Re: iSCSI: Flow Control

[Pierre Labat <pierre_labat@hp.com>]

Douglas Otis wrote:

> Pierre,
>
> What protocol are you describing?  Are you explaining TCP or your version of
> TCP?

Doug,

Well, i omitted to say where is TCP in this model, i add some lines below to
explain
and some other lines in the middle of the mail.
First of all, TCP is handled on the adapter and is a regular TCP.
The host intiates the opening of the TCP connection, then the only thing
it knows about it, is a token returned by the adapter when the adapter
opened the TCP connection. The host doesn't know the value of the TCP
variables (window, acked bytes,...) and doesn't care about it.
Then for each command, the host posts it to the adapter indicating the token
of the TCP cx it wants the command going on.

>
>
> Doug
>
> > Julian_Satran@il.ibm.com wrote:
> >
> > > Pierre,
> > >
> > > It does not matter how from where you send the data on the wire.
> > > If you have a long wire and you want to cover the latency you will
> > > send data as soon as you can and then commands get stuck  behind.
> >
> > Julian,
> >
> > The command can NOT  be stuck because there is "data on the wire".
> > Let me give you an example,
> > Let's talk again about the "pull model" adapter on the initiator.
> > Imagine you have 100Mbytes of (write) data outstanding
> > because 1000 cmds of large write commands have been posted to
> > the adapter.
> > The adapter sends this data as fast as it can. But very important,
> > the data are not tossed in any kind of buffer on the adapter.
> > What the adapter does is: pull some kbytes of data form host memory,

>
> > encapsulate it, send it on the wire. Again and again, as fast as it can.

For each command posted to the card (in the falt array), the host specifies for
which TCP connection.
When the adapter pulls some kbytes from host memory (from one write command
buffers for example) it knows to which TCP connection it is related and adds
the TCP
header accordingly. Hence instead of pushing data in a TCP connection
(as for regular networking adapter), the adapter pulls data to send it on the
wire.
To do that an algorithm could be:
- first pulls unumbered command if any posted in the array
- then command+immediate data if any posted in the array
- then data of the previous commands if any
Each time the adapter pulls command/data from the host memory, it encapsulates
that
in iSCSI+TCP/IP(regular).
In fact the adapter pulls something from host memory when it is sure it can
send it
on the wire after the current pdu being transmitted (it checks the tcp window
before
pulling something).

All that to say that we  could  have something that works just fine (in term
of throughput, no deadlocks, no command blocked by data) with this model
and only ONE tcp connexion.


Regards,

Pierre



>
> >
> > Now, imagine that a read is posted to the adapter after the 1000 writes.
> > Here is the point. The interface between the host and the adapter is not
> > a FIFO but a flat array and the adapter can works in parallel on
> > all the commands. Immediately when the host posts the read
> > (in the flat array), the adapter sees it. The adapter as soon as it
> > completes transmitting the current data PDU, sends the read command.
> >
> > The read command is not stuck behind the 100Mbytes of data.
> > The maximum latency for the command is the time to
> > transmit one iSCSI pdu on the wire.
> > That is (size of pdu)/throughput.
> >  Then the adapter continues to send the write data of the
> > 100Mbytes. And as soon as a new command will be posted,
> > it will send a command pdu immediately after the current
> > data PDU.
> >
> > Commands are not stuck behind data because there is no FIFO
> > before the wire, and because data "on the wire" doesn't block anything.
> > The wire is always able to deliver its throughput.
> >
> >
> > Regards,
> >
> > Pierre
> >