Re: iSCSI : Concerns regarding DataRN in READ Data PDU.

[julian_satran@il.ibm.com]

Pierre,

In a complex environment you have to leave to the initiator to decide when
to ack. If data goes to another device (3rd party) the TCP ack is not
enough. Data will be acked only after reaching the final place.

Julo



Pierre Labat <pierre_labat@hp.com> on 12/01/2001 19:19:09

Please respond to Pierre Labat <pierre_labat@hp.com>

To:   ips@ece.cmu.edu
cc:
Subject:  Re: iSCSI : Concerns regarding DataRN in READ Data PDU.




julian_satran@il.ibm.com wrote:

> This feature was meant for long lasting operations (tons of data) that
> could then recover
> with ease. As for complexity... it is local counter.  If that is not
> available all long lasting commands will have to provide their own
> checkpointing mechanisms or there will be no long-lasting commands.

Julian,

The target can use the TCP acks to know what data reached the initiator.
From the TCP ack the target can find the corresponding data in the READ cmd
that have been received by the initiator. And it can free the resources for
this data
and in case of connection failure it can restart from there.
It implies that the initiator sends the TCP ack only after the  data has
been
copied out of the adapter (in sever memory or alike).
It's why the DATARN doesn't add value, it is because using TCP
you can get the same benefit.

Regards,

Pierre

>
>
> Julo
>
> "VOIGT,DOUG (HP-Boise,ex1)" <doug_voigt@hp.com> on 11/01/2001 03:11:41
>
> Please respond to "VOIGT,DOUG (HP-Boise,ex1)" <doug_voigt@hp.com>
>
> To:   IPS Reflector <ips@ece.cmu.edu>
> cc:
> Subject:  RE: iSCSI : Concerns regarding DataRN in READ Data PDU.
>
> It seems to me that this feature is redundant with TCP's own delivery
> guarantees, in an effort to minimize work loss in the event of TCP
> connection failure.  I too believe this level of complexity and potential
> normal performance impact compared with accelerated recovery from
> improbable
> error events is not likely to yield a net gain.
>
> Is recoverable TCP connection loss more common than disk array controller
> failure?  Since different controllers in the same array are (quite
> reasonably) viewed as different targets (pending SAM change), iSCSI level
> failover won't compensate for controller failure.  This example leads me
to
> suggest that the benefit we are shooting for has a high likelihood of
being
> overshadowed by other failure modes whose recovery is less graceful.
>
> Doug Voigt
>
> > -----Original Message-----
> > From: Santosh Rao [mailto:santoshr@cup.hp.com]
> > Sent: Monday, January 08, 2001 9:03 PM
> > To: IPS Reflector
> > Subject: iSCSI : Concerns regarding DataRN in READ Data PDU.
> >
> >
> > Julian/All,
> >
> > Can anybody comment on the true benefit that the DataRN
> > feature provides
> > that justifies the complexity and performance issues it introduces to
> > the protocol ?
> >
> > I see the following issues with DataRN :
> >
> > 1) It adds a performance penalty in that on every READ Data PDU
> > targets will have to initialize one additional 32-bit word in the
> > header.
> >
> > 2) Extra traffic on the outbound context from initiators sending
> > NOP-OUTs to acknowledge the DataRNs.
> >
> > 3) Performance path checks for the "P" bit on every READ PDU.
> > (Alternatively, checks for the DataNumber field from the Login
> > Text Keys).
> >
> > 4)  Implementing DataRNs per task is too short-lived to provide any
> > real benefit. Implementing them per connection or session can result
> > in quick use up of the 32-bit DataRN space (since each frame consumes
> > one DataRN), adding additional complexity to the target code
> > to use the
> > "P"
> > bit appropriately.
> >
> > 5) Further, "DataNumber", which is the size of the un-acknowledged
> > DataRN window is negotiated at login time, whereas this is really
> > desired to be a dynamic variable based on the target's
> > current I/O load
> > and the number of initiators speaking to it. Thus, DataRN can end up
> > being
> > under-utilized resulting in too much NOP-OUT traffic, or it can be
> > over-utilized resulting in too much usage of the "P" bit in the READ
> > PDUs, which, again, can cause too much NOP-OUT traffic.
> >
> > 6) Considering that the DataRN generation and acknowledgement are
> > functions
> > that would typically be in SCSI hardware assist engines and there is a
> > need to
> > keep these as simple as possible, adding such features in the
> > spec means
> >
> > that these SCSI assist engines MUST implement DataRN generation
> > and acknowledgement [since initiators have no way to turn it off if a
> > target decides to use it]
> >
> > What is the true benefit of this feature ? Is it intended to
> > be used as
> > :
> >
> > a) Will allow targets to do partial freeing of their memory buffers as
> > and when DataRNs are received ?
> >
> > b) Will allow targets to send back only the un-acknowledged data (as
> > seen from the ExpDataRN received on the last NOP-OUT from the
> > initiator)
> > when the initiators retry commands with the "Retry" bit set ?
> >
> > If the benefit intended is (a), then, in order to derive this benefit,
> > targets will have to advertise their DataNumber login key to be < the
> > avg. number of frames transmitted per READ I/O. Having a DataNumber >
> > the avg. number of frames per READ I/O implies the life of
> > the task ends
> > [and buffers for the task get released] before the initiator sends
> > NOP-OUT, thereby, defeating benefit (a).
> >
> > If the benefit intended is (b) [and i do not see this
> > explicitly stated
> > in the draft], then, this is a dangerous assumption which can lead to
> > data corruption issues.
> >
> > When commands are retried with the "Retry" bit set, are
> > targets allowed
> > to send back partial data based on the last ExpDataRN they
> > received ? If
> > so, this can have multiple issues like :
> >
> > o    complexity in handling I/O underrun cases for the retried command
> >       which only saw partial data transfer from the target.
> >
> > o    If the target only sent back partial data based
> >       on its last ExpDataRN received, then, it can open up
> > possible data
> >
> >       corruption windows.
> >
> > All in all, I believe that the DataRN is adding too much
> > complexity and
> > potential performance impact to iSCSI for the benefits it may provide.
> >
> > Moreover and most importantly, the draft does not give initiators any
> > control over the usage of this feature and if a target decides to use
> > this feature, initiators will have to support it. This exposes the
> > initiators to all of the above issues without being able to turn off
> > this feature.
> >
> > I would like to therefore ask that :
> >
> > either,
> > a) DataRN support be removed from the iSCSI draft.
> >
> > or
> > b) The support for DataRN be negotiated at login time giving initators
> > control over this feature and allowing them to disable this feature.
> > This will allow SCSI hardware assist engines to skip implementation of
> > this functionality and their associated software can merely turn off
> > DataNumber in the login negotiation.
> >
> > Thanks,
> > Santosh
> >
> >
> >
> >