RE: iSCSI: Use of the A bit

[Eddy Quicksall <Eddy_Quicksall@ivivity.com>]

> why did the unit negotiate it to that size?

Well, if things are layered properly, a ULP would be able to use several
transports and at the same time. And the ULP could have dynamic needs.

The ULP should be separate enough that it only issues a "send" request to
the transport and the transport does everything it can to send it. Then, it
informs the ULP that it is finished. The ULP then re-uses the buffer for the
next "send".

The ULP should not understand the transports and especially it should not
have different code for all the different transports.

So, negotiation should depend on the transports needs, not the ULP's.

> We got to this position, since so many folks did not want to support the
positive ack.

You can do this by negotiating for ErrorRecoveryLevel = 0. Also, what is so
difficult about the positive ack when ErrorRecoveryLevel > 0 (especially
when you have all that complex code to do error recovery anyway)?

BTW, parallel SCSI can do this. Why shouldn't iSCSI? What happens when you
want to use the same ULP but upgrade to iSCSI?

Eddy

-----Original Message-----
From: John Hufferd [mailto:hufferd@us.ibm.com]
Sent: Thursday, March 14, 2002 2:36 PM
To: LEMAY,KEVIN (A-Roseville,ex1)
Cc: 'Eddy Quicksall'; BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Rod
Harrison'; ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: Use of the A bit



We got to this position, since so many folks did not want to support the
positive ack.  And this approach was put in as a compromise.  I would
suggest, that the thing we are chasing is a small corner case that may not
be smart enough to operate at anything other then ErrorRecoverylevel=0.  In
any event I do not see a problem with limiting the response to MaxBurstSize
intervals.  If the interval is too big, why did the unit negotiate it to
that size?

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: hufferd@us.ibm.com


"LEMAY,KEVIN (A-Roseville,ex1)" <kevin_lemay@agilent.com>@ece.cmu.edu on
03/14/2002 10:43:31 AM

Sent by:    owner-ips@ece.cmu.edu


To:    "'Eddy Quicksall'" <Eddy_Quicksall@ivivity.com>, "BURBRIDGE,MATTHEW
       (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>, "'Rod Harrison'"
       <rod.harrison@windriver.com>, John Hufferd/San Jose/IBM@IBMUS
cc:    "ips@ece. cmu. edu (E-mail)" <ips@ece.cmu.edu>
Subject:    RE: iSCSI: Use of the A bit



I also do not like the comment about having to wait until maxBurstsize byte
to set the A-bit.

I wanted to use it as Eddie suggested. To force the initiator to
acknowledge
the IO so that I can release resources on the target.

This will be especially useful for hardware accelerated IOs where the chip
cannot declare the IO complete until it has been Ack'd.

Could we just change the wording to:
----
For sessions with ErrorRecoveryLevel 1 or higher the target sets this bit
to
1 to indicate that it requests from the initiator a positive
acknowledgement
for the data received. The target should use the A bit moderately; the
A-bit
MAY only be set to 1 at the end of a data sequence.

On receiving a Data-In PDU with the A bit set to 1 the initiator MUST
issued
a SNACK of type DataACK. If the initiator has detected holes in the input
sequence, it MUST postpone issuing the SNACK of type ACKN until the holes
are filled.
---

This way I can always set the A bit at the end of an IO.

Kevin Lemay

-----Original Message-----
From: Eddy Quicksall [mailto:Eddy_Quicksall@ivivity.com]
Sent: Thursday, March 14, 2002 5:51 AM
To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Rod Harrison'; John
Hufferd
Cc: ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: Use of the A bit


I see a lot of responses talking about ErrorRecoveryLevel = 0. That was not
my original point. ErrorRecoveryLevel = 0 already has this solved. I am
talking about ErrorRecoveryLevel > 0 (I made an error in my first EMAIL and
the point may have been lost).

If the initiator has the option to ignore the A bit, the target will get
stuck. What is wrong with requiring the initiator to respond? Can we just
get that answered first?

I suspect that there is a concern that this may cause too much traffic. My
answer would be "don't buy a cheep target ... their lack of memory may
cause
degraded performance" (not intended for the spec, of course :-) ).

I also do not see the rational in not allowing the A bit more often than
MaxBurstSize. The ULP layers may not know anything about the transport
layers. So their resource issues are not known by the transport(s). The
MaxBurstSize is an iSCSI issue and may not apply to future protocols.

So, why not just say something like:

1) The initiator MUST send a DataACK SNACK if it sees the A bit set (and if
ErrorRecoveryLevel > 0)
2) The A bit may be set only when the F bit is set.

And leave it go at that?

Eddy

-----Original Message-----
From: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
[mailto:matthew_burbridge@hp.com]
Sent: Thursday, March 14, 2002 7:22 AM
To: 'Rod Harrison'; John Hufferd
Cc: Eddy Quicksall; ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: Use of the A bit


I think that what Santosh and others have stated is the correct think to do
for initiators.  If the revovery level is 0 and the A bit is set then the
initiator is quite with in its rights to ignore the A bit (If the target is
at fault then penalise the target and not the initiator).

IMHO changing the spec to make it more specific is not the right thing and
it could lead to inter operability problems.  However, I do think something
needs to be added so I propose adding the statement:

"If ErrorRecoveryLevel=0 then the initiator MAY ignore the A bit (i.e.
assume A=0)".

One could argue for the MAY to be a MUST but if there is a kind initiator
out there then let it do its stuff - it can only help the rogue target.

Matthew

-----Original Message-----
From: Rod Harrison [mailto:rod.harrison@windriver.com]
Sent: Thursday, March 14, 2002 9:27 AM
To: John Hufferd
Cc: Eddy Quicksall; BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); ips@ece.
cmu. edu (E-mail)
Subject: RE: iSCSI: Use of the A bit


John,

 Actually 9.7.2 says almost exactly that ...

 "For sessions with ErrorRecoveryLevel 1 or higher, the target sets
this bit to 1 to indicate that it requests a positive acknowledgement
from the initiator for the data received."

 We've interpreted that to mean our initiator should ignore the A bit
if ErrorRecoveryLevel is 0.

 Having said that it is very easy to support just the positive form
of
SNACK, so I would have no objection to removing the A bits current
link to ErrorRecoveryLevel.

 - Rod

-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
John Hufferd
Sent: Thursday, March 14, 2002 3:12 AM
To: Rod Harrison
Cc: Eddy Quicksall; BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); ips@ece.
cmu. edu (E-mail)
Subject: RE: iSCSI: Use of the A bit



Actually it does not actually say that.  It says "This is limited to
sessions that support error recovery and is implemented through the A
bit
..."

I know what I am about to say, is nit picking, but ....
ErrorRecoveryLevel=0 is an error recovery technique.

Having said that, we probably should just change that statement to
"This is
implemented through the A bit ..."


.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: hufferd@us.ibm.com


"Rod Harrison" <rod.harrison@windriver.com>@ece.cmu.edu on 03/13/2002
01:05:51 PM

Sent by:    owner-ips@ece.cmu.edu


To:    "Eddy Quicksall" <Eddy_Quicksall@ivivity.com>,
"BURBRIDGE,MATTHEW
       (HP-UnitedKingdom,ex2)" <matthew_burbridge@hp.com>, "ips@ece.
cmu.
       edu (E-mail)" <ips@ece.cmu.edu>
cc:
Subject:    RE: iSCSI: Use of the A bit



Eddy,

 You were right first time, in draft 11 section 2.5.1.5 paragraph 5
and 9.7.2 state that the A-bit is only supported if ErrorRecoveryLevel
is 1 or higher.

 - Rod

-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
Eddy Quicksall
Sent: Wednesday, March 13, 2002 8:03 PM
To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); ips@ece. cmu. edu
(E-mail)
Subject: RE: iSCSI: Use of the A bit


Then my only concern is that the initiator may ignore the A bit if it
deems that the bit is being set aggressively.

If it ignored it, then the target would be stalled waiting for he ACK.

Eddy
-----Original Message-----
From: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
[mailto:matthew_burbridge@hp.com]
Sent: Wednesday, March 13, 2002 1:39 PM
To: 'Eddy Quicksall'; ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: Use of the A bit
Importance: High


Eddy,

The target is quite within its rights to use the A bit when at
recovery level 0.  If the session is re-established due to recovery
7.11.4 then the relevant command is aborted anyway and so there is no
reason to keep hold of the data any way: With recovery level 0 there
is no recovery mechanism that requires the target to keep the data.
Therefore the A bit is redundant when the recovery level is 0.

The spec says that the initiator MUST issue a SNACK if the A bit is
set.  However, the MaxBurstSize restriction is there to prevent the
initiator from having to send a SNACK on every PDU in the case where a
target inadvertently sets the A bit in (for example) every data in
PDU. The target may set the A bit more often than the MaxBurstSize but
it should not expect a SNACK more often than this.

Matthew Burbridge
-----Original Message-----
From: Eddy Quicksall [mailto:Eddy_Quicksall@ivivity.com]
Sent: Wednesday, March 13, 2002 3:12 PM
To: ips@ece. cmu. edu (E-mail)
Subject: iSCSI: Use of the A bit


Here is a case that I want to go over and if there is not already a
solution, I think a refinement to the A bit could solve it.

The problem is that a target (perhaps an iSCSI disk drive) does not
have enough memory to transfer the full READ request so it must read
from the medium as much as it can, transmit that, when that
transmission is known to be good, read the next bunch, transmit that
and so on.

The problem we have is that the target must keep the buffer around
until the transfer has been "ack'd" via ExpStatSN. But that status
can't be sent because all of the requested data has not been sent. So
the target would have to refuse to do the command.

I was going to use the A bit for this thinking it would force the
initiator to give an "ack" but our current wording does not make this
a sure fire thing:

1) The initiator may not want to run at ErrorRecoveryLevel 1.
2) The initiator may ignore the A bit if it deems that the bit is
being set aggressively.
3) The target may set the A bit no more frequently than MaxBurstSize.

Comments?

mailto:Eddy_Quicksall@iVivity.com