RE: iSCSI: BASE64 and numerical values

["Martin, Nick (Server Storage)" <Nick.Martin@hp.com>]

Julo,

 

Although it is more compact than hex which encodes 1/2 byte at a time, b64 encodes 3/4 of a byte at a time.  When encoding b64 strings, one normally processes three bytes at a time, low addressed byte first.  When encoding and decoding numbers on little endian to b64, perhaps the process is straight forward, but I am not sure.  I am sure it is extra code relative to hex for handling lengths not a multiple of 3 bytes.  If any implementation is allowed to send numbers in b64, then all must have capability to accept them whenever parsing for a number.  This is a third redundant encoding for a number.  We could operate effectively with only one (hex).  The single code path would be well tested.  I expect several implementations (including my own) to have flaws in their first attempt at b64 encoding or decoding for numbers of arbitrary length.  I don't expect anyone to have flaws in their hexadecimal handling.

 

I have seen discussion about not allowing excess leading zeroes.  This defeats inferring the correct storage size of a number from its encoding.  Outside of security, the sizes of numbers are already fixed and this is not a problem.  Within security, I am not sure this is true.  (I am nothing like a security expert yet.)

 

Another confusion factor is numbers of arbitrary length.  Without knowing the format required internally by security algorithms which use these, I am not sure whether they will want big endian or little endian format on little endian hardware.  I hope they in fact want a b64 or hex representation in big endian a.k.a. network byte order (a string) and convert it to internal format themselves.  Suppose the algorithm wants a string in hex, but I receive a string in b64!  Whether these are used internally as numbers or strings is not relevant if they are always delivered to the algorithms as strings.

 

Although I use decimal rather than hex today, I would be willing to give it up for simpler implementations.

 

If b64 is only used for security strings and the login process can treat them as strings and pass them to security algorithms as strings, then whether the string contains b64, hex, or decimal is transparent to parsing during login.  I see no reason to require an implementation to accept b64 for login operational parameters like MaxBurstSize.

 

Unlike redundant encoding options in the spec, if my messages are redundant, you are free to ignore them ;-).

 

Thanks,

Nick

 

P.S. Due to the spam filter, my recent posting did not reach the list.  I am working on it.

 

Nick Martin

now part of the new HP

 

-----Original Message-----
From: Julian Satran [mailto:Julian_Satran@il.ibm.com]
Sent: Tuesday, May 21, 2002 7:05 PM
To: Martin, Nick (Server Storage)
Cc: ips@ece.cmu.edu; Robert Snively; Bill Studenmund
Subject: RE: iSCSI: BASE64 and numerical values

Nick,

I said already that I will say something about order in numbers - and that holds for hex or b64
and that will be the "normal"  high-endian encoding.
Why do you consider hex and b64 to be  different?

Julo

"Martin, Nick (Server Storage)" <Nick.Martin@hp.com> 05/21/2002 11:11 PM Please respond to "Martin, Nick (Server Storage)"

To:        "Robert Snively" <rsnively@Brocade.COM>, "Bill Studenmund" <wrstuden@wasabisystems.com>, Julian Satran/Haifa/IBM@IBMIL, <ips@ece.cmu.edu>         cc:                 Subject:        RE: iSCSI: BASE64 and numerical values

Robert,

My comments are below.

> -----Original Message-----
> From: Robert Snively [mailto:rsnively@Brocade.COM]
> Sent: Tuesday, May 21, 2002 1:44 PM
> To: Bill Studenmund; Julian Satran; ips@ece.cmu.edu
> Subject: RE: iSCSI: BASE64 and numerical values
>
>
> I guess that my point was that we should be transmitting the
> binary value, not the encoding of the binary value.
>
In the current draft and all those I saw preceding one can not transmit "raw binary" values during login phase.  To be transmitted within a Login, Login Response, Text, or Text Response PDU, the value must be formatted as a null terminated string of the form keyword=value.  Further the set of characters which may be used is restricted as specified.

> Especially if you care about the length, that is the most
> compact version of transmission.
>
Agreed this is the most compact, but not the most universal.  As many working in both big endian and little endian worlds can attest, a number conveyed in "binary" is not universally understood.

> You are never using them in the text-encoded format, so
> why transmit them in a text-encoded format.  The values are
> ALWAYS used only in the binary format.  TCP/IP does not
> have any requirement for them to ever be expressed in the
> text-encoded format.
>
We need to encode in some text format.  The rules for doing so need to be agreed.
The distinction between binary strings and binary numbers must be clear.  Strings of any sort will be transmitted lowest addressed byte first.  Binary strings would be encoded into text strings lowest addressed byte first.  Text encoding of numbers are always transmitted most significant digit (hex, decimal, or base64).  A conversion between native binary numeric representation and text implies native endian awareness.  When dealing with binary values on the order of say 1024 bits, one must be aware of endian issues to encode, transmit, receive, encode and use the result correctly.  Is this value used as a string, or a number?  If a string, one will want to encode and transmit low addressed byte first.  If binary one will want to encode and transmit the most significant "digit" first (either highest address or lowest address depending on native arithmetic byte order).

> If you need to input them, describe them for debug purposes, or
> print them to a screen, then put them in the
> format that is most natural for parsing to natural byte and
> word boundaries, which is hexadecimal, but would be outside the
> scope of the iSCSI document anyway.
>
Decimal is handy for humans, but not indispensable for machine to machine.
I agree with others that hexadecimal is the simplest, and least ambiguous encoding.
Still one needs to know whether a string or a binary value is being processed unless one is fortunate enough to only be concerned with big-endian (network byte order) hardware.

Prior use of Base64 was designed explicitly for binary files which are effectively binary strings. It will encode, transmit, and decode lowest byte address first.

If the intent is to use base64 to transmit large binary numbers, it could be done, but why bother when hexadecimal is so straight forward?  
The big deal with 4 bits, 6 bits, or 8 bits encoding is the loading, masking, shifting, and storing of the next bits from the correct location at the correct time from 8 bits per byte addressable memory systems.  If the whole value fits in one arithmetic register at one time, no big deal.  However for larger quantities, 4 bits (hexadecimal) and 8 bits (raw binary which is prohibited) are simple, 6 bits (base64) and 3.something bits (decimal) are not.

Thanks,
Nick

> Bob
>
>
> > -----Original Message-----
> > From: Bill Studenmund [mailto:wrstuden@wasabisystems.com]
> > Sent: Tuesday, May 21, 2002 9:28 AM
> > To: Robert Snively
> > Cc: Julian Satran; ips@ece.cmu.edu
> > Subject: RE: iSCSI: BASE64 and numerical values
> >
> >
> > On Tue, 21 May 2002, Robert Snively wrote:
> >
> > > I have been having continuing problems understanding why
> > > iSCSI is considering any use of base64, and I find myself
> > > in total agreement with Bill's clear exposition of the
> > > situation.
> > >
> > > Upon reviewing RFC-2045, it is clear that base64 is a method
> > > for containing handy binary values in a EMAIL structure that does
> > > not use TCP/IP to transmit data, but rather uses obsolete
> > > 7-bit ASCII or (slightly less obsolete) EBCDIC strings to
> > > carry the data.  TCP/IP transmits data neutral octets,
> > > so binary values can be carried in their native binary
> > > string format (allowing for a possible requirement to pad
> > > strings to an octet boundary).  The most standard way to
> > > present those binary strings on a printer or as an input
> > > value to a program is hexadecimal encoding.
> > >
> > > For those reasons, I agree with Bill that we should delete
> > > base64 from the text and use hexadecimal as he suggests.
> >
> > I must not have been totally clear. I'm not suggesting we
> > remove base64,
> > I'm suggesting we limit its use to the cases where (IMHO) it
> > makes sense.
> > While there will be much fewer cases in the text with the changes I
> > suggest, it will still be in the spec.
> >
> > Some of the cryptographic authentications throw around large binary
> > strings. Quite large (compared to the numbers and strings we
> > often use)
> > binary strings. The fact that if you support these modes of
> > authentication
> > you are supposed to be able to accept four times as much
> text in login
> > commands reflects the size of these strings.
> >
> > Base64 is a real win for these. Hex can encode 4 bits per
> byte, while
> > base64 can encode 6. Thus an n byte string will be about
> > 1.33*n bytes in
> > base 64, while it will be 2n bytes in hex. That size
> > difference is why we
> > want base64.
> >
> > The main point I've been trying to make is that binary
> > strings and numbers
> > are different things, and that encoding methods for one
> aren't really
> > appropriate for the other. Specifics: decimal & hex work great for
> > numbers, and hex and base64 work great for binary strings.
> That's it.
> > Also, the hex for numbers isn't the same as the hex for
> > binary strings.
> > One is the number in hex, the other is more like hexdump -x
> > without the
> > spaces and without the offsets.
> >
> > Take care,
> >
> > Bill
> >
> >
> >
>