Re: [Mono-list] Socket code.
On Mon, Feb 23, 2004 at 10:21:06AM +, Jonathan Gilbert wrote:
At 11:59 AM 22/02/2004 -0800, you wrote:
On Sun, 2004-02-22 at 01:17, Michal Moskal wrote:
On Sat, Feb 21, 2004 at 04:31:00PM -0800, George Farris wrote:
I have some socket code that looks something like this:
byte[] bytes = new byte[1448];
do {
len = sock.Read(bytes, 0, (int)1448);
s = Encoding.ASCII.GetString(bytes);
buf.Append(s.Substring(0,len));
Interesting, this code will work as long as the check at the bottom is
1448, any other number and it croaks??? The buffer can be any size
presumably larger than 1448 and it will work.
Note (to original poster) that if you want to read up to the end of the
stream, you should actually check 'len' against '0':
if (len 0)
error_occurred();
if (len == 0)
end_of_stream_occurred();
This ain't C -- for errors you should get exception, not -1.
You shouldn't assume that the underlying transport will always be able to
deliver chunks of the same size. The 1448 here has most likely to do with
the sending computer's TCP/IP stack. Perhaps it ties the TCP packet size
into the MTU to decrease fragmentation increase performance (this is
fairly common, as I understand). Assuming this is the case, your problem is
simply evidence that, unlike my 'ReadFully' function,
'NetworkStream::Read()' does not try to completely read the requested
buffer. Instead, it does one read attempt and then immediately returns. So,
you're limited to the size that 'Socket::Receive()' will give you.
This is the same as read(2) Unix function. This is the only reasonable
thing to do -- there are cases when you don't want to block, so you
first check if data is available and then do read(). If read() wait for
entire buffer to be filled you would block anyway.
In short, any positive value from 'Socket::Receive()' indicates that there
could be additional data waiting; a return value of 0 indicates that the
connection was shut down in that direction (thus indicating the
end-of-stream). Simply getting a smaller value than you requested does not
indicate that the end of the stream has been reached. This is distinctly
different from file I/O.
Again, I'm not sure about .NET class library, but for Unix file I/O
read() can return less then requested, and it still wouldn't indicate
end of file. If you wonder why, consider pipes that are also files.
--
: Michal Moskal :: http://www.kernel.pl/~malekith :: GCS !tv h e+++ b++
: When in doubt, use brute force. -- Ken Thompson :: UL$ C++ E--- a?
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
On Mon, 2004-02-23 at 19:37, Michal Moskal wrote: This ain't C -- for errors you should get exception, not -1. Not so. According to microsoft's guidelines, it depends on the error - if it's an error that you expect to get often, there should be a return code to indicate it. This is because of the performance cost of exceptions. There are various examples of this, such as Hashtable returning null if you ask for a non-existent key and string.IndexOf() returning -1 if the thing you are searching for is not found. -- Iain McCoy [EMAIL PROTECTED] ___ Mono-list maillist - [EMAIL PROTECTED] http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
On Mon, Feb 23, 2004 at 08:51:36PM +1100, Iain McCoy wrote: On Mon, 2004-02-23 at 19:37, Michal Moskal wrote: This ain't C -- for errors you should get exception, not -1. Not so. According to microsoft's guidelines, it depends on the error - if it's an error that you expect to get often, there should be a return code to indicate it. This is because of the performance cost of exceptions. Yes, but not for NetworkStream.Read(). Beside I wonder why exceptions cannot be fast enough. Using exceptions is sometimes very natural way of expression some algorithms, especially in functional languages. I have to benchmark it some day. -- : Michal Moskal :: http://www.kernel.pl/~malekith :: GCS !tv h e+++ b++ : When in doubt, use brute force. -- Ken Thompson :: UL$ C++ E--- a? ___ Mono-list maillist - [EMAIL PROTECTED] http://lists.ximian.com/mailman/listinfo/mono-list
RE: [Mono-list] Socket code.
I'm not sure why you're checking for specific lengths of bytes returned at
all.
This is what I use:
string Read (Stream stream)
{
StringBuilder sb = new StringBuilder ();
byte [] rgb = new byte [8 * 1024];
int cbRead;
while ((cbRead = stream.Read (rgb, 0, rgb.Length)) 0)
sb.Append (Encoding.ASCII.GetString (rgb, 0, cbRead));
return sb.ToString ();
}
Piers.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of George Farris
Sent: Sunday, February 22, 2004 5:50 PM
To: Mono list
Subject: Re: [Mono-list] Socket code.
On Mon, 2004-02-23 at 02:21, Jonathan Gilbert wrote:
Interesting, this code will work as long as the check at the bottom
is 1448, any other number and it croaks??? The buffer can be any
size presumably larger than 1448 and it will work.
Note (to original poster) that if you want to read up to the end of
the stream, you should actually check 'len' against '0':
if (len 0)
error_occurred();
if (len == 0)
end_of_stream_occurred();
You shouldn't assume that the underlying transport will always be able
to deliver chunks of the same size. The 1448 here has most likely to
do with the sending computer's TCP/IP stack. Perhaps it ties the TCP
packet size into the MTU to decrease fragmentation increase
performance (this is fairly common, as I understand). Assuming this is
the case, your problem is simply evidence that, unlike my 'ReadFully'
function, 'NetworkStream::Read()' does not try to completely read the
requested buffer. Instead, it does one read attempt and then
immediately returns. So, you're limited to the size that
'Socket::Receive()' will give you.
In short, any positive value from 'Socket::Receive()' indicates that
there could be additional data waiting; a return value of 0 indicates
that the connection was shut down in that direction (thus indicating
the end-of-stream). Simply getting a smaller value than you requested
does not indicate that the end of the stream has been reached. This is
distinctly different from file I/O.
Well I do check for len 0, here is the entire function, I also understand
about the TCP packet size not being consistent but what I don't understand
is why this actually works only like this? Seems like a bug in the stream
code.
private string read (NetworkStream stream, TcpClient c) {
int len;
string s;
StringBuilder buf = new StringBuilder();
do {
byte[] bytes = new byte[5];
len = stream.Read(bytes, 0, (int)5);
if (len 0 ) {
s = Encoding.ASCII.GetString(bytes,0,len);
buf.Append(s);
}
if (len 1448)
break;
} while (len 0);
return buf.ToString();
}
If I take the if (len 1448) out it doesn't work. If I change the 1448 to
something else, it doesn't work.
--
George Farris [EMAIL PROTECTED]
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
On Sat, Feb 21, 2004 at 04:31:00PM -0800, George Farris wrote:
I have some socket code that looks something like this:
byte[] bytes = new byte[1448];
do {
len = sock.Read(bytes, 0, (int)1448);
s = Encoding.ASCII.GetString(bytes);
buf.Append(s.Substring(0,len));
Shouldn't it be:
s = Encoding.ASCII.GetString(bytes, 0, len);
buf.Append(s);
Passing some random junk (ergh.. does new byte[] clear memory?) to
encodoer doesn't seem good idea.
--
: Michal Moskal :: http://www.kernel.pl/~malekith :: GCS !tv h e+++ b++
: When in doubt, use brute force. -- Ken Thompson :: UL$ C++ E--- a?
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
On Sun, 2004-02-22 at 01:17, Michal Moskal wrote:
On Sat, Feb 21, 2004 at 04:31:00PM -0800, George Farris wrote:
I have some socket code that looks something like this:
byte[] bytes = new byte[1448];
do {
len = sock.Read(bytes, 0, (int)1448);
s = Encoding.ASCII.GetString(bytes);
buf.Append(s.Substring(0,len));
Shouldn't it be:
s = Encoding.ASCII.GetString(bytes, 0, len);
buf.Append(s);
Well the docs on GetString say it can be just a byte[].
The interesting thing is the length which only works as 1448 which is a
number I found by printing out the len variable. If I set the byte[] to
say 1, sock.read (which should really be named stream.read as it's a
NetworkStream) always returns anywhere from 0 to 1448 bytes but never
more.
I also tried the example from the .NET docs and it doesn't work either.
It reads up to 1448 bytes and the stops. The code is here:
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemnetsocketsnetworkstreamclassreadtopic.asp?frame=true
--
George Farris [EMAIL PROTECTED]
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
Below...
On Sun, 2004-02-22 at 13:06, George Farris wrote:
snip/
Shouldn't it be:
s = Encoding.ASCII.GetString(bytes, 0, len);
buf.Append(s);
Well the docs on GetString say it can be just a byte[].
Yes, GetString(byte[]) exists, but it's equivalent to this:
string GetString (byte[] bytes)
{
return GetString (bytes, 0, bytes.Length);
}
So this is only valid if `bytes' is full. Otherwise (when you've read
fewer than `bytes.Length' bytes) the end of your string will contain
garbage data. The `GetString (bytes, 0, len)' fixes this, so GetString
will only convert valid data.
- Jon
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
Right, I did go and read up on GetString and you are of course correct.
On Sun, 2004-02-22 at 10:51, Jonathan Pryor wrote:
Below...
On Sun, 2004-02-22 at 13:06, George Farris wrote:
snip/
Shouldn't it be:
s = Encoding.ASCII.GetString(bytes, 0, len);
buf.Append(s);
Well the docs on GetString say it can be just a byte[].
Yes, GetString(byte[]) exists, but it's equivalent to this:
string GetString (byte[] bytes)
{
return GetString (bytes, 0, bytes.Length);
}
So this is only valid if `bytes' is full. Otherwise (when you've read
fewer than `bytes.Length' bytes) the end of your string will contain
garbage data. The `GetString (bytes, 0, len)' fixes this, so GetString
will only convert valid data.
- Jon
--
George Farris [EMAIL PROTECTED]
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
El dom, 22-02-2004 a las 01:31, George Farris escribió:
I have some socket code that looks something like this:
byte[] bytes = new byte[1448];
do {
len = sock.Read(bytes, 0, (int)1448);
s = Encoding.ASCII.GetString(bytes);
buf.Append(s.Substring(0,len));
if (len 1448)
break;
} while (len 0);
The key thing is this used to work with a socket size of 1460, now it is
down to 1448. If I don't set the buffer size exactly then all the data
available is not read. Is there a property to get the packet size of a
socket?
If this doesn't work, it's a bug. Please file a bug report in bugzilla
and attach a simple test case to reproduce the problem.
Thanks.
-Gonzalo
___
Mono-list maillist - [EMAIL PROTECTED]
http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] Socket code.
At 04:31 PM 21/02/2004 -0800, you wrote:
I have some socket code that looks something like this:
byte[] bytes = new byte[1448];
do {
len = sock.Read(bytes, 0, (int)1448);
s = Encoding.ASCII.GetString(bytes);
buf.Append(s.Substring(0,len));
if (len 1448)
break;
} while (len 0);
The key thing is this used to work with a socket size of 1460, now it is
down to 1448. If I don't set the buffer size exactly then all the data
available is not read. Is there a property to get the packet size of a
socket?
The property 'Available' returns the number of bytes which are waiting. In
general, you can't assume anything about the MTU, though it is reasonable
to assume that it won't be much larger than 1500, since this limit is
enforced by the underlying data transport. However, you have to keep in
mind that TCP is a stream-oriented protocol. It isn't based on the concept
of packets. Multiple packets could arrive between two calls to sock.Read().
In addition, TCP packets can exceed the data layer MTU; they will be
automatically fragmented, and then reassembled by the operating system
before your code ever sees them. Thus, you can atomically see an increase
in available bytes of more than the MTU.
Your loop looks roughly correct, except that for the Encoding function, you
should be using an overload which allows you to specify the amount of data
to translate. It just so happens that Encoding.ASCII defines a 1-to-1
correspondence between bytes and characters, but you cannot assume this.
This leads to another problem as well; a character might be split across
packet boundaries, in which case it would not decode properly. To properly
handle this, you need to collect the byte[] values until you have as much
data as you expected, and only then translate them to a string. Here is how
I would do it:
static byte[] ReadFully(Socket socket, int numBytes)
{
byte[] ret = new byte[numBytes];
int offset = 0;
while (numBytes 0)
{
int received = socket.Receive(ret, offset, numBytes, SocketFlags.None);
if (received = 0)
throw new SocketException(Connection closed by peer or network error
occurred);
offset += received;
numBytes -= received;
}
return ret;
}
static string ReadString(Socket socket, int expectedBytes)
{
byte[] bytes = ReadFully(socket, expectedBytes);
return Encoding.ASCII.GetString(bytes);
}
With this code, if you have a string that you expect to be 10,000 bytes
long, simply pass 10,000 as the parameter to 'ReadString'. Packets will be
automatically collected and put back together as appropriate.
If you are expecting TCP to allow the receiving end to Receive() the same
chunks that the sending end Send()'s, this is a flawed assumption. As I
mentioned earlier, TCP is a stream-based protocol. You can't any more
expect to automatically Receive() the same chunks that were sent using
Send() than you can expect to automatically be able to determine the size
of each Write() to a file after the file has been written. Just think of
the TCP stream as a non-packet-oriented stream of bytes. In this context,
if you have a variable-length string, you need to either send the length of
the string explicitly before you send the string, or decide on a specific
terminator. If you chose to use a terminator, then you will need a
substantially more complicated routine than the one above. Here is a
routine which I have used in the past:
static string ReadToCRLF(Socket socket)
{
ArrayList buffers = new ArrayList();
byte[] buffer = new byte[1000];
int offset = 0;
bool cr = false;
while (true)
{
int bytesToReceive = ((offset + 1 buffer.Length) !cr) ? 2 : 1;
int count = socket.Receive(buffer, offset, bytesToReceive,
SocketFlags.None);
offset += count;
if (cr)
if (buffer[offset - count] == 10)
break;
else
cr = false;
if ((count == 2) (buffer[offset - 2] == 13) (buffer[offset - 1]
== 10))
break;
if (buffer[offset - 1] == 13)
cr = true;
if (offset == buffer.Length)
{
buffers.Add(buffer);
buffer = new byte[1000];
offset = 0;
}
}
int totalBytes = buffers.Count * 1000 + offset - 2; /* remove the CRLF */
byte[] bytes = new byte[totalBytes];
int remaining = totalBytes;
offset = 0; /* re-using this variable */
while (remaining 0)
{
byte[] source;
if (buffers.Count 0) /* this is required because there might be a
CRLF spanning the boundary between the last two buffers */
{
source = (byte[])buffers[0];
buffers.RemoveAt(0); /* performance might increase slightly by using
an index instead of erasing buffers, but it shouldn't be a big issue */
}
else
source = buffer; /* the last buffer */
int numBytes = buffer.Length;
if (numBytes remaining)
numBytes = remaining;
Array.Copy(source, 0, bytes, offset, numBytes);
offset += numBytes;
remaining -= numBytes;
}
return
