Re: SSIZE_MAX
On Thu, Jan 16, 2020 at 10:19:52AM -0700, Raymond, David wrote: > Thanks, that is helpful. > > It is now clear to me that the default on OpenBSD for SSIZE_MAX is > 2^31 - 1 or greater. However, I still run into problems on writes to > a TCP/IP socket with sizes exceeding something like 32000 bytes > (probably 2^15 -1). > > Is it possible that TCP sockets have a smaller SSIZE_MAX or could this > be a bug? I am handling partial reads and writes correctly as far as > I can see. Chopping big writes into chunks smaller than 32000 seems > to solve the problem. You need to tell more details. what errno? mayeb a ktrace? Show the code? -Otto > > Dave > > On 1/16/20, Martin Wanvik wrote: > > tor. 16. jan. 2020 kl. 14:52 skrev Raymond, David : > >> > >> Hmm > >> > >> Thought I found a 2^15 -1 version of SSIZE_MAX in the includes, but I > >> guess I was mistaken. > > > > Not necessarily. What you have probably seen is _POSIX_SSIZE_MAX > > (which is almost literally > > what you wrote in your first post), defined in limits.h to be 32767 > > (2^15 - 1). But that > > specifies the minimum value that SSIZE_MAX can have and still conform > > to the POSIX standard. > > So the actual SSIZE_MAX may well be (and usually is) much bigger, as > > others have pointed out. > > > >> The real issue is whether doing write(2) to a TCP/IP socket bigger > >> than 2^15 - 1 bytes causes problems. I am not very experienced in > >> this area. > > > > > > > -- > David J. Raymond > david.raym...@nmt.edu > http://physics.nmt.edu/~raymond >
Re: SSIZE_MAX
Thanks, that is helpful. It is now clear to me that the default on OpenBSD for SSIZE_MAX is 2^31 - 1 or greater. However, I still run into problems on writes to a TCP/IP socket with sizes exceeding something like 32000 bytes (probably 2^15 -1). Is it possible that TCP sockets have a smaller SSIZE_MAX or could this be a bug? I am handling partial reads and writes correctly as far as I can see. Chopping big writes into chunks smaller than 32000 seems to solve the problem. Dave On 1/16/20, Martin Wanvik wrote: > tor. 16. jan. 2020 kl. 14:52 skrev Raymond, David : >> >> Hmm >> >> Thought I found a 2^15 -1 version of SSIZE_MAX in the includes, but I >> guess I was mistaken. > > Not necessarily. What you have probably seen is _POSIX_SSIZE_MAX > (which is almost literally > what you wrote in your first post), defined in limits.h to be 32767 > (2^15 - 1). But that > specifies the minimum value that SSIZE_MAX can have and still conform > to the POSIX standard. > So the actual SSIZE_MAX may well be (and usually is) much bigger, as > others have pointed out. > >> The real issue is whether doing write(2) to a TCP/IP socket bigger >> than 2^15 - 1 bytes causes problems. I am not very experienced in >> this area. > > -- David J. Raymond david.raym...@nmt.edu http://physics.nmt.edu/~raymond
Re: SSIZE_MAX
On Thu, Jan 16, 2020 at 09:35:38AM +, cho...@jtan.com wrote: > Raymond, David writes: > > I am confused about SSIZE_MAX and read(2)/write(2). The POSIX > > SSIZE_MAX is something like 2^15 -1. This seems to be a real > > limitation when writing to a TCP/IP socket, as I learned from > > experience. However, much larger reads and writes seem to be possible > > to files and UNIX sockets (pipes). This makes me uneasy, given the > > warning in the man pages for read(2)/write(2). > > > > Any insight on this topic would be appreciated. > > I would guess this is part of the reason why ssize_t was invented > - so that half of the numeric range could be wasted in order for a > function to be able to return -1, and/or ridiculous notions of > symmetry. Most people out there don't understand C numeric types, as a very long list of signed vs unsigned bugs in all kinds of tools show. Clamping down to a value that has less chance of confusing the numerous idiots out there seems like a valid approach to me.
Re: SSIZE_MAX
tor. 16. jan. 2020 kl. 14:52 skrev Raymond, David : > > Hmm > > Thought I found a 2^15 -1 version of SSIZE_MAX in the includes, but I > guess I was mistaken. Not necessarily. What you have probably seen is _POSIX_SSIZE_MAX (which is almost literally what you wrote in your first post), defined in limits.h to be 32767 (2^15 - 1). But that specifies the minimum value that SSIZE_MAX can have and still conform to the POSIX standard. So the actual SSIZE_MAX may well be (and usually is) much bigger, as others have pointed out. > The real issue is whether doing write(2) to a TCP/IP socket bigger > than 2^15 - 1 bytes causes problems. I am not very experienced in > this area.
Re: SSIZE_MAX
Yes, my code deals with the short reads and writes.
On 1/16/20, Otto Moerbeek wrote:
> On Thu, Jan 16, 2020 at 06:48:30AM -0700, Raymond, David wrote:
>
>> Hmm
>>
>> Thought I found a 2^15 -1 version of SSIZE_MAX in the includes, but I
>> guess I was mistaken.
>>
>> The real issue is whether doing write(2) to a TCP/IP socket bigger
>> than 2^15 - 1 bytes causes problems. I am not very experienced in
>> this area.
>
> It should not. But short read and writes are a real thing and code
> should take care, see the example in write(2).
>
>
> -Otto
>
>>
>> Dave Raymond
>>
>> On 1/15/20, Bryan Steele wrote:
>> >> I am confused about SSIZE_MAX and read(2)/write(2). The POSIX
>> >> SSIZE_MAX is something like 2^15 -1. This seems to be a real
>> >> limitation when writing to a TCP/IP socket, as I learned from
>> >> experience. However, much larger reads and writes seem to be possible
>> >> to files and UNIX sockets (pipes). This makes me uneasy, given the
>> >> warning in the man pages for read(2)/write(2).
>> >>
>> >> Any insight on this topic would be appreciated.
>> >>
>> >> --
>> >> David J. Raymond
>> >> david.raym...@nmt.edu
>> >> http://physics.nmt.edu/~raymond
>> >
>> > Not in any reasonably modern version of POSIX..
>> >
>> > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/limits.h.html
>> >
>> > {SSIZE_MAX}
>> > Maximum value for an object of type ssize_t.
>> >
>> > $ grep -R "SSIZE_MAX" /usr/include
>> > ./amd64/limits.h:#define SSIZE_MAX LONG_MAX/* max value
>> > for
>> > a ssize_t */
>> >
>> > /usr/include/sys/limits.h:
>> > #ifdef __LP64__
>> > ..
>> > # define LONG_MAX 0x7fffL
>> > ...
>> > #else
>> > ..
>> > # define LONG_MAX 0x7fffL
>> >
>> > -Bryan.
>> >
>>
>>
>> --
>> David J. Raymond
>> david.raym...@nmt.edu
>> http://physics.nmt.edu/~raymond
>>
>
--
David J. Raymond
david.raym...@nmt.edu
http://physics.nmt.edu/~raymond
Re: SSIZE_MAX
On Thu, Jan 16, 2020 at 06:48:30AM -0700, Raymond, David wrote:
> Hmm
>
> Thought I found a 2^15 -1 version of SSIZE_MAX in the includes, but I
> guess I was mistaken.
>
> The real issue is whether doing write(2) to a TCP/IP socket bigger
> than 2^15 - 1 bytes causes problems. I am not very experienced in
> this area.
It should not. But short read and writes are a real thing and code
should take care, see the example in write(2).
-Otto
>
> Dave Raymond
>
> On 1/15/20, Bryan Steele wrote:
> >> I am confused about SSIZE_MAX and read(2)/write(2). The POSIX
> >> SSIZE_MAX is something like 2^15 -1. This seems to be a real
> >> limitation when writing to a TCP/IP socket, as I learned from
> >> experience. However, much larger reads and writes seem to be possible
> >> to files and UNIX sockets (pipes). This makes me uneasy, given the
> >> warning in the man pages for read(2)/write(2).
> >>
> >> Any insight on this topic would be appreciated.
> >>
> >> --
> >> David J. Raymond
> >> david.raym...@nmt.edu
> >> http://physics.nmt.edu/~raymond
> >
> > Not in any reasonably modern version of POSIX..
> >
> > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/limits.h.html
> >
> > {SSIZE_MAX}
> > Maximum value for an object of type ssize_t.
> >
> > $ grep -R "SSIZE_MAX" /usr/include
> > ./amd64/limits.h:#define SSIZE_MAX LONG_MAX/* max value for
> > a ssize_t */
> >
> > /usr/include/sys/limits.h:
> > #ifdef __LP64__
> > ..
> > # define LONG_MAX 0x7fffL
> > ...
> > #else
> > ..
> > # define LONG_MAX 0x7fffL
> >
> > -Bryan.
> >
>
>
> --
> David J. Raymond
> david.raym...@nmt.edu
> http://physics.nmt.edu/~raymond
>
Re: SSIZE_MAX
Hmm
Thought I found a 2^15 -1 version of SSIZE_MAX in the includes, but I
guess I was mistaken.
The real issue is whether doing write(2) to a TCP/IP socket bigger
than 2^15 - 1 bytes causes problems. I am not very experienced in
this area.
Dave Raymond
On 1/15/20, Bryan Steele wrote:
>> I am confused about SSIZE_MAX and read(2)/write(2). The POSIX
>> SSIZE_MAX is something like 2^15 -1. This seems to be a real
>> limitation when writing to a TCP/IP socket, as I learned from
>> experience. However, much larger reads and writes seem to be possible
>> to files and UNIX sockets (pipes). This makes me uneasy, given the
>> warning in the man pages for read(2)/write(2).
>>
>> Any insight on this topic would be appreciated.
>>
>> --
>> David J. Raymond
>> david.raym...@nmt.edu
>> http://physics.nmt.edu/~raymond
>
> Not in any reasonably modern version of POSIX..
>
> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/limits.h.html
>
> {SSIZE_MAX}
> Maximum value for an object of type ssize_t.
>
> $ grep -R "SSIZE_MAX" /usr/include
> ./amd64/limits.h:#define SSIZE_MAX LONG_MAX/* max value for
> a ssize_t */
>
> /usr/include/sys/limits.h:
> #ifdef __LP64__
> ..
> # define LONG_MAX 0x7fffL
> ...
> #else
> ..
> # define LONG_MAX 0x7fffL
>
> -Bryan.
>
--
David J. Raymond
david.raym...@nmt.edu
http://physics.nmt.edu/~raymond
Re: SSIZE_MAX
On 16/1/20 7:35 pm, cho...@jtan.com wrote: > I would guess this is part of the reason why ssize_t was invented > - so that half of the numeric range could be wasted in order for a > function to be able to return -1, and/or ridiculous notions of > symmetry. Actually it is used with fseek… in particular the case where whence=SEEK_CUR. It allows you to go backwards relative to the current file position. -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
Re: SSIZE_MAX
Raymond, David writes: > I am confused about SSIZE_MAX and read(2)/write(2). The POSIX > SSIZE_MAX is something like 2^15 -1. This seems to be a real > limitation when writing to a TCP/IP socket, as I learned from > experience. However, much larger reads and writes seem to be possible > to files and UNIX sockets (pipes). This makes me uneasy, given the > warning in the man pages for read(2)/write(2). > > Any insight on this topic would be appreciated. I would guess this is part of the reason why ssize_t was invented - so that half of the numeric range could be wasted in order for a function to be able to return -1, and/or ridiculous notions of symmetry. Looking in /usr/src/sys/lib/libsa/read.c shows that different types of file descriptor get their own read implementation (f->f_ops->read) which I'm not awake enough to track down. read(2) states: ... The system guarantees to read the number of bytes requested if the descriptor references a normal file that has that many bytes left before the end-of- file, but in no other case. Which suggests to me that the implementations for files and network sockets are quite different and that, unsurprisingly, the version for files is able to make a lot of assumptions and shortcuts that the networking code path cannot. Note that the manual also includes in the list of potential errors: [EINVAL] nbytes was larger than SSIZE_MAX. which is totally unqualified - ie. nbytes should not be larger than SSIZE_MAX in *any* case. Also nbytes is a size_t while the function's return value is ssize_t. This all comes to you pre-coffee. Take it as you will. Matthew
Re: SSIZE_MAX
Raymond, David wrote: > The POSIX SSIZE_MAX is something like 2^15 -1. I doubt that, you better backtrack a couple of steps.
Re: SSIZE_MAX
> I am confused about SSIZE_MAX and read(2)/write(2). The POSIX
> SSIZE_MAX is something like 2^15 -1. This seems to be a real
> limitation when writing to a TCP/IP socket, as I learned from
> experience. However, much larger reads and writes seem to be possible
> to files and UNIX sockets (pipes). This makes me uneasy, given the
> warning in the man pages for read(2)/write(2).
>
> Any insight on this topic would be appreciated.
>
> --
> David J. Raymond
> david.raym...@nmt.edu
> http://physics.nmt.edu/~raymond
Not in any reasonably modern version of POSIX..
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/limits.h.html
{SSIZE_MAX}
Maximum value for an object of type ssize_t.
$ grep -R "SSIZE_MAX" /usr/include
./amd64/limits.h:#define SSIZE_MAX LONG_MAX/* max value for
a ssize_t */
/usr/include/sys/limits.h:
#ifdef __LP64__
..
# define LONG_MAX 0x7fffL
...
#else
..
# define LONG_MAX 0x7fffL
-Bryan.
SSIZE_MAX
I am confused about SSIZE_MAX and read(2)/write(2). The POSIX SSIZE_MAX is something like 2^15 -1. This seems to be a real limitation when writing to a TCP/IP socket, as I learned from experience. However, much larger reads and writes seem to be possible to files and UNIX sockets (pipes). This makes me uneasy, given the warning in the man pages for read(2)/write(2). Any insight on this topic would be appreciated. -- David J. Raymond david.raym...@nmt.edu http://physics.nmt.edu/~raymond
