Re: fix off-by-one in dup2
On Thu, Dec 05, 2013 at 06:45:22AM -0700, Eric Blake wrote: On 12/04/2013 10:51 AM, Christopher Faylor wrote: One question, though. Assuming start is == size, then the current code in CVS extends the fd table by only 1. If that happens often, the current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't it in fact be better to extend always by at least NOFILE_INCR, and to extend by (1 + start - size) only if start is size + NOFILE_INCR? Something like size_t extendby = (start = size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR; Always increasing by a minimum of NOFILE_INCR is wrong in one case - we should never increase beyond OPEN_MAX_MAX (currently 3200). dup2(0, 3199) should succeed (unless it fails with EMFILE due to rlimit, but we already know that our handling of setrlimit(RLIMIT_NOFILE) is still a bit awkward); but dup2(0, 3200) must always fail with EBADF. I think the code in CVS is still wrong: we want to increase to the larger of the value specified by the user or NOFILE_INCR to minimize repeated calloc, but we also need to cap the increase to be at most OPEN_MAX_MAX descriptors, to avoid having a table larger than what the rest of our code base will support. I made some more changes to CVS. Incidentally did you catch the fact that you broke how this worked in 1.7.26? You were taking a MAX of a signed and unsigned quantity so the signed quantity was promoted to a huge positive number. Not having NOFILE_INCR free slots after a user allocation is not fatal; No one implied it was. it means that the first allocation to a large number will not have tail padding, but the next allocation to fd+1 will allocate NOFILE_INCR slots rather than just one. My original idea of MAX(NOFILE_INCR, start - size) expresses that. That wasn't Corinna's concern. My replacement code would have called calloc for every one of: dup2(0, 32); dup2(1, 33); dup2(2, 34); Obviously there are different ways to avoid this and I chose to extend the table after the start location. That might be helpful. Tcsh, for instance, always dup's it's std descriptors to the new fds 15-19. If it does so in this order, it would have to call extend 5 times. dtable.h:#define NOFILE_INCR32 It shouldn't extend in that scenario. The table starts with 32 elements. Rather, the table starts with 256 elements; which is why dup2 wouldn't crash until dup'ing to 256 or greater before I started touching this. The table is initialized in dtable_init() with 32 elements. When it enters main, it is still 32 elements, at least according to cygheap-fdtab.size. I just checked this with gdb. cgf
Re: fix off-by-one in dup2
On Wed, Dec 04, 2013 at 01:04:08PM +0100, Corinna Vinschen wrote: On Dec 4 12:36, Corinna Vinschen wrote: On Dec 4 10:32, Corinna Vinschen wrote: Hi guys, [...etc...] The problem is still present in the current sources. [...] Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes this problem. Duh. Sorry for the confusion. One question, though. Assuming start is == size, then the current code in CVS extends the fd table by only 1. If that happens often, the current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't it in fact be better to extend always by at least NOFILE_INCR, and to extend by (1 + start - size) only if start is size + NOFILE_INCR? Something like size_t extendby = (start = size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR; ? Sorry again. Fortunately it's my WJM week... I don't think it is a common occurrence for start = size. It is usually done when something like bash dup2's stdin/stdout/stderr to a high fd. Howeer, I'll check in something which guarantees that there is always a NOFILE_INCR entries free after start. cgf
Re: fix off-by-one in dup2
On Dec 4 12:00, Christopher Faylor wrote: On Wed, Dec 04, 2013 at 01:04:08PM +0100, Corinna Vinschen wrote: On Dec 4 12:36, Corinna Vinschen wrote: On Dec 4 10:32, Corinna Vinschen wrote: Hi guys, [...etc...] The problem is still present in the current sources. [...] Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes this problem. Duh. Sorry for the confusion. One question, though. Assuming start is == size, then the current code in CVS extends the fd table by only 1. If that happens often, the current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't it in fact be better to extend always by at least NOFILE_INCR, and to extend by (1 + start - size) only if start is size + NOFILE_INCR? Something like size_t extendby = (start = size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR; ? Sorry again. Fortunately it's my WJM week... I don't think it is a common occurrence for start = size. It is usually done when something like bash dup2's stdin/stdout/stderr to a high fd. Howeer, I'll check in something which guarantees that there is always a NOFILE_INCR entries free after start. That might be helpful. Tcsh, for instance, always dup's it's std descriptors to the new fds 15-19. If it does so in this order, it would have to call extend 5 times. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpbzJcH0cDuZ.pgp Description: PGP signature
Re: fix off-by-one in dup2
On Wed, Dec 04, 2013 at 06:23:24PM +0100, Corinna Vinschen wrote: On Dec 4 12:00, Christopher Faylor wrote: On Wed, Dec 04, 2013 at 01:04:08PM +0100, Corinna Vinschen wrote: On Dec 4 12:36, Corinna Vinschen wrote: On Dec 4 10:32, Corinna Vinschen wrote: Hi guys, [...etc...] The problem is still present in the current sources. [...] Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes this problem. Duh. Sorry for the confusion. One question, though. Assuming start is == size, then the current code in CVS extends the fd table by only 1. If that happens often, the current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't it in fact be better to extend always by at least NOFILE_INCR, and to extend by (1 + start - size) only if start is size + NOFILE_INCR? Something like size_t extendby = (start = size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR; ? Sorry again. Fortunately it's my WJM week... I don't think it is a common occurrence for start = size. It is usually done when something like bash dup2's stdin/stdout/stderr to a high fd. Howeer, I'll check in something which guarantees that there is always a NOFILE_INCR entries free after start. That might be helpful. Tcsh, for instance, always dup's it's std descriptors to the new fds 15-19. If it does so in this order, it would have to call extend 5 times. dtable.h:#define NOFILE_INCR32 It shouldn't extend in that scenario. The table starts with 32 elements. cgf
Re: fix off-by-one in dup2
On 10/15/2013 08:06 AM, Christopher Faylor wrote: On Wed, Sep 25, 2013 at 05:26:25PM -0600, Eric Blake wrote: Solves the segfault here: http://cygwin.com/ml/cygwin/2013-09/msg00397.html but does not address the fact that we are still screwy with regards to rlimit. Corinna reminded me about this. Sorry for the delay in responding. I was investigating if setdtablesize should set an errno on error but it is difficult to say if it should since it seems not to be a POSIX or Linux. So, I guess we can just say that it should set EINVAL. Would you mind making that minor change and checking this in? Yikes, I still haven't done this (and was reminded by today's announcement to test snapshots). I'll try to get to it pronto. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
