Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-04 Thread Philip Boampong
On Wed, Mar 4, 2020 at 4:09 PM Ian Lance Taylor wrote: > > The links that I looked at were speculating about behavior, not > demonstrating real behavior in kernels. I don't know how to judge > whether those speculations about hypothetical behavior are correct or > not. Many things are hypothetic

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-04 Thread Ian Lance Taylor
On Wed, Mar 4, 2020 at 6:20 AM Philip Boampong wrote: > > On Sun, Mar 1, 2020 at 3:55 PM Ian Lance Taylor wrote: > > > > If dup2 can 1) close newfd; 2) receive a signal before duping oldfd to > > newfd; 3) return EINTR leaving newfd closed, then dup2 requires > > considerable care in any multi-th

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-04 Thread Philip Boampong
Thanks for the feedback. > "If the close operation fails to close fildes2, dup2() shall return -1 > without changing the open file description to which fildes2 refers." > "If close() is interrupted by a signal that is to be caught, it shall return > -1 with errno set to [EINTR] and the state of

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-04 Thread Philip Boampong
On Sun, Mar 1, 2020 at 3:55 PM Ian Lance Taylor wrote: > > If dup2 can 1) close newfd; 2) receive a signal before duping oldfd to > newfd; 3) return EINTR leaving newfd closed, then dup2 requires > considerable care in any multi-threaded program. It requires that if > one thread is calling dup2,

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-01 Thread Uli Kunitz
Ian, It is unclear how to interpret the POSIX specification regarding dup2 returning EINTR. The POSIX specification [1] of dup2 says: "If the close operation fails to close *fildes2*, *dup2*() shall return -1 without changing the open file description to which *fildes2* refers." It appears tha

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-01 Thread Michael Jones
...and if true, this notion of “it’s atomic unless it’s not” deserves a name: the “subatomic operation.” On Sun, Mar 1, 2020 at 6:56 AM Ian Lance Taylor wrote: > On Sat, Feb 29, 2020 at 9:45 PM Philip Boampong > wrote: > > > > > And dup2 is not documented to return EINTR > > > > Yes, it is. In

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-01 Thread Ian Lance Taylor
On Sat, Feb 29, 2020 at 9:45 PM Philip Boampong wrote: > > > And dup2 is not documented to return EINTR > > Yes, it is. In the same man page you quoted [1]. > | EINTR The dup2() or dup3() call was interrupted by a signal; see > | signal(7). > > > and the Linux kernel code shown above doe

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-03-01 Thread Uli Kunitz
Thanks, now I understand the concerns better. For Linux I stay with my remark, if you get an error from dup2 nothing has happened. But this is not true for POSIX, which requires the error of the close to be reported. Linux doesn't do it, if you look carefully at the code that I have sent, the r

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Philip Boampong
On Sun, Mar 1, 2020 at 3:06 AM Ian Lance Taylor wrote: > > > But if you opened newfd yourself, or if it is 0/1/2 and you never > > closed os.Std*, then you *can* dup2 safely, regardless of other > > packages. > > Those are examples where you are in charge of the FD namespace > (assuming you know t

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Kurtis Rader
On Sat, Feb 29, 2020 at 7:35 AM Uli Kunitz wrote: > My reading of the Linux kernel sources (5.3) is that dup2 will never > return -EINTR. Any necessary file closure will happen, but its return value > will be ignored. > That a specific implementation might never return EINTR as a result of a `du

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Kurtis Rader
In addition to all of the other points that have been made by Ian and others I think it is important to reinforce another point. On UNIX like systems every function call that returns a file descriptor (e.g., `open()` and `socket()`) is expected to return the lowest unused file descriptor. If the se

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Ian Lance Taylor
On Sat, Feb 29, 2020 at 12:16 PM Philip Boampong wrote: > > Can you really do that? I don't think the standard library guarantees > that it will not create a new FD behind the scenes tomorrow (nor it > exactly documents its FD usage and timing). I forgot to reply to this point. The Go standard l

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Ian Lance Taylor
On Sat, Feb 29, 2020 at 12:16 PM Philip Boampong wrote: > > On Sat, Feb 29, 2020 at 1:34 PM Ian Lance Taylor wrote: > > > > It does not make sense to use dup2 if you are not in control of the FD > > namespace. In order to use dup2 you need to specify the new FD. If > > that FD might be concurre

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Philip Boampong
> If you are getting an error nothing has happened, no replacement of newfd and > no close I wish that sentence was written on the man page. That was the way I first understood it too (and it makes more sense) but the little information I found disagree (libuv [1], python [2] (see the note about

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Uli Kunitz
On Saturday, February 29, 2020 at 9:32:11 PM UTC+1, Philip Boampong wrote > > Whether it cannot harm is what I'm trying to find out. > If newfd gets closed then a retry loop is racy, see my previous messages. > > [1] > https://stackoverflow.com/questions/23440216/race-condition-when-using-dup2

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Philip Boampong
On Sat, Feb 29, 2020 at 9:13 PM Brian Candler wrote: > > I don't quite follow. If two threads are fighting to use the target fd, then > that's just a race anyway. One case is when you have full control of the FD namespace, then you can rely on your own synchronization and do whatever you want,

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Philip Boampong
On Sat, Feb 29, 2020 at 4:35 PM Uli Kunitz wrote: > > My reading of the Linux kernel sources (5.3) is that dup2 will never return > -EINTR. Thanks, good to know. > But you will have to program the loop around the syscall anyway, because > Linux may return EBUSY, which happens if the new fd has

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Philip Boampong
On Sat, Feb 29, 2020 at 1:34 PM Ian Lance Taylor wrote: > > It does not make sense to use dup2 if you are not in control of the FD > namespace. In order to use dup2 you need to specify the new FD. If > that FD might be concurrently opened by some other package, or by the > runtime, then you can

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Brian Candler
On Saturday, 29 February 2020 12:37:24 UTC, Ian Lance Taylor wrote: > > On Sat, Feb 29, 2020 at 12:33 AM Brian Candler > wrote: > > > > Just to ask the an obvious question: is dup2() idempotent or not? > > dup2 in itself is idempotent. But I'm not sure that is a useful > question. I think i

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Uli Kunitz
My reading of the Linux kernel sources (5.3) is that dup2 will never return -EINTR. Any necessary file closure will happen, but its return value will be ignored. But you will have to program the loop around the syscall anyway, because Linux may return EBUSY, which happens if the new fd has been

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Ian Lance Taylor
On Sat, Feb 29, 2020 at 12:33 AM Brian Candler wrote: > > Just to ask the an obvious question: is dup2() idempotent or not? dup2 in itself is idempotent. But I'm not sure that is a useful question. The issue is whether some other thread in the same process can open a file at the target file des

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Ian Lance Taylor
On Fri, Feb 28, 2020 at 8:27 PM Philip Boampong wrote: > > The Go programmer is not fully in charge of the FD namespace: > libraries and the runtime can create new FDs at any time. Therefore, > unless you are sure that newfd *is* in use and know exactly what it > is, you are probably looking for t

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-29 Thread Brian Candler
Just to ask the an obvious question: is dup2() idempotent or not? > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. T

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-28 Thread Philip Boampong
Thanks for the reply. > Note that dup2() can only fail with EINTR if the new fd is currently open on > a "slow" device and the implicit close() fails due to being interrupted. I understand the condition may be rare, but I still want to know the correct way to handle it. > In my experience it is

Re: [go-nuts] How to handle EINTR from syscall.Dup2

2020-02-28 Thread Kurtis Rader
On Fri, Feb 28, 2020 at 12:41 PM wrote: > What to do on EINTR from syscall.Dup2 (Linux)? > > 1) It never happen. > 2) Retry. > 3) Take it as irrecoverable. > 4) Take it as success. > > I know this is more of an OS question, but it all started with the > asynchronous preemption announcement, and I

[go-nuts] How to handle EINTR from syscall.Dup2

2020-02-28 Thread pboampong5
(I've asked the same question already, but probably in the wrong thread, sorry for the repost.) What to do on EINTR from syscall.Dup2 (Linux)? 1) It never happen. 2) Retry. 3) Take it as irrecoverable. 4) Take it as success. I know this is more of an OS question, but it all started with the as