Re: pkg_add: signify broken pipe when running from script

[Marc Espie]

On Mon, Nov 21, 2016 at 06:26:39PM +0100, Patrik Lundin wrote:
> On Sun, Nov 20, 2016 at 12:09:23PM +0100, lvdd wrote:
> > 
> > thank you Patrik. I just wanted to report that this solved the
> > problem I am having with this since 1.5 months as well. Please read
> > http://marc.info/?l=openbsd-misc=147544336719598=2 for details.
> > 
> > I know this is not a final solution but you seem to have a deeper
> > understanding of this than I have.
> > 
> 
> Nice to know it was be helpful for someone else, I was only
> following up on the hunch from Espie though :).
> 
> Marc: Is this somehing that would be suitable to commit to pkg_add or
> some related library? If SIGPIPE has become more important since the
> added support for signed gzip headers (which i guess is related to
> this) maby it makes sense to initialize the handler to a known state?
> 
> Can you repeat the problem using my python snippet for reference?

It's on my list to have a much closer look, because I still need to figure
out exactly whether there is some deeper problem lurking.

Re: pkg_add: signify broken pipe when running from script

[Patrik Lundin]

On Sun, Nov 20, 2016 at 12:09:23PM +0100, lvdd wrote:
> 
> thank you Patrik. I just wanted to report that this solved the
> problem I am having with this since 1.5 months as well. Please read
> http://marc.info/?l=openbsd-misc=147544336719598=2 for details.
> 
> I know this is not a final solution but you seem to have a deeper
> understanding of this than I have.
> 

Nice to know it was be helpful for someone else, I was only
following up on the hunch from Espie though :).

Marc: Is this somehing that would be suitable to commit to pkg_add or
some related library? If SIGPIPE has become more important since the
added support for signed gzip headers (which i guess is related to
this) maby it makes sense to initialize the handler to a known state?

Can you repeat the problem using my python snippet for reference?

-- 
Patrik Lundin

Re: pkg_add: signify broken pipe when running from script

[lvdd]

Hi,

On Sat, 19 Nov 2016 19:58:59 +0100
Patrik Lundin  wrote:

> On Sat, Nov 19, 2016 at 06:27:16PM +0100, Marc Espie wrote:
> > Wondering if your script isn't passing some weird signal handling
> > (such as ignoring some signals) through.
> >   
> 
> Seems like you are on to something :).
> 
> Some googling led me here:
> https://blog.nelhage.com/2010/02/a-very-subtle-bug/
> 
> It talks about how python ignores SIGPIPE and shows how it can be
> worked around when using subprocess.Popen.
> 
> I can solve the problem by adding the following line
> in /usr/sbin/pkg_add: 
> ===
> $SIG{PIPE} = 'DEFAULT';
> ===
> 

thank you Patrik. I just wanted to report that this solved the
problem I am having with this since 1.5 months as well. Please read
http://marc.info/?l=openbsd-misc=147544336719598=2 for details.

I know this is not a final solution but you seem to have a deeper
understanding of this than I have.

regards
Lars 

Re: pkg_add: signify broken pipe when running from script

[Patrik Lundin]

On Sat, Nov 19, 2016 at 06:27:16PM +0100, Marc Espie wrote:
> Wondering if your script isn't passing some weird signal handling (such as
> ignoring some signals) through.
> 

Seems like you are on to something :).

Some googling led me here:
https://blog.nelhage.com/2010/02/a-very-subtle-bug/

It talks about how python ignores SIGPIPE and shows how it can be worked
around when using subprocess.Popen.

I can solve the problem by adding the following line in /usr/sbin/pkg_add:
===
$SIG{PIPE} = 'DEFAULT';
===

-- 
Patrik Lundin

Re: pkg_add: signify broken pipe when running from script

[Marc Espie]

Wondering if your script isn't passing some weird signal handling (such as
ignoring some signals) through.

I've had a few reports related to signify broken pipes, but I haven't been
able to reproduce anything, so I'm not quite up to debugging problems I can't
see...

Re: pkg_add: signify broken pipe when running from script

[Patrik Lundin]

On Sat, Nov 19, 2016 at 03:55:07PM +0100, Patrik Lundin wrote:
> Hello,
> 
> I have noticed that pkg_add will behave differently when being executed
> from a script when compared to running it manually on the command line.
> 

I forgot to mention that all the testing is done on a fresh -current
machine.

-- 
Patrik Lundin

pkg_add: signify broken pipe when running from script

[Patrik Lundin]

Hello,

I have noticed that pkg_add will behave differently when being executed
from a script when compared to running it manually on the command line.

This is creating problems for the ansible openbsd_pkg module.

This is what I see, first running manually which is fine:
===
# pkg_add nmap
quirks-2.270 signed on 2016-11-18T14:37:13Z
nmap-7.30: ok

# pkg_add nmap
quirks-2.270 signed on 2016-11-18T14:37:13Z
#
===

Deleting nmap again to start over:
===
# pkg_delete nmap
nmap-7.30: ok
Read shared items: ok
===

Now executing pkg_add from python:
===
# ./run_pkg_add.py
quirks-2.270 signed on 2016-11-18T14:37:13Z
nmap-7.30: ok

# ./run_pkg_add.py
quirks-2.270 signed on 2016-11-18T14:37:13Z
Error from 
http://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/packages/amd64/nmap-7.30.tgz
signify: write to stdout: Broken pipe
#
===

Note that the first run is successfull, while the second run throws a
broken pipe error from signify.

The python code:
===
#!/usr/bin/env python

import shlex
import subprocess

command = "pkg_add nmap"

args = shlex.split(command);

subprocess.call(args)
===

This is python-2.7.12p1.

I have been doing some print-debugging in
/usr/libdata/perl5/OpenBSD/PackageRepository.pm trying to figure out
what is going on, but so far no luck.

-- 
Patrik Lundin