Creating and running tests for a snapcraft plugin

[Joseph Rushton Wakeling]

Hello all,

I have started working on a snapcraft plugin for dub, a build/package manager 
for the D programming language.  The plugin itself is ready (at least in a 
working draft), and now I'm working on the tests.


Is there a simple way to run the tests for a single plugin, rather than the 
whole body of tests?  The snapcraft HACKING.md doesn't offer any advice on this 
point.


I'm not much of a pythonist, so apologies if this is a daft question and the 
answer is as simple as `python3 snapcraft/tests/test_dub.py` (but currently at 
least, even trying that fails with an ImportError, so I assume there's something 
else I have to do).


Thanks & best wishes,

-- Joe

--
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: systemd-resolved and snaps

[Steve Langasek]

Hi Kyle,

On Thu, Feb 23, 2017 at 01:58:07PM -0800, Kyle Fazzari wrote:
> Hey all.

> I've received a bug report on a snap where the user was running a 16.10
> Server install with the snap in question, and getting DNS errors. I've
> distilled the problem as much as I can but I cannot for the life of me
> figure out what's happening, so I thought maybe the list could point me
> in the right direction.

> Prerequisites
> =
> 
> I have a demo snap (a standalone snapcraft.yaml) that will demonstrate
> this issue[1].
> 
> Ubuntu 16.10 Server uses systemd-resolved, which means its
> /etc/resolv.conf contains a single nameserver:
> 
> nameserver 127.0.0.53
> 
> If you have others there, comment them out for the time being.
> 
> 
> Steps to reproduce
> ==
> 
> 1. Build and install the `resolved-test` snap[1]. It exposes two apps,
> `test` (which is a python2 script uses the requests lib) and `host`
> which is just the `host` utility from bind9-host.
> 
> 2. With 127.0.0.53 as the only nameserver, run `resolved-test.test`.
> Note that it fails with "Name or service not known."

acme-staging.api.letsencrypt.org is a CNAME.  You are hitting bug #1647031,
which we encountered when trying to roll out systemd-resolved by default for
17.04.  This took a while to work through, but the fix has finally landed in
zesty as of a week ago; we should now SRU the upstream change back to
yakkety.  (We should also SRU it back to xenial, but xenial needs a more
complete backport of fixes to resolved, not just a cherry-pick of this one
fix.)

Dimitri, could you handle this backport to yakkety?  Since unlike the
Desktop, Ubuntu Server does not use dnsmasq by default (which would override
resolved), this is a rather important bug there.



-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: PGP signature
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: systemd-resolved and snaps

[Stéphane Graber]

On Thu, Feb 23, 2017 at 01:58:07PM -0800, Kyle Fazzari wrote:
> Hey all.
> 
> I've received a bug report on a snap where the user was running a 16.10
> Server install with the snap in question, and getting DNS errors. I've
> distilled the problem as much as I can but I cannot for the life of me
> figure out what's happening, so I thought maybe the list could point me
> in the right direction.
> 
> 
> Prerequisites
> =
> 
> I have a demo snap (a standalone snapcraft.yaml) that will demonstrate
> this issue[1].
> 
> Ubuntu 16.10 Server uses systemd-resolved, which means its
> /etc/resolv.conf contains a single nameserver:
> 
> nameserver 127.0.0.53
> 
> If you have others there, comment them out for the time being.
> 
> 
> Steps to reproduce
> ==
> 
> 1. Build and install the `resolved-test` snap[1]. It exposes two apps,
> `test` (which is a python2 script uses the requests lib) and `host`
> which is just the `host` utility from bind9-host.
> 
> 2. With 127.0.0.53 as the only nameserver, run `resolved-test.test`.
> Note that it fails with "Name or service not known."
> 
> 3. With 127.0.0.53 as the only nameserver, run `resolved-test.host
> acme-staging.api.letsencrypt.org`. Note that it works perfectly fine.
> 
> At this point, one might suspect that this is an issue in the requests
> lib. So:
> 
> 4. Run the test from outside the snap:
> `PYTHONPATH=/snap/resolved-test/current/lib/python2.7/site-packages
> /snap/resolved-test/current/usr/bin/python
> /snap/resolved-test/current/bin/test.py`. Note that this works fine (you
> should see a dict printed).
> 
> 5. Now add another nameserver to /etc/resolv.conf (I used `nameserver
> 8.8.8.8`), and run `resolved-test.test`. Note that it works fine.
> 
> 
> Some analysis
> =
> 
> Note that when I run step (1) I get [2] in the syslog. When I run step
> (2) I get [3] in the syslog. When I run step (4) I get nothing in the
> syslog.
> 
> I'm at a loss here. Installing with devmode doesn't change these
> results. `host` always seems to work fine, but `test` never works with
> the systemd-resolved IP address. Has anyone else encountered this? Any
> advice would be appreciated.
> 
> [1]: http://pastebin.ubuntu.com/24055388/
> [2]: http://pastebin.ubuntu.com/24055470/
> [3]: http://pastebin.ubuntu.com/24055466/

Hey there,

Since acme-staging.api.letsencrypt.org is a CNAME, that may be related to:
 - https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1647031
 - https://github.com/systemd/systemd/issues/3826

What happens if you try with "host letsencrypt.org" and
"https://letsencrypt.org; (that's an A++MX instead of CNAME)?


-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com


signature.asc
Description: PGP signature
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

systemd-resolved and snaps

[Kyle Fazzari]

Hey all.

I've received a bug report on a snap where the user was running a 16.10
Server install with the snap in question, and getting DNS errors. I've
distilled the problem as much as I can but I cannot for the life of me
figure out what's happening, so I thought maybe the list could point me
in the right direction.


Prerequisites
=

I have a demo snap (a standalone snapcraft.yaml) that will demonstrate
this issue[1].

Ubuntu 16.10 Server uses systemd-resolved, which means its
/etc/resolv.conf contains a single nameserver:

nameserver 127.0.0.53

If you have others there, comment them out for the time being.


Steps to reproduce
==

1. Build and install the `resolved-test` snap[1]. It exposes two apps,
`test` (which is a python2 script uses the requests lib) and `host`
which is just the `host` utility from bind9-host.

2. With 127.0.0.53 as the only nameserver, run `resolved-test.test`.
Note that it fails with "Name or service not known."

3. With 127.0.0.53 as the only nameserver, run `resolved-test.host
acme-staging.api.letsencrypt.org`. Note that it works perfectly fine.

At this point, one might suspect that this is an issue in the requests
lib. So:

4. Run the test from outside the snap:
`PYTHONPATH=/snap/resolved-test/current/lib/python2.7/site-packages
/snap/resolved-test/current/usr/bin/python
/snap/resolved-test/current/bin/test.py`. Note that this works fine (you
should see a dict printed).

5. Now add another nameserver to /etc/resolv.conf (I used `nameserver
8.8.8.8`), and run `resolved-test.test`. Note that it works fine.


Some analysis
=

Note that when I run step (1) I get [2] in the syslog. When I run step
(2) I get [3] in the syslog. When I run step (4) I get nothing in the
syslog.

I'm at a loss here. Installing with devmode doesn't change these
results. `host` always seems to work fine, but `test` never works with
the systemd-resolved IP address. Has anyone else encountered this? Any
advice would be appreciated.

[1]: http://pastebin.ubuntu.com/24055388/
[2]: http://pastebin.ubuntu.com/24055470/
[3]: http://pastebin.ubuntu.com/24055466/

-- 
Kyle Fazzari (kyrofa)
Software Engineer
Canonical Ltd.
k...@canonical.com



signature.asc
Description: OpenPGP digital signature
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Further ABI-related issues for ldc2 snap on 14.04 ... ?

[Joseph Rushton Wakeling]

Hello all,

I've encountered a couple of further issues with the ldc2 compiler snap when 
it's installed on a 14.04 system.  I'm sharing here to confirm if these are 
indeed ABI issues which might be fixable in terms of how snapcraft constructs 
the package.


The first [1] relates to zlib, which is included as part of the D standard 
library.  This is built from a copy of the zlib C source and linked in to the 
rest of the library.


However, if I try to build a program that makes use of the zlib library modules, 
linking fails with the following error:


/usr/bin/gcc zlibtest.o -o zlibtest -L/snap/ldc2/x1/bin/../lib -lphobos2-ldc 
-ldruntime-ldc -Wl,--gc-sections -lrt -ldl -lpthread -lm -m64
/usr/bin/ld: /snap/ldc2/x1/bin/../lib/libphobos2-ldc.a(zutil.c.o): unrecognized 
relocation (0x2a) in section `.text'

/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
Error: /usr/bin/gcc failed with status: 1

This matches the error message observed in a past Ubuntu issue related to 
binutils: https://bugs.launchpad.net/ubuntu/+source/binutils/+bug/1545653


which would suggest that there's a problematic mismatch between the binutils 
used to build snap packages versus the one available on 14.04.


The second issue [2] relates to a plugin for the gold linker that is built for 
LDC to enable it to use link-time optimization.  This works fine on 16.04, 16.10 
and 17.04, but requesting LTO on 14.04 results in linking failing:


/usr/bin/gcc hello.o -o hello -Xlinker -plugin -Xlinker 
/snap/ldc2/x1/lib/LLVMgold-ldc.so -Xlinker -plugin-opt=O0 
-L/snap/ldc2/x1/bin/../lib -lphobos2-ldc -ldruntime-ldc -Wl,--gc-sections -lrt 
-ldl -lpthread -lm -m64
/usr/bin/ld: /snap/ldc2/x1/lib/LLVMgold-ldc.so: error loading plugin: 
/snap/core/current/lib/x86_64-linux-gnu/libpthread.so.0: symbol __libc_vfork, 
version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference

collect2: error: ld returned 1 exit status
Error: /usr/bin/gcc failed with status: 1

... which looks to me like it is probably a mismatch between the libc used to 
build the plugin, versus the libc of either the host system or the core snap?


I would presume that both the above would need a fix either snapcraft-side or 
core snap side ... ?  Can anyone advise/assist?


Thanks & best wishes,

-- Joe

[1] https://github.com/ldc-developers/ldc2.snap/issues/17
[2] https://github.com/ldc-developers/ldc2.snap/issues/18

--
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Position-independent code and Ubuntu 16.10

[Joseph Rushton Wakeling]

On 22/02/17 00:39, Joseph Rushton Wakeling wrote:

OK, fair enough.  In my case I think it was the C-library parts of the D
standard library that were being compiled without PIC.  Seems OK to assume this
may have been a project-specific thing, then (which is now fixed upstream).


Just for the record: this issue is fixed in the most recent revision of the 
package.

Seth, thanks for your help and advice!


--
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Experimental Python interpreter snap

[Stuart Bishop]

On 23 February 2017 at 14:45, James Henstridge
 wrote:

> So if I installed a package to $SNAP_USER_DATA for my
> "python36-jamesh.python3" interpreter, the files would end up
> somewhere under ~/snap/python36-jamesh/.
>
> If we then look at my simple hello-world example snap that uses the
> content interface to access the interpreter, $SNAP_USER_DATA now
> points to a location under ~/snap/hello-world/.  So it wouldn't see
> the additional packages installed for "python36-jamesh.python3".  In
> fact, the hello-world snap doesn't even have permission to read files
> under ~/snap/python36-jamesh, even if I put that directory on
> sys.path.

Yes, exactly. If I use the python36-jamesh.python3 interpreter from my
main shell, I can pip install libraries into $SNAP_USER_DATA and use
them. They will not be seen or cause conflicts if I use the
/usr/bin/python3, which is what I'm suggesting is an improvement. And
they will not be seen by or cause conflicts with snaps using the
content interface, no matter if they are confined or classic, which is
also a good thing because they will be embedding their dependencies.

-- 
Stuart Bishop 

-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft