Re: some console-conf start-up traceback

[Michael Hudson-Doyle]

Hi Nicolino,

Glad to hear it!

Cheers,
mwh

On 1 April 2017 at 20:14, Nicolino Curalli <n.cura...@domotz.com> wrote:

> Hi Michael,
> the current snap core on edge channel work fine.
> The problem that I report  disappeared, then I can confirm from my side
> that the bug is fixed.
>
> Cheers,
> Nicolino.
>
>
>
>
>
> Il 29/03/2017 22:00, Michael Hudson-Doyle ha scritto:
> >
> > This should be fixed in the latest edge version of the core snap. Let me
> > know how it goes if you try it!
> >
> > Cheers,
> > mwh
>
>
>
> --
> Snapcraft mailing list
> Snapcraft@lists.snapcraft.io
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/snapcraft
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: some console-conf start-up traceback

[Michael Hudson-Doyle]

On 28 March 2017 at 15:58, Michael Hudson-Doyle <
michael.hud...@canonical.com> wrote:

>
>
> On 28 March 2017 at 10:38, Michael Hudson-Doyle <
> michael.hud...@canonical.com> wrote:
>
>>
>>
>> On 28 March 2017 at 02:50, Nicolino Curalli <n.cura...@domotz.com> wrote:
>>
>>> Hi all,
>>>
>>> we have a strange output on console after the first boot: the first boot
>>> go straight without any traceback.
>>>
>>> We use our kernel snap on a orangepi plus 2e board (kernel 4.9).
>>>
>>> The traceback is the following:
>>>
>>>
>>> [  OK  ] Reached target Login Prompts.
>>> [  OK  ] Started OpenBSD Secure Shell server.
>>> [  OK  ] Reached target Multi-User System.
>>> [  OK  ] Reached target Graphical Interface.
>>>  Starting Update UTMP about System Runlevel Changes...
>>> [  OK  ] Started Update UTMP about System Runlevel Changes.
>>>
>>> Traceback (most recent call last):
>>>   File "/usr/share/subiquity/console-conf-write-login-details", line
>>> 21, in 
>>> sys.exit(write_login_details_standalone())
>>>   File "/usr/share/subiquity/console_conf/controllers/identity.py",
>>> line 136, in write_login_details_standalone
>>> write_login_details(sys.stdout, owner['username'], ips)
>>>   File "/usr/share/subiquity/console_conf/controllers/identity.py",
>>> line 115, in write_login_details
>>> sshcommands=sshcommands, host_key_info=host_key_info(),
>>> tty_name=tty_name, first_ip=ips[0]))
>>> IndexError: list index out of range
>>>
>>
>> Ah, this would be a bug. The script that generates the "ssh
>> domotz@192.168.5.107" page is clearly running before the system gets an
>> IP address (via DHCP I guess?) and crash looping until the system does have
>> an IP address.
>>
>> I'll see what I can come up with to fix this.
>>
>
> Hi again, this should be fixed by https://github.com/
> CanonicalLtd/subiquity/pull/209 (which I don't expect you to try, I'll
> let you know when it reaches the core snap in the edge channel). By far the
> hardest part of this was figuring out a way to test it!
>
> Thanks for the report.
>

This should be fixed in the latest edge version of the core snap. Let me
know how it goes if you try it!

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: some console-conf start-up traceback

[Michael Hudson-Doyle]

On 28 March 2017 at 10:38, Michael Hudson-Doyle <
michael.hud...@canonical.com> wrote:

>
>
> On 28 March 2017 at 02:50, Nicolino Curalli <n.cura...@domotz.com> wrote:
>
>> Hi all,
>>
>> we have a strange output on console after the first boot: the first boot
>> go straight without any traceback.
>>
>> We use our kernel snap on a orangepi plus 2e board (kernel 4.9).
>>
>> The traceback is the following:
>>
>>
>> [  OK  ] Reached target Login Prompts.
>> [  OK  ] Started OpenBSD Secure Shell server.
>> [  OK  ] Reached target Multi-User System.
>> [  OK  ] Reached target Graphical Interface.
>>  Starting Update UTMP about System Runlevel Changes...
>> [  OK  ] Started Update UTMP about System Runlevel Changes.
>>
>> Traceback (most recent call last):
>>   File "/usr/share/subiquity/console-conf-write-login-details", line 21,
>> in 
>> sys.exit(write_login_details_standalone())
>>   File "/usr/share/subiquity/console_conf/controllers/identity.py", line
>> 136, in write_login_details_standalone
>> write_login_details(sys.stdout, owner['username'], ips)
>>   File "/usr/share/subiquity/console_conf/controllers/identity.py", line
>> 115, in write_login_details
>> sshcommands=sshcommands, host_key_info=host_key_info(),
>> tty_name=tty_name, first_ip=ips[0]))
>> IndexError: list index out of range
>>
>
> Ah, this would be a bug. The script that generates the "ssh
> domotz@192.168.5.107" page is clearly running before the system gets an
> IP address (via DHCP I guess?) and crash looping until the system does have
> an IP address.
>
> I'll see what I can come up with to fix this.
>

Hi again, this should be fixed by
https://github.com/CanonicalLtd/subiquity/pull/209 (which I don't expect
you to try, I'll let you know when it reaches the core snap in the edge
channel). By far the hardest part of this was figuring out a way to test it!

Thanks for the report.

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: some console-conf start-up traceback

[Michael Hudson-Doyle]

On 28 March 2017 at 02:50, Nicolino Curalli  wrote:

> Hi all,
>
> we have a strange output on console after the first boot: the first boot
> go straight without any traceback.
>
> We use our kernel snap on a orangepi plus 2e board (kernel 4.9).
>
> The traceback is the following:
>
>
> [  OK  ] Reached target Login Prompts.
> [  OK  ] Started OpenBSD Secure Shell server.
> [  OK  ] Reached target Multi-User System.
> [  OK  ] Reached target Graphical Interface.
>  Starting Update UTMP about System Runlevel Changes...
> [  OK  ] Started Update UTMP about System Runlevel Changes.
>
> Traceback (most recent call last):
>   File "/usr/share/subiquity/console-conf-write-login-details", line 21,
> in 
> sys.exit(write_login_details_standalone())
>   File "/usr/share/subiquity/console_conf/controllers/identity.py", line
> 136, in write_login_details_standalone
> write_login_details(sys.stdout, owner['username'], ips)
>   File "/usr/share/subiquity/console_conf/controllers/identity.py", line
> 115, in write_login_details
> sshcommands=sshcommands, host_key_info=host_key_info(),
> tty_name=tty_name, first_ip=ips[0]))
> IndexError: list index out of range
>

Ah, this would be a bug. The script that generates the "ssh
domotz@192.168.5.107" page is clearly running before the system gets an IP
address (via DHCP I guess?) and crash looping until the system does have an
IP address.

I'll see what I can come up with to fix this.

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Part 2! Request for help / ideas to debug issue

[Michael Hudson-Doyle]

On 14 March 2017 at 23:56, John Lenton  wrote:

> As a followup, I added a mutex around pthread_create, and around the
> exec syscall, and the problem went away. This all in go's runtime; not
> a huge diff but they probably don't want the overhead (and that seems
> reasonable to me).
> Next I'm going to try to find a kernel person to take a look at this.
>

Having looked at the kernel a bit, I'm not sure that a mutex is that crazy
an idea :-) Does it impact performance in any noticeable way? clone is
fairly expensive I guess and Go programs only run ~GOMAXPROCS OS threads
anyway.  Would be interested in what any proper people you can round up
would have to say though.

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Part 2! Request for help / ideas to debug issue

[Michael Hudson-Doyle]

On 14 March 2017 at 12:21, John Lenton <john.len...@canonical.com> wrote:

> On 13 March 2017 at 21:05, Michael Hudson-Doyle
> <michael.hud...@canonical.com> wrote:
> > If I add a
> > time.Sleep(1*time.Millisecond) to a_go.go before the exec, the setuid bit
> > is respected every time.
>
> on my way to bed, I'll give your response a proper read in the
> morning, but note that my reproducer causes the issue a lot more
> frequently than in "the real world" (snap run calling snap-confine
> calling snap-exec), where delays are happening naturally (because the
> programs aren't just calling exec, they actually have work to do :-p).
> I don't have numbers for how often it happens in snappy; it's a lot
> less frequent, but often enough to be annoying when working
> interactively (and there are bug reports with these warnings/errors in
> lp already).


Oh yes, the sleep was just to allow the other threads to settle in the test
case program. It's not a solution for the real world at all.

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Part 2! Request for help / ideas to debug issue

[Michael Hudson-Doyle]

On 14 March 2017 at 01:59, John Lenton  wrote:

> This one is slightly more interesting.
>
> You need 1.8 (or patched <1.8 as per the previous thread) for this one
> to make sense; without it you're just going to get drowned in warning
> messages and not see the real issue.
>
> This one is the real issue :-)
>

Ah _hah_!


> In go, when calling syscall.Exec to a setuid root binary, sometimes
> (about 4% of the times, on my machine, but it's hardware- and
> load-dependent), the exec'ed process will find itself running with
> effective uid different to zero. That is, a setuid root binary will
> find itself running as non-root. As the process that sets up
> confinement is setuid root (in distros where setuid is favoured over
> capabilities), this means the snap app falls on its face.
>
> TODO: check if something similar happens when using caps
>
> This is *probably* a bug in Go, but it only seems to arise when using
> syscall.Exec, which as far as I can tell is unsupported (the whole
> syscall package is unsupported -- not covered by the go1 compatibility
> promise -- and its replacement, golang.org/x/sys/unix, is ominously
> missing Exec).
>
> Having said that, it might be a bug in the kernel ;-)
> And I say this because if you pin the process to a single cpu, the
> issue doesn't arise.
>
> Anyway, code to repro this is at
> https://gist.github.com/chipaca/806c90d96c437444f27f45a83d00a813
>
> on my machine,
>
> $ for i in `seq -w `; do ./a_c; done | wc -l
> 0
> $ for i in `seq -w `; do ./a_go; done | wc -l
> 394
>
> And,
>
> $ for i in `seq -w `; do taskset 2 ./a_go; done | wc -l
> 0
>
> Gnarly!
>

That's pretty exciting.  I bet this is going to have the same underlying
cause as the other bug: something some other thread in the go process is
doing is causing the kernel to ignore the setuid bit. If I add a
time.Sleep(1*time.Millisecond) to a_go.go before the exec, the setuid bit
is respected every time. It doesn't help that setuid is ignored when
tracing or that strace likes to hang when you trace a_go.

I spent a while staring at the kernel source but I don't really have any
idea how this might be happening. It might be this code
https://github.com/torvalds/linux/blob/master/security/commoncap.c#L549-L561,
but I don't know how to be sure (well, without building kernels to do
debugging-via-kprint or whatever).

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Request for help / ideas to debug issue

[Michael Hudson-Doyle]

PS: I guess I should back port that Go fix to all supported Go releases?

On 12 March 2017 at 21:37, Michael Hudson-Doyle <
michael.hud...@canonical.com> wrote:

> Before we get into this, what is the actual problem here? Just the ugly
> messages?
>
> On 11 March 2017 at 02:58, Alfonso Sanchez-Beato <alfonso.sanchez-beato@
> canonical.com> wrote:
>
>> On Fri, Mar 10, 2017 at 10:22 AM, John Lenton <john.len...@canonical.com>
>> wrote:
>>
>> > Hello!
>> >
>> > We're seeing a weird issue with either go, pthreads, or the kernel. If
>> > you're knowledgeable about one or more of those things, could you take
>> > a look? Thank you.
>> >
>> > The issue manifests as nasty warnings from the "snap run" command,
>> > which is also the first step into a snapped app or service. It looks
>> > like
>> >
>> > runtime/cgo: pthread_create failed: Resource temporarily unavailable
>> >
>> > a very stripped-down reproducer is http://pastebin.ubuntu.com/24150663/
>> >
>> > build that, run it in a loop, and you'll see a bunch of those messages
>> > (and some weirder ones, but let's take it one step at a time)
>>
>
> Turns out this was fixed in Go 1.8: https://go-review.
> googlesource.com/c/33894/
>
>
>> > if you comment out the 'import "C"' line the message will change but
>> > still happen, which makes me think that at least in part this is a Go
>> > issue (or that we're holding it wrong).
>>
>
> ... but only in the non-cgo case, you can (occasionally) still get
> messages like:
>
> runtime: failed to create new OS thread (have 5 already; errno=11)
> runtime: may need to increase max user processes (ulimit -u)
> fatal error: newosproc
>
> if you comment out the import "C". I guess we should report that upstream.
>
>
>> > Note that the exec does work; the warning seems to come from a
>> > different thread than the one doing the Exec (the other clue that
>> > points in this direction is that sometimes the message is truncated).
>> > You can verify the fact that it does run by changing the /bin/true to
>> > /bin/echo os.Args[1], but because this issue is obviously a race
>> > somewhere, this change makes it less likely to happen (from ~10% down
>> > to ~.5% of runs, in my machines).
>> >
>> > One thing that makes this harder to debug is that strace'ing the
>> > process hangs (hard, kill -9 of strace to get out) before reproducing
>> > the issue. This probably means we need to trace it at a lower level,
>> > and I don't know enough about tracing a process group from inside the
>> > kernel to be able to do that; what I can find about kernel-level
>> > tracing is around syscalls or devices.
>> >
>> > Ideas?
>> >
>>
>> I found this related thread:
>>
>> https://groups.google.com/forum/#!msg/golang-nuts/8gszDBRZh_
>> 4/lhROTfN9TxIJ
>>
>> <<
>> I believe this can happen on GNU/Linux if your program uses cgo and if
>> thread A is in the Go runtime starting up a new thread B while thread
>> C is execing a program.  The underlying cause is that while one thread
>> is calling exec the Linux kernel will fail attempts by other threads
>> to call clone by returning EAGAIN.  (Look for uses of the in_exec
>> field in the kernel sources.)
>> >>
>>
>
> Yeah, this seems to be very accurate. It's also why it seems this is a
> cosmetic problem only, some thread not calling exec fails, but well, the
> thread is about to die anyway.
>
>
>> Something like adding a little sleep removes the traces, for instance:
>>
>> http://paste.ubuntu.com/24151637/
>>
>> where the program run sleep for 1ms before calling Exec. For smaller units
>> (say, 20 us) the issue still happens.
>>
>> It looks to me that right before running main(), go creates some threads,
>> calling clone() and probably getting the race described in the thread. As
>> anyway you are running Exec I guess the traces are harmless, you do not
>> need the go threads. Nonetheless, I think that the go run time should
>> retry
>> instead of printing that trace.
>
>
> Cheers,
> mwh
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Request for help / ideas to debug issue

[Michael Hudson-Doyle]

Before we get into this, what is the actual problem here? Just the ugly
messages?

On 11 March 2017 at 02:58, Alfonso Sanchez-Beato <
alfonso.sanchez-be...@canonical.com> wrote:

> On Fri, Mar 10, 2017 at 10:22 AM, John Lenton 
> wrote:
>
> > Hello!
> >
> > We're seeing a weird issue with either go, pthreads, or the kernel. If
> > you're knowledgeable about one or more of those things, could you take
> > a look? Thank you.
> >
> > The issue manifests as nasty warnings from the "snap run" command,
> > which is also the first step into a snapped app or service. It looks
> > like
> >
> > runtime/cgo: pthread_create failed: Resource temporarily unavailable
> >
> > a very stripped-down reproducer is http://pastebin.ubuntu.com/24150663/
> >
> > build that, run it in a loop, and you'll see a bunch of those messages
> > (and some weirder ones, but let's take it one step at a time)
>

Turns out this was fixed in Go 1.8:
https://go-review.googlesource.com/c/33894/


> > if you comment out the 'import "C"' line the message will change but
> > still happen, which makes me think that at least in part this is a Go
> > issue (or that we're holding it wrong).
>

... but only in the non-cgo case, you can (occasionally) still get messages
like:

runtime: failed to create new OS thread (have 5 already; errno=11)
runtime: may need to increase max user processes (ulimit -u)
fatal error: newosproc

if you comment out the import "C". I guess we should report that upstream.


> > Note that the exec does work; the warning seems to come from a
> > different thread than the one doing the Exec (the other clue that
> > points in this direction is that sometimes the message is truncated).
> > You can verify the fact that it does run by changing the /bin/true to
> > /bin/echo os.Args[1], but because this issue is obviously a race
> > somewhere, this change makes it less likely to happen (from ~10% down
> > to ~.5% of runs, in my machines).
> >
> > One thing that makes this harder to debug is that strace'ing the
> > process hangs (hard, kill -9 of strace to get out) before reproducing
> > the issue. This probably means we need to trace it at a lower level,
> > and I don't know enough about tracing a process group from inside the
> > kernel to be able to do that; what I can find about kernel-level
> > tracing is around syscalls or devices.
> >
> > Ideas?
> >
>
> I found this related thread:
>
> https://groups.google.com/forum/#!msg/golang-nuts/8gszDBRZh_4/lhROTfN9TxIJ
>
> <<
> I believe this can happen on GNU/Linux if your program uses cgo and if
> thread A is in the Go runtime starting up a new thread B while thread
> C is execing a program.  The underlying cause is that while one thread
> is calling exec the Linux kernel will fail attempts by other threads
> to call clone by returning EAGAIN.  (Look for uses of the in_exec
> field in the kernel sources.)
> >>
>

Yeah, this seems to be very accurate. It's also why it seems this is a
cosmetic problem only, some thread not calling exec fails, but well, the
thread is about to die anyway.


> Something like adding a little sleep removes the traces, for instance:
>
> http://paste.ubuntu.com/24151637/
>
> where the program run sleep for 1ms before calling Exec. For smaller units
> (say, 20 us) the issue still happens.
>
> It looks to me that right before running main(), go creates some threads,
> calling clone() and probably getting the race described in the thread. As
> anyway you are running Exec I guess the traces are harmless, you do not
> need the go threads. Nonetheless, I think that the go run time should retry
> instead of printing that trace.


Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: a little script: snapdiff

[Michael Hudson-Doyle]

On 10 February 2017 at 19:16, Simon Fels <simon.f...@canonical.com> wrote:

> On 10.02.2017 01:44, Michael Hudson-Doyle wrote:
> > Very much inspired by debdiff, I wrote a little script to compare two
> > snaps. It looks like this:
> >
> > (master)mwhudson@aeglos:/opt/opensource/snaps/go$ snapdiff
> > go-17-mwhudson_1.snap go-17-mwhudson_2.snap
> > Only in second snap, go-17-mwhudson_2.snap:
> >
> > -rw-rw-r-- root/root   839 2017-02-03 11:14
> > test/fixedbugs/issue16331.go
> > -rw-rw-r-- root/root   686 2017-02-03 11:14
> > test/fixedbugs/issue18410.go
> >
> > Differences in meta directory:
> >
> > diff -Nur meta1/snap.yaml meta2/snap.yaml
> > --- meta1/snap.yaml2017-01-11 11:49:30.0 +1300
> > +++ meta2/snap.yaml2017-02-03 11:16:07.0 +1300
> > @@ -13,4 +13,4 @@
> >  grade: devel
> >  name: go-17-mwhudson
> >  summary: Go programming language compiler, linker, stdlib
> > -version: 1.7.4
> > +version: 1.7.5
> >
> > It's implemented in perl (for no very good reason) and you can get it
> > here: http://paste.ubuntu.com/23963902/
>
> That is really nice!
>
> Having a classic snap for it would be awesome :-)
>

I've finally pushed this, needs review of course. It's also on github at
https://github.com/mwhudson/snapdiff now.

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

getting $LDFLAGS for a classic snap build in the plugin

[Michael Hudson-Doyle]

Hi,

The magic of classic confinement snaps is all in passing special flags when
linking an executable. If I've read things right, snapcraft does this by
wrapping any command you execute with BasePlugin.run() with some shell code
that sets LDFLAGS.

Unfortunately, for the go classic snap the underlying tool does not care at
all about LDFLAGS. What I do is create a wrapper:

#!/bin/sh
exec gcc $LDFLAGS "$@"

and point the go tool at that. This works but it would be a lot cleaner if
I could just get at the linker flags in my snap's plugin. Any chance that
could be added?

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

a little script: snapdiff

[Michael Hudson-Doyle]

Very much inspired by debdiff, I wrote a little script to compare two
snaps. It looks like this:

(master)mwhudson@aeglos:/opt/opensource/snaps/go$ snapdiff
go-17-mwhudson_1.snap go-17-mwhudson_2.snap
Only in second snap, go-17-mwhudson_2.snap:

-rw-rw-r-- root/root   839 2017-02-03 11:14
test/fixedbugs/issue16331.go
-rw-rw-r-- root/root   686 2017-02-03 11:14
test/fixedbugs/issue18410.go

Differences in meta directory:

diff -Nur meta1/snap.yaml meta2/snap.yaml
--- meta1/snap.yaml 2017-01-11 11:49:30.0 +1300
+++ meta2/snap.yaml 2017-02-03 11:16:07.0 +1300
@@ -13,4 +13,4 @@
 grade: devel
 name: go-17-mwhudson
 summary: Go programming language compiler, linker, stdlib
-version: 1.7.4
+version: 1.7.5

It's implemented in perl (for no very good reason) and you can get it here:
http://paste.ubuntu.com/23963902/

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: please try my golang snap

[Michael Hudson-Doyle]

On 6 February 2017 at 02:55, Shuduo Sang  wrote:

>
>
> On Fri, Feb 3, 2017 at 4:52 PM, Shuduo Sang 
> wrote:
>
>> Hi,
>>
>> I got following output after run it:
>>
>> $ go-17-mwhudson.go-17
>> /snap/go-17-mwhudson/2/gowrapper: line 3: /snap/go-17-mwhudson/2/bin/go:
>> No such file or directory
>>
>> My snap version is:
>> $ snap version
>> snap2.21
>> snapd   2.21
>> series  16
>> ubuntu  16.04
>>
>> Please let me know what mistake I have.
>>
>
> UPDATE: I notice the go binary is referred to library from /snap/core
> $ file /snap/go-17-mwhudson/current/bin/go
> /snap/go-17-mwhudson/current/bin/go: ELF 64-bit LSB executable, x86-64,
> version 1 (SYSV), dynamically linked, interpreter
> /snap/core/current/lib/x86_64-linux-gnu/ld-2.23.so, for GNU/Linux 2.6.32,
> BuildID[sha1]=3e694c21430e17b6bba99bd1828696a784ffab8a, not stripped
>
> But I have /snap/ubuntu-core only.
>
> After refresh ubuntu-core from candidate channel then it transit to
> /snap/core now go-17 works well. Thanks zyga's help.
>

 Ah good, glad you got it working.

I think this makes sense as a way to distribute Go to people who want to
use Go on their machines. I maintain a PPA with backports of new versions
of Go to older releases of Ubuntu but it's a bit fiddly and snaps would be
better (and also get cross distro coverage). I think to get to the point of
it being a good general distribution mechanism, we need:

1) testing (so that's everyone who's downloaded it so far)
2) pick a better snap name, or set of snap names (go-16, go-17, go-18 or
go16, go17, go18 or something else?)
3) get it into the stable channel
4) building in Launchpad would surely be nice to get better architecture
coverage

Have I missed anything?

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

please try my golang snap

[Michael Hudson-Doyle]

Prompted by some upstream discussion, I've made a snap of Go (just 1.7.4
for now):

$ sudo snap install --edge --classic go-17-mwhudson

It has aliases for go-17 (which will probably be the final snap of the snap
if this ends up being official) and go (the thinking being that there will
also be go-16, go-18 etc snaps and users can select which they use by
selecting an alias).

The source is at
https://code.launchpad.net/~mwhudson/+git/gosnap/+ref/master.

Please let me know how this works for you!

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Please test my asciinema snap

[Michael Hudson-Doyle]

On 25 January 2017 at 11:28, Michael Hudson-Doyle <
michael.hud...@canonical.com> wrote:
>
> You have to do this "asciinema rec -c 'PYTHONHOME=/usr python3
> urwid/examples/pop_up.py '" instead (and even that ends up using python3
> from the core snap, not the one I have installed).
>

More generally, classic snaps introduce the potential for confusion between
the "snap world" and the "classic world". When poking about the filesystem,
this isn't so bad because we have the classic world at / but when it comes
to process state, I expect the process launched by asciinema to be in the
classic world, and the environment variables mean it isn't, quite. I guess
this is what Stuart was getting at when he talked about patching the exec*
syscalls (can you use seccomp-bpf to unset an environment var if the
executable being launched comes from a path other than /snap? Sounds pretty
hair-raising).

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Please test my asciinema snap

[Michael Hudson-Doyle]

On 25 January 2017 at 11:02, Sergio Schvezov <sergio.schve...@canonical.com>
wrote:

> On Wed, 25 Jan 2017 08:17:17 +1300, Michael Hudson-Doyle wrote:
> > Hm, I was assuming that the PYTHONHOME leaking was due to things in the
> > snap specifically (is the source to the snap available?), but are they
> set
> > by snapd or snap-confine or something?
>
> Might be easier to follow in the bug Dave logged and linked to earlier,
> but in a nutshell: snapcraft sets up  a couple of variables to get things
> going. There seems to be conflicts with classic systems but are only
> viewable now with trusty enablement.
>

Well no, what I am complaining about is something perhaps related to that
bug but not the same.


> I am guessing the snapcraft.yaml for this looks like
>
> parts:
> asciinema:
> plugin: python
> python-packages: [asciinema]
>
> I've added python3 as a part and did some environment tweaks locally and
> it works like a charm.
>

But if you did that, PYTHONHOME is presumably still set in the environment
of the command you are recording with asciinema. I was trying to record me
using a new widget in a program using urwid and getting this sort of thing:

mwhudson@aeglos:/opt/opensource$ asciinema rec -c 'python3
urwid/examples/pop_up.py '
~ Asciicast recording started.
~ Hit Ctrl-D or type "exit" to finish.
Traceback (most recent call last):
  File "urwid/examples/pop_up.py", line 3, in 
import urwid
ImportError: No module named 'urwid'
~ Asciicast recording finished.
~ Press  to upload,  to cancel.

You have to do this "asciinema rec -c 'PYTHONHOME=/usr python3
urwid/examples/pop_up.py '" instead (and even that ends up using python3
from the core snap, not the one I have installed).

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Please test my asciinema snap

[Michael Hudson-Doyle]

Hm, I was assuming that the PYTHONHOME leaking was due to things in the
snap specifically (is the source to the snap available?), but are they set
by snapd or snap-confine or something?

Cheers,
mwh

On 24 January 2017 at 00:09, Gustavo Niemeyer  wrote:

>
> I'm wondering if maybe we should simply drop all snapcraft wrappers for
> classic snaps, specifically.
>
> As it is, the amount of magic that is actually intended for strict snaps
> seems to be hurting the behavior and understanding of classic snaps. I
> doubt adding even more magic will help.
>
>
>
> On Mon, Jan 23, 2017 at 2:35 AM, Stuart Bishop <
> stuart.bis...@canonical.com> wrote:
>
>>
>>
>> On 20 January 2017 at 19:59, Mark Shuttleworth  wrote:
>>
>>>
>>> Any recommendations for dealing with those?
>>>
>>
>> Do exec* and friends need to be patched somehow, so that if processes are
>> spawned from a classic snap with targets outside snapd containment then the
>> environment is cleaned?
>>
>> I think this will affect all classic snaps that need to run subprocesses,
>> such as screen, vim, tmux... with other wrapper variables like the LD_*
>> settings leaking :-(
>>
>>
>> --
>> Stuart Bishop 
>>
>> --
>> Snapcraft mailing list
>> Snapcraft@lists.snapcraft.io
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
>> an/listinfo/snapcraft
>>
>>
>
>
> --
>
> gustavo @ http://niemeyer.net
>
> --
> Snapcraft mailing list
> Snapcraft@lists.snapcraft.io
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/snapcraft
>
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Please test my asciinema snap

[Michael Hudson-Doyle]

This confused me for a while:
https://asciinema.org/a/2ua1d08k6v8jiyy1m2uyvn8rx (spoiler: python3 inside '
asciinema rec' is python3 from the core snap).

Cheers,
mwh

On 18 January 2017 at 22:16, Mark Shuttleworth  wrote:

> Hi folks
>
> (For those of you who Gmail does not filter this email on
> as-yet-unexplained-grounds :))
>
> Please could you test my asciinema snap? Asciinema is a console video
> recording utility that's great for CLI-diven demos. If you want to make
> a quick web video of a CLI / console journey, asciinema is the ticket.
>
> An older version (0.9.8) is in 16.04. The new 1.3.0 version is now a
> snap, and it should work on 16.04. I am also interested in feedback on
> 14.04 for those of you on Trusty steeds who are blazing the
> snaps-on-trusty trail.
>
> It's a 'classic-only' snap, so you need:
>
>   sudo snap install --classic asciinema
>
> Then 'asciinema rec' starts a recording session, and you're off to the
> races.
>
> Thanks!
>
> Mark
>
>
>
>
> --
> Snapcraft mailing list
> Snapcraft@lists.snapcraft.io
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/snapcraft
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: create-user primitive

[Michael Hudson-Doyle]

On 18 January 2017 at 12:13, Manik Taneja <ma...@canonical.com> wrote:

>
>
> On Tue, Jan 17, 2017 at 1:30 PM, Michael Hudson-Doyle <
> michael.hud...@canonical.com> wrote:
>>
>>
>> Well snap create-user is mostly there exactly so that console-conf can
>> invoke it. That's also why it's hidden from snap help!
>>
> thanks for the insights michael. in addition, is there a way to-
>
>- remove the user created via create-user?
>
> I don't know.

>
>- add local users?
>
> Yes, you can you adduser with the --extrausers flag.

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Source of ssh key for first login

[Michael Hudson-Doyle]

On 21 December 2016 at 22:51, Luther Goh Lu Feng  wrote:

> During setup, console-conf will download the SSH key registered with your
> Store account, The ssh key is used for first login.
>
> Is there a way to configure Ubuntu Core such that the source of the ssh
> key can be from elsewhere? eg for cases whereby the device cant go online
> and can only access a private network. Thanks
>

If you are the brand for the image you are using (i.e., you make your own
image), you can provide a system-user assertion that will be processed
during first boot to allow you to log in with user and password. The
process for doing this is not documented anywhere that I know about though
:(

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: can't log in on ubuntu core console via sso...

[Michael Hudson-Doyle]

Oh wait, I haven't made a subiquity release with that change yet. Sorry!
(Blame baby brain)

sent from my phone, please excuse brevity

On 15 Dec 2016 08:05, "Oliver Grawert" <o...@ubuntu.com> wrote:

> hi,
> Am Donnerstag, den 15.12.2016, 07:47 +1300 schrieb Michael Hudson-
> Doyle:
> >
> > That's exactly what I implemented. Do you not get it with an image
> > built from the edge channel?
> >
> hmm, not with an updated image on the last edge core at least ... i'll
> have to check with a fresh install tomorrow ...
>
> ciao
> oli
> --
> Snapcraft mailing list
> Snapcraft@lists.snapcraft.io
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/snapcraft
>
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: can't log in on ubuntu core console via sso...

[Michael Hudson-Doyle]

sent from my phone, please excuse brevity

On 15 Dec 2016 01:19, "Oliver Grawert" <o...@ubuntu.com> wrote:

hi,
Am Mittwoch, den 14.12.2016, 13:22 +1300 schrieb Michael Hudson-Doyle:
> What version of core do you have? This should be clearer with the
> latest versions, you shouldn't see the login prompt until you set a
> password.
>
this issue comes up more often on IRC as well..

it is definitely not the case that we suppress the prompt currently
(and i guess it would just move the problem and people would ask why
the console is broken), we only show the "please press enter" message
until an account is created. after this the normal login prompt is
shown.

i think we should show some info text so people are aware that they
need to use the ssh login to create a password first to actually enable
the console based login.


That's exactly what I implemented. Do you not get it with an image built
from the edge channel?

just hiding the login prompt will only cause
more questions in the end.

ciao
oli
--
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: https://lists.ubuntu.com/
mailman/listinfo/snapcraft
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Help making images to test console-conf changes

[Michael Hudson-Doyle]

On 10 November 2016 at 00:50, Oliver Grawert <o...@ubuntu.com> wrote:

> hi,
> Am Mittwoch, den 09.11.2016, 16:07 +1300 schrieb Michael Hudson-Doyle:
> >
> >
> > I tried to test my packages first by fixing up our build of the core
> > snap (https://launchpad.net/~canonical-foundations/+snap/snappy-first
> > -boot) and then using ubuntu-image (version 0.11+real1 / revision 26
> > from the edge channel) and http://people.canonical.com/~vorlon/offici
> > al-models/dragonboard-model.assertion to make a dragonboard image but
> > it didn't boot. Are there any special steps that need to be taken
> > when making a dragonboard image? I'd love to be able to test this
> > stuff more locally than waiting for your scripts to churn out images
> > ...
>
> do you have any more info ? how does it fail the boot ? what is on the
> serial console ?
>

Sorry, no (my dragonboard doesn't have working serial). The daily images
worked fine though, so I've made 'proper' subiquity and probert releases
and uploaded to zesty and ppa:snappy-dev/image. I've tested in KVM and on
the dragonboard so testing on other boards would be a good idea before we
promote to beta or stable (what is the schedule for that, out of
curiosity?).

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Help making images to test console-conf changes

[Michael Hudson-Doyle]

On 8 November 2016 at 07:41, Michael Hudson-Doyle <
michael.hud...@canonical.com> wrote:

>
>
> On 8 November 2016 at 00:24, Oliver Grawert <o...@ubuntu.com> wrote:
>
>> hi,
>> Am Montag, den 07.11.2016, 17:27 +1300 schrieb Michael Hudson-Doyle:
>> > Hi all,
>> > I've been working on a rewrite of console-conf's networking bits,
>> > with the intent of making the UI a bit clearer and more dynamic (e.g.
>> > if you stick a USB Ethernet adapter in after you've started console-
>> > conf, it shows up in the UI).
>> > I've tested this locally but this is the sort of thing that should
>> > definitely be tested on all supported devices before it goes into
>> > core. I've put probert and subiquity packages into
>> > ppa:mwhudson/devirt but realised that I don't know how to make core
>> > snaps with custom packages on launchpad! If someone could help we
>> > with that, then I (or anyone) can make some images for people to
>> > test.
>>
>> this is exactly what
>> http://people.canonical.com/~ogra/snappy/all-snaps/daily/
>> is for ...
>>
>> just copy your binary into the image PPA, there it gets picked up for
>> the daily edge builds of the core snap and images ...
>>
>
> When in the day do the snap / image builds happen? I know how to trigger a
> snap build I think (https://launchpad.net/~snappy-dev/+snap/core/+
> request-builds ?) but not how to trigger an image build...
>
>
>> the daily auto-builds ( http://people.canonical.com/~ogra/core-builds/
>> ) only got into the edge channel, currently all other channels require
>> manual intervention before anything gets promoted into them, so you
>> cant break anything.
>>
>
> OK, done. I wasn't sure how disruptive breaking edge was :)
>

The packages mostly worked. I've made a few tweaks and uploaded new
packages for today's builds. Hopefully they work too :)

I tried to test my packages first by fixing up our build of the core snap (
https://launchpad.net/~canonical-foundations/+snap/snappy-first-boot) and
then using ubuntu-image (version 0.11+real1 / revision 26 from the edge
channel) and
http://people.canonical.com/~vorlon/official-models/dragonboard-model.assertion
to
make a dragonboard image but it didn't boot. Are there any special steps
that need to be taken when making a dragonboard image? I'd love to be able
to test this stuff more locally than waiting for your scripts to churn out
images ...

Cheers,
mwh


> just make sure to follow up if there are any regressions and take the
>> necessary actions ;)
>>
>
> Of course.
>
> Cheers,
> mwh
>
>
>> ciao
>> oli
>> --
>> Snapcraft mailing list
>> Snapcraft@lists.snapcraft.io
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
>> an/listinfo/snapcraft
>>
>>
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Help making images to test console-conf changes

[Michael Hudson-Doyle]

On 8 November 2016 at 00:24, Oliver Grawert <o...@ubuntu.com> wrote:

> hi,
> Am Montag, den 07.11.2016, 17:27 +1300 schrieb Michael Hudson-Doyle:
> > Hi all,
> > I've been working on a rewrite of console-conf's networking bits,
> > with the intent of making the UI a bit clearer and more dynamic (e.g.
> > if you stick a USB Ethernet adapter in after you've started console-
> > conf, it shows up in the UI).
> > I've tested this locally but this is the sort of thing that should
> > definitely be tested on all supported devices before it goes into
> > core. I've put probert and subiquity packages into
> > ppa:mwhudson/devirt but realised that I don't know how to make core
> > snaps with custom packages on launchpad! If someone could help we
> > with that, then I (or anyone) can make some images for people to
> > test.
>
> this is exactly what
> http://people.canonical.com/~ogra/snappy/all-snaps/daily/
> is for ...
>
> just copy your binary into the image PPA, there it gets picked up for
> the daily edge builds of the core snap and images ...
>

When in the day do the snap / image builds happen? I know how to trigger a
snap build I think (
https://launchpad.net/~snappy-dev/+snap/core/+request-builds ?) but not how
to trigger an image build...


> the daily auto-builds ( http://people.canonical.com/~ogra/core-builds/
> ) only got into the edge channel, currently all other channels require
> manual intervention before anything gets promoted into them, so you
> cant break anything.
>

OK, done. I wasn't sure how disruptive breaking edge was :)


> just make sure to follow up if there are any regressions and take the
> necessary actions ;)
>

Of course.

Cheers,
mwh


> ciao
> oli
> --
> Snapcraft mailing list
> Snapcraft@lists.snapcraft.io
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/snapcraft
>
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Help making images to test console-conf changes

[Michael Hudson-Doyle]

Hi all,

I've been working on a rewrite of console-conf's networking bits, with the
intent of making the UI a bit clearer and more dynamic (e.g. if you stick a
USB Ethernet adapter in after you've started console-conf, it shows up in
the UI).

I've tested this locally but this is the sort of thing that should
definitely be tested on all supported devices before it goes into core.
I've put probert and subiquity packages into ppa:mwhudson/devirt but
realised that I don't know how to make core snaps with custom packages on
launchpad! If someone could help we with that, then I (or anyone) can make
some images for people to test.

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: More Ubuntu-Core boot up problems...

[Michael Hudson-Doyle]

On 7 September 2016 at 21:52, Oliver Grawert  wrote:
>
> depending on the console (console-conf isnt actually great with serial)
> you will not see the "please press enter" message that starts the
> config tool though.


I'd love to know why this isn't working, btw. We're just using agetty's -f
option...

Cheers,
mwh
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft

Re: Snappy takes long time to boot when no network

[Michael Hudson-Doyle]

On 2 September 2016 at 00:44, Yann Sionneau 
wrote:

> I'm not sure I understand everything in the bug ticket as I am not a
> systemd / networkd / netplan expert at all.
>
> But :
>
> root@Paros:~# cat /etc/netplan/00-
> 00-initial-config.yaml  00-snapd-config.yaml
> root@Paros:~# cat /etc/netplan/00-initial-config.yaml
>
> network:
>  version: 2
>  ethernets:
>all:
> match:
>  name: "*"
> dhcp4: true
> root@Paros:~# cat /run/systemd/network/10-netplan-all.network
> [Match]
> Name=*
>
> [Network]
> DHCP=ipv4
> root@Paros:~# networkctl
> IDX LINK TYPE   OPERATIONAL SETUP
>   1 lo   loopback   carrier configured
>   2 sit0 sitroutableconfiguring
>   3 eth1 ether  routableconfigured
>   4 eth0 ether  routableconfigured
>   5 rndis0   ether  no-carrier  configuring
>
> 5 links listed.
>
> Does this help?
>
> eth0 is a WiFi interface.
>
> eth1 is USB ethernet device (plugged at boot)
>
>
I don't know if it's the cause of all your issues, but having both files in
/etc/netplan is a sign of console-conf / snapd version skew. It's fixed now
and new installs won't have this problem, but if you don't want to
re-install just deleted the 00-initial-config.yaml file and run sudo
netplan apply.

Cheers,
mwh


> Le 09/01/2016 à 02:30 PM, Oliver Grawert a écrit :
> > hi,
> > On Do, 2016-09-01 at 14:05 +0200, Yann Sionneau wrote:
> >> I feel like this is the bug I'm hitting.
> >> But, how do you explain that my boot is stalled even if I have 2 NICs
> >> with internet access?
> >> One is via wifi, the other is via usb-ethernet.
> >> Thanks!
> > hmm, this sounds more like a different bug ...
> > how about:
> >
> > https://launchpad.net/bugs/1618522
> >
> > ciao
> >   oli
>
>
> --
> Snapcraft mailing list
> Snapcraft@lists.snapcraft.io
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/snapcraft
>
-- 
Snapcraft mailing list
Snapcraft@lists.snapcraft.io
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/snapcraft