Re: Daemon update again

[Ludovic Courtès]

Andreas Enge  skribis:

> On Tue, Jun 09, 2015 at 05:51:00PM +0200, Ludovic Courtès wrote:
>> There’s also the fact that it’s using -std=c++0x, whereas current master
>> uses -std=c++11.  Fishy!
>
> Very fishy indeed! I think I did not run "autoreconf -vfi" correctly, or
> it failed and I did not pay attention, and was left with a mixture of old
> and new files. Now I redid it, and there is no trace left of "-std=c++0x",
> and the flags seem to be correctly said.
>
> Thank you for asking the right questions, and sorry for taking your time!

Good, no problem!

Ludo’.

Re: Daemon update again

[Andreas Enge]

On Tue, Jun 09, 2015 at 05:51:00PM +0200, Ludovic Courtès wrote:
> There’s also the fact that it’s using -std=c++0x, whereas current master
> uses -std=c++11.  Fishy!

Very fishy indeed! I think I did not run "autoreconf -vfi" correctly, or
it failed and I did not pay attention, and was left with a mixture of old
and new files. Now I redid it, and there is no trace left of "-std=c++0x",
and the flags seem to be correctly said.

Thank you for asking the right questions, and sorry for taking your time!

Andreas

Re: Daemon update again

[Ludovic Courtès]

Andreas Enge  skribis:

> On Sat, Jun 06, 2015 at 07:41:42PM +0200, Ludovic Courtès wrote:
>> Could you run “make V=1”?  Normally DEFAULT_CHROOT_DIRS is defined from
>> the command line (see ‘libstore_a_CPPFLAGS’.)
>
> There is no trace of it on the command line:
>
> g++ -DHAVE_CONFIG_H -I. -I./nix  -I./nix -I./nix/libutil -I./nix 
> -I./nix/libstore -I./nix/libstore -DNIX_STORE_DIR=\"/gnu/store\" 
> -DNIX_DATA_DIR=\"/usr/local/guix/share\" 
> -DNIX_STATE_DIR=\"/usr/local/guix/var/guix\" 
> -DNIX_LOG_DIR=\"/usr/local/guix/var/log/guix\" 
> -DNIX_CONF_DIR=\"/usr/local/guix/etc/guix\" 
> -DNIX_LIBEXEC_DIR=\"/usr/local/guix/libexec\" 
> -DNIX_BIN_DIR=\"/usr/local/guix/bin\" -DOPENSSL_PATH="\"guix-authenticate\""  
> -Wall -std=c++0x   -g -O2 -MT nix/libstore/libstore_a-build.o -MD -MP -MF 
> nix/libstore/.deps/libstore_a-build.Tpo -c -o nix/libstore/libstore_a-build.o 
> `test -f 'nix/libstore/build.cc' || echo './'`nix/libstore/build.cc
> nix/libstore/build.cc: In member function ‘void 
> nix::DerivationGoal::startBuilder()’:
> nix/libstore/build.cc:1808:91: error: ‘DEFAULT_CHROOT_DIRS’ was not declared 
> in this scope
>  PathSet dirs = 
> tokenizeString(settings.get("build-chroot-dirs", 
> string(DEFAULT_CHROOT_DIRS)));

Hmm.  What’s the value of ‘libstore_a_CPPFLAGS’ in daemon.am,
Makefile.in, and Makefile?

There’s also the fact that it’s using -std=c++0x, whereas current master
uses -std=c++11.  Fishy!

Ludo’.

Re: Daemon update again

[Andreas Enge]

On Sat, Jun 06, 2015 at 07:41:42PM +0200, Ludovic Courtès wrote:
> Could you run “make V=1”?  Normally DEFAULT_CHROOT_DIRS is defined from
> the command line (see ‘libstore_a_CPPFLAGS’.)

There is no trace of it on the command line:

g++ -DHAVE_CONFIG_H -I. -I./nix  -I./nix -I./nix/libutil -I./nix 
-I./nix/libstore -I./nix/libstore -DNIX_STORE_DIR=\"/gnu/store\" 
-DNIX_DATA_DIR=\"/usr/local/guix/share\" 
-DNIX_STATE_DIR=\"/usr/local/guix/var/guix\" 
-DNIX_LOG_DIR=\"/usr/local/guix/var/log/guix\" 
-DNIX_CONF_DIR=\"/usr/local/guix/etc/guix\" 
-DNIX_LIBEXEC_DIR=\"/usr/local/guix/libexec\" 
-DNIX_BIN_DIR=\"/usr/local/guix/bin\" -DOPENSSL_PATH="\"guix-authenticate\""  
-Wall -std=c++0x   -g -O2 -MT nix/libstore/libstore_a-build.o -MD -MP -MF 
nix/libstore/.deps/libstore_a-build.Tpo -c -o nix/libstore/libstore_a-build.o 
`test -f 'nix/libstore/build.cc' || echo './'`nix/libstore/build.cc
nix/libstore/build.cc: In member function ‘void 
nix::DerivationGoal::startBuilder()’:
nix/libstore/build.cc:1808:91: error: ‘DEFAULT_CHROOT_DIRS’ was not declared in 
this scope
 PathSet dirs = 
tokenizeString(settings.get("build-chroot-dirs", 
string(DEFAULT_CHROOT_DIRS)));

Andreas

Re: Daemon update again

[Ludovic Courtès]

Andreas Enge  skribis:

> I did a "make distclean", "./bootstrap", "./configure" and "make" on my mips

‘make’ is usually enough.

> machine. It fails with the following message:
>   CXX  nix/libstore/libstore_a-build.o
> nix/libstore/build.cc: In member function ‘void 
> nix::DerivationGoal::startBuilder()’:
> nix/libstore/build.cc:1808:91: error: ‘DEFAULT_CHROOT_DIRS’ was not declared 
> in this scope
>  PathSet dirs = 
> tokenizeString(settings.get("build-chroot-dirs", 
> string(DEFAULT_CHROOT_DIRS)));

Could you run “make V=1”?  Normally DEFAULT_CHROOT_DIRS is defined from
the command line (see ‘libstore_a_CPPFLAGS’.)

> nix/libstore/build.cc: In member function ‘void nix::Worker::waitForInput()’:
> nix/libstore/build.cc:3324:74: warning: comparison between signed and 
> unsigned integer expressions [-Wsign-compare]
>  if (!waitingForAWhile.empty() && lastWokenUp + settings.pollInterval <= 
> after) {
>   ^

Weird, I don’t have it on x86_64 with GCC 5.1.0 -Wall but ‘time_t’ is
indeed signed on all platforms (according to ) and
‘pollInterval’ is unsigned.

Thanks,
Ludo’.

Re: Daemon update again

[Andreas Enge]

I did a "make distclean", "./bootstrap", "./configure" and "make" on my mips
machine. It fails with the following message:
  CXX  nix/libstore/libstore_a-build.o
nix/libstore/build.cc: In member function ‘void 
nix::DerivationGoal::startBuilder()’:
nix/libstore/build.cc:1808:91: error: ‘DEFAULT_CHROOT_DIRS’ was not declared in 
this scope
 PathSet dirs = 
tokenizeString(settings.get("build-chroot-dirs", 
string(DEFAULT_CHROOT_DIRS)));

   ^
nix/libstore/build.cc: In member function ‘void nix::Worker::waitForInput()’:
nix/libstore/build.cc:3324:74: warning: comparison between signed and unsigned 
integer expressions [-Wsign-compare]
 if (!waitingForAWhile.empty() && lastWokenUp + settings.pollInterval <= 
after) {
  ^
Makefile:2630: recipe for target 'nix/libstore/libstore_a-build.o' failed

Andreas

Daemon update again

[Ludovic Courtès]

Commit 54c260e updates the daemon again from upstream Nix code.  Few
interesting changes this time; interesting changes include:

  • Fixed-output derivations (such as downloads) are now also run in a
chroot environment.  The difference with other derivations is that
they do not get a separate network name space, which allows them to
access the network, and they get additional files such as
/etc/resolv.conf.

  • pivot_root(2) is used in addition to chroot(2), which is claimed to
really prevent getting out of the chroot (though in practice build
processes are non-root so I don’t see how they could get away.)

  • The ‘verifyStore’ RPC (more on that soon.)

Please report any issues!

Ludo’.

Re: Daemon update

[Alexander Vorobiev]

Yes, sorry for too short a snippet:

PASS: gapplication 1 /gapplication/no-dbus
PASS: gapplication 2 /gapplication/no-appid
PASS: gapplication 3 /gapplication/properties
PASS: gapplication 4 /gapplication/app-id
ERROR: gapplication - missing test plan
ERROR: gapplication - exited with status 139 (terminated by signal 11?)
SKIP: gdbus-auth
PASS: gdbus-bz627724 1 /gdbus/bz627724
PASS: gdbus-close-pending 1 /gdbus/close-pending
PASS: gdbus-close-pending 2 /gdbus/unref-pending

and finally:


Testsuite summary for glib 2.44.0

# TOTAL: 673
# PASS:  632
# SKIP:  39
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 2


by the way, I had to

touch doc/guix-environment.1

which was missing.

Thanks,
Alex

On Wed, Jun 3, 2015 at 3:24 AM, Ludovic Courtès  wrote:

> Alexander Vorobiev  skribis:
>
> > At first I saw the exact same error in this list:
> >
> > http://lists.gnu.org/archive/html/bug-guix/2014-12/msg2.html
>
> Oh.
>
> > I then pulled again, rebooted the VM just in case, reinstalled guix, and
>
> It’s not necessary to reinstall Guix, it shouldn’t make any difference.
>
> > now I am seeing an error similar to the second error from that email:
> >
> > ERROR: gapplication - missing test plan
> > ERROR: gapplication - exited with status 139 (terminated by signal 11?)
> >
> > That email had status 136 and signal 6.
>
> Could you paste more context from the build log, so we can see which
> test is failing?  (Use ‘guix build --log-file glib’ to access the log.)
>
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alexander Vorobiev  skribis:

> At first I saw the exact same error in this list:
>
> http://lists.gnu.org/archive/html/bug-guix/2014-12/msg2.html

Oh.

> I then pulled again, rebooted the VM just in case, reinstalled guix, and

It’s not necessary to reinstall Guix, it shouldn’t make any difference.

> now I am seeing an error similar to the second error from that email:
>
> ERROR: gapplication - missing test plan
> ERROR: gapplication - exited with status 139 (terminated by signal 11?)
>
> That email had status 136 and signal 6.

Could you paste more context from the build log, so we can see which
test is failing?  (Use ‘guix build --log-file glib’ to access the log.)

Ludo’.

Re: Daemon update

[Alexander Vorobiev]

At first I saw the exact same error in this list:

http://lists.gnu.org/archive/html/bug-guix/2014-12/msg2.html

I then pulled again, rebooted the VM just in case, reinstalled guix, and
now I am seeing an error similar to the second error from that email:

ERROR: gapplication - missing test plan
ERROR: gapplication - exited with status 139 (terminated by signal 11?)

That email had status 136 and signal 6.

Thanks,
Alex

On Mon, Jun 1, 2015 at 3:23 PM, Alexander Vorobiev <
alexander.vorob...@gmail.com> wrote:

> Yes it is, happens every time I run it (make guix-binary). Could it be
> custom store and/or local cache? Or wrong version of something in the
> system I am using (latest Arch linux)?
>
> Thanks,
> Alex
>
> On Mon, Jun 1, 2015 at 2:58 PM, Ludovic Courtès  wrote:
>
>> Alexander Vorobiev  skribis:
>>
>> > That fixed tcsh, thanks. Here is the next stop - glib (libgio), seems
>> to be
>> > failing its unit tests:
>> >
>> > 
>> > PASS: defaultvalue 80 /Default Values/GZlibCompressor
>> > PASS: defaultvalue 81 /Default Values/GZlibDecompressor
>> > tap-driver.sh: internal error getting exit status
>> > tap-driver.sh: fatal: I/O or internal error
>> > Makefile:3751: recipe for target 'defaultvalue.log' failed
>> > make[8]: *** [defaultvalue.log] Error 1
>> > make[8]: Leaving directory
>> > '/tmp/nix-build-glib-2.44.0.drv-0/glib-2.44.0/gio/tests'
>>
>> Is that deterministic?  I.e., does it happen if you run the build again?
>> We haven’t experienced it on hydra.gnu.org.
>>
>> Thanks,
>> Ludo’.
>>
>
>

Re: Daemon update

[Alexander Vorobiev]

Yes it is, happens every time I run it (make guix-binary). Could it be
custom store and/or local cache? Or wrong version of something in the
system I am using (latest Arch linux)?

Thanks,
Alex

On Mon, Jun 1, 2015 at 2:58 PM, Ludovic Courtès  wrote:

> Alexander Vorobiev  skribis:
>
> > That fixed tcsh, thanks. Here is the next stop - glib (libgio), seems to
> be
> > failing its unit tests:
> >
> > 
> > PASS: defaultvalue 80 /Default Values/GZlibCompressor
> > PASS: defaultvalue 81 /Default Values/GZlibDecompressor
> > tap-driver.sh: internal error getting exit status
> > tap-driver.sh: fatal: I/O or internal error
> > Makefile:3751: recipe for target 'defaultvalue.log' failed
> > make[8]: *** [defaultvalue.log] Error 1
> > make[8]: Leaving directory
> > '/tmp/nix-build-glib-2.44.0.drv-0/glib-2.44.0/gio/tests'
>
> Is that deterministic?  I.e., does it happen if you run the build again?
> We haven’t experienced it on hydra.gnu.org.
>
> Thanks,
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alexander Vorobiev  skribis:

> That fixed tcsh, thanks. Here is the next stop - glib (libgio), seems to be
> failing its unit tests:
>
> 
> PASS: defaultvalue 80 /Default Values/GZlibCompressor
> PASS: defaultvalue 81 /Default Values/GZlibDecompressor
> tap-driver.sh: internal error getting exit status
> tap-driver.sh: fatal: I/O or internal error
> Makefile:3751: recipe for target 'defaultvalue.log' failed
> make[8]: *** [defaultvalue.log] Error 1
> make[8]: Leaving directory
> '/tmp/nix-build-glib-2.44.0.drv-0/glib-2.44.0/gio/tests'

Is that deterministic?  I.e., does it happen if you run the build again?
We haven’t experienced it on hydra.gnu.org.

Thanks,
Ludo’.

Re: Daemon update

[Alexander Vorobiev]

That fixed tcsh, thanks. Here is the next stop - glib (libgio), seems to be
failing its unit tests:


PASS: defaultvalue 80 /Default Values/GZlibCompressor
PASS: defaultvalue 81 /Default Values/GZlibDecompressor
tap-driver.sh: internal error getting exit status
tap-driver.sh: fatal: I/O or internal error
Makefile:3751: recipe for target 'defaultvalue.log' failed
make[8]: *** [defaultvalue.log] Error 1
make[8]: Leaving directory
'/tmp/nix-build-glib-2.44.0.drv-0/glib-2.44.0/gio/tests'
Makefile:3653: recipe for target 'check-TESTS' failed
make[7]: *** [check-TESTS] Error 2
make[7]: Leaving directory
'/tmp/nix-build-glib-2.44.0.drv-0/glib-2.44.0/gio/tests'
Makefile:4353: recipe for target 'check-am' failed
make[6]: *** [check-am] Error 2
make[6]: Leaving directory
'/tmp/nix-build-glib-2.44.0.drv-0/glib-2.44.0/gio/tests'
Makefile:3440: recipe for target 'check-recursive' failed
make[5]: *** [check-recursive] Error 1
make[5]: Leaving directory
'/tmp/nix-build-glib-2.44.0.drv-0/glib-2.44.0/gio/tests'

Thanks,
Alex

On Sun, May 31, 2015 at 2:14 PM, Ludovic Courtès  wrote:

> Alexander Vorobiev  skribis:
>
> > Pulled, restarted. The next stop is tcsh. The tarball at ftp.astron.com
> s
> > gone, replaced with more recent version...
>
> Fixed, thanks.
>
> > One observation: apparently coreutils refuses to be built as root so I
> had
> > to create a build user/group to give to guix-daemon.
>
> Right, and both the manual and guix-daemon warn you against running
> guix-daemon as root without --build-users-group.  :-)
>
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alexander Vorobiev  skribis:

> Pulled, restarted. The next stop is tcsh. The tarball at ftp.astron.com s
> gone, replaced with more recent version...

Fixed, thanks.

> One observation: apparently coreutils refuses to be built as root so I had
> to create a build user/group to give to guix-daemon.

Right, and both the manual and guix-daemon warn you against running
guix-daemon as root without --build-users-group.  :-)

Ludo’.

Re: Daemon update

[Alexander Vorobiev]

Pulled, restarted. The next stop is tcsh. The tarball at ftp.astron.com s
gone, replaced with more recent version...
One observation: apparently coreutils refuses to be built as root so I had
to create a build user/group to give to guix-daemon.

Thanks,
Alex

On Fri, May 29, 2015 at 3:57 PM, Ludovic Courtès  wrote:

> Alexander Vorobiev  skribis:
>
> > I have some good progress finally. I started from scratch and pulled the
> > latest from git. I am now running guix-daemon as root with only one
> option
> > --no-substitutes. The make guix-binary* ran for hours and built a lot of
> > stuff (bash, gcc, perl, etc) but stumbled upon openldap which doesn't
> seem
> > to be available at the url it wants to use:
>
> Indeed, I’ve updated the OpenLDAP URLs.
>
> > I tried modifying gnu/packages/openldap.scm to use correct url (
> > ftp://mirror.switch.ch/mirror/OpenLDAP/openldap-release/...) but that
> > didn't change anything since (I am guessing) the .drv files in store
> > weren't rebuilt and still point to the old url. How do I fix that without
> > rebuilding everything that it created so far?
>
> Changing the file does have an effect, but perhaps you modified the
> wrong file?  At any rate, if you git pull now and try again it should
> work (and it won’t rebuild everything.)
>
> > As a side note, would it be possible to modify the files you suggested
> (gnu
> > packages package-management and gnu system instal) for the custom local
> > state dir in such a way (through configure? scheme variable?) that it
> > wouldn't be hard-coded? I am guessing any update to guix would cause my
> > changes to be lost?
>
> Since you’re working from a git checkout (right?), you can always
> arrange to have your changes applied.
>
> Thanks,
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alexander Vorobiev  skribis:

> I have some good progress finally. I started from scratch and pulled the
> latest from git. I am now running guix-daemon as root with only one option
> --no-substitutes. The make guix-binary* ran for hours and built a lot of
> stuff (bash, gcc, perl, etc) but stumbled upon openldap which doesn't seem
> to be available at the url it wants to use:

Indeed, I’ve updated the OpenLDAP URLs.

> I tried modifying gnu/packages/openldap.scm to use correct url (
> ftp://mirror.switch.ch/mirror/OpenLDAP/openldap-release/...) but that
> didn't change anything since (I am guessing) the .drv files in store
> weren't rebuilt and still point to the old url. How do I fix that without
> rebuilding everything that it created so far?

Changing the file does have an effect, but perhaps you modified the
wrong file?  At any rate, if you git pull now and try again it should
work (and it won’t rebuild everything.)

> As a side note, would it be possible to modify the files you suggested (gnu
> packages package-management and gnu system instal) for the custom local
> state dir in such a way (through configure? scheme variable?) that it
> wouldn't be hard-coded? I am guessing any update to guix would cause my
> changes to be lost?

Since you’re working from a git checkout (right?), you can always
arrange to have your changes applied.

Thanks,
Ludo’.

Re: Daemon update

[Alexander Vorobiev]

I have some good progress finally. I started from scratch and pulled the
latest from git. I am now running guix-daemon as root with only one option
--no-substitutes. The make guix-binary* ran for hours and built a lot of
stuff (bash, gcc, perl, etc) but stumbled upon openldap which doesn't seem
to be available at the url it wants to use:

building path(s)
`/shared/shape_tier3/common/local/guix/store/50l6s098sy4ig2irjm1g4c8y261nxzfc-openldap-2.4.40.tgz'
starting download of
`/shared/shape_tier3/common/local/guix/store/50l6s098sy4ig2irjm1g4c8y261nxzfc-openldap-2.4.40.tgz'
from `
ftp://sunsite.cnlab-switch.ch/mirror/OpenLDAP/openldap-release/openldap-2.4.40.tgz'.
..
ERROR: In procedure getaddrinfo: Name or service not known
failed to download
"/shared/shape_tier3/common/local/guix/store/50l6s098sy4ig2irjm1g4c8y261nxzfc-openldap-2.4.40.tgz"
from "
ftp://sunsite.cnlab-switch.ch/mirror/OpenLDAP/openldap-release/openldap-2.4.40.tgz
"

I tried modifying gnu/packages/openldap.scm to use correct url (
ftp://mirror.switch.ch/mirror/OpenLDAP/openldap-release/...) but that
didn't change anything since (I am guessing) the .drv files in store
weren't rebuilt and still point to the old url. How do I fix that without
rebuilding everything that it created so far?

As a side note, would it be possible to modify the files you suggested (gnu
packages package-management and gnu system instal) for the custom local
state dir in such a way (through configure? scheme variable?) that it
wouldn't be hard-coded? I am guessing any update to guix would cause my
changes to be lost?

Thanks,
Alex

On Wed, May 27, 2015 at 3:51 PM, Ludovic Courtès  wrote:

> Alexander Vorobiev  skribis:
>
> > I modified the files (to use my paths for the cache and store) and ran
> > guix-daemon as root. Now it got much, much further! But still failed, it
> > seems while building perl. Here is the end of the log file:
> >
> > In unknown file:
> >?: 5 [load-compiled/vm
> >
> "/home/alex/.cache/guile/ccache/2.0-LE-8-2.0/home/alex/src/guix/build-aux/make-binary-tarball.scm.go"]
> > In build-aux/make-binary-tarball.scm:
> >   38: 4 [#]
> > In guix/store.scm:
> >  986: 3 [run-with-store # ...]
> > In build-aux/make-binary-tarball.scm:
> >   41: 2 [# > (state)> #]
> > In guix/store.scm:
> >  933: 1 [# #]
> >  624: 0 [build-things # #]
> >
> > guix/store.scm:624:0: In procedure build-things:
> > guix/store.scm:624:0: Throw to key `srfi-34' with args `(# > &nix-protocol-error [message: "|   |   |   bind mounting `/dev/full' to
> >
> `/shared/shape_tier3/commo/local/guix/store/828rlmpkg4bi5i28lif2a1af7g4s62yq-perl-5.16.1.drv.chroot/dev/full'"
> > status: 1] 4b4cae0>)'.
> > Makefile:4331: recipe for target 'guix-binary.x86_64-linux.tar.xz' failed
>
> You seem to be running the daemon with --debug no?  What surprises me is
> that the above is not an error message but a debugging message.  The
> actual error seems to be elsewhere.
>
> > I also tried to use --disable-chroot but it also failed while building
> > perl. The first error message seems to be during perl's configure run:
>
> If you can run the daemon as root, then do run it as root, but without
> --disable-chroot; as the manual states, --disable-chroot is a last
> resort, and it’s only needed if you cannot run the daemon as root.
>
> In the case of Perl, I wouldn’t be surprised if its Configure script
> looks at stuff from the host distro in /usr/lib and so on, which can
> lead to a failure down the path.
>
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alexander Vorobiev  skribis:

> I modified the files (to use my paths for the cache and store) and ran
> guix-daemon as root. Now it got much, much further! But still failed, it
> seems while building perl. Here is the end of the log file:
>
> In unknown file:
>?: 5 [load-compiled/vm
> "/home/alex/.cache/guile/ccache/2.0-LE-8-2.0/home/alex/src/guix/build-aux/make-binary-tarball.scm.go"]
> In build-aux/make-binary-tarball.scm:
>   38: 4 [#]
> In guix/store.scm:
>  986: 3 [run-with-store # ...]
> In build-aux/make-binary-tarball.scm:
>   41: 2 [# (state)> #]
> In guix/store.scm:
>  933: 1 [# #]
>  624: 0 [build-things # #]
>
> guix/store.scm:624:0: In procedure build-things:
> guix/store.scm:624:0: Throw to key `srfi-34' with args `(# &nix-protocol-error [message: "|   |   |   bind mounting `/dev/full' to
> `/shared/shape_tier3/commo/local/guix/store/828rlmpkg4bi5i28lif2a1af7g4s62yq-perl-5.16.1.drv.chroot/dev/full'"
> status: 1] 4b4cae0>)'.
> Makefile:4331: recipe for target 'guix-binary.x86_64-linux.tar.xz' failed

You seem to be running the daemon with --debug no?  What surprises me is
that the above is not an error message but a debugging message.  The
actual error seems to be elsewhere.

> I also tried to use --disable-chroot but it also failed while building
> perl. The first error message seems to be during perl's configure run:

If you can run the daemon as root, then do run it as root, but without
--disable-chroot; as the manual states, --disable-chroot is a last
resort, and it’s only needed if you cannot run the daemon as root.

In the case of Perl, I wouldn’t be surprised if its Configure script
looks at stuff from the host distro in /usr/lib and so on, which can
lead to a failure down the path.

Ludo’.

Re: Daemon update

[Alexander Vorobiev]

Hi,
I modified the files (to use my paths for the cache and store) and ran
guix-daemon as root. Now it got much, much further! But still failed, it
seems while building perl. Here is the end of the log file:

In unknown file:
   ?: 5 [load-compiled/vm
"/home/alex/.cache/guile/ccache/2.0-LE-8-2.0/home/alex/src/guix/build-aux/make-binary-tarball.scm.go"]
In build-aux/make-binary-tarball.scm:
  38: 4 [#]
In guix/store.scm:
 986: 3 [run-with-store # ...]
In build-aux/make-binary-tarball.scm:
  41: 2 [# #]
In guix/store.scm:
 933: 1 [# #]
 624: 0 [build-things # #]

guix/store.scm:624:0: In procedure build-things:
guix/store.scm:624:0: Throw to key `srfi-34' with args `(#)'.
Makefile:4331: recipe for target 'guix-binary.x86_64-linux.tar.xz' failed
make: [guix-binary.x86_64-linux.tar.xz] Error 1 (ignored)

/dev/full seems to be where is it supposed to be.

I also tried to use --disable-chroot but it also failed while building
perl. The first error message seems to be during perl's configure run:

Checking your choice of C compiler and flags for coherency...
I've tried to compile and run the following simple program:

#include 
int main() { printf("Ok\n"); return(0); }

I used the command:

gcc -o try -O2 -fno-strict-aliasing -pipe -fstack-protector
-I/shared/shape_tier3/common/local/guix/store/idc1i3zmd0i04h3n9ii9pz1k1wi7z16z-glibc-bootstrap-0/include
-fstack-protector
-L/shared/shape_tier3/common/local/guix/store/idc1i3zmd0i04h3n9ii9pz1k1wi7z16z-glibc-bootstrap-0/lib
try.c -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
 ./try

and I got the following output:

/shared/shape_tier3/common/local/guix/store/714jwgzxf49qw2ysv8p9zz80b1q8bv5n-binutils-bootstrap-0/bin/ld:
cannot find -lgdbm
/shared/shape_tier3/common/local/guix/store/714jwgzxf49qw2ysv8p9zz80b1q8bv5n-binutils-bootstrap-0/bin/ld:
cannot find -ldb
collect2: error: ld returned 1 exit status
I can't compile the test program.
(The supplied flags or libraries might be incorrect.)

You have a BIG problem.  Shall I abort Configure [y]
Ok.  Stopping Configure.


Thanks,
Alex


On Wed, May 27, 2015 at 10:18 AM, Ludovic Courtès  wrote:

> Alexander Vorobiev  skribis:
>
> > Ok, I have just tried to build the binary tarball on a VM where I
> > reproduced all the paths I want (basically, instead of /gnu I want
> > /shared/shape_tier3/common/local/guix) and which has c++11 compliant gcc
> --
> > that also failed.
>
> What failed exactly?
>
> Note that ‘make guix-binary.x86_64-linux.tar.xz’ produces a tarball that
> uses the exact same store directory as the host Guix.  So if you want to
> produce a binary tarball with a custom store directory, you first need
> to build Guix --with-store-dir=/custom/store.
>
> If, in addition, you want a custom $localstatedir, then at least the
> ‘guix’ recipe in (gnu packages package-management) must be adjusted, as
> well as the “/var” in ‘self-contained-tarball’ in (gnu system install).
>
> > I pulled today's git and ran guix-daemon --no-substitutes. The error
> > seems to be "unable to fork: Operation not permitted". I am running
> > guix-daemon as myself
>
> When running guix-daemon unprivileged, you need to pass --disable-chroot
> (EACCES is returned by clone(2) here since we ask for separate name
> spaces, which only root or someone with CAP_SYS_ADMIN can do.)
>
> It’s better to use run it as root though (info "(guix) Invoking
> guix-daemon").
>
> Thanks,
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alexander Vorobiev  skribis:

> Ok, I have just tried to build the binary tarball on a VM where I
> reproduced all the paths I want (basically, instead of /gnu I want
> /shared/shape_tier3/common/local/guix) and which has c++11 compliant gcc --
> that also failed.

What failed exactly?

Note that ‘make guix-binary.x86_64-linux.tar.xz’ produces a tarball that
uses the exact same store directory as the host Guix.  So if you want to
produce a binary tarball with a custom store directory, you first need
to build Guix --with-store-dir=/custom/store.

If, in addition, you want a custom $localstatedir, then at least the
‘guix’ recipe in (gnu packages package-management) must be adjusted, as
well as the “/var” in ‘self-contained-tarball’ in (gnu system install).

> I pulled today's git and ran guix-daemon --no-substitutes. The error
> seems to be "unable to fork: Operation not permitted". I am running
> guix-daemon as myself

When running guix-daemon unprivileged, you need to pass --disable-chroot
(EACCES is returned by clone(2) here since we ask for separate name
spaces, which only root or someone with CAP_SYS_ADMIN can do.)

It’s better to use run it as root though (info "(guix) Invoking
guix-daemon").

Thanks,
Ludo’.

Re: Daemon update

[Alexander Vorobiev]

Ok, I have just tried to build the binary tarball on a VM where I
reproduced all the paths I want (basically, instead of /gnu I want
/shared/shape_tier3/common/local/guix) and which has c++11 compliant gcc --
that also failed. I pulled today's git and ran guix-daemon
--no-substitutes. The error seems to be "unable to fork: Operation not
permitted". I am running guix-daemon as myself and I have all permissions
to the /.../local/guix):

GUIX_PACKAGE_PATH= \
./pre-inst-env "/usr/bin/guile" \
  "./build-aux/make-binary-tarball.scm" "x86_64-linux"
"guix-binary.x86_64-linux.tar.xz"

.


|   recursively deleting path
`/shared/shape_tier3/common/local/guix/store/m38laca1idmvx8m03dl6b5qmn8xhv9c3-guile-bootstrap-2.0.drv.chroot'
|   lock released on
`/shared/shape_tier3/common/local/guix/store/pgs6vqy03wg2m5mn2ahvi4sarx89vnls-guile-bootstrap-2.0.lock'
|   building of
`/shared/shape_tier3/common/local/guix/store/m38laca1idmvx8m03dl6b5qmn8xhv9c3-guile-bootstrap-2.0.drv':
goal destroyed
Backtrace:
In ice-9/boot-9.scm:
 157: 11 [catch #t # ...]
In unknown file:
   ?: 10 [apply-smob/1 #]
In ice-9/boot-9.scm:
  63: 9 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 8 [eval # #]
In ice-9/boot-9.scm:
2401: 7 [save-module-excursion #]
4052: 6 [#]
In unknown file:
   ?: 5 [load-compiled/vm
"/home/alex/.cache/guile/ccache/2.0-LE-8-2.0/home/alex/src/guix/build-aux/make-binary-tarball.scm.go"]
In build-aux/make-binary-tarball.scm:
  38: 4 [#]
In guix/store.scm:
 986: 3 [run-with-store # ...]
In build-aux/make-binary-tarball.scm:
  41: 2 [# #]
In guix/store.scm:
 933: 1 [# #]
 624: 0 [build-things # #]

guix/store.scm:624:0: In procedure build-things:
guix/store.scm:624:0: Throw to key `srfi-34' with args `(#)'.
Makefile:4331: recipe for target 'guix-binary.x86_64-linux.tar.xz' failed
make: [guix-binary.x86_64-linux.tar.xz] Error 1 (ignored)

Thanks,
Alex

On Fri, May 22, 2015 at 8:45 AM, Ludovic Courtès  wrote:

> Alexander Vorobiev  skribis:
>
> > Doesn't binary distribution have hardcoded /gnu/* paths?
>
> Yes it does.
>
> > I can't use those unfortunately. We have a standard configuration of
> > RHEL 6.5 installed on hundreds of servers and any modifications of the
> > root directory (and all other "standard" directories) layout are out
> > of question.
>
> OK.  Note that using a store other than /gnu/store prevents you from
> using substitutes from hydra.gnu.org (just mentioning it if that is of
> interest to you.)
>
> > Would it be too hard to add an environment variable(s) pointing to a
> > Guix's store and cache directories so that the binary build of the
> > daemon doesn't depend on the hardcoded values?  Thanks, Alex
>
> There are two issues here:
>
>   1. File names hard-coded in the RUNPATH and other places of the
>  ‘guix-daemon’ binary of the tarball.
>
>   2. The store directory that ‘guix-daemon’ targets.
>
> #2 can actually be changed at run-time via the (purposefully
> undocumented) ‘NIX_STORE_DIR’ environment variable (see ‘test-env.in’ in
> the source tree.)
>
> #1 means that if you want to run the ‘guix-daemon’ binary from the
> binary tarball, you’ll have to somehow satisfy its requirements.  You
> might be able to unpack the tarball in an arbitrary directory, and then
> define ‘LD_LIBRARY_PATH’ so that ‘guix-daemon’ finds all the .so files
> that it needs and are provided by the binary tarball (libstdc++.so,
> libgcrypt.so, libsqlite3.so, libc.so, etc.)
>
> Once ‘guix-daemon’ is running, you could run ‘guix environment guix’,
> and from there you could build Guix from source.
>
> But that is really hacky, and perhaps not particularly easier than
> building a newer GCC on your machine.
>
> Alternately, you could build a suitable binary tarball on another
> machine: just install and configure Guix with the options that you need,
> and run ‘make guix-binary.x86_64-linux.tar.xz’.
>
> WDYT?
>
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alexander Vorobiev  skribis:

> Doesn't binary distribution have hardcoded /gnu/* paths?

Yes it does.

> I can't use those unfortunately. We have a standard configuration of
> RHEL 6.5 installed on hundreds of servers and any modifications of the
> root directory (and all other "standard" directories) layout are out
> of question.

OK.  Note that using a store other than /gnu/store prevents you from
using substitutes from hydra.gnu.org (just mentioning it if that is of
interest to you.)

> Would it be too hard to add an environment variable(s) pointing to a
> Guix's store and cache directories so that the binary build of the
> daemon doesn't depend on the hardcoded values?  Thanks, Alex

There are two issues here:

  1. File names hard-coded in the RUNPATH and other places of the
 ‘guix-daemon’ binary of the tarball.

  2. The store directory that ‘guix-daemon’ targets.

#2 can actually be changed at run-time via the (purposefully
undocumented) ‘NIX_STORE_DIR’ environment variable (see ‘test-env.in’ in
the source tree.)

#1 means that if you want to run the ‘guix-daemon’ binary from the
binary tarball, you’ll have to somehow satisfy its requirements.  You
might be able to unpack the tarball in an arbitrary directory, and then
define ‘LD_LIBRARY_PATH’ so that ‘guix-daemon’ finds all the .so files
that it needs and are provided by the binary tarball (libstdc++.so,
libgcrypt.so, libsqlite3.so, libc.so, etc.)

Once ‘guix-daemon’ is running, you could run ‘guix environment guix’,
and from there you could build Guix from source.

But that is really hacky, and perhaps not particularly easier than
building a newer GCC on your machine.

Alternately, you could build a suitable binary tarball on another
machine: just install and configure Guix with the options that you need,
and run ‘make guix-binary.x86_64-linux.tar.xz’.

WDYT?

Ludo’.

Re: Daemon update

[Taylan Ulrich Bayırlı/Kammer]

Alexander Vorobiev  writes:

> Doesn't binary distribution have hardcoded /gnu/* paths? I can't use
> those unfortunately. We have a standard configuration of RHEL 6.5
> installed on hundreds of servers and any modifications of the root
> directory (and all other "standard" directories) layout are out of
> question. Would it be too hard to add an environment variable(s)
> pointing to a Guix's store and cache directories so that the binary
> build of the daemon doesn't depend on the hardcoded values?

I'm afraid it's not the daemon itself for which changing the store path
is problematic (although it's not supported at run-time AFAIK, only at
configure-time via --with-store-dir=), it's that any built packages in
/gnu/store are likely to have "hardcoded" absolute references to other
/gnu/store files.

I think this can't really be avoided either, because some pieces of
software have configuration files and such which only support plain file
paths without any dynamic parameterization, meaning the recipe for the
package has to patch said files to contain absolute references to
/gnu/store files when they have to refer to any files from their
dependencies.

For example, when sources of package A contain a dlopen() call to open a
library from package B, then A's source is patched to turn the file name
argument to dlopen() into an absolute path to package B's library file
in /gnu/store.  That is because dlopen() does not support changes to its
search path in a portable manner.  (Maybe one day we will hack dlopen to
fix this, but it's just one example.)

This would mean that to use a non-standard store directory, you will
have to build Guix packages locally, as well as Guix itself.  Unless I'm
missing something.


You could still use the binary tarballs to install Guix on a machine
where you are permitted to, so you can easily compile another Guix via
the method described at the bottom here:

https://lists.gnu.org/archive/html/guix-devel/2015-05/msg00360.html


I'm afraid this doesn't help very much, but it's all I've got. :-\

Taylan

Re: Daemon update

[Alexander Vorobiev]

Ludo', Taylan,

Doesn't binary distribution have hardcoded /gnu/* paths? I can't use those
unfortunately. We have a standard configuration of RHEL 6.5 installed on
hundreds of servers and any modifications of the root directory (and all
other "standard" directories) layout are out of question. Would it be too
hard to add an environment variable(s) pointing to a Guix's store and cache
directories so that the binary build of the daemon doesn't depend on the
hardcoded values?
Thanks,
Alex

On Thu, May 21, 2015 at 3:24 AM, Ludovic Courtès  wrote:

> Alex Vorobiev  skribis:
>
> > Does it mean it can't be installed on systems with older c++
> > compilers?
>
> On these systems, I would suggest installing from the binary tarball.
>
> Not having to build Guile, libgcrypt, and everything on such a system is
> already a good incentive to use the binary tarball.
>
> How does that sound?
>
> Thanks,
> Ludo’.
>

Re: Daemon update

[Ludovic Courtès]

Alex Vorobiev  skribis:

> Does it mean it can't be installed on systems with older c++
> compilers?

On these systems, I would suggest installing from the binary tarball.

Not having to build Guile, libgcrypt, and everything on such a system is
already a good incentive to use the binary tarball.

How does that sound?

Thanks,
Ludo’.

Re: Daemon update

[Taylan Ulrich Bayırlı/Kammer]

Alex Vorobiev  writes:

> Ludovic Courtès  gnu.org> writes:
>> bunch of C++11 lambdas here, a bit of ‘auto’ there.)  
>
> Does it mean it can't be installed on systems with older c++
> compilers?

FYI Guix can be "bootstrapped" very easily with the new binary tarballs
without needing to compile anything.

See the master branch's manual for some pitfalls that made it into the
0.8.2 release.  (Namely, extracting the binary tarball might change the
ownership and permissions of /, /root, and /var, if you don't pass
--skip-old-files to tar.)

After extracting the tarball there will be a /root/.guix-profile with
guix installed, and any user can call:

  /gnu/store/...-guix-.../bin/guix package -i guix

to create their ~/.guix-profile with guix installed.

After that, git checkouts of guix can be built via:

  guix environment guix --pure -E 'autoreconf -vif'
  guix environment guix --pure -E \
"./configure --with-libgcrypt-prefix=$(guix build libgcrypt | head -1)"
  guix environment guix --pure -E make

and for later re-compilations only the last line is necessary, and
sometimes the first.  'make install' should probably be avoided.

Hope that helps,
Taylan

Re: Daemon update

[Alex Vorobiev]

Hi,

Ludovic Courtès  gnu.org> writes:
> bunch of C++11 lambdas here, a bit of ‘auto’ there.)  

Does it mean it can't be installed on systems with older c++ compilers? I 
am on RHEL 6.5 which has gcc-4.4.7 and ironically one of my main reasons to 
try guix was to be able to install newer versions of various applications 
so that they coexisted with the ones which were shipped with the system and 
didn't break anything. My current solution is to build everything myself 
and use GNU stow to keep it separate.

Thanks,
Alex

Daemon update

[Ludovic Courtès]

Hello!

I have pushed a big update of the daemon code.  We were using a copy of
the Nix code from May 2014, and there have been worthwhile changes in
the meantime (thanks for the Nix folks!).

There have also been undesirable changes, such as the gratuitous switch
to multithreading, which entails various issues (see
.)

Thus, in our ‘nix’ branch, I have selected a the best of those changes
;-).  The main difference is that I didn’t take said
multithreading-related commits, as well as a couple of less significant,
Nix-specific changes.

The new changes include some refactorization and cosmetic changes (a
bunch of C++11 lambdas here, a bit of ‘auto’ there.)  Interesting
changes include:

  • The possibility to use regular files as GC roots, where the file
name denotes a store item; this reduces I/O compared to using
symlinks.

  • Slightly reduced I/O for GC.

  • New ‘disallowedReferences’ and ‘disallowedRequisites’ attributes for
derivations (akin to our #:allowed-references.)

  • New ‘optimiseStore’ RPC.

  • ASLR disabled in the build environment.

Please report any issues with the daemon!

Thanks,
Ludo’.

Daemon update

[Ludovic Courtès]

Commit 1303a4a fixes a use-after-free in the daemon that manifested when
compiling it with GCC 5.1 (string destructors were called before
‘execve’ calls that used the underlying ‘char *’ pointers.)

The fix is a backport of an upstream commit.  I plan to update our copy
of the Nix daemon code eventually, but it seemed wiser to do that after
the release, hence this backport.

Please report any issues.

Thanks,
Ludo’.

Daemon update: rebuild!

[Ludovic Courtès]

I made another update of the ‘nix-upstream’ submodule, so make sure to
run:

  git pull  # should update ‘nix-upstream’ along with the rest
  ( cd nix ; ./sync-with-upstream )
  make clean
  make
  make check

Please report any problems!

Ludo’.

Re: Heads-up: daemon update

[Andreas Enge]

On Tue, Jan 21, 2014 at 11:51:08PM +0100, Ludovic Courtès wrote:
> Do you have a way to reproduce it (command line + “will be downloaded” list)?
> I tried a few substitutions at random and everything went fine.

Now everything goes well for me, too. No idea what happened.

Andreas

Re: Heads-up: daemon update

[Ludovic Courtès]

Hi!

(It took me a while to respond, notably because I was chasing a
web-related bug I had just introduced in Guile.  Dog food!)

Andreas Enge  skribis:

> Now I get messages like
>guix build: error: build failed: got unexpected path 
> `/nix/store/y7hz3la98sk4lmxmsvzbvjxifz4yrnwz-tzdata-2013d' from substitute

Do you have a way to reproduce it (command line + “will be downloaded” list)?

I tried a few substitutions at random and everything went fine.

> Maybe we need to clear /nix/store on hydra and restart it to fill the store
> again?

I don’t think so (especially since it would take ges to rebuild
everything ;-)).

Ludo’.

Re: Heads-up: daemon update

[Ludovic Courtès]

Mark H Weaver  skribis:

> l...@gnu.org (Ludovic Courtès) writes:
>
>> Commit d43eb49 updates the Nix sub-module that provides the daemon’s
>> C++ code.
>>
>> To update, run:
>>
>>   ( cd nix ; ./sync-with-upstream )
>>   autoreconf -vfi
>>   ./configure ... && make && make check
>
> A few of us ran into problems with these instructions.  It led to:
>
>   "No rule to make target `nix/libutil/affinity.cc', needed by
>`nix/libutil/libutil_a-affinity.o'."
>
> We had to run "git submodule update", and then follow Ludovic's
> instructions again from the top.

Oh right, sorry.  I thought ‘sync-with-upstream’ did that.

Thanks,
Ludo’.

Re: Heads-up: daemon update

[Mark H Weaver]

l...@gnu.org (Ludovic Courtès) writes:

> Commit d43eb49 updates the Nix sub-module that provides the daemon’s
> C++ code.
>
> To update, run:
>
>   ( cd nix ; ./sync-with-upstream )
>   autoreconf -vfi
>   ./configure ... && make && make check

A few of us ran into problems with these instructions.  It led to:

  "No rule to make target `nix/libutil/affinity.cc', needed by
   `nix/libutil/libutil_a-affinity.o'."

We had to run "git submodule update", and then follow Ludovic's
instructions again from the top.

 Mark

Re: Heads-up: daemon update

[Andreas Enge]

Now I get messages like
   guix build: error: build failed: got unexpected path 
`/nix/store/y7hz3la98sk4lmxmsvzbvjxifz4yrnwz-tzdata-2013d' from substituter

Maybe we need to clear /nix/store on hydra and restart it to fill the store
again?

Andreas

Heads-up: daemon update

[Ludovic Courtès]

Commit d43eb49 updates the Nix sub-module that provides the daemon’s
C++ code.

To update, run:

  ( cd nix ; ./sync-with-upstream )
  autoreconf -vfi
  ./configure ... && make && make check

Please report any test suite failure.

The changes compared to the daemon we had so far include interesting bug
fixes and improvements (see git log below for the libstore part.)

Thanks for the Nix hackers for that! :-)

Ludo’.


commit 11cb4bfb257f18c906ef1d6f14ed450be8fa49fe
Author: Eelco Dolstra 
Date:   Wed Jan 8 17:32:40 2014 +0100

Fix checking of NAR hashes

*headdesk*
*headdesk*
*headdesk*

So since commit 22144afa8d9f8968da351618a1347072a93bd8aa, Nix hasn't
actually checked whether the content of a downloaded NAR matches the
hash specified in the manifest / NAR info file.  Urghhh...

commit 485f4740ee3ba4228ba3345873eb530466a8f42d
Author: Domen Kožar 
Date:   Wed Jan 1 18:10:48 2014 +0100

wording

commit a6add93d734279db8503951ac6466c275b3c8e4e
Author: Eelco Dolstra 
Date:   Tue Dec 10 13:13:59 2013 +0100

Garbage collector: Release locks on temporary root files

This allows processes waiting for such locks to proceed during the
trash deletion phase of the garbage collector.

commit c5b8fe315162440c1d808bc0a684a412d78bfa76
Author: Eelco Dolstra 
Date:   Thu Dec 5 14:31:57 2013 -0500

Print a trace message if a build fails due to the platform being unknown

commit 709cbe4e76e7b0f1b8abddbeb7714e194f6f8a02
Author: Eelco Dolstra 
Date:   Fri Nov 22 10:00:43 2013 +

Include  for memset

This should fix building on Illumos.

commit a478e8a7bb8c24da0ac91b7100bd0e422035c62f
Author: Eelco Dolstra 
Date:   Thu Nov 14 11:57:37 2013 +0100

Remove nix-setuid-helper

AFAIK, nobody uses it, it's not maintained, and it has no tests.

commit 89e6781cc5885cbf6284a51c0403dded62ce8bc0
Author: Eelco Dolstra 
Date:   Tue Nov 12 12:51:59 2013 +0100

Make function calls show up in stack traces again

Note that adding --show-trace prevents functions calls from being
tail-recursive, so an expression that evaluates without --show-trace
may fail with a stack overflow if --show-trace is given.

commit c0861838432ea9d8e5ea5750aadfbc59ebd6f3b4
Author: Eelco Dolstra 
Date:   Tue Oct 22 11:39:10 2013 +0200

For auto roots, show the intermediate link

I.e. "nix-store -q --roots" will now show (for example)

  /home/eelco/Dev/nixpkgs/result

rather than

  /nix/var/nix/gcroots/auto/53222qsppi12s2hkap8dm2lg8xhhyk6v

commit a737f51fd96be2866a33eb67732e034bcc65a659
Author: Eelco Dolstra 
Date:   Wed Oct 16 15:58:20 2013 +0200

Retry all SQLite operations

To deal with SQLITE_PROTOCOL, we also need to retry read-only
operations.

commit ff02f5336cd0cff0e97fbcf3c54b5b23827702d6
Author: Eelco Dolstra 
Date:   Wed Oct 16 14:51:20 2013 +0200

Fix a race in registerFailedPath()

Registering the path as failed can fail if another process does the
same thing after the call to hasPathFailed().  This is extremely
unlikely though.

commit 4bd52825734face53df2ab00052d2457d31c3c68
Author: Eelco Dolstra 
Date:   Wed Oct 16 14:46:35 2013 +0200

Convenience macros for retrying a SQLite transaction

commit bce14d0f61801f0f1c3080970619e2ca11683a4e
Author: Eelco Dolstra 
Date:   Wed Oct 16 14:36:53 2013 +0200

Don't wrap read-only queries in a transaction

There is no risk of getting an inconsistent result here: if the ID
returned by queryValidPathId() is deleted from the database
concurrently, subsequent queries involving that ID will simply fail
(since IDs are never reused).

commit 7cdefdbe732c209e13f234eb71022791909a5518
Author: Eelco Dolstra 
Date:   Wed Oct 16 14:27:36 2013 +0200

Print a distinct warning for SQLITE_PROTOCOL

commit d05bf01dbf8e36d545df9689bdec0b72
Author: Eelco Dolstra 
Date:   Wed Oct 16 14:03:22 2013 +0200

Treat SQLITE_PROTOCOL as SQLITE_BUSY

In the Hydra build farm we fairly regularly get SQLITE_PROTOCOL errors
(e.g., "querying path in database: locking protocol").  The docs for
this error code say that it "is returned if some other process is
messing with file locks and has violated the file locking protocol
that SQLite uses on its rollback journal files."  However, the SQLite
source code reveals that this error can also occur under high load:

  if( cnt>5 ){
int nDelay = 1;  /* Pause time in microseconds */
if( cnt>100 ){
  VVA_ONLY( pWal->lockError = 1; )
  return SQLITE_PROTOCOL;
}
if( cnt>=10 ) nDelay = (cnt-9)*238;  /* Max delay 21ms. Total delay 
996ms */
sqlite3OsSleep(pWal->pVfs, nDelay);
  }

i.e. if certain locks cannot be not acquired, SQLite will retry a
number of times before giving up and returing SQLITE_PROTOCOL.  The
comments say:

  Circumstances that cause a R