Re: Is using xdg-utils post-install safe?

[Jesse Gibbons]

On Mon, 2019-10-28 at 20:12 +0100, Tobias Geerinckx-Rice wrote:
> Jesse,
> 
> Jesse Gibbons 写道：
> > I'm working on packaging The Powder Toy. When it first starts, 
> > it asks if
> > the user wants to install it.
> 
> ‘Install’?  That doesn't sound good.  :-)
> 
> <
> https://github.com/The-Powder-Toy/The-Powder-Toy/blob/master/src/client/Client.cpp#L366>
> ;.
Thanks for looking it up for me. This will save me some grepping.
> 
> From my reading, that will embed a store file name into a .desktop 
> file somewhere in (each) user's $HOME, which will break sooner or 
> later.
> 
I can confirm that is an accurate summary of what it does. Letting it stay
would be an error waiting to happen. 
> > When the user clicks confirm, it tries to use
> > xdg-desktop-menu, xdg-mime, and xdg-icon-resource.
> > Is this a safe behavior with guix, or should I try to patch it 
> > out?
> > If it is safe, should I bother trying to install a desktop icon 
> > with guix if
> > the program will do that itself?
> 
> Even if I'm wrong and it's ‘safe’, I don't see how this behaviour 
> is desirable.  What is upstream trying to accomplish here?
If a user decides to build from source, installing the information on first
launch would allow the user to "install" it without moving any binary. It's
just speculation, but this could be the theory behind this questionable
design choice.
>  Do you see how this would be helpful compared to doing things 
> the normally?
I cannot think of any advantage the average user running guix would gain in
leaving it in. If anything it would make the first launch a little more
complex because there would be no icon until then. 
> 
> If not, I think we have a responsibility towards our users to 
> install a copy of the above .desktop file and patch out any 
> xdg-desktop-menu calls.  Same for MIME types and icons.
I agree. I'll work on it now and send the patch when everything works.
> 
> Kind regards,
> 
> T G-R
-- 
-Jesse

Re: Clang c++ include path

[Marius Bakke]

Mark H Weaver  writes:

> Hi Ricardo and Mathieu,
>
> Ricardo Wurmus  writes:
>
>>> When running clang on a c++ program, it cannot find c++ std libraries.
>>> That's because, those libraries path are hardcoded inside g++ compiler
>>> and clang cannot find them.
>>
>> Does this patch help?
>
> I'd like to request that fixes to LLVM/Clang be done on another branch
> for now, or at least that the fixed versions are given a different
> variable name.  The reason is that IceCat depends on Rust which depends
> on Clang, and there is a chain of *18* Rust compilers that must be built
> before IceCat can be built.  On my system, compiling all of those Rust
> compilers requires approximately *90 hours* of continuous compiling.
>
> For the sake of Guix users like myself who do not want to trust the
> build farm, it would be good for it to remain reasonably viable to build
> everything locally.

I agree.  The CI system also spends days bootstrapping the Rust compiler
chain.  Any user who tries to install IceCat during that time will have
a terrible experience.

I've been keeping "libgit2" updates in 'staging' for that reason, even
though it is below the ~300 rebuild limit.


signature.asc
Description: PGP signature

Re: Clang c++ include path

[Mark H Weaver]

Hi Ricardo and Mathieu,

Ricardo Wurmus  writes:

>> When running clang on a c++ program, it cannot find c++ std libraries.
>> That's because, those libraries path are hardcoded inside g++ compiler
>> and clang cannot find them.
>
> Does this patch help?

I'd like to request that fixes to LLVM/Clang be done on another branch
for now, or at least that the fixed versions are given a different
variable name.  The reason is that IceCat depends on Rust which depends
on Clang, and there is a chain of *18* Rust compilers that must be built
before IceCat can be built.  On my system, compiling all of those Rust
compilers requires approximately *90 hours* of continuous compiling.

For the sake of Guix users like myself who do not want to trust the
build farm, it would be good for it to remain reasonably viable to build
everything locally.

What do you think?

  Mark

Re: Is using xdg-utils post-install safe?

[Tobias Geerinckx-Rice]

Jesse,

Jesse Gibbons 写道：
I'm working on packaging The Powder Toy. When it first starts, 
it asks if

the user wants to install it.


‘Install’?  That doesn't sound good.  :-)

.

From my reading, that will embed a store file name into a .desktop 
file somewhere in (each) user's $HOME, which will break sooner or 
later.



When the user clicks confirm, it tries to use
xdg-desktop-menu, xdg-mime, and xdg-icon-resource.
Is this a safe behavior with guix, or should I try to patch it 
out?
If it is safe, should I bother trying to install a desktop icon 
with guix if

the program will do that itself?


Even if I'm wrong and it's ‘safe’, I don't see how this behaviour 
is desirable.  What is upstream trying to accomplish here?  Do you 
see how this would be helpful compared to doing things the 
normally?


If not, I think we have a responsibility towards our users to 
install a copy of the above .desktop file and patch out any 
xdg-desktop-menu calls.  Same for MIME types and icons.


Kind regards,

T G-R


signature.asc
Description: PGP signature

Re: Clang c++ include path

[Ricardo Wurmus]

Hi Mathieu,

> When running clang on a c++ program, it cannot find c++ std libraries.
> That's because, those libraries path are hardcoded inside g++ compiler
> and clang cannot find them.

Does this patch help?

--8<---cut here---start->8---
diff --git a/gnu/packages/llvm.scm b/gnu/packages/llvm.scm
index 9efb4a4841..3c59e8d26d 100644
--- a/gnu/packages/llvm.scm
+++ b/gnu/packages/llvm.scm
@@ -204,7 +204,7 @@ compiler.  In LLVM this library is called \"compiler-rt\".")
(compiler-rt (assoc-ref inputs "clang-runtime")))
(case (string->number ,(version-major
(package-version 
clang-runtime)))
- ((or 6 7)
+ ((or 6 7 8)
   ;; Link to libclang_rt files from clang-runtime.
   (substitute* "lib/Driver/ToolChain.cpp"
 (("getDriver\\(\\)\\.ResourceDir")
@@ -363,16 +363,15 @@ requirements according to version 1.1 of the OpenCL 
specification.")
 "1mf9cpgvix34xlpv0inkgl3qmdvgvp96f7sksqizri0n5xfp1cgp"))
   (file-name (string-append "libomp-" version ".tar.xz"
 (build-system cmake-build-system)
-;; XXX: Note this gets built with GCC because building with Clang itself
-;; fails (missing , even when libcxx is added as an input.)
 (arguments
  '(#:configure-flags '("-DLIBOMP_USE_HWLOC=ON"
-   "-DOPENMP_TEST_C_COMPILER=clang"
-   "-DOPENMP_TEST_CXX_COMPILER=clang++")
+   "-DCMAKE_C_COMPILER=clang"
+   "-DCMAKE_CXX_COMPILER=clang++")
#:test-target "check-libomptarget"))
 (native-inputs
  `(("clang" ,clang)
("llvm" ,llvm)
+   ("libcxx" ,libcxx)
("perl" ,perl)
("pkg-config" ,pkg-config)))
 (inputs
--8<---cut here---end--->8---


-- 
Ricardo

Re: “Guix Profiles in Practice”

[Konrad Hinsen]

Hi Pierre,

>> Jumping to symbol definitions, for example, or access to documentation.
>
> I can jump to definition with no problem.  Is your load path set
> properly?

After inspection... no. At least not in all my Guix installations.
It does work fine with the right path.

> Access to the manual has been broken forever apparently.
>
> Work-around here: https://gitlab.com/jaor/geiser/issues/252

Great, thanks! The biggest improvement in my Guix experience for a
while!

Cheers,
  Konrad.

Re: Moving Lisp libraries to lisp-xyz.scm?

[Katherine Cox-Buday]

Pierre Neidhardt  writes:

> lisp.scm currently contains:
>
> - Lisp compilers.
> - Lisp libraries.
> - A few (3-4) Lisp-based programs.
>
> I suggest we move the lisp libraries to lisp-xyz.scm.  It would also be
> more consistent with what we do for other languages.
>
> Thoughts?

This sounds like a good idea to me.

-- 
Katherine

Is using xdg-utils post-install safe?

[Jesse Gibbons]

I'm working on packaging The Powder Toy. When it first starts, it asks if
the user wants to install it. When the user clicks confirm, it tries to use
xdg-desktop-menu, xdg-mime, and xdg-icon-resource.
Is this a safe behavior with guix, or should I try to patch it out?
If it is safe, should I bother trying to install a desktop icon with guix if
the program will do that itself?
-- 
-Jesse

Re: [PATCH v3 0/9] gnu: Add poetry.

[Tanguy Le Carrour]

Hi Mathieu,

Le 10/28, Mathieu Othacehe a écrit :
> > I've just rebased my patches for Poetry because they were few weeks old.
> > If someone had time to take a look at them, it would be great!
> 
> Apart from some style related issues, your serie seems fine. Can you
> please send a v4 :)

I've just sent the new patches… but wrote "v3" in the email when I actually
named the series "v4"! ^_^'

Thanks!

-- 
Tanguy

Re: [PATCH v3 0/9] gnu: Add poetry.

[Mathieu Othacehe]

Hello Tanguy,

> I've just rebased my patches for Poetry because they were few weeks old.
> If someone had time to take a look at them, it would be great!

Apart from some style related issues, your serie seems fine. Can you
please send a v4 :)

Thanks,

Mathieu

Granularity of grub-install (was Re: Reproducibility of grub-install)

[Miguel Arruga Vivas]

Hi,

First of all, thank you for your comments.  My answers are inline,
although the bad subject was a main issue of misunderstanding.  I
changed it, as I think it reflects better my current idea.  I'm sorry
for the confusion it certainly caused.

Wed, 23 Oct 2019 11:09:07 +0200
Daniel Kiper  wrote:
> On Mon, Oct 21, 2019 at 04:30:21PM +0200, Miguel Arruga Vivas wrote:
> > Hi, everybody!
> >
> > After taking a deeper look into our (guix's) grub installation
> > procedure, I have the thought that it could be a neat idea to make
> > the boot directory an actual derivation instead something of the
> > global status.  
> 
> I do not understand what you mean by "make the boot directory an
> actual derivation instead something of the global status". Could you
> elaborate more about that?

Currently, the /boot directory is partially a system-wide directory,
even though most of its contents are direct copies of the ones from
grub's installation prefix: /usr/... in most distributions including
BSD world, /{nix,gnu}/store/-grub-2.04 in Nix/Guix.  Most of its
contents don't depend at all on the actual machine and that can be
shared between machine-dependent boot-sector/efi-variables
"activations".  These contents include all grub modules and message
object files.  On the other hand, the task of the platform-specific
installation cannot be "shared", as it must be performed directly "on
the metal".  And there is a third task between these two: the generation
of the raw image, that depends on the hardware configuration through
the provided configuration file *and* the binary installation.

> (...)
> I am not sure why grub.cfg have to be reproducible. OK, to some extent
> it has but...

Guix doesn't generate grub.cfg through grub-mkconfig, as each kernel is
also associated with the configuration for the init system too--GNU
Shepherd in Guix--, and one kernel may boot up different systems, so it
is generated with Guile scripts.  My understanding is that grub-mkconfig
is a portable tool, not only intended for this case, whose output is
system dependant.  I mixed up concepts here, sorry.

> >
> >   - /boot/grub/grubenv: IIUC, this file must be writable by grub.
> > This should not be on the store, and not sharing the path may be the
> > main problem right now to implement this.  
> 
> I do not understand this.
>
> > AFAIK, the grubenv problem requires a modification of the grub code
> > if we try to use a different path for this kind-of-modifiable file,
> > so this would require modify grub to being able to lookup for that
> > file somewhere else.  This way the global state can be made
> > explicit.  
> 
> What do you mean by "This way the global state can be made explicit."?

Sorry again.  We do not use load_env nor save_env in our
configuration.  I misunderstood the creation in grub-install.c as a
hidden requirement for it, but I'd just had to read the manual.  That
global state already is explicit enough.  I think now that its creation
should be optional, even though it's created by default as
grub-mkconfig makes use of it.  Should I send a patch for it?

> > The image installation into the device is a separate issue from the
> > binaries installation, that could be separated into two separate
> > binaries, or two steps/flags for grub-install, one for binaries
> > installation into ${boot-directory}/grub and the other one for
> > load.cfg generation and core/boot.img installation.  
> 
> Why do you need to separate this thing into two steps?

I'd like being able to have different grub /boot-like folders, maybe
through unions in the file system (as hard links) of the shared
contents and the physical system dependencies, and activate them
selectively without writing into them any more.  This is intended for
Guix's store[1], but other use cases are possible, in order to roll-back
to a previous generation of the system by only writing
core.img/grub.efi/etc. into platform-dependant locations.

As I said before, I see them now as three steps.  The middle one is
already there with grub-mkimage.  The bios-like installation can be
performed with grub-setup, but EFI systems need grub-install to copy
the image and activate it, the reason behind grub-setup deprecation if I
understand the code correctly.  And the copy of the correct files to
the boot-directory needs no modification at all of grub-install.

> > To grub-devel: I'd be able to send patches for the latter if you
> > think  
> 
> Patches are always welcome...

I'll send just to grub-devel following this mail a patch for a minor
problem I've found reading the code. 

The code for an --only-platform-installation flag (the name I'm using
right now) needs more work but I'll send it as soon as I have
something I've tested on x86 BIOS and x86_64 EFI.  From my
current understanding it would replace completely grub-setup, wouldn't
it?

Happy hacking!
Miguel

[1] https://guix.gnu.org/manual/en/html_node/Features.html

[PATCH v3 0/9] gnu: Add poetry.

[Tanguy Le Carrour]

Hi Guix!

I've just rebased my patches for Poetry because they were few weeks old.
If someone had time to take a look at them, it would be great!

Regards,

-- 
Tanguy

Re: 06/06: gnu: Add weasyprint.

[Hartmut Goebel]

Am 23.10.19 um 16:24 schrieb Ludovic Courtès:
>
> Another thing: could you add a couple of lines at the top of the patch

Done: e30808ddb33f4aa0c4940ce8596ddc71ff55ed2a

Thanks for the review.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Getting build information in to the Guix Data Service (draft patch)

[Christopher Baines]

Christopher Baines  writes:

> Ludovic Courtès  writes:
>
>> Christopher Baines  skribis:
>>
>> Should ‘BuildEvents’ be more generic and have ‘event’ be an sexp or JSON
>> string that could describe any kind of event?
>>
>> If we did that, we could keep ‘derivation’ but remove “NOT NULL” so that
>> non-derivation events can exist but we can still query
>> derivation-related events quickly.  Does that make sense?
>
> Yep, that makes sense.
>
> This seems to be the general decision about the way you use a relational
> database, do you have specific tables (types) for the data, or do you
> have a more freeform structure (columns containing sexp or JSON).
>
> There's quite a few factors to consider here, the internals of Cuirass,
> how these events are exposed through the HTTP API, how these events one
> day might be published to a WebSub hub and then what kind of
> subscriptions you might support in Cuirass (events for an individual
> derivation, all builds for an evaluation, all builds, ...). I'll think
> about it further and see if I can form an opinion either way.

So I had a think about this, and also re-read the WebSub spec [1], along
with JSON Feed which is a Atom/RSS alternative with JSON [2]. I'm still
a bit undecided about how closely to follow WebSub, as it seems more
concerned with applications like blogging, and less with moving data
between services, but I still think the principles are good. For now, I
think it's best to pick the bits of the standard that work, and leave
any that don't particularly apply.

I think a generic approach to storing the events in the database will
work better though, at least with the way the events are currently
exposed. It does make it potentially harder to expose and support
subscriptions for events for individual builds, but that can probably be
addressed later if desired.

1: https://www.w3.org/TR/websub/
2: https://jsonfeed.org/version/1

>>> +CREATE TABLE BuildEventsOutbox (
>>> +  build_event_id INTEGER NOT NULL,
>>> +  FOREIGN KEY (build_event_id) REFERENCES BuildEvents (id)
>>> +);
>>
>> These are events that have not yet been sent, right?
>
> Yep, exactly.
>
>> Thanks!
>
> Thanks for taking a look. I'll neaten up the patch a bit, add in some
> error handling and retrying for sending out the events, and think a bit
> more about the data model, then hopefully send an updated patch soon!

I've now sent an updated set of patches. The first to add events for
builds, and the second to extend this to evaluations.

I've also added in error handling to the sending of the events.

Thanks,

Chris


signature.asc
Description: PGP signature

Clang c++ include path

[Mathieu Othacehe]

Hello,

When running clang on a c++ program, it cannot find c++ std libraries.
That's because, those libraries path are hardcoded inside g++ compiler
and clang cannot find them.

This has already been discussed here:
https://issues.guix.info/issue/32773, without reaching a solution.

Running this program,

--8<---cut here---start->8---
#include 

int main()
{
  return 0;
}
--8<---cut here---end--->8---

I have this error:

--8<---cut here---start->8---
mathieu@meru:~/tmp$ ~/.guix-profile/bin/clang t.cpp 
#include 
 ^~~~
t.cpp:1:10: fatal error: 'string' file not found
1 error generated.
--8<---cut here---end--->8---

that's because:

--8<---cut here---start->8---
mathieu@meru:~/tmp$ ~/.guix-profile/bin/clang -v t.cpp 
...
#include "..." search starts here:
#include <...> search starts here:
 /gnu/store/933ijsm5wwjkvlh963lc3pxc7i3dfxcp-profile/include
 /gnu/store/i2vzqb02gxcm5c2zix92lq3ay73kbl9s-clang-8.0.0/lib/clang/8.0.0/include
 /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include
End of search list.
#include 
 ^~~~
t.cpp:1:10: fatal error: 'string' file not found
1 error generated.
--8<---cut here---end--->8---

whereas,

--8<---cut here---start->8---
mathieu@meru:~/tmp$ ~/.guix-profile/bin/g++ -v  t.cpp
...
#include "..." search starts here:
#include <...> search starts here:
 /gnu/store/933ijsm5wwjkvlh963lc3pxc7i3dfxcp-profile/include
 /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/include/c++ [1]
 
/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/include/c++/x86_64-unknown-linux-gnu
 [2]
 /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/include/c++/backward [3]
 
/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include
 
/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include-fixed
 /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include
...
--8<---cut here---end--->8---

Hardcoding the paths [1], [2] and [3] into clang would solve the
issue, but it might not be the better way to proceed. WDYT?

Mathieu

Clang c++ include path

[Mathieu Othacehe]

Message-ID: <87r22xxoy6@gmail.com>
User-agent: mu4e 1.2.0; emacs 26.3

https://issues.guix.info/issue/32773,
mathieu@meru:~/tmp$ ~/.guix-profile/bin/clang t.cpp 
t.cpp:1:10: fatal error: 'string' file not found
mathieu@meru:~/tmp$ ~/.guix-profile/bin/clang -v t.cpp 
t.cpp:1:10: fatal error: 'string' file not found
mathieu@meru:~/tmp$ ~/.guix-profile/bin/g++ -v  t.cpp
Hello,

When running clang on a c++ program, it cannot find c++ std libraries.
That's because, those libraries path are hardcoded inside g++ compiler
and clang cannot find them.

This has already been discussed here:
without reaching a solution.

Running this program,

--8<---cut here---start->8---
#include 

int main()
{
  return 0;
}
--8<---cut here---end--->8---

I have this error:

--8<---cut here---start->8---
#include 
 ^~~~
1 error generated.
--8<---cut here---end--->8---

that's because:

--8<---cut here---start->8---
...
#include "..." search starts here:
#include <...> search starts here:
 /gnu/store/933ijsm5wwjkvlh963lc3pxc7i3dfxcp-profile/include
 /gnu/store/i2vzqb02gxcm5c2zix92lq3ay73kbl9s-clang-8.0.0/lib/clang/8.0.0/include
 /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include
End of search list.
#include 
 ^~~~
1 error generated.
--8<---cut here---end--->8---

whereas,

--8<---cut here---start->8---
...
#include "..." search starts here:
#include <...> search starts here:
 /gnu/store/933ijsm5wwjkvlh963lc3pxc7i3dfxcp-profile/include
 /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/include/c++ [1]
 
/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/include/c++/x86_64-unknown-linux-gnu
 [2]
 /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/include/c++/backward [3]
 
/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include
 
/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include-fixed
 /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include
...
--8<---cut here---end--->8---

Hardcoding the paths [1], [2] and [3] into clang would solve the
issue, but it might not be the better way to proceed. WDYT?

Mathieu

[PATCH 1/2] Support publishing build events

[Christopher Baines]

---
 Makefile.am |   8 +-
 bin/cuirass-send-events.in  |  90 +++
 src/cuirass/base.scm|   9 ++-
 src/cuirass/database.scm| 142 +---
 src/cuirass/http.scm|  24 ++
 src/cuirass/send-events.scm |  69 ++
 src/schema.sql  |  12 +++
 src/sql/upgrade-5.sql   |  15 
 8 files changed, 356 insertions(+), 13 deletions(-)
 create mode 100644 bin/cuirass-send-events.in
 create mode 100644 src/cuirass/send-events.scm
 create mode 100644 src/sql/upgrade-5.sql

diff --git a/Makefile.am b/Makefile.am
index 7cea2ff..5448420 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -21,7 +21,7 @@
 # You should have received a copy of the GNU General Public License
 # along with Cuirass.  If not, see .
 
-bin_SCRIPTS = bin/cuirass bin/evaluate
+bin_SCRIPTS = bin/cuirass bin/cuirass-send-events bin/evaluate
 noinst_SCRIPTS = pre-inst-env
 
 guilesitedir = $(datarootdir)/guile/site/@GUILE_EFFECTIVE_VERSION@
@@ -45,6 +45,7 @@ dist_pkgmodule_DATA = \
   src/cuirass/database.scm \
   src/cuirass/http.scm \
   src/cuirass/logging.scm  \
+  src/cuirass/send-events.scm  \
   src/cuirass/ui.scm   \
   src/cuirass/utils.scm \
   src/cuirass/templates.scm
@@ -68,7 +69,8 @@ dist_sql_DATA =   \
   src/sql/upgrade-1.sql\
   src/sql/upgrade-2.sql\
   src/sql/upgrade-3.sql\
-  src/sql/upgrade-4.sql
+  src/sql/upgrade-4.sql\
+  src/sql/upgrade-5.sql
 
 dist_css_DATA =\
   src/static/css/bootstrap.css \
@@ -143,6 +145,7 @@ sql-check: src/schema.sql
 EXTRA_DIST = \
   .dir-locals.el \
   bin/cuirass.in \
+  bin/cuirass-send-events.in \
   bin/evaluate.in \
   bootstrap \
   build-aux/guix.scm \
@@ -202,6 +205,7 @@ generate_file = \
 # These files depend on Makefile so they are rebuilt if $(VERSION),
 # $(datadir) or other do_subst'ituted variables change.
 bin/cuirass: $(srcdir)/bin/cuirass.in
+bin/cuirass-send-events: $(srcdir)/bin/cuirass-send-events.in
 bin/evaluate: $(srcdir)/bin/evaluate.in
 $(bin_SCRIPTS): Makefile
$(generate_file); chmod +x $@
diff --git a/bin/cuirass-send-events.in b/bin/cuirass-send-events.in
new file mode 100644
index 000..5f2e678
--- /dev/null
+++ b/bin/cuirass-send-events.in
@@ -0,0 +1,90 @@
+#!/bin/sh
+# -*- scheme -*-
+# @configure_input@
+#GUILE_LOAD_PATH="@PACKAGE_LOAD_PATH@${GUILE_LOAD_PATH:+:}$GUILE_LOAD_PATH"
+#GUILE_LOAD_COMPILED_PATH="@PACKAGE_LOAD_COMPILED_PATH@${GUILE_LOAD_COMPILED_PATH:+:}$GUILE_LOAD_COMPILED_PATH"
+exec ${GUILE:-@GUILE@} --no-auto-compile -e main -s "$0" "$@"
+!#
+ cuirass -- continuous integration tool
+;;; Copyright © 2016 Mathieu Lirzin 
+;;; Copyright © 2017 Mathieu Othacehe 
+;;; Copyright © 2018 Ludovic Courtès 
+;;; Copyright © 2018 Clément Lassieur 
+;;; Copyright © 2019 Christopher Baines 
+;;;
+;;; This file is part of Cuirass.
+;;;
+;;; Cuirass is free software: you can redistribute it and/or modify
+;;; it under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation, either version 3 of the License, or
+;;; (at your option) any later version.
+;;;
+;;; Cuirass is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with Cuirass.  If not, see .
+
+(use-modules (cuirass)
+ (cuirass ui)
+ (cuirass logging)
+ (cuirass utils)
+ (cuirass send-events)
+ (guix ui)
+ (fibers)
+ (fibers channels)
+ (srfi srfi-19)
+ (ice-9 getopt-long))
+
+(define (show-help)
+  (format #t "Usage: ~a [OPTIONS]~%" (%program-name))
+  (display "Send events to the target URL.
+
+  -T  --target-url=URL  Send events to URL.
+  -D  --database=DB Use DB to store build results.
+  -h, --helpDisplay this help message")
+  (newline)
+  (show-package-information))
+
+(define %options
+  '((target-url (single-char #\T) (value #t))
+(database   (single-char #\D) (value #t))
+(help   (single-char #\h) (value #f
+
+
+;;;
+;;; Entry point.
+;;;
+
+(define* (main #:optional (args (command-line)))
+
+  ;; Always have stdout/stderr line-buffered.
+  (setvbuf (current-output-port) 'line)
+  (setvbuf (current-error-port) 'line)
+
+  (let ((opts (getopt-long args %options)))
+(parameterize
+((%program-name (car a

[PATCH 2/2] Support publishing evaluation events

[Christopher Baines]

---
 src/cuirass/base.scm | 5 +
 src/cuirass/database.scm | 6 +-
 src/cuirass/http.scm | 9 +
 3 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/src/cuirass/base.scm b/src/cuirass/base.scm
index fd10e35..943a4f2 100644
--- a/src/cuirass/base.scm
+++ b/src/cuirass/base.scm
@@ -724,6 +724,11 @@ started)."
  (checkouts (fetch-inputs spec))
  (eval-id (db-add-evaluation name checkouts)))
 (when eval-id
+  (db-add-event 'evaluation
+(time-second (current-time time-utc))
+`((#:evalutaion. ,eval-id)
+  (#:specification . ,name)
+  (#:in_progress   . #t)))
   (compile-checkouts spec (filter compile? checkouts))
   (spawn-fiber
(lambda ()
diff --git a/src/cuirass/database.scm b/src/cuirass/database.scm
index 83c0c5a..05382d7 100644
--- a/src/cuirass/database.scm
+++ b/src/cuirass/database.scm
@@ -408,7 +408,11 @@ VALUES (" spec-name ", true);")
 (define (db-set-evaluation-done eval-id)
   (with-db-critical-section db
 (sqlite-exec db "UPDATE Evaluations SET in_progress = false
-WHERE id = " eval-id ";")))
+WHERE id = " eval-id ";")
+(db-add-event 'evaluation
+  (time-second (current-time time-utc))
+  `((#:evaluation  . ,eval-id)
+(#:in_progress . #f)
 
 (define-syntax-rule (with-database body ...)
   "Run BODY with %DB-CHANNEL being dynamically bound to a channel implementing
diff --git a/src/cuirass/http.scm b/src/cuirass/http.scm
index 35e3d7f..4a3214d 100644
--- a/src/cuirass/http.scm
+++ b/src/cuirass/http.scm
@@ -347,6 +347,15 @@ Hydra format."
 (specifications-table
  (db-get-specifications))
 '(
+(("api" "evaluation-events")
+ (let* ((params (request-parameters request))
+;; 'nr parameter is mandatory to limit query size.
+(valid-params? (assq-ref params 'nr)))
+   (if valid-params?
+   (respond-json
+(object->json-string
+ (handle-events-request 'evaluation params)))
+   (respond-json-with-error 500 "Parameter not defined!"
 
 (("jobset" name)
  (respond-html
-- 
2.23.0

Re: Performance improvements

[Efraim Flashner]

On Mon, Oct 28, 2019 at 12:06:25AM +0100, Ludovic Courtès wrote:
> Hi!
> 
> Some more improvements come from these commits:
> 
>   ac19950507 gnu: bootstrap: Cache the 'bootstrap-executable' origins.
>   5166d02730 gnu: bootstrap: 'bootstrap-origin' preserves eq?-ness when no 
> changes are made.
>   d727a9343d derivations: Don't memoize 'derivation->bytevector'.
>   f5fca9a82c gexp: Cache the module to derivation mappings.
>   c57e417eff store: Allow objects in the cache to be inserted and search for 
> with 'equal?'.
>   f58b45350b gexp: Add 'imported+compiled-modules'.
>   f726f6f802 derivations: 'build-expression->derivation' caches its module 
> derivations.
>   cf7648f882 derivations: Introduce 'imported+compiled-modules'.
> 
> This caches the module-to-derivation mapping, which wasn’t cached until
> now.
> 
> Overall the goal is to have the high-level caches work nicely (the
> “object cache” in particular), rendering the low-level caches (such as
> the ‘add-data-to-store’ cache) potentially useless.  This is beneficial
> because a cache hit on the object cache saves more CPU time than a cache
> hit on a low-level cache.
> 
> Ludovic Courtès  skribis:
> 
> > $ time guix build libreoffice -nd
> > /gnu/store/8drmbhsrayr2j5lkvrwq37rg8g06hgsw-libreoffice-6.1.5.2.drv
> >
> > real0m2.142s
> > user0m2.323s
> > sys 0m0.082s
> > $ guix describe
> > Generacio 110   Oct 14 2019 08:43:33(nuna)
> >   guix bd04fe8
> > repository URL: https://git.savannah.gnu.org/git/guix.git
> > branch: master
> > commit: bd04fe878627a14533d908ccdf5b906050d6e0a4
> 
> And now:
> 
> --8<---cut here---start->8---
> $ time /tmp/new/bin/guix build libreoffice -nd
> /gnu/store/ibzap0v1c367h0j2mdd2jmlsfx9jplfb-libreoffice-6.1.5.2.drv
> 
> real0m1.658s
> user0m1.699s
> sys 0m0.073s
> $ /tmp/new/bin/guix describe
> Generacio 1 Oct 28 2019 00:03:05(nuna)
>   guix 271b3c9
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 271b3c9d16f8880f4a27b52a615cdec611322fae
> --8<---cut here---end--->8---
> 

Results are even better on a slower machine:

$ time guix build libreoffice -nd
/gnu/store/ibzap0v1c367h0j2mdd2jmlsfx9jplfb-libreoffice-6.1.5.2.drv

real0m5.408s
user0m5.186s
sys 0m0.299s

$ time ./pre-inst-env guix build libreoffice -nd
/gnu/store/ibzap0v1c367h0j2mdd2jmlsfx9jplfb-libreoffice-6.1.5.2.drv

real0m3.836s
user0m3.519s
sys 0m0.235s

$ guix describe
Generation 12   Oct 27 2019 11:53:42(current)
  ietf 8c6079e
repository URL: https://gitlab.com/efraim/guix-ietf.git
branch: master
commit: 8c6079e10fdf640ce2cdafa3c1ea24961650f451
  guix 24adcba
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 24adcbab0ad903c59793dbe141950fff5058b8a2

$ ./pre-inst-env guix describe
Git checkout:
  repository: /home/efraim/workspace/guix
  branch: master
  commit: 64e3fe04b32e2eab1144d5b0c9278eab81056bd6



-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Performance improvements

[Pierre Neidhardt]

Only 700ms to go! :)
Congrats!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: We need your feedback of the documentation videos!

[pelzflorian (Florian Pelz)]

On Sun, Oct 27, 2019 at 02:47:00PM -0500, sirgazil wrote:
> Great, thank you, Florian.
> 
> I just sent a patch that adds the link to the videos in the Help page.
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37947
> 

It looks very good, except you use tabs instead of spaces, cf.
.  If this is
pushed by someone, please untabify.  I can push no sooner than this
evening because I will not have access to the beaglebone hosting my
GPG key.

Regards,
Florian