Re: [PATCH v2] gnu: Add dub-build-system.

[Ludovic Courtès]

Hello!

Danny Milosavljevic  skribis:

>> Do you have experience using it on real DUB packages?  IOW, how complete
>> is it?  :-)
>
> Yeah, I've tested it on a small subset of all code.dlang.org packages. About 
> half of the ones I tested work fine. This is my log:

Impressive!  I guess we can consider the importer mostly ready.  :-)

>>Rather: (chmod "." #o755).
>
> Thanks. Note that it's a workaround because git-download leaves the build 
> directory read-only for some reason. Should the problem be found and fixed 
> there? Or is it on purpose?

It’s not really on purpose, it’s just that the checkout in /gnu/store is
read-only and the ‘unpack’ phase simply copies it as-is.

>>Could you add a comment above explaining why this needs to be done?
>
> Sure. (it prepares a directory which can be used to find all the (D) 
> dependencies by going just one level down; earlier versions just used 
> /gnu/store directly but that would mean if you put such a package (without 
> any subdirs in it) into your profile it would pollute the root)
>
> I'm just thinking out loud about the comment :)
>
> Something like this?
>
> ;; Prepare one new directory with all the required dependencies.
> ;; It's necessary to do this (instead of just using /gnu/store as the 
> directory) because we want to hide the libraries in subdirectories 
> lib/dub/... instead of polluting the user's profile root.

Sounds good (with wrapped lines).

>>If HOME is relied on, please add a comment explaining why.
>
> The dub build system uses it to [...]

No need to comment here, I trust you to do the right thing ;-), but
please just provide an explanation as a comment so people later can know
why things are done this way.

>> +  (if (or (zero? (system* "grep" "-q" "sourceLibrary" "package.json"))
>> +  (zero? (system* "grep" "-q" "sourceLibrary" "dub.sdl")) ; note: 
>> format is different!
>> +  (zero? (system* "grep" "-q" "sourceLibrary" "dub.json")))
>
>>Would be best to avoid calling out to ‘grep’.  At worst you can do:
>>  (define (grep string file)
>>(string-contains (call-with-input-file file get-string-all)
>> string))
>
> Thanks. I'll use it. Will it break when it can't find the file? That
> would be bad. Usually there's only one of these files available.

Then you can do, say:

  (or (not (file-exists? file))
  (string-contains …))

With these issues addressed, I think you can push the updated patch.

Thank you!

Ludo’.

Re: [PATCH v2] gnu: Add dub-build-system.

[Danny Milosavljevic]

Hi Ludo,

> Do you have experience using it on real DUB packages?  IOW, how complete
> is it?  :-)

Yeah, I've tested it on a small subset of all code.dlang.org packages. About 
half of the ones I tested work fine. This is my log:

OK: (define-public d-taggedalgebraic
OK: (define-public d-memutils
NOT: (define-public d-libasync unittest failure
NOT: (define-public d-eventcore by libasync
OK: (define-public tsv-utils-dlang [bin directory?!]
OK: (define-public d-lapack
OK: (define-public d-cblas
OK: (define-public d-money
NOT: (define-public d-units-d unit tests fail
OK: (define-public d-libevent
NOT anymore: (define-public d-llvm-d
OK: (define-public d-htmld
OK: (define-public d-cssd
OK: (define-public d-openssl
NOT: (define-public d-mail can't find [d-]openssl
NOT: (define-public d-secured can't find [d-]openssl
NOT: (define-public d-botan-math [tests crash]
NOT: (define-public d-botan [dependency on d-botan-math - which will fail]
OK: (define-public d-urld
OK: (define-public isitthere
NOT: (define-public d-imaged ; dependency on undead which is probably bad.
OK: (define-public d-matplotlib-d [huge deps]
NOT: (define-public d-universal ; tests fail
NOT: (define-public d-future ; requires d-universal - which will fail
OK: (define-public d-docopt
NOT: (define-public d-requests ; depends on d-vibe-d - which will fail
OK: (define-public d-concepts
OK: (define-public d-gtk-d ; missing gdata
NOT: (define-public d-pyd ; Python 2 should be UCS-4 but isn't.
NOT: (define-public d-vibe-core ; depends on libasync - which will fail
NOT: (define-public d-vibe-d ; depends on d-libevent 2.0.1 - which it can't 
find for some reason
OK: (define-public d-derelict-util
NOT: (define-public d-derelict-sdl2 ; tries to write into d-derelict-util's 
immutable directory
NOT: (define-public d-derelict-ft ; tries to write into d-derelict-util's 
immutable directory
NOT: (define-public d-bdb2d ; unknown error 6995216
NOT: (define-public unde ; depends on d-bdb2d - which will fail
OK: (define-public d-wave-d
OK: (define-public d-assert-that
OK: (define-public d-compile-time-unittest
NOT: (define-public d-tcenal ; requires d-compile-time-unittest 0.0.3 but can't 
find it for some reason
NOT: (define-public d-libdparse ; requires newer ldc (requires ldc 1.1.0)
NOT: (define-public dfmt ; requires d-libdparse - which will fail.
???: (define-public d-antispam
???: (define-public d-userman
???: (define-public vibenews
OK: (define-public d-syscall-d
OK: (define-public d-quantities

And I've attached the actual package definitions (these are WIP).

>For the final patch, could you add a short description under “Build Systems” 
>in guix.texi?

Sure.

>You can remove keyword parameters that don’t make sense like this one.

Yeah, but maybe I should use them instead. Will investigate :)

>s/Rust/DUB/  :-)

Whoops, thanks :)

>Rather: (chmod "." #o755).

Thanks. Note that it's a workaround because git-download leaves the build 
directory read-only for some reason. Should the problem be found and fixed 
there? Or is it on purpose?

>“vendor”?

Yeah, a Rust term which I reused here. It means a directory with all the 
bundled dependencies (here we symlink the dependencies from our store 
dynamically).

>Could you add a comment above explaining why this needs to be done?

Sure. (it prepares a directory which can be used to find all the (D) 
dependencies by going just one level down; earlier versions just used 
/gnu/store directly but that would mean if you put such a package (without any 
subdirs in it) into your profile it would pollute the root)

I'm just thinking out loud about the comment :)

Something like this?

;; Prepare one new directory with all the required dependencies.
;; It's necessary to do this (instead of just using /gnu/store as the 
directory) because we want to hide the libraries in subdirectories lib/dub/... 
instead of polluting the user's profile root.

>> +  ;(setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc"))  
>Remove.

Ok.

>If HOME is relied on, please add a comment explaining why.

The dub build system uses it to find or create the .dub directory wherein it 
puts everything it built. I've found the place in DUB where one could patch 
that out: In source/dub/dub.d it says:

private void init()
{
import std.file : tempDir;
version(Windows) {
m_dirs.systemSettings = 
Path(environment.get("ProgramData")) ~ "dub/";
m_dirs.userSettings = Path(environment.get("APPDATA")) 
~ "dub/";
} else version(Posix){
m_dirs.systemSettings = Path("/var/lib/dub/");
m_dirs.userSettings = Path(environment.get("HOME")) ~ 
".dub/";
if (!m_dirs.userSettings.absolute)
m_dirs.userSettings = Path(getcwd()) ~ 
m_dirs.userSettings;
}

m_dirs.temp = Path(tempDir);

m_config = new 

[PATCH v2] gnu: Add dub-build-system.

[Danny Milosavljevic]

* guix/build-system/dub.scm: New file.
* guix/build/dub-build-system.scm: New file.
* Makefile.am (MODULES): Add them.
---
 Makefile.am |   2 +
 guix/build-system/dub.scm   | 141 
 guix/build/dub-build-system.scm | 106 ++
 3 files changed, 249 insertions(+)
 create mode 100644 guix/build-system/dub.scm
 create mode 100644 guix/build/dub-build-system.scm

diff --git a/Makefile.am b/Makefile.am
index 360c356f1..f6622e455 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -64,6 +64,7 @@ MODULES = \
   guix/build-system/ant.scm\
   guix/build-system/cargo.scm  \
   guix/build-system/cmake.scm  \
+  guix/build-system/dub.scm\
   guix/build-system/emacs.scm  \
   guix/build-system/asdf.scm   \
   guix/build-system/glib-or-gtk.scm\
@@ -88,6 +89,7 @@ MODULES = \
   guix/build/download.scm  \
   guix/build/cargo-build-system.scm\
   guix/build/cmake-build-system.scm\
+  guix/build/dub-build-system.scm  \
   guix/build/emacs-build-system.scm\
   guix/build/asdf-build-system.scm \
   guix/build/git.scm   \
diff --git a/guix/build-system/dub.scm b/guix/build-system/dub.scm
new file mode 100644
index 0..4b8d22ad9
--- /dev/null
+++ b/guix/build-system/dub.scm
@@ -0,0 +1,141 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2013, 2014, 2015, 2016 Ludovic Courtès 
+;;; Copyright © 2013 Andreas Enge 
+;;; Copyright © 2013 Nikita Karetnikov 
+;;; Copyright © 2016 David Craven 
+;;; Copyright © 2016 Danny Milosavljevic 
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see .
+
+(define-module (guix build-system dub)
+  #:use-module (guix search-paths)
+  #:use-module (guix store)
+  #:use-module (guix utils)
+  #:use-module (guix derivations)
+  #:use-module (guix packages)
+  #:use-module (guix build-system)
+  #:use-module (guix build-system gnu)
+  #:use-module (ice-9 match)
+  #:use-module (srfi srfi-26)
+  #:export (dub-build-system))
+
+(define (default-ldc)
+  "Return the default ldc package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((ldc (resolve-interface '(gnu packages ldc
+(module-ref ldc 'ldc)))
+
+(define (default-dub)
+  "Return the default dub package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((ldc (resolve-interface '(gnu packages ldc
+(module-ref ldc 'dub)))
+
+(define %dub-build-system-modules
+  ;; Build-side modules imported by default.
+  `((guix build dub-build-system)
+(guix build syscalls)
+,@%gnu-build-system-modules))
+
+(define* (dub-build store name inputs
+  #:key
+  (tests? #t)
+  (test-target #f)
+  (configure-flags #f) ; XXX unused
+  (dub-build-flags ''())
+  (phases '(@ (guix build dub-build-system)
+  %standard-phases))
+  (outputs '("out"))
+  (search-paths '())
+  (system (%current-system))
+  (guile #f)
+  (imported-modules %dub-build-system-modules)
+  (modules '((guix build dub-build-system)
+ (guix build utils
+  "Build SOURCE using DUB, and with INPUTS."
+  (define builder
+`(begin
+   (use-modules ,@modules)
+   (dub-build #:name ,name
+#:source ,(match (assoc-ref inputs "source")
+(((? derivation? source))
+ (derivation->output-path source))
+((source)
+ source)
+(source
+ source))
+#:system ,system
+#:test-target ,test-target
+#:dub-build-flags ,dub-build-flags
+