Re: [GNUnet-developers] GSoC: Binary package distribution through GnuNet (report 1)
Christian Grothoff groth...@gnunet.org skribis: On 06/03/2015 05:15 PM, Ludovic Courtès wrote: asg...@free.fr skribis: This first week started with cleaning and organizing the draft bindings I’ve been working on previously (set up of a proper directory structure, a small Makefile, etc.), then most of the work has been on mapping GnuNet’s data structures. This week, I’m heading for the remaining “most-needed” functions (URI handling, scheduling) and some testing, and also writing a simplified version of the `gnunet-search` utility, as it could be a good test and example of usage for the bindings. OK, sounds like a good plan! Are you already using a public repo that we could look at? If not, we should set up a Git repository at gnunet.org or Savannah, whichever is more convenient. gnunet.org isn't quite setup for *public* Git's yet... OK, I have requested the creation of a repo under the Guix project on Savannah: https://savannah.gnu.org/support/?108833 Rémi: I will add you on Savannah so you can push your code there when the repo is ready. Make sure to add a ‘COPYING’ for the GNU GPL v3, as well as copyright and license headers (see how this is done in Guix.) Feel free to contact me off-list for additional details. Options handling: the standard way of writing software that uses GnuNet’s API is to let GnuNet handle the command line options: the API’s entry point, must be fed directly with the C arguments vector `argv`; it assures that all GnuNet programs have a uniform CLI. The first difficulty I encountred is: how to properly handle these options? After some work, I decided to manually craft a fake `argv` as a quick temporary fix. OK. Question for the GNUnet people: What’s the recommended way to handle argc/argv if we want to provide a library? Well, libgnunetutil contains a wrapper around getopt for our own convenience, but I'm totally fine with using a language-specific argv-parser for command-line tools. Note that GNUnet *libraries* obviously also don't take command-line arguments, so I'm not sure what the question is. Typically a Guile module cannot be passed any arguments, and the initialization that happens is just that its top-level forms are evaluated when it’s loaded. Having to explicitly call an initialization function that takes an argv would look unidiomatic. Right, same applies for (most) GNUnet APIs, with the exception of our service and program abstractions which are there to make it easy to write a 'main' function. But I don't see a need to expose those two APIs to Guile. OK, so I guess there’s nothing to worry about in the end. :-) Thanks for your feedback! Ludo’.
Re: hackage importer
Howdy! Federico Beffa be...@ieee.org skribis: On Sat, May 2, 2015 at 2:48 PM, Ludovic Courtès l...@gnu.org wrote: [...] This procedure is intimidating. I think this is partly due to its length, to the big let-values, the long identifiers, the many local variables, nested binds, etc. Ok, this procedure has now ... disappeared ... or rather it is now hidden in a huge, but invisible macro ;-) I've added support for braces delimited blocks. In so doing the complexity of an ad-hoc solution increased further and decided that it was time to study (and use) a proper parser. Yay, good idea! But, a couple of words on your remarks: - Thanks to your comment about long list of local variables I (re-)discovered the (test = expr) form of cond clauses. Very useful! - The nested use of the = function didn't look nice and the reason is that it is really meant as a way to sequence monadic functions as in (= m f1 f2 ...). Unfortunately the current version of = in guile only accepts 2 arguments (1 function), hence the nesting. It would be nice to correct that :-) Sure, I have it in my to-do list. :-) I looked at it quickly and it seems less trivial than initially envisioned though. +(define-record-type cabal-package + (make-cabal-package name version license home-page source-repository + synopsis description + executables lib test-suites + flags eval-environment) + cabal-package? + (name cabal-package-name) + (version cabal-package-version) + (license cabal-package-license) + (home-page cabal-package-home-page) + (source-repository cabal-package-source-repository) + (synopsis cabal-package-synopsis) + (description cabal-package-description) + (executables cabal-package-executables) + (lib cabal-package-library) ; 'library' is a Scheme keyword There are no keyboards in Scheme. :-) ?? Oops, these should have read “keywords”, not “keyboards.” :-) The existing tests here are fine, but they are more like integration tests (they test the whole pipeline.) Maybe it would be nice to directly exercise ‘read-cabal’ and ‘eval-cabal’ individually? It is true that the tests are for the whole pipeline, but they catch most of the problems (problems in any function along the chain) with the smallest number of tests :-). I'm not very keen in doing fine grained testing. Sorry. I've removed the test with TABs because the Cabal documentation says explicitly that they are not allowed. https://www.haskell.org/cabal/users-guide/developing-packages.html#package-descriptions But are they actually used in practice? I've changed the second test to check the use of braces (multi-line values have still to be indented). OK. From f422ea9aff3aa8425c80eaadf50628c24d54495a Mon Sep 17 00:00:00 2001 From: Federico Beffa be...@fbengineering.ch Date: Sun, 26 Apr 2015 11:22:29 +0200 Subject: [PATCH] import: hackage: Refactor parsing code and add new options. * guix/import/cabal.scm: New file. * guix/import/hackage.scm: Update to use the new Cabal parsing module. * tests/hackage.scm: Update tests. * guix/scripts/import/hackage.scm: Add new '--cabal-environment' and '--stdin' options. * doc/guix.texi: ... and document them. * Makefile.am (MODULES): Add 'guix/import/cabal.scm', 'guix/import/hackage.scm' and 'guix/scripts/import/hackage.scm'. (SCM_TESTS): Add 'tests/hackage.scm'. [...] +(define (make-stack) + Creates a simple stack closure. Actions on the generated stack are +requested by calling it with one of the following symbols as the first +argument: 'empty?, 'push!, 'top, 'pop! and 'clear!. The action 'push! is the +only one requiring a second argument corresponding to the object to be added +to the stack. + (let ((stack '())) +(lambda (msg . args) + (cond ((eqv? msg 'empty?) (null? stack)) +((eqv? msg 'push!) (set! stack (cons (first args) stack))) +((eqv? msg 'top) (if (null? stack) '() (first stack))) +((eqv? msg 'pop!) (match stack +((e r ...) (set! stack (cdr stack)) e) +(_ #f))) +((eqv? msg 'clear!) (set! stack '())) +(else #f) Fair enough. :-) I wonder what happens exactly when trying to return monadic values in the parser. +;; Stack to track the structure of nested blocks +(define context-stack (make-stack)) What about making it either a SRFI-39 parameter, or a parameter to ‘make-cabal-parser’? I’ve only read through quickly, but the rest of the file looks lean and clean! +(define* (hackage-guix-package package-name #:key +(include-test-dependencies? #t) +(read-from-stdin? #f) +(cabal-environment '())) Instead of #:read-from-stdin?, what about adding a #:port argument? That way, it would either read from PORT, or fetch
Re: RFC: building numpy against OpenBLAS.
Ricardo Wurmus ricardo.wur...@mdc-berlin.de writes: Ricardo Wurmus writes: python-numpy currently depends on Atlas, which means that it cannot be substituted with a binary built elsewhere. OpenBLAS is an alternative to Atlas and the binary can be used on all supported CPUs at runtime. This makes it possible for us to make numpy substitutable. We currently do not have a working OpenBLAS on MIPS, so the attached patch selects OpenBLAS as an input only when not on MIPS. Some additional configuration happens only unless atlas is among the inputs. If there are no objections I'd like to go ahead and push this patch to build numpy with OpenBLAS. From the comments I gather that it's not as controversial a change as I suspected. Yes, please do! Thanks, Mark
Re: RFC: building numpy against OpenBLAS.
Ricardo Wurmus ricardo.wur...@mdc-berlin.de writes: Ricardo Wurmus writes: python-numpy currently depends on Atlas, which means that it cannot be substituted with a binary built elsewhere. OpenBLAS is an alternative to Atlas and the binary can be used on all supported CPUs at runtime. This makes it possible for us to make numpy substitutable. We currently do not have a working OpenBLAS on MIPS, so the attached patch selects OpenBLAS as an input only when not on MIPS. Some additional configuration happens only unless atlas is among the inputs. If there are no objections I'd like to go ahead and push this patch to build numpy with OpenBLAS. From the comments I gather that it's not as controversial a change as I suspected. I think we should go ahead and switch to OpenBLAS on _all_ platforms, not just on MIPS. Were there any objections to that? If not, I'd advocate just doing it. Thanks! Mark
Re: RFC: building numpy against OpenBLAS.
Ricardo Wurmus ricardo.wur...@mdc-berlin.de writes: We currently do not have a working OpenBLAS on MIPS, so the attached patch selects OpenBLAS as an input only when not on MIPS. Some additional configuration happens only unless atlas is among the inputs. If there are no objections I'd like to go ahead and push this patch to build numpy with OpenBLAS. From the comments I gather that it's not as controversial a change as I suspected. I think we should go ahead and switch to OpenBLAS on _all_ platforms, not just on MIPS. Were there any objections to that? If not, I'd advocate just doing it. There may have been a misunderstanding: We can switch to OpenBLAS for all platforms _except_ MIPS because OpenBLAS currently does not build on MIPS. I was indeed confused. However, it seems that ATLAS doesn't build on MIPS either, based on its 'supported-systems' field. So, I come again to the same conclusion: that we should switch to OpenBLAS on _all_ platforms, and strip away the ATLAS-handling code in our python-numpy package. We can work on fixing OpenBLAS to MIPS later. What do you think? Thanks! Mark
Re: hackage importer
On Fri, Jun 5, 2015 at 9:30 AM, Ludovic Courtès l...@gnu.org wrote:
I've removed the test with TABs because the Cabal documentation says
explicitly that they are not allowed.
https://www.haskell.org/cabal/users-guide/developing-packages.html#package-descriptions
But are they actually used in practice?
When I prepared the very first version of the importer I did find one
case among all the ones I tried. I believe that now that package has
been update to a new version and doesn't include TABs anymore.
[...]
+(define (make-stack)
+ Creates a simple stack closure. Actions on the generated stack are
+requested by calling it with one of the following symbols as the first
+argument: 'empty?, 'push!, 'top, 'pop! and 'clear!. The action 'push! is
the
+only one requiring a second argument corresponding to the object to be added
+to the stack.
+ (let ((stack '()))
+(lambda (msg . args)
+ (cond ((eqv? msg 'empty?) (null? stack))
+((eqv? msg 'push!) (set! stack (cons (first args) stack)))
+((eqv? msg 'top) (if (null? stack) '() (first stack)))
+((eqv? msg 'pop!) (match stack
+((e r ...) (set! stack (cdr stack)) e)
+(_ #f)))
+((eqv? msg 'clear!) (set! stack '()))
+(else #f)
Fair enough. :-) I wonder what happens exactly when trying to return
monadic values in the parser.
Given that the parser repeatedly calls the tunk generated by
'make-lexer' without passing any state or knowing anything about to
which monad it may belong to, I thought that it would not work. But,
as you see, I'm new to Scheme, new to monads, and new to Lisp in
general.
+;; Stack to track the structure of nested blocks
+(define context-stack (make-stack))
What about making it either a SRFI-39 parameter, or a parameter to
‘make-cabal-parser’?
I made it a parameter. Thanks for suggesting it! It made me realize
what they are really used for :-)
Do you think it is correct to say that they serve the purpose of
special variables in Lisp? (I'm looking a little bit into Common Lisp
as well.)
+(define* (hackage-guix-package package-name #:key
+(include-test-dependencies? #t)
+(read-from-stdin? #f)
+(cabal-environment '()))
Instead of #:read-from-stdin?, what about adding a #:port argument?
That way, it would either read from PORT, or fetch from Hackage. That
seems more idiomatic and more flexible.
Absolutely! Changed.
Also please mention it in the docstring.
Done.
-(test-assert hackage-guix-package test 3
- (eval-test-with-cabal test-cabal-3))
-
-(test-assert conditional-sexp-like
- (match
-(eval-cabal-keywords
- (conditional-sexp-like test-cond-1)
- '((debug . False)))
-(('and ('or ('string-match darwin ('%current-system)) ('not '#f)) '#t)
- #t)
-(x
- (pk 'fail x #f
I’m a bit scared when we add new code *and* remove tests. ;-)
Could you add a couple of representative tests? I’m sure you ran tests
by hand at the REPL, so it should be a matter of adding them in the file.
The reason for deleting the test is that that particular function
doesn't exist anymore. The functionality that it did provide is now
integrated in the parser. So, I've added one new test which exercises
'read-cabal' with a bunch of nested conditionals.
Thanks for the review!
Fede
From 8a28ed0f3c3077ce12d4924c59e317c52a68a77e Mon Sep 17 00:00:00 2001
From: Federico Beffa be...@fbengineering.ch
Date: Sun, 26 Apr 2015 11:22:29 +0200
Subject: [PATCH] import: hackage: Refactor parsing code and add new options.
* guix/import/cabal.scm: New file.
* guix/import/hackage.scm: Update to use the new Cabal parsing module.
* tests/hackage.scm: Update tests.
* guix/scripts/import/hackage.scm: Add new '--cabal-environment' and '--stdin'
options.
* doc/guix.texi: ... and document them.
* Makefile.am (MODULES): Add 'guix/import/cabal.scm',
'guix/import/hackage.scm' and 'guix/scripts/import/hackage.scm'.
(SCM_TESTS): Add 'tests/hackage.scm'.
---
Makefile.am | 4 +
doc/guix.texi | 22 +-
guix/import/cabal.scm | 815
guix/import/hackage.scm | 703 --
guix/scripts/import/hackage.scm | 66 +++-
tests/hackage.scm | 88 +++--
6 files changed, 1017 insertions(+), 681 deletions(-)
create mode 100644 guix/import/cabal.scm
diff --git a/Makefile.am b/Makefile.am
index d54e281..b42a7f5 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -89,6 +89,8 @@ MODULES = \
guix/import/utils.scm\
guix/import/gnu.scm\
guix/import/snix.scm\
+ guix/import/cabal.scm\
+ guix/import/hackage.scm \
guix/scripts/download.scm \
guix/scripts/build.scm \
guix/scripts/archive.scm \
@@ -104,6 +106,7 @@
Re: RFC: building numpy against OpenBLAS.
Mark H Weaver m...@netris.org writes: Ricardo Wurmus ricardo.wur...@mdc-berlin.de writes: Ricardo Wurmus writes: python-numpy currently depends on Atlas, which means that it cannot be substituted with a binary built elsewhere. OpenBLAS is an alternative to Atlas and the binary can be used on all supported CPUs at runtime. This makes it possible for us to make numpy substitutable. We currently do not have a working OpenBLAS on MIPS, so the attached patch selects OpenBLAS as an input only when not on MIPS. Some additional configuration happens only unless atlas is among the inputs. If there are no objections I'd like to go ahead and push this patch to build numpy with OpenBLAS. From the comments I gather that it's not as controversial a change as I suspected. I think we should go ahead and switch to OpenBLAS on _all_ platforms, not just on MIPS. Were there any objections to that? If not, I'd advocate just doing it. Libreoffice depends on python2-numpy, so if we don't switch to OpenBLAS on all platforms, then we cannot offer binary substitutes for Libreoffice. Also, the persistent build failures of python-numpy on Hydra are probably due to its use of ATLAS, when ALTAS is built on a different build slave than python2-numpy. So, at this point we have several strong incentives to phase out the use of ATLAS. Mark
Re: RFC: building numpy against OpenBLAS.
We currently do not have a working OpenBLAS on MIPS, so the attached patch selects OpenBLAS as an input only when not on MIPS. Some additional configuration happens only unless atlas is among the inputs. If there are no objections I'd like to go ahead and push this patch to build numpy with OpenBLAS. From the comments I gather that it's not as controversial a change as I suspected. I think we should go ahead and switch to OpenBLAS on _all_ platforms, not just on MIPS. Were there any objections to that? If not, I'd advocate just doing it. There may have been a misunderstanding: We can switch to OpenBLAS for all platforms _except_ MIPS because OpenBLAS currently does not build on MIPS. ~~ Ricardo
Re: Daemon update again
I did a make distclean, ./bootstrap, ./configure and make on my mips
machine. It fails with the following message:
CXX nix/libstore/libstore_a-build.o
nix/libstore/build.cc: In member function ‘void
nix::DerivationGoal::startBuilder()’:
nix/libstore/build.cc:1808:91: error: ‘DEFAULT_CHROOT_DIRS’ was not declared in
this scope
PathSet dirs =
tokenizeStringStringSet(settings.get(build-chroot-dirs,
string(DEFAULT_CHROOT_DIRS)));
^
nix/libstore/build.cc: In member function ‘void nix::Worker::waitForInput()’:
nix/libstore/build.cc:3324:74: warning: comparison between signed and unsigned
integer expressions [-Wsign-compare]
if (!waitingForAWhile.empty() lastWokenUp + settings.pollInterval =
after) {
^
Makefile:2630: recipe for target 'nix/libstore/libstore_a-build.o' failed
Andreas
