pygnulib: progress and some questions (func_emit_autoconf_snippets)
Hello, this is a kind reminder that python branch is still alive, though it's quite difficult to find a time for coding. :-) Anyway, I think I'll finish the "import" section in about a month: right now I'm slowly processing gnulib-comp.m4 generation part of the code. I have a question regarding a function named func_emit_autoconf_snippets. TL;DR: feel free to skip to ===QUESTIONS=== section if this is a way too long introduction to the topic. :-) The questions mainly target Bruno as the original author, but I'd be very grateful for any help. Could you please explain in details how func_emit_autoconf_snippets works? The most important for me is to understand how to map it into the API I've recently implemented: currently pygnulib can create a special object, "module database", which allows to iterate over modules and dependencies very quickly. Basically it is a hash table where each entry contains yet another hash table. Given a dependency DEP, it is really easy to iterate over each demander DEM and condition COND like this: for (dem, cond) in db.demanders(dep): print(dem, cond) One can also iterate over all dependencies of the demander DEM, acquiring each dependency DEP and the condition COND: for (dep, cond) in db.dependencies(dem): print(dep, cond) So given a set of modules MODULES, I can hopefully obtain all the required information to implement func_emit_autoconf_snippets analogue. However, I can't quite understand how the modules are processed. The questions are followed; they all are relevant only for -- conditional-dependencies switch since otherwise the algorithm is trivial (just treat each module as unconditional). ===QUESTIONS=== 1. At line 4260, what's the place where $modules variable gets assigned? The nearest explicit assignment I've found is at line 5298; is it correct that this is the one whose result is used at 4260? 2. What's the logic for dependencies processing, starting at line 4298? 3. Do you have an idea how could the algorithm mapped to the described API, given that I have module database, main modules, tests modules and final modules (and hopefully every other bit of information)? There's no need to check whether module exists, since all the mentioned collections (database and modules lists) contain not strings, but a real module instances. Thank you for your help! -- With best regards, Dmitry Selyutin signature.asc Description: This is a digitally signed message part.
Re: pygnulib: progress and some questions (func_emit_autoconf_snippets)
Hi Dmitry, > 1. At line 4260, what's the place where $modules variable gets assigned? Indeed the comments for function func_emit_autoconf_snippets are misleading: This functions receives two module lists: $modules (implicit), $1 (explicit: first argument). Where does $modules come from? - For the invocation in func_import, line 5640 and 5659 it is set at line 5298. But I'm not sure this was intended this way, maybe another list should be used instead of "$main_modules"? - For the invocation in func_create_testdir, line 6288 and 6290 and 6402 and 6404 and 6417 it does not appear to be set in a consistent and correct way. (It is set differently if $single_configure and ! $single_configure.) This is probably a bug. The set of modules that _ought_ to be used here is the set of modules being used for the current configure file. That's my current understanding. But I'd really need to look at a couple of test cases in order to firmly answer the question. > 2. What's the logic for dependencies processing, starting at line 4298? First the code intersects $deps with $modules. This matters for two cases: - when --avoid is used: we don't want to invoke the initialization code for a module that is not present because the user has required it to be avoided. - when ! $single_configure and we are generating the configure file for the tests directory: we may have a dependency from a module in $tests_modules to a module in $main_modules, but since the $main_modules are in a different directory they should be all treated like --avoid'ed modules here. Then the test if func_cond_module_p "$dep"; then is based on the consideration that the unconditional modules have their initialization code already emitted (unconditionally) by lines 4261..4271. Then comes the conditional invocation of the initialization code echo " if $condition; then" echo "$shellfunc" echo ' fi' with a trivial optimization for the (frequent) case that $condition = "true". > 3. Do you have an idea how could the algorithm mapped to the described API You need db.dependencies($module) here. db.demanders(...) is not used here. db.demanders(...) will be useful, in order to implement a new command-line option --extract-reverse-dependencies. Bruno
Re: pygnulib: progress and some questions (func_emit_autoconf_snippets)
Hi Dmitry, > > 1. At line 4260, what's the place where $modules variable gets assigned? > > Indeed the comments for function func_emit_autoconf_snippets are misleading: > This functions receives two module lists: > $modules (implicit), > $1 (explicit: first argument). > > Where does $modules come from? > - For the invocation in func_import, line 5640 and 5659 > it is set at line 5298. But I'm not sure this was intended this way, maybe > another list should be used instead of "$main_modules"? > - For the invocation in func_create_testdir, line 6288 and 6290 and 6402 > and 6404 and 6417 > it does not appear to be set in a consistent and correct way. > (It is set differently if $single_configure and ! $single_configure.) > This is probably a bug. > The set of modules that _ought_ to be used here is the set of modules > being used for the current configure file. That's my current understanding. > But I'd really need to look at a couple of test cases in order to > firmly answer the question. I've understood it now: the code in func_create_testdir was not really a bug, because --conditional-dependencies was not allowed in this case anyway. Here's the patch that implements --conditional-dependencies also with --with-tests. Bruno >From 589e96475f8f2d21a83405ab0672ce95091b80e5 Mon Sep 17 00:00:00 2001 From: Bruno Haible Date: Fri, 29 Dec 2017 00:29:23 +0100 Subject: [PATCH] gnulib-tool: Make --conditional-dependencies work better. Reported by Dmitry Selyutin . * gnulib-tool (Options): Don't reject the combination of --conditional-dependencies with --with-tests. (func_emit_autoconf_snippets): Add argument referenceable_modules. Don't reference $modules. (func_import, func_create_testdir): Pass it. --- ChangeLog | 10 ++ gnulib-tool | 37 - 2 files changed, 30 insertions(+), 17 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7d7b10a..1e2e0fb 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,13 @@ +2017-12-28 Bruno Haible + + gnulib-tool: Make --conditional-dependencies work better. + Reported by Dmitry Selyutin . + * gnulib-tool (Options): Don't reject the combination of + --conditional-dependencies with --with-tests. + (func_emit_autoconf_snippets): Add argument referenceable_modules. + Don't reference $modules. + (func_import, func_create_testdir): Pass it. + 2017-12-19 Paul Eggert regex: use re_malloc etc. consistently diff --git a/gnulib-tool b/gnulib-tool index 22817c9..5f640e3 100755 --- a/gnulib-tool +++ b/gnulib-tool @@ -1505,10 +1505,6 @@ func_determine_path_separator esac # Now the only possible values of "$inctests" are true and false # (or blank but then it is irrelevant). - if test "$cond_dependencies" = true && test "$inctests" = true; then -echo "gnulib-tool: option --conditional-dependencies is not supported with --with-tests" 1>&2 -func_exit 1 - fi # Determine the minimum supported autoconf version from the project's # configure.ac. @@ -4231,7 +4227,7 @@ func_emit_autoconf_snippet () fi } -# func_emit_autoconf_snippets modules verifier toplevel disable_libtool disable_gettext +# func_emit_autoconf_snippets modules referenceable_modules verifier toplevel disable_libtool disable_gettext # collects and emit the autoconf snippets of a set of modules. # Input: # - local_gnulib_path from --local-dir @@ -4240,6 +4236,7 @@ func_emit_autoconf_snippet () # - sed_replace_include_guard_prefix # sed expression for resolving ${gl_include_guard_prefix} # - modules the list of modules. +# - referenceable_modules the list of modules which may be referenced as dependencies. # - verifier one of func_verify_module, func_verify_nontests_module, # func_verify_tests_module. It selects the subset of # $modules to consider. @@ -4252,12 +4249,13 @@ func_emit_autoconf_snippet () # invocations. func_emit_autoconf_snippets () { - verifier="$2" - toplevel="$3" - disable_libtool="$4" - disable_gettext="$5" + referenceable_modules="$2" + verifier="$3" + toplevel="$4" + disable_libtool="$5" + disable_gettext="$6" if test "$cond_dependencies" = true; then -for m in $modules; do echo $m; done | LC_ALL=C sort -u > "$tmp"/modules +for m in $referenceable_modules; do echo $m; done | LC_ALL=C sort -u > "$tmp"/modules # Emit the autoconf code for the unconditional modules. for module in $1; do eval $verifier @@ -5637,7 +5635,7 @@ s,//*$,/,' if test -n "$witness_c_macro"; then echo " m4_pushdef([gl_MODULE_INDICATOR_CONDITION], [$witness_c_macro])" fi -func_emit_autoconf_snippets "$main_modules" func_verify_module true false true +func_emit_autoconf_snippets "$main_modules" "$main_modules" func_verify_module true false true if test -n "$witness_c_macro"; then echo " m4_popdef([gl_MODULE_INDICATOR_CONDITION])" fi @@ -5
Re: pygnulib: progress and some questions (func_emit_autoconf_snippets)
Hi Bruno, thank you for the detailed answer. I think I still have to do some debugging to understand why the Python version of the algorithm for autoconf snippets generation works incorrectly, but I hope I understood how it should work. 29 дек. 2017 г. 2:32 ДП пользователь "Bruno Haible" написал: > Hi Dmitry, > > > > 1. At line 4260, what's the place where $modules variable gets > assigned? > > > > Indeed the comments for function func_emit_autoconf_snippets are > misleading: > > This functions receives two module lists: > > $modules (implicit), > > $1 (explicit: first argument). > > > > Where does $modules come from? > > - For the invocation in func_import, line 5640 and 5659 > > it is set at line 5298. But I'm not sure this was intended this way, > maybe > > another list should be used instead of "$main_modules"? > > - For the invocation in func_create_testdir, line 6288 and 6290 and > 6402 and 6404 and 6417 > > it does not appear to be set in a consistent and correct way. > > (It is set differently if $single_configure and ! $single_configure.) > > This is probably a bug. > > The set of modules that _ought_ to be used here is the set of modules > > being used for the current configure file. That's my current > understanding. > > But I'd really need to look at a couple of test cases in order to > > firmly answer the question. > > I've understood it now: the code in func_create_testdir was not really a > bug, > because --conditional-dependencies was not allowed in this case anyway. > > Here's the patch that implements --conditional-dependencies also with > --with-tests. > > Bruno >