Re: Missing Guile Bindings and False Positive Unit tests

[Derek Atkins]

Hi,

Chad Albers  writes:

> Hi Derek,
>
> Thanks for the history about Gnucash bindings.  I might be asking for
> trouble.  I just would rather interact with GnuCash rather that
> Python.  Personal preference that is pretty arbitrary.  I'm curious
> how far I can take it.

Oh, I understand.  Personally I would have rather seen Ruby bindings
instead of Python, but I didn't write them so 

> After further inspection, I now understand what you meant my "swig
> exports the functions for guile".  I found the exactly location where
> I need to add new exports in the swig interface file.  I haven't
> managed, though, to try it out yet.

Understood.  Good Luck!

-derek

>
> On Tue, Feb 21, 2017 at 11:59 AM, Derek Atkins  wrote:
>> Hi Chad,
>>
>> Please remember to CC gnucash-user on all replies so that everyone can
>> remain "in the loop".
>>
>> Also, the gnc-modules should properly export swig-exported functions
>> without a re-export.  If it stopped doing that, that would be a bug
>> IMHO.
>>
>> Note that these unit tests are probably 18 years old.
>>
>> 
>> A long long time ago (well, around GnuCash 1.6 or so), /usr/bin/gnucash
>> was a guile script.  It loaded everything in as guile modules
>> (technically it loaded GncModules) and then jumped back and forth
>> between C and Scheme.  Still, not everything was available from Scheme,
>> but I don't believe anything has been *removed* from the guile
>> bindings.
>>
>> Anyways, this structure was very hard to debug.  You couldn't
>> run it under gdb; you needed to "attach" gdb to the running process,
>> which made it nearly impossible to debug issues during application
>> loading.  Eventually the Schemer developers left the project to pursue
>> other endeavors, and those that remained eventually switched back to the
>> "main() in C" structure that we have today.
>>
>> Of course a bunch of the functionality is still in GncModules.
>> 
>>
>> Having said all that, you shouldn't need to export or re-export a
>> wrapped API.  It should "just work".  And the unit tests should be
>> properly failing when there are missing or broken APIs.
>>
>> -derek
>>
>> Chad Albers  writes:
>>
>>> I'll file a bug report.
>>>
>>> But, for the record, swig isn't exporting.  That's not what swig does.
>>>   Function exports are done by using 2 APIs - the Guile define-modules
>>> API or Guiles' C API.
>>>
>>> For example,
>>>
>>> - Some guile functions are defined here:  src/engine/swig-engine.c.
>>> This is where qof-session-new is defined.
>>> - swig-engine.c uses Guile's C API to create a module called sw_engine
>>> - swig-engine.c exports the functions - including qof-session-new -
>>> using scm_c_export(...)
>>> - Several Guile scm files use this module: (use-modules (sw_engine)).
>>> None of these scm files either (export...) or (re-export...) functions
>>> from the sw_engine module.
>>>
>>> Therefore, none of the Guile  defined modules expose sw_engine functions.
>>>
>>> There may be, though, one exception, but even that isn't exposing
>>> anything from sw_engine.  The following is speculation...
>>>
>>> - GnuCash seems to include, what I suspect, is another module system.
>>> I think it's in the module (gnucash gnc-module).  Calls such as these
>>> -   (gnc:module-system-init) - may be a part that system.
>>> - Included in that system may be code from: src/engine/gncmod-engine.c
>>> - This C code has a C function  called 
>>> 'libgncmod_engine_gnc_module_init(...)
>>> - This includes the sw_engine module.  It calls
>>> scm_c_eval_string("(use-modules (sw_engine))");
>>> - This scm_c_eval_string does not include any (exports...) or
>>> (re-exports...) in the gncmode-engine.c file.
>>>
>>> Therefore, the sw_engine functions are not available in this other
>>> module system.
>>>
>>> To make the sw_engine functions available (which is what I would
>>> like),  this may have been how it was once accomplished. The units
>>> test that pass when they shouldn't include this line:
>>>
>>>  (gnc:module-load "gnucash/engine" 0)
>>>
>>> Maybe, at one time gnc:module-load exported functions from
>>> "gnucash/engine" modules.  Or something like that.  I'm still
>>> investigating the what/why/how of the other module system.
>>>
>>> In the meantime, one solution is just to re-export sw_engine functions
>>> from (gnucash engine) or engine.scm.  That's what I'm doing for my
>>> experiments, and it works.  Perhaps I will submit a patch for that,
>>> but I'm thinking that that's a hack. I'm opening up the discussion
>>> here for a better solution, if exposing more of the Guile API is a)
>>> fixing a bug and/or b) adding a new feature.  Since much of GnuCash
>>> was written in Guile, I thought I would use that instead of Python.
>>> As is, though, there doesn't seem to be a way to even source a GnuCash
>>> using its Guile bindings.  It can be done using Python.
>>>
>>> Chad
>>>
>>> --
>>>
>>> On Sun, Feb 19, 2017 at 5:34 PM, Derek 

Re: Missing Guile Bindings and False Positive Unit tests

[Chad Albers]

Hi Derek,

Thanks for the history about Gnucash bindings.  I might be asking for
trouble.  I just would rather interact with GnuCash rather that
Python.  Personal preference that is pretty arbitrary.  I'm curious
how far I can take it.

After further inspection, I now understand what you meant my "swig
exports the functions for guile".  I found the exactly location where
I need to add new exports in the swig interface file.  I haven't
managed, though, to try it out yet.
--


On Tue, Feb 21, 2017 at 11:59 AM, Derek Atkins  wrote:
> Hi Chad,
>
> Please remember to CC gnucash-user on all replies so that everyone can
> remain "in the loop".
>
> Also, the gnc-modules should properly export swig-exported functions
> without a re-export.  If it stopped doing that, that would be a bug
> IMHO.
>
> Note that these unit tests are probably 18 years old.
>
> 
> A long long time ago (well, around GnuCash 1.6 or so), /usr/bin/gnucash
> was a guile script.  It loaded everything in as guile modules
> (technically it loaded GncModules) and then jumped back and forth
> between C and Scheme.  Still, not everything was available from Scheme,
> but I don't believe anything has been *removed* from the guile
> bindings.
>
> Anyways, this structure was very hard to debug.  You couldn't
> run it under gdb; you needed to "attach" gdb to the running process,
> which made it nearly impossible to debug issues during application
> loading.  Eventually the Schemer developers left the project to pursue
> other endeavors, and those that remained eventually switched back to the
> "main() in C" structure that we have today.
>
> Of course a bunch of the functionality is still in GncModules.
> 
>
> Having said all that, you shouldn't need to export or re-export a
> wrapped API.  It should "just work".  And the unit tests should be
> properly failing when there are missing or broken APIs.
>
> -derek
>
> Chad Albers  writes:
>
>> I'll file a bug report.
>>
>> But, for the record, swig isn't exporting.  That's not what swig does.
>>   Function exports are done by using 2 APIs - the Guile define-modules
>> API or Guiles' C API.
>>
>> For example,
>>
>> - Some guile functions are defined here:  src/engine/swig-engine.c.
>> This is where qof-session-new is defined.
>> - swig-engine.c uses Guile's C API to create a module called sw_engine
>> - swig-engine.c exports the functions - including qof-session-new -
>> using scm_c_export(...)
>> - Several Guile scm files use this module: (use-modules (sw_engine)).
>> None of these scm files either (export...) or (re-export...) functions
>> from the sw_engine module.
>>
>> Therefore, none of the Guile  defined modules expose sw_engine functions.
>>
>> There may be, though, one exception, but even that isn't exposing
>> anything from sw_engine.  The following is speculation...
>>
>> - GnuCash seems to include, what I suspect, is another module system.
>> I think it's in the module (gnucash gnc-module).  Calls such as these
>> -   (gnc:module-system-init) - may be a part that system.
>> - Included in that system may be code from: src/engine/gncmod-engine.c
>> - This C code has a C function  called 'libgncmod_engine_gnc_module_init(...)
>> - This includes the sw_engine module.  It calls
>> scm_c_eval_string("(use-modules (sw_engine))");
>> - This scm_c_eval_string does not include any (exports...) or
>> (re-exports...) in the gncmode-engine.c file.
>>
>> Therefore, the sw_engine functions are not available in this other
>> module system.
>>
>> To make the sw_engine functions available (which is what I would
>> like),  this may have been how it was once accomplished. The units
>> test that pass when they shouldn't include this line:
>>
>>  (gnc:module-load "gnucash/engine" 0)
>>
>> Maybe, at one time gnc:module-load exported functions from
>> "gnucash/engine" modules.  Or something like that.  I'm still
>> investigating the what/why/how of the other module system.
>>
>> In the meantime, one solution is just to re-export sw_engine functions
>> from (gnucash engine) or engine.scm.  That's what I'm doing for my
>> experiments, and it works.  Perhaps I will submit a patch for that,
>> but I'm thinking that that's a hack. I'm opening up the discussion
>> here for a better solution, if exposing more of the Guile API is a)
>> fixing a bug and/or b) adding a new feature.  Since much of GnuCash
>> was written in Guile, I thought I would use that instead of Python.
>> As is, though, there doesn't seem to be a way to even source a GnuCash
>> using its Guile bindings.  It can be done using Python.
>>
>> Chad
>>
>> --
>>
>> On Sun, Feb 19, 2017 at 5:34 PM, Derek Atkins  wrote:
>>> That srfi didn't exist when the tests were written, which was probably
>>> around 18 years ago.  The fact the tests don't fail just means nobody tested
>>> the tests for false positives.  Please file a bug report?
>>>
>>> There isn't a need to be-export.  Swig exports do that automatically.

Re: Missing Guile Bindings and False Positive Unit tests

[Derek Atkins]

Ted Creedon  writes:

> FYI there is a swig problem in OpenSuse leap 4.2.
>
> Its not the latest version.

And this is why we package it up such that in general you don't need
swig to build GnuCash.  :)

> Tedc

-derek

-- 
   Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
   Member, MIT Student Information Processing Board  (SIPB)
   URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH
   warl...@mit.eduPGP key available
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Missing Guile Bindings and False Positive Unit tests

[Ted Creedon]

FYI there is a swig problem in OpenSuse leap 4.2.

Its not the latest version.

Tedc

From: gnucash-devel <gnucash-devel-bounces+tcreedon=easystreet@gnucash.org> 
on behalf of Derek Atkins <warl...@mit.edu>
Sent: Tuesday, February 21, 2017 8:59:27 AM
To: Chad Albers
Cc: gnucash-devel@gnucash.org
Subject: Re: Missing Guile Bindings and False Positive Unit tests

Hi Chad,

Please remember to CC gnucash-user on all replies so that everyone can
remain "in the loop".

Also, the gnc-modules should properly export swig-exported functions
without a re-export.  If it stopped doing that, that would be a bug
IMHO.

Note that these unit tests are probably 18 years old.


A long long time ago (well, around GnuCash 1.6 or so), /usr/bin/gnucash
was a guile script.  It loaded everything in as guile modules
(technically it loaded GncModules) and then jumped back and forth
between C and Scheme.  Still, not everything was available from Scheme,
but I don't believe anything has been *removed* from the guile
bindings.

Anyways, this structure was very hard to debug.  You couldn't
run it under gdb; you needed to "attach" gdb to the running process,
which made it nearly impossible to debug issues during application
loading.  Eventually the Schemer developers left the project to pursue
other endeavors, and those that remained eventually switched back to the
"main() in C" structure that we have today.

Of course a bunch of the functionality is still in GncModules.


Having said all that, you shouldn't need to export or re-export a
wrapped API.  It should "just work".  And the unit tests should be
properly failing when there are missing or broken APIs.

-derek

Chad Albers <calb...@neomantic.com> writes:

> I'll file a bug report.
>
> But, for the record, swig isn't exporting.  That's not what swig does.
>   Function exports are done by using 2 APIs - the Guile define-modules
> API or Guiles' C API.
>
> For example,
>
> - Some guile functions are defined here:  src/engine/swig-engine.c.
> This is where qof-session-new is defined.
> - swig-engine.c uses Guile's C API to create a module called sw_engine
> - swig-engine.c exports the functions - including qof-session-new -
> using scm_c_export(...)
> - Several Guile scm files use this module: (use-modules (sw_engine)).
> None of these scm files either (export...) or (re-export...) functions
> from the sw_engine module.
>
> Therefore, none of the Guile  defined modules expose sw_engine functions.
>
> There may be, though, one exception, but even that isn't exposing
> anything from sw_engine.  The following is speculation...
>
> - GnuCash seems to include, what I suspect, is another module system.
> I think it's in the module (gnucash gnc-module).  Calls such as these
> -   (gnc:module-system-init) - may be a part that system.
> - Included in that system may be code from: src/engine/gncmod-engine.c
> - This C code has a C function  called 'libgncmod_engine_gnc_module_init(...)
> - This includes the sw_engine module.  It calls
> scm_c_eval_string("(use-modules (sw_engine))");
> - This scm_c_eval_string does not include any (exports...) or
> (re-exports...) in the gncmode-engine.c file.
>
> Therefore, the sw_engine functions are not available in this other
> module system.
>
> To make the sw_engine functions available (which is what I would
> like),  this may have been how it was once accomplished. The units
> test that pass when they shouldn't include this line:
>
>  (gnc:module-load "gnucash/engine" 0)
>
> Maybe, at one time gnc:module-load exported functions from
> "gnucash/engine" modules.  Or something like that.  I'm still
> investigating the what/why/how of the other module system.
>
> In the meantime, one solution is just to re-export sw_engine functions
> from (gnucash engine) or engine.scm.  That's what I'm doing for my
> experiments, and it works.  Perhaps I will submit a patch for that,
> but I'm thinking that that's a hack. I'm opening up the discussion
> here for a better solution, if exposing more of the Guile API is a)
> fixing a bug and/or b) adding a new feature.  Since much of GnuCash
> was written in Guile, I thought I would use that instead of Python.
> As is, though, there doesn't seem to be a way to even source a GnuCash
> using its Guile bindings.  It can be done using Python.
>
> Chad
>
> --
>
> On Sun, Feb 19, 2017 at 5:34 PM, Derek Atkins <de...@ihtfp.com> wrote:
>> That srfi didn't exist when the tests were written, which was probably
>> around 18 years ago.  The fact the tests don't fail just means nobody tested
>> the tests for false positives.  Please file a bug report?
>>
>> There isn't a need to be-export.  Swig 

Re: Missing Guile Bindings and False Positive Unit tests

[Derek Atkins]

Hi Chad,

Please remember to CC gnucash-user on all replies so that everyone can
remain "in the loop".

Also, the gnc-modules should properly export swig-exported functions
without a re-export.  If it stopped doing that, that would be a bug
IMHO.

Note that these unit tests are probably 18 years old.


A long long time ago (well, around GnuCash 1.6 or so), /usr/bin/gnucash
was a guile script.  It loaded everything in as guile modules
(technically it loaded GncModules) and then jumped back and forth
between C and Scheme.  Still, not everything was available from Scheme,
but I don't believe anything has been *removed* from the guile
bindings.

Anyways, this structure was very hard to debug.  You couldn't
run it under gdb; you needed to "attach" gdb to the running process,
which made it nearly impossible to debug issues during application
loading.  Eventually the Schemer developers left the project to pursue
other endeavors, and those that remained eventually switched back to the
"main() in C" structure that we have today.

Of course a bunch of the functionality is still in GncModules.


Having said all that, you shouldn't need to export or re-export a
wrapped API.  It should "just work".  And the unit tests should be
properly failing when there are missing or broken APIs.

-derek

Chad Albers  writes:

> I'll file a bug report.
>
> But, for the record, swig isn't exporting.  That's not what swig does.
>   Function exports are done by using 2 APIs - the Guile define-modules
> API or Guiles' C API.
>
> For example,
>
> - Some guile functions are defined here:  src/engine/swig-engine.c.
> This is where qof-session-new is defined.
> - swig-engine.c uses Guile's C API to create a module called sw_engine
> - swig-engine.c exports the functions - including qof-session-new -
> using scm_c_export(...)
> - Several Guile scm files use this module: (use-modules (sw_engine)).
> None of these scm files either (export...) or (re-export...) functions
> from the sw_engine module.
>
> Therefore, none of the Guile  defined modules expose sw_engine functions.
>
> There may be, though, one exception, but even that isn't exposing
> anything from sw_engine.  The following is speculation...
>
> - GnuCash seems to include, what I suspect, is another module system.
> I think it's in the module (gnucash gnc-module).  Calls such as these
> -   (gnc:module-system-init) - may be a part that system.
> - Included in that system may be code from: src/engine/gncmod-engine.c
> - This C code has a C function  called 'libgncmod_engine_gnc_module_init(...)
> - This includes the sw_engine module.  It calls
> scm_c_eval_string("(use-modules (sw_engine))");
> - This scm_c_eval_string does not include any (exports...) or
> (re-exports...) in the gncmode-engine.c file.
>
> Therefore, the sw_engine functions are not available in this other
> module system.
>
> To make the sw_engine functions available (which is what I would
> like),  this may have been how it was once accomplished. The units
> test that pass when they shouldn't include this line:
>
>  (gnc:module-load "gnucash/engine" 0)
>
> Maybe, at one time gnc:module-load exported functions from
> "gnucash/engine" modules.  Or something like that.  I'm still
> investigating the what/why/how of the other module system.
>
> In the meantime, one solution is just to re-export sw_engine functions
> from (gnucash engine) or engine.scm.  That's what I'm doing for my
> experiments, and it works.  Perhaps I will submit a patch for that,
> but I'm thinking that that's a hack. I'm opening up the discussion
> here for a better solution, if exposing more of the Guile API is a)
> fixing a bug and/or b) adding a new feature.  Since much of GnuCash
> was written in Guile, I thought I would use that instead of Python.
> As is, though, there doesn't seem to be a way to even source a GnuCash
> using its Guile bindings.  It can be done using Python.
>
> Chad
>
> --
>
> On Sun, Feb 19, 2017 at 5:34 PM, Derek Atkins  wrote:
>> That srfi didn't exist when the tests were written, which was probably
>> around 18 years ago.  The fact the tests don't fail just means nobody tested
>> the tests for false positives.  Please file a bug report?
>>
>> There isn't a need to be-export.  Swig exports do that automatically.
>>
>> -derek
>>
>> Sent from my mobile device. Please excuse any typos.
>>
>> - Reply message -
>> From: "Chad Albers" 
>> To: "Derek Atkins" 
>> Subject: Missing Guile Bindings and False Positive Unit tests
>> Date: Sun, Feb 19, 2017 5:23 PM
>>
>> Definitely, though, the unit tests it is checking is
>> 'qof-session-new'.  It passes, but it hasn't been exported in the
>> Guile code.
>>
>> At one time, I think it was exported from this file:
>> 'src/mine/gnucash/src/engine/engine.scm'  This file includes the
>> (sw_engine) module (defined in Guile C API), but it does not "(export
>> qof-session-new)" or "(re-export 

Re: Missing Guile Bindings and False Positive Unit tests

[Derek Atkins]

Hi,
Except for a few functions in app-utils, most of QofSession is not wrapped in 
the scheme bindings like they are in the python bindings.

-derek

Sent from my mobile device. Please excuse any typos.

- Reply message -
From: "Chad Albers" 
To: 
Subject: Missing Guile Bindings and False Positive Unit tests
Date: Sun, Feb 19, 2017 4:46 PM

Hi,

I found some problems with some unit tests for the Guile bindings.
Here's the background.

Rather than use the Python bindings, I wanted to use guile to add/manipulate
my GnuCash data.  From the Python bindings, I deduced that the entry
point into creating
transactions is via a construct called a "session".  Accordingly, I went to find
the Guile equivalent: "qof-session-new".  I wrote a Guile script which
successfully
imports GNUCash's guile bindings.  However, I was never able to successfully use
"qof-session-*" functions.  They are considered "unbound".

After further investigation, I found that GNUCash has unit tests that
specifically exercise the
'qof-session-new' function.
Here...'src/engine/test/test-create-account.scm'.  I ran
these units tests with 'make check' and they passed. I concluded that
since the unit test
passes, it must somehow have the binding for 'qof-session-new'.

'make check', though, creates a log for each unit test.  I peeked in
'test-create-account.scm.log'.
I found the following line:

;;; src/engine/test/./test-create-account.scm:30:18: warning: possibly
unbound variable `qof-session-new'

This is the same error that I have encountered in my own Guile script.
But...I also see this line
at the end of the test log.

PASS test-create-account (exit status: 0)

So the unit test cannot find 'qof-session-new' and yet the unit test
framework thinks its
passing. I wanted to let someone know that GNUCash has unit tests
(perhaps only for guile)
that are returning false positives. That's bug one.  Would you like me
to file a bug for this?

It turns out that GnuCash is not properly exposing the Guile bindings.
I consider this
a "bug", but others may disagree.  GnuCash seems to work pretty well
without them.  For my
purposes, though, I know where they are defined, why they aren't
exposed, and how to expose them.
If someone wants a patch for that, I may be able to provide it.

In the meantime, the heads up here is that there are some testing
returning false positives.

Let me know if you want any more information.
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel