Re: [GNC-dev] Import Main Matcher

[David T. via gnucash-devel]

To echo one part of John's comment, I recommend that any description of this 
process belongs in the Guide, and not the Help or on the wiki. Since chapter 3 
of the Guide is called "Importing into Gnucash," I would probably suggest to 
put it there.
David T. 

 
 
  On Wed, Aug 14, 2019 at 8:56, John Ralls wrote:   

> On Aug 13, 2019, at 6:25 PM, David Cousens  wrote:
> 
> At present the transaction importing documentation in the Help manual has no
> full description of how the import main matcher goes about the process of
> searching for and matching an imported transaction to an existing
> transaction or how it assigns a transfer account using the original mapping
> process or the more recent Bayesian approach. 
> 
> I am in the process of going through the code at present for my own
> edification and could create an expanded description of these processes in
> their current state either for inclusion in the Help manual or  as a Wiki
> page. My concern is that a detailed explanantion in the importing
> transaction section of the help manual may make that section too long
> winded. If that is not a significant issue, that would be my personal
> preference.
> 
> Does anyone have any strong preferences as to which is the better option ?

I'd rather not document internals in the T Implementation details aren't 
something that users need to worry about, so document there what it does, but 
not how. 

How belongs in comments in the code. Exported functions should have 
Doxygen-formatted comments so that they add to the API documentation 
(https://code.gnucash.org/docs/MAINT), local functions may also have 
non-Doxygen comments. Beware that a lot of our code follows the Gnome dispatch 
model where a static function is exported via a hand-written vtable, usually 
placed near the bottom of the source file. Those functions are really public 
and rate Doxygen documentation.

Bigger-picture documentation of a module's design also belongs in 
Doxygen-formatted comments. If you're not familiar with Doxygen see 
http://www.doxygen.nl/.

Regards,
John Ralls

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
  
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Import Main Matcher

[John Ralls]

> On Aug 13, 2019, at 6:25 PM, David Cousens  wrote:
> 
> At present the transaction importing documentation in the Help manual has no
> full description of how the import main matcher goes about the process of
> searching for and matching an imported transaction to an existing
> transaction or how it assigns a transfer account using the original mapping
> process or the more recent Bayesian approach. 
> 
> I am in the process of going through the code at present for my own
> edification and could create an expanded description of these processes in
> their current state either for inclusion in the Help manual or  as a Wiki
> page. My concern is that a detailed explanantion in the importing
> transaction section of the help manual may make that section too long
> winded. If that is not a significant issue, that would be my personal
> preference.
> 
> Does anyone have any strong preferences as to which is the better option ?

I'd rather not document internals in the T Implementation details aren't 
something that users need to worry about, so document there what it does, but 
not how. 

How belongs in comments in the code. Exported functions should have 
Doxygen-formatted comments so that they add to the API documentation 
(https://code.gnucash.org/docs/MAINT), local functions may also have 
non-Doxygen comments. Beware that a lot of our code follows the Gnome dispatch 
model where a static function is exported via a hand-written vtable, usually 
placed near the bottom of the source file. Those functions are really public 
and rate Doxygen documentation.

Bigger-picture documentation of a module's design also belongs in 
Doxygen-formatted comments. If you're not familiar with Doxygen see 
http://www.doxygen.nl/.

Regards,
John Ralls

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Rework GoogleTest integration

[John Ralls]

> On Aug 13, 2019, at 2:54 PM, Christian Gruber  
> wrote:
> 
> Am 13.08.19 um 02:45 schrieb John Ralls:
>> 
>>> On Aug 12, 2019, at 3:12 PM, Christian Gruber  
>>> wrote:
>>> 
>>> Following my previous thread "[GNC-dev] Contribute to GnuCash development" 
>>> I opened a new topic thread about reworking GoogleTest integration.
>>> 
>>> 
>>> At first some investigation results on bug 797344 
>>> .
>>> 
>>> In function gnc_gtest_configure() in file 
>>> common/cmake_modules/GncAddTest.cmake the two CMake variables 
>>> GTEST_INCLUDE_DIR and GMOCK_INCLUDE_DIR are set as follows:
>>> 
>>> find_path(GTEST_INCLUDE_DIR gtest/gtest.h HINTS ${GTEST_ROOT}/include 
>>> ${GMOCK_ROOT}/gtest/include /usr/include)
>>> find_path(GMOCK_INCLUDE_DIR gmock/gmock.h HINTS ${GMOCK_ROOT}/include 
>>> /usr/include)
>>> 
>>> This means, as long as GTEST_ROOT and GMOCK_ROOT are defined and refer to a 
>>> valid GoogleTest repository, header files gtest.h and gmock.h are found 
>>> there and the two variables GTEST_INCLUDE_DIR and GMOCK_INCLUDE_DIR will 
>>> refer to the include directories within this GoogleTest repository.
>>> 
>>> In contrast to that the four CMake variables GTEST_MAIN_LIB, 
>>> GTEST_SHARED_LIB, GMOCK_MAIN_LIB and GMOCK_SHARED_LIB are set in the same 
>>> function gnc_gtest_configure() as follows:
>>> 
>>> find_library(GTEST_MAIN_LIB gtest_main)
>>> find_library(GTEST_SHARED_LIB gtest)
>>> find_library(GMOCK_MAIN_LIB gmock_main)
>>> find_library(GMOCK_SHARED_LIB gmock)
>>> 
>>> This means, libraries are always searched in the default CMake search 
>>> paths. Therefore any preinstalled GoogleTest libraries will be found this 
>>> way.
>>> 
>>> This explains the behaviour described in my bug report.
>>> 
>>> 
>>> Now let's come to my ideas regarding rework of GoogleTest integration. 
>>> After further studying CMake files I have two general questions at first.
>>> 
>>> 1. Why are library targets "gtest" and "gmock" defined in GnuCash build 
>>> system instead of importing them from GoogleTest build results?
>>> 
>>> In file common/test-core/CMakeLists.txt these two targets are defined via 
>>> add_library(). The same is done within GoogleTest build system in files 
>>> googletest/CMakeLists.txt and googlemock/CMakeLists.txt. So part of the 
>>> CMake build system from GoogleTest repository is copied into GnuCash build 
>>> system.
>>> 
>>> My idea is to build and install GoogleTest completely independent from 
>>> GnuCash using its own build system and then importing targets via 
>>> find_package() into GnuCash build system. GoogleTest build system provides 
>>> CMake configuration files after installation ready for importing targets 
>>> into other projects.
>>> 
>>> CMake function find_package() is able to find package GTest via CMake 
>>> configuration files, i.e. prebuilt from source code repository, as well as 
>>> preinstalled libraries from distro using internal CMake module FindGTest 
>>> . Beginning from 
>>> CMake version 3.5 module FindGTest provides imported targets GTest::GTest 
>>> and GTest::Main.
>>> 
>>> This would avoid mixing of libraries and header files from different 
>>> locations.
>>> 
>>> Additionally the user doesn't have to define extra environment variables 
>>> GTEST_ROOT and GMOCK_ROOT anymore. Instead the user can configure CMake 
>>> search path using CMake variable CMAKE_PREFIX_PATH, if desired.
>>> 
>>> And the user can choose, which GoogleTest installation should be used by 
>>> influencing the CMake search path. So the requirement, that one can decide 
>>> whether to use preinstalled GoogleTest libraries from distro or GoogleTest 
>>> installation prebuilt from sources would be fulfilled.
>>> 
>>> 
>>> 2. Why is library target "gtest" not used directly instead of variables 
>>> GTEST_LIB, GTEST_INCLUDE_DIR?
>>> 
>>> Several test applications are defined via function gnc_add_test() by 
>>> passing a list of sources, include directories and libraries to that 
>>> function. This function gnc_add_test() is defined in file 
>>> common/cmake_modules/GncAddTest.cmake and passes the list of libraries 
>>> (TEST_LIBS_VAR_NAME) after resolving the variable names to CMake function 
>>> target_link_libraries().
>>> 
>>> My idea is to add library target "gtest" directly to that list of libraries 
>>> instead of variable GTEST_LIB, which is possible since CMake function 
>>> target_link_libraries() can handle libraries as well as CMake targets. The 
>>> advantage is, that you don't have to handle include directories separately 
>>> on your own. When passing CMake targets to function target_link_libraries() 
>>> CMake does all the stuff for you. Include directories defined for that 
>>> target are added automatically, i.e. target_include_directories() doesn't 
>>> have to be invoked to add GTEST_INCLUDE_DIR.
>>> 
>>> A further advantage is, that all test application targets depend on 

Re: [GNC-dev] Import Main Matcher

[Frank H. Ellenberger]

Hi David,

Am Mi., 14. Aug. 2019 um 03:26 Uhr schrieb David Cousens
:
>
> At present the transaction importing documentation in the Help manual has no
> full description of how the import main matcher goes about the process of
> searching for and matching an imported transaction to an existing
> transaction or how it assigns a transfer account using the original mapping
> process or the more recent Bayesian approach.
>
> I am in the process of going through the code at present for my own
> edification and could create an expanded description of these processes in
> their current state either for inclusion in the Help manual or  as a Wiki
> page. My concern is that a detailed explanantion in the importing
> transaction section of the help manual may make that section too long
> winded. If that is not a significant issue, that would be my personal
> preference.
>
> Does anyone have any strong preferences as to which is the better option ?

Your Docbook knowledge is sufficient to write directly for the Manual.
And you are smart enought to see that "The best way to migrate data
from XYZ to Gnucash" would belong into the Guide.

I fear if you put it at first in the wiki, it will stay there for years.

~Frank

> David Cousens
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] Import Main Matcher

[David Cousens]

At present the transaction importing documentation in the Help manual has no
full description of how the import main matcher goes about the process of
searching for and matching an imported transaction to an existing
transaction or how it assigns a transfer account using the original mapping
process or the more recent Bayesian approach. 

I am in the process of going through the code at present for my own
edification and could create an expanded description of these processes in
their current state either for inclusion in the Help manual or  as a Wiki
page. My concern is that a detailed explanantion in the importing
transaction section of the help manual may make that section too long
winded. If that is not a significant issue, that would be my personal
preference.

Does anyone have any strong preferences as to which is the better option ?

David Cousens



-
David Cousens
--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-Dev-f1435356.html
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] [MAINT] Cleaning up Code (the server)

[John Ralls]

> On Aug 13, 2019, at 9:14 AM, Derek Atkins  wrote:
> 
> Hi,
> 
> On Tue, August 13, 2019 12:03 pm, John Ralls wrote:
>> 
>> Yay! I've been asking for you to clean up the old Windows builds for
>> years! Now for a little preventive medicine: take a few minutes to add a
>> cron job.
> 
> eTooLazy.  ;)
> 
> Writing and testing the script would take more time than just going and
> pruning every year or two.

Nah, it's easy: find /home/builder/builds/win32/maint -ctime +180 -delete`. 

> 
>> The logs directory could also stand some vacuuming, we really don't need
>> to keep the logs around for more than a few months and they currently go
>> back to the beginning of 2018.
> 
> The logs don't take up much space, but I pruned out the 2018 logs.
> 
>> If you want you could be more aggressive. We don't need to keep release
>> builds at all, they're already all on SourceForge and the newer ones are
>> on Github as well. If you want to keep a separate backup you could keep
>> them somewhere offline. Maint and master builds are of marginal utility
>> after the release they lead up to and it gets more marginal with every
>> subsequent release. 6 months worth (two regular releases) seems about
>> right.
>> 
>> Flatpak builds should follow the same logic as Windows builds.
>> Incidentally I like the way that you've separated the flatpack build logs.
> 
> This wasn't me; this was how the build scripts were written.

Ah, duh. So I can fix that in the Windows build script.
> 
>> There don't seem to be any flatpak release builds. Should there be and if
>> so should I include them in the release downloads on SourceForge and
>> Github?
> 
> Not yet.  Remember that right now code is sitting behind a tiny straw. 
> Once I can move back to my main network then we can advertise that.

This wouldn't be advertising, it would be me downloading one flatpack at 
release and uploading it to SF and Github just as I do with Windows releases.

Regards,
John Ralls

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] [MAINT] Cleaning up Code (the server)

[Derek Atkins]

Hi,

On Tue, August 13, 2019 3:01 pm, Frank H. Ellenberger wrote:
> Hi,
>
> Am Di., 13. Aug. 2019 um 18:15 Uhr schrieb Derek Atkins :
> :
>> > Flatpak builds should follow the same logic as Windows builds.
>> > Incidentally I like the way that you've separated the flatpack build
>> logs.
>>
>> This wasn't me; this was how the build scripts were written.
>>
>> > There don't seem to be any flatpak release builds. Should there be and
>> if
>> > so should I include them in the release downloads on SourceForge and
>> > Github?
>>
>> Not yet.  Remember that right now code is sitting behind a tiny straw.
>> Once I can move back to my main network then we can advertise that.
>
> The other way: At least some should go to a broader download server.

Umm.. I don't think there is a way to "push" a build to flathub.

> This will be important next month for german linux users to get
> Onlinebanking back, if their distros do not push updates.

I dont think I'll be online then.  I suspect it'll be at least 2 more
months before I can migrate.  MAYBE I'll get lucky and be able to move
over earlier.  But very unlikely I'll be online next month.

>> I don't know why it's not building the releases.  Maybe Geert knows?
>>
>> > We only used trac for online code browsing, right? I don't see any
>> reason
>> > to keep backups of that, all of the actual data are in the
>> repositories.
>>
>> I don't think we even use it for that anymore.  I don't even have trac
>> installed.  So it's probably safe to delete.  (I'll wait for
>> confirmation
>> before I do).
>
> Go!

Done.  Usage went down from 92671240 (46%) to 69117424 (34%)

>> > So 30G for Windows builds, 24G for Flatpack builds, and 20G for trac
>> > backups = 74G. What's the remaining 15G? The repos?
>>
>> Repos.
>> Wiki data (and backups)
>> Email list data & archives
>> IRC Logs (or possibly something else.. supybot's dir is 4.1G
>> Then little bits here and there.
>
> Good to know.
>
> ~Frank

-derek

-- 
   Derek Atkins 617-623-3745
   de...@ihtfp.com www.ihtfp.com
   Computer and Internet Security Consultant

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] [MAINT] Cleaning up Code (the server)

[Frank H. Ellenberger]

Hi,

Am Di., 13. Aug. 2019 um 18:15 Uhr schrieb Derek Atkins :
:
> > Flatpak builds should follow the same logic as Windows builds.
> > Incidentally I like the way that you've separated the flatpack build logs.
>
> This wasn't me; this was how the build scripts were written.
>
> > There don't seem to be any flatpak release builds. Should there be and if
> > so should I include them in the release downloads on SourceForge and
> > Github?
>
> Not yet.  Remember that right now code is sitting behind a tiny straw.
> Once I can move back to my main network then we can advertise that.

The other way: At least some should go to a broader download server.

This will be important next month for german linux users to get
Onlinebanking back, if their distros do not push updates.

> I don't know why it's not building the releases.  Maybe Geert knows?
>
> > We only used trac for online code browsing, right? I don't see any reason
> > to keep backups of that, all of the actual data are in the repositories.
>
> I don't think we even use it for that anymore.  I don't even have trac
> installed.  So it's probably safe to delete.  (I'll wait for confirmation
> before I do).

Go!

> > So 30G for Windows builds, 24G for Flatpack builds, and 20G for trac
> > backups = 74G. What's the remaining 15G? The repos?
>
> Repos.
> Wiki data (and backups)
> Email list data & archives
> IRC Logs (or possibly something else.. supybot's dir is 4.1G
> Then little bits here and there.

Good to know.

~Frank
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] [MAINT] Cleaning up Code (the server)

[Derek Atkins]

Hi,

On Tue, August 13, 2019 12:03 pm, John Ralls wrote:
>
> Yay! I've been asking for you to clean up the old Windows builds for
> years! Now for a little preventive medicine: take a few minutes to add a
> cron job.

eTooLazy.  ;)

Writing and testing the script would take more time than just going and
pruning every year or two.

> The logs directory could also stand some vacuuming, we really don't need
> to keep the logs around for more than a few months and they currently go
> back to the beginning of 2018.

The logs don't take up much space, but I pruned out the 2018 logs.

> If you want you could be more aggressive. We don't need to keep release
> builds at all, they're already all on SourceForge and the newer ones are
> on Github as well. If you want to keep a separate backup you could keep
> them somewhere offline. Maint and master builds are of marginal utility
> after the release they lead up to and it gets more marginal with every
> subsequent release. 6 months worth (two regular releases) seems about
> right.
>
> Flatpak builds should follow the same logic as Windows builds.
> Incidentally I like the way that you've separated the flatpack build logs.

This wasn't me; this was how the build scripts were written.

> There don't seem to be any flatpak release builds. Should there be and if
> so should I include them in the release downloads on SourceForge and
> Github?

Not yet.  Remember that right now code is sitting behind a tiny straw. 
Once I can move back to my main network then we can advertise that.

I don't know why it's not building the releases.  Maybe Geert knows?

> We only used trac for online code browsing, right? I don't see any reason
> to keep backups of that, all of the actual data are in the repositories.

I don't think we even use it for that anymore.  I don't even have trac
installed.  So it's probably safe to delete.  (I'll wait for confirmation
before I do).

> So 30G for Windows builds, 24G for Flatpack builds, and 20G for trac
> backups = 74G. What's the remaining 15G? The repos?

Repos.
Wiki data (and backups)
Email list data & archives
IRC Logs (or possibly something else.. supybot's dir is 4.1G
Then little bits here and there.

> Regards,
> John Ralls

-derek

-- 
   Derek Atkins 617-623-3745
   de...@ihtfp.com www.ihtfp.com
   Computer and Internet Security Consultant

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] [MAINT] Cleaning up Code (the server)

[John Ralls]

> On Aug 13, 2019, at 7:22 AM, Derek Atkins  wrote:
> 
> Hi,
> 
> Code's /home got to be 91% full (177GB used).  So I started looking at
> the data hogs.  The #1 hog was Win32 builds.
> 
> I cleaned out all interim (non-release) builds for 2.4, 2.6, 3-pre, and
> maint builds from 2018.  This reduced us down to 89GB used, so now
> there are only 30GB of win32 data remaining.
> 
> I also found we've got about 20GB of trac backups.  We stopped using
> trac over 3 years ago.  The most recent DB backup is dated Dec 30,
> 2016.  Is there any reason to keep these around anymore?
> 
> FWIW, flatpak builds are currently using 24G, so it's not ready to be
> pruned, yet.  But I suspect I'll need to learn a process to do so
> eventually.


Yay! I've been asking for you to clean up the old Windows builds for years! Now 
for a little preventive medicine: take a few minutes to add a cron job.

The logs directory could also stand some vacuuming, we really don't need to 
keep the logs around for more than a few months and they currently go back to 
the beginning of 2018.

If you want you could be more aggressive. We don't need to keep release builds 
at all, they're already all on SourceForge and the newer ones are on Github as 
well. If you want to keep a separate backup you could keep them somewhere 
offline. Maint and master builds are of marginal utility after the release they 
lead up to and it gets more marginal with every subsequent release. 6 months 
worth (two regular releases) seems about right.

Flatpak builds should follow the same logic as Windows builds. Incidentally I 
like the way that you've separated the flatpack build logs. 

There don't seem to be any flatpak release builds. Should there be and if so 
should I include them in the release downloads on SourceForge and Github?

We only used trac for online code browsing, right? I don't see any reason to 
keep backups of that, all of the actual data are in the repositories.

So 30G for Windows builds, 24G for Flatpack builds, and 20G for trac backups = 
74G. What's the remaining 15G? The repos?

Regards,
John Ralls

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] [MAINT] Cleaning up Code (the server)

[Derek Atkins]

Hi,

Code's /home got to be 91% full (177GB used).  So I started looking at
the data hogs.  The #1 hog was Win32 builds.

I cleaned out all interim (non-release) builds for 2.4, 2.6, 3-pre, and
maint builds from 2018.  This reduced us down to 89GB used, so now
there are only 30GB of win32 data remaining.

I also found we've got about 20GB of trac backups.  We stopped using
trac over 3 years ago.  The most recent DB backup is dated Dec 30,
2016.  Is there any reason to keep these around anymore?

FWIW, flatpak builds are currently using 24G, so it's not ready to be
pruned, yet.  But I suspect I'll need to learn a process to do so
eventually.

Thanks.

-derek
-- 
   Derek Atkins 617-623-3745
   de...@ihtfp.com www.ihtfp.com
   Computer and Internet Security Consultant
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel