Re: [GNC-dev] BZ: Migration Status (2018-06-15) -- Final Testing
> Message: 6 > Date: Fri, 15 Jun 2018 17:18:05 -0400 > From: Derek Atkins > To: gnucash-devel@gnucash.org > Subject: [GNC-dev] BZ: Migration Status (2018-06-15) -- Final Testing > Message-ID: > Content-Type: text/plain > > Hi Everyone, > > I've been working on the BZ migration and finished yet another migration test. > At this point I *THINK* we've got everything the way we want in preparation for > our final migration. > > Please take a look at https://bugs.gnucash.org/ > > Peruse the bug list. Check it out. See if there are any problems. > Also check out the wiki page for other notes and status: > https://wiki.gnucash.org/wiki/Bugzilla_Administration#New_Installation > > NOTES: > 1) All users have accounts in the database, but you will need to reset >your password in order to access it. > 2) The database will be blown away when we perform the final migration, >so you will need to reset your password again when we do the final >migration. > 3) Bug creation will not work, so don't try. > 4) I have not migrated any GnomeBZ extensions -- we can always add those >later (e.g. their browse page). > > Final migration is planned for the week of June 25. > > Please let me know if you find any issues! > > Thanks, > > -derek > -- >Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory >Member, MIT Student Information Processing Board (SIPB) >URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH >warl...@mit.eduPGP key available > > > -- Hi Derek, Thanks for all your work on this. There seems to be a problem with dates/times. In the old and new websites, I changed my preference to Timezone used to display dates and times : Australia/Sydney Now: https://bugzilla.gnome.org/show_bug.cgi?id=796154 Reported: 2018-05-16 10:42 AEST by Chris Good https://bugs.gnucash.org/show_bug.cgi?id=796154 Reported: 2018-05-16 14:42 AEST by Chris Good Sorry in advance if this is something you already have mentioned. Regards, Chris Good ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
[GNC-dev] Fwd: The two modules
Thanks; this is from an old version of eguile balsheet which is already
obsolete in 3.X onwards, and I know it'll be more difficult to fix as time
goes by. There are already changes from timepair to time64, compulsory CSS,
removal of slots access... There is no active eguile maintainer anymore,
and I really cannot understand eguile.
Would you be kind to comment upon the HTML output (File > Export > Export
Report) of the standard (non-eguile) balsheet, with desired amendments? I
can try to amend with backward compatibility.
For bonus points, if you have a sample datafile with accounts and
transactions that highlight the various functionality of balsheet (e.g.
income/expense, equity, any foreign currency conversions or stock purchases
- I'm sure the wife will have them in her textbooks) I'm sure I can fix the
standard balsheet to her standards!
So, wishlist:
- datafile with example transactions
- current html report of standard balsheet
- annotated ideal report produced by standard balsheet :)
Let's keep this discussion public for external input too.
Regards
-- Forwarded message --
From: Stephen M. Butler
Date: 16 June 2018 at 00:45
Subject: The two modules
To: Christopher Lam
Hope this helps. As I mentioned in the list email, perhaps there is a
way to show the Trading Accounts if they are present and suppress if
they are not.
Also, I think it would be acceptable to have an Income/Expense Report on
a separate page but generated by the same module. That would ensure
that the Profit/Loss line (or whatever folks want to call it) would be
the same for both reports. But my wife was adamant that they are two
separate reports.
Thanks for the example as this is my first time seeing this language.
You might find that I took some shortcuts where it wasn't appropriate
for the long haul -- especially around the printing of the Profit/Loss
line. I'm still now sure what the #t #f and a couple of the other
parameters actually do -- I just got the report to look right!
--Steve
--
Stephen M Butler, PMP, PSM
stephen.m.butle...@gmail.com
kg...@arrl.net
253-350-0166
---
GnuPG Fingerprint: 8A25 9726 D439 758D D846 E5D4 282A 5477 0385 81D8
" x "")
(set! depth (max depth 1)); hack for depth=0
(if leftoverrule?
(begin
(set! lo-adjust -1)
(set! lo-cell "")))
?>
(accrec-depth accrec) 0))
)
; has sub-accounts: shift left to put balance in same column as sub-accounts
(set! rshift2 -1))
; Don't show zero amount for a placeholder -- the value to
; test for zero depends on whether or not this is a 'summary' value
; (i.e. a total of sub-accounts that are not shown separately)
(if (and (accrec-placeholder? accrec)
(if (accrec-summary? accrec)
(not (accrec-non-zero? accrec))
(gnc-numeric-zero-p (accrec-balance-num accrec
(set! showamt? #f))
(display-acc-row
maxdepth
(accrec-depth accrec)
(+ rshift rshift2)
(accrec-namelink accrec)
(if showamt?
(if (accrec-summary? accrec)
(format-comm-coll (accrec-subtotal-cc accrec))
(format-monetary (accrec-balance-mny accrec)))
""
)
(< (accrec-depth accrec) 1); total?
#f) ; leftoverrule?
(if (accrec-sublist accrec)
(begin
; recurse to deeper accounts...
(display-accounts-table-r (accrec-sublist accrec) neg? maxdepth rshift onedepth1)
; ...and then display the total
; unless there is only one depth-1 account
(if (not (and onedepth1
(= 1 (accrec-depth accrec
(display-acc-row
maxdepth
(accrec-depth accrec)
(if (> (accrec-depth accrec) 1) rshift 0)
(string-append " " )
(format-comm-coll-total (accrec-subtotal-cc accrec))
(<= (accrec-depth accrec) 1); total?
(> (accrec-depth accrec) 0))) ; leftoverrule?
)
?>
body {
font-family: ;
font-size: ;
}
table { /* table does not inherit font sizes for some reason */
font-size: ;
}
(accrec-treedepth accrec-as) 1))
(set! rshift-as 1))
(if (and (one-depth-1 accrec-li)
(> (accrec-treedepth accrec-li) 1))
(set! rshift-li 1))
(if (and (one-depth-1 accrec-eq)
(> (accrec-treedepth accrec-eq) 1))
(set!
Re: [GNC-dev] BZ: Migration Status (2018-06-15) -- Final Testing
Since that is almost the same time as release 3.2 happens there could be a flurry of users wanting to read bugs listed in the release announcement and perhaps enter new bugs or updates. I suggest that the 6.3 release announcement mention the BZ release and warn as appropriate. David C On Fri, Jun 15, 2018, 4:19 PM Derek Atkins wrote: > Hi Everyone, > > I've been working on the BZ migration and finished yet another migration > test. At this point I *THINK* we've got everything the way we want in > preparation for our final migration. > > Please take a look at https://bugs.gnucash.org/ > > Peruse the bug list. Check it out. See if there are any problems. > Also check out the wiki page for other notes and status: > https://wiki.gnucash.org/wiki/Bugzilla_Administration#New_Installation > > NOTES: > 1) All users have accounts in the database, but you will need to reset >your password in order to access it. > 2) The database will be blown away when we perform the final migration, >so you will need to reset your password again when we do the final >migration. > 3) Bug creation will not work, so don't try. > 4) I have not migrated any GnomeBZ extensions -- we can always add those >later (e.g. their browse page). > > Final migration is planned for the week of June 25. > > Please let me know if you find any issues! > > Thanks, > > -derek > -- >Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory >Member, MIT Student Information Processing Board (SIPB) >URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH >warl...@mit.eduPGP key available > ___ > gnucash-devel mailing list > gnucash-devel@gnucash.org > https://lists.gnucash.org/mailman/listinfo/gnucash-devel > ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
[GNC-dev] BZ: Migration Status (2018-06-15) -- Final Testing
Hi Everyone, I've been working on the BZ migration and finished yet another migration test. At this point I *THINK* we've got everything the way we want in preparation for our final migration. Please take a look at https://bugs.gnucash.org/ Peruse the bug list. Check it out. See if there are any problems. Also check out the wiki page for other notes and status: https://wiki.gnucash.org/wiki/Bugzilla_Administration#New_Installation NOTES: 1) All users have accounts in the database, but you will need to reset your password in order to access it. 2) The database will be blown away when we perform the final migration, so you will need to reset your password again when we do the final migration. 3) Bug creation will not work, so don't try. 4) I have not migrated any GnomeBZ extensions -- we can always add those later (e.g. their browse page). Final migration is planned for the week of June 25. Please let me know if you find any issues! Thanks, -derek -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH warl...@mit.eduPGP key available ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Slow user interface in 3.x gnucash (for large files), and potential optimization
> On Jun 15, 2018, at 12:55 PM, Christian Stimming > wrote: > > Am Freitag, 15. Juni 2018, 10:05:09 schrieb John Ralls: A very good catch indeed. But pre-constructing the string in qofbook.cpp only saves two string constructions per invocation as the vector still has to make its own copies. I guess that its much worse for you because the ancient gcc on Ubuntu 14.04 (gcc4.8) doesn't do small-string optimization. >>> >>> Is there any reason we cant use std::string& in the vector? Or do we >>> think that we might lose references there? >> >> In this instance the string is static so it would be safe to use const >> std::string&, but while there's an is_volatile type trait there's no >> is_static so we can't insert a static assert to catch cases where the >> actual std::string is on the stack. That could lead to some ugly bugs. > > The std containers are specified in a way so that they make only sense to > contain the values by-value (ok, this might be formulated a bit too short, > but > not too long ago this was a good rule of thumb). It is up to the > implementation to make optimizations (copy-on-write and such), but the > interface requires the programmer to think of the stored type as "by-value". > > Nevertheless I think my change made good use of the std::string's already > existing optimizations. In particular, the saving was so surprisingly large > here that I think gcc's std::string itself implements some sort of copy-on- > write optimization, and this means its content isn't deep-copied in these > calls. > > Some numbers from the valgrind/callgrind evaluation. I got the following > number of calls, starting from qof_query_run and skipping some uninterestings > calls in between: > > - 6 qof_query_run > - 6 g_list_sort_with_data > - 360,000 (approx.) sort_func - due to the number of splits and txn in my > file > - 360,000 xaccSplitOrder >The most expensive part in this function is this next callee: > - 360,000 qof_book_use_split_action_for_num_field > - 360,000 (approx.) qof_book_get_property > > This function in turn has approx. 1,100,000 calls to std::basic_string > constructor and destructor. The change in my commit is this: > Before my commit, these calls also caused 1,100,000 calls to > std::string::_S_create and std::string::_M_destroy i.e. the by-value > constructor and destructor. After my commit, I see a mere 22,000 calls to > std::string::_S_create and std::string::_M_destroy. > >> Besides, using strings is still leaving performance on the table: A string >> comparison is n times longer than an int comparison where n is the number >> of consecutive leading characters that are the same in the two strings. I >> got a huge speedup on loading a few months ago because GncGUID's >> operator==() was doing a character-wise comparison of the ascii >> representation instead of comparing the two int64_ts of the actual GUID. > > Sounds good. The above numbers seem to indicate that std::string already > optimizes away a long comparison as long as the string memory itself is > identical. In my opinion this is good enough of an optimization. > >> KVP paths aren't really a good use for std::string anyway: It's a lot of >> weight for something that's used as a human-readable index. Even static >> char[] is heavy for this purpose. We could get a huge boost if we use >> something like Glib's quarks [1]. >> Want to have a go at that? > > Err... this would mean a major refactoring of any access to the KVP system. > No, I don't feel motivated to work into that direction. Also, if I understnad > the above valgrind evaluation correctly, we can already achieve almost > optimum > (i.e. pointer comparison) by ensuring a set of std::string constants to be > used. I think this is the most efficient way to proceed here, and I would > just > push this commit after some more cleanup. Christian, Hardly major, particularly considering that Aaron has already completely rewritten KVP into C++ and separately changed the access from '/'-delimited string paths to std::vector of the tokens. All that's required is lifting Glib's GQuark and converting it to C++, changing KVP frame's std::vector to std::vector, and using g_quark_intern_from_static_string instead of static std::string. The situation you've tested is a bit of a special case because all of the uses of the KVP_OPTIONS keys are in one file. Any keys used in more than one file will have different static std::strings and pointer comparison won't work. Examples include anything accessed with qof_instance_set/get. That said, the mallocs/deallocs are 99% or more of the performance hit. If you're willing and have time to do it your way and have time to get it done quickly then by all means go ahead. Regards, John Ralls ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Slow user interface in 3.x gnucash (for large files), and potential optimization
Am Freitag, 15. Juni 2018, 10:05:09 schrieb John Ralls: > >> A very good catch indeed. But pre-constructing the string in > >> qofbook.cpp only saves two string constructions per invocation as the > >> vector still has to make its own copies. I guess that its much worse > >> for you because the ancient gcc on Ubuntu 14.04 (gcc4.8) doesn't do > >> small-string optimization. > > > > Is there any reason we cant use std::string& in the vector? Or do we > > think that we might lose references there? > > In this instance the string is static so it would be safe to use const > std::string&, but while there's an is_volatile type trait there's no > is_static so we can't insert a static assert to catch cases where the > actual std::string is on the stack. That could lead to some ugly bugs. The std containers are specified in a way so that they make only sense to contain the values by-value (ok, this might be formulated a bit too short, but not too long ago this was a good rule of thumb). It is up to the implementation to make optimizations (copy-on-write and such), but the interface requires the programmer to think of the stored type as "by-value". Nevertheless I think my change made good use of the std::string's already existing optimizations. In particular, the saving was so surprisingly large here that I think gcc's std::string itself implements some sort of copy-on- write optimization, and this means its content isn't deep-copied in these calls. Some numbers from the valgrind/callgrind evaluation. I got the following number of calls, starting from qof_query_run and skipping some uninterestings calls in between: - 6 qof_query_run - 6 g_list_sort_with_data - 360,000 (approx.) sort_func - due to the number of splits and txn in my file - 360,000 xaccSplitOrder The most expensive part in this function is this next callee: - 360,000 qof_book_use_split_action_for_num_field - 360,000 (approx.) qof_book_get_property This function in turn has approx. 1,100,000 calls to std::basic_string constructor and destructor. The change in my commit is this: Before my commit, these calls also caused 1,100,000 calls to std::string::_S_create and std::string::_M_destroy i.e. the by-value constructor and destructor. After my commit, I see a mere 22,000 calls to std::string::_S_create and std::string::_M_destroy. > Besides, using strings is still leaving performance on the table: A string > comparison is n times longer than an int comparison where n is the number > of consecutive leading characters that are the same in the two strings. I > got a huge speedup on loading a few months ago because GncGUID's > operator==() was doing a character-wise comparison of the ascii > representation instead of comparing the two int64_ts of the actual GUID. Sounds good. The above numbers seem to indicate that std::string already optimizes away a long comparison as long as the string memory itself is identical. In my opinion this is good enough of an optimization. > KVP paths aren't really a good use for std::string anyway: It's a lot of > weight for something that's used as a human-readable index. Even static > char[] is heavy for this purpose. We could get a huge boost if we use > something like Glib's quarks [1]. > Want to have a go at that? Err... this would mean a major refactoring of any access to the KVP system. No, I don't feel motivated to work into that direction. Also, if I understnad the above valgrind evaluation correctly, we can already achieve almost optimum (i.e. pointer comparison) by ensuring a set of std::string constants to be used. I think this is the most efficient way to proceed here, and I would just push this commit after some more cleanup. Regards, Christian ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Slow user interface in 3.x gnucash (for large files), and potential optimization
> On Jun 15, 2018, at 9:16 AM, Derek Atkins wrote: > > John Ralls writes: > >> A very good catch indeed. But pre-constructing the string in >> qofbook.cpp only saves two string constructions per invocation as the >> vector still has to make its own copies. I guess that its much worse >> for you because the ancient gcc on Ubuntu 14.04 (gcc4.8) doesn't do >> small-string optimization. > > Is there any reason we cant use std::string& in the vector? Or do we > think that we might lose references there? > In this instance the string is static so it would be safe to use const std::string&, but while there's an is_volatile type trait there's no is_static so we can't insert a static assert to catch cases where the actual std::string is on the stack. That could lead to some ugly bugs. Besides, using strings is still leaving performance on the table: A string comparison is n times longer than an int comparison where n is the number of consecutive leading characters that are the same in the two strings. I got a huge speedup on loading a few months ago because GncGUID's operator==() was doing a character-wise comparison of the ascii representation instead of comparing the two int64_ts of the actual GUID. Regards, John Ralls ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Slow user interface in 3.x gnucash (for large files), and potential optimization
John Ralls writes: > A very good catch indeed. But pre-constructing the string in > qofbook.cpp only saves two string constructions per invocation as the > vector still has to make its own copies. I guess that its much worse > for you because the ancient gcc on Ubuntu 14.04 (gcc4.8) doesn't do > small-string optimization. Is there any reason we cant use std::string& in the vector? Or do we think that we might lose references there? > KVP paths aren't really a good use for std::string anyway: It's a lot > of weight for something that's used as a human-readable index. Even > static char[] is heavy for this purpose. We could get a huge boost if > we use something like Glib's quarks [1]. They can be expanded to their > string value for storage so that we don't break file/db compatibility. > > Want to have a go at that? -derek -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH warl...@mit.eduPGP key available ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] GnuCash Bugzilla URLs -- /bugzilla or not?
Derek Atkins writes: > Adrien Monteleone writes: > >> If it is not an Apache problem, one other consideration might be an >> .htaccess rule inserted there by BZ. > > More likely an Alias vs Directory issue. > I'll play. For the record, I did get this working. >> Regards, >> Adrien -derek -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH warl...@mit.eduPGP key available ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
