Re: [GNC-dev] More need for speed improvement around XML
> On Dec 30, 2018, at 4:52 PM, John Ralls wrote:
>
>
>
>> On Dec 30, 2018, at 1:25 PM, Christian Stimming
>> wrote:
>>
>> Am Sonntag, 30. Dezember 2018, 00:44:40 CET schrieb John Ralls:
>>>> On Dec 29, 2018, at 2:16 PM, Christian Stimming
>>>> wrote:>
>>>> Am Samstag, 29. Dezember 2018, 01:45:32 CET schrieb John Ralls:
>>>>>> When saving to XML file, for each transaction the call stack with the
>>>>>> expensive code walks down like this:
>>>>>>
>>>>>> gnc_transaction_dom_tree_create
>>>>>> add_time64
>>>>>> time64_to_dom_tree
>>>>>> gnc_print_time64
>>>>>> GncDateTime::format
>>>>>> GncDateTimeImpl::format
>>>>>
>>>>> Christian,
>>>>>
>>>>> I don't have time to fully test it out right now, but give
>>>>> https://github.com/jralls/gnucash/tree/maint a go.
>>>>>
>>>>> As noted in the HEAD commit it has a side effect of recording times in
>>>>> XML
>>>>> files in UTC, no timezones. Users who want to keep their files in version
>>>>> control will like that, but it needs to be checked for backward
>>>>> compatibility with 2.6 before it can be merged into maint.
>>>>
>>>> Thanks for working at this. Indeed the main issue is that we don't need a
>>>> user-visible format conversion (which must respect the locale) but instead
>>>> a serialization to a fixed iso8601 format. Fixing this issue also means
>>>> that the XML files will no longer depend on the local timezone, which is
>>>> an issue that has been asked for a long time.
>>>>
>>>> Thanks for your patch - this fixes performance issues with
>>>> gnc_time64_to_iso8601_buff() and all of its callers. However, the
>>>> Save-to-XML function calls gnc_print_time64() instead (from
>>>> time64_to_dom_tree), which is a different code path. Consequently, your
>>>> patch alone does not modify the saved XML (just verified). Your call to
>>>> GncDateTime::format_iso8601 needs to be used in gnc_print_time64, too,
>>>> like so:
>>>>
>>>> xmlNodePtr
>>>> time64_to_dom_tree (const char* tag, const time64 time)
>>>> {
>>>>
>>>> xmlNodePtr ret;
>>>> g_return_val_if_fail (time != INT64_MAX, NULL);
>>>>
>>>> -auto date_str = gnc_print_time64 (time, "%Y-%m-%d %H:%M:%S %q");
>>>> -if (!date_str)
>>>> -return NULL;
>>>> +GncDateTime gncdt(time);
>>>> +auto sstr = gncdt.format_iso8601();
>>>> +gchar* date_str = g_strdup(sstr.c_str());
>>>>
>>>>
>>>> However, with this patch, all time64 timestamps in the XML file will then
>>>> change from time+timezone to UTC time only. To my surprise, although we
>>>> knew about this issue for such a long time and changed most timestamps
>>>> appropriately, there are still timestamps that will change date due to
>>>> this
>>>> change. With a quick glance, the "reconciled-date" timestamp contains
>>>> beginning-of-day in a local timezone. Also maybe user-entered commodity
>>>> prices. Others might still show up, oh well.
>>>>
>>>> I haven't checked for backward compatibility with 2.6, but will do during
>>>> the next days.
>>>
>>> Rats, I got the two functions scrambled.
>>>
>>> Geert and I discussed a month or two ago expanding the use of time-neutral
>>> to everywhere that we currently use day-begin and day-end except searches,
>>> but neither of us has gotten to it yet.
>>
>> For completeness: I checked loading such a file with gnucash-2.6, and as far
>> as I can tell, this runs just fine. All transactions and their dates show up
>> as usual even though the timestamps are now read as UTC instead of localtime.
>
> The timestamps have always been UTC, they've just been written oddly. It's
> explained in comments somewhere, I'll summarize: We've historically written
> timestamps as an ISO8607-like string, i.e. -MM-DD HH:MM:SS with an offset
> representing a timezone, ±HHMM. For example my TZ offset is -0800, so the
> date-time right now is 2018-12-30 14:26:24 -0800. The parser adds back the 8
> hours and stores whatever 2018-12-30 22:26:24 is in seconds since the epoch.
> Losing the offset doesn't change anything except to make less work for
> the parser and reducing the diff of two files written with different offsets.
>
> It's good news that the 2.6 parser reads them correctly.
>
> Since SQL doesn't understand offsets we've always used plain UTC date times
> in the SQL backend.
I've merged Chris Carson's PRs to speed up the scrub and to get the C++ locale
only once and I've fixed up my format_iso8601 so that it's used in both
backends instead of GncDateTime::format and pushed that as well.
Regards,
John Ralls
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] More need for speed improvement around XML
> On Dec 30, 2018, at 1:25 PM, Christian Stimming
> wrote:
>
> Am Sonntag, 30. Dezember 2018, 00:44:40 CET schrieb John Ralls:
>>> On Dec 29, 2018, at 2:16 PM, Christian Stimming
>>> wrote:>
>>> Am Samstag, 29. Dezember 2018, 01:45:32 CET schrieb John Ralls:
> When saving to XML file, for each transaction the call stack with the
> expensive code walks down like this:
>
> gnc_transaction_dom_tree_create
> add_time64
> time64_to_dom_tree
> gnc_print_time64
> GncDateTime::format
> GncDateTimeImpl::format
Christian,
I don't have time to fully test it out right now, but give
https://github.com/jralls/gnucash/tree/maint a go.
As noted in the HEAD commit it has a side effect of recording times in
XML
files in UTC, no timezones. Users who want to keep their files in version
control will like that, but it needs to be checked for backward
compatibility with 2.6 before it can be merged into maint.
>>>
>>> Thanks for working at this. Indeed the main issue is that we don't need a
>>> user-visible format conversion (which must respect the locale) but instead
>>> a serialization to a fixed iso8601 format. Fixing this issue also means
>>> that the XML files will no longer depend on the local timezone, which is
>>> an issue that has been asked for a long time.
>>>
>>> Thanks for your patch - this fixes performance issues with
>>> gnc_time64_to_iso8601_buff() and all of its callers. However, the
>>> Save-to-XML function calls gnc_print_time64() instead (from
>>> time64_to_dom_tree), which is a different code path. Consequently, your
>>> patch alone does not modify the saved XML (just verified). Your call to
>>> GncDateTime::format_iso8601 needs to be used in gnc_print_time64, too,
>>> like so:
>>>
>>> xmlNodePtr
>>> time64_to_dom_tree (const char* tag, const time64 time)
>>> {
>>>
>>>xmlNodePtr ret;
>>>g_return_val_if_fail (time != INT64_MAX, NULL);
>>>
>>> -auto date_str = gnc_print_time64 (time, "%Y-%m-%d %H:%M:%S %q");
>>> -if (!date_str)
>>> -return NULL;
>>> +GncDateTime gncdt(time);
>>> +auto sstr = gncdt.format_iso8601();
>>> +gchar* date_str = g_strdup(sstr.c_str());
>>>
>>>
>>> However, with this patch, all time64 timestamps in the XML file will then
>>> change from time+timezone to UTC time only. To my surprise, although we
>>> knew about this issue for such a long time and changed most timestamps
>>> appropriately, there are still timestamps that will change date due to
>>> this
>>> change. With a quick glance, the "reconciled-date" timestamp contains
>>> beginning-of-day in a local timezone. Also maybe user-entered commodity
>>> prices. Others might still show up, oh well.
>>>
>>> I haven't checked for backward compatibility with 2.6, but will do during
>>> the next days.
>>
>> Rats, I got the two functions scrambled.
>>
>> Geert and I discussed a month or two ago expanding the use of time-neutral
>> to everywhere that we currently use day-begin and day-end except searches,
>> but neither of us has gotten to it yet.
>
> For completeness: I checked loading such a file with gnucash-2.6, and as far
> as I can tell, this runs just fine. All transactions and their dates show up
> as usual even though the timestamps are now read as UTC instead of localtime.
The timestamps have always been UTC, they've just been written oddly. It's
explained in comments somewhere, I'll summarize: We've historically written
timestamps as an ISO8607-like string, i.e. -MM-DD HH:MM:SS with an offset
representing a timezone, ±HHMM. For example my TZ offset is -0800, so the
date-time right now is 2018-12-30 14:26:24 -0800. The parser adds back the 8
hours and stores whatever 2018-12-30 22:26:24 is in seconds since the epoch.
Losing the offset doesn't change anything except to make less work for
the parser and reducing the diff of two files written with different offsets.
It's good news that the 2.6 parser reads them correctly.
Since SQL doesn't understand offsets we've always used plain UTC date times in
the SQL backend.
Regards,
John Ralls
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] More need for speed improvement around XML
Am Sonntag, 30. Dezember 2018, 00:44:40 CET schrieb John Ralls:
> > On Dec 29, 2018, at 2:16 PM, Christian Stimming
> > wrote:>
> > Am Samstag, 29. Dezember 2018, 01:45:32 CET schrieb John Ralls:
> >>> When saving to XML file, for each transaction the call stack with the
> >>> expensive code walks down like this:
> >>>
> >>> gnc_transaction_dom_tree_create
> >>> add_time64
> >>> time64_to_dom_tree
> >>> gnc_print_time64
> >>> GncDateTime::format
> >>> GncDateTimeImpl::format
> >>
> >> Christian,
> >>
> >> I don't have time to fully test it out right now, but give
> >> https://github.com/jralls/gnucash/tree/maint a go.
> >>
> >> As noted in the HEAD commit it has a side effect of recording times in
> >> XML
> >> files in UTC, no timezones. Users who want to keep their files in version
> >> control will like that, but it needs to be checked for backward
> >> compatibility with 2.6 before it can be merged into maint.
> >
> > Thanks for working at this. Indeed the main issue is that we don't need a
> > user-visible format conversion (which must respect the locale) but instead
> > a serialization to a fixed iso8601 format. Fixing this issue also means
> > that the XML files will no longer depend on the local timezone, which is
> > an issue that has been asked for a long time.
> >
> > Thanks for your patch - this fixes performance issues with
> > gnc_time64_to_iso8601_buff() and all of its callers. However, the
> > Save-to-XML function calls gnc_print_time64() instead (from
> > time64_to_dom_tree), which is a different code path. Consequently, your
> > patch alone does not modify the saved XML (just verified). Your call to
> > GncDateTime::format_iso8601 needs to be used in gnc_print_time64, too,
> > like so:
> >
> > xmlNodePtr
> > time64_to_dom_tree (const char* tag, const time64 time)
> > {
> >
> > xmlNodePtr ret;
> > g_return_val_if_fail (time != INT64_MAX, NULL);
> >
> > -auto date_str = gnc_print_time64 (time, "%Y-%m-%d %H:%M:%S %q");
> > -if (!date_str)
> > -return NULL;
> > +GncDateTime gncdt(time);
> > +auto sstr = gncdt.format_iso8601();
> > +gchar* date_str = g_strdup(sstr.c_str());
> >
> >
> > However, with this patch, all time64 timestamps in the XML file will then
> > change from time+timezone to UTC time only. To my surprise, although we
> > knew about this issue for such a long time and changed most timestamps
> > appropriately, there are still timestamps that will change date due to
> > this
> > change. With a quick glance, the "reconciled-date" timestamp contains
> > beginning-of-day in a local timezone. Also maybe user-entered commodity
> > prices. Others might still show up, oh well.
> >
> > I haven't checked for backward compatibility with 2.6, but will do during
> > the next days.
>
> Rats, I got the two functions scrambled.
>
> Geert and I discussed a month or two ago expanding the use of time-neutral
> to everywhere that we currently use day-begin and day-end except searches,
> but neither of us has gotten to it yet.
For completeness: I checked loading such a file with gnucash-2.6, and as far
as I can tell, this runs just fine. All transactions and their dates show up
as usual even though the timestamps are now read as UTC instead of localtime.
Regards,
Christian
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] More need for speed improvement around XML
> On Dec 29, 2018, at 2:16 PM, Christian Stimming
> wrote:
>
> Am Samstag, 29. Dezember 2018, 01:45:32 CET schrieb John Ralls:
>>> When saving to XML file, for each transaction the call stack with the
>>> expensive code walks down like this:
>>>
>>> gnc_transaction_dom_tree_create
>>> add_time64
>>> time64_to_dom_tree
>>> gnc_print_time64
>>> GncDateTime::format
>>> GncDateTimeImpl::format
>>
>> Christian,
>>
>> I don't have time to fully test it out right now, but give
>> https://github.com/jralls/gnucash/tree/maint a go.
>>
>> As noted in the HEAD commit it has a side effect of recording times in XML
>> files in UTC, no timezones. Users who want to keep their files in version
>> control will like that, but it needs to be checked for backward
>> compatibility with 2.6 before it can be merged into maint.
>
> Thanks for working at this. Indeed the main issue is that we don't need a
> user-visible format conversion (which must respect the locale) but instead a
> serialization to a fixed iso8601 format. Fixing this issue also means that
> the
> XML files will no longer depend on the local timezone, which is an issue that
> has been asked for a long time.
>
> Thanks for your patch - this fixes performance issues with
> gnc_time64_to_iso8601_buff() and all of its callers. However, the Save-to-XML
> function calls gnc_print_time64() instead (from time64_to_dom_tree), which is
> a different code path. Consequently, your patch alone does not modify the
> saved XML (just verified). Your call to GncDateTime::format_iso8601 needs to
> be used in gnc_print_time64, too, like so:
>
> xmlNodePtr
> time64_to_dom_tree (const char* tag, const time64 time)
> {
> xmlNodePtr ret;
> g_return_val_if_fail (time != INT64_MAX, NULL);
> -auto date_str = gnc_print_time64 (time, "%Y-%m-%d %H:%M:%S %q");
> -if (!date_str)
> -return NULL;
> +GncDateTime gncdt(time);
> +auto sstr = gncdt.format_iso8601();
> +gchar* date_str = g_strdup(sstr.c_str());
>
>
> However, with this patch, all time64 timestamps in the XML file will then
> change from time+timezone to UTC time only. To my surprise, although we knew
> about this issue for such a long time and changed most timestamps
> appropriately, there are still timestamps that will change date due to this
> change. With a quick glance, the "reconciled-date" timestamp contains
> beginning-of-day in a local timezone. Also maybe user-entered commodity
> prices. Others might still show up, oh well.
>
> I haven't checked for backward compatibility with 2.6, but will do during the
> next days.
Rats, I got the two functions scrambled.
Geert and I discussed a month or two ago expanding the use of time-neutral to
everywhere that we currently use day-begin and day-end except searches, but
neither of us has gotten to it yet.
Regards,
John Ralls
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] More need for speed improvement around XML
Am Samstag, 29. Dezember 2018, 01:45:32 CET schrieb John Ralls:
> > When saving to XML file, for each transaction the call stack with the
> > expensive code walks down like this:
> >
> > gnc_transaction_dom_tree_create
> > add_time64
> > time64_to_dom_tree
> > gnc_print_time64
> > GncDateTime::format
> > GncDateTimeImpl::format
>
> Christian,
>
> I don't have time to fully test it out right now, but give
> https://github.com/jralls/gnucash/tree/maint a go.
>
> As noted in the HEAD commit it has a side effect of recording times in XML
> files in UTC, no timezones. Users who want to keep their files in version
> control will like that, but it needs to be checked for backward
> compatibility with 2.6 before it can be merged into maint.
Thanks for working at this. Indeed the main issue is that we don't need a
user-visible format conversion (which must respect the locale) but instead a
serialization to a fixed iso8601 format. Fixing this issue also means that the
XML files will no longer depend on the local timezone, which is an issue that
has been asked for a long time.
Thanks for your patch - this fixes performance issues with
gnc_time64_to_iso8601_buff() and all of its callers. However, the Save-to-XML
function calls gnc_print_time64() instead (from time64_to_dom_tree), which is
a different code path. Consequently, your patch alone does not modify the
saved XML (just verified). Your call to GncDateTime::format_iso8601 needs to
be used in gnc_print_time64, too, like so:
xmlNodePtr
time64_to_dom_tree (const char* tag, const time64 time)
{
xmlNodePtr ret;
g_return_val_if_fail (time != INT64_MAX, NULL);
-auto date_str = gnc_print_time64 (time, "%Y-%m-%d %H:%M:%S %q");
-if (!date_str)
-return NULL;
+GncDateTime gncdt(time);
+auto sstr = gncdt.format_iso8601();
+gchar* date_str = g_strdup(sstr.c_str());
However, with this patch, all time64 timestamps in the XML file will then
change from time+timezone to UTC time only. To my surprise, although we knew
about this issue for such a long time and changed most timestamps
appropriately, there are still timestamps that will change date due to this
change. With a quick glance, the "reconciled-date" timestamp contains
beginning-of-day in a local timezone. Also maybe user-entered commodity
prices. Others might still show up, oh well.
I haven't checked for backward compatibility with 2.6, but will do during the
next days.
Regards,
Christian
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] More need for speed improvement around XML (Re: Introduction, a story, and 50% improvement in XML loading speed)
> On Dec 28, 2018, at 2:14 PM, Christian Stimming
> wrote:
>
> Am Montag, 24. Dezember 2018, 13:38:27 CET schrieb Chris Carson:
>> TL;DR: hi! I'm a programmer! The attached patch to one line of code
>> gives a 50% reduction in XML load CPU use! Skeptical? I was.
>
> Dear Chris,
>
> thanks a lot for this patch! I'm using gnucash since many years, too, and I'm
> still somewhat unhappy with some of the reaction times in the 3.x series of
> gnucash.
>
> Now that you got a good catch on the XML loading time, maybe you are up for
> some improvements on the XML saving time...? :-) I ran some valgrind where I
> could locate an expensive code section, but I'm a bit out of practice to know
> the C++11 way of writing that code in a more efficient way. Here's what I
> found:
>
> When saving to XML file, for each transaction the call stack with the
> expensive code walks down like this:
>
> gnc_transaction_dom_tree_create
> add_time64
> time64_to_dom_tree
> gnc_print_time64
> GncDateTime::format
> GncDateTimeImpl::format
>
> and this very last function seems significantly more expensive than needed,
> slowing down saving considerably:
>
> std::string
> GncDateTimeImpl::format(const char* format) const
> {
>using Facet = boost::local_time::local_time_facet;
>std::stringstream ss;
>//The stream destructor frees the facet, so it must be heap-allocated.
>auto output_facet(new Facet(normalize_format(format).c_str()));
>// FIXME Rather than imbueing a locale below we probably should set
> std::locale::global appropriately somewhere.
>ss.imbue(std::locale(std::locale(""), output_facet));
>ss << m_time;
>return ss.str();
> }
>
> Both the operator<< and the constructor std::locale(const char*) seem to be
> quite costly operations. To me, it seems there is way too much construction/
> destruction going on for options that are constant during each saving
> operation. Maybe even the recent commit 7f1a7115 (Frank?) has slowed down
> things here even more. Maybe this could be checked in some of the unittests
> more easily, but I'm a bit out of practice there, too. Maybe someone is up
> for
> some performance boosting here...? Thanks a lot in advance!
Christian,
I don't have time to fully test it out right now, but give
https://github.com/jralls/gnucash/tree/maint a go.
As noted in the HEAD commit it has a side effect of recording times in XML
files in UTC, no timezones. Users who want to keep their files in version
control will like that, but it needs to be checked for backward compatibility
with 2.6 before it can be merged into maint.
Regards,
John Ralls
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
[GNC-dev] More need for speed improvement around XML (Re: Introduction, a story, and 50% improvement in XML loading speed)
Am Montag, 24. Dezember 2018, 13:38:27 CET schrieb Chris Carson:
> TL;DR: hi! I'm a programmer! The attached patch to one line of code
> gives a 50% reduction in XML load CPU use! Skeptical? I was.
Dear Chris,
thanks a lot for this patch! I'm using gnucash since many years, too, and I'm
still somewhat unhappy with some of the reaction times in the 3.x series of
gnucash.
Now that you got a good catch on the XML loading time, maybe you are up for
some improvements on the XML saving time...? :-) I ran some valgrind where I
could locate an expensive code section, but I'm a bit out of practice to know
the C++11 way of writing that code in a more efficient way. Here's what I
found:
When saving to XML file, for each transaction the call stack with the
expensive code walks down like this:
gnc_transaction_dom_tree_create
add_time64
time64_to_dom_tree
gnc_print_time64
GncDateTime::format
GncDateTimeImpl::format
and this very last function seems significantly more expensive than needed,
slowing down saving considerably:
std::string
GncDateTimeImpl::format(const char* format) const
{
using Facet = boost::local_time::local_time_facet;
std::stringstream ss;
//The stream destructor frees the facet, so it must be heap-allocated.
auto output_facet(new Facet(normalize_format(format).c_str()));
// FIXME Rather than imbueing a locale below we probably should set
std::locale::global appropriately somewhere.
ss.imbue(std::locale(std::locale(""), output_facet));
ss << m_time;
return ss.str();
}
Both the operator<< and the constructor std::locale(const char*) seem to be
quite costly operations. To me, it seems there is way too much construction/
destruction going on for options that are constant during each saving
operation. Maybe even the recent commit 7f1a7115 (Frank?) has slowed down
things here even more. Maybe this could be checked in some of the unittests
more easily, but I'm a bit out of practice there, too. Maybe someone is up for
some performance boosting here...? Thanks a lot in advance!
Best Regards,
Christian
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Introduction, a story, and 50% improvement in XML loading speed
Op maandag 24 december 2018 14:29:15 CET schreef Geert Janssens: > Op maandag 24 december 2018 13:38:27 CET schreef Chris Carson: > > *The Patch* > > > > I tried a couple of different fixes to this. The patch below copies off > > and validates only the bytes being consumed. It brings the user CPU to > > startup and load my XML file from ~38 seconds to ~20.5 seconds, and given > > that 4.5 seconds of that is startup I make that about a 50% improvement in > > load speed. I tried a more aggressive fix for funsies and it wasn't much > > better. > > > > I have tested this *ONLY* on the load of my largeish XML file. But the > > patched code reads well. > > > > What would you guys advise as next steps? > > No you celebrate your first patch was included in the project's code base :) > A good find, congratulations! That was meant to read "Now you celebrate..." My keyboard needs replacement. Geert ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] Introduction, a story, and 50% improvement in XML loading speed
Op maandag 24 december 2018 13:38:27 CET schreef Chris Carson: > *The Patch* > > I tried a couple of different fixes to this. The patch below copies off > and validates only the bytes being consumed. It brings the user CPU to > startup and load my XML file from ~38 seconds to ~20.5 seconds, and given > that 4.5 seconds of that is startup I make that about a 50% improvement in > load speed. I tried a more aggressive fix for funsies and it wasn't much > better. > > I have tested this *ONLY* on the load of my largeish XML file. But the > patched code reads well. > > What would you guys advise as next steps? No you celebrate your first patch was included in the project's code base :) A good find, congratulations! I have committed your fix. It is consistent with the same we also do in generic_accumulate_chars (sixtp-utils.cpp :80) For future submissions it's probably easier to generate a pull request on github. I'm looking forward to those :) Regards, Geert ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
[GNC-dev] Introduction, a story, and 50% improvement in XML loading speed
TL;DR: hi! I'm a programmer! The attached patch to one line of code
gives a 50% reduction in XML load CPU use! Skeptical? I was.
*Introduction* (of me)
My name is Chris Carson. I wrote code for a living from 1976-1986, and
then entered management but continued dabbling in code as a hobbyist. C,
C++, Unix, Linux, blah blah.
*A Story*
I have financial data in Quicken dating back to 1991. When Intuit sold off
Quicken I decided I needed a path to another financial management package.
I wrote a processor to deal with the QIF file duplicate transfer problem
(that's another story) and imported my data into Gnucash. The resulting
XML save file is 55.8Mb (uncompressed. Yes, I store it compressed.)
The XML file takes ~38 seconds of user CPU time to load on my build of the
Gnucash 3.3 maint stream. (For reference, starting Gnucash with an empty
simple account file takes ~4.5 seconds of user CPU time on my machine.) I
did a relatively tedious run of callgrind which showed that about half of
that time was being consumed dom_chars_handler(...) and
checked_char_cast(...), both in the libgnucash/backend/xml directory.
Turns out dom_chars_handler(...) is called with an enormous multi-line
string. It copies the whole thing and validates it, nibbles off a few
bytes, and returns, only to be called again with the remainder of the
enormous multi-line string to copy, validate, nibble again.
*The Patch*
I tried a couple of different fixes to this. The patch below copies off
and validates only the bytes being consumed. It brings the user CPU to
startup and load my XML file from ~38 seconds to ~20.5 seconds, and given
that 4.5 seconds of that is startup I make that about a 50% improvement in
load speed. I tried a more aggressive fix for funsies and it wasn't much
better.
I have tested this *ONLY* on the load of my largeish XML file. But the
patched code reads well.
What would you guys advise as next steps?
Patch included below signature, and separately as a file.
Regards,
Chris Carson
=
>From b4e1911f774bfc292e97cffd2492a0257d0aee3c Mon Sep 17 00:00:00 2001
From: "Christopher D. Carson"
Date: Sun, 23 Dec 2018 20:48:02 -0600
Subject: [PATCH] Performance fix in dom_chars_handler: use g_strndup instead
of g_strdup
Because the origin string can be extraordinarily long, you get more
benefit from this than you would imagine
---
libgnucash/backend/xml/sixtp-to-dom-parser.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libgnucash/backend/xml/sixtp-to-dom-parser.cpp
b/libgnucash/backend/xml/sixtp-to-dom-parser.cpp
index e6ba43039..9aba0801a 100644
--- a/libgnucash/backend/xml/sixtp-to-dom-parser.cpp
+++ b/libgnucash/backend/xml/sixtp-to-dom-parser.cpp
@@ -95,7 +95,7 @@ static gboolean dom_chars_handler (
{
if (length > 0)
{
-gchar* newtext = g_strdup (text);
+gchar* newtext = g_strndup (text,length);
xmlNodeAddContentLen ((xmlNodePtr)parent_data,
checked_char_cast (newtext), length);
g_free (newtext);
--
2.19.2
From b4e1911f774bfc292e97cffd2492a0257d0aee3c Mon Sep 17 00:00:00 2001
From: "Christopher D. Carson"
Date: Sun, 23 Dec 2018 20:48:02 -0600
Subject: [PATCH] Performance fix in dom_chars_handler: use g_strndup instead
of g_strdup
Because the origin string can be extraordinarly long, you get more
benefit from this than you would imagine
---
libgnucash/backend/xml/sixtp-to-dom-parser.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libgnucash/backend/xml/sixtp-to-dom-parser.cpp b/libgnucash/backend/xml/sixtp-to-dom-parser.cpp
index e6ba43039..9aba0801a 100644
--- a/libgnucash/backend/xml/sixtp-to-dom-parser.cpp
+++ b/libgnucash/backend/xml/sixtp-to-dom-parser.cpp
@@ -95,7 +95,7 @@ static gboolean dom_chars_handler (
{
if (length > 0)
{
-gchar* newtext = g_strdup (text);
+gchar* newtext = g_strndup (text,length);
xmlNodeAddContentLen ((xmlNodePtr)parent_data,
checked_char_cast (newtext), length);
g_free (newtext);
--
2.19.2
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: FYI internet speed tests.
Interesting numbers... but I suppose not related to GnuCash ;) ? Geert On Wednesday 23 October 2013 12:14:47 Mike Evans wrote: > Speeds are in Kilobytes so x 8 for Kilobits > > > 2013-10-21 13:16:49 (388 KB/s) > 2013-10-21 13:35:38 (798 KB/s) > 2013-10-21 14:03:15 (783 KB/s) > 2013-10-21 16:03:16 (756 KB/s) > 2013-10-21 18:03:16 (790 KB/s) > 2013-10-21 20:03:40 (291 KB/s) > 2013-10-21 22:03:22 (536 KB/s) > 2013-10-22 00:03:16 (786 KB/s) > 2013-10-22 02:03:15 (797 KB/s) > 2013-10-22 04:03:15 (796 KB/s) > 2013-10-22 06:03:16 (797 KB/s) > 2013-10-22 08:03:16 (791 KB/s) > 2013-10-22 10:03:16 (798 KB/s) > 2013-10-22 12:03:15 (795 KB/s) > 2013-10-22 14:03:16 (795 KB/s) > 2013-10-22 16:03:15 (789 KB/s) > 2013-10-22 18:03:15 (792 KB/s) > 2013-10-22 20:03:20 (608 KB/s) > 2013-10-22 22:03:17 (730 KB/s) > 2013-10-23 00:03:16 (797 KB/s) > 2013-10-23 04:03:15 (798 KB/s) > 2013-10-23 06:03:16 (790 KB/s) > 2013-10-23 08:03:15 (798 KB/s) > 2013-10-23 10:03:16 (751 KB/s) > 2013-10-23 12:03:16 (792 KB/s) > > > Slows down at about 8pm it seems. > > Mike E (neighbour) ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: FYI internet speed tests.
On Wed, 23 Oct 2013 12:14:47 +0100 Mike Evans wrote: > Speeds are in Kilobytes so x 8 for Kilobits > > > 2013-10-21 13:16:49 (388 KB/s) > 2013-10-21 13:35:38 (798 KB/s) > 2013-10-21 14:03:15 (783 KB/s) > 2013-10-21 16:03:16 (756 KB/s) > 2013-10-21 18:03:16 (790 KB/s) > 2013-10-21 20:03:40 (291 KB/s) > 2013-10-21 22:03:22 (536 KB/s) > 2013-10-22 00:03:16 (786 KB/s) > 2013-10-22 02:03:15 (797 KB/s) > 2013-10-22 04:03:15 (796 KB/s) > 2013-10-22 06:03:16 (797 KB/s) > 2013-10-22 08:03:16 (791 KB/s) > 2013-10-22 10:03:16 (798 KB/s) > 2013-10-22 12:03:15 (795 KB/s) > 2013-10-22 14:03:16 (795 KB/s) > 2013-10-22 16:03:15 (789 KB/s) > 2013-10-22 18:03:15 (792 KB/s) > 2013-10-22 20:03:20 (608 KB/s) > 2013-10-22 22:03:17 (730 KB/s) > 2013-10-23 00:03:16 (797 KB/s) > 2013-10-23 04:03:15 (798 KB/s) > 2013-10-23 06:03:16 (790 KB/s) > 2013-10-23 08:03:15 (798 KB/s) > 2013-10-23 10:03:16 (751 KB/s) > 2013-10-23 12:03:16 (792 KB/s) > > > Slows down at about 8pm it seems. > Oops, wrong list! Still, someone might be interested. :) -- Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All. ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
FYI internet speed tests.
Speeds are in Kilobytes so x 8 for Kilobits 2013-10-21 13:16:49 (388 KB/s) 2013-10-21 13:35:38 (798 KB/s) 2013-10-21 14:03:15 (783 KB/s) 2013-10-21 16:03:16 (756 KB/s) 2013-10-21 18:03:16 (790 KB/s) 2013-10-21 20:03:40 (291 KB/s) 2013-10-21 22:03:22 (536 KB/s) 2013-10-22 00:03:16 (786 KB/s) 2013-10-22 02:03:15 (797 KB/s) 2013-10-22 04:03:15 (796 KB/s) 2013-10-22 06:03:16 (797 KB/s) 2013-10-22 08:03:16 (791 KB/s) 2013-10-22 10:03:16 (798 KB/s) 2013-10-22 12:03:15 (795 KB/s) 2013-10-22 14:03:16 (795 KB/s) 2013-10-22 16:03:15 (789 KB/s) 2013-10-22 18:03:15 (792 KB/s) 2013-10-22 20:03:20 (608 KB/s) 2013-10-22 22:03:17 (730 KB/s) 2013-10-23 00:03:16 (797 KB/s) 2013-10-23 04:03:15 (798 KB/s) 2013-10-23 06:03:16 (790 KB/s) 2013-10-23 08:03:15 (798 KB/s) 2013-10-23 10:03:16 (751 KB/s) 2013-10-23 12:03:16 (792 KB/s) Slows down at about 8pm it seems. Mike E (neighbour) -- Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All. ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Report Engine Speed
On 2/19/2010 12:06 PM, Christian Stimming wrote: [...] what I can recall is this: All the "number collecting calculations" in all of the reports are horribly inefficient, or more academic: coded in the wrong level of abstraction. That is, the calculation of the balance difference between start and end of a reporting period is usually done by obtaining the list of splits (transactions) in question from within the scheme code, then summing over all of those splits. Also, the intermediate results are not cached at all, even during one report generation run, so they are calculated over and over again when the report is generated. This turns your O(n) problem into an O(n^2) problem. Thanks for the insight! It was my first instinct as well was that there was something inefficient in the way that the data behind the report was being collected. Interestingly, crude timing information suggests that the bulk of the time is in html-document-render. The renderer in budget.scm (2.2.9, at least) is returning in 3.8 wall-clock seconds, but html-document-render is taking 13.4 seconds. A quick look at html-object-renderer shows it being called 42,019 times in the roughly 12 seconds it takes, with the string-append that follows consuming a little over a second. Seems as though some structural changes in how the HTML is being generated may have some very meaningful improvements! I'll probably end up taking a stab at an eguile version this weekend. Jeff ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Report Engine Speed
Am Freitag, 19. Februar 2010 schrieb Jeff Kletsky: > I found the "optimizing reports" thread from October 2007, but not a lot > since then. Has there been any significant understanding gained since > then about what is so slow on what should be a relatively > straightforward task? Was it ever narrowed down to bad algorithms in the > reporting, or something else? I've created some of the oldest reports in there. I have never done real profiling of the time spent in the report generation - but what I can recall is this: All the "number collecting calculations" in all of the reports are horribly inefficient, or more academic: coded in the wrong level of abstraction. That is, the calculation of the balance difference between start and end of a reporting period is usually done by obtaining the list of splits (transactions) in question from within the scheme code, then summing over all of those splits. Also, the intermediate results are not cached at all, even during one report generation run, so they are calculated over and over again when the report is generated. This turns your O(n) problem into an O(n^2) problem. As I said, the problem is the level of abstaction: Most of the report code uses too low-level accessor functions into the txn data, like "querying for a list of splits", then "accumulating the sum over these splits". It would go a long way to choose more higher-level query functions here, like "querying for a balance difference for splits obeying criterion xy". The "engine" and/or the "libqof" part of gnucash in theory should provide sufficient such higher-level functions (implemented in C); it's unclear if they actually do. However, the problem doesn't stop here: Even if those few higher-level query functions were being used from the schemed side of it, it would turn out their implementation is probably highly inefficient as well, like O(n^2) instead of something less than that. But as long as nobody really used these function for time-critical applications (or cared about the time), nobody had any incentive to work on the optimization here and its functional correctness was enough as a goal. The optimization of the report generation obviously needs work on several levels. I would recommend the following: - Use profiling to identify the 10% part of the report generation that consumes most of the time. Using the time-of-day is probably good enough to identify this section. I would guess the problem is in the calculation of each single amount that is displayed in the report, and most probably the amount calculation is programmed in scheme for the most part and using the "engine" query functions only on a very low level. - Identify some higher-level query function that can fulfil the amount generation task, but refactors its calculation away from the reporting code. This higher-level query function can first be a scheme function. Once the functional correctness is somewhat verified, you can try to port this to C and/or to replace this by use of the libqof query object... Good luck! Regards, Christian ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Report Engine Speed
Always good to know what has gone before and that I'm biting off a "challenging" problem. I'll probably start with a liberal set debug statements with gettimeofday to see if I can tell if it's in the collection of the data, or the rendering of the data into HTML once all collected. Including eguile as an option is a significant and welcome improvement, in my opinion. While not what I'm used to in web development, it certainly makes things a lot easier! -jeff On 2/19/2010 7:36 AM, Derek Atkins wrote: I don't think we've done any profiling of reports to see where it's spending its time. Of course, profiling the C and Scheme separately might be... Challenging. ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Report Engine Speed
Jeff Kletsky writes: > Re-reading, I wanted to make sure that it was understood that my > intent is to try to fix some of this in a hopefully generally useful > way as I work through the reports and output I need... I suspect you'll need to invoke more 6.031. :) I don't think we've done any profiling of reports to see where it's spending its time. Of course, profiling the C and Scheme separately might be... Challenging. > Jeff -derek -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH warl...@mit.eduPGP key available ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Report Engine Speed
While we like the professional features found in GNUCash as an accounting system, we're at wits end with the reporting system. Moving to a modern HTML rendering engine is great step in the right direction. However, even in 2.3.10, the time to build a simple budget report off two-months' of personal data is 15-25 seconds on a dual-core Atom 330 running Ubuntu, which seems obscenely long. Its not noticably better on a 1.83 GHz Core Duo MacMini. That just isn't going to work for us. I found the "optimizing reports" thread from October 2007, but not a lot since then. Has there been any significant understanding gained since then about what is so slow on what should be a relatively straightforward task? Was it ever narrowed down to bad algorithms in the reporting, or something else? Before I induce any more 6.031 flashbacks, is Scheme just too much of a dog to get the task done? Thanks, Jeff ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Report Engine Speed
Re-reading, I wanted to make sure that it was understood that my intent is to try to fix some of this in a hopefully generally useful way as I work through the reports and output I need... Jeff ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
speed up file loading with many prices
Hi! On 2004-10-31 Phil Longstaff's patch to prevent duplicate pricedb entries was merged. I like this functionality but it has one drawback: Every time a new price is inserted this price is compared to all allready existing prices. This slows down file loading if there are many prices in the file. E.g. in my current datafile are about 15000 prices and it takes about 40 seconds to load. If the duplicate check is disabled the loading takes less than 2 seconds. I added a parameter check_dupl to gnc_price_list_insert and in turn to gnc_pricedb_add_price. Then check_dupl is set to FALSE when loading files and to TRUE when adding prices manually or online. Modified files: src/engine/gnc-pricedb.c src/engine/gnc-pricedb.h src/engine/gw-engine-spec.scm src/engine/test-core/test-engine-stuff.c src/backend/file/gnc-pricedb-xml-v2.c src/backend/file/io-gncbin-r.c src/backend/file/io-gncxml-v1.c src/gnome-utils/dialog-transfer.c src/gnome/dialog-price-editor.c src/gnome/druid-stock-split.c src/scm/price-quotes.scm makepatch patches for head and stable branches are attached. Herbert. gnucash-20050602-205056-herbie.head.diff.gz Description: GNU Zip compressed data gnucash-20050602-205724-herbie.stable.diff.gz Description: GNU Zip compressed data ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: speed?
Hendrik Boom <[EMAIL PROTECTED]> writes: > The account I am editing has something like 7000 or so transactions. > Could this be relevant? Yes. One way you can probably get substantial improvements in your everyday work is to set the register to only show the past 200 transactions or so. > - because gtk+ imposes a scrolling area limit af 32K pixels, > you have too handle scrollong at a higher level in the protocol. > - the higher layer is written in guile, which is interpreted (like Java) > - so scrolling slows down quite a bit. As Bill said, no guile in there. > - deleteing a transaction involves recalculating sizes for the > entire scroling area, and this also is done in guile, is > interpreted, and so each deletion might end up taking the same > order of magnitude of time as some of the analysis activities > during importing. Nope. > - Events are queued, and not subsumed. So if you press the up cursor > three times in a row, you end up displaying the register in three > separate positions, instead of counting up-cursor presses and moving > the register three lines all at once. This may actually be part of the problem. ISTR that we've seen weird behaviors with the gnome canvas and exposures (which are related to scrolling) before. In those cases, I think it was acting as if it wasn't doing any batching as you've described. > P.S. For what it's worth, my processor is a Pentium running at 39.73 > BogoMIPS; 48 meg of RAM. I'm running SuSE Linux 6.3. That's a nice test platform for slower systems. I'd like to see us running well on systems near that speed if it doesn't force too many compromises in the design. I suppose you can be our guinea pig :> -- Rob Browning <[EMAIL PROTECTED]> PGP=E80E0D04F521A094 532B97F5D64E3930 -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
Re: speed?
> > > > > > I think the bottleneck is in the Gnome Canvas, which is the basis of > > > the register display. Dave P would know more about that. > > > > I suspect this is where it's happening, but we will need to do some > > profiling to be sure. > > Is each entry field in the register a separate Gnome widget? No, there is one widget which gets moved around as needed. > Do you suppose that the canvas does its complete set of layout calculations > over again when a transaction is deleted? Or entered? Or changed? > Would it be useful if I looked into the Gnome source code about this? I'm not too familiar with the details of the canvas's operation and its use in GnuCash; Heath Martin will know more. Source code inspection is always useful, I think :) You will also want to look in src/register and src/register/gnome. > Do you happen to know what profiling tools I should look for under > Linux? Every OS I've used has a different way of doing this, and the best > one is usually discovered by word-of-mouth, not by looking in manuals. The only one I know of under linux is gprof, which has a nice set of info pages. To use it, you will need to compile gnucash with profiling enabled. You can do that by running configure with the --enable-profile argument. You will need to run make clean so that all object files get recreated. > By the way, is there a maximum allowable size to the memo field, cheque > number field, etc? (I don't care if there are current limits on how much > is displayed (these are at worst temporary(and they resize with the window > (Aha! Four nested parentheses in an English prose. Natural language > catches up to Lisp))), as long as all the data get > into the file.) The length of the string must fit in an integer, but other than that there are no limits. dave -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
Re: speed?
> On Fri, 2 Jun 2000 22:37:30 -0400 (EDT), Hendrik Boom > <[EMAIL PROTECTED]> said: Hendrik> By the way, is there a maximum allowable size to the memo Hendrik> field, cheque number field, etc? (I don't care if there are Hendrik> current limits on how much is displayed (these are at worst Hendrik> temporary(and they resize with the window (Aha! Four nested Hendrik> parentheses in an English prose. Natural language catches up Hendrik> to Lisp))), as long as all the data get into the file.) I hope that you use emacs to edit that paragraph, so you knew hw to close the parens. rob -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
Re: speed?
> > > > I think the bottleneck is in the Gnome Canvas, which is the basis of > > the register display. Dave P would know more about that. > > I suspect this is where it's happening, but we will need to do some > profiling to be sure. Is each entry field in the register a separate Gnome widget? Do you suppose that the canvas does its complete set of layout calculations over again when a transaction is deleted? Or entered? Or changed? Would it be useful if I looked into the Gnome source code about this? Do you happen to know what profiling tools I should look for under Linux? Every OS I've used has a different way of doing this, and the best one is usually discovered by word-of-mouth, not by looking in manuals. By the way, is there a maximum allowable size to the memo field, cheque number field, etc? (I don't care if there are current limits on how much is displayed (these are at worst temporary(and they resize with the window (Aha! Four nested parentheses in an English prose. Natural language catches up to Lisp))), as long as all the data get into the file.) -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
Re: speed?
> > Another topic: what sorts of duplicate transactions are you finding > after QIF import? IIRC you have mentioned two classes of problem: one > is the Opening Balance in non-first-position and one is Quicken > collapsing two splits into one. Are there other types of problem > transactions you have identified? No. The ones I have identified in the last few days are the only ones I have encountered. > > > My guesses (using psychic debugging, without reference to source code) > > is: > > ... > > These two are not right. There's no Guile code involved in the > register display at all, that I know of, and certainly not in the > actual display-construction. Well, my reputation as a psychic has always been dismal. ;-) > > > P.S. For what it's worth, my processor is a Pentium running at 39.73 > > BogoMIPS; 48 meg of RAM. I'm running SuSE Linux 6.3. > > Well, that's hardware that would kindly be called "pathetic" these > days :), but I think gnucash should be able to run fine on it. If it > doesn't, that's something we need to work on. So far, it's fine for everything I really need except video games. We have a nearly-dedicated Windows 95 machine for that. It's not supposed to have any sensitive or critical data on it; hence the desire to move from Quicken there to gnucash here. > > Bill Gribble > -- hendrik. -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
Re: speed?
> Hendrik Boom <[EMAIL PROTECTED]> writes: > > Is there a speed problem, or am I doing something wrong.? > > For large numbers of transactions, the current register code doesn't > exactly set speed records. I only have 2000 or so transactions in my > largest accounts, and the speed could be better, even though I'm on a > fast machine. I agree that it's something that needs to be addressed. > > I think the bottleneck is in the Gnome Canvas, which is the basis of > the register display. Dave P would know more about that. I suspect this is where it's happening, but we will need to do some profiling to be sure. dave -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
Re: speed?
Hendrik Boom <[EMAIL PROTECTED]> writes: > Is there a speed problem, or am I doing something wrong.? For large numbers of transactions, the current register code doesn't exactly set speed records. I only have 2000 or so transactions in my largest accounts, and the speed could be better, even though I'm on a fast machine. I agree that it's something that needs to be addressed. I think the bottleneck is in the Gnome Canvas, which is the basis of the register display. Dave P would know more about that. > I just imported *all* my Quicken accounts, and then started tracking > down duplicated entries. Another topic: what sorts of duplicate transactions are you finding after QIF import? IIRC you have mentioned two classes of problem: one is the Opening Balance in non-first-position and one is Quicken collapsing two splits into one. Are there other types of problem transactions you have identified? > My guesses (using psychic debugging, without reference to source code) > is: > (a) > - because gtk+ imposes a scrolling area limit af 32K pixels, > you have too handle scrollong at a higher level in the protocol. > - the higher layer is written in guile, which is interpreted (like Java) > - so scrolling slows down quite a bit. > (b) > - deleteing a transaction involves recalculating sizes for the entire > scroling area, and this also is done in guile, is interpreted, and > so each deletion might end up taking the same order of magnitude > of time as some of the analysis activities during importing. These two are not right. There's no Guile code involved in the register display at all, that I know of, and certainly not in the actual display-construction. > P.S. For what it's worth, my processor is a Pentium running at 39.73 > BogoMIPS; 48 meg of RAM. I'm running SuSE Linux 6.3. Well, that's hardware that would kindly be called "pathetic" these days :), but I think gnucash should be able to run fine on it. If it doesn't, that's something we need to work on. Bill Gribble -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
speed?
I'm slowly gearing up to use GnuCash on live data, and am attempting to start parallel operation with Quicken before cutting over comletely. As a result I am continuing to find problems, some of which are relatively easy to fix. But some, I suspect are not. Is there a speed problem, or am I doing something wrong.? I just imported *all* my Quicken accounts, and then started tracking down duplicated entries. Once I find one, it takes 45 seconds to delete it, starting from the moment I confirm the deletion to the moment the register has its new contents. This is the performance I expect from Java. I also notice that scrolling is quite slow. Also, if I happen to move the delete-confirmation window by grabbing it by its title bar, the window leaves a trail of ghosts, which slowly disappear at the rate of about one per second. The account I am editing has something like 7000 or so transactions. Could this be relevant? My guesses (using psychic debugging, without reference to source code) is: (a) - because gtk+ imposes a scrolling area limit af 32K pixels, you have too handle scrollong at a higher level in the protocol. - the higher layer is written in guile, which is interpreted (like Java) - so scrolling slows down quite a bit. (b) - deleteing a transaction involves recalculating sizes for the entire scroling area, and this also is done in guile, is interpreted, and so each deletion might end up taking the same order of magnitude of time as some of the analysis activities during importing. (c) - Events are queued, and not subsumed. So if you press the up cursor three times in a row, you end up displaying the register in three separate positions, instead of counting up-cursor presses and moving the register three lines all at once. I don't know if these analyses are correct, or whether the problems are in gnome/gtk+ or in the guile code for gnucash. In any case, this is probably not something to try and fix before the 1.4 code freeze. And perhaps the proper fix would be in gnome, not gnucash. Any comments? -- hendrik. P.S. For what it's worth, my processor is a Pentium running at 39.73 BogoMIPS; 48 meg of RAM. I'm running SuSE Linux 6.3. -- Gnucash Developer's List To unsubscribe send empty email to: [EMAIL PROTECTED]
