Re: testing the book zeroing code (maybe for backport?)
On Feb 6, 2008, at 9:16 PM, David Reiser wrote: > Using Tim's approach, I have: > 642 accounts > 6423 transactions > and it took 4 minutes, 6 seconds to close the books. 1minute 46 seconds for a trial balance. Dave -- David Reiser [EMAIL PROTECTED] ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: GDA: PostgreSQL empty slots table
Mark Johnson wrote:
> The slots table includes an id field which is "auto_increment".
> PostgreSQL does not implement that keyword. Instead, it appears to
> accept it, but ignore it when creating the table. (This may actually be
> libgda's PostgreSQL provider doing that.) Gnucash-gda relies upon that
> field auto-incrementing when inserts are done to the slots table. The
> result is similar to this for every insert to the slots table:
> ERROR: null value in column "slot_id" violates not-null constraint
> STATEMENT: INSERT INTO slots (obj_guid, name, slot_type, int64_val,
> string_val, double_val, timespec_val, guid_val, numeric_val_num,
> numeric_val_denom) VALUES ('77889f8da5fb434ae68891da19bef5ad',
> 'reconcile-info/last-date', 1, 1199170799, NULL, NULL, '1969-12-31',
> NULL, 0, 1)
>
> When saving to the gda backend, both MySql and sqlite have many records
> in the slots table. PostgreSQL ends up with none.
>
GDA provides 1) numerous backends and 2) an object model to build
queries and get results. From what I have seen so far, (1) is more
mature than (2). For example, some of the queries you wrote in your
other e-mail (SELECT * from t1 where t1.guid in (SELECT * from ...))
cannot be represented in the query model. However, I should be able to
create a string and execute it.
It looks as though the postgresql backend has a bug. Using the object
model, I create an XML-based representation of the table. I create the
'slot_id' column with type integer and add the 'AUTOINC' flag to it.
For mysql and sqlite, this is an INTEGER AUTO_INCREMENT column, while
for postgresql, it should be a SERIAL column. I don't really want to
special-case the code for various backends since libgda is supposed to
remove that need. There are other cases where it might be required.
SQLite, for example, does not allow more than one row to be added in an
INSERT statement, whereas mysql and postgresql do. I could speed up
initial save to a blank db by writing all accounts at once, for example,
but sqlite couldn't handle it and would need them to be written one at a
time. Note that the libgda object model does not allow more than 1 row
to be written either, so that would need to be done with SQL directly.
Mark, I am happy if you want to design any of the SQL queries to improve
them.
Phil
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: testing the book zeroing code (maybe for backport?) <>
On Thursday 07 February 2008 04:37:35 pm Derek Atkins wrote: > Mike or Penny Novack <[EMAIL PROTECTED]> writes: > > of each account as of some specified date). You could ask your testers > > to give you the time one of these things took. No point looking for > > what might be wrong with your "closing" until you KNOW something is > > wrong (takes MUCH longer than one of these). > > True.. > ~26 seconds for a Trial Balance: Reports-Income and Expense-Trial Balance Tim -- Fedora Core release 6 (Zod), Linux 2.6.22.14-72.fc6 KDE: 3.5.8-1 Fedora 16:50:01 up 11 days, 56 min, 3 users, load average: 0.16, 0.20, 0.21 "It's what you learn after you know it all that counts" John Wooden signature.asc Description: This is a digitally signed message part. ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: testing the book zeroing code (maybe for backport?) <>
Mike or Penny Novack <[EMAIL PROTECTED]> writes: >>Wow! I didn't think my algorithms were THAT bad! Okay, I'll >>see about adding a progress dialog.. Unless someone else >>can get to it first. I probably wont have time for a bit. >> >>-derek >> >> >> > Derek, > >Would you want me to take a look? Lot's of experience "tuning" > including redesigning of unwise choices for algorithms. Sure! Go for it! >Might not be your fault at all. I haven't yet looked at how GnuCash > does anything and the question here might be just how the data for an > account is stored. It shouldn't take MUCH longer to generate the two > closing transactions than it does to calculate the balance of each > income and expense account. But if GnuCash does not store the balances > (recalculated each time by examining every transaction in each such > account) then your time might not be excessive. Might be quite > reasonable. Well, it's possible that GetBalanceAsOfDate goes forward in time instead of backwards in time, which means that the more transactions you have the longer it takes. >There are exisiting things which you could use for comparison. For > example, the time required to produce an Income/Exense report or a > trial balance or ... (anything that requires obtaining the balance > of each account as of some specified date). You could ask your testers > to give you the time one of these things took. No point looking for > what might be wrong with your "closing" until you KNOW something is > wrong (takes MUCH longer than one of these). True.. > Michael -derek -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH [EMAIL PROTECTED]PGP key available ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Plot details - barchart reports
Thanks for your indications. I will start to work on it and report as soon as I have something ready, or (more likely) additional doubts. As I said, I will first have to learn A LOT, so don't expect fast patches... Thank you Davide On Feb 7, 2008 6:08 PM, Josh Sled <[EMAIL PROTECTED]> wrote: > "Davide Imbeni" <[EMAIL PROTECTED]> writes: > > I would like to change slightly the appearance of the plots in bar-chart > > reports. > > As it is now it's a bit difficult to get an idea of the level of the > bars (a > > part from the first one on the left), so I would like to add a grid to > the > > plot or, even better, have the actual numeric values visible (e.g. when > > clicking on the corresponding bar, or when pointing it without clicking, > or > > even as a label close to it) > > These things would be provided by modifying how we call libgoffice, which > provides the actual plotting. The primary binding code between gnucash > and > goffice is in src/gnome-utils/gnc-html-graph-gog.c. > > The interface between the reports and that code is through a couple of > levels… > > The report (scm code) will emit an tag with tags for the > various parameters. See src/report/report-system/html-barchart.scm. > > GtkHtml is configured for tags (with a particular class-id) to be > handled by the aforementioned gnc-html-graph-gog code. It then parses the > params/data, and converts it into goffice's interface, and renders the > report > into a pixmap. The pixmap is then returned to GtkHtml as the rendering of > the tag. > > > I don't know anything about scheme, guile, and extremely little about > html, > > and before continuing my research, I would like to get directions and > > suggestions from the experts. > > > > Where to start from? Do I have to dig into scheme? Shall I create a > custom > > report or change something in the stylesheet? > > - Make sure you know what's supported by goffice, and the details of what > it > needs (for, say, labels on chart segments). > - Modify the pie/bar-chart generator code for those new parameters. > - Modify the gnc-html-graph-gog code for those new parameters. > - Modify the reports to use the new pie/bar-chart generator code. > > > We're here and happy to help if you have any questions. > > -- > ...jsled > http://asynchronous.org/ - a=jsled; b=asynchronous.org; echo [EMAIL PROTECTED] > ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Plot details - barchart reports
"Davide Imbeni" <[EMAIL PROTECTED]> writes: > I would like to change slightly the appearance of the plots in bar-chart > reports. > As it is now it's a bit difficult to get an idea of the level of the bars (a > part from the first one on the left), so I would like to add a grid to the > plot or, even better, have the actual numeric values visible (e.g. when > clicking on the corresponding bar, or when pointing it without clicking, or > even as a label close to it) These things would be provided by modifying how we call libgoffice, which provides the actual plotting. The primary binding code between gnucash and goffice is in src/gnome-utils/gnc-html-graph-gog.c. The interface between the reports and that code is through a couple of levels… The report (scm code) will emit an tag with tags for the various parameters. See src/report/report-system/html-barchart.scm. GtkHtml is configured for tags (with a particular class-id) to be handled by the aforementioned gnc-html-graph-gog code. It then parses the params/data, and converts it into goffice's interface, and renders the report into a pixmap. The pixmap is then returned to GtkHtml as the rendering of the tag. > I don't know anything about scheme, guile, and extremely little about html, > and before continuing my research, I would like to get directions and > suggestions from the experts. > > Where to start from? Do I have to dig into scheme? Shall I create a custom > report or change something in the stylesheet? - Make sure you know what's supported by goffice, and the details of what it needs (for, say, labels on chart segments). - Modify the pie/bar-chart generator code for those new parameters. - Modify the gnc-html-graph-gog code for those new parameters. - Modify the reports to use the new pie/bar-chart generator code. We're here and happy to help if you have any questions. -- ...jsled http://asynchronous.org/ - a=jsled; b=asynchronous.org; echo [EMAIL PROTECTED] pgpwUi1k9isHg.pgp Description: PGP signature ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: Plot details - barchart reports
Hi, "Davide Imbeni" <[EMAIL PROTECTED]> writes: > Hi, > > I would like to change slightly the appearance of the plots in bar-chart > reports. > As it is now it's a bit difficult to get an idea of the level of the bars (a > part from the first one on the left), so I would like to add a grid to the > plot or, even better, have the actual numeric values visible (e.g. when > clicking on the corresponding bar, or when pointing it without clicking, or > even as a label close to it) Sounds like a good idea. It's been requested a few times. > I have had a look at the wiki page on customizing reports ( > http://wiki.gnucash.org/wiki/Custom_Reports), and at some additional > resources like the stylesheet scm files and the file > /src/report/report-system/doc/report-html.txt. > > I don't know anything about scheme, guile, and extremely little about html, > and before continuing my research, I would like to get directions and > suggestions from the experts. > > Where to start from? Do I have to dig into scheme? Shall I create a custom > report or change something in the stylesheet? I'm afraid that yes, you'll have to dig into scheme. Most likely you'll need to make a custom report or, better yet, add an option to the existing report and feed us back the patch so we can include it in future versions. > Thanks for your time. > > Davide -derek -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH [EMAIL PROTECTED]PGP key available ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: testing the book zeroing code (maybe for backport?)
Wow! I didn't think my algorithms were THAT bad! Okay, I'll see about adding a progress dialog.. Unless someone else can get to it first. I probably wont have time for a bit. -derek David Reiser <[EMAIL PROTECTED]> writes: > Using Tim's approach, I have: > 642 accounts > 6423 transactions > and it took 4 minutes, 6 seconds to close the books. -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH [EMAIL PROTECTED]PGP key available ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Plot details - barchart reports
Hi, I would like to change slightly the appearance of the plots in bar-chart reports. As it is now it's a bit difficult to get an idea of the level of the bars (a part from the first one on the left), so I would like to add a grid to the plot or, even better, have the actual numeric values visible (e.g. when clicking on the corresponding bar, or when pointing it without clicking, or even as a label close to it) I have had a look at the wiki page on customizing reports ( http://wiki.gnucash.org/wiki/Custom_Reports), and at some additional resources like the stylesheet scm files and the file /src/report/report-system/doc/report-html.txt. I don't know anything about scheme, guile, and extremely little about html, and before continuing my research, I would like to get directions and suggestions from the experts. Where to start from? Do I have to dig into scheme? Shall I create a custom report or change something in the stylesheet? Thanks for your time. Davide ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: mingwm10.dll
Hi Stephen!
1. Does /C/QT/4.2.3/bin/mingwm10.dll exist on your machine?
2. Are the directories capitalized as you indicate? (eg. "C", and "QT")
Nathan
On Feb 6, 2008 11:25 PM, Stephen Grant Brown <[EMAIL PROTECTED]> wrote:
> Hi All,
> I am tryiing to produce a distributable version of gnucash for the Vista
> operating system.
>
> I am running XP in vmaware running under Vista.
>
> I did svn update in Packaging.
> Running msys
> cd to packaging
> reset.sh run to completion
> In custom.sh
>
> QTDIR=/C/QT/4.2.3
>
> In gnucash.iss.in
>
> Source: "@[EMAIL PROTECTED]"; ...
>
> install.sh run to completion
>
> dist.sh produces
>
> mingwm10.dll does not exist
>
> In Packaging/gnucash.iss.in
>
> deleted line with
>
> Source ... mingwm.dll
>
> reset.sh run to completion
>
> install.sh run to completion
>
> dist.sh produces
>
> Error on line 84 in c:\soft\gnucash\gnucash.iss
> ... mingwm10.dll does not exist
>
> cd /soft
>
> grep -r mingwm *
>
> produces
> gnucash/build/packaging/win32/gnucash.iss:Source:
> "@[EMAIL PROTECTED]"; DestDir: "{app}\bin"; Flags: ignoreversion
> gnucash/gnucash.iss:Source: "c:/QT/4.2.3/bin\mingwm10.dll"; DestDir:
> "{app}\bin"; Flags: ignoreversion
> gnucash/repos/packaging/win32/.svn/text-base/gnucash.iss.in.svn-base:Source:
> "@[EMAIL PROTECTED]"; DestDir: "{app}\bin"; Flags: ignoreversion
> gnucash/repos/packaging/win32/gnucash.iss.in:Source:
> "@[EMAIL PROTECTED]"; DestDir: "{app}\bin"; Flags: ignoreversion
> Binary file mingw/bin/mingwm10.dll matches
> Binary file mingw/lib/libmingwthrd.a matches
> msys/home/Stephen/Packaging/.svn/text-base/gnucash.iss.in.svn-base:Source:
> "@[EMAIL PROTECTED]"; DestDir: "{app}\bin"; Flags: ignoreversionWhich
> file produces c:\soft/gnucash\gnucash.iss?Yours Sincerely Stephen Grant
> Brown
> ___
> gnucash-devel mailing list
> gnucash-devel@gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
--
<><><><><><><><><><><><><><><>
"Even if you are on the right track, you'll get run over if you just sit
there" - Will Rogers
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
GDA: PostgreSQL empty slots table
The slots table includes an id field which is "auto_increment".
PostgreSQL does not implement that keyword. Instead, it appears to
accept it, but ignore it when creating the table. (This may actually be
libgda's PostgreSQL provider doing that.) Gnucash-gda relies upon that
field auto-incrementing when inserts are done to the slots table. The
result is similar to this for every insert to the slots table:
ERROR: null value in column "slot_id" violates not-null constraint
STATEMENT: INSERT INTO slots (obj_guid, name, slot_type, int64_val,
string_val, double_val, timespec_val, guid_val, numeric_val_num,
numeric_val_denom) VALUES ('77889f8da5fb434ae68891da19bef5ad',
'reconcile-info/last-date', 1, 1199170799, NULL, NULL, '1969-12-31',
NULL, 0, 1)
When saving to the gda backend, both MySql and sqlite have many records
in the slots table. PostgreSQL ends up with none.
Mark
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
GDA register open queries
Phil,
From the svn change log:
r16924 | plongstaff | 2008-02-03 14:16:06 -0700 (Sun, 03 Feb 2008) | 8 lines
When opening a register, there are now 5 queries: 1) get tx guids for
all splits in the register 2) get all tx'es for those guids 3) get all
slots for those tx'es 4) get all splits for those tx'es 5) get all
slots for those splits. This makes register manipulation much faster.
However, more work may need to be done to ensure that we don't make the
sql line too long by splitting one or more of the queries.
Logging the queries sent to PostgreSQL shows:
Query 1:
SELECT DISTINCT t1.tx_guid FROM splits AS t1 WHERE
t1.account_guid='b0853faaca8fad4e60d2a3f6b0b70261'
(returns greater than 600 rows in my test case)
Query 2:
SELECT * FROM transactions WHERE guid IN
('00587c1825d286a198fbb90685cfb08b','0200d809a230434900026c62e6b30cdc',
... );
Query 3:
SELECT * FROM slots WHERE obj_guid IN
('a05751c70b43be0c0cd03ef4a789981a','b032cddd5bfd3095963b2b1c001fceca',
...);
Query 4:
SELECT * FROM splits WHERE tx_guid IN
('a05751c70b43be0c0cd03ef4a789981a','b032cddd5bfd3095963b2b1c001fceca',
...);
Query 5:
SELECT * FROM slots WHERE obj_guid IN
('0d4942388ef552026d66c861782bab1c','44f52793993b48d68d2435735090caa5',
...);
Phil, what do you think of the following queries?
Query A (combines Queries #1 & #2 above):
(I've tried this query successfully)
SELECT DISTINCT t2.*
FROM splits AS t1, transactions AS t2
WHERE t1.account_guid='b0853faaca8fad4e60d2a3f6b0b70261' AND t1.tx_guid
= t2.guid;
Query B (same as Query #3 above):
( I can't confirm this one due to my slots table being empty. I've
noted the cause of that in other postings.)
SELECT * FROM slots
WHERE obj_guid IN (SELECT DISTINCT t1.tx_guid FROM splits AS t1 WHERE
t1.account_guid='b0853faaca8fad4e60d2a3f6b0b70261');
Query C (same as Query #4 above)
SELECT * FROM splits
WHERE tx_guid IN (SELECT DISTINCT t1.tx_guid FROM splits AS t1 WHERE
t1.account_guid='b0853faaca8fad4e60d2a3f6b0b70261');
or rewritten as:
SELECT t2.*
FROM splits AS t1, splits AS t2
WHERE t1.account_guid='b0853faaca8fad4e60d2a3f6b0b70261' AND t1.tx_guid
= t2.tx_guid;
Query D (same as Query #5 above)
(Again with an empty slots table, I can't really try this one.)
SELECT t3.*
FROM splits AS t1, splits AS t2, slots AS t3
WHERE t1.account_guid='b0853faaca8fad4e60d2a3f6b0b70261' AND
t1.tx_guid = t2.tx_guid AND
t2.guid = t3.obj_guid;
I believe that this would address your concerns regarding the length of
the SQL query strings and save you the trouble of writing code to split
them into several queries. Plus, 5 queries are reduced to 4.
I notice that some joins and/or sub-queries are repeated in my
suggestions. This may provide the possibility of further reducing the
number of queries.
Mark
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
