[tryton-dev] Re: performance of chart of accounts?
Le 24/01/2017 à 11:03, Cédric Krier a écrit : On 2017-01-24 07:52, Richard PALO wrote: Le 23/01/2017 à 21:53, Cédric Krier a écrit : On 2017-01-23 07:16, Richard PALO wrote: Are there any tuning remedies? If not, what would things be like if we were to migrate a company with an average of over 5500 movements per year? Normally it should not depend on the number of moves. For me, the slowness is the number of queries but not the computation of the balance/debit/credit. Those computations should be almost linear (with a very small coefficient). Perhaps this could be split up to run parallel by the top (N) level(s), for example, in French PCG, run parallel queries for each class (1 .. 8) or subclass (1[0-8], 2[0-9] et cetera)... I do not think it will improve. There are very few parts that could be run in parallel like the SQL queries (but only if # accounts > 1000) and the cumulate function (but it should normally iterate only once). Also the machineries to make it parallel will probably cost more then it will benefit, without talking about the complexity it will create. Yeah, I can understand diminishing returns... will just have to avoid using + except at low levels... thanks for the explanations. Also, unfortunately it appears that the 'views' still only update the final balance field and not the debit/credit fields. I thought I already filed an issue including that way back but can't seem to find it. Is it on 'purpose' to exclude displaying the total movements debit/credit as well as, Yes it is on purpose (since 1.0). okay, though this makes it awkward online, will have to deal with this via custom reporting. eventually, the initial balance in the chart of accounts? I guess you are looking for the General Ledger instead of the Chart of Account. (which will load much faster as it is a list) in many respects, especially using filters, GL is indeed somewhat more versatile. Now that I try to compare, is there really any reason not to simply replace chart of accounts by reports->general ledger? No it makes no sense as GL is based on the CoA. CoA? The views would/should probably need to be added back, though, as they seem to be missing in both the display and in the reports. Usually, account without moves are not shown on the GL. View accounts have never any moves. But it could be discussed but probably on tryton@ where accounting guys should be. except I can't seem to post any longer neither directly nor via gmane group interface... As to the threaded performance, I've been meaning to check out using uwsgi, but it's still a bit premature to spend time on that for the moment. As I said, it will not help for this specific issue because it is the client who perform sequential request because of the design of GTK expand all feature and the lazy loading of Tryton. I did mean for other use cases, naturally. cheers -- Richard PALO -- You received this message because you are subscribed to the Google Groups "tryton-dev" group. To view this discussion on the web visit https://groups.google.com/d/msgid/tryton-dev/o67p6j%24cp8%241%40blaine.gmane.org.
Re: [tryton-dev] Re: performance of chart of accounts?
On 2017-01-24 07:52, Richard PALO wrote: > Le 23/01/2017 à 21:53, Cédric Krier a écrit : > > On 2017-01-23 07:16, Richard PALO wrote: > > > Are there any tuning remedies? If not, what would things be like if we > > > were to migrate a company with an average of over 5500 movements per year? > > > > Normally it should not depend on the number of moves. For me, the > > slowness is the number of queries but not the computation of the > > balance/debit/credit. Those computations should be almost linear (with a > > very small coefficient). > > > Perhaps this could be split up to run parallel by the top (N) level(s), > for example, in French PCG, run parallel queries for each class (1 .. 8) > or subclass (1[0-8], 2[0-9] et cetera)... I do not think it will improve. There are very few parts that could be run in parallel like the SQL queries (but only if # accounts > 1000) and the cumulate function (but it should normally iterate only once). Also the machineries to make it parallel will probably cost more then it will benefit, without talking about the complexity it will create. > > > Also, unfortunately it appears that the 'views' still only update the > > > final > > > balance field and not the debit/credit fields. > > > I thought I already filed an issue including that way back but can't seem > > > to find it. > > > Is it on 'purpose' to exclude displaying the total movements debit/credit > > > as well as, > > > > Yes it is on purpose (since 1.0). > > > > > eventually, the initial balance in the chart of accounts? > > > > I guess you are looking for the General Ledger instead of the Chart of > > Account. > > (which will load much faster as it is a list) > > > > Now that I try to compare, is there really any reason not to simply replace > chart of accounts > by reports->general ledger? No it makes no sense as GL is based on the CoA. > The views would/should probably need to be added back, though, as they seem > to be missing in both the display and in the reports. Usually, account without moves are not shown on the GL. View accounts have never any moves. But it could be discussed but probably on tryton@ where accounting guys should be. > As to the threaded performance, I've been meaning to check out using uwsgi, > but > it's still a bit premature to spend time on that for the moment. As I said, it will not help for this specific issue because it is the client who perform sequential request because of the design of GTK expand all feature and the lazy loading of Tryton. -- Cédric Krier - B2CK SPRL Email/Jabber: cedric.kr...@b2ck.com Tel: +32 472 54 46 59 Website: http://www.b2ck.com/ -- You received this message because you are subscribed to the Google Groups "tryton-dev" group. To view this discussion on the web visit https://groups.google.com/d/msgid/tryton-dev/20170124100300.GE68908%40tetsuo.
[tryton-dev] Re: performance of chart of accounts?
Le 23/01/2017 à 21:53, Cédric Krier a écrit :
On 2017-01-23 07:16, Richard PALO wrote:
[2nd resend via gmane... googlegroups seems busted as usual]
Nothing to do with google groups, we configured to not allow anonymous
post, only members can. It is configured on gmane if there was still a
web interface.
I'm not so sure, I have [tried to have, anyway] my two primary email addressses
registered via tryton{,-dev,-fr}+subscr...@googlegroups.com with both confirmed.
typically one works or the other (but not both) and neither seem to work
directly any more for some reason...at least not consistently between the
groups.
This one that finally worked was sent to group:gmane.comp.python.tryton.devel
Previously it seemed to be the opposite, I needed to send directly to
tryton*@googlegroups.com
Perhaps related to previous discussion of GL and perhaps of Issue4028 and co,
I have two bookyears in a db now under tryton 4.2 with PCG French
(neither closed yet) and a total of 103 movements (2/3 from older bookyear)
Opening the chart of accounts for the later year (2016), selecting
the only line 'Plan comptable (French)' and doing a full expand ('+')
takes on the average 4 *minutes* on a quiet system!
Full expand is quiet expensive because of the behavior of GTK.
It triggers an event on each node to open them. The client tries to
optimize as much as possible. We load all sibling children of the first
parent at once to minimize the number of queries. But it is a compromise
between manual expand and automatic expand.
It seems to peg a single core on a 32GB 12*core system (2,3G AMD 6338P)
running PostgreSQL 9.6.1 locally and directly on the socket (not the tcp port).
Usual Python threading model. The embedded server is the default
from Werkzeug which is only threaded. For better performance it is
better to use a pre-forked WSGI server.
But for this case, it will not change anything because each queries are
serialized, they depend on the result of the previous to do the next one
(read children to request the second children level).
Are there any tuning remedies? If not, what would things be like if we
were to migrate a company with an average of over 5500 movements per year?
Normally it should not depend on the number of moves. For me, the
slowness is the number of queries but not the computation of the
balance/debit/credit. Those computations should be almost linear (with a
very small coefficient).
Perhaps this could be split up to run parallel by the top (N) level(s),
for example, in French PCG, run parallel queries for each class (1 .. 8)
or subclass (1[0-8], 2[0-9] et cetera)...
Also, unfortunately it appears that the 'views' still only update the final
balance field and not the debit/credit fields.
I thought I already filed an issue including that way back but can't seem to
find it.
Is it on 'purpose' to exclude displaying the total movements debit/credit as
well as,
Yes it is on purpose (since 1.0).
eventually, the initial balance in the chart of accounts?
I guess you are looking for the General Ledger instead of the Chart of Account.
(which will load much faster as it is a list)
Now that I try to compare, is there really any reason not to simply replace
chart of accounts
by reports->general ledger? The views would/should probably need to be added
back, though, as they seem to be missing in both the display and in the reports.
This is particularly frustrating in the French PCG as the account names are
structured 'in context' (check out classe 2 or 6, for example).
Also, I just noticed too that filters don't seem to do much (on name or code)
in the
'chart of accounts' so 'general ledger' seems to much more flexible here.
As to the threaded performance, I've been meaning to check out using uwsgi, but
it's still a bit premature to spend time on that for the moment.
--
Richard PALO
--
You received this message because you are subscribed to the Google Groups
"tryton-dev" group.
To view this discussion on the web visit
https://groups.google.com/d/msgid/tryton-dev/o66tjl%2453o%241%40blaine.gmane.org.
