Hi Jim,
Thank you for your feedback, and I have update now...
1) I was able remove all .text usage in both Parser and TreeParser. GOOD.
2) BAD ... This have save 500MB,
but I still have 1.5GB of allocations in my bench ...
And now I see (using Apple Instruments) that all this is eaten by PARSER.
Not by Lexer, and not by TreeParser.
I just see endless
newPool
newPool
newPool
newPool
newPool
newPool
I will send you snapshoot off list so you can see that.
And now there is ZERO my code, which affect this.
Only ANTLR own logic...
This makes me think, that reuse() do not work as expected.
As I understand, when we do
parser( reset )
It must mark all existed allocations as free in your pool,
So next run should reuse all that. Yes?
And note, that all my calls to parser, are very similar by size.
This is just
INSERT INTO( f1, f,2 ... f9 ) VALUES ( v1, v2, ... )
I.e. Pool really should not grow much after first / second iteration of
loop. But it grows like crazy.
I think you have own test app where you did test this ...
May be just increase loop count to million or such
To see that RAM on your computer go away ...
I very hope you will be able find issue and show how fix it
in sources of ANTLR 3.4 ... Please? May be some kind of objects is not
marked as free ?
======
Also interesting fact.
v3 without reuse 22.4 sec
v3 with reuse and 1.5GB allocation 20.4 sec
v2 with reuse 19.7
So if we will be able resolve this 1.5GB "leaks", there is yet hope to be at
least not slower of v2 ...
=======================
About your hope that V3 C should be much faster of v2 C++
So far I do not see this.
I see in profiles,
parser 36% RAM only
tree parser 24% RAM only
execute of vdb engine 13% insert recs into disk (!!) db
And when I am starting go deep by parser calls ... I just see that deep is
big
sql -> sql_single -> ....
And each step down just reduce 0.5-0.8% ...
This is BODY of each rule of parser ...
And nothing really to optimize :(
Just a lots of small calls ... NilNodes, LT(), ...
=======================
My vision is that this is Nature of ANTLR ... We get many calls of parser
funcs ... Deep stack ... Although they are light they eat milliseconds ...
And fact that in C you need create structures with huge number of pointers
sometimes, then e.g NULL them, in C++ virtual table of methods is created
once per class, not once per instances ... This fact can be one of hidden
bottleneck IMO. You can workaround this, if also will extract pointers into
single separate structure, so instances will have just a single pointer.
--
Best regards,
Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc
Valentina - Joining Worlds of Information
http://www.paradigmasoft.com
[I feel the need: the need for speed]
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
