You should not be seeing more than a few newPool calls, however, if you are building a tree then this may be affecting it. The reuse stuff was not built for trees, so you may have to debug this because I won't have time to look at new use cases for some time.
I will take out the myriad duplication of function pointers over the new year all being well. Jim > -----Original Message----- > From: Ruslan Zasukhin [mailto:ruslan_zasuk...@valentina-db.com] > Sent: Thursday, November 17, 2011 12:43 PM > To: Jim Idle; antlr-interest@antlr.org > Subject: Re: [antlr-interest] [C] my v3 Parser no reuse() slower 20% > than v2. With reuse() 2GB leaks, oops. > > Hi Jim, > > Thank you for your feedback, and I have update now... > > 1) I was able remove all .text usage in both Parser and TreeParser. > GOOD. > > 2) BAD ... This have save 500MB, > but I still have 1.5GB of allocations in my bench ... > > And now I see (using Apple Instruments) that all this is eaten by > PARSER. > Not by Lexer, and not by TreeParser. > > I just see endless > newPool > newPool > newPool > newPool > newPool > newPool > > I will send you snapshoot off list so you can see that. > > And now there is ZERO my code, which affect this. > Only ANTLR own logic... > > This makes me think, that reuse() do not work as expected. > > As I understand, when we do > parser( reset ) > > It must mark all existed allocations as free in your pool, So next run > should reuse all that. Yes? > > And note, that all my calls to parser, are very similar by size. > This is just > INSERT INTO( f1, f,2 ... f9 ) VALUES ( v1, v2, ... ) > > I.e. Pool really should not grow much after first / second iteration of > loop. But it grows like crazy. > > > I think you have own test app where you did test this ... > May be just increase loop count to million or such To see that RAM on > your computer go away ... > > > I very hope you will be able find issue and show how fix it in sources > of ANTLR 3.4 ... Please? May be some kind of objects is not marked as > free ? > > > ====== > Also interesting fact. > > v3 without reuse 22.4 sec > v3 with reuse and 1.5GB allocation 20.4 sec > > v2 with reuse 19.7 > > > So if we will be able resolve this 1.5GB "leaks", there is yet hope to > be at least not slower of v2 ... > > > ======================= > About your hope that V3 C should be much faster of v2 C++ So far I do > not see this. > > I see in profiles, > parser 36% RAM only > tree parser 24% RAM only > execute of vdb engine 13% insert recs into disk (!!) db > > And when I am starting go deep by parser calls ... I just see that deep > is big > sql -> sql_single -> .... > > And each step down just reduce 0.5-0.8% ... > > This is BODY of each rule of parser ... > > And nothing really to optimize :( > Just a lots of small calls ... NilNodes, LT(), ... > > > ======================= > My vision is that this is Nature of ANTLR ... We get many calls of > parser funcs ... Deep stack ... Although they are light they eat > milliseconds ... > > > And fact that in C you need create structures with huge number of > pointers sometimes, then e.g NULL them, in C++ virtual table of > methods is created once per class, not once per instances ... This fact > can be one of hidden bottleneck IMO. You can workaround this, if also > will extract pointers into single separate structure, so instances will > have just a single pointer. > > > -- > Best regards, > > Ruslan Zasukhin > VP Engineering and New Technology > Paradigma Software, Inc > > Valentina - Joining Worlds of Information http://www.paradigmasoft.com > > [I feel the need: the need for speed] > List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.