Hey John,
About the compression/decompression, i'm not sure what they were useful for, i simply removed them and it simply gave me a nice performance boost :) Do you think these 2 performance patches would be worth including into the subversion (along with the few other bug fixes i sent today) ? Thanks, ________________________________________ From: K. John Wu [[email protected]] Sent: March-26-12 7:18 PM To: FastBit Users Cc: Dominique Prunier Subject: Re: [FastBit-users] PATCH: perf boost on top of r501 Hi, Dominique, Thanks for the performance patches. The real problem with the decompress call in patternSearch is that as is the call to decompress seems to be repeated with without any limit. It should only be called once. I will examine all other calls to decompress to make sure they are only called once.. John On 3/26/12 9:43 AM, Dominique Prunier wrote: > To go even further, it is also possible to add the attached patch __on > top__ of the previous one: > > > > · don't use bitvector::sloppyCount at all in > category::patternSearch > > · need to AND with the mast when there's no match in > query::doEvaluate > > > > Thanks, > > > > *From:*Dominique Prunier > *Sent:* Monday, March 26, 2012 11:46 AM > *To:* FastBit Users > *Subject:* PATCH: perf boost on top of r501 > > > > Hey John, > > > > I tried the performance of the sloppyCount and it seems quite > convincing. It is only 1% slower than returning constants (1 or 0) and > could potentially speedup trivial cases from time to time i guess. > > > > So i ported my performance patch on top of r501 and replaced my > “return 1 or 0” by some calls to sloppyCount. Here is what the patch > includes: > > · in category::patternSearch, skip the > decompression/compression phase (this one has a __HUGE__ impact on perf) > > · in category::patternSearch, use sloppyCount for estimation > > · in query::doEvaluate, fix returned value when term == null > (was 0, now is row count) > > · in query::doEvaluate, even more usage of > bitvector::sloppyCount instead of bitvector::cnt > > · in query::doEvaluate, removed complex OR evaluation > depending on bitmap sizes (maybe this can be restored in a less costly > way) > > > > So far, it passes all my tests (this is not a very intrusive change). > > Please tell me if this would be candidate for the next stable release. > > > > Thanks, > > > > */Dominique Prunier/**//* > > APG Lead Developper > > Logo-W4N-100dpi > > 4388, rue Saint-Denis > > Bureau 309 > > Montreal (Quebec) H2J 2L1 > > Tel. +1 514-842-6767 x310 > > Fax +1 514-842-3989 > > [email protected] <mailto:[email protected]> > > www.watch4net.com <http://www.watch4net.com/> > > / / > > /This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise private information. If you have > received it in error, please notify the sender immediately and delete > the original. Any other use of this electronic mail by you is prohibited. > > //Ce message est pour le récipiendaire désigné seulement et peut > contenir des informations privilégiées, propriétaires ou autrement > privées. Si vous l'avez reçu par erreur, S.V.P. avisez l'expéditeur > immédiatement et effacez l'original. Toute autre utilisation de ce > courrier électronique par vous est prohibée./// > > > > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
