On Thu, Aug 27, 2009 at 9:52 PM, Dan Price<[email protected]> wrote: > On Thu 27 Aug 2009 at 09:19PM, Mike Gerdts wrote: >> Time in read_dict_file improved by over 20% (17.9 sec -> 14.2 sec) by >> inlining one tiny function. Overall execution time improved by 13% >> (33.9 sec -> 29.6 sec). >> >> It seems as though there are two things to consider out of this: >> >> 1) It is not just recursive calls that are horribly expensive. >> 2) A file format that did not require parsing on every invocation >> (e.g. some sort of binary format that can just be mmap'd) would be >> very helpful. >> >> I do not believe this is unique to sparc, as I explain in >> http://mail.opensolaris.org/pipermail/pkg-discuss/2009-August/032980.html >> (seach for Athlon), clock speed alone accounts for the bulk of the >> speed differences between sparc and x86 in "pkg list". > > I think the other question to ask is whether there's a way to avoid > a full scan of the file in the first place, perhaps with an additional > table, or something else. This goes to Bart's "ALGORITHMS ALGORITHMS > ALGORITHMS" chant/rant. > > That is to say, the best way to create long lasting performance is > with good algorithms. 20% is a great win, but 2000% is more interesting.
Agreed. I wasn't overly thrilled, as this operation should take less than a second. After all, google can find 172,000,000 references to ls in 0.22 seconds. :) > I would also note that if you aren't running with > > 10050 reading manifests during search should pick an appropriate buffer > size > > It might help a little-- but its effect will depend a lot on how many > search results you are returning. Very little, I think. The file with the 700,000 lines is being read in 128 KB chunks and I am opening only a handful of manifests. > > -dp > > -- > Daniel Price, Solaris Kernel Engineering http://blogs.sun.com/dp > -- Mike Gerdts http://mgerdts.blogspot.com/ _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
