On Thu, Aug 27, 2009 at 10:07 PM, Nicolas Williams<[email protected]> wrote: > On Thu, Aug 27, 2009 at 09:19:08PM -0500, Mike Gerdts wrote: >> http://mail.opensolaris.org/pipermail/pkg-discuss/2009-August/032980.html >> >> Here's my starting point with profiling "pkg search -l ls": >> ... >> >> 667 for line in self._file_handle: >> 668 res = line.split(" ") >> 669 token = self.__unquote(res[0]) >> 670 offset = int(res[1]) >> 671 self._dict[token] = offset >> >> In other words, each line of code between 668 and 671 takes an average >> of 4.5 seconds to execute. It is parsing this file: > > As I said, LISP^WPython programmers know the value of every expression...
Indeed. Can we throw Java programmers in there too? > >> # wc -l /var/pkg/index/token_byte_offset.v1 >> 691111 /var/pkg/index/token_byte_offset.v1 >> >> Thinking that it could be an inefficient hash algorithm, I split that >> off into a separate function so it could be profiled individually. > > One thing I find odd about IPS is that it has its own indexing scheme, > and it doesn't strike me as a very good idea to home-grow your own when > you can use any number of off-the-shelf ones... > >> Time in read_dict_file improved by over 20% (17.9 sec -> 14.2 sec) by >> inlining one tiny function. Overall execution time improved by 13% >> (33.9 sec -> 29.6 sec). > > Hot loops need to execute fast. This is a hot loop. Reducing the > number of calls to read_dict_file should help even more. There is currently only one call to read_dict_file. If we could get rid of it, then we would get another 50+% boost. Even that would not competitive for this operation. :( >> It seems as though there are two things to consider out of this: >> >> 1) It is not just recursive calls that are horribly expensive. > > Yeah, hot loops matter. > > Nico > -- > -- Mike Gerdts http://mgerdts.blogspot.com/ _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
