On Sun, 2006-04-23 at 17:09 +0200, Dirk Meyer wrote: > Jason Tackaberry wrote: > > But the main benefit here is with a list of extensions that we know are > > special, beacon can do O(1) lookups: > > > > ext = os.path.splitext(self._beacon_data['name'])[1] > > special_exts = kaa.metadata.get_special_exts_for_type(ext) > > listdir_file_map = self._beacon_parent._beacon_listdir(cache=True)[1] > > for ext in special_exts: > > if basename+ext in listdir_file_map: > > mtime += listdir_file_map[basename+ext][3][stat.ST_MTIME] > > > > Maybe you want to avoid splitext() or whatever, but you get the idea. > > This is just rough code. But the above logic reduces the complexity to > > O(n) and when n=2500, that's a difference of 6.25 million iterations. > > Sounds like a good speedup. I will keep that in mind. For now, beacon > works like it is now and speedups are not very high an my todo list. I > first need to finish rom drive support.
Ok, so this logic is checked in now, and it's muuuuccch better (which we fully expected). beacon now no longer churns at 100% for a few seconds when starting the dirname=/usr/bin search. Now it completes in a fraction of a second. Tonight I'll turn my attention to the normalization stuff. Normalization accounts for about 40% of the time it takes to do the dirname query. However, 40% of that 40% is from cPickle, and there's nothing that can be done about that unless we decide to write our own simpler serialization stuff (that supports only the primitives types plus list,tuple,dict). Jason. ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Freevo-devel mailing list Freevo-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-devel
