On Tue, 2005-04-19 at 11:27 +0200, Dirk Meyer wrote: > inotify sounds nice. But it requires a special kernel build. So I > guess it should be optional in Freevo. But is sounds much better than > polling directories for changes.
It's not difficult to build the vfs so that it uses inotify if available, otherwise fall back to regular polling. I plan to add dnotify support too, which is much more widely supported in the kernel. But dnotify has more limitations and its interface is not as nice. So if inotify wasn't available, I'd use dnotify for watching directories currently being viewed in the GUI, but not for the repository-wide monitoring for auto-indexing since that requires monitoring a large number of directories and dnotify doesn't scale well. (dnotify requires keeping a file descriptor open for each directory you want to monitor, whereas with inotify you just talk to /dev/inotify.) > For Freevo I was thinking different. I create metadata and thumbnails > on the fly when it's needed or use the cache helper to create it > before. Well, the beagle-like auto-indexer is optional, of course, so with it disabled that's the behaviour I've implemented too. > What I found interesting is the sqlite db. I played with > sqlite and found using a directory pickle file much faster for normal > operations like "go into a directory". But I still need some sort of > database or index to do stuff like "show all mp3 files from 2001". Yes, pickling is going to be faster, especially when you're talking about adding a lot of data to the database. > I don't like datbases right now, they seem slow. For a huge guide, the > epg database is slower than it should be. And for a mediadb, the index > can be huge. I find sqlite pretty reasonable in performance. Yes, it's often not going to be as fast as pickling. But it also shouldn't be disgustingly slower as long as you're doing it right. For example, are you doing an individual SELECT for each file as you iterate over a directory list? That's going to be slow, of course. As with any app that uses a database, you need to modify your design to accommodate database limitations. In this case that means doing a SELECT on whole directories, rather than individual files. Also are you calling db.commit() after every insert? That'd be horrendously slow. Here are some timing examples. When dealing with 10000 simple records in a files table (4 fields: file_id, dir_id, filename, mtime): 10k INSERTs: 1.48 seconds 1 SELECT on whole directory (WHERE dir_id=1): 0.26 seconds 10k separate SELECTs (WHERE file_id=xxx): 3.71 seconds Pickling 10k tuples in a list with the same contents: Construct list and Dump: 0.32 seconds Load: 0.1 seconds So compare the 1 select (which returns 10k records) to the load pickle case. It's true that it's faster. It's even much faster. But 0.26 seconds overhead for loading a directory of 10k entries isn't horrible. Of course the actual directory load will take longer (closer to 0.9 seconds when you do os.listdir(), stat each file, query the database, and sort the records), but it's still not unreasonable. And besides, who keeps 10k files in one directory? :) In order to keep the UI responsive, things are done asynchronously where possible. > inotify works with select, so why is there a problem with directory > monitoring? File indexing may take some time, but in Freevo creating > metadata and thumbnail is very fast for one file. It's only slow if > you enter a new directory with too much files. For inotify, yes, it's clearly not worth doing that in another process. But polling large directories is another matter. The main thing is indexing. Thumbnailing even a very large image is reasonably quick. But even if it takes 0.1 seconds, that's too long. If you do a directory of large images and try to scroll the list while it's generating thumbnails for it, it will be sluggish. My design requirements are very strict in this respect. I demand a very smooth experience. Anything that's sluggish needs to go somewhere else. :) And clearly, thumbnailing a video is a worst-case scenario where it may take several seconds. So rather than having special cases (thumbnailing an image is ok to do in-process, but video we fork or use a thread or whatever), just do all indexing the same way: out of the GUI process. So the reason to put directory monitoring in another process is just to keep the design simple. If indexing needs to be done in another process and I want to have this auto-indexing feature, then I also need directory monitoring logic in that process. Why not put it all in there. Also if there are other processes, say a web server, that wants to do directory monitoring, it can just talk to the monitor process via IPC, so we don't have multiple processes polling the same directory. > Great. It would help if I have something liek the ImageCancas. I need > an imlib2 image, draw on it and put it into evas. Everything else > would be easy to rewrite, only the creating of objects is used very > often. Is it possible to create an evas object from an imlib2 image? Taking an Imlib2 image and putting it as an Evas Image object is very easy. Both use BGR32 so there's no colorspace conversion that needs to be done. But in practice you probably don't want to do this. Evas does all imaging internally, and if you try to bolt on mevas's way of doing things with Imlib2 images, you're likely to not be happy with performance. As a matter of fact, whereas evas was once tightly coupled with Imlib2, now there's nary a reference to be found. They are completely divorced, and I think Imlib2 might be on the path to deprecation within EFL. (Maybe someone who knows about EFL can comment.) Evas does support memory buffer backends. So this is essentially what BitmapCanvas is. Anyway, writing a mevas interface for evas is probably not something that can be architected easily in mevas. Probably the best bet in terms of performance (and debugging) is to wrap evas in the same API mevas uses and just have the app think it's really mevas. :) But I'm not sure -- I haven't given this too much thought. Cheers, Jason. ------------------------------------------------------- This SF.Net email is sponsored by: New Crystal Reports XI. Version 11 adds new functionality designed to reduce time involved in creating, integrating, and deploying reporting solutions. Free runtime info, new features, or free trial, at: http://www.businessobjects.com/devxi/728 _______________________________________________ Freevo-devel mailing list Freevo-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freevo-devel
