Hi I did some testing with a vfs based on a sqlite db and kaa.ipc. The first one is ok, the second one makes trouble.
First test: The old mediadb code needs 0.09 seconds for the whole operation. This includes reading the pickle file, checking the directory and the overlay directory for updates and checking all files for updates. One the same dir with an up-to-date db the db query took 0.05 seconds, the query + checking for new files took 0.008 seconds. Checking the files for updates is now a background task and overlay directory is not supported yet, but I guess the new code isn't much slower than the old one. BUT from the client point of view, the whole operation takes 0.07 seconds. This is all ipc overhead. To test if the overhead is more or less constant, I did some testings on a larger directory with 700 files. It crashed in the ipc code. The code wants to send the whole data on a non blocking socket. Tack: read the help doc: either everything is send or nothing, even for non blocking sockets. As a result, you closed the socket. I changed the code to only send 4096 bytes packages each iteration. The result is _very_ bad. It takes 2 seconds (!!!) to send the data. I replaced the sending code with afast C module (I wrote this module for a project at work, it is not part of kaa.notifier yet). The result is the same, 2 seonds. OK, more debugging. The data has a size of 200kb, on send the server sends only 100kb. The client reads this 100kb after that, nothing happens for 2 seconds. The server sends no more data. It looks to me that the unix sockets don't report the reading back to the server. It looks like a bad problem with unix sockets to me. You can test this by the current svn of kaa.vfs, you only need to change the path in test/client.py where to scan for media files. As a result, I guess we should not use ipc to transfer data from server to client. We should do the db lookup in the client and after that, let the server know that we are monitoring the query. The server will check all files and sends a signal on the progress. We should not send db results using the ipc code. Dischi -- Hello, you've reached the psychiatric hotline. If you are obsessive/compulsive, press 1 repeatedly. If you have multiple personalities, press 2, 3, 4, and 5. If you are simply paranoid, just stay on the line - we know who you are.
pgpm4jMlv1Jtr.pgp
Description: PGP signature
