(Let's move this to devel) Good job, Ed!
Some more questions: * did you compress the data? I guess xml is a bit more verbose, but compression should probably remedy that * did you look at memory usage? A protocol that is fast on desktop machines, but that requires the entire sync exchange to be kept in memory will be useless for a mobile device. Peter On Sunday 17 May 2009 09:23:06 pm Ed Bartosh wrote: > Hi, > > 2009/5/14 Damien Elmes <reso...@ichi2.net>: > > Processing single facts at a time will be memory efficient but very > > slow. A more intelligent solution would be to place an upper bound on > > the number of facts processed at once. > > 2009/5/14 Peter Bienstman <peter.bienst...@ugent.be>: > >> > An alternative would > >> > be an incremental JSON parser, but that may be complicated to > >> > implement. > >> > >> Not that I know anything about this, but would there be a difference > >> JSON/XML in that regard? > > > > I stumbled upon this: > > > > http://effbot.org/zone/element-iterparse.htm > > > > This seems like a very efficient way of doing an incremental XML parser. > > Moreover, it has the advantage that the C-compiled cElementTree module is > > available for the Python 2.5 installation on Windows Mobile, whereas JSON > > would be a Python only separate install there, and therefore much slower. > > > > That seems to point rather strongly to XML over JSON, but I'd appreciate > > feedback from people who have more experience in these matters. > > I sketched out test script for comparing xml-rpc and json-rpc > transports to investigate this topic a bit deeper. Script sends cards > and related facts from client to server using different protocols > packing them into packs(list of cards and related facts). Protocol, > amount of cards to send and amount of cards per pack are arguments of > the script. Cards and facts are artificial, but their structure and > field sizes were taken from current development version of Mnemosyne. > > I looked at the anki sources and used it's approach, but different > protocol. I also used this implementation of SimpleJSONRPCServer: > http://code.activestate.com/recipes/552751/ with small modification. > It's included into the script to simplify its usage. The only external > dependency is simplejson module, which you > can install from pypi: http://pypi.python.org/pypi/simplejson/ > > A couple of command line examples: > 1. run json-rpc client connecting to server on 192.168.255.17:8000 and > sending 10000 cards. One pack contains 1000 cards and related facts. > deck1 is the name of the deck. It should be different from one defined > for server. > synctest.py client json-rpc http://localhost:8000 deck1 10000 1000 > 2. run json-rpc server listening on 192.168.255.17:8000: > synctest.py server json-rpc 192.168.255.17 8000 deck2 10000 1000 > deck2 is the name of the deck. It should be different from the one > defined for client. > Amount of cards and cards/pack is not used at the moment, because > server only gets data from client but doesn't send anything. > > Feel free to play with the script and modify it whatever way you want. > You can get the source from my master branch on github, or look at it > and download from here: > http://github.com/bartosh/pomni/blob/609c579e09f85e12cf4c8b832e00fd3f4a13a3 >19/mnemosyne/mnemosyne/pomni/scripts/synctest.py > > Below is the link to test results table for two major cases: > 1. server and client run on one host(X60s lenovo laptop) and > communicate through loopback interface. Test results of this case have > 'loopback' type in the table. > 2. server run on the same laptop and client run on N810 handheld > device (TI OMAP 2420, 400Mhz arm processor, 128Mb RAM). Both server > and client in the same WLAN with almost no other traffic in it. This > case has type 'WLAN 802.11g'. > http://www.assembla.com/wiki/show/pomni/Sync_Test > > I'm not going to draw any conclusions in purpose. > Instead looking forward to hearing your comments. ------------------------------------------------ Peter Bienstman Ghent University, Dept. of Information Technology Sint-Pietersnieuwstraat 41, B-9000 Gent, Belgium tel: +32 9 264 34 46, fax: +32 9 264 35 93 WWW: http://photonics.intec.UGent.be email: peter.bienst...@ugent.be ------------------------------------------------ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "mnemosyne-proj-users" group. To post to this group, send email to mnemosyne-proj-users@googlegroups.com To unsubscribe from this group, send email to mnemosyne-proj-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/mnemosyne-proj-users?hl=en -~----------~----~----~----~------~----~------~--~---
