storing in db 2.22120189667 for 24438 tweets storing in memcached 1.91712522507 reading from db 7.59235692024 (multi-get) reading from memcached 1.85763192177 (single get)
With libmemcached :) much improvement :D On Aug 18, 12:31 pm, dormando <dorma...@rydia.net> wrote: > You should go read the manuals. > > Look for "multiget" or mget or get_multi or whatever functions. I don't > know what the equivalent set routines would look like for pylibmemc > offhand. It should be in the documentation. > > If not, and someone knows, they should pipe up :) > > -Dormando > > > > > > > > On Thu, 18 Aug 2011, Neeraj Agarwal wrote: > > I have pasted my code above. > > > I tried my program again by writing back the results in a file. > > perhaps mongo just pushes the request and returns so it wasn't taking > > much time while memcached is a blocking call. > > > storing in db 1.26978802681 for 11437 tweets > > storing in memcached 1.5911450386 > > reading from db 41.3619318008 (was reading > > one record at a time from db and writing it to the file) > > reading from memcached 1.78016901016 (similarly with > > memcached) > > > mongodb still winning in terms of writing.. > > > Does libmemcached groups multiple get/ set calls? Or are they even > > blocking calls? > > > Thanks for your time, > > Neeraj > > > On Aug 18, 12:08�pm, dormando <dorma...@rydia.net> wrote: > > > > 85 seconds was because of the network latency (was using EC2 with my > > > > computer. pinging time was 350 ms itself..) > > > > > Perhaps for x many number of items, it was taking x*350 ms time for > > > > making calls.. while mongo, it was sending all data at one go. > > > > > So I ran the script on the server itself: > > > > storing in db �0.108298063278 �for �1487 �items > > > > storing in memcached �0.208426952362 > > > > reading from db �0.0738799571991 > > > > reading from memcached �0.145488023758 > > > > > what am I doing wrong here? > > > > Can you attach your benchmark program? > > > > You should be using multiget to fetch back items quicker. Try using > > > pylibmc, which is based off of libmemcached. That may be able to do > > > multisets and get you better speed. The other library is pure python, I > > > think. Would be slower than a native DB driver. > > > > -Dormando
