Re: Problems about two memcached java clients: spy and gwhalin

2010-10-17 Thread dormando
I think you're supposed to read to the point where it says queues stuff in memory before sending to the server and extrapolate that writing to the queue too fast is a bad thing. On Sun, 17 Oct 2010, Shi Yu wrote: Kelvin. This is year 2010 and computer programs should not be that fragile. And

Re: Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-17 Thread Dustin
On Oct 16, 9:28 pm, Shi Yu shee...@gmail.com wrote:         MapString,String map1 = new HashMapString,String();         MapString,String map2 = new HashMapString,String();         MapString,String map3 = new HashMapString,String(); You're loading at least four million strings into two

Re: Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-17 Thread Shi Yu
Hi Dustin. I have to go on my work now so I probably wont spend any time on this issue. Please, before you suggest, try some experiment to load more than 6 million records using the same API. I would be happy to hear how you do that. I now fully rely on Whalin's API, it can handle 14 million

Re: Problems about two memcached java clients: spy and gwhalin

2010-10-17 Thread Dustin
On Oct 16, 11:11 pm, Kelvin Edmison kel...@kindsight.net wrote:   while trying to re-create this problem and point out the various errors in his code, I found that, in his test case, if I did not call Future.get() to verify the result of the set, the spyMemcached client leaked memory.  Given

Re: Problems about two memcached java clients: spy and gwhalin

2010-10-17 Thread Dustin
On Oct 16, 11:24 pm, Shi Yu shee...@gmail.com wrote: Kelvin. This is year 2010 and computer programs should not be that fragile. And I believe my code is just a fast simple toy problem trying to find out why I failed too many times in my real problem. Before I post my problem, I checked and

Re: Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-17 Thread Dustin
On Oct 16, 11:51 pm, Shi Yu shee...@gmail.com wrote: Hi Dustin. I have to go on my work now so I probably wont spend any time on this issue. Please, before you suggest, try some experiment to load more than 6 million records using the same API. I would be happy to hear how you do that.  I now

Re: Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-17 Thread Shi Yu
Thanks Dustin, marked and will try again. On Sun, Oct 17, 2010 at 2:38 AM, Dustin dsalli...@gmail.com wrote: On Oct 16, 11:51 pm, Shi Yu shee...@gmail.com wrote: Hi Dustin. I have to go on my work now so I probably wont spend any time on this issue. Please, before you suggest, try some

Problems about two memcached java clients: spy and gwhalin

2010-10-16 Thread Shi Yu
Hi, I have two problems when using memcached java clients the spymemcached (http://code.google.com/p/spymemcached/) and the gwhalin java client for memcached (http://github.com/gwhalin/Memcached-Java-Client). I found that the spymemcached failed to store more than 4.3 million records sometimes

Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-16 Thread Shi Yu
Follow up my previous question. I tried to read those 6 million key,value records out. Both API are capable, but the spymemcache API is faster (16 minutes) than the Whalin's v2.5.1 distribution (24 minutes) (http://github.com/gwhalin/Memcached-Java-Client/downloads) .Since the spymemcache cannot

Re: Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-16 Thread Dustin
On Oct 16, 5:27 pm, Shi Yu shee...@gmail.com wrote: I am really curious why spy cannot store up to 6 million... I'm quite sure you can add more than 6M items to something. Part of the problem might be that you're adding to an in-memory queue as fast as possible, not checking results, or

Re: Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-16 Thread Shi Yu
I have also tried the CacheLoader API, it pops a java GC error. The thing I haven't tried is to separate 6 million records into several objects and try CacheLoader. But I don't think it should be that fragile and complicated. I have spent a whole day on this issue, now I just rely the hybrid

Re: Updated: Problems about two memcached java clients: spy and gwhalin

2010-10-16 Thread Dustin
On Oct 16, 6:45 pm, Shi Yu shee...@gmail.com wrote: I have also tried the CacheLoader API, it pops a java GC error. The thing I haven't tried is to separate 6 million records into several objects and try CacheLoader. But I don't think it should be that fragile and complicated. I have spent a