Storage Engine ?

2012-01-09 Thread Wendy Cheng
There are archived discussions floating around about this subject, particularly with SSD. Are there open source works (implementation, src code, interface description, etc) available for people to take a look ? Thanks, Wendy

Re: Storage Engine ?

2012-01-09 Thread Wendy Cheng
On Mon, Jan 9, 2012 at 1:21 PM, Wendy Cheng wrote: > There are archived discussions floating around about this subject, > particularly with SSD. Are there open source works (implementation, > src code, interface description, etc) available for people to take a > look ? > Found an

Re: Storage Engine ?

2012-01-09 Thread Wendy Cheng
On Mon, Jan 9, 2012 at 2:21 PM, dormando wrote: >> There are archived discussions floating around about this subject, >> particularly with SSD. Are there open source works (implementation, >> src code, interface description, etc) available for people to take a >> look ? > > There is a working and

Re: Storage Engine ?

2012-01-09 Thread Wendy Cheng
On Mon, Jan 9, 2012 at 2:28 PM, Dustin wrote: > > On Monday, January 9, 2012 2:13:36 PM UTC-8, Wendy Cheng wrote: >> >> Found an expired RFC at >> http://code.google.com/p/memcached/wiki/EngineInterface. Look pretty >> old though. Wondering whether there are (open s

Re: Storage Engine ?

2012-01-10 Thread Wendy Cheng
On Tue, Jan 10, 2012 at 12:10 AM, Dustin wrote: >> By taking a quick browse on the source, it seems only "default_engine" >> is included (?) that uses RAM as a backend. Is this right ? So I >> assume for other type of backends, we have to supply our own engine by >> implementing the "ENGINE_HANDLE

Re: Storage Engine ?

2012-01-11 Thread Wendy Cheng
On Tue, Jan 10, 2012 at 7:42 PM, Dustin wrote: > > On Tuesday, January 10, 2012 4:19:26 PM UTC-8, Wendy Cheng wrote: > >> >> It seems the engine operations (hooks) are invoked as part of the >> >> network request processing logic (e.g. process_bin_set() et

Re: Storage Engine ?

2012-01-23 Thread Wendy Cheng
On Wed, Jan 11, 2012 at 10:51 AM, Dustin wrote: >   One thing that might not be obvious is that you don't have to use an item > structure to store your data.  You just need it to transport it between the > back and front. First, I need to pass a thank-you note for all the helps received, as well

elapse time for "set" command ?

2012-02-28 Thread Wendy Cheng
Does anyone have a ballpark number for "memslap --server --test=set" command ? Somehow I got ~0.6 seconds that is surprising. Intuitively, our network round-trip latency for 16K tcp data is less than 0.001 second. I expected the memslap "set" command would not go up to msec range (?). Our lab mach

Re: elapse time for "set" command ?

2012-02-28 Thread Wendy Cheng
On Tue, Feb 28, 2012 at 9:47 AM, Wendy Cheng wrote: > Does anyone have a ballpark number for "memslap --server > --test=set" command ? Somehow I got ~0.6 seconds that is surprising. > Intuitively, our network round-trip latency for 16K tcp data is less > than 0.001 second.

Re: elapse time for "set" command ?

2012-02-28 Thread Wendy Cheng
On Tue, Feb 28, 2012 at 10:02 AM, Wendy Cheng wrote: > On Tue, Feb 28, 2012 at 9:47 AM, Wendy Cheng wrote: >> Does anyone have a ballpark number for "memslap --server >> --test=set" command ? Somehow I got ~0.6 seconds that is surprising. >> Intuitively, our net

Re: elapse time for "set" command ?

2012-02-28 Thread Wendy Cheng
On Tue, Feb 28, 2012 at 11:01 AM, dormando wrote: > > You might want to consider mc-crusher as well. > https://github.com/dormando/mc-crusher > > it's still rough, but from what it can do it's very flexible. Thanks ! Will take a look soon ... Wendy

memcached tuning tips (?)

2012-03-01 Thread Wendy Cheng
Hi, We plan to start tuning work for our Memcached setup. The binaries were built on top of "engine_pu" branch (storage engine interface). A quick test run (everything on default, including storage engine itself) using "/usr/local/bin/memslap --server xyz --test=get --concurrency=1024" got a bun

Re: memcached tuning tips (?)

2012-03-02 Thread Wendy Cheng
On Thu, Mar 1, 2012 at 5:55 PM, Wendy Cheng wrote: > Hi, > > We plan to start tuning work for our Memcached setup. The binaries > were built on top of "engine_pu" branch (storage engine interface). A > quick test run (everything on default, including storage engine > i

Re: memcached tuning tips (?)

2012-03-02 Thread Wendy Cheng
On Fri, Mar 2, 2012 at 2:51 PM, Ripduman Sohan wrote: > On 2 March 2012 01:55, Wendy Cheng wrote: >> Hi, >> >> We plan to start tuning work for our Memcached setup. The binaries >> were built on top of "engine_pu" branch (storage engine interface). A >

Re: Locks, what are they for?

2012-03-07 Thread Wendy Cheng
On Tue, Feb 28, 2012 at 3:51 AM, Slawomir Pryczek wrote: > Actually i have already modified it and it seems working. It's > compatible with current client libraries. I'll post on mailing list > with some simple benchmarks if anyone's interested. > I would be interested to learn the numbers if po

connections-related memory

2012-03-07 Thread Wendy Cheng
As part of the tuning work, it is noticed that the maximum connection count is controlled by "settings.maxconns". There is a comment next to the default setting: settings.maxconns = 1024; /* to limit connections-related memory to about 5MB */ I'm wondering how the 5MB was calculated. Cou

Re: connections-related memory

2012-03-08 Thread Wendy Cheng
-- Wendy > > -- > Blog: http://www.chenzongzhi.info > Twitter: https://twitter.com/baotiao > Git: https://github.com/baotiao > > On 2012-3-8, at 上午8:40, Wendy Cheng wrote: > > As part of the tuning work, it is noticed that the maximum connection > count is controlled by

Helps on LIBEVENT_THREAD mutex ?

2012-03-14 Thread Wendy Cheng
This mutex seems to only exist with Storage Engine branch. We're working on prototyping a new storage engine on top of an NVRAM device. This morning, the test server hangs. Four worker threads are blocked waiting for cache_lock. The cache_lock holder is blocked in "notify_io_complete()" waiting fo

Re: Helps on LIBEVENT_THREAD mutex ?

2012-03-14 Thread Wendy Cheng
On Wed, Mar 14, 2012 at 11:02 AM, Wendy Cheng wrote: > This mutex seems to only exist with Storage Engine branch. > > We're working on prototyping a new storage engine on top of an NVRAM > device. This morning, the test server hangs. Four worker threads are > blocked waiting

Re: Locks, what are they for?

2012-03-27 Thread Wendy Cheng
On Sun, Mar 25, 2012 at 12:16 PM, dormando wrote: >> Thanks. Any reason to move item_lock(s) back to cache_lock ? >> >> We've done preliminary performance runs using memaslap. Memory limits >> are altered from 32MB up to 2GB. The total transaction-per-second >> numbers drop as the memory limits ar

Re: Storage Engine ?

2012-04-05 Thread Wendy Cheng
On Wed, Apr 4, 2012 at 10:47 PM, Dustin wrote: > > On Wednesday, April 4, 2012 11:48:26 AM UTC-7, Aaron Stone wrote: >> >> Hello list! Long time. The storage engine branch looks a little bit behind >> master - is github.com/memcached/memcached:engine-pu the most up to date >> branch (last commit i

Re: Storage Engine ?

2012-04-06 Thread Wendy Cheng
On Thu, Apr 5, 2012 at 10:39 PM, Dustin wrote: > > On Thursday, April 5, 2012 9:51:33 AM UTC-7, Wendy Cheng wrote: > >> >> We also have a prototype storage engine (hybrid of RAM + flash memory) >> up and running. Would be happy to help out > > That would be

Re: Memcached is not responsive to any request

2012-04-25 Thread Wendy Cheng
On Wed, Apr 25, 2012 at 8:32 AM, pcdinh wrote: > Hi all, > > I compiled and installed Memcached 1.4.3 from source: > http://memcached.googlecode.com/files/memcached-1.4.13.tar.gz > > The configuration is quite simple: ./configure --prefix=/usr/local/ > memcached > > It works if I install it on a d

Re: Memcached is not responsive to any request

2012-04-25 Thread Wendy Cheng
On Wed, Apr 25, 2012 at 9:25 AM, pcdinh wrote: > Here is the list of libevent-related packages that are installed in my > server > > root@spica:/usr/src/libevent-2.0.18-stable# dpkg --get-selections | > grep libevent > libevent-2.0-5                                  install > libevent-core-2.0-5  

Re: Memcached is not responsive to any request

2012-04-26 Thread Wendy Cheng
On Thu, Apr 26, 2012 at 2:06 AM, pcdinh wrote: > The native box is my development environment at my office. Sample OS > (Ubuntu 11.10), same libs (autoconf 2.68 ) > In development environment it works. Now I want to move the source > code to a production environment which is Xen-based > I have