How to Create Memcached Cluster
We have 3 memcached server and we are using individual IP address in application. is it possible to create Cluster if yes then please share setup recommendation step by step -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/ae3fc34d-7872-4653-9d10-fbfc46cf80afo%40googlegroups.com.
Re: Memcached cluster
Thanks guys, this is quite helpful In the same vein, can we also say that memcache replication is also a client side driven solution? -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
Also, anyone heard about repcached? Is this a patch users would need to add in order to get data replication feature? Does memcached endorse this? -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
On Tue, Mar 4, 2014 at 2:56 AM, Ranjit D'Souza bernard@gmail.com wrote: Thanks guys, this is quite helpful In the same vein, can we also say that memcache replication is also a client side driven solution? In the stock system, there is no replication - although I believe others done it with client code. If you have a large enough number of servers, losing one will just add a small percentage of extra load on your backend DB to cover the extra cache misses. -- Les Mikesell lesmikes...@gmail.com -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
FYI, I have written a memcached compatible KVS named yrmcds that implements master-slave replication. As this is slightly off-topic from the list, please ask me directly if you are interested in it: http://cybozu.github.io/yrmcds/ @ymmt2005 On Tuesday, March 4, 2014 6:33:24 PM UTC+9, Ranjit D'Souza wrote: Also, anyone heard about repcached? Is this a patch users would need to add in order to get data replication feature? Does memcached endorse this? -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Memcached cluster
I have a basic question: Does memcached support clustering and HA out of the box, or does it need some third party integration in order to do so? Thanks, Ranjit -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
On Mon, Mar 3, 2014 at 6:01 AM, Ranjit D'Souza bernard@gmail.com wrote: I have a basic question: Does memcached support clustering and HA out of the box, or does it need some third party integration in order to do so? Yes, but it is a cache, not a persistent store so the way it handles failure of a node is for the client to fetch a new copy of the cache misses from the backing storage. -- Les Mikesell lesmikes...@gmail.com -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
Thank you Is there any concept of master-slave configuration (like in Redis), and promoting the slave to master? Can you point me to a document or wiki link that gives more information on how to set up a memcached cluster? -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
There is no server-side concept of master-slave. memcached never talks to any other servers; it only responds to requests from clients. Any clustering or failover you do will need to be driven from the client. What you could do is set up a client to write to two different memcached boxes and read from one randomly; if that request fails, then read from the other. This way, if one goes down, you still have all the data in cache. Here's an article I found (but didn't read yet): http://www.linuxjournal.com/article/7451 ~Ryan On Mon, Mar 3, 2014 at 9:36 PM, Ranjit D'Souza bernard@gmail.comwrote: Thank you Is there any concept of master-slave configuration (like in Redis), and promoting the slave to master? Can you point me to a document or wiki link that gives more information on how to set up a memcached cluster? -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
On Mon, Mar 3, 2014 at 11:36 PM, Ranjit D'Souza bernard@gmail.com wrote: Thank you Is there any concept of master-slave configuration (like in Redis), and promoting the slave to master? No, the server nodes are all alike and don't need to know about each other. The clients need to know about all the servers and they distribute the data by hashing the keys. Depending on the hashing scheme the clients can rebalance the distribution when a node fails (invalidating some existing data) or continue to fail and use the backing store for the missing percentage until the node is back on line. Can you point me to a document or wiki link that gives more information on how to set up a memcached cluster? The server side is packaged for some Linux distributions. You just configure the amount of memory for it to use on each node. -- Les Mikesell lesmikes...@gmail.com -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
Hello Dormando!Thanks for your reply. Description and requirements are: - Memcached will be used a primary storage (i.e. not for caching MySQL). - It should have failover feature (in case if one server fails all connections goes to another server). Basically something like Master-Slave will be sufficient, but Master-Master architecture is more acceptable. Thanks! Sorry, I can't advocate it as primary storage. If you can't lose the data, please don't use memcached. On Sunday, March 10, 2013 1:58:16 AM UTC+2, Dormando wrote: Dear memcached community, It would be really good to build a failover memcached cluster.� As I know this functionality is not provided by default.� Hence as options - you may use CouchBase Server or something like Repcached. Both of them has disadvantages: CouchBase Server is much slower, �Repcached works with legacy memcached version 1.2.8 Based on your�experience�what is the best way to build cluster of memcached servers? Thanks in advance! Hi, This depends on why you need that second cluster and what the constraints are for it. You can do client side replication via libmemcached which will handle a lot of potential use cases. Though oftentimes people who are attempting to do this are doing so because they don't understand normal memcached clustering very well. So it'd be useful to state all of your requirements up front. Then we can make a real recommendation/etc. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
It's really a discussion for another mailing list, but if you could elaborate to me directly or to couchb...@googlegroups.com, I'd be interested in why you say Couchbase is much slower. I've not seen it that way. Full disclosure, I'm a couchbase person. I also do a lot of work on the spymemcached client. Thanks, Matt On Thu, Mar 14, 2013 at 4:47 AM, Oleksandr Drach luckyred...@gmail.comwrote: Thanks, Henrik! I will look onto Cassandra later. BTW repcached 1.2.x may fit our needs.. Anyone has used it in production? What are your feedbacks? On Monday, March 11, 2013 1:48:23 PM UTC+2, Henrik Schröder wrote: Memcached is a cache, not storage, you really shouldn't use it as such. When you set a value in memcached, you have no guarantees whatsoever that you'll be able to get the value back afterwards. You're guaranteed to get the latest value set if you get something, and you're guaranteed to not get a value if it's been deleted or has expired. But there are a lot of factors that can cause a value to be spontaneously evicted, to say nothing of the fact that you lose everything if you restart it or if the machine crashes. Also note that any replication functionality can lead to inconsistency, since there are no built-in mechanisms for resolving that, you can just hope that your failover server has the same data as the original one. If you want storage, then get a piece of software that actually offers storage, there are plenty to choose from. But memcached is probably the wrong choice for you. If you only need key-value storage, then I suggest you check out Cassandra, it scales pretty linearly in that scenario. Or you could check out hstore in Postgresql, but you probably need to make your own sharding for that. /Henrik On Mon, Mar 11, 2013 at 9:57 AM, Oleksandr Drach lucky...@gmail.comwrote: Hello Dormando! Thanks for your reply. Description and requirements are: - Memcached will be used a primary storage (i.e. not for caching MySQL). - It should have failover feature (in case if one server fails all connections goes to another server). Basically something like Master-Slave will be sufficient, but Master-Master architecture is more acceptable. Thanks! On Sunday, March 10, 2013 1:58:16 AM UTC+2, Dormando wrote: Dear memcached community, It would be really good to build a failover memcached cluster.� As I know this functionality is not provided by default.� Hence as options - you may use CouchBase Server or something like Repcached. Both of them has disadvantages: CouchBase Server is much slower, �Repcached works with legacy memcached version 1.2.8 Based on your�experience�what is the best way to build cluster of memcached servers? Thanks in advance! Hi, This depends on why you need that second cluster and what the constraints are for it. You can do client side replication via libmemcached which will handle a lot of potential use cases. Though oftentimes people who are attempting to do this are doing so because they don't understand normal memcached clustering very well. So it'd be useful to state all of your requirements up front. Then we can make a real recommendation/etc. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+...@**googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
I realize this is a Memcached list, and this isn't to suggest that Memcached isn't a rock solid cache. However it ought to be noted that other tools/services offer in-memory replication features while working as a drop-in replacement for building your own scalable Memcached architecture. Redis has master-slave capabilities; CraigsList at one point was built on 40 Redis nodes. [1] Also, if I'm not mistaken, Riak [2], MySQL Cluster [3], and AWS ElastiCache [4] (probably several more: couchdb, cassandra...) all offer more or less what you'd expect from a memcached cluster. Redis, in particular, is awesome for a variety of reasons. Here is a slideshare I did a while back on Redis. [5] [1] http://blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist/ [2] http://basho.com/riak/#architechture [3] http://dev.mysql.com/doc/ndbapi/en/ndbmemcache.html [4] http://aws.amazon.com/elasticache/ [5] http://www.slideshare.net/phpguru/redis-101-10043219 -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
Hello Dormando! Thanks for your reply. Description and requirements are: - Memcached will be used a primary storage (i.e. not for caching MySQL). - It should have failover feature (in case if one server fails all connections goes to another server). Basically something like Master-Slave will be sufficient, but Master-Master architecture is more acceptable. Thanks! On Sunday, March 10, 2013 1:58:16 AM UTC+2, Dormando wrote: Dear memcached community, It would be really good to build a failover memcached cluster.� As I know this functionality is not provided by default.� Hence as options - you may use CouchBase Server or something like Repcached. Both of them has disadvantages: CouchBase Server is much slower, �Repcached works with legacy memcached version 1.2.8 Based on your�experience�what is the best way to build cluster of memcached servers? Thanks in advance! Hi, This depends on why you need that second cluster and what the constraints are for it. You can do client side replication via libmemcached which will handle a lot of potential use cases. Though oftentimes people who are attempting to do this are doing so because they don't understand normal memcached clustering very well. So it'd be useful to state all of your requirements up front. Then we can make a real recommendation/etc. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
Dear memcached community, It would be really good to build a failover memcached cluster. As I know this functionality is not provided by default. Hence as options - you may use CouchBase Server or something like Repcached. Both of them has disadvantages: CouchBase Server is much slower, Repcached works with legacy memcached version 1.2.8 Based on your experience what is the best way to build cluster of memcached servers? Thanks in advance! Hi, This depends on why you need that second cluster and what the constraints are for it. You can do client side replication via libmemcached which will handle a lot of potential use cases. Though oftentimes people who are attempting to do this are doing so because they don't understand normal memcached clustering very well. So it'd be useful to state all of your requirements up front. Then we can make a real recommendation/etc. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Memcached cluster
Dear memcached community, It would be really good to build a failover memcached cluster. As I know this functionality is not provided by default. Hence as options - you may use CouchBase Server or something like Repcachedhttp://sourceforge.net/projects/repcached/ . Both of them has disadvantages: CouchBase Serverhttp://www.couchbase.com/memcachedis much slower, Repcached works with legacy memcached version 1.2.8 Based on your experience what is the best way to build cluster of memcached servers? Thanks in advance! -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
Don't. Consider your memcached nodes as defense for your database. If you have N physical nodes in your single logical memcached cluster, if one of the physical nodes goes away, then your database will have a burst of 1/N additional activity until the data is re-balanced to the remaining N-1 servers. Memcached is designed to hold transient data. If you are using memcached as a permanent data store, you are Doing It Wrong™. - Nelz On Tue, Mar 5, 2013 at 1:53 AM, Oleksandr Drach luckyred...@gmail.com wrote: Dear memcached community, It would be really good to build a failover memcached cluster. As I know this functionality is not provided by default. Hence as options - you may use CouchBase Server or something like Repcached. Both of them has disadvantages: CouchBase Server is much slower, Repcached works with legacy memcached version 1.2.8 Based on your experience what is the best way to build cluster of memcached servers? Thanks in advance! -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Remember: You do not need permission to do something creative and fun. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Memcached cluster
Hi Nelz, while in general I agree with you, there can be situations where something like that is really useful. In case of multiple data center deployments, you want to keep caches warm in stand-by data center(s). This can be achieved by having database replication do this job but it may be hard if what's you cache is not raw result sets but post processed data. Boris On Tue, Mar 5, 2013 at 12:53 PM, Nelz nelz9...@gmail.com wrote: Don't. Consider your memcached nodes as defense for your database. If you have N physical nodes in your single logical memcached cluster, if one of the physical nodes goes away, then your database will have a burst of 1/N additional activity until the data is re-balanced to the remaining N-1 servers. Memcached is designed to hold transient data. If you are using memcached as a permanent data store, you are Doing It Wrong™. - Nelz On Tue, Mar 5, 2013 at 1:53 AM, Oleksandr Drach luckyred...@gmail.com wrote: Dear memcached community, It would be really good to build a failover memcached cluster. As I know this functionality is not provided by default. Hence as options - you may use CouchBase Server or something like Repcached. Both of them has disadvantages: CouchBase Server is much slower, Repcached works with legacy memcached version 1.2.8 Based on your experience what is the best way to build cluster of memcached servers? Thanks in advance! -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Remember: You do not need permission to do something creative and fun. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups memcached group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: data synchronization betwean memcached cluster nodes?
On Wed, Oct 17, 2012 at 12:25 AM, Raymond Wang rmn...@gmail.com wrote: I am newbie to Memcached, and I have a questions about cluster deployment: does memcached support data synchronization betwean cluster nodes? if it does, how can I determine the scope of synchronization? No, the keys are distributed over the cluster nodes and if data is not found in the cache it is up to the client to pull a new copy from the persistent data store and refresh it in the cache. -- Les Mikesell lesmikes...@gmail.com
data synchronization betwean memcached cluster nodes?
I am newbie to Memcached, and I have a questions about cluster deployment: does memcached support data synchronization betwean cluster nodes? if it does, how can I determine the scope of synchronization? Thanks in advance.
integrity of memcached cluster
Hi, How to check the integrity of memcached cluster ? If not complete, we can add the lack immediately. Best Regards, Jumping Qu -- Don't tell me how many enemies we have, but where they are! (ADV:Perl -- It's like Java, only it lets you deliver on time and under budget.)
memcached cluster
I'd just like to check that my understanding of how a distributed memcached system works. Are these 2 assertions correct? 1) It's the client which is configured to know about multiple memcached servers; the servers themselves know nothing about being part of a cluster. 2) If you have one client configured to use servers A and B, and a 2nd client configured to use servers A,B and C, the results will be indeterminate. The clients may use the same servers for the same key, or they may not - it's just a matter of chance. Questions: 1) Does the ordering of the servers matter in the client? If client 1 uses server A and server B, while client 2 uses server B and server A, will they hash the same? 2) Does the server name matter? If client 1 uses name.memached.localnetwork and client 2 uses 192.168.0.1, will they hash the same? Thanks Rachel
Re: memcached cluster
Assertion 1 is correct. Assertion 2 and the answer to your two questions depends on the client and the type of hashing they use to determine which server to use. To understand this, check out the libketama consistent hash pagehttp://amarok.kde.org/blog/archives/363-libketama-a-consistent-hashing-algo-for-memcache-clients.html . On Fri, Mar 20, 2009 at 6:02 AM, Rachel rwill...@gmail.com wrote: I'd just like to check that my understanding of how a distributed memcached system works. Are these 2 assertions correct? 1) It's the client which is configured to know about multiple memcached servers; the servers themselves know nothing about being part of a cluster. 2) If you have one client configured to use servers A and B, and a 2nd client configured to use servers A,B and C, the results will be indeterminate. The clients may use the same servers for the same key, or they may not - it's just a matter of chance. Questions: 1) Does the ordering of the servers matter in the client? If client 1 uses server A and server B, while client 2 uses server B and server A, will they hash the same? 2) Does the server name matter? If client 1 uses name.memached.localnetwork and client 2 uses 192.168.0.1, will they hash the same? Thanks Rachel -- If you see a whole thing - it seems that it's always beautiful. Planets, lives... But up close a world's all dirt and rocks. And day to day, life's a hard job, you get tired, you lose the pattern. Ursula K. Le Guin http://www.finsel.com/words,-words,-words.aspx (My blog) - http://www.finsel.com/photo-gallery.aspx (My Photogallery) - http://www.reluctantdba.com/dbas-and-programmers/blog.aspx (My Professional Blog)
Re: memcached cluster
3. There is not a single hashing algorithm defined for use by a memcached client. so 4. There is no guarantee that two clients with the same configured set of servers, configured in the same order, will use the same server for the same key. If they use the same client software, they will use the same server for the same key. Different clients may use different software (e.g. one is Java and the other PHP) and in that case the algorithm may be different (but it does not have to be.) -- Svemir Brkic sve...@symplicity.com Symplicity Corporation http://www.symplicity.com
Re: memcached cluster
Hi Rachel, For assertation 2, yes, it's a matter of chance which key is stored in which server, but depending on which server selection algorithm you use, you can determine the probability of a key being stored on the same server from both clients. (It's higher if you use Ketama) Still, you should never run memcached with differently configured clients, since that makes it non-synchronized which can be pretty fatal for you application. For your questions, it depends again on the server selection algorithm. If you use the Ketama one, the order of the clients doesn't matter, but their names do. If you use the naive one, the names of the clients doesn't matter, but their order does. /Henrik On Fri, Mar 20, 2009 at 11:02, Rachel rwill...@gmail.com wrote: I'd just like to check that my understanding of how a distributed memcached system works. Are these 2 assertions correct? 1) It's the client which is configured to know about multiple memcached servers; the servers themselves know nothing about being part of a cluster. 2) If you have one client configured to use servers A and B, and a 2nd client configured to use servers A,B and C, the results will be indeterminate. The clients may use the same servers for the same key, or they may not - it's just a matter of chance. Questions: 1) Does the ordering of the servers matter in the client? If client 1 uses server A and server B, while client 2 uses server B and server A, will they hash the same? 2) Does the server name matter? If client 1 uses name.memached.localnetwork and client 2 uses 192.168.0.1, will they hash the same? Thanks Rachel
Re: memcached cluster
Thanks for your replies. So is this correct? 3. There is not a single hashing algorithm defined for use by a memcached client. so 4. There is no guarantee that two clients with the same configured set of servers, configured in the same order, will use the same server for the same key. Rachel
