Re: Problems with memcache running on virtual ethernet interface
Sorry yeah, IP alias. I actually think I may have found the source of my problem. I was using a connection pooler that had about max_conns connections per worker process! So despite not getting any traffic to the box the memcached connections would have been saturated. This doesn't explain the errors in my log though... I'll keep updating with my progress thanks. D On Jun 12, 12:23 am, Vladimir Vuksan wrote: > Can you send what is the command you use to invoke memcached. eth0:0 is > not really a virtual interface but an IP alias. > > Vladimir > > > > On Fri, 11 Jun 2010, David Novakovic wrote: > > We have two VPS running on linode with a virtual ethernet connection > > between them. The ifconfig info looks like the following.. (note the > > name of the interface) > > > eth0:0 Link encap:Ethernet HWaddr fe:fd:ad:e6:95:fb > > inet addr:192.168.141.205 Bcast:192.168.255.255 Mask: > > 255.255.128.0 > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > > Interrupt:28 > > > Memcached seems to run ok when it is listening on 127.0.0.1, as soon > > as i switch to the interface above it becomes unbearably slow! > > This means I can't run my memcached servers in a cluster. > > Even when I telnet in to do a "stats" sometimes the connection is > > closed by memcached after a long pause before it even sends a > > response. > > > When I did have each server running its own memcache instance on > > 127.0.0.1 (ie not clustered) I notice the follow coming up in the log > > right before I had to restart memcache to get it to work again. Does > > it mean anything to anyone? i notice some people have interesting ways > > to deal with this issue.. like the following node.js snippet I found > > on github:http://gist.github.com/322407 > > > r...@li159-251:~# tail -f /var/log/memcached.log > > event_add: No such file or directory > > event_add: No such file or directory > > > Thanks in advance for any insights! > > > David
Re: Distributed != Replicated. Another memcached vocabulary lesson
I think it's kind of ridiculous to even bother arguing about this. Memcached does what it does and it does it exceptionally well, to the point that it's used very heavily at most of the largest websites in the world. Having said that, I just googled "distributed system" and took some definitions from the first few results: A distributed system consists of a collection of autonomous computers, connected through a network and distribution middleware, which enables computers to coordinate their activities and to share the resources of the system, so that users perceive the system as a single, integrated computing facility. or, this one from Google itself: A distributed system is an application that executes a collection of protocols to coordinate the actions of multiple processes on a network, such that all components cooperate together to perform a single or small set of related tasks. Memcached clearly fits both of these definitions. The servers are autonomous but through the middleware/protocol design, they work together to perform one single task (acting as a cache/distributed hash table) and appear as a single facility to any client. I think the only problem that the OP has is that the "middleware" is embedded in the client, but this just means that the distribution is, in a way, baked into the protocol. As long as every client agrees on the protocol/hash, everything works perfectly. And there is no single point of failure, because if you lose a server, the cache as a whole continues to function. On Fri, Jun 11, 2010 at 12:54 PM, Brian Moon wrote: > After the recent thread and reading some comments on the memcached wiki I > think I know what is wrong. People see the word distributed and think it > means replicated. > > http://www.merriam-webster.com/dictionary/distribute says: > > distribute: > 1 : to divide among several or many > 2 : to spread out so as to cover something : scatter > 3 : to divide or separate especially into kinds > 4 : to use in or as an operation so as to be mathematically distributive > > POW! memcached (the solution, not the daemon) satisfies all of these > definitions to a tee. > > http://www.merriam-webster.com/dictionary/replicate says: > > replicate: > produce a replica of itself > > This is not what memcached is or does. Never claimed to. > > -- > > Brian. > > http://brian.moonspot.net/ > -- awl
Distributed != Replicated. Another memcached vocabulary lesson
After the recent thread and reading some comments on the memcached wiki I think I know what is wrong. People see the word distributed and think it means replicated. http://www.merriam-webster.com/dictionary/distribute says: distribute: 1 : to divide among several or many 2 : to spread out so as to cover something : scatter 3 : to divide or separate especially into kinds 4 : to use in or as an operation so as to be mathematically distributive POW! memcached (the solution, not the daemon) satisfies all of these definitions to a tee. http://www.merriam-webster.com/dictionary/replicate says: replicate: produce a replica of itself This is not what memcached is or does. Never claimed to. -- Brian. http://brian.moonspot.net/
Re: Problems with memcache running on virtual ethernet interface
Can you send what is the command you use to invoke memcached. eth0:0 is not really a virtual interface but an IP alias. Vladimir On Fri, 11 Jun 2010, David Novakovic wrote: We have two VPS running on linode with a virtual ethernet connection between them. The ifconfig info looks like the following.. (note the name of the interface) eth0:0Link encap:Ethernet HWaddr fe:fd:ad:e6:95:fb inet addr:192.168.141.205 Bcast:192.168.255.255 Mask: 255.255.128.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:28 Memcached seems to run ok when it is listening on 127.0.0.1, as soon as i switch to the interface above it becomes unbearably slow! This means I can't run my memcached servers in a cluster. Even when I telnet in to do a "stats" sometimes the connection is closed by memcached after a long pause before it even sends a response. When I did have each server running its own memcache instance on 127.0.0.1 (ie not clustered) I notice the follow coming up in the log right before I had to restart memcache to get it to work again. Does it mean anything to anyone? i notice some people have interesting ways to deal with this issue.. like the following node.js snippet I found on github: http://gist.github.com/322407 r...@li159-251:~# tail -f /var/log/memcached.log event_add: No such file or directory event_add: No such file or directory Thanks in advance for any insights! David
Problems with memcache running on virtual ethernet interface
Hey, We have two VPS running on linode with a virtual ethernet connection between them. The ifconfig info looks like the following.. (note the name of the interface) eth0:0Link encap:Ethernet HWaddr fe:fd:ad:e6:95:fb inet addr:192.168.141.205 Bcast:192.168.255.255 Mask: 255.255.128.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:28 Memcached seems to run ok when it is listening on 127.0.0.1, as soon as i switch to the interface above it becomes unbearably slow! This means I can't run my memcached servers in a cluster. Even when I telnet in to do a "stats" sometimes the connection is closed by memcached after a long pause before it even sends a response. When I did have each server running its own memcache instance on 127.0.0.1 (ie not clustered) I notice the follow coming up in the log right before I had to restart memcache to get it to work again. Does it mean anything to anyone? i notice some people have interesting ways to deal with this issue.. like the following node.js snippet I found on github: http://gist.github.com/322407 r...@li159-251:~# tail -f /var/log/memcached.log event_add: No such file or directory event_add: No such file or directory Thanks in advance for any insights! David
Re: Is this really Distributed?
In memcached too data is not distributed, that logic is part of client and not the server. You can't separate the server from the client. Is there a memcached client that exists that does not distribute the objects around when multiple servers are given to it? I don't know of one. Brian.
Re: Is this really Distributed?
All client server architectures are distributed. like ftp, ldap, sql servers are all distributed. How is the data distributed across many servers in these services? In this case too we have clients which are distributed. Also we can have inteliigent clients which can use different servers based on different servers. In memcached too data is not distributed, that logic is part of client and not the server. On Fri, Jun 11, 2010 at 11:13 AM, Les Mikesell wrote: > Dilip wrote: > >> Going by that definition: >> >> All client server architectures are distributed. >> >> like ftp, ldap, sql servers are all distributed. >> > > How is the data distributed across many servers in these services? > > > As i understood, in distributed systems there is no single point of >> failure. >> > > Memcache is sort of a special case in that not finding a cached item is not > a failure - it is an expected event and the client is expected to retrieve > the data from the actual source and refresh the cache. If a server fails, > this cache refresh will go to a different server. > > > but in all these cases there are single point of failures. >> >> internet is distributed because there is no single point of failure. >> >> But after looking at the link >> http://en.wikipedia.org/wiki/Distributed_computing, >> here client server architecture is termed as distributed. >> I think the responsibility of not having single point of failure is to >> have intermediary clients, which can do that. >> >> Now I think we can call it that way. >> > > A cache server failure shouldn't have any visible effect other than making > the source servers work harder while the data it held is refreshed onto the > remapped servers. > > > -- > Les Mikesell > lesmikes...@gmail.com > > -- Regards Dilip (If you are not living on the edge then you are wasting space.)
