Re: ZK recovery questions

2010-07-21 Thread Ted Dunning
My own experiments in my own environment where ZK is being used purely for coordination at a fairly low transaction rate (tens to hundreds of ops per second, mostly status updates) made me feel that disk throughput would only be detectable as an issue for pretty massively abused ZK applications. T

Re: ZK recovery questions

2010-07-21 Thread Benjamin Reed
i did a benchmark a while back to see the effect of turning off the disk. (it wasn't as big as you would think.) i had to modify the code. there is an option to turn off the sync in the config that will get you most of the performance you would get by turning off the disk entirely. ben On 07/

Re: ZK recovery questions

2010-07-20 Thread Patrick Hunt
Not having a datadir is currently not possible - the servers expect to read/write snapshot and log files. In particular the leader needs to be able to stream updates, and in some cases the entire latest snapshot, to a follower. It does this by streaming data directly from the filesystem. Patri

Re: ZK recovery questions

2010-07-20 Thread Ashwin Jayaprakash
rum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:108) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:76) Ashwin. -- View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZK-recovery-questions-tp5310116p5319775.html Sent from the zoo

Re: ZK recovery questions

2010-07-20 Thread Ashwin Jayaprakash
s can sync directly from > senior > > servers without relying on log files. In that case, I'm curious to see > what > > happens if you just redirect log files to /dev/null. Anyone tried this? > > > > Regards, > > Ashwin Jayaprakash. > > > > ---

Re: ZK recovery questions

2010-07-20 Thread Mahadev Konar
Hi Ashwin, We have seen people wanting to have something like ZooKeeper without the reliability of permanent storage and are willing to work with loosened guarantees of current Zookeeper. What you mention on log files is certainly a valid use case. It would be great to see how much throughput yo

Re: ZK recovery questions

2010-07-19 Thread Ted Dunning
They don't auto-detect. What is usually done is that the configurations on all the servers are changed and they are re-started one at a time. On Mon, Jul 19, 2010 at 8:35 PM, Ashwin Jayaprakash < ashwin.jayaprak...@gmail.com> wrote: > So, what happens > when a new replacement server has to be br

Re: ZK recovery questions

2010-07-19 Thread Ashwin Jayaprakash
eeper-user.578899.n2.nabble.com/ZK-recovery-questions-tp5310116p5315123.html Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: ZK recovery questions

2010-07-18 Thread Ted Dunning
On Sun, Jul 18, 2010 at 3:34 PM, Ashwin Jayaprakash < ashwin.jayaprak...@gmail.com> wrote: > > - If 1 out of 3 servers crashes and the log files are unrecoverable, how > do we provision a replacement server? > Just start it and it will download a snapshot from the other servers. > >- If

ZK recovery questions

2010-07-18 Thread Ashwin Jayaprakash
Hi, I've been reading the docs and trying out some basic Zookeeper examples. I have a few simple questions related to recovery. It would be good to have questions like these on the Wiki/docs to avoid noobs like me asking the same thing over and over. - If 1 out of 3 servers crashes and the lo