Re: Help to compile Zookeeper C API on a old system

2009-07-06 Thread Mahadev Konar
Hi Qian, I am not sure if it will work. You should be able to back port it such a way so that it works with gcc 3.*/4.*, but again I have never tried it. mahadev On 7/6/09 6:35 PM, "Qian Ye" wrote: > Thanks Mahadev, I follow the installation instruction in the README, > > autoreconf -i -f

Re: Help to compile Zookeeper C API on a old system

2009-07-06 Thread Qian Ye
Thanks Mahadev, I follow the installation instruction in the README, autoreconf -i -f ./configure --prefix=$dir make make install until "./configure --prefix=$dir", there is no error, however, errors came when I did make, My plan is change the compiler from gcc to g++, and solve the compile err

Re: zookeeper on ec2

2009-07-06 Thread Henry Robinson
On Mon, Jul 6, 2009 at 10:16 PM, Ted Dunning wrote: > No. This should not cause data loss. > As soon as ZK cannot replicate changes to a majority of machines, it > refuses > to take any more changes. This is old ground and is required for > correctness in the face of network partition. It is

Re: zookeeper on ec2

2009-07-06 Thread Ted Dunning
There is no full stop for our system. It (had better) will run forever. That said, we have no permanent information in ZK that is not persisted in, say, SVN. On Mon, Jul 6, 2009 at 3:06 PM, Gustavo Niemeyer wrote: > > Doing that would make the intricate and unlikely failure mode that Henry > >

Re: zookeeper on ec2

2009-07-06 Thread Gustavo Niemeyer
Hi again, (...) > ZK seemed pretty darned stable through all of this. Sounds like a nice test, and it's great to hear that ZooKeeper works well there. > The only instability that I saw was caused by excessive amounts of data in > ZK itself.  As I neared the (small) amount of memory I had allocat

Re: zookeeper on ec2

2009-07-06 Thread Ted Dunning
On Mon, Jul 6, 2009 at 12:58 PM, Gustavo Niemeyer wrote: > > can make the ZK servers appear a bit less connected. You have to plan > for > > ConnectionLoss events. > > Interesting. Note that most of these seem to be related to client issues, especially GC. If you configure in such a way as to g

Re: zookeeper on ec2

2009-07-06 Thread Ted Dunning
No. This should not cause data loss. As soon as ZK cannot replicate changes to a majority of machines, it refuses to take any more changes. This is old ground and is required for correctness in the face of network partition. It is conceivable (barely) that *exactly* the minority that were behin

Re: zookeeper on ec2

2009-07-06 Thread Patrick Hunt
Ted thanks for the info. I've created a wiki page http://wiki.apache.org/hadoop/ZooKeeper/ZooKeeperOnEC2 to capture details of running ZK on EC2. If you or anyone else would like to update it with information please do so. Regards, Patrick Ted Dunning wrote: I disagree with the original pos

Re: zookeeper on ec2

2009-07-06 Thread Evan Jones
On Jul 6, 2009, at 15:40 , Henry Robinson wrote: This is an interesting way of doing things. It seems like there is a correctness issue: if a majority of servers fail, with the remaining minority lagging the leader for some reason, won't the ensemble's current state be forever lost? This is aki

Re: zookeeper on ec2

2009-07-06 Thread Gustavo Niemeyer
Hi Ted, > b) EC2 interconnect has a lot more going on than in a dedicated VLAN.  That > can make the ZK servers appear a bit less connected.  You have to plan for > ConnectionLoss events. Interesting. > c) for highest reliability, I switched to large instances.  On reflection, I > think that was

Re: zookeeper on ec2

2009-07-06 Thread Henry Robinson
On Mon, Jul 6, 2009 at 7:38 PM, Ted Dunning wrote: > > I think that the misunderstanding is that this on-disk image is critical to > cluster function. It is not critical because it is replicated to all > cluster members. This means that any member can disappear and a new > instance can replace

Re: zookeeper on ec2

2009-07-06 Thread Ted Dunning
I disagree with the original post that this is a problem, even in EC2. Having the persistent copy on disk is exactly what makes the rolling restart work so well. I think that the misunderstanding is that this on-disk image is critical to cluster function. It is not critical because it is replicat

Re: zookeeper on ec2

2009-07-06 Thread Ted Dunning
We have used EC2 quite a bit for ZK. The basic lessons that I have learned include: a) EC2's biggest advantage after scaling and elasticity was conformity of configuration. Since you are bringing machines up and down all the time, they begin to act more like programs and you wind up with boot sc

Re: zookeeper on ec2

2009-07-06 Thread Mahadev Konar
Hi David, Answers in line: On 7/6/09 4:45 AM, "David Graf" wrote: > Hello > > I wanna set up a zookeeper ensemble on amazon's ec2 service. In my > system, zookeeper is used to run a locking service and to generate > unique id's. Currently, for testing purposes, I am only running one > instanc

Re: Help to compile Zookeeper C API on a old system

2009-07-06 Thread Mahadev Konar
Hi Qian, What issues do you face? I have never tried compiling with the configuration below, but I could give it a try in my free time to see if I can get it to compile. mahadev On 7/6/09 7:37 AM, "Qian Ye" wrote: > Hi all: > > I'm writing to ask you to do me a favor. It's urgent. For some

zookeeper on ec2

2009-07-06 Thread David Graf
Hello I wanna set up a zookeeper ensemble on amazon's ec2 service. In my system, zookeeper is used to run a locking service and to generate unique id's. Currently, for testing purposes, I am only running one instance. Now, I need to set up an ensemble to protect my system against crashes.

Help to compile Zookeeper C API on a old system

2009-07-06 Thread Qian Ye
Hi all: I'm writing to ask you to do me a favor. It's urgent. For some unchangeable reason, I have to compile "libzookeeper_st.a", "libzookeeper_mt.a" on an old system: gcc 2.96 autoconf 2.13 automake 1.4-p5 libtool 1.4.2 I cannot not compile the target lib in the usual way, and this task drives