Thanks for the detailed explanation, Mahadev and Ted. The suggestions are very valuable to us.
One additional question for how zookeeper handles errors: Let's say we have 3 zookeeper servers Z1, Z2, Z3, and 3 clients C1, C2, C3. C1 is connected to Z1. C2 is connected to Z2. C3 is connected to Z3. C3 has created a ephemeral node. What will happen if C3 and Z3 are partitioned from the rest of the world? I guess C3 should see some errors, but where will I get it (since C3 is not calling any zookeeper functions after the ephemeral node is created. I am reading http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200807.mbox/%3c386225.96676...@web31804.mail.mud.yahoo.com%3e There are 2 types of error that C3 needs to handle: 1. disconnections; 2. session expirations. Is that still valid (since it's over 1.5 years old)? I am also reading http://wiki.apache.org/hadoop/ZooKeeper/FAQ and I guess these are the only 2 types of error that C3 need to handle. Correct? Zheng On Sun, Jan 24, 2010 at 7:08 PM, Mahadev Konar <maha...@yahoo-inc.com> wrote: > Hi Zheng, >> Let's say I have 100 long-running tasks and 20 nodes. >> I want each of them to take up to 10 tasks. Each of the task should be >> taken by one and only one node. > This is exactly one of our users is using ZooKeeper for. You might want to > make it more general saying that a directory /tasks/ will have the list of > tasks that need to be processed - (in your case 0-99). Basically storing the > list of tasks also in zookeeper. The clients can then read of this list and > try creating ephemeral nodes for tasks in mytasks/ and assign themselves as > the owner of those tasks. > You also should factor in the task dying or the machine not able to start > that task. In that case the machine should just remove the ephemeral node > that it created and should let the other machines take up that task. > > Here is one minor thing that might be useful. One of the zookeeper users who > was doing exactly the same thing had the number of failures of booting up a > task stored as data in /tasks/ znode for that task. This way all the > machines can update this count and alert (to the admin) if a task cannot be > started or worked upon by a given count of machines. > > Hope this helps. > > Thanks > Mahadev > > > On 1/23/10 12:58 AM, "Zheng Shao" <zsh...@gmail.com> wrote: > >> Let's say I have 100 long-running tasks and 20 nodes. >> I want each of them to take up to 10 tasks. Each of the task should be >> taken by one and only one node. >> >> Will the following solution solve the problem? >> >> Create a directory "/mytasks" in zookeeper. >> Normally there will be 100 EPHEMERAL children in /mytasks directory, >> named from "0" to "99". >> The data of each will be the name of the node and the process id in >> the node. This data is optional but allow us to do lookup from task to >> node and process id. >> >> >> Each node will start 10 processes. >> Each process will list the directory "/mytasks" with a watcher >> If trigger by the watcher, we relist the directory. >> If we found some missing files in the range of "0" to "99", we >> create an EPHEMERAL node with no-overwrite option >> if the creation is successful, then we disable the watcher and >> start processing the corresponding task (if something goes wrong, just >> kill itself and the node will be gone) >> if not, we go back to wait for watcher. >> >> Will this work? >> >> > > -- Yours, Zheng
