Re: Cluster Configuration Issues
Great to hear. Good luck. Patrick Mark Vigeant wrote: Oh awesome! Thanks Patrick! Everything works now. Thanks a lot everyone for all your help! -Mark -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Friday, October 23, 2009 11:58 AM To: Mark Vigeant Cc: Patrick Hunt; zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Usually this indicates that either a firewall is blocking one or more of the ports, or the server isn't started on all the hosts. First verify connectivity, I usually use telnet, see detail here: https://help.maximumasp.com/KB/a445/connectivity-testing-with-ping-telnet-tracert-and-pathping-.aspx Patrick Mark Vigeant wrote: Ok, so now the clientPort is all set, but I'm running into another problem. When I try to start up zookeeper server, the error I'm getting now is that it can't open the channel to the other machine: 2009-10-223 10:39:11,941 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:quorumcnxmana...@336] - Cannot open channel to 2 at election address leela/10.7.21.216:3888 java.net.ConnectException: Connection refused I've checked to see if ports 2888 or 3888 are in use on either machine and they are not. Also, there is an open ssh between the 2 of them. I've tried on a few other ports as well and it's giving me the same issue. Do I need to tamper with port permissions? -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Friday, October 23, 2009 1:23 AM To: zookeeper-user@hadoop.apache.org; Mark Vigeant Subject: Re: Cluster Configuration Issues That usually indicates that you have a process that's already using a port that you've specified zookeeper to use. Try something like: netstat -a -n -p tcp and look at 2181/2888/3888 on the hosts you are using. Either use a different port or stop the process that's currently using. Patrick Mark Vigeant wrote: Haha well I'm an idiot. Thank you. I would end there but a new error is being thrown: 2009-10-22 18:00:17,178 - FATAL [main:quorumpeerm...@86] - Unexpected exception, exiting abnormally Java.net.BindException: Address already in use -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 6:01 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues yeah - thought this was it: you've missed the forward slash on home/mark/zookeeper (this turned up on your exception message). On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant wrote: Yeah I just figured out the problem with zoocfg.py I am running as the same user who created myid. Here's my config: zoo.cfg tickTime-2000 dataDir=home/mark/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1= hermes:2888:3888 server.2= leela:2888:3888 on the machines hermes and leela I've put myid files in /home/mark/zookeeper with the numbers 1 and 2 respectively -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 5:43 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Hi Mark - The Python error relates to not being able to find the zoocfg module - is zoocfg.py in the same directory as zkconf.py? Another couple of questions - are you running zookeeper as the same user who created myid? Can you post your entire configuration file please - copy and paste? Henry On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant wrote: Before I make the Jira, I am trying to go with Ted's advice to use the python script. Unfortunately I'm relatively unfamiliar with python so I'm having trouble running it. When I execute "Python zkconf.py" on the command line it tells me: Traceback (most recent call last): File "zkconf.py", line 27, in from zoocfg import zoocfg ImportError: No module named zoocfg The same error comes when I try to call zkcfg.py from the python interface and when I try running Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I gathered from the Usage). Any suggestions? Also, I've been using zookeeper 3.2.1 -Original Message- From: Ted Dunning [mailto:ted.dunn...@gmail.com] Sent: Thursday, October 22, 2009 4:33 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Try Patrick's utility for creating the config files and compare the result to your hand-made files. On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant wrote: The file contains the number 1 and nothing else. My other node has the number 2 (I only have 2 machines right now, I know it makes more sense to run an odd number of zookeeper nodes but I just want to make sure it works first). Any suggestions? -- Ted Dunning, CTO DeepDyve
Re: Cluster Configuration Issues
Usually this indicates that either a firewall is blocking one or more of the ports, or the server isn't started on all the hosts. First verify connectivity, I usually use telnet, see detail here: https://help.maximumasp.com/KB/a445/connectivity-testing-with-ping-telnet-tracert-and-pathping-.aspx Patrick Mark Vigeant wrote: Ok, so now the clientPort is all set, but I'm running into another problem. When I try to start up zookeeper server, the error I'm getting now is that it can't open the channel to the other machine: 2009-10-223 10:39:11,941 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:quorumcnxmana...@336] - Cannot open channel to 2 at election address leela/10.7.21.216:3888 java.net.ConnectException: Connection refused I've checked to see if ports 2888 or 3888 are in use on either machine and they are not. Also, there is an open ssh between the 2 of them. I've tried on a few other ports as well and it's giving me the same issue. Do I need to tamper with port permissions? -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Friday, October 23, 2009 1:23 AM To: zookeeper-user@hadoop.apache.org; Mark Vigeant Subject: Re: Cluster Configuration Issues That usually indicates that you have a process that's already using a port that you've specified zookeeper to use. Try something like: netstat -a -n -p tcp and look at 2181/2888/3888 on the hosts you are using. Either use a different port or stop the process that's currently using. Patrick Mark Vigeant wrote: Haha well I'm an idiot. Thank you. I would end there but a new error is being thrown: 2009-10-22 18:00:17,178 - FATAL [main:quorumpeerm...@86] - Unexpected exception, exiting abnormally Java.net.BindException: Address already in use -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 6:01 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues yeah - thought this was it: you've missed the forward slash on home/mark/zookeeper (this turned up on your exception message). On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant wrote: Yeah I just figured out the problem with zoocfg.py I am running as the same user who created myid. Here's my config: zoo.cfg tickTime-2000 dataDir=home/mark/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1= hermes:2888:3888 server.2= leela:2888:3888 on the machines hermes and leela I've put myid files in /home/mark/zookeeper with the numbers 1 and 2 respectively -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 5:43 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Hi Mark - The Python error relates to not being able to find the zoocfg module - is zoocfg.py in the same directory as zkconf.py? Another couple of questions - are you running zookeeper as the same user who created myid? Can you post your entire configuration file please - copy and paste? Henry On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant wrote: Before I make the Jira, I am trying to go with Ted's advice to use the python script. Unfortunately I'm relatively unfamiliar with python so I'm having trouble running it. When I execute "Python zkconf.py" on the command line it tells me: Traceback (most recent call last): File "zkconf.py", line 27, in from zoocfg import zoocfg ImportError: No module named zoocfg The same error comes when I try to call zkcfg.py from the python interface and when I try running Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I gathered from the Usage). Any suggestions? Also, I've been using zookeeper 3.2.1 -Original Message- From: Ted Dunning [mailto:ted.dunn...@gmail.com] Sent: Thursday, October 22, 2009 4:33 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Try Patrick's utility for creating the config files and compare the result to your hand-made files. On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant wrote: The file contains the number 1 and nothing else. My other node has the number 2 (I only have 2 machines right now, I know it makes more sense to run an odd number of zookeeper nodes but I just want to make sure it works first). Any suggestions? -- Ted Dunning, CTO DeepDyve
Re: Cluster Configuration Issues
That usually indicates that you have a process that's already using a port that you've specified zookeeper to use. Try something like: netstat -a -n -p tcp and look at 2181/2888/3888 on the hosts you are using. Either use a different port or stop the process that's currently using. Patrick Mark Vigeant wrote: Haha well I'm an idiot. Thank you. I would end there but a new error is being thrown: 2009-10-22 18:00:17,178 - FATAL [main:quorumpeerm...@86] - Unexpected exception, exiting abnormally Java.net.BindException: Address already in use -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 6:01 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues yeah - thought this was it: you've missed the forward slash on home/mark/zookeeper (this turned up on your exception message). On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant wrote: Yeah I just figured out the problem with zoocfg.py I am running as the same user who created myid. Here's my config: zoo.cfg tickTime-2000 dataDir=home/mark/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1= hermes:2888:3888 server.2= leela:2888:3888 on the machines hermes and leela I've put myid files in /home/mark/zookeeper with the numbers 1 and 2 respectively -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 5:43 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Hi Mark - The Python error relates to not being able to find the zoocfg module - is zoocfg.py in the same directory as zkconf.py? Another couple of questions - are you running zookeeper as the same user who created myid? Can you post your entire configuration file please - copy and paste? Henry On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant wrote: Before I make the Jira, I am trying to go with Ted's advice to use the python script. Unfortunately I'm relatively unfamiliar with python so I'm having trouble running it. When I execute "Python zkconf.py" on the command line it tells me: Traceback (most recent call last): File "zkconf.py", line 27, in from zoocfg import zoocfg ImportError: No module named zoocfg The same error comes when I try to call zkcfg.py from the python interface and when I try running Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I gathered from the Usage). Any suggestions? Also, I've been using zookeeper 3.2.1 -Original Message- From: Ted Dunning [mailto:ted.dunn...@gmail.com] Sent: Thursday, October 22, 2009 4:33 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Try Patrick's utility for creating the config files and compare the result to your hand-made files. On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant wrote: The file contains the number 1 and nothing else. My other node has the number 2 (I only have 2 machines right now, I know it makes more sense to run an odd number of zookeeper nodes but I just want to make sure it works first). Any suggestions? -- Ted Dunning, CTO DeepDyve
RE: Cluster Configuration Issues
Haha well I'm an idiot. Thank you. I would end there but a new error is being thrown: 2009-10-22 18:00:17,178 - FATAL [main:quorumpeerm...@86] - Unexpected exception, exiting abnormally Java.net.BindException: Address already in use -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 6:01 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues yeah - thought this was it: you've missed the forward slash on home/mark/zookeeper (this turned up on your exception message). On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant wrote: > Yeah I just figured out the problem with zoocfg.py > > I am running as the same user who created myid. Here's my config: > > zoo.cfg > > tickTime-2000 > dataDir=home/mark/zookeeper > clientPort=2181 > initLimit=5 > syncLimit=2 > server.1= hermes:2888:3888 > server.2= leela:2888:3888 > > on the machines hermes and leela I've put myid files in > /home/mark/zookeeper > with the numbers 1 and 2 respectively > -Original Message- > From: Henry Robinson [mailto:he...@cloudera.com] > Sent: Thursday, October 22, 2009 5:43 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: Cluster Configuration Issues > > Hi Mark - > > The Python error relates to not being able to find the zoocfg module - is > zoocfg.py in the same directory as zkconf.py? > > Another couple of questions - are you running zookeeper as the same user > who > created myid? Can you post your entire configuration file please - copy and > paste? > > Henry > > On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant > wrote: > > > Before I make the Jira, I am trying to go with Ted's advice to use the > > python script. > > > > Unfortunately I'm relatively unfamiliar with python so I'm having trouble > > running it. > > > > When I execute "Python zkconf.py" on the command line it tells me: > > Traceback (most recent call last): > > File "zkconf.py", line 27, in > >from zoocfg import zoocfg > > ImportError: No module named zoocfg > > > > The same error comes when I try to call zkcfg.py from the python > interface > > and when I try running > > Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I > > gathered from the Usage). Any suggestions? > > > > Also, I've been using zookeeper 3.2.1 > > -Original Message- > > From: Ted Dunning [mailto:ted.dunn...@gmail.com] > > Sent: Thursday, October 22, 2009 4:33 PM > > To: zookeeper-user@hadoop.apache.org > > Subject: Re: Cluster Configuration Issues > > > > Try Patrick's utility for creating the config files and compare the > result > > to your hand-made files. > > > > On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant > > wrote: > > > > > The file contains the number 1 and nothing else. My other node has the > > > number 2 (I only have 2 machines right now, I know it makes more sense > to > > > run an odd number of zookeeper nodes but I just want to make sure it > > works > > > first). Any suggestions? > > > > > > > > > > > -- > > Ted Dunning, CTO > > DeepDyve > > >
Re: Cluster Configuration Issues
yeah - thought this was it: you've missed the forward slash on home/mark/zookeeper (this turned up on your exception message). On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant wrote: > Yeah I just figured out the problem with zoocfg.py > > I am running as the same user who created myid. Here's my config: > > zoo.cfg > > tickTime-2000 > dataDir=home/mark/zookeeper > clientPort=2181 > initLimit=5 > syncLimit=2 > server.1= hermes:2888:3888 > server.2= leela:2888:3888 > > on the machines hermes and leela I've put myid files in > /home/mark/zookeeper > with the numbers 1 and 2 respectively > -Original Message- > From: Henry Robinson [mailto:he...@cloudera.com] > Sent: Thursday, October 22, 2009 5:43 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: Cluster Configuration Issues > > Hi Mark - > > The Python error relates to not being able to find the zoocfg module - is > zoocfg.py in the same directory as zkconf.py? > > Another couple of questions - are you running zookeeper as the same user > who > created myid? Can you post your entire configuration file please - copy and > paste? > > Henry > > On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant > wrote: > > > Before I make the Jira, I am trying to go with Ted's advice to use the > > python script. > > > > Unfortunately I'm relatively unfamiliar with python so I'm having trouble > > running it. > > > > When I execute "Python zkconf.py" on the command line it tells me: > > Traceback (most recent call last): > > File "zkconf.py", line 27, in > >from zoocfg import zoocfg > > ImportError: No module named zoocfg > > > > The same error comes when I try to call zkcfg.py from the python > interface > > and when I try running > > Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I > > gathered from the Usage). Any suggestions? > > > > Also, I've been using zookeeper 3.2.1 > > -Original Message- > > From: Ted Dunning [mailto:ted.dunn...@gmail.com] > > Sent: Thursday, October 22, 2009 4:33 PM > > To: zookeeper-user@hadoop.apache.org > > Subject: Re: Cluster Configuration Issues > > > > Try Patrick's utility for creating the config files and compare the > result > > to your hand-made files. > > > > On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant > > wrote: > > > > > The file contains the number 1 and nothing else. My other node has the > > > number 2 (I only have 2 machines right now, I know it makes more sense > to > > > run an odd number of zookeeper nodes but I just want to make sure it > > works > > > first). Any suggestions? > > > > > > > > > > > -- > > Ted Dunning, CTO > > DeepDyve > > >
Re: Cluster Configuration Issues
Missing slash on the front of home/mark/zookeeper On 10/22/09 2:55 PM, "Mark Vigeant" wrote: > Yeah I just figured out the problem with zoocfg.py > > I am running as the same user who created myid. Here's my config: > > zoo.cfg > > tickTime-2000 > dataDir=home/mark/zookeeper > clientPort=2181 > initLimit=5 > syncLimit=2 > server.1= hermes:2888:3888 > server.2= leela:2888:3888 > > on the machines hermes and leela I've put myid files in /home/mark/zookeeper > with the numbers 1 and 2 respectively > -Original Message- > From: Henry Robinson [mailto:he...@cloudera.com] > Sent: Thursday, October 22, 2009 5:43 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: Cluster Configuration Issues > > Hi Mark - > > The Python error relates to not being able to find the zoocfg module - is > zoocfg.py in the same directory as zkconf.py? > > Another couple of questions - are you running zookeeper as the same user who > created myid? Can you post your entire configuration file please - copy and > paste? > > Henry > > On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant > wrote: > >> Before I make the Jira, I am trying to go with Ted's advice to use the >> python script. >> >> Unfortunately I'm relatively unfamiliar with python so I'm having trouble >> running it. >> >> When I execute "Python zkconf.py" on the command line it tells me: >> Traceback (most recent call last): >> File "zkconf.py", line 27, in >>from zoocfg import zoocfg >> ImportError: No module named zoocfg >> >> The same error comes when I try to call zkcfg.py from the python interface >> and when I try running >> Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I >> gathered from the Usage). Any suggestions? >> >> Also, I've been using zookeeper 3.2.1 >> -Original Message- >> From: Ted Dunning [mailto:ted.dunn...@gmail.com] >> Sent: Thursday, October 22, 2009 4:33 PM >> To: zookeeper-user@hadoop.apache.org >> Subject: Re: Cluster Configuration Issues >> >> Try Patrick's utility for creating the config files and compare the result >> to your hand-made files. >> >> On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant >> wrote: >> >>> The file contains the number 1 and nothing else. My other node has the >>> number 2 (I only have 2 machines right now, I know it makes more sense to >>> run an odd number of zookeeper nodes but I just want to make sure it >> works >>> first). Any suggestions? >>> >> >> >> >> -- >> Ted Dunning, CTO >> DeepDyve >> -- Leonard Cuff lc...@valueclick.com ³This email and any files included with it may contain privileged, proprietary and/or confidential information that is for the sole use of the intended recipient(s). Any disclosure, copying, distribution, posting, or use of the information contained in or attached to this email is prohibited unless permitted by the sender. If you have received this email in error, please immediately notify the sender via return e-mail, telephone, or fax and destroy this original transmission and its included files without reading or saving it in any manner. Thank you.² This email and any files included with it may contain privileged, proprietary and/or confidential information that is for the sole use of the intended recipient(s). Any disclosure, copying, distribution, posting, or use of the information contained in or attached to this email is prohibited unless permitted by the sender. If you have received this email in error, please immediately notify the sender via return email, telephone, or fax and destroy this original transmission and its included files without reading or saving it in any manner. Thank you.
RE: Cluster Configuration Issues
Yeah I just figured out the problem with zoocfg.py I am running as the same user who created myid. Here's my config: zoo.cfg tickTime-2000 dataDir=home/mark/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1= hermes:2888:3888 server.2= leela:2888:3888 on the machines hermes and leela I've put myid files in /home/mark/zookeeper with the numbers 1 and 2 respectively -Original Message- From: Henry Robinson [mailto:he...@cloudera.com] Sent: Thursday, October 22, 2009 5:43 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Hi Mark - The Python error relates to not being able to find the zoocfg module - is zoocfg.py in the same directory as zkconf.py? Another couple of questions - are you running zookeeper as the same user who created myid? Can you post your entire configuration file please - copy and paste? Henry On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant wrote: > Before I make the Jira, I am trying to go with Ted's advice to use the > python script. > > Unfortunately I'm relatively unfamiliar with python so I'm having trouble > running it. > > When I execute "Python zkconf.py" on the command line it tells me: > Traceback (most recent call last): > File "zkconf.py", line 27, in >from zoocfg import zoocfg > ImportError: No module named zoocfg > > The same error comes when I try to call zkcfg.py from the python interface > and when I try running > Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I > gathered from the Usage). Any suggestions? > > Also, I've been using zookeeper 3.2.1 > -Original Message- > From: Ted Dunning [mailto:ted.dunn...@gmail.com] > Sent: Thursday, October 22, 2009 4:33 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: Cluster Configuration Issues > > Try Patrick's utility for creating the config files and compare the result > to your hand-made files. > > On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant > wrote: > > > The file contains the number 1 and nothing else. My other node has the > > number 2 (I only have 2 machines right now, I know it makes more sense to > > run an odd number of zookeeper nodes but I just want to make sure it > works > > first). Any suggestions? > > > > > > -- > Ted Dunning, CTO > DeepDyve >
Re: Cluster Configuration Issues
bummer, donno what to say, what version of python are you using? works for me in 2.5 and 2.6 (linux jaunty) $ ls LICENSE.txt README.html start.py start.pyc stop.py stop.pyc test3 zoocfg.py zoocfg.pyc ptestREADME.textile start.py.bak start.tmpl stop.py.bak stop.tmpl zkconf.py zoocfg.py.bak zoocfg.tmpl $ python2.5 zkconf.py ../t/zookeeper-3.2.1 ptest $ ls ptest/ cli.sh localhost:2181 localhost:2182 localhost:2183 log4j.jar log4j.properties start.sh status.sh stop.sh zookeeper.jar $ rm -fr ptest $ python2.6 zkconf.py ../t/zookeeper-3.2.1 ptest $ ls ptest/ cli.sh localhost:2181 localhost:2182 localhost:2183 log4j.jar log4j.properties start.sh status.sh stop.sh zookeeper.jar Perhaps you can try something like: PYTHONPATH=. python zkconf.py ../t/zookeeper-3.2.1 ptest that help at all? Patrick Mark Vigeant wrote: Before I make the Jira, I am trying to go with Ted's advice to use the python script. Unfortunately I'm relatively unfamiliar with python so I'm having trouble running it. When I execute "Python zkconf.py" on the command line it tells me: Traceback (most recent call last): File "zkconf.py", line 27, in from zoocfg import zoocfg ImportError: No module named zoocfg The same error comes when I try to call zkcfg.py from the python interface and when I try running Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I gathered from the Usage). Any suggestions? Also, I've been using zookeeper 3.2.1 -Original Message- From: Ted Dunning [mailto:ted.dunn...@gmail.com] Sent: Thursday, October 22, 2009 4:33 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Try Patrick's utility for creating the config files and compare the result to your hand-made files. On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant wrote: The file contains the number 1 and nothing else. My other node has the number 2 (I only have 2 machines right now, I know it makes more sense to run an odd number of zookeeper nodes but I just want to make sure it works first). Any suggestions?
Re: Cluster Configuration Issues
Hi Mark - The Python error relates to not being able to find the zoocfg module - is zoocfg.py in the same directory as zkconf.py? Another couple of questions - are you running zookeeper as the same user who created myid? Can you post your entire configuration file please - copy and paste? Henry On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant wrote: > Before I make the Jira, I am trying to go with Ted's advice to use the > python script. > > Unfortunately I'm relatively unfamiliar with python so I'm having trouble > running it. > > When I execute "Python zkconf.py" on the command line it tells me: > Traceback (most recent call last): > File "zkconf.py", line 27, in >from zoocfg import zoocfg > ImportError: No module named zoocfg > > The same error comes when I try to call zkcfg.py from the python interface > and when I try running > Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I > gathered from the Usage). Any suggestions? > > Also, I've been using zookeeper 3.2.1 > -Original Message- > From: Ted Dunning [mailto:ted.dunn...@gmail.com] > Sent: Thursday, October 22, 2009 4:33 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: Cluster Configuration Issues > > Try Patrick's utility for creating the config files and compare the result > to your hand-made files. > > On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant > wrote: > > > The file contains the number 1 and nothing else. My other node has the > > number 2 (I only have 2 machines right now, I know it makes more sense to > > run an odd number of zookeeper nodes but I just want to make sure it > works > > first). Any suggestions? > > > > > > -- > Ted Dunning, CTO > DeepDyve >
RE: Cluster Configuration Issues
Before I make the Jira, I am trying to go with Ted's advice to use the python script. Unfortunately I'm relatively unfamiliar with python so I'm having trouble running it. When I execute "Python zkconf.py" on the command line it tells me: Traceback (most recent call last): File "zkconf.py", line 27, in from zoocfg import zoocfg ImportError: No module named zoocfg The same error comes when I try to call zkcfg.py from the python interface and when I try running Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I gathered from the Usage). Any suggestions? Also, I've been using zookeeper 3.2.1 -Original Message- From: Ted Dunning [mailto:ted.dunn...@gmail.com] Sent: Thursday, October 22, 2009 4:33 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues Try Patrick's utility for creating the config files and compare the result to your hand-made files. On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant wrote: > The file contains the number 1 and nothing else. My other node has the > number 2 (I only have 2 machines right now, I know it makes more sense to > run an odd number of zookeeper nodes but I just want to make sure it works > first). Any suggestions? > -- Ted Dunning, CTO DeepDyve
Re: Cluster Configuration Issues
Try Patrick's utility for creating the config files and compare the result to your hand-made files. On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant wrote: > The file contains the number 1 and nothing else. My other node has the > number 2 (I only have 2 machines right now, I know it makes more sense to > run an odd number of zookeeper nodes but I just want to make sure it works > first). Any suggestions? > -- Ted Dunning, CTO DeepDyve
Re: Cluster Configuration Issues
I would suggest that you create a jira with the config files and your data directories attached as a compressed archive. We can take a look and pinpoint the issue. Also detail the release and command line you are using to start the cluster. https://issues.apache.org/jira/browse/ZOOKEEPER Patrick Mark Vigeant wrote: So I'm trying to get zookeeper to run again, but even though I created a file entitled "myid" and saved in the specified DataDir, the error I get when trying to run it is 2009-10-22 15:58:31, 830 FATAL [main:quorumpeerm...@82] - Invalid config, exiting abnormally Caused by: java.lang.IllegalArgumentException: home/hadoop/zookeeper/myid file is missing The file contains the number 1 and nothing else. My other node has the number 2 (I only have 2 machines right now, I know it makes more sense to run an odd number of zookeeper nodes but I just want to make sure it works first). Any suggestions? -Mark -Original Message- From: Mark Vigeant [mailto:mark.vige...@riskmetrics.com] Sent: Wednesday, October 21, 2009 9:43 AM To: zookeeper-user@hadoop.apache.org Subject: RE: Cluster Configuration Issues Thank you guys for all of your immediate help! My machines are currently down for other reasons so I'll be able to try things again, send out the errors I'm getting and such in a day or so (hopefully!) thanks again! -Mark -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Tuesday, October 20, 2009 1:55 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues You might try my ZooKeeper configuration generator if you have python handy: http://bit.ly/mBEcF The main issue that I see with your config is that each config file needs to contain a list of all the servers in the ensemble: ... syncLimit=2 server.1=host1... server.2=host2... server.3=host3... server.4=host4... where the "myid" file in the data dir for each hostX corresponds to it's server id (so myid=1 on host1, myid=2 on host2, etc...) Patrick Mark Vigeant wrote: Hey- So I'm trying to run hbase on 4 nodes, and in order to do that I need to run zookeeper in replicated mode (I could have hbase run the quorum for me, but it's suggested that I don't). I have an issue though. For some reason the id I'm assigning each server in the file "myid" in the assigned data directory is not getting read. I feel like another id is being created and put somewhere else. Does anyone have any tips on starting a zookeeper quorum? Do I create the myid file myself or do I edit one once it is created by zookeeper? This is what my config looks like: ticktime=2000 dataDir=/home/hadoop/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=hadoop1:2888:3888 The name of my machine is hadoop1, with user name hadoop. In /home/hadoop/zookeeper I've created a myid file with the number 1 in it. Mark Vigeant RiskMetrics Group, Inc.
RE: Cluster Configuration Issues
So I'm trying to get zookeeper to run again, but even though I created a file entitled "myid" and saved in the specified DataDir, the error I get when trying to run it is 2009-10-22 15:58:31, 830 FATAL [main:quorumpeerm...@82] - Invalid config, exiting abnormally Caused by: java.lang.IllegalArgumentException: home/hadoop/zookeeper/myid file is missing The file contains the number 1 and nothing else. My other node has the number 2 (I only have 2 machines right now, I know it makes more sense to run an odd number of zookeeper nodes but I just want to make sure it works first). Any suggestions? -Mark -Original Message- From: Mark Vigeant [mailto:mark.vige...@riskmetrics.com] Sent: Wednesday, October 21, 2009 9:43 AM To: zookeeper-user@hadoop.apache.org Subject: RE: Cluster Configuration Issues Thank you guys for all of your immediate help! My machines are currently down for other reasons so I'll be able to try things again, send out the errors I'm getting and such in a day or so (hopefully!) thanks again! -Mark -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Tuesday, October 20, 2009 1:55 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues You might try my ZooKeeper configuration generator if you have python handy: http://bit.ly/mBEcF The main issue that I see with your config is that each config file needs to contain a list of all the servers in the ensemble: ... syncLimit=2 server.1=host1... server.2=host2... server.3=host3... server.4=host4... where the "myid" file in the data dir for each hostX corresponds to it's server id (so myid=1 on host1, myid=2 on host2, etc...) Patrick Mark Vigeant wrote: > Hey- > > So I'm trying to run hbase on 4 nodes, and in order to do that I need to run > zookeeper in replicated mode (I could have hbase run the quorum for me, but > it's suggested that I don't). > > I have an issue though. For some reason the id I'm assigning each server in > the file "myid" in the assigned data directory is not getting read. I feel > like another id is being created and put somewhere else. Does anyone have any > tips on starting a zookeeper quorum? Do I create the myid file myself or do I > edit one once it is created by zookeeper? > > This is what my config looks like: > ticktime=2000 > dataDir=/home/hadoop/zookeeper > clientPort=2181 > initLimit=5 > syncLimit=2 > server.1=hadoop1:2888:3888 > > The name of my machine is hadoop1, with user name hadoop. In > /home/hadoop/zookeeper I've created a myid file with the number 1 in it. > > Mark Vigeant > RiskMetrics Group, Inc. > >
RE: Cluster Configuration Issues
Thank you guys for all of your immediate help! My machines are currently down for other reasons so I'll be able to try things again, send out the errors I'm getting and such in a day or so (hopefully!) thanks again! -Mark -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Tuesday, October 20, 2009 1:55 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Cluster Configuration Issues You might try my ZooKeeper configuration generator if you have python handy: http://bit.ly/mBEcF The main issue that I see with your config is that each config file needs to contain a list of all the servers in the ensemble: ... syncLimit=2 server.1=host1... server.2=host2... server.3=host3... server.4=host4... where the "myid" file in the data dir for each hostX corresponds to it's server id (so myid=1 on host1, myid=2 on host2, etc...) Patrick Mark Vigeant wrote: > Hey- > > So I'm trying to run hbase on 4 nodes, and in order to do that I need to run > zookeeper in replicated mode (I could have hbase run the quorum for me, but > it's suggested that I don't). > > I have an issue though. For some reason the id I'm assigning each server in > the file "myid" in the assigned data directory is not getting read. I feel > like another id is being created and put somewhere else. Does anyone have any > tips on starting a zookeeper quorum? Do I create the myid file myself or do I > edit one once it is created by zookeeper? > > This is what my config looks like: > ticktime=2000 > dataDir=/home/hadoop/zookeeper > clientPort=2181 > initLimit=5 > syncLimit=2 > server.1=hadoop1:2888:3888 > > The name of my machine is hadoop1, with user name hadoop. In > /home/hadoop/zookeeper I've created a myid file with the number 1 in it. > > Mark Vigeant > RiskMetrics Group, Inc. > >
Re: Cluster Configuration Issues
You might try my ZooKeeper configuration generator if you have python handy: http://bit.ly/mBEcF The main issue that I see with your config is that each config file needs to contain a list of all the servers in the ensemble: ... syncLimit=2 server.1=host1... server.2=host2... server.3=host3... server.4=host4... where the "myid" file in the data dir for each hostX corresponds to it's server id (so myid=1 on host1, myid=2 on host2, etc...) Patrick Mark Vigeant wrote: Hey- So I'm trying to run hbase on 4 nodes, and in order to do that I need to run zookeeper in replicated mode (I could have hbase run the quorum for me, but it's suggested that I don't). I have an issue though. For some reason the id I'm assigning each server in the file "myid" in the assigned data directory is not getting read. I feel like another id is being created and put somewhere else. Does anyone have any tips on starting a zookeeper quorum? Do I create the myid file myself or do I edit one once it is created by zookeeper? This is what my config looks like: ticktime=2000 dataDir=/home/hadoop/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=hadoop1:2888:3888 The name of my machine is hadoop1, with user name hadoop. In /home/hadoop/zookeeper I've created a myid file with the number 1 in it. Mark Vigeant RiskMetrics Group, Inc.
Re: Cluster Configuration Issues
HI Mark, ZooKeeper does not create the myid file in the data directory. Looking at the config file it looks like it is missing the quorum configuration for other servers. Please take alook at http://hadoop.apache.org/zookeeper/docs/r3.2.1/zookeeperAdmin.html#sc_zkMuli tServerSetup You will need to add config options for other servers in the quorum in the config file. Thanks mahadev On 10/20/09 10:12 AM, "Mark Vigeant" wrote: > Hey- > > So I'm trying to run hbase on 4 nodes, and in order to do that I need to run > zookeeper in replicated mode (I could have hbase run the quorum for me, but > it's suggested that I don't). > > I have an issue though. For some reason the id I'm assigning each server in > the file "myid" in the assigned data directory is not getting read. I feel > like another id is being created and put somewhere else. Does anyone have any > tips on starting a zookeeper quorum? Do I create the myid file myself or do I > edit one once it is created by zookeeper? > > This is what my config looks like: > ticktime=2000 > dataDir=/home/hadoop/zookeeper > clientPort=2181 > initLimit=5 > syncLimit=2 > server.1=hadoop1:2888:3888 > > The name of my machine is hadoop1, with user name hadoop. In > /home/hadoop/zookeeper I've created a myid file with the number 1 in it. > > Mark Vigeant > RiskMetrics Group, Inc. >
Re: Cluster Configuration Issues
Hi Mark - You should create the myid file yourself, as you have done. What errors are you seeing that lead you to think the id is not being read correctly? cheers, Henry On Tue, Oct 20, 2009 at 10:12 AM, Mark Vigeant wrote: > Hey- > > So I'm trying to run hbase on 4 nodes, and in order to do that I need to > run zookeeper in replicated mode (I could have hbase run the quorum for me, > but it's suggested that I don't). > > I have an issue though. For some reason the id I'm assigning each server > in the file "myid" in the assigned data directory is not getting read. I > feel like another id is being created and put somewhere else. Does anyone > have any tips on starting a zookeeper quorum? Do I create the myid file > myself or do I edit one once it is created by zookeeper? > > This is what my config looks like: > ticktime=2000 > dataDir=/home/hadoop/zookeeper > clientPort=2181 > initLimit=5 > syncLimit=2 > server.1=hadoop1:2888:3888 > > The name of my machine is hadoop1, with user name hadoop. In > /home/hadoop/zookeeper I've created a myid file with the number 1 in it. > > Mark Vigeant > RiskMetrics Group, Inc. > >
