Passing Custom Arguments for giraph.zkList
Hi, I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception: 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration Exception in thread main java.lang.IllegalArgumentException: Unable to parse custom argument: zk2:port at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288) at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147) at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197) I pass the zklist like this: Hadoop jar GRAPH.jar CLASSNAME -vip CLASS NAME -vif CLASS NAME -wc CLASS NAME -w worker number -ca giraph.zkList=zk1:port,zk2:port,zk3:port,zk4:port,zk5:port Please suggest what is wrong with this invocation. Thanks Arun Ramani
Re: Passing Custom Arguments for giraph.zkList
zk1 is supposed to be a hostname. On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun aram...@paypal.com wrote: Hi, I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception: 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration Exception in thread main java.lang.IllegalArgumentException: Unable to parse custom argument: zk2:port at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288) at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147) at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197) I pass the zklist like this: Hadoop jar GRAPH.jar CLASSNAME -vip CLASS NAME -vif CLASS NAME -wc CLASS NAME -w worker number -ca giraph.zkList=zk1:port,zk2:port,zk3:port,zk4:port,zk5:port Please suggest what is wrong with this invocation. Thanks Arun Ramani -- Claudio Martella claudio.marte...@gmail.com
Re: Passing Custom Arguments for giraph.zkList
Hi Claudio, Yes zk1, zk2, zk3, zk4 and zk5 are all zookeeper hostnames. These 5 hosts make a zookeeper quorum. Please let me know how to pass this. Thanks Arun Ramani From: Claudio Martella claudio.marte...@gmail.commailto:claudio.marte...@gmail.com Reply-To: user@giraph.apache.orgmailto:user@giraph.apache.org user@giraph.apache.orgmailto:user@giraph.apache.org Date: Thursday, August 29, 2013 2:18 PM To: user@giraph.apache.orgmailto:user@giraph.apache.org user@giraph.apache.orgmailto:user@giraph.apache.org Subject: Re: Passing Custom Arguments for giraph.zkList zk1 is supposed to be a hostname. On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun aram...@paypal.commailto:aram...@paypal.com wrote: Hi, I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception: 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration Exception in thread main java.lang.IllegalArgumentException: Unable to parse custom argument: zk2:port at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288) at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147) at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197) I pass the zklist like this: Hadoop jar GRAPH.jar CLASSNAME -vip CLASS NAME -vif CLASS NAME -wc CLASS NAME -w worker number -ca giraph.zkList=zk1:port,zk2:port,zk3:port,zk4:port,zk5:port Please suggest what is wrong with this invocation. Thanks Arun Ramani -- Claudio Martella claudio.marte...@gmail.commailto:claudio.marte...@gmail.com
Re: Passing Custom Arguments for giraph.zkList
the problem is not the format of the string, but the way you're passing it. Try passing it as -D giraph.zkList=... before the giraphrunner options. that should work. On Thu, Aug 29, 2013 at 11:47 PM, Ramani, Arun aram...@paypal.com wrote: Hi Claudio, Yes zk1, zk2, zk3, zk4 and zk5 are all zookeeper hostnames. These 5 hosts make a zookeeper quorum. Please let me know how to pass this. Thanks Arun Ramani From: Claudio Martella claudio.marte...@gmail.com Reply-To: user@giraph.apache.org user@giraph.apache.org Date: Thursday, August 29, 2013 2:18 PM To: user@giraph.apache.org user@giraph.apache.org Subject: Re: Passing Custom Arguments for giraph.zkList zk1 is supposed to be a hostname. On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun aram...@paypal.com wrote: Hi, I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception: 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one. 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration Exception in thread main java.lang.IllegalArgumentException: Unable to parse custom argument: zk2:port at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288) at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147) at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197) I pass the zklist like this: Hadoop jar GRAPH.jar CLASSNAME -vip CLASS NAME -vif CLASS NAME -wc CLASS NAME -w worker number -ca giraph.zkList=zk1:port,zk2:port,zk3:port,zk4:port,zk5:port Please suggest what is wrong with this invocation. Thanks Arun Ramani -- Claudio Martella claudio.marte...@gmail.com -- Claudio Martella claudio.marte...@gmail.com
