Hey Guys, I was able to reproduce this problem.
I was also able to fix it (without the patch in SAMZA-182). All I needed to do was update ~/.bashrc on my NM's box to have: export YARN_HOME=/tmp/hadoop-2.3.0 It appears that the YARN environment variables are somehow getting lost or not forwarded from the NM to the AM. Adding this bashrc setting makes sure that the NM gets them. I have a feeling upgrading Samza to YARN 2.3.0 will fix this, but I haven't validated yet. I will continue to investigate tomorrow. Cheers, Chris On 3/12/14 6:43 PM, "Yan Fang" <[email protected]> wrote: >I guess Sonali has the problem is because his NMs do not read the >YARN_HOME variable. That may be because the NM machine does not have >YARN_HOME set when the NM starts. > >Check this https://issues.apache.org/jira/browse/SAMZA-182 > >Thanks, > >Yan Fang > >> On Mar 12, 2014, at 6:14 PM, Chris Riccomini <[email protected]> >>wrote: >> >> Hey Sonali, >> >> I am unfamiliar with the start-yarn.sh. Looking at: >> >> >> >>https://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-yarn-project/had >>oo >> p-yarn/bin/stop-yarn.sh?revision=1370666&view=markup >> >> What version of YARN are you using? >> >> Cheers, >> Chris >> >> On 3/12/14 5:56 PM, "[email protected]" >> <[email protected]> wrote: >> >>> Hey Chris, >>> >>> Yes, I have YARN_HOME set in all the NMs pointing to the right >>> directories. I also made sure the yarn-site.xml file has the hostname >>>set. >>> >>> I start yarn using start.yarn.sh in the RM and that automatically >>>starts >>> the NMs on the slave nodes. Is that the right way to do it? >>> >>> -----Original Message----- >>> From: Chris Riccomini [mailto:[email protected]] >>> Sent: Wednesday, March 12, 2014 5:52 PM >>> To: [email protected] >>> Subject: Re: Failed to package using mvn >>> >>> Hey Sonali, >>> >>> OK, so we've validated that the NMs are able to connect, which means >>>they >>> can see the yarn-site.xml. >>> >>> How are you starting your NMs? Are you running: >>> >>> export YARN_HOME=/path/to/yarn/home >>> >>> In the CLI before starting the NM? >>> >>> For reference, we run: >>> >>> export YARN_HOME=/path/to/our/yarn-home >>> export YARN_CONF_DIR=$YARN_HOME/conf >>> >>> bin/yarn nodemanager >>> >>> With YARN_HOME pointing to a directory that has a subdirectory called >>> "conf" in it, which has a yarn-site.xml in it: >>> >>> /path/to/our/yarn-home/conf/yarn-site.xml >>> >>> This yarn-site.xml has yarn.resourcemanager.hostname set to the IP (or >>> hostname) of the resource manager: >>> >>> <property> >>> <name>yarn.resourcemanager.hostname</name> >>> <value>123.456.789.123</value> >>> </property> >>> >>> >>> Cheers, >>> Chris >>> >>> On 3/12/14 5:33 PM, "[email protected]" >>> <[email protected]> wrote: >>> >>>> I see two active nodes (I have 2 NMs running) >>>> >>>> -----Original Message----- >>>> From: Chris Riccomini [mailto:[email protected]] >>>> Sent: Wednesday, March 12, 2014 5:24 PM >>>> To: [email protected] >>>> Subject: Re: Failed to package using mvn >>>> >>>> Hey Sonali, >>>> >>>> Can you go to your ResourceManager's UI, and tell me how many active >>>> nodes you see? This should be under the "active nodes" heading. >>>> >>>> It sounds like the SamzaAppMaster is not getting the resource manager >>>> host/port from the yarn-site.xml. Usually this is due to not exporting >>>> YARN_HOME on the NodeManager before starting it. >>>> >>>> Cheers, >>>> Chris >>>> >>>> On 3/12/14 5:21 PM, "[email protected]" >>>> <[email protected]> wrote: >>>> >>>>> Okay so I was able to submit the job: >>>>> >>>>> In the nodemanager I get this error: Specifically it's trying to >>>>> connect to 0.0.0.0/8032 instead of the IP I have specified in the >>>>> yarn-site.xml file >>>>> >>>>> 2014-03-12 17:04:47 SamzaAppMaster$ [INFO] got container id: >>>>> container_1391637982288_0033_01_000001 >>>>> 2014-03-12 17:04:47 SamzaAppMaster$ [INFO] got app attempt id: >>>>> appattempt_1391637982288_0033_000001 >>>>> 2014-03-12 17:04:47 SamzaAppMaster$ [INFO] got node manager host: >>>>> svdpdac001.techlabs.accenture.com >>>>> 2014-03-12 17:04:47 SamzaAppMaster$ [INFO] got node manager port: >>>>> 38218 >>>>> 2014-03-12 17:04:47 SamzaAppMaster$ [INFO] got node manager http >>>>>port: >>>>> 8042 >>>>> 2014-03-12 17:04:47 SamzaAppMaster$ [INFO] got config: >>>>> >>>>>{task.inputs=wikipedia.#en.wikipedia,wikipedia.#en.wiktionary,wikipedi >>>>> a .#e n.wikinews, systems.wikipedia.host=irc.wikimedia.org, >>>>> systems.kafka.producer.batch.num.messages=1, >>>>> job.factory.class=org.apache.samza.job.yarn.YarnJobFactory, >>>>> systems.wikipedia.port=6667, >>>>> systems.kafka.producer.producer.type=sync, >>>>> job.name=wikipedia-feed, >>>>> >>>>>systems.kafka.consumer.zookeeper.connect=svdpdac013.techlabs.accenture >>>>>. >>>>> com :2181/, systems.kafka.samza.msg.serde=json, >>>>> >>>>>serializers.registry.json.class=org.apache.samza.serializers.JsonSerde >>>>> F >>>>> act ory, >>>>> task.class=samza.examples.wikipedia.task.WikipediaFeedStreamTask, >>>>> >>>>>yarn.package.path=hdfs://10.1.174.85:9000/samza-job-package-0.7.0-dist >>>>>. >>>>> tar >>>>> .gz, >>>>> >>>>>systems.wikipedia.samza.factory=samza.examples.wikipedia.system.Wikipe >>>>> d >>>>> iaS >>>>> ystemFactory, >>>>> >>>>>systems.kafka.samza.factory=org.apache.samza.system.kafka.KafkaSystemF >>>>> a >>>>> cto >>>>> ry, >>>>> >>>>>systems.kafka.producer.metadata.broker.list=svdpdac001.techlabs.accent >>>>> ure >>>>> . >>>>> com:6667,svdpdac015.techlabs.accenture.com:6667} >>>>> 2014-03-12 17:04:48 ClientHelper [INFO] trying to connect to RM >>>>> 0.0.0.0:8032 >>>>> 2014-03-12 17:04:48 NativeCodeLoader [WARN] Unable to load >>>>> native-hadoop library for your platform... using builtin-java classes >>>>> where applicable >>>>> 2014-03-12 17:04:48 RMProxy [INFO] Connecting to ResourceManager at >>>>> /0.0.0.0:8032 >>>>> >>>>> -----Original Message----- >>>>> From: Chris Riccomini [mailto:[email protected]] >>>>> Sent: Wednesday, March 12, 2014 4:48 PM >>>>> To: [email protected] >>>>> Subject: Re: Failed to package using mvn >>>>> >>>>> Hey Sonali, >>>>> >>>>> You need to specify a valid HDFS uri. Usually something like: >>>>> >>>>> hdfs://<hdfs name node ip>:<hdfs name node port>/path/to/tgz >>>>> >>>>> Right now, Hadoop is trying to use the package name as the HDFS host. >>>>> >>>>> Cheers, >>>>> Chris >>>>> >>>>> On 3/12/14 4:45 PM, "[email protected]" >>>>> <[email protected]> wrote: >>>>> >>>>>> I did and I can now see the hadoop-hdfs jar in /deploy/samza/lib >>>>>> folder. >>>>>> >>>>>> I do get a different error now. >>>>>> >>>>>> I uploaded the samza-job to hdfs and it resides on >>>>>> hdfs://samza-job-package-0.7.0-dist.tar.gz >>>>>> >>>>>> But when I run the job I get this exception: >>>>>> >>>>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>>>> java.net.UnknownHostException: samza-job-package-0.7.0-dist.tar.gz >>>>>> at >>>>>> >>>>>>org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUti >>>>>>l. >>>>>> jav >>>>>> a:418) >>>>>> at >>>>>> >>>>>>org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxi >>>>>> e >>>>>> s >>>>>> .ja >>>>>> va:231) >>>>>> at >>>>>> >>>>>>org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.ja >>>>>> v >>>>>> a >>>>>> :13 >>>>>> 9) >>>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:510) >>>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:453) >>>>>> at >>>>>> >>>>>>org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFi >>>>>> l >>>>>> e >>>>>> Sys >>>>>> tem.java:136) >>>>>> at >>>>>> >>>>>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2433 >>>>>>) >>>>>> at >>>>>> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88) >>>>>> at >>>>>> >>>>>>org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:246 >>>>>>7) >>>>>> at >>>>>> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449) >>>>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) >>>>>> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287) >>>>>> at >>>>>> >>>>>>org.apache.samza.job.yarn.ClientHelper.submitApplication(ClientHelper >>>>>>. >>>>>> s >>>>>> cal >>>>>> a:111) >>>>>> at org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:55) >>>>>> at org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:48) >>>>>> at org.apache.samza.job.JobRunner.run(JobRunner.scala:100) >>>>>> at org.apache.samza.job.JobRunner$.main(JobRunner.scala:75) >>>>>> at org.apache.samza.job.JobRunner.main(JobRunner.scala) >>>>>> Caused by: java.net.UnknownHostException: >>>>>> samza-job-package-0.7.0-dist.tar.gz >>>>>> ... 18 more >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Yan Fang [mailto:[email protected]] >>>>>> Sent: Wednesday, March 12, 2014 4:20 PM >>>>>> To: [email protected] >>>>>> Subject: Re: Failed to package using mvn >>>>>> >>>>>> Hi Sonali, >>>>>> >>>>>> One tip you may miss: >>>>>> >>>>>> If you had already run >>>>>> >>>>>> tar -xvf >>>>>> ./samza-job-package/target/samza-job-package-0.7.0-dist.tar.gz >>>>>> -C deploy/samza >>>>>> >>>>>> before you bundled the jar file to tar.gz. Please also remember to >>>>>> put the hdfs jar file to the deploy/samza/lib. >>>>>> >>>>>> Let me know if you miss this step. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Fang, Yan >>>>>> [email protected] >>>>>> +1 (206) 849-4108 >>>>>> >>>>>> >>>>>> On Wed, Mar 12, 2014 at 4:10 PM, Chris Riccomini >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Hey Sonali, >>>>>>> >>>>>>> Yan has made a step-by-step tutorial for this. Could you confirm >>>>>>> that you've followed the instructions, and it's still not working? >>>>>>> >>>>>>> https://issues.apache.org/jira/browse/SAMZA-181 >>>>>>> >>>>>>> Cheers, >>>>>>> Chris >>>>>>> >>>>>>> On 3/12/14 3:12 PM, "[email protected]" >>>>>>> <[email protected]> wrote: >>>>>>> >>>>>>>> So sigh! I had some Kafka issues in-between. That's fixed now. >>>>>>>> >>>>>>>> As suggested, >>>>>>>> >>>>>>>> 1. I made sure the hadoop-hdfs-2.2.0.jar is bundled with the samza >>>>>>>> job tar.gz. >>>>>>>> 2. I added the configuration to implement hdfs in the >>>>>>>> hdfs-site.xml files both on the NMs and in the /conf directory for >>>>>>>> samza >>>>>>>> >>>>>>>> I still get the No Filesystem for scheme :hdfs error. >>>>>>>> >>>>>>>> Is there anything else im missing? >>>>>>>> Thanks, >>>>>>>> Sonali >>>>>>>> >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Chris Riccomini [mailto:[email protected]] >>>>>>>> Sent: Tuesday, March 11, 2014 8:27 PM >>>>>>>> To: [email protected] >>>>>>>> Subject: Re: Failed to package using mvn >>>>>>>> >>>>>>>> Hey Yan, >>>>>>>> >>>>>>>> This looks great! I added a few requests to the JIRA, if you have >>>>>>> time. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Chris >>>>>>>> >>>>>>>>> On 3/11/14 7:20 PM, "Yan Fang" <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Hi Chris, >>>>>>>>> >>>>>>>>> Has opened an issue >>>>>>>>> SAMZA-181<https://issues.apache.org/jira/browse/SAMZA-181>and >>>>>>>>> also uploaded the patch. Let me know if there is something wrong >>>>>>>>> in my tutorial. Thank you! >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>> Fang, Yan >>>>>>>>> [email protected] >>>>>>>>> +1 (206) 849-4108 >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Mar 11, 2014 at 10:40 AM, >>>>>>>>> <[email protected]>wrote: >>>>>>>>> >>>>>>>>>> Thanks Chris, Yan, >>>>>>>>>> >>>>>>>>>> Let me try that. >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Chris Riccomini [mailto:[email protected]] >>>>>>>>>> Sent: Tuesday, March 11, 2014 10:22 AM >>>>>>>>>> To: [email protected] >>>>>>>>>> Subject: Re: Failed to package using mvn >>>>>>>>>> >>>>>>>>>> Hey Yan, >>>>>>>>>> >>>>>>>>>> Awesome!The location where you can add your .md is here: >>>>>>>>>> >>>>>>>>>> docs/learn/tutorials/0.7.0/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here's a link to the code tree: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-samza.git;a= >>>>>>>>>> t >>>>>>>>>> r >>>>>>>>>> e >>>>>>>>>> e;f >>>>>>>>>> =do >>>>>>>>>> cs >>>>>>>>>> >>>>>>>>>> /learn/tutorials/0.7.0;h=ef117f4066f14a00f50f0f6fca1790313044831 >>>>>>>>>> 2 >>>>>>>>>> ; >>>>>>>>>> h >>>>>>>>>> b=H >>>>>>>>>> EAD >>>>>>>>>> >>>>>>>>>> You can get the code here: >>>>>>>>>> >>>>>>>>>> git clone >>>>>>>>>> http://git-wip-us.apache.org/repos/asf/incubator-samza.git >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Once you write the .md, just throw it up on a JIRA, and one of >>>>>>>>>> us can merge it in. >>>>>>>>>> >>>>>>>>>> Re: hdfs-site.xml, ah ha, that's what I figured. This is good >>>>>>>>>> to >>>>>>> know. >>>>>>>>>> So >>>>>>>>>> you just copy your hdfs-site.xml from your NodeManager's conf >>>>>>>>>> directory into your local hdfs-site.xml. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>>> On 3/11/14 10:16 AM, "Yan Fang" <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Chris, >>>>>>>>>>> >>>>>>>>>>> Sure. I just do not know how/where to contribute this >>>>>>>>>>> page...*_* >>>>>>>>>>> >>>>>>>>>>> Oh, I mean the same this as you mentioned in the *Cluster >>>>>>>>>>> Installation*thread: >>>>>>>>>>> >>>>>>>>>>> *"2. Get a copy of one of your NM's yarn-site.xml and put it >>>>>>>>>>> somewhere >>>>>>>>>>> on* >>>>>>>>>>> >>>>>>>>>>> *your desktop (I usually use ~/.yarn/conf/yarn-site.xml). Note >>>>>>>>>>> that there'sa "conf" directory there. This is mandatory."* >>>>>>>>>>> >>>>>>>>>>> So I just copy the hdfs-site.xml to ~/.yarn/conf/hdfs-site.xml. >>>>>>>>>>> Thank >>>>>>>>>> you. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> >>>>>>>>>>> Fang, Yan >>>>>>>>>>> [email protected] >>>>>>>>>>> +1 (206) 849-4108 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Mar 11, 2014 at 10:10 AM, Chris Riccomini >>>>>>>>>>> <[email protected]>wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey Yan, >>>>>>>>>>>> >>>>>>>>>>>> Would you be up for contributing a tutorial page that >>>>>>>>>>>> describes >>>>>>>>>> this? >>>>>>>>>>>> This >>>>>>>>>>>> is really useful information. Our docs are just simple .md >>>>>>>>>>>> files in the main code base. >>>>>>>>>>>> >>>>>>>>>>>> Regarding step (3), is the hdfs-site.xml put into the conf >>>>>>>>>>>> folder for the NM boxes, or on the client side (where >>>>>>>>>>>> run-job.sh >>>>>>> is run)? >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Chris >>>>>>>>>>>> >>>>>>>>>>>>> On 3/11/14 10:07 AM, "Yan Fang" <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Sonali, >>>>>>>>>>>>> >>>>>>>>>>>>> The way I make Samza run with HDFS is following: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. include hdfs jar in Samza jar tar.gz. >>>>>>>>>>>>> 2. you may also want to make sure the hadoop-common.jar has >>>>>>>>>>>>> the same version as your hdfs jar. Otherwise, you may have >>>>>>>>>>>>> configuration error popping out. >>>>>>>>>>>>> 3. then put hdfs-site.xml to conf folder, the same folder >>>>>>>>>>>>> as the yarn-site.xml 4. all other steps are not changed. >>>>>>>>>>>>> >>>>>>>>>>>>> Hope this will help. Thank you. >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers, >>>>>>>>>>>>> >>>>>>>>>>>>> Fang, Yan >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> +1 (206) 849-4108 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Mar 11, 2014 at 9:25 AM, Chris Riccomini >>>>>>>>>>>>> <[email protected]>wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hey Sonali, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I believe that you need to make sure that the HDFS jar is >>>>>>>>>>>>>> in your .tar.gz file, as you've said. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If that doesn't work, you might need to define this >>>>>>>>>>>>>> setting in core-site.xml on the machine you're running >>>>>>> run-job.sh on: >>>>>>>>>>>>>> >>>>>>>>>>>>>> <property> >>>>>>>>>>>>>> <name>fs.hdfs.impl</name> >>>>>>> <value>org.apache.hadoop.hdfs.DistributedFileSystem</value> >>>>>>>>>>>>>> <description>The FileSystem for hdfs: >>>>>>>>>>>>>> uris.</description> </property> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> You might also need to configure your NodeManagers to >>>>>>>>>>>>>> have the HDFS >>>>>>>>>>>> file >>>>>>>>>>>>>> system impl as well. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I've never run Samza with HDFS, so I'm guessing here. >>>>>>>>>>>>>> Perhaps someone else on the list has been successful with >>>>>>> this? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>> Chris >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 3/10/14 3:59 PM, "[email protected]" >>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I fixed this by starting from scratch with gradlew. But >>>>>>>>>>>>>>> now when I >>>>>>>>>>>> run >>>>>>>>>>>>>> my >>>>>>>>>>>>>>> job it throws this error: >>>>>>>>>>>>>>> Exception in thread "main" java.io.IOException: No >>>>>>>>>>>>>>> FileSystem for >>>>>>>>>>>>>> scheme: >>>>>>>>>>>>>>> hdfs >>>>>>>>>>>>>>> at >>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSyste >>>>>>>>>>>>>>>m. >>>>>>>>>>>>>>> jav >>>>>>>>>>>>>>> a: >>>>>>>>>>>>>>> 242 >>>>>>>>>>>>>>> 1) >>>>>>>>>>>>>>> at >>>>>>>>>>>>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem. >>>>>>>>>>>>> j >>>>>>>>>>>>> a >>>>>>>>>>>>> v >>>>>>>>>>>>> a:2 >>>>>>>>>>>>> 428 >>>>>>>>>>>>> ) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java: >>>>>>>>>>>>>> 8 >>>>>>>>>>>>>> 8 >>>>>>>>>>>>>> ) >>>>>>>>>>>>>>> at >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java >>>>>>>>>>: >>>>>>>>>>>>> 246 >>>>>>>>>>>>> 7) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java: >>>>>>>>>>>>>> 2 >>>>>>>>>>>>>> 4 >>>>>>>>>>>>>> 4 >>>>>>>>>>>>>> 9) >>>>>>>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) >>>>>>>>>>>>>>> at >>>>>>>>>> org.apache.hadoop.fs.Path.getFileSystem(Path.java:287) >>>>>>>>>>>>>>> at >>>>>>>>>> >>>>>>>>>>>>>>> org.apache.samza.job.yarn.ClientHelper.submitApplication(Cl >>>>>>>>>>>>>>> i >>>>>>>>>>>>>>> e >>>>>>>>>>>>>>> n >>>>>>>>>>>>>>> tHe >>>>>>>>>>>>>>> lpe >>>>>>>>>>>>>>> r. >>>>>>>>>>>>>>> sc >>>>>>>>>>>>>>> al >>>>>>>>>>>>>>> a:111) >>>>>>>>>>>>>>> at >>>>>>>>>>>> org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:55) >>>>>>>>>>>>>>> at >>>>>>>>>>>> org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:48) >>>>>>>>>>>>>>> at >>>>>>>>>> org.apache.samza.job.JobRunner.run(JobRunner.scala:100) >>>>>>>>>>>>>>> at >>>>>>>>>> org.apache.samza.job.JobRunner$.main(JobRunner.scala:75) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.samza.job.JobRunner.main(JobRunner.scala) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I looked at the samza job tar.gz and it doesn't have a >>>>>>>>>>>>>>> Hadoop-hdfs >>>>>>>>>>>> jar. >>>>>>>>>>>>>>> Is that why I get this error? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Sonali >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> From: Parthasarathy, Sonali >>>>>>>>>>>>>>> Sent: Monday, March 10, 2014 11:25 AM >>>>>>>>>>>>>>> To: [email protected] >>>>>>>>>>>>>>> Subject: Failed to package using mvn >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> When I tried to do a mvn clean package of my hello-samza >>>>>>>>>>>>>>> project, I >>>>>>>>>>>> get >>>>>>>>>>>>>>> the following error. Has anyone seen this before? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project samza-wikipedia: >>>>>>>>>>>>>>> Could not resolve dependencies for project >>>>>>>>>> samza:samza-wikipedia:jar:0.7.0: >>>>>>>>>>>> Could >>>>>>>>>>>>>>> not find artifact >>>>>>>>>>>>>>> org.apache.samza:samza-kv_2.10:jar:0.7.0 >>>>>>>>>>>>>>> in apache-releases >>>>>>>>>>>> (https://repository.apache.org/content/groups/public) >>>>>>>>>>>>>> -> >>>>>>>>>>>>>>> [Help 1] >>>>>>>>>>>>>>> [ERROR] >>>>>>>>>>>>>>> [ERROR] To see the full stack trace of the errors, >>>>>>>>>>>>>>> re-run Maven with >>>>>>>>>>>>>> the >>>>>>>>>>>>>>> -e switch. >>>>>>>>>>>>>>> [ERROR] Re-run Maven using the -X switch to enable full >>>>>>>>>>>>>>> debug >>>>>>>>>>>> logging. >>>>>>>>>>>>>>> [ERROR] >>>>>>>>>>>>>>> [ERROR] For more information about the errors and >>>>>>>>>>>>>>> possible >>>>>>>>>>>> solutions, >>>>>>>>>>>>>>> please read the following articles: >>>>>>>>>>>>>>> [ERROR] [Help 1] >>>>>>>>>>>> >>>>>>>>>>>> http://cwiki.apache.org/confluence/display/MAVEN/DependencyRe >>>>>>>>>>>> s >>>>>>>>>>>> o >>>>>>>>>>>> l >>>>>>>>>>>> uti >>>>>>>>>>>> onE >>>>>>>>>>>> xce >>>>>>>>>>>>>> p >>>>>>>>>>>>>>> tion >>>>>>>>>>>>>>> [ERROR] >>>>>>>>>>>>>>> [ERROR] After correcting the problems, you can resume >>>>>>>>>>>>>>> the build with >>>>>>>>>>>>>> the >>>>>>>>>>>>>>> command >>>>>>>>>>>>>>> [ERROR] mvn <goals> -rf :samza-wikipedia >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Sonali >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sonali Parthasarathy >>>>>>>>>>>>>>> R&D Developer, Data Insights Accenture Technology Labs >>>>>>>>>>>>>>> 703-341-7432 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ________________________________ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This message is for the designated recipient only and >>>>>>>>>>>>>>> may contain privileged, proprietary, or otherwise >>>>>>>>>>>>>>> confidential >>>>>>>>>> information. >>>>>>>>>>>>>>> If >>>>>>>>>>>> you >>>>>>>>>>>>>>> have received it in error, please notify the sender >>>>>>>>>>>>>>> immediately and delete the original. Any other use of >>>>>>>>>>>>>>> the e-mail by you is >>>>>>>>>>>> prohibited. >>>>>>>>>>>>>>> Where allowed by local law, electronic communications >>>>>>>>>>>>>>> with Accenture >>>>>>>>>>>>>> and >>>>>>>>>>>>>>> its affiliates, including e-mail and instant messaging >>>>>>>>>>>>>>> (including content), may be scanned by our systems for >>>>>>>>>>>>>>> the purposes of >>>>>>>>>>>> information >>>>>>>>>>>>>>> security and assessment of internal compliance with >>>>>>>>>>>>>>> Accenture >>>>>>>>>>>> policy. >>>>>>>>>>>> >>>>>>>>>>>>>>> __________________________________________________________ >>>>>>>>>>>>>>> _ >>>>>>>>>>>>>>> _ >>>>>>>>>>>>>>> _ >>>>>>>>>>>>>>> ___ >>>>>>>>>>>>>>> ___ >>>>>>>>>>>>>>> ___ >>>>>>>>>>>>>>> __ >>>>>>>>>>>>>>> __ >>>>>>>>>>>>>>> ____________ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> www.accenture.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ________________________________ >>>>>>>>>> >>>>>>>>>> This message is for the designated recipient only and may >>>>>>>>>> contain privileged, proprietary, or otherwise confidential >>>>>>>>>> information. If you have received it in error, please notify >>>>>>>>>> the sender immediately and delete the original. Any other use >>>>>>>>>> of the e-mail by you is prohibited. Where allowed by local law, >>>>>>>>>> electronic communications with Accenture and its affiliates, >>>>>>>>>> including e-mail and instant messaging (including content), may >>>>>>>>>> be scanned by our systems for the purposes of information >>>>>>>>>> security and assessment of internal compliance with Accenture >>>>>>> policy. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ________________________________________________________________ >>>>>>>>>> _ >>>>>>>>>> _ >>>>>>>>>> _ >>>>>>>>>> ___ >>>>>>>>>> ___ >>>>>>>>>> _____________ >>>>>>>>>> >>>>>>>>>> www.accenture.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ________________________________ >>>>>>>> >>>>>>>> This message is for the designated recipient only and may contain >>>>>>>> privileged, proprietary, or otherwise confidential information. If >>>>>>>> you have received it in error, please notify the sender >>>>>>>> immediately and delete the original. Any other use of the e-mail >>>>>>>> by you is >>>>>>> prohibited. >>>>>>>> Where allowed by local law, electronic communications with >>>>>>>> Accenture and its affiliates, including e-mail and instant >>>>>>>> messaging (including content), may be scanned by our systems for >>>>>>>> the purposes of information security and assessment of internal >>>>>>>> compliance with >>>>>>> Accenture policy. >>>>>>>> __________________________________________________________________ >>>>>>>> _ >>>>>>>> _ >>>>>>>> _ >>>>>>>> _____ >>>>>>>> ____________ >>>>>>>> >>>>>>>> www.accenture.com >>>>>> >>>>>> ________________________________ >>>>>> >>>>>> This message is for the designated recipient only and may contain >>>>>> privileged, proprietary, or otherwise confidential information. If >>>>>> you have received it in error, please notify the sender immediately >>>>>> and delete the original. Any other use of the e-mail by you is >>>>>> prohibited. >>>>>> Where allowed by local law, electronic communications with Accenture >>>>>> and its affiliates, including e-mail and instant messaging >>>>>>(including >>>>>> content), may be scanned by our systems for the purposes of >>>>>> information security and assessment of internal compliance with >>>>>> Accenture policy. >>>>>> >>>>>>_____________________________________________________________________ >>>>>> _ >>>>>> _ >>>>>> ___ >>>>>> ____________ >>>>>> >>>>>> www.accenture.com >>>>> >>>>> >>>>> >>>>> ________________________________ >>>>> >>>>> This message is for the designated recipient only and may contain >>>>> privileged, proprietary, or otherwise confidential information. If >>>>>you >>>>> have received it in error, please notify the sender immediately and >>>>> delete the original. Any other use of the e-mail by you is >>>>>prohibited. >>>>> Where allowed by local law, electronic communications with Accenture >>>>> and its affiliates, including e-mail and instant messaging (including >>>>> content), may be scanned by our systems for the purposes of >>>>> information security and assessment of internal compliance with >>>>> Accenture policy. >>>>> >>>>>______________________________________________________________________ >>>>> _ >>>>> ___ >>>>> ____________ >>>>> >>>>> www.accenture.com >>>> >>>> >>>> >>>> ________________________________ >>>> >>>> This message is for the designated recipient only and may contain >>>> privileged, proprietary, or otherwise confidential information. If you >>>> have received it in error, please notify the sender immediately and >>>> delete the original. Any other use of the e-mail by you is prohibited. >>>> Where allowed by local law, electronic communications with Accenture >>>> and its affiliates, including e-mail and instant messaging (including >>>> content), may be scanned by our systems for the purposes of >>>>information >>>> security and assessment of internal compliance with Accenture policy. >>>> >>>>_______________________________________________________________________ >>>> ___ >>>> ____________ >>>> >>>> www.accenture.com >>> >>> >>> >>> ________________________________ >>> >>> This message is for the designated recipient only and may contain >>> privileged, proprietary, or otherwise confidential information. If you >>> have received it in error, please notify the sender immediately and >>> delete the original. Any other use of the e-mail by you is prohibited. >>> Where allowed by local law, electronic communications with Accenture >>>and >>> its affiliates, including e-mail and instant messaging (including >>> content), may be scanned by our systems for the purposes of information >>> security and assessment of internal compliance with Accenture policy. >>> >>>________________________________________________________________________ >>>__ >>> ____________ >>> >>> www.accenture.com >>
