Hey Yan, Awesome!The location where you can add your .md is here:
docs/learn/tutorials/0.7.0/ Here's a link to the code tree: https://git-wip-us.apache.org/repos/asf?p=incubator-samza.git;a=tree;f=docs /learn/tutorials/0.7.0;h=ef117f4066f14a00f50f0f6fca17903130448312;hb=HEAD You can get the code here: git clone http://git-wip-us.apache.org/repos/asf/incubator-samza.git Once you write the .md, just throw it up on a JIRA, and one of us can merge it in. Re: hdfs-site.xml, ah ha, that's what I figured. This is good to know. So you just copy your hdfs-site.xml from your NodeManager's conf directory into your local hdfs-site.xml. Cheers, Chris On 3/11/14 10:16 AM, "Yan Fang" <[email protected]> wrote: >Hi Chris, > >Sure. I just do not know how/where to contribute this page...*_* > >Oh, I mean the same this as you mentioned in the *Cluster >Installation*thread: > >*"2. Get a copy of one of your NM's yarn-site.xml and put it somewhere on* > >*your desktop (I usually use ~/.yarn/conf/yarn-site.xml). Note that >there'sa "conf" directory there. This is mandatory."* > >So I just copy the hdfs-site.xml to ~/.yarn/conf/hdfs-site.xml. Thank you. > >Cheers, > >Fang, Yan >[email protected] >+1 (206) 849-4108 > > >On Tue, Mar 11, 2014 at 10:10 AM, Chris Riccomini ><[email protected]>wrote: > >> Hey Yan, >> >> Would you be up for contributing a tutorial page that describes this? >>This >> is really useful information. Our docs are just simple .md files in the >> main code base. >> >> Regarding step (3), is the hdfs-site.xml put into the conf folder for >>the >> NM boxes, or on the client side (where run-job.sh is run)? >> >> Cheers, >> Chris >> >> On 3/11/14 10:07 AM, "Yan Fang" <[email protected]> wrote: >> >> >Hi Sonali, >> > >> >The way I make Samza run with HDFS is following: >> > >> >1. include hdfs jar in Samza jar tar.gz. >> >2. you may also want to make sure the hadoop-common.jar has the same >> >version as your hdfs jar. Otherwise, you may have configuration error >> >popping out. >> >3. then put hdfs-site.xml to conf folder, the same folder as the >> >yarn-site.xml >> >4. all other steps are not changed. >> > >> >Hope this will help. Thank you. >> > >> >Cheers, >> > >> >Fang, Yan >> >[email protected] >> >+1 (206) 849-4108 >> > >> > >> >On Tue, Mar 11, 2014 at 9:25 AM, Chris Riccomini >> ><[email protected]>wrote: >> > >> >> Hey Sonali, >> >> >> >> I believe that you need to make sure that the HDFS jar is in your >> >>.tar.gz >> >> file, as you've said. >> >> >> >> If that doesn't work, you might need to define this setting in >> >> core-site.xml on the machine you're running run-job.sh on: >> >> >> >> <property> >> >> <name>fs.hdfs.impl</name> >> >> <value>org.apache.hadoop.hdfs.DistributedFileSystem</value> >> >> <description>The FileSystem for hdfs: uris.</description> >> >> </property> >> >> >> >> >> >> You might also need to configure your NodeManagers to have the HDFS >>file >> >> system impl as well. >> >> >> >> I've never run Samza with HDFS, so I'm guessing here. Perhaps someone >> >>else >> >> on the list has been successful with this? >> >> >> >> Cheers, >> >> Chris >> >> >> >> On 3/10/14 3:59 PM, "[email protected]" >> >> <[email protected]> wrote: >> >> >> >> >Hello, >> >> > >> >> >I fixed this by starting from scratch with gradlew. But now when I >>run >> >>my >> >> >job it throws this error: >> >> >Exception in thread "main" java.io.IOException: No FileSystem for >> >>scheme: >> >> >hdfs >> >> > at >> >> >> >>>>>org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:242 >>>>>1) >> >> > at >> >> >>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428) >> >> > at >> >>org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88) >> >> > at >> >> >>>org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467) >> >> > at >> >>org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449) >> >> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) >> >> > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287) >> >> > at >> >> >> >>>>>org.apache.samza.job.yarn.ClientHelper.submitApplication(ClientHelper. >>>>>sc >> >>>al >> >> >a:111) >> >> > at >>org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:55) >> >> > at >>org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:48) >> >> > at org.apache.samza.job.JobRunner.run(JobRunner.scala:100) >> >> > at org.apache.samza.job.JobRunner$.main(JobRunner.scala:75) >> >> > at org.apache.samza.job.JobRunner.main(JobRunner.scala) >> >> > >> >> >I looked at the samza job tar.gz and it doesn't have a Hadoop-hdfs >>jar. >> >> >Is that why I get this error? >> >> > >> >> >Thanks, >> >> >Sonali >> >> > >> >> >From: Parthasarathy, Sonali >> >> >Sent: Monday, March 10, 2014 11:25 AM >> >> >To: [email protected] >> >> >Subject: Failed to package using mvn >> >> > >> >> >Hi, >> >> > >> >> >When I tried to do a mvn clean package of my hello-samza project, I >>get >> >> >the following error. Has anyone seen this before? >> >> > >> >> >[ERROR] Failed to execute goal on project samza-wikipedia: Could not >> >> >resolve dependencies for project samza:samza-wikipedia:jar:0.7.0: >>Could >> >> >not find artifact org.apache.samza:samza-kv_2.10:jar:0.7.0 in >> >> >apache-releases >>(https://repository.apache.org/content/groups/public) >> >>-> >> >> >[Help 1] >> >> >[ERROR] >> >> >[ERROR] To see the full stack trace of the errors, re-run Maven with >> >>the >> >> >-e switch. >> >> >[ERROR] Re-run Maven using the -X switch to enable full debug >>logging. >> >> >[ERROR] >> >> >[ERROR] For more information about the errors and possible >>solutions, >> >> >please read the following articles: >> >> >[ERROR] [Help 1] >> >> > >> >> >> >> >> >>http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionExce >> >>p >> >> >tion >> >> >[ERROR] >> >> >[ERROR] After correcting the problems, you can resume the build with >> >>the >> >> >command >> >> >[ERROR] mvn <goals> -rf :samza-wikipedia >> >> > >> >> >Thanks, >> >> >Sonali >> >> > >> >> >Sonali Parthasarathy >> >> >R&D Developer, Data Insights >> >> >Accenture Technology Labs >> >> >703-341-7432 >> >> > >> >> > >> >> >________________________________ >> >> > >> >> >This message is for the designated recipient only and may contain >> >> >privileged, proprietary, or otherwise confidential information. If >>you >> >> >have received it in error, please notify the sender immediately and >> >> >delete the original. Any other use of the e-mail by you is >>prohibited. >> >> >Where allowed by local law, electronic communications with Accenture >> >>and >> >> >its affiliates, including e-mail and instant messaging (including >> >> >content), may be scanned by our systems for the purposes of >>information >> >> >security and assessment of internal compliance with Accenture >>policy. >> >> >> >>>>>______________________________________________________________________ >>>>>__ >> >>>__ >> >> >____________ >> >> > >> >> >www.accenture.com >> >> >> >> >> >>
