Submitting and running hadoop jobs Programmatically

2011-07-26 Thread madhu phatak
Hi, I am working on a open source project Nectar where i am trying to create the hadoop jobs depending upon the user input. I was using Java Process API to run the bin/hadoop shell script to submit the jobs. But it seems not good way because the proces

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
A simple job.submit(…) OR JobClient.runJob(jobConf), submits your job right from the Java API. Does this not work for you? If not, what error do you face? Forking out and launching from a system process is a bad idea unless there's absolutely no way. On Tue, Jul 26, 2011 at 3:28 PM, madhu phatak

RE: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Devaraj K
oop/mapred uce/Job.html Devaraj K -Original Message- From: madhu phatak [mailto:phatak@gmail.com] Sent: Tuesday, July 26, 2011 3:29 PM To: common-user@hadoop.apache.org Subject: Submitting and running hadoop jobs Programmatically Hi, I am working on a open source project Nectar<

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread madhu phatak
apache.org/mapreduce/docs/current/api/org/apache/hadoop/mapred > uce/Job.html > > > > Devaraj K > > -Original Message----- > From: madhu phatak [mailto:phatak@gmail.com] > Sent: Tuesday, July 26, 2011 3:29 PM > To: common-user@hadoop.apache.org > Subject: S

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
/hadoop.apache.org/mapreduce/docs/current/api/org/apache/hadoop/mapred >> uce/Job.html >> >> >> >> Devaraj K >> >> -Original Message- >> From: madhu phatak [mailto:phatak@gmail.com] >> Sent: Tuesday, July 26, 2011 3:29 PM >> To: com

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread madhu phatak
re you want to submit > >> the > >> job. > >> > >> You can refer this docs for more info on Job API's. > >> > >> > http://hadoop.apache.org/mapreduce/docs/current/api/org/apache/hadoop/mapred > >> uce/Job.html > >> >

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
gt; >> >>     // Submit the job >> >>     job.submit(); >> >> >> >> >> >> >> >> For submitting this, need to add the hadoop jar files and configuration >> >> files in the class path of the application from where you w

RE: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Devaraj K
ication from where you want to submit >> >> the >> >> job. >> >> >> >> You can refer this docs for more info on Job API's. >> >> >> >> >> http://hadoop.apache.org/mapreduce/docs/current/api/org/apache/hadoop/mapred >

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread madhu phatak
ob? > > Devaraj K > > -Original Message- > From: Harsh J [mailto:ha...@cloudera.com] > Sent: Tuesday, July 26, 2011 5:01 PM > To: common-user@hadoop.apache.org > Subject: Re: Submitting and running hadoop jobs Programmatically > > Yes. Internally, it calls regula

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
aj K wrote: > >> Madhu, >> >>  Can you check the client logs, whether any error/exception is coming while >> submitting the job? >> >> Devaraj K >> >> -Original Message----- >> From: Harsh J [mailto:ha...@cloudera.com] >> Sent: Tues

Re: Submitting and running hadoop jobs Programmatically

2011-07-27 Thread Steve Loughran
On 27/07/11 05:55, madhu phatak wrote: Hi I am submitting the job as follows java -cp Nectar-analytics-0.0.1-SNAPSHOT.jar:/home/hadoop/hadoop-for-nectar/hadoop-0.21.0/conf/*:$HADOOP_COMMON_HOME/lib/*:$HADOOP_COMMON_HOME/* com.zinnia.nectar.regression.hadoop.primitive.jobs.SigmaJob input/book.

Re: Submitting and running hadoop jobs Programmatically

2011-07-27 Thread madhu phatak
Thank you . Will have a look on it. On Wed, Jul 27, 2011 at 3:28 PM, Steve Loughran wrote: > On 27/07/11 05:55, madhu phatak wrote: > >> Hi >> I am submitting the job as follows >> >> java -cp >> Nectar-analytics-0.0.1-**SNAPSHOT.jar:/home/hadoop/** >> hadoop-for-nectar/hadoop-0.21.**0/conf/*:$

Re: Submitting and running hadoop jobs Programmatically

2011-07-27 Thread madhu phatak
2011 at 5:11 PM, Devaraj K wrote: > > > >> Madhu, > >> > >> Can you check the client logs, whether any error/exception is coming > while > >> submitting the job? > >> > >> Devaraj K > >> > >> -Original Message-