Re: Submitting and running hadoop jobs Programmatically

2011-07-27 Thread madhu phatak
2011 at 5:11 PM, Devaraj K wrote: > > > >> Madhu, > >> > >> Can you check the client logs, whether any error/exception is coming > while > >> submitting the job? > >> > >> Devaraj K > >> > >> -Original Message-

Re: Submitting and running hadoop jobs Programmatically

2011-07-27 Thread madhu phatak
Thank you . Will have a look on it. On Wed, Jul 27, 2011 at 3:28 PM, Steve Loughran wrote: > On 27/07/11 05:55, madhu phatak wrote: > >> Hi >> I am submitting the job as follows >> >> java -cp >> Nectar-analytics-0.0.1-**SNAPSHOT.jar:/home/hadoop/** >> hadoop-for-nectar/hadoop-0.21.**0/conf/*:$

Re: Submitting and running hadoop jobs Programmatically

2011-07-27 Thread Steve Loughran
On 27/07/11 05:55, madhu phatak wrote: Hi I am submitting the job as follows java -cp Nectar-analytics-0.0.1-SNAPSHOT.jar:/home/hadoop/hadoop-for-nectar/hadoop-0.21.0/conf/*:$HADOOP_COMMON_HOME/lib/*:$HADOOP_COMMON_HOME/* com.zinnia.nectar.regression.hadoop.primitive.jobs.SigmaJob input/book.

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
aj K wrote: > >> Madhu, >> >>  Can you check the client logs, whether any error/exception is coming while >> submitting the job? >> >> Devaraj K >> >> -Original Message----- >> From: Harsh J [mailto:ha...@cloudera.com] >> Sent: Tues

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread madhu phatak
ob? > > Devaraj K > > -Original Message- > From: Harsh J [mailto:ha...@cloudera.com] > Sent: Tuesday, July 26, 2011 5:01 PM > To: common-user@hadoop.apache.org > Subject: Re: Submitting and running hadoop jobs Programmatically > > Yes. Internally, it calls regula

RE: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Devaraj K
Madhu, Can you check the client logs, whether any error/exception is coming while submitting the job? Devaraj K -Original Message- From: Harsh J [mailto:ha...@cloudera.com] Sent: Tuesday, July 26, 2011 5:01 PM To: common-user@hadoop.apache.org Subject: Re: Submitting and running

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
Yes. Internally, it calls regular submit APIs. On Tue, Jul 26, 2011 at 4:32 PM, madhu phatak wrote: > I am using JobControl.add() to add a job and running job control in > a separate thread and using JobControl.allFinished() to see all jobs > completed or not . Is this work same as Job.submit()??

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread madhu phatak
I am using JobControl.add() to add a job and running job control in a separate thread and using JobControl.allFinished() to see all jobs completed or not . Is this work same as Job.submit()?? On Tue, Jul 26, 2011 at 4:08 PM, Harsh J wrote: > Madhu, > > Do you get a specific error message / stack

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
Madhu, Do you get a specific error message / stack trace? Could you also paste your JT logs? On Tue, Jul 26, 2011 at 4:05 PM, madhu phatak wrote: > Hi >  I am using the same APIs but i am not able to run the jobs by just adding > the configuration files and jars . It never create a job in Hadoop

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread madhu phatak
Hi I am using the same APIs but i am not able to run the jobs by just adding the configuration files and jars . It never create a job in Hadoop , it just shows cleaning up staging area and fails. On Tue, Jul 26, 2011 at 3:46 PM, Devaraj K wrote: > Hi Madhu, > > You can submit the jobs using t

RE: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Devaraj K
Hi Madhu, You can submit the jobs using the Job API's programmatically from any system. The job submission code can be written this way. // Create a new Job Job job = new Job(new Configuration()); job.setJarByClass(MyJob.class); // Specify various job-specific paramet

Re: Submitting and running hadoop jobs Programmatically

2011-07-26 Thread Harsh J
A simple job.submit(…) OR JobClient.runJob(jobConf), submits your job right from the Java API. Does this not work for you? If not, what error do you face? Forking out and launching from a system process is a bad idea unless there's absolutely no way. On Tue, Jul 26, 2011 at 3:28 PM, madhu phatak