Hi all,
I am trying to figure how I can start a hadoop job porgramatically from my Java
application running in an app server. I was able to run my map reduce job using
hadoop command from hadoop master machine but my goal is to run the same job
from my java program (running on a different machin
Hi:
I am trying to run a job in my own application.
So far, I can run the job on the server which the hadoop server is running
on.
But what I expect is the hadoop server is running on server A. and the
application will run on other server (B server).
If I run the application on Server B, it will
On 11/22/2010 04:39 PM, praveen.pe...@nokia.com wrote:
Hi all, I am trying to figure how I can start a hadoop job
porgramatically from my Java application running in an app server. I
was able to run my map reduce job using hadoop command from hadoop
master machine but my goal is to run the same j
Hi Praveen,
we do. We are using the "new" org.apache.hadoop.mapreduce.* API in
Hadoop 0.20.2.
Essentially the flow is:
//
// assuming all config is on the class path
Configuration config = new Configuration();
Job job = new Job(config, "some job name");
// set in/out types
ubject: Re: Starting a Hadoop job programtically
Hi Praveen,
we do. We are using the "new" org.apache.hadoop.mapreduce.* API in Hadoop
0.20.2.
Essentially the flow is:
//
// assuming all config is on the class path
Configuration config = new Configuration();
Job
__
> From: ext Henning Blohm [mailto:henning.bl...@zfabrik.de]
> Sent: Monday, November 22, 2010 5:07 PM
> To: mapreduce-user@hadoop.apache.org
> Subject: Re: Starting a Hadoop job programtically
>
>
>
>
> Hi Praveen,
>
> we
henning.bl...@zfabrik.de]
Sent: Tuesday, November 23, 2010 3:24 AM
To: mapreduce-user@hadoop.apache.org
Subject: RE: Starting a Hadoop job programtically
Hi Praveen,
in order to submit it to the cluster, you just need to have a core-site.xml
on your classpath (or load it explicitly into your configu
ck (as I tried to explain
before and if I am not mistaken).
>
> Thanks
> Praveen
Good luck,
Henning
>
>
> __
> From: ext Henning Blohm [mailto:henning.bl...@zfabrik.de]
> Sent: Tuesday, No
taken).
Thanks
Praveen
Good luck,
Henning
From: ext Henning Blohm [mailto:henning.bl...@zfabrik.de]
Sent: Tuesday, November 23, 2010 3:24 AM
To: mapreduce-user@hadoop.apache.org
Subject: RE: Starting a Hadoop job programtically
Hi Praveen,
in order to submit it
on.java:762)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
> ... 4 more
>
> Praveen
>
>
> __
> From: ext Henning Blohm [mailto:henning.bl...@zfabrik.de]
> S
ovember 24, 2010 3:38 AM
To: mapreduce-user@hadoop.apache.org
Subject: RE: Starting a Hadoop job programtically
Hi Praveen,
looking at the Job configuration you will find properties like user.name and
more stuff that has created by substituting template values in
core-default.xml, mapred-default.x
Please check the status of job tracker since you can not find port
9001 using netstat
2010/11/25 li ping :
> Hi:
> I am trying to run a job in my own application.
> So far, I can run the job on the server which the hadoop server is running
> on.
> But what I expect is the hadoop server is runnin
Are you running a firewall on Server A? Check your firewall settings
if so, to allow connections to port 9001/etc.. I'd recommend disabling
Firewalls in development environments.
2010/11/25 li ping :
> Hi:
> I am trying to run a job in my own application.
> So far, I can run the job on the server
A possible cause is that name node and job tracker on server A are bound to
a local address such as 127.0.0.1 which server B can't see. If you are
using linux, you can check /etc/hosts to make sure that xi-pli is not bound
to a local address.
2010/11/25 li ping
> Hi:
>
> I am trying to run a jo
But I can run the application on the hadoop server. (not use the hadoop
command to run).
On Thu, Nov 25, 2010 at 4:38 PM, Jeff Zhang wrote:
> Please check the status of job tracker since you can not find port
> 9001 using netstat
>
>
>
> 2010/11/25 li ping :
> > Hi:
> > I am trying to run a jo
I made it works.
The configuration file contains some error.
fs.default.name
hdfs://xi-pli:9000
mapred.job.tracker
xi-pli:9001
Here,we'd better use the IP address instead of the host-name. because the
hostname may point to a lookback address.
When
16 matches
Mail list logo
