Configuration conf = new Configuration();
FileInputFormat.setInputPaths(conf, new Path(args[0]));
Job job = new Job(conf);
On Wed, Sep 15, 2010 at 6:48 AM, Mark Kerzner markkerz...@gmail.com wrote:
Hi,
in 0.18, it used to be
main()
JobConf conf = new JobConf(MyMR.class);
Check FileSystem and Configuration classes. Set 'fs.default.name' to point
to hdfs host (make sure it is prefixed with 'hdfs://') and via
FileSystem.get(conf)
you should get object for accessing hdfs, e.g.
conf = new Configuration();
conf.set(fs.default.name, hdfs://localhost:PORT);
hdfs =
AFAIK, there is no such method (to get a job name from client side) :(
(at least I wasn't able to find it). Via JobProfile can be
extracted job name via given id, but only JobTracker can access it (if
you try to instantiate it, you will start own job tracker).
The only solution is to directly
JobClient is able to directly connect to job tracker address (see
JobTracker constructor
with InetSocketAddress parameter). After that, getAllJobs() will
return known jobs and
you will able to find your job id there.
I would go with similar solution (with proposed one): write some lock
with job
Just make sure 'fs.default.name' (set in Configuration) is set and point to
master with 'hdfs://' prefix. Otherwise, by default, it will behave as
LocalFileSystem.
On Sat, Jun 12, 2010 at 9:08 AM, Jeff Zhang zjf...@gmail.com wrote:
You should put hadoop-*.core.jar on your classpath. and
Hi all,
I'm working on larger application that utilizes Hadoop for some
crunching tasks and utilization is
done via new job API (Job/Configuration). I've noticed how
running/completed jobs are not visible on
JobTracker web view nor are displayed via 'hadoop job -list all' when
they are started
