On Aug 2, 2010, at 9:17 AM, Erik Test wrote:
I'm trying to set a variable in my mapper class by reading an
argument from
the command line and then passing the entry to the mapper from main.
Is this
possible?
Others have already answered with the current solution of using
JobConf to
hi All,
i want to access through map-reduce java code .
plz guide me ..
--
View this message in context:
http://old.nabble.com/how-to-access-HDFS-file-system.-tp29333807p29333807.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
Hi All,
i want to access HDFS file sytem through map-reduce java code.
how can specify path for HDFS in Path constructor.
--
View this message in context:
http://old.nabble.com/how-to-access-HDFS-file-system.-tp29333820p29333820.html
Sent from the Hadoop core-user mailing list archive at
Check FileSystem and Configuration classes. Set 'fs.default.name' to point
to hdfs host (make sure it is prefixed with 'hdfs://') and via
FileSystem.get(conf)
you should get object for accessing hdfs, e.g.
conf = new Configuration();
conf.set(fs.default.name, hdfs://localhost:PORT);
hdfs =
hi Sanjiv,
you can simply intializae Path (org.apache.hadoop.fs.Path) with your
desired filename you want to create/open and then get a Filesystem
((org.apache.hadoop.fs.Path)) object. Filesystem object can do the rest for
you.
Example code snippet:
Path tmp = new Path(/tmp); //initializing the
Really? This seems pretty nice.
In the future, with your implementation, would the value always have to be
wrapped in a MyMapper instance? How would parameters be removed if
necessary?
Erik
On 3 August 2010 02:37, Owen O'Malley omal...@apache.org wrote:
On Aug 2, 2010, at 9:17 AM, Erik Test
So I am administering a 10+ node hadoop cluster and everything is going
swimmingly. Unfortunately, some relatively critical data is now being
stored on the cluster and I am being asked to create a backup solution for
hadoop in case of catasrophic failure of the data center, the application
Dan,
Here's quick and dirty solution that works.
I'm assuming that your cloud is part of a larger corporate network and that you
have your cloud, and then 'cloud aware machines', machines that have hadoop
installed, but are not part of your cloud but are where you launch jobs and
applications
On Aug 3, 2010, at 6:12 AM, Erik Test wrote:
Really? This seems pretty nice.
In the future, with your implementation, would the value always have
to be
wrapped in a MyMapper instance? How would parameters be removed if
necessary?
Sorry, I wasn't clear. I mean that if you make the
I am trying to use Distributes Cache on AWS Elastic Map educe
Using S3Fox I see I have two files in my bucket lordjoe
FeeFie.txt
and
Hello
In My main I say
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
String[] otherArgs = new
Date: Tue, 3 Aug 2010 11:02:48 -0400
Subject: Re: Backing up HDFS
From: edlinuxg...@gmail.com
To: common-user@hadoop.apache.org
Assuming you are taking the distcp approach you can mirror your
cluster with some scripting/coding. However your destination systems
can be more modest,
On Tue, Aug 3, 2010 at 11:46 AM, Michael Segel
michael_se...@hotmail.com wrote:
Date: Tue, 3 Aug 2010 11:02:48 -0400
Subject: Re: Backing up HDFS
From: edlinuxg...@gmail.com
To: common-user@hadoop.apache.org
Assuming you are taking the distcp approach you can mirror your
cluster with
I don't know of any research, but such a scenario is likely not going to
turn out so well. Hadoop is very network hungry and is designed to be run
in a datacenter. Sorry I don't have more information for you.
Alex
On Mon, Aug 2, 2010 at 9:14 PM, Rahul.V. greatness.hardn...@gmail.comwrote:
Sorry, cross posting to save time.
I now have a WIP of Cascading 1.2 that includes support for Riffle annotations.
Riffle is an Apache licensed library that includes Java annotations for marking
lifecycle and dependency methods on a 'process' object.
That is, you can create custom objects
O ok. Yes this is clear now. Thanks for the explanation
Erik
On 3 August 2010 11:34, Owen O'Malley omal...@apache.org wrote:
On Aug 3, 2010, at 6:12 AM, Erik Test wrote:
Really? This seems pretty nice.
In the future, with your implementation, would the value always have to be
wrapped in
We have recently decreased the dfs.replication value on our cluster
from 3 to 2 and see behavior similar to that described in issue
HADOOP-2270
(https://issues.apache.org/jira/browse/HADOOP-2270?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel).
Even though the parameter
16 matches
Mail list logo
