Re: Running Distributed shell in hadoop0.23

When we create a directory using distributed shell,any idea where it is created On Thu, Dec 15, 2011 at 11:57 AM, raghavendhra rahul < raghavendhrara...@gmail.com> wrote: > How to run any script using this.When i tried it shows final status as > failed. > > > On Thu, Dec 15, 2011 at 11:48 AM, rag

Re: Running Distributed shell in hadoop0.23

How to run any script using this.When i tried it shows final status as failed. On Thu, Dec 15, 2011 at 11:48 AM, raghavendhra rahul < raghavendhrara...@gmail.com> wrote: > Thanks for the help i made a mistake of creating symlinks within > modules.Now everythng is fine. > > > > On Thu, Dec 15, 201

Re: Running Distributed shell in hadoop0.23

Thanks for the help i made a mistake of creating symlinks within modules.Now everythng is fine. On Thu, Dec 15, 2011 at 11:18 AM, raghavendhra rahul < raghavendhrara...@gmail.com> wrote: > should i link the hadoop-yarn-applications-distributedshell-0.23.0.jar > also. > Without linking this jar i

Re: Running Distributed shell in hadoop0.23

should i link the hadoop-yarn-applications-distributedshell-0.23.0.jar also. Without linking this jar it throws the same error. If linked it shows at org.apache.hadoop.util.RunJar.main(RunJar.java:130) Caused by: java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile

Re: Where JobTracker stores Task'sinformation

From where Jobtracker takes information regrading which map & reduce task fails. and where it reschedules these map & reduce task (i.e. which code it executes to reschedule map & reduce task) On Thu, Dec 15, 2011 at 12:02 AM, Arun C Murthy wrote: > Take a look at JobInP

Re: Running Distributed shell in hadoop0.23

The yarn jars are likely missing from the classpath. Could you try creating the symlinks as per step 11 from http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/INSTALL?revision=1166955 ? -- Hitesh On Dec 14, 2011, at 8:35 PM, raghavendhra rahul wrote: > I get the follow

Re: Running Distributed shell in hadoop0.23

I get the following erroer by the given command to run distributed shell hadoop1@master:~/hadoop/bin$ ./hadoop jar ../modules/hadoop-yarn-applications-distributedshell-0.23.0.jar org.apache.hadoop.yarn.applications.distributedshell.Client --jar ../modules/hadoop-yarn-applications-distributedshell-0

registering service for use in input / output formats

Is there a way to pass a service to the output format? I have an object which I would like to initialize/configure outside and then pass in (since it must also be used elsewhere). So far I have been using a static singleton instance so the format can find it but I'd prefer to avoid that if poss

Re: running shell commands inside mapper

Souri, Yes and no. Oozie does something like this, but not through the shell, and you can look at how they are doing it. I don't know the details, but you will probably need to get some delegation tokens to make it work properly if you have security enabled. You should be able to copy the ja

Re: Running Distributed shell in hadoop0.23

Assuming you have a non-secure cluster setup ( the code does not handle security properly yet ), the following command would run the ls command on 5 allocated containers. $HADOOP_COMMON_HOME/bin/hadoop jar < path to hadoop-yarn-applications-distributedshell-0.24.0-SNAPSHOT.jar> org.apache.had

Re: Where JobTracker stores Task'sinformation

Take a look at JobInProgress.java. There is one object per job. Arun On Dec 14, 2011, at 1:14 AM, hadoop anis wrote: > > > > Hi Friends, > I want to know, where JobTracker stores Task's Information, > i.e. which task is being executed on which tasktracker, and h

Re: Overriding remote classes

On Wed, 14 Dec 2011 11:04:37 -0500, David Rosenstrauch wrote: > I ran into the same (known) issue. (See: > https://issues.apache.org/jira/browse/MAPREDUCE-1700) > > Doesn't look like there's a solution yet. Thanks; good to know that I'm actually doing the best I can be writing everything to be

File Side-Effect files in Streaming

Hi, is it possible use side-effect file using streaming (python)? If it is, how can i do it? Thanks.

Re: Analysing Completed Job info programmatically apart from Jobtracker GUI

On 12/14/2011 09:39 AM, arun k wrote: Hi Guys ! I want to analyse the completed Job counters like FILE/HDFS BYTES READ/WRITTEN along with other values like average map/reduce task run time. I see that Jobtracker GUI has this info but i want to programmatically retrieve these values instead of ma

Re: Overriding remote classes

On 12/14/2011 08:20 AM, John Armstrong wrote: Hi, there. I've run into an odd situation, and I'm wondering if there's a way around it; I'm trying to use Jackson for some JSON serialization in my program, and I wrote/unit-tested it to work with Jackson 1.9. Then, in integration testing, I starte

Analysing Completed Job info programmatically apart from Jobtracker GUI

Hi Guys ! I want to analyse the completed Job counters like FILE/HDFS BYTES READ/WRITTEN along with other values like average map/reduce task run time. I see that Jobtracker GUI has this info but i want to programmatically retrieve these values instead of manually noting down these values and do s

Overriding remote classes

Hi, there. I've run into an odd situation, and I'm wondering if there's a way around it; I'm trying to use Jackson for some JSON serialization in my program, and I wrote/unit-tested it to work with Jackson 1.9. Then, in integration testing, I started to see some weird version incompatibilities an

running shell commands inside mapper

Hi, I am trying to run a shell command from within a mapper. The shell command is of the form: * hadoop jar somjarfile arg1 arg2 ...* Can i do this type of operation from within a mapper? Also, can i copy the jar file in distributed cache and use it? Thanks, Souri

Where JobTracker stores Task'sinformation

Hi Friends, I want to know, where JobTracker stores Task's Information, i.e. which task is being executed on which tasktracker, and how JobTracker stores this information. If anyone know this please let me know it. Regards, Anis M.Tech. Student

Running Distributed shell in hadoop0.23

Hi, Can anyone give the procedure about how to run Distibuted shell example in hadoop yarn.So that i try to understand how applicatin master really works.