When I click on individual maps logs, it says "Aggregation is not enabled.
Try the nodemanager at slave1-machine:60933"
How could I enable aggregation?
On Sun, Jan 5, 2014 at 1:21 PM, Harsh J wrote:
> Every failed task typically carries a diagnostic message and a set of
> logs for you to invest
Hello,
Can someone provide some pointers on how to set up a HDFS file system? I
tried searching in the document but could not find anything with respect to
this.
--Ashish
I am using TextOutputFormat
Ok, the idea over here is , This output format writes to to a record
writer.. which in turn has to pass it on *some other object *where the data
is stored in mem and flushed once the block size is reached.
I want to look at that *some other object *and other helper cl
Thanks all, the following is required
Download from http://code.google.com/p/protobuf/downloads/list
$ ./configure
$ make
$ make check
$ make install
Then compile the source code
On Tue, Jan 7, 2014 at 9:46 AM, Rohith Sharma K S wrote:
> You can read Build instructions for Hadoop.
>
> htt
What OutputFormat are you using?
Once it reaches OutputFormat (specifically RecordWriter) it all depends on what
the RecordWriter does. Are you using some OutputFormat with a RecordWriter that
buffers like this?
Thanks,
+Vinod
On Jan 6, 2014, at 7:11 PM, nagarjuna kanamarlapudi
wrote:
> Th
You can read Build instructions for Hadoop.
http://svn.apache.org/repos/asf/hadoop/common/trunk/BUILDING.txt
For your problem, proto-buf not set in PATH. After setting, recheck
proto-buffer version is 2.5
From: nagarjuna kanamarlapudi [mailto:nagarjuna.kanamarlap...@gmail.com]
Sent: 07 January 2
El ene 6, 2014 10:48 PM, "nagarjuna kanamarlapudi" <
nagarjuna.kanamarlap...@gmail.com> escribió:
>
> Hi,
> I checked out the source code from
https://svn.apache.org/repos/asf/hadoop/common/trunk/
>
> I tried to compile the code with mvn.
>
> I am compiling this on a mac os X , mavericks. Any help
The error message said that the build needs protoc to be installed.
Search protobuf on the web.
Cheers
On Mon, Jan 6, 2014 at 7:47 PM, nagarjuna kanamarlapudi <
nagarjuna.kanamarlap...@gmail.com> wrote:
> Hi,
> I checked out the source code from
> https://svn.apache.org/repos/asf/hadoop/common
Hi,
I have a table in hbase named currencymaster
For Eg:
id,currency
1,INR
2,USD
Now I am dumping the text file containing currency as one of the field in
tableCurrency table in hbase using mapreduce code.
if the value from text file of currency field matches with the value of
currency master t
Hi,
I checked out the source code from
https://svn.apache.org/repos/asf/hadoop/common/trunk/
I tried to compile the code with mvn.
I am compiling this on a mac os X , mavericks. Any help is appreciated.
It failed at the following stage
[INFO] Apache Hadoop Auth Examples ..
This is not in DFSClient.
Before the output is written on to HDFS, lot of operations take place.
Like reducer output in mem reaching 90% of HDFS block size, then starting
to flush the data etc..,
So, my requirement is to have a look at that code where in I want to change
the logic a bit which s
The error is happening during Sort And Spill phase
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill
It seems like you are trying to compare two Int values and it fails during
compare
Caused by: java.lang.ArrayIndexOutOfBoundsException: 99614720
at
org.apache.hadoop.io.Writab
I have a hadoop program that I'm running with version 1.2.1 which
fails in a peculiar place. Most mappers complete without error, but
some fail with this stack trace:
java.io.IOException: Spill failed
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1297)
a
Assuming your output is going to HDFS, you want to look at DFSClient.
Reducer uses FileSystem to write the output. You need to start looking at how
DFSClient chunks the output and sends them across to the remote data-nodes.
Thanks
+Vinod
On Jan 6, 2014, at 11:07 AM, nagarjuna kanamarlapudi
wr
I want to have a look at the code where of flush operations that happens
after the reduce phase.
Reducer writes the output to OutputFormat which inturn pushes that to
memory and once it reaches 90% of chunk size it starts to flush the reducer
output.
I essentially want to look at the code of that
Can you please share how you are doing the lookup?
On Mon, Jan 6, 2014 at 4:23 AM, Ranjini Rathinam wrote:
> Hi,
>
> I have a input File of 16 fields in it.
>
> Using Mapreduce code need to load the hbase tables.
>
> The first eight has to go into one table in hbase and last eight has to
> got
Please do not tell me since last 2.5 years you have not used virtual Hadoop
environment to debug your Map Reduce application before deploying to
Production environment
No one can stop you looking at the code , Hadoop and its ecosystem is
open-source
On Mon, Jan 6, 2014 at 9:35 AM, nagarjuna kana
-- Forwarded message --
From: nagarjuna kanamarlapudi
Date: Mon, Jan 6, 2014 at 6:39 PM
Subject: Understanding MapReduce source code : Flush operations
To: mapreduce-u...@hadoop.apache.org
Hi,
I am using hadoop/ map reduce for aout 2.5 years. I want to understand the
internals o
I’m running Nutch 2.2.1 on a Hadoop cluster. I’m running 5000 links from the
DMOZ Open Directory Project. The reduce job stops exactly at 33% all the time
and it throws this exception. From the nutch mailing list, it seems that my job
is stumbling upon a repUrl value that’s null.
--
Manikandan
Based on the Exception type, it looks like something in your job is looking
for a valid value, and not finding it.
You will probably need to share the job code for people to help with this -
to my eyes, this doesn't appear to be a Hadoop configuration issue, or any
kind of problem with how the sys
-- Forwarded message --
From: nagarjuna kanamarlapudi
Date: Mon, Jan 6, 2014 at 8:09 AM
Subject: Understanding MapReduce source code : Flush operations
To: mapreduce-u...@hadoop.apache.org
Hi,
I am using hadoop/ map reduce for aout 2.5 years. I want to understand the
internals o
Hi,
I am using hadoop/ map reduce for aout 2.5 years. I want to understand the
internals of the hadoop source code.
Let me put my requirement very clear.
I want to have a look at the code where of flush operations that happens
after the reduce phase.
Reducer writes the output to OutputFormat wh
I’m trying to run Nutch 2.2.1 on a Hadoop 1.2.1 cluster. The fetch phase runs
fine. But in the next job, this error comes up
java.lang.NullPointerException
at org.apache.avro.util.Utf8.(Utf8.java:37)
at
org.apache.nutch.crawl.GeneratorReducer.setup(GeneratorReducer.java:100)
hi rajini
Can u use hive? then u can just use xpaths in ur select clause
cheers
R+
On Mon, Jan 6, 2014 at 2:44 PM, Ranjini Rathinam wrote:
> Hi,
>
> Thanks a lot .
>
> Ranjini
>
> On Fri, Jan 3, 2014 at 10:40 PM, Diego Gutierrez <
> diego.gutier...@ucsp.edu.pe> wrote:
>
>> Hi,
>>
>> I suggest
Hi,
I have a input File of 16 fields in it.
Using Mapreduce code need to load the hbase tables.
The first eight has to go into one table in hbase and last eight has to got
to another hbase table.
The data is being loaded into hbase table in 0.11 sec , but if any lookup
is being added in the ma
Hi,
Thanks a lot .
Ranjini
On Fri, Jan 3, 2014 at 10:40 PM, Diego Gutierrez <
diego.gutier...@ucsp.edu.pe> wrote:
> Hi,
>
> I suggest to use the XPath, this is a native java support for parse xml
> and json formats.
>
> For the main problem, like distcp command(
> http://hadoop.apache.org/docs
Hi all,
I just want to confirm if my understanding with Hadoop FileSystem object is
correct or not.
>From the source code of org.apache.hadoop.fs. FileSystem (either from version
>1.0.4 or 2.2.0), the method
public static FileSystem get(URI uri, Configuration conf) throws IOException
is using
Sure i will try and see Harsh :)
On Mon, Jan 6, 2014 at 12:40 PM, Harsh J wrote:
> You seem to have it right - why not give it a try before asking? :)
>
> On Mon, Jan 6, 2014 at 10:42 AM, unmesha sreeveni
> wrote:
> > Given a csv file with numeric data.
> > Need to find maximum element of each
28 matches
Mail list logo
