Can someone have a look at the patch MAPREDUCE-2457 and see if it can be
modified to work for 0.20.205?
I am very new to java and have no idea what's going on in that patch. If
you have any pointers for me, I will see if I can do it on my own.
Thanks,
Austin
On Fri, Mar 2, 2012 at 7:15 PM,
On 02/27/2012 11:53 AM, W.P. McNeill wrote:
You don't need any virtualization. Mac OS X is Linux and runs Hadoop as is.
Nitpick: OS X is NEXTSTEP based on Mach, which is a different
POSIX-compliant system from Linux.
AWS MapReduce (EMR) does not use S3 for its HDFS persistance. If it did
your S3 billing would be massive :) EMR reads all input jar files and
input data from S3, but it copies these files down to its local disk. It
then does starts the MR process, doing all HDFS reads and writes to the
local
On Mon, Mar 5, 2012 at 7:40 AM, John Conwell j...@iamjohn.me wrote:
AWS MapReduce (EMR) does not use S3 for its HDFS persistance. If it did
your S3 billing would be massive :) EMR reads all input jar files and
input data from S3, but it copies these files down to its local disk. It
then
Hi Madhu, it has the following line:
TermDocFreqArrayWritable () {}
but I'll try it with public access in case it's been called outside of my
package.
Thank you,
Mark
On Sun, Mar 4, 2012 at 9:55 PM, madhu phatak phatak@gmail.com wrote:
Hi,
Please make sure that your CustomWritable has
Streaming is good for simulation. Long running map-only processes, where pig
doesn't really help and it is simple to fire off a streaming process. You do
have to set some options so they can take a long time to return/return counters.
Russell Jurney http://datasyndrome.com
On Mar 5, 2012, at
Unfortunately, public didn't change my error ... Any other ideas? Has
anyone ran Hadoop on eclipse with custom sequence inputs ?
Thank you,
Mark
On Mon, Mar 5, 2012 at 9:58 AM, Mark question markq2...@gmail.com wrote:
Hi Madhu, it has the following line:
TermDocFreqArrayWritable () {}
but
Hi Rohini,
The similar problem was just encountered for me yesterday. But for my
situation, the max process num (ulimit -u) is set to 1024, which is too
small. And when i increase it to 100, the problem gone. But u said
Ulimit on the machine is set to unlimited, i'm not sure this will
All I see in the logs is:
2012-03-05 17:26:36,889 FATAL org.apache.hadoop.mapred.TaskTracker: Task:
attempt_201203051722_0001_m_30_1 - Killed : Java heap space
Looks like task tracker is killing the tasks. Not sure why. I increased
heap from 512 to 1G and still it fails.
On Mon, Mar 5,
Sorry for multiple emails. I did find:
2012-03-05 17:26:35,636 INFO
org.apache.pig.impl.util.SpillableMemoryManager: first memory handler call-
Usage threshold init = 715849728(699072K) used = 575921696(562423K)
committed = 715849728(699072K) max = 715849728(699072K)
2012-03-05 17:26:35,719
Hi all,
I have experience with hadoop 0.20.204 on 3 machines cluster as pilot,
now im trying to setup real cluster on 32 linux machines.
I have some question:
1. is hadoop 1.0 stable?? in hadoop site this version is indicated as
beta release
2. as you know installing and setting up hadoop
Its your use of the mapred.input.dir property, which is a reserved
name in the framework (its what FileInputFormat uses).
You have a config you extract path from:
Path input = new Path(conf.get(mapred.input.dir));
Then you do:
FileInputFormat.addInputPath(job, input);
Which internally, simply
12 matches
Mail list logo
