Thanks, I built 2.3 yesterday (checked out from from link suggested in
earlier post of this thread) without problems apart from VM running out of
memory which was fixed with
export MAVEN_OPTS=-Xmx2048m
At least, I got a message saying successful build.
Thanks for your help.
On 8 February 201
It was actually zlib compression in Python. But you can certainly use any
other compression lib. Unfortunately, this is not a built-in feature in
Pig, but I agree that would be useful.
On Fri, Feb 7, 2014 at 11:45 AM, praveenesh kumar wrote:
> Hi Park,
>
> Your explanation makes perfect sense in
In the output for a passing test, I saw:
2014-02-06 16:48:49,722 ERROR [Thread[Thread-71,5,main]]
delegation.AbstractDelegationTokenSecretManager
(AbstractDelegationTokenSecretManager.java:run(557)) -
InterruptedExcpetion recieved for ExpiredTokenRemover thread
java.lang.InterruptedException: slee
Hi Park,
Your explanation makes perfect sense in my case. Thanks for explaining what
is happening behind the scenes. I am wondering you used normal java
compression/decompression or is there a UDF already available to do this
stuff or some kind of property that we need to enable to say to PIG that
Hi Prav,
You're thinking correctly, and it's true that Pig bags are spillable.
However, spilling is no magic, meaning you can still run into OOM with huge
bags like you have here. Pig runs Spillable Memory Manager (SMM) in a
separate thread. When spilling is triggered, SMM locks bags that it's
tr
Hi all,
I have a question about reserved containers in the YARN capacity
scheduler. After reading the source code and related document, it is
not very clear. What is the purpose or practical usage of the reserved
container? thx.
Thanks Park for sharing the above configs
But I am wondering if the above config changes would make any huge
difference in my case.
As per my logs, I am very worried about this line -
INFO org.apache.hadoop.mapred.MapTask: Record too large for in-memory
buffer: 644245358 bytes
If I am understan
Hi,
I am having some failure test cases where my Application Master is
supposed to fail. But when it fails it is again started with _02 .
Is there a way for me to avoid the second instance of the Application
Master getting started? Is it re-started automatically by the RM after the
first one fa
Hi Alex,
You should give the copyFromLocal command from the namenode or any machine
that is not a datanode to get the file distributed.
On Fri, Feb 7, 2014 at 10:53 AM, Harsh J wrote:
> Hi,
>
> The 0.20.203.0rc1 is a very old version at this point. Why not use a
> more current version if you'r
Hi,
The 0.20.203.0rc1 is a very old version at this point. Why not use a
more current version if you're deploying a new cluster?
Onto your issue, your configuration XML files (core-site.xml,
hdfs-site.xml or mapred-site.xml) most likely have a "dfs.replication"
value set to 1 causing only that ma
Hi, folks!
I've deployed hadoop (0.20.203.0rc1) on 8-node cluster. After uploading
file onto hdfs I've got this file only on one of the nodes instead of being
uniformly distributed across all nodes. What can be the issue?
$HADOOP_HOME/bin/hadoop dfs -copyFromLocal ../data/rmat-20.0
/user/frolo/in
11 matches
Mail list logo