Like you said, it depends both on the kind of network you have and the type of
your workload.
Given your point about S3, I'd guess your input files/blocks are not large
enough that moving code to data trumps moving data itself to the code. When
that balance tilts a lot, especially when moving
Yes. JobTracker and TaskTracker are gone from all the 2.x release lines.
MapReduce is an application on top of YARN. That is per job - launches, starts
and finishes after it is done with its work. Once it is done, you can go look
at it in the MapReduce specific JobHistoryServer.
+Vinod
On
Reduce has three phases - shuffle, sort and reduce.
So, 33% would imply the shuffle phase end, and 66% would refer to the end of
sort phase.
Thanks,
+Vinod
On Oct 15, 2012, at 2:32 PM, Jay Vyas wrote:
Hi guys !
We all know that there are major milestones in reducers (33%, 66%)
In
in caps. If that is the case, you should try tweaking your
host-names to all lower-case.
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Sep 12, 2012, at 9:47 AM, Shumin Wu wrote:
Hi,
I am setting up a secured hdfs using Kerberos. I got NN, 2NN working just
fine
This has got nothing to do with the scheduler.
I believe this has got to do with some compilation issue. How did you build
hadoop?
Also, I found that the repo at github (which is a mirror of git repo at apache)
doesn't always pick all the commits immediately. You are better off checking
out
Replies/more questions inline.
I'm using Hadoop 0.23 on 50 machines, each connected with gigabit ethernet
and each having solely a single hard disk. I am getting the following error
repeatably for the TeraSort benchmark. TeraGen runs without error, but
TeraSort runs predictably until
Hadoop has integrated snappy via installed native libraries instead of
snappy-java.jar (ref https://issues.apache.org/jira/browse/HADOOP-7206)
- You need to have the snappy system libraries (snappy and snappy-devel)
installed before you compile hadoop. (RPMs are available on the web,
through jsvc, I don't know if the java setting
does not work after executed through jsvc. But anyway, it still complain for
the AES 256 is not supported.
Any ideas?
Thanks
Emma
-Original Message-
From: Vinod Kumar Vavilapalli [mailto:vino...@hortonworks.com]
Sent: 2012年1月20日 13
Hi,
Just today evening, I happened to run into someone who had the same
issue. After some debugging, I cornered that to the hostnames having
upper-case characters. Somehow, when DataNode or NodeManager try to
get a service ticket for their corresponding services (NameNode and
ResourceManager
You can use yarn.nodemanager.resource.memory-mb to set the limit on
each NodeManager.
You should have a good look at
http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
. It has enough information to get you a good distance.
HTH.
+Vinod
On Tue, Jan 10,
Yes, you can.
http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html#Writing_an_ApplicationMaster
should give you a very good idea and example code about this.
But, the requirements are not hard-fixed. If the scheduler cannot find
free resources on
Must be related to some kind of permissions problems.
It will help if you can paste the corresponding source code for
FileUtil.copy(). Hard to track it with different versions, so.
Thanks,
+Vinod
On Mon, Oct 3, 2011 at 9:28 PM, Raj V rajv...@yahoo.com wrote:
Eric
Yes. The owner is hdfs and
You may be missing the kerberos principal for the namenode in your
configuration used to connect to NameNode. Check your configuration for
dfs.namenode.kerberos.principal and set it to the same value as on NN.
HTH
+Vinod
On Thu, Sep 22, 2011 at 4:06 AM, Sivva svijaysand...@gmail.com wrote:
Hi
13 matches
Mail list logo
