MultipleOutputs is the way to go :)
On Tue, Mar 12, 2013 at 12:48 PM, Fatih Haltas fatih.hal...@nyu.edu wrote:
Hi Everyone,
I would like to have 2 different output (having different columns of a same
input text file.)
When I googled a bit, I got multipleoutputs classes, is this the common
What George's suggested is more of a hack. If you want to write proper
impersonation code that works despite the security toggle, follow
http://hadoop.apache.org/docs/stable/Secure_Impersonation.html.
Or in your case, alternatively, create a local user hdfs and use
that via sudo -u hdfs prefixes.
you need to disable following property..
property
namedfs.permissions.enabled/name
valuetrue/value
description
If true, enable permission checking in HDFS.
If false, permission checking is turned off,
but all other behavior is unchanged.
Switching from one parameter value
Hi All,
I am pretty new to Hadoop. I noticed that there are many test classes in hadoop
source code under the packages test. org.apache.hadoop. Can anyone explain me
or provide some pointers to understand what is the purpose of this and how to
use them. In particular I wanted to use
dfs.datanode.max.xcievers value should set across the cluster rather than
particular DataNode.
It means the upper bound on the number of files that the DataNode will
serve at any one time.
2013/3/17 Dhanasekaran Anbalagan bugcy...@gmail.com
Hi Guys,
We are having few data nodes in an
I want 20 servers, I got 7, so I want to make the most of the 7 I have. Each
of the 7 servers have: 24GB of ram, 4TB, and 8 cores.
Would it be terribly unwise of me to Run such a configuration:
. Server #1: NameNode + Master + TaskTracker(reduced
slots)
. Server
These test classes are used for unit testing.
You can run these cases to test particular function of a class.
But when we run these test case, we need some additional classes and
functions to simulate some underlying function which were called by these
test cases.
InMemoryNativeFileSystemStore is
Hi,
Thanks for the quick reply. In order to test the class
TestInMemoryNativeS3FileSystemContract and its functions what should be the
value of parameter sin my configuration files (core-site, mapred, etc.)?
Regards,
Nikhil
From: Agarwal, Nikhil
Sent: Monday, March 18, 2013 1:55 PM
To:
Hello all,
We have our dfs.name.dir configured to write to two local and one NFS
directories. The NFS server in question had to be restarted a couple
days back and that copy of the namenode data fell behind as a result.
As I understand it, restarting hadoop will take the most recent copy of
You may want to check this JIRA:
https://issues.apache.org/jira/browse/HADOOP-4885
It won't help you right know but it could allow you next time to avoid
restarting.
Regards
Bertrand
On Mon, Mar 18, 2013 at 3:52 PM, Brennon Church bren...@getjar.com wrote:
Hello all,
We have our
Hi list,
I'm using Hadoop 1.0.3 for a MapReduce task and I thought it might be a simple
job to append a Counter value and some text to the end of a file (which
ultimately will be in AWS S3). How wrong I was :)
I've been reading about o.a.h.fs.FileSystem.append and whether it does or
Need some guidance on CDH4 installation from tarballs
I have downloaded two files from
https://ccp.cloudera.com/display/SUPPORT/CDH4+Downloadable+Tarballs
*1) hadoop-0.20-mapreduce-0.20.2+1341 *(has only MRv1)*
2) hadoop-2.0.0+922 *(has HDFS+ Yarn)
I was able to install MRv1 from first file
Hello,
I am using one of the old legacy version (0.20) of hadoop for our cluster. We
have scheduled for an upgrade to the newer version within a couple of months,
but I would like to understand a couple of things before moving towards the
upgrade plan.
We have about 200 datanodes and some of
Hi,
It is not explicitly said but did you use the balancer?
http://hadoop.apache.org/docs/r1.0.4/commands_manual.html#balancer
Regards
Bertrand
On Mon, Mar 18, 2013 at 10:01 PM, Tapas Sarangi tapas.sara...@gmail.comwrote:
Hello,
I am using one of the old legacy version (0.20) of hadoop for
Hi,
Sorry about that, had it written, but thought it was obvious.
Yes, balancer is active and running on the namenode.
-Tapas
On Mar 18, 2013, at 4:43 PM, Bertrand Dechoux decho...@gmail.com wrote:
Hi,
It is not explicitly said but did you use the balancer?
And by active, it means that it does actually stops by itself? Else it
might mean that the throttling/limit might be an issue with regard to the
data volume or velocity.
What threshold is used?
About the small and big datanodes, how are they distributed with regards to
racks?
About files, how is
Appending on 1.x releases is available but not tested/supported and
can be toggled to be disabled completely. Appending works better on
2.x releases.
On Mon, Mar 18, 2013 at 9:14 PM, Tony Burton tbur...@sportingindex.com wrote:
Hi list,
I’m using Hadoop 1.0.3 for a MapReduce task and I
Just curious, why are we recommending one disables permissions rather
than trying to make them understand it?
On Mon, Mar 18, 2013 at 2:03 PM, Brahma Reddy Battula
brahmareddy.batt...@huawei.com wrote:
you need to disable following property..
property
namedfs.permissions.enabled/name
Maybe you need to modify the rackware script to make the rack balance,
ie, all the racks are the same size, on rack by 6 small nodes, one rack
by 1 large nodes.
P.S.
you need to reboot the cluster for rackware script modify.
于 2013/3/19 7:17, Bertrand Dechoux 写道:
And by active, it means that
On Mar 18, 2013, at 6:17 PM, Bertrand Dechoux decho...@gmail.com wrote:
And by active, it means that it does actually stops by itself?
Else it might mean that the throttling/limit might be an issue with regard to
the data volume or velocity.
This else is probably what's happening. I just
Hi,
On Mar 18, 2013, at 8:21 PM, 李洪忠 lhz...@hotmail.com wrote:
Maybe you need to modify the rackware script to make the rack balance, ie,
all the racks are the same size, on rack by 6 small nodes, one rack by 1
large nodes.
P.S.
you need to reboot the cluster for rackware script modify.
What do you mean that the balancer is always active? It is to be used
as a tool and it exits once it balances in a specific run (loops until
it does, but always exits at end). The balancer does balance based on
usage percentage so that is what you're probably looking for/missing.
On Tue, Mar 19,
It just unit test, so you don't need to set any parameters in configuration
files.
2013/3/18 Agarwal, Nikhil nikhil.agar...@netapp.com
Hi,
** **
Thanks for the quick reply. In order to test the class
TestInMemoryNativeS3FileSystemContract and its functions what should be the
value
I am assuming you refer to the YARN's CapacityScheduler.
The CS in YARN does support parallel job (the right term is application, or
'app', not 'job' anymore, when speaking in YARN's context) execution. If
looking at code of CapacityScheduler.java and LeafQueue.java, you can
notice it iterate
24 matches
Mail list logo
