hadoop 2.0 (cloudera cdh 4.7)
Am 21.08.2014 um 16:23 schrieb Liu, Yi A:
Which version are you using?
Regards,
Yi Liu
-Original Message-
From: norbi [mailto:no...@rocknob.de]
Sent: Wednesday, August 20, 2014 10:14 PM
To: user@hadoop.apache.org
Subject: hdfs dfsclient, possible to force
If you want to use NLineInputFormat, and also want the individual file to be
processed in the map task which prefer to be on the same task node as data
node, you need to implement and control that kind of logic by yourself.
Extend the NLineInputFormat, Override the getSplits() method, read the l
Hello,
This means that a file with names of all the files that need to be
processed and is fed to hadoop with NLineInputFormat?
If this is the case, then, how can we ensure that map processes are
scheduled in the node where blocks containing the files are stored already?
regards
rab
On Thu, Au
Hello,
Sorry for posting to the yarn issue list before.
I have a job with no reducers and 20 mappers, out of which 14 get scheduled
at a time. But strangely even if mappers reach a SUCEEDED status the
container running them is not killed. Due to which the next set of mappers
never get scheduled.
If I were you, I’ll first generate a file with those file name:
hadoop fs -ls > term_file
Then run the normal map reduce job
Felix
On Aug 21, 2014, at 1:38 AM, rab ra wrote:
> Thanks for the link. If it is not required for CFinputformat to have contents
> of the files in the map process but
Yes, that’s correct behavior.
You remove the file, but snapshot is there and it has ref for that file, so the
blocks will not be removed. Only after you delete all snapshots and original
file, then blocks are removed.
Keep in mind that blocks in datanodes are not copied for snapshot.
Regards,
Y
Which version are you using?
Regards,
Yi Liu
-Original Message-
From: norbi [mailto:no...@rocknob.de]
Sent: Wednesday, August 20, 2014 10:14 PM
To: user@hadoop.apache.org
Subject: hdfs dfsclient, possible to force storage datanode ?
hi list,
we have 52 DNs and more hundred clients the
Escape character is \, but please enclose with single quote
For example /foo/{123} should be '/foo/\{123\}'
Regards,
Yi Liu
From: varun kumar [mailto:varun@gmail.com]
Sent: Thursday, August 21, 2014 2:21 PM
To: user; praveen...@gmail.com
Subject: Re: Delete a folder name containing *
Make s
Hi Narayanan,
I've read a great blog post by Rohit Bakhshi before, recommend it to you :
http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/. I
think most your questions are covered by this blog post. Please let me know
if you have more questions.
Thanks,
Wangda Tan
On Wed, A
Sorry I forgot to add Datanode logs:
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: RemoteException in
offerService
org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache
Hi
We are using Yarn 2.4.1 build from svn as follows "package -Pdist -Dtar
-DskipTests". Build is successful and YARN is starting also successful.
However when we are trying to copy data to HDFS (copyFromLocal) we have the
following exception:
Namenode.log
java.lang.NoSuchMethodError:
org.apache.
Hi
Can some one give a example of TotalOrderPartitioner. I saw some exapmles
they had used partition file as a input and also used InputSampler.
http://grokbase.com/p/cloudera/cdh-user/12b2jydsce/totalorderpartitioner-cannot-find-partition-lst-file
Can some explain how this works.
Thanks
Pratap
On Wed, Aug 20, 2014 at 11:54 PM, Ken Krugler
wrote:
>
>
> PS - And why, oh why is "target" hard-coded all over the place in the
> mini-cluster code as the directory (from CWD) for logs, data blocks, etc?
>
>
https://issues.apache.org/jira/browse/YARN-1442
- André
--
André Kelpe
an...@concurr
Thanks for the link. If it is not required for CFinputformat to have
contents of the files in the map process but only the filename, what
changes need to be done in the code?
rab.
On 20 Aug 2014 22:59, "Felix Chern" wrote:
> I wrote a post on how to use CombineInputFormat:
>
> http://www.idryman
14 matches
Mail list logo
