Yes, but I am generating multiple output file names with prefix.
So I know their prefix size.
On 9/25/14, João Alves wrote:
> Hey Susheel,
>
> Maybe it would be a good idea to check if the string is long enough for your
> substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
Hey Susheel,
Maybe it would be a good idea to check if the string is long enough for your
substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay wrote:
> I solved it like this. This will move a file from one location to
> ano
Hi,
Which script is generated launch_container.sh file while we run "yarn
jar"? this script is located in
${yarn.nodemanager.local-dirs}/nm-local-dir/usercache/${username}/appcache/application_*\container_*/
I need to insert some command to change LOG_DIRS permission.
Thanks.
Hi!
I am not sure if this question must be posted in the Hadoop forum or in the
mongoDB one. Let´s try:
I am using the mongo-hadoop integration and wrote a MR job. I want to test it,
and foud the framework MRUnit (https://mrunit.apache.org/), which sounds great.
I am facing some difficulties w
Hi Jameel,
Datanodes on EMR are registered with namenode with their private DNS and
private hostname to Ip can only be resolved within the VPC.(through default
local DNS server attach to a aws VPC).
And, Client connecting to namenode will always get the
internal_hostname:port (socket) of datanodes
Hi Ankit,
Thanks for the info.
However, we are not using EMR. We are using our own cluster. We have tried
everything listed before in an attempt to get the nodes to register with their
public DNS name. The frustration comes from the fact that Hadoop/HDFS seemingly
ignores out attempts and stil
Hi all ,
I have a hive table partitioned with d (date) as string .
while i was running the query as
HI all,
I have a hive table partitioned with date (d) as string but while running
the query i am getting following exception.
SELECT * FROM click WHERE d>="2014-09-25" ;
FAILED: SemanticException MetaException(message:javax.jdo.JDOException:
Invocation of method "substring" on "StringExpressi
Hello,
I am having a Hadoop cluster with 1 name node and 3 data nodes. I running
sample word count job on 1GB of file which is distributed among the HDFS.
When I run the map reduce job, before even completing the mapping 100 %
reduce starts. Say for eg map 40% reduce 10% etc.
I would like to kn
Hi Guys,
I am new to yarn framework, for example I have a yarn cluster, I am
planning to submit mapreduce and storm and sprk applications,
Please correct me I am wrong.
when I submit each each jobs, It's create own instance of mapreduce,sprk
and storm application master in different nodes.
Que
see mapreduce.job.reduce.slowstart.completedmaps
It gives hint of when reduce tasks could kick off.
2014-09-26 8:36 GMT+08:00 Abdul Navaz :
> Hello,
>
> I am having a Hadoop cluster with 1 name node and 3 data nodes. I running
> sample word count job on 1GB of file which is distributed among the
The reducer starts as soon as it has data available from any one of the mappers.
The reducer keeps polling the AM and asks if any mapper has completed
processing. If so it fetches data from that mapper.
So it's not necessary for all the mappers of a task to complete for
the reducer to start process
12 matches
Mail list logo