reduce finished container statuses not present at AM

2015-06-01 Thread Grandl Robert
Hi guys,
I was running a simple Terasort job with 4 mappers and 2 reducers using Hadoop 
3.0.0-Snapshot(trunk).
I was analyzing the finished containers reported to AM in 
RMContainerAllocator.java -> getResources(), but I realized that none of the 
finished containers for reducers are sent back to AM. Is this a bug or as 
designed ? 

I remember I did a similar analysis for Hadoop 2.3 and I got finished 
containers statuses both for mappers and reducers. 

Could someone explain me what is the case ?
Thank you,Robert



Copy the content of the input splits

2012-08-22 Thread Grandl Robert
Hi,

Do you know how I can copy using Hadoop code the content of input splits into 
local files ? I saw something like InputSplit class but still not able to 
figure out how to copy it. Is there somewhere in the code something similar I 
can look at ?

Thanks,
Robert

Splitting input file

2012-08-21 Thread Grandl Robert
Hi,

I think there are many discussions about splitting the input file based on 
custom delimiters. 

However, I am not sure if there is a simple way to split text input file based 
on end of sentences(.) without writing any custom split delimiter or so. Can I 
simply specify such delimiter when I add the input into HDFS ?

Thanks,
Robert

Copy the input block files

2012-08-21 Thread Grandl Robert
Hi,

I have an input which I divided into 256 blocks into hdfs, each of 10 MB. 

My question is: How can I download some of these blocks only ? I tried from the 
Web UIĀ  and look at "Advanced view/download options" where all the input blocks 
are listed. However, when I try to specify "Chunk size to view (in bytes, up to 
file's DFS block size):" I got an error like: "Invalid input (genstamp absent)"

Do you know other way to copy the content of some blocks ? I am using Hadoop 
1.0.3

Thanks,
Robert