Personally I would just use Har :) It sounds like an interesting
project. You might find this document helpful:
http://kazman.shidler.hawaii.edu/ArchDoc.html
It was designed to help contributors navigate the HDFS source tree.
-Joey
On Thu, Jan 19, 2012 at 11:52 AM, Sesha Kumar wrote:
> I'm cu
I'm currently working on this paper where we try to improve the
performance of HDFS by combining small files into a single file (like HAR),
but this merged file contains at the beginning of each block an index file
which is similar to HAR index file. Datanode uses this index file to obtain
the sma
sesha...@gmail.com]
Sent: Wednesday, January 18, 2012 8:24 PM
To: hdfs-user@hadoop.apache.org
Subject: Re: Data processing in DFSClient
Sorry for the delay. I'm trying to implement an IEEE paper which combines a
bunch of files into a single file and when the file is requested the datanode
extracts t
Sorry for the delay. I'm trying to implement an IEEE paper which combines a
bunch of files into a single file and when the file is requested the
datanode extracts the desired file from the block and sends the file to
DFSClient.
Sesha,
What kind of processing are you attempting to do? Maybe it makes more sense
to just implement a MapReduce job rather than modifying the datanodes?
-Joey
On Mon, Jan 16, 2012 at 9:20 AM, Sesha Kumar wrote:
> Hey guys,
>
> Sorry for the typo in my last message.I have corrected it.
>
> I w
Hi Shesha,
Take a look at org.apache.hadoop.hdfs.server.datanode.BlockSender.java
Regards,
Uma
From: Sesha Kumar [sesha...@gmail.com]
Sent: Monday, January 16, 2012 7:50 PM
To: hdfs-user@hadoop.apache.org
Subject: Data processing in DFSClient
Hey guys,
Sorry f
