Hi,
I want to custom FileInputFormat.class. In order to determine which host
the specific part of a file belongs to, I need to open the file in HDFS and
read some information. It will take me nearly 500ms to open a file and get
the information I need. But now I have thousands of files to deal
wrote:
Hi,
I want to custom FileInputFormat.class. In order to determine which host
the specific part of a file belongs to, I need to open the file in HDFS and
read some information. It will take me nearly 500ms to open a file and get
the information I need. But now I have thousands of files