Custom FileInputFormat.class

2014-12-01 Thread 胡斐
Hi, I want to custom FileInputFormat.class. In order to determine which host the specific part of a file belongs to, I need to open the file in HDFS and read some information. It will take me nearly 500ms to open a file and get the information I need. But now I have thousands of files to deal

Re: Custom FileInputFormat.class

2014-12-01 Thread Pradeep Gollakota
wrote: Hi, I want to custom FileInputFormat.class. In order to determine which host the specific part of a file belongs to, I need to open the file in HDFS and read some information. It will take me nearly 500ms to open a file and get the information I need. But now I have thousands of files