Re: Nonempty files become empty after being saved to Hdfs.

hi Peter, There is no any operation log? Please using the state change log with debug level. I checked the code for creating 'symlink'. It looks like regular operation with other interfaces at NameNode. GC, directory lock, shell script may be doubtful point. -Regards Denny Ye 2012/2/24 Ti

Re: Nonempty files become empty after being saved to Hdfs.

Hi, Denny, Thanks for the info, you are right about his, after some digging, I found there were no data loss, the root cause of getting empty file is that during the file rotation, the old file was closed, the new empty new file was created/opened, then hdfs tried to create the small file(acting as

Writing files arrving via HTTP Post into HDFS - Approaches

Hi, My use case scenario involves data coming in periodically over https (post) from thousands of clients. Apache is the web server. About 50MB of data arrives every 15 minutes from each of these clients. I need to move this data into HDFS for further Mapreduce processing. Would appreciate if the

Re: Newbie question on block size calculation

Thanks very much for the clarification. So, we'd i guess ideally set the block size equal to the transfer rate for optimum results. If seek time has to be 0.5% of transfer time would i set my block size at 200MB (higher than transfer rate)? Conversely if seek time has to be 2% of transfer time wo