local file input for seqdirectory

2014-03-13 Thread Steven Cullens
Hi, I have a large number of files on the order of kilobytes on my local machine that I want to convert to a sequence file on HDFS. Whenever, I try to copy the local files to HDFS, hadoop complains about bad blocks, presumably because each block is 64mb and there are more files than blocks. In

Re: local file input for seqdirectory

2014-03-13 Thread Suneel Marthi
The workaround is to add -xm sequential. A MR version of seqdirectory was introduced in 0.8 and hence the default execution mode is MR if none is specified. On Thursday, March 13, 2014 4:12 PM, Steven Cullens srcull...@gmail.com wrote: Hi, I have a large number of files on the order of

Re: local file input for seqdirectory

2014-03-13 Thread Steven Cullens
Thanks, Suneel. On Thu, Mar 13, 2014 at 4:17 PM, Suneel Marthi suneel_mar...@yahoo.comwrote: The workaround is to add -xm sequential. A MR version of seqdirectory was introduced in 0.8 and hence the default execution mode is MR if none is specified. On Thursday, March 13, 2014 4:12