issue about read file from HDFS

hi,mailist: when HDFS 's file is in appending ,no other reader can get data from this file,so when i do some statistics action every five minuters use hive read hdfs file ,i can not read data ,any good way can offer? thanks

Re: issue about read file from HDFS

One suggestion is change your hive partition, add a hive partition every five minutes, and your HDFS file also roller every five minutes. On Wed, Dec 4, 2013 at 11:56 AM, ch huang justlo...@gmail.com wrote: hi,mailist: when HDFS 's file is in appending ,no other reader can get

Re: issue about read file from HDFS

it seems not a good suggestion,get lot of partition dir and data file will be a big compact to NN On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu azury...@gmail.com wrote: One suggestion is change your hive partition, add a hive partition every five minutes, and your HDFS file also roller every

Re: issue about read file from HDFS

Ok, so you go for yourself. On Wed, Dec 4, 2013 at 12:15 PM, ch huang justlo...@gmail.com wrote: it seems not a good suggestion,get lot of partition dir and data file will be a big compact to NN On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu azury...@gmail.com wrote: One suggestion is change