Yes. Renaming the file or moving it will help you avoid a lot of issues. If it's possible it will be even better to write a set of files to a temp directory and then moving the directory to some staging/processing dir as it would reduce the load on the NameNode.
Thanks Sudhan S On Wed, Aug 17, 2011 at 11:11 PM, Adam Shook <ash...@clearedgeit.com> wrote: > Sadly, I don't have control over naming the files. They are being ingested > in HDFS by powers out of my control. I'll mess around with the modification > times and see if I can get a good solution. If anyone knows of a way that > seems less hackish, I am all ears. > > Thanks, Adam > > > > > -----Original Message----- > From: David Rosenstrauch [mailto:dar...@darose.net] > Sent: Wednesday, August 17, 2011 1:22 PM > To: hdfs-user@hadoop.apache.org > Subject: Re: HDFS File being written > > On 08/17/2011 12:57 PM, Adam Shook wrote: > > Hello All, > > > > Is there any clean way to tell from the API (v0.20.2) that a file in HDFS > is currently being written to? I've seen some exceptions before related to > it, but I was hoping there is a clean way and Google isn't turning anything > up for me. > > > > Thanks! > > -- Adam > > You might be able to do it to some extent using > FileStatus.getModificationTime() > ( > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileStatus.html#getModificationTime() > ), > but this would really be a hack, IMO, and not something you should rely on. > > I think you'd be better off either a) writing the file to a temp > directory, or b) writing it with a .tmp extension, and then moving or > renaming it once the file write is complete. > > HTH, > > DR > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 10.0.1392 / Virus Database: 1520/3840 - Release Date: 08/17/11 >