Yes. Renaming the file or moving it will help you avoid a lot of issues. If
it's possible it will be even better to write a set of files to a temp
directory and then moving the directory to some staging/processing dir as it
would reduce the load on the NameNode.

Thanks
Sudhan S

On Wed, Aug 17, 2011 at 11:11 PM, Adam Shook <ash...@clearedgeit.com> wrote:

> Sadly, I don't have control over naming the files.  They are being ingested
> in HDFS by powers out of my control.  I'll mess around with the modification
> times and see if I can get a good solution.  If anyone knows of a way that
> seems less hackish, I am all ears.
>
> Thanks, Adam
>
>
>
>
> -----Original Message-----
> From: David Rosenstrauch [mailto:dar...@darose.net]
> Sent: Wednesday, August 17, 2011 1:22 PM
> To: hdfs-user@hadoop.apache.org
> Subject: Re: HDFS File being written
>
> On 08/17/2011 12:57 PM, Adam Shook wrote:
> > Hello All,
> >
> > Is there any clean way to tell from the API (v0.20.2) that a file in HDFS
> is currently being written to?  I've seen some exceptions before related to
> it, but I was hoping there is a clean way and Google isn't turning anything
> up for me.
> >
> > Thanks!
> > -- Adam
>
> You might be able to do it to some extent using
> FileStatus.getModificationTime()
> (
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileStatus.html#getModificationTime()
> ),
> but this would really be a hack, IMO, and not something you should rely on.
>
> I think you'd be better off either a) writing the file to a temp
> directory, or b) writing it with a .tmp extension, and then moving or
> renaming it once the file write is complete.
>
> HTH,
>
> DR
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 10.0.1392 / Virus Database: 1520/3840 - Release Date: 08/17/11
>

Reply via email to