You've been misled by the GUI you use, I'm afraid. Many DEs (Desktop Environments) consider ~-suffix files as hidden but not the general standards (try ls for example, or even shell expansions, it will ignore . prefixes, but not ~ suffixes) :)
To answer specifically though, no, the base FileInputFormat does not recognize ~ today, but if you want it to, you can pass a custom path filter to your InputFormat's implementation for when it calls the listStatus method. On Wed, Apr 3, 2013 at 3:16 PM, Mohammad Tariq <donta...@gmail.com> wrote: > Hello Harsh, > > Thank you for the response. I am sorry for being unclear. > Actually I was talking about the backup files which end with "~" > I mean these files are not visible normally, but my job is able to > see them. Does FileInputFormat behave in the same way for "~" > as it does in the case of "." and "_"? > > Thanks. > > Warm Regards, > Tariq > https://mtariq.jux.com/ > cloudfront.blogspot.com > > > On Wed, Apr 3, 2013 at 7:45 AM, Harsh J <ha...@cloudera.com> wrote: >> >> Not quite sure if I got your question. These tidbits may help though, >> from what I can understand: >> >> * LocalFileSystem's listing uses Java's APIs for file/dir listing, and >> has no concept of what a hidden file is on its own. It retrieves the >> whole list. >> * MR's FileInputFormat (and normal derivatives) does filter away "." >> and "_" starting path names, from added input paths to the job. >> >> On Wed, Apr 3, 2013 at 3:09 AM, Mohammad Tariq <donta...@gmail.com> wrote: >> > >> > Warm Regards, >> > Tariq >> > https://mtariq.jux.com/ >> > cloudfront.blogspot.com >> > >> > >> > ---------- Forwarded message ---------- >> > From: Mohammad Tariq <donta...@gmail.com> >> > Date: Tue, Apr 2, 2013 at 5:16 PM >> > Subject: MapReduce on Local files >> > To: mapreduce-u...@hadoop.apache.org >> > >> > >> > Hello list, >> > >> > Is a MR job capable of reading even the hidden temp files >> > present >> > inside a directory located on my local FS?I have noticed this thing >> > today >> > for the first time because till now I never tried running MR jobs on >> > local >> > files. >> > >> > Thank you so much for your time? >> > >> > Warm Regards, >> > Tariq >> > https://mtariq.jux.com/ >> > cloudfront.blogspot.com >> > >> >> >> >> -- >> Harsh J > > -- Harsh J
