Re: Regular expressions in fs paths?
I want to unsubscribe from this mailing list On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson charles.robert...@gmail.com wrote: Hi all, Is it possible to use regular expressions in fs commands? Specifically, I want to use the copy (-cp) and move (-mv) commands on all files in a directory that match a pattern (the pattern being all files that do not end in '.tmp'). Can this be done? Thanks, Charles
Re: Regular expressions in fs paths?
Yes you can : hadoop fs -ls /tmp/myfiles* I would recommend first using -ls in order to verify you are selecting the right files. #Mahesh : do you need some help doing this ? On 10.09.2014 13:46, Mahesh Khandewal wrote: I want to unsubscribe from this mailing list On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson charles.robert...@gmail.com mailto:charles.robert...@gmail.com wrote: Hi all, Is it possible to use regular expressions in fs commands? Specifically, I want to use the copy (-cp) and move (-mv) commands on all files in a directory that match a pattern (the pattern being all files that do not end in '.tmp'). Can this be done? Thanks, Charles
Re: Regular expressions in fs paths?
Hi Georgi, Thanks for your reply. Won't hadoop fs -ls /tmp/myfiles* return all files that begin with 'myfiles' in the tmp directory? What I don't understand is how I can specify a pattern that excludes files ending in '.tmp'. I have tried using the normal regular expression syntax for this ^(.tmp) but it tries to match it literally. Regards, Charles On 10 September 2014 13:07, Georgi Ivanov iva...@vesseltracker.com wrote: Yes you can : hadoop fs -ls /tmp/myfiles* I would recommend first using -ls in order to verify you are selecting the right files. #Mahesh : do you need some help doing this ? On 10.09.2014 13:46, Mahesh Khandewal wrote: I want to unsubscribe from this mailing list On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson charles.robert...@gmail.com wrote: Hi all, Is it possible to use regular expressions in fs commands? Specifically, I want to use the copy (-cp) and move (-mv) commands on all files in a directory that match a pattern (the pattern being all files that do not end in '.tmp'). Can this be done? Thanks, Charles
Re: Regular expressions in fs paths?
HDFS doesn't support he full range of glob matching you will find in Linux. If you want to exclude all files from a directory listing that meet a certain criteria try doing your listing and using grep -v to exclude the matching records.
Re: Regular expressions in fs paths?
I solved this in the end by using a shell script (initiated by an oozie shell action) to use grep and loop through the results - didn't have to use -v option, as the -e option gives you access to a fuller range of regular expression functionality. Thanks for your help (again!) Rich. Charles On 10 September 2014 16:50, Rich Haase rdha...@gmail.com wrote: HDFS doesn't support he full range of glob matching you will find in Linux. If you want to exclude all files from a directory listing that meet a certain criteria try doing your listing and using grep -v to exclude the matching records.
