Re: Proposal for reading from / writing to archive files

2020-05-28 Thread Robert Bradshaw
On Thu, May 28, 2020 at 9:34 AM Chamikara Jayalath wrote: > Thanks for the contribution. This sounds very interesting. Few comments. > > * | fileio.MatchFiles('hdfs://path/to/*.zip') | fileio.ExtractMatches() | > fileio.MatchAll() > > We usually either do

Re: Proposal for reading from / writing to archive files

2020-05-28 Thread Chamikara Jayalath
Thanks for the contribution. This sounds very interesting. Few comments. * | fileio.MatchFiles('hdfs://path/to/*.zip') | fileio.ExtractMatches() | fileio.MatchAll() We usually either do 'fileio.MatchFiles('hdfs://path/to/*.zip')' or 'fileio.MatchAll()'. Former to read a specific glob and latter

Proposal for reading from / writing to archive files

2020-05-27 Thread Ashwin Ramaswami
I have a requirement where I need to read from / write to archive files (such as .tar, .zip). Essentially, I'd like to treat the entire .zip file I read from as a filesystem, so that I can only get the files I need that are within the archive. This is useful, because some archive formats such