Recursive nested wildcard directory walking in Spark

Hi! My name is James, and I’m working on a question there doesn’t seem to be a lot of answers about online. I was hoping spark/hadoop gurus could shed some light on this. I have a data feed on NFS that looks like /foobar/.gz Currently I have a spark scala job that calls

Re: Recursive nested wildcard directory walking in Spark

antir.com> Cc: "user@spark.apache.org" <user@spark.apache.org> Subject: Re: Recursive nested wildcard directory walking in Spark Have you seen this thread ? http://search-hadoop.com/m/q3RTt2uhMX1UhnCc1=Re+Does+sc+newAPIHadoopFil e+support+multiple+directories+o

Re: Recursive nested wildcard directory walking in Spark

Have you seen this thread ? http://search-hadoop.com/m/q3RTt2uhMX1UhnCc1=Re+Does+sc+newAPIHadoopFile+support+multiple+directories+or+nested+directories+ FYI On Wed, Dec 9, 2015 at 11:18 AM, James Ding wrote: > Hi! > > My name is James, and I’m working on a question there