Is 'hadoop' / 'hdfs' command accessible to your python script ? If so, you can call 'hdfs dfs -ls' from python.
Cheers On Sat, Jan 23, 2016 at 4:08 AM, Andrew Holway < andrew.hol...@otternetworks.de> wrote: > Hello, > > I would like to make a list of files (parquet or json) in a specific > HDFS directory with python so I can do some logic on which files to > load into a dataframe. > > Any ideas? > > Thanks, > > Andrew > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
