have you tried --files <your h5 file>? > On Jun 15, 2016, at 18:50, ar7 <ashraag...@gmail.com> wrote: > > I am using PySpark 1.6.1 for my spark application. I have additional modules > which I am loading using the argument --py-files. I also have a h5 file > which I need to access from one of the modules for initializing the > ApolloNet. > > Is there any way I could access those files from the modules if I put them > in the same archive? I tried this approach but it was throwing an error > because the files are not there in every worker. I can think of one solution > which is copying the file to each of the workers but I want to know if there > are better ways to do it? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Adding-h5-files-in-a-zip-to-use-with-PySpark-tp27173.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org >
--------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
