Good timing! I encountered that same issue recently and to address it, I changed the default Class.forName call to Utils.classForName. See my patch at https://github.com/apache/spark/pull/1916. After that change, my bin/pyspark --jars worked.
On Wed, Aug 13, 2014 at 11:47 PM, Tassilo Klein <[email protected]> wrote: > Thanks. This was already helping a bit. But the examples don't use custom > InputFormats. Rather, org.apache fully qualified InputFormat. If I want to > use my own custom InputFormat in form of .class (or jar) how can I use it? > I > tried providing it to pyspark with --jars <myCustomInputFormat.jar> > > and then using sc.newAPIHadoopFile(path, > <myCustomFullyQualifiedPackageName.ClassName>, .....) > > However, that didn't work as it couldn't find the class. > > Any other idea? > > Thanks so far, > -Tassilo > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Using-Hadoop-InputFormat-in-Python-tp12067p12092.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
