You can use Hadoop APi & provide input/output reader & hadoop configuration file to read the data. Regards Mayur
Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Wed, May 28, 2014 at 7:22 PM, Laurent T <laurent.thou...@ldmobile.net>wrote: > Hi, > > I have a bunch of files that are bz2 compressed but do not have the > extension .bz2 > Is there anyway to force spark to read them as bz2 files using sc.textFile > ? > > FYI, if i add the .bz2 extension to the file it works fine but the process > that creates those files can't do that and i'd like to find another way to > make this work than renaming all the files before executing my Spark job. > > Thanks > Regards > Laurent > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Reading-bz2-files-that-do-not-end-with-bz2-tp6473.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
