Re: issue with spark and bson input

2014-08-06 Thread Dmitriy Selivanov
Finally I made it work. The trick was in asSubclass method: val mongoRDD = sc.newAPIHadoopFile(file:///root/jobs/dump/input.bson, classOf[BSONFileInputFormat].asSubclass(classOf[org.apache.hadoop.mapreduce.lib.input.FileInputFormat[Object, BSONObject]]), classOf[Object], classOf[BSONObject],

issue with spark and bson input

2014-08-05 Thread Dmitriy Selivanov
Hello, I have issue when try to use bson file as spark input. I use mongo-hadoop-connector 1.3.0 and spark 1.0.0: val sparkConf = new SparkConf() val sc = new SparkContext(sparkConf) val config = new Configuration() config.set(mongo.job.input.format,