sequenceFile() calls hadoopFile() where: val confBroadcast = broadcast(new SerializableConfiguration(hadoopConfiguration))
You can set the parameter in sc.hadoopConfiguration before calling sc.sequenceFile(). Cheers On Sun, Jun 28, 2015 at 9:23 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com> wrote: > I can do this > > val hadoopConf = new Configuration(sc.hadoopConfiguration) > > * hadoopConf.set("mapreduce.input.fileinputformat.split.maxsize", > "67108864")* > > sc.newAPIHadoopFile( > > path + "/*.avro", > > classOf[AvroKeyInputFormat[GenericRecord]], > > classOf[AvroKey[GenericRecord]], > > classOf[NullWritable], > > hadoopConf) > > > But i cant do the same with > > sc.sequenceFile("path", classOf[Text], classOf[Text]) > How can i achieve the same with sequenceFile > -- > Deepak > >