Hello Community members, I am getting error while reading large JSON file in spark,
*Code:* val landingVisitor = sqlContext.read.json("s3n://hist-ngdp/lvisitor/lvisitor-01-aug.json") *Error:* 16/10/18 07:30:30 ERROR Executor: Exception in task 8.0 in stage 0.0 (TID 8) java.io.IOException: Too many bytes before newline: 2147483648 at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:249) at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174) at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:135) at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67) at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:237) What would be resolution for the same ? Thanks in Advance ! -- Yours Aye, Chetan Khatri.