newAPIHadoopFile throws a JsonMappingException: Infinite recursion (StackOverflowError) error

I am trying to create a new JavaPairRDD from data in an HDFS file. My code is: sparkContext = new JavaSparkContext("yarn-client", "SumFramesPerTimeUnit", sparkConf); JavaPairRDD inputRDD = sparkContext.newAPIHadoopFile(fileFilter, FixedLengthInputFormat.class,

RE: submitting a spark job using yarn-client and getting NoClassDefFoundError: org/apache/spark/Logging

ncher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) I’ve checked and the class does live in the spark assembly. Any thoughts as what might be wrong? Best Regards, David R Robison Senior Systems Engineer [cid:image004.png@01D19182.F24CA3E0] From: Dav

RE: Problem submitting a spark job using yarn-client as master

r of the JavaSparkContext. I have a logging call right after creating the SparkContext and it is never executied. Any idea what I’m doing wrong? David Best Regards, David R Robison Senior Systems Engineer [cid:image004.png@01D19182.F24CA3E0] From: Rohit Verma [mailto:rohit.ve...@rokittech.com]

Problem submitting a spark job using yarn-client as master

I am trying to submit a spark job through the yarn-client master setting. The job gets created and submitted to the clients but immediately errors out. Here is the relevant portion of the log: 15:39:37,385 INFO [org.apache.spark.deploy.yarn.Client] (default task-1) Requesting a new

creating a javaRDD using newAPIHadoopFile and FixedLengthInputFormat

I am trying to create a Spark javaRDD using the newAPIHadoopFile and the FixedLengthInputFormat. Here is my code snippit, Configuration config = new Configuration(); config.setInt(FixedLengthInputFormat.FIXED_RECORD_LENGTH, JPEG_INDEX_SIZE); config.set("fs.hdfs.impl",