unsubscribe
Hi Deng.
Thanks for the response.
Is it possible to load sequence files parallely and process each of it in
parallel...?
Regards
Vinoth Sankar
On Fri, Oct 30, 2015 at 2:56 PM Deng Ching-Mallete <och...@apache.org>
wrote:
> Hi,
>
> You seem to be creating a new RDD for each
logs which are present
inside map,filter closures. But logs outside these closures are getting
printed as usual. If i can't get the logger prints inside these closures
how do i debug them ?
Thanks
Vinoth Sankar
On Wed, Oct 28, 2015 at 8:29 PM Adrian Tanase <atan...@adobe.com> wrote:
files = javaSparkContext.parallelize(fileList)
Iterator localIterator = files.toLocalIterator();
Regards
Vinoth Sankar
. And i'm always getting "Active Jobs" as 1 in Spark
UI. Am I missing anything.? How do I Check whether my Spark Jobs are
paralleled or not ?
Regards
Vinoth Sankar
I'm just reading data from HDFS through Spark. It throws
*java.lang.ClassCastException:
org.apache.hadoop.io.LongWritable cannot be cast to
org.apache.hadoop.io.BytesWritable* at line no 6. I never used LongWritable
in my code, no idea how the data was in that format.
Note : I'm not using
es.nn",
"spark.hadoop.dfs.namenode.rpc-address.nn",
"spark.hadoop.dfs.namenode.http-address.nn" and other core-site & hdfs-site
conf properties in SparkConf Object. But still i get UnknownHostException.
Regards
Vinoth Sankar
)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.net.UnknownHostException: ABC ... 38 more
Regards
Vinoth Sankar
