Can you give us a bit more information ?
how you packaged the code into jar
command you used for execution
version of Spark
related log snippet
Thanks
On Mon, Jun 6, 2016 at 10:43 AM, Daniel Haviv <
daniel.ha...@veracity-group.com> wrote:
> Hi,
> I'm wrapped the following code into a jar:
>
>
Hi,
I'm wrapped the following code into a jar:
val test = sc.parallelize(Seq(("daniel", "a"), ("daniel", "b"), ("test", "1)")))
val agg = test.groupByKey()
agg.collect.foreach(r=>{println(r._1)})
The result of groupByKey is an empty RDD, when I'm trying the same
code using the spark-shell it's
this:
emptyRDD.mapPartitionsWithIndex(...), but it doesn't work. So it seems it is
not suitable to use emptyRDD to start tasks on the worker side.It is so
appreciated if you can give me some suggestions.Thanks.
See
http://spark.apache.org/docs/0.8.1/api/core/org/apache/spark/rdd/EmptyRDD.html
On Nov 14, 2014, at 2:09 AM, Deep Pradhan pradhandeep1...@gmail.com wrote:
How to create an empty RDD in Spark?
Thank You
If I remember correctly, EmptyRDD is private [spark]
You can create an empty RDD using the spark context:
val emptyRdd = sc.emptyRDD
-kr, Gerard.
On Fri, Nov 14, 2014 at 11:22 AM, Deep Pradhan pradhandeep1...@gmail.com
wrote:
To get an empty RDD, I did this:
I have an rdd with one
It looks like an Scala issue. Seems like the implicit conversion to
ArrayOps does not apply if the type is Array[Nothing].
Try giving a type to the empty RDD:
val emptyRdd: RDD[Any] = sc.EmptyRDD
emptyRdd.collect.foreach(println) // prints a line return
-kr, Gerard.
On Fri, Nov 14, 2014
