Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-21 Thread Grega Kešpret
Hi, is this fixed in master? Grega On Thu, May 14, 2015 at 7:50 PM, Michael Armbrust wrote: > End of the month is the target: > https://cwiki.apache.org/confluence/display/SPARK/Wiki+Homepage > > On Thu, May 14, 2015 at 3:45 AM, Ishwardeep Singh < > ishwardeep.si...@impetus.co.in> wrote: > >>

Shuffle size difference - operations on RDD vs. operations on SchemaRDD

2014-09-21 Thread Grega Kešpret
Hi, I am seeing different shuffle write sizes when using SchemaRDD (versus normal RDD). I'm doing the following: case class DomainObj(a: String, b: String, c: String, d: String) val logs: RDD[String] = sc.textFile(...) val filtered: RDD[String] = logs.filter(...) val myDomainObjects: RDD[DomainO

Re: java.io.IOException: Failed to save output of task

2014-05-22 Thread Grega Kešpret
I have since resolved the issue. The problem was that multiple rdds were trying to write to the same s3 bucket. Grega -- [image: Inline image 1] *Grega Kešpret* Analytics engineer Celtra — Rich Media Mobile Advertising celtra.com <http://www.celtra.com/> | @celtramobile<http://www.tw

java.io.IOException: Failed to save output of task

2014-05-21 Thread Grega Kešpret
xecutor.Executor$TaskRunner.run(Executor.scala:178) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Grega -- [image: Inline