Hi,
is this fixed in master?
Grega
On Thu, May 14, 2015 at 7:50 PM, Michael Armbrust
wrote:
> End of the month is the target:
> https://cwiki.apache.org/confluence/display/SPARK/Wiki+Homepage
>
> On Thu, May 14, 2015 at 3:45 AM, Ishwardeep Singh <
> ishwardeep.si...@impetus.co.in> wrote:
>
>>
Hi,
I am seeing different shuffle write sizes when using SchemaRDD (versus
normal RDD). I'm doing the following:
case class DomainObj(a: String, b: String, c: String, d: String)
val logs: RDD[String] = sc.textFile(...)
val filtered: RDD[String] = logs.filter(...)
val myDomainObjects: RDD[DomainO
I have since resolved the issue. The problem was that multiple rdds were
trying to write to the same s3 bucket.
Grega
--
[image: Inline image 1]
*Grega Kešpret*
Analytics engineer
Celtra — Rich Media Mobile Advertising
celtra.com <http://www.celtra.com/> |
@celtramobile<http://www.tw
xecutor.Executor$TaskRunner.run(Executor.scala:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Grega
--
[image: Inline