Re: TaskCommitDenied (Driver denied task commit)

2016-01-22 Thread Arun Luthra
>>>>> with a count will mask this exception because the coordination does not >>>>> get >>>>> triggered in non save/write operations. >>>>> >>>>> On Thu, Jan 21, 2016 at 2:46 PM Holden Karau >>>>> wrote: >&g

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Arun Luthra
ng which most quickly jumps >>>>> out to me is groupByKey which could be causing some problems - whats the >>>>> distribution of keys like? Try replacing the groupByKey with a count() and >>>>> see if the pipeline works up until that stage. Also 1G of

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Arun Luthra
t;>> >>>>> >>>>> >>>>> 16/01/21 21:52:11 WARN NativeCodeLoader: Unable to load native-hadoop >>>>> library for your platform... using builtin-java classes where applicable >>>>> >>>>> 16/01/21 21:52:14 WAR

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Holden Karau
21 21:52:14 WARN MetricsSystem: Using default name DAGScheduler >>>> for source because spark.app.id is not set. >>>> >>>> spark.yarn.driver.memoryOverhead is set but does not apply in client >>>> mode. >>>> >>>> 16/01/21 21:52:1

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Arun Luthra
ng default name DAGScheduler >>> for source because spark.app.id is not set. >>> >>> spark.yarn.driver.memoryOverhead is set but does not apply in client >>> mode. >>> >>> 16/01/21 21:52:16 WARN DomainSocketFactory: The short-circuit local >>

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Josh Rosen
t;> in memory! (computed 60.2 MB so far) >> >> 16/01/21 21:52:52 WARN MemoryStore: Persisting block broadcast_4 to disk >> instead. >> >> [Stage 1:============>(2260 + 7) >> / 2262]16/01/21 21:57:24 WARN TaskSetMa

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Holden Karau
==>(2260 + 7) / > 2262]16/01/21 21:57:24 WARN TaskSetManager: Lost task 1440.1 in stage 1.0 > (TID 4530, --): TaskCommitDenied (Driver denied task commit) for job: 1, > partition: 1440, attempt: 4530 > > [Stage 1:==

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Arun Luthra
WARN MemoryStore: Persisting block broadcast_4 to disk instead. [Stage 1:>(2260 + 7) / 2262]16/01/21 21:57:24 WARN TaskSetManager: Lost task 1440.1 in stage 1.0 (TID 4530, --): TaskCommitDenied (Driver denied task commit) for job: 1, partit

Re: TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Holden Karau
nied (Driver denied task commit) for job: 1, > partition: 2168, attempt: 4436 > > > Is there a solution for this? Increase driver memory? I'm using just 1G > driver memory but ideally I won't have to increase it. > > The RDD being processed has 2262 partitions. > >

TaskCommitDenied (Driver denied task commit)

2016-01-21 Thread Arun Luthra
Example warning: 16/01/21 21:57:57 WARN TaskSetManager: Lost task 2168.0 in stage 1.0 (TID 4436, XXX): TaskCommitDenied (Driver denied task commit) for job: 1, partition: 2168, attempt: 4436 Is there a solution for this? Increase driver memory? I'm using just 1G driver memory but idea