hi there I am trying to descrease my app's running time in worker node. I
checked the log and found the most time-wasting part is below:15/06/08 16:14:23
INFO storage.MemoryStore: Block broadcast_0 stored as values in memory
(estimated size 2.1 KB, free 353.3 MB)
15/06/08 16:14:42 INFO exec
May be you should check in your driver UI and see if there's any GC time
involved etc.
Thanks
Best Regards
On Mon, Jun 8, 2015 at 5:45 PM, wrote:
> hi there
>
> I am trying to descrease my app's running time in worker node. I
> checked the log and found the most time-wasting part is below
Best Regards
On Tue, Jun 9, 2015 at 2:09 PM, wrote:
> Only 1 minor GC, 0.07s.
>
>
>
>
> Thanks&Best regards!
> San.Luo
>
> - 原始邮件 -
> 发件人:Akhil Das
> 收件人:罗辉
> 抄送人:user
> 主题:Re: How to decrease the time of storing
Only 1 minor GC, 0.07s.
Thanks&Best regards!
San.Luo
- 原始邮件 -
发件人:Akhil Das
收件人:罗辉
抄送人:user
主题:Re: How to decrease the time of storing block in memory
日期:2015年06月09日 15点02分
May be you should check in your driver UI and see if there's an
> /opt/data/shellcompare/data/user" + j + "/pgs/intermediateResult/result" +
> i + ".txt 600")
>
> pipeModify2.collect()
>
> sc.stop()
> }
> }
>
>
>
>
> Thanks&Best regards!
> San.Luo
llcompare/data/user" + j +
"/pgs/intermediateResult/result" + i + ".txt 600")
pipeModify2.collect()sc.stop()
}
}
--------------------
Thanks&Best regards!
San.Luo
- 原始邮件 -
发件人:Akhil Das
收件人:罗辉
抄送人:user
主题:Re: Re: How to decrease the time of storing block i
i + ".txt 600")
pipeModify2.collect() sc.stop()
}
}
--------------------
Thanks&Best regards!
San.Luo
- 原始邮件 -
发件人:Akhil Das
收件人:罗辉
抄送人:user
主题:Re: Re: How to decrease the time of storing block in memory
日期:2015年06月09日 16点51分
Is it that task taking 1