Re: How do I find the time taken by each step in a stage in a Spark Job

Yes, the Spark UI has some information but, it's not that helpful to find out which particular stage is taking time. On Wed, Jun 28, 2017 at 12:51 AM, 萝卜丝炒饭 <1427357...@qq.com> wrote: > You can find the information from the spark UI > > ---Original--- > *From:* "SRK" >

Re: How do I find the time taken by each step in a stage in a Spark Job

You can find the information from the spark UI ---Original--- From: "SRK" Date: 2017/6/28 02:36:37 To: "user"; Subject: How do I find the time taken by each step in a stage in a Spark Job Hi, How do I find the time taken by each step in a

How do I find the time taken by each step in a stage in a Spark Job

Hi, How do I find the time taken by each step in a stage in spark job? Also, how do I find the bottleneck in each step and if a stage is skipped because of the RDDs being persisted in streaming? I am trying to identify which step in a job is taking time in my Streaming job. Thanks! -- View