what program do u use to profile Spark? On Fri, Jun 23, 2017 at 3:07 PM, Marcelo Vanzin <van...@cloudera.com> wrote:
> That thread looks like the connection between the Spark process and > jvisualvm. It's expected to show high up when doing sampling if the > app is not doing much else. > > On Fri, Jun 23, 2017 at 10:46 AM, Reth RM <reth.ik...@gmail.com> wrote: > > Running a spark job on local machine and profiler results indicate that > > highest time spent in sun.rmi.transport.tcp.TCPTransport$ > ConnectionHandler. > > Screenshot of profiler result can be seen here : https://jpst.it/10i-V > > > > Spark job(program) is performing IO (sc.wholeTextFile method of spark > apis), > > Reads files from local file system and analyses the text to obtain > tokens. > > > > Any thoughts and suggestions? > > > > Thanks. > > > > > > -- > Marcelo > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
