Re: Query on Profiling Spark Code

2018-07-31 Thread Aakash Basu
t; > *From:* Aakash Basu [mailto:aakash.spark@gmail.com] > *Sent:* Tuesday, July 17, 2018 12:41 PM > *To:* user > *Subject:* Query on Profiling Spark Code > > > > Hi guys, > > > > I'm trying to profile my Spark code on cProfiler and check where more time > is tak

RE: Query on Profiling Spark Code

2018-07-31 Thread Patil, Prashasth
Hi Aakash, On a related note, you may want to try SparkLens for profiling which is quite helpful in my opinion. -Prash From: Aakash Basu [mailto:aakash.spark@gmail.com] Sent: Tuesday, July 17, 2018 12:41 PM To: user Subject: Query on Profiling Spark Code Hi guys, I'm trying to profile my

Query on Profiling Spark Code

2018-07-17 Thread Aakash Basu
Hi guys, I'm trying to profile my Spark code on cProfiler and check where more time is taken. I found the most time taken is by some socket object, which I'm quite clueless of, as to where it is used. Can anyone shed some light on this?