Hi Rohit, Thanks for sharing this great tool. I tried running a spark job with the tool, but it failed with an *IncompatibleClassChangeError *Exception.
I have opened an issue on Github.( https://github.com/qubole/sparklens/issues/1) Shmuel On Thu, Mar 22, 2018 at 5:05 PM, Shmuel Blitz <shmuel.bl...@similarweb.com> wrote: > Thanks. > > We will give this a try and report back. > > Shmuel > > On Thu, Mar 22, 2018 at 4:22 PM, Rohit Karlupia <roh...@qubole.com> wrote: > >> Thanks everyone! >> Please share how it works and how it doesn't. Both help. >> >> Fawaze, just made few changes to make this work with spark 1.6. Can you >> please try building from branch *spark_1.6* >> >> thanks, >> rohitk >> >> >> >> On Thu, Mar 22, 2018 at 10:18 AM, Fawze Abujaber <fawz...@gmail.com> >> wrote: >> >>> It's super amazing .... i see it was tested on spark 2.0.0 and above, >>> what about Spark 1.6 which is still part of Cloudera's main versions? >>> >>> We have a vast Spark applications with version 1.6.0 >>> >>> On Thu, Mar 22, 2018 at 6:38 AM, Holden Karau <hol...@pigscanfly.ca> >>> wrote: >>> >>>> Super exciting! I look forward to digging through it this weekend. >>>> >>>> On Wed, Mar 21, 2018 at 9:33 PM ☼ R Nair (रविशंकर नायर) < >>>> ravishankar.n...@gmail.com> wrote: >>>> >>>>> Excellent. You filled a missing link. >>>>> >>>>> Best, >>>>> Passion >>>>> >>>>> On Wed, Mar 21, 2018 at 11:36 PM, Rohit Karlupia <roh...@qubole.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Happy to announce the availability of Sparklens as open source >>>>>> project. It helps in understanding the scalability limits of spark >>>>>> applications and can be a useful guide on the path towards tuning >>>>>> applications for lower runtime or cost. >>>>>> >>>>>> Please clone from here: https://github.com/qubole/sparklens >>>>>> Old blogpost: https://www.qubole.com/blog/introducing-quboles-sp >>>>>> ark-tuning-tool/ >>>>>> >>>>>> thanks, >>>>>> rohitk >>>>>> >>>>>> PS: Thanks for the patience. It took couple of months to get back on >>>>>> this. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> -- >>>> Twitter: https://twitter.com/holdenkarau >>>> >>> >>> >> > > > -- > Shmuel Blitz > Big Data Developer > Email: shmuel.bl...@similarweb.com > www.similarweb.com > <https://www.facebook.com/SimilarWeb/> > <https://www.linkedin.com/company/429838/> > <https://twitter.com/similarweb> > -- Shmuel Blitz Big Data Developer Email: shmuel.bl...@similarweb.com www.similarweb.com <https://www.facebook.com/SimilarWeb/> <https://www.linkedin.com/company/429838/> <https://twitter.com/similarweb>
