Hello spark community,
Adding a new topic. - Spark UI - Dynamic allocation - Tuning of jobs - Collecting spark metrics for monitoring and alerting - For those who prefer to use Pandas API on Spark since the release of Spark 3.2, What are some important notes for those users? For example, what are the additional factors affecting the Spark performance using Pandas API on Spark? How to tune them in addition to the conventional Spark tuning methods applied to Spark SQL users. - Spark internals and/or comparing spark 3 and 2 (I can take care of this if the community finds the topic interesting) Le jeu. 9 mars 2023 à 10:14, Winston Lai <weiruanl...@gmail.com> a écrit : > Hi everyone, > > I would like to add one topic to Saurabh's list as well. > > - Spark UI > - Dynamic allocation > - Tuning of jobs > - Collecting spark metrics for monitoring and alerting > - For those who prefer to use Pandas API on Spark since the release of > Spark 3.2, What are some important notes for those users? For example, what > are the additional factors affecting the Spark performance using Pandas API > on Spark? How to tune them in addition to the conventional Spark tuning > methods applied to Spark SQL users. > > > Thank You & Best Regards > Winston Lai > ------------------------------ > *From:* Saurabh Gulati <saurabh.gul...@fedex.com> > *Sent:* Thursday, March 9, 2023 5:04:35 PM > *To:* Mich Talebzadeh <mich.talebza...@gmail.com>; Deepak Sharma < > deepakmc...@gmail.com> > *Cc:* Denny Lee <denny.g....@gmail.com>; Sofia’s World < > mmistr...@gmail.com>; User <user@spark.apache.org>; Winston Lai < > weiruanl...@gmail.com>; ashok34...@yahoo.com <ashok34...@yahoo.com>; asma > zgolli <zgollia...@gmail.com>; karan alang <karan.al...@gmail.com> > *Subject:* Re: [EXTERNAL] Re: Online classes for spark topics > > Hey guys, > Its a nice idea and appreciate the effort you guys are taking. > I can add to the list of topics which might be of interest: > > - Spark UI > - Dynamic allocation > - Tuning of jobs > - Collecting spark metrics for monitoring and alerting > > > HTH > ------------------------------ > *From:* Mich Talebzadeh <mich.talebza...@gmail.com> > *Sent:* 09 March 2023 09:00 > *To:* Deepak Sharma <deepakmc...@gmail.com> > *Cc:* Denny Lee <denny.g....@gmail.com>; Sofia’s World < > mmistr...@gmail.com>; User <user@spark.apache.org>; Winston Lai < > weiruanl...@gmail.com>; ashok34...@yahoo.com <ashok34...@yahoo.com>; asma > zgolli <zgollia...@gmail.com>; karan alang <karan.al...@gmail.com> > *Subject:* [EXTERNAL] Re: Online classes for spark topics > > *Caution! This email originated outside of FedEx. Please do not open > attachments or click links from an unknown or suspicious origin*. > Hi Deepak, > > The priority list of topics is a very good point. The theard owner > mentioned Spark on k8s, Data Science and Spark Structured Streaming. What > other topics need to be included I guess it depends on demand.. I suggest > we wait a couple of days to see the demand . > > We just need to create a draft list of topics of interest and share them > in the forum to get the priority order. > > Well that is my thoughts. > > Cheers > > > > > > view my Linkedin profile > <https://urldefense.com/v3/__https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/__;!!BL9GA0TyTA!diGzOFoBDwbNwJnASVo09eytoXzhdUa03QoTvaP_HPka86FafTU8DqCg4fPtTSe08Y9ycp0Uie5nWTlKlGtK81xy3kU6Mg$> > > > https://en.everybodywiki.com/Mich_Talebzadeh > <https://urldefense.com/v3/__https://en.everybodywiki.com/Mich_Talebzadeh__;!!BL9GA0TyTA!diGzOFoBDwbNwJnASVo09eytoXzhdUa03QoTvaP_HPka86FafTU8DqCg4fPtTSe08Y9ycp0Uie5nWTlKlGtK81wx_fbKIQ$> > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Thu, 9 Mar 2023 at 06:13, Deepak Sharma <deepakmc...@gmail.com> wrote: > > I can prepare some topics and present as well , if we have a prioritised > list of topics already . > > On Thu, 9 Mar 2023 at 11:42 AM, Denny Lee <denny.g....@gmail.com> wrote: > > We used to run Spark webinars on the Apache Spark LinkedIn group > <https://urldefense.com/v3/__https://www.linkedin.com/company/apachespark/?viewAsMember=true__;!!BL9GA0TyTA!diGzOFoBDwbNwJnASVo09eytoXzhdUa03QoTvaP_HPka86FafTU8DqCg4fPtTSe08Y9ycp0Uie5nWTlKlGtK81ykBUs6eA$> > but > honestly the turnout was pretty low. We had dove into various features. > If there are particular topics that. you would like to discuss during a > live session, please let me know and we can try to restart them. HTH! > > On Wed, Mar 8, 2023 at 9:45 PM Sofia’s World <mmistr...@gmail.com> wrote: > > +1 > > On Wed, Mar 8, 2023 at 10:40 PM Winston Lai <weiruanl...@gmail.com> wrote: > > +1, any webinar on Spark related topic is appreciated 👍 > > Thank You & Best Regards > Winston Lai > ------------------------------ > *From:* asma zgolli <zgollia...@gmail.com> > *Sent:* Thursday, March 9, 2023 5:43:06 AM > *To:* karan alang <karan.al...@gmail.com> > *Cc:* Mich Talebzadeh <mich.talebza...@gmail.com>; ashok34...@yahoo.com < > ashok34...@yahoo.com>; User <user@spark.apache.org> > *Subject:* Re: Online classes for spark topics > > +1 > > Le mer. 8 mars 2023 à 21:32, karan alang <karan.al...@gmail.com> a écrit : > > +1 .. I'm happy to be part of these discussions as well ! > > > > > On Wed, Mar 8, 2023 at 12:27 PM Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > > Hi, > > I guess I can schedule this work over a course of time. I for myself can > contribute plus learn from others. > > So +1 for me. > > Let us see if anyone else is interested. > > HTH > > > > view my Linkedin profile > <https://urldefense.com/v3/__https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/__;!!BL9GA0TyTA!diGzOFoBDwbNwJnASVo09eytoXzhdUa03QoTvaP_HPka86FafTU8DqCg4fPtTSe08Y9ycp0Uie5nWTlKlGtK81xy3kU6Mg$> > > > https://en.everybodywiki.com/Mich_Talebzadeh > <https://urldefense.com/v3/__https://en.everybodywiki.com/Mich_Talebzadeh__;!!BL9GA0TyTA!diGzOFoBDwbNwJnASVo09eytoXzhdUa03QoTvaP_HPka86FafTU8DqCg4fPtTSe08Y9ycp0Uie5nWTlKlGtK81wx_fbKIQ$> > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Wed, 8 Mar 2023 at 17:48, ashok34...@yahoo.com <ashok34...@yahoo.com> > wrote: > > > Hello Mich. > > Greetings. Would you be able to arrange for Spark Structured Streaming > learning webinar.? > > This is something I haven been struggling with recently. it will be very > helpful. > > Thanks and Regard > > AK > On Tuesday, 7 March 2023 at 20:24:36 GMT, Mich Talebzadeh < > mich.talebza...@gmail.com> wrote: > > > Hi, > > This might be a worthwhile exercise on the assumption that the > contributors will find the time and bandwidth to chip in so to speak. > > I am sure there are many but on top of my head I can think of Holden Karau > for k8s, and Sean Owen for data science stuff. They are both very > experienced. > > Anyone else 🤔 > > HTH > > > > view my Linkedin profile > <https://urldefense.com/v3/__https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/__;!!BL9GA0TyTA!diGzOFoBDwbNwJnASVo09eytoXzhdUa03QoTvaP_HPka86FafTU8DqCg4fPtTSe08Y9ycp0Uie5nWTlKlGtK81xy3kU6Mg$> > > > https://en.everybodywiki.com/Mich_Talebzadeh > <https://urldefense.com/v3/__https://en.everybodywiki.com/Mich_Talebzadeh__;!!BL9GA0TyTA!diGzOFoBDwbNwJnASVo09eytoXzhdUa03QoTvaP_HPka86FafTU8DqCg4fPtTSe08Y9ycp0Uie5nWTlKlGtK81wx_fbKIQ$> > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Tue, 7 Mar 2023 at 19:17, ashok34...@yahoo.com.INVALID > <ashok34...@yahoo.com.invalid> wrote: > > Hello gurus, > > Does Spark arranges online webinars for special topics like Spark on K8s, > data science and Spark Structured Streaming? > > I would be most grateful if experts can share their experience with > learners with intermediate knowledge like myself. Hopefully we will find > the practical experiences told valuable. > > Respectively, > > AK > > > > > -- Asma ZGOLLI PhD in Big Data - Applied Machine Learning
