Re: Question related to parallelism using structed streaming parallelism

2023-03-21 Thread Mich Talebzadeh
or download it from here https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies Limited view my Linkedin profile

Re: Question related to parallelism using structed streaming parallelism

2023-03-21 Thread Sean Owen
Yes more specifically, you can't ask for executors once the app starts, in SparkConf like that. You set this when you launch it against a Spark cluster in spark-submit or otherwise. On Tue, Mar 21, 2023 at 4:23 AM Mich Talebzadeh wrote: > Hi Emmanouil, > > This means that your job is running on

Topics for Spark online classes & webinars, next steps

2023-03-21 Thread Mich Talebzadeh
Hi all, As you may be aware we are proposing to set-up community classes and webinars for Spark interest group or simply for those who could benefit from them. @Denny Lee and myself had a discussion on how to put this framework forward. The idea is first and foremost getting support from