Thanks for that. How do you rate the performance of Jupyter W/Spark on K8s compared to the same on a cluster of VMs (example Dataproc).
Also somehow a related question (may be naive as well). For example, Google offers a lot of standard ML libraries for example built into a data warehouse like BigQuery. What does the Jupyter notebook offer that others don't? view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Mon, 5 Sept 2022 at 12:47, Holden Karau <hol...@pigscanfly.ca> wrote: > I’ve run Jupyter w/Spark on K8s, haven’t tried it with Dataproc personally. > > The Spark K8s pod scheduler is now more pluggable for Yunikorn and Volcano > can be used with less effort. > > On Mon, Sep 5, 2022 at 7:44 AM Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > >> >> Hi, >> >> >> Has anyone got experience of running Jupyter on dataproc versus Jupyter >> notebook on GKE (k8). >> >> >> I have not looked at this for a while but my understanding is that Spark >> on GKE/k8 is not yet performed. This is classic Spark with Python/Pyspark. >> >> >> Also I would like to know the state of spark with Volcano. Has progress >> made on that front. >> >> >> Regards, >> >> >> Mich >> >> >> view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> https://en.everybodywiki.com/Mich_Talebzadeh >> >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> > -- > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >
