Re: Deploying Spark on Kubernetes Operator

Thanks for being interested in the Spark Kubernetes Operator. Because the initial PR is large so it is split into several PRs which are good to review and merge. And seems the initial series of PRs to merge the codes into the repo is not done yet. For example, you can see there is PR to add the op

Deploying Spark on Kubernetes Operator

Hello Spark team, I hope this email finds you well. I am writing on behalf of my team, which is looking to deploy a Spark Kubernetes Operator in our cluster. We have come across this latest initiative on GitHub to develop a Java based operator

SPIP: Adding work load identity to Spark on Kubernetes documents (supersedes Secret Management)

Hi, I would like to propose that the current Secret Management in Spark Kubernetes documentation to include the more secure credentials Workload identity) for Spark to access Kubernetes services. Both Google Clou

Spark on Kubernetes focused workshops

Team, is there a meetup or workshop focused on Spark on Kubernetes? If not, any interest in creating a once a month sync to exchange notes and best practices? Thanks, Janak

Re: Reason/rationale for using a service for the driver when running Spark on Kubernetes?

destruction. On Wed, 2 Mar 2022 at 21:42, Simon van der Veldt < simon.vanderve...@gmail.com> wrote: > Hi Spark devs, > > Does anyone know what the reason/rationale was for using/creating a > service for the driver when running Spark on Kubernetes? > The PR (https://github.

Reason/rationale for using a service for the driver when running Spark on Kubernetes?

Hi Spark devs, Does anyone know what the reason/rationale was for using/creating a service for the driver when running Spark on Kubernetes? The PR (https://github.com/apache/spark/pull/19717) and the commit (https://github.com/mccheah/spark/commit/3f4060c340d6bac412e8819c4388ccba226efcf3) that

RE: [Fork] ]RE: One click to run Spark on Kubernetes

customers to focus on running Spark application on their EKS cluster. Thanks, Janak From: Mich Talebzadeh Sent: Wednesday, February 23, 2022 11:54 AM To: Agarwal, Janak Cc: Spark dev list Subject: RE: [EXTERNAL] [Fork] ]RE: One click to run Spark on Kubernetes CAUTION: This email originated from

Re: One click to run Spark on Kubernetes

mit. >>>>> However, that depends on setting up access permission, use of service >>>>> accounts, pulling the correct dockerfiles for the driver and the >>>>> executors. >>>>> Those details add to the complexity. >>>>> >

Re: [Fork] ]RE: One click to run Spark on Kubernetes

d EKS-managed/self-managed EC2 based node groups. > > > > Thanks, > > Janak > > > > *From:* Mich Talebzadeh > *Sent:* Wednesday, February 23, 2022 10:46 AM > *To:* Agarwal, Janak > *Cc:* Spark dev list > *Subject:* RE: [EXTERNAL] [Fork] ]RE: One click to r

Re: One click to run Spark on Kubernetes

;>>> >>>> Thanks >>>> >>>> >>>> >>>>view my Linkedin profile >>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>> >>>> >>>> https://en.everybodywiki.com/M

RE: [Fork] ]RE: One click to run Spark on Kubernetes

[Fork] ]RE: One click to run Spark on Kubernetes CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi Janak, Are you talking about EKS Fargate? Thanks

Re: [Fork] ]RE: One click to run Spark on Kubernetes

elps. Want to give EMR on EKS a > spin as you decide your path forward? > > > > > > Best, > > Janak > > > > *From:* Sarath Annareddy > *Sent:* Wednesday, February 23, 2022 7:41 AM > *To:* bo yang > *Cc:* Mich Talebzadeh ; Spark Dev List < > d

Re: One click to run Spark on Kubernetes

gt;>> >>> >>> https://en.everybodywiki.com/Mich_Talebzadeh >>> >>> >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>>

Re: One click to run Spark on Kubernetes

he complexity. >>>> >>>> Thanks >>>> >>>> >>>>view my Linkedin profile >>>> >>>> >>>> >>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>> >>>> >>&g

Re: One click to run Spark on Kubernetes

everybodywiki.com/Mich_Talebzadeh >> >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly

Re: One click to run Spark on Kubernetes

any other property which may arise >> from relying on this email's technical content is explicitly disclaimed. The >> author will in no case be liable for any monetary damages arising from such >> loss, damage or destruction. >> >> >> >>>

Re: One click to run Spark on Kubernetes

no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Wed, 23 Feb 2022 at 04:06, bo yang wrote: > >> Hi Spark Community, >> >> We built an open source tool to deploy and run Spark on Kubernetes with a >>

Re: One click to run Spark on Kubernetes

#x27;s technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Wed, 23 Feb 2022 at 04:06, bo yang wrote: > Hi Spark Community, > > We built an open source tool to deploy and run Spark on Kube

Re: One click to run Spark on Kubernetes

;>> The one click tool intends to hide these details, so people could just >>>> submit Spark and do not need to deal with too many deployment details. >>>> >>>> On Tue, Feb 22, 2022 at 8:09 PM Bitfox wrote: >>>> >>>>> Can it be a clu

Re: One click to run Spark on Kubernetes

one node? >>> >>> Thanks >>> >>> On Wed, Feb 23, 2022 at 12:06 PM bo yang wrote: >>> >>>> Hi Spark Community, >>>> >>>> We built an open source tool to deploy and run Spark on Kubernetes with >>>> a one

Re: One click to run Spark on Kubernetes

12:06 PM bo yang wrote: >> >>> Hi Spark Community, >>> >>> We built an open source tool to deploy and run Spark on Kubernetes with >>> a one click command. For example, on AWS, it could automatically create an >>> EKS cluster, node group, NGINX i

Re: One click to run Spark on Kubernetes

ion of spark? or just the standalone node? > > Thanks > > On Wed, Feb 23, 2022 at 12:06 PM bo yang wrote: > >> Hi Spark Community, >> >> We built an open source tool to deploy and run Spark on Kubernetes with a >> one click command. For example, on AWS, it co

One click to run Spark on Kubernetes

Hi Spark Community, We built an open source tool to deploy and run Spark on Kubernetes with a one click command. For example, on AWS, it could automatically create an EKS cluster, node group, NGINX ingress, and Spark Operator. Then you will be able to use curl or a CLI tool to submit Spark

Re: Spark on Kubernetes scheduler variety

I >>>> may say I doubt whether such an approach and the so-called democratization >>>> of Spark on whatever platform is really should be of great focus. >>>> >>>> Having worked on Google Dataproc <https://cloud.google.com/dataproc> (A >>

Re: Spark on Kubernetes scheduler variety

ort plus >>> talent on batch scheduling on Kubernetes could be rewarding. However, if I >>> may say I doubt whether such an approach and the so-called democratization >>> of Spark on whatever platform is really should be of great focus. >>> >>> Having wor

Re: Spark on Kubernetes scheduler variety

;>>>>>> Mich >>>>>>>> >>>>>>>> >>>>>>>>view my Linkedin profile >>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>>>>>> >&

Re: Spark on Kubernetes scheduler variety

gt;>> >>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility >>>>>>> for any loss, damage or destruction of data or any other property which >>>>>>> may >>>>>>> arise from relying on this

Re: Spark on Kubernetes scheduler variety

gt;>>> arising from such loss, damage or destruction. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, 25 Jun 2021 at 09:45, Yikun Jiang >>>>>> wrote: >>>>>> >>>>>>&

Re: Spark on Kubernetes scheduler variety

o I created the spark-volcano[1] org in github to make it happen. >>>>>> >>>>>> [3] >>>>>> https://github.com/huawei-cloudnative/spark/commit/6c1f37525f026353eaead34216d47dad653f13a4 >>>>>> >>>>>> >>>>>> And &g

Re: Spark on Kubernetes scheduler variety

353eaead34216d47dad653f13a4 >>>>> >>>>> >>>>> And >>>>> Regards, >>>>> Yikun >>>>> >>>>> >>>>> Yikun Jiang 于2021年6月25日周五 上午11:53写道： >>>>> >>>>>> Hi, folks. >>>>&g

Re: Spark on Kubernetes scheduler variety

tion deployment validation from > our partners in China, like JingDong, XiaoHongShu, VIPshop. > > We will also prepare to propose an initial design and POC[3] on a shared > branch (based on spark master branch) where we can collaborate on it, so I > created the spark-volcano[1] org

Re: Spark on Kubernetes scheduler variety

rt. Also, there were also some production deployment >>>>> validation >>>>> from our partners in China, like JingDong, XiaoHongShu, VIPshop. >>>>> >>>>> We will also prepare to propose an initial design and POC[3] on a >>>>> sha

Re: Spark on Kubernetes scheduler variety

t;>> We will also prepare to propose an initial design and POC[3] on a >>>> shared branch (based on spark master branch) where we can collaborate on >>>> it, so I created the spark-volcano[1] org in github to make it happen. >>>> >>>> Pls feel free to comment on it [2] if you

Re: Spark on Kubernetes scheduler variety

no[1] org in github to make it happen. >>> >>> Pls feel free to comment on it [2] if you guys have any questions or >>> concerns. >>> >>> [1] https://github.com/spark-volcano >>> [2] https://github.com/spark-volcano/spark/issues/1 >>> [3] >>&

Re: Spark on Kubernetes scheduler variety

e and express a different point of view on >>>> this roadmap. >>>> >>>> >>>> I believe from a technical point of view spending time and effort plus >>>> talent on batch scheduling on Kubernetes could be rewarding. However, if I >>&

Re: Spark on Kubernetes scheduler variety

; > > Regards, > Yikun > > Holden Karau 于2021年6月25日周五 上午12:00写道： > >> Hi Mich, >> >> I certainly think making Spark on Kubernetes run well is going to be a >> challenge. However I think, and I could be wrong about this as well, that >> in terms of

Re: Spark on Kubernetes scheduler variety

if you guys have any questions or > concerns. > > [1] https://github.com/spark-volcano > [2] https://github.com/spark-volcano/spark/issues/1 > [3] https://github.com/spark-volcano-wip/spark-3-volcano > > Regards, > Yikun > > Holden Karau 于2021年6月25日周五 上午12:00写道： > >>

Re: Spark on Kubernetes scheduler variety

/issues/1 [3] https://github.com/spark-volcano-wip/spark-3-volcano Regards, Yikun Holden Karau 于2021年6月25日周五 上午12:00写道： > Hi Mich, > > I certainly think making Spark on Kubernetes run well is going to be a > challenge. However I think, and I could be wrong about this as well, that &

Re: Spark on Kubernetes scheduler variety

Hi Holden, Thank you for your points. I guess coming from a corporate world I had an oversight on how an open source project like Spark does leverage resources and interest :). As @KlausMa kindly volunteered it would be good to hear scheduling ideas on Spark on Kubernetes and of course as I am

Re: Spark on Kubernetes scheduler variety

Hi Mich, I certainly think making Spark on Kubernetes run well is going to be a challenge. However I think, and I could be wrong about this as well, that in terms of cluster managers Kubernetes is likely to be our future. Talking with people I don't hear about new standalone, YARN or

Re: Spark on Kubernetes scheduler variety

so-called democratization >> of Spark on whatever platform is really should be of great focus. >> >> Having worked on Google Dataproc <https://cloud.google.com/dataproc> (A fully >> managed and highly scalable service for running Apache Spark, Hadoop and >> more rec

Re: Spark on Kubernetes scheduler variety

gle Dataproc <https://cloud.google.com/dataproc> (A fully >> managed and highly scalable service for running Apache Spark, Hadoop and >> more recently other artefacts) for that past two years, and Spark on >> Kubernetes on-premise, I have come to the conclusion that Spark is not

Re: Spark on Kubernetes scheduler variety

om/dataproc> (A fully >> managed and highly scalable service for running Apache Spark, Hadoop and >> more recently other artefacts) for that past two years, and Spark on >> Kubernetes on-premise, I have come to the conclusion that Spark is not a >> beast that that one

Re: Spark on Kubernetes scheduler variety

t focus. > > Having worked on Google Dataproc <https://cloud.google.com/dataproc> (A fully > managed and highly scalable service for running Apache Spark, Hadoop and > more recently other artefacts) for that past two years, and Spark on > Kubernetes on-premise, I have come to

Re: Spark on Kubernetes scheduler variety

democratization of Spark on whatever platform is really should be of great focus. Having worked on Google Dataproc <https://cloud.google.com/dataproc> (A fully managed and highly scalable service for running Apache Spark, Hadoop and more recently other artefacts) for that past two years, and Sp

Re: Spark on Kubernetes scheduler variety

I think these approaches are good, but there are limitations (eg dynamic scaling) without us making changes inside of the Spark Kube scheduler. Certainly whichever scheduler extensions we add support for we should collaborate with the people developing those extensions insofar as they are interest

Re: Spark on Kubernetes scheduler variety

Hi, Regarding your point and I quote ".. I know that one of the Spark on Kube operators supports volcano/kube-batch so I was thinking that might be a place I would start exploring..." There seems to be ongoing work on say Volcano as part of Cloud Native Computing Foundation

Spark on Kubernetes scheduler variety

Hi Folks, I'm continuing my adventures to make Spark on containers party and I was wondering if folks have experience with the different batch scheduler options that they prefer? I was thinking so that we can better support dynamic allocation it might make sense for us to support using different s

Spark on Kubernetes plan for 2.4

On behalf of folks who work on Spark on Kubernetes, I would like to share a doc <https://docs.google.com/document/d/17Hy5tbAoIYbRv6bzZ4_MKb0ah30VpQHljvPx5sEnzhM/edit#> on the plan for Spark on Kubernetes features and changes for the upcoming 2.4 release. Please take a look if you are inte

Re: Spark on Kubernetes Builder Pattern Design Document

> >1. We started building Spark on Kubernetes on a fork and was prepared >to merge our work directly into master, >2. Discussion on https://issues.apache.org/jira/browse/SPARK-18278 led >us to move down the path of working on a fork first. We would harden the >

Re: Spark on Kubernetes Builder Pattern Design Document

Spark on Kubernetes on a fork and was prepared to merge our work directly into master, Discussion on https://issues.apache.org/jira/browse/SPARK-18278 led us to move down the path of working on a fork first. We would harden the fork, have the fork become used more widely to prove its value and

Re: Spark on Kubernetes Builder Pattern Design Document

line spark development, not a separate fork. On Mon, Feb 5, 2018 at 1:17 PM, Matt Cheah wrote: > Hi everyone, > > > > While we were building the Spark on Kubernetes integration, we realized > that some of the abstractions we introduced for building the driver > applicatio

Spark on Kubernetes Builder Pattern Design Document

Hi everyone, While we were building the Spark on Kubernetes integration, we realized that some of the abstractions we introduced for building the driver application in spark-submit, and building executor pods in the scheduler backend, could be improved for better readability and clarity. We

Announcing Spark on Kubernetes release 0.5.0

The Spark on Kubernetes development community is pleased to announce release 0.5.0 of Apache Spark with Kubernetes as a native scheduler back-end! This release includes a few bug fixes and the following features: - Spark R support - Kubernetes 1.8 support - Mounts emptyDir volumes for

Announcing Spark on Kubernetes release 0.4.0

The Spark on Kubernetes development community is pleased to announce release 0.4.0 of Apache Spark with native Kubernetes scheduler back-end! The dev community is planning to use this release as the reference for upstreaming native kubernetes capability over the Spark 2.3 release cycle. This

Re: SPIP: Spark on Kubernetes

Li >> Andrew Ash >> 李书明 >> Gary Lucas >> Ismael Mejia >> Jean-Baptiste Onofré >> Alexander Bezzubov >> duyanghao >> elmiko >> Sudarshan Kadambi >> Varun Katta >> Matt Cheah >> Edward Zhang >> Vaquar Khan >> >> &

Re: SPIP: Spark on Kubernetes

ing-started> and >>>user-facing <https://apache-spark-on-k8s.github.io/userdocs/> docu >>>mentation >>>- 10+ consistent code contributors from different organizations >>> >>> <https://apache-spark-on-k8s.github.io/use

Re: SPIP: Spark on Kubernetes

2017 at 10:42 PM, Reynold Xin wrote: > This has passed, hasn't it? > > On Tue, Aug 15, 2017 at 5:33 PM Anirudh Ramanathan > wrote: > >> Spark on Kubernetes effort has been developed separately in a fork, and >> linked back from the Apache Spark project as an experimenta

Re: SPIP: Spark on Kubernetes

This has passed, hasn't it? On Tue, Aug 15, 2017 at 5:33 PM Anirudh Ramanathan wrote: > Spark on Kubernetes effort has been developed separately in a fork, and > linked back from the Apache Spark project as an experimental backend > <http://spark.apache.org/docs/latest/clus

Re: SPIP: Spark on Kubernetes

he >>> committers. >>> >>> Separating out the cluster managers (into an as yet undecided new home) >>> appears far more disruptive and a high risk change for the short term. >>> However, when there is enough community support behind that effort, tracked &g

Re: SPIP: Spark on Kubernetes

(into an as yet undecided new home) >> appears far more disruptive and a high risk change for the short term. >> However, when there is enough community support behind that effort, tracked >> in 19700 <https://issues.apache.org/jira/browse/SPARK-19700>; and if >

Re: SPIP: Spark on Kubernetes

unity support behind that effort, tracked > in 19700 <https://issues.apache.org/jira/browse/SPARK-19700>; and if that > is realized in the future, it wouldn't be difficult to switch over > Kubernetes, YARN and Mesos to using the pluggable API. Currently, in my > opinion, with t

Re: SPIP: Spark on Kubernetes

g2012 Sent: Wednesday, August 23, 2017 5:47:16 AM To: dev@spark.apache.org Subject: Re: SPIP: Spark on Kubernetes +1 (non-binding) I am specifically interested in setting up testing environment for my company's Spark use and also expecting more comprehensive documents on getting development env

Re: SPIP: Spark on Kubernetes

August 23, 2017 5:47:16 AM To: dev@spark.apache.org Subject: Re: SPIP: Spark on Kubernetes +1 (non-binding) I am specifically interested in setting up testing environment for my company's Spark use and also expecting more comprehensive documents on getting development env setup in case of bug

Re: SPIP: Spark on Kubernetes

whole development lifecycle can be done, but hope developer guide can have more information. Thanks Edward Zhang -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-tp22147p22210.html Sent from the Apache Spark Developers List mailing

Re: SPIP: Spark on Kubernetes

uggable API. Currently, in my opinion, with the integration tests, active users, and a community of maintainers, Spark-on-Kubernetes would add minimal overhead and benefit a large (and growing) class of users. Lastly, the RSS is indeed separate and a value-add that we would love to share with ot

Re: SPIP: Spark on Kubernetes

gt;user-facing <https://apache-spark-on-k8s.github.io/userdocs/> docu >>mentation >> - 10+ consistent code contributors from different organizations >> >> <https://apache-spark-on-k8s.github.io/userdocs/contribute.html#project-contributions> >>

Re: SPIP: Spark on Kubernetes

;t seem necessary for kubernetes support, or specific to it. If its nice to have, and you want to add it to kubernetes first before other cluster managers, fine, but seems separate from this proposal. On Tue, Aug 15, 2017 at 10:32 AM, Anirudh Ramanathan < fox...@google.com.invalid> wrote: &

Re: SPIP: Spark on Kubernetes

+1 (non-binding) We are evaluating Kubernetes for a variety of data processing workloads. Spark is the natural choice for some of these workloads. Native Spark on Kubernetes is of interest to us as it brings in dynamic allocation, resource isolation and improved notions of security. -- View

Re: SPIP: Spark on Kubernetes

ubject: Re: SPIP: Spark on Kubernetes There are a fair number of people (myself included) who have interest in making scheduler back-ends fully pluggable. That will represent a significant impact to core spark architecture, with corresponding risk. Adding the kubernetes back-end in a manner simi

Re: SPIP: Spark on Kubernetes

+1 (non-binding) -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-tp22147p22195.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com

Re: SPIP: Spark on Kubernetes

ort at that time, was probably an incorrect design > choice on my part. > > > Regards, > Mridul > > On Tue, Aug 15, 2017 at 8:32 AM, Anirudh Ramanathan > wrote: > > Spark on Kubernetes effort has been developed separately in a fork, and > > linked back from the Apac

Re: SPIP: Spark on Kubernetes

probably an incorrect design choice on my part. Regards, Mridul On Tue, Aug 15, 2017 at 8:32 AM, Anirudh Ramanathan wrote: > Spark on Kubernetes effort has been developed separately in a fork, and > linked back from the Apache Spark project as an experimental backend. We're > ~6 m

Re: SPIP: Spark on Kubernetes

> >> > >> We're moving large amounts of infrastructure from a combination of open > >> source and homegrown cluster management systems to unify on Kubernetes and > >> want to bring Spark workloads along with us. > >> > >> On Tue, Aug 15,

Fwd: SPIP: Spark on Kubernetes

-- Forwarded message -- From: Timothy Chen Date: Thu, Aug 17, 2017 at 2:48 PM Subject: Re: SPIP: Spark on Kubernetes To: Marcelo Vanzin Hi Marcelo, Agree with your points, and I had that same thought around Resource staging server and like to share that with Spark on Mesos

Re: SPIP: Spark on Kubernetes

+1 (non-binding) peace o/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: SPIP: Spark on Kubernetes

#x27;t see any of those as blockers for an initial version. So adding my +1. On Tue, Aug 15, 2017 at 8:32 AM, Anirudh Ramanathan wrote: > Spark on Kubernetes effort has been developed separately in a fork, and > linked back from the Apache Spark project as an experimental backend. We'

Re: SPIP: Spark on Kubernetes

t; > >> +1 (non-binding) > >> > >> We're moving large amounts of infrastructure from a combination of open > >> source and homegrown cluster management systems to unify on Kubernetes and > >> want to bring Spark workloads along with us. > >>

Re: SPIP: Spark on Kubernetes

;> +1 (non-binding) >> >> >> >> We're moving large amounts of infrastructure from a combination of open >> >> source and homegrown cluster management systems to unify on Kubernetes >> and >> >> want to bring Spark workloads a

Re: SPIP: Spark on Kubernetes

megrown cluster management systems to unify on Kubernetes > and > >> want to bring Spark workloads along with us. > >> > >> On Tue, Aug 15, 2017 at 2:29 PM, liyinan926 > wrote: > >>> > >>> +1 (non-binding) &

Re: SPIP: Spark on Kubernetes

n-binding) >>> >>> We're moving large amounts of infrastructure from a combination of >open >>> source and homegrown cluster management systems to unify on >Kubernetes and >>> want to bring Spark workloads along with us. >>> >>> On Tue, Au

Re: SPIP: Spark on Kubernetes

rk workloads along with us. >> >> On Tue, Aug 15, 2017 at 2:29 PM, liyinan926 wrote: >>> >>> +1 (non-binding) >>> >>> >>> >>> -- >>> View this message in context: >>

Re: SPIP: Spark on Kubernetes

> >> +1 (non-binding) >> >> >> >> -- >> View this message in context: http://apache-spark-developers >> -list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-tp22147p22164.html >> Sent from the Apache Spark Developers List mailing list archive

回复： SPIP: Spark on Kubernetes

the expanding Kubernetes community. On Tue, Aug 15, 2017 at 8:43 AM, Erik Erlandson wrote: +1 (non-binding) On Tue, Aug 15, 2017 at 8:32 AM, Anirudh Ramanathan wrote: Spark on Kubernetes effort has been developed separately in a fork, and linked back from the Apache Spark project as an

Re: SPIP: Spark on Kubernetes

-binding) > > > > -- > View this message in context: http://apache-spark- > developers-list.1001551.n3.nabble.com/SPIP-Spark-on- > Kubernetes-tp22147p22164.html > Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. > > -

Re: SPIP: Spark on Kubernetes

+1 (non-binding) -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-tp22147p22164.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com

Re: SPIP: Spark on Kubernetes

timize uptake and retention of >> Apache Spark among the members of the expanding Kubernetes community. >> >> On Tue, Aug 15, 2017 at 8:43 AM, Erik Erlandson >> wrote: >> >>> +1 (non-binding) >>> >>> >>> On Tue, Aug 15, 2017 at 8:32 A

Re: SPIP: Spark on Kubernetes

, Erik Erlandson > wrote: > >> +1 (non-binding) >> >> >> On Tue, Aug 15, 2017 at 8:32 AM, Anirudh Ramanathan >> wrote: >> >>> Spark on Kubernetes effort has been developed separately in a fork, and >>> linked back from the Apache Spar

Re: SPIP: Spark on Kubernetes

represents a way to optimize uptake and retention of Apache Spark among the members of the expanding Kubernetes community. On Tue, Aug 15, 2017 at 8:43 AM, Erik Erlandson wrote: > +1 (non-binding) > > > On Tue, Aug 15, 2017 at 8:32 AM, Anirudh Ramanathan > wrote: > >> Spark on Ku

Re: SPIP: Spark on Kubernetes

ork required to maintain it. > > On Tue, Aug 15, 2017 at 9:51 AM William Benton wrote: > >> +1 (non-binding) >> >> On Tue, Aug 15, 2017 at 10:32 AM, Anirudh Ramanathan < >> fox...@google.com.invalid> wrote: >> >>> Spark on Kubernetes effort has bee

Re: SPIP: Spark on Kubernetes

017 at 10:32 AM, Anirudh Ramanathan < > fox...@google.com.invalid> wrote: > >> Spark on Kubernetes effort has been developed separately in a fork, and >> linked back from the Apache Spark project as an experimental backend >> <http://spark.apache.org/docs/latest/c

Re: SPIP: Spark on Kubernetes

+1 (non-binding) On Tue, Aug 15, 2017 at 10:32 AM, Anirudh Ramanathan < fox...@google.com.invalid> wrote: > Spark on Kubernetes effort has been developed separately in a fork, and > linked back from the Apache Spark project as an experimental backend > <http://spark.apac

Re: SPIP: Spark on Kubernetes

e in context: >> http://apache-spark-developers-list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-tp22147p22150.html >> Sent from the Apache Spark Developers List mailing list archive at >> Nabble.com. >> >> ---

Re: SPIP: Spark on Kubernetes

+1 (non-binding) Thanks, Kimoon On Tue, Aug 15, 2017 at 9:19 AM, Sean Suchter wrote: > +1 (non-binding) > > > > -- > View this message in context: http://apache-spark- > developers-list.1001551.n3.nabble.com/SPIP-Spark-on- > Kubernetes-tp22147p22150.html > Sent from

Re: SPIP: Spark on Kubernetes

+1 (non-binding) -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-tp22147p22150.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com

Re: SPIP: Spark on Kubernetes

+1 (non-binding) On Tue, Aug 15, 2017 at 8:32 AM, Anirudh Ramanathan wrote: > Spark on Kubernetes effort has been developed separately in a fork, and > linked back from the Apache Spark project as an experimental backend > <http://spark.apache.org/docs/latest/cluster-overview.

Apache Spark on Kubernetes: New Release for Spark 2.2

The Apache Spark on Kubernetes Community Development Project is pleased to announce the latest release of Apache Spark with native Scheduler Backend for Kubernetes! Features provided in this release include: - Cluster-mode submission of Spark jobs to a Kubernetes cluster - Support

Re: An Update on Spark on Kubernetes [Jun 23]

ssions (link > > <https://docs.google.com/document/u/1/d/1pnF38NF6N5eM8DlK088XUW85Vms4V2uTsGZvSp8MNIA/edit> >) > > *Recent talks and updates*: > >- Spark on Kubernetes (Spark Summit 2017) - link ><https://spark-summit.org/2017/events/apache-spark-on-ku

An Update on Spark on Kubernetes [Jun 23]

ent (Tencent) - Planning of work around Spark and HDFS on Kubernetes for Q3 2017 - Weekly meeting notes and discussions (link <https://docs.google.com/document/u/1/d/1pnF38NF6N5eM8DlK088XUW85Vms4V2uTsGZvSp8MNIA/edit> ) *Recent talks and updates*: - Spark on Kubernetes (Spark S

Re: Spark on Kubernetes: Birds-of-a-Feather Session 12:50pm 6/6

nterest and applications for running deep learning on Apache Spark platform (e.g., BigDL, TensorFrames, Caffe/TensorFlow-on-Spark, etc.) in the community. In this BoF discussion, we would like to cover related topics such as… BoF Discussion-Apache Spark on Kubernetes - Erik Erlandson <https://sp

  1   2   >