Re: An Update on Spark on Kubernetes [Jun 23]

[Reynold Xin]

Thanks, Anirudh. This is super helpful!

On Fri, Jun 23, 2017 at 9:50 AM, Anirudh Ramanathan 
wrote:

> *Project Description: *Kubernetes cluster manager integration that
> enables native support for submitting Spark applications to a kubernetes
> cluster. The submitted applications can make use of Kubernetes native
> constructs.
>
> *JIRA*: 18278 
> *Upstream Kubernetes Issue*: 34377
> 
>
> *Summary of activity in the last 2 weeks*:
>
>- PySpark support being added by ifilolenko (Bloomberg)
>- HDFS locality fixes by kimoonkim (PepperData)
>- Minor bug fixes and performance improvement (Tencent)
>- Planning of work around Spark and HDFS on Kubernetes for Q3 2017
>- Weekly meeting notes and discussions (link
>
> 
>)
>
> *Recent talks and updates*:
>
>- Spark on Kubernetes (Spark Summit 2017) - link
>
>- HDFS on Kubernetes - Lessons Learned (Spark Summit 2017) - link
>
>
>
> For more details and to contribute to our discussions/efforts, please
> attend our weekly public meeting, or join our slack/mailing list. (link
> )
>

An Update on Spark on Kubernetes [Jun 23]

[Anirudh Ramanathan]

*Project Description: *Kubernetes cluster manager integration that enables
native support for submitting Spark applications to a kubernetes cluster.
The submitted applications can make use of Kubernetes native constructs.

*JIRA*: 18278 
*Upstream Kubernetes Issue*: 34377


*Summary of activity in the last 2 weeks*:

   - PySpark support being added by ifilolenko (Bloomberg)
   - HDFS locality fixes by kimoonkim (PepperData)
   - Minor bug fixes and performance improvement (Tencent)
   - Planning of work around Spark and HDFS on Kubernetes for Q3 2017
   - Weekly meeting notes and discussions (link
   

   )

*Recent talks and updates*:

   - Spark on Kubernetes (Spark Summit 2017) - link
   
   - HDFS on Kubernetes - Lessons Learned (Spark Summit 2017) - link
   


For more details and to contribute to our discussions/efforts, please
attend our weekly public meeting, or join our slack/mailing list. (link
)