Re: ASF board report draft for Feb 2023

[Matei Zaharia]

Sounds good!On Feb 6, 2023, at 8:16 PM, Dongjoon Hyun  wrote:Thank you, Matei. Could you include the following addtionally?1. Liang-Chi is preparing v3.3.2 (This month).    https://lists.apache.org/thread/nwzr3o2cxyyf6sbb37b8yylgcvmbtp162. Since Spark 3.4.0, we attached SBOM to Apache Spark Maven artifacts [SPARK-41893] in line with other ASF projects.    https://cwiki.apache.org/confluence/display/COMDEV/SBOMThanks,DongjoonOn Mon, Feb 6, 2023 at 6:13 PM Matei Zaharia  wrote:Hi all,

It’s time to send our quarterly report to the ASF board this Wednesday (Feb 8th). Here is a draft; let me know if you have suggestions:

===

Issues for the board:

- None

Project status:

- We cut the branch Spark 3.4.0 on Jan 24th 2023. The community is working on bug fixes, tests, stability and docs.
- We released Apache Spark 3.2.3, a bug fix release for the 3.2 line, on Nov 28th 2022.
- Votes on the Spark Project Improvement Proposals (SPIPs) for "Asynchronous Offset Management in Structured Streaming" and "Better Spark UI scalability and Driver stability for large applications" passed.
- The DStream API will be deprecated in the upcoming Apache Spark 3.4 release to focus work on the Structured Streaming APIs. [SPARK-42075]

Trademarks:

- No changes since the last report.

Latest releases:

- Spark 3.2.3 was released on Nov 28, 2022.
- Spark 3.3.1 was released on Oct 25, 2022.
- Spark 3.3.0 was released on June 16, 2022.

Committers and PMC:

- The latest committer was added on Oct 2nd, 2022 (Yikun Jiang).
- The latest PMC member was added on June 28th, 2022 (Huaxin Gao).

===

Re: 自动回复: Re: Spark on Kube (virtua) coffee/tea/pop times

[Mich Talebzadeh]

Hi,

Sounds like we are repeatedly getting out of office messages in Chinese
from this member. Is there any way an admin disables this account To save
usfrom  being spammed?


Thanks


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 8 Feb 2023 at 19:47, xyqiao  wrote:

> 这是来自QQ邮箱的假期自动回复邮件。
>
> 您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。
>

Re: Spark on Kube (virtua) coffee/tea/pop times

[Mich Talebzadeh]

That sounds like a good plan Holden!


Let us go for it


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 8 Feb 2023 at 20:12, Holden Karau  wrote:

> My thought here was that it's more focused on getting to understand each
> other's goals / priorities and less solving any specific problem.
>
> For example, I know that some folks running on EKS have different
> priorities than folks running on-prem.
>
> We might (later on) make a roadmap doc if that seems necessary, but I'm
> hoping that just an understanding of folks priorities and challenges will
> make it easier for us to all collaborate.
>
> On Wed, Feb 8, 2023 at 11:47 AM Mich Talebzadeh 
> wrote:
>
>> Hi all,
>>
>> Is this going to be a brainstorming meeting or there will be a prior
>> agenda to work around it?
>>
>> thanks
>>
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 8 Feb 2023 at 18:33, Mich Talebzadeh 
>> wrote:
>>
>>> Ok Colin thanks for clarification
>>>
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 8 Feb 2023 at 18:08, Colin Williams <
>>> colin.williams.seat...@gmail.com> wrote:
>>>
 I'm sorry you misunderstood.  The context is migrating jobs to Spark on
 k8s.

 On Wed, Feb 8, 2023, 8:31 AM Mich Talebzadeh 
 wrote:

> Hi Colin,
>
> Thanks for your reply.
>
>
> I think both Yarn and Kubernetes are cluster managers plus Standalone
> and Remotely Mesos. So I gather this discussion will focus on Spark on k8s
> unless I am mistaken.
>
>
> HTH,
>
>
> Mich
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any loss, damage or destruction of data or any other property which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary damages
> arising from such loss, damage or destruction.
>
>
>
>
> On Wed, 8 Feb 2023 at 16:02, Colin Williams <
> colin.williams.seat...@gmail.com> wrote:
>
>> Cloud or non cloud. Yarn, etc.
>>
>> On Wed, Feb 8, 2023, 7:57 AM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>> Hi Colin,
>>>
>>> Environments meaning different (cloud)  vendors or hosts?
>>>
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>> for any loss, damage or destruction of data or any other property which 
>>> may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary 
>>> damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 8 Feb 2023 at 03:42, Colin Williams <
>>> colin.williams.seat...@gmail.com> wrote:
>>>
 I wouldn't mind attending or viewing a recording depending on
 availability. I'm interested in challenges and solutions to porting 
 Spark
 jobs between environments.

 On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov <
 bolshakov.de...@gmail.com> wrote:

> Hello,
>
> I am also interested, 

Re: Spark on Kube (virtua) coffee/tea/pop times

[Holden Karau]

My thought here was that it's more focused on getting to understand each
other's goals / priorities and less solving any specific problem.

For example, I know that some folks running on EKS have different
priorities than folks running on-prem.

We might (later on) make a roadmap doc if that seems necessary, but I'm
hoping that just an understanding of folks priorities and challenges will
make it easier for us to all collaborate.

On Wed, Feb 8, 2023 at 11:47 AM Mich Talebzadeh 
wrote:

> Hi all,
>
> Is this going to be a brainstorming meeting or there will be a prior
> agenda to work around it?
>
> thanks
>
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 8 Feb 2023 at 18:33, Mich Talebzadeh 
> wrote:
>
>> Ok Colin thanks for clarification
>>
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 8 Feb 2023 at 18:08, Colin Williams <
>> colin.williams.seat...@gmail.com> wrote:
>>
>>> I'm sorry you misunderstood.  The context is migrating jobs to Spark on
>>> k8s.
>>>
>>> On Wed, Feb 8, 2023, 8:31 AM Mich Talebzadeh 
>>> wrote:
>>>
 Hi Colin,

 Thanks for your reply.


 I think both Yarn and Kubernetes are cluster managers plus Standalone
 and Remotely Mesos. So I gather this discussion will focus on Spark on k8s
 unless I am mistaken.


 HTH,


 Mich


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Wed, 8 Feb 2023 at 16:02, Colin Williams <
 colin.williams.seat...@gmail.com> wrote:

> Cloud or non cloud. Yarn, etc.
>
> On Wed, Feb 8, 2023, 7:57 AM Mich Talebzadeh <
> mich.talebza...@gmail.com> wrote:
>
>> Hi Colin,
>>
>> Environments meaning different (cloud)  vendors or hosts?
>>
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>> for any loss, damage or destruction of data or any other property which 
>> may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 8 Feb 2023 at 03:42, Colin Williams <
>> colin.williams.seat...@gmail.com> wrote:
>>
>>> I wouldn't mind attending or viewing a recording depending on
>>> availability. I'm interested in challenges and solutions to porting 
>>> Spark
>>> jobs between environments.
>>>
>>> On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov <
>>> bolshakov.de...@gmail.com> wrote:
>>>
 Hello,

 I am also interested, please add me to the conf.

 ср, 8 февр. 2023 г., 07:21 Jayabindu Singh :

> Greetings everyone!
> I am super new to this group and currently leading some work to
> deploy spark on k8 for my company o9 Solutions.
> I would love to join the discussion.
> I am in PST.
>
> Regards
> Jay
>
> Sent from my iPhone
>
> On Feb 7, 2023, at 3:57 PM, Mich Talebzadeh <
> mich.talebza...@gmail.com> wrote:
>
> 
> Could be interesting. Need to summarise where we are with Spark on
> k8s and what market demands.
>
> My personal experience with Volcano was not that impressive 樂. So

Re: Spark on Kube (virtua) coffee/tea/pop times

[Mich Talebzadeh]

Hi all,

Is this going to be a brainstorming meeting or there will be a prior agenda
to work around it?

thanks



   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 8 Feb 2023 at 18:33, Mich Talebzadeh 
wrote:

> Ok Colin thanks for clarification
>
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 8 Feb 2023 at 18:08, Colin Williams <
> colin.williams.seat...@gmail.com> wrote:
>
>> I'm sorry you misunderstood.  The context is migrating jobs to Spark on
>> k8s.
>>
>> On Wed, Feb 8, 2023, 8:31 AM Mich Talebzadeh 
>> wrote:
>>
>>> Hi Colin,
>>>
>>> Thanks for your reply.
>>>
>>>
>>> I think both Yarn and Kubernetes are cluster managers plus Standalone
>>> and Remotely Mesos. So I gather this discussion will focus on Spark on k8s
>>> unless I am mistaken.
>>>
>>>
>>> HTH,
>>>
>>>
>>> Mich
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 8 Feb 2023 at 16:02, Colin Williams <
>>> colin.williams.seat...@gmail.com> wrote:
>>>
 Cloud or non cloud. Yarn, etc.

 On Wed, Feb 8, 2023, 7:57 AM Mich Talebzadeh 
 wrote:

> Hi Colin,
>
> Environments meaning different (cloud)  vendors or hosts?
>
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any loss, damage or destruction of data or any other property which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary damages
> arising from such loss, damage or destruction.
>
>
>
>
> On Wed, 8 Feb 2023 at 03:42, Colin Williams <
> colin.williams.seat...@gmail.com> wrote:
>
>> I wouldn't mind attending or viewing a recording depending on
>> availability. I'm interested in challenges and solutions to porting Spark
>> jobs between environments.
>>
>> On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov <
>> bolshakov.de...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I am also interested, please add me to the conf.
>>>
>>> ср, 8 февр. 2023 г., 07:21 Jayabindu Singh :
>>>
 Greetings everyone!
 I am super new to this group and currently leading some work to
 deploy spark on k8 for my company o9 Solutions.
 I would love to join the discussion.
 I am in PST.

 Regards
 Jay

 Sent from my iPhone

 On Feb 7, 2023, at 3:57 PM, Mich Talebzadeh <
 mich.talebza...@gmail.com> wrote:

 
 Could be interesting. Need to summarise where we are with Spark on
 k8s and what market demands.

 My personal experience with Volcano was not that impressive 樂. So
 may be a summary will do.where we are currently with spark on k8s

 I am on  Greenwich Mean Time but I can take part in late sessions
 if needed


 HTH


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility
 for any loss, damage or destruction of data or any other property 
 which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in 

Re: Spark on Kube (virtua) coffee/tea/pop times

[Mich Talebzadeh]

Ok Colin thanks for clarification



   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 8 Feb 2023 at 18:08, Colin Williams <
colin.williams.seat...@gmail.com> wrote:

> I'm sorry you misunderstood.  The context is migrating jobs to Spark on
> k8s.
>
> On Wed, Feb 8, 2023, 8:31 AM Mich Talebzadeh 
> wrote:
>
>> Hi Colin,
>>
>> Thanks for your reply.
>>
>>
>> I think both Yarn and Kubernetes are cluster managers plus Standalone and
>> Remotely Mesos. So I gather this discussion will focus on Spark on k8s
>> unless I am mistaken.
>>
>>
>> HTH,
>>
>>
>> Mich
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 8 Feb 2023 at 16:02, Colin Williams <
>> colin.williams.seat...@gmail.com> wrote:
>>
>>> Cloud or non cloud. Yarn, etc.
>>>
>>> On Wed, Feb 8, 2023, 7:57 AM Mich Talebzadeh 
>>> wrote:
>>>
 Hi Colin,

 Environments meaning different (cloud)  vendors or hosts?



view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Wed, 8 Feb 2023 at 03:42, Colin Williams <
 colin.williams.seat...@gmail.com> wrote:

> I wouldn't mind attending or viewing a recording depending on
> availability. I'm interested in challenges and solutions to porting Spark
> jobs between environments.
>
> On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov <
> bolshakov.de...@gmail.com> wrote:
>
>> Hello,
>>
>> I am also interested, please add me to the conf.
>>
>> ср, 8 февр. 2023 г., 07:21 Jayabindu Singh :
>>
>>> Greetings everyone!
>>> I am super new to this group and currently leading some work to
>>> deploy spark on k8 for my company o9 Solutions.
>>> I would love to join the discussion.
>>> I am in PST.
>>>
>>> Regards
>>> Jay
>>>
>>> Sent from my iPhone
>>>
>>> On Feb 7, 2023, at 3:57 PM, Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>
>>> 
>>> Could be interesting. Need to summarise where we are with Spark on
>>> k8s and what market demands.
>>>
>>> My personal experience with Volcano was not that impressive 樂. So
>>> may be a summary will do.where we are currently with spark on k8s
>>>
>>> I am on  Greenwich Mean Time but I can take part in late sessions
>>> if needed
>>>
>>>
>>> HTH
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>> for any loss, damage or destruction of data or any other property which 
>>> may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary 
>>> damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 7 Feb 2023 at 23:37, John Zhuge  wrote:
>>>
 Awesome, count me in!
 PST

 On Tue, Feb 7, 2023 at 3:34 PM Andrew Melo 
 wrote:

> I'm Central US time (AKA UTC -6:00)
>
> On Tue, Feb 7, 2023 at 5:32 PM Holden Karau 
> wrote:
> >
> > Awesome, I guess I should have asked folks for timezones that
> they’re in.
> >
> > On Tue, Feb 7, 2023 at 3:30 PM Andrew Melo <
> andrew.m...@gmail.com> wrote:
> >>
> >> Hello Holden,
> >>
> >> We are interested in Spark on k8s and would like the

Re: Spark on Kube (virtua) coffee/tea/pop times

[Colin Williams]

I'm sorry you misunderstood.  The context is migrating jobs to Spark on k8s.

On Wed, Feb 8, 2023, 8:31 AM Mich Talebzadeh 
wrote:

> Hi Colin,
>
> Thanks for your reply.
>
>
> I think both Yarn and Kubernetes are cluster managers plus Standalone and
> Remotely Mesos. So I gather this discussion will focus on Spark on k8s
> unless I am mistaken.
>
>
> HTH,
>
>
> Mich
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 8 Feb 2023 at 16:02, Colin Williams <
> colin.williams.seat...@gmail.com> wrote:
>
>> Cloud or non cloud. Yarn, etc.
>>
>> On Wed, Feb 8, 2023, 7:57 AM Mich Talebzadeh 
>> wrote:
>>
>>> Hi Colin,
>>>
>>> Environments meaning different (cloud)  vendors or hosts?
>>>
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Wed, 8 Feb 2023 at 03:42, Colin Williams <
>>> colin.williams.seat...@gmail.com> wrote:
>>>
 I wouldn't mind attending or viewing a recording depending on
 availability. I'm interested in challenges and solutions to porting Spark
 jobs between environments.

 On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov <
 bolshakov.de...@gmail.com> wrote:

> Hello,
>
> I am also interested, please add me to the conf.
>
> ср, 8 февр. 2023 г., 07:21 Jayabindu Singh :
>
>> Greetings everyone!
>> I am super new to this group and currently leading some work to
>> deploy spark on k8 for my company o9 Solutions.
>> I would love to join the discussion.
>> I am in PST.
>>
>> Regards
>> Jay
>>
>> Sent from my iPhone
>>
>> On Feb 7, 2023, at 3:57 PM, Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>> 
>> Could be interesting. Need to summarise where we are with Spark on
>> k8s and what market demands.
>>
>> My personal experience with Volcano was not that impressive 樂. So
>> may be a summary will do.where we are currently with spark on k8s
>>
>> I am on  Greenwich Mean Time but I can take part in late sessions if
>> needed
>>
>>
>> HTH
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>> for any loss, damage or destruction of data or any other property which 
>> may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 7 Feb 2023 at 23:37, John Zhuge  wrote:
>>
>>> Awesome, count me in!
>>> PST
>>>
>>> On Tue, Feb 7, 2023 at 3:34 PM Andrew Melo 
>>> wrote:
>>>
 I'm Central US time (AKA UTC -6:00)

 On Tue, Feb 7, 2023 at 5:32 PM Holden Karau 
 wrote:
 >
 > Awesome, I guess I should have asked folks for timezones that
 they’re in.
 >
 > On Tue, Feb 7, 2023 at 3:30 PM Andrew Melo 
 wrote:
 >>
 >> Hello Holden,
 >>
 >> We are interested in Spark on k8s and would like the opportunity
 to
 >> speak with devs about what we're looking for slash better ways
 to use
 >> spark.
 >>
 >> Thanks!
 >> Andrew
 >>
 >> On Tue, Feb 7, 2023 at 5:24 PM Holden Karau <
 hol...@pigscanfly.ca> wrote:
 >> >
 >> > Hi Folks,
 >> >
 >> > It seems like we could maybe use some additional shared
 context around Spark on Kube so I’d like to try and schedule a virtual
 coffee session.
 >> >
 >> > Who all would be interested in virtual adventures around Spark
 on Kube development?
 >> >
 >> > No pressure if the idea of hanging out in a virtual chat with
 coffee and Spark devs does not sound like your 

Re: Spark on Kube (virtua) coffee/tea/pop times

[Holden Karau]

Ok y'all so it looks like the majority of interested folks are in pacific
timezone with a few in other north american and european timezones. So I've
made a doodle with a lot of pacific teams for next week
https://doodle.com/meeting/participate/id/e16z3MGa and I was thinking I'll
make another doodle for the following week with more european friendly
times.
Let me know what folks think :)

On Tue, Feb 7, 2023 at 3:23 PM Holden Karau  wrote:

> Hi Folks,
>
> It seems like we could maybe use some additional shared context around
> Spark on Kube so I’d like to try and schedule a virtual coffee session.
>
> Who all would be interested in virtual adventures around Spark on Kube
> development?
>
> No pressure if the idea of hanging out in a virtual chat with coffee and
> Spark devs does not sound like your thing, just trying to make something
> informal so we can have a better understanding of everyone’s goals here.
>
> Cheers,
>
> Holden :)
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>


-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Re: Spark on Kube (virtua) coffee/tea/pop times

[Mich Talebzadeh]

Hi Colin,

Thanks for your reply.


I think both Yarn and Kubernetes are cluster managers plus Standalone and
Remotely Mesos. So I gather this discussion will focus on Spark on k8s
unless I am mistaken.


HTH,


Mich


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 8 Feb 2023 at 16:02, Colin Williams <
colin.williams.seat...@gmail.com> wrote:

> Cloud or non cloud. Yarn, etc.
>
> On Wed, Feb 8, 2023, 7:57 AM Mich Talebzadeh 
> wrote:
>
>> Hi Colin,
>>
>> Environments meaning different (cloud)  vendors or hosts?
>>
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 8 Feb 2023 at 03:42, Colin Williams <
>> colin.williams.seat...@gmail.com> wrote:
>>
>>> I wouldn't mind attending or viewing a recording depending on
>>> availability. I'm interested in challenges and solutions to porting Spark
>>> jobs between environments.
>>>
>>> On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov <
>>> bolshakov.de...@gmail.com> wrote:
>>>
 Hello,

 I am also interested, please add me to the conf.

 ср, 8 февр. 2023 г., 07:21 Jayabindu Singh :

> Greetings everyone!
> I am super new to this group and currently leading some work to deploy
> spark on k8 for my company o9 Solutions.
> I would love to join the discussion.
> I am in PST.
>
> Regards
> Jay
>
> Sent from my iPhone
>
> On Feb 7, 2023, at 3:57 PM, Mich Talebzadeh 
> wrote:
>
> 
> Could be interesting. Need to summarise where we are with Spark on k8s
> and what market demands.
>
> My personal experience with Volcano was not that impressive 樂. So may
> be a summary will do.where we are currently with spark on k8s
>
> I am on  Greenwich Mean Time but I can take part in late sessions if
> needed
>
>
> HTH
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any loss, damage or destruction of data or any other property which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary damages
> arising from such loss, damage or destruction.
>
>
>
>
> On Tue, 7 Feb 2023 at 23:37, John Zhuge  wrote:
>
>> Awesome, count me in!
>> PST
>>
>> On Tue, Feb 7, 2023 at 3:34 PM Andrew Melo 
>> wrote:
>>
>>> I'm Central US time (AKA UTC -6:00)
>>>
>>> On Tue, Feb 7, 2023 at 5:32 PM Holden Karau 
>>> wrote:
>>> >
>>> > Awesome, I guess I should have asked folks for timezones that
>>> they’re in.
>>> >
>>> > On Tue, Feb 7, 2023 at 3:30 PM Andrew Melo 
>>> wrote:
>>> >>
>>> >> Hello Holden,
>>> >>
>>> >> We are interested in Spark on k8s and would like the opportunity
>>> to
>>> >> speak with devs about what we're looking for slash better ways to
>>> use
>>> >> spark.
>>> >>
>>> >> Thanks!
>>> >> Andrew
>>> >>
>>> >> On Tue, Feb 7, 2023 at 5:24 PM Holden Karau 
>>> wrote:
>>> >> >
>>> >> > Hi Folks,
>>> >> >
>>> >> > It seems like we could maybe use some additional shared context
>>> around Spark on Kube so I’d like to try and schedule a virtual coffee
>>> session.
>>> >> >
>>> >> > Who all would be interested in virtual adventures around Spark
>>> on Kube development?
>>> >> >
>>> >> > No pressure if the idea of hanging out in a virtual chat with
>>> coffee and Spark devs does not sound like your thing, just trying to 
>>> make
>>> something informal so we can have a better understanding of everyone’s
>>> goals here.
>>> >> >
>>> >> > Cheers,
>>> >> >
>>> >> > Holden :)
>>> >> > --
>>> >> > Twitter: https://twitter.com/holdenkarau
>>> >> > Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9
>>> 

Re: Spark on Kube (virtua) coffee/tea/pop times

[Colin Williams]

Cloud or non cloud. Yarn, etc.

On Wed, Feb 8, 2023, 7:57 AM Mich Talebzadeh 
wrote:

> Hi Colin,
>
> Environments meaning different (cloud)  vendors or hosts?
>
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 8 Feb 2023 at 03:42, Colin Williams <
> colin.williams.seat...@gmail.com> wrote:
>
>> I wouldn't mind attending or viewing a recording depending on
>> availability. I'm interested in challenges and solutions to porting Spark
>> jobs between environments.
>>
>> On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov 
>> wrote:
>>
>>> Hello,
>>>
>>> I am also interested, please add me to the conf.
>>>
>>> ср, 8 февр. 2023 г., 07:21 Jayabindu Singh :
>>>
 Greetings everyone!
 I am super new to this group and currently leading some work to deploy
 spark on k8 for my company o9 Solutions.
 I would love to join the discussion.
 I am in PST.

 Regards
 Jay

 Sent from my iPhone

 On Feb 7, 2023, at 3:57 PM, Mich Talebzadeh 
 wrote:

 
 Could be interesting. Need to summarise where we are with Spark on k8s
 and what market demands.

 My personal experience with Volcano was not that impressive 樂. So may
 be a summary will do.where we are currently with spark on k8s

 I am on  Greenwich Mean Time but I can take part in late sessions if
 needed


 HTH


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Tue, 7 Feb 2023 at 23:37, John Zhuge  wrote:

> Awesome, count me in!
> PST
>
> On Tue, Feb 7, 2023 at 3:34 PM Andrew Melo 
> wrote:
>
>> I'm Central US time (AKA UTC -6:00)
>>
>> On Tue, Feb 7, 2023 at 5:32 PM Holden Karau 
>> wrote:
>> >
>> > Awesome, I guess I should have asked folks for timezones that
>> they’re in.
>> >
>> > On Tue, Feb 7, 2023 at 3:30 PM Andrew Melo 
>> wrote:
>> >>
>> >> Hello Holden,
>> >>
>> >> We are interested in Spark on k8s and would like the opportunity to
>> >> speak with devs about what we're looking for slash better ways to
>> use
>> >> spark.
>> >>
>> >> Thanks!
>> >> Andrew
>> >>
>> >> On Tue, Feb 7, 2023 at 5:24 PM Holden Karau 
>> wrote:
>> >> >
>> >> > Hi Folks,
>> >> >
>> >> > It seems like we could maybe use some additional shared context
>> around Spark on Kube so I’d like to try and schedule a virtual coffee
>> session.
>> >> >
>> >> > Who all would be interested in virtual adventures around Spark
>> on Kube development?
>> >> >
>> >> > No pressure if the idea of hanging out in a virtual chat with
>> coffee and Spark devs does not sound like your thing, just trying to make
>> something informal so we can have a better understanding of everyone’s
>> goals here.
>> >> >
>> >> > Cheers,
>> >> >
>> >> > Holden :)
>> >> > --
>> >> > Twitter: https://twitter.com/holdenkarau
>> >> > Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9
>> >> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>> >
>> > --
>> > Twitter: https://twitter.com/holdenkarau
>> > Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9
>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>> --
> John Zhuge
>

Re: Spark on Kube (virtua) coffee/tea/pop times

[Mich Talebzadeh]

Hi Colin,

Environments meaning different (cloud)  vendors or hosts?



   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 8 Feb 2023 at 03:42, Colin Williams <
colin.williams.seat...@gmail.com> wrote:

> I wouldn't mind attending or viewing a recording depending on
> availability. I'm interested in challenges and solutions to porting Spark
> jobs between environments.
>
> On Tue, Feb 7, 2023 at 7:34 PM Denis Bolshakov 
> wrote:
>
>> Hello,
>>
>> I am also interested, please add me to the conf.
>>
>> ср, 8 февр. 2023 г., 07:21 Jayabindu Singh :
>>
>>> Greetings everyone!
>>> I am super new to this group and currently leading some work to deploy
>>> spark on k8 for my company o9 Solutions.
>>> I would love to join the discussion.
>>> I am in PST.
>>>
>>> Regards
>>> Jay
>>>
>>> Sent from my iPhone
>>>
>>> On Feb 7, 2023, at 3:57 PM, Mich Talebzadeh 
>>> wrote:
>>>
>>> 
>>> Could be interesting. Need to summarise where we are with Spark on k8s
>>> and what market demands.
>>>
>>> My personal experience with Volcano was not that impressive 樂. So may
>>> be a summary will do.where we are currently with spark on k8s
>>>
>>> I am on  Greenwich Mean Time but I can take part in late sessions if
>>> needed
>>>
>>>
>>> HTH
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 7 Feb 2023 at 23:37, John Zhuge  wrote:
>>>
 Awesome, count me in!
 PST

 On Tue, Feb 7, 2023 at 3:34 PM Andrew Melo 
 wrote:

> I'm Central US time (AKA UTC -6:00)
>
> On Tue, Feb 7, 2023 at 5:32 PM Holden Karau 
> wrote:
> >
> > Awesome, I guess I should have asked folks for timezones that
> they’re in.
> >
> > On Tue, Feb 7, 2023 at 3:30 PM Andrew Melo 
> wrote:
> >>
> >> Hello Holden,
> >>
> >> We are interested in Spark on k8s and would like the opportunity to
> >> speak with devs about what we're looking for slash better ways to
> use
> >> spark.
> >>
> >> Thanks!
> >> Andrew
> >>
> >> On Tue, Feb 7, 2023 at 5:24 PM Holden Karau 
> wrote:
> >> >
> >> > Hi Folks,
> >> >
> >> > It seems like we could maybe use some additional shared context
> around Spark on Kube so I’d like to try and schedule a virtual coffee
> session.
> >> >
> >> > Who all would be interested in virtual adventures around Spark on
> Kube development?
> >> >
> >> > No pressure if the idea of hanging out in a virtual chat with
> coffee and Spark devs does not sound like your thing, just trying to make
> something informal so we can have a better understanding of everyone’s
> goals here.
> >> >
> >> > Cheers,
> >> >
> >> > Holden :)
> >> > --
> >> > Twitter: https://twitter.com/holdenkarau
> >> > Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9
> >> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau
> >
> > --
> > Twitter: https://twitter.com/holdenkarau
> > Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9
> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
> --
 John Zhuge

>>>