Re: Time for release v3.3.2

2023-01-30 Thread Maxim Gekk 
+1

On Tue, Jan 31, 2023 at 10:12 AM John Zhuge  wrote:

> +1 Thanks Liang-Chi for driving the release!
>
> On Mon, Jan 30, 2023 at 10:26 PM Yuming Wang  wrote:
>
>> +1
>>
>> On Tue, Jan 31, 2023 at 12:18 PM yangjie01  wrote:
>>
>>> +1 Thanks Liang-Chi!
>>>
>>>
>>>
>>> YangJie
>>>
>>>
>>>
>>> *发件人**: *huaxin gao 
>>> *日期**: *2023年1月31日 星期二 10:03
>>> *收件人**: *Dongjoon Hyun 
>>> *抄送**: *Hyukjin Kwon , Chao Sun ,
>>> "L. C. Hsieh" , Spark dev list 
>>> *主题**: *Re: Time for release v3.3.2
>>>
>>>
>>>
>>> +1 Thanks Liang-Chi!
>>>
>>>
>>>
>>> On Mon, Jan 30, 2023 at 6:01 PM Dongjoon Hyun 
>>> wrote:
>>>
>>> +1
>>>
>>>
>>>
>>> Thank you so much, Liang-Chi.
>>>
>>> 3.3.2 release will help 3.4.0 release too because they share many bug
>>> fixes.
>>>
>>>
>>>
>>> Dongjoon
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Jan 30, 2023 at 5:56 PM Hyukjin Kwon 
>>> wrote:
>>>
>>> +100!
>>>
>>>
>>>
>>> On Tue, 31 Jan 2023 at 10:54, Chao Sun  wrote:
>>>
>>> +1, thanks Liang-Chi for volunteering!
>>>
>>> Chao
>>>
>>> On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh  wrote:
>>> >
>>> > Hi Spark devs,
>>> >
>>> > As you know, it has been 4 months since Spark 3.3.1 was released on
>>> > 2022/10, it seems a good time to think about next maintenance release,
>>> > i.e. Spark 3.3.2.
>>> >
>>> > I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
>>> >
>>> > What do you think?
>>> >
>>> > I am willing to volunteer for Spark 3.3.2 if there is consensus about
>>> > this maintenance release.
>>> >
>>> > Thank you.
>>> >
>>> > -
>>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>> >
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>
>
> --
> John Zhuge
>

Re: Time for release v3.3.2

2023-01-30 Thread John Zhuge 
+1 Thanks Liang-Chi for driving the release!

On Mon, Jan 30, 2023 at 10:26 PM Yuming Wang  wrote:

> +1
>
> On Tue, Jan 31, 2023 at 12:18 PM yangjie01  wrote:
>
>> +1 Thanks Liang-Chi!
>>
>>
>>
>> YangJie
>>
>>
>>
>> *发件人**: *huaxin gao 
>> *日期**: *2023年1月31日 星期二 10:03
>> *收件人**: *Dongjoon Hyun 
>> *抄送**: *Hyukjin Kwon , Chao Sun ,
>> "L. C. Hsieh" , Spark dev list 
>> *主题**: *Re: Time for release v3.3.2
>>
>>
>>
>> +1 Thanks Liang-Chi!
>>
>>
>>
>> On Mon, Jan 30, 2023 at 6:01 PM Dongjoon Hyun 
>> wrote:
>>
>> +1
>>
>>
>>
>> Thank you so much, Liang-Chi.
>>
>> 3.3.2 release will help 3.4.0 release too because they share many bug
>> fixes.
>>
>>
>>
>> Dongjoon
>>
>>
>>
>>
>>
>> On Mon, Jan 30, 2023 at 5:56 PM Hyukjin Kwon  wrote:
>>
>> +100!
>>
>>
>>
>> On Tue, 31 Jan 2023 at 10:54, Chao Sun  wrote:
>>
>> +1, thanks Liang-Chi for volunteering!
>>
>> Chao
>>
>> On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh  wrote:
>> >
>> > Hi Spark devs,
>> >
>> > As you know, it has been 4 months since Spark 3.3.1 was released on
>> > 2022/10, it seems a good time to think about next maintenance release,
>> > i.e. Spark 3.3.2.
>> >
>> > I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
>> >
>> > What do you think?
>> >
>> > I am willing to volunteer for Spark 3.3.2 if there is consensus about
>> > this maintenance release.
>> >
>> > Thank you.
>> >
>> > -
>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>> >
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

-- 
John Zhuge

Re: Time for release v3.3.2

2023-01-30 Thread Yuming Wang 
+1

On Tue, Jan 31, 2023 at 12:18 PM yangjie01  wrote:

> +1 Thanks Liang-Chi!
>
>
>
> YangJie
>
>
>
> *发件人**: *huaxin gao 
> *日期**: *2023年1月31日 星期二 10:03
> *收件人**: *Dongjoon Hyun 
> *抄送**: *Hyukjin Kwon , Chao Sun ,
> "L. C. Hsieh" , Spark dev list 
> *主题**: *Re: Time for release v3.3.2
>
>
>
> +1 Thanks Liang-Chi!
>
>
>
> On Mon, Jan 30, 2023 at 6:01 PM Dongjoon Hyun 
> wrote:
>
> +1
>
>
>
> Thank you so much, Liang-Chi.
>
> 3.3.2 release will help 3.4.0 release too because they share many bug
> fixes.
>
>
>
> Dongjoon
>
>
>
>
>
> On Mon, Jan 30, 2023 at 5:56 PM Hyukjin Kwon  wrote:
>
> +100!
>
>
>
> On Tue, 31 Jan 2023 at 10:54, Chao Sun  wrote:
>
> +1, thanks Liang-Chi for volunteering!
>
> Chao
>
> On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh  wrote:
> >
> > Hi Spark devs,
> >
> > As you know, it has been 4 months since Spark 3.3.1 was released on
> > 2022/10, it seems a good time to think about next maintenance release,
> > i.e. Spark 3.3.2.
> >
> > I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
> >
> > What do you think?
> >
> > I am willing to volunteer for Spark 3.3.2 if there is consensus about
> > this maintenance release.
> >
> > Thank you.
> >
> > -
> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> >
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: Time for release v3.3.2

2023-01-30 Thread yangjie01 
+1 Thanks Liang-Chi!

YangJie

发件人: huaxin gao 
日期: 2023年1月31日 星期二 10:03
收件人: Dongjoon Hyun 
抄送: Hyukjin Kwon , Chao Sun , "L. C. 
Hsieh" , Spark dev list 
主题: Re: Time for release v3.3.2

+1 Thanks Liang-Chi!

On Mon, Jan 30, 2023 at 6:01 PM Dongjoon Hyun 
mailto:dongjoon.h...@gmail.com>> wrote:
+1

Thank you so much, Liang-Chi.
3.3.2 release will help 3.4.0 release too because they share many bug fixes.

Dongjoon


On Mon, Jan 30, 2023 at 5:56 PM Hyukjin Kwon 
mailto:gurwls...@gmail.com>> wrote:
+100!

On Tue, 31 Jan 2023 at 10:54, Chao Sun 
mailto:sunc...@apache.org>> wrote:
+1, thanks Liang-Chi for volunteering!

Chao

On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh 
mailto:vii...@gmail.com>> wrote:
>
> Hi Spark devs,
>
> As you know, it has been 4 months since Spark 3.3.1 was released on
> 2022/10, it seems a good time to think about next maintenance release,
> i.e. Spark 3.3.2.
>
> I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
>
> What do you think?
>
> I am willing to volunteer for Spark 3.3.2 if there is consensus about
> this maintenance release.
>
> Thank you.
>
> -
> To unsubscribe e-mail: 
> dev-unsubscr...@spark.apache.org
>

-
To unsubscribe e-mail: 
dev-unsubscr...@spark.apache.org

Re: Time for release v3.3.2

2023-01-30 Thread huaxin gao 
+1 Thanks Liang-Chi!

On Mon, Jan 30, 2023 at 6:01 PM Dongjoon Hyun 
wrote:

> +1
>
> Thank you so much, Liang-Chi.
> 3.3.2 release will help 3.4.0 release too because they share many bug
> fixes.
>
> Dongjoon
>
>
> On Mon, Jan 30, 2023 at 5:56 PM Hyukjin Kwon  wrote:
>
>> +100!
>>
>> On Tue, 31 Jan 2023 at 10:54, Chao Sun  wrote:
>>
>>> +1, thanks Liang-Chi for volunteering!
>>>
>>> Chao
>>>
>>> On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh  wrote:
>>> >
>>> > Hi Spark devs,
>>> >
>>> > As you know, it has been 4 months since Spark 3.3.1 was released on
>>> > 2022/10, it seems a good time to think about next maintenance release,
>>> > i.e. Spark 3.3.2.
>>> >
>>> > I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
>>> >
>>> > What do you think?
>>> >
>>> > I am willing to volunteer for Spark 3.3.2 if there is consensus about
>>> > this maintenance release.
>>> >
>>> > Thank you.
>>> >
>>> > -
>>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>> >
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>

Re: Time for release v3.3.2

2023-01-30 Thread Dongjoon Hyun 
+1

Thank you so much, Liang-Chi.
3.3.2 release will help 3.4.0 release too because they share many bug fixes.

Dongjoon


On Mon, Jan 30, 2023 at 5:56 PM Hyukjin Kwon  wrote:

> +100!
>
> On Tue, 31 Jan 2023 at 10:54, Chao Sun  wrote:
>
>> +1, thanks Liang-Chi for volunteering!
>>
>> Chao
>>
>> On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh  wrote:
>> >
>> > Hi Spark devs,
>> >
>> > As you know, it has been 4 months since Spark 3.3.1 was released on
>> > 2022/10, it seems a good time to think about next maintenance release,
>> > i.e. Spark 3.3.2.
>> >
>> > I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
>> >
>> > What do you think?
>> >
>> > I am willing to volunteer for Spark 3.3.2 if there is consensus about
>> > this maintenance release.
>> >
>> > Thank you.
>> >
>> > -
>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>> >
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: Time for release v3.3.2

2023-01-30 Thread Hyukjin Kwon 
+100!

On Tue, 31 Jan 2023 at 10:54, Chao Sun  wrote:

> +1, thanks Liang-Chi for volunteering!
>
> Chao
>
> On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh  wrote:
> >
> > Hi Spark devs,
> >
> > As you know, it has been 4 months since Spark 3.3.1 was released on
> > 2022/10, it seems a good time to think about next maintenance release,
> > i.e. Spark 3.3.2.
> >
> > I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
> >
> > What do you think?
> >
> > I am willing to volunteer for Spark 3.3.2 if there is consensus about
> > this maintenance release.
> >
> > Thank you.
> >
> > -
> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> >
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: Time for release v3.3.2

2023-01-30 Thread Chao Sun 
+1, thanks Liang-Chi for volunteering!

Chao

On Mon, Jan 30, 2023 at 5:51 PM L. C. Hsieh  wrote:
>
> Hi Spark devs,
>
> As you know, it has been 4 months since Spark 3.3.1 was released on
> 2022/10, it seems a good time to think about next maintenance release,
> i.e. Spark 3.3.2.
>
> I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).
>
> What do you think?
>
> I am willing to volunteer for Spark 3.3.2 if there is consensus about
> this maintenance release.
>
> Thank you.
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Time for release v3.3.2

2023-01-30 Thread L. C. Hsieh 
Hi Spark devs,

As you know, it has been 4 months since Spark 3.3.1 was released on
2022/10, it seems a good time to think about next maintenance release,
i.e. Spark 3.3.2.

I'm thinking of the release of Spark 3.3.2 this Feb (2023/02).

What do you think?

I am willing to volunteer for Spark 3.3.2 if there is consensus about
this maintenance release.

Thank you.

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Re: spark+kafka+dynamic resource allocation

2023-01-30 Thread Mich Talebzadeh 
Sure, I suggest that you add a note to that Jira and express your interest.

HTH



   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 30 Jan 2023 at 01:56, Lingzhe Sun  wrote:

> Hi Mich，
>
> Thanks for the information. I myself think this open issue should have
> higher priority as more streaming application is being built using spark.
> Hope this becomes a feature in the coming future.
>
> --
> BR
> Lingzhe Sun
>
>
> *From:* Mich Talebzadeh 
> *Date:* 2023-01-30 02:14
> *To:* Lingzhe Sun 
> *CC:* ashok34...@yahoo.com; User 
> *Subject:* Re: Re: spark+kafka+dynamic resource allocation
> Hi,
>
> Spark Structured Streaming currently does not support dynamic allocation
> (see SPARK-24815: Structured Streaming should support dynamic allocation
> ). which is still open
>
>
> Autoscaling in Cloud offerings
> like
> Google Dataproc does not support Spark Structured Streaming either
>
>
> HTH
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Sun, 29 Jan 2023 at 01:49, Lingzhe Sun  wrote:
>
>> Thank you for the response. But the reference does not seem to be
>> answering any of those questions.
>>
>> BS
>> Lingzhe Sun
>>
>>
>> *From:* ashok34...@yahoo.com
>> *Date:* 2023-01-29 04:01
>> *To:* User ; Lingzhe Sun 
>> *Subject:* Re: spark+kafka+dynamic resource allocation
>> Hi,
>>
>> Worth checking this link
>>
>>
>> https://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation
>>
>> On Saturday, 28 January 2023 at 06:18:28 GMT, Lingzhe Sun <
>> lingzhe@hirain.com> wrote:
>>
>>
>> Hi all,
>>
>> I'm wondering if dynamic resource allocation works in spark+kafka
>> streaming applications. Here're some questions:
>>
>>- Will structured streaming be supported?
>>- Is the number of consumers always equal to the number of the
>>partitions of subscribed topic (let's say there's only one topic)?
>>- If consumers is evenly distributed across executors, will newly
>>added executor(through dynamic resource allocation) trigger a consumer
>>reassignment?
>>- Would it be simply a bad idea to use dynamic resource allocation in
>>streaming app, because there's no way to scale down number of executors
>>unless no data is coming in?
>>
>> Any thoughts are welcomed.
>>
>> Lingzhe Sun
>> Hirain Technology
>>
>>