Re: Re: [ANNOUNCE] New PMC member: Arvid Heise

[Roman Khachatryan]

Congratulations!

Regards,
Roman

On Thu, Jun 17, 2021 at 5:56 AM Xingbo Huang  wrote:
>
> Congratulations, Arvid!
>
> Best,
> Xingbo
>
> Yun Tang  于2021年6月17日周四 上午10:49写道：
>
> > Congratulations, Arvid
> >
> > Best
> > Yun Tang
> > 
> > From: Yun Gao 
> > Sent: Thursday, June 17, 2021 10:46
> > To: Jingsong Li ; dev 
> > Subject: Re: Re: [ANNOUNCE] New PMC member: Arvid Heise
> >
> > Congratulations, Arvid!
> >
> > Best,
> > Yun
> >
> >
> > --
> > Sender:Jingsong Li
> > Date:2021/06/17 10:41:29
> > Recipient:dev
> > Theme:Re: [ANNOUNCE] New PMC member: Arvid Heise
> >
> > Congratulations, Arvid!
> >
> > Best,
> > Jingsong
> >
> > On Thu, Jun 17, 2021 at 6:52 AM Matthias J. Sax  wrote:
> >
> > > Congrats!
> > >
> > > On 6/16/21 6:06 AM, Leonard Xu wrote:
> > > > Congratulations, Arvid!
> > > >
> > > >
> > > >> 在 2021年6月16日，20:08，Till Rohrmann  写道：
> > > >>
> > > >> Congratulations, Arvid!
> > > >>
> > > >> Cheers,
> > > >> Till
> > > >>
> > > >> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG 
> > > wrote:
> > > >>
> > > >>> Congratulations, Arvid!
> > > >>>
> > > >>> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
> > > >>>
> > >  Congratulations, Arvid!
> > > 
> > > 
> > > 
> > >  --
> > >  Sent from:
> > > >>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> > > 
> > > >>>
> > > >
> > >
> >
> >
> > --
> > Best, Jingsong Lee
> >
> >

[jira] [Created] (FLINK-23016) Job client must be a Coordination Request Gateway when submit a job on web ui

[wen qi (Jira)]

wen qi created FLINK-23016:
--

 Summary: Job client must be a Coordination Request Gateway when 
submit a job on web ui 
 Key: FLINK-23016
 URL: https://issues.apache.org/jira/browse/FLINK-23016
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Web Frontend
Affects Versions: 1.13.1
 Environment: flink: 1.13.1

flink-cdc: com.alibaba.ververica:flink-connector-postgres-cdc:1.4.0

jdk:1.8
Reporter: wen qi
 Attachments: WechatIMG10.png, WechatIMG11.png, WechatIMG8.png

I used flink cdc to collect data,and use table api to  transfer data  and write 
to another table.

That's all ritht when I run code in IDE and submit jar of jobs use cli, but web 
ui

When I use StreamTableEnvironment.from('table-path').execute(), it's failed! 

please check my attachments , it seems that a  bug of web ui bug ? 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Re: [ANNOUNCE] New PMC member: Xintong Song

[Xingbo Huang]

Congratulations, Xintong!

Best,
Xingbo

Yun Gao  于2021年6月17日周四 上午10:46写道：

> Congratulations, Xintong!
>
> Best,
> Yun
>
>
> --
> Sender:Jingsong Li
> Date:2021/06/17 10:41:22
> Recipient:dev
> Theme:Re: [ANNOUNCE] New PMC member: Xintong Song
>
> Congratulations, Xintong!
>
> Best,
> Jingsong
>
> On Thu, Jun 17, 2021 at 10:26 AM Yun Tang  wrote:
>
> > Congratulations, Xintong!
> >
> > Best
> > Yun Tang
> > 
> > From: Leonard Xu 
> > Sent: Wednesday, June 16, 2021 21:05
> > To: dev (dev@flink.apache.org) 
> > Subject: Re: [ANNOUNCE] New PMC member: Xintong Song
> >
> >
> > Congratulations, Xintong!
> >
> >
> > Best,
> > Leonard
> > > 在 2021年6月16日，20:07，Till Rohrmann  写道：
> > >
> > > Congratulations, Xintong!
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG 
> wrote:
> > >
> > >> Congratulations, Xintong!
> > >>
> > >>
> > >> Jiayi Liao  于2021年6月16日周三 下午7:30写道：
> > >>
> > 
> >  
> >  Congratulations Xintong!
> > 
> >  On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang  >
> >  wrote:
> > 
> > > Congratulations, Xintong!
> > >
> > >
> > >
> > > --
> > > Sent from:
> > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> > 
> > 
> > >>>
> > >>
> >
> >
>
> --
> Best, Jingsong Lee
>
>

Re: Re: [ANNOUNCE] New PMC member: Arvid Heise

[Xingbo Huang]

Congratulations, Arvid!

Best,
Xingbo

Yun Tang  于2021年6月17日周四 上午10:49写道：

> Congratulations, Arvid
>
> Best
> Yun Tang
> 
> From: Yun Gao 
> Sent: Thursday, June 17, 2021 10:46
> To: Jingsong Li ; dev 
> Subject: Re: Re: [ANNOUNCE] New PMC member: Arvid Heise
>
> Congratulations, Arvid!
>
> Best,
> Yun
>
>
> --
> Sender:Jingsong Li
> Date:2021/06/17 10:41:29
> Recipient:dev
> Theme:Re: [ANNOUNCE] New PMC member: Arvid Heise
>
> Congratulations, Arvid!
>
> Best,
> Jingsong
>
> On Thu, Jun 17, 2021 at 6:52 AM Matthias J. Sax  wrote:
>
> > Congrats!
> >
> > On 6/16/21 6:06 AM, Leonard Xu wrote:
> > > Congratulations, Arvid!
> > >
> > >
> > >> 在 2021年6月16日，20:08，Till Rohrmann  写道：
> > >>
> > >> Congratulations, Arvid!
> > >>
> > >> Cheers,
> > >> Till
> > >>
> > >> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG 
> > wrote:
> > >>
> > >>> Congratulations, Arvid!
> > >>>
> > >>> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
> > >>>
> >  Congratulations, Arvid!
> > 
> > 
> > 
> >  --
> >  Sent from:
> > >>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> > 
> > >>>
> > >
> >
>
>
> --
> Best, Jingsong Lee
>
>

[jira] [Created] (FLINK-23015) Implement streaming window Deduplicate operator

[JING ZHANG (Jira)]

JING ZHANG created FLINK-23015:
--

 Summary: Implement streaming window Deduplicate operator
 Key: FLINK-23015
 URL: https://issues.apache.org/jira/browse/FLINK-23015
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Runtime
Reporter: JING ZHANG






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23014) Support streaming window Deduplicate in planner

[JING ZHANG (Jira)]

JING ZHANG created FLINK-23014:
--

 Summary: Support streaming window Deduplicate in planner
 Key: FLINK-23014
 URL: https://issues.apache.org/jira/browse/FLINK-23014
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: JING ZHANG






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23013) Introduce faker source connector

[Jingsong Lee (Jira)]

Jingsong Lee created FLINK-23013:


 Summary: Introduce faker source connector
 Key: FLINK-23013
 URL: https://issues.apache.org/jira/browse/FLINK-23013
 Project: Flink
  Issue Type: New Feature
  Components: Table SQL / API
Reporter: Jingsong Lee


We already have datagen connector.

But sometimes, we need a more real datagen connector which can produce more 
natural random records.

We can integrate [https://github.com/DiUS/java-faker] and introduce a built-in 
faker connector.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Re: [ANNOUNCE] New PMC member: Arvid Heise

[Yun Tang]

Congratulations, Arvid

Best
Yun Tang

From: Yun Gao 
Sent: Thursday, June 17, 2021 10:46
To: Jingsong Li ; dev 
Subject: Re: Re: [ANNOUNCE] New PMC member: Arvid Heise

Congratulations, Arvid!

Best,
Yun


--
Sender:Jingsong Li
Date:2021/06/17 10:41:29
Recipient:dev
Theme:Re: [ANNOUNCE] New PMC member: Arvid Heise

Congratulations, Arvid!

Best,
Jingsong

On Thu, Jun 17, 2021 at 6:52 AM Matthias J. Sax  wrote:

> Congrats!
>
> On 6/16/21 6:06 AM, Leonard Xu wrote:
> > Congratulations, Arvid!
> >
> >
> >> 在 2021年6月16日，20:08，Till Rohrmann  写道：
> >>
> >> Congratulations, Arvid!
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG 
> wrote:
> >>
> >>> Congratulations, Arvid!
> >>>
> >>> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
> >>>
>  Congratulations, Arvid!
> 
> 
> 
>  --
>  Sent from:
> >>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> 
> >>>
> >
>


--
Best, Jingsong Lee

Re: Re: [ANNOUNCE] New PMC member: Xintong Song

[Yun Gao]

Congratulations, Xintong!

Best,
Yun


--
Sender:Jingsong Li
Date:2021/06/17 10:41:22
Recipient:dev
Theme:Re: [ANNOUNCE] New PMC member: Xintong Song

Congratulations, Xintong!

Best,
Jingsong

On Thu, Jun 17, 2021 at 10:26 AM Yun Tang  wrote:

> Congratulations, Xintong!
>
> Best
> Yun Tang
> 
> From: Leonard Xu 
> Sent: Wednesday, June 16, 2021 21:05
> To: dev (dev@flink.apache.org) 
> Subject: Re: [ANNOUNCE] New PMC member: Xintong Song
>
>
> Congratulations, Xintong!
>
>
> Best,
> Leonard
> > 在 2021年6月16日，20:07，Till Rohrmann  写道：
> >
> > Congratulations, Xintong!
> >
> > Cheers,
> > Till
> >
> > On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:
> >
> >> Congratulations, Xintong!
> >>
> >>
> >> Jiayi Liao  于2021年6月16日周三 下午7:30写道：
> >>
> 
>  
>  Congratulations Xintong!
> 
>  On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
>  wrote:
> 
> > Congratulations, Xintong!
> >
> >
> >
> > --
> > Sent from:
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> 
> 
> >>>
> >>
>
>

-- 
Best, Jingsong Lee

Re: Re: [ANNOUNCE] New PMC member: Arvid Heise

[Yun Gao]

Congratulations, Arvid!

Best,
Yun


--
Sender:Jingsong Li
Date:2021/06/17 10:41:29
Recipient:dev
Theme:Re: [ANNOUNCE] New PMC member: Arvid Heise

Congratulations, Arvid!

Best,
Jingsong

On Thu, Jun 17, 2021 at 6:52 AM Matthias J. Sax  wrote:

> Congrats!
>
> On 6/16/21 6:06 AM, Leonard Xu wrote:
> > Congratulations, Arvid!
> >
> >
> >> 在 2021年6月16日，20:08，Till Rohrmann  写道：
> >>
> >> Congratulations, Arvid!
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG 
> wrote:
> >>
> >>> Congratulations, Arvid!
> >>>
> >>> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
> >>>
>  Congratulations, Arvid!
> 
> 
> 
>  --
>  Sent from:
> >>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> 
> >>>
> >
>


-- 
Best, Jingsong Lee

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Jingsong Li]

Congratulations, Arvid!

Best,
Jingsong

On Thu, Jun 17, 2021 at 6:52 AM Matthias J. Sax  wrote:

> Congrats!
>
> On 6/16/21 6:06 AM, Leonard Xu wrote:
> > Congratulations, Arvid!
> >
> >
> >> 在 2021年6月16日，20:08，Till Rohrmann  写道：
> >>
> >> Congratulations, Arvid!
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG 
> wrote:
> >>
> >>> Congratulations, Arvid!
> >>>
> >>> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
> >>>
>  Congratulations, Arvid!
> 
> 
> 
>  --
>  Sent from:
> >>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> 
> >>>
> >
>


-- 
Best, Jingsong Lee

Re: [ANNOUNCE] New PMC member: Xintong Song

[Jingsong Li]

Congratulations, Xintong!

Best,
Jingsong

On Thu, Jun 17, 2021 at 10:26 AM Yun Tang  wrote:

> Congratulations, Xintong!
>
> Best
> Yun Tang
> 
> From: Leonard Xu 
> Sent: Wednesday, June 16, 2021 21:05
> To: dev (dev@flink.apache.org) 
> Subject: Re: [ANNOUNCE] New PMC member: Xintong Song
>
>
> Congratulations, Xintong!
>
>
> Best,
> Leonard
> > 在 2021年6月16日，20:07，Till Rohrmann  写道：
> >
> > Congratulations, Xintong!
> >
> > Cheers,
> > Till
> >
> > On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:
> >
> >> Congratulations, Xintong!
> >>
> >>
> >> Jiayi Liao  于2021年6月16日周三 下午7:30写道：
> >>
> 
>  
>  Congratulations Xintong!
> 
>  On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
>  wrote:
> 
> > Congratulations, Xintong!
> >
> >
> >
> > --
> > Sent from:
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> 
> 
> >>>
> >>
>
>

-- 
Best, Jingsong Lee

[jira] [Created] (FLINK-23012) Add v1.13 docs link in "Pick Docs Version" for master branch

[Jark Wu (Jira)]

Jark Wu created FLINK-23012:
---

 Summary: Add v1.13 docs link in "Pick Docs Version" for master 
branch
 Key: FLINK-23012
 URL: https://issues.apache.org/jira/browse/FLINK-23012
 Project: Flink
  Issue Type: Improvement
  Components: Documentation
Affects Versions: 1.14.0
Reporter: Jark Wu
 Attachments: image-2021-06-17-10-26-29-125.png

 !image-2021-06-17-10-26-29-125.png|thumbnail! 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New PMC member: Xintong Song

[Yun Tang]

Congratulations, Xintong!

Best
Yun Tang

From: Leonard Xu 
Sent: Wednesday, June 16, 2021 21:05
To: dev (dev@flink.apache.org) 
Subject: Re: [ANNOUNCE] New PMC member: Xintong Song


Congratulations, Xintong!


Best,
Leonard
> 在 2021年6月16日，20:07，Till Rohrmann  写道：
>
> Congratulations, Xintong!
>
> Cheers,
> Till
>
> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:
>
>> Congratulations, Xintong!
>>
>>
>> Jiayi Liao  于2021年6月16日周三 下午7:30写道：
>>

 
 Congratulations Xintong!

 On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
 wrote:

> Congratulations, Xintong!
>
>
>
> --
> Sent from:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/


>>>
>>

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Matthias J. Sax]

Congrats!

On 6/16/21 6:06 AM, Leonard Xu wrote:
> Congratulations, Arvid!
> 
> 
>> 在 2021年6月16日，20:08，Till Rohrmann  写道：
>>
>> Congratulations, Arvid!
>>
>> Cheers,
>> Till
>>
>> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:
>>
>>> Congratulations, Arvid!
>>>
>>> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
>>>
 Congratulations, Arvid!



 --
 Sent from:
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

>>>
> 

Re: [DISCUSS] Dashboard/HistoryServer authentication

[Konstantin Knauf]

Hi Gabor,

> However representing Kerberos as completely new feature is not true
because
it's already in since Flink makes authentication at least with HDFS and
Hbase through Kerberos.

True, that is one way to look at it, but there are differences, too:
Control Plane vs Data Plane, Core vs Connectors.

> Adding OIDC or OAuth2 has the exact same concerns what you've guys just
raised. Why exactly these? If you think this would be beneficial we can
discuss it in detail

That's exactly my point. Once we start adding authx support, we will sooner
or later discuss other options besides Kerberos, too. A user who would like
to use OAuth can not easily use Kerberos, right?
That is one of the reasons I am skeptical about adding initial authx
support.

> Related authorization you've mentioned it can be complicated over time.
Can
you show us an example? We've knowledge with couple of open source
components
but authorization was never a horror complex story. I personally have the
most experience with Spark which I think is quite simple and stable. Users
can be viewers/admins
and jobs started by others can't be modified. If you can share an example
over-complication we can discuss on facts.

Authorization is a new aspect that needs to be considered for every
addition to the REST API. In the future users might ask for additional
roles (e.g. an editor), user-defined roles and you've already mentioned
job-level permissions yourself. And keep in mind that there might also be
larger additions in the future like the flink-sql-gateway. Contributions
like this become more expensive the more aspects we need to consider.

In general, I believe, it is important that the community focuses its
efforts where we can generate the most value to the user and - personally -
I don't think there is much to gain by extending Flink's scope in that
direction. Of course, this is not black and white and there are other valid
opinions.

Thanks,

Konstantin

On Wed, Jun 16, 2021 at 7:38 PM Gabor Somogyi 
wrote:

> Hi Konstantin,
>
> Thanks for the response. Related new feature introduction in case of Basic
> auth I tend to agree, anything else can be chosen.
>
> However representing Kerberos as completely new feature is not true because
> it's already in since Flink makes authentication at least with HDFS and
> Hbase through Kerberos.
> The main problem with the actual Kerberos implementation is that it
> contains several bugs and only partially implemented. Following your
> suggestion can we agree that we
> skip the Basic auth implementation and finish an already started Kerberos
> story by adding History Server and Job Dashboard authentication?
>
> Adding OIDC or OAuth2 has the exact same concerns what you've guys just
> raised. Why exactly these? If you think this would be beneficial we can
> discuss it in detail
> but as a side story it would be good to finish a halfway done Kerberos
> story.
>
> Related authorization you've mentioned it can be complicated over time. Can
> you show us an example? We've knowledge with couple of open source
> components
> but authorization was never a horror complex story. I personally have the
> most experience with Spark which I think is quite simple and stable. Users
> can be viewers/admins
> and jobs started by others can't be modified. If you can share an example
> over-complication we can discuss on facts.
>
> Thank you in advance!
>
> BR,
> G
>
>
> On Wed, Jun 16, 2021 at 5:42 PM Konstantin Knauf 
> wrote:
>
> > Hi everyone,
> >
> > sorry for joining late and thanks for the insightful discussion.
> >
> > In general, I'd personally prefer not to increase the surface area of
> > Apache Flink unless there is a good reason. It seems we all agree that
> > authx is not part of the core value proposition of Apache Flink, so if we
> > can delegate this problem to a more specialized tool, I am in favor of
> > that. Apache Flink is already huge and a lot of work goes into
> maintenance,
> > so I personally have become more sensitive to this aspect over time.
> >
> > If we add support for Basic Auth and Kerberos now, users will sooner or
> > later ask for OIDC, LDAP, SAML,... I acknowledge that Kerberos is widely
> > used in the corporate, on-premises context, but isn't the focus moving
> more
> > towards more web-friendly standards like OIDC/OAuth 2.0? If we only want
> to
> > support a single protocol, there is an argument to be made that it should
> > be OIDC and Dex [1,2] as a bridge to everything else. Have OIDC or OAuth2
> > been considered instead of Kerberos? How do you see the market moving?
> But
> > as I said before, in my opinion we can generate more value by investing
> > into other areas of Apache Flink.
> >
> > Authorization also has the potential to become more fine-grained and
> > complex over time: you already mentioned restricting the actions that a
> > specific user can do in a cluster.
> >
> > Cheers,
> >
> > Konstantin
> >
> > [1] https://github.com/dexidp/dex
> > [2] https://github.com/dexidp

Re: [DISCUSS] Dashboard/HistoryServer authentication

[Gabor Somogyi]

Hi Konstantin,

Thanks for the response. Related new feature introduction in case of Basic
auth I tend to agree, anything else can be chosen.

However representing Kerberos as completely new feature is not true because
it's already in since Flink makes authentication at least with HDFS and
Hbase through Kerberos.
The main problem with the actual Kerberos implementation is that it
contains several bugs and only partially implemented. Following your
suggestion can we agree that we
skip the Basic auth implementation and finish an already started Kerberos
story by adding History Server and Job Dashboard authentication?

Adding OIDC or OAuth2 has the exact same concerns what you've guys just
raised. Why exactly these? If you think this would be beneficial we can
discuss it in detail
but as a side story it would be good to finish a halfway done Kerberos
story.

Related authorization you've mentioned it can be complicated over time. Can
you show us an example? We've knowledge with couple of open source
components
but authorization was never a horror complex story. I personally have the
most experience with Spark which I think is quite simple and stable. Users
can be viewers/admins
and jobs started by others can't be modified. If you can share an example
over-complication we can discuss on facts.

Thank you in advance!

BR,
G


On Wed, Jun 16, 2021 at 5:42 PM Konstantin Knauf  wrote:

> Hi everyone,
>
> sorry for joining late and thanks for the insightful discussion.
>
> In general, I'd personally prefer not to increase the surface area of
> Apache Flink unless there is a good reason. It seems we all agree that
> authx is not part of the core value proposition of Apache Flink, so if we
> can delegate this problem to a more specialized tool, I am in favor of
> that. Apache Flink is already huge and a lot of work goes into maintenance,
> so I personally have become more sensitive to this aspect over time.
>
> If we add support for Basic Auth and Kerberos now, users will sooner or
> later ask for OIDC, LDAP, SAML,... I acknowledge that Kerberos is widely
> used in the corporate, on-premises context, but isn't the focus moving more
> towards more web-friendly standards like OIDC/OAuth 2.0? If we only want to
> support a single protocol, there is an argument to be made that it should
> be OIDC and Dex [1,2] as a bridge to everything else. Have OIDC or OAuth2
> been considered instead of Kerberos? How do you see the market moving? But
> as I said before, in my opinion we can generate more value by investing
> into other areas of Apache Flink.
>
> Authorization also has the potential to become more fine-grained and
> complex over time: you already mentioned restricting the actions that a
> specific user can do in a cluster.
>
> Cheers,
>
> Konstantin
>
> [1] https://github.com/dexidp/dex
> [2] https://github.com/dexidp/dex/issues/1903
>
>
> On Wed, Jun 16, 2021 at 11:44 AM Gabor Somogyi 
> wrote:
>
>> Hi Till,
>>
>> Did you have the chance to take a look at the doc? Not yet seen any
>> update.
>>
>> BR,
>> G
>>
>>
>> On Wed, Jun 9, 2021 at 1:43 PM Till Rohrmann 
>> wrote:
>>
>> > Thanks for the update Gabor. I'll take a look and respond in the
>> document.
>> >
>> > Cheers,
>> > Till
>> >
>> > On Wed, Jun 9, 2021 at 12:59 PM Gabor Somogyi <
>> gabor.g.somo...@gmail.com>
>> > wrote:
>> >
>> >> Hi Till,
>> >>
>> >> Your proxy suggestion has been considered in-depth and updated the FLIP
>> >> accordingly.
>> >> We've considered 2 proxy implementation (Nginx and Squid) but according
>> >> to our analysis and testing it's not suitable for the mentioned
>> use-cases.
>> >> Please take a look at the rejected alternatives for detailed
>> explanation.
>> >>
>> >> Thanks for your time in advance!
>> >>
>> >> BR,
>> >> G
>> >>
>> >>
>> >> On Fri, Jun 4, 2021 at 3:31 PM Till Rohrmann 
>> >> wrote:
>> >>
>> >>> As I've said I am not a security expert and that's why I have to ask
>> for
>> >>> clarification, Gabor. You are saying that if we configure a
>> truststore for
>> >>> the REST endpoint with a single trusted certificate which has been
>> >>> generated by the operator of the Flink cluster, then the attacker can
>> >>> generate a new certificate, sign it and then talk to the Flink
>> cluster if
>> >>> he has access to the node on which the REST endpoint runs? My
>> understanding
>> >>> was that you need the corresponding private key which in my proposed
>> setup
>> >>> would be under the control of the operator as well (e.g. stored in a
>> >>> keystore on the same machine but guarded by some secret). That way
>> (if I am
>> >>> not mistaken), only the entity which has access to the keystore is
>> able to
>> >>> talk to the Flink cluster.
>> >>>
>> >>> Maybe we are also getting our wires crossed here and are talking about
>> >>> different things.
>> >>>
>> >>> Thanks for listing the pros and cons of Kerberos. Concerning what
>> other
>> >>> authentication mechanisms are used in the industry, I am not 100%
>> sure.
>> >>>
>> >>> C

Re: [DISCUSS] FLIP-171: Async Sink

[Hausmann, Steffen]

Hi Danny,

Right now, I'd expect the core of the Async Sink (without third party 
dependencies) to live in its own submodule. For instance 
`flink-connector-async` as part of `flink-connectors`.

I'm currently planning to implement three different sinks to verify that the 
design of the sink if flexible enough to support different services: Amazon 
Kinesis Data Streams, Amazon Kinesis Data Firehose, and Amazon DynamoDB. But 
I'm not sure where to actually put them. To keep is simple, I'd start with a 
module that contains all AWS specific connectors. However, it has the obvious 
disadvantage that if someone wants to use a single sink, they would need to 
pull in all dependencies for all supported services that are included in this 
module (mainly the AWS SDK for these services). But I don't know how much of a 
problem that's going to be in practice. If the respective jar grows too big 
because all the included dependencies, that's certainly not going to work. But 
for now I'd just give it a try and then start a discussion once I have more 
data to share.

What's more interesting is whether that module should be part of the Flink code 
base or live somewhere else. I'd be great to get some feedback from the 
community on this.

Regarding the Kinesis Data Streams sink, I fully agree that it would be nice to 
remove the dependency to the KPL. So it seems to be desirable to keep the 
existing and the new FLIP-171  based implementation in separate modules. 
Otherwise people would be forced to pull in the KPL dependencies, even if they 
are only using the new implementation. In addition, the new implementation will 
not support the exact same functionality as the existing one: the KPL 
implements a very optimized form of aggregation on a shard level [1] by 
maintaining a mapping of shards and their respective key spaces. The new 
implementation can in principle support aggregation as well, but only on a 
partition key level, which may lead to less efficient aggregation and higher 
latencies.

Cheers, Steffen

[1] 
https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-concepts.html#kinesis-kpl-concepts-aggretation



On 15.06.21, 19:52, "Cranmer, Danny"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



Hey Steffen,

I have a few questions regarding the FLIP:
1. Where do you expect the core code to live, would it be in an existing 
module (say flink-clients) or would you introduce a new module?
2. Which destination implementations do you intend to ship with this FLIP? 
I see an example with Kinesis but you also list a bunch of other candidates.
3. For the Kinesis implementation, would you add the Sink to the existing 
flink-connector-kinesis repo, or create a new module? Reason I ask is that the 
existing Kinesis Sink depends on KPL and has a heavy transitive dependency 
chain, removing this would substantially reduce application size and clean the 
dependency chain

Thanks,

On 10/06/2021, 09:09, "Hausmann, Steffen"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



Hey Piotrek,

Thanks for your comments on the FLIP. I'll address your second question 
first, as I think it's more central to this FLIP. Just looking at the AWS 
ecosystem, there are several sinks with overlapping functionality. I've chosen 
AWS sinks here because I'm most familiar with those, but a similar argument 
applies more generically for destination that support async ingest.

There is, for instance, a sink for Amazon Kinesis Data Streams that is 
part of Apache Flink [1], a sink for Amazon Kinesis Data Firehose [2], a sink 
for Amazon DynamoDB [3], and a sink for Amazon Timestream [4]. All these sinks 
have implemented their own mechanisms for batching, persisting, and retrying 
events. And I'm not sure if all of them properly participate in checkpointing. 
[3] even seems to closely mirror [1] as it contains references to the Kinesis 
Producer Library, which is unrelated to Amazon DynamoDB.

These sinks predate FLIP-143. But as batching, persisting, and retrying 
capabilities do not seem to be part of FLIP-143, I'd argue that we would end up 
with similar duplication, even if these sinks were rewritten today based on 
FLIP-143. And that's the idea of FLIP-171: abstract away these commonly 
required capabilities so that it becomes easy to create support for a wide 
range of destination without having to think about batching, retries, 
checkpointing, etc. I've included an example in the FLIP [5] that shows that it 
only takes a couple of lines of code to implement a sink with exactly-once 
semantics. To be fair, the example is lacking robust failure handling and some 
more advanced capabilities of [1], but I th

Re: [DISCUSS] Dashboard/HistoryServer authentication

[Konstantin Knauf]

Hi everyone,

sorry for joining late and thanks for the insightful discussion.

In general, I'd personally prefer not to increase the surface area of
Apache Flink unless there is a good reason. It seems we all agree that
authx is not part of the core value proposition of Apache Flink, so if we
can delegate this problem to a more specialized tool, I am in favor of
that. Apache Flink is already huge and a lot of work goes into maintenance,
so I personally have become more sensitive to this aspect over time.

If we add support for Basic Auth and Kerberos now, users will sooner or
later ask for OIDC, LDAP, SAML,... I acknowledge that Kerberos is widely
used in the corporate, on-premises context, but isn't the focus moving more
towards more web-friendly standards like OIDC/OAuth 2.0? If we only want to
support a single protocol, there is an argument to be made that it should
be OIDC and Dex [1,2] as a bridge to everything else. Have OIDC or OAuth2
been considered instead of Kerberos? How do you see the market moving? But
as I said before, in my opinion we can generate more value by investing
into other areas of Apache Flink.

Authorization also has the potential to become more fine-grained and
complex over time: you already mentioned restricting the actions that a
specific user can do in a cluster.

Cheers,

Konstantin

[1] https://github.com/dexidp/dex
[2] https://github.com/dexidp/dex/issues/1903


On Wed, Jun 16, 2021 at 11:44 AM Gabor Somogyi 
wrote:

> Hi Till,
>
> Did you have the chance to take a look at the doc? Not yet seen any update.
>
> BR,
> G
>
>
> On Wed, Jun 9, 2021 at 1:43 PM Till Rohrmann  wrote:
>
> > Thanks for the update Gabor. I'll take a look and respond in the
> document.
> >
> > Cheers,
> > Till
> >
> > On Wed, Jun 9, 2021 at 12:59 PM Gabor Somogyi  >
> > wrote:
> >
> >> Hi Till,
> >>
> >> Your proxy suggestion has been considered in-depth and updated the FLIP
> >> accordingly.
> >> We've considered 2 proxy implementation (Nginx and Squid) but according
> >> to our analysis and testing it's not suitable for the mentioned
> use-cases.
> >> Please take a look at the rejected alternatives for detailed
> explanation.
> >>
> >> Thanks for your time in advance!
> >>
> >> BR,
> >> G
> >>
> >>
> >> On Fri, Jun 4, 2021 at 3:31 PM Till Rohrmann 
> >> wrote:
> >>
> >>> As I've said I am not a security expert and that's why I have to ask
> for
> >>> clarification, Gabor. You are saying that if we configure a truststore
> for
> >>> the REST endpoint with a single trusted certificate which has been
> >>> generated by the operator of the Flink cluster, then the attacker can
> >>> generate a new certificate, sign it and then talk to the Flink cluster
> if
> >>> he has access to the node on which the REST endpoint runs? My
> understanding
> >>> was that you need the corresponding private key which in my proposed
> setup
> >>> would be under the control of the operator as well (e.g. stored in a
> >>> keystore on the same machine but guarded by some secret). That way (if
> I am
> >>> not mistaken), only the entity which has access to the keystore is
> able to
> >>> talk to the Flink cluster.
> >>>
> >>> Maybe we are also getting our wires crossed here and are talking about
> >>> different things.
> >>>
> >>> Thanks for listing the pros and cons of Kerberos. Concerning what other
> >>> authentication mechanisms are used in the industry, I am not 100% sure.
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Fri, Jun 4, 2021 at 11:09 AM Gabor Somogyi <
> gabor.g.somo...@gmail.com>
> >>> wrote:
> >>>
>  > I did not mean for the user to sign its own certificates but for the
>  operator of the cluster. Once the user request hits the proxy, it
> should no
>  longer be under his control. I think I do not fully understand yet
> why this
>  would not work.
>  I said it's not solving the authentication problem over any proxy.
> Even
>  if the operator is signing the certificate one can have access to an
>  internal node.
>  Such case anybody can craft certificates which is accepted by the
>  server. When it's accepted a bad guy can cancel jobs causing huge
> impacts.
> 
>  > Also, I am missing a bit the comparison of Kerberos to other
>  authentication mechanisms and why they were rejected in favour of
> Kerberos.
>  PROS:
>  * Since it's not depending on cloud provider and/or k8s or bare-metal
>  etc. deployment it's the biggest plus
>  * Centralized with tools and no need to write tons of tools around
>  * There are clients/tools on almost all OS-es and several languages
>  * Super huge users are using it for years in production w/o huge
> issues
>  * Provides cross-realm trust possibility amongst other features
>  * Several open source components using it which could increase
>  compatibility
> 
>  CONS:
>  * Not everybody using kerberos
>  * It would increase the code footprint but this is true for many
>  features

Re: State migration scenario's

[Marlo Ploemen]

Hi Yun, 

Thanks for your response. If I understand correctly, for a particular state 
descriptor you want to improve the evolution of the underlying data structure? 
Are there other typical scenario’s that you encounter when running a stateful 
dataflow graph (e.g. merging data structures, changing underlying class (not 
modifying)) etc. 

Best,
Marlo

> Op 9 jun. 2021, om 10:10 heeft Yun Tang  het volgende 
> geschreven:
> 
> Hi Marlo,
> 
> One of the scenarios that we're trying to improve is to add or remove one 
> field in state serializer.
> Users might add or remove one field during their schema evolution, state 
> processor could help it with another offline job while state migration could 
> help it once we restart the new job.
> 
> Best
> Yun Tang
> 
> From: Marlo Ploemen 
> Sent: Wednesday, June 9, 2021 15:57
> To: dev@flink.apache.org 
> Subject: State migration scenario's
> 
> Hi community,
> 
> I am looking into the data migration and schema evolution process for 
> stateful streaming jobs. Currently, there is no orchestration support for 
> performing these job evolutions and no in-job state migration or schema 
> evolution syntax (as this is part of the separate state processor API). I am 
> looking for examples (e.g. Github repositories) or scenarios of stateful 
> streaming jobs where the orchestration of their state evolution process can 
> improve development quality.
> 
> Best, Marlo

Re: [ANNOUNCE] New PMC member: Xintong Song

[Jark Wu]

Congratulations, Xintong! Well deserved!

Best,
Jark

On Wed, 16 Jun 2021 at 21:06, Leonard Xu  wrote:

>
> Congratulations, Xintong!
>
>
> Best,
> Leonard
> > 在 2021年6月16日，20:07，Till Rohrmann  写道：
> >
> > Congratulations, Xintong!
> >
> > Cheers,
> > Till
> >
> > On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:
> >
> >> Congratulations, Xintong!
> >>
> >>
> >> Jiayi Liao  于2021年6月16日周三 下午7:30写道：
> >>
> 
>  
>  Congratulations Xintong!
> 
>  On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
>  wrote:
> 
> > Congratulations, Xintong!
> >
> >
> >
> > --
> > Sent from:
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> 
> 
> >>>
> >>
>
>

Re: [DISCUSS] Feedback Collection Jira Bot

[Robert Metzger]

Very sorry for the delayed response.

Regarding tickets with the "test-instability" label (topic 1): I'm usually
assigning a fixVersion to the next release of the branch where the failure
occurred, when I'm opening a test failure ticket. Others seem to do that
too. Hence my comment that not checking tickets with a fixVersion set by
Flink bot is good (because test failures should always stay "Critical"
until we've understood what's going on)
I see that it is a bit contradicting that Critical test instabilities
receive no attention for 14 days, but that seems to be the norm given the
current number of incoming test instabilities.

On Wed, Jun 16, 2021 at 2:05 PM Till Rohrmann  wrote:

> Another example for category 4 would be the ticket where we collect
> breaking API changes for Flink 2.0 [1]. The idea behind this ticket is to
> collect things to consider when developing the next major version.
> Admittedly, we have never seen the benefits of collecting the breaking
> changes because we haven't started Flink 2.x yet. Also, it is not clear how
> relevant these tickets are right now.
>
> [1] https://issues.apache.org/jira/browse/FLINK-3957
>
> Cheers,
> Till
>
> On Wed, Jun 16, 2021 at 11:42 AM Konstantin Knauf 
> wrote:
>
> > Hi everyone,
> >
> > thank you for all the feedback so far. I believe we have four different
> > topics by now:
> >
> > 1 about *test-instability tickets* raised by Robert. Waiting for feedback
> > by Robert.
> >
> > 2 about *aggressiveness of stale-assigned *rule raised by Timo. Waiting
> > for feedback by Timo and others.
> >
> > 3 about *excluding issues with a fixVersion* raised by Konstantin, Till.
> > Waiting for more feedback by the community as it involves general changes
> > to how we deal with fixVersion.
> >
> > 4 about *excluding issues with a specific-label* raised by Arvid.
> >
> > I've already written something about 1-3. Regarding 4:
> >
> > How do we make sure that these don't become stale? I think, there have
> > been a few "long-term efforts" in the past that never got the attention
> > that we initially wanted. Is this just about the ability to collect
> tickets
> > under an umbrella to document a future effort? Maybe for the example of
> > DataStream replacing DataSet how would this look like in Jira?
> >
> > Cheers,
> >
> > Konstantin
> >
> >
> > On Tue, Jun 8, 2021 at 11:31 AM Till Rohrmann 
> > wrote:
> >
> >> I like this idea. It would then be the responsibility of the component
> >> maintainers to manage the lifecycle explicitly.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Mon, Jun 7, 2021 at 1:48 PM Arvid Heise  wrote:
> >>
> >> > One more idea for the bot. Could we have a label to exclude certain
> >> tickets
> >> > from the life-cycle?
> >> >
> >> > I'm thinking about long-term tickets such as improving DataStream to
> >> > eventually replace DataSet. We would collect ideas over the next
> couple
> >> of
> >> > weeks without any visible progress on the implementation.
> >> >
> >> > On Fri, May 21, 2021 at 2:06 PM Konstantin Knauf 
> >> > wrote:
> >> >
> >> > > Hi Timo,
> >> > >
> >> > > Thanks for joining the discussion. All rules except the unassigned
> >> rule
> >> > do
> >> > > not apply to Sub-Tasks actually (like deprioritization, closing).
> >> > > Additionally, activity on a Sub-Taks counts as activity for the
> >> parent.
> >> > So,
> >> > > the parent ticket would not be touched by the bot as long as there
> is
> >> a
> >> > > single Sub-Task that has a discussion or an update. If you
> experience
> >> > > something different, this is a bug.
> >> > >
> >> > > Is there a reason why it is important to assign all Sub-Tasks to the
> >> same
> >> > > person immediately? I am not sure if this kind "reserving tickets"
> is
> >> a
> >> > > good idea in general to be honest.
> >> > >
> >> > > Cheers,
> >> > >
> >> > > Konstantin
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Fri, May 21, 2021 at 12:00 PM Timo Walther 
> >> > wrote:
> >> > >
> >> > > > Hi Konstantin,
> >> > > >
> >> > > > thanks for starting this discussion. I was also about to provide
> >> some
> >> > > > feedback because I have the feeling that the bot is too aggressive
> >> at
> >> > > > the moment.
> >> > > >
> >> > > > Even a 14 days interval is a short period of time for bigger
> efforts
> >> > > > that might include several subtasks. Currently, if we split an
> issue
> >> > > > into subtasks usually most subtasks are assigned to the same
> person.
> >> > But
> >> > > > the bot requires us to update all subtasks again after 7 days.
> >> Could we
> >> > > > disable the bot for subtasks or extend the period to 30 days?
> >> > > >
> >> > > > The core problem in the past was that we had issues laying around
> >> > > > untouched for years. Luckily, this is solved with the bot now. But
> >> > going
> >> > > > from years to 7 days spams the mail box quite a bit.
> >> > > >
> >> > > > Regards,
> >> > > > Timo
> >> > > >
> >> > > >
> >> > > > On 21.05.21 09:22, Konstantin Knauf wrote:
> >

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Leonard Xu]

Congratulations, Arvid!


> 在 2021年6月16日，20:08，Till Rohrmann  写道：
> 
> Congratulations, Arvid!
> 
> Cheers,
> Till
> 
> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:
> 
>> Congratulations, Arvid!
>> 
>> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
>> 
>>> Congratulations, Arvid!
>>> 
>>> 
>>> 
>>> --
>>> Sent from:
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>>> 
>> 

Re: [ANNOUNCE] New PMC member: Xintong Song

[Leonard Xu]

Congratulations, Xintong!


Best,
Leonard
> 在 2021年6月16日，20:07，Till Rohrmann  写道：
> 
> Congratulations, Xintong!
> 
> Cheers,
> Till
> 
> On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:
> 
>> Congratulations, Xintong!
>> 
>> 
>> Jiayi Liao  于2021年6月16日周三 下午7:30写道：
>> 
 
 
 Congratulations Xintong!
 
 On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
 wrote:
 
> Congratulations, Xintong!
> 
> 
> 
> --
> Sent from:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
 
 
>>> 
>> 

Re: Unsubscribe Dev-Mail

[Till Rohrmann]

Hi,

you have to send a mail to dev-unsubscr...@flink.apache.org in order to
unsubscribe from Flink's dev ML.

Cheers,
Till

On Wed, Jun 16, 2021 at 12:04 PM 雒正林  wrote:

> Unsubscribe

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Till Rohrmann]

Congratulations, Arvid!

Cheers,
Till

On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:

> Congratulations, Arvid!
>
> Nicholas Jiang  于2021年6月16日周三 下午7:25写道：
>
> > Congratulations, Arvid!
> >
> >
> >
> > --
> > Sent from:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> >
>

Re: [ANNOUNCE] New PMC member: Xintong Song

[Till Rohrmann]

Congratulations, Xintong!

Cheers,
Till

On Wed, Jun 16, 2021 at 1:47 PM JING ZHANG  wrote:

> Congratulations, Xintong!
>
>
> Jiayi Liao  于2021年6月16日周三 下午7:30写道：
>
> > >
> > > 
> > > Congratulations Xintong!
> > >
> > > On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
> > > wrote:
> > >
> > >> Congratulations, Xintong!
> > >>
> > >>
> > >>
> > >> --
> > >> Sent from:
> > >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> > >
> > >
> >
>

Re: [DISCUSS] Feedback Collection Jira Bot

[Till Rohrmann]

Another example for category 4 would be the ticket where we collect
breaking API changes for Flink 2.0 [1]. The idea behind this ticket is to
collect things to consider when developing the next major version.
Admittedly, we have never seen the benefits of collecting the breaking
changes because we haven't started Flink 2.x yet. Also, it is not clear how
relevant these tickets are right now.

[1] https://issues.apache.org/jira/browse/FLINK-3957

Cheers,
Till

On Wed, Jun 16, 2021 at 11:42 AM Konstantin Knauf  wrote:

> Hi everyone,
>
> thank you for all the feedback so far. I believe we have four different
> topics by now:
>
> 1 about *test-instability tickets* raised by Robert. Waiting for feedback
> by Robert.
>
> 2 about *aggressiveness of stale-assigned *rule raised by Timo. Waiting
> for feedback by Timo and others.
>
> 3 about *excluding issues with a fixVersion* raised by Konstantin, Till.
> Waiting for more feedback by the community as it involves general changes
> to how we deal with fixVersion.
>
> 4 about *excluding issues with a specific-label* raised by Arvid.
>
> I've already written something about 1-3. Regarding 4:
>
> How do we make sure that these don't become stale? I think, there have
> been a few "long-term efforts" in the past that never got the attention
> that we initially wanted. Is this just about the ability to collect tickets
> under an umbrella to document a future effort? Maybe for the example of
> DataStream replacing DataSet how would this look like in Jira?
>
> Cheers,
>
> Konstantin
>
>
> On Tue, Jun 8, 2021 at 11:31 AM Till Rohrmann 
> wrote:
>
>> I like this idea. It would then be the responsibility of the component
>> maintainers to manage the lifecycle explicitly.
>>
>> Cheers,
>> Till
>>
>> On Mon, Jun 7, 2021 at 1:48 PM Arvid Heise  wrote:
>>
>> > One more idea for the bot. Could we have a label to exclude certain
>> tickets
>> > from the life-cycle?
>> >
>> > I'm thinking about long-term tickets such as improving DataStream to
>> > eventually replace DataSet. We would collect ideas over the next couple
>> of
>> > weeks without any visible progress on the implementation.
>> >
>> > On Fri, May 21, 2021 at 2:06 PM Konstantin Knauf 
>> > wrote:
>> >
>> > > Hi Timo,
>> > >
>> > > Thanks for joining the discussion. All rules except the unassigned
>> rule
>> > do
>> > > not apply to Sub-Tasks actually (like deprioritization, closing).
>> > > Additionally, activity on a Sub-Taks counts as activity for the
>> parent.
>> > So,
>> > > the parent ticket would not be touched by the bot as long as there is
>> a
>> > > single Sub-Task that has a discussion or an update. If you experience
>> > > something different, this is a bug.
>> > >
>> > > Is there a reason why it is important to assign all Sub-Tasks to the
>> same
>> > > person immediately? I am not sure if this kind "reserving tickets" is
>> a
>> > > good idea in general to be honest.
>> > >
>> > > Cheers,
>> > >
>> > > Konstantin
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Fri, May 21, 2021 at 12:00 PM Timo Walther 
>> > wrote:
>> > >
>> > > > Hi Konstantin,
>> > > >
>> > > > thanks for starting this discussion. I was also about to provide
>> some
>> > > > feedback because I have the feeling that the bot is too aggressive
>> at
>> > > > the moment.
>> > > >
>> > > > Even a 14 days interval is a short period of time for bigger efforts
>> > > > that might include several subtasks. Currently, if we split an issue
>> > > > into subtasks usually most subtasks are assigned to the same person.
>> > But
>> > > > the bot requires us to update all subtasks again after 7 days.
>> Could we
>> > > > disable the bot for subtasks or extend the period to 30 days?
>> > > >
>> > > > The core problem in the past was that we had issues laying around
>> > > > untouched for years. Luckily, this is solved with the bot now. But
>> > going
>> > > > from years to 7 days spams the mail box quite a bit.
>> > > >
>> > > > Regards,
>> > > > Timo
>> > > >
>> > > >
>> > > > On 21.05.21 09:22, Konstantin Knauf wrote:
>> > > > > Hi Robert,
>> > > > >
>> > > > > Could you elaborate on your comment on test instabilities? Would
>> test
>> > > > > instabilities always get a fixVersion then?
>> > > > >
>> > > > > Background: Test instabilities are supposed to be Critical.
>> Critical
>> > > > > tickets are deprioritized if they are unassigned and have not
>> > received
>> > > an
>> > > > > update for 14 days.
>> > > > >
>> > > > > Cheers,
>> > > > >
>> > > > > Konstantin
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Thu, May 20, 2021 at 9:34 AM Robert Metzger <
>> rmetz...@apache.org>
>> > > > wrote:
>> > > > >
>> > > > >> +1
>> > > > >> This would also cover test instabilities, which I personally
>> believe
>> > > > should
>> > > > >> not be auto-deprioritized until they've been analyzed.
>> > > > >>
>> > > > >> On Wed, May 19, 2021 at 1:46 PM Till Rohrmann <
>> trohrm...@apache.org
>> > >
>> > > > >> wrote:
>> > > > >>
>> > > > >>> I like this idea. +1 for yo

Re: [ANNOUNCE] New PMC member: Arvid Heise

[JING ZHANG]

Congratulations, Arvid!

Nicholas Jiang  于2021年6月16日周三 下午7:25写道：

> Congratulations, Arvid!
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>

Re: [ANNOUNCE] New PMC member: Xintong Song

[JING ZHANG]

Congratulations, Xintong!


Jiayi Liao  于2021年6月16日周三 下午7:30写道：

> >
> > 
> > Congratulations Xintong!
> >
> > On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
> > wrote:
> >
> >> Congratulations, Xintong!
> >>
> >>
> >>
> >> --
> >> Sent from:
> >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> >
> >
>

Re: [ANNOUNCE] New PMC member: Xintong Song

[Jiayi Liao]

>
> 
> Congratulations Xintong!
>
> On Wed, Jun 16, 2021 at 7:24 PM Nicholas Jiang 
> wrote:
>
>> Congratulations, Xintong!
>>
>>
>>
>> --
>> Sent from:
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>
>

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Nicholas Jiang]

Congratulations, Arvid!



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

Re: [ANNOUNCE] New PMC member: Xintong Song

[Nicholas Jiang]

Congratulations, Xintong!



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Hausmann, Steffen]

Congrats, Arvid!

On 16.06.21, 13:22, "Dian Fu"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



Congratulations, Arvid!

> 2021年6月16日 下午7:16，Benchao Li  写道：
>
> Congratulations, Arvid!
>
> Thomas Weise  于2021年6月16日周三 下午6:44写道：
>
>> Congratulations!
>>
>> On Wed, Jun 16, 2021 at 12:04 PM 刘建刚  wrote:
>>>
>>> Congratulations, Arvid!
>>>
>>> Best
>>>
>>> Xintong Song  于2021年6月16日周三 下午5:51写道：
>>>
 Congratulations, Arvid~!

 Thank you~

 Xintong Song



 On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
 wrote:

> Congratulations, Arvid!
>
> On Wed, Jun 16, 2021 at 11:30 AM Yangze Guo 
>> wrote:
>
>> Congrats, Arvid!
>>
>> Best,
>> Yangze Guo
>>
>> On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren 
> wrote:
>>>
>>> Congratulations Arvid!
>>>
>>> --
>>> Best Regards,
>>>
>>> Qingsheng Ren
>>> Email: renqs...@gmail.com
>>> On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz <
> dwysakow...@apache.org>,
>> wrote:
 Hi all!

 I'm very happy to announce that Arvid Heise has joined the
>> Flink
 PMC!

 Congratulations and welcome Arvid!

 Best,
 Dawid
>>
>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>

>>
>
>
> --
>
> Best,
> Benchao Li





Amazon Web Services EMEA SARL
38 avenue John F. Kennedy, L-1855 Luxembourg
Sitz der Gesellschaft: L-1855 Luxemburg
eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284

Amazon Web Services EMEA SARL, Niederlassung Deutschland
Marcel-Breuer-Str. 12, D-80807 Muenchen
Sitz der Zweigniederlassung: Muenchen
eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, 
USt-ID DE317013094

Re: [ANNOUNCE] New PMC member: Xintong Song

[Dian Fu]

Congratulations, Xintong!

> 2021年6月16日 下午7:16，Benchao Li  写道：
> 
> Congratulations, Xintong!
> 
> Yang Wang  于2021年6月16日周三 下午6:48写道：
> 
>> Congratulations Xintong!
>> 
>> Best,
>> Yang
>> 
>> Thomas Weise  于2021年6月16日周三 下午6:46写道：
>> 
>>> Congratulations!
>>> 
>>> On Wed, Jun 16, 2021 at 12:14 PM Nirvana <1572139...@qq.com> wrote:
 
 Congratulations!
 
 
 ---Original---
 From: "Arvid Heise">>> Date: Wed, Jun 16, 2021 18:12 PM
 To: "dev">>> Subject: Re: [ANNOUNCE] New PMC member: Xintong Song
 
 
 Congratulations!
 
 On Wed, Jun 16, 2021 at 12:05 PM 刘建刚 >> wrote:
 
 > Congrats, Xintong!
 >
 > Best.
 >
 > Xintong Song >>> >
 > > Thanks all for the support.
 > > It's my honor to be part of the community and work with all
>> of
>>> you great
 > > people.
 > >
 > > Thank you~
 > >
 > > Xintong Song
 > >
 > >
 > > On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf <
>>> kna...@apache.org>
 > > wrote:
 > >
 > > > Congratulations, Xintong!
 > > >
 > > > On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren <
>>> renqs...@gmail.com>
 > > wrote:
 > > >
 > > > > Congratulations Xintong!
 > > > >
 > > > > --
 > > > > Best Regards,
 > > > >
 > > > > Qingsheng Ren
 > > > > Email: renqs...@gmail.com
 > > > > On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz <
 > > dwysakow...@apache.org
 > > > >,
 > > > > wrote:
 > > > > > Hi all!
 > > > > >
 > > > > > I'm very happy to announce that Xintong Song
>>> has joined the Flink
 > > PMC!
 > > > > >
 > > > > > Congratulations and welcome Xintong!
 > > > > >
 > > > > > Best,
 > > > > > Dawid
 > > > >
 > > >
 > > >
 > > > --
 > > >
 > > > Konstantin Knauf
 > > >
 > > > https://twitter.com/snntrable
 > > >
 > > > https://github.com/knaufk
 > > >
 > >
 >
>>> 
>> 
> 
> 
> -- 
> 
> Best,
> Benchao Li

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Dian Fu]

Congratulations, Arvid!

> 2021年6月16日 下午7:16，Benchao Li  写道：
> 
> Congratulations, Arvid!
> 
> Thomas Weise  于2021年6月16日周三 下午6:44写道：
> 
>> Congratulations!
>> 
>> On Wed, Jun 16, 2021 at 12:04 PM 刘建刚  wrote:
>>> 
>>> Congratulations, Arvid!
>>> 
>>> Best
>>> 
>>> Xintong Song  于2021年6月16日周三 下午5:51写道：
>>> 
 Congratulations, Arvid~!
 
 Thank you~
 
 Xintong Song
 
 
 
 On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
 wrote:
 
> Congratulations, Arvid!
> 
> On Wed, Jun 16, 2021 at 11:30 AM Yangze Guo 
>> wrote:
> 
>> Congrats, Arvid!
>> 
>> Best,
>> Yangze Guo
>> 
>> On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren 
> wrote:
>>> 
>>> Congratulations Arvid!
>>> 
>>> --
>>> Best Regards,
>>> 
>>> Qingsheng Ren
>>> Email: renqs...@gmail.com
>>> On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz <
> dwysakow...@apache.org>,
>> wrote:
 Hi all!
 
 I'm very happy to announce that Arvid Heise has joined the
>> Flink
 PMC!
 
 Congratulations and welcome Arvid!
 
 Best,
 Dawid
>> 
> 
> 
> --
> 
> Konstantin Knauf
> 
> https://twitter.com/snntrable
> 
> https://github.com/knaufk
> 
 
>> 
> 
> 
> -- 
> 
> Best,
> Benchao Li

Re: [ANNOUNCE] New PMC member: Xintong Song

[Benchao Li]

Congratulations, Xintong!

Yang Wang  于2021年6月16日周三 下午6:48写道：

> Congratulations Xintong!
>
> Best,
> Yang
>
> Thomas Weise  于2021年6月16日周三 下午6:46写道：
>
> > Congratulations!
> >
> > On Wed, Jun 16, 2021 at 12:14 PM Nirvana <1572139...@qq.com> wrote:
> > >
> > > Congratulations!
> > >
> > >
> > > ---Original---
> > > From: "Arvid Heise" > > Date: Wed, Jun 16, 2021 18:12 PM
> > > To: "dev" > > Subject: Re: [ANNOUNCE] New PMC member: Xintong Song
> > >
> > >
> > > Congratulations!
> > >
> > > On Wed, Jun 16, 2021 at 12:05 PM 刘建刚  > wrote:
> > >
> > > > Congrats, Xintong!
> > > >
> > > > Best.
> > > >
> > > > Xintong Song  > > >
> > > > > Thanks all for the support.
> > > > > It's my honor to be part of the community and work with all
> of
> > you great
> > > > > people.
> > > > >
> > > > > Thank you~
> > > > >
> > > > > Xintong Song
> > > > >
> > > > >
> > > > > On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf <
> > kna...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Congratulations, Xintong!
> > > > > >
> > > > > > On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren <
> > renqs...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Congratulations Xintong!
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards,
> > > > > > >
> > > > > > > Qingsheng Ren
> > > > > > > Email: renqs...@gmail.com
> > > > > > > On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz <
> > > > > dwysakow...@apache.org
> > > > > > >,
> > > > > > > wrote:
> > > > > > > > Hi all!
> > > > > > > >
> > > > > > > > I'm very happy to announce that Xintong Song
> > has joined the Flink
> > > > > PMC!
> > > > > > > >
> > > > > > > > Congratulations and welcome Xintong!
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Dawid
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Konstantin Knauf
> > > > > >
> > > > > > https://twitter.com/snntrable
> > > > > >
> > > > > > https://github.com/knaufk
> > > > > >
> > > > >
> > > >
> >
>


-- 

Best,
Benchao Li

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Benchao Li]

Congratulations, Arvid!

Thomas Weise  于2021年6月16日周三 下午6:44写道：

> Congratulations!
>
> On Wed, Jun 16, 2021 at 12:04 PM 刘建刚  wrote:
> >
> > Congratulations, Arvid!
> >
> > Best
> >
> > Xintong Song  于2021年6月16日周三 下午5:51写道：
> >
> > > Congratulations, Arvid~!
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > >
> > > On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
> > > wrote:
> > >
> > > > Congratulations, Arvid!
> > > >
> > > > On Wed, Jun 16, 2021 at 11:30 AM Yangze Guo 
> wrote:
> > > >
> > > > > Congrats, Arvid!
> > > > >
> > > > > Best,
> > > > > Yangze Guo
> > > > >
> > > > > On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren 
> > > > wrote:
> > > > > >
> > > > > > Congratulations Arvid!
> > > > > >
> > > > > > --
> > > > > > Best Regards,
> > > > > >
> > > > > > Qingsheng Ren
> > > > > > Email: renqs...@gmail.com
> > > > > > On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz <
> > > > dwysakow...@apache.org>,
> > > > > wrote:
> > > > > > > Hi all!
> > > > > > >
> > > > > > > I'm very happy to announce that Arvid Heise has joined the
> Flink
> > > PMC!
> > > > > > >
> > > > > > > Congratulations and welcome Arvid!
> > > > > > >
> > > > > > > Best,
> > > > > > > Dawid
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Konstantin Knauf
> > > >
> > > > https://twitter.com/snntrable
> > > >
> > > > https://github.com/knaufk
> > > >
> > >
>


-- 

Best,
Benchao Li

Re: [ANNOUNCE] New PMC member: Xintong Song

[Yang Wang]

Congratulations Xintong!

Best,
Yang

Thomas Weise  于2021年6月16日周三 下午6:46写道：

> Congratulations!
>
> On Wed, Jun 16, 2021 at 12:14 PM Nirvana <1572139...@qq.com> wrote:
> >
> > Congratulations!
> >
> >
> > ---Original---
> > From: "Arvid Heise" > Date: Wed, Jun 16, 2021 18:12 PM
> > To: "dev" > Subject: Re: [ANNOUNCE] New PMC member: Xintong Song
> >
> >
> > Congratulations!
> >
> > On Wed, Jun 16, 2021 at 12:05 PM 刘建刚  wrote:
> >
> > > Congrats, Xintong!
> > >
> > > Best.
> > >
> > > Xintong Song  > >
> > > > Thanks all for the support.
> > > > It's my honor to be part of the community and work with all of
> you great
> > > > people.
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > > On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf <
> kna...@apache.org>
> > > > wrote:
> > > >
> > > > > Congratulations, Xintong!
> > > > >
> > > > > On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren <
> renqs...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Congratulations Xintong!
> > > > > >
> > > > > > --
> > > > > > Best Regards,
> > > > > >
> > > > > > Qingsheng Ren
> > > > > > Email: renqs...@gmail.com
> > > > > > On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz <
> > > > dwysakow...@apache.org
> > > > > >,
> > > > > > wrote:
> > > > > > > Hi all!
> > > > > > >
> > > > > > > I'm very happy to announce that Xintong Song
> has joined the Flink
> > > > PMC!
> > > > > > >
> > > > > > > Congratulations and welcome Xintong!
> > > > > > >
> > > > > > > Best,
> > > > > > > Dawid
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Konstantin Knauf
> > > > >
> > > > > https://twitter.com/snntrable
> > > > >
> > > > > https://github.com/knaufk
> > > > >
> > > >
> > >
>

Re: [ANNOUNCE] New PMC member: Xintong Song

[Thomas Weise]

Congratulations!

On Wed, Jun 16, 2021 at 12:14 PM Nirvana <1572139...@qq.com> wrote:
>
> Congratulations!
>
>
> ---Original---
> From: "Arvid Heise" Date: Wed, Jun 16, 2021 18:12 PM
> To: "dev" Subject: Re: [ANNOUNCE] New PMC member: Xintong Song
>
>
> Congratulations!
>
> On Wed, Jun 16, 2021 at 12:05 PM 刘建刚 
> > Congrats, Xintong!
> >
> > Best.
> >
> > Xintong Song  >
> > > Thanks all for the support.
> > > It's my honor to be part of the community and work with all of you 
> great
> > > people.
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > > On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
>  > > wrote:
> > >
> > > > Congratulations, Xintong!
> > > >
> > > > On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren 
>  > > wrote:
> > > >
> > > > > Congratulations Xintong!
> > > > >
> > > > > --
> > > > > Best Regards,
> > > > >
> > > > > Qingsheng Ren
> > > > > Email: renqs...@gmail.com
> > > > > On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz <
> > > dwysakow...@apache.org
> > > > >,
> > > > > wrote:
> > > > > > Hi all!
> > > > > >
> > > > > > I'm very happy to announce that Xintong Song has 
> joined the Flink
> > > PMC!
> > > > > >
> > > > > > Congratulations and welcome Xintong!
> > > > > >
> > > > > > Best,
> > > > > > Dawid
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Konstantin Knauf
> > > >
> > > > https://twitter.com/snntrable
> > > >
> > > > https://github.com/knaufk
> > > >
> > >
> >

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Thomas Weise]

Congratulations!

On Wed, Jun 16, 2021 at 12:04 PM 刘建刚  wrote:
>
> Congratulations, Arvid!
>
> Best
>
> Xintong Song  于2021年6月16日周三 下午5:51写道：
>
> > Congratulations, Arvid~!
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
> > wrote:
> >
> > > Congratulations, Arvid!
> > >
> > > On Wed, Jun 16, 2021 at 11:30 AM Yangze Guo  wrote:
> > >
> > > > Congrats, Arvid!
> > > >
> > > > Best,
> > > > Yangze Guo
> > > >
> > > > On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren 
> > > wrote:
> > > > >
> > > > > Congratulations Arvid!
> > > > >
> > > > > --
> > > > > Best Regards,
> > > > >
> > > > > Qingsheng Ren
> > > > > Email: renqs...@gmail.com
> > > > > On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz <
> > > dwysakow...@apache.org>,
> > > > wrote:
> > > > > > Hi all!
> > > > > >
> > > > > > I'm very happy to announce that Arvid Heise has joined the Flink
> > PMC!
> > > > > >
> > > > > > Congratulations and welcome Arvid!
> > > > > >
> > > > > > Best,
> > > > > > Dawid
> > > >
> > >
> > >
> > > --
> > >
> > > Konstantin Knauf
> > >
> > > https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >

Re: [ANNOUNCE] New PMC member: Xintong Song

[liufangliang789]

Re: [ANNOUNCE] New PMC member: Xintong Song

[Nirvana]

Congratulations!


---Original---
From: "Arvid Heise"https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >
>

Re: [ANNOUNCE] New PMC member: Xintong Song

[Nirvana]

Congratulations!



---Original---
From: "Arvid Heise"https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >
>

Re: [ANNOUNCE] New PMC member: Xintong Song

[Arvid Heise]

Congratulations!

On Wed, Jun 16, 2021 at 12:05 PM 刘建刚  wrote:

> Congrats, Xintong!
>
> Best.
>
> Xintong Song  于2021年6月16日周三 下午5:51写道：
>
> > Thanks all for the support.
> > It's my honor to be part of the community and work with all of you great
> > people.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> > On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
> > wrote:
> >
> > > Congratulations, Xintong!
> > >
> > > On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren 
> > wrote:
> > >
> > > > Congratulations Xintong!
> > > >
> > > > --
> > > > Best Regards,
> > > >
> > > > Qingsheng Ren
> > > > Email: renqs...@gmail.com
> > > > On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz <
> > dwysakow...@apache.org
> > > >,
> > > > wrote:
> > > > > Hi all!
> > > > >
> > > > > I'm very happy to announce that Xintong Song has joined the Flink
> > PMC!
> > > > >
> > > > > Congratulations and welcome Xintong!
> > > > >
> > > > > Best,
> > > > > Dawid
> > > >
> > >
> > >
> > > --
> > >
> > > Konstantin Knauf
> > >
> > > https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >
>

[jira] [Created] (FLINK-23011) FLIP-27 sources are generating non-deterministic results when using event time

[Piotr Nowojski (Jira)]

Piotr Nowojski created FLINK-23011:
--

 Summary: FLIP-27 sources are generating non-deterministic results 
when using event time
 Key: FLINK-23011
 URL: https://issues.apache.org/jira/browse/FLINK-23011
 Project: Flink
  Issue Type: New Feature
  Components: API / DataStream
Affects Versions: 1.12.4, 1.13.1, 1.14.0
 Environment: 

Reporter: Piotr Nowojski


FLIP-27 sources currently start in the {{StreamStatus.IDLE}} state and they 
switch to {{ACTIVE}} only after emitting first {{Watermark}}. Until this 
happens, downstream operators are ignoring {{IDLE}} inputs from calculating the 
input (min) watermark. 

An extreme example to what problem this leads to, are completely bogus results 
if for example one FLIP-27 source subtask is slower than others for some reason:
{code:java}
env.getConfig().setAutoWatermarkInterval(2000);
env.setParallelism(2);
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(Integer.MAX_VALUE, 
10));

DataStream eventStream =
env.fromSource(
new NumberSequenceSource(0, Long.MAX_VALUE),
WatermarkStrategy.forMonotonousTimestamps()
.withTimestampAssigner(new 
LongTimestampAssigner()),
"NumberSequenceSource")
.map(
new RichMapFunction() {
@Override
public Long map(Long value) throws Exception {
if (getRuntimeContext().getIndexOfThisSubtask() 
== 0) {
Thread.sleep(1);
}
return 1L;
}
});

eventStream.windowAll(TumblingEventTimeWindows.of(Time.seconds(1))).sum(0).print();

(...)
private static class LongTimestampAssigner implements 
SerializableTimestampAssigner {
private long counter = 0;

@Override
public long extractTimestamp(Long record, long recordTimeStamp) {
return counter++;
}
}
{code}
In such case, after 2 seconds ({{setAutoWatermarkInterval}}) the not throttled 
subtask (subTaskId == 1) generates very high watermarks. The other source 
subtask (subTaskId == 0) emits very low watermarks. If the non throttled 
watermark reaches the downstream {{WindowOperator}} first, while the other 
input channel is still idle, it will take those high watermarks as combined 
input watermark for the the whole {{WindowOperator}}. When the input channel 
from the throttled source subtask finally receives it's {{ACTIVE}} status and a 
much lower watermark, that's already too late.

Actual output of the example program:
{noformat}
1596
2000
1000
1000
1000
1000
1000
1000
(...)
{noformat}
while the expected output should be always "2000" (2000 records fitting in 
every 1 second global window)
{noformat}
2000
2000
2000
2000
(...)
{noformat}.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New PMC member: Xintong Song

[刘建刚]

Congrats, Xintong!

Best.

Xintong Song  于2021年6月16日周三 下午5:51写道：

> Thanks all for the support.
> It's my honor to be part of the community and work with all of you great
> people.
>
> Thank you~
>
> Xintong Song
>
>
> On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
> wrote:
>
> > Congratulations, Xintong!
> >
> > On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren 
> wrote:
> >
> > > Congratulations Xintong!
> > >
> > > --
> > > Best Regards,
> > >
> > > Qingsheng Ren
> > > Email: renqs...@gmail.com
> > > On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz <
> dwysakow...@apache.org
> > >,
> > > wrote:
> > > > Hi all!
> > > >
> > > > I'm very happy to announce that Xintong Song has joined the Flink
> PMC!
> > > >
> > > > Congratulations and welcome Xintong!
> > > >
> > > > Best,
> > > > Dawid
> > >
> >
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>

Unsubscribe Dev-Mail

[雒正林]

Unsubscribe

Re: [ANNOUNCE] New PMC member: Arvid Heise

[刘建刚]

Congratulations, Arvid!

Best

Xintong Song  于2021年6月16日周三 下午5:51写道：

> Congratulations, Arvid~!
>
> Thank you~
>
> Xintong Song
>
>
>
> On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf 
> wrote:
>
> > Congratulations, Arvid!
> >
> > On Wed, Jun 16, 2021 at 11:30 AM Yangze Guo  wrote:
> >
> > > Congrats, Arvid!
> > >
> > > Best,
> > > Yangze Guo
> > >
> > > On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren 
> > wrote:
> > > >
> > > > Congratulations Arvid!
> > > >
> > > > --
> > > > Best Regards,
> > > >
> > > > Qingsheng Ren
> > > > Email: renqs...@gmail.com
> > > > On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz <
> > dwysakow...@apache.org>,
> > > wrote:
> > > > > Hi all!
> > > > >
> > > > > I'm very happy to announce that Arvid Heise has joined the Flink
> PMC!
> > > > >
> > > > > Congratulations and welcome Arvid!
> > > > >
> > > > > Best,
> > > > > Dawid
> > >
> >
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Xintong Song]

Congratulations, Arvid~!

Thank you~

Xintong Song



On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf  wrote:

> Congratulations, Arvid!
>
> On Wed, Jun 16, 2021 at 11:30 AM Yangze Guo  wrote:
>
> > Congrats, Arvid!
> >
> > Best,
> > Yangze Guo
> >
> > On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren 
> wrote:
> > >
> > > Congratulations Arvid!
> > >
> > > --
> > > Best Regards,
> > >
> > > Qingsheng Ren
> > > Email: renqs...@gmail.com
> > > On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz <
> dwysakow...@apache.org>,
> > wrote:
> > > > Hi all!
> > > >
> > > > I'm very happy to announce that Arvid Heise has joined the Flink PMC!
> > > >
> > > > Congratulations and welcome Arvid!
> > > >
> > > > Best,
> > > > Dawid
> >
>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>

Re: [ANNOUNCE] New PMC member: Xintong Song

[Xintong Song]

Thanks all for the support.
It's my honor to be part of the community and work with all of you great
people.

Thank you~

Xintong Song


On Wed, Jun 16, 2021 at 5:31 PM Konstantin Knauf  wrote:

> Congratulations, Xintong!
>
> On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren  wrote:
>
> > Congratulations Xintong!
> >
> > --
> > Best Regards,
> >
> > Qingsheng Ren
> > Email: renqs...@gmail.com
> > On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz  >,
> > wrote:
> > > Hi all!
> > >
> > > I'm very happy to announce that Xintong Song has joined the Flink PMC!
> > >
> > > Congratulations and welcome Xintong!
> > >
> > > Best,
> > > Dawid
> >
>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>

Re: [DISCUSS] Dashboard/HistoryServer authentication

[Gabor Somogyi]

Hi Till,

Did you have the chance to take a look at the doc? Not yet seen any update.

BR,
G


On Wed, Jun 9, 2021 at 1:43 PM Till Rohrmann  wrote:

> Thanks for the update Gabor. I'll take a look and respond in the document.
>
> Cheers,
> Till
>
> On Wed, Jun 9, 2021 at 12:59 PM Gabor Somogyi 
> wrote:
>
>> Hi Till,
>>
>> Your proxy suggestion has been considered in-depth and updated the FLIP
>> accordingly.
>> We've considered 2 proxy implementation (Nginx and Squid) but according
>> to our analysis and testing it's not suitable for the mentioned use-cases.
>> Please take a look at the rejected alternatives for detailed explanation.
>>
>> Thanks for your time in advance!
>>
>> BR,
>> G
>>
>>
>> On Fri, Jun 4, 2021 at 3:31 PM Till Rohrmann 
>> wrote:
>>
>>> As I've said I am not a security expert and that's why I have to ask for
>>> clarification, Gabor. You are saying that if we configure a truststore for
>>> the REST endpoint with a single trusted certificate which has been
>>> generated by the operator of the Flink cluster, then the attacker can
>>> generate a new certificate, sign it and then talk to the Flink cluster if
>>> he has access to the node on which the REST endpoint runs? My understanding
>>> was that you need the corresponding private key which in my proposed setup
>>> would be under the control of the operator as well (e.g. stored in a
>>> keystore on the same machine but guarded by some secret). That way (if I am
>>> not mistaken), only the entity which has access to the keystore is able to
>>> talk to the Flink cluster.
>>>
>>> Maybe we are also getting our wires crossed here and are talking about
>>> different things.
>>>
>>> Thanks for listing the pros and cons of Kerberos. Concerning what other
>>> authentication mechanisms are used in the industry, I am not 100% sure.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Fri, Jun 4, 2021 at 11:09 AM Gabor Somogyi 
>>> wrote:
>>>
 > I did not mean for the user to sign its own certificates but for the
 operator of the cluster. Once the user request hits the proxy, it should no
 longer be under his control. I think I do not fully understand yet why this
 would not work.
 I said it's not solving the authentication problem over any proxy. Even
 if the operator is signing the certificate one can have access to an
 internal node.
 Such case anybody can craft certificates which is accepted by the
 server. When it's accepted a bad guy can cancel jobs causing huge impacts.

 > Also, I am missing a bit the comparison of Kerberos to other
 authentication mechanisms and why they were rejected in favour of Kerberos.
 PROS:
 * Since it's not depending on cloud provider and/or k8s or bare-metal
 etc. deployment it's the biggest plus
 * Centralized with tools and no need to write tons of tools around
 * There are clients/tools on almost all OS-es and several languages
 * Super huge users are using it for years in production w/o huge issues
 * Provides cross-realm trust possibility amongst other features
 * Several open source components using it which could increase
 compatibility

 CONS:
 * Not everybody using kerberos
 * It would increase the code footprint but this is true for many
 features (as a side note I'm here to maintain it)

 Feel free to add your points because it only represents a single
 viewpoint.
 Also if you have any better option for strong authentication please
 share it and we can consider the pros/cons here.

 BR,
 G


 On Fri, Jun 4, 2021 at 10:32 AM Till Rohrmann 
 wrote:

> I did not mean for the user to sign its own certificates but for the
> operator of the cluster. Once the user request hits the proxy, it should 
> no
> longer be under his control. I think I do not fully understand yet why 
> this
> would not work.
>
> What I would like to avoid is to add more complexity into Flink if
> there is an easy solution which fulfills the requirements. That's why I
> would like to exercise thoroughly through the different alternatives. 
> Also,
> I am missing a bit the comparison of Kerberos to other authentication
> mechanisms and why they were rejected in favour of Kerberos.
>
> Cheers,
> Till
>
> On Fri, Jun 4, 2021 at 10:26 AM Gyula Fóra  wrote:
>
>> Hi!
>>
>> I think there might be possible alternatives but it seems Kerberos on
>> the rest endpoint ticks all the right boxes and provides a super clean 
>> and
>> simple solution for strong authentication.
>>
>> I wouldn’t even consider sidecar proxies etc if we can solve it in
>> such a simple way as proposed by G.
>>
>> Cheers
>> Gyula
>>
>> On Fri, 4 Jun 2021 at 10:03, Till Rohrmann 
>> wrote:
>>
>>> I am not saying that we shouldn't add a strong authentication
>>> mechanism if there are good re

Re: [DISCUSS] Feedback Collection Jira Bot

[Konstantin Knauf]

Hi everyone,

thank you for all the feedback so far. I believe we have four different
topics by now:

1 about *test-instability tickets* raised by Robert. Waiting for feedback
by Robert.

2 about *aggressiveness of stale-assigned *rule raised by Timo. Waiting for
feedback by Timo and others.

3 about *excluding issues with a fixVersion* raised by Konstantin, Till.
Waiting for more feedback by the community as it involves general changes
to how we deal with fixVersion.

4 about *excluding issues with a specific-label* raised by Arvid.

I've already written something about 1-3. Regarding 4:

How do we make sure that these don't become stale? I think, there have been
a few "long-term efforts" in the past that never got the attention that we
initially wanted. Is this just about the ability to collect tickets under
an umbrella to document a future effort? Maybe for the example of
DataStream replacing DataSet how would this look like in Jira?

Cheers,

Konstantin


On Tue, Jun 8, 2021 at 11:31 AM Till Rohrmann  wrote:

> I like this idea. It would then be the responsibility of the component
> maintainers to manage the lifecycle explicitly.
>
> Cheers,
> Till
>
> On Mon, Jun 7, 2021 at 1:48 PM Arvid Heise  wrote:
>
> > One more idea for the bot. Could we have a label to exclude certain
> tickets
> > from the life-cycle?
> >
> > I'm thinking about long-term tickets such as improving DataStream to
> > eventually replace DataSet. We would collect ideas over the next couple
> of
> > weeks without any visible progress on the implementation.
> >
> > On Fri, May 21, 2021 at 2:06 PM Konstantin Knauf 
> > wrote:
> >
> > > Hi Timo,
> > >
> > > Thanks for joining the discussion. All rules except the unassigned rule
> > do
> > > not apply to Sub-Tasks actually (like deprioritization, closing).
> > > Additionally, activity on a Sub-Taks counts as activity for the parent.
> > So,
> > > the parent ticket would not be touched by the bot as long as there is a
> > > single Sub-Task that has a discussion or an update. If you experience
> > > something different, this is a bug.
> > >
> > > Is there a reason why it is important to assign all Sub-Tasks to the
> same
> > > person immediately? I am not sure if this kind "reserving tickets" is a
> > > good idea in general to be honest.
> > >
> > > Cheers,
> > >
> > > Konstantin
> > >
> > >
> > >
> > >
> > >
> > > On Fri, May 21, 2021 at 12:00 PM Timo Walther 
> > wrote:
> > >
> > > > Hi Konstantin,
> > > >
> > > > thanks for starting this discussion. I was also about to provide some
> > > > feedback because I have the feeling that the bot is too aggressive at
> > > > the moment.
> > > >
> > > > Even a 14 days interval is a short period of time for bigger efforts
> > > > that might include several subtasks. Currently, if we split an issue
> > > > into subtasks usually most subtasks are assigned to the same person.
> > But
> > > > the bot requires us to update all subtasks again after 7 days. Could
> we
> > > > disable the bot for subtasks or extend the period to 30 days?
> > > >
> > > > The core problem in the past was that we had issues laying around
> > > > untouched for years. Luckily, this is solved with the bot now. But
> > going
> > > > from years to 7 days spams the mail box quite a bit.
> > > >
> > > > Regards,
> > > > Timo
> > > >
> > > >
> > > > On 21.05.21 09:22, Konstantin Knauf wrote:
> > > > > Hi Robert,
> > > > >
> > > > > Could you elaborate on your comment on test instabilities? Would
> test
> > > > > instabilities always get a fixVersion then?
> > > > >
> > > > > Background: Test instabilities are supposed to be Critical.
> Critical
> > > > > tickets are deprioritized if they are unassigned and have not
> > received
> > > an
> > > > > update for 14 days.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Konstantin
> > > > >
> > > > >
> > > > >
> > > > > On Thu, May 20, 2021 at 9:34 AM Robert Metzger <
> rmetz...@apache.org>
> > > > wrote:
> > > > >
> > > > >> +1
> > > > >> This would also cover test instabilities, which I personally
> believe
> > > > should
> > > > >> not be auto-deprioritized until they've been analyzed.
> > > > >>
> > > > >> On Wed, May 19, 2021 at 1:46 PM Till Rohrmann <
> trohrm...@apache.org
> > >
> > > > >> wrote:
> > > > >>
> > > > >>> I like this idea. +1 for your proposal Konstantin.
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Till
> > > > >>>
> > > > >>> On Wed, May 19, 2021 at 1:30 PM Konstantin Knauf <
> > > > >> konstan...@ververica.com
> > > > 
> > > > >>> wrote:
> > > > >>>
> > > >  Hi everyone,
> > > > 
> > > >  Till and I recently discussed whether we should disable the
> > > >  "stale-blocker", "stale-critical", "stale-major" and
> "stale-minor"
> > > > >> rules
> > > >  for tickets that have a fixVersion set. This would allow people
> to
> > > > plan
> > > > >>> the
> > > >  upcoming release without tickets being deprioritized by the bot
> > > during
> > > > >>> the
> > > >  release cycle.
> > > > 
> >

Re: [ANNOUNCE] New PMC member: Xintong Song

[Konstantin Knauf]

Congratulations, Xintong!

On Wed, Jun 16, 2021 at 11:30 AM Qingsheng Ren  wrote:

> Congratulations Xintong!
>
> --
> Best Regards,
>
> Qingsheng Ren
> Email: renqs...@gmail.com
> On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz ,
> wrote:
> > Hi all!
> >
> > I'm very happy to announce that Xintong Song has joined the Flink PMC!
> >
> > Congratulations and welcome Xintong!
> >
> > Best,
> > Dawid
>


-- 

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Konstantin Knauf]

Congratulations, Arvid!

On Wed, Jun 16, 2021 at 11:30 AM Yangze Guo  wrote:

> Congrats, Arvid!
>
> Best,
> Yangze Guo
>
> On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren  wrote:
> >
> > Congratulations Arvid!
> >
> > --
> > Best Regards,
> >
> > Qingsheng Ren
> > Email: renqs...@gmail.com
> > On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz ,
> wrote:
> > > Hi all!
> > >
> > > I'm very happy to announce that Arvid Heise has joined the Flink PMC!
> > >
> > > Congratulations and welcome Arvid!
> > >
> > > Best,
> > > Dawid
>


-- 

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

Re: [ANNOUNCE] New PMC member: Xintong Song

[Qingsheng Ren]

Congratulations Xintong!

--
Best Regards,

Qingsheng Ren
Email: renqs...@gmail.com
On Jun 16, 2021, 5:23 PM +0800, Dawid Wysakowicz , 
wrote:
> Hi all!
>
> I'm very happy to announce that Xintong Song has joined the Flink PMC!
>
> Congratulations and welcome Xintong!
>
> Best,
> Dawid

Re: [ANNOUNCE] New PMC member: Xintong Song

[Paul Lam]

Congrats, Xintong! Well deserved!

Best,
Paul Lam

> 2021年6月16日 17:23，Dawid Wysakowicz  写道：
> 
> Hi all!
> 
> I'm very happy to announce that Xintong Song has joined the Flink PMC!
> 
> Congratulations and welcome Xintong!
> 
> Best,
> Dawid
> 

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Yangze Guo]

Congrats, Arvid!

Best,
Yangze Guo

On Wed, Jun 16, 2021 at 5:27 PM Qingsheng Ren  wrote:
>
> Congratulations Arvid!
>
> --
> Best Regards,
>
> Qingsheng Ren
> Email: renqs...@gmail.com
> On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz , 
> wrote:
> > Hi all!
> >
> > I'm very happy to announce that Arvid Heise has joined the Flink PMC!
> >
> > Congratulations and welcome Arvid!
> >
> > Best,
> > Dawid

Re: [ANNOUNCE] New PMC member: Xintong Song

[Yangze Guo]

Congrats, Xintong!

Best,
Yangze Guo

On Wed, Jun 16, 2021 at 5:23 PM Dawid Wysakowicz  wrote:
>
> Hi all!
>
> I'm very happy to announce that Xintong Song has joined the Flink PMC!
>
> Congratulations and welcome Xintong!
>
> Best,
> Dawid

Re: [ANNOUNCE] New PMC member: Arvid Heise

[Qingsheng Ren]

Congratulations Arvid!

--
Best Regards,

Qingsheng Ren
Email: renqs...@gmail.com
On Jun 16, 2021, 5:21 PM +0800, Dawid Wysakowicz , 
wrote:
> Hi all!
>
> I'm very happy to announce that Arvid Heise has joined the Flink PMC!
>
> Congratulations and welcome Arvid!
>
> Best,
> Dawid

[ANNOUNCE] New PMC member: Xintong Song

[Dawid Wysakowicz]

Hi all!

I'm very happy to announce that Xintong Song has joined the Flink PMC!

Congratulations and welcome Xintong!

Best,
Dawid



OpenPGP_signature
Description: OpenPGP digital signature

[ANNOUNCE] New PMC member: Arvid Heise

[Dawid Wysakowicz]

Hi all!

I'm very happy to announce that Arvid Heise has joined the Flink PMC!

Congratulations and welcome Arvid!

Best,
Dawid



OpenPGP_signature
Description: OpenPGP digital signature

[jira] [Created] (FLINK-23010) HivePartitionFetcherContextBase::getComparablePartitionValueList can return partitions that don't exist

[Rui Li (Jira)]

Rui Li created FLINK-23010:
--

 Summary: 
HivePartitionFetcherContextBase::getComparablePartitionValueList can return 
partitions that don't exist
 Key: FLINK-23010
 URL: https://issues.apache.org/jira/browse/FLINK-23010
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Hive
Reporter: Rui Li
Assignee: Rui Li






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23009) Bump up Guava in Kinesis Connector

[Emre Kartoglu (Jira)]

Emre Kartoglu created FLINK-23009:
-

 Summary: Bump up Guava in Kinesis Connector
 Key: FLINK-23009
 URL: https://issues.apache.org/jira/browse/FLINK-23009
 Project: Flink
  Issue Type: Improvement
Affects Versions: 1.14.0, 1.12.5, 1.13.2
Reporter: Emre Kartoglu


*Background*

We maintain a copy of the Flink connector in our AWS GitHub group: 
[https://github.com/awslabs/amazon-kinesis-connector-flink]

We've recently upgraded the Guava library in our AWS copy as the version we 
were using was quite old and had incompatible interface with the later and more 
commonly used Guava versions. As part of this ticket we'll be applying the same 
changes in the Flink repo 
https://github.com/apache/flink/tree/master/flink-connectors/flink-connector-kinesis
 

*Scope*
 * Upgrade Guava library in pom.xml
 * Switch to 3-arg version of Guava Futures.addCallback method call, as the old 
2-arg version is no longer supported

*Result*

All existing and new tests should pass

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Got multiple issues when running the tutorial project "table-walkthrough" on IDEA

[Guowei Ma]

Hi, Lingfeng

These job errors you posted happened when the job(`SpendReport`) was
running on the IDE?
According to my understanding, this document[1] & repository[2] mean that
the example is to be run in docker, not in IDE.

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/try-flink/table_api/
[2] https://github.com/apache/flink-playgrounds

Best,
Guowei


On Tue, Jun 15, 2021 at 10:29 AM Lingfeng Pu  wrote:

> Hi,
>
> I'm currently trying to debug the tutorial project "table-walkthrough" on
> IDEA on a standalone Flink environment. I installed the required software
> (Java 8 or 11, Maven, Docker) according to the tutorial. I'll provide some
> key environment info about the running environment before describing the
> issues as follows:
>
> 1. OS: Fedora 34;
> 2. Java version: 1.8;
> 3.Maven version: 3.6.3;
> 4. Docker version: 20.10.7;
> 5. Flink version: 1.13.1;
> 6. Scala version: 2.11;
> 7. No Kafka, Yarn, etc. are installed.
>
> *The issues are:*
> When I run the code on IDEA, it returns me WARNs and exceptions. Due to
> this tutorial project did not require things to be installed like Kafka,
> Yarn, I couldn't find any proper solution(s) for my problem after searching
> on the web. *The full issue report is shown below:*
>
>
> *=*
> /usr/java/jdk1.8.0_291-amd64/bin/java
> -javaagent:/var/lib/snapd/snap/intellij-idea-community/302/lib/idea_rt.jar=39087:/var/lib/snapd/snap/intellij-idea-community/302/bin
> -Dfile.encoding=UTF-8 -classpath
> /usr/java/jdk1.8.0_291-amd64/jre/lib/charsets.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/deploy.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/cldrdata.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/dnsns.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/jaccess.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/jfxrt.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/localedata.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/nashorn.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/sunec.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/sunjce_provider.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/sunpkcs11.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/ext/zipfs.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/javaws.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/jce.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/jfr.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/jfxswt.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/jsse.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/management-agent.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/plugin.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/resources.jar:/usr/java/jdk1.8.0_291-amd64/jre/lib/rt.jar:/home/AkatsukiG5/IdeaProjects/flink-playgrounds-master/table-walkthrough/target/classes:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-table-api-java/1.12.1/flink-table-api-java-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-table-common/1.12.1/flink-table-common-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-connector-files/1.12.1/flink-connector-files-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-connector-base/1.12.1/flink-connector-base-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-shaded-asm-7/7.1-12.0/flink-shaded-asm-7-7.1-12.0.jar:/home/AkatsukiG5/Documents/mavenRepo/org/slf4j/slf4j-api/1.7.15/slf4j-api-1.7.15.jar:/home/AkatsukiG5/Documents/mavenRepo/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/force-shading/1.12.1/force-shading-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-table-api-java-bridge_2.11/1.12.1/flink-table-api-java-bridge_2.11-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-java/1.12.1/flink-java-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/commons/commons-math3/3.5/commons-math3-3.5.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-streaming-java_2.11/1.12.1/flink-streaming-java_2.11-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-file-sink-common/1.12.1/flink-file-sink-common-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-shaded-guava/18.0-12.0/flink-shaded-guava-18.0-12.0.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-streaming-scala_2.11/1.12.1/flink-streaming-scala_2.11-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/apache/flink/flink-scala_2.11/1.12.1/flink-scala_2.11-1.12.1.jar:/home/AkatsukiG5/Documents/mavenRepo/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar:/home/AkatsukiG5/Documents/mavenRepo/org/scala-lang/scala-library/2.11.12/scala-library-2.11.12.jar:/home/AkatsukiG5/Documents/mavenRepo/org/scala-lang/scala-compiler/2.11.12/scala-compiler-2.11.12.jar:/home/AkatsukiG5/Documents/mavenRepo/org/scala-la

[jira] [Created] (FLINK-23008) Upload file not exist when submit job use RestClient

[YufeiLiu (Jira)]

YufeiLiu created FLINK-23008:


 Summary: Upload file not exist when submit job use RestClient
 Key: FLINK-23008
 URL: https://issues.apache.org/jira/browse/FLINK-23008
 Project: Flink
  Issue Type: Bug
  Components: Runtime / REST
Affects Versions: 1.12.3
Reporter: YufeiLiu


Upload jar file using {{PipelineOptions.JARS}} or {{JobGraph.addJar}}, client 
will wrapped files as a request and post to server. If some file doesn't exist, 
the request will just interrupt and no exception. Client keep waiting response 
and server doesn't receive request, until trigger channel idle check and retry.
I think we should add a file exist check before submit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-23007) Introduce java code splitter to split generated code

[Jingsong Lee (Jira)]

Jingsong Lee created FLINK-23007:


 Summary: Introduce java code splitter to split generated code
 Key: FLINK-23007
 URL: https://issues.apache.org/jira/browse/FLINK-23007
 Project: Flink
  Issue Type: New Feature
  Components: Table SQL / Runtime
Reporter: Jingsong Lee
 Fix For: 1.14.0


Current lots of "code of method of class grows beyond 64k".

We have fixed by:

https://issues.apache.org/jira/browse/FLINK-15430

https://issues.apache.org/jira/browse/FLINK-16589

But this approach is not perfect. It has many corner cases.

 

We can introduce a post java class splitter, based on antlr4, anaysis code and 
split method to multi-methods.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)