Congratulations!
Best regards,
Xuannan
On Thu, Jun 6, 2024 at 9:53 AM Hangxiang Yu wrote:
>
> Congratulations, Rui !
>
> On Thu, Jun 6, 2024 at 9:18 AM Lincoln Lee wrote:
>
> > Congratulations, Rui!
> >
> > Best,
> > Lincoln Lee
> >
> >
> > Lijie Wang 于2024年6月6日周四 09:11写道：
> >
> > >
Congratulations!
Best regards,
Xuannan
On Thu, Jun 6, 2024 at 9:20 AM Jiabao Sun wrote:
>
> Congratulations, Weijie!
>
> Best,
> Jiabao
>
> Lincoln Lee 于2024年6月6日周四 09:17写道：
>
> > Congratulations, Weijie!
> >
> > Best,
> > Lincoln Lee
> >
> >
> > ConradJam 于2024年6月5日周三 10:46写道：
> >
> > >
Xuannan Su created FLINK-35475:
--
Summary: Introduce isInternalSorterSupport to OperatorAttributes
Key: FLINK-35475
URL: https://issues.apache.org/jira/browse/FLINK-35475
Project: Flink
Issue
+1 (non-binding)
Best regards,
Xuannan
On Mon, May 27, 2024 at 3:43 PM Jark Wu wrote:
>
> +1 (binding)
>
> Best,
> Jark
>
> On Mon, 27 May 2024 at 14:29, Hang Ruan wrote:
>
> > +1 (non-binding)
> >
> > Best,
> > Hang
> >
> > gongzhongqiang 于2024年5月27日周一 14:16写道：
> >
> > > +1 (non-binding)
> >
Xuannan Su created FLINK-35461:
--
Summary: Improve Runtime Configuration for Flink 2.0
Key: FLINK-35461
URL: https://issues.apache.org/jira/browse/FLINK-35461
Project: Flink
Issue Type
Hi devs,
I'm happy to announce that FLIP-450: Improve Runtime Configuration for
Flink 2.0[1] has been accepted with 3 approving
votes (3 bindings) [2]:
- Rui Fan (binding)
- Weijie Guo (binding)
- Xintong Song (binding)
There are no disapproving votes.
Thanks again to everyone who participated
guo
> wrote:
>
> > +1(binding)
> >
> > Best regards,
> >
> > Weijie
> >
> >
> > Rui Fan <1996fan...@gmail.com> 于2024年5月15日周三 17:50写道：
> >
> > > +1(binding)
> > >
> > > Best,
> > > Rui
> > >
> >
they are going to be removed for v2, it would be good to
> explicitly document the thinking and impact of the deprecations and
> consequent removals,
> Kind regards, David.
>
> From: Xuannan Su
> Date: Tuesday, 14 May 2024 at 02:08
> To: dev@flink.apache.org
> Su
Hi Jane,
Thanks for driving this effort! And +1 for the proposed changes.
I have one comment on the migration plan.
For options to be moved to another module/package, I think we have to
mark the old option deprecated in 1.20 for it to be removed in 2.0,
according to the API compatibility
Xuannan Su created FLINK-35359:
--
Summary: General Improvement to Configuration for Flink 2.0
Key: FLINK-35359
URL: https://issues.apache.org/jira/browse/FLINK-35359
Project: Flink
Issue Type
Hi everyone,
Thanks for all the feedback about the FLIP-450: Improve Runtime
Configuration for Flink 2.0 [1] [2].
I'd like to start a vote for it. The vote will be open for at least 72
hours(excluding weekends,until MAY 20, 12:00AM GMT) unless there is an
objection or an insufficient number of
te. The FLIP looks good to me. +1 for it.
>
> Regards,
> Jeyhun
>
> On Mon, May 13, 2024 at 4:45 AM Xuannan Su wrote:
>
> > Hi Jeyhun,
> >
> > Thanks for the comment!
> >
> > Yes, we intended to remove the StreamPiplineOptions in 2.0. I updated
&
, +1 for this proposal.
> >
> > Best,
> > Rui
> >
> > On Sat, May 11, 2024 at 10:20 AM Xuannan Su wrote:
> >
> > > Hi Rui,
> > >
> > > Thanks for the suggestion!
> > >
> > > I updated the description of
> > &g
and legacy
> > > hybrid shuffle mode, and the behavior change of overdraft network
> > buffers.
> > > Therefore, it might be nicer to provide an implementation plan with a
> > list
> > > of related tasks in the FLIP. This should not block the FLIP though.
, it might be nicer to provide an implementation plan with a list
> of related tasks in the FLIP. This should not block the FLIP though.
>
> Best,
>
> Xintong
>
>
>
> On Thu, Apr 25, 2024 at 4:35 PM Xuannan Su wrote:
>
> > Hi all,
> >
> > I'd like to star
Hi all,
I'd like to start a discussion on FLIP-450: Improve Runtime
Configuration for Flink 2.0 [1]. As Flink moves toward 2.0, we have
revisited all runtime configurations and identified several
improvements to enhance user-friendliness and maintainability. In this
FLIP, we aim to refine the
Hi devs,
I'm happy to announce that FLIP-442: General Improvement to
Configuration for Flink 2.0[1] has been accepted with 9 approving
votes (4 bindings) [2]:
- Rui Fan (binding)
- Zakelly Lan (binding)
- Xintong Song (binding)
- Yuxin Tan (non-binding)
- Zhu Zhu (binding)
- Jeyhun Karimov
Hi all,
Thank you all! Closing the vote. The result will be announced in a
separate email.
Best regards,
Xuannan
On Fri, Apr 19, 2024 at 10:58 AM gongzhongqiang
wrote:
>
> +1 (non-binding)
>
>
> Best,
> Zhongqiang Gong
>
> Xuannan Su 于2024年4月17日周三 13:02写道：
>
>
Hi everyone,
Thanks for all the feedback about the FLIP-442: General Improvement to
Configuration for Flink 2.0 [1] [2].
I'd like to start a vote for it. The vote will be open for at least 72
hours(excluding weekends,until APR 22, 12:00AM GMT) unless there is an
objection or an insufficient
Congratulations Zakelly!
Best regards,
Xuannan
On Mon, Apr 15, 2024 at 4:31 PM Jing Ge wrote:
>
> Congratulations Zakelly!
>
> Best regards,
> Jing
>
> On Mon, Apr 15, 2024 at 4:26 PM Xia Sun wrote:
>
> > Congratulations Zakelly!
> >
> > Best,
> > Xia
> >
> > Leonard Xu 于2024年4月15日周一
Congratulations, Lincoln!
Best regards,
Xuannan
On Tue, Apr 16, 2024 at 10:04 AM Hang Ruan wrote:
>
> Congratulations, Lincoln!
>
> Best,
> Hang
>
> yh z 于2024年4月16日周二 09:14写道：
>
> > Congratulations, Lincoln!
> >
> > Best,
> > Yunhong (Swuferhong)
> >
> >
> > Swapnal Varma 于2024年4月15日周一
gt; Regarding the new classes, we propose updating the
> > `ConfigOptionsDocGenerator` to throw an exception and fail the build
> > if it detects any new class missing the proper annotation in the
> > future.
>
> Thanks for your feedback, it sounds good to me.
>
> Best,
&
Xuannan Su created FLINK-35089:
--
Summary: Two input AbstractStreamOperator may throw NPE when
receiving RecordAttributes
Key: FLINK-35089
URL: https://issues.apache.org/jira/browse/FLINK-35089
Project
> > We will relocate these ConfigOptions to a class that is included
> > in the documentation generation.
>
> Would it make sense to define also in the FLIP the options class for
> these variables? For example, GPUDriverOptions?
>
> Best,
> Muhammet
>
> On 2024-04
dy annotated
> with @PublicEvolving.
>
>
> Best,
> Zakelly
>
>
> On Tue, Apr 9, 2024 at 4:21 PM Xuannan Su wrote:
>
> > Hi all,
> >
> > I'd like to start a discussion on FLIP-442: General Improvement to
> > Configuration for Flink 2.0 [1]. As Flink moves t
Hi all,
I'd like to start a discussion on FLIP-442: General Improvement to
Configuration for Flink 2.0 [1]. As Flink moves toward 2.0, we aim to
provide users with a better experience with the existing
configuration. This FLIP proposes several general improvements to the
current configuration.
+1 (non-binding)
Best regards,
Xuannan
On Wed, Mar 27, 2024 at 6:28 PM Yanfei Lei wrote:
>
> Hi everyone,
>
> Thanks for all the feedback about the FLIP-425: Asynchronous Execution
> Model [1]. The discussion thread is here [2].
>
> The vote will be open for at least 72 hours unless there is an
+1 (non-binding)
Best,
Xuannan
On Wed, Mar 27, 2024 at 6:23 PM Zakelly Lan wrote:
>
> Hi devs,
>
> I'd like to start a vote on the FLIP-424: Asynchronous State APIs [1]. The
> discussion thread is here [2].
>
> The vote will be open for at least 72 hours unless there is an objection or
>
Congratulations!
Best regards,
Xuannan
On Fri, Mar 22, 2024 at 9:17 AM Charles Zhang wrote:
>
> Congratulations!
>
> Best wishes,
> Charles Zhang
> from Apache InLong
>
>
> Jeyhun Karimov 于2024年3月22日周五 04:16写道：
>
> > Great news! Congratulations!
> >
> > Regards,
> > Jeyhun
> >
> > On Thu, Mar
Congratulations! Thanks for all the great work!
Best regards,
Xuannan
On Tue, Mar 19, 2024 at 1:31 PM Yu Li wrote:
>
> Congrats and thanks all for the efforts!
>
> Best Regards,
> Yu
>
> On Tue, 19 Mar 2024 at 11:51, gongzhongqiang
> wrote:
> >
> > Congrats! Thanks to everyone involved!
> >
>
+1 (non-binding)
- Verified signature and checksum
- Verified that source distribution does not contain binaries
- Built from source code successfully
- Reviewed the release announcement PR
Best regards,
Xuannan
On Tue, Mar 12, 2024 at 2:18 PM Hang Ruan wrote:
>
> +1 (non-binding)
>
> -
Hi Zakelly,
Thanks for the quick response.
> It will be used in callback chaining cases where some branch within one
> callback does nothing. I'm in favor of short phrases to express the
> functionalities. Thus I suggest `completedVoidFuture` or `voidFuture`, WDTY?
I'd prefer
Hi Zakelly,
Thank you for the proposal! Please find my comments below.
1. The name `emptyFuture` seems a little unintuitive, and it is hard
to understand in what use case the `emptyFuture` should be used. If I
understand correctly, it is similar to the
FutureUtils#completedVoidFuture. How about
+1 (non-binding)
Best,
Xuannan
On Tue, Feb 27, 2024 at 9:37 AM Xintong Song wrote:
>
> +1 (binding)
>
> Best,
>
> Xintong
>
>
>
> On Mon, Feb 26, 2024 at 6:10 PM weijie guo
> wrote:
>
> > Hi everyone,
> >
> >
> > Thanks for all the feedback about the FLIP-410: Config, Context and
> >
+1 (non-binding)
Best,
Xuannan
On Tue, Feb 27, 2024 at 9:38 AM Xintong Song wrote:
>
> +1 (binding)
>
> Best,
>
> Xintong
>
>
>
> On Mon, Feb 26, 2024 at 6:09 PM weijie guo
> wrote:
>
> > Hi everyone,
> >
> >
> > Thanks for all the feedback about the FLIP-409: DataStream V2 Building
> >
+1 (non-binding)
Best,
Xuannan
On Tue, Feb 27, 2024 at 9:36 AM Xintong Song wrote:
>
> +1 (binding)
>
> Best,
>
> Xintong
>
>
>
> On Mon, Feb 26, 2024 at 6:08 PM weijie guo
> wrote:
>
> > Hi everyone,
> >
> >
> > Thanks for all the feedback about the FLIP-408: [Umbrella] Introduce
> >
Hi devs,
I'm happy to announce that FLIP-331: Support EndOfStreamTrigger and
isOutputOnlyAfterEndOfStream operator attribute to optimize task
deployment [1] has been accepted with 6 approving votes (4 binding)
[2]:
- Xintong Song (binding)
- Rui Fan (binding)
- Weijie Guo (binding)
- Dong Lin
n-binding)
> >
> > Best,
> > Hang
> >
> > Dong Lin 于2024年2月5日周一 11:08写道：
> >
> > > Thanks for the FLIP.
> > >
> > > +1 (binding)
> > >
> > > Best,
> > > Dong
> > >
> > > On Wed, Jan 3
oduce some concepts like ConnectedStreams. But we hope that streams
> > > > will be more clearly defined in the DataStream API, otherwise we will
> > end
> > > > up going the same way as the original API, which you have to understand
> > > > `JoinedS
Hi everyone,
Thanks for all the feedback about the FLIP-331: Support
EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute
to optimize task deployment [1] [2].
I'd like to start a vote for it. The vote will be open for at least 72
hours(excluding weekends,until Feb 5, 12:00AM
Hi all,
Thanks for the comments and suggestions. If there are no further
comments, I will open the voting thread tomorrow.
Best regards,
Xuannan
On Fri, Jan 26, 2024 at 3:16 PM Dong Lin wrote:
>
> Thanks Xuannan for the update!
>
> +1 (binding)
>
> On Wed, Jan 10, 2024 at
Hi Weijie,
Thanks for the FLIP! I have a few questions regarding the FLIP.
1. +1 to only use XXXParititionStream if users only need to use the
configurable PartitionStream. If there are use cases for both,
perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or
Hi Weijie,
Thank you for driving the design of the new DataStream API. I have a
few questions regarding the FLIP:
1. In the partitioning section, it says that "broadcast can only be
used as a side-input of other Inputs." Could you clarify what is meant
by "side-input"? If I understand correctly,
Hi Weijie,
Thanks for driving the work! There are indeed many pain points in the
current DataStream API, which are challenging to resolve with its
existing design. It is a great opportunity to propose a new DataStream
API that tackles these issues. I like the way we've divided the FLIP
into
+1 (non-binding)
Best,
Xuannan
On Thu, Jan 25, 2024 at 10:15 AM Lijie Wang wrote:
>
> +1 (binding)
>
> Best,
> Lijie
>
> Yanfei Lei 于2024年1月25日周四 10:06写道：
>
> > +1 (binding)
> >
> > Hangxiang Yu 于2024年1月25日周四 10:00写道：
> > >
> > > +1 (binding)
> > >
> > > On Thu, Jan 25, 2024 at 8:49 AM Rui
Xuannan Su created FLINK-34085:
--
Summary: Remove deprecated string configuration keys in Flink 2.0
Key: FLINK-34085
URL: https://issues.apache.org/jira/browse/FLINK-34085
Project: Flink
Issue
Xuannan Su created FLINK-34084:
--
Summary: Deprecate unused configuration in
BinaryInput/OutputFormat and FileInput/OutputFormat
Key: FLINK-34084
URL: https://issues.apache.org/jira/browse/FLINK-34084
Xuannan Su created FLINK-34083:
--
Summary: Deprecate string configuration keys and unused constants
in ConfigConstants
Key: FLINK-34083
URL: https://issues.apache.org/jira/browse/FLINK-34083
Project
Hi devs,
I'm happy to announce that FLIP-405: Migrate string configuration key
to ConfigOption [1] has been accepted with 4 approving votes (3
binding) [2]:
- Rui Fan (binding)
- Hang Ruan (non-binding)
- Xintong Song (binding)
- Zhu Zhu (binding)
There are no disapproving votes.
Thanks again
Hi all,
Thank you all! Closing the vote. The result will be announced in a
separate email.
Best regards,
Xuannan
On Fri, Jan 12, 2024 at 2:44 PM Zhu Zhu wrote:
>
> +1 (binding)
>
> Thanks,
> Zhu
>
> Xuannan Su 于2024年1月12日周五 14:24写道：
>
> > Hi all,
> >
>
Hi Zakelly,
Thanks for driving this. +1 to removing the LEGACY mode.
Best regards,
Xuannan
On Mon, Jan 15, 2024 at 3:22 AM Danny Cranmer wrote:
>
> +1 to removing LEGACY mode in Flink 2.0. Thanks for driving.
>
> Danny,
>
> On Sat, 13 Jan 2024, 08:20 Yanfei Lei, wrote:
>
> > Thanks Zakelly
, 2024 at 1:36 PM Xuannan Su wrote:
>
> Hi all,
>
> During voting, we identified two improvements we'd like to make to the FLIP:
>
> - We will mark the getBytes(String key, byte[] defaultValue) and
> setBytes(String key, byte[] bytes) methods as @Internal, as they are
> intend
n wrote:
>
> > +1(non-binding)
> >
> > Best,
> > Hang
> >
> > Rui Fan <1996fan...@gmail.com> 于2024年1月8日周一 13:04写道：
> >
> > > +1(binding)
> > >
> > > Best,
> > > Rui
> > >
> > > On Mon, Jan 8, 2024
+1 (non-binding)
Best,
Xuannan
On Thu, Jan 11, 2024 at 10:28 AM Xuyang wrote:
>
> +1 (non-binding)--
>
> Best！
> Xuyang
>
>
>
>
>
> 在 2024-01-11 10:00:11，"Yang Wang" 写道：
> >+1 (binding)
> >
> >
> >Best,
> >Yang
> >
> >On Thu, Jan 11, 2024 at 9:53 AM liu ron wrote:
> >
> >> +1
.backend.incremental. It means we use
> > >> > > > > > `execution.checkpointing.rocksdb-incremental` or
> > >> > > > > > `execution.checkpointing.rocksdb-incremental.enabled`.
> > >> > > > > >
> > >> > > > > &g
Hi all,
After several rounds of offline discussions with Xingtong and Jinhao,
we have decided to narrow the scope of the FLIP. It will now focus on
introducing OperatorAttributes that indicate whether an operator emits
records only after inputs have ended. We will also use the attribute
to
s good. Is the requirement to send an email thread about the voting?
> What else is needed? What's the passing criteria?
>
> Best
> Lu
>
> On Sun, Jan 7, 2024 at 5:41 PM Xuannan Su wrote:
>>
>> Hi Liu,
>>
>> The voting thread has been open for a long time. W
Hi everyone,
Thanks for all the feedback about the FLIP-405: Migrate string
configuration key to ConfigOption [1] [2].
I'd like to start a vote for it. The vote will be open for at least 72
hours(excluding weekends,until Jan 11, 12:00AM GMT) unless there is an
objection or an insufficient number
Hi all,
Thanks for the discussion. I think all the comments and questions have
been addressed. I will open the voting thread today.
Best,
Xuannan
On Tue, Jan 2, 2024 at 11:59 AM Xuannan Su wrote:
>
> Hi all,
>
> Thank you for all your comments! The FLIP has been updated
> acco
gt; appreciated! I realize there is already a voting thread "[VOTE] FLIP-329: Add
> operator attribute to specify support for object-reuse". What else do we need?
>
> Best
> Lu
>
> On Fri, Jan 5, 2024 at 12:46 AM Xuannan Su wrote:
>>
>> Hi Lu,
>>
>&g
>> I was interested in getting clarification as to what you meant by “or
>>> access references…”, to see if it covered this situation:
>>>
>>> StreamX —forward--> operator1
>>> StreamX —forward--> operator2
>>>
>>> If operator1 mod
Hang Ruan wrote:
>>
>> Hi, Rui Fan.
>>
>> Thanks for this FLIP.
>>
>> I think the key of LOCAL_NUMBER_TASK_MANAGER is better as
>> 'minicluster.number-of-taskmanagers' or 'minicluster.taskmanager-number'
>> instead of 'minicluster.number-taskmanager'.
>>
>&
Hi Zakelly,
Thanks for driving this! The organization of the configuration option
in the FLIP looks much cleaner and easier to understand. +1 to the
FLIP.
Just some questions from me.
1. I think the change to the ConfigOptions should be put in the
`Public Interface` section, instead of
Hi Xintong and Rui,
Thanks for the quick feedback and the suggestions.
> 1. I think the default value for `TASK_MANAGER_LOG_PATH_KEY` should be "no
> default".
I have considered both ways of describing the default value. However,
I found out that some of the configurations, such as
erialized to a json string, and we can only write/read them at once.
> The CACHE_FILE_BLOB_KEY is special, its type is byte[], we need to
> store it by the setBytes/getBytes.
>
> Also, I have an offline discussion
Xuannan Su created FLINK-33890:
--
Summary: Determine the initial status before receiving the first
RecordAttributes
Key: FLINK-33890
URL: https://issues.apache.org/jira/browse/FLINK-33890
Project: Flink
Thanks for driving this!
+1(non-binding)
Best,
Xuannan
On Thu, Dec 14, 2023 at 11:04 AM Xintong Song wrote:
>
> +1 (binding)
>
> Best,
>
> Xintong
>
>
>
> On Thu, Dec 14, 2023 at 10:05 AM Jiabao Sun
> wrote:
>
> > Thanks Matthias for this hard work!
> >
> > +1(non-binding)
> >
> > Best,
> >
Hi Rui,
We are currently revisiting all the configurations for Flink 2.0, and
it turns out that many string-based configurations in
`ConfigConstants` are deprecated and have been replaced by
`ConfigOptions`. Since `ConfigOptions` offers many advantages over
string-based configurations for the end
Xuannan Su created FLINK-33810:
--
Summary: Propagate RecordAttributes that contains
isProcessingBacklog status
Key: FLINK-33810
URL: https://issues.apache.org/jira/browse/FLINK-33810
Project: Flink
Xuannan Su created FLINK-33399:
--
Summary: Support switching from batch to stream mode for
KeyedCoProcessOperator and IntervalJoinOperator
Key: FLINK-33399
URL: https://issues.apache.org/jira/browse/FLINK-33399
Xuannan Su created FLINK-33398:
--
Summary: Support switching from batch to stream mode for one input
stream operator
Key: FLINK-33398
URL: https://issues.apache.org/jira/browse/FLINK-33398
Project: Flink
Hi all,
There is still a discussion ongoing. I will cancel the vote for now.
Best Regards,
Xuannan
On Thu, Oct 19, 2023 at 2:08 PM Dong Lin wrote:
>
> Thanks for the FLIP!
>
> +1 (binding)
>
> On Wed, Oct 18, 2023 at 10:25 AM Xuannan Su wrote:
>
> > Hi all,
>
Hi all,
We would like to start the vote for FLIP-329: Add operator attribute
to specify support for object-reuse[1]. This FLIP was discussed in
this thread [2].
The vote will be open until at least Oct 22nd (at least 72 hours),
following the consensus voting process.
Cheers,
Xuannan
[1]
Hi all,
We would like to start the vote for FLIP-328: Allow source operators
to determine isProcessingBacklog based on watermark lag [1]. This FLIP
was discussed in this thread [2].
The vote will be open until at least Oct 21st (at least 72 hours),
following the consensus voting process.
Hi Martijn,
Do you have further comments regarding the FLIP? If not, I'd like to
move forward and start the voting in two days.
Best regards,
Xuannan
On Sat, Oct 7, 2023 at 3:02 PM Xuannan Su wrote:
>
> Hi Martijn,
>
> Sorry for the late reply. I don't think it is feasible to alw
Congratulations Jane!
Best,
Xuannan
On Mon, Oct 16, 2023 at 10:21 AM Yun Tang wrote:
>
> Congratulations, Jane!
>
> Best
> Yun Tang
>
> From: Rui Fan <1996fan...@gmail.com>
> Sent: Monday, October 16, 2023 10:16
> To: dev@flink.apache.org
> Cc:
compatibility.
>
> Best regards,
>
> Martijn
>
> On Tue, Sep 26, 2023 at 1:32 AM Xuannan Su wrote:
> >
> > Hi all,
> >
> > We would like to revive the discussion and provide a quick update on
> > the recent work of the FLIP. We have implemented a POC[1]
Xuannan Su created FLINK-33202:
--
Summary: FLIP-327: Support switching from batch to stream mode to
improve throughput when processing backlog data
Key: FLINK-33202
URL: https://issues.apache.org/jira/browse/FLINK
operator in the chain.*
> >>>>>
> >>>>>
> >>>>> If I understand you correctly, the hard coding objectReusedCompliant
> >>>>> should have higher priority over the configuration, the checking logic
> >>>>> shoul
sure if it's enough to support this feature only in FLIP-27 Source.
> Although we are pushing the sourceFunction API to be removed, these APIs will
> be survive one or two versions in flink repo before they are actually removed.
>
> Best,
> Leonard
>
> >
> > Best
>
> I thought FLIP-328 will compete with FLIP-309 while setting the value of
> the backlog. Understood. Thanks for the hint.
>
> Best regards,
> Jing
>
> On Wed, Sep 6, 2023 at 12:12 PM Xuannan Su wrote:
>
> > Hi Jing,
> >
> > Thank you for the clari
ng mode. All data before that range will
> be considered as backlog and needed to be processed in the batch mode,
> like, e.g. the Present Perfect Progressive tense used in English language.
>
> Best regards,
> Jing
>
> On Thu, Aug 31, 2023 at 4:45 AM Xuannan Su wrote:
>
> >
report isProcessingBacklog: 1.
> From the source; 2. Judged by the watermark lag. What is the priority
> between them?
> For example, what is the status isProcessingBacklog when the source report
> `isProcessingBacklog=false` and the watermark lag exceeds the threshold?
>
> Best,
>
Hi all,
I would like to share some updates on FLIP-327. Dong and I have had a
series of discussions and have made several refinements to the FLIP.
The major change to the FLIP is to allow the input of the one-input
operator to be automatically sorted during backlog processing. When
combined with
g
>
> [1]
> https://github.com/apache/flink/blob/2c50b4e956305426f478b726d4de4a640a16b810/flink-core/src/main/java/org/apache/flink/api/common/eventtime/WatermarkStrategy.java#L236
>
> On Wed, Aug 30, 2023 at 4:06 AM Xuannan Su wrote:
>
>> Hi Jing,
>>
>> Than
ulated exactly? E.g. Watermark
> lag = A - B with A is the timestamp of the watermark emitted to the
> downstream and B is(this is the part I am not really sure after reading
> the FLIP).
>
> Best regards,
> Jing
>
>
> On Mon, Aug 21, 2023 at 9:03 AM Xuannan Su wrote
>
> On Tue, 15 Aug 2023 at 12:04, Xintong Song <mailto:tonysong...@gmail.com>> wrote:
>
>> Sounds good to me.
>>
>> It is true that, if we are introducing the generalized watermark, there
>> will be other watermark related concepts / configurations
hanging it later, if needed, with
> a regular deprecation and migration process. Just making my suggestions.
>
>
> Best,
>
> Xintong
>
>
>
> On Mon, Aug 14, 2023 at 12:00 PM Xuannan Su wrote:
>
> > Hi Xintong,
> >
> > Thanks for the reply.
> &
n the watermark for
> emitting the record attribute? Why not use timestamps in the records? I
> don't see any concern in using watermarks. Just wondering if there's any
> deep considerations behind this.
>
> Best,
>
> Xintong
>
>
>
> On Thu, Aug 3, 2023 at 3:03 PM Xuann
Hi all,
I am opening this thread to discuss FLIP-328: Allow source operators to
determine isProcessingBacklog based on watermark lag[1]. We had a several
discussions with Dong Ling about the design, and thanks for all the valuable
advice.
The FLIP aims to target the use-case where user want
bject
> reusing; false: force deep-copying; unknown: depends on
> pipeline.object-reuse config.
>
>
> Best regards,
> Jing
>
>
> [1] https://en.wikipedia.org/wiki/JavaBeans
>
> On Mon, Jul 3, 2023 at 4:25 AM Xuannan Su wrote:
>
> > Hi all,
> &g
Hi all,
Dong(cc'ed) and I are opening this thread to discuss our proposal to
add operator attribute to allow operator to specify support for
object-reuse [1].
Currently, the default configuration for pipeline.object-reuse is set
to false to avoid data corruption, which can result in suboptimal
Xuannan Su created FLINK-32476:
--
Summary: Support configuring object-reuse for internal operators
Key: FLINK-32476
URL: https://issues.apache.org/jira/browse/FLINK-32476
Project: Flink
Issue
Hi all,
Dong(cc'ed) and I are opening this thread to discuss our proposal to
enhance the watermark to properly support processing-time temporal
join, which has been documented in FLIP-326 [1].
We want to support the use case where the records from the probe side
of the processing-time temporal
Xuannan Su created FLINK-32008:
--
Summary: Protobuf format throws exception with Map datatype
Key: FLINK-32008
URL: https://issues.apache.org/jira/browse/FLINK-32008
Project: Flink
Issue Type
Xuannan Su created FLINK-31944:
--
Summary: Protobuf format throw
com.google.protobuf.InvalidProtocolBufferException
Key: FLINK-31944
URL: https://issues.apache.org/jira/browse/FLINK-31944
Project: Flink
Xuannan Su created FLINK-31943:
--
Summary: Multiple t_env cause class loading problem
Key: FLINK-31943
URL: https://issues.apache.org/jira/browse/FLINK-31943
Project: Flink
Issue Type: Bug
Xuannan Su created FLINK-30607:
--
Summary: Table.to_pandas doesn't support Map type
Key: FLINK-30607
URL: https://issues.apache.org/jira/browse/FLINK-30607
Project: Flink
Issue Type: Bug
Xuannan Su created FLINK-30258:
--
Summary: PyFlink supports closing loop back server
Key: FLINK-30258
URL: https://issues.apache.org/jira/browse/FLINK-30258
Project: Flink
Issue Type
Xuannan Su created FLINK-30078:
--
Summary: Temporal join should finish when the left table is
bounded and finished
Key: FLINK-30078
URL: https://issues.apache.org/jira/browse/FLINK-30078
Project: Flink
1 - 100 of 175 matches
Mail list logo
