Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

2024-07-26 Thread Yu Chen
Hi Junrui, Thanks for driving this. It’s really helpful for user to explore the job they submitted. It's an important cornerstone of the FLINK-33230 since there are no stream-graph informations serialized. I'd like to base on this flip to expose operator level metrics and the presentation of

Re: [DISCUSS] FLIP-XXX Amazon SQS Source Connector

2024-07-26 Thread Saurabh Singh
Hi Li Wang, Thanks for the review and appreciate your feedback. I completely understand your concern and agree with it. Our goal is to provide users of connectors with a consistent and coherent ecosystem, free of any issues. To ensure that, we are closely monitoring/reviewing the SQS Sink implemen

Re: [DISCUSS] FLIP-471: Fixing watermark idleness timeout accounting

2024-07-26 Thread Arvid Heise
Hi Piotr, > > 1. `RelativeClock` looks like a super-interface of > > `org.apache.flink.util.clock.Clock`. Should we also reflect that > > accordingly by extending? This should not break anything. > > It should be fine to do so, but to me the question is if we should do so? > It doesn't give any be

Re: [DISCUSS] FLIP-XXX Amazon SQS Source Connector

2024-07-26 Thread Li Wang
Hi Saurabh, Thanks for the FLIP. Given the ongoing effort to implement the SQS sink connector ( https://cwiki.apache.org/confluence/display/FLINK/FLIP-438%3A+Amazon+SQS+Sink+Connector), it is important to consider how the SQS source connector supports this development, ensuring a unified framework

Re: [DISCUSS] FLIP-471: Fixing watermark idleness timeout accounting

2024-07-26 Thread Piotr Nowojski
Hi Zakelly, > Could we just use `org.apache.flink.util.clock.Clock` instead of introducing a > new one? We can not use it as is, because of this java doc in it: * Relative Time * * This time advances at the same speed as the absolute time, but the timestamps can only * be referred to relativ

[jira] [Created] (FLINK-35904) Test harness for async state processing operators

2024-07-26 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-35904: --- Summary: Test harness for async state processing operators Key: FLINK-35904 URL: https://issues.apache.org/jira/browse/FLINK-35904 Project: Flink Issue Type: S

Re: [DISCUSS] FLIP-471: Fixing watermark idleness timeout accounting

2024-07-26 Thread Zakelly Lan
Hi Piotr, I'd +1 for this idea. I partially share the same concern with Arvid. Could we just use `org.apache.flink.util.clock.Clock` instead of introducing a new one? Thus only a new implementation is required IIUC. My thinking is that clock is a general need for timing. How to generate time or wh

Re: [DISCUSS] FLIP-XXX: Introduce CLI API compatibility guarantees

2024-07-26 Thread Ferenc Csaky
Hi, I prepared an initial FLIP with the already covered details, PTAL [1]. Best, Ferenc [1] https://docs.google.com/document/d/1heInCDO7WrVKRVDfRzRzrMF9ljS3Tt_JH5TLdKwHLvU/edit?usp=sharing On Wednesday, 24 July 2024 at 03:39, Xintong Song wrote: > > > Sounds good to me. Thank you, Ferenc

Re: [VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission

2024-07-26 Thread Zhanghao Chen
Thanks for driving it. +1 (non-binding) Best, Zhanghao Chen From: Junrui Lee Sent: Friday, July 26, 2024 11:02 To: dev@flink.apache.org Subject: [VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission Hi everyone, Thanks for all the feedback about FLIP-46

Re: [DISCUSS] FLIP-469: Supports Adaptive Optimization of StreamGraph

2024-07-26 Thread Junrui Lee
Hi all, Thank you for all the feedback and suggestions so far. If there are no further comments, we will open the voting thread next Monday, July 29, 2024. Best regards, Junrui Ron Liu 于2024年7月26日周五 14:17写道: > Thanks for your updated, the FLIP looks good to me. > > Junrui Lee 于2024年7月26日周五 1