Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)
Congratulations! On Wed, Mar 29, 2023 at 4:11 AM Zhu Zhu wrote: > Congratulations! > > Cheers, > Zhu > > Jing Zhang 于2023年3月28日周二 23:56写道： > > > > Congratulations! > > > > Best, > > Jing Zhang > > > > Rui Fan <1996fan...@gmail.com> 于2023年3月28日周二 22:25写道： > > > > > Congratulations! > > > > > > Best, > > > Rui Fan > > > > > > On Tue, Mar 28, 2023 at 15:37 Guowei Ma wrote: > > > > > > > Congratulations! > > > > > > > > Best, > > > > Guowei > > > > > > > > > > > > On Tue, Mar 28, 2023 at 12:02 PM Yuxin Tan > > > wrote: > > > > > > > >> Congratulations！ > > > >> > > > >> Best, > > > >> Yuxin > > > >> > > > >> > > > >> Guanghui Zhang 于2023年3月28日周二 11:06写道： > > > >> > > > >>> Congratulations！ > > > >>> > > > >>> Best, > > > >>> Zhang Guanghui > > > >>> > > > >>> Hang Ruan 于2023年3月28日周二 10:29写道： > > > >>> > > > >>> > Congratulations! > > > >>> > > > > >>> > Best, > > > >>> > Hang > > > >>> > > > > >>> > yu zelin 于2023年3月28日周二 10:27写道： > > > >>> > > > > >>> >> Congratulations! > > > >>> >> > > > >>> >> Best, > > > >>> >> Yu Zelin > > > >>> >> > > > >>> >> 2023年3月27日 17:23，Yu Li 写道： > > > >>> >> > > > >>> >> Dear Flinkers, > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> As you may have noticed, we are pleased to announce that Flink > Table > > > >>> Store has joined the Apache Incubator as a separate project called > > > Apache > > > >>> Paimon(incubating) [1] [2] [3]. The new project still aims at > building > > > a > > > >>> streaming data lake platform for high-speed data ingestion, change > data > > > >>> tracking and efficient real-time analytics, with the vision of > > > supporting a > > > >>> larger ecosystem and establishing a vibrant and neutral open source > > > >>> community. > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> We would like to thank everyone for their great support and > efforts > > > >>> for the Flink Table Store project, and warmly welcome everyone to > join > > > the > > > >>> development and activities of the new project. Apache Flink will > > > continue > > > >>> to be one of the first-class citizens supported by Paimon, and we > > > believe > > > >>> that the Flink and Paimon communities will maintain close > cooperation. > > > >>> >> > > > >>> >> > > > >>> >> 亲爱的Flinkers， > > > >>> >> > > > >>> >> > > > >>> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache > > > >>> >> 孵化器独立孵化 [1] [2] [3]。新项目的名字是 > > > >>> >> Apache > > > >>> > > > > Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。 > > > >>> >> > > > >>> >> > > > >>> >> 在这里我们要感谢大家对 Flink Table Store > > > >>> >> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon > > > >>> 支持的主力计算引擎之一，我们也相信 > > > >>> >> Flink 和 Paimon 社区将继续保持密切合作。 > > > >>> >> > > > >>> >> > > > >>> >> Best Regards, > > > >>> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) > > > >>> >> > > > >>> >> 致礼， > > > >>> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） > > > >>> >> > > > >>> >> [1] https://paimon.apache.org/ > > > >>> >> [2] https://github.com/apache/incubator-paimon > > > >>> >> [3] > > > >>> > https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> > > > >> > > > > -- Best regards, Sergey
Re: [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs
Thanks Jane driving this flip! And very glad to see the lively discussion which has inspired me a lot! I'd like to share some thoughts: as many of you mentioned, FLIP-113[1] formally introduced the Calcite's hints syntax and supported the dynamic table option(i.e., table hints), and then FLIP-229[2] completed the support for join hints(i.e., query hints), back to the problem that this proposal is trying to solve, I remembered the issue FLINK-24254[3] created by Timo, which wanted to introduce a new config option to achieve fine-grained source/sink configuration(a bit different from the current dynamic table option). I also agree with this categorization, so in general we may have three categories of hints (with possible new ones in the future): 1. dynamic table options 2. query hints (join hints and possibly more support in the future) 3. configuration hints (fine-grained config vs global config) So which category does the operator-level state ttl configuration fall into? Considering the current global configuration "table.exec.state.ttl", I would put it in the third category, so I agree with Shuo's mention of modifying the title of the flip, e.g. "Extend exec plan to support operator-level state ttl configuration" Of course, this extends the original goal of FLINK-24254, not only to source/sink, but to all stateful operators, but how to design a configuration hints to support source/sink and all other operators in general, I don't have a clear answer yet, maybe this is more suitable to be discussed in a separate flip(hints or a lower level based configuration,e.g., exec plan, are solutions at different levels, each with its own advantages and disadvantages, and there seems to be no single answer to what is the best way to do fine-grained configurations) So +1 for this proposal (before a more perfect configuration hints which can balance the ease of use and generality) [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-113%3A+Supports+Dynamic+Table+Options+for+Flink+SQL [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-229%3A+Introduces+Join+Hint+for+Flink+SQL+Batch+Job [3] https://issues.apache.org/jira/browse/FLINK-24254 Best, Lincoln Lee Martijn Visser 于2023年3月28日周二 22:49写道： > Hi Jane, > > Thanks for creating the FLIP. In general I'm not a fan of using the query > plan for enabling these kinds of use cases. It introduces a different way > of submitting SQL jobs in our already extensive list of possibilities, > making things complicated. I would have a preference for using hints, given > that we explicitly mention hints for "Operator resource constraints" [1]. > For me, that feels like a more natural fit for this use case. > > I would like to get @Timo Walther his opinion on this > topic too. > > [1] > > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/ > > On Mon, Mar 27, 2023 at 10:22 PM Jing Ge > wrote: > > > Hi Jane, > > > > Thanks for clarifying it. As far as I am concerned, the issue is where to > > keep the user's job metadata, i.e. SQL script (to make the discussion > > easier, let's ignore config). As long as FLIP-190 is only used for > > migration/upgrade, SQL is the single source of truth. Once the compiled > > plan has been modified, in this case ttls, the user's job metadata will > be > > distributed into two different places. Each time when the SQL needs > > changes, extra effort will be required to take care of the modification > in > > the compiled plan. > > > > Examples: > > > > 1. If we try to start the same SQL with a new Flink cluster (one type of > > "restart") without knowing the modified compiled plan. The old > > performance issue will rise again. This might happen when multiple users > > are working on the same project who run a working SQL job, get > performance > > issues, and have no clue since nothing has been changed. Or one user is > > working on many SQL jobs who might lose the overview of which SQL job has > > modified plans or not. > > 2. If a SQL has been changed in a backwards compatible way and (re)start > > with a given savepoint(NO_CLAIM), the version2 json plan has to be made > > based on version1, as I mentioned previously, which means each time when > > the SQL got changed, the related compiled plan need modification too. > > Beyond that , it would also be easily forgotten to do it if there were no > > connection between the SQL and the related modified compiled plan. The > SQL > > job will have the performance issue again after the change. > > 3. Another scenario would be running a backwards compatible SQL job with > an > > upgraded FLink version, additional upgrade logic or guideline should be > > developed for e.g. ttl modification in the compiled plan, because > upgraded > > Flink engine underneath might lead to a different ttl setting. > > 4. The last scenario is just like you described that SQL has been changed > > significantly so that the compiled operators will be changed too. The > easy
[jira] [Created] (FLINK-31651) Improve logging of granting/revoking leadership in JobMasterServiceLeadershipRunner to INFO level
Lijie Wang created FLINK-31651: -- Summary: Improve logging of granting/revoking leadership in JobMasterServiceLeadershipRunner to INFO level Key: FLINK-31651 URL: https://issues.apache.org/jira/browse/FLINK-31651 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Affects Versions: 1.15.4, 1.16.1, 1.17.0 Reporter: Lijie Wang Currently, the log level of granting/revoking leadership of JobMasterServiceLeadershipRunner is DEBUG. However, we usually configure it to INFO level in production jobs, which make it hard to understand the behaviour from Flink's logs because JobMasterServiceLeadershipRunner may suddenly be stopped. I suggest to improve the logging to INFO level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31650) Incorrect busyMsTimePerSecond metric value for FINISHED task
Lijie Wang created FLINK-31650: -- Summary: Incorrect busyMsTimePerSecond metric value for FINISHED task Key: FLINK-31650 URL: https://issues.apache.org/jira/browse/FLINK-31650 Project: Flink Issue Type: Bug Components: Runtime / Metrics, Runtime / REST Affects Versions: 1.16.1, 1.17.0 Reporter: Lijie Wang Attachments: busyMsTimePerSecond.png As shown in the figure below, the busyMsTimePerSecond of the FINISHED task is 100%, which is obviously unreasonable. !busyMsTimePerSecond.png|width=827,height=341! -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)
Congratulations! Cheers, Zhu Jing Zhang 于2023年3月28日周二 23:56写道： > > Congratulations! > > Best, > Jing Zhang > > Rui Fan <1996fan...@gmail.com> 于2023年3月28日周二 22:25写道： > > > Congratulations! > > > > Best, > > Rui Fan > > > > On Tue, Mar 28, 2023 at 15:37 Guowei Ma wrote: > > > > > Congratulations! > > > > > > Best, > > > Guowei > > > > > > > > > On Tue, Mar 28, 2023 at 12:02 PM Yuxin Tan > > wrote: > > > > > >> Congratulations！ > > >> > > >> Best, > > >> Yuxin > > >> > > >> > > >> Guanghui Zhang 于2023年3月28日周二 11:06写道： > > >> > > >>> Congratulations！ > > >>> > > >>> Best, > > >>> Zhang Guanghui > > >>> > > >>> Hang Ruan 于2023年3月28日周二 10:29写道： > > >>> > > >>> > Congratulations! > > >>> > > > >>> > Best, > > >>> > Hang > > >>> > > > >>> > yu zelin 于2023年3月28日周二 10:27写道： > > >>> > > > >>> >> Congratulations! > > >>> >> > > >>> >> Best, > > >>> >> Yu Zelin > > >>> >> > > >>> >> 2023年3月27日 17:23，Yu Li 写道： > > >>> >> > > >>> >> Dear Flinkers, > > >>> >> > > >>> >> > > >>> >> > > >>> >> As you may have noticed, we are pleased to announce that Flink Table > > >>> Store has joined the Apache Incubator as a separate project called > > Apache > > >>> Paimon(incubating) [1] [2] [3]. The new project still aims at building > > a > > >>> streaming data lake platform for high-speed data ingestion, change data > > >>> tracking and efficient real-time analytics, with the vision of > > supporting a > > >>> larger ecosystem and establishing a vibrant and neutral open source > > >>> community. > > >>> >> > > >>> >> > > >>> >> > > >>> >> We would like to thank everyone for their great support and efforts > > >>> for the Flink Table Store project, and warmly welcome everyone to join > > the > > >>> development and activities of the new project. Apache Flink will > > continue > > >>> to be one of the first-class citizens supported by Paimon, and we > > believe > > >>> that the Flink and Paimon communities will maintain close cooperation. > > >>> >> > > >>> >> > > >>> >> 亲爱的Flinkers， > > >>> >> > > >>> >> > > >>> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache > > >>> >> 孵化器独立孵化 [1] [2] [3]。新项目的名字是 > > >>> >> Apache > > >>> > > Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。 > > >>> >> > > >>> >> > > >>> >> 在这里我们要感谢大家对 Flink Table Store > > >>> >> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon > > >>> 支持的主力计算引擎之一，我们也相信 > > >>> >> Flink 和 Paimon 社区将继续保持密切合作。 > > >>> >> > > >>> >> > > >>> >> Best Regards, > > >>> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) > > >>> >> > > >>> >> 致礼， > > >>> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） > > >>> >> > > >>> >> [1] https://paimon.apache.org/ > > >>> >> [2] https://github.com/apache/incubator-paimon > > >>> >> [3] > > >>> https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal > > >>> >> > > >>> >> > > >>> >> > > >>> > > >> > >
Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)
Congratulations! Best, Jing Zhang Rui Fan <1996fan...@gmail.com> 于2023年3月28日周二 22:25写道： > Congratulations! > > Best, > Rui Fan > > On Tue, Mar 28, 2023 at 15:37 Guowei Ma wrote: > > > Congratulations! > > > > Best, > > Guowei > > > > > > On Tue, Mar 28, 2023 at 12:02 PM Yuxin Tan > wrote: > > > >> Congratulations！ > >> > >> Best, > >> Yuxin > >> > >> > >> Guanghui Zhang 于2023年3月28日周二 11:06写道： > >> > >>> Congratulations！ > >>> > >>> Best, > >>> Zhang Guanghui > >>> > >>> Hang Ruan 于2023年3月28日周二 10:29写道： > >>> > >>> > Congratulations! > >>> > > >>> > Best, > >>> > Hang > >>> > > >>> > yu zelin 于2023年3月28日周二 10:27写道： > >>> > > >>> >> Congratulations! > >>> >> > >>> >> Best, > >>> >> Yu Zelin > >>> >> > >>> >> 2023年3月27日 17:23，Yu Li 写道： > >>> >> > >>> >> Dear Flinkers, > >>> >> > >>> >> > >>> >> > >>> >> As you may have noticed, we are pleased to announce that Flink Table > >>> Store has joined the Apache Incubator as a separate project called > Apache > >>> Paimon(incubating) [1] [2] [3]. The new project still aims at building > a > >>> streaming data lake platform for high-speed data ingestion, change data > >>> tracking and efficient real-time analytics, with the vision of > supporting a > >>> larger ecosystem and establishing a vibrant and neutral open source > >>> community. > >>> >> > >>> >> > >>> >> > >>> >> We would like to thank everyone for their great support and efforts > >>> for the Flink Table Store project, and warmly welcome everyone to join > the > >>> development and activities of the new project. Apache Flink will > continue > >>> to be one of the first-class citizens supported by Paimon, and we > believe > >>> that the Flink and Paimon communities will maintain close cooperation. > >>> >> > >>> >> > >>> >> 亲爱的Flinkers， > >>> >> > >>> >> > >>> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache > >>> >> 孵化器独立孵化 [1] [2] [3]。新项目的名字是 > >>> >> Apache > >>> > Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。 > >>> >> > >>> >> > >>> >> 在这里我们要感谢大家对 Flink Table Store > >>> >> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon > >>> 支持的主力计算引擎之一，我们也相信 > >>> >> Flink 和 Paimon 社区将继续保持密切合作。 > >>> >> > >>> >> > >>> >> Best Regards, > >>> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) > >>> >> > >>> >> 致礼， > >>> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） > >>> >> > >>> >> [1] https://paimon.apache.org/ > >>> >> [2] https://github.com/apache/incubator-paimon > >>> >> [3] > >>> https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal > >>> >> > >>> >> > >>> >> > >>> > >> >
[jira] [Created] (FLINK-31649) [Connectors / JDBC] Fix archunit violations
João Boto created FLINK-31649: - Summary: [Connectors / JDBC] Fix archunit violations Key: FLINK-31649 URL: https://issues.apache.org/jira/browse/FLINK-31649 Project: Flink Issue Type: Improvement Components: Connectors / JDBC Reporter: João Boto -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs
Hi Jane, Thanks for creating the FLIP. In general I'm not a fan of using the query plan for enabling these kinds of use cases. It introduces a different way of submitting SQL jobs in our already extensive list of possibilities, making things complicated. I would have a preference for using hints, given that we explicitly mention hints for "Operator resource constraints" [1]. For me, that feels like a more natural fit for this use case. I would like to get @Timo Walther his opinion on this topic too. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/ On Mon, Mar 27, 2023 at 10:22 PM Jing Ge wrote: > Hi Jane, > > Thanks for clarifying it. As far as I am concerned, the issue is where to > keep the user's job metadata, i.e. SQL script (to make the discussion > easier, let's ignore config). As long as FLIP-190 is only used for > migration/upgrade, SQL is the single source of truth. Once the compiled > plan has been modified, in this case ttls, the user's job metadata will be > distributed into two different places. Each time when the SQL needs > changes, extra effort will be required to take care of the modification in > the compiled plan. > > Examples: > > 1. If we try to start the same SQL with a new Flink cluster (one type of > "restart") without knowing the modified compiled plan. The old > performance issue will rise again. This might happen when multiple users > are working on the same project who run a working SQL job, get performance > issues, and have no clue since nothing has been changed. Or one user is > working on many SQL jobs who might lose the overview of which SQL job has > modified plans or not. > 2. If a SQL has been changed in a backwards compatible way and (re)start > with a given savepoint(NO_CLAIM), the version2 json plan has to be made > based on version1, as I mentioned previously, which means each time when > the SQL got changed, the related compiled plan need modification too. > Beyond that , it would also be easily forgotten to do it if there were no > connection between the SQL and the related modified compiled plan. The SQL > job will have the performance issue again after the change. > 3. Another scenario would be running a backwards compatible SQL job with an > upgraded FLink version, additional upgrade logic or guideline should be > developed for e.g. ttl modification in the compiled plan, because upgraded > Flink engine underneath might lead to a different ttl setting. > 4. The last scenario is just like you described that SQL has been changed > significantly so that the compiled operators will be changed too. The easy > way is to start a fresh new tuning. But since there was a tuning for the > last SQL. User has to compare both compiled plans and copy/paste some ttls > that might still work. > > A virtualization tool could help but might not reduce those efforts > significantly, since the user behaviour is changed enormously. > > I was aware that the json string might be large. Doing(EXECUTE PLAN 'json > plan as string') is intended to avoid dealing with files for most common > cases where the json string has common length. > > Anyway, it should be fine, if it is only recommended for advanced use cases > where users are aware of those efforts. > > Best regards, > Jing > > On Sat, Mar 25, 2023 at 3:54 PM Jane Chan wrote: > > > Hi Leonard, Jing and Shengkai, > > > > Thanks so much for your insightful comments. Here are my thoughts > > > > @Shengkai > > > 1. How the Gateway users use this feature? As far as I know, the > EXEUCTE > > PLAN only supports local file right now. Is it possible to extend this > > syntax to allow for reading plan files from remote file systems? > > > > Nice catch! Currently, the "COMPILE PLAN" and "EXECUTE PLAN" statements > > only support a local file path without the scheme (see > > TableEnvironmentImpl.java#L773 > > < > https://github.com/apache/flink/blob/80ee512f00a9a8873926626d66cdcc97164c4595/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java#L773 > >). > > It's reasonable to extend the support to Flink's FileSystem. Besides, the > > JSON plan should also be added to the resource cleaning mechanism for the > > Gateway mode, just like we do with the "ADD JAR" operation, cleaning it > up > > when the session ends. I will take your suggestion and make changes to > FLIP. > > > > > 2. I would like to inquire if there are any limitations on this > feature? > > I have encountered several instances where the data did not expire in the > > upstream operator, but it expired in the downstream operator, resulting > in > > abnormal calculation results or direct exceptions thrown by the operator > > (e.g. rank operator). Can we limit that the expiration time of downstream > > operator data should be greater than or equal to the expiration time of > > upstream operator data? > > > > This is an excellent point. In fact, the current state TTL is based on >
Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)
Congratulations! Best, Rui Fan On Tue, Mar 28, 2023 at 15:37 Guowei Ma wrote: > Congratulations! > > Best, > Guowei > > > On Tue, Mar 28, 2023 at 12:02 PM Yuxin Tan wrote: > >> Congratulations！ >> >> Best, >> Yuxin >> >> >> Guanghui Zhang 于2023年3月28日周二 11:06写道： >> >>> Congratulations！ >>> >>> Best, >>> Zhang Guanghui >>> >>> Hang Ruan 于2023年3月28日周二 10:29写道： >>> >>> > Congratulations! >>> > >>> > Best, >>> > Hang >>> > >>> > yu zelin 于2023年3月28日周二 10:27写道： >>> > >>> >> Congratulations! >>> >> >>> >> Best, >>> >> Yu Zelin >>> >> >>> >> 2023年3月27日 17:23，Yu Li 写道： >>> >> >>> >> Dear Flinkers, >>> >> >>> >> >>> >> >>> >> As you may have noticed, we are pleased to announce that Flink Table >>> Store has joined the Apache Incubator as a separate project called Apache >>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a >>> streaming data lake platform for high-speed data ingestion, change data >>> tracking and efficient real-time analytics, with the vision of supporting a >>> larger ecosystem and establishing a vibrant and neutral open source >>> community. >>> >> >>> >> >>> >> >>> >> We would like to thank everyone for their great support and efforts >>> for the Flink Table Store project, and warmly welcome everyone to join the >>> development and activities of the new project. Apache Flink will continue >>> to be one of the first-class citizens supported by Paimon, and we believe >>> that the Flink and Paimon communities will maintain close cooperation. >>> >> >>> >> >>> >> 亲爱的Flinkers， >>> >> >>> >> >>> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache >>> >> 孵化器独立孵化 [1] [2] [3]。新项目的名字是 >>> >> Apache >>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。 >>> >> >>> >> >>> >> 在这里我们要感谢大家对 Flink Table Store >>> >> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon >>> 支持的主力计算引擎之一，我们也相信 >>> >> Flink 和 Paimon 社区将继续保持密切合作。 >>> >> >>> >> >>> >> Best Regards, >>> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) >>> >> >>> >> 致礼， >>> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） >>> >> >>> >> [1] https://paimon.apache.org/ >>> >> [2] https://github.com/apache/incubator-paimon >>> >> [3] >>> https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal >>> >> >>> >> >>> >> >>> >>
Re: [VOTE] Release flink-connector-mongodb, release candidate #1
+1 (non-binding) * Hashes and Signatures look good * All required files on dist.apache.org * Tag is present in Github * Verified source archive does not contain any binary files * Source archive builds using maven Cheers, Hong > On 28 Mar 2023, at 12:56, Samrat Deb wrote: > > CAUTION: This email originated from outside of the organization. Do not click > links or open attachments unless you can confirm the sender and know the > content is safe. > > > > +1 (non-binding) > > > - checked Source archive builds using maven > > - verified Hashes and Signatures > > - verified Tag is present in Github > > - checked dist.apache.org, required files present > > - verified source archive > > > Bests, > > Samrat > > On Mon, Mar 20, 2023 at 2:08 PM Leonard Xu wrote: > >> >>> + 1 (non-binding) >>> >>> Best, >>> Jiaba >> >> Hi, Jiabao >> >> Thanks for help to verify the release candidate. But we need to do the >> verification firstly and then post the verified items when in a release >> candidate vote thread. >> >> The community does not require verification of all items in the document >> [1], each contributor can determine the items they need to verify based on >> the modules they are familiar with and the features they care about. >> >> Best, >> Leonard >> >> [1] >> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+Release >> >> >> >> >> >>> >>> 2023年3月17日 上午4:32，Martijn Visser 写道： +1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Thu, Mar 16, 2023 at 11:21 AM Danny Cranmer >> wrote: > Hi everyone, > Please review and vote on the release candidate # for the version , as > follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > > The complete staging area is available for your review, which includes: > * JIRA release notes [1], > * the official Apache source release to be deployed to dist.apache.org > [2], > which are signed with the key with fingerprint > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag v1.0.0-rc1 [5], > * website pull request listing the new release [6]. > > Given this is a new connector I will follow-up with a blog, but it >> will not > block the release. > > The vote will be open for at least 72 hours (ending 2023-03-21 11:00 >> UTC). > It is adopted by majority approval, with at least 3 PMC affirmative >> votes. > > Thanks, > Danny > > [1] > > >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352386 > [2] > > >> https://dist.apache.org/repos/dist/dev/flink/flink-connector-mongodb-1.0.0-rc1/ > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > [4] >> https://repository.apache.org/content/repositories/orgapacheflink-1598 > [5] > >> https://github.com/apache/flink-connector-mongodb/releases/tag/v1.0.0-rc1 > [6] https://github.com/apache/flink-web/pull/622 > >>> >> >>
Re: [DISCUSS] FLIP-304: Pluggable failure handling for Apache Flink
Hi Panagiotis, How about to introduce a config option to control which error handling plugins should be used? It is more flexible for deployments. Additionally, it can also enable users to explicitly specify the order that the plugins take effects. Thanks, Zhu Gen Luo 于2023年3月27日周一 15:02写道： > > Thanks for the summary! > > Also +1 to support custom restart strategies in a different FLIP, > as long as we can make sure that the plugin interface won't be > changed when the restart strategy interface is introduced. > > To achieve this, maybe we should think well how the handler > would cooperate with the restart strategy, like would it executes b > efore the strategy (e.g. some strategy may use the tag), or after > it (e.g. some metric reporting handler may use the handling result). > Though we can implement in one way, and extend if the other is > really necessary by someone. > > Besides, instead of using either of the names, shall we just make > them two subclasses named FailureEnricher and FailureListener? > The former executes synchronously and can modify the context, > while the latter executes asynchronously and has a read-only view > of context. In this way we can make sure a handler behaves in > the expected way. > > > On Thu, Mar 23, 2023 at 5:19 PM Zhu Zhu wrote: > > > +1 to support custom restart strategies in a different FLIP. > > > > It's fine to have a different plugin for custom restart strategy. > > If so, since we do not treat the FLIP-304 plugin as a common failure > > handler, but instead mainly targets to add labels to errors, I would > > +1 for the name `FailureEnricher`. > > > > Thanks, > > Zhu > > > > David Morávek 于2023年3月23日周四 15:51写道： > > > > > > > > > > > One additional remark on introducing it as an async operation: We would > > > > need a new configuration parameter to define the timeout for such a > > > > listener call, wouldn't we? > > > > > > > > > > This could be left up to the implementor to handle. > > > > > > What about adding an extra method getNamespace() to the Listener > > interface > > > > which returns an Optional. > > > > > > > > > > I'd avoid mixing an additional concept into this. We can simply have a > > new > > > method that returns a set of keys the listener can output. We can > > validate > > > this at the JM startup time and fail fast (since it's a configuration > > > error) if there is an overlap. If the listener outputs the key that is > > not > > > allowed to, I wouldn't be afraid to call into a fatal error handler since > > > it's an invalid implementation. > > > > > > Best, > > > D. > > > > > > On Thu, Mar 23, 2023 at 8:34 AM Matthias Pohl > > > wrote: > > > > > > > Sounds good. Two points I want to add: > > > > > > > >- Listener execution should be independent — however we need a way > > to > > > > > enforce a Label key/key-prefix is only assigned to a single Listener, > > > > > thinking of a validation step both at Listener init and runtime > > stages > > > > > > > > > What about adding an extra method getNamespace() to the Listener > > interface > > > > which returns an Optional. Therefore, the implementation/the > > user > > > > can decide depending on the use case whether it's necessary to have > > > > separate namespaces for the key/value pairs or not. On the Flink side, > > we > > > > would just merge the different maps considering their namespaces. > > > > > > > > A flaw of this approach is that if a user decides to use the same > > namespace > > > > for multiple listeners, how is an error in one of the listeners > > represented > > > > in the outcome? We would have to overwrite either the successful > > listener's > > > > result or the failed ones. I wanted to share it, anyway. > > > > > > > > One additional remark on introducing it as an async operation: We would > > > > need a new configuration parameter to define the timeout for such a > > > > listener call, wouldn't we? > > > > > > > > Matthias > > > > > > > > On Wed, Mar 22, 2023 at 4:56 PM Panagiotis Garefalakis < > > pga...@apache.org> > > > > wrote: > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > Thanks for the valuable comments! > > > > > Excited to see this is an area of interest for the community! > > > > > > > > > > Summarizing some of the main points raised along with my thoughts: > > > > > > > > > >- Labels (Key/Value) pairs are more expressive than Tags > > (Strings) so > > > > >using the former is a good idea — I am also debating if we want to > > > > > return > > > > >multiple KV pairs per Listener (one could argue that we could > > split > > > > the > > > > >logic in multiple Listeners to support that) > > > > >- An immutable context along with data returned using the > > interface > > > > >method implementations is a better approach than a mutable > > Collection > > > > >- Listener execution should be independent — however we need a > > way to > > > > >enforce a Label key/key-prefix is only assigned to a single > >
Re: [VOTE] Release flink-connector-mongodb, release candidate #1
+1 (non-binding) - checked Source archive builds using maven - verified Hashes and Signatures - verified Tag is present in Github - checked dist.apache.org, required files present - verified source archive Bests, Samrat On Mon, Mar 20, 2023 at 2:08 PM Leonard Xu wrote: > > > + 1 (non-binding) > > > > Best, > > Jiaba > > Hi, Jiabao > > Thanks for help to verify the release candidate. But we need to do the > verification firstly and then post the verified items when in a release > candidate vote thread. > > The community does not require verification of all items in the document > [1], each contributor can determine the items they need to verify based on > the modules they are familiar with and the features they care about. > > Best, > Leonard > > [1] > https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+Release > > > > > > > > > > >> 2023年3月17日 上午4:32，Martijn Visser 写道： > >> > >> +1 (binding) > >> > >> - Validated hashes > >> - Verified signature > >> - Verified that no binaries exist in the source archive > >> - Build the source with Maven > >> - Verified licenses > >> - Verified web PRs > >> > >> On Thu, Mar 16, 2023 at 11:21 AM Danny Cranmer > > >> wrote: > >> > >>> Hi everyone, > >>> Please review and vote on the release candidate # for the version , as > >>> follows: > >>> [ ] +1, Approve the release > >>> [ ] -1, Do not approve the release (please provide specific comments) > >>> > >>> > >>> The complete staging area is available for your review, which includes: > >>> * JIRA release notes [1], > >>> * the official Apache source release to be deployed to dist.apache.org > >>> [2], > >>> which are signed with the key with fingerprint > >>> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3], > >>> * all artifacts to be deployed to the Maven Central Repository [4], > >>> * source code tag v1.0.0-rc1 [5], > >>> * website pull request listing the new release [6]. > >>> > >>> Given this is a new connector I will follow-up with a blog, but it > will not > >>> block the release. > >>> > >>> The vote will be open for at least 72 hours (ending 2023-03-21 11:00 > UTC). > >>> It is adopted by majority approval, with at least 3 PMC affirmative > votes. > >>> > >>> Thanks, > >>> Danny > >>> > >>> [1] > >>> > >>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352386 > >>> [2] > >>> > >>> > https://dist.apache.org/repos/dist/dev/flink/flink-connector-mongodb-1.0.0-rc1/ > >>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS > >>> [4] > https://repository.apache.org/content/repositories/orgapacheflink-1598 > >>> [5] > >>> > https://github.com/apache/flink-connector-mongodb/releases/tag/v1.0.0-rc1 > >>> [6] https://github.com/apache/flink-web/pull/622 > >>> > > > >
Re: [VOTE] Release flink-connector-aws 4.1.0, release candidate #1
+1 (non-binding) - checked Source archive builds using maven - verified Hashes and Signatures - verified Tag is present in Github - checked dist.apache.org, required files present - verified source archive Bests, Samrat On Tue, Mar 28, 2023 at 5:12 PM Teoh, Hong wrote: > +1 (non-binding) > > * Hashes and Signatures look good > * All required files on dist.apache.org > * Tag is present in Github > * Verified source archive does not contain any binary files > * Source archive builds using maven > > > Cheers, > Hong > > > On 22 Mar 2023, at 07:21, Robert Metzger wrote: > > > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > > > > > +1 (binding) > > > > Note, in the NOTICE files, the copyright year is still only until 2022. I > > don't think it's a blocker, but should be updated. > > > > + Checked the website update > > + checked the source archive for copied code (no licenses are listed in > the > > root NOTICE file) > > + src archive contents build locally > > > > > > On Thu, Mar 16, 2023 at 1:11 PM Danny Cranmer > > wrote: > > > >> Hi everyone, > >> Please review and vote on the flink-connector-aws release candidate #1 > for > >> the version 4.1.0, as follows: > >> [ ] +1, Approve the release > >> [ ] -1, Do not approve the release (please provide specific comments) > >> > >> > >> The complete staging area is available for your review, which includes: > >> * JIRA release notes [1], > >> * the official Apache source release to be deployed to dist.apache.org > >> [2], > >> which are signed with the key with fingerprint > >> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3], > >> * all artifacts to be deployed to the Maven Central Repository [4], > >> * source code tag v4.1.0-rc1 [5], > >> * website pull request listing the new release [6]. > >> > >> The vote will be open for at least 72 hours (ending 2023-03-21 13:00 > UTC). > >> It is adopted by majority approval, with at least 3 PMC affirmative > votes. > >> > >> Thanks, > >> Danny > >> > >> [1] > >> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352646 > >> [2] > >> > https://dist.apache.org/repos/dist/dev/flink/flink-connector-aws-4.1.0-rc1/ > >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS > >> [4] > >> https://repository.apache.org/content/repositories/orgapacheflink-1599/ > >> [5] > https://github.com/apache/flink-connector-aws/releases/tag/v4.1.0-rc1 > >> [6] https://github.com/apache/flink-web/pull/623 > >> > >
Re: [VOTE] FLIP-284: Making AsyncSinkWriter Flush triggers adjustable
Thank you Ahmed for driving the change. This new feature will help flink developers with more control on their job in SinkWriter. +1 (non binding) Bests, Samrat On Mon, Mar 27, 2023 at 12:34 PM Danny Cranmer wrote: > Thanks for driving this Ahmed. This change will benefit future and current > sink implementations by giving the developer more flexibility. A good > addition following configurable rate limiting strategies [1]. > > +1 (binding) > > Thanks, > Danny > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-242%3A+Introduce+configurable+RateLimitingStrategy+for+Async+Sink > > On Sun, Mar 26, 2023 at 2:27 PM Ahmed Hamdy wrote: > > > Dear Flink developers, > > > > I want to move discussion on this FLIP[1] to voting. Discussion thread[2] > > has been inactive for more than 2 months. Please feel free to add your > > feedback below. > > > > The vote will be open for at least 72 > > hours unless there is an objection or not enough votes. > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-284+%3A+Making+AsyncSinkWriter+Flush+triggers+adjustable > > [2] https://lists.apache.org/thread/hmh2xoy34jm4r81txm3x1wv27d6vnpkw > > >
Re: [VOTE] Release flink-connector-aws 4.1.0, release candidate #1
+1 (non-binding) * Hashes and Signatures look good * All required files on dist.apache.org * Tag is present in Github * Verified source archive does not contain any binary files * Source archive builds using maven Cheers, Hong > On 22 Mar 2023, at 07:21, Robert Metzger wrote: > > CAUTION: This email originated from outside of the organization. Do not click > links or open attachments unless you can confirm the sender and know the > content is safe. > > > > +1 (binding) > > Note, in the NOTICE files, the copyright year is still only until 2022. I > don't think it's a blocker, but should be updated. > > + Checked the website update > + checked the source archive for copied code (no licenses are listed in the > root NOTICE file) > + src archive contents build locally > > > On Thu, Mar 16, 2023 at 1:11 PM Danny Cranmer > wrote: > >> Hi everyone, >> Please review and vote on the flink-connector-aws release candidate #1 for >> the version 4.1.0, as follows: >> [ ] +1, Approve the release >> [ ] -1, Do not approve the release (please provide specific comments) >> >> >> The complete staging area is available for your review, which includes: >> * JIRA release notes [1], >> * the official Apache source release to be deployed to dist.apache.org >> [2], >> which are signed with the key with fingerprint >> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3], >> * all artifacts to be deployed to the Maven Central Repository [4], >> * source code tag v4.1.0-rc1 [5], >> * website pull request listing the new release [6]. >> >> The vote will be open for at least 72 hours (ending 2023-03-21 13:00 UTC). >> It is adopted by majority approval, with at least 3 PMC affirmative votes. >> >> Thanks, >> Danny >> >> [1] >> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352646 >> [2] >> https://dist.apache.org/repos/dist/dev/flink/flink-connector-aws-4.1.0-rc1/ >> [3] https://dist.apache.org/repos/dist/release/flink/KEYS >> [4] >> https://repository.apache.org/content/repositories/orgapacheflink-1599/ >> [5] https://github.com/apache/flink-connector-aws/releases/tag/v4.1.0-rc1 >> [6] https://github.com/apache/flink-web/pull/623 >>
[jira] [Created] (FLINK-31648) Elasticsearch supports retrying writes when concurrency issues occur
陈磊 created FLINK-31648: -- Summary: Elasticsearch supports retrying writes when concurrency issues occur Key: FLINK-31648 URL: https://issues.apache.org/jira/browse/FLINK-31648 Project: Flink Issue Type: New Feature Components: Connectors / ElasticSearch Affects Versions: 1.14.6 Reporter: 陈磊 Elasticsearch supports retrying writes when concurrency issues occur.The exception information is as follows: ElasticsearchException[Elasticsearch exception [type=version_conflict_engine_exception, reason=[**][**]: version conflict, current version [2] is different than the one provided [1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)
Congratulations! Best, Guowei On Tue, Mar 28, 2023 at 12:02 PM Yuxin Tan wrote: > Congratulations！ > > Best, > Yuxin > > > Guanghui Zhang 于2023年3月28日周二 11:06写道： > >> Congratulations！ >> >> Best, >> Zhang Guanghui >> >> Hang Ruan 于2023年3月28日周二 10:29写道： >> >> > Congratulations! >> > >> > Best, >> > Hang >> > >> > yu zelin 于2023年3月28日周二 10:27写道： >> > >> >> Congratulations! >> >> >> >> Best, >> >> Yu Zelin >> >> >> >> 2023年3月27日 17:23，Yu Li 写道： >> >> >> >> Dear Flinkers, >> >> >> >> >> >> >> >> As you may have noticed, we are pleased to announce that Flink Table >> Store has joined the Apache Incubator as a separate project called Apache >> Paimon(incubating) [1] [2] [3]. The new project still aims at building a >> streaming data lake platform for high-speed data ingestion, change data >> tracking and efficient real-time analytics, with the vision of supporting a >> larger ecosystem and establishing a vibrant and neutral open source >> community. >> >> >> >> >> >> >> >> We would like to thank everyone for their great support and efforts >> for the Flink Table Store project, and warmly welcome everyone to join the >> development and activities of the new project. Apache Flink will continue >> to be one of the first-class citizens supported by Paimon, and we believe >> that the Flink and Paimon communities will maintain close cooperation. >> >> >> >> >> >> 亲爱的Flinkers， >> >> >> >> >> >> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache >> >> 孵化器独立孵化 [1] [2] [3]。新项目的名字是 >> >> Apache >> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。 >> >> >> >> >> >> 在这里我们要感谢大家对 Flink Table Store >> >> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon >> 支持的主力计算引擎之一，我们也相信 >> >> Flink 和 Paimon 社区将继续保持密切合作。 >> >> >> >> >> >> Best Regards, >> >> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) >> >> >> >> 致礼， >> >> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） >> >> >> >> [1] https://paimon.apache.org/ >> >> [2] https://github.com/apache/incubator-paimon >> >> [3] >> https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal >> >> >> >> >> >> >> >
[jira] [Created] (FLINK-31647) Introduce remote storage tier reader
Yuxin Tan created FLINK-31647: - Summary: Introduce remote storage tier reader Key: FLINK-31647 URL: https://issues.apache.org/jira/browse/FLINK-31647 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31646) Introduce remote storage tier
Yuxin Tan created FLINK-31646: - Summary: Introduce remote storage tier Key: FLINK-31646 URL: https://issues.apache.org/jira/browse/FLINK-31646 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31644) Introduce disk tier
Yuxin Tan created FLINK-31644: - Summary: Introduce disk tier Key: FLINK-31644 URL: https://issues.apache.org/jira/browse/FLINK-31644 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31645) Introduce disk tier reader
Yuxin Tan created FLINK-31645: - Summary: Introduce disk tier reader Key: FLINK-31645 URL: https://issues.apache.org/jira/browse/FLINK-31645 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31643) Introduce a temporary switch to enable the new shuffle implementation
Yuxin Tan created FLINK-31643: - Summary: Introduce a temporary switch to enable the new shuffle implementation Key: FLINK-31643 URL: https://issues.apache.org/jira/browse/FLINK-31643 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31641) Introduce memory tier
Yuxin Tan created FLINK-31641: - Summary: Introduce memory tier Key: FLINK-31641 URL: https://issues.apache.org/jira/browse/FLINK-31641 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31642) Introduce memory tier reader
Yuxin Tan created FLINK-31642: - Summary: Introduce memory tier reader Key: FLINK-31642 URL: https://issues.apache.org/jira/browse/FLINK-31642 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31640) Introduce segment index tracker
Yuxin Tan created FLINK-31640: - Summary: Introduce segment index tracker Key: FLINK-31640 URL: https://issues.apache.org/jira/browse/FLINK-31640 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31639) Introduce tiered store memory manager
Yuxin Tan created FLINK-31639: - Summary: Introduce tiered store memory manager Key: FLINK-31639 URL: https://issues.apache.org/jira/browse/FLINK-31639 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31638) Downstream supports reading buffers from tiered store
Yuxin Tan created FLINK-31638: - Summary: Downstream supports reading buffers from tiered store Key: FLINK-31638 URL: https://issues.apache.org/jira/browse/FLINK-31638 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31637) Introduce buffer accumulator tier
Yuxin Tan created FLINK-31637: - Summary: Introduce buffer accumulator tier Key: FLINK-31637 URL: https://issues.apache.org/jira/browse/FLINK-31637 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31636) Upstream supports reading buffers from tiered store
Yuxin Tan created FLINK-31636: - Summary: Upstream supports reading buffers from tiered store Key: FLINK-31636 URL: https://issues.apache.org/jira/browse/FLINK-31636 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31635) Support writing records to the new tiered store architecture
Yuxin Tan created FLINK-31635: - Summary: Support writing records to the new tiered store architecture Key: FLINK-31635 URL: https://issues.apache.org/jira/browse/FLINK-31635 Project: Flink Issue Type: Sub-task Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)
Exciting! If this ends up working well, Wikimedia Foundation would love to try it out! On Mon, Mar 27, 2023 at 8:39 AM Matthias Pohl via user < u...@flink.apache.org> wrote: > Congratulations and good luck with pushing the project forward. > > On Mon, Mar 27, 2023 at 2:35 PM Jing Ge via user > wrote: > >> Congrats! >> >> Best regards, >> Jing >> >> On Mon, Mar 27, 2023 at 2:32 PM Leonard Xu wrote: >> >>> Congratulations! >>> >>> >>> Best, >>> Leonard >>> >>> On Mar 27, 2023, at 5:23 PM, Yu Li wrote: >>> >>> Dear Flinkers, >>> >>> >>> >>> As you may have noticed, we are pleased to announce that Flink Table Store >>> has joined the Apache Incubator as a separate project called Apache >>> Paimon(incubating) [1] [2] [3]. The new project still aims at building a >>> streaming data lake platform for high-speed data ingestion, change data >>> tracking and efficient real-time analytics, with the vision of supporting a >>> larger ecosystem and establishing a vibrant and neutral open source >>> community. >>> >>> >>> >>> We would like to thank everyone for their great support and efforts for the >>> Flink Table Store project, and warmly welcome everyone to join the >>> development and activities of the new project. Apache Flink will continue >>> to be one of the first-class citizens supported by Paimon, and we believe >>> that the Flink and Paimon communities will maintain close cooperation. >>> >>> >>> 亲爱的Flinkers， >>> >>> >>> 正如您可能已经注意到的，我们很高兴地宣布，Flink Table Store 已经正式加入 Apache >>> 孵化器独立孵化 [1] [2] [3]。新项目的名字是 >>> Apache >>> Paimon(incubating)，仍致力于打造一个支持高速数据摄入、流式数据订阅和高效实时分析的新一代流式湖仓平台。此外，新项目将支持更加丰富的生态，并建立一个充满活力和中立的开源社区。 >>> >>> >>> 在这里我们要感谢大家对 Flink Table Store >>> 项目的大力支持和投入，并热烈欢迎大家加入新项目的开发和社区活动。Apache Flink 将继续作为 Paimon 支持的主力计算引擎之一，我们也相信 >>> Flink 和 Paimon 社区将继续保持密切合作。 >>> >>> >>> Best Regards, >>> Yu (on behalf of the Apache Flink PMC and Apache Paimon PPMC) >>> >>> 致礼， >>> 李钰（谨代表 Apache Flink PMC 和 Apache Paimon PPMC） >>> >>> [1] https://paimon.apache.org/ >>> [2] https://github.com/apache/incubator-paimon >>> [3] https://cwiki.apache.org/confluence/display/INCUBATOR/PaimonProposal >>> >>> >>>
