Re: [VOTE] FLIP-254 Redis Streams connector
Hi Martijn, Thanks for driving this forward, but from the discussion thread [1], it seems one question from Konstantin is still not answered: does this FLIP only aim to add source/sink connectors for Redis Streams, and we will create another one to support regular Redis data structures (as LookupTableSource or Sink)? I think it's necessary to confirm the scope of the FLIP before voting (smile). Best Regards, Yu [1] https://lists.apache.org/thread/9pftp3x10602nzr9c0ln4h9zt8hg2rjk On Mon, 12 Sept 2022 at 22:05, Ryan Skraba wrote: > Hello! There's quite a bit of existing code and it looks like there's > interest and community willing to contribute to this connector with 2 > implementations already in the flink-connector-redis repo[1]. > > There's a couple of points that should probably be fixed in the FLIP: some > typos such as "provide at-least guarantees" and the initial version should > not be 1.0.0 given that version 1.1.5 was already released in its previous > incarnation[2]. > > In principle: +1 (non-binding) > > All my best, Ryan > > [1]: https://github.com/apache/flink-connector-redis/pulls > [2]: > https://mvnrepository.com/artifact/org.apache.flink/flink-connector-redis > > > > On Mon, Sep 12, 2022 at 10:20 AM Zheng Yu Chen > wrote: > > > +1 (non-binding) > > > > Martijn Visser 于2022年9月12日周一 15:58写道： > > > > > Hi everyone, > > > > > > With no comments provided in the discussion thread, I'm opening a vote > > > thread on FLIP-254: Redis Streams connector: > > > > > > FLIP: > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-254%3A+Redis+Streams+Connector > > > > > > > > > The vote will be open for at least 72h. > > > > > > Best regards, > > > > > > Martijn > > > https://twitter.com/MartijnVisser82 > > > https://github.com/MartijnVisser > > > > > >
[jira] [Created] (FLINK-29276) Flush all memory in SortBufferMemTable.clear
Jingsong Lee created FLINK-29276: Summary: Flush all memory in SortBufferMemTable.clear Key: FLINK-29276 URL: https://issues.apache.org/jira/browse/FLINK-29276 Project: Flink Issue Type: Improvement Reporter: Jingsong Lee Now BinaryInMemorySortBuffer.reset will keep one page. We could free all memory. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29275) Temporal Table function: Cannot add expression of different type to set
Tally created FLINK-29275:
-
Summary: Temporal Table function: Cannot add expression of
different type to set
Key: FLINK-29275
URL: https://issues.apache.org/jira/browse/FLINK-29275
Project: Flink
Issue Type: Bug
Affects Versions: 1.15.2
Reporter: Tally
I am useing the temporal table funciton to join two stream like this, but got
this error. Any ways to solve this?
{code:java}
Exception in thread "main" java.lang.AssertionError: Cannot add expression of
different type to set:
set type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE" order_id,
DECIMAL(32, 2) price, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency,
TIMESTAMP(3) order_time, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime,
VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency0, BIGINT conversion_rate,
TIMESTAMP(3) update_time, TIMESTAMP_LTZ(3) *PROCTIME* proctime0) NOT NULL
expression type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
order_id, DECIMAL(32, 2) price, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
currency, TIMESTAMP(3) order_time, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL
proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency0, BIGINT
conversion_rate, TIMESTAMP(3) update_time, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL
proctime0) NOT NULL
set is rel#61:LogicalCorrelate.NONE.any.None:
0.[NONE].[NONE](left=HepRelVertex#59,right=HepRelVertex#60,correlation=$cor0,joinType=inner,requiredColumns={4})
expression is LogicalJoin(condition=[__TEMPORAL_JOIN_CONDITION($4, $7,
__TEMPORAL_JOIN_CONDITION_PRIMARY_KEY($5))], joinType=[inner])
LogicalProject(order_id=[$0], price=[$1], currency=[$2], order_time=[$3],
proctime=[PROCTIME()])
LogicalTableScan(table=[[default_catalog, default_database, orders]])
LogicalProject(currency=[$0], conversion_rate=[$1], update_time=[$2],
proctime=[PROCTIME()])
LogicalTableScan(table=[[default_catalog, default_database,
currency_rates]])
{code}
Fact Table:
{code:java}
CREATE TABLE `orders` (
order_id STRING,
price DECIMAL(32,2),
currency STRING,
order_time TIMESTAMP(3),
proctime as PROCTIME()
) WITH (
'properties.bootstrap.servers' = '127.0.0.1:9092',
'properties.group.id' = 'test',
'scan.topic-partition-discovery.interval' = '1',
'connector' = 'kafka',
'format' = 'json',
'scan.startup.mode' = 'latest-offset',
'topic' = 'test1'
) {code}
Build Table:
{code:java}
CREATE TABLE `currency_rates` (
currency STRING,
conversion_rate BIGINT,
update_time TIMESTAMP(3),
proctime as PROCTIME()
) WITH (
'properties.bootstrap.servers' = '127.0.0.1:9092',
'properties.group.id' = 'test',
'scan.topic-partition-discovery.interval' = '1',
'connector' = 'kafka',
'format' = 'json',
'scan.startup.mode' = 'latest-offset',
'topic' = 'test3'
) {code}
The way to generate table function:
{code:java}
TemporalTableFunction table_rate = tEnv.from("currency_rates")
.createTemporalTableFunction("update_time", "currency");
tEnv.registerFunction("rates", table_rate); {code}
Join logic:
{code:java}
SELECT
order_id,
price,
s.currency,
conversion_rate,
order_time
FROM orders AS o,
LATERAL TABLE (rates(o.proctime)) AS s
WHERE o.currency = s.currency {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29274) HiveServer2EndpointITCase.testGetFunctionWithPattern failed with Persistence Manager has been closed
Xingbo Huang created FLINK-29274:
Summary: HiveServer2EndpointITCase.testGetFunctionWithPattern
failed with Persistence Manager has been closed
Key: FLINK-29274
URL: https://issues.apache.org/jira/browse/FLINK-29274
Project: Flink
Issue Type: Bug
Components: Connectors / Hive
Affects Versions: 1.16.0
Reporter: Xingbo Huang
Fix For: 1.16.0
{code:java}
4.6807800Z Sep 13 02:07:54 [ERROR]
org.apache.flink.table.endpoint.hive.HiveServer2EndpointITCase.testGetFunctionWithPattern
Time elapsed: 22.127 s <<< ERROR!
2022-09-13T02:07:54.6813586Z Sep 13 02:07:54 java.sql.SQLException:
javax.jdo.JDOFatalUserException: Persistence Manager has been closed
2022-09-13T02:07:54.6815315Z Sep 13 02:07:54at
org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401)
2022-09-13T02:07:54.6816917Z Sep 13 02:07:54at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266)
2022-09-13T02:07:54.6818338Z Sep 13 02:07:54at
org.apache.flink.table.endpoint.hive.HiveServer2EndpointITCase.lambda$testGetFunctionWithPattern$29(HiveServer2EndpointITCase.java:542)
2022-09-13T02:07:54.6819988Z Sep 13 02:07:54at
org.apache.flink.table.endpoint.hive.HiveServer2EndpointITCase.runGetObjectTest(HiveServer2EndpointITCase.java:633)
2022-09-13T02:07:54.6821484Z Sep 13 02:07:54at
org.apache.flink.table.endpoint.hive.HiveServer2EndpointITCase.runGetObjectTest(HiveServer2EndpointITCase.java:621)
2022-09-13T02:07:54.6823318Z Sep 13 02:07:54at
org.apache.flink.table.endpoint.hive.HiveServer2EndpointITCase.testGetFunctionWithPattern(HiveServer2EndpointITCase.java:539)
2022-09-13T02:07:54.6824711Z Sep 13 02:07:54at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-09-13T02:07:54.6825817Z Sep 13 02:07:54at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-09-13T02:07:54.6827003Z Sep 13 02:07:54at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-09-13T02:07:54.6828259Z Sep 13 02:07:54at
java.lang.reflect.Method.invoke(Method.java:498)
2022-09-13T02:07:54.6829478Z Sep 13 02:07:54at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
2022-09-13T02:07:54.6830717Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
2022-09-13T02:07:54.6832444Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
2022-09-13T02:07:54.6834028Z Sep 13 02:07:54at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
2022-09-13T02:07:54.6835304Z Sep 13 02:07:54at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
2022-09-13T02:07:54.6836734Z Sep 13 02:07:54at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
2022-09-13T02:07:54.6838257Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
2022-09-13T02:07:54.6839775Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
2022-09-13T02:07:54.6841400Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
2022-09-13T02:07:54.6843309Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
2022-09-13T02:07:54.6845300Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
2022-09-13T02:07:54.6846879Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
2022-09-13T02:07:54.6848406Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
2022-09-13T02:07:54.6849760Z Sep 13 02:07:54at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
2022-09-13T02:07:54.6851297Z Sep 13 02:07:54at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
2022-09-13T02:07:54.6853032Z Sep 13 02:07:54at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
2022-09-13T02:07:54.6854384Z Sep 13 02:07:54at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
2022-09-13T02:07:54.6856052Z Sep 13 02:07:54at
[jira] [Created] (FLINK-29273) Page not enough Exception in SortBufferMemTable
Jingsong Lee created FLINK-29273: Summary: Page not enough Exception in SortBufferMemTable Key: FLINK-29273 URL: https://issues.apache.org/jira/browse/FLINK-29273 Project: Flink Issue Type: Bug Components: Table Store Reporter: Jingsong Lee Fix For: table-store-0.3.0, table-store-0.2.1 Attachments: image-2022-09-13-11-03-07-855.png !image-2022-09-13-11-03-07-855.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congratulations Martijn! Best regards, Qingsheng > On Sep 9, 2022, at 23:08, Timo Walther wrote: > > Hi everyone, > > I'm very happy to announce that Martijn Visser has joined the Flink PMC! > > Martijn has helped the community in many different ways over the past months. > Externalizing the connectors from the Flink repo to their own repository, > continously updating dependencies, and performing other project-wide > refactorings. He is constantly coordinating contributions, connecting > stakeholders, finding committers for contributions, driving release syncs, > and helping in making the ASF a better place (e.g. by using Matomo an > ASF-compliant tracking solution for all projects). > > Congratulations and welcome, Martijn! > > Cheers, > Timo Walther > (On behalf of the Apache Flink PMC)
[jira] [Created] (FLINK-29272) Document DataStream API (DataStream to Table) for table store
Jingsong Lee created FLINK-29272: Summary: Document DataStream API (DataStream to Table) for table store Key: FLINK-29272 URL: https://issues.apache.org/jira/browse/FLINK-29272 Project: Flink Issue Type: New Feature Components: Table Store Reporter: Jingsong Lee Fix For: table-store-0.3.0, table-store-0.2.1 We can have documentation to describe how to convert from DataStream to Table to write to TableStore. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats, Martijn! Well deserved! Best, Paul Lam > 2022年9月13日 10:53，Jingsong Li 写道： > > Congrats, Martijn! > > Best, > Jingsong > > On Tue, Sep 13, 2022 at 10:48 AM Yang Wang wrote: >> >> Congrats, Martijn! >> >> Best, >> Yang >> >> Lijie Wang 于2022年9月13日周二 10:10写道： >> >>> Congratulations, Martijn! >>> >>> Best, >>> Lijie >>> >>> yuxia 于2022年9月13日周二 09:52写道： >>> Congrats, Martijn! Best regards, Yuxia - 原始邮件 - 发件人: "Steven Wu" 收件人: "dev" 发送时间: 星期二, 2022年 9 月 13日 上午 5:33:47 主题: Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser Congrats, Martijn! On Mon, Sep 12, 2022 at 1:49 PM Alexander Fedulov wrote: > Congrats, Martijn! > > On Mon, Sep 12, 2022 at 10:06 AM Jing Ge wrote: > >> Congrats! >> >> On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang wrote: >> >>> Congrats! >>> >>> On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser < > martijnvis...@apache.org >>> >>> wrote: >>> Thank you all :) Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen < > jam.gz...@gmail.com >>> : > Congratulations, Martijn > > > > Timo Walther 于2022年9月9日周五 23:08写道： > >> Hi everyone, >> >> I'm very happy to announce that Martijn Visser has joined the > Flink PMC! >> >> Martijn has helped the community in many different ways over the >> past >> months. Externalizing the connectors from the Flink repo to their >> own >> repository, continously updating dependencies, and performing > other >> project-wide refactorings. He is constantly coordinating >>> contributions, >> connecting stakeholders, finding committers for >>> contributions, >>> driving >> release syncs, and helping in making the ASF a better place (e.g. >> by >> using Matomo an ASF-compliant tracking solution for all > projects). >> >> Congratulations and welcome, Martijn! >> >> Cheers, >> Timo Walther >> (On behalf of the Apache Flink PMC) >> > > > -- > Best > > ConradJam > >>> >> > >>>
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats, Martijn! Best, Jingsong On Tue, Sep 13, 2022 at 10:48 AM Yang Wang wrote: > > Congrats, Martijn! > > Best, > Yang > > Lijie Wang 于2022年9月13日周二 10:10写道： > > > Congratulations, Martijn! > > > > Best, > > Lijie > > > > yuxia 于2022年9月13日周二 09:52写道： > > > > > Congrats, Martijn! > > > > > > Best regards, > > > Yuxia > > > > > > - 原始邮件 - > > > 发件人: "Steven Wu" > > > 收件人: "dev" > > > 发送时间: 星期二, 2022年 9 月 13日 上午 5:33:47 > > > 主题: Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser > > > > > > Congrats, Martijn! > > > > > > On Mon, Sep 12, 2022 at 1:49 PM Alexander Fedulov > > > wrote: > > > > > > > Congrats, Martijn! > > > > > > > > On Mon, Sep 12, 2022 at 10:06 AM Jing Ge wrote: > > > > > > > > > Congrats! > > > > > > > > > > On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang > > > wrote: > > > > > > > > > > > Congrats! > > > > > > > > > > > > On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser < > > > > martijnvis...@apache.org > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Thank you all :) > > > > > > > > > > > > > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen < > > > > jam.gz...@gmail.com > > > > > >: > > > > > > > > > > > > > > > Congratulations, Martijn > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > I'm very happy to announce that Martijn Visser has joined the > > > > Flink > > > > > > > PMC! > > > > > > > > > > > > > > > > > > Martijn has helped the community in many different ways over > > > the > > > > > past > > > > > > > > > months. Externalizing the connectors from the Flink repo to > > > their > > > > > own > > > > > > > > > repository, continously updating dependencies, and performing > > > > other > > > > > > > > > project-wide refactorings. He is constantly coordinating > > > > > > contributions, > > > > > > > > > connecting stakeholders, finding committers for > > contributions, > > > > > > driving > > > > > > > > > release syncs, and helping in making the ASF a better place > > > (e.g. > > > > > by > > > > > > > > > using Matomo an ASF-compliant tracking solution for all > > > > projects). > > > > > > > > > > > > > > > > > > Congratulations and welcome, Martijn! > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > Timo Walther > > > > > > > > > (On behalf of the Apache Flink PMC) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best > > > > > > > > > > > > > > > > ConradJam > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
[jira] [Created] (FLINK-29271) Change to byte array from bytebuffer to improve performance when reading parquet file
jiangjiguang0719 created FLINK-29271: Summary: Change to byte array from bytebuffer to improve performance when reading parquet file Key: FLINK-29271 URL: https://issues.apache.org/jira/browse/FLINK-29271 Project: Flink Issue Type: Improvement Reporter: jiangjiguang0719 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats, Martijn! Best, Yang Lijie Wang 于2022年9月13日周二 10:10写道： > Congratulations, Martijn! > > Best, > Lijie > > yuxia 于2022年9月13日周二 09:52写道： > > > Congrats, Martijn! > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Steven Wu" > > 收件人: "dev" > > 发送时间: 星期二, 2022年 9 月 13日 上午 5:33:47 > > 主题: Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser > > > > Congrats, Martijn! > > > > On Mon, Sep 12, 2022 at 1:49 PM Alexander Fedulov > > wrote: > > > > > Congrats, Martijn! > > > > > > On Mon, Sep 12, 2022 at 10:06 AM Jing Ge wrote: > > > > > > > Congrats! > > > > > > > > On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang > > wrote: > > > > > > > > > Congrats! > > > > > > > > > > On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser < > > > martijnvis...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > Thank you all :) > > > > > > > > > > > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen < > > > jam.gz...@gmail.com > > > > >: > > > > > > > > > > > > > Congratulations, Martijn > > > > > > > > > > > > > > > > > > > > > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > I'm very happy to announce that Martijn Visser has joined the > > > Flink > > > > > > PMC! > > > > > > > > > > > > > > > > Martijn has helped the community in many different ways over > > the > > > > past > > > > > > > > months. Externalizing the connectors from the Flink repo to > > their > > > > own > > > > > > > > repository, continously updating dependencies, and performing > > > other > > > > > > > > project-wide refactorings. He is constantly coordinating > > > > > contributions, > > > > > > > > connecting stakeholders, finding committers for > contributions, > > > > > driving > > > > > > > > release syncs, and helping in making the ASF a better place > > (e.g. > > > > by > > > > > > > > using Matomo an ASF-compliant tracking solution for all > > > projects). > > > > > > > > > > > > > > > > Congratulations and welcome, Martijn! > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Timo Walther > > > > > > > > (On behalf of the Apache Flink PMC) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best > > > > > > > > > > > > > > ConradJam > > > > > > > > > > > > > > > > > > > > > > > > > > > >
[jira] [Created] (FLINK-29270) Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE
Muhammad Hilmi Al Fatih created FLINK-29270: --- Summary: Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE Key: FLINK-29270 URL: https://issues.apache.org/jira/browse/FLINK-29270 Project: Flink Issue Type: Bug Components: Connectors / Kafka, Runtime / Metrics Affects Versions: 1.14.4 Reporter: Muhammad Hilmi Al Fatih Attachments: image-2022-09-13-11-41-51-759.png, image-2022-09-13-11-42-45-297.png, image-2022-09-13-11-43-38-226.png, image-2022-09-13-11-43-50-441.png I did a very simple kafka-to-kafka pipeline with flink 1.14.4 with checkpoint. I did 4 tests to confirm: for each old API (FlinkKafkaConsumer) and new API (KafkaSource), I run AT_LEAST_ONCE and EXACLTY_ONCE. However, the metrics shown by producer is not correct when doing EXACTLY_ONCE semantics in both APIs. To understand the situation, I also check the incoming rate metrics for both source and sink topics. So here are the summary: h3. Old API h4. At Least Once !image-2022-09-13-11-41-51-759.png|width=616,height=190! h4. Exactly Once !image-2022-09-13-11-42-45-297.png! h3. New API (KafkaSource) h4. At Least Once !image-2022-09-13-11-43-38-226.png! h4. Exactly Once !image-2022-09-13-11-43-50-441.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congratulations, Martijn! Best, Lijie yuxia 于2022年9月13日周二 09:52写道： > Congrats, Martijn! > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Steven Wu" > 收件人: "dev" > 发送时间: 星期二, 2022年 9 月 13日 上午 5:33:47 > 主题: Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser > > Congrats, Martijn! > > On Mon, Sep 12, 2022 at 1:49 PM Alexander Fedulov > wrote: > > > Congrats, Martijn! > > > > On Mon, Sep 12, 2022 at 10:06 AM Jing Ge wrote: > > > > > Congrats! > > > > > > On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang > wrote: > > > > > > > Congrats! > > > > > > > > On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser < > > martijnvis...@apache.org > > > > > > > > wrote: > > > > > > > > > Thank you all :) > > > > > > > > > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen < > > jam.gz...@gmail.com > > > >: > > > > > > > > > > > Congratulations, Martijn > > > > > > > > > > > > > > > > > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > I'm very happy to announce that Martijn Visser has joined the > > Flink > > > > > PMC! > > > > > > > > > > > > > > Martijn has helped the community in many different ways over > the > > > past > > > > > > > months. Externalizing the connectors from the Flink repo to > their > > > own > > > > > > > repository, continously updating dependencies, and performing > > other > > > > > > > project-wide refactorings. He is constantly coordinating > > > > contributions, > > > > > > > connecting stakeholders, finding committers for contributions, > > > > driving > > > > > > > release syncs, and helping in making the ASF a better place > (e.g. > > > by > > > > > > > using Matomo an ASF-compliant tracking solution for all > > projects). > > > > > > > > > > > > > > Congratulations and welcome, Martijn! > > > > > > > > > > > > > > Cheers, > > > > > > > Timo Walther > > > > > > > (On behalf of the Apache Flink PMC) > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best > > > > > > > > > > > > ConradJam > > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats, Martijn! Best regards, Yuxia - 原始邮件 - 发件人: "Steven Wu" 收件人: "dev" 发送时间: 星期二, 2022年 9 月 13日 上午 5:33:47 主题: Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser Congrats, Martijn! On Mon, Sep 12, 2022 at 1:49 PM Alexander Fedulov wrote: > Congrats, Martijn! > > On Mon, Sep 12, 2022 at 10:06 AM Jing Ge wrote: > > > Congrats! > > > > On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang wrote: > > > > > Congrats! > > > > > > On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser < > martijnvis...@apache.org > > > > > > wrote: > > > > > > > Thank you all :) > > > > > > > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen < > jam.gz...@gmail.com > > >: > > > > > > > > > Congratulations, Martijn > > > > > > > > > > > > > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > I'm very happy to announce that Martijn Visser has joined the > Flink > > > > PMC! > > > > > > > > > > > > Martijn has helped the community in many different ways over the > > past > > > > > > months. Externalizing the connectors from the Flink repo to their > > own > > > > > > repository, continously updating dependencies, and performing > other > > > > > > project-wide refactorings. He is constantly coordinating > > > contributions, > > > > > > connecting stakeholders, finding committers for contributions, > > > driving > > > > > > release syncs, and helping in making the ASF a better place (e.g. > > by > > > > > > using Matomo an ASF-compliant tracking solution for all > projects). > > > > > > > > > > > > Congratulations and welcome, Martijn! > > > > > > > > > > > > Cheers, > > > > > > Timo Walther > > > > > > (On behalf of the Apache Flink PMC) > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best > > > > > > > > > > ConradJam > > > > > > > > > > > > > > >
Re: Recommended way to Enable SSL Flink Kubernetes Operator
Hi Biao I think this modify basic-example FlinkDeployment should load the existing keystore although I am not certain re-using the webhook keystore recommended. apiVersion: flink.apache.org/v1beta1 kind: FlinkDeployment metadata: name: basic-example spec: image: flink:1.15 flinkVersion: v1_15 flinkConfiguration: taskmanager.numberOfTaskSlots: "2" serviceAccount: flink jobManager: resource: memory: "2048m" cpu: 1 taskManager: resource: memory: "2048m" cpu: 1 podTemplate: apiVersion: v1 kind: Pod metadata: name: pod-template spec: containers: - name: flink-main-container volumeMounts: - mountPath: /certs name: keystore volumes: - name: keystore secret: defaultMode: 420 items: - key: keystore.p12 path: keystore.p12 secretName: webhook-server-cert job: jarURI: local:///opt/flink/examples/streaming/StateMachineExample.jar parallelism: 2 upgradeMode: stateless Verify with curl curl -v -k https://basic-example-rest:8081 * Trying 172.21.126.88:8081... * Connected to basic-example-rest (172.21.126.88) port 8081 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: /etc/ssl/certs * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 * ALPN, server did not agree to a protocol * Server certificate: * subject: CN=FlinkDeployment Validator * start date: Sep 12 17:38:37 2022 GMT * expire date: Dec 11 17:38:37 2022 GMT * issuer: CN=FlinkDeployment Validator * SSL certificate verify result: self signed certificate (18), continuing anyway. > GET / HTTP/1.1 > Host: basic-example-rest:8081 > User-Agent: curl/7.74.0 > Accept: */* From: Hao t Chang Date: Friday, September 9, 2022 at 11:10 AM To: dev@flink.apache.org Subject: [EXTERNAL] Re: Recommended way to Enable SSL Flink Kubernetes Operator Hi Biao thanks for the quick reply. The helm chart uses a standard Deployment to mount the keystore onto the webhook container using volumes/volumeMounts for the operator but it’s not clear to me how to mount the keystore using the FlinkDeployment CRD[2] for a Flink application.
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats, Martijn! On Mon, Sep 12, 2022 at 1:49 PM Alexander Fedulov wrote: > Congrats, Martijn! > > On Mon, Sep 12, 2022 at 10:06 AM Jing Ge wrote: > > > Congrats! > > > > On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang wrote: > > > > > Congrats! > > > > > > On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser < > martijnvis...@apache.org > > > > > > wrote: > > > > > > > Thank you all :) > > > > > > > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen < > jam.gz...@gmail.com > > >: > > > > > > > > > Congratulations, Martijn > > > > > > > > > > > > > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > I'm very happy to announce that Martijn Visser has joined the > Flink > > > > PMC! > > > > > > > > > > > > Martijn has helped the community in many different ways over the > > past > > > > > > months. Externalizing the connectors from the Flink repo to their > > own > > > > > > repository, continously updating dependencies, and performing > other > > > > > > project-wide refactorings. He is constantly coordinating > > > contributions, > > > > > > connecting stakeholders, finding committers for contributions, > > > driving > > > > > > release syncs, and helping in making the ASF a better place (e.g. > > by > > > > > > using Matomo an ASF-compliant tracking solution for all > projects). > > > > > > > > > > > > Congratulations and welcome, Martijn! > > > > > > > > > > > > Cheers, > > > > > > Timo Walther > > > > > > (On behalf of the Apache Flink PMC) > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best > > > > > > > > > > ConradJam > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats, Martijn! On Mon, Sep 12, 2022 at 10:06 AM Jing Ge wrote: > Congrats! > > On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang wrote: > > > Congrats! > > > > On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser > > > wrote: > > > > > Thank you all :) > > > > > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen >: > > > > > > > Congratulations, Martijn > > > > > > > > > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > > > > > Hi everyone, > > > > > > > > > > I'm very happy to announce that Martijn Visser has joined the Flink > > > PMC! > > > > > > > > > > Martijn has helped the community in many different ways over the > past > > > > > months. Externalizing the connectors from the Flink repo to their > own > > > > > repository, continously updating dependencies, and performing other > > > > > project-wide refactorings. He is constantly coordinating > > contributions, > > > > > connecting stakeholders, finding committers for contributions, > > driving > > > > > release syncs, and helping in making the ASF a better place (e.g. > by > > > > > using Matomo an ASF-compliant tracking solution for all projects). > > > > > > > > > > Congratulations and welcome, Martijn! > > > > > > > > > > Cheers, > > > > > Timo Walther > > > > > (On behalf of the Apache Flink PMC) > > > > > > > > > > > > > > > > > -- > > > > Best > > > > > > > > ConradJam > > > > > > > > > >
[jira] [Created] (FLINK-29269) Setup CI logging
Chesnay Schepler created FLINK-29269: Summary: Setup CI logging Key: FLINK-29269 URL: https://issues.apache.org/jira/browse/FLINK-29269 Project: Flink Issue Type: Sub-task Components: Connectors / ElasticSearch Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: elasticsearch-3.0.0 Logging isn't setup on CI, which breaks some tests that rely on certain message being logged. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29268) Sync 1.16.0 code to ES repo
Chesnay Schepler created FLINK-29268: Summary: Sync 1.16.0 code to ES repo Key: FLINK-29268 URL: https://issues.apache.org/jira/browse/FLINK-29268 Project: Flink Issue Type: Sub-task Components: Connectors / ElasticSearch Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: elasticsearch-3.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: HELP: New repository 'flink-connector-opensearch' under ASF organization
Thank you very much, Chesnay! Best Regards, Andriy Redko On Mon, Sep 12, 2022, 2:01 PM Chesnay Schepler wrote: > - created repo https://github.com/apache/flink-connector-opensearch > - Conectors/Opensearch component added to JIRA > - opensearch-1.0.0 version added to JIRA > > On 02/09/2022 19:43, Andriy Redko wrote: > > Hello Everyone, > > > > Kindly asking PMCs for help with creating new > 'flink-connector-opensearch' repository > > under ASF organization to host Apache Flink OpenSearch connector(s) [1], > [2], [3], [4], > > following the initiative to move connectors out of core. > > > > Thank you very much in advance. > > > > Best Regards, > > Andriy Redko > > > > > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors > > [2] https://www.mail-archive.com/dev@flink.apache.org/msg58911.html > > [3] https://lists.apache.org/thread/jls0vqc7jb84jp14j4jok1pqfgo2cl30 > > [4] https://lists.apache.org/thread/4bms24983g38q956rp8qmm4bpdo4361s > > > >
Re: HELP: New repository 'flink-connector-opensearch' under ASF organization
- created repo https://github.com/apache/flink-connector-opensearch - Conectors/Opensearch component added to JIRA - opensearch-1.0.0 version added to JIRA On 02/09/2022 19:43, Andriy Redko wrote: Hello Everyone, Kindly asking PMCs for help with creating new 'flink-connector-opensearch' repository under ASF organization to host Apache Flink OpenSearch connector(s) [1], [2], [3], [4], following the initiative to move connectors out of core. Thank you very much in advance. Best Regards, Andriy Redko [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors [2] https://www.mail-archive.com/dev@flink.apache.org/msg58911.html [3] https://lists.apache.org/thread/jls0vqc7jb84jp14j4jok1pqfgo2cl30 [4] https://lists.apache.org/thread/4bms24983g38q956rp8qmm4bpdo4361s
[jira] [Created] (FLINK-29267) Support external type systems in DDL
Timo Walther created FLINK-29267:
Summary: Support external type systems in DDL
Key: FLINK-29267
URL: https://issues.apache.org/jira/browse/FLINK-29267
Project: Flink
Issue Type: Improvement
Components: Connectors / JDBC, Formats (JSON, Avro, Parquet, ORC,
SequenceFile), Table SQL / Ecosystem
Reporter: Timo Walther
Assignee: Timo Walther
Many connectors and formats require supporting external data types. Postgres
users request UUID support, Avro users require enum support, etc.
FLINK-19869 implemented support for Postgres UUIDs poorly and event impacts
pipelines with regular strings.
The long-term solution should be user-defined types in Flink. This is however a
bigger effort that requires a FLIP and a bigger amount of resources.
As a mid-term solution, we should offer a consistent approach based on DDL
options that allows to define a mapping from Flink type system to the external
type system. I suggest the following:
{code}
CREATE TABLE MyTable (
...
) WITH(
'mapping.data-types' = ': '
)
{code}
The mapping defines a map from Flink data type to external data type. The
external data type should be string parsable. This works for most connectors
and formats (e.g. Avro schema string).
Examples:
{code}
CREATE TABLE MyTable (
regular_col STRING,
uuid_col STRING,
point_col ARRAY,
box_col ARRAY>
) WITH(
'mapping.data-types' = 'uuid_col: uuid, point_col: point, box_col: box'
)
{code}
We provide a table of supported mapping data types. E.g. the {{point}} type is
always maped to {{ARRAY}}. In general we choose a data type in Flink
that comes closest to the required functionality.
Future work:
In theory, we can also offer mapping of field names. It might be a requirement
that Flink's column name is different from the external system's one.
{code}
CREATE TABLE MyTable (
...
) WITH(
'mapping.names' = ': '
)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29266) Cleanup ES root pom
Chesnay Schepler created FLINK-29266: Summary: Cleanup ES root pom Key: FLINK-29266 URL: https://issues.apache.org/jira/browse/FLINK-29266 Project: Flink Issue Type: Technical Debt Components: Build System, Connectors / ElasticSearch Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: elasticsearch-3.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29265) Update ES version to 3.0-SNAPSHOT
Chesnay Schepler created FLINK-29265: Summary: Update ES version to 3.0-SNAPSHOT Key: FLINK-29265 URL: https://issues.apache.org/jira/browse/FLINK-29265 Project: Flink Issue Type: Technical Debt Components: Connectors / ElasticSearch Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: elasticsearch-3.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29264) Give ES parent module a proper artifact ID
Chesnay Schepler created FLINK-29264: Summary: Give ES parent module a proper artifact ID Key: FLINK-29264 URL: https://issues.apache.org/jira/browse/FLINK-29264 Project: Flink Issue Type: Improvement Components: Connectors / ElasticSearch Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: elasticsearch-3.0.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] FLIP-254 Redis Streams connector
Hello! There's quite a bit of existing code and it looks like there's interest and community willing to contribute to this connector with 2 implementations already in the flink-connector-redis repo[1]. There's a couple of points that should probably be fixed in the FLIP: some typos such as "provide at-least guarantees" and the initial version should not be 1.0.0 given that version 1.1.5 was already released in its previous incarnation[2]. In principle: +1 (non-binding) All my best, Ryan [1]: https://github.com/apache/flink-connector-redis/pulls [2]: https://mvnrepository.com/artifact/org.apache.flink/flink-connector-redis On Mon, Sep 12, 2022 at 10:20 AM Zheng Yu Chen wrote: > +1 (non-binding) > > Martijn Visser 于2022年9月12日周一 15:58写道： > > > Hi everyone, > > > > With no comments provided in the discussion thread, I'm opening a vote > > thread on FLIP-254: Redis Streams connector: > > > > FLIP: > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-254%3A+Redis+Streams+Connector > > > > > > The vote will be open for at least 72h. > > > > Best regards, > > > > Martijn > > https://twitter.com/MartijnVisser82 > > https://github.com/MartijnVisser > > >
[jira] [Created] (FLINK-29263) Remove ElasticSearch connector from apache/flink repo
Chesnay Schepler created FLINK-29263: Summary: Remove ElasticSearch connector from apache/flink repo Key: FLINK-29263 URL: https://issues.apache.org/jira/browse/FLINK-29263 Project: Flink Issue Type: Sub-task Components: Connectors / ElasticSearch Reporter: Chesnay Schepler Fix For: 1.17.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29262) Update documentation
Chesnay Schepler created FLINK-29262: Summary: Update documentation Key: FLINK-29262 URL: https://issues.apache.org/jira/browse/FLINK-29262 Project: Flink Issue Type: Sub-task Components: Documentation Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.16.0, 1.15.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29261) Consider using FAIL_ON_UNKNOWN_PROPERTIES in the Operator
Matyas Orhidi created FLINK-29261: - Summary: Consider using FAIL_ON_UNKNOWN_PROPERTIES in the Operator Key: FLINK-29261 URL: https://issues.apache.org/jira/browse/FLINK-29261 Project: Flink Issue Type: Bug Reporter: Matyas Orhidi The operator cannot be downgraded, once the CR specification is written to the `status` Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "mode" (class org.apache.flink.kubernetes.operator.crd.spec.FlinkDeploymentSpec), not marked as ignorable (12 known properties: "restartNonce", "imagePullPolicy", "ingress", "flinkConfiguration", "serviceAccount", "image", "job", "podTemplate", "jobManager", "logConfiguration", "flinkVersion", "taskManager"]) at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: org.apache.flink.kubernetes.operator.crd.spec.FlinkDeploymentSpec["mode"]) at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61) at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:1127) at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1989) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1700) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1678) at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:319) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:176) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322) at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4650) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2831) at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:3295) at org.apache.flink.kubernetes.operator.reconciler.ReconciliationUtils.deserializeSpecWithMeta(ReconciliationUtils.java:288) ... 18 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29260) Auto-wipe exclusion list after updating reference version
Chesnay Schepler created FLINK-29260: Summary: Auto-wipe exclusion list after updating reference version Key: FLINK-29260 URL: https://issues.apache.org/jira/browse/FLINK-29260 Project: Flink Issue Type: Sub-task Components: Release System Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.16.0, 1.15.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29259) Update release guide
Chesnay Schepler created FLINK-29259: Summary: Update release guide Key: FLINK-29259 URL: https://issues.apache.org/jira/browse/FLINK-29259 Project: Flink Issue Type: Sub-task Reporter: Chesnay Schepler Assignee: Chesnay Schepler New minor version: * wipe exclusions on master after release New patch version X.Y.Z: * verify no new exclusions were added since last release and/or confirm whether they were discussed anywhere * wipe exclusions on release-X.Y branch after release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29258) Add SAVEPOINT_ONLY upgrade mode
Maximilian Michels created FLINK-29258:
--
Summary: Add SAVEPOINT_ONLY upgrade mode
Key: FLINK-29258
URL: https://issues.apache.org/jira/browse/FLINK-29258
Project: Flink
Issue Type: New Feature
Components: Deployment / Kubernetes
Reporter: Maximilian Michels
Users may want to explicitly upgrade from a savepoint. the {{SAVEPOINT}}
upgrade mode cannot ensure this. When the savepoint creation fails, which might
be temporary, the operator falls back to using the latest checkpoint.
To prevent this from happening, we could add a SAVEPOINT_ONLY upgrade mode
which does not proceed when the savepoint creation fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29257) Kubernetes operator docs are vague about upgrade mode
Maximilian Michels created FLINK-29257: -- Summary: Kubernetes operator docs are vague about upgrade mode Key: FLINK-29257 URL: https://issues.apache.org/jira/browse/FLINK-29257 Project: Flink Issue Type: Improvement Components: Documentation, Kubernetes Operator Reporter: Maximilian Michels Assignee: Maximilian Michels Users are confused how the SAVEPOINT ugprade mode works. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29256) Docs for k8s operator webhook
Márton Balassi created FLINK-29256: -- Summary: Docs for k8s operator webhook Key: FLINK-29256 URL: https://issues.apache.org/jira/browse/FLINK-29256 Project: Flink Issue Type: Improvement Components: Documentation, Kubernetes Operator Affects Versions: kubernetes-operator-1.2.0 Reporter: Márton Balassi Assignee: Gabor Somogyi Fix For: kubernetes-operator-1.2.0 Please add an admission control section to the architecture page describing the webhook and call out the certificate renewal feature there explicitly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29255) FLIP-258 - Enforce binary compatibility in patch releases
Chesnay Schepler created FLINK-29255: Summary: FLIP-258 - Enforce binary compatibility in patch releases Key: FLINK-29255 URL: https://issues.apache.org/jira/browse/FLINK-29255 Project: Flink Issue Type: Improvement Components: Build System Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.16.0, 1.15.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29254) RpcGateway should have some form of Close() method
Chesnay Schepler created FLINK-29254: Summary: RpcGateway should have some form of Close() method Key: FLINK-29254 URL: https://issues.apache.org/jira/browse/FLINK-29254 Project: Flink Issue Type: Technical Debt Components: Runtime / RPC Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.17.0 In a simple client-server model the gateway constitutes the client, which will generally have to allocate some resources to communicate with the server. There is however currently no way for a user to close a gateway, hence these resources will generally leak (unless the underlying RPC implementation either magically fixes that somehow or doesn't allocate resources for clients in the first place). -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] FLIP-254 Redis Streams connector
+1 (non-binding) Martijn Visser 于2022年9月12日周一 15:58写道： > Hi everyone, > > With no comments provided in the discussion thread, I'm opening a vote > thread on FLIP-254: Redis Streams connector: > > FLIP: > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-254%3A+Redis+Streams+Connector > > > The vote will be open for at least 72h. > > Best regards, > > Martijn > https://twitter.com/MartijnVisser82 > https://github.com/MartijnVisser >
[RESULT][VOTE] FLIP-258 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
The voting period has elapsed and we have unanimously accepted the FLIP. +1 votes: * Konstantin (binding) * Xintong (binding) * Xinbgo (binding) * Martijn (binding) * Yu (binding) * Jingsong (binding) * Zhu (binding) * Danny (binding) * Thomas (binding) * Alexander (non-binding) On 08/09/2022 23:03, Alexander Fedulov wrote: +1 (non-binding) On Wed, Sep 7, 2022 at 9:28 PM Thomas Weise wrote: +1 On Wed, Sep 7, 2022 at 4:48 AM Danny Cranmer wrote: +1 On Wed, 7 Sept 2022, 07:32 Zhu Zhu, wrote: +1 Thanks, Zhu Jingsong Li 于2022年9月6日周二 19:49写道： +1 On Tue, Sep 6, 2022 at 7:11 PM Yu Li wrote: +1 Thanks for the efforts, Chesnay Best Regards, Yu On Tue, 6 Sept 2022 at 18:17, Martijn Visser < martijnvis...@apache.org wrote: +1 (binding) Op di 6 sep. 2022 om 11:59 schreef Xingbo Huang < hxbks...@gmail.com : Thanks Chesnay for driving this, +1 Best, Xingbo Xintong Song 于2022年9月6日周二 17:57写道： +1 Best, Xintong On Tue, Sep 6, 2022 at 5:55 PM Konstantin Knauf < kna...@apache.org> wrote: +1. Thanks, Chesnay. Am Di., 6. Sept. 2022 um 11:51 Uhr schrieb Chesnay Schepler < ches...@apache.org>: Since no one objected in the discuss thread, let's vote! FLIP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 The vote will be open for at least 72h. Regards, Chesnay -- https://twitter.com/snntrable https://github.com/knaufk
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats! On Mon, Sep 12, 2022 at 9:38 AM Daisy Tsang wrote: > Congrats! > > On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser > wrote: > > > Thank you all :) > > > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen : > > > > > Congratulations, Martijn > > > > > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > > > Hi everyone, > > > > > > > > I'm very happy to announce that Martijn Visser has joined the Flink > > PMC! > > > > > > > > Martijn has helped the community in many different ways over the past > > > > months. Externalizing the connectors from the Flink repo to their own > > > > repository, continously updating dependencies, and performing other > > > > project-wide refactorings. He is constantly coordinating > contributions, > > > > connecting stakeholders, finding committers for contributions, > driving > > > > release syncs, and helping in making the ASF a better place (e.g. by > > > > using Matomo an ASF-compliant tracking solution for all projects). > > > > > > > > Congratulations and welcome, Martijn! > > > > > > > > Cheers, > > > > Timo Walther > > > > (On behalf of the Apache Flink PMC) > > > > > > > > > > > > > -- > > > Best > > > > > > ConradJam > > > > > >
Re: [NEWS][DISCUSS] Akka moves to BSL licence
Hi, Thanks Chesnay for this clarifying blog post. I was off on Friday, it was merged while I was gone but LGTM. Best Etienne Le 08/09/2022 à 12:25, Chesnay Schepler a écrit : I have prepared a blogpost for the whole situation: https://github.com/apache/flink-web/pull/570 Any feedback/comments are highly appreciated; if you feel another question should be answered, or something in the the post shouldn't be said, don't hesitate to comment. On 07/09/2022 15:03, Etienne Chauchot wrote: Hi all, I'd like to share a concerning news. I've just read that Akka will move from ASFv2 license to BSLv1.1 (1) I guess this license is considered Category X (2) by the ASF and cannot be included in ASF projects. Let's discuss possible solutions. Best Etienne [1] https://www.lightbend.com/blog/why-we-are-changing-the-license-for-akka [2] https://www.apache.org/legal/resolved.html#category-x
[VOTE] FLIP-254 Redis Streams connector
Hi everyone, With no comments provided in the discussion thread, I'm opening a vote thread on FLIP-254: Redis Streams connector: FLIP: https://cwiki.apache.org/confluence/display/FLINK/FLIP-254%3A+Redis+Streams+Connector The vote will be open for at least 72h. Best regards, Martijn https://twitter.com/MartijnVisser82 https://github.com/MartijnVisser
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Congrats! On Mon, Sep 12, 2022 at 9:32 AM Martijn Visser wrote: > Thank you all :) > > Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen : > > > Congratulations, Martijn > > > > > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > > > Hi everyone, > > > > > > I'm very happy to announce that Martijn Visser has joined the Flink > PMC! > > > > > > Martijn has helped the community in many different ways over the past > > > months. Externalizing the connectors from the Flink repo to their own > > > repository, continously updating dependencies, and performing other > > > project-wide refactorings. He is constantly coordinating contributions, > > > connecting stakeholders, finding committers for contributions, driving > > > release syncs, and helping in making the ASF a better place (e.g. by > > > using Matomo an ASF-compliant tracking solution for all projects). > > > > > > Congratulations and welcome, Martijn! > > > > > > Cheers, > > > Timo Walther > > > (On behalf of the Apache Flink PMC) > > > > > > > > > -- > > Best > > > > ConradJam > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Martijn Visser
Thank you all :) Op zo 11 sep. 2022 om 13:58 schreef Zheng Yu Chen : > Congratulations, Martijn > > > > Timo Walther 于2022年9月9日周五 23:08写道： > > > Hi everyone, > > > > I'm very happy to announce that Martijn Visser has joined the Flink PMC! > > > > Martijn has helped the community in many different ways over the past > > months. Externalizing the connectors from the Flink repo to their own > > repository, continously updating dependencies, and performing other > > project-wide refactorings. He is constantly coordinating contributions, > > connecting stakeholders, finding committers for contributions, driving > > release syncs, and helping in making the ASF a better place (e.g. by > > using Matomo an ASF-compliant tracking solution for all projects). > > > > Congratulations and welcome, Martijn! > > > > Cheers, > > Timo Walther > > (On behalf of the Apache Flink PMC) > > > > > -- > Best > > ConradJam >
[jira] [Created] (FLINK-29253) DefaultJobmanagerRunnerRegistry#localCleanupAsync calls close instead of closeAsync
Matthias Pohl created FLINK-29253:
-
Summary: DefaultJobmanagerRunnerRegistry#localCleanupAsync calls
close instead of closeAsync
Key: FLINK-29253
URL: https://issues.apache.org/jira/browse/FLINK-29253
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.15.2, 1.16.0, 1.17.0
Reporter: Matthias Pohl
{{DefaultJobmanagerRunnerRegistry#localCleanupAsync}} is meant to be called
from the main thread. The current implementation calls {{close}} on the
{{JobManagerRunner}} instead of {{closeAsync}}. This results in a blocking call
on the {{Dispatcher}}'s main thread which we want to avoid.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
