Re: [VOTE] Creating an Apache Flink slack workspace

2022-05-18 Thread Peter Huang
+1 (non-binding) Best Regards Peter Huang On Wed, May 18, 2022 at 9:33 PM Leonard Xu wrote: > Thanks Xintong for driving this. > > +1 > > Best, > Leonard > > > 2022年5月19日 上午11:11,Zhou, Brian 写道: > > > > +1 (non-binding) Slack is a better place for code sharing and quick > discussion. > > >

Re: [VOTE] Creating an Apache Flink slack workspace

2022-05-18 Thread Leonard Xu
Thanks Xintong for driving this. +1 Best, Leonard > 2022年5月19日 上午11:11,Zhou, Brian 写道: > > +1 (non-binding) Slack is a better place for code sharing and quick > discussion. > > Regards, > Brian Zhou > > -Original Message- > From: Yun Tang > Sent: Thursday, May 19, 2022 10:32 AM

Re: [DISCUSS] Next Flink Kubernetes Operator release timeline

2022-05-18 Thread Gyula Fóra
Hi Thomas! Thank you for raising your concerns. I agree that we should document the compatibility guarantees that we expect to provide going forward. Since releasing 0.1 (v1alpha1) we added a great deal of new core features. This required some modification to the CR obviously but actually it

[jira] [Created] (FLINK-27690) Add Pulsar Source connector document

2022-05-18 Thread LuNng Wang (Jira)
LuNng Wang created FLINK-27690: -- Summary: Add Pulsar Source connector document Key: FLINK-27690 URL: https://issues.apache.org/jira/browse/FLINK-27690 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-27689) Pulsar Connector support PulsarSchema

2022-05-18 Thread LuNng Wang (Jira)
LuNng Wang created FLINK-27689: -- Summary: Pulsar Connector support PulsarSchema Key: FLINK-27689 URL: https://issues.apache.org/jira/browse/FLINK-27689 Project: Flink Issue Type: Improvement

RE: [VOTE] Creating an Apache Flink slack workspace

2022-05-18 Thread Zhou, Brian
+1 (non-binding) Slack is a better place for code sharing and quick discussion. Regards, Brian Zhou -Original Message- From: Yun Tang Sent: Thursday, May 19, 2022 10:32 AM To: dev Subject: Re: [VOTE] Creating an Apache Flink slack workspace [EXTERNAL EMAIL] Thanks Xintong for

Re: [VOTE] FLIP-229: Introduces Join Hint for Flink SQL Batch Job

2022-05-18 Thread Yun Tang
Thanks for driving, +1 (binding) Best Yun Tang From: Jark Wu Sent: Wednesday, May 18, 2022 23:09 To: dev Subject: Re: [VOTE] FLIP-229: Introduces Join Hint for Flink SQL Batch Job +1(binding) Best, Jark On Wed, 18 May 2022 at 14:18, Jingsong Li wrote: > +1

Re: [VOTE] Creating an Apache Flink slack workspace

2022-05-18 Thread Yun Tang
Thanks Xintong for driving this. +1 from my side. Best Yun Tang From: Zhu Zhu Sent: Wednesday, May 18, 2022 17:08 To: dev Subject: Re: [VOTE] Creating an Apache Flink slack workspace +1 (binding) Thanks, Zhu Timo Walther 于2022年5月18日周三 16:52写道: > > +1

Re:[ANNOUNCE] Call for Presentations for ApacheCon Asia 2022 streaming track

2022-05-18 Thread hamster
退订 At 2022-05-18 19:47:47, "Yu Li" wrote: >Hi everyone, > >ApacheCon Asia [1] will feature the Streaming track for the second year. >Please don't hesitate to submit your proposal if there is an interesting >project or Flink experience you would like to share with us! > >The conference will be

Re: [DISCUSS] Next Flink Kubernetes Operator release timeline

2022-05-18 Thread Thomas Weise
I think before we release 1.0, we need to define and document the compatibility guarantees. At the moment, the CR changes frequently and as was pointed out earlier, there isn't any automation to ensure changes are backward compatible. In addition, our documentation still refers to upgrade as a

[jira] [Created] (FLINK-27688) Pluggable backend for EventUtils

2022-05-18 Thread Jira
Márton Balassi created FLINK-27688: -- Summary: Pluggable backend for EventUtils Key: FLINK-27688 URL: https://issues.apache.org/jira/browse/FLINK-27688 Project: Flink Issue Type: New Feature

RE: Kafka Sink Key and Value Avro Schema Usage Issues

2022-05-18 Thread Ghiya, Jay (GE Healthcare)
Also forgot to attach the information regarding how did I convert the avro objects to bytes in the approach that I took with deprecated kafka producer. protected byte[] getValueBytes(Value value) { DatumWriter valWriter = new SpecificDatumWriter( Value.getSchema());

[jira] [Created] (FLINK-27687) SpanningWrapper shouldn't assume temp folder exists

2022-05-18 Thread Jira
Gaël Renoux created FLINK-27687: --- Summary: SpanningWrapper shouldn't assume temp folder exists Key: FLINK-27687 URL: https://issues.apache.org/jira/browse/FLINK-27687 Project: Flink Issue

Kafka Sink Key and Value Avro Schema Usage Issues

2022-05-18 Thread Ghiya, Jay (GE Healthcare)
Hi Team, This is regarding Flink Kafka Sink. We have a usecase where we have headers and both key and value as Avro Schema. Below is the expectation in terms of intuitiveness for avro kafka key and value: KafkaSink.>builder() .setBootstrapServers(cloudkafkaBrokerAPI)

[jira] [Created] (FLINK-27686) Only patch status when the status actually changed

2022-05-18 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-27686: -- Summary: Only patch status when the status actually changed Key: FLINK-27686 URL: https://issues.apache.org/jira/browse/FLINK-27686 Project: Flink Issue Type:

Re: [VOTE] FLIP-229: Introduces Join Hint for Flink SQL Batch Job

2022-05-18 Thread Jark Wu
+1(binding) Best, Jark On Wed, 18 May 2022 at 14:18, Jingsong Li wrote: > +1 Thanks for driving. > > Best, > Jingsong > > On Wed, May 18, 2022 at 1:33 PM godfrey he wrote: > > > Thanks Xuyang for driving this, +1(binding) > > > > Best, > > Godfrey > > > > Xuyang 于2022年5月17日周二 10:21写道: > > >

Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-05-18 Thread Jark Wu
Hi Godfrey, Regarding Table API for CTAS, "Table#createTableAs(tablePath)" seems a little strange to me. Usually, the parameter after AS should be the query, but the query is in front of AS. I slightly prefer a method on TableEnvironment besides "createTable" (i.e. a special createTable with

Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-05-18 Thread Jark Wu
Hi Mang, Thanks for proposing this, CTAS is a very important API for batch users. I think the key problem of this FLIP is the ACID semantics of the CTAS operation. We care most about two parts of the semantics: 1) Atomicity: the created table should be rolled back if the write is failed. 2)

[jira] [Created] (FLINK-27685) Add scale subresource

2022-05-18 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-27685: -- Summary: Add scale subresource Key: FLINK-27685 URL: https://issues.apache.org/jira/browse/FLINK-27685 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-27684) FlinkKafkaConsumerBase could record partitions offset when GROUP_OFFSETS

2022-05-18 Thread SilkyAlex (Jira)
SilkyAlex created FLINK-27684: - Summary: FlinkKafkaConsumerBase could record partitions offset when GROUP_OFFSETS Key: FLINK-27684 URL: https://issues.apache.org/jira/browse/FLINK-27684 Project: Flink

[ANNOUNCE] Call for Presentations for ApacheCon Asia 2022 streaming track

2022-05-18 Thread Yu Li
Hi everyone, ApacheCon Asia [1] will feature the Streaming track for the second year. Please don't hesitate to submit your proposal if there is an interesting project or Flink experience you would like to share with us! The conference will be online (virtual) and the talks will be pre-recorded.

[jira] [Created] (FLINK-27683) Insert into (column1, column2) Values(.....) can't work with sql Hints

2022-05-18 Thread Xin Yang (Jira)
Xin Yang created FLINK-27683: Summary: Insert into (column1, column2) Values(.) can't work with sql Hints Key: FLINK-27683 URL: https://issues.apache.org/jira/browse/FLINK-27683 Project: Flink

[jira] [Created] (FLINK-27682) [JUnit5 Migration] Migrate ComparatorTestBase to Junit5

2022-05-18 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-27682: --- Summary: [JUnit5 Migration] Migrate ComparatorTestBase to Junit5 Key: FLINK-27682 URL: https://issues.apache.org/jira/browse/FLINK-27682 Project: Flink

Re: [DISCUSS] FLIP-222: Support full query lifecycle statements in SQL client

2022-05-18 Thread Jark Wu
Hi Paul, 1) SHOW QUERIES +1 to add finished time, but it would be better to call it "end_time" to keep aligned with names in Web UI. 2) DROP QUERY I think we shouldn't throw exceptions for batch jobs, otherwise, how to stop batch queries? At present, I don't think "DROP" is a suitable keyword

Re: [VOTE] Creating an Apache Flink slack workspace

2022-05-18 Thread Zhu Zhu
+1 (binding) Thanks, Zhu Timo Walther 于2022年5月18日周三 16:52写道: > > +1 (binding) > > Thanks, > Timo > > > On 17.05.22 20:44, Gyula Fóra wrote: > > +1 (binding) > > > > On Tue, 17 May 2022 at 19:52, Yufei Zhang wrote: > > > >> +1 (nonbinding) > >> > >> On Tue, May 17, 2022 at 5:29 PM Márton

[jira] [Created] (FLINK-27681) Improve the availability of Flink when the RocksDB file is corrupted.

2022-05-18 Thread ming li (Jira)
ming li created FLINK-27681: --- Summary: Improve the availability of Flink when the RocksDB file is corrupted. Key: FLINK-27681 URL: https://issues.apache.org/jira/browse/FLINK-27681 Project: Flink

[jira] [Created] (FLINK-27680) Disable PulsarSinkITCase on JDK 11

2022-05-18 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-27680: -- Summary: Disable PulsarSinkITCase on JDK 11 Key: FLINK-27680 URL: https://issues.apache.org/jira/browse/FLINK-27680 Project: Flink Issue Type: Technical

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-18 Thread Qingsheng Ren
Hi Jark and Alexander, Thanks for your comments! I’m also OK to introduce common table options. I prefer to introduce a new DefaultLookupCacheOptions class for holding these option definitions because putting all options into FactoryUtil would make it a bit ”crowded” and not well categorized.

Could not copy native libraries - Permission denied

2022-05-18 Thread Zain Haider Nemati
Hi, We are using flink version 1.13 with a kafka source and a kinesis sink with a parallelism of 3. On submitting the job I get this error Could not copy native binaries to temp directory /tmp/amazon-kinesis-producer-native-binaries Followed by permission denied even though all the permissions

Re: [VOTE] Creating an Apache Flink slack workspace

2022-05-18 Thread Timo Walther
+1 (binding) Thanks, Timo On 17.05.22 20:44, Gyula Fóra wrote: +1 (binding) On Tue, 17 May 2022 at 19:52, Yufei Zhang wrote: +1 (nonbinding) On Tue, May 17, 2022 at 5:29 PM Márton Balassi wrote: +1 (binding) On Tue, May 17, 2022 at 11:00 AM Jingsong Li wrote: Thank Xintong for

[jira] [Created] (FLINK-27679) Support append-only table for log store.

2022-05-18 Thread Zheng Hu (Jira)
Zheng Hu created FLINK-27679: Summary: Support append-only table for log store. Key: FLINK-27679 URL: https://issues.apache.org/jira/browse/FLINK-27679 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27678) Support append-only table for file store.

2022-05-18 Thread Zheng Hu (Jira)
Zheng Hu created FLINK-27678: Summary: Support append-only table for file store. Key: FLINK-27678 URL: https://issues.apache.org/jira/browse/FLINK-27678 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27677) Kubernetes reuse rest.bind-port, but do not support a range of ports

2022-05-18 Thread tartarus (Jira)
tartarus created FLINK-27677: Summary: Kubernetes reuse rest.bind-port, but do not support a range of ports Key: FLINK-27677 URL: https://issues.apache.org/jira/browse/FLINK-27677 Project: Flink

[jira] [Created] (FLINK-27676) Output records from on_timer are behind the triggering watermark in PyFlink

2022-05-18 Thread Juntao Hu (Jira)
Juntao Hu created FLINK-27676: - Summary: Output records from on_timer are behind the triggering watermark in PyFlink Key: FLINK-27676 URL: https://issues.apache.org/jira/browse/FLINK-27676 Project: Flink

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-18 Thread Jark Wu
Hi Alex, 1) retry logic I think we can extract some common retry logic into utilities, e.g. RetryUtils#tryTimes(times, call). This seems independent of this FLIP and can be reused by DataStream users. Maybe we can open an issue to discuss this and where to put it. 2) cache ConfigOptions I'm fine

Re: [DISCUSS] FLIP-222: Support full query lifecycle statements in SQL client

2022-05-18 Thread Paul Lam
Hi Godfrey, Thanks a lot for your inputs! 'SHOW QUERIES' lists all jobs in the cluster, no limit on APIs (DataStream or SQL) or clients (SQL client or CLI). Under the hook, it’s based on ClusterClient#listJobs, the same with Flink CLI. I think it’s okay to have non-SQL jobs listed in SQL

Re: [VOTE] FLIP-229: Introduces Join Hint for Flink SQL Batch Job

2022-05-18 Thread Jingsong Li
+1 Thanks for driving. Best, Jingsong On Wed, May 18, 2022 at 1:33 PM godfrey he wrote: > Thanks Xuyang for driving this, +1(binding) > > Best, > Godfrey > > Xuyang 于2022年5月17日周二 10:21写道: > > > > Hi, everyone. > > Thanks for your feedback for FLIP-229: Introduces Join Hint for Flink > SQL