Re: [VOTE] FLIP-243: Dedicated Opensearch connectors

2022-08-18 Thread Matthias Pohl
I'm gonna revive this one here. I guess, having a dedicated OpenSearch connector makes sense considering that the ElasticSearch project drifts away... Thanks for looking into it Andriy. +1 (binding) On 2022/07/28 08:18:20 Martijn Visser wrote: > Hi Andy, > > For future threads, it's best to st

Re: [VOTE] Apache Flink Table Store 0.2.0, release candidate #2

2022-08-18 Thread Nicholas Jiang
Hi all! +1 for the release (non-binding). I've verified the jar with SQL client and listed the check items as follows: * Compiled the sources and built the source distribution - PASSED * Ran through Quick Start Guide - PASSED * Checked Spark 2.3.4&3.3.0 reader and catalog with table store jar -

[jira] [Created] (FLINK-29029) Fix the bug of CSV format doesn't work in Thread Mode

2022-08-18 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-29029: Summary: Fix the bug of CSV format doesn't work in Thread Mode Key: FLINK-29029 URL: https://issues.apache.org/jira/browse/FLINK-29029 Project: Flink Issue T

[jira] [Created] (FLINK-29030) Consider printing a warning if a Pojo contains a generic type

2022-08-18 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-29030: Summary: Consider printing a warning if a Pojo contains a generic type Key: FLINK-29030 URL: https://issues.apache.org/jira/browse/FLINK-29030 Project: Flink

[jira] [Created] (FLINK-29031) FlinkKinesisConsumerTest.testSourceSynchronization failed with AssertionFailedError

2022-08-18 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-29031: Summary: FlinkKinesisConsumerTest.testSourceSynchronization failed with AssertionFailedError Key: FLINK-29031 URL: https://issues.apache.org/jira/browse/FLINK-29031 P

[jira] [Created] (FLINK-29032) Kafka Consume from timestamp catch exception : Caused by: java.lang.IllegalArgumentException: Invalid negative offset

2022-08-18 Thread HunterHunter (Jira)
HunterHunter created FLINK-29032: Summary: Kafka Consume from timestamp catch exception : Caused by: java.lang.IllegalArgumentException: Invalid negative offset Key: FLINK-29032 URL: https://issues.apache.org/jira

Re: [DISCUSS] FLIP-256 Support Job Dynamic Parameter With Flink Rest Api

2022-08-18 Thread Danny Cranmer
+1 thanks for driving this FLIP. We actually have an internally forked equivalent of this, which is on our list to try to upstream. Your proposal would *almost* work "off the shelf" for us. We have an inverted order or priority: - Rest API / Flink CLI > User Code > Cluster Config This is because

Re: [ANNOUNCE] New Apache Flink Committer - Junhan Yang

2022-08-18 Thread Timo Walther
Congratulations and welcome to the committer team :-) Regards, Timo On 18.08.22 07:19, Lijie Wang wrote: Congratulations, Junhan! Best, Lijie Leonard Xu 于2022年8月18日周四 11:31写道: Congratulations, Junhan! Best, 2022年8月18日 上午11:27,Zhipeng Zhang 写道: Congratulations, Junhan! Xintong Song 于

Re: [Discuss] Let's Session Cluster JobManager take a breather (FLIP-257: Flink JobManager Process Split)

2022-08-18 Thread Zheng Yu Chen
Thank you for your interest in this program. This proposal only takes effect for Session Cluster Mode, and will not take effect for Application Mode. I would like to answer some of your previous questions about patterns and practical cases first, and then discuss some implementation details later.

Re: [ANNOUNCE] New Apache Flink Committer - Junhan Yang

2022-08-18 Thread Jark Wu
Congrats and welcome Junhan! Cheers, Jark > 2022年8月18日 17:59,Timo Walther 写道: > > Congratulations and welcome to the committer team :-) > > Regards, > Timo > > On 18.08.22 07:19, Lijie Wang wrote: >> Congratulations, Junhan! >> Best, >> Lijie >> Leonard Xu 于2022年8月18日周四 11:31写道: >>> Congratu

Re: [Discuss] Let's Session Cluster JobManager take a breather (FLIP-257: Flink JobManager Process Split)

2022-08-18 Thread Zheng Yu Chen
Zheng Yu Chen 18:00 (2分钟前) 发送至 dev Thank you for your interest in this program. This proposal only takes effect for Session Cluster Mode, and will not take effect for Application Mode. I would like to answer some of your previous questions about patterns and practical cases first, and then discuss

Re: [Discuss] Let's Session Cluster JobManager take a breather (FLIP-257: Flink JobManager Process Split)

2022-08-18 Thread Zheng Yu Chen
You're right, this does add to the complexity of their communication coordination I can understand what you mean is similar to ngnix, load balancing to different SessionClusters in the front, rather than one more component. In fact, I have tried this myself, and it seems to solve the problem of hig

[jira] [Created] (FLINK-29033) Add timestamp to operator resource listener context

2022-08-18 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-29033: -- Summary: Add timestamp to operator resource listener context Key: FLINK-29033 URL: https://issues.apache.org/jira/browse/FLINK-29033 Project: Flink Issue Type: I

[jira] [Created] (FLINK-29034) HYBRID_FULL result partition type is not yet reConsumable

2022-08-18 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-29034: -- Summary: HYBRID_FULL result partition type is not yet reConsumable Key: FLINK-29034 URL: https://issues.apache.org/jira/browse/FLINK-29034 Project: Flink Issue T

Re: [DISCUSS] FLIP-256 Support Job Dynamic Parameter With Flink Rest Api

2022-08-18 Thread Teoh, Hong
+1 to this FLIP. This is very useful for teams building a Flink platform to run jobs from an external user +1 on Danny's comment on adding a configuration to allow inverted order of overrides. However, might it be better to include an "override" toggle in the REST API itself? That way we can

Re: [DISCUSS] FLIP-256 Support Job Dynamic Parameter With Flink Rest Api

2022-08-18 Thread Gyula Fóra
+1 for the proposal. @Hong : I feel that the inverted override flag should be a cluster setting and not something the user can override at will. I fear that this might defeat the purpose of the feature itself. So I think we should add a config for this but not expose it on the rest api Gyula On

Re: [DISCUSS] FLIP-256 Support Job Dynamic Parameter With Flink Rest Api

2022-08-18 Thread Teoh, Hong
@Gyula: I was thinking of the use case where the managed platform also handles the job submission. Setup #1 External user : Creating jar Managed platform : Submit job Managed platform : Manage Flink cluster As compared to Setup #2 External user : Creating

[jira] [Created] (FLINK-29035) ExpressionReducer does not work with jar resources

2022-08-18 Thread Timo Walther (Jira)
Timo Walther created FLINK-29035: Summary: ExpressionReducer does not work with jar resources Key: FLINK-29035 URL: https://issues.apache.org/jira/browse/FLINK-29035 Project: Flink Issue Type

[jira] [Created] (FLINK-29036) Code examples on the Data Sources page have errors

2022-08-18 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-29036: --- Summary: Code examples on the Data Sources page have errors Key: FLINK-29036 URL: https://issues.apache.org/jira/browse/FLINK-29036 Project: Flink Issue Type:

[jira] [Created] (FLINK-29037) Flink Histogram not emitting sum while using Metrics Reporter

2022-08-18 Thread Sarang Vadali (Jira)
Sarang Vadali created FLINK-29037: - Summary: Flink Histogram not emitting sum while using Metrics Reporter Key: FLINK-29037 URL: https://issues.apache.org/jira/browse/FLINK-29037 Project: Flink

Re: Flink running same task on different Task Manager

2022-08-18 Thread Great Info
Kindly help with this, I got stuck -> If so, I think you can set Task1 and Task2 to the same parallelism and set them in the same slot sharing group. In this way, Task1 and Task2 will be deployed into the same slot(That is, the same task manager). *Updating task details * *Task1- Source some stati

Re: [ANNOUNCE] New Apache Flink Committer - Junhan Yang

2022-08-18 Thread Jing Ge
Congrats Junhan! Best regards, Jing On Thu, Aug 18, 2022 at 12:05 PM Jark Wu wrote: > Congrats and welcome Junhan! > > Cheers, > Jark > > > 2022年8月18日 17:59,Timo Walther 写道: > > > > Congratulations and welcome to the committer team :-) > > > > Regards, > > Timo > > > > On 18.08.22 07:19, Lijie

Re: [ANNOUNCE] New Apache Flink Committer - Lijie Wang

2022-08-18 Thread Jing Ge
Congrats Lijie! Best regards, Jing On Thu, Aug 18, 2022 at 8:40 AM Terry Wang wrote: > Congratulations, Lijie! > > On Thu, Aug 18, 2022 at 11:31 AM Leonard Xu wrote: > > > Congratulations, Lijie! > > > > Best, > > Leonard > > > > > 2022年8月18日 上午11:26,Zhipeng Zhang 写道: > > > > > > Congratulati

Re: [DISCUSS] FLIP-256 Support Job Dynamic Parameter With Flink Rest Api

2022-08-18 Thread Jing Ge
+1 for the FLIP. I like this FLIP. It is very useful. I'd like to ask one more question: " - The internal parameters passed in by the user contains the external parameter. At this point, we can see that the user configures the same two parameters in the *external parameter* and *interna

[jira] [Created] (FLINK-29038) AsyncWaitOperatorTest.testProcessingTimeRepeatedCompleteOrderedWithRetry failed with AssertionError

2022-08-18 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-29038: Summary: AsyncWaitOperatorTest.testProcessingTimeRepeatedCompleteOrderedWithRetry failed with AssertionError Key: FLINK-29038 URL: https://issues.apache.org/jira/browse/FLINK-290

[jira] [Created] (FLINK-29039) RowData produced by LineBytesInputFormat is reused, but DeserializationSchemaAdapter#Reader only shallow copies produced data, thus result will always be the last row va

2022-08-18 Thread Marco A. Villalobos (Jira)
Marco A. Villalobos created FLINK-29039: --- Summary: RowData produced by LineBytesInputFormat is reused, but DeserializationSchemaAdapter#Reader only shallow copies produced data, thus result will always be the last row value

Re: [ANNOUNCE] New Apache Flink Committer - Junhan Yang

2022-08-18 Thread Guowei Ma
Congratulations, Junhan! Best, Guowei On Fri, Aug 19, 2022 at 6:01 AM Jing Ge wrote: > Congrats Junhan! > > Best regards, > Jing > > On Thu, Aug 18, 2022 at 12:05 PM Jark Wu wrote: > > > Congrats and welcome Junhan! > > > > Cheers, > > Jark > > > > > 2022年8月18日 17:59,Timo Walther 写道: > > > >

Re: Flink running same task on different Task Manager

2022-08-18 Thread Lijie Wang
Hi Great, -> Will these methods work? I think it will not work. It can control that the slots are evenly distributed on the TM, but cannot control the correspondence between tasks and slots. For example, the Flink cluster has 2 TMs, each TM has 4 slots(so total 8 slots in this cluster), and your j

[jira] [Created] (FLINK-29040) When using the JDBC Catalog, the custom cannot be applied because it is fixed in the code

2022-08-18 Thread Zhimin Geng (Jira)
Zhimin Geng created FLINK-29040: --- Summary: When using the JDBC Catalog, the custom cannot be applied because it is fixed in the code Key: FLINK-29040 URL: https://issues.apache.org/jira/browse/FLINK-29040