[jira] [Created] (FLINK-36020) exception: Time should be non negative

2024-08-08 Thread siyuan zheng (Jira)
siyuan zheng created FLINK-36020: Summary: exception: Time should be non negative Key: FLINK-36020 URL: https://issues.apache.org/jira/browse/FLINK-36020 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-36019) flink_ha目录下的文件,可以通过配置自定义权限

2024-08-08 Thread yanggang (Jira)
yanggang created FLINK-36019: Summary: flink_ha目录下的文件,可以通过配置自定义权限 Key: FLINK-36019 URL: https://issues.apache.org/jira/browse/FLINK-36019 Project: Flink Issue Type: New Feature Affects Versio

[jira] [Created] (FLINK-36018) Support lazy scale down to avoid frequent rescaling

2024-08-08 Thread Rui Fan (Jira)
Rui Fan created FLINK-36018: --- Summary: Support lazy scale down to avoid frequent rescaling Key: FLINK-36018 URL: https://issues.apache.org/jira/browse/FLINK-36018 Project: Flink Issue Type: Improve

[jira] [Created] (FLINK-36017) Support Passing Metadata to Formats in KafkaDynamicSink

2024-08-08 Thread Kevin Lam (Jira)
Kevin Lam created FLINK-36017: - Summary: Support Passing Metadata to Formats in KafkaDynamicSink Key: FLINK-36017 URL: https://issues.apache.org/jira/browse/FLINK-36017 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-474: Store operator name and UID in state metadata

2024-08-08 Thread Gabor Somogyi
Hi Zakelly, Thanks for the feedback, let me elaborate on this. In short Databricks has created a much more user friendly solution[1] for state observability (based on Flink's state processor API) than what we have now. Up until now our state processor API was good enough but now we're lagging be

[jira] [Created] (FLINK-36016) Synchronize initialization time and clock usage

2024-08-08 Thread Zdenek Tison (Jira)
Zdenek Tison created FLINK-36016: Summary: Synchronize initialization time and clock usage Key: FLINK-36016 URL: https://issues.apache.org/jira/browse/FLINK-36016 Project: Flink Issue Type:

[jira] [Created] (FLINK-36015) Align rescale parameters

2024-08-08 Thread Zdenek Tison (Jira)
Zdenek Tison created FLINK-36015: Summary: Align rescale parameters Key: FLINK-36015 URL: https://issues.apache.org/jira/browse/FLINK-36015 Project: Flink Issue Type: Sub-task Compo

[jira] [Created] (FLINK-36014) Align the desired and sufficient resources definiton in Executing and WaitForResources states

2024-08-08 Thread Zdenek Tison (Jira)
Zdenek Tison created FLINK-36014: Summary: Align the desired and sufficient resources definiton in Executing and WaitForResources states Key: FLINK-36014 URL: https://issues.apache.org/jira/browse/FLINK-36014

[jira] [Created] (FLINK-36013) Introduce the transition from Restarting to CreatingExecutionGraph state

2024-08-08 Thread Zdenek Tison (Jira)
Zdenek Tison created FLINK-36013: Summary: Introduce the transition from Restarting to CreatingExecutionGraph state Key: FLINK-36013 URL: https://issues.apache.org/jira/browse/FLINK-36013 Project: Fli

[jira] [Created] (FLINK-36012) Integrate StateTransitionManager into WaitingForResources state

2024-08-08 Thread Zdenek Tison (Jira)
Zdenek Tison created FLINK-36012: Summary: Integrate StateTransitionManager into WaitingForResources state Key: FLINK-36012 URL: https://issues.apache.org/jira/browse/FLINK-36012 Project: Flink

[jira] [Created] (FLINK-36011) Generalize RescaleManager to become StateTransitionManager

2024-08-08 Thread Zdenek Tison (Jira)
Zdenek Tison created FLINK-36011: Summary: Generalize RescaleManager to become StateTransitionManager Key: FLINK-36011 URL: https://issues.apache.org/jira/browse/FLINK-36011 Project: Flink Is

[jira] [Created] (FLINK-36010) Duplicate Job Submission returns Succeeded on single execution even if Global state is FAILED

2024-08-08 Thread Ahmed Hamdy (Jira)
Ahmed Hamdy created FLINK-36010: --- Summary: Duplicate Job Submission returns Succeeded on single execution even if Global state is FAILED Key: FLINK-36010 URL: https://issues.apache.org/jira/browse/FLINK-36010

[RESULT][VOTE] FLIP-472: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

2024-08-08 Thread Zdenek Tison
Hi everyone, I'm delighted to announce that FLIP-472 [1] has been accepted. There were 5 votes in favor: - Rui Fan (binding) - Yuanfeng Hu (no-binding) - David Moravek (binding) - Matthias Pohl (binding) - Yuepeng Pan (no-binding) There were no votes against it. [1] https://cwiki.apache.org/con

[jira] [Created] (FLINK-36009) Change configs to use CondigOptions

2024-08-08 Thread Jira
João Boto created FLINK-36009: - Summary: Change configs to use CondigOptions Key: FLINK-36009 URL: https://issues.apache.org/jira/browse/FLINK-36009 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-474: Store operator name and UID in state metadata

2024-08-08 Thread Márton Balassi
Hi team, Thank you for the proposal, Gabor and the review, Zakelly. Let me provide more context on the intended outcome, let us agree on the desirability of that first and then we can come back to the technical details. The pain point that we observed with multiple users is that their operations

Re:Re:Re: [DISCUSS] FLIP-473: Introduce New SQL Operators Based on Asynchronous State APIs

2024-08-08 Thread Xuyang
Hi, everyone. I have updated the FLIP. Here are the newly added sections: In theory, the implementation of AsyncStateOperator and SyncStateOperator differs only in their state handling. Their state schemas, business logic, and others are the same. Therefore, within the same Flink version,

Re: [DISCUSS] FLIP-XXX Amazon SQS Source Connector

2024-08-08 Thread Saurabh Singh
Hi Ahmed, Yes, you're correct. Currently, we're utilizing the "record emitter" to send messages into the queue for deletion. However, for the actual deletion process, which is dependent on the checkpoints, we've been using the source reader class because it allows us to override the notifyCheckpoi

Re: [DISCUSS] FLIP-474: Store operator name and UID in state metadata

2024-08-08 Thread Zakelly Lan
Hi Gabor, Thanks for the proposal! However, I find it a little strange. Are you saying the user can set the operator uid but then doesn't know what they set when debugging? Otherwise, is the `OperatorIdentifier.forUid("my-uid")` feasible? I understand your point about potential cross-team work, bu

Flink Jackson Exception when using JsonDeserialisationSchema

2024-08-08 Thread Taher Koitawala
Hi All, I am reading from Kafka with Json schema and i get the following error message I am using flink 1.19 and tried the same on 1.18 objectNode is imported as : import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.node.ObjectNode; KafkaSource kafkaSource = KafkaSour

[jira] [Created] (FLINK-36008) Guidance for connector docs contributions

2024-08-08 Thread Lorenzo Nicora (Jira)
Lorenzo Nicora created FLINK-36008: -- Summary: Guidance for connector docs contributions Key: FLINK-36008 URL: https://issues.apache.org/jira/browse/FLINK-36008 Project: Flink Issue Type: Imp

Re: [DISCUSS] Release flink-connector-mongodb 2.0.0

2024-08-08 Thread Leonard Xu
Thanks Jiabao for kicking off this thread, +1 for the proposal. Feel free to ping me if you need help from PMC member side. Best, Leonard > 2024年8月8日 下午5:04,Jiabao Sun 写道: > > Hi, > > I would like to propose to create a release for flink-connector-mongodb > 2.0.0. > > From the last release

[DISCUSS] Release flink-connector-mongodb 2.0.0

2024-08-08 Thread Jiabao Sun
Hi, I would like to propose to create a release for flink-connector-mongodb 2.0.0. >From the last release (1.2.0), we mainly achieved: 1. Support upsert into sharded collections[1] 2. Support MongoDB 7.0[2] 3. Support fine-grained configuration to control filter push down[3] I can help in drivi

[jira] [Created] (FLINK-36007) cdc load and register factory in once search

2024-08-08 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-36007: - Summary: cdc load and register factory in once search Key: FLINK-36007 URL: https://issues.apache.org/jira/browse/FLINK-36007 Project: Flink Issue Type: Im

Re: [DISCUSS] Release flink-connector-pulsar 2.0.0

2024-08-08 Thread Jiabao Sun
Sorry, I wrote the wrong title. It should be the release of flink-connector-mongodb 2.0.0. This discussion thread is closed. I will start a new discussion thread. Please ignore this. Best, Jiabao On 2024/08/08 08:54:19 Jiabao Sun wrote: > Hi, > > I would like to propose to create a release

[DISCUSS] Release flink-connector-pulsar 2.0.0

2024-08-08 Thread Jiabao Sun
Hi, I would like to propose to create a release for flink-connector-mongodb 2.0.0. >From the last release (1.2.0), we mainly achieved: 1. Support upsert into sharded collections[1] 2. Support MongoDB 7.0[2] 3. Support fine-grained configuration to control filter push down[3] I can help in drivi

[jira] [Created] (FLINK-36006) Add documentations for Flink CDC CLI

2024-08-08 Thread yux (Jira)
yux created FLINK-36006: --- Summary: Add documentations for Flink CDC CLI Key: FLINK-36006 URL: https://issues.apache.org/jira/browse/FLINK-36006 Project: Flink Issue Type: Bug Components: Flin

Re:[VOTE] FLIP-472: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

2024-08-08 Thread Yuepeng Pan
Thanks for driving it !+1(non-binding) Best, Yuepeng Pan At 2024-08-05 15:11:04, "Zdenek Tison" wrote: >Hi everyone, > > >Thanks for all the feedback about FLIP-472: Aligning timeout logic in the >AdaptiveScheduler's WaitingForResources and Executing states [1]. The >discussion thread can be foun