Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-04-19 Thread via GitHub
tweise merged PR #23777: URL: https://github.com/apache/flink/pull/23777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-04-19 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-2066761675 CI is green: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58998=results -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-04-18 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-2064985125 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-04-18 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-2064139948 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-01-26 Thread via GitHub
leonardBang commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1911749157 > @leonardBang , thanks for sharing your concerns, I understand where they are coming from. Here are my thoughts: > > * This is probably the trickiest source to migrate, since

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-01-25 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1910235989 @leonardBang , thanks for sharing your concerns, I understand where they are coming from. Here are my thoughts: - This is probably the trickiest source to migrate, since it

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-01-18 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1899072559 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-01-18 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1898812128 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2024-01-16 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1453506424 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/DoubleEmittingSourceReaderWithCheckpointsInBetween.java: ## @@

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-12-03 Thread via GitHub
tweise commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1837588962 LGTM from my side, based on a quick review after @mxm had already covered the details. Regarding complexity: FLIP-27 sources are without doubt more complex for the source

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-12-03 Thread via GitHub
tweise commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1413195035 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/DataGenerators.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-12-03 Thread via GitHub
tweise commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1413186514 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/DoubleEmittingSourceReaderWithCheckpointsInBetween.java: ## @@ -0,0

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-28 Thread via GitHub
mxm commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1829727020 LGTM. I would appreciate just another set of eyes before merging. Do you think you could have a look @zentol? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-24 Thread via GitHub
mxm commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1404209068 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/DataGenerators.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1824713681 > We could then keep all the existing code and just wrap it. The execution model is fundamentally different - a while loop that runs directly in the function vs an async loop that

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403579962 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/SourceReaderWithCheckpointsLatch.java: ## @@ -0,0 +1,158 @@ +/* +

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1824683968 There are some things that can probably be done, but not in the scope of this PR. You can see how much complexity of the new Source API is already hidden behind the

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
mxm commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403558590 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/SourceReaderWithCheckpointsLatch.java: ## @@ -0,0 +1,158 @@ +/* + *

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on PR #23777: URL: https://github.com/apache/flink/pull/23777#issuecomment-1824456923 @mxm thanks! Addressed your comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403378092 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/functions/IndexLookupGeneratorFunction.java: ## @@ -0,0 +1,180 @@ +/* + *

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403399379 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/SourceReaderWithCheckpointsLatch.java: ## @@ -0,0 +1,158 @@ +/* +

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403389661 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/functions/IndexLookupGeneratorFunction.java: ## @@ -0,0 +1,180 @@ +/* + *

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403385649 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/SourceReaderWithCheckpointsLatch.java: ## @@ -0,0 +1,158 @@ +/* +

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403381199 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/SourceReaderWithCheckpointsLatch.java: ## @@ -0,0 +1,158 @@ +/* +

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
afedulov commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403378092 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/functions/IndexLookupGeneratorFunction.java: ## @@ -0,0 +1,180 @@ +/* + *

Re: [PR] [FLINK-28048][connectors] Introduce Source API alternative to FiniteTestSource [flink]

2023-11-23 Thread via GitHub
mxm commented on code in PR #23777: URL: https://github.com/apache/flink/pull/23777#discussion_r1403292650 ## flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/functions/IndexLookupGeneratorFunction.java: ## @@ -0,0 +1,180 @@ +/* + *