This is an automated email from the ASF dual-hosted git repository. sewen pushed a change to branch release-1.11 in repository https://gitbox.apache.org/repos/asf/flink.git.
from bc9f0fc [FLINK-16753][checkpointing] Use CheckpointException to wrap exceptions thrown from AsyncCheckpointRunnable (#14072) new c54b12b [backport] Backport some core changes that were part of the FLIP-27 File Source commit new b0bfe8b [backport] Relevant test mock changes from "[FLINK-18907][test] Refactor MockSourceReader" new 21283ee [backport] Relevant test mock changes from "[FLINK-18907][task] Add test coverage for watermarks with chained sources" new c7d2b4f [backport] Relevant test mock changes from "[FLINK-18907][task] Fix and add test coverage for watermarks gauges with chained sources" new bb234ca [backport] Relevant test mock changes from "[FLINK-18907][test] Add stream status forwarding test for chained sources" new 9c558bc [refactor][tests] Move some source test utils from flink-connector-base to flink-core new 0f7ff28 [FLINK-19457][core] Add a number sequence generating source for the FLIP-27 source API. new 99e9a90 [FLINK-19492][core] Consolidate Source Events between Source API and Split Reader API new e4e0f23 [FLINK-19448][connector base] Explicitly check for un-expected condition that would leave an inconsistent state new 253ffd2 [FLINK-19427][FLINK-19489][tests] Fix test conditions for 'SplitFetcherTest.testNotifiesWhenGoingIdleConcurrent()' new 1131e05 [FLINK-18044][Connectors/Common] Add the subtask index information to the SourceReaderContext. (#12647) new dffc124 [hotfix][runtime] Remove commented-out annotation in SourceOperator new 37ccdbe [FLINK-18820] Emit MAX_WATERMARK at the end in SourceOperator new 1be08c1 [hotfix] Replace finally block with JUnit After method in SourceOperatorTest. new 83e3613 [hotfix] Only close the SourceReader and EventTimeLogic in the SourceOperator if they are not null. new 4d413df [hotfix] Add a waitUntil() method to the CommonTestUtils. new 9dce8f0 Backport of "[FLINK-19698][connector/common] Let SplitEnumerator/SourceReader/SourceCoordinator implement CheckpointListener." new 2d07471 [FLINK-19698][connector/common] Add a close() method to the SplitReader. new 216f469 [FLINK-19384][core] Add common permissive exception signatures to all methods of Source. new 2b4d69d [build] Add exclusions for added default methods in SourceReader and SplitEnumerator interfaces. new ae23ab6 [refactor][DataStream API] Make DataStreamUtils.collect() methods more flexible. The 21 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../SingleThreadMultiplexSourceReaderBase.java | 63 ++++- .../base/source/reader/SourceReaderBase.java | 11 +- .../base/source/reader/fetcher/SplitFetcher.java | 10 +- .../source/reader/splitreader/SplitReader.java | 7 + .../base/source/reader/SourceReaderBaseTest.java | 16 +- .../base/source/reader/SourceReaderTestBase.java | 2 +- .../reader/fetcher/SplitFetcherManagerTest.java | 22 ++ .../source/reader/fetcher/SplitFetcherTest.java | 63 ++--- .../base/source/reader/mocks/MockSourceReader.java | 5 + .../source/reader/mocks/MockSplitEnumerator.java | 2 +- .../base/source/reader/mocks/MockSplitReader.java | 3 + .../source/reader/mocks/TestingSplitReader.java | 20 ++ flink-core/pom.xml | 5 +- .../apache/flink/api/connector/source/Source.java | 16 +- .../flink/api/connector/source/SourceReader.java | 19 +- .../api/connector/source/SourceReaderContext.java | 5 + .../api/connector/source/SplitEnumerator.java | 17 +- .../connector}/source/event/NoMoreSplitsEvent.java | 2 +- .../connector}/source/event/RequestSplitEvent.java | 15 +- .../connector/source/lib/NumberSequenceSource.java | 253 +++++++++++++++++++++ .../source/lib/util/IteratorSourceEnumerator.java | 90 ++++++++ .../source/lib/util/IteratorSourceReader.java | 148 ++++++++++++ .../source/lib/util/IteratorSourceSplit.java | 34 ++- .../source/lib/NumberSequenceSourceTest.java | 91 ++++++++ .../api/connector/source/mocks/MockSource.java | 12 +- .../connector/source/mocks/MockSourceReader.java | 98 ++++++-- .../source/mocks/MockSplitEnumerator.java | 21 +- .../source}/mocks/TestingReaderContext.java | 11 +- .../source}/mocks/TestingReaderOutput.java | 2 +- .../runtime/checkpoint/CheckpointCoordinator.java | 7 +- .../OperatorCoordinatorCheckpointContext.java | 19 +- .../coordination/OperatorCoordinator.java | 42 ++-- .../coordination/OperatorCoordinatorHolder.java | 22 +- .../RecreateOnResetOperatorCoordinator.java | 13 +- .../source/coordinator/SourceCoordinator.java | 24 +- .../coordinator/SourceCoordinatorProvider.java | 2 +- .../checkpoint/CheckpointCoordinatorTest.java | 53 +++++ .../CheckpointCoordinatorTestingUtils.java | 19 +- .../CoordinatorEventsExactlyOnceITCase.java | 2 +- .../coordination/MockOperatorCoordinator.java | 2 +- .../OperatorCoordinatorHolderTest.java | 2 +- .../RecreateOnResetOperatorCoordinatorTest.java | 3 +- .../coordination/TestingOperatorCoordinator.java | 2 +- .../coordinator/SourceCoordinatorProviderTest.java | 2 +- .../source/coordinator/SourceCoordinatorTest.java | 12 +- .../coordinator/SourceCoordinatorTestBase.java | 5 +- .../streaming/api/datastream/DataStreamUtils.java | 136 ++++++++++- .../streaming/api/operators/SourceOperator.java | 55 ++++- .../api/operators/SourceOperatorFactory.java | 10 +- .../collect/CollectSinkOperatorCoordinator.java | 2 +- .../runtime/tasks/SourceOperatorStreamTask.java | 8 +- .../api/operators/SourceOperatorTest.java | 156 ++++++++----- .../source/SourceOperatorEventTimeTest.java | 14 +- .../operators/source/TestingSourceOperator.java | 13 ++ .../tasks/SourceOperatorStreamTaskTest.java | 57 +++-- .../flink/core/testutils/CommonTestUtils.java | 29 +++ .../source/lib/IteratorSourcesITCase.java | 90 ++++++++ 57 files changed, 1590 insertions(+), 274 deletions(-) rename {flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base => flink-core/src/main/java/org/apache/flink/api/connector}/source/event/NoMoreSplitsEvent.java (96%) rename {flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base => flink-core/src/main/java/org/apache/flink/api/connector}/source/event/RequestSplitEvent.java (83%) create mode 100644 flink-core/src/main/java/org/apache/flink/api/connector/source/lib/NumberSequenceSource.java create mode 100644 flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceEnumerator.java create mode 100644 flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceReader.java copy flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingSourceSplit.java => flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceSplit.java (55%) create mode 100644 flink-core/src/test/java/org/apache/flink/api/connector/source/lib/NumberSequenceSourceTest.java rename {flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader => flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderContext.java (93%) rename {flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader => flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderOutput.java (96%) create mode 100644 flink-tests/src/test/java/org/apache/flink/api/connector/source/lib/IteratorSourcesITCase.java