This is an automated email from the ASF dual-hosted git repository. sewen pushed a change to branch release-1.11 in repository https://gitbox.apache.org/repos/asf/flink.git.
from c24185d [hotfix] Adjust japicmp exclusion to permit adding methods to SourceReaderContext and SplitEnumeratorContext interfaces new 9190335 [backport] Backport some core changes that were part of the FLIP-27 File Source commit new ecc6a8a [backport] Relevant test mock changes from "[FLINK-18907][test] Refactor MockSourceReader" new f79a3a7 [backport] Relevant test mock changes from "[FLINK-18907][task] Add test coverage for watermarks with chained sources" new cd3ffa5 [backport] Relevant test mock changes from "[FLINK-18907][task] Fix and add test coverage for watermarks gauges with chained sources" new 242ced9 [backport] Relevant test mock changes from "[FLINK-18907][test] Add stream status forwarding test for chained sources" new 7916ad7 [refactor][tests] Move some source test utils from flink-connector-base to flink-core new 09ea814 [FLINK-19457][core] Add a number sequence generating source for the FLIP-27 source API. new 301046a [FLINK-19492][core] Consolidate Source Events between Source API and Split Reader API new cdc0721 [FLINK-19448][connector base] Explicitly check for un-expected condition that would leave an inconsistent state new dd2ac7a [FLINK-19427][FLINK-19489][tests] Fix test conditions for 'SplitFetcherTest.testNotifiesWhenGoingIdleConcurrent()' new 5b2cc0e [FLINK-18044][Connectors/Common] Add the subtask index information to the SourceReaderContext. (#12647) new 1f823bd [hotfix][runtime] Remove commented-out annotation in SourceOperator new b3ea6da [FLINK-18820] Emit MAX_WATERMARK at the end in SourceOperator new 8ad5da9 [hotfix] Replace finally block with JUnit After method in SourceOperatorTest. new 3463da5 [hotfix] Only close the SourceReader and EventTimeLogic in the SourceOperator if they are not null. new 5787a70 [hotfix] Add a waitUntil() method to the CommonTestUtils. new 5f63e34 Backport of "[FLINK-19698][connector/common] Let SplitEnumerator/SourceReader/SourceCoordinator implement CheckpointListener." new 2a77589 [FLINK-19698][connector/common] Add a close() method to the SplitReader. new abe4c4b [FLINK-19384][core] Add common permissive exception signatures to all methods of Source. new 9cb2419 [build] Add exclusions for added default methods in SourceReader and SplitEnumerator interfaces. new 1a8ff89 [refactor][DataStream API] Make DataStreamUtils.collect() methods more flexible. The 21 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../SingleThreadMultiplexSourceReaderBase.java | 63 ++++- .../base/source/reader/SourceReaderBase.java | 11 +- .../base/source/reader/fetcher/SplitFetcher.java | 10 +- .../source/reader/splitreader/SplitReader.java | 7 + .../base/source/reader/SourceReaderBaseTest.java | 16 +- .../base/source/reader/SourceReaderTestBase.java | 2 +- .../reader/fetcher/SplitFetcherManagerTest.java | 22 ++ .../source/reader/fetcher/SplitFetcherTest.java | 63 ++--- .../base/source/reader/mocks/MockSourceReader.java | 5 + .../source/reader/mocks/MockSplitEnumerator.java | 2 +- .../base/source/reader/mocks/MockSplitReader.java | 3 + .../source/reader/mocks/TestingSplitReader.java | 20 ++ flink-core/pom.xml | 5 +- .../apache/flink/api/connector/source/Source.java | 16 +- .../flink/api/connector/source/SourceReader.java | 19 +- .../api/connector/source/SourceReaderContext.java | 5 + .../api/connector/source/SplitEnumerator.java | 17 +- .../connector}/source/event/NoMoreSplitsEvent.java | 2 +- .../connector}/source/event/RequestSplitEvent.java | 15 +- .../connector/source/lib/NumberSequenceSource.java | 253 +++++++++++++++++++++ .../source/lib/util/IteratorSourceEnumerator.java | 90 ++++++++ .../source/lib/util/IteratorSourceReader.java | 148 ++++++++++++ .../source/lib/util/IteratorSourceSplit.java | 34 ++- .../source/lib/NumberSequenceSourceTest.java | 91 ++++++++ .../api/connector/source/mocks/MockSource.java | 12 +- .../connector/source/mocks/MockSourceReader.java | 98 ++++++-- .../source/mocks/MockSplitEnumerator.java | 21 +- .../source}/mocks/TestingReaderContext.java | 11 +- .../source}/mocks/TestingReaderOutput.java | 2 +- .../runtime/checkpoint/CheckpointCoordinator.java | 7 +- .../OperatorCoordinatorCheckpointContext.java | 19 +- .../coordination/OperatorCoordinator.java | 42 ++-- .../coordination/OperatorCoordinatorHolder.java | 22 +- .../RecreateOnResetOperatorCoordinator.java | 13 +- .../source/coordinator/SourceCoordinator.java | 24 +- .../coordinator/SourceCoordinatorProvider.java | 2 +- .../checkpoint/CheckpointCoordinatorTest.java | 53 +++++ .../CheckpointCoordinatorTestingUtils.java | 19 +- .../CoordinatorEventsExactlyOnceITCase.java | 2 +- .../coordination/MockOperatorCoordinator.java | 2 +- .../OperatorCoordinatorHolderTest.java | 2 +- .../RecreateOnResetOperatorCoordinatorTest.java | 3 +- .../coordination/TestingOperatorCoordinator.java | 2 +- .../coordinator/SourceCoordinatorProviderTest.java | 2 +- .../source/coordinator/SourceCoordinatorTest.java | 12 +- .../coordinator/SourceCoordinatorTestBase.java | 5 +- .../streaming/api/datastream/DataStreamUtils.java | 136 ++++++++++- .../streaming/api/operators/SourceOperator.java | 55 ++++- .../api/operators/SourceOperatorFactory.java | 10 +- .../collect/CollectSinkOperatorCoordinator.java | 2 +- .../runtime/tasks/SourceOperatorStreamTask.java | 8 +- .../api/operators/SourceOperatorTest.java | 156 ++++++++----- .../source/SourceOperatorEventTimeTest.java | 14 +- .../operators/source/TestingSourceOperator.java | 13 ++ .../tasks/SourceOperatorStreamTaskTest.java | 57 +++-- .../flink/core/testutils/CommonTestUtils.java | 29 +++ .../source/lib/IteratorSourcesITCase.java | 90 ++++++++ 57 files changed, 1590 insertions(+), 274 deletions(-) rename {flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base => flink-core/src/main/java/org/apache/flink/api/connector}/source/event/NoMoreSplitsEvent.java (96%) rename {flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base => flink-core/src/main/java/org/apache/flink/api/connector}/source/event/RequestSplitEvent.java (83%) create mode 100644 flink-core/src/main/java/org/apache/flink/api/connector/source/lib/NumberSequenceSource.java create mode 100644 flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceEnumerator.java create mode 100644 flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceReader.java copy flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingSourceSplit.java => flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceSplit.java (55%) create mode 100644 flink-core/src/test/java/org/apache/flink/api/connector/source/lib/NumberSequenceSourceTest.java rename {flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader => flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderContext.java (93%) rename {flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader => flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderOutput.java (96%) create mode 100644 flink-tests/src/test/java/org/apache/flink/api/connector/source/lib/IteratorSourcesITCase.java