[GitHub] [flink] zhijiangW commented on a change in pull request #8242: [FLINK-6227][network] Introduce the DataConsumptionException for downstream task failure
zhijiangW commented on a change in pull request #8242: [FLINK-6227][network]
Introduce the DataConsumptionException for downstream task failure
URL: https://github.com/apache/flink/pull/8242#discussion_r281005899
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/LocalInputChannel.java
##
@@ -127,7 +128,7 @@ void requestSubpartition(int subpartitionIndex) throws
IOException, InterruptedE
if (increaseBackoff()) {
Review comment:
The `IOException` should be wrapped into `DataConsumptionException` here,
not only for `PartitionNotFound`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #8242: [FLINK-6227][network] Introduce the DataConsumptionException for downstream task failure
zhijiangW commented on a change in pull request #8242: [FLINK-6227][network]
Introduce the DataConsumptionException for downstream task failure
URL: https://github.com/apache/flink/pull/8242#discussion_r281005734
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/LocalInputChannelTest.java
##
@@ -389,25 +401,80 @@ public void testGetNextAfterPartitionReleased() throws
Exception {
try {
channel.getNextBuffer();
fail("Did not throw expected CancelTaskException");
- } catch (CancelTaskException ignored) {
+ } catch (DataConsumptionException ignored) {
}
channel.releaseAllResources();
assertFalse(channel.getNextBuffer().isPresent());
}
+ /**
+* Tests the {@link PartitionNotFoundException} is thrown during
requesting partition and
+* wrapped into {@link DataConsumptionException}.
+*/
+ @Test
+ public void testDataConsumptionExceptionDuringPartitionRequest() throws
Exception {
+ Tuple2 backoff = new Tuple2<>(0, 0);
+
+ ResultPartitionManager partitionManager = new
ResultPartitionManager();
+
+ ResultPartitionID partitionId = new ResultPartitionID();
+
+ LocalInputChannel ch = createLocalInputChannel(
+ InputChannelTestUtils.createSingleInputGate(1),
partitionId, partitionManager, backoff);
+
+ try {
+ ch.requestSubpartition(0);
+ fail("Did not throw expected DataConsumptionException");
+ } catch (IOException ex) {
+ assertTrue(ExceptionUtils.findThrowable(ex,
PartitionNotFoundException.class).isPresent());
+ verifyDataConsumptionException(partitionId, ex);
+ }
+ }
+
+ /**
+* Tests the {@link DataConsumptionException} is wrapped when {@link
ResultSubpartitionView#getNextBuffer()}
+* throws an exception.
+*/
+ @Test
+ public void testDataConsumptionExceptionDuringGetNextBuffer() throws
Exception {
+ Tuple2 backoff = new Tuple2<>(0, 0);
+
+ ResultSubpartitionView view =
mock(ResultSubpartitionView.class);
+ when(view.getNextBuffer()).thenThrow(new IOException("Expected
test exception"));
Review comment:
yes
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #8242: [FLINK-6227][network] Introduce the DataConsumptionException for downstream task failure
zhijiangW commented on a change in pull request #8242: [FLINK-6227][network]
Introduce the DataConsumptionException for downstream task failure
URL: https://github.com/apache/flink/pull/8242#discussion_r281005469
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java
##
@@ -162,10 +164,13 @@ void assignExclusiveSegments(List
segments) {
public void requestSubpartition(int subpartitionIndex) throws
IOException, InterruptedException {
if (partitionRequestClient == null) {
// Create a client and request the partition
- partitionRequestClient = connectionManager
- .createPartitionRequestClient(connectionId);
+ try {
+ partitionRequestClient =
connectionManager.createPartitionRequestClient(connectionId);
+ } catch (RemoteTransportException ex) {
Review comment:
It should check `IOException` here and only wrap for
`RemoteTransportException` instance.
For the case of `LocalTransportException` it should not need to restart
producer side.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
