[
https://issues.apache.org/jira/browse/GOBBLIN-1359?focusedWorklogId=543194&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-543194
]
ASF GitHub Bot logged work on GOBBLIN-1359:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 27/Jan/21 23:46
Start Date: 27/Jan/21 23:46
Worklog Time Spent: 10m
Work Description: autumnust commented on a change in pull request #3214:
URL: https://github.com/apache/incubator-gobblin/pull/3214#discussion_r565719362
##########
File path:
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/StreamModelTaskRunner.java
##########
@@ -86,7 +87,10 @@ protected void run() throws Exception {
// The cancel is not propagated to the extractor's record generator when
it has been turned into a hot Flowable
// by publish, so set the shutdownRequested flag on cancel to stop the
extractor
- Flowable streamWithShutdownOnCancel = connectableStream.doOnCancel(() ->
this.shutdownRequested.set(true));
+ Flowable streamWithShutdownOnCancel = connectableStream.doOnCancel(() -> {
+ this.shutdownRequested.set(true);
+ this.extractor.shutdown();
Review comment:
I don't quite remember how was the failure propagated when the slow
write happens, but feels like the right fix here is to add finally block in the
place where `extractor.shutdown()` is called in `Task` (e.g. line 514) after
catching failure, instead of forcing extractor closure here.
##########
File path:
gobblin-core-base/src/main/java/org/apache/gobblin/source/extractor/extract/FlushingExtractor.java
##########
@@ -298,6 +298,14 @@ protected void publishTaskOutput() throws IOException {
this.flushPublisher.get().publish(Collections.singletonList(workUnitState));
}
+ @Override
+ public void shutdown() {
Review comment:
What's the reason super.shutdown is not called, is this intentional ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 543194)
Time Spent: 3h 10m (was: 3h)
> Fix the error that IOException thrown by writer be swallowed by Reactivex
> --------------------------------------------------------------------------
>
> Key: GOBBLIN-1359
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1359
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> we have a bug in Reactivex code which swallows the exception and leaves the
> task hanging
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
