[
https://issues.apache.org/jira/browse/FLINK-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643392#comment-15643392
]
ASF GitHub Bot commented on FLINK-4391:
---------------------------------------
Github user bjlovegithub commented on the issue:
https://github.com/apache/flink/pull/2629
Maybe this is a solution. We will not think about changing the type of
`checkpointLock`, which is an `Object` ans quite efficient, and not change the
order of `broadcastBarriers` and `operator.snapshotState()`. By placing
**pause** `EmitterThread` codes in `StreamTask.performCheckpoint()`, like this:
` private boolean performCheckpoint(CheckpointMetaData
checkpointMetaData) throws Exception {
LOG.debug("Starting checkpoint {} on task {}",
checkpointMetaData.getCheckpointId(), getName());
synchronized (lock) {
if (isRunning) {
// stop working threads first.
for (StreamOperator operator :
operatorChain.getAllOperators()) {
if (operator instanceof
AsyncWaitOperator) {
operator.pauseEmitterThread();
}
}
// broadcast barriers after snapshot operators'
states.
operatorChain.broadcastCheckpointBarrier(
checkpointMetaData.getCheckpointId(), checkpointMetaData.getTimestamp()
);
checkpointState(checkpointMetaData);
return true;
} else {
return false;
}
}
}`
> Provide support for asynchronous operations over streams
> --------------------------------------------------------
>
> Key: FLINK-4391
> URL: https://issues.apache.org/jira/browse/FLINK-4391
> Project: Flink
> Issue Type: New Feature
> Components: DataStream API
> Reporter: Jamie Grier
> Assignee: david.wang
>
> Many Flink users need to do asynchronous processing driven by data from a
> DataStream. The classic example would be joining against an external
> database in order to enrich a stream with extra information.
> It would be nice to add general support for this type of operation in the
> Flink API. Ideally this could simply take the form of a new operator that
> manages async operations, keeps so many of them in flight, and then emits
> results to downstream operators as the async operations complete.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
