[
https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349824#comment-15349824
]
ASF GitHub Bot commented on APEXCORE-222:
-----------------------------------------
Github user sandeshh commented on a diff in the pull request:
https://github.com/apache/apex-core/pull/350#discussion_r68493910
--- Diff:
engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java ---
@@ -769,7 +769,11 @@ public void
processHeartbeatResponse(ContainerHeartbeatResponse rsp)
}
if (rsp.committedWindowId != lastCommittedWindowId) {
+
lastCommittedWindowId = rsp.committedWindowId;
+
+ bufferServer.purge(lastCommittedWindowId);
--- End diff --
Working on that, here are the current finding.
1. Previous purge kept one extra window before the committed window.
2. Couldn't find the history for that in GitHub. That change is present
atleast from 2013.
Next steps
Will conduct few test scenario to see why we need a window before the
committed window in the buffer server.
> Delegate Buffer Server purge to StreamingContainer
> --------------------------------------------------
>
> Key: APEXCORE-222
> URL: https://issues.apache.org/jira/browse/APEXCORE-222
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Thomas Weise
> Assignee: Sandesh
>
> Currently the purge requests are sent to the buffer servers from the app
> master. This interaction exists parallel to the heartbeat protocol. Instead,
> the committed window ID that is propagated through the heartbeat response can
> be used in StreamingContainer to initiate the purge with the local buffer
> server, similar to how the committed callback on the operator occurs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
