Pramod Immaneni created APEXMALHAR-2017:
-------------------------------------------
Summary: Use pre checkpoint notification to optimize operator IO
Key: APEXMALHAR-2017
URL: https://issues.apache.org/jira/browse/APEXMALHAR-2017
Project: Apache Apex Malhar
Issue Type: Improvement
Reporter: Pramod Immaneni
Assignee: Pramod Immaneni
Currently many output operators enforce persistence of data on endWindow by
calling flush, hflush or equivalent calls. This was done to help recovery.
Doing this always ensures that the data corresponding to checkpoint state at
recovery is always present.
A recent addition to the engine lets the operators know about an impending
checkpoint just before it happens using a callback. Operators can now enforce
persistence of data one time in this in this callback instead of end of every
window. This results in better performance as data is not being frequently
written to persistent storage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
