[
https://issues.apache.org/jira/browse/APEXMALHAR-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580363#comment-16580363
]
Pramod Immaneni commented on APEXMALHAR-2565:
---------------------------------------------
The in-memory state is checkpointed to a persistent store, by default hdfs, so
no data should be lost. There are also efficient state management and storage
mechanisms available if there is a lot of data to hold in state. The
checkpointing is done periodically as the processing moves forward so you are
capturing a fairly recent snapshot of the state, to recover from in case of
failures. In the documentation and in the public domain you will also be able
to find some presentations and videos on more advanced topics like exactly once
processing.
> Creation of IMDG for data persistence as of now this is handled by In memory
> , tied to sticky session
> -----------------------------------------------------------------------------------------------------
>
> Key: APEXMALHAR-2565
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2565
> Project: Apache Apex Malhar
> Issue Type: Improvement
> Components: AppData
> Reporter: Mahesh
> Priority: Major
>
> The Apex Malhar has the In memory that is warmed up ( data loaded) to the In
> memory on the startup, that is Data is also lost when the Node goes down,
> this has to be handled with a cluster of IMDG, that can increase the
> performance of the processing and also Fault-tolerant, with the currentÂ
> in-memory Model and with sticky Node (session), if Node fails the same has to
> be restarted . Pls look into this if this is an improvement adds value or not
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
