We have a requirement where large amount of data (key/value) is kept in memory for a long time (few hours).
I am in need of a scalable operator which can be backed by a database/filesystem but provides this functionality in a fault-tolerant manner. So I wanted to see what option I have in Malhar library to achieve this and the only option I could find is - com.datatorrent.lib.db.cache.AbstractDBLookupCacheBasedOperator. The above expects a BackupStore but there is only one such implementation of it - JDBCLookupCacheBackedOperator and I don't know whether that is fault-tolerant. Do we have other options in Malhar? It will be good to have an operator that buckets tuples and load/offload buckets in memory in Malhar. Please recommend alternatives. Thanks, Chandni
