[ https://issues.apache.org/jira/browse/GEODE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashvin resolved GEODE-10. ------------------------- Resolution: Fixed > HDFS Integration > ---------------- > > Key: GEODE-10 > URL: https://issues.apache.org/jira/browse/GEODE-10 > Project: Geode > Issue Type: New Feature > Components: hdfs > Reporter: Dan Smith > Assignee: Ashvin > Attachments: GEODE-HDFSPersistence-Draft-060715-2109-21516.pdf > > > Ability to persist data on HDFS had been under development for GemFire. It > was part of the latest code drop, GEODE-8. As part of this feature we are > proposing some changes to the HdfsStore management API (see attached doc for > details). > # The current API has nested configuration for compaction and async queue. > This nested structure forces user to execute multiple steps to manage a > store. It also does not seem to be consistent with other management APIs > # Some member names in current API are confusing > HDFS Integration: Geode as a transactional layer that microbatches data out > to Hadoop. This capability makes Geode a NoSQL store that can sit on top of > Hadoop and parallelize the process of moving data from the in memory tier > into Hadoop, making it very useful for capturing and processing fast data > while making it available for Hadoop jobs relatively quickly. The key > requirements being met here are > # Ingest data into HDFS parallely > # Cache bloom filters and allow fast lookups of individual elements > # Have programmable policies for deciding what stays in memory > # Roll files in HDFS > # Index data that is in memory > # Have expiration policies that allows the transactional set to decay out > older data > # Solution needs to support replicated and partitioned regions -- This message was sent by Atlassian JIRA (v6.3.4#6332)