hudi-bot opened a new issue, #14476: URL: https://github.com/apache/hudi/issues/14476
[https://github.com/vinothchandar/incubator-hudi/commit/dd578947ec1db9388038f0a1863a90b3761cd571] has some test code. Seems to work with appends also.. ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-95 - Type: New Feature --- ## Comments 04/Aug/19 15:34;semanticbeeng;https://apacheignite.readme.io/docs/durable-memory _"Ignite can be used as a caching layer (aka. data grid) above an existing 3rd party database: RDBMS, NoSQL, or HDFS."_ The goal of this is to make possible using Ignite FS / in-memory HDFS. A more general goal would be to abstract over HDSF persistence to allow multiple implementations 1. current 2. Ignite in-memory HDFS 3. HopsFS https://www.logicalclocks.com/millions-and-millions-of-files-deep-learning-at-scale-with-hopsfs/ https://www.usenix.org/conference/fast17/technical-sessions/presentation/niazi https://twitter.com/semanticbeeng/status/1144877062446039040 Even further, would be good to ease [#InMemoryComputing|https://twitter.com/hashtag/InMemoryComputing] and allow parallelization between persistence and computation with some form of "eventual consistency" at the end (and join between the two tasks. Thoughts? ;;; --- 05/Aug/19 16:04;vinoth;[~semanticbeeng] Hi.. the work pending here is to actually test out in a real-world cluster for stability and performance.. Functionality wise, Ignite can work end-end already IIUC ;;; -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
