Take a look at: https://github.com/apache/incubator-blur/blob/master/blur-kvs
https://github.com/apache/incubator-blur/blob/master/blur-kvs/src/main/java/org/apache/blur/kvs/HdfsKeyValueStore.java https://github.com/apache/incubator-blur/blob/master/blur-kvs/src/test/java/org/apache/blur/kvs/HdfsKeyValueStoreTest.java The HdfsKeyValueStore took the place of the WAL in Blur. It's an inmemory cache that needs to be purged (and written to disk) from time to time. Let me know if you have questions. Aaron On Thu, Dec 15, 2016 at 7:22 AM, Ravikumar Govindarajan < [email protected]> wrote: > I remember seeing Blur writing transaction logs (WAL) in HDFS but is > removed as of today... > > Was re-implementing WAL-log for a particular use-case in Blur. Would be > great if community experienced any issues during WAL-writes in HDFS & could > share them here... > > I was looking at corner cases where WAL files could go corrupt. I guess > there should be many scenarios but just off the top of my head > > - Program crashing on OOM/Seg-fault, with open WAL streams to HDFS > - Name-Node, Data-Node differing on length of written block-data > - Data-Node(s) failing to ack sync calls correctly > > Any insights is much appreciated > > -- > Ravi >
