Hi, I am new to Akka streams, I am currently trying to implement a Source that takes continuous input from RocksDB. The idea is to use RocksDB as a queue from which the source will constantly pull data, perform some stream operations and finally store it into a database store like Cassandra or MongoDB.
My code looks like this Source.repeat(NotUsed.getInstance()). .map(readRocksDB()) .runWith(storeMessages(), ActorMaterializer.create(system)); private Source<byte[], NotUsed> getMessageSource(RocksDB db) { Source<byte[], NotUsed> s = Source.from(getValues(db)); return s; } private Flow<File, List<Document>, NotUsed> readRocksDB() { return Flow.of(File.class).flatMapConcat(f -> { RocksDB db = RocksDB.open(f.getAbsolutePath()); return this.getMessageSource(db) .mapAsync(numOfThreads, this::parseMessageBytes); }); } storeMessage() is responsible for storing the processed data into a different database (Cassandra for example). I am expecting a workload of 1 mil writes per second, so I am using RocksDB to store raw before processing instead of Akka mailbox. Since I am new, I would like to get a feed back on this approach. Thanks, Jerry -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.