I am running a simulation which generates a million records every second. I’m writing them to Kafka and reading them through Akka Streams. I’m performing a few aggregations on this data and writing the output back to Kafka.
The data contains a timestamp based on which the aggregations are grouped. Using the timestamps, I’m creating windows of data and performing aggregation on these windows. Since there are a million records each second, the aggregations are taking about 40 seconds for one million records. This is really slow because new data is being generated and written to Kafka every second. I referred this blog post for performing window aggregations: https://dvirgiln.github.io/akka-streams-windowing/ Is there any better way to perform these aggregations in lesser time(preferably less than one second) using Akka Streams? -- ***************************************************************************************************** ** New discussion forum: https://discuss.akka.io/ replacing akka-user google-group soon. ** This group will soon be put into read-only mode, and replaced by discuss.akka.io ** More details: https://akka.io/blog/news/2018/03/13/discuss.akka.io-announced ***************************************************************************************************** >>>>>>>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/akka-user/cc8f1634-76d2-4f31-a226-950c37a239cd%40googlegroups.com.
