Also, what version of streams is this? There is no reason why the stream version could not produce 6 figure numbers, so something else is going on. Have you profiled it?
-Endre On Fri, Apr 1, 2016 at 1:12 PM, Viktor Klang <viktor.kl...@gmail.com> wrote: > Hi Greg, > > I don't know why the performance is low in your example, but there are > several problematic aspects of how you're measuring. > > For any benchmark on the JVM, plaese use JMH > <http://openjdk.java.net/projects/code-tools/jmh/> (for sbt projects, see > sbt-jmh <https://github.com/ktoso/sbt-jmh>) > > On Thu, Mar 31, 2016 at 10:44 PM, tigerfoot <gzol...@gmail.com> wrote: > >> Hello, >> >> I'm using a simple consumer pulled right out of the examples in the >> github repo: >> >> count = 0 >> val graph = GraphDSL.create(Consumer[Array[Byte], String](provider)) >> { implicit b => kafka => >> import GraphDSL.Implicits._ >> type In = ConsumerRecord[Array[Byte], String] >> val show = Flow[In].map{ i => count += 1; if(i.value=="done") >> println(s"[$id] time 2 ($count): "+(System.currentTimeMillis() - now)); i } >> >> kafka.messages ~> show ~> Consumer.record2commit ~> kafka.commit >> >> SourceShape(kafka.confirmation) >> } >> now = System.currentTimeMillis() >> val control = Source.fromGraph(graph) >> .to(shutdownAsOnComplete) >> .run() >> >> This works fine, but its really slow. When I pre-populate a topic with >> 1000 messages it took 2.2 sec to digest them. >> When I dispensed with the Akka stream logic and just used a trivial while >> loop with consumer.poll() and consumer.comitSync >> I got eye-watering performance--6-figure messages/sec. >> >> To rule out Akka streams I timed a "hello world" stream, and again got a >> very fast throughput number. >> >> My suspicion is that something in reactive-kafka's ManualCommitConsumer class >> is causing the slowness but I'm not sure. >> >> Any ideas appreciated. >> Greg >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Cheers, > √ > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
