I'm asking this question here because Rediscala uses Akka IO. So maybe someone here could provide some insight into what's going on.
I'm trying to run a simple test by putting 1 million keys-values into Redis (running on the same machine). For 100,000 keys it is really fast. However, the performance degrades a lot when I bump the number of keys to 1 million. The max. heap space is 12G and I'm running this on a Macbook pro. As you can see the network write drops significantly after sometime. Not sure what's going on here. Any help would be really appreciated. NOTE: The time measurements are just to get an estimate. I plan to use a micro benchmark library for my final measurements. I'm using the following versions: *"com.etaty.rediscala" %% "rediscala" % "1.4.0" scalaVersion := "2.11.4"* Here is the code. package redisbenchmark import akka.util.ByteString import scala.concurrent.{Future} import redis.RedisClient import java.util.UUID object RedisLocalPerf { def main(args:Array[String]) = { implicit val akkaSystem = akka.actor.ActorSystem() var numberRuns = 1000 //default to 100 var size = 1 if( args.length == 1 ) numberRuns = Integer.parseInt(args(0)) val s = """How to explain ZeroMQ? Some of us start by saying all the wonderful things it does. It's sockets on steroids. It's like mailboxes with routing. It's fast! Others try to share their moment of enlightenment, that zap-pow-kaboom satori paradigm-shift moment when it all became obvious. Things just become simpler. Complexity goes away. It opens the mind. Others try to explain by comparison. It's smaller, simpler, but still looks familiar. Personally, I like to remember why we made ZeroMQ at all, because that's most likely where you, the reader, still are today.How to explain ZeroMQ? Some of us start by saying all the wonderful things it does. It's sockets on steroids. It's like mailboxes with routing. It's fast! Others try to share their moment of enlightenment, that zap-pow-kaboom satori paradigm-shift moment when it all became obvious. Things just become simpler. Complexity goes away. It opens the mind. Others try to explain by comparison. It's smaller, simpler, but still looks familiar. Personally, I like to remember why we made ZeroMQ at all, because that's most likely where""" val msgSize = s.getBytes.size val redis = RedisClient() implicit val ec = redis.executionContext val futurePong = redis.ping() println("Ping sent!") futurePong.map(pong => { println(s"Redis replied with a $pong") }) val random = UUID.randomUUID().toString val start = System.currentTimeMillis() val result: Seq[Future[Boolean]] = for {i <- 1 to numberRuns} yield { redis.set(random + i.toString, ByteString(s)) } val res: Future[List[Boolean]] = Future.sequence(result.toList) val end = System.currentTimeMillis() val diff = (end - start) println(s"for msgSize $msgSize and numOfRuns [$numberRuns] time is $diff ms ") akkaSystem.shutdown() } } <https://lh5.googleusercontent.com/-GRKgNYGTkzo/VJUO-aIKjkI/AAAAAAAAvSY/9ofgk_acLOk/s1600/rediscala_1millionkeys.png> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.