Spark Streaming unable to handle production Kafka load

2014-09-24 Thread maddenpj
that feels excessive to me as currently we're processing this data load on a single node mapreduce cluster, an m3.xlarge (4 cores, 15 GB memory). Thanks,Patrick -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-unable-to-handle-production

Re: Spark Streaming unable to handle production Kafka load

2014-09-24 Thread maddenpj
this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-unable-to-handle-production-Kafka-load-tp15077p15080.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: Spark Streaming unable to handle production Kafka load

2014-09-24 Thread maddenpj
the connection pool? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-unable-to-handle-production-Kafka-load-tp15077p15081.html Sent from the Apache Spark User List mailing list archive at Nabble.com