Hello Todd,
Thank you for your suggestion! I have first tried increasing the Driver
memory to 2G and it worked without any problems, but I will also test with
the parameters and values you've shared.
Kind regards,
Emre Sevinç
http://www.bigindustries.be/
On Fri, Feb 20, 2015 at 3:25 PM, Todd
Hello,
We are building a Spark Streaming application that listens to a directory
on HDFS, and uses the SolrJ library to send newly detected files to a Solr
server. When we put 10.000 files to the directory it is listening to, it
starts to process them by sending the files to our Solr server but
Hi Emre,
Have you tried adjusting these:
.set(spark.akka.frameSize, 500).set(spark.akka.askTimeout,
30).set(spark.core.connection.ack.wait.timeout, 600)
-Todd
On Fri, Feb 20, 2015 at 8:14 AM, Emre Sevinc emre.sev...@gmail.com wrote:
Hello,
We are building a Spark Streaming application that
