.com bit1...@163.com]
*Enviado el:* jueves, 16 de abril de 2015 10:58
*Para:* González Salgado, Miquel; user
*Asunto:* Re: Streaming problems running 24x7
From your description, looks like the data processing speed is far behind
the data receiving speed
Could you try to increase the core
: bit1...@163.com mailto:bit1...@163.com [mailto:bit1...@163.com
mailto:bit1...@163.com]
Enviado el: jueves, 16 de abril de 2015 10:58
Para: González Salgado, Miquel; user
Asunto: Re: Streaming problems running 24x7
From your description, looks like the data processing speed is far behind
Asunto: Re: Streaming problems running 24x7
I used to hit this issue when my processing time exceeds the batch duration.
Here's a few workarounds:
- Use storage level MEMORY_AND_DISK
- Enable WAL and check pointing
Above two will slow down things a little bit.
If you want low latency, what you
()
ssc.awaitTermination()
}
}
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Streaming-problems-running-24x7-tp22518.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
)
})
}
}
ssc.start()
ssc.awaitTermination()
}
}
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Streaming-problems-running-24x7-tp22518.html
Sent from the Apache Spark User List mailing list archive at Nabble.com