http://shop.oreilly.com/product/0636920047568.do has quite good information on it. For Kafka, you need to start with approximation that processing of each partition is a separate task that need to be executed, so you need to plan number of cores correspondingly.
Srinivas V at "Thu, 16 Apr 2020 22:49:15 +0530" wrote: SV> Hello, SV> Can someone point me to a good video or document which takes about performance tuning for structured streaming app? SV> I am looking especially for listening to Kafka topics say 5 topics each with 100 portions . SV> Trying to figure out best cluster size and number of executors and cores required. -- With best wishes, Alex Ott http://alexott.net/ Twitter: alexott_en (English), alexott (Russian) --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
