Hi, I'm testing Beam on Spark. I use spark example code WordCount processing 1G data file, cost 1 minutes. However, I use Beam example code WordCount processing the same file, cost 30minutes. My Spark parameter is : --deploy-mode client --executor-memory 1g --num-executors 1 --driver-memory 1g My Spark version is 2.3.1, Beam version is 2.5 Is there any optimization method? Thank you.
- How to optimize the performance of Beam on Spark 段丁瑞
- Re: How to optimize the performance of Beam on ... Robert Bradshaw
- Re: How to optimize the performance of Beam... Tim Robertson
- Re: How to optimize the performance of Beam on ... Jean-Baptiste Onofré
- Re: How to optimize the performance of Beam... Jean-Baptiste Onofré
- Re: How to optimize the performance of ... Jean-Baptiste Onofré
- Re: Re: How to optimize the perform... 段丁瑞
- Re: Re: How to optimize the pe... Tim Robertson
- Re: Re: How to optimize th... 段丁瑞
- Re: Re: How to optimiz... Robert Bradshaw
- Re: Re: How to optimiz... Tim Robertson