Hi, I am new to Flink community. I am interested in comparing Spark’s feature and performance vs. Spark. Does anyone know if there is any benchmark or test available for testing Spark performance on servers that has 32 plus cores and 256GB plus memory?
Thanks -yanping From: Fabian Hueske [mailto:[email protected]] Sent: Sunday, July 05, 2015 10:18 AM To: [email protected] Subject: Re: Benchmark results between Flink and Spark Thanks for sharing, Slim! I had a look at the report (except for two pages which were not available in the preview). It compares four different tasks on a setup with 4 rather small nodes (8 cores, 16GB memory). I could not find which versions of Flink and Spark were compared. The comparison tasks are: 1) WordCount for "batch processing" 2) KMeans for "Machine-learning" 3) PageRank for "Graph-processing" 4) Some kind of relational query (details probably in the two missing pages) Flink outperforms Spark in all tasks except WordCount. The results should not be taken too serious due to the small number of nodes and low number of different tasks (only one task characterizes a task category). K-Means and Wordcount are certainly not representative for the very diverse categories machine-learning and "batch processing". Same applies for relational processing which could be a single table aggregation or a cascade of a dozen joins. The results are very motivating though :-) I hope to see more independent benchmarks in the future. 2015-07-05 19:02 GMT+02:00 Stephan Ewen <[email protected]<mailto:[email protected]>>: Hi Slim! Thank you for the link. Unfortunately, I cannot access the contents. I always get a "connection closed" error. Anybody else experiences something similar? Stephan On Sun, Jul 5, 2015 at 6:37 PM, Slim Baltagi <[email protected]<mailto:[email protected]>> wrote: Hi Apache Flink outperforms Apache Spark in processing machine learning & graph algorithms and relational queries but not in batch processing! The results were published in the proceedings of the 18th International Conference, Business Information Systems 2015, Poznań, Poland, June 24-26, 2015. Thanks to our friend Google, Chapter 3: Evaluating New Approaches of Big Data Analytics Frameworks by Norman Spangenberg, Martin Roth and Bogdan Franczyk is available for preview at http://goo.gl/WocQci at pages 28-37. Enjoy! Slim Baltagi http://www.SparkBigData.com -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Benchmark-results-between-Flink-and-Spark-tp1940.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
