Re: TCP/IP speedup

2015-08-02 Thread Michael Segel
This may seem like a silly question… but in following Mark’s link, the presentation talks about the TPC-DS benchmark. Here’s my question… what benchmark results? If you go over to the TPC.org http://tpc.org/ website they have no TPC-DS benchmarks listed. (Either audited or unaudited) So

Re: TCP/IP speedup

2015-08-02 Thread Steve Loughran
On 1 Aug 2015, at 18:26, Ruslan Dautkhanov dautkha...@gmail.commailto:dautkha...@gmail.com wrote: If your network is bandwidth-bound, you'll see setting jumbo frames (MTU 9000) may increase bandwidth up to ~20%.

Re: TCP/IP speedup

2015-08-01 Thread Mark Hamstra
https://spark-summit.org/2015/events/making-sense-of-spark-performance/ On Sat, Aug 1, 2015 at 3:24 PM, Simon Edelhaus edel...@gmail.com wrote: Hi All! How important would be a significant performance improvement to TCP/IP itself, in terms of overall job performance improvement. Which part

Re: TCP/IP speedup

2015-08-01 Thread Simon Edelhaus
H 2% huh. -- ttfn Simon Edelhaus California 2015 On Sat, Aug 1, 2015 at 3:45 PM, Mark Hamstra m...@clearstorydata.com wrote: https://spark-summit.org/2015/events/making-sense-of-spark-performance/ On Sat, Aug 1, 2015 at 3:24 PM, Simon Edelhaus edel...@gmail.com wrote: Hi All!

TCP/IP speedup

2015-08-01 Thread Simon Edelhaus
Hi All! How important would be a significant performance improvement to TCP/IP itself, in terms of overall job performance improvement. Which part would be most significantly accelerated? Would it be HDFS? -- ttfn Simon Edelhaus California 2015

Re: TCP/IP speedup

2015-08-01 Thread Ruslan Dautkhanov
If your network is bandwidth-bound, you'll see setting jumbo frames (MTU 9000) may increase bandwidth up to ~20%. http://docs.hortonworks.com/HDP2Alpha/index.htm#Hardware_Recommendations_for_Hadoop.htm Enabling Jumbo Frames across the cluster improves bandwidth If Spark workload is not network