Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

Thank you for your inputs. Will test it out and share my findings On Thursday, July 14, 2016, CosminC wrote: > Didn't have the time to investigate much further, but the one thing that > popped out is that partitioning was no longer working on 1.6.1. This would > definitely

Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

Didn't have the time to investigate much further, but the one thing that popped out is that partitioning was no longer working on 1.6.1. This would definitely explain the 2x performance loss. Checking 1.5.1 Spark logs for the same application showed that our partitioner was working correctly, and

Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

You can get some good pointers in this JIRA https://issues.apache.org/jira/browse/SPARK-15796 Dibyendu On Thu, Jul 14, 2016 at 12:53 AM, Sunita wrote: > I am facing the same issue. Upgrading to Spark1.6 is causing hugh > performance > loss. Could you solve this issue?

Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

I am facing the same issue. Upgrading to Spark1.6 is causing hugh performance loss. Could you solve this issue? I am also attempting memory settings as mentioned http://spark.apache.org/docs/latest/configuration.html#memory-management But its not making a lot of difference. Appreciate your inputs