Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

Thank you for your inputs. Will test it out and share my findings On Thursday, July 14, 2016, CosminC wrote: > Didn't have the time to investigate much further, but the one thing that > popped out is that partitioning was no longer working on 1.6.1. This would > definitely

Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

Didn't have the time to investigate much further, but the one thing that popped out is that partitioning was no longer working on 1.6.1. This would definitely explain the 2x performance loss. Checking 1.5.1 Spark logs for the same application showed that our partitioner was working correctly, and

Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

You can get some good pointers in this JIRA https://issues.apache.org/jira/browse/SPARK-15796 Dibyendu On Thu, Jul 14, 2016 at 12:53 AM, Sunita wrote: > I am facing the same issue. Upgrading to Spark1.6 is causing hugh > performance > loss. Could you solve this issue?

Re: Severe Spark Streaming performance degradation after upgrading to 1.6.1

I am facing the same issue. Upgrading to Spark1.6 is causing hugh performance loss. Could you solve this issue? I am also attempting memory settings as mentioned http://spark.apache.org/docs/latest/configuration.html#memory-management But its not making a lot of difference. Appreciate your inputs

Re: [REPOST] Severe Spark Streaming performance degradation after upgrading to 1.6.1

urday 4 June 2016 at 12:33 >> *To: *Florin Broască <florin.broa...@gmail.com> >> *Cc: *David Newberger <david.newber...@wandcorp.com>, Adrian Tanase < >> atan...@adobe.com>, "user@spark.apache.org" <user@spark.apache.org>, >> ciobanu <ciob.

Re: [REPOST] Severe Spark Streaming performance degradation after upgrading to 1.6.1

<david.newber...@wandcorp.com>, Adrian Tanase < > atan...@adobe.com>, "user@spark.apache.org" <user@spark.apache.org>, > ciobanu <ciob...@adobe.com> > *Subject: *Re: [REPOST] Severe Spark Streaming performance degradation > after upgrading to 1.6.

Re: [REPOST] Severe Spark Streaming performance degradation after upgrading to 1.6.1

e.org" <user@spark.apache.org>, ciobanu <ciob...@adobe.com> Subject: Re: [REPOST] Severe Spark Streaming performance degradation after upgrading to 1.6.1 batch interval I meant thx Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=A

RE: [REPOST] Severe Spark Streaming performance degradation after upgrading to 1.6.1

What does your processing time look like. Is it consistently within that 20sec micro batch window? David Newberger From: Adrian Tanase [mailto:atan...@adobe.com] Sent: Friday, June 3, 2016 8:14 AM To: user@spark.apache.org Cc: Cosmin Ciobanu Subject: [REPOST] Severe Spark Streaming performance

[REPOST] Severe Spark Streaming performance degradation after upgrading to 1.6.1

Hi all, Trying to repost this question from a colleague on my team, somehow his subscription is not active: http://apache-spark-user-list.1001560.n3.nabble.com/Severe-Spark-Streaming-performance-degradation-after-upgrading-to-1-6-1-td27056.html Appreciate any thoughts, -adrian