> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Severe-Spark-Streaming-performance-degradation-after-upgrading-to-1-6-1-tp27056p27334.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> --
the partitioning issue is no longer
present there.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Severe-Spark-Streaming-performance-degradation-after-upgrading-to-1-6-1-tp27056p27334.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
/apache-spark-user-list.1001560.n3.nabble.com/Severe-Spark-Streaming-performance-degradation-after-upgrading-to-1-6-1-tp27056p27330.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>
on this
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Severe-Spark-Streaming-performance-degradation-after-upgrading-to-1-6-1-tp27056p27330.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
urday 4 June 2016 at 12:33
>> *To: *Florin Broască <florin.broa...@gmail.com>
>> *Cc: *David Newberger <david.newber...@wandcorp.com>, Adrian Tanase <
>> atan...@adobe.com>, "user@spark.apache.org" <user@spark.apache.org>,
>> ciobanu <ciob.
<david.newber...@wandcorp.com>, Adrian Tanase <
> atan...@adobe.com>, "user@spark.apache.org" <user@spark.apache.org>,
> ciobanu <ciob...@adobe.com>
> *Subject: *Re: [REPOST] Severe Spark Streaming performance degradation
> after upgrading to 1.6.
e.org" <user@spark.apache.org>, ciobanu
<ciob...@adobe.com>
Subject: Re: [REPOST] Severe Spark Streaming performance degradation after
upgrading to 1.6.1
batch interval I meant
thx
Dr Mich Talebzadeh
LinkedIn
https://www.linkedin.com/profile/view?id=A
What does your processing time look like. Is it consistently within that 20sec
micro batch window?
David Newberger
From: Adrian Tanase [mailto:atan...@adobe.com]
Sent: Friday, June 3, 2016 8:14 AM
To: user@spark.apache.org
Cc: Cosmin Ciobanu
Subject: [REPOST] Severe Spark Streaming performance
Hi all,
Trying to repost this question from a colleague on my team, somehow his
subscription is not active:
http://apache-spark-user-list.1001560.n3.nabble.com/Severe-Spark-Streaming-performance-degradation-after-upgrading-to-1-6-1-td27056.html
Appreciate any thoughts,
-adrian
Hi,
I've developed a POC Spark Streaming application.
But it seems to perform better on my development machine than on our cluster.
I submit it to yarn on our cloudera cluster.
But my first question is more detailed:
In de application UI (:4040) I see in the streaming section that the batch
What were the number of cores in the executor? It could be that you had
only one core in the executor which did all the 50 tasks serially so 50
task X 15 ms = ~ 1 second.
Could you take a look at the task details in the stage page to see when the
tasks were added to see whether it explains the 5
I am not sure why you are getting node_local and not process_local. Also
there is probably not a good documentation other than that configuration
page - http://spark.apache.org/docs/latest/configuration.html (search for
locality)
On Thu, Jul 9, 2015 at 5:51 AM, Michel Hubert
12 matches
Mail list logo
