[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259801#comment-17259801 ] David Wyles commented on SPARK-33635: - Thanks a lot for getting down to the route cause and

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles edited comment on SPARK-33635 at 12/29/20, 8:48 PM:

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles edited comment on SPARK-33635 at 12/29/20, 4:54 PM:

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles edited comment on SPARK-33635 at 12/29/20, 4:34 PM:

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles commented on SPARK-33635: - [~gsomogyi] I now have my results. I was so unhappy about

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17251911#comment-17251911 ] David Wyles commented on SPARK-33635: - [~gsomogyi] Just so you know, I'm still doing this. I've

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247879#comment-17247879 ] David Wyles commented on SPARK-33635: - [~gsomogyi]  "try to turn off Kafka consumer caching. Apart

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247878#comment-17247878 ] David Wyles commented on SPARK-33635: - "Since you're measuring speed I've ported the Kafka source

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247875#comment-17247875 ] David Wyles edited comment on SPARK-33635 at 12/11/20, 12:12 PM: - I'll

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247875#comment-17247875 ] David Wyles edited comment on SPARK-33635 at 12/11/20, 12:11 PM: - I'll

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247875#comment-17247875 ] David Wyles edited comment on SPARK-33635 at 12/11/20, 12:11 PM: - I'll

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247875#comment-17247875 ] David Wyles commented on SPARK-33635: - I'll give all those a go and get back to you. > Performance

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246430#comment-17246430 ] David Wyles commented on SPARK-33635: - Having performed my tests I can conclude that the kafka

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245329#comment-17245329 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:41 PM: --- Apart from

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245329#comment-17245329 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:33 PM: --- The diffs

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245329#comment-17245329 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:32 PM: --- The diffs

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245329#comment-17245329 ] David Wyles commented on SPARK-33635: - The diffs on that library between 2.4.5 and 3.0.0 are very

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245304#comment-17245304 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:20 PM: --- Fair

[jira] [Issue Comment Deleted] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wyles updated SPARK-33635: Comment: was deleted (was: Also note, in my example - the only thing that changes is the spark

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245309#comment-17245309 ] David Wyles commented on SPARK-33635: - Also note, in my example - the only thing that changes is the

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245306#comment-17245306 ] David Wyles commented on SPARK-33635: - Is anyone even able to confirm my results are not just unique

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245304#comment-17245304 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:00 PM: --- Fair

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245304#comment-17245304 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 3:53 PM: --- Fair

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245304#comment-17245304 ] David Wyles commented on SPARK-33635: - Fair point, the library I suspect is spark-sql-kafka-0-10, is

[jira] [Updated] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wyles updated SPARK-33635: Description: I have observed a slowdown in the reading of data from kafka on all of our systems

[jira] [Created] (SPARK-33635) Performance regression in Kafka read

David Wyles created SPARK-33635: --- Summary: Performance regression in Kafka read Key: SPARK-33635 URL: https://issues.apache.org/jira/browse/SPARK-33635 Project: Spark Issue Type: Bug