[jira] [Updated] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-33206: - Description: SPARK-21501 changed the spark shuffle index service to be based on memory instead

[jira] [Commented] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218306#comment-17218306 ] Lars Francke commented on SPARK-33206: -- I used YourKit (thank you for the free lice

[jira] [Updated] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-33206: - Description: dSPARK-21501 changed the spark shuffle index service to be based on memory instead

[jira] [Commented] (SPARK-21501) Spark shuffle index cache size should be memory based

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218285#comment-17218285 ] Lars Francke commented on SPARK-21501: -- Just FYI for others stumbling across this:

[jira] [Updated] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-33206: - Attachment: image001(1).png > Spark Shuffle Index Cache calculates memory usage wrong >

[jira] [Created] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
Lars Francke created SPARK-33206: Summary: Spark Shuffle Index Cache calculates memory usage wrong Key: SPARK-33206 URL: https://issues.apache.org/jira/browse/SPARK-33206 Project: Spark Issue

[jira] [Commented] (SPARK-30196) Bump lz4-java version to 1.7.0

2020-01-10 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013039#comment-17013039 ] Lars Francke commented on SPARK-30196: -- Excellent, thank you! > Bump lz4-java vers

[jira] [Commented] (SPARK-30196) Bump lz4-java version to 1.7.0

2020-01-08 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010475#comment-17010475 ] Lars Francke commented on SPARK-30196: -- Sorry I didn't have time to check yet but i

[jira] [Commented] (SPARK-30196) Bump lz4-java version to 1.7.0

2019-12-29 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17005066#comment-17005066 ] Lars Francke commented on SPARK-30196: -- FYI: This seems to have broken Spark 3 on M

[jira] [Commented] (SPARK-29226) Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.

2019-11-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979868#comment-16979868 ] Lars Francke commented on SPARK-29226: -- Thanks for the quick follow-up. I did not c

[jira] [Commented] (SPARK-29226) Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.

2019-11-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979697#comment-16979697 ] Lars Francke commented on SPARK-29226: -- This causes the following exception for me:

[jira] [Commented] (SPARK-25390) data source V2 API refactoring

2019-06-25 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872684#comment-16872684 ] Lars Francke commented on SPARK-25390: -- Is there any kind of end-user documentation

[jira] [Created] (SPARK-24049) Add a feature to not start speculative tasks before an absolute minimum amount of time has passed

2018-04-22 Thread Lars Francke (JIRA)
Lars Francke created SPARK-24049: Summary: Add a feature to not start speculative tasks before an absolute minimum amount of time has passed Key: SPARK-24049 URL: https://issues.apache.org/jira/browse/SPARK-24049

[jira] [Updated] (SPARK-24049) Add a feature to not start speculative tasks when average task duration is less than a configurable absolute number

2018-04-22 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-24049: - Summary: Add a feature to not start speculative tasks when average task duration is less than a c

[jira] [Comment Edited] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-10-15 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15578914#comment-15578914 ] Lars Francke edited comment on SPARK-650 at 10/15/16 11:22 PM: --

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-10-15 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15578914#comment-15578914 ] Lars Francke commented on SPARK-650: I can only come up with three reasons at the momen

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2016-10-15 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15578875#comment-15578875 ] Lars Francke commented on SPARK-650: I also have to disagree with this being a duplicat

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-21 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15341381#comment-15341381 ] Lars Francke commented on SPARK-12177: -- I have looked at the code and have only mino

[jira] [Updated] (SPARK-572) Forbid update of static mutable variables

2015-07-29 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-572: --- Description: Consider the following piece of code: {code} object Foo { var xx = -1 def main() { x

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2015-07-29 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645919#comment-14645919 ] Lars Francke commented on SPARK-650: Not [~matei] but I think this would be a good idea

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

2015-06-29 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605657#comment-14605657 ] Lars Francke commented on SPARK-2447: - Hey Ted et. al, thanks for the work on this. S

[jira] [Created] (SPARK-8462) Documentation fixes for Spark SQL

2015-06-18 Thread Lars Francke (JIRA)
Lars Francke created SPARK-8462: --- Summary: Documentation fixes for Spark SQL Key: SPARK-8462 URL: https://issues.apache.org/jira/browse/SPARK-8462 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-7366) Support multi-line JSON objects

2015-05-05 Thread Lars Francke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14528489#comment-14528489 ] Lars Francke commented on SPARK-7366: - This would be very helpful. ESRI has a RecordRe