[jira] [Commented] (SPARK-32931) Unevaluable Expressions are not foldable

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198131#comment-17198131 ] Apache Spark commented on SPARK-32931: -- User 'gatorsmile' has created a pull request for this

[jira] [Assigned] (SPARK-32931) Unevaluable Expressions are not foldable

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32931: Assignee: Xiao Li (was: Apache Spark) > Unevaluable Expressions are not foldable >

[jira] [Assigned] (SPARK-32931) Unevaluable Expressions are not foldable

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32931: Assignee: Apache Spark (was: Xiao Li) > Unevaluable Expressions are not foldable >

[jira] [Commented] (SPARK-32931) Unevaluable Expressions are not foldable

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198132#comment-17198132 ] Apache Spark commented on SPARK-32931: -- User 'gatorsmile' has created a pull request for this

[jira] [Updated] (SPARK-32886) '.../jobs/undefined' link from "Event Timeline" in jobs page

2020-09-17 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li updated SPARK-32886: Affects Version/s: 2.4.4 > '.../jobs/undefined' link from "Event Timeline" in jobs page >

[jira] [Updated] (SPARK-32906) Struct field names should not change after normalizing floats

2020-09-17 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32906: Fix Version/s: 3.1.0 3.0.2 > Struct field names should not change after

[jira] [Commented] (SPARK-32906) Struct field names should not change after normalizing floats

2020-09-17 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198116#comment-17198116 ] L. C. Hsieh commented on SPARK-32906: - This was resolved by

[jira] [Resolved] (SPARK-32906) Struct field names should not change after normalizing floats

2020-09-17 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-32906. - Assignee: Takeshi Yamamuro Resolution: Fixed > Struct field names should not change after

[jira] [Commented] (SPARK-32932) AQE local reader optimizer breaks repartitioning for dynamic partition overwrite

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198108#comment-17198108 ] Apache Spark commented on SPARK-32932: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32932) AQE local reader optimizer breaks repartitioning for dynamic partition overwrite

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32932: Assignee: Apache Spark > AQE local reader optimizer breaks repartitioning for dynamic

[jira] [Commented] (SPARK-32932) AQE local reader optimizer breaks repartitioning for dynamic partition overwrite

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198107#comment-17198107 ] Apache Spark commented on SPARK-32932: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32932) AQE local reader optimizer breaks repartitioning for dynamic partition overwrite

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32932: Assignee: (was: Apache Spark) > AQE local reader optimizer breaks repartitioning for

[jira] [Created] (SPARK-32932) AQE local reader optimizer breaks repartitioning for dynamic partition overwrite

2020-09-17 Thread Manu Zhang (Jira)
Manu Zhang created SPARK-32932: -- Summary: AQE local reader optimizer breaks repartitioning for dynamic partition overwrite Key: SPARK-32932 URL: https://issues.apache.org/jira/browse/SPARK-32932

[jira] [Created] (SPARK-32931) Unevaluable Expressions are not foldable

2020-09-17 Thread Xiao Li (Jira)
Xiao Li created SPARK-32931: --- Summary: Unevaluable Expressions are not foldable Key: SPARK-32931 URL: https://issues.apache.org/jira/browse/SPARK-32931 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-32930) Replace deprecated isFile/isDirectory methods

2020-09-17 Thread William Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Hyun resolved SPARK-32930. -- Resolution: Incomplete > Replace deprecated isFile/isDirectory methods >

[jira] [Closed] (SPARK-32930) Replace deprecated isFile/isDirectory methods

2020-09-17 Thread William Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Hyun closed SPARK-32930. > Replace deprecated isFile/isDirectory methods > - > >

[jira] [Commented] (SPARK-32930) Replace deprecated isFile/isDirectory methods

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198088#comment-17198088 ] Apache Spark commented on SPARK-32930: -- User 'williamhyun' has created a pull request for this

[jira] [Commented] (SPARK-32930) Replace deprecated isFile/isDirectory methods

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198087#comment-17198087 ] Apache Spark commented on SPARK-32930: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-32930) Replace deprecated isFile/isDirectory methods

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32930: Assignee: Apache Spark > Replace deprecated isFile/isDirectory methods >

[jira] [Assigned] (SPARK-32930) Replace deprecated isFile/isDirectory methods

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32930: Assignee: (was: Apache Spark) > Replace deprecated isFile/isDirectory methods >

[jira] [Created] (SPARK-32930) Replace deprecated isFile/isDirectory methods

2020-09-17 Thread William Hyun (Jira)
William Hyun created SPARK-32930: Summary: Replace deprecated isFile/isDirectory methods Key: SPARK-32930 URL: https://issues.apache.org/jira/browse/SPARK-32930 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32908. -- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution:

[jira] [Assigned] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32908: Assignee: Maxim Gekk > percentile_approx() returns incorrect results >

[jira] [Updated] (SPARK-32067) [K8S] Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-09-17 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Yu updated SPARK-32067: - Description: THE BUG: The bug is reproducible by spark-submit two different apps (app1 and app2) with

[jira] [Updated] (SPARK-32067) [K8S] Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-09-17 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Yu updated SPARK-32067: - Summary: [K8S] Executor pod template ConfigMap of ongoing submission got inadvertently altered by

[jira] [Commented] (SPARK-32511) Add dropFields method to Column class

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198026#comment-17198026 ] Apache Spark commented on SPARK-32511: -- User 'fqaiser94' has created a pull request for this issue:

[jira] [Commented] (SPARK-32511) Add dropFields method to Column class

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198025#comment-17198025 ] Apache Spark commented on SPARK-32511: -- User 'fqaiser94' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32902) Logging plan changes for AQE

2020-09-17 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32902. -- Fix Version/s: 3.1.0 Assignee: Takeshi Yamamuro Resolution: Fixed

[jira] [Resolved] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-09-17 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32635. -- Fix Version/s: 3.1.0 3.0.2 Assignee: Peter Toth

[jira] [Updated] (SPARK-32889) orc table column name doesn't support special characters.

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32889: -- Affects Version/s: (was: 3.0.1) 3.1.0 > orc table column name

[jira] [Updated] (SPARK-32889) orc table column name doesn't support special characters.

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32889: -- Issue Type: Improvement (was: Bug) > orc table column name doesn't support special

[jira] [Resolved] (SPARK-32889) orc table column name doesn't support special characters.

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32889. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29761

[jira] [Assigned] (SPARK-32889) orc table column name doesn't support special characters.

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32889: - Assignee: jason jin > orc table column name doesn't support special characters. >

[jira] [Updated] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32926: -- Parent: SPARK-25075 Issue Type: Sub-task (was: Test) > Add Scala 2.13 build test in

[jira] [Resolved] (SPARK-32909) Pass all `sql/hive-thriftserver` module UTs in Scala 2.13

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32909. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29783

[jira] [Assigned] (SPARK-32909) Pass all `sql/hive-thriftserver` module UTs in Scala 2.13

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32909: - Assignee: Yang Jie > Pass all `sql/hive-thriftserver` module UTs in Scala 2.13 >

[jira] [Commented] (SPARK-29900) make relation lookup behavior consistent within Spark SQL

2020-09-17 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197965#comment-17197965 ] Terry Kim commented on SPARK-29900: --- These are the outputs from postgres and mysql. I also created a

[jira] [Assigned] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32926: - Assignee: Dongjoon Hyun > Add Scala 2.13 build test in GitHub Action >

[jira] [Resolved] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32926. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29793

[jira] [Created] (SPARK-32929) StreamSuite failure on IBM Z: - SPARK-20432: union one stream with itself

2020-09-17 Thread Michael Munday (Jira)
Michael Munday created SPARK-32929: -- Summary: StreamSuite failure on IBM Z: - SPARK-20432: union one stream with itself Key: SPARK-32929 URL: https://issues.apache.org/jira/browse/SPARK-32929

[jira] [Updated] (SPARK-32928) Non-deterministic expressions should not be reordered inside AND and OR

2020-09-17 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-32928: --- Description: Using the splitDisjunctivePredicates and splitConjunctivePredicates helper methods

[jira] [Created] (SPARK-32928) Non-deterministic expressions should not be reordered inside AND and OR

2020-09-17 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-32928: -- Summary: Non-deterministic expressions should not be reordered inside AND and OR Key: SPARK-32928 URL: https://issues.apache.org/jira/browse/SPARK-32928 Project: Spark

[jira] [Commented] (SPARK-32927) Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197928#comment-17197928 ] Apache Spark commented on SPARK-32927: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32927) Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32927: Assignee: (was: Apache Spark) > Bitwise OR, AND and XOR should have similar

[jira] [Assigned] (SPARK-32927) Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32927: Assignee: Apache Spark > Bitwise OR, AND and XOR should have similar canonicalization

[jira] [Commented] (SPARK-32927) Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197927#comment-17197927 ] Apache Spark commented on SPARK-32927: -- User 'tanelk' has created a pull request for this issue:

[jira] [Updated] (SPARK-32927) Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND

2020-09-17 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-32927: --- Priority: Minor (was: Major) > Bitwise OR, AND and XOR should have similar canonicalization rules

[jira] [Updated] (SPARK-32927) Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND

2020-09-17 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-32927: --- Priority: Major (was: Minor) > Bitwise OR, AND and XOR should have similar canonicalization rules

[jira] [Created] (SPARK-32927) Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND

2020-09-17 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-32927: -- Summary: Bitwise OR, AND and XOR should have similar canonicalization rules to boolean OR and AND Key: SPARK-32927 URL: https://issues.apache.org/jira/browse/SPARK-32927

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32635: -- Affects Version/s: 2.1.3 > When pyspark.sql.functions.lit() function is used with dataframe

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32635: -- Affects Version/s: 2.2.3 > When pyspark.sql.functions.lit() function is used with dataframe

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32635: -- Affects Version/s: 2.3.4 > When pyspark.sql.functions.lit() function is used with dataframe

[jira] [Commented] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197909#comment-17197909 ] Apache Spark commented on SPARK-32926: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32926: Assignee: Apache Spark > Add Scala 2.13 build test in GitHub Action >

[jira] [Assigned] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32926: Assignee: (was: Apache Spark) > Add Scala 2.13 build test in GitHub Action >

[jira] [Created] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-32926: - Summary: Add Scala 2.13 build test in GitHub Action Key: SPARK-32926 URL: https://issues.apache.org/jira/browse/SPARK-32926 Project: Spark Issue Type:

[jira] [Updated] (SPARK-32926) Add Scala 2.13 build test in GitHub Action

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32926: -- Component/s: (was: Spark Core) > Add Scala 2.13 build test in GitHub Action >

[jira] [Assigned] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32858: Assignee: Apache Spark > UnwrapCastInBinaryComparison: support other numeric types >

[jira] [Commented] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197897#comment-17197897 ] Apache Spark commented on SPARK-32858: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32858: Assignee: (was: Apache Spark) > UnwrapCastInBinaryComparison: support other numeric

[jira] [Commented] (SPARK-32925) Support push-based shuffle in multiple deployment environments

2020-09-17 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197893#comment-17197893 ] Min Shen commented on SPARK-32925: -- cc [~dongjoon], [~holden], [~dbtsai] for comments on k8s shuffle

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32635: -- Component/s: (was: PySpark) > When pyspark.sql.functions.lit() function is used with

[jira] [Created] (SPARK-32925) Support push-based shuffle in multiple deployment environments

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32925: Summary: Support push-based shuffle in multiple deployment environments Key: SPARK-32925 URL: https://issues.apache.org/jira/browse/SPARK-32925 Project: Spark

[jira] [Updated] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-09-17 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-32858: - Summary: UnwrapCastInBinaryComparison: support other numeric types (was: UnwrapCastInBinaryComparison:

[jira] [Created] (SPARK-32924) Web UI sort on duration is wrong

2020-09-17 Thread t oo (Jira)
t oo created SPARK-32924: Summary: Web UI sort on duration is wrong Key: SPARK-32924 URL: https://issues.apache.org/jira/browse/SPARK-32924 Project: Spark Issue Type: Bug Components: Web

[jira] [Updated] (SPARK-32924) Web UI sort on duration is wrong

2020-09-17 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-32924: - Attachment: ui_sort.png > Web UI sort on duration is wrong > > >

[jira] [Created] (SPARK-32923) Add support to properly handle different type of stage retries

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32923: Summary: Add support to properly handle different type of stage retries Key: SPARK-32923 URL: https://issues.apache.org/jira/browse/SPARK-32923 Project: Spark

[jira] [Created] (SPARK-32922) Add support for ShuffleBlockFetcherIterator to read from merged shuffle partitions and to fallback to original shuffle blocks if encountering failures

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32922: Summary: Add support for ShuffleBlockFetcherIterator to read from merged shuffle partitions and to fallback to original shuffle blocks if encountering failures Key: SPARK-32922 URL:

[jira] [Created] (SPARK-32921) Extend MapOutputTracker to support tracking and serving the metadata about each merged shuffle partitions for a given shuffle in push-based shuffle scenario

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32921: Summary: Extend MapOutputTracker to support tracking and serving the metadata about each merged shuffle partitions for a given shuffle in push-based shuffle scenario Key: SPARK-32921

[jira] [Created] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32920: Summary: Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage Key: SPARK-32920

[jira] [Created] (SPARK-32919) Add support in Spark driver to coordinate the shuffle map stage in push-based shuffle by selecting external shuffle services for merging shuffle partitions

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32919: Summary: Add support in Spark driver to coordinate the shuffle map stage in push-based shuffle by selecting external shuffle services for merging shuffle partitions Key: SPARK-32919

[jira] [Created] (SPARK-32918) RPC implementation to support control plane coordination for push-based shuffle

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32918: Summary: RPC implementation to support control plane coordination for push-based shuffle Key: SPARK-32918 URL: https://issues.apache.org/jira/browse/SPARK-32918 Project:

[jira] [Updated] (SPARK-32738) thread safe endpoints may hang due to fatal error

2020-09-17 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-32738: Fix Version/s: 3.0.2 > thread safe endpoints may hang due to fatal error >

[jira] [Created] (SPARK-32917) Add support for executors to push shuffle blocks after successful map task completion

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32917: Summary: Add support for executors to push shuffle blocks after successful map task completion Key: SPARK-32917 URL: https://issues.apache.org/jira/browse/SPARK-32917

[jira] [Created] (SPARK-32916) Add support for external shuffle service in YARN deployment mode to leverage push-based shuffle

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32916: Summary: Add support for external shuffle service in YARN deployment mode to leverage push-based shuffle Key: SPARK-32916 URL: https://issues.apache.org/jira/browse/SPARK-32916

[jira] [Created] (SPARK-32915) RPC implementation to support pushing and merging shuffle blocks

2020-09-17 Thread Min Shen (Jira)
Min Shen created SPARK-32915: Summary: RPC implementation to support pushing and merging shuffle blocks Key: SPARK-32915 URL: https://issues.apache.org/jira/browse/SPARK-32915 Project: Spark

[jira] [Commented] (SPARK-32914) Avoid calling dataType multiple times for each expression

2020-09-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197796#comment-17197796 ] Yuming Wang commented on SPARK-32914: - After this change: {noformat} --- Execution profile --- Total

[jira] [Assigned] (SPARK-32887) Example command in https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be changed

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32887: - Assignee: Udbhav Agrawal > Example command in >

[jira] [Resolved] (SPARK-32887) Example command in https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be changed

2020-09-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32887. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-32914) Avoid calling dataType multiple times for each expression

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32914: Assignee: (was: Apache Spark) > Avoid calling dataType multiple times for each

[jira] [Commented] (SPARK-32914) Avoid calling dataType multiple times for each expression

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197778#comment-17197778 ] Apache Spark commented on SPARK-32914: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-32914) Avoid calling dataType multiple times for each expression

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1719#comment-1719 ] Apache Spark commented on SPARK-32914: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32914) Avoid calling dataType multiple times for each expression

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32914: Assignee: Apache Spark > Avoid calling dataType multiple times for each expression >

[jira] [Created] (SPARK-32914) Avoid calling dataType multiple times for each expression

2020-09-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-32914: --- Summary: Avoid calling dataType multiple times for each expression Key: SPARK-32914 URL: https://issues.apache.org/jira/browse/SPARK-32914 Project: Spark

[jira] [Commented] (SPARK-32898) totalExecutorRunTimeMs is too big

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197763#comment-17197763 ] Apache Spark commented on SPARK-32898: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32898) totalExecutorRunTimeMs is too big

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32898: Assignee: Apache Spark > totalExecutorRunTimeMs is too big >

[jira] [Commented] (SPARK-32898) totalExecutorRunTimeMs is too big

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197762#comment-17197762 ] Apache Spark commented on SPARK-32898: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32898) totalExecutorRunTimeMs is too big

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32898: Assignee: (was: Apache Spark) > totalExecutorRunTimeMs is too big >

[jira] [Commented] (SPARK-29900) make relation lookup behavior consistent within Spark SQL

2020-09-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197742#comment-17197742 ] Wenchen Fan commented on SPARK-29900: - I think it was due to Spark wanting to make SHOW TABLES and

[jira] [Commented] (SPARK-32913) Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197741#comment-17197741 ] Apache Spark commented on SPARK-32913: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32913) Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32913: Assignee: (was: Apache Spark) > Improve ExecutorDecommissionInfo and

[jira] [Commented] (SPARK-32913) Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197740#comment-17197740 ] Apache Spark commented on SPARK-32913: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32913) Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32913: Assignee: Apache Spark > Improve ExecutorDecommissionInfo and ExecutorDecommissionState

[jira] [Assigned] (SPARK-32911) UnsafeExternalSorter.SpillableIterator cannot spill after reading all records

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32911: Assignee: (was: Apache Spark) > UnsafeExternalSorter.SpillableIterator cannot spill

[jira] [Assigned] (SPARK-32911) UnsafeExternalSorter.SpillableIterator cannot spill after reading all records

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32911: Assignee: Apache Spark > UnsafeExternalSorter.SpillableIterator cannot spill after

[jira] [Commented] (SPARK-32911) UnsafeExternalSorter.SpillableIterator cannot spill after reading all records

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197737#comment-17197737 ] Apache Spark commented on SPARK-32911: -- User 'tomvanbussel' has created a pull request for this

[jira] [Commented] (SPARK-30283) V2 Command logical plan should use UnresolvedV2Relation for a table

2020-09-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197735#comment-17197735 ] Wenchen Fan commented on SPARK-30283: - Yea we should resume the work, and probably do ALTER TABLE as

[jira] [Commented] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197733#comment-17197733 ] Apache Spark commented on SPARK-32908: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197734#comment-17197734 ] Apache Spark commented on SPARK-32908: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-32913) Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

2020-09-17 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32913: - Summary: Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases (was: Improve

  1   2   >