[jira] [Resolved] (SPARK-32955) An item in the navigation bar in the WebUI has a wrong link.

2020-09-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32955. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29821

[jira] [Commented] (SPARK-32306) `approx_percentile` in Spark SQL gives incorrect results

2020-09-21 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199825#comment-17199825 ] Jungtaek Lim commented on SPARK-32306: -- 50% percentile should give you median, not average. So that

[jira] [Commented] (SPARK-32306) `approx_percentile` in Spark SQL gives incorrect results

2020-09-21 Thread Sean Malory (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199817#comment-17199817 ] Sean Malory commented on SPARK-32306: - I expect the result to be (5 + 8) / 2 = 6.5. >

[jira] [Resolved] (SPARK-32949) Add timestamp_seconds to SparkR

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32949. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29822

[jira] [Assigned] (SPARK-32949) Add timestamp_seconds to SparkR

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32949: - Assignee: Maciej Szymkiewicz > Add timestamp_seconds to SparkR >

[jira] [Assigned] (SPARK-32351) Partially pushed partition filters are not explained

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32351: Assignee: (was: Apache Spark) > Partially pushed partition filters are not explained

[jira] [Commented] (SPARK-32351) Partially pushed partition filters are not explained

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199802#comment-17199802 ] Apache Spark commented on SPARK-32351: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32951) Foldable propagation from Aggregate

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32951: - Assignee: Peter Toth > Foldable propagation from Aggregate >

[jira] [Resolved] (SPARK-32951) Foldable propagation from Aggregate

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32951. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29816

[jira] [Commented] (SPARK-32351) Partially pushed partition filters are not explained

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199800#comment-17199800 ] Apache Spark commented on SPARK-32351: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32351) Partially pushed partition filters are not explained

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32351: Assignee: Apache Spark > Partially pushed partition filters are not explained >

[jira] [Updated] (SPARK-32961) PySpark CSV read with UTF-16 encoding is not working correctly

2020-09-21 Thread Bui Bao Anh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bui Bao Anh updated SPARK-32961: Description: There are weird characters in the output when printing out to console or writing to

[jira] [Updated] (SPARK-32961) PySpark CSV read with UTF-16 encoding is not working correctly

2020-09-21 Thread Bui Bao Anh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bui Bao Anh updated SPARK-32961: Attachment: pyspark df.png > PySpark CSV read with UTF-16 encoding is not working correctly >

[jira] [Updated] (SPARK-32961) PySpark CSV read with UTF-16 encoding is not working correctly

2020-09-21 Thread Bui Bao Anh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bui Bao Anh updated SPARK-32961: Attachment: pandas df.png > PySpark CSV read with UTF-16 encoding is not working correctly >

[jira] [Updated] (SPARK-32961) PySpark CSV read with UTF-16 encoding is not working correctly

2020-09-21 Thread Bui Bao Anh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bui Bao Anh updated SPARK-32961: Description: There are weird characters in the output when printing out to console or writing to

[jira] [Created] (SPARK-32961) PySpark CSV read with UTF-16 encoding is not working correctly

2020-09-21 Thread Bui Bao Anh (Jira)
Bui Bao Anh created SPARK-32961: --- Summary: PySpark CSV read with UTF-16 encoding is not working correctly Key: SPARK-32961 URL: https://issues.apache.org/jira/browse/SPARK-32961 Project: Spark

[jira] [Updated] (SPARK-32708) Query optimization fails to reuse exchange with DataSourceV2

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32708: -- Affects Version/s: (was: 2.4.8) 2.4.7 > Query optimization fails

[jira] [Updated] (SPARK-32708) Query optimization fails to reuse exchange with DataSourceV2

2020-09-21 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-32708: --- Affects Version/s: (was: 2.4.7) 2.4.8 > Query optimization fails

[jira] [Updated] (SPARK-32708) Query optimization fails to reuse exchange with DataSourceV2

2020-09-21 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-32708: --- Fix Version/s: 2.4.8 > Query optimization fails to reuse exchange with DataSourceV2 >

[jira] [Commented] (SPARK-32960) Provide better exception on temporary view against DataFrameWriterV2

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199713#comment-17199713 ] Apache Spark commented on SPARK-32960: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-32960) Provide better exception on temporary view against DataFrameWriterV2

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32960: Assignee: (was: Apache Spark) > Provide better exception on temporary view against

[jira] [Assigned] (SPARK-32960) Provide better exception on temporary view against DataFrameWriterV2

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32960: Assignee: Apache Spark > Provide better exception on temporary view against

[jira] [Commented] (SPARK-32960) Provide better exception on temporary view against DataFrameWriterV2

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199711#comment-17199711 ] Apache Spark commented on SPARK-32960: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Created] (SPARK-32960) Provide better exception on temporary view against DataFrameWriterV2

2020-09-21 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-32960: Summary: Provide better exception on temporary view against DataFrameWriterV2 Key: SPARK-32960 URL: https://issues.apache.org/jira/browse/SPARK-32960 Project: Spark

[jira] [Commented] (SPARK-32312) Upgrade Apache Arrow to 1.0.0

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199704#comment-17199704 ] Apache Spark commented on SPARK-32312: -- User 'ueshin' has created a pull request for this issue:

[jira] [Updated] (SPARK-31882) DAG-viz is not rendered correctly with pagination.

2020-09-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li updated SPARK-31882: Affects Version/s: 2.4.4 > DAG-viz is not rendered correctly with pagination. >

[jira] [Updated] (SPARK-32718) remove unnecessary keywords for interval units

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32718: -- Fix Version/s: 3.0.2 > remove unnecessary keywords for interval units >

[jira] [Commented] (SPARK-30882) Inaccurate results with higher precision ApproximatePercentile results

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199663#comment-17199663 ] Dongjoon Hyun commented on SPARK-30882: --- +1 for [~maxgekk]'s assessment. > Inaccurate results

[jira] [Updated] (SPARK-32928) Non-deterministic expressions should not be reordered inside AND and OR

2020-09-21 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-32928: --- Labels: correctness (was: CorrectnessBug) > Non-deterministic expressions should not be reordered

[jira] [Assigned] (SPARK-32959) Fix the "Relation: view text" test in DataSourceV2SQLSuite

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32959: Assignee: (was: Apache Spark) > Fix the "Relation: view text" test in

[jira] [Commented] (SPARK-32959) Fix the "Relation: view text" test in DataSourceV2SQLSuite

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199647#comment-17199647 ] Apache Spark commented on SPARK-32959: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-32959) Fix the "Relation: view text" test in DataSourceV2SQLSuite

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199648#comment-17199648 ] Apache Spark commented on SPARK-32959: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32959) Fix the "Relation: view text" test in DataSourceV2SQLSuite

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32959: Assignee: Apache Spark > Fix the "Relation: view text" test in DataSourceV2SQLSuite >

[jira] [Updated] (SPARK-32928) Non-deterministic expressions should not be reordered inside AND and OR

2020-09-21 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-32928: --- Labels: CorrectnessBug (was: ) > Non-deterministic expressions should not be reordered inside AND

[jira] [Commented] (SPARK-32928) Non-deterministic expressions should not be reordered inside AND and OR

2020-09-21 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199646#comment-17199646 ] Tanel Kiis commented on SPARK-32928: One more point, where this can manifest is FilterExec

[jira] [Created] (SPARK-32959) Fix the "Relation: view text" test in DataSourceV2SQLSuite

2020-09-21 Thread Terry Kim (Jira)
Terry Kim created SPARK-32959: - Summary: Fix the "Relation: view text" test in DataSourceV2SQLSuite Key: SPARK-32959 URL: https://issues.apache.org/jira/browse/SPARK-32959 Project: Spark Issue

[jira] [Assigned] (SPARK-32948) Optimize Json expression chain

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32948: Assignee: L. C. Hsieh (was: Apache Spark) > Optimize Json expression chain >

[jira] [Commented] (SPARK-32948) Optimize Json expression chain

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199643#comment-17199643 ] Apache Spark commented on SPARK-32948: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-32948) Optimize Json expression chain

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199644#comment-17199644 ] Apache Spark commented on SPARK-32948: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32948) Optimize Json expression chain

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32948: Assignee: Apache Spark (was: L. C. Hsieh) > Optimize Json expression chain >

[jira] [Created] (SPARK-32958) Column pruning for JsonToStructs

2020-09-21 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-32958: --- Summary: Column pruning for JsonToStructs Key: SPARK-32958 URL: https://issues.apache.org/jira/browse/SPARK-32958 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-32957) Add a GitHub Actions job to run WebUI tests with Chrome

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199637#comment-17199637 ] Apache Spark commented on SPARK-32957: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32957) Add a GitHub Actions job to run WebUI tests with Chrome

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32957: Assignee: Kousuke Saruta (was: Apache Spark) > Add a GitHub Actions job to run WebUI

[jira] [Commented] (SPARK-32957) Add a GitHub Actions job to run WebUI tests with Chrome

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199635#comment-17199635 ] Apache Spark commented on SPARK-32957: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32957) Add a GitHub Actions job to run WebUI tests with Chrome

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32957: Assignee: Apache Spark (was: Kousuke Saruta) > Add a GitHub Actions job to run WebUI

[jira] [Created] (SPARK-32957) Add a GitHub Actions job to run WebUI tests with Chrome

2020-09-21 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-32957: -- Summary: Add a GitHub Actions job to run WebUI tests with Chrome Key: SPARK-32957 URL: https://issues.apache.org/jira/browse/SPARK-32957 Project: Spark

[jira] [Commented] (SPARK-27345) Expose approx_percentile in python api

2020-09-21 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199618#comment-17199618 ] Maxim Gekk commented on SPARK-27345: The function was exposed recently by

[jira] [Commented] (SPARK-19335) Spark should support doing an efficient DataFrame Upsert via JDBC

2020-09-21 Thread Aaron Lucas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199617#comment-17199617 ] Aaron Lucas commented on SPARK-19335: - +1 this is a big feature that we are currently having to work

[jira] [Commented] (SPARK-32306) `approx_percentile` in Spark SQL gives incorrect results

2020-09-21 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199615#comment-17199615 ] Maxim Gekk commented on SPARK-32306: [~seanmalory] Which result do you expect? 8? >

[jira] [Resolved] (SPARK-30882) Inaccurate results with higher precision ApproximatePercentile results

2020-09-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30882. -- Resolution: Duplicate > Inaccurate results with higher precision ApproximatePercentile

[jira] [Commented] (SPARK-30882) Inaccurate results with higher precision ApproximatePercentile results

2020-09-21 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199602#comment-17199602 ] Maxim Gekk commented on SPARK-30882: This PR [https://github.com/apache/spark/pull/29784] solves the

[jira] [Commented] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199591#comment-17199591 ] Dongjoon Hyun commented on SPARK-32908: --- I also added `2.3.4` to the affected version and 2.2.x

[jira] [Updated] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32908: -- Affects Version/s: 2.3.4 > percentile_approx() returns incorrect results >

[jira] [Updated] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32908: -- Affects Version/s: (was: 3.0.2) (was: 2.4.8)

[jira] [Commented] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199580#comment-17199580 ] Dongjoon Hyun commented on SPARK-32908: --- Hi, All. I added a correctness label. >

[jira] [Updated] (SPARK-32908) percentile_approx() returns incorrect results

2020-09-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32908: -- Labels: correctness (was: ) > percentile_approx() returns incorrect results >

[jira] [Updated] (SPARK-32886) '.../jobs/undefined' link from "Event Timeline" in jobs page

2020-09-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-32886: - Fix Version/s: (was: 2.4.8) > '.../jobs/undefined' link from "Event Timeline" in jobs page

[jira] [Created] (SPARK-32956) Duplicate Columns in a csv file

2020-09-21 Thread Punit Shah (Jira)
Punit Shah created SPARK-32956: -- Summary: Duplicate Columns in a csv file Key: SPARK-32956 URL: https://issues.apache.org/jira/browse/SPARK-32956 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-32886) '.../jobs/undefined' link from "Event Timeline" in jobs page

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199480#comment-17199480 ] Apache Spark commented on SPARK-32886: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-32886) '.../jobs/undefined' link from "Event Timeline" in jobs page

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199481#comment-17199481 ] Apache Spark commented on SPARK-32886: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32954) Add jakarta.servlet-api test dependency to yarn module to avoid classpath badcase of UTs

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32954: Assignee: (was: Apache Spark) > Add jakarta.servlet-api test dependency to yarn

[jira] [Assigned] (SPARK-32954) Add jakarta.servlet-api test dependency to yarn module to avoid classpath badcase of UTs

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32954: Assignee: Apache Spark > Add jakarta.servlet-api test dependency to yarn module to avoid

[jira] [Commented] (SPARK-32954) Add jakarta.servlet-api test dependency to yarn module to avoid classpath badcase of UTs

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199470#comment-17199470 ] Apache Spark commented on SPARK-32954: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-32718) remove unnecessary keywords for interval units

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199459#comment-17199459 ] Apache Spark commented on SPARK-32718: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32718) remove unnecessary keywords for interval units

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199460#comment-17199460 ] Apache Spark commented on SPARK-32718: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32944) Avoid push down Filter through Project when it will hurts performance

2020-09-21 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199456#comment-17199456 ] Erik Krogen commented on SPARK-32944: - [~viirya] I'm curious if you can provide some examples of

[jira] [Commented] (SPARK-32955) An item in the navigation bar in the WebUI has a wrong link.

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199455#comment-17199455 ] Apache Spark commented on SPARK-32955: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32955) An item in the navigation bar in the WebUI has a wrong link.

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32955: Assignee: Kousuke Saruta (was: Apache Spark) > An item in the navigation bar in the

[jira] [Commented] (SPARK-32955) An item in the navigation bar in the WebUI has a wrong link.

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199452#comment-17199452 ] Apache Spark commented on SPARK-32955: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-32954) Add jakarta.servlet-api test dependency to yarn module to avoid classpath badcase of UTs

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Description: When I tried to verify that the resource-managers yarn module passed all UTs in Scala

[jira] [Assigned] (SPARK-32955) An item in the navigation bar in the WebUI has a wrong link.

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32955: Assignee: Apache Spark (was: Kousuke Saruta) > An item in the navigation bar in the

[jira] [Updated] (SPARK-32954) Add jakarta.servlet-api test dependency to yarn module to avoid classpath badcase of UTs

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Description: When I tried to verify that the resource-managers yarn module passed all UTs in Scala

[jira] [Updated] (SPARK-32954) Add jakarta.servlet-api test dependency to yarn module to avoid classpath badcase of UTs

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Summary: Add jakarta.servlet-api test dependency to yarn module to avoid classpath badcase of UTs

[jira] [Updated] (SPARK-32954) Add jakarta.servlet-api test dependency to yarn module to avoid badcase of UTs

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Summary: Add jakarta.servlet-api test dependency to yarn module to avoid badcase of UTs (was: Add

[jira] [Updated] (SPARK-32954) Add jakarta.servlet-api test dependency to resource-managers yarn module to avoid badcase of UTs

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Summary: Add jakarta.servlet-api test dependency to resource-managers yarn module to avoid badcase of

[jira] [Updated] (SPARK-32954) Add jakarta.servlet-api test dependency to

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Summary: Add jakarta.servlet-api test dependency to (was: Let resource-managers yarn module can be

[jira] [Commented] (SPARK-32949) Add timestamp_seconds to SparkR

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199444#comment-17199444 ] Apache Spark commented on SPARK-32949: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32949) Add timestamp_seconds to SparkR

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32949: Assignee: Apache Spark > Add timestamp_seconds to SparkR >

[jira] [Assigned] (SPARK-32949) Add timestamp_seconds to SparkR

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32949: Assignee: (was: Apache Spark) > Add timestamp_seconds to SparkR >

[jira] [Updated] (SPARK-32954) Let resource-managers yarn module can be tested independently

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Description: When I tried to verify that the resource-managers yarn module passed all UTs in Scala

[jira] [Commented] (SPARK-32938) Spark can not cast long value from Kafka

2020-09-21 Thread Matthias Seiler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199427#comment-17199427 ] Matthias Seiler commented on SPARK-32938: - When I load the stream from Kafka, the schema is

[jira] [Created] (SPARK-32955) An item in the navigation bar in the WebUI has a wrong link.

2020-09-21 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-32955: -- Summary: An item in the navigation bar in the WebUI has a wrong link. Key: SPARK-32955 URL: https://issues.apache.org/jira/browse/SPARK-32955 Project: Spark

[jira] [Updated] (SPARK-32954) Let resource-managers yarn module can be tested independently

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Description: When I tried to verify that the resource-managers yarn module passed all UTs in Scala

[jira] [Updated] (SPARK-32954) Let resource-managers yarn module can be tested independently

2020-09-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32954: - Description: When I tried to verify that the resource-managers yarn module passed all UTs in Scala

[jira] [Created] (SPARK-32954) Let resource-managers yarn module can be tested independently

2020-09-21 Thread Yang Jie (Jira)
Yang Jie created SPARK-32954: Summary: Let resource-managers yarn module can be tested independently Key: SPARK-32954 URL: https://issues.apache.org/jira/browse/SPARK-32954 Project: Spark Issue

[jira] [Commented] (SPARK-27428) Test "metrics StatsD sink with Timer " fails on BigEndian

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199417#comment-17199417 ] Apache Spark commented on SPARK-27428: -- User 'mundaym' has created a pull request for this issue:

[jira] [Assigned] (SPARK-27428) Test "metrics StatsD sink with Timer " fails on BigEndian

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27428: Assignee: Apache Spark > Test "metrics StatsD sink with Timer " fails on BigEndian >

[jira] [Commented] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199416#comment-17199416 ] Apache Spark commented on SPARK-20044: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-27428) Test "metrics StatsD sink with Timer " fails on BigEndian

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27428: Assignee: (was: Apache Spark) > Test "metrics StatsD sink with Timer " fails on

[jira] [Resolved] (SPARK-32886) '.../jobs/undefined' link from "Event Timeline" in jobs page

2020-09-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-32886. -- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution:

[jira] [Assigned] (SPARK-32886) '.../jobs/undefined' link from "Event Timeline" in jobs page

2020-09-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-32886: Assignee: Zhen Li (was: Apache Spark) > '.../jobs/undefined' link from "Event Timeline"

[jira] [Commented] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199391#comment-17199391 ] Apache Spark commented on SPARK-32953: -- User 'lidavidm' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32953: Assignee: (was: Apache Spark) > Lower memory usage in toPandas with Arrow

[jira] [Assigned] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32953: Assignee: Apache Spark > Lower memory usage in toPandas with Arrow self_destruct >

[jira] [Created] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

2020-09-21 Thread David Li (Jira)
David Li created SPARK-32953: Summary: Lower memory usage in toPandas with Arrow self_destruct Key: SPARK-32953 URL: https://issues.apache.org/jira/browse/SPARK-32953 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32850) Simply the RPC message flow of decommission

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199377#comment-17199377 ] Apache Spark commented on SPARK-32850: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Created] (SPARK-32952) Test failure on IBM Z: CoalesceShufflePartitionsSuite: - determining the number of reducers: complex query 1

2020-09-21 Thread Michael Munday (Jira)
Michael Munday created SPARK-32952: -- Summary: Test failure on IBM Z: CoalesceShufflePartitionsSuite: - determining the number of reducers: complex query 1 Key: SPARK-32952 URL:

[jira] [Updated] (SPARK-32952) Test failure on IBM Z: CoalesceShufflePartitionsSuite: - determining the number of reducers: complex query 1

2020-09-21 Thread Michael Munday (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Munday updated SPARK-32952: --- Environment: Linux on IBM Z (s390x). (was: *{color:#FF}{color}*Linux on IBM Z

[jira] [Commented] (SPARK-32951) Foldable propagation from Aggregate

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199325#comment-17199325 ] Apache Spark commented on SPARK-32951: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-32951) Foldable propagation from Aggregate

2020-09-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32951: Assignee: Apache Spark > Foldable propagation from Aggregate >

  1   2   >