[jira] [Updated] (SPARK-32247) scipy installation fails with PyPy

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32247: - Fix Version/s: 3.0.2 2.4.8 > scipy installation fails with PyPy >

[jira] [Commented] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218743#comment-17218743 ] Hyukjin Kwon commented on SPARK-33213: -- cc [~bryanc] FYI > Upgrade Apache Arrow to 2.0.0 >

[jira] [Commented] (SPARK-33217) Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218734#comment-17218734 ] Apache Spark commented on SPARK-33217: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33217) Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33217: Assignee: Apache Spark > Set upper bound of Pandas and PyArrow version in GitHub Actions

[jira] [Assigned] (SPARK-33217) Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33217: Assignee: (was: Apache Spark) > Set upper bound of Pandas and PyArrow version in

[jira] [Comment Edited] (SPARK-33189) Support PyArrow 2.0.0+

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218727#comment-17218727 ] Hyukjin Kwon edited comment on SPARK-33189 at 10/22/20, 4:26 AM: - This

[jira] [Updated] (SPARK-33190) Set upperbound of PyArrow version in GitHub Actions

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33190: - Fix Version/s: (was: 2.4.8) > Set upperbound of PyArrow version in GitHub Actions >

[jira] [Commented] (SPARK-33190) Set upperbound of PyArrow version in GitHub Actions

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218730#comment-17218730 ] Hyukjin Kwon commented on SPARK-33190: -- This was reverted in branch-2.4. See SPARK-33217 > Set

[jira] [Updated] (SPARK-33189) Support PyArrow 2.0.0+

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33189: - Fix Version/s: (was: 2.4.8) > Support PyArrow 2.0.0+ > -- > >

[jira] [Commented] (SPARK-33189) Support PyArrow 2.0.0+

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218727#comment-17218727 ] Hyukjin Kwon commented on SPARK-33189: -- This was reverted at

[jira] [Created] (SPARK-33217) Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4

2020-10-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33217: Summary: Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 Key: SPARK-33217 URL: https://issues.apache.org/jira/browse/SPARK-33217

[jira] [Updated] (SPARK-33217) Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33217: - Reporter: Hyukjin Kwon (was: Dongjoon Hyun) > Set upper bound of Pandas and PyArrow version in

[jira] [Resolved] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-21 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-33212. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29843

[jira] [Assigned] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-21 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-33212: --- Assignee: Chao Sun > Move to shaded clients for Hadoop 3.x profile >

[jira] [Closed] (SPARK-33216) Set upper bound of Pandas version in GitHub Actions in branch-2.4

2020-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-33216. - > Set upper bound of Pandas version in GitHub Actions in branch-2.4 >

[jira] [Resolved] (SPARK-33216) Set upper bound of Pandas version in GitHub Actions in branch-2.4

2020-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33216. --- Resolution: Duplicate > Set upper bound of Pandas version in GitHub Actions in branch-2.4 >

[jira] [Resolved] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33210. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30121

[jira] [Assigned] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33210: --- Assignee: Maxim Gekk > Set the rebasing mode for parquet INT96 type to `EXCEPTION` by

[jira] [Assigned] (SPARK-33216) Set upper bound of Pandas version in GitHub Actions in branch-2.4

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33216: Assignee: (was: Apache Spark) > Set upper bound of Pandas version in GitHub Actions

[jira] [Commented] (SPARK-33216) Set upper bound of Pandas version in GitHub Actions in branch-2.4

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218705#comment-17218705 ] Apache Spark commented on SPARK-33216: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33216) Set upper bound of Pandas version in GitHub Actions in branch-2.4

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33216: Assignee: Apache Spark > Set upper bound of Pandas version in GitHub Actions in

[jira] [Created] (SPARK-33216) Set upper bound of Pandas version in GitHub Actions in branch-2.4

2020-10-21 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33216: - Summary: Set upper bound of Pandas version in GitHub Actions in branch-2.4 Key: SPARK-33216 URL: https://issues.apache.org/jira/browse/SPARK-33216 Project: Spark

[jira] [Commented] (SPARK-33215) Speed up event log download by skipping UI rebuild

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218683#comment-17218683 ] Apache Spark commented on SPARK-33215: -- User 'baohe-zhang' has created a pull request for this

[jira] [Assigned] (SPARK-33215) Speed up event log download by skipping UI rebuild

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33215: Assignee: Apache Spark > Speed up event log download by skipping UI rebuild >

[jira] [Assigned] (SPARK-33215) Speed up event log download by skipping UI rebuild

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33215: Assignee: (was: Apache Spark) > Speed up event log download by skipping UI rebuild >

[jira] [Assigned] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33203: - Assignee: Alessandro Patti (was: Apache Spark) > Pyspark ml tests failing with

[jira] [Resolved] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33203. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30104

[jira] [Created] (SPARK-33215) Speed up event log download by skipping UI rebuild

2020-10-21 Thread Baohe Zhang (Jira)
Baohe Zhang created SPARK-33215: --- Summary: Speed up event log download by skipping UI rebuild Key: SPARK-33215 URL: https://issues.apache.org/jira/browse/SPARK-33215 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33189) Support PyArrow 2.0.0+

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218675#comment-17218675 ] Apache Spark commented on SPARK-33189: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-33189) Support PyArrow 2.0.0+

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218674#comment-17218674 ] Apache Spark commented on SPARK-33189: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-19297) Add ability for --packages tag to pull latest version

2020-10-21 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aoyuan Liao resolved SPARK-19297. - Target Version/s: 3.0.1 Resolution: Fixed > Add ability for --packages tag to pull

[jira] [Commented] (SPARK-19297) Add ability for --packages tag to pull latest version

2020-10-21 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218660#comment-17218660 ] Aoyuan Liao commented on SPARK-19297: - latest.release can be used. {code:java} // bin/spark-submit

[jira] [Commented] (SPARK-21529) Improve the error message for unsupported Uniontype

2020-10-21 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218653#comment-17218653 ] Aoyuan Liao commented on SPARK-21529: - [~teabot] I think catalyst still doesn't support uniontype.

[jira] [Updated] (SPARK-33197) Changes to spark.sql.analyzer.maxIterations do not take effect at runtime

2020-10-21 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33197: - Affects Version/s: (was: 3.0.0) 3.0.2 > Changes to

[jira] [Commented] (SPARK-33214) HiveExternalCatalogVersionsSuite shouldn't use or delete hard-coded /tmp directory

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218619#comment-17218619 ] Apache Spark commented on SPARK-33214: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33214) HiveExternalCatalogVersionsSuite shouldn't use or delete hard-coded /tmp directory

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33214: Assignee: (was: Apache Spark) > HiveExternalCatalogVersionsSuite shouldn't use or

[jira] [Assigned] (SPARK-33214) HiveExternalCatalogVersionsSuite shouldn't use or delete hard-coded /tmp directory

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33214: Assignee: Apache Spark > HiveExternalCatalogVersionsSuite shouldn't use or delete

[jira] [Created] (SPARK-33214) HiveExternalCatalogVersionsSuite shouldn't use or delete hard-coded /tmp directory

2020-10-21 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-33214: --- Summary: HiveExternalCatalogVersionsSuite shouldn't use or delete hard-coded /tmp directory Key: SPARK-33214 URL: https://issues.apache.org/jira/browse/SPARK-33214

[jira] [Resolved] (SPARK-33202) Fix BlockManagerDecommissioner to return the correct migration status

2020-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33202. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30116

[jira] [Assigned] (SPARK-33202) Fix BlockManagerDecommissioner to return the correct migration status

2020-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33202: - Assignee: Dongjoon Hyun > Fix BlockManagerDecommissioner to return the correct

[jira] [Created] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

2020-10-21 Thread Chao Sun (Jira)
Chao Sun created SPARK-33213: Summary: Upgrade Apache Arrow to 2.0.0 Key: SPARK-33213 URL: https://issues.apache.org/jira/browse/SPARK-33213 Project: Spark Issue Type: Dependency upgrade

[jira] [Assigned] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33212: Assignee: (was: Apache Spark) > Move to shaded clients for Hadoop 3.x profile >

[jira] [Assigned] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33212: Assignee: Apache Spark > Move to shaded clients for Hadoop 3.x profile >

[jira] [Commented] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218588#comment-17218588 ] Apache Spark commented on SPARK-33212: -- User 'sunchao' has created a pull request for this issue:

[jira] [Updated] (SPARK-29250) Upgrade to Hadoop 3.2.2

2020-10-21 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-29250: - Summary: Upgrade to Hadoop 3.2.2 (was: Upgrade to Hadoop 3.2.1 and move to shaded client) > Upgrade

[jira] [Created] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-21 Thread Chao Sun (Jira)
Chao Sun created SPARK-33212: Summary: Move to shaded clients for Hadoop 3.x profile Key: SPARK-33212 URL: https://issues.apache.org/jira/browse/SPARK-33212 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33064) Spark-shell does not display accented chara

2020-10-21 Thread Laurent GUEMAPPE (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent GUEMAPPE updated SPARK-33064: - Description: It seems to be a duplicate of *FLEX-18425*, which is duplicate of

[jira] [Updated] (SPARK-33211) Early state store eviction for left semi stream-stream join

2020-10-21 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-33211: - Parent: SPARK-32883 Issue Type: Sub-task (was: Improvement) > Early state store eviction for

[jira] [Created] (SPARK-33211) Early state store eviction for left semi stream-stream join

2020-10-21 Thread Cheng Su (Jira)
Cheng Su created SPARK-33211: Summary: Early state store eviction for left semi stream-stream join Key: SPARK-33211 URL: https://issues.apache.org/jira/browse/SPARK-33211 Project: Spark Issue

[jira] [Commented] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218564#comment-17218564 ] Apache Spark commented on SPARK-33210: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218563#comment-17218563 ] Apache Spark commented on SPARK-33210: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33210: Assignee: Apache Spark > Set the rebasing mode for parquet INT96 type to `EXCEPTION` by

[jira] [Assigned] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33210: Assignee: (was: Apache Spark) > Set the rebasing mode for parquet INT96 type to

[jira] [Created] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33210: -- Summary: Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default Key: SPARK-33210 URL: https://issues.apache.org/jira/browse/SPARK-33210 Project: Spark

[jira] [Assigned] (SPARK-33205) Bump snappy-java version to 1.1.8

2020-10-21 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-33205: --- Assignee: Takeshi Yamamuro > Bump snappy-java version to 1.1.8 >

[jira] [Resolved] (SPARK-33205) Bump snappy-java version to 1.1.8

2020-10-21 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-33205. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30120

[jira] [Updated] (SPARK-33209) Clean up unit test file UnsupportedOperationsSuite.scala

2020-10-21 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-33209: - Parent: SPARK-32883 Issue Type: Sub-task (was: Improvement) > Clean up unit test file

[jira] [Created] (SPARK-33209) Clean up unit test file UnsupportedOperationsSuite.scala

2020-10-21 Thread Cheng Su (Jira)
Cheng Su created SPARK-33209: Summary: Clean up unit test file UnsupportedOperationsSuite.scala Key: SPARK-33209 URL: https://issues.apache.org/jira/browse/SPARK-33209 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-21 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218545#comment-17218545 ] Cheng Su commented on SPARK-33207: -- Thank [~yumwang] for bringing up the issue. We don't need to launch

[jira] [Updated] (SPARK-33197) Changes to spark.sql.analyzer.maxIterations do not take effect at runtime

2020-10-21 Thread Yuning Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuning Zhang updated SPARK-33197: - Affects Version/s: 3.0.0 > Changes to spark.sql.analyzer.maxIterations do not take effect at

[jira] [Issue Comment Deleted] (SPARK-13493) json to DataFrame to parquet does not respect case sensitiveness

2020-10-21 Thread Hariharan Karthikeyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hariharan Karthikeyan updated SPARK-13493: -- Comment: was deleted (was: !Screen Shot 2020-10-21 at 11.02.34 AM.png! ) >

[jira] [Commented] (SPARK-26764) [SPIP] Spark Relational Cache

2020-10-21 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218374#comment-17218374 ] Nicholas Chammas commented on SPARK-26764: -- The SPIP PDF references a design doc, but I'm not

[jira] [Commented] (SPARK-13493) json to DataFrame to parquet does not respect case sensitiveness

2020-10-21 Thread Hariharan Karthikeyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218342#comment-17218342 ] Hariharan Karthikeyan commented on SPARK-13493: --- !Screen Shot 2020-10-21 at 11.02.34

[jira] [Updated] (SPARK-13493) json to DataFrame to parquet does not respect case sensitiveness

2020-10-21 Thread Hariharan Karthikeyan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hariharan Karthikeyan updated SPARK-13493: -- Attachment: Screen Shot 2020-10-21 at 11.02.34 AM.png > json to DataFrame to

[jira] [Created] (SPARK-33208) Update the document of SparkSession#sql

2020-10-21 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-33208: --- Summary: Update the document of SparkSession#sql Key: SPARK-33208 URL: https://issues.apache.org/jira/browse/SPARK-33208 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-33206: - Description: SPARK-21501 changed the spark shuffle index service to be based on memory instead

[jira] [Commented] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218306#comment-17218306 ] Lars Francke commented on SPARK-33206: -- I used YourKit (thank you for the free license!) and it

[jira] [Updated] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33207: Description: We only need to read 1 bucket, but it still launch 200 tasks. {code:sql} create

[jira] [Commented] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218290#comment-17218290 ] Yuming Wang commented on SPARK-33207: - cc [~chengsu] > Reduce the number of tasks launched after

[jira] [Created] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33207: --- Summary: Reduce the number of tasks launched after bucket pruning Key: SPARK-33207 URL: https://issues.apache.org/jira/browse/SPARK-33207 Project: Spark Issue

[jira] [Updated] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-33206: - Description: dSPARK-21501 changed the spark shuffle index service to be based on memory

[jira] [Commented] (SPARK-21501) Spark shuffle index cache size should be memory based

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218285#comment-17218285 ] Lars Francke commented on SPARK-21501: -- Just FYI for others stumbling across this: This has a bug

[jira] [Updated] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Francke updated SPARK-33206: - Attachment: image001(1).png > Spark Shuffle Index Cache calculates memory usage wrong >

[jira] [Created] (SPARK-33206) Spark Shuffle Index Cache calculates memory usage wrong

2020-10-21 Thread Lars Francke (Jira)
Lars Francke created SPARK-33206: Summary: Spark Shuffle Index Cache calculates memory usage wrong Key: SPARK-33206 URL: https://issues.apache.org/jira/browse/SPARK-33206 Project: Spark

[jira] [Commented] (SPARK-33205) Bump snappy-java version to 1.1.8

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218267#comment-17218267 ] Apache Spark commented on SPARK-33205: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33205) Bump snappy-java version to 1.1.8

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33205: Assignee: Apache Spark > Bump snappy-java version to 1.1.8 >

[jira] [Assigned] (SPARK-33205) Bump snappy-java version to 1.1.8

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33205: Assignee: (was: Apache Spark) > Bump snappy-java version to 1.1.8 >

[jira] [Created] (SPARK-33205) Bump snappy-java version to 1.1.8

2020-10-21 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-33205: Summary: Bump snappy-java version to 1.1.8 Key: SPARK-33205 URL: https://issues.apache.org/jira/browse/SPARK-33205 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218251#comment-17218251 ] Apache Spark commented on SPARK-33204: -- User 'akiyamaneko' has created a pull request for this

[jira] [Assigned] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33204: Assignee: (was: Apache Spark) > `Event Timeline` in Spark Job UI sometimes cannot

[jira] [Assigned] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33204: Assignee: Apache Spark > `Event Timeline` in Spark Job UI sometimes cannot be opened >

[jira] [Assigned] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33204: Assignee: Apache Spark > `Event Timeline` in Spark Job UI sometimes cannot be opened >

[jira] [Updated] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-21 Thread akiyamaneko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] akiyamaneko updated SPARK-33204: Summary: `Event Timeline` in Spark Job UI sometimes cannot be opened (was: `Event Timeline` in

[jira] [Updated] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot open

2020-10-21 Thread akiyamaneko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] akiyamaneko updated SPARK-33204: Attachment: reproduce.gif > `Event Timeline` in Spark Job UI sometimes cannot open >

[jira] [Created] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot open

2020-10-21 Thread akiyamaneko (Jira)
akiyamaneko created SPARK-33204: --- Summary: `Event Timeline` in Spark Job UI sometimes cannot open Key: SPARK-33204 URL: https://issues.apache.org/jira/browse/SPARK-33204 Project: Spark Issue

[jira] [Commented] (SPARK-33160) Allow saving/loading INT96 in parquet w/o rebasing

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218216#comment-17218216 ] Apache Spark commented on SPARK-33160: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Resolved] (SPARK-33196) Expose filtered aggregation API

2020-10-21 Thread Erwan Guyomarc'h (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erwan Guyomarc'h resolved SPARK-33196. -- Resolution: Won't Do > Expose filtered aggregation API >

[jira] [Commented] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218163#comment-17218163 ] Apache Spark commented on SPARK-33203: -- User 'AlessandroPatti' has created a pull request for this

[jira] [Assigned] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33203: Assignee: Apache Spark > Pyspark ml tests failing with rounding errors >

[jira] [Commented] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218162#comment-17218162 ] Apache Spark commented on SPARK-33203: -- User 'AlessandroPatti' has created a pull request for this

[jira] [Assigned] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33203: Assignee: Apache Spark > Pyspark ml tests failing with rounding errors >

[jira] [Assigned] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33203: Assignee: (was: Apache Spark) > Pyspark ml tests failing with rounding errors >

[jira] [Updated] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Alessandro Patti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Patti updated SPARK-33203: - Description: The tests _{{pyspark.ml.recommendation}}_ and

[jira] [Created] (SPARK-33203) Pyspark ml tests failing with rounding errors

2020-10-21 Thread Alessandro Patti (Jira)
Alessandro Patti created SPARK-33203: Summary: Pyspark ml tests failing with rounding errors Key: SPARK-33203 URL: https://issues.apache.org/jira/browse/SPARK-33203 Project: Spark Issue

[jira] [Commented] (SPARK-32785) interval with dangling part should not results null

2020-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218155#comment-17218155 ] Apache Spark commented on SPARK-32785: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-32185) User Guide - Monitoring

2020-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218143#comment-17218143 ] Hyukjin Kwon commented on SPARK-32185: -- [~a7prasad] is there any update on this :-)? > User Guide

[jira] [Resolved] (SPARK-32881) NoSuchElementException occurs during decommissioning

2020-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32881. --- Fix Version/s: 3.1.0 Assignee: Holden Karau Resolution: Fixed This is