[jira] [Updated] (SPARK-35484) structured streaming multiple queries can not be execute concurrently

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35484: - Target Version/s: (was: 3.1.1) > structured streaming multiple queries can not be execute conc

[jira] [Resolved] (SPARK-35485) WARN TaskSchedulerImpl: Initial job has not accepted any resources;

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35485. -- Resolution: Invalid It's very likely environment issue in your network or configuration errors

[jira] [Resolved] (SPARK-35511) Spark computes all rows during count() on a parquet file

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35511. -- Resolution: Duplicate > Spark computes all rows during count() on a parquet file > ---

[jira] [Commented] (SPARK-35518) Critical Vulnerabilities: log4j_log4j 1.2.17 shipped

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351546#comment-17351546 ] Hyukjin Kwon commented on SPARK-35518: -- Latest version of log4j is 1.2.17. We'll ha

[jira] [Updated] (SPARK-35523) Fix the default value in Data Source Options page

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35523: - Summary: Fix the default value in Data Source Options page (was: Fix the default value properly

[jira] [Commented] (SPARK-35524) Pass objects as parameters to SparkSQL UDFs

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351541#comment-17351541 ] Hyukjin Kwon commented on SPARK-35524: -- It wouldn't make much sense for SQL to supp

[jira] [Resolved] (SPARK-35524) Pass objects as parameters to SparkSQL UDFs

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35524. -- Resolution: Won't Fix > Pass objects as parameters to SparkSQL UDFs >

[jira] [Updated] (SPARK-35524) Pass objects as parameters to SparkSQL UDFs

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35524: - Issue Type: Improvement (was: New Feature) > Pass objects as parameters to SparkSQL UDFs >

[jira] [Updated] (SPARK-35524) Pass objects as parameters to SparkSQL UDFs

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35524: - Flags: (was: Important) > Pass objects as parameters to SparkSQL UDFs > --

[jira] [Updated] (SPARK-35524) Pass objects as parameters to SparkSQL UDFs

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35524: - Target Version/s: (was: 3.2.0) > Pass objects as parameters to SparkSQL UDFs > ---

[jira] [Updated] (SPARK-35525) Define UDTs in schemas using string format

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35525: - Flags: (was: Important) > Define UDTs in schemas using string format > ---

[jira] [Updated] (SPARK-35525) Define UDTs in schemas using string format

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35525: - Issue Type: Improvement (was: New Feature) > Define UDTs in schemas using string format > -

[jira] [Updated] (SPARK-35528) Add CREATE TABLE with OPTIONS at Data Source Options pages

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35528: - Summary: Add CREATE TABLE with OPTIONS at Data Source Options pages (was: Add more methods for

[jira] [Updated] (SPARK-35525) Define UDTs in schemas using string format

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35525: - Target Version/s: (was: 3.2.0) > Define UDTs in schemas using string format >

[jira] [Commented] (SPARK-35507) Move Python 3.9 installtation to the docker image for GitHub Actions

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351538#comment-17351538 ] Hyukjin Kwon commented on SPARK-35507: -- I will leave it assigned to you [~dongjoon]

[jira] [Assigned] (SPARK-35507) Move Python 3.9 installtation to the docker image for GitHub Actions

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35507: Assignee: Dongjoon Hyun > Move Python 3.9 installtation to the docker image for GitHub Ac

[jira] [Created] (SPARK-35529) Add fallback metrics for hash aggregate

2021-05-25 Thread Cheng Su (Jira)
Cheng Su created SPARK-35529: Summary: Add fallback metrics for hash aggregate Key: SPARK-35529 URL: https://issues.apache.org/jira/browse/SPARK-35529 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-35528) Add more methods for Data Source Options pages

2021-05-25 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35528: --- Summary: Add more methods for Data Source Options pages Key: SPARK-35528 URL: https://issues.apache.org/jira/browse/SPARK-35528 Project: Spark Issue Type: Sub-

[jira] [Assigned] (SPARK-35440) Add language type to `ExpressionInfo` for UDF

2021-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35440: --- Assignee: Linhong Liu > Add language type to `ExpressionInfo` for UDF > ---

[jira] [Resolved] (SPARK-35440) Add language type to `ExpressionInfo` for UDF

2021-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35440. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32587 [https://gith

[jira] [Assigned] (SPARK-35527) Fix HiveExternalCatalogVersionsSuite to pass with Java 11

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35527: Assignee: Kousuke Saruta (was: Apache Spark) > Fix HiveExternalCatalogVersionsSuite to p

[jira] [Commented] (SPARK-35527) Fix HiveExternalCatalogVersionsSuite to pass with Java 11

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351482#comment-17351482 ] Apache Spark commented on SPARK-35527: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35527) Fix HiveExternalCatalogVersionsSuite to pass with Java 11

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35527: Assignee: Apache Spark (was: Kousuke Saruta) > Fix HiveExternalCatalogVersionsSuite to p

[jira] [Created] (SPARK-35527) Fix HiveExternalCatalogVersionsSuite to pass with Java 11

2021-05-25 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35527: -- Summary: Fix HiveExternalCatalogVersionsSuite to pass with Java 11 Key: SPARK-35527 URL: https://issues.apache.org/jira/browse/SPARK-35527 Project: Spark

[jira] [Commented] (SPARK-35526) Re-cleanup `procedure syntax is deprecated` compilation warning in Scala 2.13

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351481#comment-17351481 ] Apache Spark commented on SPARK-35526: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-35526) Re-cleanup `procedure syntax is deprecated` compilation warning in Scala 2.13

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35526: Assignee: (was: Apache Spark) > Re-cleanup `procedure syntax is deprecated` compilati

[jira] [Assigned] (SPARK-35526) Re-cleanup `procedure syntax is deprecated` compilation warning in Scala 2.13

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35526: Assignee: Apache Spark > Re-cleanup `procedure syntax is deprecated` compilation warning

[jira] [Commented] (SPARK-35526) Re-cleanup `procedure syntax is deprecated` compilation warning in Scala 2.13

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351480#comment-17351480 ] Apache Spark commented on SPARK-35526: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-34271) Use majorMinorPatchVersion for Hive version parsing

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351478#comment-17351478 ] Apache Spark commented on SPARK-34271: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-35526) Re-cleanup `procedure syntax is deprecated` compilation warning in Scala 2.13

2021-05-25 Thread Yang Jie (Jira)
Yang Jie created SPARK-35526: Summary: Re-cleanup `procedure syntax is deprecated` compilation warning in Scala 2.13 Key: SPARK-35526 URL: https://issues.apache.org/jira/browse/SPARK-35526 Project: Spark

[jira] [Updated] (SPARK-34859) Vectorized parquet reader needs synchronization among pages for column index

2021-05-25 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-34859: - Priority: Critical (was: Major) > Vectorized parquet reader needs synchronization among pages for colum

[jira] [Commented] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

2021-05-25 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351473#comment-17351473 ] Yang Jie commented on SPARK-35496: -- ok [~dongjoon] > Upgrade Scala 2.13 to 2.13.7 > --

[jira] [Updated] (SPARK-35378) Eagerly execute non-root Command

2021-05-25 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Eagerly execute non-root Command (was: Eagerly execute Command) > Eagerly execute non-root

[jira] [Updated] (SPARK-34859) Vectorized parquet reader needs synchronization among pages for column index

2021-05-25 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-34859: - Labels: correctness (was: ) > Vectorized parquet reader needs synchronization among pages for column in

[jira] [Comment Edited] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2021-05-25 Thread dc-heros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351464#comment-17351464 ] dc-heros edited comment on SPARK-30696 at 5/26/21, 3:54 AM:

[jira] [Commented] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2021-05-25 Thread dc-heros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351464#comment-17351464 ] dc-heros commented on SPARK-30696: -- fromUTCtime and toUTCtime produced wrong result on

[jira] [Assigned] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30696: Assignee: (was: Apache Spark) > Wrong result of the combination of from_utc_timestamp

[jira] [Assigned] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30696: Assignee: Apache Spark > Wrong result of the combination of from_utc_timestamp and to_utc

[jira] [Commented] (SPARK-30696) Wrong result of the combination of from_utc_timestamp and to_utc_timestamp

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351463#comment-17351463 ] Apache Spark commented on SPARK-30696: -- User 'dgd-contributor' has created a pull r

[jira] [Assigned] (SPARK-35512) pyspark partitionBy may encounter 'OverflowError: cannot convert float infinity to integer'

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35512: Assignee: Apache Spark > pyspark partitionBy may encounter 'OverflowError: cannot convert

[jira] [Assigned] (SPARK-35512) pyspark partitionBy may encounter 'OverflowError: cannot convert float infinity to integer'

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35512: Assignee: (was: Apache Spark) > pyspark partitionBy may encounter 'OverflowError: can

[jira] [Commented] (SPARK-35512) pyspark partitionBy may encounter 'OverflowError: cannot convert float infinity to integer'

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351461#comment-17351461 ] Apache Spark commented on SPARK-35512: -- User 'nolanliou' has created a pull request

[jira] [Updated] (SPARK-34859) Vectorized parquet reader needs synchronization among pages for column index

2021-05-25 Thread Li Xian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Xian updated SPARK-34859: Description: the current implementation has a problem. the pages returned by `readNextFilteredRowGroup` m

[jira] [Updated] (SPARK-34859) Vectorized parquet reader needs synchronization among pages for column index

2021-05-25 Thread Li Xian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Xian updated SPARK-34859: Attachment: part-0-bee08cae-04cd-491c-9602-4c66791af3d0-c000.snappy.parquet > Vectorized parquet read

[jira] [Resolved] (SPARK-32195) Standardize warning types and messages

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32195. -- Resolution: Done > Standardize warning types and messages > --

[jira] [Resolved] (SPARK-32194) Standardize exceptions in PySpark

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32194. -- Fix Version/s: 3.2.0 Assignee: Hyukjin Kwon Resolution: Fixed Fixed in https:/

[jira] [Assigned] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35496: Assignee: Apache Spark > Upgrade Scala 2.13 to 2.13.7 > > >

[jira] [Assigned] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35496: Assignee: (was: Apache Spark) > Upgrade Scala 2.13 to 2.13.7 > --

[jira] [Created] (SPARK-35525) Define UDTs in schemas using string format

2021-05-25 Thread Julian Shalaby (Jira)
Julian Shalaby created SPARK-35525: -- Summary: Define UDTs in schemas using string format Key: SPARK-35525 URL: https://issues.apache.org/jira/browse/SPARK-35525 Project: Spark Issue Type: Ne

[jira] [Updated] (SPARK-35524) Pass objects as parameters to SparkSQL UDFs

2021-05-25 Thread Julian Shalaby (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Shalaby updated SPARK-35524: --- Shepherd: (was: Sean R. Owen) > Pass objects as parameters to SparkSQL UDFs >

[jira] [Created] (SPARK-35524) Pass objects as parameters to SparkSQL UDFs

2021-05-25 Thread Julian Shalaby (Jira)
Julian Shalaby created SPARK-35524: -- Summary: Pass objects as parameters to SparkSQL UDFs Key: SPARK-35524 URL: https://issues.apache.org/jira/browse/SPARK-35524 Project: Spark Issue Type: N

[jira] [Updated] (SPARK-35455) Enhance EliminateUnnecessaryJoin

2021-05-25 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35455: -- Priority: Major (was: Minor) > Enhance EliminateUnnecessaryJoin > >

[jira] [Updated] (SPARK-35455) Enhance EliminateUnnecessaryJoin

2021-05-25 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35455: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Enhance EliminateUnnecessaryJoi

[jira] [Commented] (SPARK-35332) Not Coalesce shuffle partitions when cache table

2021-05-25 Thread Xianghao Lu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351426#comment-17351426 ] Xianghao Lu commented on SPARK-35332: - Great, thank you very much for your work [~ul

[jira] [Commented] (SPARK-35504) count distinct asterisk

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351425#comment-17351425 ] Hyukjin Kwon commented on SPARK-35504: -- Thanks for confirmation and investigation m

[jira] [Created] (SPARK-35523) Fix the default value properly in Data Source Options page

2021-05-25 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35523: --- Summary: Fix the default value properly in Data Source Options page Key: SPARK-35523 URL: https://issues.apache.org/jira/browse/SPARK-35523 Project: Spark Issu

[jira] [Resolved] (SPARK-35521) List Python 3.8 installed libraries in build_and_test workflow

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35521. -- Resolution: Duplicate > List Python 3.8 installed libraries in build_and_test workflow > -

[jira] [Resolved] (SPARK-35506) Run tests with Python 3.9 in GitHub Actions

2021-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35506. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32657 [https://gi

[jira] [Assigned] (SPARK-35522) Introduce BinaryOps for BinaryType

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35522: Assignee: (was: Apache Spark) > Introduce BinaryOps for BinaryType >

[jira] [Assigned] (SPARK-35522) Introduce BinaryOps for BinaryType

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35522: Assignee: Apache Spark > Introduce BinaryOps for BinaryType > ---

[jira] [Commented] (SPARK-35522) Introduce BinaryOps for BinaryType

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351373#comment-17351373 ] Apache Spark commented on SPARK-35522: -- User 'xinrong-databricks' has created a pul

[jira] [Created] (SPARK-35522) Introduce BinaryOps for BinaryType

2021-05-25 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35522: Summary: Introduce BinaryOps for BinaryType Key: SPARK-35522 URL: https://issues.apache.org/jira/browse/SPARK-35522 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-35516) Storage UI tab Storage Level tool tip correction

2021-05-25 Thread Lidiya Nixon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351346#comment-17351346 ] Lidiya Nixon commented on SPARK-35516: -- I have raised a fix for this https://githu

[jira] [Commented] (SPARK-35516) Storage UI tab Storage Level tool tip correction

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351344#comment-17351344 ] Apache Spark commented on SPARK-35516: -- User 'lidiyag' has created a pull request f

[jira] [Assigned] (SPARK-35516) Storage UI tab Storage Level tool tip correction

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35516: Assignee: Apache Spark > Storage UI tab Storage Level tool tip correction > -

[jira] [Assigned] (SPARK-35516) Storage UI tab Storage Level tool tip correction

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35516: Assignee: (was: Apache Spark) > Storage UI tab Storage Level tool tip correction > --

[jira] [Resolved] (SPARK-35513) Upgrade joda-time to 2.10.10

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35513. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32661 [https://

[jira] [Assigned] (SPARK-35513) Upgrade joda-time to 2.10.10

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35513: - Assignee: Vinod KC > Upgrade joda-time to 2.10.10 > > >

[jira] [Resolved] (SPARK-35505) Remove APIs that have been deprecated in Koalas.

2021-05-25 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35505. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Updated] (SPARK-35514) Automatically update version index of DocSearch via release-tag.sh

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35514: -- Fix Version/s: (was: 3.1.2) 3.1.3 > Automatically update version index

[jira] [Commented] (SPARK-35521) List Python 3.8 installed libraries in build_and_test workflow

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351218#comment-17351218 ] Apache Spark commented on SPARK-35521: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-35521) List Python 3.8 installed libraries in build_and_test workflow

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35521: Assignee: Apache Spark > List Python 3.8 installed libraries in build_and_test workflow >

[jira] [Assigned] (SPARK-35521) List Python 3.8 installed libraries in build_and_test workflow

2021-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35521: Assignee: (was: Apache Spark) > List Python 3.8 installed libraries in build_and_test

[jira] [Updated] (SPARK-35341) Introduce ExtentionDtypeOps

2021-05-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35341: - Description: {{Now ___and, __or,_ ___rand, and __ror___ are not data type based.}}

[jira] [Created] (SPARK-35521) List Python 3.8 installed libraries in build_and_test workflow

2021-05-25 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35521: Summary: List Python 3.8 installed libraries in build_and_test workflow Key: SPARK-35521 URL: https://issues.apache.org/jira/browse/SPARK-35521 Project: Spark

[jira] [Updated] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35496: -- Description: This issue aims to upgrade to Scala 2.13.7. Scala 2.13.6 released(https://github

[jira] [Updated] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35496: -- Description: Scala 2.13.6 released(https://github.com/scala/scala/releases/tag/v2.13.6) Howev

[jira] [Commented] (SPARK-35496) Upgrade Scala 2.13 from 2.13.5 to 2.13.6

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351187#comment-17351187 ] Dongjoon Hyun commented on SPARK-35496: --- Hi, [~LuciferYang]. Let's reuse this issu

[jira] [Reopened] (SPARK-35496) Upgrade Scala 2.13 from 2.13.5 to 2.13.6

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-35496: --- Assignee: (was: Apache Spark) > Upgrade Scala 2.13 from 2.13.5 to 2.13.6 > ---

[jira] [Updated] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

2021-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35496: -- Summary: Upgrade Scala 2.13 to 2.13.7 (was: Upgrade Scala 2.13 from 2.13.5 to 2.13.6) > Upgr

[jira] [Resolved] (SPARK-35514) Automatically update version index of DocSearch via release-tag.sh

2021-05-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35514. Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by p

[jira] [Updated] (SPARK-35520) Spark-SQL test fails on IBM Z for certain config combinations.

2021-05-25 Thread Simrit Kaur (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simrit Kaur updated SPARK-35520: Description: Some queries of SQL related test cases: in-joins.sql, in-order-by.sql,  not-in-group-

[jira] [Created] (SPARK-35520) Spark-SQL test fails on IBM Z for certain config combinations.

2021-05-25 Thread Simrit Kaur (Jira)
Simrit Kaur created SPARK-35520: --- Summary: Spark-SQL test fails on IBM Z for certain config combinations. Key: SPARK-35520 URL: https://issues.apache.org/jira/browse/SPARK-35520 Project: Spark

[jira] [Commented] (SPARK-35517) Critical Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar

2021-05-25 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351134#comment-17351134 ] Vinod KC commented on SPARK-35517: -- [~ldeflandre], In Spark 3.2.0, SPARK-34784 upgraded

[jira] [Updated] (SPARK-35517) Critical Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar

2021-05-25 Thread Louis DEFLANDRE (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Louis DEFLANDRE updated SPARK-35517: Description: Vulnerabilities scanner is highlighting following CRITICAL vulnerabilities in

[jira] [Updated] (SPARK-35518) Critical Vulnerabilities: log4j_log4j 1.2.17 shipped

2021-05-25 Thread Louis DEFLANDRE (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Louis DEFLANDRE updated SPARK-35518: Description: Vulnerabilities scanner is highlighting following CRITICAL vulnerabilities in

[jira] [Updated] (SPARK-35519) Critical Vulnerabilities: nimbusds_nimbus-jose-jwt 4.41.1 shipped

2021-05-25 Thread Louis DEFLANDRE (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Louis DEFLANDRE updated SPARK-35519: Description: Vulnerabilities scanner is highlighting following CRITICAL vulnerabilities in

[jira] [Created] (SPARK-35519) Critical Vulnerabilities: nimbusds_nimbus-jose-jwt 4.41.1 shipped

2021-05-25 Thread Louis DEFLANDRE (Jira)
Louis DEFLANDRE created SPARK-35519: --- Summary: Critical Vulnerabilities: nimbusds_nimbus-jose-jwt 4.41.1 shipped Key: SPARK-35519 URL: https://issues.apache.org/jira/browse/SPARK-35519 Project: Spar

[jira] [Issue Comment Deleted] (SPARK-10388) Public dataset loader interface

2021-05-25 Thread Gaurav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Kumar updated SPARK-10388: - Comment: was deleted (was: I want to work on this issue [~mengxr], yet I am new to opensource. I

[jira] [Updated] (SPARK-35517) Critical Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar

2021-05-25 Thread Louis DEFLANDRE (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Louis DEFLANDRE updated SPARK-35517: Summary: Critical Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-

[jira] [Created] (SPARK-35518) Critical Vulnerabilities: log4j_log4j 1.2.17 shipped

2021-05-25 Thread Louis DEFLANDRE (Jira)
Louis DEFLANDRE created SPARK-35518: --- Summary: Critical Vulnerabilities: log4j_log4j 1.2.17 shipped Key: SPARK-35518 URL: https://issues.apache.org/jira/browse/SPARK-35518 Project: Spark Is

[jira] [Commented] (SPARK-10388) Public dataset loader interface

2021-05-25 Thread Gaurav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351097#comment-17351097 ] Gaurav Kumar commented on SPARK-10388: -- I want to work on this issue [~mengxr], yet

[jira] [Created] (SPARK-35517) Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar

2021-05-25 Thread Louis DEFLANDRE (Jira)
Louis DEFLANDRE created SPARK-35517: --- Summary: Vulnerabilities: jackson-databind 2.4.0 shipped with htrace-core4-4.1.0-incubating.jar Key: SPARK-35517 URL: https://issues.apache.org/jira/browse/SPARK-35517

[jira] [Created] (SPARK-35516) Storage UI tab Storage Level tool tip correction

2021-05-25 Thread jobit mathew (Jira)
jobit mathew created SPARK-35516: Summary: Storage UI tab Storage Level tool tip correction Key: SPARK-35516 URL: https://issues.apache.org/jira/browse/SPARK-35516 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35396) Support to manual close/release entries in MemoryStore and InMemoryRelation instead of replying on GC

2021-05-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-35396: - Issue Type: Improvement (was: New Feature) Priority: Minor (was: Major) > Support to man

[jira] [Assigned] (SPARK-35396) Support to manual close/release entries in MemoryStore and InMemoryRelation instead of replying on GC

2021-05-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35396: Assignee: Apache Spark > Support to manual close/release entries in MemoryStore and InMem

[jira] [Resolved] (SPARK-35396) Support to manual close/release entries in MemoryStore and InMemoryRelation instead of replying on GC

2021-05-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35396. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32534 [https://gi

[jira] [Assigned] (SPARK-35447) optimize skew join before coalescing shuffle partitions

2021-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35447: --- Assignee: Wenchen Fan > optimize skew join before coalescing shuffle partitions > -

[jira] [Resolved] (SPARK-35447) optimize skew join before coalescing shuffle partitions

2021-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35447. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32594 [https://gith

[jira] [Resolved] (SPARK-29223) Kafka source: offset by timestamp - allow specifying timestamp for "all partitions"

2021-05-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-29223. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32609 [https://gi

  1   2   >