[jira] [Resolved] (SPARK-37113) Upgrade Parquet to 1.12.2

2021-10-25 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-37113. - Fix Version/s: 3.2.1 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37113) Upgrade Parquet to 1.12.2

2021-10-25 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-37113: --- Assignee: Chao Sun > Upgrade Parquet to 1.12.2 > - > >

[jira] [Comment Edited] (SPARK-37068) Confusing tgz filename for download

2021-10-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433205#comment-17433205 ] Hyukjin Kwon edited comment on SPARK-37068 at 10/26/21, 5:11 AM: - The

[jira] [Resolved] (SPARK-36845) Inline type hint files

2021-10-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36845. -- Resolution: Done > Inline type hint files > -- > > Key:

[jira] [Resolved] (SPARK-37096) Where clause and where operator will report error on varchar column type

2021-10-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37096. -- Resolution: Duplicate > Where clause and where operator will report error on varchar column

[jira] [Commented] (SPARK-37110) Add java17 support for spark pull request builds

2021-10-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434113#comment-17434113 ] Hyukjin Kwon commented on SPARK-37110: -- we should probably set the scheduled job instead of making

[jira] [Commented] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434114#comment-17434114 ] Hyukjin Kwon commented on SPARK-37109: -- Jenkins will be dropped at the end of this year. We could

[jira] [Commented] (SPARK-37111) RDD file loading APIs throw URISyntaxException when there is a colon in the file path

2021-10-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434112#comment-17434112 ] Hyukjin Kwon commented on SPARK-37111: -- This is from Hadoop's limitation I believe. > RDD file

[jira] [Commented] (SPARK-37116) Allow sequences (tuples and lists) as pivot values argument in PySpark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434111#comment-17434111 ] Apache Spark commented on SPARK-37116: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37116) Allow sequences (tuples and lists) as pivot values argument in PySpark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37116: Assignee: Apache Spark > Allow sequences (tuples and lists) as pivot values argument in

[jira] [Assigned] (SPARK-37116) Allow sequences (tuples and lists) as pivot values argument in PySpark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37116: Assignee: (was: Apache Spark) > Allow sequences (tuples and lists) as pivot values

[jira] [Commented] (SPARK-37116) Allow sequences (tuples and lists) as pivot values argument in PySpark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434110#comment-17434110 ] Apache Spark commented on SPARK-37116: -- User 'dchvn' has created a pull request for this issue:

[jira] [Commented] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434109#comment-17434109 ] Apache Spark commented on SPARK-37098: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434108#comment-17434108 ] Apache Spark commented on SPARK-37098: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-37116) Allow sequences (tuples and lists) as pivot values argument in PySpark

2021-10-25 Thread dch nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434107#comment-17434107 ] dch nguyen commented on SPARK-37116: work on this :v   > Allow sequences (tuples and lists) as

[jira] [Created] (SPARK-37116) Allow sequences (tuples and lists) as pivot values argument in PySpark

2021-10-25 Thread dch nguyen (Jira)
dch nguyen created SPARK-37116: -- Summary: Allow sequences (tuples and lists) as pivot values argument in PySpark Key: SPARK-37116 URL: https://issues.apache.org/jira/browse/SPARK-37116 Project: Spark

[jira] [Commented] (SPARK-37113) Upgrade Parquet to 1.12.2

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434106#comment-17434106 ] Apache Spark commented on SPARK-37113: -- User 'sunchao' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-37096) Where clause and where operator will report error on varchar column type

2021-10-25 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434104#comment-17434104 ] Saurabh Chawla edited comment on SPARK-37096 at 10/26/21, 4:54 AM: ---

[jira] [Commented] (SPARK-37096) Where clause and where operator will report error on varchar column type

2021-10-25 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434104#comment-17434104 ] Saurabh Chawla commented on SPARK-37096: This already fixed as the part of

[jira] [Assigned] (SPARK-37036) Add util function to raise advice warning for pandas API on Spark.

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37036: Assignee: Apache Spark > Add util function to raise advice warning for pandas API on

[jira] [Assigned] (SPARK-37036) Add util function to raise advice warning for pandas API on Spark.

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37036: Assignee: (was: Apache Spark) > Add util function to raise advice warning for pandas

[jira] [Commented] (SPARK-37036) Add util function to raise advice warning for pandas API on Spark.

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434096#comment-17434096 ] Apache Spark commented on SPARK-37036: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-37036) Add util function to raise advice warning for pandas API on Spark.

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434095#comment-17434095 ] Apache Spark commented on SPARK-37036: -- User 'itholic' has created a pull request for this issue:

[jira] [Updated] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-37098: - Fix Version/s: (was: 3.1.3) > Alter table properties should invalidate cache >

[jira] [Commented] (SPARK-37115) Replace HiveClient call with hive shim

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434088#comment-17434088 ] Apache Spark commented on SPARK-37115: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37115) Replace HiveClient call with hive shim

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37115: Assignee: (was: Apache Spark) > Replace HiveClient call with hive shim >

[jira] [Commented] (SPARK-37115) Replace HiveClient call with hive shim

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434087#comment-17434087 ] Apache Spark commented on SPARK-37115: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37115) Replace HiveClient call with hive shim

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37115: Assignee: Apache Spark > Replace HiveClient call with hive shim >

[jira] [Commented] (SPARK-37051) The filter operator gets wrong results in ORC's char type

2021-10-25 Thread wang-zhun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434086#comment-17434086 ] wang-zhun commented on SPARK-37051: --- We have the same problem, tcpds-q7 does not work. The

[jira] [Created] (SPARK-37115) Replace HiveClient call with hive shim

2021-10-25 Thread angerszhu (Jira)
angerszhu created SPARK-37115: - Summary: Replace HiveClient call with hive shim Key: SPARK-37115 URL: https://issues.apache.org/jira/browse/SPARK-37115 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37114) Support Submitting Jobs to Cloud Providers

2021-10-25 Thread Naga Vijayapuram (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naga Vijayapuram updated SPARK-37114: - Description: To be able to submit jobs to cloud providers, `spark-submit` can be

[jira] [Updated] (SPARK-37114) Support Submitting Jobs to Cloud Providers

2021-10-25 Thread Naga Vijayapuram (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naga Vijayapuram updated SPARK-37114: - Description: To be able to submit jobs to cloud providers, `spark-submit` can be

[jira] [Created] (SPARK-37114) Support Submitting Jobs to Cloud Providers

2021-10-25 Thread Naga Vijayapuram (Jira)
Naga Vijayapuram created SPARK-37114: Summary: Support Submitting Jobs to Cloud Providers Key: SPARK-37114 URL: https://issues.apache.org/jira/browse/SPARK-37114 Project: Spark Issue

[jira] [Commented] (SPARK-37043) Cancel all running job after AQE plan finished

2021-10-25 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434071#comment-17434071 ] XiDuo You commented on SPARK-37043: --- [~xkrogen] sorry for the late reply, I agree with mark this issue

[jira] [Assigned] (SPARK-37113) Upgrade Parquet to 1.12.2

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37113: Assignee: (was: Apache Spark) > Upgrade Parquet to 1.12.2 >

[jira] [Commented] (SPARK-37113) Upgrade Parquet to 1.12.2

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434067#comment-17434067 ] Apache Spark commented on SPARK-37113: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37113) Upgrade Parquet to 1.12.2

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37113: Assignee: Apache Spark > Upgrade Parquet to 1.12.2 > - > >

[jira] [Created] (SPARK-37113) Upgrade Parquet to 1.12.2

2021-10-25 Thread Chao Sun (Jira)
Chao Sun created SPARK-37113: Summary: Upgrade Parquet to 1.12.2 Key: SPARK-37113 URL: https://issues.apache.org/jira/browse/SPARK-37113 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-37112) Fix MiMa failure with Scala 2.13

2021-10-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37112. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34382

[jira] [Commented] (SPARK-33325) Spark executors pod are not shutting down when losing driver connection

2021-10-25 Thread Mihir Kelkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433993#comment-17433993 ] Mihir Kelkar commented on SPARK-33325: -- [~hadrien.kohl], [~hyukjin.kwon] : We are seeing this in

[jira] [Commented] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433962#comment-17433962 ] Apache Spark commented on SPARK-37011: -- User 'ueshin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp resolved SPARK-37011. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34384

[jira] [Assigned] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp reassigned SPARK-37011: --- Assignee: Shane Knapp > Upgrade flake8 to 3.9.0 or above in Jenkins >

[jira] [Commented] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433954#comment-17433954 ] Shane Knapp commented on SPARK-37011: - from the test build

[jira] [Commented] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433931#comment-17433931 ] Apache Spark commented on SPARK-37011: -- User 'shaneknapp' has created a pull request for this

[jira] [Assigned] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37011: Assignee: (was: Apache Spark) > Upgrade flake8 to 3.9.0 or above in Jenkins >

[jira] [Assigned] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37011: Assignee: Apache Spark > Upgrade flake8 to 3.9.0 or above in Jenkins >

[jira] [Commented] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433923#comment-17433923 ] Shane Knapp commented on SPARK-37011: - PR w/the updated conda config:

[jira] [Commented] (SPARK-37011) Upgrade flake8 to 3.9.0 or above in Jenkins

2021-10-25 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433916#comment-17433916 ] Shane Knapp commented on SPARK-37011: - done.   {noformat} (py36)

[jira] [Assigned] (SPARK-37102) Missing dependencies for hadoop-azure

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37102: Assignee: Apache Spark > Missing dependencies for hadoop-azure >

[jira] [Assigned] (SPARK-37102) Missing dependencies for hadoop-azure

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37102: Assignee: (was: Apache Spark) > Missing dependencies for hadoop-azure >

[jira] [Commented] (SPARK-37102) Missing dependencies for hadoop-azure

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433914#comment-17433914 ] Apache Spark commented on SPARK-37102: -- User 'vmalakhin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36956) model prediction in .mllib avoid conversion to breeze vector

2021-10-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-36956. Resolution: Fixed > model prediction in .mllib avoid conversion to breeze vector >

[jira] [Assigned] (SPARK-36956) model prediction in .mllib avoid conversion to breeze vector

2021-10-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-36956: -- Assignee: zhengruifeng > model prediction in .mllib avoid conversion to breeze vector >

[jira] [Commented] (SPARK-37102) Missing dependencies for hadoop-azure

2021-10-25 Thread Vasily Malakhin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433880#comment-17433880 ] Vasily Malakhin commented on SPARK-37102: - If I get following exclusions removed from

[jira] [Resolved] (SPARK-37105) Pass all UTs in `sql/hive` with Java 17

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37105. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34376

[jira] [Assigned] (SPARK-37105) Pass all UTs in `sql/hive` with Java 17

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37105: - Assignee: Yang Jie > Pass all UTs in `sql/hive` with Java 17 >

[jira] [Assigned] (SPARK-37072) Pass all UTs in `repl` with Java 17

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37072: - Assignee: Yang Jie > Pass all UTs in `repl` with Java 17 >

[jira] [Resolved] (SPARK-37072) Pass all UTs in `repl` with Java 17

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37072. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34368

[jira] [Resolved] (SPARK-37106) Pass all UTs in `yarn` with Java 17

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37106. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34381

[jira] [Assigned] (SPARK-37106) Pass all UTs in `yarn` with Java 17

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37106: - Assignee: Yang Jie > Pass all UTs in `yarn` with Java 17 >

[jira] [Assigned] (SPARK-37112) Fix MiMa failure with Scala 2.13

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37112: Assignee: Kousuke Saruta (was: Apache Spark) > Fix MiMa failure with Scala 2.13 >

[jira] [Assigned] (SPARK-37112) Fix MiMa failure with Scala 2.13

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37112: Assignee: Apache Spark (was: Kousuke Saruta) > Fix MiMa failure with Scala 2.13 >

[jira] [Commented] (SPARK-37112) Fix MiMa failure with Scala 2.13

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433873#comment-17433873 ] Apache Spark commented on SPARK-37112: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-37112) Fix MiMa failure with Scala 2.13

2021-10-25 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-37112: -- Summary: Fix MiMa failure with Scala 2.13 Key: SPARK-37112 URL: https://issues.apache.org/jira/browse/SPARK-37112 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-35437) Use expressions to filter Hive partitions at client side

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35437. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 32583

[jira] [Assigned] (SPARK-35437) Use expressions to filter Hive partitions at client side

2021-10-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35437: - Assignee: dzcxzl > Use expressions to filter Hive partitions at client side >

[jira] [Commented] (SPARK-37043) Cancel all running job after AQE plan finished

2021-10-25 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433834#comment-17433834 ] Erik Krogen commented on SPARK-37043: - Converted to subtask of SPARK-37063. > Cancel all running

[jira] [Updated] (SPARK-37043) Cancel all running job after AQE plan finished

2021-10-25 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-37043: Parent: SPARK-37063 Issue Type: Sub-task (was: Improvement) > Cancel all running job

[jira] [Updated] (SPARK-37111) RDD file loading APIs throw URISyntaxException when there is a colon in the file path

2021-10-25 Thread Brady Tello (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brady Tello updated SPARK-37111: Description: When a colon is present in a path to a file, many of Spark's RDD file loading APIs

[jira] [Created] (SPARK-37111) RDD file loading APIs throw URISyntaxException when there is a colon in the file path

2021-10-25 Thread Brady Tello (Jira)
Brady Tello created SPARK-37111: --- Summary: RDD file loading APIs throw URISyntaxException when there is a colon in the file path Key: SPARK-37111 URL: https://issues.apache.org/jira/browse/SPARK-37111

[jira] [Created] (SPARK-37110) Add java17 support for spark pull request builds

2021-10-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37110: --- Summary: Add java17 support for spark pull request builds Key: SPARK-37110 URL: https://issues.apache.org/jira/browse/SPARK-37110 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433799#comment-17433799 ] Yuming Wang commented on SPARK-37109: - cc [~shaneknapp] > Install Java 17 on all of the Jenkins

[jira] [Updated] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37109: Summary: Install Java 17 on all of the Jenkins workers (was: Add Java 17 in our Jenkins) >

[jira] [Created] (SPARK-37109) Add Java 17 in our Jenkins

2021-10-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37109: --- Summary: Add Java 17 in our Jenkins Key: SPARK-37109 URL: https://issues.apache.org/jira/browse/SPARK-37109 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-34883) Setting CSV reader option "multiLine" to "true" causes URISyntaxException when colon is in file path

2021-10-25 Thread Brady Tello (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433793#comment-17433793 ] Brady Tello commented on SPARK-34883: - This issue actually affects more APIs than the JSON multiline

[jira] [Assigned] (SPARK-37106) Pass all UTs in `yarn` with Java 17

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37106: Assignee: Apache Spark > Pass all UTs in `yarn` with Java 17 >

[jira] [Assigned] (SPARK-37106) Pass all UTs in `yarn` with Java 17

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37106: Assignee: (was: Apache Spark) > Pass all UTs in `yarn` with Java 17 >

[jira] [Commented] (SPARK-37106) Pass all UTs in `yarn` with Java 17

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433723#comment-17433723 ] Apache Spark commented on SPARK-37106: -- User 'LuciferYang' has created a pull request for this

[jira] [Updated] (SPARK-37106) Pass all UTs in `yarn` with Java 17

2021-10-25 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-37106: - Description: {code:java} build/mvn clean install -DskipTests -pl resource-managers/yarn -am -Pyarn

[jira] [Commented] (SPARK-37082) Implement histogram_numeric aggregate function in spark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433667#comment-17433667 ] Apache Spark commented on SPARK-37082: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37082) Implement histogram_numeric aggregate function in spark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37082: Assignee: (was: Apache Spark) > Implement histogram_numeric aggregate function in

[jira] [Assigned] (SPARK-37082) Implement histogram_numeric aggregate function in spark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37082: Assignee: Apache Spark > Implement histogram_numeric aggregate function in spark >

[jira] [Commented] (SPARK-37082) Implement histogram_numeric aggregate function in spark

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433668#comment-17433668 ] Apache Spark commented on SPARK-37082: -- User 'AngersZh' has created a pull request for this

[jira] [Created] (SPARK-37108) Expose make_date expression in R

2021-10-25 Thread Leona Yoda (Jira)
Leona Yoda created SPARK-37108: -- Summary: Expose make_date expression in R Key: SPARK-37108 URL: https://issues.apache.org/jira/browse/SPARK-37108 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433635#comment-17433635 ] Apache Spark commented on SPARK-37098: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433614#comment-17433614 ] Kent Yao commented on SPARK-37098: -- Issuse resolved by [https://github.com/apache/spark/pull/34365] >

[jira] [Resolved] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37098. -- Fix Version/s: 3.3.0 3.2.1 3.1.3 Resolution: Fixed >

[jira] [Assigned] (SPARK-37098) Alter table properties should invalidate cache

2021-10-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37098: Assignee: XiDuo You > Alter table properties should invalidate cache >

[jira] [Commented] (SPARK-37105) Pass all UTs in `sql/hive` with Java 17

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433595#comment-17433595 ] Apache Spark commented on SPARK-37105: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-37105) Pass all UTs in `sql/hive` with Java 17

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37105: Assignee: (was: Apache Spark) > Pass all UTs in `sql/hive` with Java 17 >

[jira] [Assigned] (SPARK-37105) Pass all UTs in `sql/hive` with Java 17

2021-10-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37105: Assignee: Apache Spark > Pass all UTs in `sql/hive` with Java 17 >

[jira] [Commented] (SPARK-36057) Support volcano/alternative schedulers

2021-10-25 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433576#comment-17433576 ] Yikun Jiang commented on SPARK-36057: - [~seedjeffwan] Hi, jiaxin, thanks for you attention, FYI