[jira] [Assigned] (SPARK-40863) Upgrade dropwizard metrics from 4.2.10 to 4.2.12

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40863: Assignee: Apache Spark > Upgrade dropwizard metrics from 4.2.10 to 4.2.12 > -

[jira] [Assigned] (SPARK-40863) Upgrade dropwizard metrics from 4.2.10 to 4.2.12

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40863: Assignee: (was: Apache Spark) > Upgrade dropwizard metrics from 4.2.10 to 4.2.12 > --

[jira] [Commented] (SPARK-40863) Upgrade dropwizard metrics from 4.2.10 to 4.2.12

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17621524#comment-17621524 ] Apache Spark commented on SPARK-40863: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-40864) Remove pip/setuptools dynamic upgrade

2022-10-21 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-40864: --- Summary: Remove pip/setuptools dynamic upgrade Key: SPARK-40864 URL: https://issues.apache.org/jira/browse/SPARK-40864 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-40865) Upgrade jodatime to 2.12.0

2022-10-21 Thread Yang Jie (Jira)
Yang Jie created SPARK-40865: Summary: Upgrade jodatime to 2.12.0 Key: SPARK-40865 URL: https://issues.apache.org/jira/browse/SPARK-40865 Project: Spark Issue Type: Improvement Componen

[jira] [Assigned] (SPARK-40865) Upgrade jodatime to 2.12.0

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40865: Assignee: Apache Spark > Upgrade jodatime to 2.12.0 > -- > >

[jira] [Assigned] (SPARK-40865) Upgrade jodatime to 2.12.0

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40865: Assignee: (was: Apache Spark) > Upgrade jodatime to 2.12.0 >

[jira] [Commented] (SPARK-40865) Upgrade jodatime to 2.12.0

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17621893#comment-17621893 ] Apache Spark commented on SPARK-40865: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-40866) Rename Check Spark repo as Check Spark Docker repo in GA

2022-10-21 Thread Qian Sun (Jira)
Qian Sun created SPARK-40866: Summary: Rename Check Spark repo as Check Spark Docker repo in GA Key: SPARK-40866 URL: https://issues.apache.org/jira/browse/SPARK-40866 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-40866) Rename Check Spark repo as Check Spark Docker repo in GA

2022-10-21 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40866. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 18 [https://github.

[jira] [Assigned] (SPARK-40866) Rename Check Spark repo as Check Spark Docker repo in GA

2022-10-21 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40866: --- Assignee: Qian Sun > Rename Check Spark repo as Check Spark Docker repo in GA > ---

[jira] [Resolved] (SPARK-40839) [Python] Implement `DataFrame.sample`

2022-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40839. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38310 [https://gi

[jira] [Commented] (SPARK-40750) Migrate type check failures of math expressions onto error classes

2022-10-21 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622114#comment-17622114 ] BingKun Pan commented on SPARK-40750: - I will work on it. > Migrate type check fail

[jira] [Created] (SPARK-40867) Flaky test ProtobufCatalystDataConversionSuite

2022-10-21 Thread Yang Jie (Jira)
Yang Jie created SPARK-40867: Summary: Flaky test ProtobufCatalystDataConversionSuite Key: SPARK-40867 URL: https://issues.apache.org/jira/browse/SPARK-40867 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-40867) Flaky test ProtobufCatalystDataConversionSuite

2022-10-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622141#comment-17622141 ] Yang Jie commented on SPARK-40867: -- cc [~rangadi]  and [~sanysand...@gmail.com]  > Fla

[jira] [Assigned] (SPARK-40864) Remove pip/setuptools dynamic upgrade

2022-10-21 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-40864: --- Assignee: Yikun Jiang > Remove pip/setuptools dynamic upgrade > ---

[jira] [Resolved] (SPARK-40864) Remove pip/setuptools dynamic upgrade

2022-10-21 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40864. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 17 [https://github.

[jira] [Updated] (SPARK-18188) Add checksum for block of broadcast

2022-10-21 Thread bettermouse (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bettermouse updated SPARK-18188: Description: {color:red}着色文本{color}There is an understanding issue for a long time: https://issue

[jira] [Created] (SPARK-40868) Avoid introducing too many partitions when bucketed scan disabled by sql planner

2022-10-21 Thread Xingchao, Zhang (Jira)
Xingchao, Zhang created SPARK-40868: --- Summary: Avoid introducing too many partitions when bucketed scan disabled by sql planner Key: SPARK-40868 URL: https://issues.apache.org/jira/browse/SPARK-40868

[jira] [Assigned] (SPARK-40868) Avoid introducing too many partitions when bucketed scan disabled by sql planner

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40868: Assignee: (was: Apache Spark) > Avoid introducing too many partitions when bucketed s

[jira] [Assigned] (SPARK-40868) Avoid introducing too many partitions when bucketed scan disabled by sql planner

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40868: Assignee: Apache Spark > Avoid introducing too many partitions when bucketed scan disable

[jira] [Commented] (SPARK-40868) Avoid introducing too many partitions when bucketed scan disabled by sql planner

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622178#comment-17622178 ] Apache Spark commented on SPARK-40868: -- User 'xingchaozh' has created a pull reques

[jira] [Created] (SPARK-40869) KubernetesConf.getResourceNamePrefix creates invalid name prefixes

2022-10-21 Thread Tobias Stadler (Jira)
Tobias Stadler created SPARK-40869: -- Summary: KubernetesConf.getResourceNamePrefix creates invalid name prefixes Key: SPARK-40869 URL: https://issues.apache.org/jira/browse/SPARK-40869 Project: Spark

[jira] [Assigned] (SPARK-40869) KubernetesConf.getResourceNamePrefix creates invalid name prefixes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40869: Assignee: Apache Spark > KubernetesConf.getResourceNamePrefix creates invalid name prefix

[jira] [Commented] (SPARK-40869) KubernetesConf.getResourceNamePrefix creates invalid name prefixes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622187#comment-17622187 ] Apache Spark commented on SPARK-40869: -- User 'tobiasstadler' has created a pull req

[jira] [Assigned] (SPARK-40869) KubernetesConf.getResourceNamePrefix creates invalid name prefixes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40869: Assignee: (was: Apache Spark) > KubernetesConf.getResourceNamePrefix creates invalid

[jira] [Commented] (SPARK-40869) KubernetesConf.getResourceNamePrefix creates invalid name prefixes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622188#comment-17622188 ] Apache Spark commented on SPARK-40869: -- User 'tobiasstadler' has created a pull req

[jira] [Created] (SPARK-40870) Upgrade docker actions to cleanup warning

2022-10-21 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-40870: --- Summary: Upgrade docker actions to cleanup warning Key: SPARK-40870 URL: https://issues.apache.org/jira/browse/SPARK-40870 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-40871) Upgrade actions/script to v6

2022-10-21 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-40871: --- Summary: Upgrade actions/script to v6 Key: SPARK-40871 URL: https://issues.apache.org/jira/browse/SPARK-40871 Project: Spark Issue Type: Sub-task Com

[jira] [Resolved] (SPARK-40854) Change default serialization from 'broken' CSV to Spark DF JSON

2022-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40854. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38300 [https://gi

[jira] [Assigned] (SPARK-40854) Change default serialization from 'broken' CSV to Spark DF JSON

2022-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40854: Assignee: Martin Grund > Change default serialization from 'broken' CSV to Spark DF JSON

[jira] [Commented] (SPARK-40750) Migrate type check failures of math expressions onto error classes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622214#comment-17622214 ] Apache Spark commented on SPARK-40750: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-40750) Migrate type check failures of math expressions onto error classes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40750: Assignee: (was: Apache Spark) > Migrate type check failures of math expressions onto

[jira] [Assigned] (SPARK-40750) Migrate type check failures of math expressions onto error classes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40750: Assignee: Apache Spark > Migrate type check failures of math expressions onto error class

[jira] [Commented] (SPARK-40750) Migrate type check failures of math expressions onto error classes

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622215#comment-17622215 ] Apache Spark commented on SPARK-40750: -- User 'panbingkun' has created a pull reques

[jira] [Created] (SPARK-40872) Fallback to original shuffle block when a push-merged shuffle chunk is zero-size

2022-10-21 Thread gaoyajun02 (Jira)
gaoyajun02 created SPARK-40872: -- Summary: Fallback to original shuffle block when a push-merged shuffle chunk is zero-size Key: SPARK-40872 URL: https://issues.apache.org/jira/browse/SPARK-40872 Project:

[jira] [Updated] (SPARK-40872) Fallback to original shuffle block when a push-merged shuffle chunk is zero-size

2022-10-21 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-40872: --- Parent: SPARK-33235 Issue Type: Sub-task (was: Improvement) > Fallback to original shuffle

[jira] [Assigned] (SPARK-40872) Fallback to original shuffle block when a push-merged shuffle chunk is zero-size

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40872: Assignee: (was: Apache Spark) > Fallback to original shuffle block when a push-merged

[jira] [Commented] (SPARK-40872) Fallback to original shuffle block when a push-merged shuffle chunk is zero-size

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622233#comment-17622233 ] Apache Spark commented on SPARK-40872: -- User 'gaoyajun02' has created a pull reques

[jira] [Assigned] (SPARK-40872) Fallback to original shuffle block when a push-merged shuffle chunk is zero-size

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40872: Assignee: Apache Spark > Fallback to original shuffle block when a push-merged shuffle ch

[jira] [Updated] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-40873: --- Attachment: part-0.parquet > Spark doesn't see some Parquet columns written from r-arrow > -

[jira] [Created] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
Daniel Darabos created SPARK-40873: -- Summary: Spark doesn't see some Parquet columns written from r-arrow Key: SPARK-40873 URL: https://issues.apache.org/jira/browse/SPARK-40873 Project: Spark

[jira] [Commented] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622251#comment-17622251 ] Daniel Darabos commented on SPARK-40873: Oh, I think I got it! With debug loggin

[jira] [Commented] (SPARK-32380) sparksql cannot access hive table while data in hbase

2022-10-21 Thread Mehul Thakkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622266#comment-17622266 ] Mehul Thakkar commented on SPARK-32380: --- I am waiting for the permanent fix for th

[jira] [Commented] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2022-10-21 Thread Mehul Thakkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622291#comment-17622291 ] Mehul Thakkar commented on SPARK-34210: --- Which Spark release should I use to get t

[jira] [Assigned] (SPARK-40865) Upgrade jodatime to 2.12.0

2022-10-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-40865: Assignee: Yang Jie > Upgrade jodatime to 2.12.0 > -- > >

[jira] [Resolved] (SPARK-40865) Upgrade jodatime to 2.12.0

2022-10-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40865. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38329 [https://gi

[jira] [Updated] (SPARK-40865) Upgrade jodatime to 2.12.0

2022-10-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40865: - Priority: Minor (was: Major) > Upgrade jodatime to 2.12.0 > -- > >

[jira] [Resolved] (SPARK-40863) Upgrade dropwizard metrics from 4.2.10 to 4.2.12

2022-10-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40863. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38328 [https://gi

[jira] [Assigned] (SPARK-40863) Upgrade dropwizard metrics from 4.2.10 to 4.2.12

2022-10-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-40863: Assignee: Yang Jie > Upgrade dropwizard metrics from 4.2.10 to 4.2.12 > -

[jira] (SPARK-32380) sparksql cannot access hive table while data in hbase

2022-10-21 Thread Mehul Thakkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380 ] Mehul Thakkar deleted comment on SPARK-32380: --- was (Author: JIRAUSER297345): I am waiting for the permanent fix for this issue in Spark 3.x > sparksql cannot access hive table while data i

[jira] [Created] (SPARK-40874) Fix Python UDFs with broadcasts when encryption is enabled

2022-10-21 Thread Peter Toth (Jira)
Peter Toth created SPARK-40874: -- Summary: Fix Python UDFs with broadcasts when encryption is enabled Key: SPARK-40874 URL: https://issues.apache.org/jira/browse/SPARK-40874 Project: Spark Issue

[jira] [Updated] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-40874: --- Summary: Fix broadcasts in Python UDFs when encryption is enabled (was: Fix Python UDFs with broadc

[jira] [Commented] (SPARK-40873) Spark doesn't see some Parquet columns written from r-arrow

2022-10-21 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622311#comment-17622311 ] Daniel Darabos commented on SPARK-40873: This works on the R side for dropping t

[jira] [Commented] (SPARK-40861) CVE-2022-42889 upgrade commons text library to 1.10.0 in spark 3.0.0

2022-10-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-40861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622322#comment-17622322 ] Bjørn Jørgensen commented on SPARK-40861: - [SPARK-40801|https://issues.apache.or

[jira] [Commented] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2022-10-21 Thread zhangzhanchang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622328#comment-17622328 ] zhangzhanchang commented on SPARK-34210: 3.0.1 Fix > Cannot create a record rea

[jira] [Comment Edited] (SPARK-40861) CVE-2022-42889 upgrade commons text library to 1.10.0 in spark 3.0.0

2022-10-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-40861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622322#comment-17622322 ] Bjørn Jørgensen edited comment on SPARK-40861 at 10/21/22 3:24 PM: ---

[jira] [Assigned] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40874: Assignee: Apache Spark > Fix broadcasts in Python UDFs when encryption is enabled > -

[jira] [Assigned] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40874: Assignee: (was: Apache Spark) > Fix broadcasts in Python UDFs when encryption is enab

[jira] [Commented] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622333#comment-17622333 ] Apache Spark commented on SPARK-40874: -- User 'peter-toth' has created a pull reques

[jira] [Closed] (SPARK-40861) CVE-2022-42889 upgrade commons text library to 1.10.0 in spark 3.0.0

2022-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-40861. - > CVE-2022-42889 upgrade commons text library to 1.10.0 in spark 3.0.0 > ---

[jira] [Resolved] (SPARK-40861) CVE-2022-42889 upgrade commons text library to 1.10.0 in spark 3.0.0

2022-10-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40861. --- Target Version/s: (was: 3.0.0) Resolution: Invalid According to the community ve

[jira] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874 ] Peter Toth deleted comment on SPARK-40874: was (Author: petertoth): The following Pyspark script: {noformat} bin/pyspark --conf spark.io.encryption.enabled=true ... bar = {"a": "aa", "b": "

[jira] [Updated] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-40874: --- The following Pyspark script: {noformat} bin/pyspark --conf spark.io.encryption.enabled=true ... bar =

[jira] [Updated] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-40874: --- Description: The following Pyspark script: {noformat} bin/pyspark --conf spark.io.encryption.enabled

[jira] [Commented] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

2022-10-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622378#comment-17622378 ] Steve Loughran commented on SPARK-38934: sounds like there is a race condition,

[jira] [Commented] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622446#comment-17622446 ] Apache Spark commented on SPARK-40796: -- User 'amaliujia' has created a pull request

[jira] [Commented] (SPARK-32380) sparksql cannot access hive table while data in hbase

2022-10-21 Thread Mehul Thakkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622497#comment-17622497 ] Mehul Thakkar commented on SPARK-32380: --- [~meimile] Since changes were merged on J

[jira] [Comment Edited] (SPARK-32380) sparksql cannot access hive table while data in hbase

2022-10-21 Thread Mehul Thakkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622497#comment-17622497 ] Mehul Thakkar edited comment on SPARK-32380 at 10/21/22 9:36 PM: -

[jira] [Comment Edited] (SPARK-32380) sparksql cannot access hive table while data in hbase

2022-10-21 Thread Mehul Thakkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622497#comment-17622497 ] Mehul Thakkar edited comment on SPARK-32380 at 10/21/22 10:07 PM:

[jira] [Assigned] (SPARK-40862) Unexpected operators when rewriting scalar subqueries with non-deterministic expressions

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40862: Assignee: Apache Spark > Unexpected operators when rewriting scalar subqueries with non-d

[jira] [Assigned] (SPARK-40862) Unexpected operators when rewriting scalar subqueries with non-deterministic expressions

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40862: Assignee: (was: Apache Spark) > Unexpected operators when rewriting scalar subqueries

[jira] [Commented] (SPARK-40862) Unexpected operators when rewriting scalar subqueries with non-deterministic expressions

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622508#comment-17622508 ] Apache Spark commented on SPARK-40862: -- User 'allisonwang-db' has created a pull re

[jira] [Commented] (SPARK-40862) Unexpected operators when rewriting scalar subqueries with non-deterministic expressions

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622509#comment-17622509 ] Apache Spark commented on SPARK-40862: -- User 'allisonwang-db' has created a pull re

[jira] [Commented] (SPARK-40875) Add .agg() to Connect DSL

2022-10-21 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622510#comment-17622510 ] Rui Wang commented on SPARK-40875: -- I am working on this. > Add .agg() to Connect DSL

[jira] [Created] (SPARK-40875) Add .agg() to Connect DSL

2022-10-21 Thread Rui Wang (Jira)
Rui Wang created SPARK-40875: Summary: Add .agg() to Connect DSL Key: SPARK-40875 URL: https://issues.apache.org/jira/browse/SPARK-40875 Project: Spark Issue Type: Sub-task Components:

[jira] [Created] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2022-10-21 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created SPARK-40876: --- Summary: Spark's Vectorized ParquetReader should support type promotions Key: SPARK-40876 URL: https://issues.apache.org/jira/browse/SPARK-40876 Project: Spark

[jira] [Updated] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2022-10-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated SPARK-40876: Description: Currently, when reading Parquet table using Spark's `VectorizedColumnReader`,

[jira] [Commented] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2022-10-21 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622514#comment-17622514 ] Chao Sun commented on SPARK-40876: -- Yes, Spark doesn't support int -> long for Parquet.

[jira] [Commented] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2022-10-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622517#comment-17622517 ] Alexey Kudinkin commented on SPARK-40876: - Alright, troubleshooting this more, t

[jira] [Comment Edited] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2022-10-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622518#comment-17622518 ] Alexey Kudinkin edited comment on SPARK-40876 at 10/21/22 11:53 PM: --

[jira] [Commented] (SPARK-40876) Spark's Vectorized ParquetReader should support type promotions

2022-10-21 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622518#comment-17622518 ] Alexey Kudinkin commented on SPARK-40876: - > This is because Parquet Int32, whic

[jira] [Commented] (SPARK-39404) Unable to query _metadata in streaming if getBatch returns multiple logical nodes in the DataFrame

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622520#comment-17622520 ] Apache Spark commented on SPARK-39404: -- User 'Yaohua628' has created a pull request

[jira] [Commented] (SPARK-39404) Unable to query _metadata in streaming if getBatch returns multiple logical nodes in the DataFrame

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622521#comment-17622521 ] Apache Spark commented on SPARK-39404: -- User 'Yaohua628' has created a pull request

[jira] [Commented] (SPARK-40796) Check the generated python protos in GitHub Actions

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622522#comment-17622522 ] Apache Spark commented on SPARK-40796: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-40877) Reimplement `crosstab` with dataframe operations

2022-10-21 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-40877: - Summary: Reimplement `crosstab` with dataframe operations Key: SPARK-40877 URL: https://issues.apache.org/jira/browse/SPARK-40877 Project: Spark Issue Type

[jira] [Assigned] (SPARK-40877) Reimplement `crosstab` with dataframe operations

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40877: Assignee: Apache Spark > Reimplement `crosstab` with dataframe operations > -

[jira] [Commented] (SPARK-40877) Reimplement `crosstab` with dataframe operations

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622527#comment-17622527 ] Apache Spark commented on SPARK-40877: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-40877) Reimplement `crosstab` with dataframe operations

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40877: Assignee: (was: Apache Spark) > Reimplement `crosstab` with dataframe operations > --

[jira] [Commented] (SPARK-40877) Reimplement `crosstab` with dataframe operations

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622528#comment-17622528 ] Apache Spark commented on SPARK-40877: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-40871) Upgrade actions/script to v6

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622530#comment-17622530 ] Apache Spark commented on SPARK-40871: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-40871) Upgrade actions/script to v6

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622531#comment-17622531 ] Apache Spark commented on SPARK-40871: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-40871) Upgrade actions/script to v6

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40871: Assignee: (was: Apache Spark) > Upgrade actions/script to v6 > --

[jira] [Commented] (SPARK-40752) Migrate type check failures of misc expressions onto error classes

2022-10-21 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622532#comment-17622532 ] BingKun Pan commented on SPARK-40752: - I work on it. > Migrate type check failures

[jira] [Assigned] (SPARK-40871) Upgrade actions/script to v6

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40871: Assignee: Apache Spark > Upgrade actions/script to v6 > > >

[jira] [Resolved] (SPARK-40871) Upgrade actions/script to v6

2022-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40871. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38341 [https://gi

[jira] [Assigned] (SPARK-40871) Upgrade actions/script to v6

2022-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40871: Assignee: Yikun Jiang > Upgrade actions/script to v6 > > >

[jira] [Assigned] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40874: Assignee: Peter Toth > Fix broadcasts in Python UDFs when encryption is enabled > ---

[jira] [Resolved] (SPARK-40874) Fix broadcasts in Python UDFs when encryption is enabled

2022-10-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40874. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38334 [https://gi

[jira] [Assigned] (SPARK-40870) Upgrade docker actions to cleanup warning

2022-10-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40870: Assignee: Apache Spark > Upgrade docker actions to cleanup warning >

  1   2   >