[jira] [Created] (SPARK-40145) Create infra image when cut down branches

2022-08-18 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-40145: --- Summary: Create infra image when cut down branches Key: SPARK-40145 URL: https://issues.apache.org/jira/browse/SPARK-40145 Project: Spark Issue Type: Improveme

[jira] [Assigned] (SPARK-40107) Pull out empty2null conversion from FileFormatWriter

2022-08-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40107: --- Assignee: Allison Wang > Pull out empty2null conversion from FileFormatWriter > ---

[jira] [Resolved] (SPARK-40107) Pull out empty2null conversion from FileFormatWriter

2022-08-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40107. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37539 [https://gith

[jira] [Assigned] (SPARK-39791) In Spark 3.0 standalone cluster mode, unable to customize driver JVM path

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39791: Assignee: (was: Apache Spark) > In Spark 3.0 standalone cluster mode, unable to custo

[jira] [Commented] (SPARK-39791) In Spark 3.0 standalone cluster mode, unable to customize driver JVM path

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581682#comment-17581682 ] Apache Spark commented on SPARK-39791: -- User 'obobj' has created a pull request for

[jira] [Assigned] (SPARK-39791) In Spark 3.0 standalone cluster mode, unable to customize driver JVM path

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39791: Assignee: Apache Spark > In Spark 3.0 standalone cluster mode, unable to customize driver

[jira] [Assigned] (SPARK-40144) Standalone log-view can't load new

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40144: Assignee: Apache Spark > Standalone log-view can't load new > --

[jira] [Assigned] (SPARK-40144) Standalone log-view can't load new

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40144: Assignee: (was: Apache Spark) > Standalone log-view can't load new > ---

[jira] [Commented] (SPARK-40144) Standalone log-view can't load new

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581664#comment-17581664 ] Apache Spark commented on SPARK-40144: -- User 'obobj' has created a pull request for

[jira] [Commented] (SPARK-40144) Standalone log-view can't load new

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581665#comment-17581665 ] Apache Spark commented on SPARK-40144: -- User 'obobj' has created a pull request for

[jira] [Assigned] (SPARK-30115) Improve limit only query on datasource table

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30115: Assignee: Apache Spark > Improve limit only query on datasource table > -

[jira] [Assigned] (SPARK-30115) Improve limit only query on datasource table

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30115: Assignee: (was: Apache Spark) > Improve limit only query on datasource table > --

[jira] [Created] (SPARK-40144) Standalone log-view can't load new

2022-08-18 Thread Obobj (Jira)
Obobj created SPARK-40144: - Summary: Standalone log-view can't load new Key: SPARK-40144 URL: https://issues.apache.org/jira/browse/SPARK-40144 Project: Spark Issue Type: Bug Components: S

[jira] [Commented] (SPARK-40005) Self-contained examples with parameter descriptions in PySpark documentation

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581658#comment-17581658 ] Hyukjin Kwon commented on SPARK-40005: -- I removed ML since ML has its own dedicated

[jira] [Updated] (SPARK-40005) Self-contained examples with parameter descriptions in PySpark documentation

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40005: - Description: This JIRA aims to improve PySpark documentation in: - {{pyspark}} - {{pyspark.sql}}

[jira] [Resolved] (SPARK-35542) Bucketizer created for multiple columns with parameters splitsArray,  inputCols and outputCols can not be loaded after saving it.

2022-08-18 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-35542. Fix Version/s: 3.3.1 3.1.4 3.2.3 3.4.0

[jira] [Resolved] (SPARK-40120) Make pyspark.sql.readwriter examples self-contained

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40120. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37570 [https://gi

[jira] [Assigned] (SPARK-40120) Make pyspark.sql.readwriter examples self-contained

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40120: Assignee: Hyukjin Kwon > Make pyspark.sql.readwriter examples self-contained > --

[jira] [Commented] (SPARK-40143) ANSI mode: allow explicitly casting fraction strings as Integral types

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581635#comment-17581635 ] Apache Spark commented on SPARK-40143: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-40143) ANSI mode: allow explicitly casting fraction strings as Integral types

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581633#comment-17581633 ] Apache Spark commented on SPARK-40143: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-40143) ANSI mode: allow explicitly casting fraction strings as Integral types

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40143: Assignee: Gengliang Wang (was: Apache Spark) > ANSI mode: allow explicitly casting frac

[jira] [Assigned] (SPARK-40143) ANSI mode: allow explicitly casting fraction strings as Integral types

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40143: Assignee: Apache Spark (was: Gengliang Wang) > ANSI mode: allow explicitly casting frac

[jira] [Commented] (SPARK-37130) why spark-X.X.X-bin-without-hadoop.tgz does not provide spark-hive_X.jar (and spark-hive-thriftserver_X.jar)

2022-08-18 Thread Victor Tso (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581631#comment-17581631 ] Victor Tso commented on SPARK-37130: I have the same question. We want to have Hadoo

[jira] [Updated] (SPARK-40143) ANSI mode: allow explicitly casting fraction strings as Integral types

2022-08-18 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-40143: --- Summary: ANSI mode: allow explicitly casting fraction strings as Integral types (was: ANSI

[jira] [Created] (SPARK-40143) ANSI mode: explicitly casting String as Integral types should allow fraction strings

2022-08-18 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-40143: -- Summary: ANSI mode: explicitly casting String as Integral types should allow fraction strings Key: SPARK-40143 URL: https://issues.apache.org/jira/browse/SPARK-40143

[jira] [Commented] (SPARK-40142) Make pyspark.sql.functions examples self-contained

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581628#comment-17581628 ] Apache Spark commented on SPARK-40142: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-40142) Make pyspark.sql.functions examples self-contained

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40142: Assignee: (was: Apache Spark) > Make pyspark.sql.functions examples self-contained >

[jira] [Assigned] (SPARK-40142) Make pyspark.sql.functions examples self-contained

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40142: Assignee: Apache Spark > Make pyspark.sql.functions examples self-contained > ---

[jira] [Commented] (SPARK-40142) Make pyspark.sql.functions examples self-contained

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581627#comment-17581627 ] Apache Spark commented on SPARK-40142: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-40142) Make pyspark.sql.functions examples self-contained

2022-08-18 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-40142: Summary: Make pyspark.sql.functions examples self-contained Key: SPARK-40142 URL: https://issues.apache.org/jira/browse/SPARK-40142 Project: Spark Issue Type

[jira] [Resolved] (SPARK-39271) Upgrade pandas to 1.4.3

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39271. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37567 [https://gi

[jira] [Commented] (SPARK-40081) Add Document Parameters for pyspark.sql.streaming.query

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581606#comment-17581606 ] Hyukjin Kwon commented on SPARK-40081: -- [~dcoliversun] just asking. are you working

[jira] [Assigned] (SPARK-39271) Upgrade pandas to 1.4.3

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39271: Assignee: Yikun Jiang > Upgrade pandas to 1.4.3 > --- > >

[jira] [Commented] (SPARK-37751) Apache Commons Crypto doesn't support Java 11

2022-08-18 Thread Qiyuan Gong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581582#comment-17581582 ] Qiyuan Gong commented on SPARK-37751: - Hi [~benoit_roy] . Some hot fix for this issu

[jira] [Commented] (SPARK-40140) REST API for SQL level information does not show information on running queries

2022-08-18 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581580#comment-17581580 ] ming95 commented on SPARK-40140: I'm interested in this issue and I can fix it, 

[jira] [Assigned] (SPARK-40141) Task listener overloads no longer needed with JDK 8+

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40141: Assignee: (was: Apache Spark) > Task listener overloads no longer needed with JDK 8+

[jira] [Commented] (SPARK-40141) Task listener overloads no longer needed with JDK 8+

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581569#comment-17581569 ] Apache Spark commented on SPARK-40141: -- User 'ryan-johnson-databricks' has created

[jira] [Assigned] (SPARK-40141) Task listener overloads no longer needed with JDK 8+

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40141: Assignee: Apache Spark > Task listener overloads no longer needed with JDK 8+ > -

[jira] [Updated] (SPARK-36462) Allow Spark on Kube to operate without polling or watchers

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36462: -- Affects Version/s: 3.4.0 (was: 3.2.0) (wa

[jira] [Assigned] (SPARK-36462) Allow Spark on Kube to operate without polling or watchers

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36462: - Assignee: Holden Karau > Allow Spark on Kube to operate without polling or watchers > -

[jira] [Resolved] (SPARK-36462) Allow Spark on Kube to operate without polling or watchers

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36462. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36433 [https://

[jira] [Created] (SPARK-40141) Task listener overloads no longer needed with JDK 8+

2022-08-18 Thread Ryan Johnson (Jira)
Ryan Johnson created SPARK-40141: Summary: Task listener overloads no longer needed with JDK 8+ Key: SPARK-40141 URL: https://issues.apache.org/jira/browse/SPARK-40141 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-40106) Task failure handlers should always run if the task failed

2022-08-18 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-40106: -- Assignee: Ryan Johnson > Task failure handlers should always run if the task failed > ---

[jira] [Resolved] (SPARK-40106) Task failure handlers should always run if the task failed

2022-08-18 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-40106. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37531 [https://github

[jira] [Assigned] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4: Assignee: (was: Apache Spark) > Add config to toggle whether to automatically add def

[jira] [Assigned] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4: Assignee: Apache Spark > Add config to toggle whether to automatically add default values

[jira] [Reopened] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-4: --- Assignee: (was: Daniel) This is reverted via https://github.com/apache/spark/commit/5

[jira] [Updated] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-4: -- Fix Version/s: (was: 3.4.0) > Add config to toggle whether to automatically add default va

[jira] [Comment Edited] (SPARK-37751) Apache Commons Crypto doesn't support Java 11

2022-08-18 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581506#comment-17581506 ] Benoit Roy edited comment on SPARK-37751 at 8/18/22 8:10 PM: -

[jira] [Updated] (SPARK-37544) sequence over dates with month interval is producing incorrect results

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37544: -- Fix Version/s: 3.1.4 > sequence over dates with month interval is producing incorrect results

[jira] [Updated] (SPARK-40134) Update ORC to 1.7.6

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40134: -- Affects Version/s: 3.4.0 > Update ORC to 1.7.6 > --- > > Key:

[jira] [Updated] (SPARK-40134) Update ORC to 1.7.6

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40134: -- Affects Version/s: 3.3.0 > Update ORC to 1.7.6 > --- > > Key:

[jira] [Updated] (SPARK-40134) Update ORC to 1.7.6

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40134: -- Affects Version/s: (was: 3.4.0) > Update ORC to 1.7.6 > --- > >

[jira] [Updated] (SPARK-40134) Update ORC to 1.7.6

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40134: -- Fix Version/s: 3.4.0 > Update ORC to 1.7.6 > --- > > Key: SPAR

[jira] [Updated] (SPARK-40134) Update ORC to 1.7.6

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40134: -- Fix Version/s: (was: 3.4.0) > Update ORC to 1.7.6 > --- > >

[jira] [Resolved] (SPARK-40134) Update ORC to 1.7.6

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40134. --- Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue resolved by pul

[jira] [Assigned] (SPARK-40134) Update ORC to 1.7.6

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40134: - Assignee: William Hyun > Update ORC to 1.7.6 > --- > >

[jira] [Comment Edited] (SPARK-37751) Apache Commons Crypto doesn't support Java 11

2022-08-18 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581506#comment-17581506 ] Benoit Roy edited comment on SPARK-37751 at 8/18/22 7:54 PM: -

[jira] [Comment Edited] (SPARK-37751) Apache Commons Crypto doesn't support Java 11

2022-08-18 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581506#comment-17581506 ] Benoit Roy edited comment on SPARK-37751 at 8/18/22 7:48 PM: -

[jira] [Comment Edited] (SPARK-37751) Apache Commons Crypto doesn't support Java 11

2022-08-18 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581506#comment-17581506 ] Benoit Roy edited comment on SPARK-37751 at 8/18/22 7:48 PM: -

[jira] [Commented] (SPARK-37751) Apache Commons Crypto doesn't support Java 11

2022-08-18 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581506#comment-17581506 ] Benoit Roy commented on SPARK-37751: Hello, we have also encountered this issue afte

[jira] [Commented] (SPARK-39399) proxy-user not working for Spark on k8s in cluster deploy mode

2022-08-18 Thread Shrikant Prasad (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581495#comment-17581495 ] Shrikant Prasad commented on SPARK-39399: - [~dongjoon] [~hyukjin.kwon] Can you p

[jira] [Commented] (SPARK-39993) Spark on Kubernetes doesn't filter data by date

2022-08-18 Thread Shrikant Prasad (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581492#comment-17581492 ] Shrikant Prasad commented on SPARK-39993: - [~h.liashchuk] The code snippet you h

[jira] [Resolved] (SPARK-40136) Incorrect fragment of query context

2022-08-18 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40136. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37566 [https://github.com

[jira] [Resolved] (SPARK-40124) Update TPCDS v1.4 q32 for Plan Stability tests

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40124. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37554 [https://

[jira] [Assigned] (SPARK-40124) Update TPCDS v1.4 q32 for Plan Stability tests

2022-08-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40124: - Assignee: Kapil Singh > Update TPCDS v1.4 q32 for Plan Stability tests > --

[jira] [Commented] (SPARK-40000) Add config to toggle whether to automatically add default values for INSERTs without user-specified fields

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581440#comment-17581440 ] Apache Spark commented on SPARK-4: -- User 'dtenedor' has created a pull request

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581424#comment-17581424 ] Yeachan Park commented on SPARK-27142: -- Hi all, thanks a lot for working on this. I

[jira] [Comment Edited] (SPARK-27142) Provide REST API for SQL level information

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581424#comment-17581424 ] Yeachan Park edited comment on SPARK-27142 at 8/18/22 4:14 PM: ---

[jira] [Created] (SPARK-40140) REST API for SQL level information does not show information on running queries

2022-08-18 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-40140: Summary: REST API for SQL level information does not show information on running queries Key: SPARK-40140 URL: https://issues.apache.org/jira/browse/SPARK-40140 Proje

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Created] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-40139: Summary: Filter jobs by Job Group in UI & API Key: SPARK-40139 URL: https://issues.apache.org/jira/browse/SPARK-40139 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-38909) Encapsulate LevelDB used by ExternalShuffleBlockResolver and YarnShuffleService as LocalDB

2022-08-18 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-38909: --- Assignee: Yang Jie > Encapsulate LevelDB used by ExternalShuffleBlockResolv

[jira] [Resolved] (SPARK-38909) Encapsulate LevelDB used by ExternalShuffleBlockResolver and YarnShuffleService as LocalDB

2022-08-18 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38909. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 362

[jira] [Resolved] (SPARK-39975) Upgrade rocksdbjni to 7.4.5

2022-08-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39975. -- Fix Version/s: 3.4.0 Assignee: Yang Jie Resolution: Fixed Resolved by https://

[jira] [Commented] (SPARK-21487) WebUI-Executors Page results in "Request is a replay (34) attack"

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581363#comment-17581363 ] Apache Spark commented on SPARK-21487: -- User 'santosh-d3vpl3x' has created a pull r

[jira] [Commented] (SPARK-21487) WebUI-Executors Page results in "Request is a replay (34) attack"

2022-08-18 Thread Santosh Pingale (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581351#comment-17581351 ] Santosh Pingale commented on SPARK-21487: - I believe this still an issue with ke

[jira] [Resolved] (SPARK-40087) Support multiple Column drop in R

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40087. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37526 [https://gi

[jira] [Assigned] (SPARK-40087) Support multiple Column drop in R

2022-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40087: Assignee: Santosh Pingale > Support multiple Column drop in R > -

[jira] [Assigned] (SPARK-40120) Make pyspark.sql.readwriter examples self-contained

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40120: Assignee: Apache Spark > Make pyspark.sql.readwriter examples self-contained > --

[jira] [Assigned] (SPARK-40120) Make pyspark.sql.readwriter examples self-contained

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40120: Assignee: (was: Apache Spark) > Make pyspark.sql.readwriter examples self-contained >

[jira] [Commented] (SPARK-40120) Make pyspark.sql.readwriter examples self-contained

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581347#comment-17581347 ] Apache Spark commented on SPARK-40120: -- User 'HyukjinKwon' has created a pull reque

[jira] [Updated] (SPARK-39993) Spark on Kubernetes doesn't filter data by date

2022-08-18 Thread Hanna Liashchuk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanna Liashchuk updated SPARK-39993: Description: I'm creating a Dataset with type date and saving it into s3. When I read it a

[jira] [Assigned] (SPARK-40138) Implement DataFrame.mode

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40138: Assignee: Apache Spark > Implement DataFrame.mode > > >

[jira] [Commented] (SPARK-40138) Implement DataFrame.mode

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581315#comment-17581315 ] Apache Spark commented on SPARK-40138: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-40138) Implement DataFrame.mode

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581316#comment-17581316 ] Apache Spark commented on SPARK-40138: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-40138) Implement DataFrame.mode

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40138: Assignee: (was: Apache Spark) > Implement DataFrame.mode > >

[jira] [Created] (SPARK-40138) Implement DataFrame.mode

2022-08-18 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-40138: - Summary: Implement DataFrame.mode Key: SPARK-40138 URL: https://issues.apache.org/jira/browse/SPARK-40138 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-35542) Bucketizer created for multiple columns with parameters splitsArray,  inputCols and outputCols can not be loaded after saving it.

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35542: Assignee: Weichen Xu (was: Apache Spark) > Bucketizer created for multiple columns with

[jira] [Commented] (SPARK-35542) Bucketizer created for multiple columns with parameters splitsArray,  inputCols and outputCols can not be loaded after saving it.

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581272#comment-17581272 ] Apache Spark commented on SPARK-35542: -- User 'WeichenXu123' has created a pull requ

[jira] [Assigned] (SPARK-35542) Bucketizer created for multiple columns with parameters splitsArray,  inputCols and outputCols can not be loaded after saving it.

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35542: Assignee: Apache Spark (was: Weichen Xu) > Bucketizer created for multiple columns with

[jira] [Assigned] (SPARK-39271) Upgrade pandas to 1.4.3

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39271: Assignee: Apache Spark > Upgrade pandas to 1.4.3 > --- > >

[jira] [Assigned] (SPARK-39271) Upgrade pandas to 1.4.3

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39271: Assignee: (was: Apache Spark) > Upgrade pandas to 1.4.3 > --- > >

[jira] [Commented] (SPARK-39271) Upgrade pandas to 1.4.3

2022-08-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581266#comment-17581266 ] Apache Spark commented on SPARK-39271: -- User 'Yikun' has created a pull request for

[jira] [Updated] (SPARK-39271) Upgrade pandas to 1.4.3

2022-08-18 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-39271: Summary: Upgrade pandas to 1.4.3 (was: Upgrade pandas to 1.4.2) > Upgrade pandas to 1.4.3 > -

[jira] [Assigned] (SPARK-35542) Bucketizer created for multiple columns with parameters splitsArray,  inputCols and outputCols can not be loaded after saving it.

2022-08-18 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-35542: -- Assignee: Weichen Xu > Bucketizer created for multiple columns with parameters splitsArray, 

  1   2   >