[jira] [Created] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

Haejoon Lee created SPARK-43433: --- Summary: Match `GroupBy.nth` behavior with new pandas behavior Key: SPARK-43433 URL: https://issues.apache.org/jira/browse/SPARK-43433 Project: Spark Issue

[jira] [Resolved] (SPARK-43425) Add TimestampNTZType to ColumnarBatchRow

[ https://issues.apache.org/jira/browse/SPARK-43425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43425. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41103

[jira] [Assigned] (SPARK-43425) Add TimestampNTZType to ColumnarBatchRow

[ https://issues.apache.org/jira/browse/SPARK-43425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43425: - Assignee: Fokko Driesprong > Add TimestampNTZType to ColumnarBatchRow >

[jira] [Created] (SPARK-43432) Fix `min_periods` for Rolling to work same as pandas

Haejoon Lee created SPARK-43432: --- Summary: Fix `min_periods` for Rolling to work same as pandas Key: SPARK-43432 URL: https://issues.apache.org/jira/browse/SPARK-43432 Project: Spark Issue

[jira] [Commented] (SPARK-40395) Provide query context in AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-40395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721168#comment-17721168 ] Jia Fan commented on SPARK-40395: - Already supported by [https://github.com/apache/spark/pull/37483] ？

[jira] [Commented] (SPARK-40174) Add query contexts to SparkException

[ https://issues.apache.org/jira/browse/SPARK-40174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721169#comment-17721169 ] Jia Fan commented on SPARK-40174: - Already supported by [https://github.com/apache/spark/pull/37483] ？  

[jira] [Assigned] (SPARK-43361) Allow deserializing protobuf enum fields as integers

[ https://issues.apache.org/jira/browse/SPARK-43361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43361: Assignee: Parth Upadhyay > Allow deserializing protobuf enum fields as integers >

[jira] [Resolved] (SPARK-43361) Allow deserializing protobuf enum fields as integers

[ https://issues.apache.org/jira/browse/SPARK-43361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43361. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41075

[jira] [Resolved] (SPARK-43428) Move some class utils to common/utils

[ https://issues.apache.org/jira/browse/SPARK-43428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-43428. --- Fix Version/s: 3.5.0 Resolution: Fixed > Move some class utils to

[jira] [Assigned] (SPARK-43316) Add more CTE SQL tests

[ https://issues.apache.org/jira/browse/SPARK-43316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43316: --- Assignee: Runyao.Chen > Add more CTE SQL tests > -- > >

[jira] [Resolved] (SPARK-43316) Add more CTE SQL tests

[ https://issues.apache.org/jira/browse/SPARK-43316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43316. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40989

[jira] [Created] (SPARK-43431) Spark Connect Scala Client should not expose `collectResult`

Rui Wang created SPARK-43431: Summary: Spark Connect Scala Client should not expose `collectResult` Key: SPARK-43431 URL: https://issues.apache.org/jira/browse/SPARK-43431 Project: Spark Issue

[jira] [Created] (SPARK-43430) ExecutePlanRequest should have the ability to set request options.

Martin Grund created SPARK-43430: Summary: ExecutePlanRequest should have the ability to set request options. Key: SPARK-43430 URL: https://issues.apache.org/jira/browse/SPARK-43430 Project: Spark

[jira] [Commented] (SPARK-43421) Implement changelog checkpointing for RocksDB state store

[ https://issues.apache.org/jira/browse/SPARK-43421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721085#comment-17721085 ] Hudson commented on SPARK-43421: User 'chaoqin-li1123' has created a pull request for this issue:

[jira] [Updated] (SPARK-43429) Add default/active SparkSession APIs

[ https://issues.apache.org/jira/browse/SPARK-43429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-43429: -- Epic Link: SPARK-42554 > Add default/active SparkSession APIs >

[jira] [Created] (SPARK-43429) Add default/active SparkSession APIs

Herman van Hövell created SPARK-43429: - Summary: Add default/active SparkSession APIs Key: SPARK-43429 URL: https://issues.apache.org/jira/browse/SPARK-43429 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43418) add SparkSession.Builder.getOrCreate()

[ https://issues.apache.org/jira/browse/SPARK-43418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-43418. --- Fix Version/s: 3.5.0 Resolution: Fixed > add

[jira] [Updated] (SPARK-43418) add SparkSession.Builder.getOrCreate()

[ https://issues.apache.org/jira/browse/SPARK-43418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-43418: -- Epic Link: SPARK-42554 > add SparkSession.Builder.getOrCreate() >

[jira] [Comment Edited] (SPARK-43421) Implement changelog checkpointing for RocksDB state store

[ https://issues.apache.org/jira/browse/SPARK-43421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721037#comment-17721037 ] Chaoqin Li edited comment on SPARK-43421 at 5/9/23 6:19 PM: Design doc:

[jira] [Created] (SPARK-43428) Move some class utils to common/utils

Rui Wang created SPARK-43428: Summary: Move some class utils to common/utils Key: SPARK-43428 URL: https://issues.apache.org/jira/browse/SPARK-43428 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-43235) ClientDistributedCacheManager doesn't set the LocalResourceVisibility.PRIVATE if isPublic throws exception

[ https://issues.apache.org/jira/browse/SPARK-43235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720979#comment-17720979 ] Pralabh Kumar commented on SPARK-43235: --- can any one please look into this . If ok I can create PR

[jira] [Assigned] (SPARK-41966) Add `CharType` and `TimestampNTZType` to PySpark API references

[ https://issues.apache.org/jira/browse/SPARK-41966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41966: - Assignee: Ruifeng Zheng (was: Apache Spark) > Add `CharType` and `TimestampNTZType`

[jira] [Assigned] (SPARK-43383) Add statistics rowCount for LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-43383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43383: - Assignee: yikaifei > Add statistics rowCount for LocalRelation >

[jira] [Resolved] (SPARK-43383) Add statistics rowCount for LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-43383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43383. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41064

[jira] [Resolved] (SPARK-42579) Extend function.lit() to match Literal.apply()

[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-42579. -- Resolution: Fixed > Extend function.lit() to match Literal.apply() >

[jira] [Assigned] (SPARK-43357) Spark AWS Glue date partition push down broken

[ https://issues.apache.org/jira/browse/SPARK-43357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43357: --- Assignee: Stijn De Haes > Spark AWS Glue date partition push down broken >

[jira] [Resolved] (SPARK-43357) Spark AWS Glue date partition push down broken

[ https://issues.apache.org/jira/browse/SPARK-43357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43357. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41035

[jira] [Updated] (SPARK-43427) Unsigned integer types are deserialized as signed numeric equivalents

[ https://issues.apache.org/jira/browse/SPARK-43427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43427: --- Description: h2. Issue Protobuf supports unsigned integer types, including `uint32` and

[jira] [Resolved] (SPARK-42604) Implement functions.typedlit

[ https://issues.apache.org/jira/browse/SPARK-42604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42604. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40355

[jira] [Assigned] (SPARK-42604) Implement functions.typedlit

[ https://issues.apache.org/jira/browse/SPARK-42604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42604: Assignee: Yang Jie > Implement functions.typedlit > > >

[jira] [Created] (SPARK-43427) Unsigned integer types are deserialized as signed numeric equivalents

Parth Upadhyay created SPARK-43427: -- Summary: Unsigned integer types are deserialized as signed numeric equivalents Key: SPARK-43427 URL: https://issues.apache.org/jira/browse/SPARK-43427 Project:

[jira] [Created] (SPARK-43426) Introducing a new SparkListenerEvent for jobs running in AQE mode

Faiz Halde created SPARK-43426: -- Summary: Introducing a new SparkListenerEvent for jobs running in AQE mode Key: SPARK-43426 URL: https://issues.apache.org/jira/browse/SPARK-43426 Project: Spark

[jira] (SPARK-43251) Assign a name to the error class _LEGACY_ERROR_TEMP_2015

[ https://issues.apache.org/jira/browse/SPARK-43251 ] Liang Yan deleted comment on SPARK-43251: --- was (Author: JIRAUSER299156): I will work on this issue. > Assign a name to the error class _LEGACY_ERROR_TEMP_2015 >

[jira] [Resolved] (SPARK-43423) Retry when `test_gmm` fails

[ https://issues.apache.org/jira/browse/SPARK-43423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43423. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41101

[jira] [Assigned] (SPARK-43423) Retry when `test_gmm` fails

[ https://issues.apache.org/jira/browse/SPARK-43423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43423: Assignee: Ruifeng Zheng > Retry when `test_gmm` fails > --- > >

[jira] [Updated] (SPARK-43425) Add TimestampNTZType to ColumnarBatchRow

[ https://issues.apache.org/jira/browse/SPARK-43425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43425: Fix Version/s: (was: 3.5.0) (was: 3.4.1) > Add TimestampNTZType to

[jira] [Created] (SPARK-43425) Add TimestampNTZType to ColumnarBatchRow

Fokko Driesprong created SPARK-43425: Summary: Add TimestampNTZType to ColumnarBatchRow Key: SPARK-43425 URL: https://issues.apache.org/jira/browse/SPARK-43425 Project: Spark Issue Type:

[jira] [Commented] (SPARK-43157) TreeNode tags can become corrupted and hang driver when the dataset is cached

[ https://issues.apache.org/jira/browse/SPARK-43157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720872#comment-17720872 ] ASF GitHub Bot commented on SPARK-43157: User 'robreeves' has created a pull request for this

[jira] [Commented] (SPARK-43390) DSv2 allows CTAS/RTAS to reserve schema nullability

[ https://issues.apache.org/jira/browse/SPARK-43390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720869#comment-17720869 ] ASF GitHub Bot commented on SPARK-43390: User 'pan3793' has created a pull request for this

[jira] [Commented] (SPARK-43390) DSv2 allows CTAS/RTAS to reserve schema nullability

[ https://issues.apache.org/jira/browse/SPARK-43390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720868#comment-17720868 ] ASF GitHub Bot commented on SPARK-43390: User 'pan3793' has created a pull request for this

[jira] [Assigned] (SPARK-43390) DSv2 allows CTAS/RTAS to reserve schema nullability

[ https://issues.apache.org/jira/browse/SPARK-43390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43390: --- Assignee: Cheng Pan > DSv2 allows CTAS/RTAS to reserve schema nullability >

[jira] [Resolved] (SPARK-43390) DSv2 allows CTAS/RTAS to reserve schema nullability

[ https://issues.apache.org/jira/browse/SPARK-43390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43390. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41070

[jira] [Commented] (SPARK-43286) Update CBC mode in aes_encrypt()/aes_decrypt() to not use KDF

[ https://issues.apache.org/jira/browse/SPARK-43286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720866#comment-17720866 ] ASF GitHub Bot commented on SPARK-43286: User 'sweisdb' has created a pull request for this

[jira] [Assigned] (SPARK-43286) Update CBC mode in aes_encrypt()/aes_decrypt() to not use KDF

[ https://issues.apache.org/jira/browse/SPARK-43286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-43286: Assignee: Steve Weis > Update CBC mode in aes_encrypt()/aes_decrypt() to not use KDF >

[jira] [Resolved] (SPARK-43286) Update CBC mode in aes_encrypt()/aes_decrypt() to not use KDF

[ https://issues.apache.org/jira/browse/SPARK-43286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43286. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40969

[jira] [Commented] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720864#comment-17720864 ] Martin Grund commented on SPARK-43351: -- 1.1 we had the same questions with Python and Scala and

[jira] [Comment Edited] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720831#comment-17720831 ] melin edited comment on SPARK-43338 at 5/9/23 7:38 AM: --- If the same hive database

[jira] [Commented] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720831#comment-17720831 ] melin commented on SPARK-43338: --- If the same hive database has parquet and hudi table, does

[jira] [Created] (SPARK-43424) Support vanila JDBC CHAR/VARCHAR through STS

Kent Yao created SPARK-43424: Summary: Support vanila JDBC CHAR/VARCHAR through STS Key: SPARK-43424 URL: https://issues.apache.org/jira/browse/SPARK-43424 Project: Spark Issue Type:

[jira] [Created] (SPARK-43423) Retry when `test_gmm` fails

Ruifeng Zheng created SPARK-43423: - Summary: Retry when `test_gmm` fails Key: SPARK-43423 URL: https://issues.apache.org/jira/browse/SPARK-43423 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-43422) Tags are lost on LogicalRelation when adding _metadata

Jan-Ole Sasse created SPARK-43422: - Summary: Tags are lost on LogicalRelation when adding _metadata Key: SPARK-43422 URL: https://issues.apache.org/jira/browse/SPARK-43422 Project: Spark

[jira] [Commented] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720798#comment-17720798 ] Kent Yao commented on SPARK-43338: -- Why not just use Catalog V2 API to implement a separate catalog

[jira] [Created] (SPARK-43420) Make DisableUnnecessaryBucketedScan smart with table cache

XiDuo You created SPARK-43420: - Summary: Make DisableUnnecessaryBucketedScan smart with table cache Key: SPARK-43420 URL: https://issues.apache.org/jira/browse/SPARK-43420 Project: Spark Issue

[jira] [Commented] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720794#comment-17720794 ] melin commented on SPARK-43338: --- You understand a big feature. Only one hms is accessed in a sparksession.