[jira] [Resolved] (SPARK-34420) Throw exception if non-streaming Deduplicate is not replaced by aggregate

[ https://issues.apache.org/jira/browse/SPARK-34420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34420. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31547

[jira] [Created] (SPARK-34421) Custom functions can't be used in temporary views with CTEs

Lauri Koobas created SPARK-34421: Summary: Custom functions can't be used in temporary views with CTEs Key: SPARK-34421 URL: https://issues.apache.org/jira/browse/SPARK-34421 Project: Spark

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282871#comment-17282871 ] Attila Zsolt Piros commented on SPARK-34389: @Ranju from the logs you attached it is clear

[jira] [Commented] (SPARK-34409) haversine distance spark function

[ https://issues.apache.org/jira/browse/SPARK-34409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282860#comment-17282860 ] Darshan Jani commented on SPARK-34409: -- Tagging some of members based previous Jira tickets to

[jira] [Commented] (SPARK-34410) Vincenty distance spark function

[ https://issues.apache.org/jira/browse/SPARK-34410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282859#comment-17282859 ] Darshan Jani commented on SPARK-34410: -- Tagging some of members based previous Jira tickets to

[jira] [Assigned] (SPARK-34127) Support table valued command

[ https://issues.apache.org/jira/browse/SPARK-34127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34127: Assignee: (was: Apache Spark) > Support table valued command >

[jira] [Commented] (SPARK-34127) Support table valued command

[ https://issues.apache.org/jira/browse/SPARK-34127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282844#comment-17282844 ] Apache Spark commented on SPARK-34127: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34127) Support table valued command

[ https://issues.apache.org/jira/browse/SPARK-34127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34127: Assignee: Apache Spark > Support table valued command > > >

[jira] [Resolved] (SPARK-34408) Refactor spark.udf.register to share the same path to generate UDF instance

[ https://issues.apache.org/jira/browse/SPARK-34408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34408. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31537

[jira] [Assigned] (SPARK-34408) Refactor spark.udf.register to share the same path to generate UDF instance

[ https://issues.apache.org/jira/browse/SPARK-34408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34408: Assignee: Hyukjin Kwon > Refactor spark.udf.register to share the same path to generate

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282825#comment-17282825 ] Ranju commented on SPARK-34389: --- Attila, I had a question on pod Creation timeout if you see this

[jira] [Resolved] (SPARK-34419) Move PartitionTransforms from java to scala directory

[ https://issues.apache.org/jira/browse/SPARK-34419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34419. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31546

[jira] [Assigned] (SPARK-34419) Move PartitionTransforms from java to scala directory

[ https://issues.apache.org/jira/browse/SPARK-34419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-34419: --- Assignee: Chao Sun > Move PartitionTransforms from java to scala directory >

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282821#comment-17282821 ] Ranju commented on SPARK-34389: --- Attila, Thanks for your response I also analyzed this pod Creation

[jira] [Commented] (SPARK-34420) Throw exception if non-streaming Deduplicate is not replaced by aggregate

[ https://issues.apache.org/jira/browse/SPARK-34420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282756#comment-17282756 ] Apache Spark commented on SPARK-34420: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34420) Throw exception if non-streaming Deduplicate is not replaced by aggregate

[ https://issues.apache.org/jira/browse/SPARK-34420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34420: Assignee: Apache Spark (was: L. C. Hsieh) > Throw exception if non-streaming

[jira] [Assigned] (SPARK-34420) Throw exception if non-streaming Deduplicate is not replaced by aggregate

[ https://issues.apache.org/jira/browse/SPARK-34420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34420: Assignee: L. C. Hsieh (was: Apache Spark) > Throw exception if non-streaming

[jira] [Commented] (SPARK-34420) Throw exception if non-streaming Deduplicate is not replaced by aggregate

[ https://issues.apache.org/jira/browse/SPARK-34420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282755#comment-17282755 ] Apache Spark commented on SPARK-34420: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-34420) Throw exception if non-streaming Deduplicate is not replaced by aggregate

L. C. Hsieh created SPARK-34420: --- Summary: Throw exception if non-streaming Deduplicate is not replaced by aggregate Key: SPARK-34420 URL: https://issues.apache.org/jira/browse/SPARK-34420 Project:

[jira] [Assigned] (SPARK-34419) Move PartitionTransforms from java to scala directory

[ https://issues.apache.org/jira/browse/SPARK-34419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34419: Assignee: Apache Spark > Move PartitionTransforms from java to scala directory >

[jira] [Assigned] (SPARK-34419) Move PartitionTransforms from java to scala directory

[ https://issues.apache.org/jira/browse/SPARK-34419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34419: Assignee: (was: Apache Spark) > Move PartitionTransforms from java to scala

[jira] [Commented] (SPARK-34419) Move PartitionTransforms from java to scala directory

[ https://issues.apache.org/jira/browse/SPARK-34419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282698#comment-17282698 ] Apache Spark commented on SPARK-34419: -- User 'sunchao' has created a pull request for this issue:

[jira] [Created] (SPARK-34419) Move PartitionTransforms from java to scala directory

Chao Sun created SPARK-34419: Summary: Move PartitionTransforms from java to scala directory Key: SPARK-34419 URL: https://issues.apache.org/jira/browse/SPARK-34419 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282614#comment-17282614 ] Attila Zsolt Piros commented on SPARK-34389: [~ranju] the description of this config is a

[jira] [Assigned] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-32953: Assignee: David Li > Lower memory usage in toPandas with Arrow self_destruct >

[jira] [Resolved] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-32953. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 29818

[jira] [Resolved] (SPARK-31949) Add spark.default.parallelism in SQLConf for isolated across session

[ https://issues.apache.org/jira/browse/SPARK-31949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-31949. - Resolution: Won't Fix > Add spark.default.parallelism in SQLConf for isolated across session >

[jira] [Assigned] (SPARK-34417) org.apache.spark.sql.DataFrameNaFunctions.fillMap(values: Seq[(String, Any)]) fails for column name having a dot

[ https://issues.apache.org/jira/browse/SPARK-34417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34417: Assignee: (was: Apache Spark) >

[jira] [Assigned] (SPARK-34417) org.apache.spark.sql.DataFrameNaFunctions.fillMap(values: Seq[(String, Any)]) fails for column name having a dot

[ https://issues.apache.org/jira/browse/SPARK-34417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34417: Assignee: Apache Spark > org.apache.spark.sql.DataFrameNaFunctions.fillMap(values:

[jira] [Commented] (SPARK-34417) org.apache.spark.sql.DataFrameNaFunctions.fillMap(values: Seq[(String, Any)]) fails for column name having a dot

[ https://issues.apache.org/jira/browse/SPARK-34417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282563#comment-17282563 ] Apache Spark commented on SPARK-34417: -- User 'amandeep-sharma' has created a pull request for this

[jira] [Commented] (SPARK-34418) Check v1 TRUNCATE TABLE preserves partitions

[ https://issues.apache.org/jira/browse/SPARK-34418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282509#comment-17282509 ] Apache Spark commented on SPARK-34418: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34418) Check v1 TRUNCATE TABLE preserves partitions

[ https://issues.apache.org/jira/browse/SPARK-34418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34418: Assignee: (was: Apache Spark) > Check v1 TRUNCATE TABLE preserves partitions >

[jira] [Assigned] (SPARK-34418) Check v1 TRUNCATE TABLE preserves partitions

[ https://issues.apache.org/jira/browse/SPARK-34418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34418: Assignee: Apache Spark > Check v1 TRUNCATE TABLE preserves partitions >

[jira] [Commented] (SPARK-34418) Check v1 TRUNCATE TABLE preserves partitions

[ https://issues.apache.org/jira/browse/SPARK-34418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282508#comment-17282508 ] Apache Spark commented on SPARK-34418: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-34418) Check v1 TRUNCATE TABLE preserves partitions

Maxim Gekk created SPARK-34418: -- Summary: Check v1 TRUNCATE TABLE preserves partitions Key: SPARK-34418 URL: https://issues.apache.org/jira/browse/SPARK-34418 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282433#comment-17282433 ] Ranju commented on SPARK-34389: --- Hi Attila ,  Please help me in understanding

[jira] [Updated] (SPARK-34417) org.apache.spark.sql.DataFrameNaFunctions.fillMap(values: Seq[(String, Any)]) fails for column name having a dot

[ https://issues.apache.org/jira/browse/SPARK-34417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amandeep Sharma updated SPARK-34417: Description: Code to reproduce the issue: {code:java} import

[jira] [Updated] (SPARK-34417) org.apache.spark.sql.DataFrameNaFunctions.fillMap(values: Seq[(String, Any)]) fails for column name having a dot

[ https://issues.apache.org/jira/browse/SPARK-34417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amandeep Sharma updated SPARK-34417: Description: Code to reproduce the issue: {code:java} import

[jira] [Created] (SPARK-34417) org.apache.spark.sql.DataFrameNaFunctions.fillMap(values: Seq[(String, Any)]) fails for column name having a dot

Amandeep Sharma created SPARK-34417: --- Summary: org.apache.spark.sql.DataFrameNaFunctions.fillMap(values: Seq[(String, Any)]) fails for column name having a dot Key: SPARK-34417 URL:

[jira] [Assigned] (SPARK-34416) Support avroSchemaUrl in addition to avroSchema

[ https://issues.apache.org/jira/browse/SPARK-34416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34416: Assignee: (was: Apache Spark) > Support avroSchemaUrl in addition to avroSchema >

[jira] [Assigned] (SPARK-34416) Support avroSchemaUrl in addition to avroSchema

[ https://issues.apache.org/jira/browse/SPARK-34416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34416: Assignee: Apache Spark > Support avroSchemaUrl in addition to avroSchema >

[jira] [Commented] (SPARK-34416) Support avroSchemaUrl in addition to avroSchema

[ https://issues.apache.org/jira/browse/SPARK-34416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282360#comment-17282360 ] Apache Spark commented on SPARK-34416: -- User 'uzadude' has created a pull request for this issue:

[jira] [Created] (SPARK-34416) Support avroSchemaUrl in addition to avroSchema

Ohad Raviv created SPARK-34416: -- Summary: Support avroSchemaUrl in addition to avroSchema Key: SPARK-34416 URL: https://issues.apache.org/jira/browse/SPARK-34416 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-34415) Use randomization as a possibly better technique than grid search in optimizing hyperparameters

[ https://issues.apache.org/jira/browse/SPARK-34415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282315#comment-17282315 ] Apache Spark commented on SPARK-34415: -- User 'PhillHenry' has created a pull request for this

[jira] [Assigned] (SPARK-34415) Use randomization as a possibly better technique than grid search in optimizing hyperparameters

[ https://issues.apache.org/jira/browse/SPARK-34415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34415: Assignee: (was: Apache Spark) > Use randomization as a possibly better technique

[jira] [Assigned] (SPARK-34415) Use randomization as a possibly better technique than grid search in optimizing hyperparameters

[ https://issues.apache.org/jira/browse/SPARK-34415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34415: Assignee: Apache Spark > Use randomization as a possibly better technique than grid

[jira] [Commented] (SPARK-34415) Use randomization as a possibly better technique than grid search in optimizing hyperparameters

[ https://issues.apache.org/jira/browse/SPARK-34415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282313#comment-17282313 ] Apache Spark commented on SPARK-34415: -- User 'PhillHenry' has created a pull request for this

[jira] [Created] (SPARK-34415) Use randomization as a possibly better technique than grid search in optimizing hyperparameters

Phillip Henry created SPARK-34415: - Summary: Use randomization as a possibly better technique than grid search in optimizing hyperparameters Key: SPARK-34415 URL: https://issues.apache.org/jira/browse/SPARK-34415