[jira] [Assigned] (SPARK-33641) Invalidate new char-like type in public APIs that result incorrect results

[ https://issues.apache.org/jira/browse/SPARK-33641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33641: Assignee: Apache Spark > Invalidate new char-like type in public APIs that result

[jira] [Assigned] (SPARK-33641) Invalidate new char-like type in public APIs that result incorrect results

[ https://issues.apache.org/jira/browse/SPARK-33641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33641: Assignee: (was: Apache Spark) > Invalidate new char-like type in public APIs that

[jira] [Commented] (SPARK-33641) Invalidate new char-like type in public APIs that result incorrect results

[ https://issues.apache.org/jira/browse/SPARK-33641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242978#comment-17242978 ] Apache Spark commented on SPARK-33641: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Created] (SPARK-33641) Invalidate new char-like type in public APIs that result incorrect results

Kent Yao created SPARK-33641: Summary: Invalidate new char-like type in public APIs that result incorrect results Key: SPARK-33641 URL: https://issues.apache.org/jira/browse/SPARK-33641 Project: Spark

[jira] [Commented] (SPARK-33620) Task not started after filtering

[ https://issues.apache.org/jira/browse/SPARK-33620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242966#comment-17242966 ] Vladislav Sterkhov commented on SPARK-33620: [~hyukjin.kwon] Hello. I created associated

[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers

[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242960#comment-17242960 ] Apache Spark commented on SPARK-26218: -- User 'luluorta' has created a pull request for this issue:

[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers

[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242959#comment-17242959 ] Apache Spark commented on SPARK-26218: -- User 'luluorta' has created a pull request for this issue:

[jira] [Updated] (SPARK-33628) Use the Hive.getPartitionsByNames method instead of Hive.getPartitions in the HiveClientImpl

[ https://issues.apache.org/jira/browse/SPARK-33628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinhai updated SPARK-33628: --- Description: When partitions are tracked by the catalog, that will compute all custom partition locations,

[jira] [Updated] (SPARK-33628) Use the Hive.getPartitionsByNames method instead of Hive.getPartitions in the HiveClientImpl

[ https://issues.apache.org/jira/browse/SPARK-33628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinhai updated SPARK-33628: --- Description: When partitions are tracked by the catalog, that will compute all custom partition locations,

[jira] [Comment Edited] (SPARK-33628) Use the Hive.getPartitionsByNames method instead of Hive.getPartitions in the HiveClientImpl

[ https://issues.apache.org/jira/browse/SPARK-33628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242951#comment-17242951 ] jinhai edited comment on SPARK-33628 at 12/3/20, 6:40 AM: -- And,

[jira] [Comment Edited] (SPARK-33628) Use the Hive.getPartitionsByNames method instead of Hive.getPartitions in the HiveClientImpl

[ https://issues.apache.org/jira/browse/SPARK-33628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242951#comment-17242951 ] jinhai edited comment on SPARK-33628 at 12/3/20, 6:38 AM: -- And,

[jira] [Commented] (SPARK-33628) Use the Hive.getPartitionsByNames method instead of Hive.getPartitions in the HiveClientImpl

[ https://issues.apache.org/jira/browse/SPARK-33628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242951#comment-17242951 ] jinhai commented on SPARK-33628: And, Hive.getPartitions will cause MetaStoreClient close connection.

[jira] [Commented] (SPARK-33472) IllegalArgumentException when applying RemoveRedundantSorts before EnsureRequirements

[ https://issues.apache.org/jira/browse/SPARK-33472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242945#comment-17242945 ] Apache Spark commented on SPARK-33472: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-33472) IllegalArgumentException when applying RemoveRedundantSorts before EnsureRequirements

[ https://issues.apache.org/jira/browse/SPARK-33472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242944#comment-17242944 ] Apache Spark commented on SPARK-33472: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-33640) Extend connection timeout to DB server for DB2IntegrationSuite and its variants

[ https://issues.apache.org/jira/browse/SPARK-33640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33640: Assignee: Kousuke Saruta (was: Apache Spark) > Extend connection timeout to DB server

[jira] [Commented] (SPARK-33640) Extend connection timeout to DB server for DB2IntegrationSuite and its variants

[ https://issues.apache.org/jira/browse/SPARK-33640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242932#comment-17242932 ] Apache Spark commented on SPARK-33640: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-33640) Extend connection timeout to DB server for DB2IntegrationSuite and its variants

[ https://issues.apache.org/jira/browse/SPARK-33640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242931#comment-17242931 ] Apache Spark commented on SPARK-33640: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33640) Extend connection timeout to DB server for DB2IntegrationSuite and its variants

[ https://issues.apache.org/jira/browse/SPARK-33640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33640: Assignee: Apache Spark (was: Kousuke Saruta) > Extend connection timeout to DB server

[jira] [Created] (SPARK-33640) Extend connection timeout to DB server for DB2IntegrationSuite and its variants

Kousuke Saruta created SPARK-33640: -- Summary: Extend connection timeout to DB server for DB2IntegrationSuite and its variants Key: SPARK-33640 URL: https://issues.apache.org/jira/browse/SPARK-33640

[jira] [Commented] (SPARK-33545) Support Fallback Storage during Worker decommission

[ https://issues.apache.org/jira/browse/SPARK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242929#comment-17242929 ] Dongjoon Hyun commented on SPARK-33545: --- BTW, I'm not sure about the progress of the other worker

[jira] [Commented] (SPARK-33545) Support Fallback Storage during Worker decommission

[ https://issues.apache.org/jira/browse/SPARK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242928#comment-17242928 ] Dongjoon Hyun commented on SPARK-33545: --- It's because I developed this for K8s graceful

[jira] [Commented] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242922#comment-17242922 ] Apache Spark commented on SPARK-33636: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242923#comment-17242923 ] Apache Spark commented on SPARK-33636: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33615) Add an integration test of K8S for archive feature

[ https://issues.apache.org/jira/browse/SPARK-33615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33615: Assignee: (was: Apache Spark) > Add an integration test of K8S for archive feature >

[jira] [Assigned] (SPARK-33615) Add an integration test of K8S for archive feature

[ https://issues.apache.org/jira/browse/SPARK-33615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33615: Assignee: Apache Spark > Add an integration test of K8S for archive feature >

[jira] [Commented] (SPARK-33615) Add an integration test of K8S for archive feature

[ https://issues.apache.org/jira/browse/SPARK-33615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242912#comment-17242912 ] Apache Spark commented on SPARK-33615: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-33615) Add an integration test of K8S for archive feature

[ https://issues.apache.org/jira/browse/SPARK-33615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242911#comment-17242911 ] Apache Spark commented on SPARK-33615: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-33598) Support Java Class with circular references

[ https://issues.apache.org/jira/browse/SPARK-33598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242901#comment-17242901 ] Hyukjin Kwon commented on SPARK-33598: -- [~jacklzg] areyou able to test in the higher versions of

[jira] [Resolved] (SPARK-33620) Task not started after filtering

[ https://issues.apache.org/jira/browse/SPARK-33620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33620. -- Resolution: Invalid > Task not started after filtering > > >

[jira] [Commented] (SPARK-33620) Task not started after filtering

[ https://issues.apache.org/jira/browse/SPARK-33620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242899#comment-17242899 ] Hyukjin Kwon commented on SPARK-33620: -- For a question, let's interact in the mailing list before

[jira] [Commented] (SPARK-33638) Full support of V2 table creation in DataStreamWriter.toTable API

[ https://issues.apache.org/jira/browse/SPARK-33638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242898#comment-17242898 ] Yuanjian Li commented on SPARK-33638: - There is already some discussion in the PR comments:

[jira] [Commented] (SPARK-33629) spark.buffer.size not applied in driver from pyspark

[ https://issues.apache.org/jira/browse/SPARK-33629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242895#comment-17242895 ] Hyukjin Kwon commented on SPARK-33629: -- Thank you [~gsomogyi]. > spark.buffer.size not applied in

[jira] [Commented] (SPARK-33637) Spark sql drop non-existent table will not report an error after failure

[ https://issues.apache.org/jira/browse/SPARK-33637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242894#comment-17242894 ] Hyukjin Kwon commented on SPARK-33637: -- [~guihuawen] can you show the current input/output and

[jira] [Commented] (SPARK-33632) to_date doesn't behave as documented

[ https://issues.apache.org/jira/browse/SPARK-33632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242893#comment-17242893 ] Hyukjin Kwon commented on SPARK-33632: -- cc [~XuanYuan] and [~maxgekk] FYI > to_date doesn't behave

[jira] [Created] (SPARK-33639) The external table does not specify a location

guihuawen created SPARK-33639: - Summary: The external table does not specify a location Key: SPARK-33639 URL: https://issues.apache.org/jira/browse/SPARK-33639 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33520) make CrossValidator/TrainValidateSplit/OneVsRest Reader/Writer support Python backend estimator/evaluator

[ https://issues.apache.org/jira/browse/SPARK-33520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-33520: --- Description: Currently, pyspark support third-party library to define python backend

[jira] [Updated] (SPARK-33520) make CrossValidator/TrainValidateSplit/OneVsRest Reader/Writer support Python backend estimator/evaluator

[ https://issues.apache.org/jira/browse/SPARK-33520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-33520: --- Summary: make CrossValidator/TrainValidateSplit/OneVsRest Reader/Writer support Python backend

[jira] [Created] (SPARK-33638) Full support of V2 table creation in DataStreamWriter.toTable API

Yuanjian Li created SPARK-33638: --- Summary: Full support of V2 table creation in DataStreamWriter.toTable API Key: SPARK-33638 URL: https://issues.apache.org/jira/browse/SPARK-33638 Project: Spark

[jira] [Created] (SPARK-33637) Spark sql drop non-existent table will not report an error after failure

guihuawen created SPARK-33637: - Summary: Spark sql drop non-existent table will not report an error after failure Key: SPARK-33637 URL: https://issues.apache.org/jira/browse/SPARK-33637 Project: Spark

[jira] [Commented] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242883#comment-17242883 ] Apache Spark commented on SPARK-33636: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242882#comment-17242882 ] Apache Spark commented on SPARK-33636: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-33545) Support Fallback Storage during Worker decommission

[ https://issues.apache.org/jira/browse/SPARK-33545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242862#comment-17242862 ] wuyi commented on SPARK-33545: -- Hi [~dongjoon] Why this's a subtask of k8s GA instead of

[jira] [Commented] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242855#comment-17242855 ] Apache Spark commented on SPARK-33636: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33636: - Target Version/s: (was: 3.1.0) > Add StringIndexerModel.labelsArray and deprecate

[jira] [Updated] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33636: - Fix Version/s: 3.1.0 > Add StringIndexerModel.labelsArray and deprecate

[jira] [Updated] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33636: - Issue Type: New Feature (was: Improvement) > Add StringIndexerModel.labelsArray and deprecate

[jira] [Created] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

Hyukjin Kwon created SPARK-33636: Summary: Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels Key: SPARK-33636 URL: https://issues.apache.org/jira/browse/SPARK-33636 Project:

[jira] [Resolved] (SPARK-33636) Add StringIndexerModel.labelsArray and deprecate StringIndexerModel.labels

[ https://issues.apache.org/jira/browse/SPARK-33636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33636. -- Target Version/s: 3.1.0 Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-33610) Imputer transform skip head() job

[ https://issues.apache.org/jira/browse/SPARK-33610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-33610. -- Resolution: Resolved > Imputer transform skip head() job > -

[jira] [Commented] (SPARK-22798) Add multiple column support to PySpark StringIndexer

[ https://issues.apache.org/jira/browse/SPARK-22798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242853#comment-17242853 ] Hyukjin Kwon commented on SPARK-22798: -- There was a bit of mistake during JIRA creation.

[jira] [Commented] (SPARK-32896) Add DataStreamWriter.toTable API

[ https://issues.apache.org/jira/browse/SPARK-32896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242844#comment-17242844 ] Shixiong Zhu commented on SPARK-32896: -- The method name is changed to `toTable` in 

[jira] [Updated] (SPARK-32896) Add DataStreamWriter.toTable API

[ https://issues.apache.org/jira/browse/SPARK-32896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-32896: - Description: For now, there's no way to write to the table (especially catalog table) even the

[jira] [Updated] (SPARK-32896) Add DataStreamWriter.toTable API

[ https://issues.apache.org/jira/browse/SPARK-32896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-32896: - Summary: Add DataStreamWriter.toTable API (was: Add DataStreamWriter.table API) > Add

[jira] [Resolved] (SPARK-31953) Add Spark Structured Streaming History Server Support

[ https://issues.apache.org/jira/browse/SPARK-31953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-31953. -- Fix Version/s: 3.1.0 Resolution: Done > Add Spark Structured Streaming History Server

[jira] [Assigned] (SPARK-31953) Add Spark Structured Streaming History Server Support

[ https://issues.apache.org/jira/browse/SPARK-31953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-31953: Assignee: Genmao Yu > Add Spark Structured Streaming History Server Support >

[jira] [Commented] (SPARK-22798) Add multiple column support to PySpark StringIndexer

[ https://issues.apache.org/jira/browse/SPARK-22798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242836#comment-17242836 ] Apache Spark commented on SPARK-22798: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-29679) Make interval type camparable and orderable

[ https://issues.apache.org/jira/browse/SPARK-29679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29679: - Fix Version/s: (was: 3.0.0) > Make interval type camparable and orderable >

[jira] [Updated] (SPARK-30048) Enable aggregates with interval type values for RelationalGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-30048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30048: - Fix Version/s: (was: 3.0.0) > Enable aggregates with interval type values for

[jira] [Reopened] (SPARK-29385) Make `INTERVAL` values comparable

[ https://issues.apache.org/jira/browse/SPARK-29385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reopened SPARK-29385: -- > Make `INTERVAL` values comparable > - > > Key:

[jira] [Resolved] (SPARK-29385) Make `INTERVAL` values comparable

[ https://issues.apache.org/jira/browse/SPARK-29385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29385. -- Fix Version/s: (was: 3.0.0) Resolution: Won't Fix > Make `INTERVAL` values

[jira] [Commented] (SPARK-33621) Add a way to inject data source rewrite rules

[ https://issues.apache.org/jira/browse/SPARK-33621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242739#comment-17242739 ] Apache Spark commented on SPARK-33621: -- User 'aokolnychyi' has created a pull request for this

[jira] [Commented] (SPARK-33621) Add a way to inject data source rewrite rules

[ https://issues.apache.org/jira/browse/SPARK-33621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242740#comment-17242740 ] Apache Spark commented on SPARK-33621: -- User 'aokolnychyi' has created a pull request for this

[jira] [Assigned] (SPARK-33621) Add a way to inject data source rewrite rules

[ https://issues.apache.org/jira/browse/SPARK-33621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33621: Assignee: Apache Spark > Add a way to inject data source rewrite rules >

[jira] [Assigned] (SPARK-33621) Add a way to inject data source rewrite rules

[ https://issues.apache.org/jira/browse/SPARK-33621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33621: Assignee: (was: Apache Spark) > Add a way to inject data source rewrite rules >

[jira] [Updated] (SPARK-33631) Clean up `spark.core.connection.ack.wait.timeout` from `configuration.md`

[ https://issues.apache.org/jira/browse/SPARK-33631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33631: -- Affects Version/s: (was: 3.1.0) 2.0.2 2.1.3

[jira] [Updated] (SPARK-33631) Clean up `spark.core.connection.ack.wait.timeout` from `configuration.md`

[ https://issues.apache.org/jira/browse/SPARK-33631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33631: -- Issue Type: Bug (was: Improvement) > Clean up `spark.core.connection.ack.wait.timeout` from

[jira] [Resolved] (SPARK-33631) Clean up `spark.core.connection.ack.wait.timeout` from `configuration.md`

[ https://issues.apache.org/jira/browse/SPARK-33631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33631. --- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution:

[jira] [Assigned] (SPARK-33631) Clean up `spark.core.connection.ack.wait.timeout` from `configuration.md`

[ https://issues.apache.org/jira/browse/SPARK-33631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33631: - Assignee: Yang Jie > Clean up `spark.core.connection.ack.wait.timeout` from

[jira] [Resolved] (SPARK-33627) Add new function UNIX_SECONDS, UNIX_MILLIS and UNIX_MICROS

[ https://issues.apache.org/jira/browse/SPARK-33627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33627. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30566

[jira] [Commented] (SPARK-33504) The application log in the Spark history server contains sensitive attributes such as password that should be redated instead of plain text

[ https://issues.apache.org/jira/browse/SPARK-33504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242657#comment-17242657 ] Apache Spark commented on SPARK-33504: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Commented] (SPARK-33504) The application log in the Spark history server contains sensitive attributes such as password that should be redated instead of plain text

[ https://issues.apache.org/jira/browse/SPARK-33504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242656#comment-17242656 ] Apache Spark commented on SPARK-33504: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Updated] (SPARK-33504) The application log in the Spark history server contains sensitive attributes such as password that should be redated instead of plain text

[ https://issues.apache.org/jira/browse/SPARK-33504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-33504: -- Fix Version/s: (was: 3.0.2) > The application log in the Spark history server contains

[jira] [Updated] (SPARK-26341) Expose executor memory metrics at the stage level, in the Stages tab

[ https://issues.apache.org/jira/browse/SPARK-26341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26341: --- Affects Version/s: (was: 3.1.0) 3.0.1 > Expose executor memory metrics at the

[jira] [Assigned] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33071: --- Assignee: wuyi > Join with ambiguous column succeeding but giving wrong output >

[jira] [Resolved] (SPARK-33536) Incorrect join results when joining twice with the same DF

[ https://issues.apache.org/jira/browse/SPARK-33536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33536. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30488

[jira] [Assigned] (SPARK-33536) Incorrect join results when joining twice with the same DF

[ https://issues.apache.org/jira/browse/SPARK-33536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33536: --- Assignee: wuyi > Incorrect join results when joining twice with the same DF >

[jira] [Resolved] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33071. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30488

[jira] [Commented] (SPARK-33141) capture SQL configs when creating permanent views

[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242518#comment-17242518 ] Apache Spark commented on SPARK-33141: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-33141) capture SQL configs when creating permanent views

[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242517#comment-17242517 ] Apache Spark commented on SPARK-33141: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-33635) Performance regression in Kafka read

[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wyles updated SPARK-33635: Description: I have observed a slowdown in the reading of data from kafka on all of our systems

[jira] [Created] (SPARK-33635) Performance regression in Kafka read

David Wyles created SPARK-33635: --- Summary: Performance regression in Kafka read Key: SPARK-33635 URL: https://issues.apache.org/jira/browse/SPARK-33635 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33212: Assignee: Apache Spark > Move to shaded clients for Hadoop 3.x profile >

[jira] [Assigned] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33212: Assignee: (was: Apache Spark) > Move to shaded clients for Hadoop 3.x profile >

[jira] [Assigned] (SPARK-33634) use Analyzer in PlanResolutionSuite

[ https://issues.apache.org/jira/browse/SPARK-33634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33634: Assignee: Wenchen Fan (was: Apache Spark) > use Analyzer in PlanResolutionSuite >

[jira] [Commented] (SPARK-33634) use Analyzer in PlanResolutionSuite

[ https://issues.apache.org/jira/browse/SPARK-33634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242506#comment-17242506 ] Apache Spark commented on SPARK-33634: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33634) use Analyzer in PlanResolutionSuite

[ https://issues.apache.org/jira/browse/SPARK-33634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33634: Assignee: Apache Spark (was: Wenchen Fan) > use Analyzer in PlanResolutionSuite >

[jira] [Updated] (SPARK-33005) Kubernetes GA Preparation

[ https://issues.apache.org/jira/browse/SPARK-33005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33005: -- Labels: releasenotes (was: ) > Kubernetes GA Preparation > - > >

[jira] [Updated] (SPARK-33626) Allow k8s integration tests to assert both driver and executor logs for expected log(s)

[ https://issues.apache.org/jira/browse/SPARK-33626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33626: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Improvement) > Allow k8s integration

[jira] [Resolved] (SPARK-33626) Allow k8s integration tests to assert both driver and executor logs for expected log(s)

[ https://issues.apache.org/jira/browse/SPARK-33626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33626. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30568

[jira] [Assigned] (SPARK-33626) Allow k8s integration tests to assert both driver and executor logs for expected log(s)

[ https://issues.apache.org/jira/browse/SPARK-33626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33626: - Assignee: Prashant Sharma > Allow k8s integration tests to assert both driver and

[jira] [Updated] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33212: -- Fix Version/s: (was: 3.1.0) > Move to shaded clients for Hadoop 3.x profile >

[jira] [Reopened] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-33212: --- Assignee: (was: Chao Sun) This is reverted via SPARK-33618 . We will retry this with

[jira] [Commented] (SPARK-33576) PythonException: An exception was thrown from a UDF: 'OSError: Invalid IPC message: negative bodyLength'.

[ https://issues.apache.org/jira/browse/SPARK-33576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242493#comment-17242493 ] Darshat commented on SPARK-33576: - Thanks [~bryanc] , yes this does look similar if not the same issue.

[jira] [Created] (SPARK-33634) use Analyzer in PlanResolutionSuite

Wenchen Fan created SPARK-33634: --- Summary: use Analyzer in PlanResolutionSuite Key: SPARK-33634 URL: https://issues.apache.org/jira/browse/SPARK-33634 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-33619) GetMapValueUtil code generation error

[ https://issues.apache.org/jira/browse/SPARK-33619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33619: --- Assignee: Leanken.Lin > GetMapValueUtil code generation error >

[jira] [Resolved] (SPARK-33619) GetMapValueUtil code generation error

[ https://issues.apache.org/jira/browse/SPARK-33619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33619. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30560

[jira] [Updated] (SPARK-32910) Remove UninterruptibleThread usage from KafkaOffsetReader

[ https://issues.apache.org/jira/browse/SPARK-32910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-32910: -- Affects Version/s: (was: 3.2.0) 3.1.0 > Remove

[jira] [Updated] (SPARK-33633) Expose fine grained state information on SS UI

[ https://issues.apache.org/jira/browse/SPARK-33633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-33633: -- Affects Version/s: (was: 3.2.0) 3.1.0 > Expose fine grained state

[jira] [Updated] (SPARK-33633) Expose fine grained state information on SS UI

[ https://issues.apache.org/jira/browse/SPARK-33633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-33633: -- Description: SPARK-33223 provides aggregated information but in order to find out the

[jira] [Created] (SPARK-33633) Expose fine grained state information on SS UI

Gabor Somogyi created SPARK-33633: - Summary: Expose fine grained state information on SS UI Key: SPARK-33633 URL: https://issues.apache.org/jira/browse/SPARK-33633 Project: Spark Issue Type:

  1   2   >