[jira] [Reopened] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

[ https://issues.apache.org/jira/browse/SPARK-34406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-34406: -- > When we submit spark core tasks frequently, the submitted nodes will have a > lot of resource p

[jira] [Commented] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

[ https://issues.apache.org/jira/browse/SPARK-34406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281560#comment-17281560 ] hao commented on SPARK-34406: - Yes, sir. I'm using the Yarn cluster mode. What I mean here i

[jira] [Resolved] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34389. -- Resolution: Not A Problem > Spark job on Kubernetes scheduled For Zero or less than minimum nu

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281544#comment-17281544 ] Hyukjin Kwon commented on SPARK-34389: -- I think this is more a question. I will ten

[jira] [Commented] (SPARK-34392) Invalid ID for offset-based ZoneId since Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-34392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281543#comment-17281543 ] Hyukjin Kwon commented on SPARK-34392: -- cc [~maxgekk] FYI > Invalid ID for offset-

[jira] [Resolved] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

[ https://issues.apache.org/jira/browse/SPARK-34406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34406. -- Resolution: Won't Fix > When we submit spark core tasks frequently, the submitted nodes will h

[jira] [Commented] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

[ https://issues.apache.org/jira/browse/SPARK-34406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281541#comment-17281541 ] Hyukjin Kwon commented on SPARK-34406: -- You should use Yarn cluster mode that will

[jira] [Updated] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

[ https://issues.apache.org/jira/browse/SPARK-34406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34406: - Priority: Major (was: Critical) > When we submit spark core tasks frequently, the submitted nod

[jira] [Updated] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34407: -- Affects Version/s: 2.4.7 > KubernetesClusterSchedulerBackend.stop should clean up K8s resource

[jira] [Assigned] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34407: - Assignee: Dongjoon Hyun > KubernetesClusterSchedulerBackend.stop should clean up K8s re

[jira] [Resolved] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34407. --- Fix Version/s: 3.1.2 2.4.8 3.0.2 Resolution: Fix

[jira] [Commented] (SPARK-34389) Spark job on Kubernetes scheduled For Zero or less than minimum number of executors and Wait indefinitely under resource starvation

[ https://issues.apache.org/jira/browse/SPARK-34389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281534#comment-17281534 ] Ranju commented on SPARK-34389: --- Yes I understood this why there is no retry logic and tha

[jira] [Assigned] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34405: - Assignee: iteblog > The getMetricsSnapshot method of the PrometheusServlet class has a

[jira] [Resolved] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34405. --- Fix Version/s: 3.0.2 3.1.1 Resolution: Fixed Issue resolved by pul

[jira] [Updated] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34405: -- Fix Version/s: (was: 3.1.1) 3.1.2 > The getMetricsSnapshot method of th

[jira] [Updated] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34405: -- Affects Version/s: 3.1.1 3.2.0 3.1.0 > The getMe

[jira] [Updated] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34407: -- Affects Version/s: 2.3.4 > KubernetesClusterSchedulerBackend.stop should clean up K8s resource

[jira] [Created] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

Dongjoon Hyun created SPARK-34407: - Summary: KubernetesClusterSchedulerBackend.stop should clean up K8s resources Key: SPARK-34407 URL: https://issues.apache.org/jira/browse/SPARK-34407 Project: Spark

[jira] [Updated] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34407: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > KubernetesClusterSchedulerBacke

[jira] [Commented] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281523#comment-17281523 ] Apache Spark commented on SPARK-34407: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34407: Assignee: (was: Apache Spark) > KubernetesClusterSchedulerBackend.stop should clean u

[jira] [Assigned] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34407: Assignee: Apache Spark > KubernetesClusterSchedulerBackend.stop should clean up K8s resou

[jira] [Commented] (SPARK-34407) KubernetesClusterSchedulerBackend.stop should clean up K8s resources

[ https://issues.apache.org/jira/browse/SPARK-34407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281522#comment-17281522 ] Apache Spark commented on SPARK-34407: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281506#comment-17281506 ] Apache Spark commented on SPARK-34405: -- User '397090770' has created a pull request

[jira] [Commented] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281505#comment-17281505 ] Apache Spark commented on SPARK-34405: -- User '397090770' has created a pull request

[jira] [Assigned] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34405: Assignee: Apache Spark > The getMetricsSnapshot method of the PrometheusServlet class has

[jira] [Assigned] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

[ https://issues.apache.org/jira/browse/SPARK-34405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34405: Assignee: (was: Apache Spark) > The getMetricsSnapshot method of the PrometheusServle

[jira] [Created] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

hao created SPARK-34406: --- Summary: When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure Key: SPARK-34406 URL: https://issues.apache.org/jira/browse/SPARK-34406 Pr

[jira] [Created] (SPARK-34405) The getMetricsSnapshot method of the PrometheusServlet class has a wrong value

iteblog created SPARK-34405: --- Summary: The getMetricsSnapshot method of the PrometheusServlet class has a wrong value Key: SPARK-34405 URL: https://issues.apache.org/jira/browse/SPARK-34405 Project: Spark

[jira] [Updated] (SPARK-34352) Improve SQLQueryTestSuite so as could run on windows system

[ https://issues.apache.org/jira/browse/SPARK-34352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34352: - Fix Version/s: (was: 3.1.2) 3.2.0 > Improve SQLQueryTestSuite so as could

[jira] [Assigned] (SPARK-29465) Unable to configure SPARK UI (spark.ui.port) in spark yarn cluster mode.

[ https://issues.apache.org/jira/browse/SPARK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29465: Assignee: Vishwas Nalka > Unable to configure SPARK UI (spark.ui.port) in spark yarn clus

[jira] [Updated] (SPARK-29465) Unable to configure SPARK UI (spark.ui.port) in spark yarn cluster mode.

[ https://issues.apache.org/jira/browse/SPARK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29465: - Priority: Minor (was: Major) > Unable to configure SPARK UI (spark.ui.port) in spark yarn clust

[jira] [Assigned] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-33571: Assignee: Maxim Gekk > Handling of hybrid to proleptic calendar when reading and writing

[jira] [Assigned] (SPARK-34395) Clean up unused code for code simplifications

[ https://issues.apache.org/jira/browse/SPARK-34395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-34395: Assignee: yikf > Clean up unused code for code simplifications >

[jira] [Resolved] (SPARK-34395) Clean up unused code for code simplifications

[ https://issues.apache.org/jira/browse/SPARK-34395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34395. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31510 [https://gi

[jira] [Resolved] (SPARK-34352) Improve SQLQueryTestSuite so as could run on windows system

[ https://issues.apache.org/jira/browse/SPARK-34352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34352. -- Fix Version/s: 3.1.2 Resolution: Fixed Fixed in https://github.com/apache/spark/pull/31

[jira] [Assigned] (SPARK-33566) Incorrectly Parsing CSV file

[ https://issues.apache.org/jira/browse/SPARK-33566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33566: Assignee: Yang Jie > Incorrectly Parsing CSV file > > >

[jira] [Assigned] (SPARK-33565) python/run-tests.py calling python3.8

[ https://issues.apache.org/jira/browse/SPARK-33565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33565: Assignee: Shane Knapp (was: Apache Spark) > python/run-tests.py calling python3.8 >

[jira] [Assigned] (SPARK-33376) Remove the option of "sharesHadoopClasses" in Hive IsolatedClientLoader

[ https://issues.apache.org/jira/browse/SPARK-33376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33376: Assignee: Chao Sun (was: Apache Spark) > Remove the option of "sharesHadoopClasses" in H

[jira] [Assigned] (SPARK-33303) Deduplicate deterministic PythonUDF calls

[ https://issues.apache.org/jira/browse/SPARK-33303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33303: Assignee: Peter Toth (was: Apache Spark) > Deduplicate deterministic PythonUDF calls > -

[jira] [Assigned] (SPARK-32215) Expose end point on Master so that it can be informed about decommissioned workers out of band

[ https://issues.apache.org/jira/browse/SPARK-32215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32215: Assignee: Devesh Agrawal > Expose end point on Master so that it can be informed about de

[jira] [Assigned] (SPARK-33297) Intermittent Compilation failure In GitHub Actions after SBT upgrade

[ https://issues.apache.org/jira/browse/SPARK-33297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33297: Assignee: Hyukjin Kwon (was: Apache Spark) > Intermittent Compilation failure In GitHub

[jira] [Assigned] (SPARK-32450) Upgrade pycodestyle to 2.6.0

[ https://issues.apache.org/jira/browse/SPARK-32450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32450: Assignee: L. C. Hsieh (was: Apache Spark) > Upgrade pycodestyle to 2.6.0 > -

[jira] [Assigned] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32613: Assignee: Devesh Agrawal > DecommissionWorkerSuite has started failing sporadically again

[jira] [Assigned] (SPARK-33190) Set upperbound of PyArrow version in GitHub Actions

[ https://issues.apache.org/jira/browse/SPARK-33190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33190: Assignee: Hyukjin Kwon (was: Apache Spark) > Set upperbound of PyArrow version in GitHub

[jira] [Assigned] (SPARK-31830) Consistent error handling for datetime formatting functions

[ https://issues.apache.org/jira/browse/SPARK-31830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31830: Assignee: Kent Yao (was: Apache Spark) > Consistent error handling for datetime formatti

[jira] [Assigned] (SPARK-32074) Update AppVeyor R to 4.0.2

[ https://issues.apache.org/jira/browse/SPARK-32074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32074: Assignee: Hyukjin Kwon > Update AppVeyor R to 4.0.2 > -- > >

[jira] [Assigned] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32282: Assignee: Terry Kim (was: Apache Spark) > Improve EnsureRquirement.reorderJoinKeys to ha

[jira] [Assigned] (SPARK-30880) Delete Sphinx Makefile cruft

[ https://issues.apache.org/jira/browse/SPARK-30880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30880: Assignee: Nicholas Chammas > Delete Sphinx Makefile cruft >

[jira] [Assigned] (SPARK-31598) LegacySimpleTimestampFormatter incorrectly interprets pre-Gregorian timestamps

[ https://issues.apache.org/jira/browse/SPARK-31598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31598: Assignee: Bruce Robbins > LegacySimpleTimestampFormatter incorrectly interprets pre-Grego

[jira] [Assigned] (SPARK-31000) Add ability to set table description in the catalog

[ https://issues.apache.org/jira/browse/SPARK-31000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31000: Assignee: Nicholas Chammas > Add ability to set table description in the catalog > --

[jira] [Assigned] (SPARK-31517) SparkR::orderBy with multiple columns descending produces error

[ https://issues.apache.org/jira/browse/SPARK-31517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31517: Assignee: Michael Chirico > SparkR::orderBy with multiple columns descending produces err

[jira] [Assigned] (SPARK-30889) Add version information to the configuration of Worker

[ https://issues.apache.org/jira/browse/SPARK-30889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30889: Assignee: jiaan.geng > Add version information to the configuration of Worker > -

[jira] [Assigned] (SPARK-30856) SQLContext retains reference to unusable instance after SparkContext restarted

[ https://issues.apache.org/jira/browse/SPARK-30856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30856: Assignee: Alex Favaro > SQLContext retains reference to unusable instance after SparkCont

[jira] [Assigned] (SPARK-30988) Add more edge-case exercising values to stats tests

[ https://issues.apache.org/jira/browse/SPARK-30988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30988: Assignee: Maxim Gekk > Add more edge-case exercising values to stats tests >

[jira] [Assigned] (SPARK-30839) Add version information for Spark configuration

[ https://issues.apache.org/jira/browse/SPARK-30839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30839: Assignee: jiaan.geng > Add version information for Spark configuration >

[jira] [Assigned] (SPARK-30821) Executor pods with multiple containers will not be rescheduled unless all containers fail

[ https://issues.apache.org/jira/browse/SPARK-30821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30821: Assignee: Shiqi Sun (was: Apache Spark) > Executor pods with multiple containers will no

[jira] [Assigned] (SPARK-30795) Spark SQL codegen's code() interpolator should treat escapes like Scala's StringContext.s()

[ https://issues.apache.org/jira/browse/SPARK-30795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30795: Assignee: Kris Mok > Spark SQL codegen's code() interpolator should treat escapes like Sc

[jira] [Assigned] (SPARK-30733) Fix SparkR tests per testthat and R version upgrade

[ https://issues.apache.org/jira/browse/SPARK-30733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30733: Assignee: Hyukjin Kwon > Fix SparkR tests per testthat and R version upgrade > --

[jira] [Assigned] (SPARK-28646) Allow usage of `count` only for parameterless aggregate function

[ https://issues.apache.org/jira/browse/SPARK-28646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28646: Assignee: jiaan.geng (was: Apache Spark) > Allow usage of `count` only for parameterless

[jira] [Assigned] (SPARK-27497) Spark wipes out bucket spec in metastore when updating table stats

[ https://issues.apache.org/jira/browse/SPARK-27497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27497: Assignee: Bruce Robbins > Spark wipes out bucket spec in metastore when updating table st

[jira] [Assigned] (SPARK-28367) Kafka connector infinite wait because metadata never updated

[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28367: Assignee: Gabor Somogyi > Kafka connector infinite wait because metadata never updated >

[jira] [Assigned] (SPARK-33670) Verify the partition provider is Hive in v1 SHOW TABLE EXTENDED

[ https://issues.apache.org/jira/browse/SPARK-33670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33670: Assignee: Maxim Gekk (was: Apache Spark) > Verify the partition provider is Hive in v1 S

[jira] [Assigned] (SPARK-33697) UnionExec should require column ordering in RemoveRedundantProjects

[ https://issues.apache.org/jira/browse/SPARK-33697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33697: Assignee: Allison Wang (was: Apache Spark) > UnionExec should require column ordering in

[jira] [Assigned] (SPARK-33803) Sort table properties by key in DESCRIBE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-33803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33803: Assignee: Hyukjin Kwon (was: Apache Spark) > Sort table properties by key in DESCRIBE TA

[jira] [Assigned] (SPARK-33901) Char and Varchar display error after DDLs

[ https://issues.apache.org/jira/browse/SPARK-33901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33901: Assignee: Kent Yao (was: Apache Spark) > Char and Varchar display error after DDLs > ---

[jira] [Assigned] (SPARK-33907) Only prune columns of from_json if parsing options is empty

[ https://issues.apache.org/jira/browse/SPARK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33907: Assignee: L. C. Hsieh (was: Apache Spark) > Only prune columns of from_json if parsing o

[jira] [Assigned] (SPARK-34181) Update build doc help document

[ https://issues.apache.org/jira/browse/SPARK-34181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34181: Assignee: angerszhu > Update build doc help document > -- > >

[jira] [Assigned] (SPARK-25872) Add an optimizer tracker for TPC-DS queries

[ https://issues.apache.org/jira/browse/SPARK-25872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-25872: --- Assignee: wuyi (was: Apache Spark) > Add an optimizer tracker for TPC-DS queries > ---

[jira] [Commented] (SPARK-34080) Add UnivariateFeatureSelector to deprecate existing selectors

[ https://issues.apache.org/jira/browse/SPARK-34080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281478#comment-17281478 ] Apache Spark commented on SPARK-34080: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-32736) Avoid caching the removed decommissioned executors in TaskSchedulerImpl

[ https://issues.apache.org/jira/browse/SPARK-32736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-32736: --- Assignee: wuyi (was: Apache Spark) > Avoid caching the removed decommissioned executors in

[jira] [Assigned] (SPARK-32689) HiveSerDeReadWriteSuite and ScriptTransformationSuite are currently failed under hive1.2 profile in branch-3.0 and master

[ https://issues.apache.org/jira/browse/SPARK-32689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-32689: --- Assignee: L. C. Hsieh > HiveSerDeReadWriteSuite and ScriptTransformationSuite are currently

[jira] [Assigned] (SPARK-33139) protect setActiveSession and clearActiveSession

[ https://issues.apache.org/jira/browse/SPARK-33139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33139: --- Assignee: jiahong.li > protect setActiveSession and clearActiveSession > --

[jira] [Assigned] (SPARK-33105) Broken installation of source packages on AppVeyor

[ https://issues.apache.org/jira/browse/SPARK-33105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33105: --- Assignee: Maciej Szymkiewicz > Broken installation of source packages on AppVeyor > ---

[jira] [Assigned] (SPARK-33169) Check propagation of datasource options to underlying file system for built-in file-based datasources

[ https://issues.apache.org/jira/browse/SPARK-33169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33169: --- Assignee: Maxim Gekk > Check propagation of datasource options to underlying file system fo

[jira] [Assigned] (SPARK-33165) Remove dependencies(scalatest,scalactic) from Benchmark

[ https://issues.apache.org/jira/browse/SPARK-33165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33165: --- Assignee: Takeshi Yamamuro > Remove dependencies(scalatest,scalactic) from Benchmark >

[jira] [Assigned] (SPARK-33265) Rename classOf[Seq] to classOf[scala.collection.Seq] in PostgresIntegrationSuite for Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-33265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33265: --- Assignee: Kousuke Saruta (was: Apache Spark) > Rename classOf[Seq] to classOf[scala.collec

[jira] [Assigned] (SPARK-33860) Make CatalystTypeConverters.convertToCatalyst match special Array value

[ https://issues.apache.org/jira/browse/SPARK-33860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33860: --- Assignee: ulysses you > Make CatalystTypeConverters.convertToCatalyst match special Array v

[jira] [Assigned] (SPARK-34116) Separate state store numKeys metric test

[ https://issues.apache.org/jira/browse/SPARK-34116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34116: --- Assignee: L. C. Hsieh (was: Apache Spark) > Separate state store numKeys metric test > ---

[jira] [Assigned] (SPARK-34027) ALTER TABLE .. RECOVER PARTITIONS doesn't refresh cache

[ https://issues.apache.org/jira/browse/SPARK-34027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34027: --- Assignee: Maxim Gekk (was: Apache Spark) > ALTER TABLE .. RECOVER PARTITIONS doesn't refre

[jira] [Resolved] (SPARK-34374) Use standard methods to extract keys or values from a Map.

[ https://issues.apache.org/jira/browse/SPARK-34374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34374. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31484 [https://gi

[jira] [Assigned] (SPARK-34374) Use standard methods to extract keys or values from a Map.

[ https://issues.apache.org/jira/browse/SPARK-34374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-34374: Assignee: Yang Jie > Use standard methods to extract keys or values from a Map. > ---

[jira] [Resolved] (SPARK-34194) Queries that only touch partition columns shouldn't scan through all files

[ https://issues.apache.org/jira/browse/SPARK-34194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-34194. -- Resolution: Won't Fix > Queries that only touch partition columns shouldn't scan throu

[jira] [Commented] (SPARK-34194) Queries that only touch partition columns shouldn't scan through all files

[ https://issues.apache.org/jira/browse/SPARK-34194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281363#comment-17281363 ] Cheng Su commented on SPARK-34194: -- [~nchammas] - I think metadata-only query on partit

[jira] [Commented] (SPARK-34404) Support Avro datasource options to control datetime rebasing in read

[ https://issues.apache.org/jira/browse/SPARK-34404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281348#comment-17281348 ] Apache Spark commented on SPARK-34404: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-34404) Support Avro datasource options to control datetime rebasing in read

[ https://issues.apache.org/jira/browse/SPARK-34404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34404: Assignee: Apache Spark (was: Maxim Gekk) > Support Avro datasource options to control da

[jira] [Assigned] (SPARK-34404) Support Avro datasource options to control datetime rebasing in read

[ https://issues.apache.org/jira/browse/SPARK-34404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34404: Assignee: Maxim Gekk (was: Apache Spark) > Support Avro datasource options to control da

[jira] [Updated] (SPARK-34404) Support Avro datasource options to control datetime rebasing in read

[ https://issues.apache.org/jira/browse/SPARK-34404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34404: --- Description: Add new Avro option similar to the SQL configs  {{spark.sql.legacy.parquet.datetimeRebas

[jira] [Created] (SPARK-34404) Support Avro datasource options to control datetime rebasing in read

Maxim Gekk created SPARK-34404: -- Summary: Support Avro datasource options to control datetime rebasing in read Key: SPARK-34404 URL: https://issues.apache.org/jira/browse/SPARK-34404 Project: Spark

[jira] [Updated] (SPARK-34168) Support DPP in AQE When the join is Broadcast hash join before applying the AQE rules

[ https://issues.apache.org/jira/browse/SPARK-34168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34168: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Support DPP in AQE When

[jira] [Commented] (SPARK-34194) Queries that only touch partition columns shouldn't scan through all files

[ https://issues.apache.org/jira/browse/SPARK-34194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281269#comment-17281269 ] Nicholas Chammas commented on SPARK-34194: -- It's not clear to me whether SPARK-

[jira] [Updated] (SPARK-34344) Have functionality to trace back Spark SQL queries from the application ID that got submitted on YARN

[ https://issues.apache.org/jira/browse/SPARK-34344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpan Bhandari updated SPARK-34344: --- Component/s: SQL > Have functionality to trace back Spark SQL queries from the application I

[jira] [Updated] (SPARK-34344) Have functionality to trace back Spark SQL queries from the application ID that got submitted on YARN

[ https://issues.apache.org/jira/browse/SPARK-34344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpan Bhandari updated SPARK-34344: --- Component/s: (was: Spark Submit) > Have functionality to trace back Spark SQL queries fr

[jira] [Assigned] (SPARK-34168) Support DPP in AQE When the join is Broadcast hash join before applying the AQE rules

[ https://issues.apache.org/jira/browse/SPARK-34168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34168: --- Assignee: Ke Jia > Support DPP in AQE When the join is Broadcast hash join before applying

[jira] [Resolved] (SPARK-34168) Support DPP in AQE When the join is Broadcast hash join before applying the AQE rules

[ https://issues.apache.org/jira/browse/SPARK-34168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34168. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31258 [https://gith

[jira] [Assigned] (SPARK-34403) Remove dependency to commons-httpclient, is not used and has vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-34403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34403: Assignee: (was: Apache Spark) > Remove dependency to commons-httpclient, is not used

[jira] [Assigned] (SPARK-34403) Remove dependency to commons-httpclient, is not used and has vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-34403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34403: Assignee: Apache Spark > Remove dependency to commons-httpclient, is not used and has vul

[jira] [Commented] (SPARK-34403) Remove dependency to commons-httpclient, is not used and has vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-34403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281180#comment-17281180 ] Apache Spark commented on SPARK-34403: -- User 'ssainz' has created a pull request fo

[jira] [Commented] (SPARK-34403) Remove dependency to commons-httpclient, is not used and has vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-34403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281181#comment-17281181 ] Apache Spark commented on SPARK-34403: -- User 'ssainz' has created a pull request fo

[jira] [Commented] (SPARK-34403) Remove dependency to commons-httpclient, is not used and has vulnerabilities.

[ https://issues.apache.org/jira/browse/SPARK-34403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281179#comment-17281179 ] Sergio Sainz commented on SPARK-34403: -- PR: https://github.com/apache/spark/pull/31

  1   2   3   >