[jira] [Assigned] (SPARK-46694) Drop the assumptions of 'hive version < 2.0' in Hive version related tests

2024-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-46694: Assignee: Kent Yao > Drop the assumptions of 'hive version < 2.0' in Hive version related tests

[jira] [Resolved] (SPARK-46694) Drop the assumptions of 'hive version < 2.0' in Hive version related tests

2024-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46694. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44700

[jira] [Updated] (SPARK-46696) In ResourceProfileManager, function calls should occur after variable declarations.

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46696: --- Labels: pull-request-available (was: ) > In ResourceProfileManager, function calls should

[jira] [Created] (SPARK-46696) In ResourceProfileManager, function calls should occur after variable declarations.

2024-01-11 Thread liangyongyuan (Jira)
liangyongyuan created SPARK-46696: - Summary: In ResourceProfileManager, function calls should occur after variable declarations. Key: SPARK-46696 URL: https://issues.apache.org/jira/browse/SPARK-46696

[jira] [Updated] (SPARK-46695) Always setting hive.execution.engine to mr

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46695: --- Labels: pull-request-available (was: ) > Always setting hive.execution.engine to mr >

[jira] [Created] (SPARK-46695) Always setting hive.execution.engine to mr

2024-01-11 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-46695: - Summary: Always setting hive.execution.engine to mr Key: SPARK-46695 URL: https://issues.apache.org/jira/browse/SPARK-46695 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-46694) Drop the assumptions of 'hive version < 2.0' in Hive version related tests

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46694: --- Labels: pull-request-available (was: ) > Drop the assumptions of 'hive version < 2.0' in

[jira] [Created] (SPARK-46694) Drop the assumptions of 'hive version < 2.0' in Hive version related tests

2024-01-11 Thread Kent Yao (Jira)
Kent Yao created SPARK-46694: Summary: Drop the assumptions of 'hive version < 2.0' in Hive version related tests Key: SPARK-46694 URL: https://issues.apache.org/jira/browse/SPARK-46694 Project: Spark

[jira] [Updated] (SPARK-46429) avoid duplicate Classes and Resources in classpath of SPARK_HOME/jars/*.jar

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46429: - Affects Version/s: (was: 3.5.2) > avoid duplicate Classes and Resources in classpath of

[jira] [Updated] (SPARK-46684) CoGroup.applyInPandas/Arrow should pass arguments properly

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46684: - Fix Version/s: 3.5.1 > CoGroup.applyInPandas/Arrow should pass arguments properly >

[jira] [Assigned] (SPARK-46684) CoGroup.applyInPandas/Arrow should pass arguments properly

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46684: Assignee: Takuya Ueshin > CoGroup.applyInPandas/Arrow should pass arguments properly >

[jira] [Resolved] (SPARK-46684) CoGroup.applyInPandas/Arrow should pass arguments properly

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46684. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44695

[jira] [Resolved] (SPARK-46588) Interrupt when executing ANALYSIS phase

2024-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46588. -- Resolution: Information Provided Jira is not a suitable place for questions, you'd better use the

[jira] [Updated] (SPARK-46693) Inject LocalLimitExec when matching OffsetAndLimit or LimitAndOffset

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46693: --- Labels: pull-request-available (was: ) > Inject LocalLimitExec when matching

[jira] [Created] (SPARK-46693) Inject LocalLimitExec when matching OffsetAndLimit or LimitAndOffset

2024-01-11 Thread Nick Young (Jira)
Nick Young created SPARK-46693: -- Summary: Inject LocalLimitExec when matching OffsetAndLimit or LimitAndOffset Key: SPARK-46693 URL: https://issues.apache.org/jira/browse/SPARK-46693 Project: Spark

[jira] [Commented] (SPARK-46588) Interrupt when executing ANALYSIS phase

2024-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805873#comment-17805873 ] Kent Yao commented on SPARK-46588: -- You can call sc.setInterruptOnCancel(true) to interrupt the running

[jira] [Updated] (SPARK-46612) Clickhouse's JDBC throws `java.lang.IllegalArgumentException: Unknown data type: string` when write array string with Apache Spark scala

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46612: --- Labels: pull-request-available (was: ) > Clickhouse's JDBC throws

[jira] [Resolved] (SPARK-46650) Replace AtomicBoolean with volatile boolean

2024-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46650. -- Resolution: Not A Problem > Replace AtomicBoolean with volatile boolean >

[jira] [Updated] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-25895: --- Labels: pull-request-available (was: ) > No test to compare Zstd and Lz4 Compression

[jira] [Updated] (SPARK-46692) Fix potential issues with environment variable transmission `PYTHON_TO_TEST`

2024-01-11 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-46692: Summary: Fix potential issues with environment variable transmission `PYTHON_TO_TEST` (was: Fix

[jira] [Updated] (SPARK-46692) Fix potential issues with environment variable transmission `PYTHON_TO_TEST` in `build_python`

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46692: --- Labels: pull-request-available (was: ) > Fix potential issues with environment variable

[jira] [Assigned] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46383: --- Assignee: Utkarsh Agarwal > Reduce Driver Heap Usage by Reducing the Lifespan of

[jira] [Resolved] (SPARK-46383) Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46383. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44321

[jira] [Resolved] (SPARK-46640) RemoveRedundantAliases does not account for SubqueryExpression when removing aliases

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46640. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-46692) Fix potential issues with environment variable transmission `PYTHON_TO_TEST` in `build_python`

2024-01-11 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-46692: Summary: Fix potential issues with environment variable transmission `PYTHON_TO_TEST` in

[jira] [Assigned] (SPARK-46640) RemoveRedundantAliases does not account for SubqueryExpression when removing aliases

2024-01-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46640: --- Assignee: Nikhil Sheoran > RemoveRedundantAliases does not account for SubqueryExpression

[jira] [Created] (SPARK-46692) Fix potential issues with environment variable transmission `$PYTHON_TO_TEST` in `build_python`

2024-01-11 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-46692: --- Summary: Fix potential issues with environment variable transmission `$PYTHON_TO_TEST` in `build_python` Key: SPARK-46692 URL: https://issues.apache.org/jira/browse/SPARK-46692

[jira] [Assigned] (SPARK-46670) Make DataSourceManager isolated and self clone-able

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46670: Assignee: Hyukjin Kwon > Make DataSourceManager isolated and self clone-able >

[jira] [Resolved] (SPARK-46670) Make DataSourceManager isolated and self clone-able

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46670. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44681

[jira] [Updated] (SPARK-46686) Basic support of SparkSession based Python UDF profiler

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46686: --- Labels: pull-request-available (was: ) > Basic support of SparkSession based Python UDF

[jira] [Created] (SPARK-46691) Support profiling on WindowInPandasExec

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46691: - Summary: Support profiling on WindowInPandasExec Key: SPARK-46691 URL: https://issues.apache.org/jira/browse/SPARK-46691 Project: Spark Issue Type:

[jira] [Created] (SPARK-46690) Support profiling on FlatMapCoGroupsInBatchExec

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46690: - Summary: Support profiling on FlatMapCoGroupsInBatchExec Key: SPARK-46690 URL: https://issues.apache.org/jira/browse/SPARK-46690 Project: Spark Issue

[jira] [Created] (SPARK-46689) Support profiling on FlatMapGroupsInBatchExec

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46689: - Summary: Support profiling on FlatMapGroupsInBatchExec Key: SPARK-46689 URL: https://issues.apache.org/jira/browse/SPARK-46689 Project: Spark Issue Type:

[jira] [Created] (SPARK-46688) Support profiling on AggregateInPandasExec

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46688: - Summary: Support profiling on AggregateInPandasExec Key: SPARK-46688 URL: https://issues.apache.org/jira/browse/SPARK-46688 Project: Spark Issue Type:

[jira] [Created] (SPARK-46687) Implement memory-profiler

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46687: - Summary: Implement memory-profiler Key: SPARK-46687 URL: https://issues.apache.org/jira/browse/SPARK-46687 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-46686) Basic support of SparkSession based Python UDF profiler

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46686: - Summary: Basic support of SparkSession based Python UDF profiler Key: SPARK-46686 URL: https://issues.apache.org/jira/browse/SPARK-46686 Project: Spark

[jira] [Created] (SPARK-46685) Introduce SparkSession based PySpark UDF profiler

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46685: - Summary: Introduce SparkSession based PySpark UDF profiler Key: SPARK-46685 URL: https://issues.apache.org/jira/browse/SPARK-46685 Project: Spark Issue

[jira] [Assigned] (SPARK-46667) XML: Throw error on multiple XML data source

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46667: Assignee: Sandip Agarwala > XML: Throw error on multiple XML data source >

[jira] [Resolved] (SPARK-46667) XML: Throw error on multiple XML data source

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46667. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44685

[jira] [Resolved] (SPARK-46682) Upgrade `curator` to 5.6.0

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46682. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44694

[jira] [Updated] (SPARK-46683) Write a subquery generator that generates subqueries of different variations to increase testing coverage in this area

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46683: --- Labels: correctness pull-request-available testing (was: correctness testing) > Write a

[jira] [Updated] (SPARK-46684) CoGroup.applyInPandas/Arrow should pass arguments properly

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46684: --- Labels: pull-request-available (was: ) > CoGroup.applyInPandas/Arrow should pass arguments

[jira] [Updated] (SPARK-46665) Remove assertPandasOnSparkEqual

2024-01-11 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46665: Summary: Remove assertPandasOnSparkEqual (was: Remove Pandas dependency for pyspark.testing) >

[jira] [Updated] (SPARK-46665) Remove assertPandasOnSparkEqual

2024-01-11 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46665: Description: Remove deprecated API (was: We should not make pyspark.testing depending on

[jira] [Created] (SPARK-46684) CoGroup.applyInPandas/Arrow should pass arguments properly

2024-01-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-46684: - Summary: CoGroup.applyInPandas/Arrow should pass arguments properly Key: SPARK-46684 URL: https://issues.apache.org/jira/browse/SPARK-46684 Project: Spark

[jira] [Created] (SPARK-46683) Write a subquery generator that generates subqueries of different variations to increase testing coverage in this area

2024-01-11 Thread Andy Lam (Jira)
Andy Lam created SPARK-46683: Summary: Write a subquery generator that generates subqueries of different variations to increase testing coverage in this area Key: SPARK-46683 URL:

[jira] [Created] (SPARK-46682) Upgrade `curator` to 5.6.0

2024-01-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46682: - Summary: Upgrade `curator` to 5.6.0 Key: SPARK-46682 URL: https://issues.apache.org/jira/browse/SPARK-46682 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-46368) Support `readyz` in REST Submission API

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46368. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44692

[jira] [Updated] (SPARK-46671) InferFiltersFromConstraint rule is creating a redundant filter

2024-01-11 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-46671: - Description: while bring my old PR which uses a different approach to the ConstraintPropagation algorithm (

[jira] [Reopened] (SPARK-46671) InferFiltersFromConstraint rule is creating a redundant filter

2024-01-11 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif reopened SPARK-46671: -- After further analysis , I believe , that what I said originally in the ticket is valid and that the code Does

[jira] [Resolved] (SPARK-46655) Skip query context catching in DataFrame methods

2024-01-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46655. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44501

[jira] [Updated] (SPARK-46368) Support `readyz` in REST Submission API

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46368: -- Summary: Support `readyz` in REST Submission API (was: Support `readyz` API) > Support

[jira] [Updated] (SPARK-46368) Support `readyz` API

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46368: --- Labels: pull-request-available (was: ) > Support `readyz` API > > >

[jira] [Resolved] (SPARK-46680) Upgrade Apache commons-pool2 to 2.12.0

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46680. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44683

[jira] [Updated] (SPARK-46681) Refactor `ExecutorFailureTracker#maxNumExecutorFailures` to avoid unnecessary computations when `MAX_EXECUTOR_FAILURES` is configured

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46681: --- Labels: pull-request-available (was: ) > Refactor

[jira] [Created] (SPARK-46681) Refactor `ExecutorFailureTracker#maxNumExecutorFailures` to avoid unnecessary computations when `MAX_EXECUTOR_FAILURES` is configured

2024-01-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-46681: Summary: Refactor `ExecutorFailureTracker#maxNumExecutorFailures` to avoid unnecessary computations when `MAX_EXECUTOR_FAILURES` is configured Key: SPARK-46681 URL:

[jira] [Updated] (SPARK-46680) Upgrade Apache commons-pool2 to 2.12.0

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46680: --- Labels: pull-request-available (was: ) > Upgrade Apache commons-pool2 to 2.12.0 >

[jira] [Created] (SPARK-46680) Upgrade Apache commons-pool2 to 2.12.0

2024-01-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-46680: Summary: Upgrade Apache commons-pool2 to 2.12.0 Key: SPARK-46680 URL: https://issues.apache.org/jira/browse/SPARK-46680 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-46368) Support `/readyz` API

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46368: - Assignee: Dongjoon Hyun > Support `/readyz` API > - > >

[jira] [Updated] (SPARK-46368) Support `readyz` API

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46368: -- Summary: Support `readyz` API (was: Support `/readyz` API) > Support `readyz` API >

[jira] [Commented] (SPARK-44638) Unable to read from JDBC data sources when using custom schema containing varchar

2024-01-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805505#comment-17805505 ] Kent Yao commented on SPARK-44638: -- Can you reproduce this issue on 3.5.0 or master branch? > Unable

[jira] [Created] (SPARK-46679) Encoders with multiple inheritance - Key not found: T

2024-01-11 Thread Andoni Teso (Jira)
Andoni Teso created SPARK-46679: --- Summary: Encoders with multiple inheritance - Key not found: T Key: SPARK-46679 URL: https://issues.apache.org/jira/browse/SPARK-46679 Project: Spark Issue

[jira] [Updated] (SPARK-46679) Encoders with multiple inheritance - Key not found: T

2024-01-11 Thread Andoni Teso (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andoni Teso updated SPARK-46679: Attachment: spark_test.zip > Encoders with multiple inheritance - Key not found: T >

[jira] [Updated] (SPARK-46679) Encoders with multiple inheritance - Key not found: T

2024-01-11 Thread Andoni Teso (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andoni Teso updated SPARK-46679: Description: Since version 3.4, I've been experiencing the following error when using encoders.

[jira] [Assigned] (SPARK-46678) Set datanucleus.autoStartMechanismMode=ignored to clean the wall of noisy logs

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46678: - Assignee: Kent Yao > Set datanucleus.autoStartMechanismMode=ignored to clean the wall

[jira] [Resolved] (SPARK-46678) Set datanucleus.autoStartMechanismMode=ignored to clean the wall of noisy logs

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46678. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44687

[jira] [Updated] (SPARK-46678) Set datanucleus.autoStartMechanismMode=ignored to clean the wall of noisy logs

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46678: --- Labels: pull-request-available (was: ) > Set datanucleus.autoStartMechanismMode=ignored to

[jira] [Created] (SPARK-46678) Set datanucleus.autoStartMechanismMode=ignored to clean the wall of noisy logs

2024-01-11 Thread Kent Yao (Jira)
Kent Yao created SPARK-46678: Summary: Set datanucleus.autoStartMechanismMode=ignored to clean the wall of noisy logs Key: SPARK-46678 URL: https://issues.apache.org/jira/browse/SPARK-46678 Project:

[jira] [Resolved] (SPARK-46672) Upgrade log4j2 to 2.22.1

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46672. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44682

[jira] [Assigned] (SPARK-46672) Upgrade log4j2 to 2.22.1

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46672: - Assignee: Yang Jie > Upgrade log4j2 to 2.22.1 > > >

[jira] [Resolved] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46675. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44686

[jira] [Assigned] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46675: - Assignee: Cheng Pan > Remove unused inferTimestampNTZ in ParquetReadSupport >

[jira] [Assigned] (SPARK-46641) Add maxBytesPerTrigger threshold option

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46641: -- Assignee: (was: Apache Spark) > Add maxBytesPerTrigger threshold option >

[jira] [Assigned] (SPARK-46641) Add maxBytesPerTrigger threshold option

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46641: -- Assignee: Apache Spark > Add maxBytesPerTrigger threshold option >

[jira] [Assigned] (SPARK-46676) dropDuplicatesWithinWatermark throws error on canonicalizing plan

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46676: -- Assignee: (was: Apache Spark) > dropDuplicatesWithinWatermark throws error on

[jira] [Assigned] (SPARK-46676) dropDuplicatesWithinWatermark throws error on canonicalizing plan

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46676: -- Assignee: Apache Spark > dropDuplicatesWithinWatermark throws error on

[jira] [Assigned] (SPARK-46676) dropDuplicatesWithinWatermark throws error on canonicalizing plan

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46676: -- Assignee: (was: Apache Spark) > dropDuplicatesWithinWatermark throws error on

[jira] [Assigned] (SPARK-46676) dropDuplicatesWithinWatermark throws error on canonicalizing plan

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46676: -- Assignee: Apache Spark > dropDuplicatesWithinWatermark throws error on

[jira] [Assigned] (SPARK-46641) Add maxBytesPerTrigger threshold option

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46641: -- Assignee: (was: Apache Spark) > Add maxBytesPerTrigger threshold option >

[jira] [Updated] (SPARK-46676) dropDuplicatesWithinWatermark throws error on canonicalizing plan

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46676: --- Labels: pull-request-available (was: ) > dropDuplicatesWithinWatermark throws error on

[jira] [Assigned] (SPARK-46641) Add maxBytesPerTrigger threshold option

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46641: -- Assignee: Apache Spark > Add maxBytesPerTrigger threshold option >

[jira] [Assigned] (SPARK-46665) Remove Pandas dependency for pyspark.testing

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46665: -- Assignee: Apache Spark > Remove Pandas dependency for pyspark.testing >

[jira] [Assigned] (SPARK-46665) Remove Pandas dependency for pyspark.testing

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46665: -- Assignee: (was: Apache Spark) > Remove Pandas dependency for pyspark.testing >

[jira] [Assigned] (SPARK-46641) Add maxBytesPerTrigger threshold option

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46641: -- Assignee: (was: Apache Spark) > Add maxBytesPerTrigger threshold option >

[jira] [Assigned] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46675: -- Assignee: Apache Spark > Remove unused inferTimestampNTZ in ParquetReadSupport >

[jira] [Assigned] (SPARK-46660) ReattachExecute requests do not refresh aliveness of SessionHolder

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46660: -- Assignee: (was: Apache Spark) > ReattachExecute requests do not refresh

[jira] [Assigned] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46675: -- Assignee: (was: Apache Spark) > Remove unused inferTimestampNTZ in

[jira] [Assigned] (SPARK-46660) ReattachExecute requests do not refresh aliveness of SessionHolder

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46660: -- Assignee: Apache Spark > ReattachExecute requests do not refresh aliveness of

[jira] [Assigned] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46675: -- Assignee: (was: Apache Spark) > Remove unused inferTimestampNTZ in

[jira] [Assigned] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46675: -- Assignee: Apache Spark > Remove unused inferTimestampNTZ in ParquetReadSupport >

[jira] [Created] (SPARK-46676) dropDuplicatesWithinWatermark throws error on canonicalizing plan

2024-01-11 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-46676: Summary: dropDuplicatesWithinWatermark throws error on canonicalizing plan Key: SPARK-46676 URL: https://issues.apache.org/jira/browse/SPARK-46676 Project: Spark

[jira] [Resolved] (SPARK-46671) InferFiltersFromConstraint rule is creating a redundant filter

2024-01-11 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif resolved SPARK-46671. -- Resolution: Not A Bug > InferFiltersFromConstraint rule is creating a redundant filter >

[jira] [Commented] (SPARK-46671) InferFiltersFromConstraint rule is creating a redundant filter

2024-01-11 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805434#comment-17805434 ] Asif commented on SPARK-46671: -- on further thoughts , I am wrong.. There should be 2 separate isNotNull

[jira] [Commented] (SPARK-46671) InferFiltersFromConstraint rule is creating a redundant filter

2024-01-11 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805435#comment-17805435 ] Asif commented on SPARK-46671: -- so closing the ticket > InferFiltersFromConstraint rule is creating a

[jira] [Updated] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46675: --- Labels: pull-request-available (was: ) > Remove unused inferTimestampNTZ in

[jira] [Created] (SPARK-46675) Remove unused inferTimestampNTZ in ParquetReadSupport

2024-01-11 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-46675: - Summary: Remove unused inferTimestampNTZ in ParquetReadSupport Key: SPARK-46675 URL: https://issues.apache.org/jira/browse/SPARK-46675 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-46668) Parallelize Sphinx build of Python API docs

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46668: Assignee: Nicholas Chammas > Parallelize Sphinx build of Python API docs >

[jira] [Resolved] (SPARK-46668) Parallelize Sphinx build of Python API docs

2024-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46668. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44680