[jira] [Assigned] (SPARK-52356) Enable divide-by-zero for boolean mod/rmod with ANSI enabled

2025-06-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-52356: - Assignee: Xinrong Meng > Enable divide-by-zero for boolean mod/rmod with ANSI enabled >

[jira] [Resolved] (SPARK-52356) Enable divide-by-zero for boolean mod/rmod with ANSI enabled

2025-06-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-52356. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51058 [https://

[jira] [Resolved] (SPARK-52288) Avoid INVALID_ARRAY_INDEX in split when ANSI mode is on

2025-06-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-52288. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51006 [https://

[jira] [Assigned] (SPARK-52288) Avoid INVALID_ARRAY_INDEX in split when ANSI mode is on

2025-06-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-52288: - Assignee: Xinrong Meng > Avoid INVALID_ARRAY_INDEX in split when ANSI mode is on > ---

[jira] [Resolved] (SPARK-52249) Enable divide-by-zero with ANSI enabled

2025-05-30 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-52249. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50972 [https://

[jira] [Assigned] (SPARK-52249) Enable divide-by-zero with ANSI enabled

2025-05-30 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-52249: - Assignee: Xinrong Meng > Enable divide-by-zero with ANSI enabled >

[jira] [Created] (SPARK-52026) Block pandas API on Spark on ANSI mode

2025-05-07 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-52026: - Summary: Block pandas API on Spark on ANSI mode Key: SPARK-52026 URL: https://issues.apache.org/jira/browse/SPARK-52026 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-51982) Pandas API on Spark supports ANSI mode

2025-05-01 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51982: - Summary: Pandas API on Spark supports ANSI mode Key: SPARK-51982 URL: https://issues.apache.org/jira/browse/SPARK-51982 Project: Spark Issue Type: New Feat

[jira] [Created] (SPARK-51983) Prepare tests for pandas API on Spark with ANSI mode enabled

2025-05-01 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51983: - Summary: Prepare tests for pandas API on Spark with ANSI mode enabled Key: SPARK-51983 URL: https://issues.apache.org/jira/browse/SPARK-51983 Project: Spark

[jira] [Assigned] (SPARK-50131) Update isin to accept DataFrame to work as IN subquery

2025-04-15 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-50131: - Assignee: Takuya Ueshin > Update isin to accept DataFrame to work as IN subquery >

[jira] [Resolved] (SPARK-50131) Update isin to accept DataFrame to work as IN subquery

2025-04-15 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50131. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50470 [https://

[jira] [Created] (SPARK-51701) Move test objects to a separate file.

2025-04-02 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51701: - Summary: Move test objects to a separate file. Key: SPARK-51701 URL: https://issues.apache.org/jira/browse/SPARK-51701 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-51608) Better log exception in python udf worker.

2025-03-28 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51608. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49890 [https://

[jira] [Assigned] (SPARK-51619) Support UDT in Arrow-optimized Python UDF.

2025-03-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-51619: - Assignee: Takuya Ueshin > Support UDT in Arrow-optimized Python UDF. >

[jira] [Resolved] (SPARK-51619) Support UDT in Arrow-optimized Python UDF.

2025-03-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51619. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50417 [https://

[jira] [Created] (SPARK-51619) Support UDT in Arrow-optimized Python UDF.

2025-03-26 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51619: - Summary: Support UDT in Arrow-optimized Python UDF. Key: SPARK-51619 URL: https://issues.apache.org/jira/browse/SPARK-51619 Project: Spark Issue Type: Sub-

[jira] [Resolved] (SPARK-51326) Remove LazyExpression proto message.

2025-02-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51326. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50093 [https://

[jira] [Assigned] (SPARK-51326) Remove LazyExpression proto message.

2025-02-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-51326: - Assignee: Takuya Ueshin > Remove LazyExpression proto message. > --

[jira] [Created] (SPARK-51326) Remove LazyExpression proto message.

2025-02-26 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51326: - Summary: Remove LazyExpression proto message. Key: SPARK-51326 URL: https://issues.apache.org/jira/browse/SPARK-51326 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-51313) Fix timestamp format for PySparkLogger

2025-02-25 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-51313: - Assignee: Takuya Ueshin > Fix timestamp format for PySparkLogger >

[jira] [Resolved] (SPARK-51313) Fix timestamp format for PySparkLogger

2025-02-25 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51313. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50081 [https://

[jira] [Created] (SPARK-51313) Fix timestamp format for PySparkLogger

2025-02-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51313: - Summary: Fix timestamp format for PySparkLogger Key: SPARK-51313 URL: https://issues.apache.org/jira/browse/SPARK-51313 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-51274) PySparkLogger should respect the expected keyword arguments.

2025-02-21 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51274. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50032 [https://

[jira] [Assigned] (SPARK-51274) PySparkLogger should respect the expected keyword arguments.

2025-02-21 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-51274: - Assignee: Takuya Ueshin > PySparkLogger should respect the expected keyword arguments.

[jira] [Created] (SPARK-51274) PySparkLogger should respect the expected keyword arguments.

2025-02-20 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51274: - Summary: PySparkLogger should respect the expected keyword arguments. Key: SPARK-51274 URL: https://issues.apache.org/jira/browse/SPARK-51274 Project: Spark

[jira] [Created] (SPARK-51212) Add a separated PySpark package for Spark Connect by default.

2025-02-13 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51212: - Summary: Add a separated PySpark package for Spark Connect by default. Key: SPARK-51212 URL: https://issues.apache.org/jira/browse/SPARK-51212 Project: Spark

[jira] [Resolved] (SPARK-51127) Kill the Python worker on idle timeout.

2025-02-07 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51127. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49843 [https://

[jira] [Assigned] (SPARK-51127) Kill the Python worker on idle timeout.

2025-02-07 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-51127: - Assignee: Takuya Ueshin > Kill the Python worker on idle timeout. > ---

[jira] [Created] (SPARK-51127) Kill the Python worker on idle timeout.

2025-02-06 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51127: - Summary: Kill the Python worker on idle timeout. Key: SPARK-51127 URL: https://issues.apache.org/jira/browse/SPARK-51127 Project: Spark Issue Type: Improve

[jira] [Resolved] (SPARK-51099) Add logs when the Python worker looks stuck.

2025-02-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51099. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49818 [https://

[jira] [Assigned] (SPARK-51099) Add logs when the Python worker looks stuck.

2025-02-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-51099: - Assignee: Takuya Ueshin > Add logs when the Python worker looks stuck. > --

[jira] [Created] (SPARK-51099) Add logs when the Python worker looks stuck.

2025-02-05 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51099: - Summary: Add logs when the Python worker looks stuck. Key: SPARK-51099 URL: https://issues.apache.org/jira/browse/SPARK-51099 Project: Spark Issue Type: Im

[jira] [Resolved] (SPARK-50393) Introduce common TableArg for Spark Classic and Spark Connect

2025-02-04 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50393. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49782 [https://

[jira] [Created] (SPARK-51058) Avoid using jvm.SparkSession

2025-02-01 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51058: - Summary: Avoid using jvm.SparkSession Key: SPARK-51058 URL: https://issues.apache.org/jira/browse/SPARK-51058 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-51056) Avoid unnecessary object creation in SparkSession.getActiveSession.

2025-02-01 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-51056: - Assignee: Takuya Ueshin > Avoid unnecessary object creation in SparkSession.getActiveSe

[jira] [Resolved] (SPARK-51056) Avoid unnecessary object creation in SparkSession.getActiveSession.

2025-02-01 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-51056. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49758 [https://

[jira] [Created] (SPARK-51056) Avoid unnecessary object creation in SparkSession.getActiveSession.

2025-01-31 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-51056: - Summary: Avoid unnecessary object creation in SparkSession.getActiveSession. Key: SPARK-51056 URL: https://issues.apache.org/jira/browse/SPARK-51056 Project: Spark

[jira] [Assigned] (SPARK-50968) Fix the usage of `Column.__new__`.

2025-01-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-50968: - Assignee: Takuya Ueshin > Fix the usage of `Column.__new__`. >

[jira] [Resolved] (SPARK-50968) Fix the usage of `Column.__new__`.

2025-01-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50968. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49631 [https://

[jira] [Created] (SPARK-50968) Fix the usage of `Column.__new__`.

2025-01-23 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50968: - Summary: Fix the usage of `Column.__new__`. Key: SPARK-50968 URL: https://issues.apache.org/jira/browse/SPARK-50968 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-50909) Setup faulthandler in PythonPlannerRunners.

2025-01-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50909. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 49592 [https://

[jira] [Assigned] (SPARK-50909) Setup faulthandler in PythonPlannerRunners.

2025-01-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-50909: - Assignee: Takuya Ueshin > Setup faulthandler in PythonPlannerRunners. > ---

[jira] [Created] (SPARK-50909) Setup faulthandler in PythonPlannerRunners.

2025-01-21 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50909: - Summary: Setup faulthandler in PythonPlannerRunners. Key: SPARK-50909 URL: https://issues.apache.org/jira/browse/SPARK-50909 Project: Spark Issue Type: Imp

[jira] [Assigned] (SPARK-50601) Support withColumn in subqueries

2025-01-13 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-50601: - Assignee: Takuya Ueshin > Support withColumn in subqueries > --

[jira] [Resolved] (SPARK-50601) Support withColumn in subqueries

2025-01-13 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50601. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49386 [https://

[jira] [Resolved] (SPARK-50789) The inputs for typed aggregations should be analyzed.

2025-01-13 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50789. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49449 [https://

[jira] [Created] (SPARK-50789) The inputs for typed aggregations should be analyzed.

2025-01-10 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50789: - Summary: The inputs for typed aggregations should be analyzed. Key: SPARK-50789 URL: https://issues.apache.org/jira/browse/SPARK-50789 Project: Spark Issue

[jira] [Created] (SPARK-50778) Add metadataColumn to PySpark DataFrame

2025-01-09 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50778: - Summary: Add metadataColumn to PySpark DataFrame Key: SPARK-50778 URL: https://issues.apache.org/jira/browse/SPARK-50778 Project: Spark Issue Type: Improve

[jira] [Resolved] (SPARK-50694) Support renames in subqueries

2025-01-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50694. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49336 [https://

[jira] [Assigned] (SPARK-50694) Support renames in subqueries

2025-01-06 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-50694: - Assignee: Takuya Ueshin > Support renames in subqueries > -

[jira] [Resolved] (SPARK-50693) The inputs for TypedScalaUdf should be analyzed

2024-12-30 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50693. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49327 [https://

[jira] [Assigned] (SPARK-50693) The inputs for TypedScalaUdf should be analyzed

2024-12-30 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-50693: - Assignee: Takuya Ueshin > The inputs for TypedScalaUdf should be analyzed > ---

[jira] [Resolved] (SPARK-50134) Support Spark Connect

2024-12-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50134. --- Assignee: Takuya Ueshin Resolution: Fixed > Support Spark Connect > --

[jira] [Updated] (SPARK-50601) Support withColumn in subqueries

2024-12-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-50601: -- Component/s: PySpark > Support withColumn in subqueries > > >

[jira] [Updated] (SPARK-50694) Support renames in subqueries

2024-12-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-50694: -- Component/s: SQL > Support renames in subqueries > - > >

[jira] [Updated] (SPARK-50601) Support withColumn in subqueries

2024-12-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-50601: -- Component/s: SQL > Support withColumn in subqueries > > >

[jira] [Updated] (SPARK-50601) Support withColumn in subqueries

2024-12-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-50601: -- Summary: Support withColumn in subqueries (was: Support subqueries in withColumn) > Support

[jira] [Created] (SPARK-50694) Support renames in subqueries

2024-12-27 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50694: - Summary: Support renames in subqueries Key: SPARK-50694 URL: https://issues.apache.org/jira/browse/SPARK-50694 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-50693) The inputs for TypedScalaUdf should be analyzed

2024-12-27 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50693: - Summary: The inputs for TypedScalaUdf should be analyzed Key: SPARK-50693 URL: https://issues.apache.org/jira/browse/SPARK-50693 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-50602) Transpose should show a proper error message when invalid index columns are specified

2024-12-19 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-50602: - Assignee: Takuya Ueshin > Transpose should show a proper error message when invalid ind

[jira] [Resolved] (SPARK-50602) Transpose should show a proper error message when invalid index columns are specified

2024-12-19 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-50602. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49219 [https://

[jira] [Updated] (SPARK-50602) Transpose should show a proper error message when invalid index columns are specified

2024-12-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-50602: -- Description: {code:python} >>> df = spark.range(10).transpose(sf.col("id") + 1) Traceback (mos

[jira] [Created] (SPARK-50601) Support subqueries in withColumn

2024-12-17 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50601: - Summary: Support subqueries in withColumn Key: SPARK-50601 URL: https://issues.apache.org/jira/browse/SPARK-50601 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-49995) Add named argument support to more TVFs

2024-11-05 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-49995. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48503 [https://

[jira] [Created] (SPARK-50196) DataFrameQueryContext in Spark Connect Python client has a different fragment than Spark Classic

2024-10-31 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50196: - Summary: DataFrameQueryContext in Spark Connect Python client has a different fragment than Spark Classic Key: SPARK-50196 URL: https://issues.apache.org/jira/browse/SPARK-50196

[jira] [Created] (SPARK-50134) Support Spark Connect

2024-10-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50134: - Summary: Support Spark Connect Key: SPARK-50134 URL: https://issues.apache.org/jira/browse/SPARK-50134 Project: Spark Issue Type: Sub-task Compon

[jira] [Created] (SPARK-50133) Support conversion to table argument

2024-10-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50133: - Summary: Support conversion to table argument Key: SPARK-50133 URL: https://issues.apache.org/jira/browse/SPARK-50133 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-50132) Update join to support lateral join

2024-10-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50132: - Summary: Update join to support lateral join Key: SPARK-50132 URL: https://issues.apache.org/jira/browse/SPARK-50132 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-50131) Update isin to accept DataFrame to work as IN subquery

2024-10-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50131: - Summary: Update isin to accept DataFrame to work as IN subquery Key: SPARK-50131 URL: https://issues.apache.org/jira/browse/SPARK-50131 Project: Spark Issu

[jira] [Created] (SPARK-50130) Add scalar and exists DataFrame APIs

2024-10-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50130: - Summary: Add scalar and exists DataFrame APIs Key: SPARK-50130 URL: https://issues.apache.org/jira/browse/SPARK-50130 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-50063) Support Variant in Spark Connect Scala client

2024-10-21 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50063: - Summary: Support Variant in Spark Connect Scala client Key: SPARK-50063 URL: https://issues.apache.org/jira/browse/SPARK-50063 Project: Spark Issue Type: S

[jira] [Created] (SPARK-49995) Add named argument support to more TVFs

2024-10-16 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-49995: - Summary: Add named argument support to more TVFs Key: SPARK-49995 URL: https://issues.apache.org/jira/browse/SPARK-49995 Project: Spark Issue Type: Improve

[jira] [Assigned] (SPARK-49009) Make Column APIs and functions accept Enums.

2024-07-26 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-49009: - Assignee: Takuya Ueshin > Make Column APIs and functions accept Enums. > --

[jira] [Resolved] (SPARK-49009) Make Column APIs and functions accept Enums.

2024-07-26 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-49009. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47495 [https://

[jira] [Created] (SPARK-49009) Make Column APIs and functions accept Enums.

2024-07-25 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-49009: - Summary: Make Column APIs and functions accept Enums. Key: SPARK-49009 URL: https://issues.apache.org/jira/browse/SPARK-49009 Project: Spark Issue Type: Im

[jira] [Resolved] (SPARK-48996) Allow bare literals for __and__ and __or__ of Column

2024-07-25 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-48996. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47474 [https://

[jira] [Assigned] (SPARK-48996) Allow bare literals for __and__ and __or__ of Column

2024-07-25 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-48996: - Assignee: Takuya Ueshin > Allow bare literals for __and__ and __or__ of Column > --

[jira] [Created] (SPARK-48996) Allow bare literals for __and__ and __or__ of Column

2024-07-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-48996: - Summary: Allow bare literals for __and__ and __or__ of Column Key: SPARK-48996 URL: https://issues.apache.org/jira/browse/SPARK-48996 Project: Spark Issue

[jira] [Created] (SPARK-48872) Reduce the overhead of _capture_call_site

2024-07-11 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-48872: - Summary: Reduce the overhead of _capture_call_site Key: SPARK-48872 URL: https://issues.apache.org/jira/browse/SPARK-48872 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-48862) Avoid calling `_proto_to_string` when INFO level is not enabled.

2024-07-10 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-48862: - Summary: Avoid calling `_proto_to_string` when INFO level is not enabled. Key: SPARK-48862 URL: https://issues.apache.org/jira/browse/SPARK-48862 Project: Spark

[jira] [Resolved] (SPARK-48798) Introduce `spark.profile.render` for SparkSession-based profiling

2024-07-08 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-48798. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47202 [https://

[jira] [Assigned] (SPARK-48566) [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

2024-06-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-48566: - Assignee: Daniel > [Bug] Partition indices are incorrect when UDTF analyze() uses both

[jira] [Resolved] (SPARK-48566) [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

2024-06-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-48566. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46918 [https://

[jira] [Resolved] (SPARK-47921) Fix ExecuteJobTag creation in ExecuteHolder

2024-04-24 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47921. --- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pul

[jira] [Resolved] (SPARK-47346) Make daemon mode configurable when creating Python workers

2024-03-15 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47346. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45468 [https://

[jira] [Assigned] (SPARK-47346) Make daemon mode configurable when creating Python workers

2024-03-15 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-47346: - Assignee: Allison Wang > Make daemon mode configurable when creating Python workers > -

[jira] [Updated] (SPARK-47251) Block invalid types from the `args` argument for `sql` command

2024-03-01 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47251: -- Summary: Block invalid types from the `args` argument for `sql` command (was: Block invalid t

[jira] [Created] (SPARK-47251) Block invalid types from the `arg` argument for `sql` command

2024-03-01 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-47251: - Summary: Block invalid types from the `arg` argument for `sql` command Key: SPARK-47251 URL: https://issues.apache.org/jira/browse/SPARK-47251 Project: Spark

[jira] [Resolved] (SPARK-47214) Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments

2024-02-28 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47214. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45319 [https://

[jira] [Assigned] (SPARK-47214) Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments

2024-02-28 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-47214: - Assignee: Daniel > Create API for 'analyze' method to differentiate constant NULL argum

[jira] [Assigned] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

2024-02-26 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-47079: - Assignee: Desmond Cheong > Unable to create PySpark dataframe containing Variant column

[jira] [Resolved] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

2024-02-26 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47079. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45131 [https://

[jira] [Resolved] (SPARK-47035) Protocol for client side StreamingQueryListener

2024-02-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47035. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45091 [https://

[jira] [Updated] (SPARK-47137) Add getAll to spark.conf for feature parity with Scala

2024-02-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47137: -- Summary: Add getAll to spark.conf for feature parity with Scala (was: Add getAll for spark.co

[jira] [Updated] (SPARK-47137) Add getAll for spark.conf for feature parity with Scala

2024-02-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47137: -- Summary: Add getAll for spark.conf for feature parity with Scala (was: Add getAll for pyspark

[jira] [Created] (SPARK-47137) Add getAll for pyspark.sql.conf for feature parity with Scala

2024-02-22 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-47137: - Summary: Add getAll for pyspark.sql.conf for feature parity with Scala Key: SPARK-47137 URL: https://issues.apache.org/jira/browse/SPARK-47137 Project: Spark

[jira] [Resolved] (SPARK-47069) Introduce `spark.profile.show/dump` for SparkSession-based profiling

2024-02-22 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47069. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45129 [https://

[jira] [Updated] (SPARK-47027) Use temporary directories for profiler test outputs

2024-02-12 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47027: -- Summary: Use temporary directories for profiler test outputs (was: Move TestUtils to the gene

[jira] [Created] (SPARK-47027) Move TestUtils to the generic testing utils.

2024-02-12 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-47027: - Summary: Move TestUtils to the generic testing utils. Key: SPARK-47027 URL: https://issues.apache.org/jira/browse/SPARK-47027 Project: Spark Issue Type: Te

[jira] [Resolved] (SPARK-47002) Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn

2024-02-08 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47002. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45062 [https://

  1   2   3   4   5   6   7   8   9   10   >