[jira] [Resolved] (SPARK-47921) Fix ExecuteJobTag creation in ExecuteHolder

[ https://issues.apache.org/jira/browse/SPARK-47921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47921. --- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-47346) Make daemon mode configurable when creating Python workers

[ https://issues.apache.org/jira/browse/SPARK-47346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47346. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45468

[jira] [Assigned] (SPARK-47346) Make daemon mode configurable when creating Python workers

[ https://issues.apache.org/jira/browse/SPARK-47346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-47346: - Assignee: Allison Wang > Make daemon mode configurable when creating Python workers >

[jira] [Updated] (SPARK-47251) Block invalid types from the `args` argument for `sql` command

[ https://issues.apache.org/jira/browse/SPARK-47251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47251: -- Summary: Block invalid types from the `args` argument for `sql` command (was: Block invalid

[jira] [Created] (SPARK-47251) Block invalid types from the `arg` argument for `sql` command

Takuya Ueshin created SPARK-47251: - Summary: Block invalid types from the `arg` argument for `sql` command Key: SPARK-47251 URL: https://issues.apache.org/jira/browse/SPARK-47251 Project: Spark

[jira] [Resolved] (SPARK-47214) Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments

[ https://issues.apache.org/jira/browse/SPARK-47214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47214. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45319

[jira] [Assigned] (SPARK-47214) Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments

[ https://issues.apache.org/jira/browse/SPARK-47214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-47214: - Assignee: Daniel > Create API for 'analyze' method to differentiate constant NULL

[jira] [Assigned] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

[ https://issues.apache.org/jira/browse/SPARK-47079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-47079: - Assignee: Desmond Cheong > Unable to create PySpark dataframe containing Variant

[jira] [Resolved] (SPARK-47079) Unable to create PySpark dataframe containing Variant columns

[ https://issues.apache.org/jira/browse/SPARK-47079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47079. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45131

[jira] [Resolved] (SPARK-47035) Protocol for client side StreamingQueryListener

[ https://issues.apache.org/jira/browse/SPARK-47035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47035. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45091

[jira] [Updated] (SPARK-47137) Add getAll to spark.conf for feature parity with Scala

[ https://issues.apache.org/jira/browse/SPARK-47137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47137: -- Summary: Add getAll to spark.conf for feature parity with Scala (was: Add getAll for

[jira] [Updated] (SPARK-47137) Add getAll for spark.conf for feature parity with Scala

[ https://issues.apache.org/jira/browse/SPARK-47137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47137: -- Summary: Add getAll for spark.conf for feature parity with Scala (was: Add getAll for

[jira] [Created] (SPARK-47137) Add getAll for pyspark.sql.conf for feature parity with Scala

Takuya Ueshin created SPARK-47137: - Summary: Add getAll for pyspark.sql.conf for feature parity with Scala Key: SPARK-47137 URL: https://issues.apache.org/jira/browse/SPARK-47137 Project: Spark

[jira] [Resolved] (SPARK-47069) Introduce `spark.profile.show/dump` for SparkSession-based profiling

[ https://issues.apache.org/jira/browse/SPARK-47069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47069. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45129

[jira] [Updated] (SPARK-47027) Use temporary directories for profiler test outputs

[ https://issues.apache.org/jira/browse/SPARK-47027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-47027: -- Summary: Use temporary directories for profiler test outputs (was: Move TestUtils to the

[jira] [Created] (SPARK-47027) Move TestUtils to the generic testing utils.

Takuya Ueshin created SPARK-47027: - Summary: Move TestUtils to the generic testing utils. Key: SPARK-47027 URL: https://issues.apache.org/jira/browse/SPARK-47027 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-47002) Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn

[ https://issues.apache.org/jira/browse/SPARK-47002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-47002. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45062

[jira] [Assigned] (SPARK-47002) Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn

[ https://issues.apache.org/jira/browse/SPARK-47002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-47002: - Assignee: Daniel > Enforce that 'AnalyzeResult' 'orderBy' field is a list of >

[jira] [Resolved] (SPARK-46691) Support profiling on WindowInPandasExec

[ https://issues.apache.org/jira/browse/SPARK-46691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-46691. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45035

[jira] [Assigned] (SPARK-46688) Support profiling on AggregateInPandasExec

[ https://issues.apache.org/jira/browse/SPARK-46688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-46688: - Assignee: Xinrong Meng > Support profiling on AggregateInPandasExec >

[jira] [Resolved] (SPARK-46688) Support profiling on AggregateInPandasExec

[ https://issues.apache.org/jira/browse/SPARK-46688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-46688. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45035

[jira] [Assigned] (SPARK-46691) Support profiling on WindowInPandasExec

[ https://issues.apache.org/jira/browse/SPARK-46691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-46691: - Assignee: Xinrong Meng > Support profiling on WindowInPandasExec >

[jira] [Assigned] (SPARK-46966) Create API for 'analyze' method to indicate subset of input table columns to select

[ https://issues.apache.org/jira/browse/SPARK-46966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-46966: - Assignee: Daniel > Create API for 'analyze' method to indicate subset of input table

[jira] [Resolved] (SPARK-46966) Create API for 'analyze' method to indicate subset of input table columns to select

[ https://issues.apache.org/jira/browse/SPARK-46966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-46966. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45007

[jira] [Resolved] (SPARK-46687) Implement memory-profiler

[ https://issues.apache.org/jira/browse/SPARK-46687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-46687. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44775

[jira] [Created] (SPARK-46691) Support profiling on WindowInPandasExec

Takuya Ueshin created SPARK-46691: - Summary: Support profiling on WindowInPandasExec Key: SPARK-46691 URL: https://issues.apache.org/jira/browse/SPARK-46691 Project: Spark Issue Type:

[jira] [Created] (SPARK-46690) Support profiling on FlatMapCoGroupsInBatchExec

Takuya Ueshin created SPARK-46690: - Summary: Support profiling on FlatMapCoGroupsInBatchExec Key: SPARK-46690 URL: https://issues.apache.org/jira/browse/SPARK-46690 Project: Spark Issue

[jira] [Created] (SPARK-46689) Support profiling on FlatMapGroupsInBatchExec

Takuya Ueshin created SPARK-46689: - Summary: Support profiling on FlatMapGroupsInBatchExec Key: SPARK-46689 URL: https://issues.apache.org/jira/browse/SPARK-46689 Project: Spark Issue Type:

[jira] [Created] (SPARK-46688) Support profiling on AggregateInPandasExec

Takuya Ueshin created SPARK-46688: - Summary: Support profiling on AggregateInPandasExec Key: SPARK-46688 URL: https://issues.apache.org/jira/browse/SPARK-46688 Project: Spark Issue Type:

[jira] [Created] (SPARK-46687) Implement memory-profiler

Takuya Ueshin created SPARK-46687: - Summary: Implement memory-profiler Key: SPARK-46687 URL: https://issues.apache.org/jira/browse/SPARK-46687 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-46686) Basic support of SparkSession based Python UDF profiler

Takuya Ueshin created SPARK-46686: - Summary: Basic support of SparkSession based Python UDF profiler Key: SPARK-46686 URL: https://issues.apache.org/jira/browse/SPARK-46686 Project: Spark

[jira] [Created] (SPARK-46685) Introduce SparkSession based PySpark UDF profiler

Takuya Ueshin created SPARK-46685: - Summary: Introduce SparkSession based PySpark UDF profiler Key: SPARK-46685 URL: https://issues.apache.org/jira/browse/SPARK-46685 Project: Spark Issue

[jira] [Created] (SPARK-46684) CoGroup.applyInPandas/Arrow should pass arguments properly

Takuya Ueshin created SPARK-46684: - Summary: CoGroup.applyInPandas/Arrow should pass arguments properly Key: SPARK-46684 URL: https://issues.apache.org/jira/browse/SPARK-46684 Project: Spark

[jira] [Resolved] (SPARK-46040) Update API for 'analyze' partitioning/ordering columns to support general expressions

[ https://issues.apache.org/jira/browse/SPARK-46040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-46040. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43946

[jira] [Resolved] (SPARK-45746) Return specific error messages if UDTF 'analyze' method accepts or returns wrong values

[ https://issues.apache.org/jira/browse/SPARK-45746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45746. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-45810) Create API to stop consuming rows from the input table

[ https://issues.apache.org/jira/browse/SPARK-45810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45810. --- Assignee: Daniel Resolution: Fixed Issue resolved by pull request 43682

[jira] [Created] (SPARK-45813) Return the observed metrics from commands

Takuya Ueshin created SPARK-45813: - Summary: Return the observed metrics from commands Key: SPARK-45813 URL: https://issues.apache.org/jira/browse/SPARK-45813 Project: Spark Issue Type:

[jira] [Created] (SPARK-45656) Fix observation when named observations with the same name on different datasets.

Takuya Ueshin created SPARK-45656: - Summary: Fix observation when named observations with the same name on different datasets. Key: SPARK-45656 URL: https://issues.apache.org/jira/browse/SPARK-45656

[jira] [Resolved] (SPARK-45524) Initial support for Python data source read API

[ https://issues.apache.org/jira/browse/SPARK-45524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45524. --- Fix Version/s: 4.0.0 Assignee: Allison Wang Resolution: Fixed Issue

[jira] [Created] (SPARK-45620) Fix user-facing APIs related to Python UDTF to use camelCase.

Takuya Ueshin created SPARK-45620: - Summary: Fix user-facing APIs related to Python UDTF to use camelCase. Key: SPARK-45620 URL: https://issues.apache.org/jira/browse/SPARK-45620 Project: Spark

[jira] [Resolved] (SPARK-45523) Return useful error message if UDTF returns None for non-nullable column

[ https://issues.apache.org/jira/browse/SPARK-45523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45523. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-45619) Apply the observed metrics to Observation object.

Takuya Ueshin created SPARK-45619: - Summary: Apply the observed metrics to Observation object. Key: SPARK-45619 URL: https://issues.apache.org/jira/browse/SPARK-45619 Project: Spark Issue

[jira] [Resolved] (SPARK-45577) Fix UserDefinedPythonTableFunctionAnalyzeRunner to pass folded values from named arguments

[ https://issues.apache.org/jira/browse/SPARK-45577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45577. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-45577) Fix UserDefinedPythonTableFunctionAnalyzeRunner to pass folded values from named arguments

Takuya Ueshin created SPARK-45577: - Summary: Fix UserDefinedPythonTableFunctionAnalyzeRunner to pass folded values from named arguments Key: SPARK-45577 URL: https://issues.apache.org/jira/browse/SPARK-45577

[jira] [Resolved] (SPARK-45505) Refactor analyzeInPython function to make it reusable

[ https://issues.apache.org/jira/browse/SPARK-45505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45505. --- Fix Version/s: 4.0.0 Assignee: Allison Wang Resolution: Fixed Issue

[jira] [Resolved] (SPARK-45402) Add API for 'analyze' method to return a buffer to be consumed on each class creation

[ https://issues.apache.org/jira/browse/SPARK-45402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45402. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-45494) Introduce read/write a byte array util functions for PythonWorkerUtils

Takuya Ueshin created SPARK-45494: - Summary: Introduce read/write a byte array util functions for PythonWorkerUtils Key: SPARK-45494 URL: https://issues.apache.org/jira/browse/SPARK-45494 Project:

[jira] [Created] (SPARK-45441) Introduce more util functions for PythonWorkerUtils

Takuya Ueshin created SPARK-45441: - Summary: Introduce more util functions for PythonWorkerUtils Key: SPARK-45441 URL: https://issues.apache.org/jira/browse/SPARK-45441 Project: Spark Issue

[jira] [Created] (SPARK-45405) Refactor Python UDTF execution

Takuya Ueshin created SPARK-45405: - Summary: Refactor Python UDTF execution Key: SPARK-45405 URL: https://issues.apache.org/jira/browse/SPARK-45405 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-45362) Project out PARTITION BY expressions before 'eval' method consumes input rows

[ https://issues.apache.org/jira/browse/SPARK-45362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45362. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-45266) Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used

[ https://issues.apache.org/jira/browse/SPARK-45266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45266. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-45266) Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used

Takuya Ueshin created SPARK-45266: - Summary: Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used Key: SPARK-45266 URL:

[jira] [Resolved] (SPARK-45118) Refactor converters for complex types to short cut when the element types don't need converters

[ https://issues.apache.org/jira/browse/SPARK-45118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45118. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-45159) Handle named arguments only when necessary

Takuya Ueshin created SPARK-45159: - Summary: Handle named arguments only when necessary Key: SPARK-45159 URL: https://issues.apache.org/jira/browse/SPARK-45159 Project: Spark Issue Type:

[jira] [Created] (SPARK-45118) Refactor converters for complex types to short cut when the element types don't need converters

Takuya Ueshin created SPARK-45118: - Summary: Refactor converters for complex types to short cut when the element types don't need converters Key: SPARK-45118 URL: https://issues.apache.org/jira/browse/SPARK-45118

[jira] [Resolved] (SPARK-44901) Add API in 'analyze' method to return partitioning/ordering expressions

[ https://issues.apache.org/jira/browse/SPARK-44901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44901. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-44952) Add named argument support for aggregate Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-44952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44952. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-44952) Add named argument support for aggregate Pandas UDFs

Takuya Ueshin created SPARK-44952: - Summary: Add named argument support for aggregate Pandas UDFs Key: SPARK-44952 URL: https://issues.apache.org/jira/browse/SPARK-44952 Project: Spark Issue

[jira] [Resolved] (SPARK-44918) Add named argument support for scalar Python/Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-44918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44918. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-44918) Add named argument support for scalar Python/Pandas UDFs

Takuya Ueshin created SPARK-44918: - Summary: Add named argument support for scalar Python/Pandas UDFs Key: SPARK-44918 URL: https://issues.apache.org/jira/browse/SPARK-44918 Project: Spark

[jira] [Resolved] (SPARK-44748) Query execution to support PARTITION BY and ORDER BY clause for table arguments

[ https://issues.apache.org/jira/browse/SPARK-44748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44748. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-44876) Enable and fix test_parity_arrow_python_udf

[ https://issues.apache.org/jira/browse/SPARK-44876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44876: -- Description: {{pyspark.sql.tests.connect.test_parity_arrow_python_udf}} is not listed in

[jira] [Created] (SPARK-44876) Enable and fix test_parity_arrow_python_udf

Takuya Ueshin created SPARK-44876: - Summary: Enable and fix test_parity_arrow_python_udf Key: SPARK-44876 URL: https://issues.apache.org/jira/browse/SPARK-44876 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-44834) Add SQL query test suites for Python UDTFs

[ https://issues.apache.org/jira/browse/SPARK-44834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44834. --- Fix Version/s: 3.5.0 Assignee: Allison Wang Resolution: Fixed Issue

[jira] [Resolved] (SPARK-44836) Refactor Arrow Python UDTF

[ https://issues.apache.org/jira/browse/SPARK-44836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44836. --- Fix Version/s: 3.5.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-44836) Refactor Arrow Python UDTF

Takuya Ueshin created SPARK-44836: - Summary: Refactor Arrow Python UDTF Key: SPARK-44836 URL: https://issues.apache.org/jira/browse/SPARK-44836 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-44749) Support named arguments in Python UDTF

[ https://issues.apache.org/jira/browse/SPARK-44749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44749. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-44749) Support named arguments in Python UDTF

Takuya Ueshin created SPARK-44749: - Summary: Support named arguments in Python UDTF Key: SPARK-44749 URL: https://issues.apache.org/jira/browse/SPARK-44749 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44561. --- Fix Version/s: 3.5.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Resolved] (SPARK-44433) Implement termination of Python process for foreachBatch & streaming listener

[ https://issues.apache.org/jira/browse/SPARK-44433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44433. --- Assignee: Wei Liu Resolution: Fixed Issue resolved by pull request 42283

[jira] [Resolved] (SPARK-44663) Disable arrow optimization by default for Python UDTFs

[ https://issues.apache.org/jira/browse/SPARK-44663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44663. --- Fix Version/s: 3.5.0 Assignee: Allison Wang Resolution: Fixed Issue

[jira] [Resolved] (SPARK-44644) Improve error messages for creating Python UDTFs with pickling errors

[ https://issues.apache.org/jira/browse/SPARK-44644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44644. --- Fix Version/s: 4.0.0 Target Version/s: 3.5.0 Assignee: Allison Wang

[jira] [Resolved] (SPARK-44648) Set up memory limits for analyze in Python.

[ https://issues.apache.org/jira/browse/SPARK-44648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44648. --- Assignee: Takuya Ueshin Resolution: Fixed Issue resolved by pull request 42328

[jira] [Created] (SPARK-44648) Set up memory limits for analyze in Python.

Takuya Ueshin created SPARK-44648: - Summary: Set up memory limits for analyze in Python. Key: SPARK-44648 URL: https://issues.apache.org/jira/browse/SPARK-44648 Project: Spark Issue Type:

[jira] [Created] (SPARK-44643) __repr__ broken for Row when the field is empty Row

Takuya Ueshin created SPARK-44643: - Summary: __repr__ broken for Row when the field is empty Row Key: SPARK-44643 URL: https://issues.apache.org/jira/browse/SPARK-44643 Project: Spark Issue

[jira] [Updated] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44561: -- Fix Version/s: (was: 4.0.0) > Fix AssertionError when converting UDTF output to a complex

[jira] [Assigned] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-44561: - Assignee: (was: Allison Wang) > Fix AssertionError when converting UDTF output to

[jira] [Assigned] (SPARK-44559) Improve error messages for Python UDTF arrow type casts

[ https://issues.apache.org/jira/browse/SPARK-44559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-44559: - Assignee: Allison Wang > Improve error messages for Python UDTF arrow type casts >

[jira] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561 ] Takuya Ueshin deleted comment on SPARK-44561: --- was (Author: ueshin): Issue resolved by pull request 42191 https://github.com/apache/spark/pull/42191 > Fix AssertionError when converting

[jira] [Updated] (SPARK-44614) Add missing packages in setup.py

[ https://issues.apache.org/jira/browse/SPARK-44614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44614: -- Fix Version/s: 3.5.0 > Add missing packages in setup.py > >

[jira] [Resolved] (SPARK-44614) Add missing packages in setup.py

[ https://issues.apache.org/jira/browse/SPARK-44614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44614. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Resolved] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44561. --- Fix Version/s: 4.0.0 Assignee: Allison Wang Resolution: Fixed Issue

[jira] [Created] (SPARK-44614) Add missing packages in setup.py

Takuya Ueshin created SPARK-44614: - Summary: Add missing packages in setup.py Key: SPARK-44614 URL: https://issues.apache.org/jira/browse/SPARK-44614 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-44603) Add pyspark.testing to setup.py

[ https://issues.apache.org/jira/browse/SPARK-44603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44603. --- Fix Version/s: 3.5.0 Assignee: Amanda Liu Resolution: Fixed Issue resolved

[jira] [Updated] (SPARK-44479) Support Python UDTFs with empty schema

[ https://issues.apache.org/jira/browse/SPARK-44479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44479: -- Fix Version/s: 3.5.0 > Support Python UDTFs with empty schema >

[jira] [Resolved] (SPARK-43968) Improve error messages for Python UDTFs with wrong number of outputs

[ https://issues.apache.org/jira/browse/SPARK-43968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-43968. --- Fix Version/s: 4.0.0 Assignee: Allison Wang Resolution: Fixed Issue

[jira] [Resolved] (SPARK-44533) Add support for accumulator, broadcast, and Spark files in Python UDTF's analyze.

[ https://issues.apache.org/jira/browse/SPARK-44533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44533. --- Assignee: Takuya Ueshin Resolution: Fixed Issue resolved by pull request 42135

[jira] [Resolved] (SPARK-44479) Support Python UDTFs with empty schema

[ https://issues.apache.org/jira/browse/SPARK-44479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44479. --- Assignee: Takuya Ueshin Resolution: Fixed Issue resolved by pull request 42161

[jira] [Resolved] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44503. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-44533) Add support for accumulator, broadcast, and Spark files in Python UDTF's analyze.

Takuya Ueshin created SPARK-44533: - Summary: Add support for accumulator, broadcast, and Spark files in Python UDTF's analyze. Key: SPARK-44533 URL: https://issues.apache.org/jira/browse/SPARK-44533

[jira] [Updated] (SPARK-44479) Support Python UDTFs with empty schema

[ https://issues.apache.org/jira/browse/SPARK-44479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44479: -- Description: Support UDTFs with empty schema, for example: {code:python} >>> class TestUDTF:

[jira] [Updated] (SPARK-44479) Support Python UDTFs with empty schema

[ https://issues.apache.org/jira/browse/SPARK-44479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44479: -- Description: Support UDTFs with empty schema, for example: {code:python} >>> class TestUDTF:

[jira] [Created] (SPARK-44479) Support Python UDTFs with empty schema

Takuya Ueshin created SPARK-44479: - Summary: Support Python UDTFs with empty schema Key: SPARK-44479 URL: https://issues.apache.org/jira/browse/SPARK-44479 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-44395) Update table function arguments to require parentheses around identifier after the TABLE keyword

[ https://issues.apache.org/jira/browse/SPARK-44395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44395. --- Fix Version/s: 3.5.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-44380) Support for UDTF to analyze in Python

Takuya Ueshin created SPARK-44380: - Summary: Support for UDTF to analyze in Python Key: SPARK-44380 URL: https://issues.apache.org/jira/browse/SPARK-44380 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44249) Refactor PythonUDTFRunner to send its return type separately

Takuya Ueshin created SPARK-44249: - Summary: Refactor PythonUDTFRunner to send its return type separately Key: SPARK-44249 URL: https://issues.apache.org/jira/browse/SPARK-44249 Project: Spark

[jira] [Updated] (SPARK-44233) Support an outer outer context in subquery resolution

[ https://issues.apache.org/jira/browse/SPARK-44233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44233: -- Description: {code:python} >>> sql("select * from range(8) t, lateral (select * from t) s")

[jira] [Updated] (SPARK-44233) Support an outer outer context in subquery resolution

[ https://issues.apache.org/jira/browse/SPARK-44233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44233: -- Description: {code:java} >>> sql("select * from range(8) t, lateral (select * from t) s")

[jira] [Updated] (SPARK-44233) Support an outer outer context in subquery resolution

[ https://issues.apache.org/jira/browse/SPARK-44233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44233: -- Description: {code:java} >>> sql("select * from range(8) t, lateral (select * from t) s")

[jira] [Created] (SPARK-44233) Support an outer outer context in subquery resolution

Takuya Ueshin created SPARK-44233: - Summary: Support an outer outer context in subquery resolution Key: SPARK-44233 URL: https://issues.apache.org/jira/browse/SPARK-44233 Project: Spark

  1   2   3   4   5   6   7   8   9   >