[jira] [Updated] (SPARK-50607) Passdown fileLength when reading parquet footer to avoid call HDFS namenode in executor side

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50607: --- Labels: pull-request-available (was: ) > Passdown fileLength when reading parquet footer to

[jira] [Assigned] (SPARK-50604) Extract MariaDBDatabaseOnDocker and upgrade Mariadb docker image version

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50604: --- Assignee: Pan Bingkun > Extract MariaDBDatabaseOnDocker and upgrade Mariadb docker image ve

[jira] [Resolved] (SPARK-50604) Extract MariaDBDatabaseOnDocker and upgrade Mariadb docker image version

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50604. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49059 [https://gith

[jira] [Resolved] (SPARK-50596) Upgrade Py4J from 0.10.9.7 to 0.10.9.8

2024-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50596. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49214 [https://gi

[jira] [Resolved] (SPARK-50504) Enable SQL pipe syntax by default

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50504. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49079 [https://gith

[jira] [Updated] (SPARK-50609) Eliminate warnings in docker image building

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50609: --- Labels: pull-request-available (was: ) > Eliminate warnings in docker image building >

[jira] [Created] (SPARK-50609) Eliminate warnings in docker image building

2024-12-17 Thread Pan Bingkun (Jira)
Pan Bingkun created SPARK-50609: --- Summary: Eliminate warnings in docker image building Key: SPARK-50609 URL: https://issues.apache.org/jira/browse/SPARK-50609 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-50608) Malformed configuration page caused by unclosed tags

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50608: --- Labels: pull-request-available (was: ) > Malformed configuration page caused by unclosed ta

[jira] [Assigned] (SPARK-50609) Eliminate warnings in docker image building

2024-12-17 Thread Pan Bingkun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pan Bingkun reassigned SPARK-50609: --- Assignee: Pan Bingkun > Eliminate warnings in docker image building > -

[jira] [Resolved] (SPARK-50599) Create the DataEncoder trait that allows for Avro and UnsafeRow encoding

2024-12-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-50599. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48944 [https://gi

[jira] [Created] (SPARK-50608) Malformed configuration page caused by unclosed tags

2024-12-17 Thread Kent Yao (Jira)
Kent Yao created SPARK-50608: Summary: Malformed configuration page caused by unclosed tags Key: SPARK-50608 URL: https://issues.apache.org/jira/browse/SPARK-50608 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-50607) Passdown fileLength when reading parquet footer to avoid call HDFS namenode in executor side

2024-12-17 Thread EdisonWang (Jira)
EdisonWang created SPARK-50607: -- Summary: Passdown fileLength when reading parquet footer to avoid call HDFS namenode in executor side Key: SPARK-50607 URL: https://issues.apache.org/jira/browse/SPARK-50607

[jira] [Updated] (SPARK-50606) NPE on uninitiated SessionHolder

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50606: --- Labels: pull-request-available (was: ) > NPE on uninitiated SessionHolder > ---

[jira] [Updated] (SPARK-40993) Migrate markdown style README to python/docs/development/testing.rst

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-40993: --- Labels: pull-request-available (was: ) > Migrate markdown style README to python/docs/devel

[jira] [Updated] (SPARK-50604) Extract MariaDBDatabaseOnDocker and upgrade Mariadb docker image version

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50604: --- Labels: pull-request-available (was: ) > Extract MariaDBDatabaseOnDocker and upgrade Mariad

[jira] [Updated] (SPARK-50604) Extract MariaDBDatabaseOnDocker and upgrade Mariadb docker image version

2024-12-17 Thread Pan Bingkun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pan Bingkun updated SPARK-50604: Summary: Extract MariaDBDatabaseOnDocker and upgrade Mariadb docker image version (was: Extract M

[jira] [Created] (SPARK-50604) Extract MariaDB image common conf as MariaDBDatabaseOnDocker

2024-12-17 Thread Pan Bingkun (Jira)
Pan Bingkun created SPARK-50604: --- Summary: Extract MariaDB image common conf as MariaDBDatabaseOnDocker Key: SPARK-50604 URL: https://issues.apache.org/jira/browse/SPARK-50604 Project: Spark I

[jira] [Updated] (SPARK-50603) basePath can cause failures on streaming reads for file sources

2024-12-17 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford updated SPARK-50603: - Description: Using the `basePath` option on a streaming read from a file source can cause the q

[jira] [Updated] (SPARK-50603) basePath can cause failures on streaming reads for file sources

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50603: --- Labels: pull-request-available (was: ) > basePath can cause failures on streaming reads for

[jira] [Updated] (SPARK-50603) basePath can cause failures on streaming reads for file sources

2024-12-17 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford updated SPARK-50603: - Description: Using the `basePath` option on a streaming read from a file source can cause the q

[jira] [Updated] (SPARK-50602) Transpose should show a proper error message when invalid index columns are specified

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50602: --- Labels: pull-request-available (was: ) > Transpose should show a proper error message when

[jira] [Updated] (SPARK-50603) basePath can cause failures on streaming reads for file sources

2024-12-17 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford updated SPARK-50603: - Description: Using the `basePath` option on a streaming read from a file source can cause the q

[jira] [Updated] (SPARK-50603) basePath can cause failures on streaming reads for file sources

2024-12-17 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford updated SPARK-50603: - Description: Using the `basePath` option on a streaming read from a file source can cause the q

[jira] [Created] (SPARK-50603) basePath can cause failures on streaming reads for file sources

2024-12-17 Thread Adam Binford (Jira)
Adam Binford created SPARK-50603: Summary: basePath can cause failures on streaming reads for file sources Key: SPARK-50603 URL: https://issues.apache.org/jira/browse/SPARK-50603 Project: Spark

[jira] [Updated] (SPARK-50602) Transpose should show a proper error message when invalid index columns are specified

2024-12-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-50602: -- Description: {code:python} >>> df = spark.range(10).transpose(sf.col("id") + 1) Traceback (mos

[jira] [Created] (SPARK-50601) Support subqueries in withColumn

2024-12-17 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-50601: - Summary: Support subqueries in withColumn Key: SPARK-50601 URL: https://issues.apache.org/jira/browse/SPARK-50601 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-50600) Send optimize plan during ReadyForExecution event

2024-12-17 Thread Jean-Francois Desjeans Gauthier (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Francois Desjeans Gauthier updated SPARK-50600: Description: As part of SPARK-44145, a callback was added

[jira] [Updated] (SPARK-50600) Send optimize plan during ReadyForExecution event

2024-12-17 Thread Jean-Francois Desjeans Gauthier (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Francois Desjeans Gauthier updated SPARK-50600: Description: As part of SPARK-44145, a callback was added

[jira] [Created] (SPARK-50600) Send optimize plan during ReadyForExecution event

2024-12-17 Thread Jean-Francois Desjeans Gauthier (Jira)
Jean-Francois Desjeans Gauthier created SPARK-50600: --- Summary: Send optimize plan during ReadyForExecution event Key: SPARK-50600 URL: https://issues.apache.org/jira/browse/SPARK-50600

[jira] [Resolved] (SPARK-50598) Extend CTE logical plan classes with recursion parameter

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50598. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49180 [https://gith

[jira] [Assigned] (SPARK-50598) Extend CTE logical plan classes with recursion parameter

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50598: --- Assignee: Milan Stefanovic > Extend CTE logical plan classes with recursion parameter > ---

[jira] [Updated] (SPARK-50598) Extend CTE logical plan classes with recursion parameter

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50598: --- Labels: pull-request-available (was: ) > Extend CTE logical plan classes with recursion par

[jira] [Created] (SPARK-50598) Extend CTE logical plan classes with recursion parameter

2024-12-17 Thread Milan Stefanovic (Jira)
Milan Stefanovic created SPARK-50598: Summary: Extend CTE logical plan classes with recursion parameter Key: SPARK-50598 URL: https://issues.apache.org/jira/browse/SPARK-50598 Project: Spark

[jira] [Commented] (SPARK-49087) Move all Connect invocations of system.internal function to the proper namespace

2024-12-17 Thread Pengfei Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906476#comment-17906476 ] Pengfei Xu commented on SPARK-49087: Start working on this one. > Move all Connect

[jira] [Resolved] (SPARK-50597) Refactor batch creation style in Optimizer

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50597. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49208 [https://gith

[jira] [Updated] (SPARK-49636) Remove the ANSI config suggestion in INVALID_ARRAY_INDEX and INVALID_ARRAY_INDEX_IN_ELEMENT_AT

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49636: --- Labels: pull-request-available starter (was: starter) > Remove the ANSI config suggestion i

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: Apache Spark > Introduce an initial implementation of the single-pass Analy

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: (was: Apache Spark) > Introduce an initial implementation of the single

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: (was: Apache Spark) > Introduce an initial implementation of the single

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: Apache Spark > Introduce an initial implementation of the single-pass Analy

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: Apache Spark > Introduce an initial implementation of the single-pass Analy

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

2024-12-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: (was: Apache Spark) > Introduce an initial implementation of the single

[jira] [Updated] (SPARK-24497) ANSI SQL: Recursive query

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24497: Parent: (was: SPARK-30374) Issue Type: New Feature (was: Sub-task) > ANSI SQL: Recurs

[jira] [Updated] (SPARK-24497) ANSI SQL: Recursive query

2024-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24497: Epic Link: SPARK-35030 > ANSI SQL: Recursive query > - > >

[jira] [Assigned] (SPARK-50588) `build-docs` skips building R docs on host when `SKIP_RDOC=1`

2024-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50588: Assignee: Cheng Pan > `build-docs` skips building R docs on host when `SKIP_RDOC=1` > ---

[jira] [Resolved] (SPARK-50588) `build-docs` skips building R docs on host when `SKIP_RDOC=1`

2024-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50588. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49200 [https://gi

[jira] [Updated] (SPARK-50595) Structured streaming job failed due to checkpoint file does not exist

2024-12-17 Thread Tianqi Wan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianqi Wan updated SPARK-50595: --- Summary: Structured streaming job failed due to checkpoint file does not exist (was: Temp file stor