[jira] [Commented] (SPARK-50595) Temp file storing state doesn't renamed to delta file, but was read by next task

[ https://issues.apache.org/jira/browse/SPARK-50595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906303#comment-17906303 ] Tianqi Wan commented on SPARK-50595: I found that the delta file was deleted by anot

[jira] [Updated] (SPARK-50594) Align the gRPC-related Python package version and pin protobuf version in page.yml and docs/Dockerfile

[ https://issues.apache.org/jira/browse/SPARK-50594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50594: --- Labels: pull-request-available (was: ) > Align the gRPC-related Python package version and

[jira] [Assigned] (SPARK-50594) Align the gRPC-related Python package version and pin protobuf version in page.yml and docs/Dockerfile

[ https://issues.apache.org/jira/browse/SPARK-50594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-50594: Assignee: Yang Jie > Align the gRPC-related Python package version and pin protobuf version in >

[jira] [Resolved] (SPARK-50594) Align the gRPC-related Python package version and pin protobuf version in page.yml and docs/Dockerfile

[ https://issues.apache.org/jira/browse/SPARK-50594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-50594. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49205 [https://github.com

[jira] [Updated] (SPARK-50595) Temp file storing state doesn't renamed to delta file, but was read by next task

[ https://issues.apache.org/jira/browse/SPARK-50595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianqi Wan updated SPARK-50595: --- Description: Structured streaming job failed due to delta file does not exist !Snipaste_2024-12-17_

[jira] [Updated] (SPARK-50595) Temp file storing state doesn't renamed to delta file, but was read by next task

[ https://issues.apache.org/jira/browse/SPARK-50595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianqi Wan updated SPARK-50595: --- Attachment: Snipaste_2024-12-17_10-38-42.png > Temp file storing state doesn't renamed to delta file

[jira] [Created] (SPARK-50595) Temp file storing state doesn't renamed to delta file, but was read by next task

Tianqi Wan created SPARK-50595: -- Summary: Temp file storing state doesn't renamed to delta file, but was read by next task Key: SPARK-50595 URL: https://issues.apache.org/jira/browse/SPARK-50595 Project:

[jira] [Updated] (SPARK-50594) Align the gRPC-related Python package version and pin protobuf version in page.yml and docs/Dockerfile

[ https://issues.apache.org/jira/browse/SPARK-50594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-50594: - Issue Type: Improvement (was: Bug) > Align the gRPC-related Python package version and pin protobuf ver

[jira] [Updated] (SPARK-50594) Align the gRPC-related Python package version and pin protobuf version in page.yml and docs/Dockerfile

[ https://issues.apache.org/jira/browse/SPARK-50594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-50594: - Component/s: Build > Align the gRPC-related Python package version and pin protobuf version in > page.y

[jira] [Created] (SPARK-50594) Align the gRPC-related Python package version and pin protobuf version in page.yml and docs/Dockerfile

Yang Jie created SPARK-50594: Summary: Align the gRPC-related Python package version and pin protobuf version in page.yml and docs/Dockerfile Key: SPARK-50594 URL: https://issues.apache.org/jira/browse/SPARK-50594

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Issue Type: Bug (was: Question) > Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled b

[jira] [Updated] (SPARK-50593) SPJ: Support truncate transform

[ https://issues.apache.org/jira/browse/SPARK-50593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50593: --- Labels: pull-request-available (was: ) > SPJ: Support truncate transform >

[jira] [Updated] (SPARK-50593) SPJ: Support truncate transform

[ https://issues.apache.org/jira/browse/SPARK-50593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated SPARK-50593: -- Description: Truncate transform is not supported in SPJ. Support was added for write distribution and

[jira] [Created] (SPARK-50593) SPJ: Support truncate transform

Szehon Ho created SPARK-50593: - Summary: SPJ: Support truncate transform Key: SPARK-50593 URL: https://issues.apache.org/jira/browse/SPARK-50593 Project: Spark Issue Type: Sub-task Comp

[jira] [Updated] (SPARK-49839) SPJ: Skip shuffles if possible for sorts

[ https://issues.apache.org/jira/browse/SPARK-49839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated SPARK-49839: -- Parent: SPARK-48613 Issue Type: Sub-task (was: New Feature) > SPJ: Skip shuffles if possible

[jira] [Comment Edited] (SPARK-50560) randstr(-1) causes Java stacktrace and argument types are too restrictive

[ https://issues.apache.org/jira/browse/SPARK-50560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906228#comment-17906228 ] Daniel edited comment on SPARK-50560 at 12/17/24 1:06 AM: -- I ha

[jira] [Resolved] (SPARK-50592) Make 3.5 daily build able to manually trigger

[ https://issues.apache.org/jira/browse/SPARK-50592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50592. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49207 [https://

[jira] [Assigned] (SPARK-50592) Make 3.5 daily build able to manually trigger

[ https://issues.apache.org/jira/browse/SPARK-50592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50592: - Assignee: Ruifeng Zheng > Make 3.5 daily build able to manually trigger > -

[jira] [Updated] (SPARK-50578) Add support for new version of state metadata for TransformWithStateInPandas

[ https://issues.apache.org/jira/browse/SPARK-50578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50578: --- Labels: pull-request-available (was: ) > Add support for new version of state metadata for

[jira] [Updated] (SPARK-50580) Upgrade log4j2 to 2.24.3

[ https://issues.apache.org/jira/browse/SPARK-50580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50580: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade log4j2 to 2.24.

[jira] [Resolved] (SPARK-50580) Upgrade log4j2 to 2.24.3

[ https://issues.apache.org/jira/browse/SPARK-50580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50580. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49189 [https://

[jira] [Updated] (SPARK-50592) Make 3.5 daily build able to manually trigger

[ https://issues.apache.org/jira/browse/SPARK-50592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50592: --- Labels: pull-request-available (was: ) > Make 3.5 daily build able to manually trigger > --

[jira] [Created] (SPARK-50592) Make 3.5 daily build able to manually trigger

Ruifeng Zheng created SPARK-50592: - Summary: Make 3.5 daily build able to manually trigger Key: SPARK-50592 URL: https://issues.apache.org/jira/browse/SPARK-50592 Project: Spark Issue Type: S

[jira] [Commented] (SPARK-46156) Add missing parameters for Pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-46156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906058#comment-17906058 ] Shril Kumar commented on SPARK-46156: - Hi [~itholic], can you assign me something th

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Description: The task was run successfully using spark-sql -f sqlfile, but the spark-web ui showed some

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Description: The task was run successfully using spark-sql -f sqlfile, but the spark-web ui showed some

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Description: The task was run successfully using spark-sql -f sqlfile, but the spark-web ui showed some

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Attachment: spark web ui01.png > Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled bec

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Attachment: spark web ui 02_17343514806851.png > Using spark-sql -f sqlfile, some jobs failed with "Sta

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Attachment: (was: 企业微信截图_17343514629522.png) > Using spark-sql -f sqlfile, some jobs failed with "S

[jira] [Created] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

Chen Xia created SPARK-50591: Summary: Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down" Key: SPARK-50591 URL: https://issues.apache.org/jira/browse/SPARK-50591

[jira] [Updated] (SPARK-50591) Using spark-sql -f sqlfile, some jobs failed with "Stage cancelled because SparkContext was shut down"

[ https://issues.apache.org/jira/browse/SPARK-50591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Xia updated SPARK-50591: - Attachment: 企业微信截图_17343514629522.png > Using spark-sql -f sqlfile, some jobs failed with "Stage cancel

[jira] [Assigned] (SPARK-50587) Remove unsupported option `--retry-all-errors` from `release-build.sh`

[ https://issues.apache.org/jira/browse/SPARK-50587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-50587: Assignee: Yang Jie > Remove unsupported option `--retry-all-errors` from `release-build.sh` > ---

[jira] [Resolved] (SPARK-50587) Remove unsupported option `--retry-all-errors` from `release-build.sh`

[ https://issues.apache.org/jira/browse/SPARK-50587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-50587. -- Fix Version/s: 3.5.4 Resolution: Fixed Issue resolved by pull request 49201 [https://github.com

[jira] [Assigned] (SPARK-50581) Dataset.observe() is not working with UDAF

[ https://issues.apache.org/jira/browse/SPARK-50581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50581: Assignee: Tom Sisso > Dataset.observe() is not working with UDAF > --

[jira] [Resolved] (SPARK-50581) Dataset.observe() is not working with UDAF

[ https://issues.apache.org/jira/browse/SPARK-50581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50581. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49190 [https://gi

[jira] [Updated] (SPARK-50590) Skip uncessary image build and push

[ https://issues.apache.org/jira/browse/SPARK-50590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50590: --- Labels: pull-request-available (was: ) > Skip uncessary image build and push >

[jira] [Created] (SPARK-50590) Skip uncessary image build and push

Ruifeng Zheng created SPARK-50590: - Summary: Skip uncessary image build and push Key: SPARK-50590 URL: https://issues.apache.org/jira/browse/SPARK-50590 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-50589) Avoid produce duplicate non-cheap expressions when push down predicate

Mingliang Zhu created SPARK-50589: - Summary: Avoid produce duplicate non-cheap expressions when push down predicate Key: SPARK-50589 URL: https://issues.apache.org/jira/browse/SPARK-50589 Project: Spa

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: Apache Spark > Introduce an initial implementation of the single-pass Analy

[jira] [Assigned] (SPARK-50472) Introduce an initial implementation of the single-pass Analyzer

[ https://issues.apache.org/jira/browse/SPARK-50472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50472: -- Assignee: (was: Apache Spark) > Introduce an initial implementation of the single

[jira] [Updated] (SPARK-50587) Remove unsupported option `--retry-all-errors` from `release-build.sh`

[ https://issues.apache.org/jira/browse/SPARK-50587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50587: --- Labels: pull-request-available (was: ) > Remove unsupported option `--retry-all-errors` fro

[jira] [Updated] (SPARK-50588) `build-docs` skips building R docs on host when `SKIP_RDOC=1`

[ https://issues.apache.org/jira/browse/SPARK-50588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50588: --- Labels: pull-request-available (was: ) > `build-docs` skips building R docs on host when `S

[jira] [Created] (SPARK-50588) `build-docs` skips building R docs on host when `SKIP_RDOC=1`

Cheng Pan created SPARK-50588: - Summary: `build-docs` skips building R docs on host when `SKIP_RDOC=1` Key: SPARK-50588 URL: https://issues.apache.org/jira/browse/SPARK-50588 Project: Spark Issu