[jira] [Updated] (SPARK-41253) Make K8s volcano IT work in Github Action

[ https://issues.apache.org/jira/browse/SPARK-41253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41253: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Improvement) > Make K8s volcano IT wor

[jira] [Updated] (SPARK-41552) Upgrade `kubernetes-client` to 6.3.1

[ https://issues.apache.org/jira/browse/SPARK-41552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41552: -- Summary: Upgrade `kubernetes-client` to 6.3.1 (was: Upgrade kubernetes-client to 6.3.1) > Up

[jira] [Resolved] (SPARK-42802) Customized K8s Scheduler GA

[ https://issues.apache.org/jira/browse/SPARK-42802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42802. --- Fix Version/s: 3.4.0 Target Version/s: 3.4.0 Assignee: Dongjoon Hyun

[jira] [Updated] (SPARK-42802) Customized K8s Scheduler GA

[ https://issues.apache.org/jira/browse/SPARK-42802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42802: -- Labels: releasenotes (was: ) > Customized K8s Scheduler GA > --- > >

[jira] [Updated] (SPARK-42802) Customized K8s Scheduler GA

[ https://issues.apache.org/jira/browse/SPARK-42802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42802: -- Description: Since Apache Spark 3.3.1, we have been supported the customized K8s scheduler via

[jira] [Assigned] (SPARK-42792) Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming

[ https://issues.apache.org/jira/browse/SPARK-42792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42792: Assignee: Anish Shrigondekar > Add support to track FLUSH_WRITE_BYTES for RocksDB state s

[jira] [Resolved] (SPARK-42792) Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming

[ https://issues.apache.org/jira/browse/SPARK-42792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42792. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40427 [https://gi

[jira] [Updated] (SPARK-42706) Document the Spark SQL error classes in user-facing documentation.

[ https://issues.apache.org/jira/browse/SPARK-42706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-42706: - Fix Version/s: 3.4.1 > Document the Spark SQL error classes in user-facing documentation. >

[jira] [Updated] (SPARK-40941) Use Java 17 in K8s Dockerfile by default and remove `Dockerfile.java17`

[ https://issues.apache.org/jira/browse/SPARK-40941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40941: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Improvement) > Use Java 17 in K8s Dock

[jira] [Updated] (SPARK-41552) Upgrade kubernetes-client to 6.3.1

[ https://issues.apache.org/jira/browse/SPARK-41552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41552: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Improvement) > Upgrade kubernetes-clie

[jira] [Updated] (SPARK-41502) Upgrade the minimum Minikube version to 1.28.0

[ https://issues.apache.org/jira/browse/SPARK-41502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41502: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Test) > Upgrade the minimum Minikube v

[jira] [Updated] (SPARK-42148) Upgrade `kubernetes-client` to 6.4.0

[ https://issues.apache.org/jira/browse/SPARK-42148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42148: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Improvement) > Upgrade `kubernetes-cli

[jira] [Updated] (SPARK-42154) Enable Volcano unit tests and integration tests in GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-42154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42154: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Improvement) > Enable Volcano unit tes

[jira] [Updated] (SPARK-42150) Upgrade Volcano to 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-42150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42150: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Improvement) > Upgrade Volcano to 1.7.

[jira] [Updated] (SPARK-42545) Remove `experimental` from Volcano docs

[ https://issues.apache.org/jira/browse/SPARK-42545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42545: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Documentation) > Remove `experimental`

[jira] [Updated] (SPARK-42532) Update YuniKorn documentation with v1.2

[ https://issues.apache.org/jira/browse/SPARK-42532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42532: -- Parent: SPARK-42802 Issue Type: Sub-task (was: Documentation) > Update YuniKorn docum

[jira] [Created] (SPARK-42802) Customized K8s Scheduler GA

Dongjoon Hyun created SPARK-42802: - Summary: Customized K8s Scheduler GA Key: SPARK-42802 URL: https://issues.apache.org/jira/browse/SPARK-42802 Project: Spark Issue Type: Umbrella

[jira] [Assigned] (SPARK-42801) Fix Flaky ClientE2ETestSuite

[ https://issues.apache.org/jira/browse/SPARK-42801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42801: - Assignee: Dongjoon Hyun > Fix Flaky ClientE2ETestSuite > >

[jira] [Resolved] (SPARK-42801) Fix Flaky ClientE2ETestSuite

[ https://issues.apache.org/jira/browse/SPARK-42801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42801. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40434 [https://

[jira] [Assigned] (SPARK-42801) Fix Flaky ClientE2ETestSuite

[ https://issues.apache.org/jira/browse/SPARK-42801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42801: Assignee: (was: Apache Spark) > Fix Flaky ClientE2ETestSuite > --

[jira] [Assigned] (SPARK-42801) Fix Flaky ClientE2ETestSuite

[ https://issues.apache.org/jira/browse/SPARK-42801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42801: Assignee: Apache Spark > Fix Flaky ClientE2ETestSuite > > >

[jira] [Commented] (SPARK-42801) Fix Flaky ClientE2ETestSuite

[ https://issues.apache.org/jira/browse/SPARK-42801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700513#comment-17700513 ] Apache Spark commented on SPARK-42801: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-42750) Support INSERT INTO by name

[ https://issues.apache.org/jira/browse/SPARK-42750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700509#comment-17700509 ] Xinsen commented on SPARK-42750: May I solve it? I'm interested in this. And by the way,

[jira] [Created] (SPARK-42801) Fix Flaky ClientE2ETestSuite

Dongjoon Hyun created SPARK-42801: - Summary: Fix Flaky ClientE2ETestSuite Key: SPARK-42801 URL: https://issues.apache.org/jira/browse/SPARK-42801 Project: Spark Issue Type: Bug Comp

[jira] [Commented] (SPARK-42706) Document the Spark SQL error classes in user-facing documentation.

[ https://issues.apache.org/jira/browse/SPARK-42706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700505#comment-17700505 ] Apache Spark commented on SPARK-42706: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-42800) Implement ml function {array_to_vector, vector_to_array}

[ https://issues.apache.org/jira/browse/SPARK-42800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42800: Assignee: Apache Spark > Implement ml function {array_to_vector, vector_to_array} > -

[jira] [Assigned] (SPARK-42800) Implement ml function {array_to_vector, vector_to_array}

[ https://issues.apache.org/jira/browse/SPARK-42800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42800: Assignee: (was: Apache Spark) > Implement ml function {array_to_vector, vector_to_arr

[jira] [Commented] (SPARK-42800) Implement ml function {array_to_vector, vector_to_array}

[ https://issues.apache.org/jira/browse/SPARK-42800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700496#comment-17700496 ] Apache Spark commented on SPARK-42800: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-42800) Implement ml function {array_to_vector, vector_to_array}

Ruifeng Zheng created SPARK-42800: - Summary: Implement ml function {array_to_vector, vector_to_array} Key: SPARK-42800 URL: https://issues.apache.org/jira/browse/SPARK-42800 Project: Spark Is

[jira] [Updated] (SPARK-42799) Update SBT build `xercesImpl` version to match with pom.xml

[ https://issues.apache.org/jira/browse/SPARK-42799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42799: -- Affects Version/s: 3.3.1 3.3.0 3.4.0

[jira] [Updated] (SPARK-42797) Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages

[ https://issues.apache.org/jira/browse/SPARK-42797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42797: - Fix Version/s: 3.4.1 (was: 3.4.0) > Spark Connect - Grammatical improveme

[jira] [Assigned] (SPARK-42765) Enable importing `pandas_udf` from `pyspark.sql.connect.functions`

[ https://issues.apache.org/jira/browse/SPARK-42765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42765: Assignee: Xinrong Meng > Enable importing `pandas_udf` from `pyspark.sql.connect.function

[jira] [Resolved] (SPARK-42765) Enable importing `pandas_udf` from `pyspark.sql.connect.functions`

[ https://issues.apache.org/jira/browse/SPARK-42765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42765. -- Fix Version/s: 3.4.1 Resolution: Fixed Issue resolved by pull request 40388 [https://gi

[jira] [Resolved] (SPARK-42797) Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages

[ https://issues.apache.org/jira/browse/SPARK-42797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42797. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40428 [https://gi

[jira] [Assigned] (SPARK-42799) Update SBT build `xercesImpl` version to match with pom.xml

[ https://issues.apache.org/jira/browse/SPARK-42799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42799: Assignee: Apache Spark > Update SBT build `xercesImpl` version to match with pom.xml > --

[jira] [Assigned] (SPARK-42797) Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages

[ https://issues.apache.org/jira/browse/SPARK-42797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42797: Assignee: Allan Folting > Spark Connect - Grammatical improvements to Spark Overview and

[jira] [Assigned] (SPARK-42799) Update SBT build `xercesImpl` version to match with pom.xml

[ https://issues.apache.org/jira/browse/SPARK-42799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42799: Assignee: (was: Apache Spark) > Update SBT build `xercesImpl` version to match with p

[jira] [Commented] (SPARK-42799) Update SBT build `xercesImpl` version to match with pom.xml

[ https://issues.apache.org/jira/browse/SPARK-42799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700492#comment-17700492 ] Apache Spark commented on SPARK-42799: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-42799) Update SBT build `xercesImpl` version to match with pom.xml

Dongjoon Hyun created SPARK-42799: - Summary: Update SBT build `xercesImpl` version to match with pom.xml Key: SPARK-42799 URL: https://issues.apache.org/jira/browse/SPARK-42799 Project: Spark

[jira] [Resolved] (SPARK-42666) Fix `createDataFrame` to work properly with rows and schema

[ https://issues.apache.org/jira/browse/SPARK-42666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-42666. - Fix Version/s: 3.4.0 Resolution: Duplicate Resolved from SPARK-42679 > Fix `createDataFr

[jira] [Assigned] (SPARK-42798) Upgrade protobuf-java to 3.22.2

[ https://issues.apache.org/jira/browse/SPARK-42798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42798: Assignee: Apache Spark > Upgrade protobuf-java to 3.22.2 > --

[jira] [Assigned] (SPARK-42798) Upgrade protobuf-java to 3.22.2

[ https://issues.apache.org/jira/browse/SPARK-42798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42798: Assignee: (was: Apache Spark) > Upgrade protobuf-java to 3.22.2 > ---

[jira] [Commented] (SPARK-42798) Upgrade protobuf-java to 3.22.2

[ https://issues.apache.org/jira/browse/SPARK-42798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700484#comment-17700484 ] Apache Spark commented on SPARK-42798: -- User 'LuciferYang' has created a pull reque

[jira] [Updated] (SPARK-42571) Provide a mode to replace Py4J for local communication

[ https://issues.apache.org/jira/browse/SPARK-42571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42571: - Affects Version/s: 3.5.0 (was: 3.4.0) > Provide a mode to replace Py4

[jira] [Updated] (SPARK-42729) Update Submitting Applications page for Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-42729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42729: - Affects Version/s: 3.5.0 (was: 3.4.0) > Update Submitting Application

[jira] [Updated] (SPARK-42798) Upgrade protobuf-java to 3.22.2

[ https://issues.apache.org/jira/browse/SPARK-42798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42798: - Description: * [https://github.com/protocolbuffers/protobuf/releases/tag/v22.1] * [https://github.com/p

[jira] [Updated] (SPARK-42798) Upgrade protobuf-java to 3.22.2

[ https://issues.apache.org/jira/browse/SPARK-42798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42798: - Environment: (was: * [https://github.com/protocolbuffers/protobuf/releases/tag/v22.1] * https://git

[jira] [Created] (SPARK-42798) Upgrade protobuf-java to 3.22.2

Yang Jie created SPARK-42798: Summary: Upgrade protobuf-java to 3.22.2 Key: SPARK-42798 URL: https://issues.apache.org/jira/browse/SPARK-42798 Project: Spark Issue Type: Improvement Com

[jira] [Assigned] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42794: Assignee: Huanli Wang > Increase the lockAcquireTimeoutMs for acquiring the RocksDB state

[jira] [Resolved] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42794. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40425 [https://gi

[jira] [Updated] (SPARK-42374) User-facing documentation

[ https://issues.apache.org/jira/browse/SPARK-42374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Folting updated SPARK-42374: -- Summary: User-facing documentation (was: User-facing documentaiton) > User-facing documentati

[jira] [Updated] (SPARK-42796) Support TimestampNTZ in Cached Batch

[ https://issues.apache.org/jira/browse/SPARK-42796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42796: - Fix Version/s: 3.4.1 > Support TimestampNTZ in Cached Batch > --

[jira] [Updated] (SPARK-42422) Upgrade `maven-shade-plugin` to 3.4.1

[ https://issues.apache.org/jira/browse/SPARK-42422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42422: -- Affects Version/s: 3.4.0 (was: 3.5.0) > Upgrade `maven-shade-plugin

[jira] [Commented] (SPARK-42775) approx_percentile produces wrong results for large decimals.

[ https://issues.apache.org/jira/browse/SPARK-42775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700448#comment-17700448 ] Apache Spark commented on SPARK-42775: -- User 'chenhao-db' has created a pull reques

[jira] [Assigned] (SPARK-42775) approx_percentile produces wrong results for large decimals.

[ https://issues.apache.org/jira/browse/SPARK-42775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42775: Assignee: (was: Apache Spark) > approx_percentile produces wrong results for large de

[jira] [Assigned] (SPARK-42775) approx_percentile produces wrong results for large decimals.

[ https://issues.apache.org/jira/browse/SPARK-42775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42775: Assignee: Apache Spark > approx_percentile produces wrong results for large decimals. > -

[jira] [Assigned] (SPARK-42797) Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages

[ https://issues.apache.org/jira/browse/SPARK-42797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42797: Assignee: Apache Spark > Spark Connect - Grammatical improvements to Spark Overview and S

[jira] [Commented] (SPARK-42797) Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages

[ https://issues.apache.org/jira/browse/SPARK-42797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700444#comment-17700444 ] Apache Spark commented on SPARK-42797: -- User 'allanf-db' has created a pull request

[jira] [Assigned] (SPARK-42797) Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages

[ https://issues.apache.org/jira/browse/SPARK-42797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42797: Assignee: (was: Apache Spark) > Spark Connect - Grammatical improvements to Spark Ove

[jira] [Created] (SPARK-42797) Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages

Allan Folting created SPARK-42797: - Summary: Spark Connect - Grammatical improvements to Spark Overview and Spark Connect Overview doc pages Key: SPARK-42797 URL: https://issues.apache.org/jira/browse/SPARK-42797

[jira] [Updated] (SPARK-42496) Introducing Spark Connect on the main page and adding Spark Connect Overview page

[ https://issues.apache.org/jira/browse/SPARK-42496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Folting updated SPARK-42496: -- Summary: Introducing Spark Connect on the main page and adding Spark Connect Overview page (w

[jira] [Updated] (SPARK-42789) Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Description: Benchmark result: {noformat} Running benchmark: Benchmark rewrite GetJsonObjects R

[jira] [Resolved] (SPARK-42793) `connect` module requires `build_profile_flags`

[ https://issues.apache.org/jira/browse/SPARK-42793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42793. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40424 [https://gi

[jira] [Assigned] (SPARK-42793) `connect` module requires `build_profile_flags`

[ https://issues.apache.org/jira/browse/SPARK-42793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42793: Assignee: Dongjoon Hyun > `connect` module requires `build_profile_flags` > -

[jira] [Resolved] (SPARK-42757) Implement textFile for DataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-42757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42757. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40377 [https://gi

[jira] [Assigned] (SPARK-42757) Implement textFile for DataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-42757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42757: Assignee: BingKun Pan > Implement textFile for DataFrameReader >

[jira] [Resolved] (SPARK-42731) Update Spark Configuration

[ https://issues.apache.org/jira/browse/SPARK-42731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42731. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40416 [https://gi

[jira] [Assigned] (SPARK-42731) Update Spark Configuration

[ https://issues.apache.org/jira/browse/SPARK-42731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42731: Assignee: Hyukjin Kwon > Update Spark Configuration > -- > >

[jira] [Resolved] (SPARK-42508) Extract the common .ml classes to `mllib-common`

[ https://issues.apache.org/jira/browse/SPARK-42508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42508. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/40097 > Extract the common .m

[jira] [Updated] (SPARK-42508) Extract the common .ml classes to `mllib-common`

[ https://issues.apache.org/jira/browse/SPARK-42508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42508: - Fix Version/s: 3.5.0 > Extract the common .ml classes to `mllib-common` > --

[jira] [Commented] (SPARK-42792) Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming

[ https://issues.apache.org/jira/browse/SPARK-42792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700413#comment-17700413 ] Apache Spark commented on SPARK-42792: -- User 'anishshri-db' has created a pull requ

[jira] [Assigned] (SPARK-42792) Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming

[ https://issues.apache.org/jira/browse/SPARK-42792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42792: Assignee: (was: Apache Spark) > Add support to track FLUSH_WRITE_BYTES for RocksDB st

[jira] [Assigned] (SPARK-42792) Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming

[ https://issues.apache.org/jira/browse/SPARK-42792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42792: Assignee: Apache Spark > Add support to track FLUSH_WRITE_BYTES for RocksDB state store f

[jira] [Commented] (SPARK-42796) Support TimestampNTZ in Cached Batch

[ https://issues.apache.org/jira/browse/SPARK-42796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700414#comment-17700414 ] Apache Spark commented on SPARK-42796: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-42796) Support TimestampNTZ in Cached Batch

[ https://issues.apache.org/jira/browse/SPARK-42796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42796: Assignee: Apache Spark (was: Gengliang Wang) > Support TimestampNTZ in Cached Batch > --

[jira] [Commented] (SPARK-42796) Support TimestampNTZ in Cached Batch

[ https://issues.apache.org/jira/browse/SPARK-42796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700412#comment-17700412 ] Apache Spark commented on SPARK-42796: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-42796) Support TimestampNTZ in Cached Batch

[ https://issues.apache.org/jira/browse/SPARK-42796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42796: Assignee: Gengliang Wang (was: Apache Spark) > Support TimestampNTZ in Cached Batch > --

[jira] [Created] (SPARK-42796) Support TimestampNTZ in Cached Batch

Gengliang Wang created SPARK-42796: -- Summary: Support TimestampNTZ in Cached Batch Key: SPARK-42796 URL: https://issues.apache.org/jira/browse/SPARK-42796 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huanli Wang updated SPARK-42794: Description: We are seeing query failure which is caused by RocksDB acquisition failure for the r

[jira] [Commented] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700402#comment-17700402 ] Apache Spark commented on SPARK-42794: -- User 'huanliwang-db' has created a pull req

[jira] [Assigned] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42794: Assignee: (was: Apache Spark) > Increase the lockAcquireTimeoutMs for acquiring the R

[jira] [Assigned] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42794: Assignee: Apache Spark > Increase the lockAcquireTimeoutMs for acquiring the RocksDB stat

[jira] [Created] (SPARK-42795) Create analyzer golden file based test suite

Daniel created SPARK-42795: -- Summary: Create analyzer golden file based test suite Key: SPARK-42795 URL: https://issues.apache.org/jira/browse/SPARK-42795 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huanli Wang updated SPARK-42794: Priority: Minor (was: Major) > Increase the lockAcquireTimeoutMs for acquiring the RocksDB state

[jira] [Commented] (SPARK-42793) `connect` module requires `build_profile_flags`

[ https://issues.apache.org/jira/browse/SPARK-42793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700398#comment-17700398 ] Apache Spark commented on SPARK-42793: -- User 'dongjoon-hyun' has created a pull req

[jira] [Updated] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huanli Wang updated SPARK-42794: Description: We are seeing query failure which is caused by RocksDB acquisition failure for the r

[jira] [Updated] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

[ https://issues.apache.org/jira/browse/SPARK-42794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huanli Wang updated SPARK-42794: Description: We are seeing query failure which is caused by RocksDB acquisition failure for the r

[jira] [Assigned] (SPARK-42793) `connect` module requires `build_profile_flags`

[ https://issues.apache.org/jira/browse/SPARK-42793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42793: Assignee: (was: Apache Spark) > `connect` module requires `build_profile_flags` > ---

[jira] [Assigned] (SPARK-42793) `connect` module requires `build_profile_flags`

[ https://issues.apache.org/jira/browse/SPARK-42793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42793: Assignee: Apache Spark > `connect` module requires `build_profile_flags` > --

[jira] [Commented] (SPARK-42793) `connect` module requires `build_profile_flags`

[ https://issues.apache.org/jira/browse/SPARK-42793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700397#comment-17700397 ] Apache Spark commented on SPARK-42793: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-42794) Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming

Huanli Wang created SPARK-42794: --- Summary: Increase the lockAcquireTimeoutMs for acquiring the RocksDB state store in Structure Streaming Key: SPARK-42794 URL: https://issues.apache.org/jira/browse/SPARK-42794

[jira] [Created] (SPARK-42793) `connect` module requires `build_profile_flags`

Dongjoon Hyun created SPARK-42793: - Summary: `connect` module requires `build_profile_flags` Key: SPARK-42793 URL: https://issues.apache.org/jira/browse/SPARK-42793 Project: Spark Issue Type:

[jira] [Created] (SPARK-42792) Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming

Anish Shrigondekar created SPARK-42792: -- Summary: Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming Key: SPARK-42792 URL: https://issues.apache.org/jira/browse/SPARK-42792

[jira] [Commented] (SPARK-42792) Add support to track FLUSH_WRITE_BYTES for RocksDB state store for streaming

[ https://issues.apache.org/jira/browse/SPARK-42792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700387#comment-17700387 ] Anish Shrigondekar commented on SPARK-42792: Will send the PR soon - cc - [~

[jira] [Commented] (SPARK-41775) Implement training functions as input

[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700332#comment-17700332 ] Apache Spark commented on SPARK-41775: -- User 'rithwik-db' has created a pull reques

[jira] [Created] (SPARK-42791) Create golden file test framework for analysis

Daniel created SPARK-42791: -- Summary: Create golden file test framework for analysis Key: SPARK-42791 URL: https://issues.apache.org/jira/browse/SPARK-42791 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-41775) Implement training functions as input

[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700333#comment-17700333 ] Apache Spark commented on SPARK-41775: -- User 'rithwik-db' has created a pull reques

[jira] [Updated] (SPARK-42774) Expose VectorTypes API for DataSourceV2 Batch Scans

[ https://issues.apache.org/jira/browse/SPARK-42774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micah Kornfield updated SPARK-42774: Priority: Minor (was: Major) > Expose VectorTypes API for DataSourceV2 Batch Scans >

[jira] [Commented] (SPARK-42787) in spark-py docker images, arrowkeys do not work in (scala) spark-shell

[ https://issues.apache.org/jira/browse/SPARK-42787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700322#comment-17700322 ] Bjørn Jørgensen commented on SPARK-42787: - Have a look at https://github.com/apa

[jira] [Assigned] (SPARK-42779) Allow V2 writes to indicate advisory partition size

[ https://issues.apache.org/jira/browse/SPARK-42779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42779: Assignee: Apache Spark > Allow V2 writes to indicate advisory partition size > --

  1   2   3   >