[jira] [Created] (SPARK-44185) Inconsistent path qualifying between catalog and data operations

Kent Yao created SPARK-44185: Summary: Inconsistent path qualifying between catalog and data operations Key: SPARK-44185 URL: https://issues.apache.org/jira/browse/SPARK-44185 Project: Spark

[jira] [Commented] (SPARK-44180) DistributionAndOrderingUtils should apply ResolveTimeZone

[ https://issues.apache.org/jira/browse/SPARK-44180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736974#comment-17736974 ] Snoot.io commented on SPARK-44180: -- User 'pan3793' has created a pull request for this issue:

[jira] [Commented] (SPARK-43926) Add array_agg, array_size, cardinality, count_min_sketch,mask,named_struct,json_* to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736973#comment-17736973 ] Snoot.io commented on SPARK-43926: -- User 'ivoson' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44184) Remove a wrong doc about ARROW_PRE_0_15_IPC_FORMAT

[ https://issues.apache.org/jira/browse/SPARK-44184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44184. --- Fix Version/s: 3.3.3 3.5.0 3.4.2 Resolution:

[jira] [Assigned] (SPARK-44184) Remove a wrong doc about ARROW_PRE_0_15_IPC_FORMAT

[ https://issues.apache.org/jira/browse/SPARK-44184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44184: - Assignee: Dongjoon Hyun > Remove a wrong doc about ARROW_PRE_0_15_IPC_FORMAT >

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736945#comment-17736945 ] Ritika Maheshwari commented on SPARK-43514: --- Somehow your code is invoking the hashFunction on

[jira] [Assigned] (SPARK-44064) Maven test `ProductAggSuite` aborted

[ https://issues.apache.org/jira/browse/SPARK-44064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44064: - Assignee: Yang Jie > Maven test `ProductAggSuite` aborted >

[jira] [Resolved] (SPARK-44064) Maven test `ProductAggSuite` aborted

[ https://issues.apache.org/jira/browse/SPARK-44064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44064. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41654

[jira] [Closed] (SPARK-44083) Spark streaming: Add max pending microbatches conf to skip scheduling new mircobatch

[ https://issues.apache.org/jira/browse/SPARK-44083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anil Dasari closed SPARK-44083. --- > Spark streaming: Add max pending microbatches conf to skip scheduling new > mircobatch >

[jira] [Resolved] (SPARK-44083) Spark streaming: Add max pending microbatches conf to skip scheduling new mircobatch

[ https://issues.apache.org/jira/browse/SPARK-44083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anil Dasari resolved SPARK-44083. - Resolution: Won't Do > Spark streaming: Add max pending microbatches conf to skip scheduling

[jira] [Commented] (SPARK-44083) Spark streaming: Add max pending microbatches conf to skip scheduling new mircobatch

[ https://issues.apache.org/jira/browse/SPARK-44083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736937#comment-17736937 ] Anil Dasari commented on SPARK-44083: - [~kabhwan] Thanks for the details. i will close the Jira for

[jira] [Commented] (SPARK-44175) Remove useless lib64 path link in dockerfile

[ https://issues.apache.org/jira/browse/SPARK-44175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736935#comment-17736935 ] Harish Gontu commented on SPARK-44175: -- Can i pick up this task ?   > Remove useless lib64 path

[jira] [Commented] (SPARK-44176) Change apt to apt-get and remove useless cleanup

[ https://issues.apache.org/jira/browse/SPARK-44176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736934#comment-17736934 ] Harish Gontu commented on SPARK-44176: -- Can I pick up this task ? > Change apt to apt-get and

[jira] [Commented] (SPARK-44177) Add 'set -eo pipefail' to entrypoint and quote variables

[ https://issues.apache.org/jira/browse/SPARK-44177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736933#comment-17736933 ] Harish Gontu commented on SPARK-44177: -- Can i pick up this task ? > Add 'set -eo pipefail' to

[jira] [Created] (SPARK-44184) Remove a wrong doc about ARROW_PRE_0_15_IPC_FORMAT

Dongjoon Hyun created SPARK-44184: - Summary: Remove a wrong doc about ARROW_PRE_0_15_IPC_FORMAT Key: SPARK-44184 URL: https://issues.apache.org/jira/browse/SPARK-44184 Project: Spark Issue

[jira] [Created] (SPARK-44183) Increate PyArrow minimum version to 4.0.0

Dongjoon Hyun created SPARK-44183: - Summary: Increate PyArrow minimum version to 4.0.0 Key: SPARK-44183 URL: https://issues.apache.org/jira/browse/SPARK-44183 Project: Spark Issue Type:

[jira] [Created] (SPARK-44182) Use Spark 3.5.0 in Python and Spark Connect docs

Dongjoon Hyun created SPARK-44182: - Summary: Use Spark 3.5.0 in Python and Spark Connect docs Key: SPARK-44182 URL: https://issues.apache.org/jira/browse/SPARK-44182 Project: Spark Issue

[jira] [Updated] (SPARK-43831) Build and Run Spark on Java 21

[ https://issues.apache.org/jira/browse/SPARK-43831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43831: -- Description: - [https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html] ||JDK

[jira] [Resolved] (SPARK-44181) Upgrade Apache Kafka to 3.4.1

[ https://issues.apache.org/jira/browse/SPARK-44181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44181. --- Fix Version/s: 3.5.0 Assignee: Bjørn Jørgensen Resolution: Fixed This is

[jira] [Commented] (SPARK-44181) Upgrade Apache Kafka to 3.4.1

[ https://issues.apache.org/jira/browse/SPARK-44181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736914#comment-17736914 ] Mike K commented on SPARK-44181: User 'bjornjorgensen' has created a pull request for this issue:

[jira] [Created] (SPARK-44181) Upgrade Apache Kafka to 3.4.1

Bjørn Jørgensen created SPARK-44181: --- Summary: Upgrade Apache Kafka to 3.4.1 Key: SPARK-44181 URL: https://issues.apache.org/jira/browse/SPARK-44181 Project: Spark Issue Type: Dependency

[jira] [Commented] (SPARK-44112) Drop Java 8 Support

[ https://issues.apache.org/jira/browse/SPARK-44112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736898#comment-17736898 ] Dongjoon Hyun commented on SPARK-44112: --- That's true. In addition to that, there are more reasons

[jira] [Commented] (SPARK-44134) Can't set resources (GPU/FPGA) to 0 when they are set to positive value in spark-defaults.conf

[ https://issues.apache.org/jira/browse/SPARK-44134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736897#comment-17736897 ] Dongjoon Hyun commented on SPARK-44134: --- Oh, thank you for updating the fix versions, [~tgraves].

[jira] [Resolved] (SPARK-44178) Positional parameters in sql()

[ https://issues.apache.org/jira/browse/SPARK-44178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44178. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41698

[jira] [Resolved] (SPARK-44174) Fix QueryExecutionErrorsSuite

[ https://issues.apache.org/jira/browse/SPARK-44174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44174. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41722

[jira] [Assigned] (SPARK-44174) Fix QueryExecutionErrorsSuite

[ https://issues.apache.org/jira/browse/SPARK-44174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44174: - Assignee: Yang Jie > Fix QueryExecutionErrorsSuite > - > >

[jira] [Created] (SPARK-44180) DistributionAndOrderingUtils should apply ResolveTimeZone

Cheng Pan created SPARK-44180: - Summary: DistributionAndOrderingUtils should apply ResolveTimeZone Key: SPARK-44180 URL: https://issues.apache.org/jira/browse/SPARK-44180 Project: Spark Issue

[jira] [Commented] (SPARK-43773) Implement 'levenshtein(str1, str2[, threshold])' functions in python client

[ https://issues.apache.org/jira/browse/SPARK-43773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736866#comment-17736866 ] Nikita Awasthi commented on SPARK-43773: User 'panbingkun' has created a pull request for this

[jira] [Commented] (SPARK-43769) Implement `levenshtein(str1, str2[, max_distance])` functions

[ https://issues.apache.org/jira/browse/SPARK-43769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736865#comment-17736865 ] Nikita Awasthi commented on SPARK-43769: User 'panbingkun' has created a pull request for this

[jira] [Commented] (SPARK-44171) Assign names to the error class _LEGACY_ERROR_TEMP_[2279-2282] & delete some unused error classes

[ https://issues.apache.org/jira/browse/SPARK-44171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736864#comment-17736864 ] Nikita Awasthi commented on SPARK-44171: User 'panbingkun' has created a pull request for this

[jira] [Commented] (SPARK-43493) Add a max distance argument to the levenshtein() function

[ https://issues.apache.org/jira/browse/SPARK-43493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736863#comment-17736863 ] Nikita Awasthi commented on SPARK-43493: User 'panbingkun' has created a pull request for this

[jira] [Created] (SPARK-44179) When a task failed and the inferred task for that task is still executing, the number of dynamically scheduled executors will be calculated incorrectly

liangyongyuan created SPARK-44179: - Summary: When a task failed and the inferred task for that task is still executing, the number of dynamically scheduled executors will be calculated incorrectly Key: SPARK-44179

[jira] [Updated] (SPARK-44178) Positional parameters in sql()

[ https://issues.apache.org/jira/browse/SPARK-44178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44178: - Description: As a follow-up to the parameterized query we added recently, we’d like to support

[jira] [Updated] (SPARK-44178) Positional parameters in sql()

[ https://issues.apache.org/jira/browse/SPARK-44178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44178: - Reporter: Max Gekk (was: Takuya Ueshin) > Positional parameters in sql() >

[jira] [Created] (SPARK-44178) Positional parameters in sql()

Max Gekk created SPARK-44178: Summary: Positional parameters in sql() Key: SPARK-44178 URL: https://issues.apache.org/jira/browse/SPARK-44178 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44178) Positional parameters in sql()

[ https://issues.apache.org/jira/browse/SPARK-44178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44178: - Fix Version/s: (was: 3.4.0) > Positional parameters in sql() > -- > >

[jira] [Updated] (SPARK-44178) Positional parameters in sql()

[ https://issues.apache.org/jira/browse/SPARK-44178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44178: - Affects Version/s: 3.5.0 (was: 3.4.0) > Positional parameters in sql() >

[jira] [Assigned] (SPARK-44178) Positional parameters in sql()

[ https://issues.apache.org/jira/browse/SPARK-44178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-44178: Assignee: Max Gekk (was: Takuya Ueshin) > Positional parameters in sql() >

[jira] [Created] (SPARK-44177) Add 'set -eo pipefail' to entrypoint and quote variables

Yikun Jiang created SPARK-44177: --- Summary: Add 'set -eo pipefail' to entrypoint and quote variables Key: SPARK-44177 URL: https://issues.apache.org/jira/browse/SPARK-44177 Project: Spark Issue

[jira] [Created] (SPARK-44176) Change apt to apt-get and remove useless cleanup

Yikun Jiang created SPARK-44176: --- Summary: Change apt to apt-get and remove useless cleanup Key: SPARK-44176 URL: https://issues.apache.org/jira/browse/SPARK-44176 Project: Spark Issue Type:

[jira] [Created] (SPARK-44175) Remove useless lib64 path link in dockerfile

Yikun Jiang created SPARK-44175: --- Summary: Remove useless lib64 path link in dockerfile Key: SPARK-44175 URL: https://issues.apache.org/jira/browse/SPARK-44175 Project: Spark Issue Type:

[jira] [Created] (SPARK-44174) Fix QueryExecutionErrorsSuite for Java 21

Yang Jie created SPARK-44174: Summary: Fix QueryExecutionErrorsSuite for Java 21 Key: SPARK-44174 URL: https://issues.apache.org/jira/browse/SPARK-44174 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44174) Fix QueryExecutionErrorsSuite

[ https://issues.apache.org/jira/browse/SPARK-44174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44174: - Summary: Fix QueryExecutionErrorsSuite (was: Fix QueryExecutionErrorsSuite for Java 21) > Fix

[jira] [Resolved] (SPARK-44168) Add Apache Spark 3.4.1 Dockerfiles

[ https://issues.apache.org/jira/browse/SPARK-44168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44168. - Fix Version/s: 3.4.1 Resolution: Fixed > Add Apache Spark 3.4.1 Dockerfiles >

[jira] [Updated] (SPARK-44171) Assign names to the error class _LEGACY_ERROR_TEMP_[2279-2282] & delete some unused error classes

[ https://issues.apache.org/jira/browse/SPARK-44171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44171: Summary: Assign names to the error class _LEGACY_ERROR_TEMP_[2279-2282] & delete some unused

[jira] [Updated] (SPARK-44171) Assign names to the error class _LEGACY_ERROR_TEMP_[2279-2282]

[ https://issues.apache.org/jira/browse/SPARK-44171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44171: Summary: Assign names to the error class _LEGACY_ERROR_TEMP_[2279-2282] (was: Assign names to