[jira] [Created] (SPARK-47474) Revert change of SPARK-47461 and add some comments

2024-03-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-47474: Summary: Revert change of SPARK-47461 and add some comments Key: SPARK-47474 URL: https://issues.apache.org/jira/browse/SPARK-47474 Project: Spark Issue Type:

[jira] [Commented] (SPARK-44101) Support pandas 2

2024-03-19 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828607#comment-17828607 ] Haejoon Lee commented on SPARK-44101: - Got it. Thanks for the notice, [~dongjoon] ! > Support

[jira] [Updated] (SPARK-47473) Correctness issue of converting postgres INFINITIES timestamps

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47473: --- Labels: correctness pull-request-available (was: correctness) > Correctness issue of

[jira] [Updated] (SPARK-47473) Correctness issue of converting postgres INFINITIES timestamps

2024-03-19 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-47473: - Labels: correctness (was: ) > Correctness issue of converting postgres INFINITIES timestamps >

[jira] [Created] (SPARK-47473) Correctness issue of converting postgres INFINITIES timestamps

2024-03-19 Thread Kent Yao (Jira)
Kent Yao created SPARK-47473: Summary: Correctness issue of converting postgres INFINITIES timestamps Key: SPARK-47473 URL: https://issues.apache.org/jira/browse/SPARK-47473 Project: Spark

[jira] [Assigned] (SPARK-47007) Add SortMap function

2024-03-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-47007: Assignee: Stevo Mitric > Add SortMap function > > > Key:

[jira] [Resolved] (SPARK-47007) Add SortMap function

2024-03-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-47007. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45069

[jira] [Updated] (SPARK-47471) Support order-insensitive lateral column alias

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47471: --- Labels: pull-request-available (was: ) > Support order-insensitive lateral column alias >

[jira] [Assigned] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47472: - Assignee: Dongjoon Hyun > Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile` >

[jira] [Updated] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47472: --- Labels: pull-request-available (was: ) > Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile` >

[jira] [Resolved] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47472. --- Fix Version/s: 3.4.3 Resolution: Fixed Issue resolved by pull request 45595

[jira] [Created] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

2024-03-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47472: - Summary: Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile` Key: SPARK-47472 URL: https://issues.apache.org/jira/browse/SPARK-47472 Project: Spark Issue

[jira] [Updated] (SPARK-47471) Support order-insensitive lateral column alias

2024-03-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-47471: Component/s: SQL (was: Block Manager) > Support order-insensitive lateral

[jira] [Resolved] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47470. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45597

[jira] [Created] (SPARK-47471) Support order-insensitive lateral column alias

2024-03-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-47471: --- Summary: Support order-insensitive lateral column alias Key: SPARK-47471 URL: https://issues.apache.org/jira/browse/SPARK-47471 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47470: - Assignee: Dongjoon Hyun > Ignore `IntentionallyFaultyConnectionProvider` error in

[jira] [Updated] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47470: -- Summary: Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite` (was: Make

[jira] [Updated] (SPARK-47470) Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47470: --- Labels: pull-request-available (was: ) > Make `CliSuite` ignores

[jira] [Created] (SPARK-47470) Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error

2024-03-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47470: - Summary: Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error Key: SPARK-47470 URL: https://issues.apache.org/jira/browse/SPARK-47470 Project:

[jira] [Resolved] (SPARK-47330) XML: Add XmlExpressionsSuite

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47330. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45445

[jira] [Assigned] (SPARK-47330) XML: Add XmlExpressionsSuite

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47330: Assignee: Yousof Hosny > XML: Add XmlExpressionsSuite > - >

[jira] [Updated] (SPARK-47469) Add availNow tests for TWS operator

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47469: --- Labels: pull-request-available (was: ) > Add availNow tests for TWS operator >

[jira] [Comment Edited] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828556#comment-17828556 ] Dongjoon Hyun edited comment on SPARK-44101 at 3/20/24 1:00 AM: I added

[jira] [Assigned] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44101: - Assignee: Haejoon Lee > Support pandas 2 > > > Key:

[jira] [Updated] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44101: -- Labels: releasenotes (was: ) > Support pandas 2 > > > Key:

[jira] [Commented] (SPARK-44101) Support pandas 2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828556#comment-17828556 ] Dongjoon Hyun commented on SPARK-44101: --- I added `releasenotes` label to this JIRA. > Support

[jira] [Closed] (SPARK-44149) Support DataFrame Merge API

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44149. - > Support DataFrame Merge API > --- > > Key: SPARK-44149 >

[jira] [Updated] (SPARK-46207) Support MergeInto in DataFrameWriterV2

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46207: -- Parent: SPARK-44111 Issue Type: Sub-task (was: New Feature) > Support MergeInto in

[jira] [Resolved] (SPARK-47468) Exclude `logback` dependency from SBT like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47468. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45594

[jira] [Updated] (SPARK-47432) Add `pyarrow` upper bound requirement, `<13.0.0`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47432: -- Fix Version/s: 3.4.3 > Add `pyarrow` upper bound requirement, `<13.0.0` >

[jira] [Resolved] (SPARK-47449) Refactor and split list/timer unit tests

2024-03-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47449. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45573

[jira] [Updated] (SPARK-46349) Prevent Multiple SortOrders for an Expression

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46349: --- Labels: pull-request-available (was: ) > Prevent Multiple SortOrders for an Expression >

[jira] [Resolved] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47290. --- Resolution: Invalid > Extend CustomTaskMetric to allow metric values from multiple sources

[jira] [Closed] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-47290. - Sure. I closed this. > Extend CustomTaskMetric to allow metric values from multiple sources >

[jira] [Resolved] (SPARK-47464) Update labeler.yml

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47464. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45590

[jira] [Updated] (SPARK-47464) Update labeler.yml for module common/sketch and common/variant

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47464: -- Summary: Update labeler.yml for module common/sketch and common/variant (was: Update

[jira] [Updated] (SPARK-47468) Exclude `logback` dependency from SBT like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47468: -- Summary: Exclude `logback` dependency from SBT like Maven (was: Exclude `logback` from SBT

[jira] [Assigned] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47468: - Assignee: Dongjoon Hyun > Exclude `logback` from SBT dependency like Maven >

[jira] [Updated] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47468: --- Labels: pull-request-available (was: ) > Exclude `logback` from SBT dependency like Maven

[jira] [Created] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

2024-03-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47468: - Summary: Exclude `logback` from SBT dependency like Maven Key: SPARK-47468 URL: https://issues.apache.org/jira/browse/SPARK-47468 Project: Spark Issue

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in

[jira] [Updated] (SPARK-47359) StringTranslate (all collations)

2024-03-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47359: - Description: Enable collation support for the *StringTranslate* built-in string function in

[jira] [Created] (SPARK-47467) Error message regressed when creating hive table with illegal column name

2024-03-19 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-47467: Summary: Error message regressed when creating hive table with illegal column name Key: SPARK-47467 URL: https://issues.apache.org/jira/browse/SPARK-47467 Project:

[jira] [Updated] (SPARK-47449) Refactor and split list/timer unit tests

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47449: --- Labels: pull-request-available (was: ) > Refactor and split list/timer unit tests >

[jira] [Commented] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

2024-03-19 Thread Parth Chandra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828481#comment-17828481 ] Parth Chandra commented on SPARK-47290: --- [~dongjoon] can we close this issue? As discussed in the

[jira] [Resolved] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47461. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45587

[jira] [Assigned] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47461: - Assignee: Yang Jie > Remove unused private function

[jira] [Commented] (SPARK-38958) Override S3 Client in Spark Write/Read calls

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828458#comment-17828458 ] ASF GitHub Bot commented on SPARK-38958: steveloughran commented on PR #6550: URL:

[jira] [Created] (SPARK-47466) Add PySpark DataFrame method to return iterator of PyArrow RecordBatches

2024-03-19 Thread Ian Cook (Jira)
Ian Cook created SPARK-47466: Summary: Add PySpark DataFrame method to return iterator of PyArrow RecordBatches Key: SPARK-47466 URL: https://issues.apache.org/jira/browse/SPARK-47466 Project: Spark

[jira] [Updated] (SPARK-47365) Add _toArrow() DataFrame method to PySpark

2024-03-19 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Component/s: Connect > Add _toArrow() DataFrame method to PySpark >

[jira] [Created] (SPARK-47465) Remove experimental tag from toArrow() PySpark DataFrame method

2024-03-19 Thread Ian Cook (Jira)
Ian Cook created SPARK-47465: Summary: Remove experimental tag from toArrow() PySpark DataFrame method Key: SPARK-47465 URL: https://issues.apache.org/jira/browse/SPARK-47465 Project: Spark

[jira] [Updated] (SPARK-47256) Assign error classes to FILTER expression errors

2024-03-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-47256: - Description: Choose a proper name for the error class *_LEGACY_ERROR_TEMP_102[4-7]* defined in

[jira] [Updated] (SPARK-47365) Add _toArrow() DataFrame method to PySpark

2024-03-19 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Description: Over in the Apache Arrow community, we hear from a lot of users who want to return the

[jira] [Updated] (SPARK-47365) Add _toArrow() DataFrame method to PySpark

2024-03-19 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Summary: Add _toArrow() DataFrame method to PySpark (was: Add toArrow() DataFrame method to PySpark)

[jira] [Resolved] (SPARK-47458) Incorrect to calculate the concurrent task number

2024-03-19 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-47458. --- Fix Version/s: 4.0.0 Assignee: Bobby Wang Resolution: Fixed > Incorrect to

[jira] [Updated] (SPARK-44280) Add convertJavaTimestampToTimestamp in JDBCDialect API

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44280: --- Labels: pull-request-available (was: ) > Add convertJavaTimestampToTimestamp in

[jira] [Resolved] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4+

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47457. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45585

[jira] [Updated] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-03-19 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-47463: -- Issue Type: Bug (was: Test) > An error occurred while pushing down the filter of if expression for

[jira] [Updated] (SPARK-47464) Update labeler.yml

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47464: --- Labels: pull-request-available (was: ) > Update labeler.yml > -- > >

[jira] [Created] (SPARK-47464) Update labeler.yml

2024-03-19 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-47464: --- Summary: Update labeler.yml Key: SPARK-47464 URL: https://issues.apache.org/jira/browse/SPARK-47464 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47458) Incorrect to calculate the concurrent task number

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47458: --- Labels: pull-request-available (was: ) > Incorrect to calculate the concurrent task number

[jira] [Updated] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-03-19 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-47463: -- Description: Reproduce: {code:java} create table t1(c1 int) using iceberg; select * from (select

[jira] [Updated] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-03-19 Thread Zhen Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-47463: -- Description: Reproduce: {code:java} create table t1(c1 int) using iceberg;select * from (select if(c1

[jira] [Updated] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47463: --- Labels: pull-request-available (was: ) > An error occurred while pushing down the filter

[jira] [Created] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-03-19 Thread Zhen Wang (Jira)
Zhen Wang created SPARK-47463: - Summary: An error occurred while pushing down the filter of if expression for iceberg datasource. Key: SPARK-47463 URL: https://issues.apache.org/jira/browse/SPARK-47463

[jira] [Updated] (SPARK-47462) Align the mapping ways of other unsigned numeric types with TINYINT UNSGINED in MySQLDialect

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47462: --- Labels: pull-request-available (was: ) > Align the mapping ways of other unsigned numeric

[jira] [Created] (SPARK-47462) Align the mapping ways of other unsigned numeric types with TINYINT UNSGINED in MySQLDialect

2024-03-19 Thread Kent Yao (Jira)
Kent Yao created SPARK-47462: Summary: Align the mapping ways of other unsigned numeric types with TINYINT UNSGINED in MySQLDialect Key: SPARK-47462 URL: https://issues.apache.org/jira/browse/SPARK-47462

[jira] [Updated] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47461: --- Labels: pull-request-available (was: ) > Remove unused private function

[jira] [Updated] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

2024-03-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-47461: - Summary: Remove unused private function `totalRunningTasksPerResourceProfile` from

[jira] [Created] (SPARK-47461) Remove the unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

2024-03-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-47461: Summary: Remove the unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager` Key: SPARK-47461 URL: https://issues.apache.org/jira/browse/SPARK-47461

[jira] [Assigned] (SPARK-47440) SQLServer does not support LIKE operator in binary comparison

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47440: -- Assignee: (was: Apache Spark) > SQLServer does not support LIKE operator in

[jira] [Assigned] (SPARK-47440) SQLServer does not support LIKE operator in binary comparison

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47440: -- Assignee: Apache Spark > SQLServer does not support LIKE operator in binary

[jira] [Updated] (SPARK-47460) Fix daily scheduled jobs for branch-3.4 & branch-3.5

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47460: --- Labels: pull-request-available (was: ) > Fix daily scheduled jobs for branch-3.4 &

[jira] [Created] (SPARK-47460) Fix daily scheduled jobs for branch-3.4 & branch-3.5

2024-03-19 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-47460: --- Summary: Fix daily scheduled jobs for branch-3.4 & branch-3.5 Key: SPARK-47460 URL: https://issues.apache.org/jira/browse/SPARK-47460 Project: Spark Issue

[jira] [Commented] (SPARK-47440) SQLServer does not support LIKE operator in binary comparison

2024-03-19 Thread Stefan Bukorovic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828215#comment-17828215 ] Stefan Bukorovic commented on SPARK-47440: -- I want to work on this issue.  Opened a PR for

[jira] [Resolved] (SPARK-47454) Split `pyspark.sql.tests.test_dataframe`

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47454. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45580

[jira] [Updated] (SPARK-47458) Incorrect to calculate the concurrent task number

2024-03-19 Thread Bobby Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bobby Wang updated SPARK-47458: --- Summary: Incorrect to calculate the concurrent task number (was: Wrong to calculate the concurrent

[jira] [Created] (SPARK-47459) Cancel running stage if the result is empty relation

2024-03-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-47459: --- Summary: Cancel running stage if the result is empty relation Key: SPARK-47459 URL: https://issues.apache.org/jira/browse/SPARK-47459 Project: Spark Issue

[jira] [Updated] (SPARK-47459) Cancel running stage if the result is empty relation

2024-03-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-47459: Attachment: task stack trace.png > Cancel running stage if the result is empty relation >

[jira] [Created] (SPARK-47458) Wrong to calculate the concurrent task number

2024-03-19 Thread Bobby Wang (Jira)
Bobby Wang created SPARK-47458: -- Summary: Wrong to calculate the concurrent task number Key: SPARK-47458 URL: https://issues.apache.org/jira/browse/SPARK-47458 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4+

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47457: -- Summary: Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4+ (was:

[jira] [Assigned] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47457: - Assignee: Dongjoon Hyun > Fix `IsolatedClientLoader.supportsHadoopShadedClient` to

[jira] [Updated] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47457: -- Component/s: SQL (was: Spark Core) > Fix

[jira] [Updated] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4

2024-03-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47457: --- Labels: pull-request-available (was: ) > Fix

[jira] [Created] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4

2024-03-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47457: - Summary: Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4 Key: SPARK-47457 URL: https://issues.apache.org/jira/browse/SPARK-47457

[jira] [Resolved] (SPARK-47452) Use `Ubuntu 22.04` in `dev/infra/Dockerfile`

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47452. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45576

[jira] [Updated] (SPARK-47456) Support ORC Brotli codec

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47456: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Support ORC Brotli

[jira] [Assigned] (SPARK-47456) Support ORC Brotli codec

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47456: - Assignee: dzcxzl > Support ORC Brotli codec > > >

[jira] [Resolved] (SPARK-47456) Support ORC Brotli codec

2024-03-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47456. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45584