[jira] [Updated] (SPARK-45306) Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans

[ https://issues.apache.org/jira/browse/SPARK-45306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45306: --- Labels: pull-request-available (was: ) > Make `InMemoryColumnarBenchmark` use AQE-aware

[jira] [Updated] (SPARK-45306) Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans

[ https://issues.apache.org/jira/browse/SPARK-45306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45306: - Affects Version/s: 3.5.1 > Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans >

[jira] [Created] (SPARK-45306) Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans

Yang Jie created SPARK-45306: Summary: Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans Key: SPARK-45306 URL: https://issues.apache.org/jira/browse/SPARK-45306 Project: Spark

[jira] [Updated] (SPARK-45305) Remove JDK 8 workaround added SPARK-32999

[ https://issues.apache.org/jira/browse/SPARK-45305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45305: --- Labels: pull-request-available (was: ) > Remove JDK 8 workaround added SPARK-32999 >

[jira] [Created] (SPARK-45305) Remove JDK 8 workaround added SPARK-32999

Hyukjin Kwon created SPARK-45305: Summary: Remove JDK 8 workaround added SPARK-32999 Key: SPARK-45305 URL: https://issues.apache.org/jira/browse/SPARK-45305 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-45304) Remove test classloader workaround for SBT build

[ https://issues.apache.org/jira/browse/SPARK-45304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45304: --- Labels: pull-request-available (was: ) > Remove test classloader workaround for SBT build

[jira] [Updated] (SPARK-37508) Add CONTAINS() function

[ https://issues.apache.org/jira/browse/SPARK-37508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-37508: --- Labels: pull-request-available (was: ) > Add CONTAINS() function > ---

[jira] [Created] (SPARK-45304) Remove test classloader workaround for SBT build

Hyukjin Kwon created SPARK-45304: Summary: Remove test classloader workaround for SBT build Key: SPARK-45304 URL: https://issues.apache.org/jira/browse/SPARK-45304 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45303) Remove JDK 8/11 workaround in KryoSerializerBenchmark

[ https://issues.apache.org/jira/browse/SPARK-45303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45303: --- Labels: pull-request-available (was: ) > Remove JDK 8/11 workaround in

[jira] [Created] (SPARK-45303) Remove JDK 8/11 workaround in KryoSerializerBenchmark

Hyukjin Kwon created SPARK-45303: Summary: Remove JDK 8/11 workaround in KryoSerializerBenchmark Key: SPARK-45303 URL: https://issues.apache.org/jira/browse/SPARK-45303 Project: Spark Issue

[jira] [Updated] (SPARK-45302) Remove PID communication between Python workers when no demon is used

[ https://issues.apache.org/jira/browse/SPARK-45302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45302: --- Labels: pull-request-available (was: ) > Remove PID communication between Python workers

[jira] [Created] (SPARK-45302) Remove PID communication between Python workers when no demon is used

Hyukjin Kwon created SPARK-45302: Summary: Remove PID communication between Python workers when no demon is used Key: SPARK-45302 URL: https://issues.apache.org/jira/browse/SPARK-45302 Project: Spark

[jira] [Updated] (SPARK-28932) Maven install fails on JDK11

[ https://issues.apache.org/jira/browse/SPARK-28932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-28932: --- Labels: pull-request-available (was: ) > Maven install fails on JDK11 >

[jira] [Created] (SPARK-45301) Remove org.scala-lang scala-library added for JDK 11 workaround

Hyukjin Kwon created SPARK-45301: Summary: Remove org.scala-lang scala-library added for JDK 11 workaround Key: SPARK-45301 URL: https://issues.apache.org/jira/browse/SPARK-45301 Project: Spark

[jira] [Updated] (SPARK-44539) Upgrade RoaringBitmap to 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-44539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44539: --- Labels: pull-request-available (was: ) > Upgrade RoaringBitmap to 1.0.0 >

[jira] [Updated] (SPARK-45300) Remove JDK 8 workaround in TimestampFormatterSuite

[ https://issues.apache.org/jira/browse/SPARK-45300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45300: --- Labels: pull-request-available (was: ) > Remove JDK 8 workaround in

[jira] [Created] (SPARK-45300) Remove JDK 8 workaround in TimestampFormatterSuite

Hyukjin Kwon created SPARK-45300: Summary: Remove JDK 8 workaround in TimestampFormatterSuite Key: SPARK-45300 URL: https://issues.apache.org/jira/browse/SPARK-45300 Project: Spark Issue

[jira] [Updated] (SPARK-45300) Remove JDK 8 workaround in TimestampFormatterSuite

[ https://issues.apache.org/jira/browse/SPARK-45300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45300: - Priority: Minor (was: Major) > Remove JDK 8 workaround in TimestampFormatterSuite >

[jira] [Updated] (SPARK-45299) Remove JDK 8 workaround in UtilsSuite

[ https://issues.apache.org/jira/browse/SPARK-45299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45299: --- Labels: pull-request-available (was: ) > Remove JDK 8 workaround in UtilsSuite >

[jira] [Updated] (SPARK-45299) Remove JDK 8 workaround in UtilsSuite

[ https://issues.apache.org/jira/browse/SPARK-45299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45299: - Description: (was: In "Kill process, we don't need the JDK 8 workaround anymore) > Remove

[jira] [Created] (SPARK-45299) Remove JDK 8 workaround in UtilsSuite

Hyukjin Kwon created SPARK-45299: Summary: Remove JDK 8 workaround in UtilsSuite Key: SPARK-45299 URL: https://issues.apache.org/jira/browse/SPARK-45299 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-45298) Remove the workaround for JDK-8228469 in SPARK-31959 test

[ https://issues.apache.org/jira/browse/SPARK-45298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45298: --- Labels: pull-request-available (was: ) > Remove the workaround for JDK-8228469 in

[jira] [Created] (SPARK-45298) Remove the workaround for JDK-8228469 in SPARK-31959 test

Hyukjin Kwon created SPARK-45298: Summary: Remove the workaround for JDK-8228469 in SPARK-31959 test Key: SPARK-45298 URL: https://issues.apache.org/jira/browse/SPARK-45298 Project: Spark

[jira] [Updated] (SPARK-45298) Remove the workaround for JDK-8228469 in SPARK-31959 test

[ https://issues.apache.org/jira/browse/SPARK-45298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45298: - Issue Type: Test (was: Improvement) > Remove the workaround for JDK-8228469 in SPARK-31959

[jira] [Updated] (SPARK-45297) Remove workaround for dateformatter added in SPARK-31827

[ https://issues.apache.org/jira/browse/SPARK-45297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45297: --- Labels: pull-request-available (was: ) > Remove workaround for dateformatter added in

[jira] [Created] (SPARK-45297) Remove workaround for dateformatter added in SPARK-31827

Hyukjin Kwon created SPARK-45297: Summary: Remove workaround for dateformatter added in SPARK-31827 Key: SPARK-45297 URL: https://issues.apache.org/jira/browse/SPARK-45297 Project: Spark

[jira] [Updated] (SPARK-45296) Comment out unused JDK 11 related in dev/run-tests.py

[ https://issues.apache.org/jira/browse/SPARK-45296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45296: --- Labels: pull-request-available (was: ) > Comment out unused JDK 11 related in

[jira] [Created] (SPARK-45296) Comment out unused JDK 11 related in dev/run-tests.py

Hyukjin Kwon created SPARK-45296: Summary: Comment out unused JDK 11 related in dev/run-tests.py Key: SPARK-45296 URL: https://issues.apache.org/jira/browse/SPARK-45296 Project: Spark Issue

[jira] [Updated] (SPARK-45295) Remove Utils.isMemberClass workaround for JDK 8

[ https://issues.apache.org/jira/browse/SPARK-45295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45295: --- Labels: pull-request-available (was: ) > Remove Utils.isMemberClass workaround for JDK 8 >

[jira] [Created] (SPARK-45295) Remove Utils.isMemberClass workaround for JDK 8

Hyukjin Kwon created SPARK-45295: Summary: Remove Utils.isMemberClass workaround for JDK 8 Key: SPARK-45295 URL: https://issues.apache.org/jira/browse/SPARK-45295 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45294) Use JDK 17 in Binder integration for PySpark live notebooks

[ https://issues.apache.org/jira/browse/SPARK-45294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45294. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43077

[jira] [Assigned] (SPARK-45294) Use JDK 17 in Binder integration for PySpark live notebooks

[ https://issues.apache.org/jira/browse/SPARK-45294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45294: Assignee: Hyukjin Kwon > Use JDK 17 in Binder integration for PySpark live notebooks >

[jira] [Updated] (SPARK-45294) Use JDK 17 in Binder integration for PySpark live notebooks

[ https://issues.apache.org/jira/browse/SPARK-45294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45294: --- Labels: pull-request-available (was: ) > Use JDK 17 in Binder integration for PySpark live

[jira] [Updated] (SPARK-44539) Upgrade RoaringBitmap to 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-44539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44539: Summary: Upgrade RoaringBitmap to 1.0.0 (was: Upgrade RoaringBitmap to 0.9.46) > Upgrade

[jira] [Created] (SPARK-45294) Use JDK 17 in Binder integration for PySpark live notebooks

Hyukjin Kwon created SPARK-45294: Summary: Use JDK 17 in Binder integration for PySpark live notebooks Key: SPARK-45294 URL: https://issues.apache.org/jira/browse/SPARK-45294 Project: Spark

[jira] [Updated] (SPARK-45276) Replace Java 8 and Java 11 installed in the Dockerfile with Java

[ https://issues.apache.org/jira/browse/SPARK-45276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45276: --- Labels: pull-request-available (was: ) > Replace Java 8 and Java 11 installed in the

[jira] [Resolved] (SPARK-45293) Install Java 17 for docker

[ https://issues.apache.org/jira/browse/SPARK-45293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan resolved SPARK-45293. - Resolution: Duplicate https://issues.apache.org/jira/browse/SPARK-45276 > Install Java 17 for

[jira] [Updated] (SPARK-45293) Install Java 17 for docker

[ https://issues.apache.org/jira/browse/SPARK-45293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45293: --- Labels: pull-request-available (was: ) > Install Java 17 for docker >

[jira] [Created] (SPARK-45293) Install Java 17 for docker

BingKun Pan created SPARK-45293: --- Summary: Install Java 17 for docker Key: SPARK-45293 URL: https://issues.apache.org/jira/browse/SPARK-45293 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-45207) Implement Error Enrichment for Scala Client

[ https://issues.apache.org/jira/browse/SPARK-45207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45207: Assignee: Yihong He > Implement Error Enrichment for Scala Client >

[jira] [Assigned] (SPARK-45240) Implement Error Enrichment for Python Client

[ https://issues.apache.org/jira/browse/SPARK-45240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45240: Assignee: Yihong He > Implement Error Enrichment for Python Client >

[jira] [Resolved] (SPARK-45207) Implement Error Enrichment for Scala Client

[ https://issues.apache.org/jira/browse/SPARK-45207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45207. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42987

[jira] [Resolved] (SPARK-45240) Implement Error Enrichment for Python Client

[ https://issues.apache.org/jira/browse/SPARK-45240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45240. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43034

[jira] [Updated] (SPARK-43259) Assign a name to the error class _LEGACY_ERROR_TEMP_2024

[ https://issues.apache.org/jira/browse/SPARK-43259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43259: --- Labels: pull-request-available starter (was: starter) > Assign a name to the error class

[jira] [Updated] (SPARK-43900) Support optimize skewed partitions even if introduce extra shuffle

[ https://issues.apache.org/jira/browse/SPARK-43900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43900: --- Labels: pull-request-available (was: ) > Support optimize skewed partitions even if

[jira] [Resolved] (SPARK-45279) Attach plan_id for all logical plan

[ https://issues.apache.org/jira/browse/SPARK-45279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45279. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43055

[jira] [Updated] (SPARK-42947) Spark Thriftserver LDAP should not use DN pattern if user contains domain

[ https://issues.apache.org/jira/browse/SPARK-42947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42947: --- Labels: pull-request-available (was: ) > Spark Thriftserver LDAP should not use DN pattern

[jira] [Assigned] (SPARK-45279) Attach plan_id for all logical plan

[ https://issues.apache.org/jira/browse/SPARK-45279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45279: - Assignee: Ruifeng Zheng > Attach plan_id for all logical plan >

[jira] [Updated] (SPARK-44077) Session Configs were not getting honored in RDDs

[ https://issues.apache.org/jira/browse/SPARK-44077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44077: --- Labels: pull-request-available (was: ) > Session Configs were not getting honored in RDDs

[jira] [Commented] (SPARK-31177) DataFrameReader.csv incorrectly reads gzip encoded CSV from S3 when it has non-".gz" extension

[ https://issues.apache.org/jira/browse/SPARK-31177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768464#comment-17768464 ] Mark Waddle commented on SPARK-31177: - [~Minskya]the resolution is “incomplete”, so I don’t think

[jira] [Resolved] (SPARK-45286) Add back Matomo analytics to release docs

[ https://issues.apache.org/jira/browse/SPARK-45286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-45286. -- Fix Version/s: 3.3.4 3.5.1 4.0.0

[jira] [Created] (SPARK-45292) Remove Guava from shared classes from IsolatedClientLoader

Cheng Pan created SPARK-45292: - Summary: Remove Guava from shared classes from IsolatedClientLoader Key: SPARK-45292 URL: https://issues.apache.org/jira/browse/SPARK-45292 Project: Spark Issue

[jira] [Updated] (SPARK-44366) Migrate antlr4 from 4.9 to 4.10+

[ https://issues.apache.org/jira/browse/SPARK-44366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44366: --- Labels: pull-request-available (was: ) > Migrate antlr4 from 4.9 to 4.10+ >

[jira] [Updated] (SPARK-44170) Migrating Junit4 to Junit5

[ https://issues.apache.org/jira/browse/SPARK-44170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44170: --- Labels: pull-request-available (was: ) > Migrating Junit4 to Junit5 >

[jira] [Updated] (SPARK-45291) Use unknown query execution id instead of no such app when id is invalid

[ https://issues.apache.org/jira/browse/SPARK-45291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45291: --- Labels: pull-request-available (was: ) > Use unknown query execution id instead of no such

[jira] [Created] (SPARK-45291) Use unknown query execution id instead of no such app when id is invalid

Kent Yao created SPARK-45291: Summary: Use unknown query execution id instead of no such app when id is invalid Key: SPARK-45291 URL: https://issues.apache.org/jira/browse/SPARK-45291 Project: Spark

[jira] [Commented] (SPARK-45282) Join loses records for cached datasets

[ https://issues.apache.org/jira/browse/SPARK-45282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768399#comment-17768399 ] Yuming Wang commented on SPARK-45282: - cc [~ulysses] [~cloud_fan] > Join loses records for cached

[jira] [Commented] (SPARK-44366) Migrate antlr4 from 4.9 to 4.10+

[ https://issues.apache.org/jira/browse/SPARK-44366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768393#comment-17768393 ] Yang Jie commented on SPARK-44366: -- +1 > Migrate antlr4 from 4.9 to 4.10+ >

[jira] [Updated] (SPARK-45289) ClassCastException when reading Delta table on AWS S3

[ https://issues.apache.org/jira/browse/SPARK-45289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanawat Panmongkol updated SPARK-45289: --- Description: When attempting to read a Delta table from S3 using version 3.5.0, a