[jira] [Commented] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844388#comment-17844388 ] Dongjoon Hyun commented on SPARK-48037: --- Thank you, [~dzcxzl]. I raised the priority to `Blocker

[jira] [Assigned] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48037: - Assignee: dzcxzl > SortShuffleWriter lacks shuffle write related metrics result

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Affects Version/s: 3.4.3 3.5.1 4.0.0

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Target Version/s: 4.0.0, 3.5.2, 3.4.4 > SortShuffleWriter lacks shuffle write related metr

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Labels: correctness (was: pull-request-available) > SortShuffleWriter lacks shuffle wr

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Priority: Blocker (was: Major) > SortShuffleWriter lacks shuffle write related metr

[jira] [Resolved] (SPARK-41547) Reenable ANSI mode in pyspark.sql.tests.connect.test_connect_functions

[ https://issues.apache.org/jira/browse/SPARK-41547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41547. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46432 [https

[jira] [Resolved] (SPARK-48169) Use lazy BadRecordException cause for StaxXmlParser and JacksonParser

[ https://issues.apache.org/jira/browse/SPARK-48169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48169. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46438 [https

[jira] [Resolved] (SPARK-48165) Update `ap-loader` to 3.0-9

[ https://issues.apache.org/jira/browse/SPARK-48165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48165. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46427 [https

[jira] [Resolved] (SPARK-48173) CheckAnalsis should see the entire query plan

[ https://issues.apache.org/jira/browse/SPARK-48173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48173. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46439 [https

[jira] [Assigned] (SPARK-48173) CheckAnalsis should see the entire query plan

[ https://issues.apache.org/jira/browse/SPARK-48173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48173: - Assignee: Wenchen Fan > CheckAnalsis should see the entire query p

[jira] [Assigned] (SPARK-48171) Clean up the use of deprecated APIs related to `o.rocksdb.Logger`

[ https://issues.apache.org/jira/browse/SPARK-48171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48171: - Assignee: Yang Jie > Clean up the use of deprecated APIs related to `o.rocksdb.Log

[jira] [Resolved] (SPARK-48171) Clean up the use of deprecated APIs related to `o.rocksdb.Logger`

[ https://issues.apache.org/jira/browse/SPARK-48171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48171. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46436 [https

[jira] [Resolved] (SPARK-48163) Disable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command`

[ https://issues.apache.org/jira/browse/SPARK-48163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48163. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46425 [https

[jira] [Updated] (SPARK-48139) Re-enable `SparkSessionE2ESuite.interrupt tag`

[ https://issues.apache.org/jira/browse/SPARK-48139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48139: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Re-ena

[jira] [Updated] (SPARK-48164) Re-enable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command`

[ https://issues.apache.org/jira/browse/SPARK-48164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48164: -- Component/s: Tests > Re-enable `SparkConnectServiceSuite.SPARK-43923: commands send eve

[jira] [Updated] (SPARK-48164) Re-enable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command`

[ https://issues.apache.org/jira/browse/SPARK-48164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48164: -- Target Version/s: 4.0.0 > Re-enable `SparkConnectServiceSuite.SPARK-43923: commands s

[jira] [Assigned] (SPARK-48163) Disable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command`

[ https://issues.apache.org/jira/browse/SPARK-48163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48163: - Assignee: Dongjoon Hyun > Disable `SparkConnectServiceSuite.SPARK-43923: commands s

[jira] [Created] (SPARK-48164) Re-enable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command`

Dongjoon Hyun created SPARK-48164: - Summary: Re-enable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command` Key: SPARK-48164 URL: https://issues.apache.org/jira/browse/SPARK-48164

[jira] [Created] (SPARK-48163) Disable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command`

Dongjoon Hyun created SPARK-48163: - Summary: Disable `SparkConnectServiceSuite.SPARK-43923: commands send events - get_resources_command` Key: SPARK-48163 URL: https://issues.apache.org/jira/browse/SPARK-48163

[jira] [Resolved] (SPARK-48141) Update the Oracle docker image version used for test and integration to use Oracle Database 23ai Free

[ https://issues.apache.org/jira/browse/SPARK-48141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48141. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46399 [https

[jira] [Assigned] (SPARK-48141) Update the Oracle docker image version used for test and integration to use Oracle Database 23ai Free

[ https://issues.apache.org/jira/browse/SPARK-48141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48141: - Assignee: Luca Canali > Update the Oracle docker image version used for t

[jira] [Resolved] (SPARK-48150) Fix nullability of try_parse_json

[ https://issues.apache.org/jira/browse/SPARK-48150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48150. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46409 [https

[jira] [Resolved] (SPARK-48153) Run `build` job of `build_and_test.yml` only if needed

[ https://issues.apache.org/jira/browse/SPARK-48153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48153. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46412 [https

[jira] [Updated] (SPARK-48153) Run `build` job of `build_and_test.yml` only if needed

[ https://issues.apache.org/jira/browse/SPARK-48153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48153: -- Summary: Run `build` job of `build_and_test.yml` only if needed (was: Run `build` job only

[jira] [Assigned] (SPARK-48151) `build_and_test.yml` should use `Volcano` 1.7.0 for `branch-3.4/3.5`

[ https://issues.apache.org/jira/browse/SPARK-48151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48151: - Assignee: Dongjoon Hyun > `build_and_test.yml` should use `Volcano` 1.7.0 for `bra

[jira] [Resolved] (SPARK-48151) `build_and_test.yml` should use `Volcano` 1.7.0 for `branch-3.4/3.5`

[ https://issues.apache.org/jira/browse/SPARK-48151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48151. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46410 [https

[jira] [Created] (SPARK-48151) `build_and_test.yml` should use `Volcano` 1.7.0 for `branch-3.4/3.5`

Dongjoon Hyun created SPARK-48151: - Summary: `build_and_test.yml` should use `Volcano` 1.7.0 for `branch-3.4/3.5` Key: SPARK-48151 URL: https://issues.apache.org/jira/browse/SPARK-48151 Project

[jira] [Resolved] (SPARK-48149) Serialize `build_python.yml` to run a single Python version per cron schedule

[ https://issues.apache.org/jira/browse/SPARK-48149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48149. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46407 [https

[jira] [Assigned] (SPARK-48149) Serialize `build_python.yml` to run a single Python version per cron schedule

[ https://issues.apache.org/jira/browse/SPARK-48149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48149: - Assignee: Dongjoon Hyun > Serialize `build_python.yml` to run a single Python vers

[jira] [Created] (SPARK-48149) Serialize `build_python.yml` to run a single Python version per cron schedule

Dongjoon Hyun created SPARK-48149: - Summary: Serialize `build_python.yml` to run a single Python version per cron schedule Key: SPARK-48149 URL: https://issues.apache.org/jira/browse/SPARK-48149

[jira] [Resolved] (SPARK-48145) Remove logDebug and logTrace with MDC in java structured logging framework

[ https://issues.apache.org/jira/browse/SPARK-48145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48145. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46405 [https

[jira] [Updated] (HIVE-25468) Create/Drop functions should be authorized in HMS

[ https://issues.apache.org/jira/browse/HIVE-25468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated HIVE-25468: - Fix Version/s: 4.0.0 3.1.3 > Create/Drop functions should be authori

Re: [VOTE] Apache Hive 2.3.10 Release Candidate 1

+1 (non-binding) Thank you so much all! Dongjoon. On 2024/05/06 13:57:09 Cheng Pan wrote: > +1 (non-binding) > > Pass integration test with Apache Spark[1] and Apache Kyuubi[2]. > > [1] https://github.com/apache/spark/pull/45372 > [2] https://github.com/apache/kyuubi/pull/6328 > > Thanks, >

Re: Orc 1.9.3 - changes around precision of decimal numbers?

Your PR seems to have irrelevant change. For example, upgrading `net.sf.opencsv: opencsv:2.3` to `com.opencsv:opencsv:5.9`. Could you remove `opencsv` change first to narrow down? Dongjoon. On 2024/05/06 14:55:20 Dmitriy Fingerman wrote: > Hello ORC Devs, > > I am working on upgrading the

Re: ORC 2.0.1 Release

Thank you so much, William. It would be helpful for preparing Apache Spark 4.0.0 release too. Dongjoon. On Sun, May 5, 2024 at 9:19 PM Gang Wu wrote: > Thanks William! > > There are some fixes on the C++ side waiting for the release. > > Best, > Gang > > On Mon, May 6, 2024 at 12:12 PM

[jira] [Updated] (SPARK-48138) Disable a flaky `SparkSessionE2ESuite.interrupt tag` test

[ https://issues.apache.org/jira/browse/SPARK-48138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48138: -- Description: - https://github.com/apache/spark/actions/runs/8962353911/job/24611130573

[jira] [Updated] (SPARK-48139) Re-enable `SparkSessionE2ESuite.interrupt tag`

[ https://issues.apache.org/jira/browse/SPARK-48139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48139: -- Description: (was: - https://github.com/apache/spark/actions/runs/8962353911/job

[jira] [Updated] (SPARK-48139) Re-enable `SparkSessionE2ESuite.interrupt tag`

[ https://issues.apache.org/jira/browse/SPARK-48139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48139: -- Description: - https://github.com/apache/spark/actions/runs/8962353911/job/24611130573

[jira] [Created] (SPARK-48138) Disable a flaky `SparkSessionE2ESuite.interrupt tag` test

Dongjoon Hyun created SPARK-48138: - Summary: Disable a flaky `SparkSessionE2ESuite.interrupt tag` test Key: SPARK-48138 URL: https://issues.apache.org/jira/browse/SPARK-48138 Project: Spark

[jira] [Resolved] (SPARK-48136) Always upload Spark Connect log files

[ https://issues.apache.org/jira/browse/SPARK-48136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48136. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46393 [https

Re: ASF board report draft for May

+1 for Holden's comment. Yes, it would be great to mention `it` as "soon". (If Wenchen release it on Monday, we can simply mention the release) In addition, Apache Spark PMC received an official notice from ASF Infra team. https://lists.apache.org/thread/rgy1cg17tkd3yox7qfq87ht12sqclkbg >

[jira] [Updated] (SPARK-48135) Run `buf` and `ui` only in PR builders and Java 21 Daily CI

[ https://issues.apache.org/jira/browse/SPARK-48135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48135: -- Summary: Run `buf` and `ui` only in PR builders and Java 21 Daily CI (was: Run `but` and `ui

[jira] [Updated] (SPARK-48132) Run `k8s-integration-tests` only in PR builder and Daily CIs

[ https://issues.apache.org/jira/browse/SPARK-48132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48132: -- Summary: Run `k8s-integration-tests` only in PR builder and Daily CIs (was: Run `k8s

[jira] [Updated] (SPARK-48116) Run `pyspark-pandas*` only in PR builder and Daily Python CIs

[ https://issues.apache.org/jira/browse/SPARK-48116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48116: -- Summary: Run `pyspark-pandas*` only in PR builder and Daily Python CIs (was: Run `pyspark

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Description: h2. ASF INFRA POLICY - https://infra.apache.org/github-actions-policy.html h2

[jira] [Resolved] (SPARK-48131) Unify MDC key `mdc.taskName` and `task_name`

[ https://issues.apache.org/jira/browse/SPARK-48131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48131. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46386 [https

[jira] [Updated] (SPARK-48128) BitwiseCount / bit_count generated code for boolean inputs fails to compile

[ https://issues.apache.org/jira/browse/SPARK-48128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48128: -- Affects Version/s: 3.4.3 3.3.4 3.5.1

[jira] [Updated] (SPARK-48128) BitwiseCount / bit_count generated code for boolean inputs fails to compile

[ https://issues.apache.org/jira/browse/SPARK-48128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48128: -- Issue Type: Bug (was: Improvement) > BitwiseCount / bit_count generated code for bool

[jira] [Resolved] (SPARK-48129) Provide a constant table schema in PySpark for querying structured logs

[ https://issues.apache.org/jira/browse/SPARK-48129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48129. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46384 [https

[jira] [Resolved] (SPARK-48128) BitwiseCount / bit_count generated code for boolean inputs fails to compile

[ https://issues.apache.org/jira/browse/SPARK-48128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48128. --- Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution

[jira] [Resolved] (SPARK-48124) Disable structured logging for Interpreter by default

[ https://issues.apache.org/jira/browse/SPARK-48124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48124. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46383 [https

[jira] [Resolved] (SPARK-48127) Fix `dev/scalastyle` to check `hadoop-cloud` and `jvm-profiler` modules

[ https://issues.apache.org/jira/browse/SPARK-48127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48127. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46376 [https

[jira] [Resolved] (SPARK-48116) Run `pyspark-pandas*` in PR builder and Daily Python CIs

[ https://issues.apache.org/jira/browse/SPARK-48116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48116. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46367 [https

[jira] [Updated] (SPARK-48127) Fix `dev/scalastyle` to check `hadoop-cloud` and `jvm-profiler` modules

[ https://issues.apache.org/jira/browse/SPARK-48127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48127: -- Summary: Fix `dev/scalastyle` to check `hadoop-cloud` and `jvm-profiler` modules (was: Fix

[jira] [Created] (SPARK-48127) Fix `dev/scalastyle` to check `hadoop-cloud` and `jvm-profile` modules

Dongjoon Hyun created SPARK-48127: - Summary: Fix `dev/scalastyle` to check `hadoop-cloud` and `jvm-profile` modules Key: SPARK-48127 URL: https://issues.apache.org/jira/browse/SPARK-48127 Project

[jira] [Resolved] (SPARK-48121) Promote ` KubernetesDriverConf` to `DeveloperApi`

[ https://issues.apache.org/jira/browse/SPARK-48121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48121. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46373 [https

[jira] [Resolved] (SPARK-48120) Enable autolink to SPARK jira issue

[ https://issues.apache.org/jira/browse/SPARK-48120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48120. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 11 [https

[jira] [Resolved] (SPARK-48114) ErrorClassesJsonReader complies template regex on every template resolution

[ https://issues.apache.org/jira/browse/SPARK-48114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48114. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46365 [https

[jira] [Updated] (SPARK-45923) Spark Kubernetes Operator

[ https://issues.apache.org/jira/browse/SPARK-45923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45923: -- Affects Version/s: kubernetes-operator-0.1.0 (was: 4.0.0) > Sp

[jira] [Created] (SPARK-48120) Enable autolink to SPARK jira issue

Dongjoon Hyun created SPARK-48120: - Summary: Enable autolink to SPARK jira issue Key: SPARK-48120 URL: https://issues.apache.org/jira/browse/SPARK-48120 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-48119) Promote ` KubernetesDriverSpec` to `DeveloperApi`

[ https://issues.apache.org/jira/browse/SPARK-48119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48119. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46371 [https

[jira] [Resolved] (SPARK-48118) Support SPARK_SQL_LEGACY_CREATE_HIVE_TABLE env variable

[ https://issues.apache.org/jira/browse/SPARK-48118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48118. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46369 [https

[jira] [Created] (SPARK-48118) Support SPARK_SQL_LEGACY_CREATE_HIVE_TABLE env variable

Dongjoon Hyun created SPARK-48118: - Summary: Support SPARK_SQL_LEGACY_CREATE_HIVE_TABLE env variable Key: SPARK-48118 URL: https://issues.apache.org/jira/browse/SPARK-48118 Project: Spark

[jira] [Updated] (SPARK-48116) Run `pyspark-pandas*` in PR builder and Daily Python CIs

[ https://issues.apache.org/jira/browse/SPARK-48116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48116: -- Summary: Run `pyspark-pandas*` in PR builder and Daily Python CIs (was: Move `pyspark-pandas

[jira] [Resolved] (SPARK-48111) Move tpcds-1g and docker-integration-tests to daily scheduled jobs

[ https://issues.apache.org/jira/browse/SPARK-48111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48111. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46361 [https

[jira] [Resolved] (SPARK-48110) Remove all Maven compilation build

[ https://issues.apache.org/jira/browse/SPARK-48110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48110. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46360 [https

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Description: h2. ASF INFRA POLICY - https://infra.apache.org/github-actions-policy.html h2

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Attachment: Screenshot 2024-05-02 at 23.56.05.png > Reduce GitHub Action usage accord

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Attachment: (was: Screenshot 2024-05-02 at 20.59.18.png) > Reduce GitHub Action us

[jira] [Resolved] (SPARK-48107) Exclude tests from Python distribution

[ https://issues.apache.org/jira/browse/SPARK-48107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48107. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46354 [https

[jira] [Assigned] (SPARK-48107) Exclude tests from Python distribution

[ https://issues.apache.org/jira/browse/SPARK-48107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48107: - Assignee: Nicholas Chammas > Exclude tests from Python distribut

[jira] [Resolved] (SPARK-48106) Use `Python 3.11` in `pyspark` tests of `build_and_test.yml`

[ https://issues.apache.org/jira/browse/SPARK-48106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48106. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46353 [https

[jira] [Resolved] (SPARK-48103) Promote ` KubernetesDriverBuilder` to `DeveloperApi`

[ https://issues.apache.org/jira/browse/SPARK-48103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48103. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46357 [https

[jira] [Resolved] (SPARK-48109) Enable `k8s-integration-tests` only for `kubernetes` module change

[ https://issues.apache.org/jira/browse/SPARK-48109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48109. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46356 [https

[jira] [Resolved] (SPARK-48108) Skip `tpcds-1g` and `docker-integration-tests` tests from `RocksDB UI-Backend` job

[ https://issues.apache.org/jira/browse/SPARK-48108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48108. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46355 [https

[jira] [Assigned] (SPARK-48109) Enable `k8s-integration-tests` only for `kubernetes` module change

[ https://issues.apache.org/jira/browse/SPARK-48109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48109: - Assignee: Dongjoon Hyun > Enable `k8s-integration-tests` only for `kubernetes` mod

[jira] [Created] (SPARK-48109) Enable `k8s-integration-tests` only for `kubernetes` module change

Dongjoon Hyun created SPARK-48109: - Summary: Enable `k8s-integration-tests` only for `kubernetes` module change Key: SPARK-48109 URL: https://issues.apache.org/jira/browse/SPARK-48109 Project: Spark

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Description: h2. ASF INFRA POLICY - https://infra.apache.org/github-actions-policy.html h2

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Description: h2. ASF INFRA POLICY - https://infra.apache.org/github-actions-policy.html h2

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Attachment: (was: Screenshot 2024-05-02 at 13.18.42.png) > Reduce GitHub Action us

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Description: h2. ASF INFRA POLICY - https://infra.apache.org/github-actions-policy.html h2

[jira] [Updated] (SPARK-48094) Reduce GitHub Action usage according to ASF project allowance

[ https://issues.apache.org/jira/browse/SPARK-48094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48094: -- Attachment: Screenshot 2024-05-02 at 20.59.18.png > Reduce GitHub Action usage accord

[jira] [Assigned] (SPARK-48106) Use `Python 3.11` in `pyspark` tests of `build_and_test.yml`

[ https://issues.apache.org/jira/browse/SPARK-48106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48106: - Assignee: Dongjoon Hyun > Use `Python 3.11` in `pyspark` tests of `build_and_test.

[jira] [Updated] (SPARK-48106) Use `Python 3.11` in `pyspark` tests of `build_and_test.yml`

[ https://issues.apache.org/jira/browse/SPARK-48106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48106: -- Description: - https://docs.python.org/3/whatsnew/3.11.html#summary-release-highlights bq

[jira] [Updated] (SPARK-48106) Use `Python 3.11` in `pyspark` tests of `build_and_test.yml`

[ https://issues.apache.org/jira/browse/SPARK-48106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48106: -- Description: - https://docs.python.org/3/whatsnew/3.11.html#summary-release-highlights

[jira] [Created] (SPARK-48106) Use `Python 3.11` in `pyspark` tests of `build_and_test.yml`

Dongjoon Hyun created SPARK-48106: - Summary: Use `Python 3.11` in `pyspark` tests of `build_and_test.yml` Key: SPARK-48106 URL: https://issues.apache.org/jira/browse/SPARK-48106 Project: Spark

[jira] [Assigned] (SPARK-48104) Run `publish_snapshot.yml` once per day

[ https://issues.apache.org/jira/browse/SPARK-48104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48104: - Assignee: Dongjoon Hyun > Run `publish_snapshot.yml` once per

[jira] [Resolved] (SPARK-48104) Run `publish_snapshot.yml` once per day

[ https://issues.apache.org/jira/browse/SPARK-48104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48104. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46352 [https

[jira] [Created] (SPARK-48104) Run `publish_snapshot.yml` once per day

Dongjoon Hyun created SPARK-48104: - Summary: Run `publish_snapshot.yml` once per day Key: SPARK-48104 URL: https://issues.apache.org/jira/browse/SPARK-48104 Project: Spark Issue Type: Sub

[jira] [Resolved] (SPARK-47671) Enable structured logging in log4j2.properties.template and update `configuration.md`

[ https://issues.apache.org/jira/browse/SPARK-47671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47671. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46349 [https

[jira] [Resolved] (SPARK-48099) Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)`

[ https://issues.apache.org/jira/browse/SPARK-48099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48099. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46347 [https

[jira] [Assigned] (SPARK-48099) Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)`

[ https://issues.apache.org/jira/browse/SPARK-48099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48099: - Assignee: Dongjoon Hyun > Run `maven-build` test only on `Java 21 on MacOS 14 (Ap

[jira] [Assigned] (SPARK-48097) Limit GHA job execution time to up to 3 hours in `build_and_test.yml`

[ https://issues.apache.org/jira/browse/SPARK-48097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48097: - Assignee: Dongjoon Hyun > Limit GHA job execution time to up to 3 ho

[jira] [Resolved] (SPARK-48097) Limit GHA job execution time to up to 3 hours in `build_and_test.yml`

[ https://issues.apache.org/jira/browse/SPARK-48097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48097. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46344 [https

[jira] [Updated] (SPARK-48099) Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)`

[ https://issues.apache.org/jira/browse/SPARK-48099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48099: -- Description: `Java 21 on MacOS 14` is the fastest Maven test and covers both Java 17

[jira] [Updated] (SPARK-48099) Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)`

[ https://issues.apache.org/jira/browse/SPARK-48099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48099: -- Description: !Screenshot 2024-05-02 at 14.59.14.png!width=100%! > Run `maven-build` t

[jira] [Updated] (SPARK-48099) Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)`

[ https://issues.apache.org/jira/browse/SPARK-48099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48099: -- Description: !Screenshot 2024-05-02 at 14.59.14.png! (was: !Screenshot 2024-05-02

[jira] [Updated] (SPARK-48099) Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)`

[ https://issues.apache.org/jira/browse/SPARK-48099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48099: -- Description: !Screenshot 2024-05-02 at 14.59.14.png|width=100%! (was: !Screenshot 2024-05

[jira] [Created] (SPARK-48099) Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)`

Dongjoon Hyun created SPARK-48099: - Summary: Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)` Key: SPARK-48099 URL: https://issues.apache.org/jira/browse/SPARK-48099 Project

<    1   2   3   4   5   6   7   8   9   10   >