[jira] [Resolved] (SPARK-48552) multi-line CSV schema inference should also throw FAILED_READ_FILE

2024-06-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48552. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46890

[jira] [Assigned] (SPARK-48552) multi-line CSV schema inference should also throw FAILED_READ_FILE

2024-06-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48552: Assignee: Wenchen Fan > multi-line CSV schema inference should also throw

[jira] [Created] (SPARK-48560) Make StreamingQueryListener.spark settable

2024-06-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48560: Summary: Make StreamingQueryListener.spark settable Key: SPARK-48560 URL: https://issues.apache.org/jira/browse/SPARK-48560 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-47952) Support retrieving the real SparkConnectService GRPC address and port programmatically when running on Yarn

2024-06-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47952: Assignee: TakawaAkirayo (was: Adam Binford) > Support retrieving the real

[jira] [Resolved] (SPARK-47952) Support retrieving the real SparkConnectService GRPC address and port programmatically when running on Yarn

2024-06-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47952. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46182

[jira] [Assigned] (SPARK-47952) Support retrieving the real SparkConnectService GRPC address and port programmatically when running on Yarn

2024-06-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47952: Assignee: Adam Binford > Support retrieving the real SparkConnectService GRPC address

[jira] [Resolved] (SPARK-48550) Directly use the parent Window class

2024-06-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48550. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46892

[jira] [Assigned] (SPARK-48550) Directly use the parent Window class

2024-06-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48550: Assignee: Ruifeng Zheng > Directly use the parent Window class >

[jira] [Assigned] (SPARK-48533) Add test for cached schema

2024-06-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48533: Assignee: Ruifeng Zheng > Add test for cached schema > -- > >

[jira] [Resolved] (SPARK-48533) Add test for cached schema

2024-06-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48533. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46871

[jira] [Created] (SPARK-48534) Support interruptOperation in streaming queries

2024-06-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48534: Summary: Support interruptOperation in streaming queries Key: SPARK-48534 URL: https://issues.apache.org/jira/browse/SPARK-48534 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48523) Add `grpc_max_message_size ` description to `client-connection-string.md`

2024-06-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48523. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46862

[jira] [Assigned] (SPARK-48523) Add `grpc_max_message_size ` description to `client-connection-string.md`

2024-06-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48523: Assignee: BingKun Pan > Add `grpc_max_message_size ` description to

[jira] [Resolved] (SPARK-48485) Support interruptTag and interruptAll in streaming queries

2024-06-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48485. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46819

[jira] [Assigned] (SPARK-48482) dropDuplicates and dropDuplicatesWithinWatermark should accept varargs

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48482: Assignee: Wei Liu > dropDuplicates and dropDuplicatesWithinWatermark should accept

[jira] [Resolved] (SPARK-48482) dropDuplicates and dropDuplicatesWithinWatermark should accept varargs

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48482. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46817

[jira] [Assigned] (SPARK-48508) Client Side RPC optimization for Spark Connect

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48508: Assignee: Ruifeng Zheng > Client Side RPC optimization for Spark Connect >

[jira] [Resolved] (SPARK-48508) Client Side RPC optimization for Spark Connect

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48508. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46848

[jira] [Resolved] (SPARK-48507) Use Hadoop 3.3.6 winutils in `build_sparkr_window`

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48507. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46846

[jira] [Assigned] (SPARK-48507) Use Hadoop 3.3.6 winutils in `build_sparkr_window`

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48507: Assignee: BingKun Pan > Use Hadoop 3.3.6 winutils in `build_sparkr_window` >

[jira] [Assigned] (SPARK-48504) Parent Window class for Spark Connect and Spark Classic

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48504: Assignee: Ruifeng Zheng > Parent Window class for Spark Connect and Spark Classic >

[jira] [Resolved] (SPARK-48504) Parent Window class for Spark Connect and Spark Classic

2024-06-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48504. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46841

[jira] [Resolved] (SPARK-48496) Use static regex Pattern instances in common/utils JavaUtils

2024-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48496. -- Fix Version/s: 4.0.0 Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-48489) Throw an user-facing error when reading invalid schema from text DataSource

2024-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48489. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46823

[jira] [Assigned] (SPARK-48489) Throw an user-facing error when reading invalid schema from text DataSource

2024-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48489: Assignee: Stefan Bukorovic > Throw an user-facing error when reading invalid schema from

[jira] [Assigned] (SPARK-48374) Support additional PyArrow Table column types

2024-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48374: Assignee: Ian Cook > Support additional PyArrow Table column types >

[jira] [Resolved] (SPARK-48374) Support additional PyArrow Table column types

2024-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48374. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46688

[jira] [Assigned] (SPARK-48220) Allow passing PyArrow Table to createDataFrame()

2024-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48220: Assignee: Ian Cook > Allow passing PyArrow Table to createDataFrame() >

[jira] [Resolved] (SPARK-48220) Allow passing PyArrow Table to createDataFrame()

2024-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48220. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46529

[jira] [Created] (SPARK-48485) Support interruptTag and interruptAll in streaming queries

2024-05-31 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48485: Summary: Support interruptTag and interruptAll in streaming queries Key: SPARK-48485 URL: https://issues.apache.org/jira/browse/SPARK-48485 Project: Spark

[jira] [Assigned] (SPARK-48474) Fix the class name of the log in `SparkSubmitArguments` & `SparkSubmit`

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48474: Assignee: BingKun Pan > Fix the class name of the log in `SparkSubmitArguments` &

[jira] [Resolved] (SPARK-48474) Fix the class name of the log in `SparkSubmitArguments` & `SparkSubmit`

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48474. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46808

[jira] [Assigned] (SPARK-48467) Upgrade Maven to 3.9.7

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48467: Assignee: BingKun Pan > Upgrade Maven to 3.9.7 > -- > >

[jira] [Resolved] (SPARK-48467) Upgrade Maven to 3.9.7

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48467. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46798

[jira] [Assigned] (SPARK-47716) SQLQueryTestSuite flaky case due to view name conflict

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47716: Assignee: Jack Chen > SQLQueryTestSuite flaky case due to view name conflict >

[jira] [Resolved] (SPARK-47716) SQLQueryTestSuite flaky case due to view name conflict

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47716. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45855

[jira] [Resolved] (SPARK-48461) Replace NullPointerExceptions with proper error classes in AssertNotNull expression

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48461. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46793

[jira] [Assigned] (SPARK-48446) Update SS Doc of dropDuplicatesWithinWatermark to use the right syntax

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48446: Assignee: Yuchen Liu > Update SS Doc of dropDuplicatesWithinWatermark to use the right

[jira] [Resolved] (SPARK-48446) Update SS Doc of dropDuplicatesWithinWatermark to use the right syntax

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48446. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46797

[jira] [Resolved] (SPARK-48475) Optimize _get_jvm_function in PySpark.

2024-05-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48475. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46809

[jira] [Resolved] (SPARK-48464) Refactor SQLConfSuite and StatisticsSuite

2024-05-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48464. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46796

[jira] [Resolved] (SPARK-48454) Directly use the parent dataframe class

2024-05-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48454. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46785

[jira] [Assigned] (SPARK-48454) Directly use the parent dataframe class

2024-05-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48454: Assignee: Ruifeng Zheng > Directly use the parent dataframe class >

[jira] [Resolved] (SPARK-48442) Add parenthesis to awaitTermination call

2024-05-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48442. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46779

[jira] [Created] (SPARK-48459) Implement DataFrameQueryContext in Spark Connect

2024-05-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48459: Summary: Implement DataFrameQueryContext in Spark Connect Key: SPARK-48459 URL: https://issues.apache.org/jira/browse/SPARK-48459 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48445) Don't inline UDFs with non-cheap children in CollapseProject

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48445. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46780

[jira] [Assigned] (SPARK-48445) Don't inline UDFs with non-cheap children in CollapseProject

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48445: Assignee: Kelvin Jiang > Don't inline UDFs with non-cheap children in CollapseProject >

[jira] [Commented] (SPARK-23015) spark-submit fails when submitting several jobs in parallel

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850230#comment-17850230 ] Hyukjin Kwon commented on SPARK-23015: -- Fixed in https://github.com/apache/spark/pull/43706 >

[jira] [Reopened] (SPARK-23015) spark-submit fails when submitting several jobs in parallel

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-23015: -- > spark-submit fails when submitting several jobs in parallel >

[jira] [Resolved] (SPARK-23015) spark-submit fails when submitting several jobs in parallel

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23015. -- Fix Version/s: 4.0.0 Resolution: Fixed > spark-submit fails when submitting several

[jira] [Assigned] (SPARK-42965) metadata mismatch for StructField when running some tests.

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42965: Assignee: Ruifeng Zheng > metadata mismatch for StructField when running some tests. >

[jira] [Resolved] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48322. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46636

[jira] [Resolved] (SPARK-42965) metadata mismatch for StructField when running some tests.

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42965. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46636

[jira] [Assigned] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48322: Assignee: Ruifeng Zheng > Drop internal metadata in `DataFrame.schema` >

[jira] [Resolved] (SPARK-48438) Directly use the parent column class

2024-05-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48438. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46775

[jira] [Resolved] (SPARK-48434) Make printSchema use the cached schema

2024-05-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48434. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46764

[jira] [Assigned] (SPARK-48434) Make printSchema use the cached schema

2024-05-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48434: Assignee: Ruifeng Zheng > Make printSchema use the cached schema >

[jira] [Assigned] (SPARK-48432) Unnecessary Integer unboxing in UnivocityParser

2024-05-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48432: Assignee: Vladimir Golubev > Unnecessary Integer unboxing in UnivocityParser >

[jira] [Resolved] (SPARK-48432) Unnecessary Integer unboxing in UnivocityParser

2024-05-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48432. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46759

[jira] [Resolved] (SPARK-48425) Replaces pyspark-connect to pyspark_connect for its output name

2024-05-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48425. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46751

[jira] [Assigned] (SPARK-48425) Replaces pyspark-connect to pyspark_connect for its output name

2024-05-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48425: Assignee: Hyukjin Kwon > Replaces pyspark-connect to pyspark_connect for its output name

[jira] [Updated] (SPARK-48425) Replaces pyspark-connect to pyspark_connect for its output name

2024-05-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48425: - Description: The issue is at setuptools from 69.X.X. It replaces dash in package name to

[jira] [Created] (SPARK-48425) Replaces pyspark-connect to pyspark_connect for its output name

2024-05-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48425: Summary: Replaces pyspark-connect to pyspark_connect for its output name Key: SPARK-48425 URL: https://issues.apache.org/jira/browse/SPARK-48425 Project: Spark

[jira] [Resolved] (SPARK-48424) Make dev/is-changed.py to return true it it fails

2024-05-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48424. -- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-48424) Make dev/is-changed.py to return true it it fails

2024-05-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48424: Assignee: Hyukjin Kwon > Make dev/is-changed.py to return true it it fails >

[jira] [Created] (SPARK-48424) Make dev/is-changed.py to return true it it fails

2024-05-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48424: Summary: Make dev/is-changed.py to return true it it fails Key: SPARK-48424 URL: https://issues.apache.org/jira/browse/SPARK-48424 Project: Spark Issue

[jira] [Resolved] (SPARK-48370) Checkpoint and localCheckpoint in Scala Spark Connect client

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48370. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46683

[jira] [Assigned] (SPARK-48370) Checkpoint and localCheckpoint in Scala Spark Connect client

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48370: Assignee: Hyukjin Kwon > Checkpoint and localCheckpoint in Scala Spark Connect client >

[jira] [Assigned] (SPARK-48393) Move a group of constants to `pyspark.util`

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48393: Assignee: Ruifeng Zheng > Move a group of constants to `pyspark.util` >

[jira] [Resolved] (SPARK-48393) Move a group of constants to `pyspark.util`

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48393. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46710

[jira] [Reopened] (SPARK-48379) Cancel build during a PR when a new commit is pushed

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-48379: -- Assignee: (was: Stefan Kandic) Reverted in

[jira] [Updated] (SPARK-48379) Cancel build during a PR when a new commit is pushed

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48379: - Fix Version/s: (was: 4.0.0) > Cancel build during a PR when a new commit is pushed >

[jira] [Resolved] (SPARK-48389) Remove obsolete workflow cancel_duplicate_workflow_runs

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48389. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46703

[jira] [Assigned] (SPARK-48389) Remove obsolete workflow cancel_duplicate_workflow_runs

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48389: Assignee: Hyukjin Kwon > Remove obsolete workflow cancel_duplicate_workflow_runs >

[jira] [Created] (SPARK-48389) Remove obsolete workflow cancel_duplicate_workflow_runs

2024-05-22 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48389: Summary: Remove obsolete workflow cancel_duplicate_workflow_runs Key: SPARK-48389 URL: https://issues.apache.org/jira/browse/SPARK-48389 Project: Spark

[jira] [Assigned] (SPARK-48379) Cancel build during a PR when a new commit is pushed

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48379: Assignee: Stefan Kandic > Cancel build during a PR when a new commit is pushed >

[jira] [Resolved] (SPARK-48379) Cancel build during a PR when a new commit is pushed

2024-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48379. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46689

[jira] [Resolved] (SPARK-48341) Allow Spark Connect plugins to use QueryTest in their tests

2024-05-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48341. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46667

[jira] [Created] (SPARK-48370) Checkpoint and localCheckpoint in Scala Spark Connect client

2024-05-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48370: Summary: Checkpoint and localCheckpoint in Scala Spark Connect client Key: SPARK-48370 URL: https://issues.apache.org/jira/browse/SPARK-48370 Project: Spark

[jira] [Updated] (SPARK-48370) Checkpoint and localCheckpoint in Scala Spark Connect client

2024-05-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48370: - Issue Type: Improvement (was: Bug) > Checkpoint and localCheckpoint in Scala Spark Connect

[jira] [Resolved] (SPARK-48367) Fix lint-scala for scalafmt to detect properly

2024-05-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48367. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46679

[jira] [Assigned] (SPARK-48367) Fix lint-scala for scalafmt to detect properly

2024-05-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48367: Assignee: Hyukjin Kwon > Fix lint-scala for scalafmt to detect properly >

[jira] [Created] (SPARK-48367) Fix lint-scala for scalafmt to detect properly

2024-05-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48367: Summary: Fix lint-scala for scalafmt to detect properly Key: SPARK-48367 URL: https://issues.apache.org/jira/browse/SPARK-48367 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48363) Cleanup some redundant codes in `from_xml`

2024-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48363. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46674

[jira] [Assigned] (SPARK-48363) Cleanup some redundant codes in `from_xml`

2024-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48363: Assignee: BingKun Pan > Cleanup some redundant codes in `from_xml` >

[jira] [Resolved] (SPARK-48340) Support TimestampNTZ infer schema miss prefer_timestamp_ntz

2024-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48340. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 4

[jira] [Assigned] (SPARK-48340) Support TimestampNTZ infer schema miss prefer_timestamp_ntz

2024-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48340: Assignee: angerszhu > Support TimestampNTZ infer schema miss prefer_timestamp_ntz >

[jira] [Resolved] (SPARK-48258) Implement DataFrame.checkpoint and DataFrame.localCheckpoint

2024-05-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48258. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46570

[jira] [Resolved] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

2024-05-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48333. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46654

[jira] [Assigned] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

2024-05-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48333: Assignee: Ruifeng Zheng > Test `test_sorting_functions_with_column` with same `Column` >

[jira] [Assigned] (SPARK-48319) Test `assert_true` and `raise_error` with the same error class as Spark Classic

2024-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48319: Assignee: Ruifeng Zheng > Test `assert_true` and `raise_error` with the same error class

[jira] [Resolved] (SPARK-48319) Test `assert_true` and `raise_error` with the same error class as Spark Classic

2024-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48319. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46633

[jira] [Assigned] (SPARK-48317) Enable test_udtf_with_analyze_using_archive and test_udtf_with_analyze_using_file

2024-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48317: Assignee: Hyukjin Kwon > Enable test_udtf_with_analyze_using_archive and >

[jira] [Resolved] (SPARK-48317) Enable test_udtf_with_analyze_using_archive and test_udtf_with_analyze_using_file

2024-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48317. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46632

[jira] [Assigned] (SPARK-48316) Fix comments for SparkFrameMethodsParityTests.test_coalesce and test_repartition

2024-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48316: Assignee: Hyukjin Kwon > Fix comments for SparkFrameMethodsParityTests.test_coalesce and

[jira] [Resolved] (SPARK-48316) Fix comments for SparkFrameMethodsParityTests.test_coalesce and test_repartition

2024-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48316. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46629

[jira] [Updated] (SPARK-48316) Fix comments for SparkFrameMethodsParityTests.test_coalesce and test_repartition

2024-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48316: - Summary: Fix comments for SparkFrameMethodsParityTests.test_coalesce and test_repartition

[jira] [Created] (SPARK-48317) Enable test_udtf_with_analyze_using_archive and test_udtf_with_analyze_using_file

2024-05-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48317: Summary: Enable test_udtf_with_analyze_using_archive and test_udtf_with_analyze_using_file Key: SPARK-48317 URL: https://issues.apache.org/jira/browse/SPARK-48317

[jira] [Updated] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48238: - Parent: (was: SPARK-47970) Issue Type: Bug (was: Sub-task) > Spark fail to start

[jira] [Created] (SPARK-48316) Enable SparkFrameMethodsParityTests.test_coalesce and test_repartition

2024-05-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48316: Summary: Enable SparkFrameMethodsParityTests.test_coalesce and test_repartition Key: SPARK-48316 URL: https://issues.apache.org/jira/browse/SPARK-48316 Project:

  1   2   3   4   5   6   7   8   9   10   >