[jira] [Created] (SPARK-43569) Remove workaround for HADOOP-14067

2023-05-17 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43569: --- Summary: Remove workaround for HADOOP-14067 Key: SPARK-43569 URL: https://issues.apache.org/jira/browse/SPARK-43569 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43568) Enable CategoricalIndexTests.test_categories_setter for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43568: --- Summary: Enable CategoricalIndexTests.test_categories_setter for pandas 2.0.0. Key: SPARK-43568 URL: https://issues.apache.org/jira/browse/SPARK-43568 Project: Spark

[jira] [Assigned] (SPARK-43548) Remove workaround for HADOOP-16255

2023-05-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43548: - Assignee: BingKun Pan > Remove workaround for HADOOP-16255 >

[jira] [Created] (SPARK-43567) Enable CategoricalIndexTests.test_factorize for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43567: --- Summary: Enable CategoricalIndexTests.test_factorize for pandas 2.0.0. Key: SPARK-43567 URL: https://issues.apache.org/jira/browse/SPARK-43567 Project: Spark

[jira] [Resolved] (SPARK-43548) Remove workaround for HADOOP-16255

2023-05-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43548. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41209

[jira] [Created] (SPARK-43566) Enable CategoricalTests.test_categories_setter for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43566: --- Summary: Enable CategoricalTests.test_categories_setter for pandas 2.0.0. Key: SPARK-43566 URL: https://issues.apache.org/jira/browse/SPARK-43566 Project: Spark

[jira] [Created] (SPARK-43565) Enable CategoricalTests.test_as_ordered_unordered for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43565: --- Summary: Enable CategoricalTests.test_as_ordered_unordered for pandas 2.0.0. Key: SPARK-43565 URL: https://issues.apache.org/jira/browse/SPARK-43565 Project: Spark

[jira] [Created] (SPARK-43564) Enable CategoricalTests.test_factorize for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43564: --- Summary: Enable CategoricalTests.test_factorize for pandas 2.0.0. Key: SPARK-43564 URL: https://issues.apache.org/jira/browse/SPARK-43564 Project: Spark Issue

[jira] [Created] (SPARK-43563) Enable CsvTests.test_read_csv_with_squeeze for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43563: --- Summary: Enable CsvTests.test_read_csv_with_squeeze for pandas 2.0.0. Key: SPARK-43563 URL: https://issues.apache.org/jira/browse/SPARK-43563 Project: Spark

[jira] [Created] (SPARK-43562) Enable DataFrameTests.test_append for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43562: --- Summary: Enable DataFrameTests.test_append for pandas 2.0.0. Key: SPARK-43562 URL: https://issues.apache.org/jira/browse/SPARK-43562 Project: Spark Issue

[jira] [Created] (SPARK-43561) Enable DataFrameConversionTests.test_to_latex for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43561: --- Summary: Enable DataFrameConversionTests.test_to_latex for pandas 2.0.0. Key: SPARK-43561 URL: https://issues.apache.org/jira/browse/SPARK-43561 Project: Spark

[jira] [Created] (SPARK-43559) Enable DataFrameSlowTests.test_iteritems for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43559: --- Summary: Enable DataFrameSlowTests.test_iteritems for pandas 2.0.0. Key: SPARK-43559 URL: https://issues.apache.org/jira/browse/SPARK-43559 Project: Spark

[jira] [Created] (SPARK-43560) Enable DataFrameSlowTests.test_mad for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43560: --- Summary: Enable DataFrameSlowTests.test_mad for pandas 2.0.0. Key: SPARK-43560 URL: https://issues.apache.org/jira/browse/SPARK-43560 Project: Spark Issue

[jira] [Created] (SPARK-43558) Enable DataFrameSlowTests.test_product for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43558: --- Summary: Enable DataFrameSlowTests.test_product for pandas 2.0.0. Key: SPARK-43558 URL: https://issues.apache.org/jira/browse/SPARK-43558 Project: Spark Issue

[jira] [Created] (SPARK-43557) Enable DataFrameSlowTests.test_between_time for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43557: --- Summary: Enable DataFrameSlowTests.test_between_time for pandas 2.0.0. Key: SPARK-43557 URL: https://issues.apache.org/jira/browse/SPARK-43557 Project: Spark

[jira] [Created] (SPARK-43556) Enable DataFrameSlowTests.test_describe for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43556: --- Summary: Enable DataFrameSlowTests.test_describe for pandas 2.0.0. Key: SPARK-43556 URL: https://issues.apache.org/jira/browse/SPARK-43556 Project: Spark

[jira] [Created] (SPARK-43555) Enable GroupByTests.test_groupby_multiindex_columns for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43555: --- Summary: Enable GroupByTests.test_groupby_multiindex_columns for pandas 2.0.0. Key: SPARK-43555 URL: https://issues.apache.org/jira/browse/SPARK-43555 Project: Spark

[jira] [Created] (SPARK-43554) Enable GroupByTests.test_basic_stat_funcs for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43554: --- Summary: Enable GroupByTests.test_basic_stat_funcs for pandas 2.0.0. Key: SPARK-43554 URL: https://issues.apache.org/jira/browse/SPARK-43554 Project: Spark

[jira] [Created] (SPARK-43553) Enable GroupByTests.test_mad for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43553: --- Summary: Enable GroupByTests.test_mad for pandas 2.0.0. Key: SPARK-43553 URL: https://issues.apache.org/jira/browse/SPARK-43553 Project: Spark Issue Type:

[jira] [Created] (SPARK-43552) Enable GroupByTests.test_nth for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43552: --- Summary: Enable GroupByTests.test_nth for pandas 2.0.0. Key: SPARK-43552 URL: https://issues.apache.org/jira/browse/SPARK-43552 Project: Spark Issue Type:

[jira] [Created] (SPARK-43551) Enable GroupByTests.test_prod for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43551: --- Summary: Enable GroupByTests.test_prod for pandas 2.0.0. Key: SPARK-43551 URL: https://issues.apache.org/jira/browse/SPARK-43551 Project: Spark Issue Type:

[jira] [Created] (SPARK-43550) Enable SeriesTests.test_factorize for pandas 2.0.0.

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43550: --- Summary: Enable SeriesTests.test_factorize for pandas 2.0.0. Key: SPARK-43550 URL: https://issues.apache.org/jira/browse/SPARK-43550 Project: Spark Issue

[jira] [Commented] (SPARK-43549) Assign a name to the error class _LEGACY_ERROR_TEMP_0035

2023-05-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723779#comment-17723779 ] BingKun Pan commented on SPARK-43549: - I work on it. > Assign a name to the error class

[jira] [Created] (SPARK-43549) Assign a name to the error class _LEGACY_ERROR_TEMP_0035

2023-05-17 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43549: --- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_0035 Key: SPARK-43549 URL: https://issues.apache.org/jira/browse/SPARK-43549 Project: Spark Issue

[jira] [Assigned] (SPARK-43547) Update "Supported Pandas API" page to point out the proper pandas docs

2023-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43547: Assignee: Haejoon Lee > Update "Supported Pandas API" page to point out the proper

[jira] [Resolved] (SPARK-43547) Update "Supported Pandas API" page to point out the proper pandas docs

2023-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43547. -- Fix Version/s: 3.4.1 Resolution: Fixed Issue resolved by pull request 41208

[jira] [Comment Edited] (SPARK-40964) Cannot run spark history server with shaded hadoop jar

2023-05-17 Thread Shuaipeng Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723777#comment-17723777 ] Shuaipeng Lee edited comment on SPARK-40964 at 5/18/23 2:53 AM: Thanks

[jira] [Commented] (SPARK-40964) Cannot run spark history server with shaded hadoop jar

2023-05-17 Thread Shuaipeng Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723777#comment-17723777 ] Shuaipeng Lee commented on SPARK-40964: --- Thanks for your commits. I rebuild the hadoop-client-api

[jira] [Updated] (SPARK-43548) Remove workaround for HADOOP-16255

2023-05-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43548: Component/s: Structured Streaming (was: SQL) > Remove workaround for

[jira] [Created] (SPARK-43548) Remove workaround for HADOOP-16255

2023-05-17 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43548: --- Summary: Remove workaround for HADOOP-16255 Key: SPARK-43548 URL: https://issues.apache.org/jira/browse/SPARK-43548 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-43488) bitmap function

2023-05-17 Thread Jia Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723772#comment-17723772 ] Jia Fan commented on SPARK-43488: - Hi, [~cloud_fan] If we want achieve this feature. Are we should

[jira] [Updated] (SPARK-43503) Deserialisation Failure on State Store Schema Evolution (Spark Structured Streaming)

2023-05-17 Thread Varun Arora (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Arora updated SPARK-43503: Priority: Critical (was: Major) > Deserialisation Failure on State Store Schema Evolution (Spark

[jira] [Assigned] (SPARK-43022) protobuf functions

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43022: Assignee: Yang Jie > protobuf functions > -- > > Key:

[jira] [Resolved] (SPARK-43022) protobuf functions

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43022. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40654

[jira] [Updated] (SPARK-43542) Define a new error class and apply for the case where streaming query fails due to concurrent run of streaming query with same checkpoint

2023-05-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-43542: - Affects Version/s: 3.5.0 (was: 1.6.3) > Define a new error class and

[jira] [Created] (SPARK-43547) Update "Supported Pandas API" page to point out the proper pandas docs

2023-05-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43547: --- Summary: Update "Supported Pandas API" page to point out the proper pandas docs Key: SPARK-43547 URL: https://issues.apache.org/jira/browse/SPARK-43547 Project: Spark

[jira] [Updated] (SPARK-43544) Fix nested MapType behavior in Pandas UDF

2023-05-17 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-43544: - Summary: Fix nested MapType behavior in Pandas UDF (was: Standardize nested non-atomic input

[jira] [Commented] (SPARK-43509) Support creating multiple sessions for Spark Connect in PySpark

2023-05-17 Thread Ignite TC Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723755#comment-17723755 ] Ignite TC Bot commented on SPARK-43509: --- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-43546) Complete Pandas UDF parity tests

2023-05-17 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-43546: Summary: Complete Pandas UDF parity tests Key: SPARK-43546 URL: https://issues.apache.org/jira/browse/SPARK-43546 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-43545) Remove outdated UNSUPPORTED_DATA_TYPE_FOR_ARROW_CONVERSION

2023-05-17 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-43545: Summary: Remove outdated UNSUPPORTED_DATA_TYPE_FOR_ARROW_CONVERSION Key: SPARK-43545 URL: https://issues.apache.org/jira/browse/SPARK-43545 Project: Spark

[jira] [Created] (SPARK-43544) Standardize nested non-atomic input type support in Pandas UDF

2023-05-17 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-43544: Summary: Standardize nested non-atomic input type support in Pandas UDF Key: SPARK-43544 URL: https://issues.apache.org/jira/browse/SPARK-43544 Project: Spark

[jira] [Created] (SPARK-43543) Standardize Nested Complex DataTypes Support

2023-05-17 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-43543: Summary: Standardize Nested Complex DataTypes Support Key: SPARK-43543 URL: https://issues.apache.org/jira/browse/SPARK-43543 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43436) Upgrade rocksdbjni to 8.1.1.1

2023-05-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43436. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41122

[jira] [Assigned] (SPARK-43436) Upgrade rocksdbjni to 8.1.1.1

2023-05-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43436: - Assignee: Yang Jie > Upgrade rocksdbjni to 8.1.1.1 > - > >

[jira] [Created] (SPARK-43542) Define a new error class and apply for the case where streaming query fails due to concurrent run of streaming query with same checkpoint

2023-05-17 Thread Eric Marnadi (Jira)
Eric Marnadi created SPARK-43542: Summary: Define a new error class and apply for the case where streaming query fails due to concurrent run of streaming query with same checkpoint Key: SPARK-43542 URL:

[jira] [Updated] (SPARK-43541) Incorrect column resolution on FULL OUTER JOIN with USING

2023-05-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-43541: - Description: This was tested on Spark 3.3.2 and Spark 3.4.0. {code} Causes

[jira] [Created] (SPARK-43541) Incorrect column resolution on FULL OUTER JOIN with USING

2023-05-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-43541: Summary: Incorrect column resolution on FULL OUTER JOIN with USING Key: SPARK-43541 URL: https://issues.apache.org/jira/browse/SPARK-43541 Project: Spark Issue

[jira] [Resolved] (SPARK-43537) Upgrade the asm deps in the tools module to 9.4

2023-05-17 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-43537. -- Fix Version/s: 3.5.0 Assignee: Yang Jie Resolution: Fixed Resolved by

[jira] [Updated] (SPARK-43537) Upgrade the asm deps in the tools module to 9.4

2023-05-17 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-43537: - Priority: Minor (was: Major) > Upgrade the asm deps in the tools module to 9.4 >

[jira] [Updated] (SPARK-43540) Add working directory into classpath on the driver in K8S cluster mode

2023-05-17 Thread Fei Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang updated SPARK-43540: - Summary: Add working directory into classpath on the driver in K8S cluster mode (was: Add current

[jira] [Updated] (SPARK-43540) Add current working directory into classpath on the driver in K8S cluster mode

2023-05-17 Thread Fei Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang updated SPARK-43540: - Description: In Yarn cluster modes, the passed files/jars are able to be accessed in the classloader.

[jira] [Updated] (SPARK-43540) Add current working directory into classpath on the driver in K8S cluster mode

2023-05-17 Thread Fei Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang updated SPARK-43540: - Description: In Yarn cluster modes, the passed files/jars are able to be accessed in the classloader.

[jira] [Updated] (SPARK-43540) Add current working directory into classpath on the driver in K8S cluster mode

2023-05-17 Thread Fei Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang updated SPARK-43540: - Description: In Yarn cluster modes, the passed files/jars are able to be accessed in the classloader.

[jira] [Created] (SPARK-43540) Add current working directory into classpath on the driver in K8S cluster mode

2023-05-17 Thread Fei Wang (Jira)
Fei Wang created SPARK-43540: Summary: Add current working directory into classpath on the driver in K8S cluster mode Key: SPARK-43540 URL: https://issues.apache.org/jira/browse/SPARK-43540 Project:

[jira] [Commented] (SPARK-43537) Upgrade the asm deps in the tools module to 9.4

2023-05-17 Thread GridGain Integration (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723478#comment-17723478 ] GridGain Integration commented on SPARK-43537: -- User 'LuciferYang' has created a pull

[jira] [Resolved] (SPARK-43535) Adjust the ImportOrderChecker rule to resolve long-standing import issues

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43535. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41184

[jira] [Assigned] (SPARK-43535) Adjust the ImportOrderChecker rule to resolve long-standing import issues

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43535: Assignee: BingKun Pan > Adjust the ImportOrderChecker rule to resolve long-standing import

[jira] [Comment Edited] (SPARK-43538) Spark Homebrew Formulae currently depends on non-officially-supported Java 20

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723433#comment-17723433 ] Yuming Wang edited comment on SPARK-43538 at 5/17/23 12:36 PM: --- Yes. I

[jira] [Commented] (SPARK-43538) Spark Homebrew Formulae currently depends on non-officially-supported Java 20

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723433#comment-17723433 ] Yuming Wang commented on SPARK-43538: - Yes. I think so. > Spark Homebrew Formulae currently depends

[jira] [Comment Edited] (SPARK-43538) Spark Homebrew Formulae currently depends on non-officially-supported Java 20

2023-05-17 Thread Ghislain Fourny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723430#comment-17723430 ] Ghislain Fourny edited comment on SPARK-43538 at 5/17/23 12:04 PM: ---

[jira] [Commented] (SPARK-43538) Spark Homebrew Formulae currently depends on non-officially-supported Java 20

2023-05-17 Thread Ghislain Fourny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723430#comment-17723430 ] Ghislain Fourny commented on SPARK-43538: - Thanks, Yuming Wang! Does it mean the Homebrew

[jira] [Commented] (SPARK-43538) Spark Homebrew Formulae currently depends on non-officially-supported Java 20

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723429#comment-17723429 ] Yuming Wang commented on SPARK-43538: - We have not tested on Java 20, because Java 20 is not LTS. >

[jira] [Created] (SPARK-43539) Assign a name to the error class _LEGACY_ERROR_TEMP_0003

2023-05-17 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43539: --- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_0003 Key: SPARK-43539 URL: https://issues.apache.org/jira/browse/SPARK-43539 Project: Spark Issue

[jira] [Commented] (SPARK-43536) Statsd sink reporter reports incorrect counter metrics.

2023-05-17 Thread Ignite TC Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723415#comment-17723415 ] Ignite TC Bot commented on SPARK-43536: --- User 'venkateshbalaji99' has created a pull request for

[jira] [Updated] (SPARK-43538) Spark Homebrew Formulae currently depends on non-officially-supported Java 20

2023-05-17 Thread Ghislain Fourny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ghislain Fourny updated SPARK-43538: Summary: Spark Homebrew Formulae currently depends on non-officially-supported Java 20

[jira] [Updated] (SPARK-43538) Spark Homebrew recipe currently depends on non-officially-supported Java 20

2023-05-17 Thread Ghislain Fourny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ghislain Fourny updated SPARK-43538: Summary: Spark Homebrew recipe currently depends on non-officially-supported Java 20

[jira] [Created] (SPARK-43538) Spark Homebrew recipe falls back to non-officially-supported Java 20

2023-05-17 Thread Ghislain Fourny (Jira)
Ghislain Fourny created SPARK-43538: --- Summary: Spark Homebrew recipe falls back to non-officially-supported Java 20 Key: SPARK-43538 URL: https://issues.apache.org/jira/browse/SPARK-43538 Project:

[jira] [Created] (SPARK-43537) Upgrade the asm deps in the tools module to 9.4

2023-05-17 Thread Yang Jie (Jira)
Yang Jie created SPARK-43537: Summary: Upgrade the asm deps in the tools module to 9.4 Key: SPARK-43537 URL: https://issues.apache.org/jira/browse/SPARK-43537 Project: Spark Issue Type:

[jira] [Created] (SPARK-43536) Statsd sink reporter reports incorrect counter metrics.

2023-05-17 Thread Abhishek Modi (Jira)
Abhishek Modi created SPARK-43536: - Summary: Statsd sink reporter reports incorrect counter metrics. Key: SPARK-43536 URL: https://issues.apache.org/jira/browse/SPARK-43536 Project: Spark

[jira] [Commented] (SPARK-43535) Adjust the ImportOrderChecker rule to resolve long-standing import issues

2023-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723389#comment-17723389 ] ASF GitHub Bot commented on SPARK-43535: User 'panbingkun' has created a pull request for this

[jira] [Commented] (SPARK-43535) Adjust the ImportOrderChecker rule to resolve long-standing import issues

2023-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723390#comment-17723390 ] ASF GitHub Bot commented on SPARK-43535: User 'panbingkun' has created a pull request for this

[jira] [Comment Edited] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

2023-05-17 Thread caican (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723388#comment-17723388 ] caican edited comment on SPARK-43526 at 5/17/23 9:03 AM: - [~yumwang]  Tpcds

[jira] [Comment Edited] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

2023-05-17 Thread caican (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723388#comment-17723388 ] caican edited comment on SPARK-43526 at 5/17/23 9:02 AM: - [~yumwang]  Tpcds

[jira] [Updated] (SPARK-43523) Memory leak in Spark UI

2023-05-17 Thread Amine Bagdouri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amine Bagdouri updated SPARK-43523: --- Description: We have a distributed Spark application running on Azure HDInsight using Spark

[jira] [Commented] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

2023-05-17 Thread caican (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723388#comment-17723388 ] caican commented on SPARK-43526: [~yumwang]  We plan to use shuffledHashJoin preferentially to eliminate

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

2023-05-17 Thread caican (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-17-16-54-59-053.png > when shuffle hash join is enabled, q95 performance

[jira] [Updated] (SPARK-43526) when shuffle hash join is enabled, q95 performance deteriorates

2023-05-17 Thread caican (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caican updated SPARK-43526: --- Attachment: image-2023-05-17-16-53-42-302.png > when shuffle hash join is enabled, q95 performance

[jira] [Comment Edited] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-05-17 Thread Svyatoslav Semenyuk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723216#comment-17723216 ] Svyatoslav Semenyuk edited comment on SPARK-43514 at 5/17/23 8:35 AM:

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-05-17 Thread Svyatoslav Semenyuk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Svyatoslav Semenyuk updated SPARK-43514: Description: We designed a function that joins two DFs on common column with some

[jira] [Comment Edited] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-05-17 Thread Svyatoslav Semenyuk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723216#comment-17723216 ] Svyatoslav Semenyuk edited comment on SPARK-43514 at 5/17/23 8:34 AM:

[jira] [Commented] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723373#comment-17723373 ] Yuming Wang commented on SPARK-43534: - https://github.com/apache/spark/pull/41195 > Add

[jira] [Updated] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43534: Attachment: hadoop log jars.png > Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active

[jira] [Updated] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43534: Description: Build Spark: {code:sh} ./dev/make-distribution.sh --name default --tgz -Phive

[jira] [Created] (SPARK-43535) Adjust the ImportOrderChecker rule to resolve long-standing import issues

2023-05-17 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43535: --- Summary: Adjust the ImportOrderChecker rule to resolve long-standing import issues Key: SPARK-43535 URL: https://issues.apache.org/jira/browse/SPARK-43535 Project:

[jira] [Resolved] (SPARK-43128) Streaming progress struct (especially in Scala)

2023-05-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-43128. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40892

[jira] [Assigned] (SPARK-43128) Streaming progress struct (especially in Scala)

2023-05-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-43128: Assignee: Yang Jie > Streaming progress struct (especially in Scala) >

[jira] [Updated] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43534: Description: Build Spark: {code:sh} ./dev/make-distribution.sh --name default --tgz -Phive

[jira] [Updated] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43534: Attachment: log4j-slf4j2-impl-2.20.0.jar > Add log4j-1.2-api and log4j-slf4j2-impl to classpath

[jira] [Updated] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43534: Attachment: log4j-1.2-api-2.20.0.jar > Add log4j-1.2-api and log4j-slf4j2-impl to classpath if

[jira] [Updated] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43534: Description: Build Spark: {code:sh} ./dev/make-distribution.sh --name provided --tgz -Phive

[jira] [Created] (SPARK-43534) Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided

2023-05-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-43534: --- Summary: Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided Key: SPARK-43534 URL: https://issues.apache.org/jira/browse/SPARK-43534

[jira] [Resolved] (SPARK-43509) Support creating multiple sessions for Spark Connect in PySpark

2023-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43509. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41013

[jira] [Assigned] (SPARK-43509) Support creating multiple sessions for Spark Connect in PySpark

2023-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43509: Assignee: Martin Grund > Support creating multiple sessions for Spark Connect in PySpark

[jira] [Resolved] (SPARK-43132) Add foreach streaming API in Python

2023-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43132. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41026