[jira] [Assigned] (SPARK-44280) Add convertJavaTimestampToTimestamp in JDBCDialect API

2023-08-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44280: --- Assignee: Mingkang Li > Add convertJavaTimestampToTimestamp in JDBCDialect API > --

[jira] [Resolved] (SPARK-44280) Add convertJavaTimestampToTimestamp in JDBCDialect API

2023-08-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44280. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41843 [https://gith

[jira] [Updated] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

2023-08-01 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhao updated SPARK-44627: - Description: When the resultSet exists a timestmp column and it's value is null, but column define is n

[jira] [Updated] (SPARK-44572) Clean up unused installers ASAP

2023-08-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44572: -- Summary: Clean up unused installers ASAP (was: Clean up unused installer ASAP) > Clean up un

[jira] [Updated] (SPARK-44572) Clean up unused installer ASAP

2023-08-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44572: -- Summary: Clean up unused installer ASAP (was: Clean up unused files ASAP) > Clean up unused i

[jira] [Updated] (SPARK-43043) Improve the performance of MapOutputTracker.updateMapOutput

2023-08-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-43043: - Fix Version/s: 3.5.0 (was: 3.4.1) > Improve the performance of MapOutputT

[jira] [Assigned] (SPARK-44630) Revert SPARK-43043 Improve the performance of MapOutputTracker.updateMapOutput

2023-08-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44630: Assignee: Dongjoon Hyun > Revert SPARK-43043 Improve the performance of MapOutputTracker.

[jira] [Resolved] (SPARK-44630) Revert SPARK-43043 Improve the performance of MapOutputTracker.updateMapOutput

2023-08-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44630. -- Fix Version/s: 3.4.2 Resolution: Fixed Issue resolved by pull request 42285 [https://gi

[jira] [Updated] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

2023-08-01 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhao updated SPARK-44627: - Priority: Minor (was: Major) > org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRow

[jira] [Commented] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

2023-08-01 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750115#comment-17750115 ] Min Zhao commented on SPARK-44627: -- !image-2023-08-02-14-01-54-447.png! it only update

[jira] [Updated] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

2023-08-01 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhao updated SPARK-44627: - Description: When the resultSet exists a timestmp column and it's value is null, In the row it generate

[jira] [Updated] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

2023-08-01 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhao updated SPARK-44627: - Attachment: image-2023-08-02-14-01-54-447.png > org.apache.spark.sql.execution.datasources.jdbc.JdbcUtil

[jira] [Updated] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

2023-08-01 Thread Min Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhao updated SPARK-44627: - Description: When the resultSet exists a timestmp column and it's value is null, In the row it generate

[jira] [Resolved] (SPARK-44555) Use checkError() to check Exception in command Suite & assign some error class names

2023-08-01 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-44555. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-44555) Use checkError() to check Exception in command Suite & assign some error class names

2023-08-01 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-44555: Assignee: BingKun Pan > Use checkError() to check Exception in command Suite & assign some error

[jira] [Created] (SPARK-44632) DiskBlockManager should check and be able to handle stale directories

2023-08-01 Thread Kent Yao (Jira)
Kent Yao created SPARK-44632: Summary: DiskBlockManager should check and be able to handle stale directories Key: SPARK-44632 URL: https://issues.apache.org/jira/browse/SPARK-44632 Project: Spark

[jira] [Created] (SPARK-44631) Remove session-based directory when the isolated session cache is evicted

2023-08-01 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-44631: Summary: Remove session-based directory when the isolated session cache is evicted Key: SPARK-44631 URL: https://issues.apache.org/jira/browse/SPARK-44631 Project: Sp

[jira] [Updated] (SPARK-44588) Migrated shuffle blocks are encrypted multiple times when io.encryption is enabled

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44588: -- Fix Version/s: 3.3.3 > Migrated shuffle blocks are encrypted multiple times when io.encryption

[jira] [Updated] (SPARK-44588) Migrated shuffle blocks are encrypted multiple times when io.encryption is enabled

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44588: -- Fix Version/s: 3.4.2 > Migrated shuffle blocks are encrypted multiple times when io.encryption

[jira] [Updated] (SPARK-44600) Make `repl` module daily test pass

2023-08-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44600: - Description: [https://github.com/apache/spark/actions/runs/5727123477/job/15518895421]   {code:java} -

[jira] [Resolved] (SPARK-44607) Remove unused function `containsNestedColumn` from `Filter`

2023-08-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-44607. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42239 [https://github.com

[jira] [Assigned] (SPARK-44607) Remove unused function `containsNestedColumn` from `Filter`

2023-08-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-44607: Assignee: Yang Jie > Remove unused function `containsNestedColumn` from `Filter` > --

[jira] [Created] (SPARK-44630) Revert SPARK-43043 Improve the performance of MapOutputTracker.updateMapOutput

2023-08-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-44630: - Summary: Revert SPARK-43043 Improve the performance of MapOutputTracker.updateMapOutput Key: SPARK-44630 URL: https://issues.apache.org/jira/browse/SPARK-44630 Proj

[jira] [Created] (SPARK-44629) Publish PySpark Test Guidelines webpage

2023-08-01 Thread Amanda Liu (Jira)
Amanda Liu created SPARK-44629: -- Summary: Publish PySpark Test Guidelines webpage Key: SPARK-44629 URL: https://issues.apache.org/jira/browse/SPARK-44629 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-43241) MultiIndex.append not checking names for equality

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43241: Affects Version/s: 4.0.0 (was: 3.5.0) > MultiIndex.append not checking

[jira] [Updated] (SPARK-42621) Add `inclusive` parameter for date_range

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42621: Affects Version/s: 4.0.0 (was: 3.5.0) > Add `inclusive` parameter for d

[jira] [Updated] (SPARK-42620) Add `inclusive` parameter for (DataFrame|Series).between_time

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42620: Affects Version/s: 4.0.0 (was: 3.5.0) > Add `inclusive` parameter for (

[jira] [Updated] (SPARK-43194) PySpark 3.4.0 cannot convert timestamp-typed objects to pandas with pandas 2.0

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43194: Affects Version/s: 4.0.0 (was: 3.4.0) > PySpark 3.4.0 cannot convert ti

[jira] [Updated] (SPARK-42619) Add `show_counts` parameter for DataFrame.info

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42619: Affects Version/s: 4.0.0 (was: 3.5.0) > Add `show_counts` parameter for

[jira] [Updated] (SPARK-42617) Support `isocalendar`

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42617: Affects Version/s: 4.0.0 (was: 3.5.0) > Support `isocalendar` > ---

[jira] [Updated] (SPARK-43271) Match behavior with DataFrame.reindex with specifying `index`.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43271: Affects Version/s: 4.0.0 (was: 3.5.0) > Match behavior with DataFrame.r

[jira] [Updated] (SPARK-43451) Enable RollingTests.test_rolling_count for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43451: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable RollingTests.test_rollin

[jira] [Updated] (SPARK-43282) Investigate DataFrame.sort_values with pandas behavior.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43282: Affects Version/s: 4.0.0 (was: 3.5.0) > Investigate DataFrame.sort_valu

[jira] [Updated] (SPARK-43245) Fix DatetimeIndex.microsecond to return 'int32' instead of 'int64' type of Index.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43245: Affects Version/s: 4.0.0 (was: 3.5.0) > Fix DatetimeIndex.microsecond t

[jira] [Updated] (SPARK-43433) Match `GroupBy.nth` behavior with new pandas behavior

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43433: Affects Version/s: 4.0.0 (was: 3.5.0) > Match `GroupBy.nth` behavior wi

[jira] [Updated] (SPARK-43432) Fix `min_periods` for Rolling to work same as pandas

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43432: Affects Version/s: 4.0.0 (was: 3.5.0) > Fix `min_periods` for Rolling t

[jira] [Updated] (SPARK-43291) Match behavior for DataFrame.cov on string DataFrame

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43291: Affects Version/s: 4.0.0 (was: 3.5.0) > Match behavior for DataFrame.co

[jira] [Updated] (SPARK-43295) Make DataFrameGroupBy.sum support for string type columns

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43295: Affects Version/s: 4.0.0 (was: 3.5.0) > Make DataFrameGroupBy.sum suppo

[jira] [Created] (SPARK-44628) Clear some unused codes in "***Errors" and extract some common logic

2023-08-01 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-44628: --- Summary: Clear some unused codes in "***Errors" and extract some common logic Key: SPARK-44628 URL: https://issues.apache.org/jira/browse/SPARK-44628 Project: Spark

[jira] [Updated] (SPARK-43460) Enable OpsOnDiffFramesGroupByTests.test_groupby_different_lengths for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43460: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable OpsOnDiffFramesGroupByTe

[jira] [Updated] (SPARK-43453) Enable OpsOnDiffFramesEnabledTests.test_concat_column_axis for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43453: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable OpsOnDiffFramesEnabledTe

[jira] [Updated] (SPARK-43459) Enable OpsOnDiffFramesGroupByTests.test_groupby_multiindex_columns for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43459: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable OpsOnDiffFramesGroupByTe

[jira] [Updated] (SPARK-43476) Enable SeriesStringTests.test_string_replace for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43476: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable SeriesStringTests.test_s

[jira] [Updated] (SPARK-43458) Enable SeriesConversionTests.test_to_latex for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43458: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable SeriesConversionTests.te

[jira] [Updated] (SPARK-43452) Enable RollingTests.test_groupby_rolling_count for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43452: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable RollingTests.test_groupb

[jira] [Updated] (SPARK-43462) Enable SeriesDateTimeTests.test_date_subtraction for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43462: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable SeriesDateTimeTests.test

[jira] [Updated] (SPARK-43477) Enable SeriesStringTests.test_string_rsplit for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43477: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable SeriesStringTests.test_s

[jira] [Updated] (SPARK-43497) Enable StatsTests.test_cov_corr_meta for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43497: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable StatsTests.test_cov_corr

[jira] [Updated] (SPARK-43498) Enable StatsTests.test_axis_on_dataframe for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43498: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable StatsTests.test_axis_on_

[jira] [Updated] (SPARK-43478) Enable SeriesStringTests.test_string_split for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43478: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable SeriesStringTests.test_s

[jira] [Updated] (SPARK-43506) Enable ArrowTests.test_toPandas_empty_columns for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43506: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable ArrowTests.test_toPandas

[jira] [Updated] (SPARK-43499) Enable StatsTests.test_stat_functions_with_no_numeric_columns for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43499: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable StatsTests.test_stat_fun

[jira] [Updated] (SPARK-43561) Enable DataFrameConversionTests.test_to_latex for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43561: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DataFrameConversionTests

[jira] [Updated] (SPARK-43562) Enable DataFrameTests.test_append for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43562: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DataFrameTests.test_appe

[jira] [Updated] (SPARK-43533) Enable MultiIndex test for IndexesTests.test_difference

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43533: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable MultiIndex test for Inde

[jira] [Updated] (SPARK-43563) Enable CsvTests.test_read_csv_with_squeeze for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43563: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable CsvTests.test_read_csv_w

[jira] [Updated] (SPARK-43570) Enable DateOpsTests.test_rsub for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43570: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DateOpsTests.test_rsub f

[jira] [Updated] (SPARK-43608) Enable IndexesTests.test_union for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43608: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable IndexesTests.test_union

[jira] [Updated] (SPARK-43705) Enable TimedeltaIndexTests.test_properties for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43705: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable TimedeltaIndexTests.test

[jira] [Updated] (SPARK-43644) Enable DatetimeIndexTests.test_indexer_between_time for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43644: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DatetimeIndexTests.test_

[jira] [Updated] (SPARK-43606) Enable IndexesTests.test_index_basic for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43606: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable IndexesTests.test_index_

[jira] [Updated] (SPARK-43567) Enable CategoricalIndexTests.test_factorize for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43567: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable CategoricalIndexTests.te

[jira] [Updated] (SPARK-43571) Enable DateOpsTests.test_sub for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43571: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DateOpsTests.test_sub fo

[jira] [Updated] (SPARK-43607) Enable IndexesTests.test_intersection for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43607: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable IndexesTests.test_inters

[jira] [Updated] (SPARK-43568) Enable CategoricalIndexTests.test_categories_setter for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43568: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable CategoricalIndexTests.te

[jira] [Updated] (SPARK-43633) Enable CategoricalIndexTests.test_remove_categories for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43633: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable CategoricalIndexTests.te

[jira] [Updated] (SPARK-43811) Enable DataFrameTests.test_reindex for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43811: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DataFrameTests.test_rein

[jira] [Updated] (SPARK-43869) Enable GroupBySlowTests for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43869: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable GroupBySlowTests for pan

[jira] [Updated] (SPARK-43709) Enable NamespaceTests.test_date_range for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43709: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable NamespaceTests.test_date

[jira] [Updated] (SPARK-43812) Enable DataFrameTests.test_all for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43812: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DataFrameTests.test_all

[jira] [Updated] (SPARK-43871) Enable SeriesDateTimeTests for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43871: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable SeriesDateTimeTests for

[jira] [Updated] (SPARK-43872) Enable DataFramePlotMatplotlibTests for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43872: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DataFramePlotMatplotlibT

[jira] [Updated] (SPARK-43873) Enable DataFrameSlowTests for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43873: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable DataFrameSlowTests for p

[jira] [Updated] (SPARK-43870) Enable SeriesTests for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43870: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable SeriesTests for pandas 2

[jira] [Updated] (SPARK-43874) Enable GroupByTests for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43874: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable GroupByTests for pandas

[jira] [Updated] (SPARK-43875) Enable CategoricalTests for pandas 2.0.0.

2023-08-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-43875: Affects Version/s: 4.0.0 (was: 3.5.0) > Enable CategoricalTests for pan

[jira] [Updated] (SPARK-44624) Spark Connect reattachable Execute when initial ExecutePlan didn't reach server

2023-08-01 Thread Juliusz Sompolski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-44624: -- Epic Link: SPARK-43754 > Spark Connect reattachable Execute when initial ExecutePlan d

[jira] [Updated] (SPARK-44624) Spark Connect reattachable Execute when initial ExecutePlan didn't reach server

2023-08-01 Thread Juliusz Sompolski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-44624: -- Description: If the ExecutePlan never reached the server, a ReattachExecute will fail

[jira] [Updated] (SPARK-44624) Spark Connect reattachable Execute when initial ExecutePlan didn't reach server

2023-08-01 Thread Juliusz Sompolski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-44624: -- Summary: Spark Connect reattachable Execute when initial ExecutePlan didn't reach serv

[jira] [Created] (SPARK-44627) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data

2023-08-01 Thread Min Zhao (Jira)
Min Zhao created SPARK-44627: Summary: org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils#resultSetToRows produces wrong data Key: SPARK-44627 URL: https://issues.apache.org/jira/browse/SPARK-44627

[jira] [Resolved] (SPARK-42941) Add support for streaming listener in Python

2023-08-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42941. -- Resolution: Fixed Issue resolved by pull request 42250 [https://github.com/apache/spark/pull/4

[jira] [Commented] (SPARK-42730) Update Spark Standalone Mode - Starting a Cluster Manually

2023-08-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750044#comment-17750044 ] Hyukjin Kwon commented on SPARK-42730: -- Please go ahead. Refs: [https://spark.apach

[jira] [Assigned] (SPARK-44218) Customize diff log in assertDataFrameEqual error message format

2023-08-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44218: Assignee: Amanda Liu > Customize diff log in assertDataFrameEqual error message format >

[jira] [Resolved] (SPARK-44218) Customize diff log in assertDataFrameEqual error message format

2023-08-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44218. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-44626) Followup on streaming query termination when client session is timed out for Spark Connect

2023-08-01 Thread Bo Gao (Jira)
Bo Gao created SPARK-44626: -- Summary: Followup on streaming query termination when client session is timed out for Spark Connect Key: SPARK-44626 URL: https://issues.apache.org/jira/browse/SPARK-44626 Projec

[jira] [Assigned] (SPARK-44588) Migrated shuffle blocks are encrypted multiple times when io.encryption is enabled

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44588: - Assignee: Henry Mai > Migrated shuffle blocks are encrypted multiple times when io.encr

[jira] [Resolved] (SPARK-44588) Migrated shuffle blocks are encrypted multiple times when io.encryption is enabled

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44588. --- Fix Version/s: 3.5.0 Resolution: Fixed > Migrated shuffle blocks are encrypted multip

[jira] [Resolved] (SPARK-44563) Upgrade Apache Arrow to 13.0.0

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44563. --- Resolution: Duplicate > Upgrade Apache Arrow to 13.0.0 > -- > >

[jira] [Closed] (SPARK-44563) Upgrade Apache Arrow to 13.0.0

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44563. - > Upgrade Apache Arrow to 13.0.0 > -- > > Key: SPARK-445

[jira] [Created] (SPARK-44625) Spark Connect clean up abandoned executions

2023-08-01 Thread Juliusz Sompolski (Jira)
Juliusz Sompolski created SPARK-44625: - Summary: Spark Connect clean up abandoned executions Key: SPARK-44625 URL: https://issues.apache.org/jira/browse/SPARK-44625 Project: Spark Issue T

[jira] [Assigned] (SPARK-44601) Make `hive-thriftserver` module daily test pass

2023-08-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-44601: Assignee: Yang Jie > Make `hive-thriftserver` module daily test pass > --

[jira] [Resolved] (SPARK-44601) Make `hive-thriftserver` module daily test pass

2023-08-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-44601. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-44624) Wrap retries around initial streaming GRPC call in connect

2023-08-01 Thread Juliusz Sompolski (Jira)
Juliusz Sompolski created SPARK-44624: - Summary: Wrap retries around initial streaming GRPC call in connect Key: SPARK-44624 URL: https://issues.apache.org/jira/browse/SPARK-44624 Project: Spark

[jira] [Assigned] (SPARK-44480) Add option for thread pool to perform maintenance for RocksDB/HDFS State Store Providers

2023-08-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-44480: Assignee: Eric Marnadi > Add option for thread pool to perform maintenance for RocksDB/HD

[jira] [Resolved] (SPARK-44480) Add option for thread pool to perform maintenance for RocksDB/HDFS State Store Providers

2023-08-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-44480. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42066 [https://gi

[jira] [Assigned] (SPARK-44623) Upgrade commons-lang3 to 3.13.0

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44623: - Assignee: Dongjoon Hyun > Upgrade commons-lang3 to 3.13.0 > ---

[jira] [Resolved] (SPARK-44623) Upgrade commons-lang3 to 3.13.0

2023-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44623. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42269 [https://

[jira] [Commented] (SPARK-29497) Cannot assign instance of java.lang.invoke.SerializedLambda to field

2023-08-01 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-29497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749991#comment-17749991 ] Herman van Hövell commented on SPARK-29497: --- I have added a check for this to

[jira] [Resolved] (SPARK-44613) Add Encoders.scala to Spark Connect Scala Client

2023-08-01 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44613. --- Fix Version/s: 3.5.0 Resolution: Fixed > Add Encoders.scala to Spark Connect

[jira] [Updated] (SPARK-44616) Hive Generic UDF support no longer supports short-circuiting of argument evaluation

2023-08-01 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-44616: --- Description: PR [https://github.com/apache/spark/pull/39555] changed DeferredObject to no longer co

  1   2   >