[jira] [Commented] (SPARK-36006) Migrate ALTER TABLE ADD/RENAME COLUMNS command to the new resolution framework

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373912#comment-17373912 ] Apache Spark commented on SPARK-36006: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-36006) Migrate ALTER TABLE ADD/RENAME COLUMNS command to the new resolution framework

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373911#comment-17373911 ] Apache Spark commented on SPARK-36006: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36006) Migrate ALTER TABLE ADD/RENAME COLUMNS command to the new resolution framework

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36006: Assignee: (was: Apache Spark) > Migrate ALTER TABLE ADD/RENAME COLUMNS command to

[jira] [Assigned] (SPARK-36006) Migrate ALTER TABLE ADD/RENAME COLUMNS command to the new resolution framework

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36006: Assignee: Apache Spark > Migrate ALTER TABLE ADD/RENAME COLUMNS command to the new

[jira] [Created] (SPARK-36006) Migrate ALTER TABLE ADD/RENAME COLUMNS command to the new resolution framework

2021-07-02 Thread Terry Kim (Jira)
Terry Kim created SPARK-36006: - Summary: Migrate ALTER TABLE ADD/RENAME COLUMNS command to the new resolution framework Key: SPARK-36006 URL: https://issues.apache.org/jira/browse/SPARK-36006 Project:

[jira] [Created] (SPARK-36005) in spark3.1.2 version The canCast method of type of char/varchar needs to be consistent with StringType

2021-07-02 Thread Sun BiaoBiao (Jira)
Sun BiaoBiao created SPARK-36005: Summary: in spark3.1.2 version The canCast method of type of char/varchar needs to be consistent with StringType Key: SPARK-36005 URL:

[jira] [Commented] (SPARK-36004) Update MiMa and audit Scala/Java API changes

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373859#comment-17373859 ] Apache Spark commented on SPARK-36004: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-36004) Update MiMa and audit Scala/Java API changes

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36004: Assignee: Apache Spark (was: Dongjoon Hyun) > Update MiMa and audit Scala/Java API

[jira] [Assigned] (SPARK-36004) Update MiMa and audit Scala/Java API changes

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36004: Assignee: Dongjoon Hyun (was: Apache Spark) > Update MiMa and audit Scala/Java API

[jira] [Assigned] (SPARK-36004) Update MiMa and audit API changes

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36004: - Assignee: Dongjoon Hyun > Update MiMa and audit API changes >

[jira] [Updated] (SPARK-36004) Update MiMa and audit Scala/Java API changes

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36004: -- Summary: Update MiMa and audit Scala/Java API changes (was: Update MiMa and audit API

[jira] [Updated] (SPARK-36004) Update MiMa and audit API changes

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36004: -- Target Version/s: 3.2.0 > Update MiMa and audit API changes >

[jira] [Created] (SPARK-36004) Update MiMa and audit API changes

2021-07-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-36004: - Summary: Update MiMa and audit API changes Key: SPARK-36004 URL: https://issues.apache.org/jira/browse/SPARK-36004 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-35974) Spark submit REST cluster/standalone mode - launching an s3a jar with STS

2021-07-02 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-35974: - Affects Version/s: (was: 2.4.6) 2.4.8 Description: {code:java}

[jira] [Reopened] (SPARK-35974) Spark submit REST cluster/standalone mode - launching an s3a jar with STS

2021-07-02 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reopened SPARK-35974: -- v2.4.8 is less than 2 months old > Spark submit REST cluster/standalone mode - launching an s3a jar with STS >

[jira] [Commented] (SPARK-35993) Flaky test: org.apache.spark.sql.execution.streaming.state.RocksDBSuite.ensure that concurrent update and cleanup consistent versions

2021-07-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373855#comment-17373855 ] Jungtaek Lim commented on SPARK-35993: -- Thanks for reporting [~gsomogyi]! The test is marked as

[jira] [Updated] (SPARK-35993) Flaky test: org.apache.spark.sql.execution.streaming.state.RocksDBSuite.ensure that concurrent update and cleanup consistent versions

2021-07-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-35993: - Affects Version/s: (was: 3.1.2) 3.2.0 > Flaky test: >

[jira] [Commented] (SPARK-33349) ExecutorPodsWatchSnapshotSource: Kubernetes client has been closed

2021-07-02 Thread Jim Kleckner (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373849#comment-17373849 ] Jim Kleckner commented on SPARK-33349: -- [~redsk] can you confirm that 

[jira] [Created] (SPARK-36003) Implement unary operator `invert` of numeric ps.Series/Index

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36003: Summary: Implement unary operator `invert` of numeric ps.Series/Index Key: SPARK-36003 URL: https://issues.apache.org/jira/browse/SPARK-36003 Project: Spark

[jira] [Commented] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

2021-07-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373839#comment-17373839 ] Xinrong Meng commented on SPARK-36002: -- How do you think about that? [~yikunkero] > Consolidate

[jira] [Created] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36002: Summary: Consolidate tests for data-type-based operations of decimal Series Key: SPARK-36002 URL: https://issues.apache.org/jira/browse/SPARK-36002 Project: Spark

[jira] [Created] (SPARK-36001) Assume result's index to be disordered in tests with operations on different Series

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36001: Summary: Assume result's index to be disordered in tests with operations on different Series Key: SPARK-36001 URL: https://issues.apache.org/jira/browse/SPARK-36001

[jira] [Created] (SPARK-36000) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36000: Summary: Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled Key: SPARK-36000 URL: https://issues.apache.org/jira/browse/SPARK-36000

[jira] [Resolved] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35996. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33196

[jira] [Assigned] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35996: - Assignee: Dongjoon Hyun > Setting version to 3.3.0-SNAPSHOT >

[jira] [Commented] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373753#comment-17373753 ] Apache Spark commented on SPARK-35995: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373754#comment-17373754 ] Apache Spark commented on SPARK-35995: -- User 'ueshin' has created a pull request for this issue:

[jira] [Created] (SPARK-35999) Make from_csv/to_csv to handle day-time intervals properly

2021-07-02 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35999: -- Summary: Make from_csv/to_csv to handle day-time intervals properly Key: SPARK-35999 URL: https://issues.apache.org/jira/browse/SPARK-35999 Project: Spark

[jira] [Created] (SPARK-35998) Make from_csv/to_csv to handle year-month intervals properly

2021-07-02 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35998: -- Summary: Make from_csv/to_csv to handle year-month intervals properly Key: SPARK-35998 URL: https://issues.apache.org/jira/browse/SPARK-35998 Project: Spark

[jira] [Created] (SPARK-35997) Implement comparison operators for CategoricalDtype in pandas API on Spark

2021-07-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35997: Summary: Implement comparison operators for CategoricalDtype in pandas API on Spark Key: SPARK-35997 URL: https://issues.apache.org/jira/browse/SPARK-35997 Project:

[jira] [Commented] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373714#comment-17373714 ] Apache Spark commented on SPARK-35996: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373712#comment-17373712 ] Apache Spark commented on SPARK-35996: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35996: Assignee: Apache Spark > Setting version to 3.3.0-SNAPSHOT >

[jira] [Assigned] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35996: Assignee: (was: Apache Spark) > Setting version to 3.3.0-SNAPSHOT >

[jira] [Created] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35996: - Summary: Setting version to 3.3.0-SNAPSHOT Key: SPARK-35996 URL: https://issues.apache.org/jira/browse/SPARK-35996 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-35990) Remove avro-sbt plugin dependency

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35990. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33190

[jira] [Resolved] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35995. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33194

[jira] [Assigned] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35995: - Assignee: Dongjoon Hyun > Enable GitHub Action build_and_test on branch-3.2 >

[jira] [Updated] (SPARK-35994) Publish snapshot from branch-3.2

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35994: -- Fix Version/s: (was: 3.3.0) 3.2.0 > Publish snapshot from branch-3.2 >

[jira] [Resolved] (SPARK-35994) Publish snapshot from branch-3.2

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35994. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33192

[jira] [Assigned] (SPARK-35994) Publish snapshot from branch-3.2

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35994: - Assignee: Dongjoon Hyun > Publish snapshot from branch-3.2 >

[jira] [Commented] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373694#comment-17373694 ] Apache Spark commented on SPARK-35995: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-35785) Cleanup support for RocksDB instance

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373695#comment-17373695 ] Apache Spark commented on SPARK-35785: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-35981) Use check_exact=False in StatsTest.test_cov_corr_meta to loosen the check precision

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373693#comment-17373693 ] Apache Spark commented on SPARK-35981: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35995: Assignee: (was: Apache Spark) > Enable GitHub Action build_and_test on branch-3.2 >

[jira] [Assigned] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35995: Assignee: Apache Spark > Enable GitHub Action build_and_test on branch-3.2 >

[jira] [Commented] (SPARK-35981) Use check_exact=False in StatsTest.test_cov_corr_meta to loosen the check precision

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373691#comment-17373691 ] Apache Spark commented on SPARK-35981: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373692#comment-17373692 ] Apache Spark commented on SPARK-35995: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-35995) Enable GitHub Action build_and_test on branch-3.2

2021-07-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35995: - Summary: Enable GitHub Action build_and_test on branch-3.2 Key: SPARK-35995 URL: https://issues.apache.org/jira/browse/SPARK-35995 Project: Spark Issue

[jira] [Assigned] (SPARK-35994) Publish snapshot from branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35994: Assignee: Apache Spark > Publish snapshot from branch-3.2 >

[jira] [Assigned] (SPARK-35994) Publish snapshot from branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35994: Assignee: (was: Apache Spark) > Publish snapshot from branch-3.2 >

[jira] [Commented] (SPARK-35994) Publish snapshot from branch-3.2

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373665#comment-17373665 ] Apache Spark commented on SPARK-35994: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-35994) Publish snapshot from branch-3.2

2021-07-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35994: - Summary: Publish snapshot from branch-3.2 Key: SPARK-35994 URL: https://issues.apache.org/jira/browse/SPARK-35994 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35992. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33189

[jira] [Assigned] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35992: - Assignee: Dongjoon Hyun > Upgrade ORC to 1.6.9 > > >

[jira] [Assigned] (SPARK-35985) File source V2 ignores partition filters when empty readDataSchema

2021-07-02 Thread Apache Spark (Jira)
ema}} of the {{FileScan}} is empty, partition > filters are not pushed down and all data is scanned. > Some examples where this happens: > {code:java} > scala> spark.sql("SELECT count(*) FROM parq WHERE day=20210702").explain > == Physical Plan == > *(2

[jira] [Assigned] (SPARK-35985) File source V2 ignores partition filters when empty readDataSchema

2021-07-02 Thread Apache Spark (Jira)
So when the {{readDataSchema}} of the {{FileScan}} is empty, partition > filters are not pushed down and all data is scanned. > Some examples where this happens: > {code:java} > scala> spark.sql("SELECT count(*) FROM parq WHERE day=20210702").explain > == Physic

[jira] [Commented] (SPARK-35985) File source V2 ignores partition filters when empty readDataSchema

2021-07-02 Thread Apache Spark (Jira)
only wants to know > how many entries there are, and is not interested of their context. > So when the {{readDataSchema}} of the {{FileScan}} is empty, partition > filters are not pushed down and all data is scanned. > Some examples where this happens: > {code:java} > scala> spark.sql("SELECT c

[jira] [Commented] (SPARK-34883) Setting CSV reader option "multiLine" to "true" causes URISyntaxException when colon is in file path

2021-07-02 Thread Mike Pieters (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373552#comment-17373552 ] Mike Pieters commented on SPARK-34883: -- I've got the same error here when I try to run: {code:java}

[jira] [Commented] (SPARK-23814) Couldn't read file with colon in name and new line character in one of the field.

2021-07-02 Thread Mike Pieters (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373526#comment-17373526 ] Mike Pieters commented on SPARK-23814: -- I also got the same error in version 3.0.1 > Couldn't read

[jira] [Created] (SPARK-35993) Flaky test: org.apache.spark.sql.execution.streaming.state.RocksDBSuite.ensure that concurrent update and cleanup consistent versions

2021-07-02 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-35993: - Summary: Flaky test: org.apache.spark.sql.execution.streaming.state.RocksDBSuite.ensure that concurrent update and cleanup consistent versions Key: SPARK-35993 URL:

[jira] [Commented] (SPARK-35990) Remove avro-sbt plugin dependency

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373443#comment-17373443 ] Apache Spark commented on SPARK-35990: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35990) Remove avro-sbt plugin dependency

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35990: Assignee: Apache Spark (was: Kousuke Saruta) > Remove avro-sbt plugin dependency >

[jira] [Commented] (SPARK-35990) Remove avro-sbt plugin dependency

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373442#comment-17373442 ] Apache Spark commented on SPARK-35990: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35990) Remove avro-sbt plugin dependency

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35990: Assignee: Kousuke Saruta (was: Apache Spark) > Remove avro-sbt plugin dependency >

[jira] [Updated] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35992: -- Description: This issue aims to upgrade Apache ORC to 1.6.9 to bring ORC encryption masking

[jira] [Updated] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35992: -- Priority: Critical (was: Major) > Upgrade ORC to 1.6.9 > > >

[jira] [Assigned] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35992: Assignee: (was: Apache Spark) > Upgrade ORC to 1.6.9 > > >

[jira] [Assigned] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35992: Assignee: Apache Spark > Upgrade ORC to 1.6.9 > > >

[jira] [Commented] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373439#comment-17373439 ] Apache Spark commented on SPARK-35992: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-35992) Upgrade ORC to 1.6.9

2021-07-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35992: - Summary: Upgrade ORC to 1.6.9 Key: SPARK-35992 URL: https://issues.apache.org/jira/browse/SPARK-35992 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-35991) Add PlanStability suite for TPCH

2021-07-02 Thread angerszhu (Jira)
angerszhu created SPARK-35991: - Summary: Add PlanStability suite for TPCH Key: SPARK-35991 URL: https://issues.apache.org/jira/browse/SPARK-35991 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-35990) Remove avro-sbt plugin dependency

2021-07-02 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35990: -- Summary: Remove avro-sbt plugin dependency Key: SPARK-35990 URL: https://issues.apache.org/jira/browse/SPARK-35990 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34632) Can we create 'SessionState' with a username in 'HiveClientImpl'

2021-07-02 Thread HonglunChen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373428#comment-17373428 ] HonglunChen commented on SPARK-34632: - Yes, we can do that. I just want Spark to support this by

[jira] [Commented] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373408#comment-17373408 ] Apache Spark commented on SPARK-35989: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35989: Assignee: Apache Spark > Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled >

[jira] [Assigned] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35989: Assignee: (was: Apache Spark) > Do not remove REPARTITION_BY_NUM shuffle if AQE is

[jira] [Commented] (SPARK-34632) Can we create 'SessionState' with a username in 'HiveClientImpl'

2021-07-02 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373410#comment-17373410 ] dzcxzl commented on SPARK-34632: You can use the default Authenticator to get the username through ugi.

[jira] [Updated] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35989: -- Description: The shuffle origin is `REPARTITION_BY_NUM` if user specify an exact partition number

[jira] [Updated] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35989: -- Environment: (was: The shuffle origin is `REPARTITION_BY_NUM` if user specify an exact partition

[jira] [Created] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread XiDuo You (Jira)
XiDuo You created SPARK-35989: - Summary: Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled Key: SPARK-35989 URL: https://issues.apache.org/jira/browse/SPARK-35989 Project: Spark Issue

[jira] [Assigned] (SPARK-35988) The implementation for RocksDBStateStoreProvider

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35988: Assignee: (was: Apache Spark) > The implementation for RocksDBStateStoreProvider >

[jira] [Commented] (SPARK-35988) The implementation for RocksDBStateStoreProvider

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373396#comment-17373396 ] Apache Spark commented on SPARK-35988: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-35988) The implementation for RocksDBStateStoreProvider

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35988: Assignee: Apache Spark > The implementation for RocksDBStateStoreProvider >

[jira] [Commented] (SPARK-35988) The implementation for RocksDBStateStoreProvider

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373397#comment-17373397 ] Apache Spark commented on SPARK-35988: -- User 'xuanyuanking' has created a pull request for this

[jira] [Created] (SPARK-35988) The implementation for RocksDBStateStoreProvider

2021-07-02 Thread Yuanjian Li (Jira)
Yuanjian Li created SPARK-35988: --- Summary: The implementation for RocksDBStateStoreProvider Key: SPARK-35988 URL: https://issues.apache.org/jira/browse/SPARK-35988 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35987) The ANSI flags of Sum and Avg should be kept after being copied

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373365#comment-17373365 ] Apache Spark commented on SPARK-35987: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-35987) The ANSI flags of Sum and Avg should be kept after being copied

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35987: Assignee: Apache Spark (was: Gengliang Wang) > The ANSI flags of Sum and Avg should be

[jira] [Commented] (SPARK-35987) The ANSI flags of Sum and Avg should be kept after being copied

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373364#comment-17373364 ] Apache Spark commented on SPARK-35987: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-35987) The ANSI flags of Sum and Avg should be kept after being copied

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35987: Assignee: Gengliang Wang (was: Apache Spark) > The ANSI flags of Sum and Avg should be

[jira] [Created] (SPARK-35987) The ANSI flags of Sum and Avg should be kept after being copied

2021-07-02 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35987: -- Summary: The ANSI flags of Sum and Avg should be kept after being copied Key: SPARK-35987 URL: https://issues.apache.org/jira/browse/SPARK-35987 Project: Spark

[jira] [Resolved] (SPARK-35981) Use check_exact=False in StatsTest.test_cov_corr_meta to loosen the check precision

2021-07-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35981. -- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-35986) fix pyspark.rdd.RDD.histogram's buckets argument

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35986: Assignee: (was: Apache Spark) > fix pyspark.rdd.RDD.histogram's buckets argument >

[jira] [Assigned] (SPARK-35986) fix pyspark.rdd.RDD.histogram's buckets argument

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35986: Assignee: Apache Spark > fix pyspark.rdd.RDD.histogram's buckets argument >

[jira] [Commented] (SPARK-35986) fix pyspark.rdd.RDD.histogram's buckets argument

2021-07-02 Thread Tomas Pereira de Vasconcelos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373352#comment-17373352 ] Tomas Pereira de Vasconcelos commented on SPARK-35986: -- Pull request: 

[jira] [Commented] (SPARK-35986) fix pyspark.rdd.RDD.histogram's buckets argument

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373353#comment-17373353 ] Apache Spark commented on SPARK-35986: -- User 'tpvasconcelos' has created a pull request for this

[jira] [Updated] (SPARK-35986) fix pyspark.rdd.RDD.histogram's buckets argument

2021-07-02 Thread Tomas Pereira de Vasconcelos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomas Pereira de Vasconcelos updated SPARK-35986: - Description: I originally opened an issue and created a PR in

[jira] [Created] (SPARK-35986) fix pyspark.rdd.RDD.histogram's buckets argument

2021-07-02 Thread Tomas Pereira de Vasconcelos (Jira)
Tomas Pereira de Vasconcelos created SPARK-35986: Summary: fix pyspark.rdd.RDD.histogram's buckets argument Key: SPARK-35986 URL: https://issues.apache.org/jira/browse/SPARK-35986

[jira] [Updated] (SPARK-35785) Cleanup support for RocksDB instance

2021-07-02 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-35785: - Fix Version/s: 3.2.0 > Cleanup support for RocksDB instance >

[jira] [Commented] (SPARK-35785) Cleanup support for RocksDB instance

2021-07-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373319#comment-17373319 ] Apache Spark commented on SPARK-35785: -- User 'xuanyuanking' has created a pull request for this

  1   2   >