[jira] [Resolved] (SPARK-40071) Update plugins to latest versions

2022-08-13 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40071. -- Assignee: BingKun Pan Resolution: Fixed Resolved by

[jira] [Resolved] (SPARK-40047) Exclude unused xalan transitive dependency from htmlunit

2022-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40047. --- Resolution: Invalid It turns out the previous exclusion was incomplete in SBT and

[jira] [Closed] (SPARK-40047) Exclude unused xalan transitive dependency from htmlunit

2022-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-40047. - > Exclude unused xalan transitive dependency from htmlunit >

[jira] [Updated] (SPARK-40047) Exclude unused xalan transitive dependency from htmlunit

2022-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40047: -- Fix Version/s: (was: 3.4.0) (was: 3.3.1)

[jira] [Reopened] (SPARK-40047) Exclude unused xalan transitive dependency from htmlunit

2022-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-40047: --- Assignee: (was: Yang Jie) > Exclude unused xalan transitive dependency from htmlunit

[jira] [Commented] (SPARK-40047) Exclude unused xalan transitive dependency from htmlunit

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579271#comment-17579271 ] Apache Spark commented on SPARK-40047: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-40001) Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS

2022-08-13 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-40001: -- Assignee: Daniel > Add config to make DEFAULT values in JSON tables mutually

[jira] [Resolved] (SPARK-40001) Add config to make DEFAULT values in JSON tables mutually exclusive with SQLConf.JSON_GENERATOR_IGNORE_NULL_FIELDS

2022-08-13 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-40001. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37431

[jira] [Assigned] (SPARK-39926) Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans

2022-08-13 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-39926: -- Assignee: Daniel > Fix bug in existence DEFAULT value lookups for non-vectorized

[jira] [Resolved] (SPARK-39926) Fix bug in existence DEFAULT value lookups for non-vectorized Parquet scans

2022-08-13 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-39926. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37501

[jira] [Assigned] (SPARK-40070) Upgrade Apache ZooKeeper to 3.8.0

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40070: Assignee: (was: Apache Spark) > Upgrade Apache ZooKeeper to 3.8.0 >

[jira] [Commented] (SPARK-40070) Upgrade Apache ZooKeeper to 3.8.0

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579245#comment-17579245 ] Apache Spark commented on SPARK-40070: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-40070) Upgrade Apache ZooKeeper to 3.8.0

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40070: Assignee: Apache Spark > Upgrade Apache ZooKeeper to 3.8.0 >

[jira] [Updated] (SPARK-40071) Update plugins to latest versions

2022-08-13 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-40071: Description: include: # maven-compiler-plugin (from 3.8.1 to 3.10.1) # maven-jar-plugin (from

[jira] [Commented] (SPARK-40071) Update plugins to latest versions

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579240#comment-17579240 ] Apache Spark commented on SPARK-40071: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-40071) Update plugins to latest versions

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40071: Assignee: Apache Spark > Update plugins to latest versions >

[jira] [Assigned] (SPARK-40071) Update plugins to latest versions

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40071: Assignee: (was: Apache Spark) > Update plugins to latest versions >

[jira] [Commented] (SPARK-40071) Update plugins to latest versions

2022-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579239#comment-17579239 ] Apache Spark commented on SPARK-40071: -- User 'panbingkun' has created a pull request for this

[jira] [Created] (SPARK-40071) Update plugins to latest versions

2022-08-13 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-40071: --- Summary: Update plugins to latest versions Key: SPARK-40071 URL: https://issues.apache.org/jira/browse/SPARK-40071 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-40070) Upgrade Apache ZooKeeper to 3.8.0

2022-08-13 Thread Jira
Bjørn Jørgensen created SPARK-40070: --- Summary: Upgrade Apache ZooKeeper to 3.8.0 Key: SPARK-40070 URL: https://issues.apache.org/jira/browse/SPARK-40070 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40070) Upgrade Apache ZooKeeper to 3.8.0

2022-08-13 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-40070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-40070: Priority: Minor (was: Major) > Upgrade Apache ZooKeeper to 3.8.0 >

[jira] [Updated] (SPARK-40048) Cached partitions are traversed multiple times (invalidating Accumulator consistency)

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-40048: Summary: Cached partitions are traversed multiple times (invalidating Accumulator consistency) (was: Partitions

[jira] [Updated] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-40048: Description: We are trying to use Accumulators to count RDDs without having to force `.count()` on them for

[jira] [Updated] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-40048: Description: We are trying to use Accumulators to count RDDs without having to force `.count()` on them for

[jira] [Updated] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-40048: Description: We are trying to use Accumulators to count RDDs without having to force `.count()` on them for

[jira] [Updated] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-40048: Affects Version/s: 3.2.1 > Partitions are traversed multiple times invalidating Accumulator consistency >

[jira] [Commented] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579231#comment-17579231 ] sam commented on SPARK-40048: - [~hyukjin.kwon] Unfortunately bumping to 3.2.1 did not fix the issue: ```

[jira] [Comment Edited] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579229#comment-17579229 ] sam edited comment on SPARK-40048 at 8/13/22 9:30 AM: -- We tried `3.2.1` and I'm now

[jira] [Commented] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579229#comment-17579229 ] sam commented on SPARK-40048: - We tried `3.2.1` and I'm now looking at

[jira] [Commented] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579226#comment-17579226 ] sam commented on SPARK-40048: - Thanks [~hyukjin.kwon], but we hit a number of issues trying to bump Spark

[jira] [Updated] (SPARK-40048) Partitions are traversed multiple times invalidating Accumulator consistency

2022-08-13 Thread sam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam updated SPARK-40048: Description: We are trying to use Accumulators to count RDDs without having to force `.count()` on them for

[jira] [Commented] (SPARK-40062) Spark - Creating Sub Folder while writing to Partitioned Hive Table

2022-08-13 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579217#comment-17579217 ] ming95 commented on SPARK-40062: I can't reproduce the problem. Can you give the specific