[jira] [Updated] (SPARK-34238) Unify output of SHOW PARTITIONS and pass output attributes

2021-01-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-34238: -- Summary: Unify output of SHOW PARTITIONS and pass output attributes (was: Unify output of SHOW PARTIT

[jira] [Created] (SPARK-34238) Unify output of SHOW PARTITIONS

2021-01-25 Thread angerszhu (Jira)
angerszhu created SPARK-34238: - Summary: Unify output of SHOW PARTITIONS Key: SPARK-34238 URL: https://issues.apache.org/jira/browse/SPARK-34238 Project: Spark Issue Type: Sub-task Comp

[jira] [Assigned] (SPARK-34237) Add more metrics (fallback, spill) for object hash aggregate

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34237: Assignee: (was: Apache Spark) > Add more metrics (fallback, spill) for object hash ag

[jira] [Assigned] (SPARK-34237) Add more metrics (fallback, spill) for object hash aggregate

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34237: Assignee: Apache Spark > Add more metrics (fallback, spill) for object hash aggregate > -

[jira] [Commented] (SPARK-34237) Add more metrics (fallback, spill) for object hash aggregate

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271908#comment-17271908 ] Apache Spark commented on SPARK-34237: -- User 'c21' has created a pull request for t

[jira] [Commented] (SPARK-34236) v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271906#comment-17271906 ] Apache Spark commented on SPARK-34236: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-34236) v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34236: Assignee: (was: Apache Spark) > v2 Overwrite w/ null static partition raise Cannot tr

[jira] [Assigned] (SPARK-34236) v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34236: Assignee: Apache Spark > v2 Overwrite w/ null static partition raise Cannot translate exp

[jira] [Commented] (SPARK-34236) v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271905#comment-17271905 ] Apache Spark commented on SPARK-34236: -- User 'yaooqinn' has created a pull request

[jira] [Created] (SPARK-34237) Add more metrics (fallback, spill) for object hash aggregate

2021-01-25 Thread Cheng Su (Jira)
Cheng Su created SPARK-34237: Summary: Add more metrics (fallback, spill) for object hash aggregate Key: SPARK-34237 URL: https://issues.apache.org/jira/browse/SPARK-34237 Project: Spark Issue T

[jira] [Updated] (SPARK-34236) v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

2021-01-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34236: - Summary: v2 Overwrite w/ null static partition raise Cannot translate expression to source filter: null

[jira] [Created] (SPARK-34236) v2 Overwrite will null static partition raise Cannot translate expression to source filter: null

2021-01-25 Thread Kent Yao (Jira)
Kent Yao created SPARK-34236: Summary: v2 Overwrite will null static partition raise Cannot translate expression to source filter: null Key: SPARK-34236 URL: https://issues.apache.org/jira/browse/SPARK-34236

[jira] [Commented] (SPARK-34218) Add Scala 2.13 packaging and publishing

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271894#comment-17271894 ] Apache Spark commented on SPARK-34218: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-34218) Add Scala 2.13 packaging and publishing

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271893#comment-17271893 ] Apache Spark commented on SPARK-34218: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34235) Make spark.sql.hive as a private package

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34235: Assignee: (was: Apache Spark) > Make spark.sql.hive as a private package > --

[jira] [Assigned] (SPARK-34235) Make spark.sql.hive as a private package

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34235: Assignee: Apache Spark > Make spark.sql.hive as a private package > -

[jira] [Commented] (SPARK-34235) Make spark.sql.hive as a private package

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271887#comment-17271887 ] Apache Spark commented on SPARK-34235: -- User 'xuanyuanking' has created a pull requ

[jira] [Created] (SPARK-34235) Make spark.sql.hive as a private package

2021-01-25 Thread Yuanjian Li (Jira)
Yuanjian Li created SPARK-34235: --- Summary: Make spark.sql.hive as a private package Key: SPARK-34235 URL: https://issues.apache.org/jira/browse/SPARK-34235 Project: Spark Issue Type: Improvemen

[jira] [Resolved] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34229. -- Fix Version/s: 3.1.2 2.4.8 3.0.2 Resolution: Fixed

[jira] [Updated] (SPARK-34232) redact credentials not working when log slow event enabled

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34232: -- Summary: redact credentials not working when log slow event enabled (was: [CORE] redact crede

[jira] [Assigned] (SPARK-34223) NPE for static partion with null in InsertIntoHadoopFsRelationCommand

2021-01-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34223: --- Assignee: Kent Yao > NPE for static partion with null in InsertIntoHadoopFsRelationCommand

[jira] [Resolved] (SPARK-34223) NPE for static partion with null in InsertIntoHadoopFsRelationCommand

2021-01-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34223. - Fix Version/s: 3.0.2 3.1.1 Resolution: Fixed Issue resolved by pull re

[jira] [Updated] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-34203: Fix Version/s: 3.0.2 > FileSource table null partition can not be dropped > --

[jira] [Assigned] (SPARK-34234) Remove TreeNodeException that didn't work

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34234: Assignee: (was: Apache Spark) > Remove TreeNodeException that didn't work > -

[jira] [Assigned] (SPARK-34234) Remove TreeNodeException that didn't work

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34234: Assignee: Apache Spark > Remove TreeNodeException that didn't work >

[jira] [Commented] (SPARK-34234) Remove TreeNodeException that didn't work

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271793#comment-17271793 ] Apache Spark commented on SPARK-34234: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-34233) NPE for char padding in binary comparison

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271791#comment-17271791 ] Apache Spark commented on SPARK-34233: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-34233) NPE for char padding in binary comparison

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34233: Assignee: Apache Spark > NPE for char padding in binary comparison >

[jira] [Commented] (SPARK-34233) NPE for char padding in binary comparison

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271789#comment-17271789 ] Apache Spark commented on SPARK-34233: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-34233) NPE for char padding in binary comparison

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34233: Assignee: (was: Apache Spark) > NPE for char padding in binary comparison > -

[jira] [Updated] (SPARK-34234) Remove TreeNodeException that didn't work

2021-01-25 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-34234: --- Description: TreeNodeException causes the error msg not clear and it didn't work well. Because the T

[jira] [Created] (SPARK-34234) Remove TreeNodeException that didn't work

2021-01-25 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-34234: -- Summary: Remove TreeNodeException that didn't work Key: SPARK-34234 URL: https://issues.apache.org/jira/browse/SPARK-34234 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-34233) NPE for char padding in binary comparison

2021-01-25 Thread Kent Yao (Jira)
Kent Yao created SPARK-34233: Summary: NPE for char padding in binary comparison Key: SPARK-34233 URL: https://issues.apache.org/jira/browse/SPARK-34233 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-34232) [CORE] redact credentials not working when log slow event enabled

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271773#comment-17271773 ] Apache Spark commented on SPARK-34232: -- User 'warrenzhu25' has created a pull reque

[jira] [Assigned] (SPARK-34232) [CORE] redact credentials not working when log slow event enabled

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34232: Assignee: (was: Apache Spark) > [CORE] redact credentials not working when log slow e

[jira] [Assigned] (SPARK-34232) [CORE] redact credentials not working when log slow event enabled

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34232: Assignee: Apache Spark > [CORE] redact credentials not working when log slow event enable

[jira] [Commented] (SPARK-31768) add getMetrics in Evaluators

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271772#comment-17271772 ] Apache Spark commented on SPARK-31768: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-31768) add getMetrics in Evaluators

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271771#comment-17271771 ] Apache Spark commented on SPARK-31768: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-34232) [CORE] redact credentials not working when log slow event enabled

2021-01-25 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-34232: Summary: [CORE] redact credentials not working when log slow event enabled Key: SPARK-34232 URL: https://issues.apache.org/jira/browse/SPARK-34232 Project: Spark

[jira] [Resolved] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34203. --- Fix Version/s: 3.1.2 3.2.0 Assignee: Maxim Gekk Resolution

[jira] [Commented] (SPARK-34182) [AVRO] Improve error messages when matching Catalyst-to-Avro schemas

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271715#comment-17271715 ] Apache Spark commented on SPARK-34182: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-34182) [AVRO] Improve error messages when matching Catalyst-to-Avro schemas

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34182: Assignee: (was: Apache Spark) > [AVRO] Improve error messages when matching Catalyst-

[jira] [Assigned] (SPARK-34182) [AVRO] Improve error messages when matching Catalyst-to-Avro schemas

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34182: Assignee: Apache Spark > [AVRO] Improve error messages when matching Catalyst-to-Avro sch

[jira] [Updated] (SPARK-34187) Use available offset range obtained during polling when checking offset validation

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34187: -- Fix Version/s: 2.4.8 > Use available offset range obtained during polling when checking offset

[jira] [Created] (SPARK-34231) [AVRO][TEST] AvroSuite has test failure when run from IDE due to bad loading of resource file

2021-01-25 Thread Erik Krogen (Jira)
Erik Krogen created SPARK-34231: --- Summary: [AVRO][TEST] AvroSuite has test failure when run from IDE due to bad loading of resource file Key: SPARK-34231 URL: https://issues.apache.org/jira/browse/SPARK-34231

[jira] [Commented] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271688#comment-17271688 ] Apache Spark commented on SPARK-34203: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-34187) Use available offset range obtained during polling when checking offset validation

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271687#comment-17271687 ] Apache Spark commented on SPARK-34187: -- User 'viirya' has created a pull request fo

[jira] [Updated] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34229: -- Target Version/s: 2.4.8, 3.0.2, 3.1.1 (was: 3.0.2, 3.1.1) > Avro should read decimal values w

[jira] [Updated] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34229: -- Affects Version/s: 2.4.7 > Avro should read decimal values with the file schema >

[jira] [Created] (SPARK-34230) Let AQE determine the right parallelism in DistributionAndOrderingUtils

2021-01-25 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-34230: Summary: Let AQE determine the right parallelism in DistributionAndOrderingUtils Key: SPARK-34230 URL: https://issues.apache.org/jira/browse/SPARK-34230 Proje

[jira] [Updated] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34229: -- Description: {code:java} scala> sql("SELECT 3.14 a").write.format("avro").save("/tmp/avro") sc

[jira] [Commented] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271665#comment-17271665 ] Apache Spark commented on SPARK-34229: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34229: Assignee: Dongjoon Hyun (was: Apache Spark) > Avro should read decimal values with the f

[jira] [Assigned] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34229: Assignee: Apache Spark (was: Dongjoon Hyun) > Avro should read decimal values with the f

[jira] [Created] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34229: - Summary: Avro should read decimal values with the file schema Key: SPARK-34229 URL: https://issues.apache.org/jira/browse/SPARK-34229 Project: Spark Issue

[jira] [Assigned] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34229: - Target Version/s: 3.0.2, 3.1.1 Assignee: Dongjoon Hyun > Avro should read d

[jira] [Updated] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34229: -- Priority: Blocker (was: Major) > Avro should read decimal values with the file schema > -

[jira] [Updated] (SPARK-34229) Avro should read decimal values with the file schema

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34229: -- Labels: correctness (was: ) > Avro should read decimal values with the file schema >

[jira] [Updated] (SPARK-34187) Use available offset range obtained during polling when checking offset validation

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34187: -- Fix Version/s: (was: 3.1.1) 3.1.2 3.0.2 > Use availa

[jira] [Commented] (SPARK-34184) Remove redundant repartition nodes in the optimizer

2021-01-25 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271632#comment-17271632 ] Anton Okolnychyi commented on SPARK-34184: -- {code:java} ignore("ordered dist

[jira] [Commented] (SPARK-34187) Use available offset range obtained during polling when checking offset validation

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271603#comment-17271603 ] Apache Spark commented on SPARK-34187: -- User 'viirya' has created a pull request fo

[jira] [Created] (SPARK-34228) Create tunable fuzzy logic for dynamic allocation

2021-01-25 Thread Holden Karau (Jira)
Holden Karau created SPARK-34228: Summary: Create tunable fuzzy logic for dynamic allocation Key: SPARK-34228 URL: https://issues.apache.org/jira/browse/SPARK-34228 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33726) Duplicate field names causes wrong answers during aggregation

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271600#comment-17271600 ] Apache Spark commented on SPARK-33726: -- User 'yliou' has created a pull request for

[jira] [Commented] (SPARK-33726) Duplicate field names causes wrong answers during aggregation

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271599#comment-17271599 ] Apache Spark commented on SPARK-33726: -- User 'yliou' has created a pull request for

[jira] [Resolved] (SPARK-34192) Move char padding to write side

2021-01-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34192. - Fix Version/s: 3.1.1 Assignee: Kent Yao Resolution: Fixed > Move char padding to

[jira] [Commented] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271595#comment-17271595 ] Apache Spark commented on SPARK-34203: -- User 'MaxGekk' has created a pull request f

[jira] [Updated] (SPARK-34212) For parquet table, after changing the precision and scale of decimal type in hive, spark reads incorrect value

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34212: -- Target Version/s: 3.0.2, 3.1.1 (was: 3.1.1) > For parquet table, after changing the precision

[jira] [Updated] (SPARK-34212) For parquet table, after changing the precision and scale of decimal type in hive, spark reads incorrect value

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34212: -- Target Version/s: 3.1.1 > For parquet table, after changing the precision and scale of decimal

[jira] [Updated] (SPARK-34212) For parquet table, after changing the precision and scale of decimal type in hive, spark reads incorrect value

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34212: -- Priority: Blocker (was: Major) > For parquet table, after changing the precision and scale of

[jira] [Assigned] (SPARK-34218) Add Scala 2.13 packaging and publishing

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34218: - Assignee: Dongjoon Hyun > Add Scala 2.13 packaging and publishing > ---

[jira] [Resolved] (SPARK-34218) Add Scala 2.13 packaging and publishing

2021-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34218. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31311 [https://

[jira] [Assigned] (SPARK-34227) WindowFunctionFrame should clear its states during preparation

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34227: Assignee: Wenchen Fan (was: Apache Spark) > WindowFunctionFrame should clear its states

[jira] [Commented] (SPARK-34227) WindowFunctionFrame should clear its states during preparation

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271366#comment-17271366 ] Apache Spark commented on SPARK-34227: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-34227) WindowFunctionFrame should clear its states during preparation

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34227: Assignee: Apache Spark (was: Wenchen Fan) > WindowFunctionFrame should clear its states

[jira] [Created] (SPARK-34227) WindowFunctionFrame should clear its states during preparation

2021-01-25 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-34227: --- Summary: WindowFunctionFrame should clear its states during preparation Key: SPARK-34227 URL: https://issues.apache.org/jira/browse/SPARK-34227 Project: Spark

[jira] [Updated] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucian Timar updated SPARK-34225: - Description:  Whenever I try to use jars or files whose paths contain spaces  FileNotFoundExcept

[jira] [Updated] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucian Timar updated SPARK-34225: - Description:  Whenever I try to use jars or files whose paths contain spaces  FileNotFoundExcept

[jira] [Updated] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucian Timar updated SPARK-34225: - Component/s: Spark Core > Jars or file paths which contain spaces are generating FileNotFoundExc

[jira] [Updated] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucian Timar updated SPARK-34225: - Description:   {code:java} c:>spark-shell --files "c:\Program Files\...\myjar.jar" or c:>spark-s

[jira] [Commented] (SPARK-34226) Reduce RepartitionOperation num partitions to its child max row

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271322#comment-17271322 ] Apache Spark commented on SPARK-34226: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-34226) Reduce RepartitionOperation num partitions to its child max row

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34226: Assignee: Apache Spark > Reduce RepartitionOperation num partitions to its child max row

[jira] [Commented] (SPARK-34226) Reduce RepartitionOperation num partitions to its child max row

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271320#comment-17271320 ] Apache Spark commented on SPARK-34226: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-34226) Reduce RepartitionOperation num partitions to its child max row

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34226: Assignee: (was: Apache Spark) > Reduce RepartitionOperation num partitions to its chi

[jira] [Created] (SPARK-34226) Reduce RepartitionOperation num partitions to its child max row

2021-01-25 Thread ulysses you (Jira)
ulysses you created SPARK-34226: --- Summary: Reduce RepartitionOperation num partitions to its child max row Key: SPARK-34226 URL: https://issues.apache.org/jira/browse/SPARK-34226 Project: Spark

[jira] [Updated] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucian Timar updated SPARK-34225: - Description:   {code:java} c:>spark-shell --files "c:\Program Files\...\myjar.jar" or c:>spark-s

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

2021-01-25 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271308#comment-17271308 ] Gabor Somogyi commented on SPARK-34198: --- +1 on this, I've started to review it bac

[jira] [Assigned] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34224: Assignee: (was: Apache Spark) > Ensure all resource opened by `Source.fromXXX` method

[jira] [Assigned] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34224: Assignee: Apache Spark > Ensure all resource opened by `Source.fromXXX` method are closed

[jira] [Commented] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271307#comment-17271307 ] Apache Spark commented on SPARK-34224: -- User 'LuciferYang' has created a pull reque

[jira] [Updated] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucian Timar updated SPARK-34225: - Description:   {code:java} c:>spark-shell --files "c:\Program%20Files\...\myjar.jar" or c:>spark

[jira] [Updated] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucian Timar updated SPARK-34225: - Environment: (was:   {code:java} c:>spark-shell --files "c:\Program%20Files\...\myjar.jar" o

[jira] [Created] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

2021-01-25 Thread Lucian Timar (Jira)
Lucian Timar created SPARK-34225: Summary: Jars or file paths which contain spaces are generating FileNotFoundException exception Key: SPARK-34225 URL: https://issues.apache.org/jira/browse/SPARK-34225

[jira] [Assigned] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34203: Assignee: (was: Apache Spark) > FileSource table null partition can not be dropped >

[jira] [Assigned] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34203: Assignee: Apache Spark > FileSource table null partition can not be dropped > ---

[jira] [Commented] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271304#comment-17271304 ] Apache Spark commented on SPARK-34203: -- User 'MaxGekk' has created a pull request f

[jira] [Updated] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-25 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-34224: - Issue Type: Bug (was: Improvement) > Ensure all resource opened by `Source.fromXXX` method are closed >

[jira] [Created] (SPARK-34224) Ensure all resource opened by `Source.fromXXX` method are closed

2021-01-25 Thread Yang Jie (Jira)
Yang Jie created SPARK-34224: Summary: Ensure all resource opened by `Source.fromXXX` method are closed Key: SPARK-34224 URL: https://issues.apache.org/jira/browse/SPARK-34224 Project: Spark Iss

[jira] [Commented] (SPARK-34185) Review and fix issues in API docs

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271279#comment-17271279 ] Apache Spark commented on SPARK-34185: -- User 'xuanyuanking' has created a pull requ

[jira] [Assigned] (SPARK-34223) NPE for static partion with null in InsertIntoHadoopFsRelationCommand

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34223: Assignee: Apache Spark > NPE for static partion with null in InsertIntoHadoopFsRelationCo

[jira] [Assigned] (SPARK-34223) NPE for static partion with null in InsertIntoHadoopFsRelationCommand

2021-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34223: Assignee: (was: Apache Spark) > NPE for static partion with null in InsertIntoHadoopF

  1   2   >