[jira] [Commented] (SPARK-34811) Redact fs.s3a.access.key like secret and token

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305629#comment-17305629 ] Apache Spark commented on SPARK-34811: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-34811) Redact fs.s3a.access.key like secret and token

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305628#comment-17305628 ] Apache Spark commented on SPARK-34811: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34811) Redact fs.s3a.access.key like secret and token

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34811: Assignee: (was: Apache Spark) > Redact fs.s3a.access.key like secret and token >

[jira] [Assigned] (SPARK-34811) Redact fs.s3a.access.key like secret and token

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34811: Assignee: Apache Spark > Redact fs.s3a.access.key like secret and token > ---

[jira] [Created] (SPARK-34811) Redact fs.s3a.access.key like secret and token

2021-03-20 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34811: - Summary: Redact fs.s3a.access.key like secret and token Key: SPARK-34811 URL: https://issues.apache.org/jira/browse/SPARK-34811 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-34810) Update PostgreSQL test with the latest results

2021-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34810. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31910 [https://gith

[jira] [Assigned] (SPARK-34810) Update PostgreSQL test with the latest results

2021-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34810: --- Assignee: William Hyun > Update PostgreSQL test with the latest results > -

[jira] [Commented] (SPARK-34796) Codegen compilation error for query with LIMIT operator and without AQE

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305621#comment-17305621 ] Apache Spark commented on SPARK-34796: -- User 'c21' has created a pull request for t

[jira] [Commented] (SPARK-34810) Update PostgreSQL test with the latest results

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305606#comment-17305606 ] Apache Spark commented on SPARK-34810: -- User 'williamhyun' has created a pull reque

[jira] [Assigned] (SPARK-34810) Update PostgreSQL test with the latest results

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34810: Assignee: Apache Spark > Update PostgreSQL test with the latest results > ---

[jira] [Assigned] (SPARK-34810) Update PostgreSQL test with the latest results

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34810: Assignee: (was: Apache Spark) > Update PostgreSQL test with the latest results >

[jira] [Commented] (SPARK-34810) Update PostgreSQL test with the latest results

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305605#comment-17305605 ] Apache Spark commented on SPARK-34810: -- User 'williamhyun' has created a pull reque

[jira] [Created] (SPARK-34810) Update PostgreSQL test with the latest results

2021-03-20 Thread William Hyun (Jira)
William Hyun created SPARK-34810: Summary: Update PostgreSQL test with the latest results Key: SPARK-34810 URL: https://issues.apache.org/jira/browse/SPARK-34810 Project: Spark Issue Type: Im

[jira] [Updated] (SPARK-34790) Fail in fetch shuffle blocks in batch when i/o encryption is enabled.

2021-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34790: -- Priority: Critical (was: Major) > Fail in fetch shuffle blocks in batch when i/o encryption i

[jira] [Commented] (SPARK-34801) java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.metadata.Hive.loadPartition

2021-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305591#comment-17305591 ] Hyukjin Kwon commented on SPARK-34801: -- Does this happen in Apache Spark too? inste

[jira] [Assigned] (SPARK-34809) Enable spark.hadoopRDD.ignoreEmptySplits by default

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34809: Assignee: Apache Spark > Enable spark.hadoopRDD.ignoreEmptySplits by default > --

[jira] [Assigned] (SPARK-34809) Enable spark.hadoopRDD.ignoreEmptySplits by default

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34809: Assignee: (was: Apache Spark) > Enable spark.hadoopRDD.ignoreEmptySplits by default >

[jira] [Commented] (SPARK-34809) Enable spark.hadoopRDD.ignoreEmptySplits by default

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305588#comment-17305588 ] Apache Spark commented on SPARK-34809: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-34809) Enable spark.hadoopRDD.ignoreEmptySplits by default

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305587#comment-17305587 ] Apache Spark commented on SPARK-34809: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-34809) Enable spark.hadoopRDD.ignoreEmptySplits by default

2021-03-20 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34809: - Summary: Enable spark.hadoopRDD.ignoreEmptySplits by default Key: SPARK-34809 URL: https://issues.apache.org/jira/browse/SPARK-34809 Project: Spark Issue T

[jira] [Updated] (SPARK-28220) join foldable condition not pushed down when parent filter is totally pushed down

2021-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28220: -- Issue Type: Improvement (was: Bug) > join foldable condition not pushed down when parent filt

[jira] [Assigned] (SPARK-28220) join foldable condition not pushed down when parent filter is totally pushed down

2021-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-28220: --- Assignee: Yuming Wang > join foldable condition not pushed down when parent filter is total

[jira] [Resolved] (SPARK-28220) join foldable condition not pushed down when parent filter is totally pushed down

2021-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-28220. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31857 [https://gith

[jira] [Commented] (SPARK-34808) Removes outer join if it only has distinct on streamed side

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305433#comment-17305433 ] Apache Spark commented on SPARK-34808: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-34808) Removes outer join if it only has distinct on streamed side

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34808: Assignee: Apache Spark > Removes outer join if it only has distinct on streamed side > --

[jira] [Commented] (SPARK-34808) Removes outer join if it only has distinct on streamed side

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305432#comment-17305432 ] Apache Spark commented on SPARK-34808: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-34808) Removes outer join if it only has distinct on streamed side

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34808: Assignee: (was: Apache Spark) > Removes outer join if it only has distinct on streame

[jira] [Assigned] (SPARK-34808) Removes outer join if it only has distinct on streamed side

2021-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34808: --- Assignee: (was: Yuming Wang) > Removes outer join if it only has distinct on streamed s

[jira] [Created] (SPARK-34808) Removes outer join if it only has distinct on streamed side

2021-03-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34808: --- Summary: Removes outer join if it only has distinct on streamed side Key: SPARK-34808 URL: https://issues.apache.org/jira/browse/SPARK-34808 Project: Spark Is

[jira] [Commented] (SPARK-34674) Spark app on k8s doesn't terminate without call to sparkContext.stop() method

2021-03-20 Thread Sergey Kotlov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305419#comment-17305419 ] Sergey Kotlov commented on SPARK-34674: --- Thanks for response, [~dongjoon] I haven

[jira] [Assigned] (SPARK-34806) Helper class for batch Dataset.observe()

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34806: Assignee: (was: Apache Spark) > Helper class for batch Dataset.observe() > --

[jira] [Commented] (SPARK-34806) Helper class for batch Dataset.observe()

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305379#comment-17305379 ] Apache Spark commented on SPARK-34806: -- User 'EnricoMi' has created a pull request

[jira] [Assigned] (SPARK-34806) Helper class for batch Dataset.observe()

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34806: Assignee: Apache Spark > Helper class for batch Dataset.observe() > -

[jira] [Commented] (SPARK-34807) Push down filter through window after TransposeWindow

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305371#comment-17305371 ] Apache Spark commented on SPARK-34807: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-34807) Push down filter through window after TransposeWindow

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34807: Assignee: Apache Spark > Push down filter through window after TransposeWindow >

[jira] [Assigned] (SPARK-34807) Push down filter through window after TransposeWindow

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34807: Assignee: (was: Apache Spark) > Push down filter through window after TransposeWindow

[jira] [Commented] (SPARK-34807) Push down filter through window after TransposeWindow

2021-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305370#comment-17305370 ] Apache Spark commented on SPARK-34807: -- User 'wangyum' has created a pull request f

[jira] [Created] (SPARK-34807) Push down filter through window after TransposeWindow

2021-03-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34807: --- Summary: Push down filter through window after TransposeWindow Key: SPARK-34807 URL: https://issues.apache.org/jira/browse/SPARK-34807 Project: Spark Issue Typ

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-03-20 Thread Devaraj Kavali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305327#comment-17305327 ] Devaraj Kavali edited comment on SPARK-18105 at 3/20/21, 7:09 AM:

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-03-20 Thread Devaraj Kavali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305327#comment-17305327 ] Devaraj Kavali commented on SPARK-18105: We are still seeing the issue with Spar