[jira] [Commented] (SPARK-29343) Eliminate sorts without limit in the subquery of Join/Aggregation

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159693#comment-17159693 ] Apache Spark commented on SPARK-29343: -- User 'ulysses-you' has created a pull request for this

[jira] [Comment Edited] (SPARK-15118) spark couldn't get hive properyties in hive-site.xml

2020-07-16 Thread xiepengjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159692#comment-17159692 ] xiepengjie edited comment on SPARK-15118 at 7/17/20, 5:52 AM: -- Maybe we

[jira] [Comment Edited] (SPARK-15118) spark couldn't get hive properyties in hive-site.xml

2020-07-16 Thread xiepengjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159692#comment-17159692 ] xiepengjie edited comment on SPARK-15118 at 7/17/20, 5:50 AM: -- Maybe we

[jira] [Reopened] (SPARK-15118) spark couldn't get hive properyties in hive-site.xml

2020-07-16 Thread xiepengjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiepengjie reopened SPARK-15118: Maybe we should merge HiveConf too: org.apache.spark.sql.internal.BaseSessionStateBuilder#conf ```

[jira] [Created] (SPARK-32343) CSV predicate pushdown for nested fields

2020-07-16 Thread jobit mathew (Jira)
jobit mathew created SPARK-32343: Summary: CSV predicate pushdown for nested fields Key: SPARK-32343 URL: https://issues.apache.org/jira/browse/SPARK-32343 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159677#comment-17159677 ] Apache Spark commented on SPARK-32018: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159675#comment-17159675 ] Apache Spark commented on SPARK-32018: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-32342) Kafka events are missing magic byte

2020-07-16 Thread Sridhar Baddela (Jira)
Sridhar Baddela created SPARK-32342: --- Summary: Kafka events are missing magic byte Key: SPARK-32342 URL: https://issues.apache.org/jira/browse/SPARK-32342 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-32341) wish to add mutiple filter in rdd function

2020-07-16 Thread gaokui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaokui updated SPARK-32341: --- Description: when i use spark rdd . i often use to read kafka data. but kafka data has lots of kinds data

[jira] [Updated] (SPARK-32341) wish to add mutiple filter in rdd function

2020-07-16 Thread gaokui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaokui updated SPARK-32341: --- Description: when i use spark rdd . i often use to read kafka data. but kafka data has lots of kinds data

[jira] [Updated] (SPARK-32341) wish to add mutiple filter in rdd function

2020-07-16 Thread gaokui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaokui updated SPARK-32341: --- Issue Type: New Feature (was: Bug) > wish to add mutiple filter in rdd function >

[jira] [Created] (SPARK-32341) wish to add mutiple filter in rdd function

2020-07-16 Thread gaokui (Jira)
gaokui created SPARK-32341: -- Summary: wish to add mutiple filter in rdd function Key: SPARK-32341 URL: https://issues.apache.org/jira/browse/SPARK-32341 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-07-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32094. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29114

[jira] [Assigned] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-07-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32094: Assignee: Hyukjin Kwon > Patch cloudpickle.py with typing module side-effect fix >

[jira] [Updated] (SPARK-32340) Move cloudpickle into python/lib as a thridparty library

2020-07-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32340: - Issue Type: Improvement (was: Bug) > Move cloudpickle into python/lib as a thridparty library

[jira] [Created] (SPARK-32340) Move cloudpickle into python/lib as a thridparty library

2020-07-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32340: Summary: Move cloudpickle into python/lib as a thridparty library Key: SPARK-32340 URL: https://issues.apache.org/jira/browse/SPARK-32340 Project: Spark

[jira] [Commented] (SPARK-32145) ThriftCLIService.GetOperationStatus should include exception's stack trace to the error message

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159624#comment-17159624 ] Apache Spark commented on SPARK-32145: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-32145) ThriftCLIService.GetOperationStatus should include exception's stack trace to the error message

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159623#comment-17159623 ] Apache Spark commented on SPARK-32145: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-32339) Improve MLlib BLAS native acceleration docs

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159610#comment-17159610 ] Apache Spark commented on SPARK-32339: -- User 'xwu99' has created a pull request for this issue:

[jira] [Commented] (SPARK-32339) Improve MLlib BLAS native acceleration docs

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159609#comment-17159609 ] Apache Spark commented on SPARK-32339: -- User 'xwu99' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32339) Improve MLlib BLAS native acceleration docs

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32339: Assignee: Apache Spark > Improve MLlib BLAS native acceleration docs >

[jira] [Assigned] (SPARK-32339) Improve MLlib BLAS native acceleration docs

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32339: Assignee: (was: Apache Spark) > Improve MLlib BLAS native acceleration docs >

[jira] [Updated] (SPARK-32339) Improve MLlib BLAS native acceleration docs

2020-07-16 Thread Xiaochang Wu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochang Wu updated SPARK-32339: - Summary: Improve MLlib BLAS native acceleration docs (was: Improve MLlib BLAS acceleration

[jira] [Created] (SPARK-32339) Improve MLlib BLAS acceleration docs

2020-07-16 Thread Xiaochang Wu (Jira)
Xiaochang Wu created SPARK-32339: Summary: Improve MLlib BLAS acceleration docs Key: SPARK-32339 URL: https://issues.apache.org/jira/browse/SPARK-32339 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-32338) Add overload for slice that accepts Columns or Int

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32338: Assignee: Apache Spark > Add overload for slice that accepts Columns or Int >

[jira] [Commented] (SPARK-32338) Add overload for slice that accepts Columns or Int

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159591#comment-17159591 ] Apache Spark commented on SPARK-32338: -- User 'nvander1' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32338) Add overload for slice that accepts Columns or Int

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32338: Assignee: (was: Apache Spark) > Add overload for slice that accepts Columns or Int >

[jira] [Created] (SPARK-32338) Add overload for slice that accepts Columns or Int

2020-07-16 Thread Nikolas Vanderhoof (Jira)
Nikolas Vanderhoof created SPARK-32338: -- Summary: Add overload for slice that accepts Columns or Int Key: SPARK-32338 URL: https://issues.apache.org/jira/browse/SPARK-32338 Project: Spark

[jira] [Assigned] (SPARK-32337) Show initial plan in AQE plan tree string

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32337: Assignee: (was: Apache Spark) > Show initial plan in AQE plan tree string >

[jira] [Assigned] (SPARK-32337) Show initial plan in AQE plan tree string

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32337: Assignee: Apache Spark > Show initial plan in AQE plan tree string >

[jira] [Commented] (SPARK-32337) Show initial plan in AQE plan tree string

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159583#comment-17159583 ] Apache Spark commented on SPARK-32337: -- User 'allisonwang-db' has created a pull request for this

[jira] [Updated] (SPARK-32337) Show initial plan in AQE plan tree string

2020-07-16 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-32337: - Description: Currently the tree string for {{AdaptiveSparkPlanExec}} only shows the current

[jira] [Created] (SPARK-32337) Show initial plan in AQE plan tree string

2020-07-16 Thread Allison Wang (Jira)
Allison Wang created SPARK-32337: Summary: Show initial plan in AQE plan tree string Key: SPARK-32337 URL: https://issues.apache.org/jira/browse/SPARK-32337 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-31924) Create remote shuffle service reference implementation

2020-07-16 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159550#comment-17159550 ] BoYang edited comment on SPARK-31924 at 7/16/20, 11:21 PM: --- We created a short 

[jira] [Comment Edited] (SPARK-31924) Create remote shuffle service reference implementation

2020-07-16 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159550#comment-17159550 ] BoYang edited comment on SPARK-31924 at 7/16/20, 11:21 PM: --- We created a short 

[jira] [Commented] (SPARK-31924) Create remote shuffle service reference implementation

2020-07-16 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159550#comment-17159550 ] BoYang commented on SPARK-31924: We created a short [design

[jira] [Created] (SPARK-32336) 11 Critical & 4 High severity issues in Apcahe Spark 3.0.0 - dependency libraries

2020-07-16 Thread Albert Baker (Jira)
Albert Baker created SPARK-32336: Summary: 11 Critical & 4 High severity issues in Apcahe Spark 3.0.0 - dependency libraries Key: SPARK-32336 URL: https://issues.apache.org/jira/browse/SPARK-32336

[jira] [Commented] (SPARK-32253) Make readability better in the test result logs

2020-07-16 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159443#comment-17159443 ] L. C. Hsieh commented on SPARK-32253: - Thanks [~Gengliang.Wang] > Make readability better in the

[jira] [Resolved] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32335. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29136

[jira] [Assigned] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32335: - Assignee: Dongjoon Hyun > Remove Python2 test from K8s IT >

[jira] [Updated] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32335: -- Parent: SPARK-29909 Issue Type: Sub-task (was: Test) > Remove Python2 test from K8s

[jira] [Updated] (SPARK-29909) Drop Python 2 and Python 3.4 and 3.5.

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29909: -- Labels: release-notes (was: ) > Drop Python 2 and Python 3.4 and 3.5. >

[jira] [Updated] (SPARK-29909) Drop Python 2 and Python 3.4 and 3.5.

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29909: -- Target Version/s: 3.1.0 > Drop Python 2 and Python 3.4 and 3.5. >

[jira] [Assigned] (SPARK-32310) ML params default value parity

2020-07-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-32310: -- Assignee: Huaxin Gao > ML params default value parity > -- > >

[jira] [Resolved] (SPARK-32310) ML params default value parity

2020-07-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-32310. Resolution: Fixed > ML params default value parity > -- > >

[jira] [Resolved] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32018. --- Fix Version/s: 3.1.0 3.0.1 Assignee: Wenchen Fan

[jira] [Commented] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159381#comment-17159381 ] Apache Spark commented on SPARK-32335: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32335: Assignee: (was: Apache Spark) > Remove Python2 test from K8s IT >

[jira] [Commented] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159380#comment-17159380 ] Apache Spark commented on SPARK-32335: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32335: Assignee: Apache Spark > Remove Python2 test from K8s IT >

[jira] [Updated] (SPARK-32335) Remove Python2 test from K8s IT

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32335: -- Summary: Remove Python2 test from K8s IT (was: Remove Python2 from K8s IT) > Remove Python2

[jira] [Created] (SPARK-32334) Investigate commonizing Columnar and Row data transformations

2020-07-16 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-32334: - Summary: Investigate commonizing Columnar and Row data transformations Key: SPARK-32334 URL: https://issues.apache.org/jira/browse/SPARK-32334 Project: Spark

[jira] [Created] (SPARK-32335) Remove Python2 from K8s IT

2020-07-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-32335: - Summary: Remove Python2 from K8s IT Key: SPARK-32335 URL: https://issues.apache.org/jira/browse/SPARK-32335 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-32315) Provide an explanation error message when calling require

2020-07-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-32315: -- Assignee: dzlabs > Provide an explanation error message when calling require >

[jira] [Commented] (SPARK-32307) Aggression that use map type input UDF as group expression can fail

2020-07-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159350#comment-17159350 ] Dongjoon Hyun commented on SPARK-32307: --- Got it. Thanks, [~Ngone51]! > Aggression that use map

[jira] [Commented] (SPARK-32253) Make readability better in the test result logs

2020-07-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159340#comment-17159340 ] Gengliang Wang commented on SPARK-32253: I find this annoying too. So I try creating a simple

[jira] [Resolved] (SPARK-32315) Provide an explanation error message when calling require

2020-07-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-32315. Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29115

[jira] [Commented] (SPARK-32151) Kafka does not allow Partition Rebalance Handling

2020-07-16 Thread Ed Mitchell (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159330#comment-17159330 ] Ed Mitchell commented on SPARK-32151: - Yeah - I think you're right. It's going to be in the write

[jira] [Commented] (SPARK-32151) Kafka does not allow Partition Rebalance Handling

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159322#comment-17159322 ] Gabor Somogyi commented on SPARK-32151: --- I see, this case Structured Streaming is not affected. >

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

2020-07-16 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159318#comment-17159318 ] Erik Krogen commented on SPARK-32037: - I think it might be helpful to frame the discussion around

[jira] [Commented] (SPARK-32151) Kafka does not allow Partition Rebalance Handling

2020-07-16 Thread Ed Mitchell (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159294#comment-17159294 ] Ed Mitchell commented on SPARK-32151: - I could do that if I wanted to start back at the beginning or

[jira] [Commented] (SPARK-30276) Support Filter expression allows simultaneous use of DISTINCT

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159289#comment-17159289 ] Apache Spark commented on SPARK-30276: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-32151) Kafka does not allow Partition Rebalance Handling

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159278#comment-17159278 ] Gabor Somogyi commented on SPARK-32151: --- Have you set `auto.offset.reset`? > Kafka does not allow

[jira] [Commented] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159275#comment-17159275 ] Apache Spark commented on SPARK-32332: -- User 'andygrove' has created a pull request for this issue:

[jira] [Commented] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159273#comment-17159273 ] Apache Spark commented on SPARK-32332: -- User 'andygrove' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32332: Assignee: (was: Apache Spark) > AQE doesn't adequately allow for Columnar Processing

[jira] [Assigned] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32332: Assignee: Apache Spark > AQE doesn't adequately allow for Columnar Processing extension

[jira] [Resolved] (SPARK-30648) Support filters pushdown in JSON datasource

2020-07-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30648. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27366

[jira] [Assigned] (SPARK-30648) Support filters pushdown in JSON datasource

2020-07-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30648: Assignee: Maxim Gekk > Support filters pushdown in JSON datasource >

[jira] [Commented] (SPARK-32253) Make readability better in the test result logs

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159264#comment-17159264 ] Apache Spark commented on SPARK-32253: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-32253) Make readability better in the test result logs

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32253: Assignee: Apache Spark > Make readability better in the test result logs >

[jira] [Assigned] (SPARK-32253) Make readability better in the test result logs

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32253: Assignee: (was: Apache Spark) > Make readability better in the test result logs >

[jira] [Created] (SPARK-32333) Drop references to Master

2020-07-16 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-32333: - Summary: Drop references to Master Key: SPARK-32333 URL: https://issues.apache.org/jira/browse/SPARK-32333 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

2020-07-16 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-32332: - Summary: AQE doesn't adequately allow for Columnar Processing extension Key: SPARK-32332 URL: https://issues.apache.org/jira/browse/SPARK-32332 Project: Spark

[jira] [Comment Edited] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159244#comment-17159244 ] Gabor Somogyi edited comment on SPARK-28367 at 7/16/20, 2:22 PM: - I'm

[jira] [Comment Edited] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159244#comment-17159244 ] Gabor Somogyi edited comment on SPARK-28367 at 7/16/20, 2:22 PM: - I'm

[jira] [Commented] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159244#comment-17159244 ] Gabor Somogyi commented on SPARK-28367: --- I'm going to have a chat w/ the Flink guys how they've

[jira] [Updated] (SPARK-32315) Provide an explanation error message when calling require

2020-07-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-32315: - Priority: Trivial (was: Minor) > Provide an explanation error message when calling require >

[jira] [Updated] (SPARK-32329) Rename HADOOP2_MODULE_PROFILES to HADOOP_MODULE_PROFILES

2020-07-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-32329: - Priority: Trivial (was: Minor) > Rename HADOOP2_MODULE_PROFILES to HADOOP_MODULE_PROFILES >

[jira] [Comment Edited] (SPARK-32257) [SQL Parser] Report Error for invalid usage of SET command

2020-07-16 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159224#comment-17159224 ] Takeshi Yamamuro edited comment on SPARK-32257 at 7/16/20, 1:48 PM:

[jira] [Commented] (SPARK-32257) [SQL Parser] Report Error for invalid usage of SET command

2020-07-16 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159224#comment-17159224 ] Takeshi Yamamuro commented on SPARK-32257: -- Can I take this over if you don't have much time.

[jira] [Assigned] (SPARK-32331) Keep advanced statistics when pruning partitions

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32331: Assignee: Apache Spark > Keep advanced statistics when pruning partitions >

[jira] [Assigned] (SPARK-32331) Keep advanced statistics when pruning partitions

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32331: Assignee: (was: Apache Spark) > Keep advanced statistics when pruning partitions >

[jira] [Commented] (SPARK-32331) Keep advanced statistics when pruning partitions

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159215#comment-17159215 ] Apache Spark commented on SPARK-32331: -- User 'peter-toth' has created a pull request for this

[jira] [Commented] (SPARK-32331) Keep advanced statistics when pruning partitions

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159214#comment-17159214 ] Apache Spark commented on SPARK-32331: -- User 'peter-toth' has created a pull request for this

[jira] [Commented] (SPARK-32321) Rollback SPARK-26267 workaround since KAFKA-7703 resolved

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159209#comment-17159209 ] Apache Spark commented on SPARK-32321: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-32321) Rollback SPARK-26267 workaround since KAFKA-7703 resolved

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32321: Assignee: Apache Spark > Rollback SPARK-26267 workaround since KAFKA-7703 resolved >

[jira] [Assigned] (SPARK-32321) Rollback SPARK-26267 workaround since KAFKA-7703 resolved

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32321: Assignee: (was: Apache Spark) > Rollback SPARK-26267 workaround since KAFKA-7703

[jira] [Commented] (SPARK-32321) Rollback SPARK-26267 workaround since KAFKA-7703 resolved

2020-07-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159206#comment-17159206 ] Apache Spark commented on SPARK-32321: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Created] (SPARK-32331) Keep advanced statistics when pruning partitions

2020-07-16 Thread Peter Toth (Jira)
Peter Toth created SPARK-32331: -- Summary: Keep advanced statistics when pruning partitions Key: SPARK-32331 URL: https://issues.apache.org/jira/browse/SPARK-32331 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-32234) Spark sql commands are failing on select Queries for the orc tables

2020-07-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32234: --- Assignee: Saurabh Chawla > Spark sql commands are failing on select Queries for the orc

[jira] [Resolved] (SPARK-32234) Spark sql commands are failing on select Queries for the orc tables

2020-07-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32234. - Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-32272) SET TIME ZONE standard sql command support

2020-07-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32272: --- Assignee: Kent Yao > SET TIME ZONE standard sql command support >

[jira] [Resolved] (SPARK-32272) SET TIME ZONE standard sql command support

2020-07-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32272. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29064

[jira] [Resolved] (SPARK-32015) Remote inheritable thread local variables after spark context is stopped

2020-07-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-32015. -- Resolution: Not A Problem > Remote inheritable thread local variables after spark context is

[jira] [Comment Edited] (SPARK-32321) Rollback SPARK-26267 workaround since KAFKA-7703 resolved

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159160#comment-17159160 ] Gabor Somogyi edited comment on SPARK-32321 at 7/16/20, 12:14 PM: -- I've

[jira] [Commented] (SPARK-32321) Rollback SPARK-26267 workaround since KAFKA-7703 resolved

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159160#comment-17159160 ] Gabor Somogyi commented on SPARK-32321: --- I've double checked that KAFKA-7703 resolved in 2.5.0

[jira] [Commented] (SPARK-23829) spark-sql-kafka source in spark 2.3 causes reading stream failure frequently

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159109#comment-17159109 ] Gabor Somogyi commented on SPARK-23829: --- Since this is race issue there is no consistent way to

[jira] [Commented] (SPARK-32321) Rollback SPARK-26267 workaround since KAFKA-7703 resolved

2020-07-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159061#comment-17159061 ] Gabor Somogyi commented on SPARK-32321: --- After I've had a deeper look my point is not to drop the

[jira] [Commented] (SPARK-32307) Aggression that use map type input UDF as group expression can fail

2020-07-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159020#comment-17159020 ] wuyi commented on SPARK-32307: -- Hi [~dongjoon] Please see my response here

  1   2   >