[jira] [Resolved] (SPARK-30833) Extract o.a.s.sql.catalyst.util.ParseMode to external project

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30833. -- Resolution: Won't Fix > Extract o.a.s.sql.catalyst.util.ParseMode to external project > --

[jira] [Commented] (SPARK-30833) Extract o.a.s.sql.catalyst.util.ParseMode to external project

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044201#comment-17044201 ] Hyukjin Kwon commented on SPARK-30833: -- It just needs one line change in User's app

[jira] [Resolved] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30855. -- Resolution: Duplicate > Issue using 'explode' function followed by a (*)star expand selection

[jira] [Created] (SPARK-30945) Preserve JVM information when executor heartbeat timeout

2020-02-24 Thread wangzhun (Jira)
wangzhun created SPARK-30945: Summary: Preserve JVM information when executor heartbeat timeout Key: SPARK-30945 URL: https://issues.apache.org/jira/browse/SPARK-30945 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-24 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043634#comment-17043634 ] Peter Toth edited comment on SPARK-30855 at 2/25/20 7:29 AM: -

[jira] [Commented] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-24 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044175#comment-17044175 ] zhengruifeng commented on SPARK-30932: -- refering to https://issues.apache.org/jira/

[jira] [Comment Edited] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-24 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043634#comment-17043634 ] Peter Toth edited comment on SPARK-30855 at 2/25/20 7:15 AM: -

[jira] [Updated] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-24 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30932: - Attachment: signature.diff added_ml_class 1_process_script.sh >

[jira] [Assigned] (SPARK-30885) V1 table name should be fully qualified if catalog name is provided

2020-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30885: --- Assignee: Terry Kim > V1 table name should be fully qualified if catalog name is provided >

[jira] [Resolved] (SPARK-30885) V1 table name should be fully qualified if catalog name is provided

2020-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30885. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27642 [https://gith

[jira] [Commented] (SPARK-30853) Error communicating with MapOutputTracker issue

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044127#comment-17044127 ] Hyukjin Kwon commented on SPARK-30853: -- [~yumwang] Can you describe some more conte

[jira] [Resolved] (SPARK-30926) Same SQL on CSV and on Parquet gives different result

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30926. -- Resolution: Not A Problem > Same SQL on CSV and on Parquet gives different result > --

[jira] [Commented] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044121#comment-17044121 ] Hyukjin Kwon commented on SPARK-30941: -- Probably we should rather document this lim

[jira] [Resolved] (SPARK-30936) Forwards-compatibility in JsonProtocol in broken

2020-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30936. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27680 [https://gith

[jira] [Assigned] (SPARK-30936) Forwards-compatibility in JsonProtocol in broken

2020-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30936: --- Assignee: Shixiong Zhu > Forwards-compatibility in JsonProtocol in broken > ---

[jira] [Resolved] (SPARK-30887) Add version information to the configuration of Deploy

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30887. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27668 [https://gi

[jira] [Assigned] (SPARK-30887) Add version information to the configuration of Deploy

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30887: Assignee: jiaan.geng > Add version information to the configuration of Deploy > -

[jira] [Resolved] (SPARK-30917) The behaviour of UnaryMinus should not depend on SQLConf.get

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30917. -- Resolution: Later > The behaviour of UnaryMinus should not depend on SQLConf.get > ---

[jira] [Resolved] (SPARK-30939) StringIndexer setOutputCols does not set output cols

2020-02-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30939. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27684 [https://gi

[jira] [Resolved] (SPARK-30901) [DOC] In streaming-kinesis-integration.md, the initialPosition method changed

2020-02-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30901. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27652 [https://gi

[jira] [Assigned] (SPARK-30901) [DOC] In streaming-kinesis-integration.md, the initialPosition method changed

2020-02-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30901: Assignee: DavidXU > [DOC] In streaming-kinesis-integration.md, the initialPosition method

[jira] [Updated] (SPARK-30901) [DOC] In streaming-kinesis-integration.md, the initialPosition method changed

2020-02-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30901: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > [DOC] In streaming-kin

[jira] [Created] (SPARK-30944) Use latest URL for Google Cloud Storage mirror of Maven Central

2020-02-24 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-30944: -- Summary: Use latest URL for Google Cloud Storage mirror of Maven Central Key: SPARK-30944 URL: https://issues.apache.org/jira/browse/SPARK-30944 Project: Spark

[jira] [Commented] (SPARK-25603) Generalize Nested Column Pruning

2020-02-24 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043941#comment-17043941 ] Takeshi Yamamuro commented on SPARK-25603: -- nvm, just a remainder :) ok. > Gen

[jira] [Created] (SPARK-30943) Show "batch ID" in tool tip string for Structured Streaming UI graphs

2020-02-24 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-30943: Summary: Show "batch ID" in tool tip string for Structured Streaming UI graphs Key: SPARK-30943 URL: https://issues.apache.org/jira/browse/SPARK-30943 Project: Spark

[jira] [Updated] (SPARK-30826) LIKE returns wrong result from external table using parquet

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30826: -- Affects Version/s: 2.0.2 > LIKE returns wrong result from external table using parquet > -

[jira] [Updated] (SPARK-30826) LIKE returns wrong result from external table using parquet

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30826: -- Affects Version/s: 2.1.3 > LIKE returns wrong result from external table using parquet > -

[jira] [Updated] (SPARK-30826) LIKE returns wrong result from external table using parquet

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30826: -- Affects Version/s: 2.2.3 > LIKE returns wrong result from external table using parquet > -

[jira] [Updated] (SPARK-30826) LIKE returns wrong result from external table using parquet

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30826: -- Affects Version/s: 2.3.4 > LIKE returns wrong result from external table using parquet > -

[jira] [Updated] (SPARK-30826) LIKE returns wrong result from external table using parquet

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30826: -- Labels: correctness (was: ) > LIKE returns wrong result from external table using parquet > -

[jira] [Resolved] (SPARK-30870) catalyst inception of lateral view explode with struct raise a Catalyst error

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30870. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27675 [https://

[jira] [Assigned] (SPARK-30870) catalyst inception of lateral view explode with struct raise a Catalyst error

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30870: - Assignee: Peter Toth > catalyst inception of lateral view explode with struct raise a C

[jira] [Commented] (SPARK-30928) ML, GraphX 3.0 QA: API: Binary incompatible changes

2020-02-24 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043874#comment-17043874 ] Huaxin Gao commented on SPARK-30928: I went through the v30excludes list to check if

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

2020-02-24 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043829#comment-17043829 ] Min Shen commented on SPARK-30602: -- [~shanyu], I have listed a few key differences betw

[jira] [Updated] (SPARK-30942) Fix the warning for requiring cores to be limiting resource

2020-02-24 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-30942: -- Summary: Fix the warning for requiring cores to be limiting resource (was: Fix the warning fo

[jira] [Created] (SPARK-30942) Fix the warning for requireing cores to be limiting resource

2020-02-24 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-30942: - Summary: Fix the warning for requireing cores to be limiting resource Key: SPARK-30942 URL: https://issues.apache.org/jira/browse/SPARK-30942 Project: Spark

[jira] [Commented] (SPARK-30926) Same SQL on CSV and on Parquet gives different result

2020-02-24 Thread Mayank Asthana (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043791#comment-17043791 ] Mayank Asthana commented on SPARK-30926:   In your CSV code, {code:java} val re

[jira] [Commented] (SPARK-25603) Generalize Nested Column Pruning

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043758#comment-17043758 ] Dongjoon Hyun commented on SPARK-25603: --- cc [~dbtsai] > Generalize Nested Column

[jira] [Commented] (SPARK-25603) Generalize Nested Column Pruning

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043757#comment-17043757 ] Dongjoon Hyun commented on SPARK-25603: --- Sorry for being late, [~maropu]. I was bu

[jira] [Commented] (SPARK-30870) catalyst inception of lateral view explode with struct raise a Catalyst error

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043691#comment-17043691 ] Dongjoon Hyun commented on SPARK-30870: --- cc [~dbtsai] > catalyst inception of lat

[jira] [Commented] (SPARK-30870) catalyst inception of lateral view explode with struct raise a Catalyst error

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043686#comment-17043686 ] Dongjoon Hyun commented on SPARK-30870: --- Thank you for reporting, [~tprelle-ubi].

[jira] [Updated] (SPARK-30870) catalyst inception of lateral view explode with struct raise a Catalyst error

2020-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30870: -- Parent: SPARK-25603 Issue Type: Sub-task (was: Bug) > catalyst inception of lateral v

[jira] [Updated] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

2020-02-24 Thread David Roher (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Roher updated SPARK-30941: Description: It is possible to create a Row that has fields with the same name when calling `coll

[jira] [Created] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

2020-02-24 Thread David Roher (Jira)
David Roher created SPARK-30941: --- Summary: PySpark Row can be instantiated with duplicate field names Key: SPARK-30941 URL: https://issues.apache.org/jira/browse/SPARK-30941 Project: Spark Issu

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-24 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043634#comment-17043634 ] Peter Toth commented on SPARK-30855: I've checked that this is a duplicate of SPARK-

[jira] [Created] (SPARK-30940) Remove meaningless attributeId when Explain SQL query

2020-02-24 Thread Xin Wu (Jira)
Xin Wu created SPARK-30940: -- Summary: Remove meaningless attributeId when Explain SQL query Key: SPARK-30940 URL: https://issues.apache.org/jira/browse/SPARK-30940 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-26825) Spark Structure Streaming job failing when submitted in cluster mode

2020-02-24 Thread Shyam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043593#comment-17043593 ] Shyam commented on SPARK-26825: --- [~asdaraujo] [~gsomogyi] I am facing the same issue in sp

[jira] [Created] (SPARK-30939) StringIndexer setOutputCols does not set output cols

2020-02-24 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-30939: Summary: StringIndexer setOutputCols does not set output cols Key: SPARK-30939 URL: https://issues.apache.org/jira/browse/SPARK-30939 Project: Spark Issue Ty

[jira] [Updated] (SPARK-30897) The nullability of ArrayExists should not depend on SQLConf.get

2020-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30897: Summary: The nullability of ArrayExists should not depend on SQLConf.get (was: The behavior of Ar

[jira] [Updated] (SPARK-30898) The nullablity of MakeDecimal should not depend on SQLConf.get

2020-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30898: Summary: The nullablity of MakeDecimal should not depend on SQLConf.get (was: The behavior of Mak

[jira] [Created] (SPARK-30938) BinaryClassificationMetrics optimization

2020-02-24 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30938: Summary: BinaryClassificationMetrics optimization Key: SPARK-30938 URL: https://issues.apache.org/jira/browse/SPARK-30938 Project: Spark Issue Type: Improve

[jira] [Resolved] (SPARK-30927) StreamingQueryManager should avoid keeping reference to terminated StreamingQuery

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30927. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27678 [https://gi

[jira] [Assigned] (SPARK-30927) StreamingQueryManager should avoid keeping reference to terminated StreamingQuery

2020-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30927: Assignee: Shixiong Zhu > StreamingQueryManager should avoid keeping reference to terminat

[jira] [Created] (SPARK-30937) Migration guide for Hive 2.3

2020-02-24 Thread wuyi (Jira)
wuyi created SPARK-30937: Summary: Migration guide for Hive 2.3 Key: SPARK-30937 URL: https://issues.apache.org/jira/browse/SPARK-30937 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Resolved] (SPARK-28880) ANSI SQL: Nested bracketed comments

2020-02-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-28880. Assignee: jiaan.geng Resolution: Fixed This issue is resolved in https://github.com/