[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Updated] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengyao piao updated SPARK-21581: -- Description: Hi all I'm using Spark2.x on cdh5.11 I have a json file as follows.

[jira] [Created] (SPARK-21581) Spark 2.x distinct return incorrect result

2017-07-30 Thread shengyao piao (JIRA)
shengyao piao created SPARK-21581: - Summary: Spark 2.x distinct return incorrect result Key: SPARK-21581 URL: https://issues.apache.org/jira/browse/SPARK-21581 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21580) There's a bug with `Group by ordinal`

2017-07-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106804#comment-16106804 ] Xiao Li commented on SPARK-21580: - https://github.com/apache/spark/pull/18779 > There's a bug with

[jira] [Updated] (SPARK-21580) There's a bug with `Group by ordinal`

2017-07-30 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-21580: Description: create temporary view data as select * from values (1, 1), (1, 2), (2, 1), (2, 2),

[jira] [Created] (SPARK-21580) There's a bug with `Group by ordinal`

2017-07-30 Thread liuxian (JIRA)
liuxian created SPARK-21580: --- Summary: There's a bug with `Group by ordinal` Key: SPARK-21580 URL: https://issues.apache.org/jira/browse/SPARK-21580 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21567) Dataset with Tuple of type alias throws error

2017-07-30 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106781#comment-16106781 ] Liang-Chi Hsieh commented on SPARK-21567: - If there is no more questions about this, we can close

[jira] [Created] (SPARK-21579) dropTempView has a critical BUG

2017-07-30 Thread ant_nebula (JIRA)
ant_nebula created SPARK-21579: -- Summary: dropTempView has a critical BUG Key: SPARK-21579 URL: https://issues.apache.org/jira/browse/SPARK-21579 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-21575) Eliminate needless synchronization in java-R serialization

2017-07-30 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-21575. - Resolution: Fixed Assignee: Iurii Antykhovych Fix Version/s: 2.3.0 > Eliminate

[jira] [Updated] (SPARK-16683) Group by does not work after multiple joins of the same dataframe

2017-07-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16683: Labels: correctness (was: ) > Group by does not work after multiple joins of the same dataframe >

[jira] [Updated] (SPARK-16683) Group by does not work after multiple joins of the same dataframe

2017-07-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16683: Component/s: (was: Optimizer) SQL > Group by does not work after multiple joins of

[jira] [Resolved] (SPARK-20944) Move SortShuffleWriter.shouldBypassMergeSort to SortShuffleManager

2017-07-30 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20944. -- Resolution: Later I am resolving this per the discussion in the PR but thanks for taking a

[jira] [Commented] (SPARK-21578) Consolidate redundant SparkContext constructors due to SI-8479

2017-07-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106679#comment-16106679 ] Dongjoon Hyun commented on SPARK-21578: --- I made a PR for this. -

[jira] [Created] (SPARK-21578) Consolidate redundant SparkContext constructors due to SI-8479

2017-07-30 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-21578: - Summary: Consolidate redundant SparkContext constructors due to SI-8479 Key: SPARK-21578 URL: https://issues.apache.org/jira/browse/SPARK-21578 Project: Spark

[jira] [Closed] (SPARK-21574) set hive.exec.max.dynamic.partitions lose effect

2017-07-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-21574. - Resolution: Duplicate Hi, [~q79969786]. Thank you for report, but this is a duplicated issue of

[jira] [Commented] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

2017-07-30 Thread Albert Chu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106609#comment-16106609 ] Albert Chu commented on SPARK-21570: Yeah, that's understandable. IIRC, on most network file

[jira] [Comment Edited] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

2017-07-30 Thread Albert Chu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106607#comment-16106607 ] Albert Chu edited comment on SPARK-21570 at 7/30/17 6:57 PM: - (Oops,

[jira] [Comment Edited] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

2017-07-30 Thread Albert Chu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106607#comment-16106607 ] Albert Chu edited comment on SPARK-21570 at 7/30/17 6:56 PM: - (Oops,

[jira] [Commented] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

2017-07-30 Thread Albert Chu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106607#comment-16106607 ] Albert Chu commented on SPARK-21570: FWIW (and my scala and Spark code knowledge is not elite), in

[jira] [Commented] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106591#comment-16106591 ] Sean Owen commented on SPARK-21570: --- Got it. I still think that when it writes the archive to a local

[jira] [Commented] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

2017-07-30 Thread Albert Chu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106585#comment-16106585 ] Albert Chu commented on SPARK-21570: My assumption was the "__spark_libs__695301535722158702.zip"

[jira] [Assigned] (SPARK-21297) Add count in 'JDBC/ODBC Server' page.

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21297: - Assignee: guoxiaolongzte Priority: Trivial (was: Minor) > Add count in 'JDBC/ODBC Server'

[jira] [Resolved] (SPARK-21297) Add count in 'JDBC/ODBC Server' page.

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21297. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18525

[jira] [Commented] (SPARK-20433) Update jackson-databind to 2.6.7

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106577#comment-16106577 ] Sean Owen commented on SPARK-20433: --- [~Davidho] I tried updating to Jackson 2.7.x for another change,

[jira] [Commented] (SPARK-20433) Update jackson-databind to 2.6.7

2017-07-30 Thread David Hodeffi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106536#comment-16106536 ] David Hodeffi commented on SPARK-20433: --- Latest major + minor version. I think compatibility is a

[jira] [Commented] (SPARK-21552) Add decimal type support to ArrowWriter.

2017-07-30 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106532#comment-16106532 ] Wes McKinney commented on SPARK-21552: -- cc @cpcloud -- I linked some associated issues in Arrow

[jira] [Comment Edited] (SPARK-21552) Add decimal type support to ArrowWriter.

2017-07-30 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106532#comment-16106532 ] Wes McKinney edited comment on SPARK-21552 at 7/30/17 3:40 PM: --- cc

[jira] [Commented] (SPARK-21569) Internal Spark class needs to be kryo-registered

2017-07-30 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106530#comment-16106530 ] Ryan Williams commented on SPARK-21569: --- Main.scala: {code} package com.foo import

[jira] [Commented] (SPARK-20433) Update jackson-databind to 2.6.7

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106521#comment-16106521 ] Sean Owen commented on SPARK-20433: --- To the latest minor version? because they are not entirely

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-07-30 Thread Arthur Rand (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106501#comment-16106501 ] Arthur Rand commented on SPARK-16742: - Hello [~vanzin], I addressed the comments for the second PR

[jira] [Resolved] (SPARK-21577) Issue is handling too many aggregations

2017-07-30 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21577. -- Resolution: Invalid I don't think this JIRA describes a particular issue / suggestion but

[jira] [Commented] (SPARK-21577) Issue is handling too many aggregations

2017-07-30 Thread Kannan Subramanian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106489#comment-16106489 ] Kannan Subramanian commented on SPARK-21577: Please any one answer for this issue. Regards

[jira] [Created] (SPARK-21577) Issue is handling too many aggregations

2017-07-30 Thread Kannan Subramanian (JIRA)
Kannan Subramanian created SPARK-21577: -- Summary: Issue is handling too many aggregations Key: SPARK-21577 URL: https://issues.apache.org/jira/browse/SPARK-21577 Project: Spark Issue

[jira] [Commented] (SPARK-20433) Update jackson-databind to 2.6.7

2017-07-30 Thread David Hodeffi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106481#comment-16106481 ] David Hodeffi commented on SPARK-20433: --- I am trying to understand, why we are not updating to the

[jira] [Resolved] (SPARK-21576) Spark caching difference between 2.0.2 and 2.1.1

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21576. --- Resolution: Invalid No, JIRA isn't for questions, but for propsing concrete changes or reporting

[jira] [Created] (SPARK-21576) Spark caching difference between 2.0.2 and 2.1.1

2017-07-30 Thread Yan (JIRA)
Yan created SPARK-21576: --- Summary: Spark caching difference between 2.0.2 and 2.1.1 Key: SPARK-21576 URL: https://issues.apache.org/jira/browse/SPARK-21576 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-21256) Add WithSQLConf to Catalyst Test

2017-07-30 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh resolved SPARK-21256. - Resolution: Fixed > Add WithSQLConf to Catalyst Test >

[jira] [Resolved] (SPARK-21572) Add description on how to exit the spark-shell in the welcome message

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21572. --- Resolution: Won't Fix > Add description on how to exit the spark-shell in the welcome message >

[jira] [Commented] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

2017-07-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106368#comment-16106368 ] Sean Owen commented on SPARK-21570: --- I'm not sure how you're running it or what creates the file --

[jira] [Resolved] (SPARK-21554) Spark Hive reporting pyspark.sql.utils.AnalysisException: u'Table not found: XXX' when run on yarn cluster

2017-07-30 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21554. -- Resolution: Invalid I am resolving this as invalid. > Spark Hive reporting

[jira] [Commented] (SPARK-21533) "configure(...)" method not called when using Hive Generic UDFs

2017-07-30 Thread Dean Gurvitz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106356#comment-16106356 ] Dean Gurvitz commented on SPARK-21533: -- Takeshi, I have already found workaround for my specific