[jira] [Commented] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400981#comment-15400981 ] Apache Spark commented on SPARK-16818: -- User 'ericl' has created a pull request for this issue:

[jira] [Commented] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400966#comment-15400966 ] Reynold Xin commented on SPARK-16818: - I've merged this in master, but this still needs to be merged

[jira] [Updated] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16818: Fix Version/s: (was: 2.0.1) > Exchange reuse incorrectly reuses scans over different sets of

[jira] [Resolved] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16818. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400942#comment-15400942 ] Xiao Li commented on SPARK-16275: - It sounds like both of you are fine to remove the Hive's hash UDF.

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400924#comment-15400924 ] Wenchen Fan commented on SPARK-16275: - It's weird if we have 2 hash implementations, one for hive

[jira] [Updated] (SPARK-16819) Exception in thread “main” org.apache.spark.SparkException: Application application finished with failed status

[ https://issues.apache.org/jira/browse/SPARK-16819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asmaa Ali updated SPARK-16819: --- Description: What is the reason of this exception ?!

[jira] [Updated] (SPARK-16819) Exception in thread “main” org.apache.spark.SparkException: Application application finished with failed status

[ https://issues.apache.org/jira/browse/SPARK-16819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asmaa Ali updated SPARK-16819: --- Description: What is the reason of this exception ?!

[jira] [Created] (SPARK-16819) Exception in thread “main” org.apache.spark.SparkException: Application application finished with failed status

Asmaa Ali created SPARK-16819: -- Summary: Exception in thread “main” org.apache.spark.SparkException: Application application finished with failed status Key: SPARK-16819 URL:

[jira] [Commented] (SPARK-16475) Broadcast Hint for SQL Queries

[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400881#comment-15400881 ] Apache Spark commented on SPARK-16475: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16818: Assignee: Apache Spark > Exchange reuse incorrectly reuses scans over different sets of

[jira] [Commented] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400853#comment-15400853 ] Apache Spark commented on SPARK-16818: -- User 'ericl' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16818: Assignee: (was: Apache Spark) > Exchange reuse incorrectly reuses scans over

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400854#comment-15400854 ] Xiao Li commented on SPARK-16275: -

[jira] [Created] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

Eric Liang created SPARK-16818: -- Summary: Exchange reuse incorrectly reuses scans over different sets of partitions Key: SPARK-16818 URL: https://issues.apache.org/jira/browse/SPARK-16818 Project: Spark

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400849#comment-15400849 ] Xiao Li commented on SPARK-16275: - Let me check it. Thanks! > Implement all the Hive fallback functions

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400846#comment-15400846 ] Reynold Xin commented on SPARK-16275: - How difficult would it be to provide a native hash

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400844#comment-15400844 ] Xiao Li commented on SPARK-16275: - Yeah, many queries are using it. Below is the list: auto_join_nulls

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400783#comment-15400783 ] Reynold Xin commented on SPARK-16275: - What do we use Hive's hash function for? Are there queries in

[jira] [Created] (SPARK-16817) Enable storing of shuffle data in Alluxio

Tim Bisson created SPARK-16817: -- Summary: Enable storing of shuffle data in Alluxio Key: SPARK-16817 URL: https://issues.apache.org/jira/browse/SPARK-16817 Project: Spark Issue Type: New

[jira] [Comment Edited] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400751#comment-15400751 ] snehil suresh wakchaure edited comment on SPARK-5992 at 7/30/16 5:43 PM:

[jira] [Commented] (SPARK-16275) Implement all the Hive fallback functions

[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400756#comment-15400756 ] Xiao Li commented on SPARK-16275: - [~rxin] What is the plan for {{hash}}? If we use our version, it

[jira] [Comment Edited] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400751#comment-15400751 ] snehil suresh wakchaure edited comment on SPARK-5992 at 7/30/16 5:28 PM:

[jira] [Commented] (SPARK-16518) Schema Compatibility of Parquet Data Source

[ https://issues.apache.org/jira/browse/SPARK-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400755#comment-15400755 ] Chanh Le commented on SPARK-16518: -- Did we have a patch for that? Right now I have this error too. >

[jira] [Comment Edited] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400751#comment-15400751 ] snehil suresh wakchaure edited comment on SPARK-5992 at 7/30/16 5:31 PM:

[jira] [Commented] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400751#comment-15400751 ] snehil suresh wakchaure commented on SPARK-5992: Hello, just curious to know if I can

[jira] [Resolved] (SPARK-16800) Fix Java Examples that throw exception

[ https://issues.apache.org/jira/browse/SPARK-16800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16800. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16800) Fix Java Examples that throw exception

[ https://issues.apache.org/jira/browse/SPARK-16800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16800: -- Assignee: Bryan Cutler > Fix Java Examples that throw exception >

[jira] [Updated] (SPARK-16696) unused broadcast variables should call destroy instead of unpersist

[ https://issues.apache.org/jira/browse/SPARK-16696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16696: -- Assignee: Weichen Xu Affects Version/s: (was: 2.1.0) Priority: Minor

[jira] [Resolved] (SPARK-16696) unused broadcast variables should call destroy instead of unpersist

[ https://issues.apache.org/jira/browse/SPARK-16696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16696. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14333

[jira] [Commented] (SPARK-16816) Add api to get JavaSparkContext from SparkSession

[ https://issues.apache.org/jira/browse/SPARK-16816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400681#comment-15400681 ] Apache Spark commented on SPARK-16816: -- User 'phalodi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16816) Add api to get JavaSparkContext from SparkSession

[ https://issues.apache.org/jira/browse/SPARK-16816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16816: Assignee: Apache Spark > Add api to get JavaSparkContext from SparkSession >

[jira] [Assigned] (SPARK-16816) Add api to get JavaSparkContext from SparkSession

[ https://issues.apache.org/jira/browse/SPARK-16816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16816: Assignee: (was: Apache Spark) > Add api to get JavaSparkContext from SparkSession >

[jira] [Created] (SPARK-16816) Add api to get JavaSparkContext from SparkSession

sandeep purohit created SPARK-16816: --- Summary: Add api to get JavaSparkContext from SparkSession Key: SPARK-16816 URL: https://issues.apache.org/jira/browse/SPARK-16816 Project: Spark

[jira] [Commented] (SPARK-14204) [SQL] Failure to register URL-derived JDBC driver on executors in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-14204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400668#comment-15400668 ] Apache Spark commented on SPARK-14204: -- User 'mchalek' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-16700) StructType doesn't accept Python dicts anymore

[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400658#comment-15400658 ] Jay Teguh Wijaya Purwanto edited comment on SPARK-16700 at 7/30/16 12:34 PM:

[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400658#comment-15400658 ] Jay Teguh Wijaya Purwanto commented on SPARK-16700: --- When using `Row` object, but with

[jira] [Resolved] (SPARK-16694) Use for/foreach rather than map for Unit expressions whose side effects are required

[ https://issues.apache.org/jira/browse/SPARK-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16694. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14332

[jira] [Closed] (SPARK-16797) Repartiton call w/ 0 partitions drops data

[ https://issues.apache.org/jira/browse/SPARK-16797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-16797. - Resolution: Duplicate Fix Version/s: (was: 2.0.0) > Repartiton call w/ 0 partitions drops

[jira] [Reopened] (SPARK-16797) Repartiton call w/ 0 partitions drops data

[ https://issues.apache.org/jira/browse/SPARK-16797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-16797: --- > Repartiton call w/ 0 partitions drops data > -- > >

[jira] [Created] (SPARK-16815) Dataset[List[T]] leads to ArrayStoreException

TobiasP created SPARK-16815: --- Summary: Dataset[List[T]] leads to ArrayStoreException Key: SPARK-16815 URL: https://issues.apache.org/jira/browse/SPARK-16815 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16797) Repartiton call w/ 0 partitions drops data

[ https://issues.apache.org/jira/browse/SPARK-16797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400542#comment-15400542 ] Dongjoon Hyun commented on SPARK-16797: --- Oh, I see. Never mind, [~bjeffrey]. > Repartiton call w/

[jira] [Commented] (SPARK-16807) Optimize some ABS() statements

[ https://issues.apache.org/jira/browse/SPARK-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400532#comment-15400532 ] Kazuaki Ishizaki commented on SPARK-16807: -- Interesting if we can ensure {{x - y}} is not

[jira] [Commented] (SPARK-16814) Fix deprecated use of ParquetWriter in Parquet test suites

[ https://issues.apache.org/jira/browse/SPARK-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400512#comment-15400512 ] Apache Spark commented on SPARK-16814: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16814) Fix deprecated use of ParquetWriter in Parquet test suites

[ https://issues.apache.org/jira/browse/SPARK-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16814: Assignee: Apache Spark > Fix deprecated use of ParquetWriter in Parquet test suites >

[jira] [Assigned] (SPARK-16814) Fix deprecated use of ParquetWriter in Parquet test suites

[ https://issues.apache.org/jira/browse/SPARK-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16814: Assignee: (was: Apache Spark) > Fix deprecated use of ParquetWriter in Parquet test

[jira] [Created] (SPARK-16814) Fix deprecated use of ParquetWriter in Parquet test suites

holdenk created SPARK-16814: --- Summary: Fix deprecated use of ParquetWriter in Parquet test suites Key: SPARK-16814 URL: https://issues.apache.org/jira/browse/SPARK-16814 Project: Spark Issue Type: