[jira] [Created] (SPARK-10623) turning on predicate pushdown throws nonsuch element exception when RDD is empty

Ram Sriharsha created SPARK-10623: - Summary: turning on predicate pushdown throws nonsuch element exception when RDD is empty Key: SPARK-10623 URL: https://issues.apache.org/jira/browse/SPARK-10623

[jira] [Updated] (SPARK-10507) reject temporal expressions such as timestamp - timestamp at parse time

[ https://issues.apache.org/jira/browse/SPARK-10507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10507: - Description: TIMESTAMP - TIMESTAMP in ISO-SQL should return an interval type which SPARK

[jira] [Created] (SPARK-10622) Race condition between scheduler and YARN executor status update

Marcelo Vanzin created SPARK-10622: -- Summary: Race condition between scheduler and YARN executor status update Key: SPARK-10622 URL: https://issues.apache.org/jira/browse/SPARK-10622 Project: Spark

[jira] [Updated] (SPARK-10623) turning on predicate pushdown throws nonsuch element exception when RDD is empty

[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram Sriharsha updated SPARK-10623: -- Assignee: Zhan Zhang > turning on predicate pushdown throws nonsuch element exception when RDD

[jira] [Commented] (SPARK-10388) Public dataset loader interface

[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745729#comment-14745729 ] Xiangrui Meng commented on SPARK-10388: --- [~lewuathe] Thanks for the discussion! Agree that it would

[jira] [Commented] (SPARK-10606) Cube/Rollup/GrpSet doesn't create the correct plan when group by is on something other than an AttributeReference

[ https://issues.apache.org/jira/browse/SPARK-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745758#comment-14745758 ] Michael Armbrust commented on SPARK-10606: -- cc [~chenghao] > Cube/Rollup/GrpSet doesn't create

[jira] [Resolved] (SPARK-10134) Improve the performance of Binary Comparison

[ https://issues.apache.org/jira/browse/SPARK-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10134. -- Resolution: Not A Problem > Improve the performance of Binary Comparison >

[jira] [Issue Comment Deleted] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Jin updated SPARK-4036: --- Comment: was deleted (was: Are we still actively working on this task? I have some work experience on CRF

[jira] [Assigned] (SPARK-10618) Refactoring and adding test for Mesos coarse-grained Scheduler

[ https://issues.apache.org/jira/browse/SPARK-10618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10618: Assignee: Apache Spark > Refactoring and adding test for Mesos coarse-grained Scheduler >

[jira] [Assigned] (SPARK-10618) Refactoring and adding test for Mesos coarse-grained Scheduler

[ https://issues.apache.org/jira/browse/SPARK-10618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10618: Assignee: (was: Apache Spark) > Refactoring and adding test for Mesos coarse-grained

[jira] [Commented] (SPARK-10618) Refactoring and adding test for Mesos coarse-grained Scheduler

[ https://issues.apache.org/jira/browse/SPARK-10618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745846#comment-14745846 ] Apache Spark commented on SPARK-10618: -- User 'SleepyThread' has created a pull request for this

[jira] [Commented] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

[ https://issues.apache.org/jira/browse/SPARK-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745875#comment-14745875 ] Brooks Johnson commented on SPARK-4226: --- I believe your last sql statement should look like this:

[jira] [Created] (SPARK-10621) Audit function names in FunctionRegistry and corresponding method names shown in functions.scala and functions.py

Yin Huai created SPARK-10621: Summary: Audit function names in FunctionRegistry and corresponding method names shown in functions.scala and functions.py Key: SPARK-10621 URL:

[jira] [Commented] (SPARK-10614) SystemClock uses non-monotonic time in its wait logic

[ https://issues.apache.org/jira/browse/SPARK-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745068#comment-14745068 ] Steve Loughran commented on SPARK-10614: Fix options? # switch {{SystemClock}} to use

[jira] [Created] (SPARK-10613) Reduce LocalNode tests dependency on SQLContext

Andrew Or created SPARK-10613: - Summary: Reduce LocalNode tests dependency on SQLContext Key: SPARK-10613 URL: https://issues.apache.org/jira/browse/SPARK-10613 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745032#comment-14745032 ] Apache Spark commented on SPARK-10611: -- User 'mingyukim' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10611: Assignee: (was: Apache Spark) > Configuration object thread safety issue in

[jira] [Assigned] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10611: Assignee: Apache Spark > Configuration object thread safety issue in NewHadoopRDD >

[jira] [Assigned] (SPARK-10613) Reduce LocalNode tests dependency on SQLContext

[ https://issues.apache.org/jira/browse/SPARK-10613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10613: Assignee: Andrew Or (was: Apache Spark) > Reduce LocalNode tests dependency on

[jira] [Assigned] (SPARK-10613) Reduce LocalNode tests dependency on SQLContext

[ https://issues.apache.org/jira/browse/SPARK-10613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10613: Assignee: Apache Spark (was: Andrew Or) > Reduce LocalNode tests dependency on

[jira] [Commented] (SPARK-10613) Reduce LocalNode tests dependency on SQLContext

[ https://issues.apache.org/jira/browse/SPARK-10613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745037#comment-14745037 ] Apache Spark commented on SPARK-10613: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10608) turn off reduce tasks locality as default to avoid bad cases

[ https://issues.apache.org/jira/browse/SPARK-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10608: Assignee: (was: Apache Spark) > turn off reduce tasks locality as default to avoid

[jira] [Assigned] (SPARK-10608) turn off reduce tasks locality as default to avoid bad cases

[ https://issues.apache.org/jira/browse/SPARK-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10608: Assignee: Apache Spark > turn off reduce tasks locality as default to avoid bad cases >

[jira] [Commented] (SPARK-10608) turn off reduce tasks locality as default to avoid bad cases

[ https://issues.apache.org/jira/browse/SPARK-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745038#comment-14745038 ] Apache Spark commented on SPARK-10608: -- User 'liyezhang556520' has created a pull request for this

[jira] [Created] (SPARK-10614) SystemClock uses non-monotonic time in its wait logic

Steve Loughran created SPARK-10614: -- Summary: SystemClock uses non-monotonic time in its wait logic Key: SPARK-10614 URL: https://issues.apache.org/jira/browse/SPARK-10614 Project: Spark

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745073#comment-14745073 ] Ivan Vergiliev commented on SPARK-2984: --- I'm also seeing this - Spark 1.3.1, Spark Standalone,

[jira] [Comment Edited] (SPARK-6613) Starting stream from checkpoint causes Streaming tab to throw error

[ https://issues.apache.org/jira/browse/SPARK-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745596#comment-14745596 ] Jeff Nadler edited comment on SPARK-6613 at 9/15/15 3:22 PM: - I'm seeing

[jira] [Assigned] (SPARK-9987) Create local aggregate operator

[ https://issues.apache.org/jira/browse/SPARK-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9987: --- Assignee: (was: Apache Spark) > Create local aggregate operator >

[jira] [Commented] (SPARK-9987) Create local aggregate operator

[ https://issues.apache.org/jira/browse/SPARK-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745615#comment-14745615 ] Apache Spark commented on SPARK-9987: - User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-6613) Starting stream from checkpoint causes Streaming tab to throw error

[ https://issues.apache.org/jira/browse/SPARK-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745596#comment-14745596 ] Jeff Nadler commented on SPARK-6613: I'm seeing almost the same problem in 1.4.1.When recovering

[jira] [Commented] (SPARK-10577) [PySpark] DataFrame hint for broadcast join

[ https://issues.apache.org/jira/browse/SPARK-10577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745620#comment-14745620 ] Apache Spark commented on SPARK-10577: -- User 'Jianfeng-chs' has created a pull request for this

[jira] [Assigned] (SPARK-10577) [PySpark] DataFrame hint for broadcast join

[ https://issues.apache.org/jira/browse/SPARK-10577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10577: Assignee: Apache Spark > [PySpark] DataFrame hint for broadcast join >

[jira] [Assigned] (SPARK-9987) Create local aggregate operator

[ https://issues.apache.org/jira/browse/SPARK-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9987: --- Assignee: Apache Spark > Create local aggregate operator > --- >

[jira] [Created] (SPARK-10620) Look into whether accumulator mechanism can replace TaskMetrics

Patrick Wendell created SPARK-10620: --- Summary: Look into whether accumulator mechanism can replace TaskMetrics Key: SPARK-10620 URL: https://issues.apache.org/jira/browse/SPARK-10620 Project: Spark

[jira] [Updated] (SPARK-10620) Look into whether accumulator mechanism can replace TaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10620: Description: This task is simply to explore whether the internal representation used by

[jira] [Commented] (SPARK-10620) Look into whether accumulator mechanism can replace TaskMetrics

[ https://issues.apache.org/jira/browse/SPARK-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745690#comment-14745690 ] Patrick Wendell commented on SPARK-10620: - /cc [~imranr] and [~srowen] for any comments. In my

[jira] [Commented] (SPARK-10591) False negative in QueryTest.checkAnswer

[ https://issues.apache.org/jira/browse/SPARK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745200#comment-14745200 ] Cheng Lian commented on SPARK-10591: Updated JIRA description. Actually we are handling {{NaN}}

[jira] [Commented] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745278#comment-14745278 ] Steve Loughran commented on SPARK-6152: --- what changes? Yes, making sure there are no regressions.

[jira] [Created] (SPARK-10616) GradientBoostedTrees stuck with 2958359 features train data

Wenmin Wu created SPARK-10616: - Summary: GradientBoostedTrees stuck with 2958359 features train data Key: SPARK-10616 URL: https://issues.apache.org/jira/browse/SPARK-10616 Project: Spark Issue

[jira] [Assigned] (SPARK-10614) SystemClock uses non-monotonic time in its wait logic

[ https://issues.apache.org/jira/browse/SPARK-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10614: Assignee: Apache Spark > SystemClock uses non-monotonic time in its wait logic >

[jira] [Assigned] (SPARK-10614) SystemClock uses non-monotonic time in its wait logic

[ https://issues.apache.org/jira/browse/SPARK-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10614: Assignee: (was: Apache Spark) > SystemClock uses non-monotonic time in its wait logic

[jira] [Commented] (SPARK-10614) SystemClock uses non-monotonic time in its wait logic

[ https://issues.apache.org/jira/browse/SPARK-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745239#comment-14745239 ] Apache Spark commented on SPARK-10614: -- User 'steveloughran' has created a pull request for this

[jira] [Resolved] (SPARK-10491) move RowMatrix.dspr to BLAS

[ https://issues.apache.org/jira/browse/SPARK-10491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10491. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8663

[jira] [Assigned] (SPARK-10577) [PySpark] DataFrame hint for broadcast join

[ https://issues.apache.org/jira/browse/SPARK-10577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10577: Assignee: (was: Apache Spark) > [PySpark] DataFrame hint for broadcast join >

[jira] [Updated] (SPARK-10300) Use tags to control which tests to run depending on changes being tested

[ https://issues.apache.org/jira/browse/SPARK-10300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-10300: --- Assignee: Marcelo Vanzin Fix Version/s: 1.6.0 Issue Type: Improvement (was:

[jira] [Resolved] (SPARK-10300) Use tags to control which tests to run depending on changes being tested

[ https://issues.apache.org/jira/browse/SPARK-10300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10300. Resolution: Fixed > Use tags to control which tests to run depending on changes being

[jira] [Created] (SPARK-10619) Can't sort columns on Executor Page

Thomas Graves created SPARK-10619: - Summary: Can't sort columns on Executor Page Key: SPARK-10619 URL: https://issues.apache.org/jira/browse/SPARK-10619 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-10511) Source releases should not include maven jars

[ https://issues.apache.org/jira/browse/SPARK-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10511: Assignee: Luciano Resende > Source releases should not include maven jars >

[jira] [Commented] (SPARK-3181) Add Robust Regression Algorithm with Huber Estimator

[ https://issues.apache.org/jira/browse/SPARK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745992#comment-14745992 ] Apache Spark commented on SPARK-3181: - User 'fjiang6' has created a pull request for this issue:

[jira] [Commented] (SPARK-3181) Add Robust Regression Algorithm with Huber Estimator

[ https://issues.apache.org/jira/browse/SPARK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745993#comment-14745993 ] Apache Spark commented on SPARK-3181: - User 'fjiang6' has created a pull request for this issue:

[jira] [Updated] (SPARK-10535) Support for recommendUsersForProducts and recommendProductsForUsers in matrix factorization model for PySpark

[ https://issues.apache.org/jira/browse/SPARK-10535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10535: -- Assignee: Vladimir Vladimirov > Support for recommendUsersForProducts and

[jira] [Updated] (SPARK-10623) turning on predicate pushdown throws nonsuch element exception when RDD is empty

[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10623: Component/s: SQL > turning on predicate pushdown throws nonsuch element exception when RDD

[jira] [Resolved] (SPARK-10437) Support aggregation expressions in Order By

[ https://issues.apache.org/jira/browse/SPARK-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10437. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8599

[jira] [Reopened] (SPARK-10300) Use tags to control which tests to run depending on changes being tested

[ https://issues.apache.org/jira/browse/SPARK-10300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-10300: > Use tags to control which tests to run depending on changes being tested >

[jira] [Updated] (SPARK-10534) ORDER BY clause allows only columns that are present in SELECT statement

[ https://issues.apache.org/jira/browse/SPARK-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10534: - Description: When invoking query SELECT EmployeeID from Employees order by

[jira] [Updated] (SPARK-10504) aggregate where NULL is defined as the value expression aborts when SUM used

[ https://issues.apache.org/jira/browse/SPARK-10504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10504: - Description: In ISO-SQL the context would determine an implicit type for NULL or one

[jira] [Updated] (SPARK-10505) windowed form of count ( star ) fails with No handler for udf class

[ https://issues.apache.org/jira/browse/SPARK-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10505: - Description: The following statement will parse/execute in Hive 0.13 but fails in SPARK.

[jira] [Commented] (SPARK-10503) incorrect predicate evaluation involving NULL value

[ https://issues.apache.org/jira/browse/SPARK-10503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746098#comment-14746098 ] Michael Armbrust commented on SPARK-10503: -- Spark version? > incorrect predicate evaluation

[jira] [Commented] (SPARK-10504) aggregate where NULL is defined as the value expression aborts when SUM used

[ https://issues.apache.org/jira/browse/SPARK-10504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746097#comment-14746097 ] Michael Armbrust commented on SPARK-10504: -- I believe this is fixed in Spark 1.5? > aggregate

[jira] [Commented] (SPARK-10485) IF expression is not correctly resolved when one of the options have NullType

[ https://issues.apache.org/jira/browse/SPARK-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746102#comment-14746102 ] Michael Armbrust commented on SPARK-10485: -- I think this is fixed in Spark 1.5? > IF expression

[jira] [Resolved] (SPARK-6946) Add visualization of logical and physical plans for SQL/DataFrames

[ https://issues.apache.org/jira/browse/SPARK-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-6946. -- Resolution: Fixed Fix Version/s: 1.5.0 Target Version/s: 1.5.0 > Add visualization of

[jira] [Reopened] (SPARK-6943) Graphically show the RDD DAG on the UI

[ https://issues.apache.org/jira/browse/SPARK-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-6943: -- > Graphically show the RDD DAG on the UI > -- > > Key:

[jira] [Resolved] (SPARK-6944) Mechanism to associate generic operator scope with RDD's

[ https://issues.apache.org/jira/browse/SPARK-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-6944. -- Resolution: Fixed > Mechanism to associate generic operator scope with RDD's >

[jira] [Resolved] (SPARK-6945) Provide SQL tab in the Spark UI

[ https://issues.apache.org/jira/browse/SPARK-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-6945. -- Resolution: Fixed Fix Version/s: 1.5.0 Target Version/s: 1.5.0 > Provide SQL tab in

[jira] [Commented] (SPARK-10511) Source releases should not include maven jars

[ https://issues.apache.org/jira/browse/SPARK-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745976#comment-14745976 ] Apache Spark commented on SPARK-10511: -- User 'lresende' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10511) Source releases should not include maven jars

[ https://issues.apache.org/jira/browse/SPARK-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10511: Assignee: (was: Apache Spark) > Source releases should not include maven jars >

[jira] [Assigned] (SPARK-10511) Source releases should not include maven jars

[ https://issues.apache.org/jira/browse/SPARK-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10511: Assignee: Apache Spark > Source releases should not include maven jars >

[jira] [Updated] (SPARK-10535) Support for recommendUsersForProducts and recommendProductsForUsers in matrix factorization model for PySpark

[ https://issues.apache.org/jira/browse/SPARK-10535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10535: -- Shepherd: Xiangrui Meng Target Version/s: 1.6.0 Issue Type: New Feature

[jira] [Resolved] (SPARK-4678) A SQL query with subquery fails with TreeNodeException

[ https://issues.apache.org/jira/browse/SPARK-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4678. - Resolution: Cannot Reproduce I can't reproduce this. Please reopen if you can with Spark

[jira] [Resolved] (SPARK-4758) Make metastore_db in-memory for HiveContext

[ https://issues.apache.org/jira/browse/SPARK-4758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4758. - Resolution: Won't Fix I'm going to close this stale ticket. Please reopen if you intend

[jira] [Resolved] (SPARK-4629) Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing Parquet files

[ https://issues.apache.org/jira/browse/SPARK-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4629. - Resolution: Fixed > Spark SQL uses Hadoop Configuration in a thread-unsafe manner when

[jira] [Resolved] (SPARK-4559) Adding support for ucase and lcase

[ https://issues.apache.org/jira/browse/SPARK-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4559. - Resolution: Fixed Fix Version/s: 1.5.0 we have {{upper}} and {{lower}} now. >

[jira] [Resolved] (SPARK-4780) Support executing multiple statements in sql(...)

[ https://issues.apache.org/jira/browse/SPARK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4780. - Resolution: Won't Fix This is a limitation of the Hive parser. I believe that using the

[jira] [Resolved] (SPARK-5084) HiveQuerySuite throws when using MySQL as Hive metastore storage

[ https://issues.apache.org/jira/browse/SPARK-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5084. - Resolution: Cannot Reproduce I think this is fixed in 1.5. Please reopen if you can

[jira] [Resolved] (SPARK-5109) Loading multiple parquet files into a single SchemaRDD

[ https://issues.apache.org/jira/browse/SPARK-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5109. - Resolution: Fixed Fix Version/s: 1.5.0 Methods for passing a list of files is now

[jira] [Resolved] (SPARK-5139) select table_alias.* with joins and selecting column names from inner queries not supported

[ https://issues.apache.org/jira/browse/SPARK-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5139. - Resolution: Cannot Reproduce I believe this is fixed in Spark 1.5. Please reopen if you

[jira] [Resolved] (SPARK-5060) Spark driver main thread hanging after SQL insert in Parquet file

[ https://issues.apache.org/jira/browse/SPARK-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5060. - Resolution: Cannot Reproduce This code has changed a lot in Spark 1.5, so I'm going to

[jira] [Resolved] (SPARK-5030) Approximated cardinality with HyperLogLog UDAF

[ https://issues.apache.org/jira/browse/SPARK-5030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5030. - Resolution: Duplicate > Approximated cardinality with HyperLogLog UDAF >

[jira] [Resolved] (SPARK-4278) SparkSQL job failing with java.lang.ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-4278. - Resolution: Fixed Assignee: Yin Huai Fix Version/s: 1.5.0 In Spark 1.5, if the type of

[jira] [Resolved] (SPARK-4869) The variable names in IF statement of Spark SQL doesn't resolve to its value.

[ https://issues.apache.org/jira/browse/SPARK-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4869. - Resolution: Cannot Reproduce I can't reproduce this in Spark 1.5. Please reopen if you

[jira] [Resolved] (SPARK-4794) Wrong parse of GROUP BY query

[ https://issues.apache.org/jira/browse/SPARK-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4794. - Resolution: Cannot Reproduce I think we have fixed our resolution logic here, but please

[jira] [Resolved] (SPARK-4838) StackOverflowError when serialization task

[ https://issues.apache.org/jira/browse/SPARK-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4838. - Resolution: Cannot Reproduce I'm going to close this barring any new information on how

[jira] [Resolved] (SPARK-8152) Dataframe Join Ignores Condition

[ https://issues.apache.org/jira/browse/SPARK-8152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8152. - Resolution: Cannot Reproduce I can't reproduce this in Spark 1.5, but please reopen if

[jira] [Resolved] (SPARK-8165) sqlContext.createDataFrame(dataWithoutHeader, csvSchema) type conversion error after .cache

[ https://issues.apache.org/jira/browse/SPARK-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8165. - Resolution: Not A Problem This is perhaps confusing, but the issue here is that caching

[jira] [Resolved] (SPARK-8109) TestSQLContext's static initialization is run during MiMa tests, causing SparkContexts to be created

[ https://issues.apache.org/jira/browse/SPARK-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8109. - Resolution: Fixed I think [~andrewor14] has removed TestSQLContext so I'm closing this

[jira] [Commented] (SPARK-8165) sqlContext.createDataFrame(dataWithoutHeader, csvSchema) type conversion error after .cache

[ https://issues.apache.org/jira/browse/SPARK-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746269#comment-14746269 ] Michael Armbrust commented on SPARK-8165: - I'd suggest looking at

[jira] [Resolved] (SPARK-5421) SparkSql throw OOM at shuffle

[ https://issues.apache.org/jira/browse/SPARK-5421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5421. - Resolution: Fixed Fix Version/s: 1.5.0 Spark 1.5 should have taken care of this.

[jira] [Resolved] (SPARK-5397) Assigning aliases to several return values of an UDF

[ https://issues.apache.org/jira/browse/SPARK-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5397. - Resolution: Fixed Fix Version/s: 1.5.0 Tested in 1.5 and this query now parses

[jira] [Resolved] (SPARK-5410) Error parsing scientific notation in a select statement

[ https://issues.apache.org/jira/browse/SPARK-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5410. - Resolution: Fixed I tested and in Spark 1.5 the HiveContext can parse {{2.2E10}}.

[jira] [Commented] (SPARK-5391) SparkSQL fails to create tables with custom JSON SerDe

[ https://issues.apache.org/jira/browse/SPARK-5391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746285#comment-14746285 ] Michael Armbrust commented on SPARK-5391: - Is this still a problem? Is there a reason you aren't

[jira] [Resolved] (SPARK-5314) java.lang.OutOfMemoryError in SparkSQL with GROUP BY

[ https://issues.apache.org/jira/browse/SPARK-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5314. - Resolution: Fixed Fix Version/s: 1.5.0 1.5 now uses sort based aggregation with

[jira] [Resolved] (SPARK-5306) Support for a NotEqualsFilter in the filter PrunedFilteredScan pushdown

[ https://issues.apache.org/jira/browse/SPARK-5306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5306. - Resolution: Fixed Fix Version/s: 1.3.0 We added {{Not}} in Spark 1.3. > Support

[jira] [Resolved] (SPARK-5302) Add support for SQLContext "partition" columns

[ https://issues.apache.org/jira/browse/SPARK-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5302. - Resolution: Fixed Fix Version/s: 1.4.0 This is supported now. > Add support for

[jira] [Resolved] (SPARK-1369) HiveUDF wrappers are slow

[ https://issues.apache.org/jira/browse/SPARK-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1369. - Resolution: Won't Fix As we add more and more native functions this is less important. >

[jira] [Resolved] (SPARK-2068) Remove other uses of @transient lazy val in physical plan nodes

[ https://issues.apache.org/jira/browse/SPARK-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2068. - Resolution: Won't Fix > Remove other uses of @transient lazy val in physical plan nodes >

[jira] [Resolved] (SPARK-2063) Creating a SchemaRDD via sql() does not correctly resolve nested types

[ https://issues.apache.org/jira/browse/SPARK-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2063. - Resolution: Fixed Assignee: Michael Armbrust (was: Cheng Lian) Fix

[jira] [Resolved] (SPARK-1363) Add streaming support for Spark SQL module

[ https://issues.apache.org/jira/browse/SPARK-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1363. - Resolution: Duplicate > Add streaming support for Spark SQL module >

[jira] [Resolved] (SPARK-1893) Explain why running Spark Hive locally freezes

[ https://issues.apache.org/jira/browse/SPARK-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1893. - Resolution: Cannot Reproduce Closing stale issue. > Explain why running Spark Hive

[jira] [Resolved] (SPARK-2607) SchemaRDD unionall prevents caching

[ https://issues.apache.org/jira/browse/SPARK-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2607. - Resolution: Fixed Assignee: Michael Armbrust Fix Version/s: 1.5.0 I

[jira] [Resolved] (SPARK-2695) Figure out a good way to handle NullType columns.

[ https://issues.apache.org/jira/browse/SPARK-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2695. - Resolution: Fixed Fix Version/s: 1.5.0 The original problem here (saving null to

<    1   2   3   >