[jira] [Assigned] (SPARK-17167) Issue Exceptions when Analyze Table on In-Memory Cataloged Tables

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17167: Assignee: (was: Apache Spark) > Issue Exceptions when Analyze Table on In-Memory Catal

[jira] [Commented] (SPARK-17167) Issue Exceptions when Analyze Table on In-Memory Cataloged Tables

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429266#comment-15429266 ] Apache Spark commented on SPARK-17167: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-17167) Issue Exceptions when Analyze Table on In-Memory Cataloged Tables

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17167: Assignee: Apache Spark > Issue Exceptions when Analyze Table on In-Memory Cataloged Tables

[jira] [Commented] (SPARK-16961) Utils.randomizeInPlace does not shuffle arrays uniformly

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429280#comment-15429280 ] Apache Spark commented on SPARK-16961: -- User 'yanboliang' has created a pull request

[jira] [Commented] (SPARK-17148) NodeManager exit because of exception “Executor is not registered”

2016-08-20 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429282#comment-15429282 ] cen yuhai commented on SPARK-17148: --- I don't know the root cause right now, I can't und

[jira] [Commented] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429326#comment-15429326 ] Herman van Hovell commented on SPARK-17164: --- I tried this in Hive enabled Spark

[jira] [Comment Edited] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429326#comment-15429326 ] Herman van Hovell edited comment on SPARK-17164 at 8/20/16 11:13 AM: --

[jira] [Commented] (SPARK-17159) Improve FileInputDStream.findNewFiles list performance

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429329#comment-15429329 ] Apache Spark commented on SPARK-17159: -- User 'steveloughran' has created a pull requ

[jira] [Assigned] (SPARK-17159) Improve FileInputDStream.findNewFiles list performance

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17159: Assignee: Apache Spark > Improve FileInputDStream.findNewFiles list performance >

[jira] [Assigned] (SPARK-17159) Improve FileInputDStream.findNewFiles list performance

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17159: Assignee: (was: Apache Spark) > Improve FileInputDStream.findNewFiles list performance

[jira] [Created] (SPARK-17168) CSV with header is incorrectly read if file is partitioned

2016-08-20 Thread Mathieu D (JIRA)
Mathieu D created SPARK-17168: - Summary: CSV with header is incorrectly read if file is partitioned Key: SPARK-17168 URL: https://issues.apache.org/jira/browse/SPARK-17168 Project: Spark Issue Ty

[jira] [Commented] (SPARK-17086) QuantileDiscretizer throws InvalidArgumentException (parameter splits given invalid value) on valid data

2016-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429348#comment-15429348 ] Sean Owen commented on SPARK-17086: --- Yeah sounds good -- feel free to make a PR. > Qua

[jira] [Created] (SPARK-17169) To use scala macros to update code when SharedParamsCodeGen.scala changed

2016-08-20 Thread Qian Huang (JIRA)
Qian Huang created SPARK-17169: -- Summary: To use scala macros to update code when SharedParamsCodeGen.scala changed Key: SPARK-17169 URL: https://issues.apache.org/jira/browse/SPARK-17169 Project: Spark

[jira] [Updated] (SPARK-16320) Document G1 heap region's effect on spark 2.0 vs 1.6

2016-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16320: -- Assignee: Sean Owen Priority: Minor (was: Critical) Component/s: Documentation Issue

[jira] [Commented] (SPARK-16320) Document G1 heap region's effect on spark 2.0 vs 1.6

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429357#comment-15429357 ] Apache Spark commented on SPARK-16320: -- User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-17046) prevent user using dataframe.select with empty param list

2016-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17046: -- Affects Version/s: (was: 2.1.0) Priority: Minor (was: Major) > prevent user using dat

[jira] [Resolved] (SPARK-17046) prevent user using dataframe.select with empty param list

2016-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17046. --- Resolution: Won't Fix > prevent user using dataframe.select with empty param list > -

[jira] [Created] (SPARK-17170) Enable whole partition pruning for InMemoryTableScanExec

2016-08-20 Thread Patrick Woody (JIRA)
Patrick Woody created SPARK-17170: - Summary: Enable whole partition pruning for InMemoryTableScanExec Key: SPARK-17170 URL: https://issues.apache.org/jira/browse/SPARK-17170 Project: Spark Is

[jira] [Assigned] (SPARK-17170) Enable whole partition pruning for InMemoryTableScanExec

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17170: Assignee: (was: Apache Spark) > Enable whole partition pruning for InMemoryTableScanEx

[jira] [Assigned] (SPARK-17170) Enable whole partition pruning for InMemoryTableScanExec

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17170: Assignee: Apache Spark > Enable whole partition pruning for InMemoryTableScanExec > --

[jira] [Commented] (SPARK-17170) Enable whole partition pruning for InMemoryTableScanExec

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429390#comment-15429390 ] Apache Spark commented on SPARK-17170: -- User 'pwoody' has created a pull request for

[jira] [Commented] (SPARK-16508) Fix documentation warnings found by R CMD check

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429401#comment-15429401 ] Apache Spark commented on SPARK-16508: -- User 'felixcheung' has created a pull reques

[jira] [Created] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread cen yuhai (JIRA)
cen yuhai created SPARK-17171: - Summary: DAG will list all partitions in the graph Key: SPARK-17171 URL: https://issues.apache.org/jira/browse/SPARK-17171 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-17171: -- Attachment: dag2.png dag1.png > DAG will list all partitions in the graph > ---

[jira] [Updated] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-17171: -- Description: When querying data from a partitioned table, DAG will list all partitions in the graph.It

[jira] [Updated] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-17171: -- Description: When querying data from a partitioned table, DAG will list all partitions in the graph.It

[jira] [Updated] (SPARK-17104) LogicalRelation.newInstance should follow the semantics of MultiInstanceRelation

2016-08-20 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17104: Assignee: Liang-Chi Hsieh > LogicalRelation.newInstance should follow the semantics of > MultiInst

[jira] [Resolved] (SPARK-17104) LogicalRelation.newInstance should follow the semantics of MultiInstanceRelation

2016-08-20 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17104. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull req

[jira] [Commented] (SPARK-17168) CSV with header is incorrectly read if file is partitioned

2016-08-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429435#comment-15429435 ] Takeshi Yamamuro commented on SPARK-17168: -- Why is having a header in each part

[jira] [Updated] (SPARK-17124) RelationalGroupedDataset.agg should be order preserving and allow duplicate column names

2016-08-20 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17124: Assignee: Peter Lee > RelationalGroupedDataset.agg should be order preserving and allow duplicate

[jira] [Resolved] (SPARK-17124) RelationalGroupedDataset.agg should be order preserving and allow duplicate column names

2016-08-20 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17124. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull req

[jira] [Updated] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17171: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > DAG will list all partitions

[jira] [Commented] (SPARK-17168) CSV with header is incorrectly read if file is partitioned

2016-08-20 Thread Mathieu D (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429443#comment-15429443 ] Mathieu D commented on SPARK-17168: --- This is error-prone, because the scenario I show w

[jira] [Created] (SPARK-17172) pyspak hiveContext can not create UDF: Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.

2016-08-20 Thread Andrew Davidson (JIRA)
Andrew Davidson created SPARK-17172: --- Summary: pyspak hiveContext can not create UDF: Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext. Key: SPARK-17172 URL: https://iss

[jira] [Commented] (SPARK-17172) pyspak hiveContext can not create UDF: Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.

2016-08-20 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429463#comment-15429463 ] Andrew Davidson commented on SPARK-17172: - related bug report : https://issues.ap

[jira] [Commented] (SPARK-17172) pyspak hiveContext can not create UDF: Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.

2016-08-20 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429465#comment-15429465 ] Andrew Davidson commented on SPARK-17172: - attached a notebook that demonstrates

[jira] [Updated] (SPARK-17172) pyspak hiveContext can not create UDF: Py4JJavaError: An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.

2016-08-20 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson updated SPARK-17172: Attachment: hiveUDFBug.ipynb hiveUDFBug.html > pyspak hiveContext can not c

[jira] [Commented] (SPARK-17163) Decide on unified multinomial and binary logistic regression interfaces

2016-08-20 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429489#comment-15429489 ] Seth Hendrickson commented on SPARK-17163: -- Just to sum up some key points: 1.

[jira] [Assigned] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17024: Assignee: Apache Spark > Weird behaviour of the DataFrame when a column name contains dots

[jira] [Assigned] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17024: Assignee: (was: Apache Spark) > Weird behaviour of the DataFrame when a column name co

[jira] [Commented] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429500#comment-15429500 ] Apache Spark commented on SPARK-17024: -- User 'izeigerman' has created a pull request

[jira] [Updated] (SPARK-12666) spark-shell --packages cannot load artifacts which are publishLocal'd by SBT

2016-08-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12666: --- Assignee: Bryan Cutler > spark-shell --packages cannot load artifacts which are publishLocal'd by SBT

[jira] [Resolved] (SPARK-12666) spark-shell --packages cannot load artifacts which are publishLocal'd by SBT

2016-08-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-12666. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Fixed for 2.0.1 and 2.1.0 by

[jira] [Created] (SPARK-17173) Refactor R mllib for easier ml implementations

2016-08-20 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-17173: Summary: Refactor R mllib for easier ml implementations Key: SPARK-17173 URL: https://issues.apache.org/jira/browse/SPARK-17173 Project: Spark Issue Type: Im

[jira] [Assigned] (SPARK-17173) Refactor R mllib for easier ml implementations

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17173: Assignee: (was: Apache Spark) > Refactor R mllib for easier ml implementations > -

[jira] [Commented] (SPARK-17173) Refactor R mllib for easier ml implementations

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429515#comment-15429515 ] Apache Spark commented on SPARK-17173: -- User 'felixcheung' has created a pull reques

[jira] [Assigned] (SPARK-17173) Refactor R mllib for easier ml implementations

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17173: Assignee: Apache Spark > Refactor R mllib for easier ml implementations >

[jira] [Resolved] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-20 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-17090. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14717 [https://github.com/ap

[jira] [Created] (SPARK-17174) Provide support for Timestamp type Column in add_months function to return HH:mm:ss

2016-08-20 Thread Amit Baghel (JIRA)
Amit Baghel created SPARK-17174: --- Summary: Provide support for Timestamp type Column in add_months function to return HH:mm:ss Key: SPARK-17174 URL: https://issues.apache.org/jira/browse/SPARK-17174 Pro

[jira] [Commented] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429577#comment-15429577 ] Apache Spark commented on SPARK-17171: -- User 'cenyuhai' has created a pull request f

[jira] [Assigned] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17171: Assignee: (was: Apache Spark) > DAG will list all partitions in the graph > --

[jira] [Assigned] (SPARK-17171) DAG will list all partitions in the graph

2016-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17171: Assignee: Apache Spark > DAG will list all partitions in the graph > -

[jira] [Updated] (SPARK-17174) Provide support for Timestamp type Column in add_months function to return HH:mm:ss

2016-08-20 Thread Amit Baghel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Baghel updated SPARK-17174: Description: add_months function currently supports Date types. If Column is Timestamp type then i

[jira] [Updated] (SPARK-17174) Provide support for Timestamp type Column in add_months function to return HH:mm:ss

2016-08-20 Thread Amit Baghel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Baghel updated SPARK-17174: Description: add_months function currently supports Date types. If Column is Timestamp type then i

[jira] [Created] (SPARK-17175) Add a expert formula as default value to aggregationDepth of SharedParam

2016-08-20 Thread Qian Huang (JIRA)
Qian Huang created SPARK-17175: -- Summary: Add a expert formula as default value to aggregationDepth of SharedParam Key: SPARK-17175 URL: https://issues.apache.org/jira/browse/SPARK-17175 Project: Spark

[jira] [Updated] (SPARK-17175) Add a expert formula to aggregationDepth of SharedParam

2016-08-20 Thread Qian Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qian Huang updated SPARK-17175: --- Summary: Add a expert formula to aggregationDepth of SharedParam (was: Add a expert formula as defau

[jira] [Created] (SPARK-17176) Task are sorted by "Index" in Stage Page.

2016-08-20 Thread cen yuhai (JIRA)
cen yuhai created SPARK-17176: - Summary: Task are sorted by "Index" in Stage Page. Key: SPARK-17176 URL: https://issues.apache.org/jira/browse/SPARK-17176 Project: Spark Issue Type: Improvement