[jira] [Comment Edited] (SPARK-13298) DAG visualization does not render correctly for jobs

2016-02-15 Thread Lucas Woltmann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148232#comment-15148232 ] Lucas Woltmann edited comment on SPARK-13298 at 2/16/16 7:57 AM: - Looks

[jira] [Updated] (SPARK-13298) DAG visualization does not render correctly for jobs

2016-02-15 Thread Lucas Woltmann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Woltmann updated SPARK-13298: --- Attachment: dag_full.png > DAG visualization does not render correctly for jobs >

[jira] [Commented] (SPARK-13298) DAG visualization does not render correctly for jobs

2016-02-15 Thread Lucas Woltmann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148232#comment-15148232 ] Lucas Woltmann commented on SPARK-13298: Looks like .cache() breaks it. DAG without cache():

[jira] [Comment Edited] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-15 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148205#comment-15148205 ] dylanzhou edited comment on SPARK-13183 at 2/16/16 7:48 AM: @Sean Owen maybe

[jira] [Comment Edited] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-15 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148205#comment-15148205 ] dylanzhou edited comment on SPARK-13183 at 2/16/16 7:46 AM: @Sean Owen maybe

[jira] [Comment Edited] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-15 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148205#comment-15148205 ] dylanzhou edited comment on SPARK-13183 at 2/16/16 7:45 AM: There is a memory

[jira] [Resolved] (SPARK-13221) GroupingSets Returns an Incorrect Results

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-13221. - Resolution: Fixed > GroupingSets Returns an Incorrect Results >

[jira] [Reopened] (SPARK-13221) GroupingSets Returns an Incorrect Results

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-13221: - > GroupingSets Returns an Incorrect Results > - > >

[jira] [Resolved] (SPARK-13221) GroupingSets Returns an Incorrect Results

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-13221. - Resolution: Resolved Fix Version/s: 2.0.0 > GroupingSets Returns an Incorrect Results >

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148212#comment-15148212 ] Xiao Li commented on SPARK-1: - Tried join, intersect and except in 2.0. Works fine! For example,

[jira] [Comment Edited] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-15 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148205#comment-15148205 ] dylanzhou edited comment on SPARK-13183 at 2/16/16 7:36 AM: There is a memory

[jira] [Reopened] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-15 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dylanzhou reopened SPARK-13183: --- 确实存在内存泄露问题,最后堆内存会耗尽,报错java.lang.OutOfMemoryError: Java heap

[jira] [Created] (SPARK-13336) Add non-numerical summaries to DataFrame.describe

2016-02-15 Thread Ian Hellstrom (JIRA)
Ian Hellstrom created SPARK-13336: - Summary: Add non-numerical summaries to DataFrame.describe Key: SPARK-13336 URL: https://issues.apache.org/jira/browse/SPARK-13336 Project: Spark Issue

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148202#comment-15148202 ] Xiao Li commented on SPARK-1: - Interesting. This query has specified the seed. Thus, it should return

[jira] [Comment Edited] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-15 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148200#comment-15148200 ] Maciej Bryński edited comment on SPARK-13283 at 2/16/16 7:30 AM: - Yep.

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-15 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148200#comment-15148200 ] Maciej Bryński commented on SPARK-13283: Yep. For MySQL this could look like this: {code}

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148196#comment-15148196 ] Adrian Wang commented on SPARK-13283: - So the problem here is that "from" is a reserved word in

[jira] [Updated] (SPARK-13335) Optimize Data Frames collect_list and collect_set with declarative aggregates

2016-02-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-13335: --- Priority: Minor (was: Major) > Optimize Data Frames collect_list and collect_set with declarative

[jira] [Commented] (SPARK-13335) Optimize Data Frames collect_list and collect_set with declarative aggregates

2016-02-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148199#comment-15148199 ] Matt Cheah commented on SPARK-13335: I have a prototypical patch for this and can submit a PR

[jira] [Updated] (SPARK-13335) Optimize Data Frames collect_list and collect_set with declarative aggregates

2016-02-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-13335: --- Summary: Optimize Data Frames collect_list and collect_set with declarative aggregates (was:

[jira] [Updated] (SPARK-13335) Optimize collect_list and collect_set with declarative aggregates

2016-02-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-13335: --- Component/s: SQL > Optimize collect_list and collect_set with declarative aggregates >

[jira] [Created] (SPARK-13335) Optimize collect_list and collect_set with declarative aggregates

2016-02-15 Thread Matt Cheah (JIRA)
Matt Cheah created SPARK-13335: -- Summary: Optimize collect_list and collect_set with declarative aggregates Key: SPARK-13335 URL: https://issues.apache.org/jira/browse/SPARK-13335 Project: Spark

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-15 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148189#comment-15148189 ] Maciej Bryński commented on SPARK-13283: No it's not fixed. Problem is in:

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148187#comment-15148187 ] Xiao Li commented on SPARK-1: - The current solution also has performance penalty. That has been

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148186#comment-15148186 ] Xiao Li commented on SPARK-1: - You will get the right result if you cache the first DF {code} //

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148182#comment-15148182 ] Xiao Li commented on SPARK-1: - This is a known issue. The same issue exists in CTE with

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148171#comment-15148171 ] Adrian Wang commented on SPARK-13283: - See comments from SPARK-13297, this have been fixed in master

[jira] [Assigned] (SPARK-13334) ML KMeansModel/BisectingKMeansModel should be set parent

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13334: Assignee: Apache Spark > ML KMeansModel/BisectingKMeansModel should be set parent >

[jira] [Commented] (SPARK-13334) ML KMeansModel/BisectingKMeansModel should be set parent

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148168#comment-15148168 ] Apache Spark commented on SPARK-13334: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13334) ML KMeansModel/BisectingKMeansModel should be set parent

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13334: Assignee: (was: Apache Spark) > ML KMeansModel/BisectingKMeansModel should be set

[jira] [Updated] (SPARK-13334) ML KMeansModel/BisectingKMeansModel/QuantileDiscretizerModel should be set parent

2016-02-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-13334: Summary: ML KMeansModel/BisectingKMeansModel/QuantileDiscretizerModel should be set parent (was:

[jira] [Updated] (SPARK-13334) ML KMeansModel/BisectingKMeansModel should be set parent

2016-02-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-13334: Summary: ML KMeansModel/BisectingKMeansModel should be set parent (was: ML

[jira] [Updated] (SPARK-13334) ML KMeansModel / BisectingKMeansModel should be set parent

2016-02-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-13334: Description: ML KMeansModel / BisectingKMeansModel / QuantileDiscretizer should be set parent. I

[jira] [Updated] (SPARK-13334) ML KMeansModel / BisectingKMeansModel should be set parent

2016-02-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-13334: Description: ML KMeansModel / BisectingKMeansModel / QuantileDiscretizer should be set parent. I

[jira] [Updated] (SPARK-13334) ML KMeansModel / BisectingKMeansModel should be set parent

2016-02-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-13334: Description: ML KMeansModel / BisectingKMeansModel / QuantileDiscretizer should be set parent. I

[jira] [Updated] (SPARK-13334) ML KMeansModel / BisectingKMeansModel should be set parent

2016-02-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-13334: Description: ML KMeansModel / BisectingKMeansModel / QuantileDiscretizer should be set parent. I

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148163#comment-15148163 ] Xiao Li commented on SPARK-1: - Glad to work on this issue. Let me try it. Will keep you posted.

[jira] [Created] (SPARK-13334) ML KMeansModel / BisectingKMeansModel should be set parent

2016-02-15 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13334: --- Summary: ML KMeansModel / BisectingKMeansModel should be set parent Key: SPARK-13334 URL: https://issues.apache.org/jira/browse/SPARK-13334 Project: Spark

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148135#comment-15148135 ] Joseph K. Bradley commented on SPARK-1: --- I haven't tested with 1.5 yet, but I assume it

[jira] [Updated] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-1: -- Affects Version/s: 1.6.1 1.4.2 > DataFrame filter + randn +

[jira] [Created] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-15 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-1: - Summary: DataFrame filter + randn + unionAll has bad interaction Key: SPARK-1 URL: https://issues.apache.org/jira/browse/SPARK-1 Project: Spark

[jira] [Commented] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

2016-02-15 Thread Qian Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148099#comment-15148099 ] Qian Huang commented on SPARK-4036: --- Hi, I have created a spark package,

[jira] [Assigned] (SPARK-13332) Decimal datatype support for SQL pow

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13332: Assignee: Apache Spark > Decimal datatype support for SQL pow >

[jira] [Assigned] (SPARK-13332) Decimal datatype support for SQL pow

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13332: Assignee: (was: Apache Spark) > Decimal datatype support for SQL pow >

[jira] [Commented] (SPARK-13332) Decimal datatype support for SQL pow

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148074#comment-15148074 ] Apache Spark commented on SPARK-13332: -- User 'yucai' has created a pull request for this issue:

[jira] [Created] (SPARK-13332) Decimal datatype support for SQL pow

2016-02-15 Thread yucai (JIRA)
yucai created SPARK-13332: - Summary: Decimal datatype support for SQL pow Key: SPARK-13332 URL: https://issues.apache.org/jira/browse/SPARK-13332 Project: Spark Issue Type: Bug Components:

[jira] [Resolved] (SPARK-13018) Replace example code in mllib-pmml-model-export.md using include_example

2016-02-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13018. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11126

[jira] [Resolved] (SPARK-13097) Extend Binarizer to allow Double AND Vector inputs

2016-02-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13097. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10976

[jira] [Commented] (SPARK-2183) Avoid loading/shuffling data twice in self-join query

2016-02-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148065#comment-15148065 ] Xiao Li commented on SPARK-2183: This problem still exists, right? I guess it might hurt the performance

[jira] [Created] (SPARK-13331) Spark network encryption optimization

2016-02-15 Thread Dong Chen (JIRA)
Dong Chen created SPARK-13331: - Summary: Spark network encryption optimization Key: SPARK-13331 URL: https://issues.apache.org/jira/browse/SPARK-13331 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-02-15 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148023#comment-15148023 ] SaintBacchus commented on SPARK-12316: -- [~tgraves] The application would not hit the same condition

[jira] [Assigned] (SPARK-13330) PYTHONHASHSEED is not propgated to executor

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13330: Assignee: (was: Apache Spark) > PYTHONHASHSEED is not propgated to executor >

[jira] [Commented] (SPARK-13330) PYTHONHASHSEED is not propgated to executor

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148014#comment-15148014 ] Apache Spark commented on SPARK-13330: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13330) PYTHONHASHSEED is not propgated to executor

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13330: Assignee: Apache Spark > PYTHONHASHSEED is not propgated to executor >

[jira] [Created] (SPARK-13330) PYTHONHASHSEED is not propgated to executor

2016-02-15 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-13330: -- Summary: PYTHONHASHSEED is not propgated to executor Key: SPARK-13330 URL: https://issues.apache.org/jira/browse/SPARK-13330 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11381) Replace example code in mllib-linear-methods.md using include_example

2016-02-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147993#comment-15147993 ] Xusen Yin commented on SPARK-11381: --- [~somi...@us.ibm.com] Are you still interested in working on it?

[jira] [Updated] (SPARK-11337) Make example code in user guide testable

2016-02-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11337: -- Description: The example code in the user guide is embedded in the markdown and hence it is not easy

[jira] [Updated] (SPARK-11337) Make example code in user guide testable

2016-02-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11337: -- Description: The example code in the user guide is embedded in the markdown and hence it is not easy

[jira] [Updated] (SPARK-11337) Make example code in user guide testable

2016-02-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11337: -- Description: The example code in the user guide is embedded in the markdown and hence it is not easy

[jira] [Updated] (SPARK-11337) Make example code in user guide testable

2016-02-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11337: -- Description: The example code in the user guide is embedded in the markdown and hence it is not easy

[jira] [Updated] (SPARK-11337) Make example code in user guide testable

2016-02-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11337: -- Description: The example code in the user guide is embedded in the markdown and hence it is not easy

[jira] [Assigned] (SPARK-13329) Considering output for statistics of logicol plan

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13329: Assignee: Davies Liu (was: Apache Spark) > Considering output for statistics of logicol

[jira] [Assigned] (SPARK-13329) Considering output for statistics of logicol plan

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13329: Assignee: Apache Spark (was: Davies Liu) > Considering output for statistics of logicol

[jira] [Commented] (SPARK-13329) Considering output for statistics of logicol plan

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147974#comment-15147974 ] Apache Spark commented on SPARK-13329: -- User 'davies' has created a pull request for this issue:

[jira] [Created] (SPARK-13329) Considering output for statistics of logicol plan

2016-02-15 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13329: -- Summary: Considering output for statistics of logicol plan Key: SPARK-13329 URL: https://issues.apache.org/jira/browse/SPARK-13329 Project: Spark Issue Type:

[jira] [Commented] (SPARK-13038) PySpark ml.pipeline support export/import

2016-02-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147941#comment-15147941 ] Xusen Yin commented on SPARK-13038: --- I start working on it. > PySpark ml.pipeline support

[jira] [Commented] (SPARK-13323) Type cast support in type inference during merging types.

2016-02-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147929#comment-15147929 ] Hyukjin Kwon commented on SPARK-13323: -- {code} sqlCtx.createDataFrame([["a"], [1]]).show() {code}

[jira] [Updated] (SPARK-13328) Possible Poor read performance for broadcast variables with dynamic resource allocation

2016-02-15 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated SPARK-13328: Summary: Possible Poor read performance for broadcast variables with dynamic resource

[jira] [Updated] (SPARK-13328) Possible poor read performance for broadcast variables with dynamic resource allocation

2016-02-15 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated SPARK-13328: Summary: Possible poor read performance for broadcast variables with dynamic resource

[jira] [Comment Edited] (SPARK-13328) Poor read performance for broadcast variables with dynamic resource allocation

2016-02-15 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147896#comment-15147896 ] Nezih Yigitbasi edited comment on SPARK-13328 at 2/15/16 11:30 PM: ---

[jira] [Commented] (SPARK-13328) Poor read performance for broadcast variables with dynamic resource allocation

2016-02-15 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147896#comment-15147896 ] Nezih Yigitbasi commented on SPARK-13328: - Although this long time can be reduced by decreasing

[jira] [Commented] (SPARK-12675) Executor dies because of ClassCastException and causes timeout

2016-02-15 Thread William Dixon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147895#comment-15147895 ] William Dixon commented on SPARK-12675: --- I see this issue in Spark 1.5.2 running in local,

[jira] [Created] (SPARK-13328) Poor read performance for broadcast variables with dynamic resource allocation

2016-02-15 Thread Nezih Yigitbasi (JIRA)
Nezih Yigitbasi created SPARK-13328: --- Summary: Poor read performance for broadcast variables with dynamic resource allocation Key: SPARK-13328 URL: https://issues.apache.org/jira/browse/SPARK-13328

[jira] [Updated] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-12583: - Target Version/s: 1.6.1 > spark shuffle fails with mesos after 2mins >

[jira] [Commented] (SPARK-13323) Type cast support in type inference during merging types.

2016-02-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147862#comment-15147862 ] Hyukjin Kwon commented on SPARK-13323: -- Let me add some codes here to reproduce in an hour. > Type

[jira] [Comment Edited] (SPARK-13323) Type cast support in type inference during merging types.

2016-02-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147851#comment-15147851 ] Hyukjin Kwon edited comment on SPARK-13323 at 2/15/16 10:43 PM: [~davies]

[jira] [Commented] (SPARK-13323) Type cast support in type inference during merging types.

2016-02-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147851#comment-15147851 ] Hyukjin Kwon commented on SPARK-13323: -- [~davies] Yes it's complicated but dealimg with numeric

[jira] [Commented] (SPARK-13327) colnames()<- allows invalid column names

2016-02-15 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147843#comment-15147843 ] Oscar D. Lara Yejas commented on SPARK-13327: - I'm working on this one > colnames()<- allows

[jira] [Created] (SPARK-13327) colnames()<- allows invalid column names

2016-02-15 Thread Oscar D. Lara Yejas (JIRA)
Oscar D. Lara Yejas created SPARK-13327: --- Summary: colnames()<- allows invalid column names Key: SPARK-13327 URL: https://issues.apache.org/jira/browse/SPARK-13327 Project: Spark Issue

[jira] [Created] (SPARK-13326) Dataset in spark 2.0.0-SNAPSHOT missing columns

2016-02-15 Thread koert kuipers (JIRA)
koert kuipers created SPARK-13326: - Summary: Dataset in spark 2.0.0-SNAPSHOT missing columns Key: SPARK-13326 URL: https://issues.apache.org/jira/browse/SPARK-13326 Project: Spark Issue

[jira] [Commented] (SPARK-12969) Exception while casting a spark supported date formatted "string" to "date" data type.

2016-02-15 Thread Ankit Jindal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147829#comment-15147829 ] Ankit Jindal commented on SPARK-12969: -- Hi Jais, i am running java program directly, and following

[jira] [Assigned] (SPARK-13325) Create a high-quality 64-bit hashcode expression

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13325: Assignee: Apache Spark > Create a high-quality 64-bit hashcode expression >

[jira] [Assigned] (SPARK-13325) Create a high-quality 64-bit hashcode expression

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13325: Assignee: (was: Apache Spark) > Create a high-quality 64-bit hashcode expression >

[jira] [Commented] (SPARK-13325) Create a high-quality 64-bit hashcode expression

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147824#comment-15147824 ] Apache Spark commented on SPARK-13325: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Created] (SPARK-13325) Create a high-quality 64-bit hashcode expression

2016-02-15 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-13325: - Summary: Create a high-quality 64-bit hashcode expression Key: SPARK-13325 URL: https://issues.apache.org/jira/browse/SPARK-13325 Project: Spark

[jira] [Commented] (SPARK-12969) Exception while casting a spark supported date formatted "string" to "date" data type.

2016-02-15 Thread Jais Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147710#comment-15147710 ] Jais Sebastian commented on SPARK-12969: Hi Ankit, Don't use spark submit. Try the following 1.

[jira] [Commented] (SPARK-12969) Exception while casting a spark supported date formatted "string" to "date" data type.

2016-02-15 Thread Ankit Jindal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147694#comment-15147694 ] Ankit Jindal commented on SPARK-12969: -- Hi Jais, Yes, i have tested your program in client mode

[jira] [Commented] (SPARK-12969) Exception while casting a spark supported date formatted "string" to "date" data type.

2016-02-15 Thread Jais Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147684#comment-15147684 ] Jais Sebastian commented on SPARK-12969: Hi Ankit, Have you tested the program in client mode ?

[jira] [Commented] (SPARK-12759) Spark should fail fast if --executor-memory is too small for spark to start

2016-02-15 Thread Daniel Jalova (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147682#comment-15147682 ] Daniel Jalova commented on SPARK-12759: --- I will work on this, thanks. > Spark should fail fast if

[jira] [Commented] (SPARK-13320) Confusing error message for Dataset API when using sum("*")

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147663#comment-15147663 ] Apache Spark commented on SPARK-13320: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13320) Confusing error message for Dataset API when using sum("*")

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13320: Assignee: Apache Spark > Confusing error message for Dataset API when using sum("*") >

[jira] [Assigned] (SPARK-13320) Confusing error message for Dataset API when using sum("*")

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13320: Assignee: (was: Apache Spark) > Confusing error message for Dataset API when using

[jira] [Commented] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-15 Thread Bertrand Bossy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147642#comment-15147642 ] Bertrand Bossy commented on SPARK-12583: [~marmbrus] If this could make it into 1.6.1, that would

[jira] [Assigned] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12583: Assignee: (was: Apache Spark) > spark shuffle fails with mesos after 2mins >

[jira] [Assigned] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12583: Assignee: Apache Spark > spark shuffle fails with mesos after 2mins >

[jira] [Commented] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147614#comment-15147614 ] Apache Spark commented on SPARK-12583: -- User 'bbossy' has created a pull request for this issue:

[jira] [Commented] (SPARK-13323) Type cast support in type inference during merging types.

2016-02-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147591#comment-15147591 ] Davies Liu commented on SPARK-13323: HiveTypeCoercion is pretty complicated, we may don't want to

[jira] [Comment Edited] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip

2016-02-15 Thread Paulo Villegas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147434#comment-15147434 ] Paulo Villegas edited comment on SPARK-4563 at 2/15/16 2:59 PM: Hi. I

[jira] [Commented] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip

2016-02-15 Thread Paulo Villegas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147434#comment-15147434 ] Paulo Villegas commented on SPARK-4563: --- Hi. I would have a use case for this functionality: when

[jira] [Commented] (SPARK-13297) [SQL] Backticks cannot be escaped in column names

2016-02-15 Thread Grzegorz Chilkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147371#comment-15147371 ] Grzegorz Chilkiewicz commented on SPARK-13297: -- I've verified it on:

  1   2   >