[jira] [Assigned] (SPARK-17072) generate table level stats:stats generation/storing/loading

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17072: Assignee: (was: Apache Spark) > generate table level stats:stats generation/storing/lo

[jira] [Commented] (SPARK-17072) generate table level stats:stats generation/storing/loading

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427692#comment-15427692 ] Apache Spark commented on SPARK-17072: -- User 'wzhfy' has created a pull request for

[jira] [Assigned] (SPARK-17072) generate table level stats:stats generation/storing/loading

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17072: Assignee: Apache Spark > generate table level stats:stats generation/storing/loading > ---

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427687#comment-15427687 ] Felix Cheung commented on SPARK-16581: -- I think JVM<->R is closely related to RBacke

[jira] [Comment Edited] (SPARK-15816) SQL server based on Postgres protocol

2016-08-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427671#comment-15427671 ] Takeshi Yamamuro edited comment on SPARK-15816 at 8/19/16 6:07 AM:

[jira] [Commented] (SPARK-15816) SQL server based on Postgres protocol

2016-08-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427671#comment-15427671 ] Takeshi Yamamuro commented on SPARK-15816: -- [~sarutak] I just posted the design

[jira] [Updated] (SPARK-15816) SQL server based on Postgres protocol

2016-08-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-15816: - Attachment: New_SQL_Server_for_Spark.pdf > SQL server based on Postgres protocol > --

[jira] [Commented] (SPARK-17140) Add initial model to MultinomialLogisticRegression

2016-08-18 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427664#comment-15427664 ] Seth Hendrickson commented on SPARK-17140: -- I can take this one. > Add initial

[jira] [Commented] (SPARK-16822) Support latex in scaladoc with MathJax

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427661#comment-15427661 ] Apache Spark commented on SPARK-16822: -- User 'jagadeesanas2' has created a pull requ

[jira] [Created] (SPARK-17151) Decide how to handle inferring number of classes in Multinomial logistic regression

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17151: Summary: Decide how to handle inferring number of classes in Multinomial logistic regression Key: SPARK-17151 URL: https://issues.apache.org/jira/browse/SPARK-17151

[jira] [Updated] (SPARK-16216) CSV data source does not write date and timestamp correctly

2016-08-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16216: Target Version/s: 2.0.1, 2.1.0 Priority: Blocker (was: Major) > CSV data source does n

[jira] [Commented] (SPARK-16533) Spark application not handling preemption messages

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427625#comment-15427625 ] Apache Spark commented on SPARK-16533: -- User 'angolon' has created a pull request fo

[jira] [Assigned] (SPARK-16533) Spark application not handling preemption messages

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16533: Assignee: Apache Spark > Spark application not handling preemption messages >

[jira] [Assigned] (SPARK-16533) Spark application not handling preemption messages

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16533: Assignee: (was: Apache Spark) > Spark application not handling preemption messages > -

[jira] [Assigned] (SPARK-17150) Support SQL generation for inline tables

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17150: Assignee: (was: Apache Spark) > Support SQL generation for inline tables > ---

[jira] [Assigned] (SPARK-17150) Support SQL generation for inline tables

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17150: Assignee: Apache Spark > Support SQL generation for inline tables > --

[jira] [Commented] (SPARK-17150) Support SQL generation for inline tables

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427589#comment-15427589 ] Apache Spark commented on SPARK-17150: -- User 'petermaxlee' has created a pull reques

[jira] [Created] (SPARK-17150) Support SQL generation for inline tables

2016-08-18 Thread Peter Lee (JIRA)
Peter Lee created SPARK-17150: - Summary: Support SQL generation for inline tables Key: SPARK-17150 URL: https://issues.apache.org/jira/browse/SPARK-17150 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-17145) Object with many fields causes Seq Serialization Bug

2016-08-18 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427565#comment-15427565 ] Liwei Lin commented on SPARK-17145: --- hi [~abdulla16] can you try https://github.com/apa

[jira] [Commented] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2016-08-18 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427563#comment-15427563 ] Yanbo Liang commented on SPARK-17137: - I think we should provide transparent interfac

[jira] [Commented] (SPARK-17149) array.sql for testing array related functions

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427557#comment-15427557 ] Apache Spark commented on SPARK-17149: -- User 'petermaxlee' has created a pull reques

[jira] [Assigned] (SPARK-17149) array.sql for testing array related functions

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17149: Assignee: (was: Apache Spark) > array.sql for testing array related functions > --

[jira] [Assigned] (SPARK-17149) array.sql for testing array related functions

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17149: Assignee: Apache Spark > array.sql for testing array related functions > -

[jira] [Commented] (SPARK-16914) NodeManager crash when spark are registering executor infomartion into leveldb

2016-08-18 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427556#comment-15427556 ] cen yuhai commented on SPARK-16914: --- [~jerryshao] hi, saisai, I think SPARK-14963 is us

[jira] [Created] (SPARK-17149) array.sql for testing array related functions

2016-08-18 Thread Peter Lee (JIRA)
Peter Lee created SPARK-17149: - Summary: array.sql for testing array related functions Key: SPARK-17149 URL: https://issues.apache.org/jira/browse/SPARK-17149 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-17148) NodeManager exit because of exception “Executor is not registered”

2016-08-18 Thread cen yuhai (JIRA)
cen yuhai created SPARK-17148: - Summary: NodeManager exit because of exception “Executor is not registered” Key: SPARK-17148 URL: https://issues.apache.org/jira/browse/SPARK-17148 Project: Spark

[jira] [Commented] (SPARK-17136) Design optimizer interface for ML algorithms

2016-08-18 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427539#comment-15427539 ] Yanbo Liang commented on SPARK-17136: - I would like to know that users' own optimizer

[jira] [Comment Edited] (SPARK-17139) Add model summary for MultinomialLogisticRegression

2016-08-18 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427519#comment-15427519 ] Weichen Xu edited comment on SPARK-17139 at 8/19/16 3:05 AM: -

[jira] [Comment Edited] (SPARK-17138) Python API for multinomial logistic regression

2016-08-18 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427518#comment-15427518 ] Weichen Xu edited comment on SPARK-17138 at 8/19/16 3:06 AM: -

[jira] [Comment Edited] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

2016-08-18 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427529#comment-15427529 ] Yanbo Liang edited comment on SPARK-17134 at 8/19/16 3:04 AM: -

[jira] [Commented] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

2016-08-18 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427529#comment-15427529 ] Yanbo Liang commented on SPARK-17134: - This is interesting. We also trying to use BLA

[jira] [Commented] (SPARK-17139) Add model summary for MultinomialLogisticRegression

2016-08-18 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427519#comment-15427519 ] Weichen Xu commented on SPARK-17139: I will work on it and create PR soon, thanks. >

[jira] [Commented] (SPARK-17138) Python API for multinomial logistic regression

2016-08-18 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427518#comment-15427518 ] Weichen Xu commented on SPARK-17138: I will work on it and create PR soon, thanks. >

[jira] [Updated] (SPARK-16947) Support type coercion and foldable expression for inline tables

2016-08-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16947: Fix Version/s: 2.0.1 > Support type coercion and foldable expression for inline tables > --

[jira] [Commented] (SPARK-17069) Expose spark.range() as table-valued function in SQL

2016-08-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427469#comment-15427469 ] Reynold Xin commented on SPARK-17069: - I've also backported this into branch-2.0 sinc

[jira] [Updated] (SPARK-17069) Expose spark.range() as table-valued function in SQL

2016-08-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17069: Fix Version/s: 2.0.1 > Expose spark.range() as table-valued function in SQL > -

[jira] [Created] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-08-18 Thread Robert Conrad (JIRA)
Robert Conrad created SPARK-17147: - Summary: Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets Key: SPARK-17147 URL: https://issues.apache.org/jira/browse/SPARK-17147 Project: S

[jira] [Resolved] (SPARK-16947) Support type coercion and foldable expression for inline tables

2016-08-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16947. - Resolution: Fixed > Support type coercion and foldable expression for inline tables > ---

[jira] [Updated] (SPARK-16947) Support type coercion and foldable expression for inline tables

2016-08-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16947: Fix Version/s: 2.1.0 > Support type coercion and foldable expression for inline tables > --

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427429#comment-15427429 ] Sital Kedia commented on SPARK-16922: - Kryo > Query with Broadcast Hash join fails

[jira] [Commented] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-18 Thread Qian Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427425#comment-15427425 ] Qian Huang commented on SPARK-17090: Gotcha. I will do the api first. > Make tree ag

[jira] [Commented] (SPARK-17141) MinMaxScaler behaves weird when min and max have the same value and some values are NaN

2016-08-18 Thread Alberto Bonsanto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427409#comment-15427409 ] Alberto Bonsanto commented on SPARK-17141: -- Crude data. | id|chicken|jam|roast

[jira] [Created] (SPARK-17146) Add RandomizedSearch to the CrossValidator API

2016-08-18 Thread Manoj Kumar (JIRA)
Manoj Kumar created SPARK-17146: --- Summary: Add RandomizedSearch to the CrossValidator API Key: SPARK-17146 URL: https://issues.apache.org/jira/browse/SPARK-17146 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-17143) pyspark unable to create UDF: java.lang.RuntimeException: org.apache.hadoop.fs.FileAlreadyExistsException: Parent path is not a directory: /tmp tmp

2016-08-18 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427394#comment-15427394 ] Andrew Davidson commented on SPARK-17143: - See email from user's group. I was abl

[jira] [Created] (SPARK-17145) Object with many fields causes Seq Serialization Bug

2016-08-18 Thread Abdulla Al-Qawasmeh (JIRA)
Abdulla Al-Qawasmeh created SPARK-17145: --- Summary: Object with many fields causes Seq Serialization Bug Key: SPARK-17145 URL: https://issues.apache.org/jira/browse/SPARK-17145 Project: Spark

[jira] [Assigned] (SPARK-17144) Removal of useless CreateHiveTableAsSelectLogicalPlan

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17144: Assignee: (was: Apache Spark) > Removal of useless CreateHiveTableAsSelectLogicalPlan

[jira] [Commented] (SPARK-17144) Removal of useless CreateHiveTableAsSelectLogicalPlan

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427383#comment-15427383 ] Apache Spark commented on SPARK-17144: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-17144) Removal of useless CreateHiveTableAsSelectLogicalPlan

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17144: Assignee: Apache Spark > Removal of useless CreateHiveTableAsSelectLogicalPlan > -

[jira] [Created] (SPARK-17144) Removal of useless CreateHiveTableAsSelectLogicalPlan

2016-08-18 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17144: --- Summary: Removal of useless CreateHiveTableAsSelectLogicalPlan Key: SPARK-17144 URL: https://issues.apache.org/jira/browse/SPARK-17144 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-17081) Empty strings not preserved which causes SQLException: mismatching column value count

2016-08-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427380#comment-15427380 ] Xiao Li commented on SPARK-17081: - Can you try to reproduce it in Spark 2.0? Thanks! > E

[jira] [Commented] (SPARK-17143) pyspark unable to create UDF: java.lang.RuntimeException: org.apache.hadoop.fs.FileAlreadyExistsException: Parent path is not a directory: /tmp tmp

2016-08-18 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427278#comment-15427278 ] Andrew Davidson commented on SPARK-17143: - given the exception metioned an issue

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427264#comment-15427264 ] Davies Liu commented on SPARK-16922: Which serializer are you using? java serializer

[jira] [Updated] (SPARK-17143) pyspark unable to create UDF: java.lang.RuntimeException: org.apache.hadoop.fs.FileAlreadyExistsException: Parent path is not a directory: /tmp tmp

2016-08-18 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson updated SPARK-17143: Attachment: udfBug.html This html version of the notebook shows the output when run in my d

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427259#comment-15427259 ] Sital Kedia commented on SPARK-16922: - >> Could you also try to disable the dense mod

[jira] [Updated] (SPARK-17143) pyspark unable to create UDF: java.lang.RuntimeException: org.apache.hadoop.fs.FileAlreadyExistsException: Parent path is not a directory: /tmp tmp

2016-08-18 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson updated SPARK-17143: Attachment: udfBug.ipynb The attached notebook demonstrated the reported bug. Note it inclu

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427250#comment-15427250 ] Sital Kedia commented on SPARK-16922: - The failure is deterministic, we are reproduci

[jira] [Created] (SPARK-17143) pyspark unable to create UDF: java.lang.RuntimeException: org.apache.hadoop.fs.FileAlreadyExistsException: Parent path is not a directory: /tmp tmp

2016-08-18 Thread Andrew Davidson (JIRA)
Andrew Davidson created SPARK-17143: --- Summary: pyspark unable to create UDF: java.lang.RuntimeException: org.apache.hadoop.fs.FileAlreadyExistsException: Parent path is not a directory: /tmp tmp Key: SPARK-17143

[jira] [Comment Edited] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427241#comment-15427241 ] Davies Liu edited comment on SPARK-16922 at 8/18/16 9:58 PM: -

[jira] [Commented] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-08-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427241#comment-15427241 ] Davies Liu commented on SPARK-16922: Is this failure determistic or not? Happened on

[jira] [Commented] (SPARK-17142) Complex query triggers binding error in HashAggregateExec

2016-08-18 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427236#comment-15427236 ] Josh Rosen commented on SPARK-17142: Interestingly, this query executes fine if the r

[jira] [Created] (SPARK-17142) Complex query triggers binding error in HashAggregateExec

2016-08-18 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17142: -- Summary: Complex query triggers binding error in HashAggregateExec Key: SPARK-17142 URL: https://issues.apache.org/jira/browse/SPARK-17142 Project: Spark Issue T

[jira] [Updated] (SPARK-17142) Complex query triggers binding error in HashAggregateExec

2016-08-18 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17142: --- Description: The following example runs successfully on Spark 2.0.0 but fails in the current master

[jira] [Commented] (SPARK-17133) Improvements to linear methods in Spark

2016-08-18 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427144#comment-15427144 ] Xin Ren commented on SPARK-17133: - hi [~sethah] I'd like to help on this, please count me

[jira] [Commented] (SPARK-16508) Fix documentation warnings found by R CMD check

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427134#comment-15427134 ] Apache Spark commented on SPARK-16508: -- User 'junyangq' has created a pull request f

[jira] [Commented] (SPARK-16904) Removal of Hive Built-in Hash Functions and TestHiveFunctionRegistry

2016-08-18 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427069#comment-15427069 ] Tejas Patil commented on SPARK-16904: - Is Spark's hashing function semantically equiv

[jira] [Updated] (SPARK-16077) Python UDF may fail because of six

2016-08-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16077: - Fix Version/s: 1.6.3 > Python UDF may fail because of six > -- > >

[jira] [Commented] (SPARK-17141) MinMaxScaler behaves weird when min and max have the same value and some values are NaN

2016-08-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426988#comment-15426988 ] Sean Owen commented on SPARK-17141: --- Summarize the reproduction here? best to put it al

[jira] [Created] (SPARK-17141) MinMaxScaler behaves weird when min and max have the same value and some values are NaN

2016-08-18 Thread Alberto Bonsanto (JIRA)
Alberto Bonsanto created SPARK-17141: Summary: MinMaxScaler behaves weird when min and max have the same value and some values are NaN Key: SPARK-17141 URL: https://issues.apache.org/jira/browse/SPARK-17141

[jira] [Commented] (SPARK-17132) binaryFiles method can't handle paths with embedded commas

2016-08-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426962#comment-15426962 ] Sean Owen commented on SPARK-17132: --- Yeah, that would be a solution. It actually affect

[jira] [Commented] (SPARK-17132) binaryFiles method can't handle paths with embedded commas

2016-08-18 Thread Maximilian Najork (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426965#comment-15426965 ] Maximilian Najork commented on SPARK-17132: --- I tried escaping the commas prior

[jira] [Created] (SPARK-17140) Add initial model to MultinomialLogisticRegression

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17140: Summary: Add initial model to MultinomialLogisticRegression Key: SPARK-17140 URL: https://issues.apache.org/jira/browse/SPARK-17140 Project: Spark Is

[jira] [Created] (SPARK-17138) Python API for multinomial logistic regression

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17138: Summary: Python API for multinomial logistic regression Key: SPARK-17138 URL: https://issues.apache.org/jira/browse/SPARK-17138 Project: Spark Issue

[jira] [Created] (SPARK-17139) Add model summary for MultinomialLogisticRegression

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17139: Summary: Add model summary for MultinomialLogisticRegression Key: SPARK-17139 URL: https://issues.apache.org/jira/browse/SPARK-17139 Project: Spark I

[jira] [Created] (SPARK-17137) Add compressed support for multinomial logistic regression coefficients

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17137: Summary: Add compressed support for multinomial logistic regression coefficients Key: SPARK-17137 URL: https://issues.apache.org/jira/browse/SPARK-17137 Proje

[jira] [Created] (SPARK-17136) Design optimizer interface for ML algorithms

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17136: Summary: Design optimizer interface for ML algorithms Key: SPARK-17136 URL: https://issues.apache.org/jira/browse/SPARK-17136 Project: Spark Issue Ty

[jira] [Updated] (SPARK-17133) Improvements to linear methods in Spark

2016-08-18 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hendrickson updated SPARK-17133: - Description: This JIRA is for tracking several improvements that we should make to Linear

[jira] [Created] (SPARK-17135) Consolidate code in linear/logistic regression where possible

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17135: Summary: Consolidate code in linear/logistic regression where possible Key: SPARK-17135 URL: https://issues.apache.org/jira/browse/SPARK-17135 Project: Spark

[jira] [Updated] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-18 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hendrickson updated SPARK-17090: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-17133 > Make tree aggregat

[jira] [Created] (SPARK-17134) Use level 2 BLAS operations in LogisticAggregator

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17134: Summary: Use level 2 BLAS operations in LogisticAggregator Key: SPARK-17134 URL: https://issues.apache.org/jira/browse/SPARK-17134 Project: Spark Iss

[jira] [Created] (SPARK-17133) Improvements to linear methods in Spark

2016-08-18 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17133: Summary: Improvements to linear methods in Spark Key: SPARK-17133 URL: https://issues.apache.org/jira/browse/SPARK-17133 Project: Spark Issue Type: U

[jira] [Created] (SPARK-17132) binaryFiles method can't handle paths with embedded commas

2016-08-18 Thread Maximilian Najork (JIRA)
Maximilian Najork created SPARK-17132: - Summary: binaryFiles method can't handle paths with embedded commas Key: SPARK-17132 URL: https://issues.apache.org/jira/browse/SPARK-17132 Project: Spark

[jira] [Updated] (SPARK-16981) For CSV files nullValue is not respected for Date/Time data type

2016-08-18 Thread Lev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lev updated SPARK-16981: Priority: Critical (was: Major) > For CSV files nullValue is not respected for Date/Time data type > -

[jira] [Commented] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-18 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426876#comment-15426876 ] DB Tsai commented on SPARK-17090: - Since having a formula of determining the aggregation

[jira] [Commented] (SPARK-15694) Implement ScriptTransformation in sql/core

2016-08-18 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426858#comment-15426858 ] Tejas Patil commented on SPARK-15694: - PR for part #1 : https://github.com/apache/spa

[jira] [Commented] (SPARK-17130) SparseVectors.apply and SparseVectors.toArray have different returns when creating with a illegal indices

2016-08-18 Thread Jon Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426821#comment-15426821 ] Jon Zhong commented on SPARK-17130: --- Thanks for posting the code. The problem is solved

[jira] [Assigned] (SPARK-15694) Implement ScriptTransformation in sql/core

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15694: Assignee: (was: Apache Spark) > Implement ScriptTransformation in sql/core > -

[jira] [Commented] (SPARK-15694) Implement ScriptTransformation in sql/core

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426815#comment-15426815 ] Apache Spark commented on SPARK-15694: -- User 'tejasapatil' has created a pull reques

[jira] [Assigned] (SPARK-15694) Implement ScriptTransformation in sql/core

2016-08-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15694: Assignee: Apache Spark > Implement ScriptTransformation in sql/core >

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426807#comment-15426807 ] Shivaram Venkataraman commented on SPARK-16581: --- I am not sure the issues a

[jira] [Issue Comment Deleted] (SPARK-16581) Making JVM backend calling functions public

2016-08-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-16581: -- Comment: was deleted (was: I am not sure the issues are very related though 1.

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-08-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426806#comment-15426806 ] Shivaram Venkataraman commented on SPARK-16581: --- I am not sure the issues a

[jira] [Commented] (SPARK-17131) Code generation fails when running SQL expressions against a wide dataset (thousands of columns)

2016-08-18 Thread Iaroslav Zeigerman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426801#comment-15426801 ] Iaroslav Zeigerman commented on SPARK-17131: Having a different exception whe

[jira] [Commented] (SPARK-6832) Handle partial reads in SparkR JVM to worker communication

2016-08-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426798#comment-15426798 ] Shivaram Venkataraman commented on SPARK-6832: -- I think we can add a new meth

[jira] [Resolved] (SPARK-17130) SparseVectors.apply and SparseVectors.toArray have different returns when creating with a illegal indices

2016-08-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17130. --- Resolution: Duplicate Oh yeah but along the way the validation is also all moved into the constructo

[jira] [Commented] (SPARK-17130) SparseVectors.apply and SparseVectors.toArray have different returns when creating with a illegal indices

2016-08-18 Thread Jon Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426780#comment-15426780 ] Jon Zhong commented on SPARK-17130: --- Yep, I wrote a comment there but I deleted since I

[jira] [Created] (SPARK-17131) Code generation fails when running SQL expressions against a wide dataset (thousands of columns)

2016-08-18 Thread Iaroslav Zeigerman (JIRA)
Iaroslav Zeigerman created SPARK-17131: -- Summary: Code generation fails when running SQL expressions against a wide dataset (thousands of columns) Key: SPARK-17131 URL: https://issues.apache.org/jira/browse/S

[jira] [Commented] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-18 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426562#comment-15426562 ] Seth Hendrickson commented on SPARK-17090: -- I'm not working on it. Please feel f

[jira] [Commented] (SPARK-17086) QuantileDiscretizer throws InvalidArgumentException (parameter splits given invalid value) on valid data

2016-08-18 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426549#comment-15426549 ] Barry Becker commented on SPARK-17086: -- I think I agree with the discussion. Here is

[jira] [Commented] (SPARK-17130) SparseVectors.apply and SparseVectors.toArray have different returns when creating with a illegal indices

2016-08-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426490#comment-15426490 ] Sean Owen commented on SPARK-17130: --- Yeah, didn't you just comment on https://github.co

[jira] [Created] (SPARK-17130) SparseVectors.apply and SparseVectors.toArray have different returns when creating with a illegal indices

2016-08-18 Thread Jon Zhong (JIRA)
Jon Zhong created SPARK-17130: - Summary: SparseVectors.apply and SparseVectors.toArray have different returns when creating with a illegal indices Key: SPARK-17130 URL: https://issues.apache.org/jira/browse/SPARK-1713

  1   2   >