[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573981#comment-14573981 ] Konstantin Shaposhnikov commented on SPARK-8118: Name of the logger has be

[jira] [Updated] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shaposhnikov updated SPARK-8122: --- Description: _enableLogForwarding()_ doesn't hold to the created loggers that

[jira] [Updated] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shaposhnikov updated SPARK-8122: --- Summary: ParquetRelation.enableLogForwarding() may fail to configure loggers

[jira] [Updated] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shaposhnikov updated SPARK-8122: --- Description: Another problem with _enableLogForwarding()_ is that it doesn't

[jira] [Updated] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shaposhnikov updated SPARK-8122: --- Priority: Minor (was: Major) > ParquetRelation.enableLogForwarding() may fail

[jira] [Commented] (SPARK-8122) A few problems in ParquetRelation.enableLogForwarding()

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573978#comment-14573978 ] Konstantin Shaposhnikov commented on SPARK-8122: SPARK-8118 is for the fir

[jira] [Commented] (SPARK-8122) A few problems in ParquetRelation.enableLogForwarding()

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573973#comment-14573973 ] Konstantin Shaposhnikov commented on SPARK-8122: I believe that currently

[jira] [Updated] (SPARK-8122) A few problems in ParquetRelation.enableLogForwarding()

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8122: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5463 > A few problems in ParquetRelation.enable

[jira] [Commented] (SPARK-8122) A few problems in ParquetRelation.enableLogForwarding()

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573966#comment-14573966 ] Reynold Xin commented on SPARK-8122: Thanks for filing. What's the relationship betwee

[jira] [Created] (SPARK-8122) A few problems in ParquetRelation.enableLogForwarding()

2015-06-04 Thread Konstantin Shaposhnikov (JIRA)
Konstantin Shaposhnikov created SPARK-8122: -- Summary: A few problems in ParquetRelation.enableLogForwarding() Key: SPARK-8122 URL: https://issues.apache.org/jira/browse/SPARK-8122 Project: Spa

[jira] [Created] (SPARK-8121) "spark.sql.parquet.output.committer.class" is overriden by "spark.sql.sources.outputCommitterClass"

2015-06-04 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-8121: - Summary: "spark.sql.parquet.output.committer.class" is overriden by "spark.sql.sources.outputCommitterClass" Key: SPARK-8121 URL: https://issues.apache.org/jira/browse/SPARK-8121

[jira] [Commented] (SPARK-8071) Run PySpark dataframe.rollup/cube test failed

2015-06-04 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573952#comment-14573952 ] Cheng Hao commented on SPARK-8071: -- I couldn't reproduce that with scala API, and also it

[jira] [Assigned] (SPARK-8119) Spark will set total executor when some executors fail.

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8119: --- Assignee: (was: Apache Spark) > Spark will set total executor when some executors fail. >

[jira] [Assigned] (SPARK-8119) Spark will set total executor when some executors fail.

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8119: --- Assignee: Apache Spark > Spark will set total executor when some executors fail. > --

[jira] [Commented] (SPARK-8119) Spark will set total executor when some executors fail.

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573946#comment-14573946 ] Apache Spark commented on SPARK-8119: - User 'SaintBacchus' has created a pull request

[jira] [Commented] (SPARK-7819) Isolated Hive Client Loader appears to cause Native Library libMapRClient.4.0.2-mapr.so already loaded in another classloader error

2015-06-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573944#comment-14573944 ] Yin Huai commented on SPARK-7819: - [~nemccarthy] Thank you for the update! Glad to hear th

[jira] [Commented] (SPARK-8071) Run PySpark dataframe.rollup/cube test failed

2015-06-04 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573935#comment-14573935 ] Cheng Hao commented on SPARK-8071: -- Can you try `df.cube('name', 'age').count().show()`?

[jira] [Commented] (SPARK-7819) Isolated Hive Client Loader appears to cause Native Library libMapRClient.4.0.2-mapr.so already loaded in another classloader error

2015-06-04 Thread Nathan McCarthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573929#comment-14573929 ] Nathan McCarthy commented on SPARK-7819: @Yin - looks like might build was just a

[jira] [Reopened] (SPARK-8096) how to convert dataframe field to LabelPoint

2015-06-04 Thread bofei.xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bofei.xiao reopened SPARK-8096: --- I'm sorry i haven't express my question clearly! > how to convert dataframe field to LabelPoint > ---

[jira] [Created] (SPARK-8120) Typos in warning message in sql/types.py

2015-06-04 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-8120: Summary: Typos in warning message in sql/types.py Key: SPARK-8120 URL: https://issues.apache.org/jira/browse/SPARK-8120 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-8096) how to convert dataframe field to label and features

2015-06-04 Thread bofei.xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bofei.xiao updated SPARK-8096: -- Description: how to convert the dataframe to RDD[LabelPoint] dataframe with fields target,age,sex,height

[jira] [Updated] (SPARK-8096) how to convert dataframe field to LabelPoint

2015-06-04 Thread bofei.xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bofei.xiao updated SPARK-8096: -- Summary: how to convert dataframe field to LabelPoint (was: how to convert dataframe field to label and

[jira] [Updated] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8118: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5463 > Turn off noisy log output produced by Pa

[jira] [Updated] (SPARK-8119) Spark will set total executor when some executors fail.

2015-06-04 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8119: Description: DynamicAllocation will set the total executor to a little number when it wants to kill

[jira] [Updated] (SPARK-8116) sc.range() doesn't match python range()

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8116: --- Target Version/s: 1.4.1 (was: 1.4.0, 1.4.1) > sc.range() doesn't match python range() > -

[jira] [Created] (SPARK-8119) Spark will set total executor when some executors fail.

2015-06-04 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8119: --- Summary: Spark will set total executor when some executors fail. Key: SPARK-8119 URL: https://issues.apache.org/jira/browse/SPARK-8119 Project: Spark Issue Typ

[jira] [Commented] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573908#comment-14573908 ] Reynold Xin commented on SPARK-8056: I'm not actively working on this. Feel free to ta

[jira] [Created] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2015-06-04 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-8118: - Summary: Turn off noisy log output produced by Parquet 1.7.0 Key: SPARK-8118 URL: https://issues.apache.org/jira/browse/SPARK-8118 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-7119) ScriptTransform doesn't consider the output data type

2015-06-04 Thread zhichao-li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573889#comment-14573889 ] zhichao-li commented on SPARK-7119: --- This workaround query can be executed correctly and

[jira] [Commented] (SPARK-7119) ScriptTransform doesn't consider the output data type

2015-06-04 Thread zhichao-li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573888#comment-14573888 ] zhichao-li commented on SPARK-7119: --- This workaround query can be executed correctly and

[jira] [Issue Comment Deleted] (SPARK-7119) ScriptTransform doesn't consider the output data type

2015-06-04 Thread zhichao-li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhichao-li updated SPARK-7119: -- Comment: was deleted (was: This workaround query can be executed correctly and there's a simple fix for

[jira] [Updated] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8056: --- Assignee: (was: Reynold Xin) > Design an easier way to construct schema for both Scala and Python

[jira] [Comment Edited] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

2015-06-04 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573874#comment-14573874 ] Ilya Ganelin edited comment on SPARK-8056 at 6/5/15 12:35 AM: --

[jira] [Commented] (SPARK-8056) Design an easier way to construct schema for both Scala and Python

2015-06-04 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573874#comment-14573874 ] Ilya Ganelin commented on SPARK-8056: - [~rxin] Are you actively working on this? I thi

[jira] [Updated] (SPARK-8106) Set derby.system.durability=test in order to speed up Hive compatibility tests

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8106: --- Component/s: Tests Build > Set derby.system.durability=test in order to speed up Hive

[jira] [Resolved] (SPARK-8106) Set derby.system.durability=test in order to speed up Hive compatibility tests

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8106. Resolution: Fixed Fix Version/s: 1.5.0 > Set derby.system.durability=test in order to speed u

[jira] [Updated] (SPARK-7417) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dependencies

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7417: - Assignee: Burak Yavuz > Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dependencies >

[jira] [Closed] (SPARK-7417) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dependencies

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-7417. Resolution: Fixed > Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dependencies > --

[jira] [Updated] (SPARK-7417) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dependencies

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7417: - Target Version/s: 1.3.2, 1.4.0 (was: 1.4.0) > Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect depe

[jira] [Updated] (SPARK-7418) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite search for artifacts

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7418: - Target Version/s: 1.3.2, 1.4.0 (was: 1.4.0) > Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite search for a

[jira] [Commented] (SPARK-7417) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dependencies

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573842#comment-14573842 ] Andrew Or commented on SPARK-7417: -- This should be resolved by: branch-1.4+: 8014e1f6bb87

[jira] [Updated] (SPARK-7418) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite search for artifacts

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7418: - Fix Version/s: 1.4.0 1.3.2 > Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite search for

[jira] [Updated] (SPARK-7417) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dependencies

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-7417: - Fix Version/s: 1.4.0 1.3.2 > Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite neglect dep

[jira] [Closed] (SPARK-7418) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite search for artifacts

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-7418. Resolution: Fixed Assignee: Burak Yavuz > Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite search for ar

[jira] [Commented] (SPARK-7418) Flaky test: o.a.s.deploy.SparkSubmitUtilsSuite search for artifacts

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573840#comment-14573840 ] Andrew Or commented on SPARK-7418: -- This should be resolved by: branch-1.4+: 8014e1f6bb8

[jira] [Assigned] (SPARK-8114) Remove wildcard import on TestSQLContext._

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8114: --- Assignee: Apache Spark > Remove wildcard import on TestSQLContext._ > ---

[jira] [Commented] (SPARK-8114) Remove wildcard import on TestSQLContext._

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573829#comment-14573829 ] Apache Spark commented on SPARK-8114: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-8114) Remove wildcard import on TestSQLContext._

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8114: --- Assignee: (was: Apache Spark) > Remove wildcard import on TestSQLContext._ >

[jira] [Commented] (SPARK-8071) Run PySpark dataframe.rollup/cube test failed

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573828#comment-14573828 ] Reynold Xin commented on SPARK-8071: [~chenghao] can you take a look at this? Cube is

[jira] [Assigned] (SPARK-8113) SQL module test cleanup

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8113: --- Assignee: Reynold Xin (was: Apache Spark) > SQL module test cleanup > --

[jira] [Commented] (SPARK-8113) SQL module test cleanup

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573822#comment-14573822 ] Apache Spark commented on SPARK-8113: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-8113) SQL module test cleanup

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8113: --- Assignee: Apache Spark (was: Reynold Xin) > SQL module test cleanup > --

[jira] [Commented] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-06-04 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573820#comment-14573820 ] DB Tsai commented on SPARK-7008: Do you see better convergence rate when LBFGS is used? >

[jira] [Updated] (SPARK-8071) Run PySpark dataframe.rollup/cube test failed

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8071: --- Description: I run test for Spark, and failed on PySpark, details are: {code} File "/xxx/Spark/python/

[jira] [Assigned] (SPARK-8117) Push codegen into Expression

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8117: --- Assignee: Davies Liu (was: Apache Spark) > Push codegen into Expression > --

[jira] [Assigned] (SPARK-8117) Push codegen into Expression

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8117: --- Assignee: Apache Spark (was: Davies Liu) > Push codegen into Expression > --

[jira] [Commented] (SPARK-8117) Push codegen into Expression

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573807#comment-14573807 ] Apache Spark commented on SPARK-8117: - User 'davies' has created a pull request for th

[jira] [Assigned] (SPARK-8112) Received block event count through the StreamingListener can be negative

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8112: --- Assignee: Apache Spark (was: Shixiong Zhu) > Received block event count through the Streamin

[jira] [Assigned] (SPARK-8112) Received block event count through the StreamingListener can be negative

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8112: --- Assignee: Shixiong Zhu (was: Apache Spark) > Received block event count through the Streamin

[jira] [Commented] (SPARK-8112) Received block event count through the StreamingListener can be negative

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573796#comment-14573796 ] Apache Spark commented on SPARK-8112: - User 'zsxwing' has created a pull request for t

[jira] [Resolved] (SPARK-8098) Show correct length of bytes on log page

2015-06-04 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-8098. -- Resolution: Fixed Fix Version/s: 1.5.0 1.4.1 1.3.2

[jira] [Updated] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8095: - Target Version/s: 1.3.2, 1.4.1, 1.5.0 > Spark package dependencies not resolved when package is in local-i

[jira] [Updated] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8095: - Assignee: Burak Yavuz > Spark package dependencies not resolved when package is in local-ivy-cache > -

[jira] [Created] (SPARK-8117) Push codegen into Expression

2015-06-04 Thread Davies Liu (JIRA)
Davies Liu created SPARK-8117: - Summary: Push codegen into Expression Key: SPARK-8117 URL: https://issues.apache.org/jira/browse/SPARK-8117 Project: Spark Issue Type: Bug Components: SQ

[jira] [Assigned] (SPARK-6419) GenerateOrdering does not support BinaryType and complex types.

2015-06-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-6419: - Assignee: Davies Liu > GenerateOrdering does not support BinaryType and complex types. >

[jira] [Assigned] (SPARK-7184) Investigate turning codegen on by default

2015-06-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-7184: - Assignee: Davies Liu > Investigate turning codegen on by default > --

[jira] [Assigned] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8095: --- Assignee: Apache Spark > Spark package dependencies not resolved when package is in local-ivy

[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573784#comment-14573784 ] Apache Spark commented on SPARK-8095: - User 'andrewor14' has created a pull request fo

[jira] [Assigned] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8095: --- Assignee: (was: Apache Spark) > Spark package dependencies not resolved when package is i

[jira] [Commented] (SPARK-7306) SPARK-7224 broke build with jdk6

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573781#comment-14573781 ] Apache Spark commented on SPARK-7306: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-7205) Support local ivy cache in --packages

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573779#comment-14573779 ] Apache Spark commented on SPARK-7205: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-7224) Mock repositories for testing with --packages

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573780#comment-14573780 ] Apache Spark commented on SPARK-7224: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-8071) Run PySpark dataframe.rollup/cube test failed

2015-06-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573771#comment-14573771 ] Davies Liu commented on SPARK-8071: --- It failed in Scala side, cc [~rxin] > Run PySpark

[jira] [Commented] (SPARK-8116) sc.range() doesn't match python range()

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573752#comment-14573752 ] Apache Spark commented on SPARK-8116: - User 'belisarius222' has created a pull request

[jira] [Assigned] (SPARK-8116) sc.range() doesn't match python range()

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8116: --- Assignee: (was: Apache Spark) > sc.range() doesn't match python range() > ---

[jira] [Assigned] (SPARK-8116) sc.range() doesn't match python range()

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8116: --- Assignee: Apache Spark > sc.range() doesn't match python range() > --

[jira] [Commented] (SPARK-7536) Audit MLlib Python API for 1.4

2015-06-04 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573736#comment-14573736 ] Joseph K. Bradley commented on SPARK-7536: -- OK, I'll see about checking through t

[jira] [Created] (SPARK-8116) sc.range() doesn't match python range()

2015-06-04 Thread Ted Blackman (JIRA)
Ted Blackman created SPARK-8116: --- Summary: sc.range() doesn't match python range() Key: SPARK-8116 URL: https://issues.apache.org/jira/browse/SPARK-8116 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-7991) Python DataFrame: support passing a list into describe

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7991: --- Assignee: Apache Spark > Python DataFrame: support passing a list into describe > ---

[jira] [Assigned] (SPARK-7991) Python DataFrame: support passing a list into describe

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7991: --- Assignee: (was: Apache Spark) > Python DataFrame: support passing a list into describe >

[jira] [Commented] (SPARK-7991) Python DataFrame: support passing a list into describe

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573678#comment-14573678 ] Apache Spark commented on SPARK-7991: - User 'ameyc' has created a pull request for thi

[jira] [Updated] (SPARK-8109) TestSQLContext's static initialization is run during MiMa tests, causing SparkContexts to be created

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-8109: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-8113 > TestSQLContext's static initiali

[jira] [Commented] (SPARK-8068) Add confusionMatrix method at class MulticlassMetrics in pyspark/mllib

2015-06-04 Thread Ai He (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573676#comment-14573676 ] Ai He commented on SPARK-8068: -- Hi, Joseph, am I supposed to solve this issue or just let the

[jira] [Created] (SPARK-8115) Remove TestData

2015-06-04 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8115: -- Summary: Remove TestData Key: SPARK-8115 URL: https://issues.apache.org/jira/browse/SPARK-8115 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Created] (SPARK-8114) Remove wildcard import on TestSQLContext._

2015-06-04 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8114: -- Summary: Remove wildcard import on TestSQLContext._ Key: SPARK-8114 URL: https://issues.apache.org/jira/browse/SPARK-8114 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-8113) SQL module test cleanup

2015-06-04 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8113: -- Summary: SQL module test cleanup Key: SPARK-8113 URL: https://issues.apache.org/jira/browse/SPARK-8113 Project: Spark Issue Type: Umbrella Components:

[jira] [Assigned] (SPARK-7546) Example code for ML Pipelines feature transformations

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7546: --- Assignee: Ram Sriharsha (was: Apache Spark) > Example code for ML Pipelines feature transfor

[jira] [Assigned] (SPARK-7546) Example code for ML Pipelines feature transformations

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7546: --- Assignee: Apache Spark (was: Ram Sriharsha) > Example code for ML Pipelines feature transfor

[jira] [Commented] (SPARK-7546) Example code for ML Pipelines feature transformations

2015-06-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573625#comment-14573625 ] Apache Spark commented on SPARK-7546: - User 'harsha2010' has created a pull request fo

[jira] [Commented] (SPARK-8111) SparkR shell should display Spark logo and version banner on startup

2015-06-04 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573574#comment-14573574 ] Shivaram Venkataraman commented on SPARK-8111: -- The code will need to go in

[jira] [Updated] (SPARK-8111) SparkR shell should display Spark logo and version banner on startup

2015-06-04 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-8111: - Labels: Starter (was: ) > SparkR shell should display Spark logo and version bann

[jira] [Commented] (SPARK-8112) Received block event count through the StreamingListener can be negative

2015-06-04 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573573#comment-14573573 ] Tathagata Das commented on SPARK-8112: -- Take a look at SPARK-8080 > Received block e

[jira] [Updated] (SPARK-8112) Received block event count through the StreamingListener can be negative

2015-06-04 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-8112: - Priority: Minor (was: Major) > Received block event count through the StreamingListener can be ne

[jira] [Created] (SPARK-8112) Received block event count through the StreamingListener can be negative

2015-06-04 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-8112: Summary: Received block event count through the StreamingListener can be negative Key: SPARK-8112 URL: https://issues.apache.org/jira/browse/SPARK-8112 Project: Spark

[jira] [Commented] (SPARK-8080) Custom Receiver.store with Iterator type do not give correct count at Spark UI

2015-06-04 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573566#comment-14573566 ] Tathagata Das commented on SPARK-8080: -- [~zsxwing] Take a look at the screenshot atta

[jira] [Updated] (SPARK-5479) PySpark on yarn mode need to support non-local python files

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5479: - Component/s: YARN > PySpark on yarn mode need to support non-local python files >

[jira] [Updated] (SPARK-5479) PySpark on yarn mode need to support non-local python files

2015-06-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5479: - Affects Version/s: 1.4.0 > PySpark on yarn mode need to support non-local python files > -

[jira] [Created] (SPARK-8111) SparkR shell should display Spark logo and version banner on startup

2015-06-04 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-8111: Summary: SparkR shell should display Spark logo and version banner on startup Key: SPARK-8111 URL: https://issues.apache.org/jira/browse/SPARK-8111 Project: Spark

[jira] [Updated] (SPARK-8110) DAG visualizations sometimes look weird in Python

2015-06-04 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-8110: - Attachment: Screen Shot 2015-06-04 at 1.51.32 PM.png Screen Shot 2015-06-04 at 1.51

[jira] [Assigned] (SPARK-7440) Remove physical Distinct operator in favor of Aggregate

2015-06-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-7440: -- Assignee: Reynold Xin > Remove physical Distinct operator in favor of Aggregate > -

  1   2   3   >