[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-06-30 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609629#comment-14609629 ] Cheng Hao commented on SPARK-8653: -- What do you think [~rxin]? > Add constraint for Chil

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-06-30 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609627#comment-14609627 ] Cheng Hao commented on SPARK-8653: -- Yes, I agree that we cannot make a clear cut, as peop

[jira] [Commented] (SPARK-8621) crosstab exception when one of the value is empty

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609621#comment-14609621 ] Apache Spark commented on SPARK-8621: - User 'cloud-fan' has created a pull request for

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609611#comment-14609611 ] Reynold Xin commented on SPARK-8653: I'm not opposed to the idea, but we have already

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-06-30 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609609#comment-14609609 ] Cheng Hao commented on SPARK-8653: -- For most of the Mathematical expressions, we can get

[jira] [Created] (SPARK-8751) Check missing and add user guide for MLlib Python API

2015-06-30 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-8751: -- Summary: Check missing and add user guide for MLlib Python API Key: SPARK-8751 URL: https://issues.apache.org/jira/browse/SPARK-8751 Project: Spark Issue Type: I

[jira] [Resolved] (SPARK-8748) Move castability test out from Cast case class into Cast object

2015-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8748. Resolution: Fixed Fix Version/s: 1.5.0 > Move castability test out from Cast case class into

[jira] [Commented] (SPARK-8750) Remove the closure in functions.callUdf

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609600#comment-14609600 ] Apache Spark commented on SPARK-8750: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-8750) Remove the closure in functions.callUdf

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8750: --- Assignee: Reynold Xin (was: Apache Spark) > Remove the closure in functions.callUdf > --

[jira] [Assigned] (SPARK-8750) Remove the closure in functions.callUdf

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8750: --- Assignee: Apache Spark (was: Reynold Xin) > Remove the closure in functions.callUdf > --

[jira] [Created] (SPARK-8750) Remove the closure in functions.callUdf

2015-06-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8750: -- Summary: Remove the closure in functions.callUdf Key: SPARK-8750 URL: https://issues.apache.org/jira/browse/SPARK-8750 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-6101) Create a SparkSQL DataSource API implementation for DynamoDB

2015-06-30 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609598#comment-14609598 ] Murtaza Kanchwala commented on SPARK-6101: -- https://github.com/cfregly/spark-dyna

[jira] [Assigned] (SPARK-8749) Remove HiveTypeCoercion trait

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8749: --- Assignee: Apache Spark (was: Reynold Xin) > Remove HiveTypeCoercion trait >

[jira] [Commented] (SPARK-8749) Remove HiveTypeCoercion trait

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609594#comment-14609594 ] Apache Spark commented on SPARK-8749: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-8749) Remove HiveTypeCoercion trait

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8749: --- Assignee: Reynold Xin (was: Apache Spark) > Remove HiveTypeCoercion trait >

[jira] [Created] (SPARK-8749) Remove HiveTypeCoercion trait

2015-06-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8749: -- Summary: Remove HiveTypeCoercion trait Key: SPARK-8749 URL: https://issues.apache.org/jira/browse/SPARK-8749 Project: Spark Issue Type: Improvement Com

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609583#comment-14609583 ] Reynold Xin commented on SPARK-8653: Implicit type casts should be up to the query eng

[jira] [Assigned] (SPARK-8653) Add constraint for Children expression for data type

2015-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-8653: -- Assignee: Reynold Xin > Add constraint for Children expression for data type >

[jira] [Assigned] (SPARK-8647) Potential issues with the constant hashCode

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8647: --- Assignee: Apache Spark > Potential issues with the constant hashCode > -

[jira] [Assigned] (SPARK-8647) Potential issues with the constant hashCode

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8647: --- Assignee: (was: Apache Spark) > Potential issues with the constant hashCode > --

[jira] [Commented] (SPARK-8653) Add constraint for Children expression for data type

2015-06-30 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609579#comment-14609579 ] Davies Liu commented on SPARK-8653: --- [~rxin] With the new `ExpectsInputTypes`, we still

[jira] [Commented] (SPARK-8647) Potential issues with the constant hashCode

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609580#comment-14609580 ] Apache Spark commented on SPARK-8647: - User 'aloknsingh' has created a pull request fo

[jira] [Assigned] (SPARK-8748) Move castability test out from Cast case class into Cast object

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8748: --- Assignee: Reynold Xin (was: Apache Spark) > Move castability test out from Cast case class i

[jira] [Assigned] (SPARK-8748) Move castability test out from Cast case class into Cast object

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8748: --- Assignee: Apache Spark (was: Reynold Xin) > Move castability test out from Cast case class i

[jira] [Commented] (SPARK-8748) Move castability test out from Cast case class into Cast object

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609576#comment-14609576 ] Apache Spark commented on SPARK-8748: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-8748) Move castability test out from Cast case class into Cast object

2015-06-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8748: -- Summary: Move castability test out from Cast case class into Cast object Key: SPARK-8748 URL: https://issues.apache.org/jira/browse/SPARK-8748 Project: Spark Is

[jira] [Comment Edited] (SPARK-6101) Create a SparkSQL DataSource API implementation for DynamoDB

2015-06-30 Thread venu k tangirala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609569#comment-14609569 ] venu k tangirala edited comment on SPARK-6101 at 7/1/15 5:21 AM: ---

[jira] [Commented] (SPARK-6101) Create a SparkSQL DataSource API implementation for DynamoDB

2015-06-30 Thread venu k tangirala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609569#comment-14609569 ] venu k tangirala commented on SPARK-6101: - Hi Chris, does this include writing bac

[jira] [Commented] (SPARK-8745) Remove GenerateMutableProjection

2015-06-30 Thread Akhil Thatipamula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609566#comment-14609566 ] Akhil Thatipamula commented on SPARK-8745: -- [~rxin] I will work on this. > Remov

[jira] [Assigned] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8746: --- Assignee: Apache Spark > Need to update download link for Hive 0.13.1 jars (HiveComparisonTes

[jira] [Assigned] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8746: --- Assignee: (was: Apache Spark) > Need to update download link for Hive 0.13.1 jars (HiveCo

[jira] [Commented] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609563#comment-14609563 ] Apache Spark commented on SPARK-8746: - User 'ckadner' has created a pull request for t

[jira] [Assigned] (SPARK-8747) fix EqualNullSafe for binary type

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8747: --- Assignee: (was: Apache Spark) > fix EqualNullSafe for binary type > -

[jira] [Assigned] (SPARK-8747) fix EqualNullSafe for binary type

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8747: --- Assignee: Apache Spark > fix EqualNullSafe for binary type >

[jira] [Commented] (SPARK-8747) fix EqualNullSafe for binary type

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609562#comment-14609562 ] Apache Spark commented on SPARK-8747: - User 'cloud-fan' has created a pull request for

[jira] [Created] (SPARK-8747) fix EqualNullSafe for binary type

2015-06-30 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-8747: -- Summary: fix EqualNullSafe for binary type Key: SPARK-8747 URL: https://issues.apache.org/jira/browse/SPARK-8747 Project: Spark Issue Type: Bug Compone

[jira] [Commented] (SPARK-8742) Improve SparkR error messages for DataFrame API

2015-06-30 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609558#comment-14609558 ] Shivaram Venkataraman commented on SPARK-8742: -- Thanks [~falaki] for creating

[jira] [Commented] (SPARK-8535) PySpark : Can't create DataFrame from Pandas dataframe with no explicit column name

2015-06-30 Thread Yuri Saito (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609552#comment-14609552 ] Yuri Saito commented on SPARK-8535: --- Could you change assignee from no-assignee to me?

[jira] [Created] (SPARK-8746) Need to update download link for Hive 0.13.1 jars (HiveComparisonTest)

2015-06-30 Thread Christian Kadner (JIRA)
Christian Kadner created SPARK-8746: --- Summary: Need to update download link for Hive 0.13.1 jars (HiveComparisonTest) Key: SPARK-8746 URL: https://issues.apache.org/jira/browse/SPARK-8746 Project: S

[jira] [Resolved] (SPARK-8699) Select command not working for SparkR built on Spark Version: 1.4.0 and R 3.2.0

2015-06-30 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-8699. -- Resolution: Not A Problem > Select command not working for SparkR built on Spark

[jira] [Comment Edited] (SPARK-8699) Select command not working for SparkR built on Spark Version: 1.4.0 and R 3.2.0

2015-06-30 Thread Kamlesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609545#comment-14609545 ] Kamlesh Kumar edited comment on SPARK-8699 at 7/1/15 4:39 AM: --

[jira] [Updated] (SPARK-8699) Select command not working for SparkR built on Spark Version: 1.4.0 and R 3.2.0

2015-06-30 Thread Kamlesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamlesh Kumar updated SPARK-8699: - Priority: Trivial (was: Critical) > Select command not working for SparkR built on Spark Version:

[jira] [Commented] (SPARK-8699) Select command not working for SparkR built on Spark Version: 1.4.0 and R 3.2.0

2015-06-30 Thread Kamlesh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609545#comment-14609545 ] Kamlesh Kumar commented on SPARK-8699: -- Thanks Shivaram, it works. > Select command

[jira] [Created] (SPARK-8745) Remove GenerateMutableProjection

2015-06-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8745: -- Summary: Remove GenerateMutableProjection Key: SPARK-8745 URL: https://issues.apache.org/jira/browse/SPARK-8745 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-6990) Add Java linting script

2015-06-30 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609531#comment-14609531 ] Yu Ishikawa edited comment on SPARK-6990 at 7/1/15 3:59 AM: I

[jira] [Commented] (SPARK-6990) Add Java linting script

2015-06-30 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609531#comment-14609531 ] Yu Ishikawa commented on SPARK-6990: I think it would be nice to execute `mvn checksty

[jira] [Updated] (SPARK-8366) When task fails and append a new one, the ExecutorAllocationManager can't sense the new tasks

2015-06-30 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] meiyoula updated SPARK-8366: Description: I use the *dynamic executor allocation* function. When an executor is killed, all running task

[jira] [Updated] (SPARK-8366) When task fails and append a new one, the ExecutorAllocationManager can't sense the new tasks

2015-06-30 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] meiyoula updated SPARK-8366: Description: I use the *dynamic executor allocation* function. When an executor is killed, all running task

[jira] [Updated] (SPARK-8366) When task fails and append a new one, the ExecutorAllocationManager can't sense the new tasks

2015-06-30 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] meiyoula updated SPARK-8366: Description: I use the *dynamic executor allocation* function. When an executor is killed, all running task

[jira] [Resolved] (SPARK-8535) PySpark : Can't create DataFrame from Pandas dataframe with no explicit column name

2015-06-30 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8535. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7124 [https://github.com/

[jira] [Commented] (SPARK-6990) Add Java linting script

2015-06-30 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609509#comment-14609509 ] Yu Ishikawa commented on SPARK-6990: Please assign this issue to me? > Add Java linti

[jira] [Commented] (SPARK-8529) Set metadata for MinMaxScaler

2015-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609497#comment-14609497 ] Joseph K. Bradley commented on SPARK-8529: -- Here's an example of setting the meta

[jira] [Commented] (SPARK-6602) Replace direct use of Akka with Spark RPC interface

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609481#comment-14609481 ] Apache Spark commented on SPARK-6602: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-8628) Race condition in AbstractSparkSQLParser.parse

2015-06-30 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609477#comment-14609477 ] Vinod KC commented on SPARK-8628: - Can you please assign this to me > Race condition in A

[jira] [Commented] (SPARK-3444) Provide a way to easily change the log level in the Spark shell while running

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609455#comment-14609455 ] Apache Spark commented on SPARK-3444: - User 'vanzin' has created a pull request for th

[jira] [Created] (SPARK-8744) StringIndexerModel should have public constructor

2015-06-30 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-8744: Summary: StringIndexerModel should have public constructor Key: SPARK-8744 URL: https://issues.apache.org/jira/browse/SPARK-8744 Project: Spark Issue

[jira] [Commented] (SPARK-8743) Deregister Codahale metrics for streaming when StreamingContext is closed

2015-06-30 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609437#comment-14609437 ] Neelesh Srinivas Salian commented on SPARK-8743: I would like to work on t

[jira] [Assigned] (SPARK-8313) Support Spark Packages containing R code with --packages

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8313: --- Assignee: (was: Apache Spark) > Support Spark Packages containing R code with --packages

[jira] [Commented] (SPARK-8313) Support Spark Packages containing R code with --packages

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609403#comment-14609403 ] Apache Spark commented on SPARK-8313: - User 'brkyvz' has created a pull request for th

[jira] [Assigned] (SPARK-8313) Support Spark Packages containing R code with --packages

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8313: --- Assignee: Apache Spark > Support Spark Packages containing R code with --packages > -

[jira] [Commented] (SPARK-8318) Spark Streaming Starter JIRAs

2015-06-30 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609399#comment-14609399 ] Tathagata Das commented on SPARK-8318: -- I think the starter label is not very easy to

[jira] [Comment Edited] (SPARK-8318) Spark Streaming Starter JIRAs

2015-06-30 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609399#comment-14609399 ] Tathagata Das edited comment on SPARK-8318 at 7/1/15 1:19 AM: --

[jira] [Created] (SPARK-8743) Deregister Codahale metrics for streaming when StreamingContext is closed

2015-06-30 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-8743: Summary: Deregister Codahale metrics for streaming when StreamingContext is closed Key: SPARK-8743 URL: https://issues.apache.org/jira/browse/SPARK-8743 Project: Spa

[jira] [Closed] (SPARK-6892) Recovery from checkpoint will also reuse the application id when write eventLog in yarn-cluster mode

2015-06-30 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-6892. Resolution: Not A Problem > Recovery from checkpoint will also reuse the application id when write

[jira] [Created] (SPARK-8742) Improve SparkR error messages for DataFrame API

2015-06-30 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-8742: - Summary: Improve SparkR error messages for DataFrame API Key: SPARK-8742 URL: https://issues.apache.org/jira/browse/SPARK-8742 Project: Spark Issue Type: B

[jira] [Resolved] (SPARK-8727) Add missing python api

2015-06-30 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8727. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7114 [https://github.com/

[jira] [Resolved] (SPARK-8741) Remove e and pi from DataFrame functions

2015-06-30 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8741. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7137 [https://github.com/

[jira] [Resolved] (SPARK-7739) Improve ChiSqSelector example code in the user guide

2015-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7739. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7029 [https

[jira] [Resolved] (SPARK-8738) Generate better error message in Python for AnalysisException

2015-06-30 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8738. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7135 [https://github.com/

[jira] [Commented] (SPARK-5016) GaussianMixtureEM should distribute matrix inverse for large numFeatures, k

2015-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609218#comment-14609218 ] Joseph K. Bradley commented on SPARK-5016: -- Perhaps you could take [~MechCoder]'s

[jira] [Updated] (SPARK-8125) Accelerate ParquetRelation2 metadata discovery

2015-06-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-8125: -- Shepherd: Cheng Lian > Accelerate ParquetRelation2 metadata discovery >

[jira] [Commented] (SPARK-5016) GaussianMixtureEM should distribute matrix inverse for large numFeatures, k

2015-06-30 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609194#comment-14609194 ] Feynman Liang commented on SPARK-5016: -- While this doesn't solve the high dimensional

[jira] [Commented] (SPARK-5480) GraphX pageRank: java.lang.ArrayIndexOutOfBoundsException:

2015-06-30 Thread Thomas Dudziak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609189#comment-14609189 ] Thomas Dudziak commented on SPARK-5480: --- We get the same exception when running a so

[jira] [Commented] (SPARK-8471) Implement Discrete Cosine Transform feature transformer

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609186#comment-14609186 ] Apache Spark commented on SPARK-8471: - User 'feynmanliang' has created a pull request

[jira] [Commented] (SPARK-8741) Remove e and pi from DataFrame functions

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609149#comment-14609149 ] Apache Spark commented on SPARK-8741: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-8741) Remove e and pi from DataFrame functions

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8741: --- Assignee: Reynold Xin (was: Apache Spark) > Remove e and pi from DataFrame functions > -

[jira] [Assigned] (SPARK-8741) Remove e and pi from DataFrame functions

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8741: --- Assignee: Apache Spark (was: Reynold Xin) > Remove e and pi from DataFrame functions > -

[jira] [Commented] (SPARK-8596) Install and configure RStudio server on Spark EC2

2015-06-30 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609148#comment-14609148 ] Shivaram Venkataraman commented on SPARK-8596: -- I think the assumption is tha

[jira] [Created] (SPARK-8741) Remove e and pi from DataFrame functions

2015-06-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-8741: -- Summary: Remove e and pi from DataFrame functions Key: SPARK-8741 URL: https://issues.apache.org/jira/browse/SPARK-8741 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

2015-06-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609144#comment-14609144 ] Josh Rosen commented on SPARK-4452: --- I've linked this to the Project Tungsten JIRA epic,

[jira] [Comment Edited] (SPARK-6785) DateUtils can not handle date before 1970/01/01 correctly

2015-06-30 Thread Christian Kadner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549366#comment-14549366 ] Christian Kadner edited comment on SPARK-6785 at 6/30/15 9:50 PM: --

[jira] [Updated] (SPARK-6360) For Spark 1.1 and 1.2, after any RDD transformations, calling saveAsParquetFile over a SchemaRDD with decimal or UDT column throws

2015-06-30 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6360: -- Description: Spark shell session for reproduction (use {{:paste}}): {noformat} import org.apache.spark.s

[jira] [Commented] (SPARK-8596) Install and configure RStudio server on Spark EC2

2015-06-30 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609127#comment-14609127 ] Vincent Warmerdam commented on SPARK-8596: -- Mhm. I seem to stumble on another iss

[jira] [Assigned] (SPARK-8740) Support GitHub OAuth tokens in dev/merge_spark_pr.py

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8740: --- Assignee: Josh Rosen (was: Apache Spark) > Support GitHub OAuth tokens in dev/merge_spark_pr

[jira] [Commented] (SPARK-8740) Support GitHub OAuth tokens in dev/merge_spark_pr.py

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609126#comment-14609126 ] Apache Spark commented on SPARK-8740: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-8740) Support GitHub OAuth tokens in dev/merge_spark_pr.py

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8740: --- Assignee: Apache Spark (was: Josh Rosen) > Support GitHub OAuth tokens in dev/merge_spark_pr

[jira] [Assigned] (SPARK-7902) SQL UDF doesn't support UDT in PySpark

2015-06-30 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-7902: - Assignee: Davies Liu > SQL UDF doesn't support UDT in PySpark > -

[jira] [Created] (SPARK-8740) Support GitHub OAuth tokens in dev/merge_spark_pr.py

2015-06-30 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-8740: - Summary: Support GitHub OAuth tokens in dev/merge_spark_pr.py Key: SPARK-8740 URL: https://issues.apache.org/jira/browse/SPARK-8740 Project: Spark Issue Type: Impr

[jira] [Assigned] (SPARK-8738) Generate better error message in Python for AnalysisException

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8738: --- Assignee: Davies Liu (was: Apache Spark) > Generate better error message in Python for Analy

[jira] [Updated] (SPARK-8389) Expose KafkaRDDs offsetRange in Python

2015-06-30 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-8389: - Issue Type: Sub-task (was: New Feature) Parent: SPARK-8337 > Expose KafkaRDDs offsetRange

[jira] [Commented] (SPARK-8738) Generate better error message in Python for AnalysisException

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609115#comment-14609115 ] Apache Spark commented on SPARK-8738: - User 'davies' has created a pull request for th

[jira] [Assigned] (SPARK-8738) Generate better error message in Python for AnalysisException

2015-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8738: --- Assignee: Apache Spark (was: Davies Liu) > Generate better error message in Python for Analy

[jira] [Issue Comment Deleted] (SPARK-8128) Schema Merging Broken: Dataframe Fails to Recognize Column in Schema

2015-06-30 Thread Brad Willard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Willard updated SPARK-8128: Comment: was deleted (was: This is only broken on queries. I can load the dataframe with the unquer

[jira] [Reopened] (SPARK-8389) Expose KafkaRDDs offsetRange in Python

2015-06-30 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reopened SPARK-8389: -- Reopening this issue because it represent one of the two parity problems that is addressed by SPARK

[jira] [Commented] (SPARK-8128) Schema Merging Broken: Dataframe Fails to Recognize Column in Schema

2015-06-30 Thread Brad Willard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609100#comment-14609100 ] Brad Willard commented on SPARK-8128: - This is only broken on queries. I can load the

[jira] [Commented] (SPARK-8337) KafkaUtils.createDirectStream for python is lacking API/feature parity with the Scala/Java version

2015-06-30 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609102#comment-14609102 ] Tathagata Das commented on SPARK-8337: -- I will let you guys figure out the implementa

[jira] [Commented] (SPARK-4727) Add "dimensional" RDDs (time series, spatial)

2015-06-30 Thread David Sabater (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609095#comment-14609095 ] David Sabater commented on SPARK-4727: -- [~sandyr] is actually working on something si

[jira] [Updated] (SPARK-8563) Bug that IndexedRowMatrix.computeSVD() yields the U with wrong numCols

2015-06-30 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8563: - Affects Version/s: 1.4.1 1.0.2 1.1.1

[jira] [Updated] (SPARK-8563) Bug that IndexedRowMatrix.computeSVD() yields the U with wrong numCols

2015-06-30 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8563: - Assignee: 19 Lee > Bug that IndexedRowMatrix.computeSVD() yields the U with wrong numCols > --

[jira] [Commented] (SPARK-7338) Survival Modelling - Cox proportional hazards

2015-06-30 Thread David Sabater (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609089#comment-14609089 ] David Sabater commented on SPARK-7338: -- Just wanted to mention Madlib implements it a

  1   2   3   >