[jira] [Updated] (SPARK-10502) tidy up the exception message text to be less verbose/"User friendly"

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10502: -- Issue Type: Improvement (was: Bug) > tidy up the exception message text to be less verbose/"User

[jira] [Updated] (SPARK-7825) Poor performance in Cross Product due to no combine operations for small files.

2015-09-09 Thread Tang Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tang Yan updated SPARK-7825: Affects Version/s: (was: 1.3.1) (was: 1.2.2) (was:

[jira] [Created] (SPARK-10511) Source releases should not include maven jars

2015-09-09 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-10511: --- Summary: Source releases should not include maven jars Key: SPARK-10511 URL: https://issues.apache.org/jira/browse/SPARK-10511 Project: Spark Issue

[jira] [Commented] (SPARK-10444) Remove duplication in Mesos schedulers

2015-09-09 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736434#comment-14736434 ] Iulian Dragos commented on SPARK-10444: --- Another example of duplicated logic:

[jira] [Commented] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736389#comment-14736389 ] Apache Spark commented on SPARK-10512: -- User 'yu-iskw' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10512: Assignee: (was: Apache Spark) > Fix @since when a function doesn't have doc >

[jira] [Assigned] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10512: Assignee: Apache Spark > Fix @since when a function doesn't have doc >

[jira] [Assigned] (SPARK-10274) Add @since annotation to pyspark.mllib.fpm

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10274: Assignee: Apache Spark > Add @since annotation to pyspark.mllib.fpm >

[jira] [Commented] (SPARK-10274) Add @since annotation to pyspark.mllib.fpm

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736286#comment-14736286 ] Apache Spark commented on SPARK-10274: -- User 'yu-iskw' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10274) Add @since annotation to pyspark.mllib.fpm

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10274: Assignee: (was: Apache Spark) > Add @since annotation to pyspark.mllib.fpm >

[jira] [Updated] (SPARK-10507) timestamp - timestamp

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10507: -- Priority: Minor (was: Major) > timestamp - timestamp > -- > >

[jira] [Commented] (SPARK-10507) timestamp - timestamp

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736385#comment-14736385 ] Sean Owen commented on SPARK-10507: --- (Can you improve the title and description please?) > timestamp -

[jira] [Updated] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-10512: Description: When I tried to add @since to a function which doesn't have doc, @since didn't go

[jira] [Resolved] (SPARK-10111) StringIndexerModel lacks of method "labels"

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10111. --- Resolution: Duplicate > StringIndexerModel lacks of method "labels" >

[jira] [Updated] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-10512: Description: When I tried to add @since to a function which doesn't have doc, @since didn't go

[jira] [Commented] (SPARK-10275) Add @since annotation to pyspark.mllib.random

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736297#comment-14736297 ] Apache Spark commented on SPARK-10275: -- User 'yu-iskw' has created a pull request for this issue:

[jira] [Commented] (SPARK-7425) spark.ml Predictor should support other numeric types for label

2015-09-09 Thread Glenn Weidner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736295#comment-14736295 ] Glenn Weidner commented on SPARK-7425: -- Unit tests for other numeric types have not been added. >

[jira] [Assigned] (SPARK-10275) Add @since annotation to pyspark.mllib.random

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10275: Assignee: (was: Apache Spark) > Add @since annotation to pyspark.mllib.random >

[jira] [Assigned] (SPARK-10275) Add @since annotation to pyspark.mllib.random

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10275: Assignee: Apache Spark > Add @since annotation to pyspark.mllib.random >

[jira] [Created] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Yu Ishikawa (JIRA)
Yu Ishikawa created SPARK-10512: --- Summary: Fix @since when a function doesn't have doc Key: SPARK-10512 URL: https://issues.apache.org/jira/browse/SPARK-10512 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10276) Add @since annotation to pyspark.mllib.recommendation

2015-09-09 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736346#comment-14736346 ] Yu Ishikawa commented on SPARK-10276: - [~mengxr] should we add `@since` = to the class methods with

[jira] [Commented] (SPARK-10309) Some tasks failed with Unable to acquire memory

2015-09-09 Thread Naden Franciscus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736304#comment-14736304 ] Naden Franciscus commented on SPARK-10309: -- Still working on the physical plan but we have been

[jira] [Comment Edited] (SPARK-10309) Some tasks failed with Unable to acquire memory

2015-09-09 Thread Naden Franciscus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736304#comment-14736304 ] Naden Franciscus edited comment on SPARK-10309 at 9/9/15 6:43 AM: -- Still

[jira] [Resolved] (SPARK-10227) sbt build on Scala 2.11 fails

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10227. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8433

[jira] [Updated] (SPARK-10227) sbt build on Scala 2.11 fails

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10227: -- Assignee: Luc Bourlier > sbt build on Scala 2.11 fails > - > >

[jira] [Updated] (SPARK-10327) Cache Table is not working while subquery has alias in its project list

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10327: -- Assignee: Cheng Hao > Cache Table is not working while subquery has alias in its project list >

[jira] [Updated] (SPARK-10441) Cannot write timestamp to JSON

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10441: -- Assignee: Yin Huai > Cannot write timestamp to JSON > -- > >

[jira] [Updated] (SPARK-4752) Classifier based on artificial neural network

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4752: - Assignee: Alexander Ulanov > Classifier based on artificial neural network >

[jira] [Updated] (SPARK-10507) reject temporal expressions such as timestamp - timestamp at parse time

2015-09-09 Thread N Campbell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] N Campbell updated SPARK-10507: --- Description: TIMESTAMP - TIMESTAMP in ISO-SQL should return an interval type which SPARK does not

[jira] [Updated] (SPARK-10316) respect non-deterministic expressions in PhysicalOperation

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10316: -- Assignee: Wenchen Fan > respect non-deterministic expressions in PhysicalOperation >

[jira] [Updated] (SPARK-10501) support UUID as an atomic type

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10501: -- Priority: Minor (was: Major) Component/s: SQL Issue Type: Improvement (was: Bug) >

[jira] [Created] (SPARK-10513) Springleaf Marketing Response

2015-09-09 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-10513: --- Summary: Springleaf Marketing Response Key: SPARK-10513 URL: https://issues.apache.org/jira/browse/SPARK-10513 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-10513) Springleaf Marketing Response

2015-09-09 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736648#comment-14736648 ] Yanbo Liang commented on SPARK-10513: - I will work on this dataset. > Springleaf Marketing Response

[jira] [Commented] (SPARK-9578) Stemmer feature transformer

2015-09-09 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736695#comment-14736695 ] yuhao yang commented on SPARK-9578: --- A better choice for LDA seems to be lemmatization. Yet that

[jira] [Commented] (SPARK-9564) Spark 1.5.0 Testing Plan

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736594#comment-14736594 ] Sean Owen commented on SPARK-9564: -- Now that 1.5.0 is released, can this be closed? Or else I'm unclear

[jira] [Created] (SPARK-10514) Minimum ratio of registered resources [ spark.scheduler.minRegisteredResourcesRatio] is not enabled for Mesos Coarse Grained mode

2015-09-09 Thread Akash Mishra (JIRA)
Akash Mishra created SPARK-10514: Summary: Minimum ratio of registered resources [ spark.scheduler.minRegisteredResourcesRatio] is not enabled for Mesos Coarse Grained mode Key: SPARK-10514 URL:

[jira] [Commented] (SPARK-10301) For struct type, if parquet's global schema has less fields than a file's schema, data reading will fail

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736947#comment-14736947 ] Apache Spark commented on SPARK-10301: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-10428) Struct fields read from parquet are mis-aligned

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736949#comment-14736949 ] Apache Spark commented on SPARK-10428: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736879#comment-14736879 ] Sean Owen commented on SPARK-10493: --- That much should be OK. zipPartitions only makes sense if you

[jira] [Created] (SPARK-10515) When kill executor, there is no need to seed RequestExecutors to AM

2015-09-09 Thread KaiXinXIaoLei (JIRA)
KaiXinXIaoLei created SPARK-10515: - Summary: When kill executor, there is no need to seed RequestExecutors to AM Key: SPARK-10515 URL: https://issues.apache.org/jira/browse/SPARK-10515 Project: Spark

[jira] [Resolved] (SPARK-8793) error/warning with pyspark WholeTextFiles.first

2015-09-09 Thread Diana Carroll (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diana Carroll resolved SPARK-8793. -- Resolution: Not A Problem this is no longer occurring. > error/warning with pyspark

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Glenn Strycker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736869#comment-14736869 ] Glenn Strycker commented on SPARK-10493: The RDD I am using has the form ((String, String),

[jira] [Commented] (SPARK-2960) Spark executables fail to start via symlinks

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736923#comment-14736923 ] Apache Spark commented on SPARK-2960: - User 'jerryshao' has created a pull request for this issue:

[jira] [Commented] (SPARK-10515) When kill executor, there is no need to seed RequestExecutors to AM

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736853#comment-14736853 ] Apache Spark commented on SPARK-10515: -- User 'KaiXinXiaoLei' has created a pull request for this

[jira] [Assigned] (SPARK-10515) When kill executor, there is no need to seed RequestExecutors to AM

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10515: Assignee: Apache Spark > When kill executor, there is no need to seed RequestExecutors to

[jira] [Assigned] (SPARK-10515) When kill executor, there is no need to seed RequestExecutors to AM

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10515: Assignee: (was: Apache Spark) > When kill executor, there is no need to seed

[jira] [Commented] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736961#comment-14736961 ] Davies Liu commented on SPARK-10512: As we discussed here

[jira] [Closed] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-10512. -- Resolution: Won't Fix > Fix @since when a function doesn't have doc >

[jira] [Updated] (SPARK-10517) Console "Output" field is empty when using DataFrameWriter.json

2015-09-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-10517: --- Attachment: screenshot-1.png > Console "Output" field is empty when using

[jira] [Commented] (SPARK-10512) Fix @since when a function doesn't have doc

2015-09-09 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736973#comment-14736973 ] Yu Ishikawa commented on SPARK-10512: - [~davies] oh, I see. Thank you for letting me know. > Fix

[jira] [Assigned] (SPARK-7874) Add a global setting for the fine-grained mesos scheduler that limits the number of concurrent tasks of a job

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7874: --- Assignee: Apache Spark > Add a global setting for the fine-grained mesos scheduler that

[jira] [Assigned] (SPARK-7874) Add a global setting for the fine-grained mesos scheduler that limits the number of concurrent tasks of a job

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7874: --- Assignee: (was: Apache Spark) > Add a global setting for the fine-grained mesos

[jira] [Created] (SPARK-10516) Add values as a property to DenseVector in PySpark

2015-09-09 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-10516: - Summary: Add values as a property to DenseVector in PySpark Key: SPARK-10516 URL: https://issues.apache.org/jira/browse/SPARK-10516 Project: Spark Issue

[jira] [Created] (SPARK-10517) Console "Output" field is empty when using DataFrameWriter.json

2015-09-09 Thread JIRA
Maciej Bryński created SPARK-10517: -- Summary: Console "Output" field is empty when using DataFrameWriter.json Key: SPARK-10517 URL: https://issues.apache.org/jira/browse/SPARK-10517 Project: Spark

[jira] [Commented] (SPARK-10514) Minimum ratio of registered resources [ spark.scheduler.minRegisteredResourcesRatio] is not enabled for Mesos Coarse Grained mode

2015-09-09 Thread Akash Mishra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737103#comment-14737103 ] Akash Mishra commented on SPARK-10514: -- Created a pull request

[jira] [Resolved] (SPARK-10117) Implement SQL data source API for reading LIBSVM data

2015-09-09 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10117. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8537

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Glenn Strycker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737001#comment-14737001 ] Glenn Strycker commented on SPARK-10493: In this example, our RDDs are partitioned with a hash

[jira] [Updated] (SPARK-10517) Console "Output" field is empty when using DataFrameWriter.json

2015-09-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-10517: --- Attachment: (was: screenshot-1.png) > Console "Output" field is empty when using

[jira] [Assigned] (SPARK-10514) Minimum ratio of registered resources [ spark.scheduler.minRegisteredResourcesRatio] is not enabled for Mesos Coarse Grained mode

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10514: Assignee: Apache Spark > Minimum ratio of registered resources [ >

[jira] [Updated] (SPARK-10517) Console "Output" field is empty when using DataFrameWriter.json

2015-09-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-10517: --- Attachment: screenshot-1.png > Console "Output" field is empty when using

[jira] [Comment Edited] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Glenn Strycker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737055#comment-14737055 ] Glenn Strycker edited comment on SPARK-10493 at 9/9/15 3:40 PM: I'm still

[jira] [Updated] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Glenn Strycker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glenn Strycker updated SPARK-10493: --- Attachment: reduceByKey_example_001.scala I'm still working on checking unit tests and

[jira] [Commented] (SPARK-10441) Cannot write timestamp to JSON

2015-09-09 Thread Don Drake (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736986#comment-14736986 ] Don Drake commented on SPARK-10441: --- Got it, thanks for the clarification. > Cannot write timestamp to

[jira] [Commented] (SPARK-7874) Add a global setting for the fine-grained mesos scheduler that limits the number of concurrent tasks of a job

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736984#comment-14736984 ] Apache Spark commented on SPARK-7874: - User 'dragos' has created a pull request for this issue:

[jira] [Updated] (SPARK-10517) Console "Output" field is empty when using DataFrameWriter.json

2015-09-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-10517: --- Description: On HTTP application UI "Output" field is empty when using DataFrameWriter.json.

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737051#comment-14737051 ] Sean Owen commented on SPARK-10493: --- I think you still have the same issue with zipPartitions, unless

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737050#comment-14737050 ] Sean Owen commented on SPARK-10493: --- I think you still have the same issue with zipPartitions, unless

[jira] [Assigned] (SPARK-10514) Minimum ratio of registered resources [ spark.scheduler.minRegisteredResourcesRatio] is not enabled for Mesos Coarse Grained mode

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10514: Assignee: (was: Apache Spark) > Minimum ratio of registered resources [ >

[jira] [Commented] (SPARK-10514) Minimum ratio of registered resources [ spark.scheduler.minRegisteredResourcesRatio] is not enabled for Mesos Coarse Grained mode

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737095#comment-14737095 ] Apache Spark commented on SPARK-10514: -- User 'SleepyThread' has created a pull request for this

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737730#comment-14737730 ] Sean Owen commented on SPARK-10493: --- checkpoint doesn't materialize the RDD, which is why it occurred

[jira] [Updated] (SPARK-9998) Create local intersect operator

2015-09-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9998: - Assignee: Shixiong Zhu > Create local intersect operator > --- > >

[jira] [Updated] (SPARK-9997) Create local Expand operator

2015-09-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9997: - Assignee: Shixiong Zhu > Create local Expand operator > > >

[jira] [Commented] (SPARK-6724) Model import/export for FPGrowth

2015-09-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737802#comment-14737802 ] Joseph K. Bradley commented on SPARK-6724: -- Now that the 1.5 release stuff is over, yes! Thanks

[jira] [Commented] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737801#comment-14737801 ] Shivaram Venkataraman commented on SPARK-10523: --- cc [~mengxr] [~ekhliang] > SparkR formula

[jira] [Updated] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-10523: -- Component/s: SparkR > SparkR formula syntax to turn strings/factors into

[jira] [Updated] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Warmerdam updated SPARK-10523: -- Issue Type: Improvement (was: Bug) > SparkR formula syntax to turn strings/factors

[jira] [Updated] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-10523: -- Component/s: ML > SparkR formula syntax to turn strings/factors into numerics

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Glenn Strycker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737727#comment-14737727 ] Glenn Strycker commented on SPARK-10493: I already have that added in my code that I'm testing...

[jira] [Commented] (SPARK-10522) Nanoseconds part of Timestamp should be positive in parquet

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737738#comment-14737738 ] Apache Spark commented on SPARK-10522: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10522) Nanoseconds part of Timestamp should be positive in parquet

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10522: Assignee: Apache Spark > Nanoseconds part of Timestamp should be positive in parquet >

[jira] [Assigned] (SPARK-10522) Nanoseconds part of Timestamp should be positive in parquet

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10522: Assignee: (was: Apache Spark) > Nanoseconds part of Timestamp should be positive in

[jira] [Commented] (SPARK-10520) dates cannot be summarised in SparkR

2015-09-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737743#comment-14737743 ] Reynold Xin commented on SPARK-10520: - Is the idea here to support aggregation functions on date and

[jira] [Commented] (SPARK-10520) dates cannot be summarised in SparkR

2015-09-09 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737757#comment-14737757 ] Vincent Warmerdam commented on SPARK-10520: --- It just occured to me that there is a very similar

[jira] [Updated] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Warmerdam updated SPARK-10523: -- Description: In normal (non SparkR) R the formula syntax enables strings or factors to

[jira] [Commented] (SPARK-9715) Store numFeatures in all ML PredictionModel types

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737797#comment-14737797 ] Apache Spark commented on SPARK-9715: - User 'sethah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-9715) Store numFeatures in all ML PredictionModel types

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9715: --- Assignee: (was: Apache Spark) > Store numFeatures in all ML PredictionModel types >

[jira] [Assigned] (SPARK-9715) Store numFeatures in all ML PredictionModel types

2015-09-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9715: --- Assignee: Apache Spark > Store numFeatures in all ML PredictionModel types >

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737681#comment-14737681 ] Sean Owen commented on SPARK-10493: --- If the RDD is a result of reduceByKey, I agree that the keys

[jira] [Comment Edited] (SPARK-10520) dates cannot be summarised in SparkR

2015-09-09 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737757#comment-14737757 ] Vincent Warmerdam edited comment on SPARK-10520 at 9/9/15 10:58 PM:

[jira] [Updated] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Warmerdam updated SPARK-10523: -- Description: In normal (non SparkR) R the formula syntax enables strings or factors

[jira] [Updated] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Warmerdam updated SPARK-10523: -- Description: In normal (non SparkR) R the formula syntax enables strings or factors to

[jira] [Created] (SPARK-10523) SparkR formula syntax to turn strings/factors into numerics

2015-09-09 Thread Vincent Warmerdam (JIRA)
Vincent Warmerdam created SPARK-10523: - Summary: SparkR formula syntax to turn strings/factors into numerics Key: SPARK-10523 URL: https://issues.apache.org/jira/browse/SPARK-10523 Project: Spark

[jira] [Commented] (SPARK-10487) MLlib model fitting causes DataFrame write to break with OutOfMemory exception

2015-09-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737790#comment-14737790 ] Joseph K. Bradley commented on SPARK-10487: --- Does this failure require there to be an ML model

[jira] [Commented] (SPARK-10520) dates cannot be summarised in SparkR

2015-09-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737805#comment-14737805 ] Shivaram Venkataraman commented on SPARK-10520: --- [~rxin] Yeah the idea here is to support

[jira] [Updated] (SPARK-9996) Create local nested loop join operator

2015-09-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9996: - Assignee: Shixiong Zhu > Create local nested loop join operator > --

[jira] [Updated] (SPARK-9992) Create local sample operator

2015-09-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9992: - Assignee: Shixiong Zhu > Create local sample operator > > >

[jira] [Updated] (SPARK-9994) Create local TopK operator

2015-09-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9994: - Assignee: Shixiong Zhu > Create local TopK operator > -- > > Key:

[jira] [Updated] (SPARK-9990) Create local hash join operator

2015-09-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9990: - Assignee: Shixiong Zhu > Create local hash join operator > --- > >

[jira] [Resolved] (SPARK-9730) Sort Merge Join for Full Outer Join

2015-09-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9730. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8579

[jira] [Commented] (SPARK-10493) reduceByKey not returning distinct results

2015-09-09 Thread Glenn Strycker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737735#comment-14737735 ] Glenn Strycker commented on SPARK-10493: Of course. I have count statements everywhere in order

  1   2   >