[jira] [Commented] (SPARK-10251) Some internal spark classes are not registered with kryo

2015-09-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739770#comment-14739770 ] Reynold Xin commented on SPARK-10251: - we should have one test suite for that -- just run some basic

[jira] [Comment Edited] (SPARK-10489) GraphX dataframe wrapper

2015-09-10 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735206#comment-14735206 ] Feynman Liang edited comment on SPARK-10489 at 9/10/15 11:52 PM: - Doing

[jira] [Commented] (SPARK-10110) StringIndexer lacks of parameter "handleInvalid".

2015-09-10 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740235#comment-14740235 ] Yanbo Liang commented on SPARK-10110: - Since this issue is fixed by SPARK-10027, do you mind to close

[jira] [Assigned] (SPARK-10050) Support collecting data of MapType in DataFrame

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10050: Assignee: (was: Apache Spark) > Support collecting data of MapType in DataFrame >

[jira] [Assigned] (SPARK-10050) Support collecting data of MapType in DataFrame

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10050: Assignee: Apache Spark > Support collecting data of MapType in DataFrame >

[jira] [Commented] (SPARK-10050) Support collecting data of MapType in DataFrame

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740089#comment-14740089 ] Apache Spark commented on SPARK-10050: -- User 'sun-rui' has created a pull request for this issue:

[jira] [Updated] (SPARK-7770) Should GBT validationTol be relative tolerance?

2015-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7770: - Assignee: Yanbo Liang > Should GBT validationTol be relative tolerance? >

[jira] [Resolved] (SPARK-10023) Unified DecisionTreeParams "checkpointInterval" between Scala and Python API.

2015-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10023. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8528

[jira] [Created] (SPARK-10557) Publish Spark 1.5.0 on Maven central

2015-09-10 Thread Marko Asplund (JIRA)
Marko Asplund created SPARK-10557: - Summary: Publish Spark 1.5.0 on Maven central Key: SPARK-10557 URL: https://issues.apache.org/jira/browse/SPARK-10557 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-7770) Should GBT validationTol be relative tolerance?

2015-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7770: - Shepherd: Joseph K. Bradley > Should GBT validationTol be relative tolerance? >

[jira] [Updated] (SPARK-7770) Should GBT validationTol be relative tolerance?

2015-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7770: - Target Version/s: 1.6.0 > Should GBT validationTol be relative tolerance? >

[jira] [Updated] (SPARK-10027) Add Python API missing methods for ml.feature

2015-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10027: -- Assignee: Yanbo Liang > Add Python API missing methods for ml.feature >

[jira] [Resolved] (SPARK-10027) Add Python API missing methods for ml.feature

2015-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10027. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8313

[jira] [Commented] (SPARK-6724) Model import/export for FPGrowth

2015-09-10 Thread Meethu Mathew (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740160#comment-14740160 ] Meethu Mathew commented on SPARK-6724: -- [~josephkb] I will take a look into it and update the PR

[jira] [Commented] (SPARK-10552) Connection String for SparkR to Cassandra

2015-09-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740167#comment-14740167 ] Sean Owen commented on SPARK-10552: --- Can you provide a description? this doesn't specify an issue as

[jira] [Created] (SPARK-10546) Check partitionId's range in ExternalSorter#spill()

2015-09-10 Thread Ted Yu (JIRA)
Ted Yu created SPARK-10546: -- Summary: Check partitionId's range in ExternalSorter#spill() Key: SPARK-10546 URL: https://issues.apache.org/jira/browse/SPARK-10546 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-10049) Support collecting data of ArraryType in DataFrame

2015-09-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-10049. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request

[jira] [Updated] (SPARK-10049) Support collecting data of ArraryType in DataFrame

2015-09-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-10049: -- Assignee: Sun Rui > Support collecting data of ArraryType in DataFrame >

[jira] [Assigned] (SPARK-10547) Streamline / improve style of Java API tests

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10547: Assignee: Apache Spark (was: Sean Owen) > Streamline / improve style of Java API tests >

[jira] [Commented] (SPARK-10056) PySpark Row - Support for row["columnName"] syntax

2015-09-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739482#comment-14739482 ] Maciej Bryński commented on SPARK-10056: [~davies] Is there a chance that PR from

[jira] [Updated] (SPARK-10548) Concurrent execution in SQL does not work

2015-09-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10548: -- Assignee: Andrew Or > Concurrent execution in SQL does not work >

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

2015-09-10 Thread Aliaksei Belablotski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739499#comment-14739499 ] Aliaksei Belablotski commented on SPARK-10528: -- Thanks a lot Marcelo. Yes, Windows is

[jira] [Commented] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-09-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739502#comment-14739502 ] Mark Grover commented on SPARK-9790: I was waiting on SPARK-8167 to get committed. That just committed

[jira] [Resolved] (SPARK-9990) Create local hash join operator

2015-09-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-9990. -- Resolution: Fixed Fix Version/s: 1.6.0 > Create local hash join operator >

[jira] [Created] (SPARK-10547) Streamline / improve style of Java API tests

2015-09-10 Thread Sean Owen (JIRA)
Sean Owen created SPARK-10547: - Summary: Streamline / improve style of Java API tests Key: SPARK-10547 URL: https://issues.apache.org/jira/browse/SPARK-10547 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-10547) Streamline / improve style of Java API tests

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10547: Assignee: Sean Owen (was: Apache Spark) > Streamline / improve style of Java API tests >

[jira] [Assigned] (SPARK-10542) The PySpark 1.5 closure serializer can't serialize a namedtuple instance.

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10542: Assignee: Davies Liu (was: Apache Spark) > The PySpark 1.5 closure serializer can't

[jira] [Assigned] (SPARK-10542) The PySpark 1.5 closure serializer can't serialize a namedtuple instance.

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10542: Assignee: Apache Spark (was: Davies Liu) > The PySpark 1.5 closure serializer can't

[jira] [Commented] (SPARK-10542) The PySpark 1.5 closure serializer can't serialize a namedtuple instance.

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739468#comment-14739468 ] Apache Spark commented on SPARK-10542: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-10547) Streamline / improve style of Java API tests

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739464#comment-14739464 ] Apache Spark commented on SPARK-10547: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-10544) Serialization of Python namedtuple subclasses in functions / closures is broken

2015-09-10 Thread Doug Bateman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739492#comment-14739492 ] Doug Bateman commented on SPARK-10544: -- This also fails in Spark 1.5 pyspark

[jira] [Created] (SPARK-10548) Concurrent execution in SQL does not work

2015-09-10 Thread Andrew Or (JIRA)
Andrew Or created SPARK-10548: - Summary: Concurrent execution in SQL does not work Key: SPARK-10548 URL: https://issues.apache.org/jira/browse/SPARK-10548 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

2015-09-10 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739516#comment-14739516 ] Marcelo Vanzin commented on SPARK-10528: I think a lot of code in this area changed between 1.4

[jira] [Closed] (SPARK-10544) Serialization of Python namedtuple subclasses in functions / closures is broken

2015-09-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-10544. -- Resolution: Duplicate Fix Version/s: (was: 1.5.1) Target Version/s: 1.5.1 >

[jira] [Assigned] (SPARK-10546) Check partitionId's range in ExternalSorter#spill()

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10546: Assignee: Apache Spark > Check partitionId's range in ExternalSorter#spill() >

[jira] [Commented] (SPARK-10546) Check partitionId's range in ExternalSorter#spill()

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739411#comment-14739411 ] Apache Spark commented on SPARK-10546: -- User 'tedyu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10546) Check partitionId's range in ExternalSorter#spill()

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10546: Assignee: (was: Apache Spark) > Check partitionId's range in ExternalSorter#spill() >

[jira] [Created] (SPARK-10549) scala 2.11 spark on yarn with security - Repl doesn't work

2015-09-10 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-10549: - Summary: scala 2.11 spark on yarn with security - Repl doesn't work Key: SPARK-10549 URL: https://issues.apache.org/jira/browse/SPARK-10549 Project: Spark

[jira] [Resolved] (SPARK-10443) Refactor SortMergeOuterJoin to reduce duplication

2015-09-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10443. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8596

[jira] [Created] (SPARK-10550) SQLListener error constructing extended SQLContext

2015-09-10 Thread shao lo (JIRA)
shao lo created SPARK-10550: --- Summary: SQLListener error constructing extended SQLContext Key: SPARK-10550 URL: https://issues.apache.org/jira/browse/SPARK-10550 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-9990) Create local hash join operator

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739567#comment-14739567 ] Apache Spark commented on SPARK-9990: - User 'andrewor14' has created a pull request for this issue:

[jira] [Resolved] (SPARK-10056) PySpark Row - Support for row["columnName"] syntax

2015-09-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10056. Resolution: Fixed Assignee: Yanbo Liang Fix Version/s: 1.6.0 > PySpark Row -

[jira] [Commented] (SPARK-8939) YARN EC2 default setting fails with IllegalArgumentException

2015-09-10 Thread Heji Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739578#comment-14739578 ] Heji Kim commented on SPARK-8939: - I was trying to upgrade to 1.5 today and could not submit drivers due

[jira] [Created] (SPARK-10551) Successful task-end event after task failed due to executor loss

2015-09-10 Thread Ryan Williams (JIRA)
Ryan Williams created SPARK-10551: - Summary: Successful task-end event after task failed due to executor loss Key: SPARK-10551 URL: https://issues.apache.org/jira/browse/SPARK-10551 Project: Spark

[jira] [Closed] (SPARK-10397) Make Python's SparkContext self-descriptive on "print sc"

2015-09-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-10397. -- Resolution: Won't Fix > Make Python's SparkContext self-descriptive on "print sc" >

[jira] [Commented] (SPARK-10551) Successful task-end event after task failed due to executor loss

2015-09-10 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739615#comment-14739615 ] Ryan Williams commented on SPARK-10551: --- Here is the full event log:

[jira] [Commented] (SPARK-10551) Successful task-end event after task failed due to executor loss

2015-09-10 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739619#comment-14739619 ] Ryan Williams commented on SPARK-10551: --- The same behavior is observable on a second task,

[jira] [Updated] (SPARK-10551) Successful task-end event after task failed due to executor loss

2015-09-10 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Williams updated SPARK-10551: -- Description: Doing forensics on some failed Spark applications and seeing nonsensical things

[jira] [Commented] (SPARK-10551) Successful task-end event after task failed due to executor loss

2015-09-10 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739630#comment-14739630 ] Ryan Williams commented on SPARK-10551: --- Something else I just noticed: both of the

[jira] [Commented] (SPARK-10277) Add @since annotation to pyspark.mllib.regression

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738252#comment-14738252 ] Apache Spark commented on SPARK-10277: -- User 'yu-iskw' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10277) Add @since annotation to pyspark.mllib.regression

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10277: Assignee: Apache Spark > Add @since annotation to pyspark.mllib.regression >

[jira] [Assigned] (SPARK-10277) Add @since annotation to pyspark.mllib.regression

2015-09-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10277: Assignee: (was: Apache Spark) > Add @since annotation to pyspark.mllib.regression >

[jira] [Created] (SPARK-10531) AppId is set as AppName in status rest api

2015-09-10 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-10531: -- Summary: AppId is set as AppName in status rest api Key: SPARK-10531 URL: https://issues.apache.org/jira/browse/SPARK-10531 Project: Spark Issue Type:

[jira] [Created] (SPARK-10539) Intersection Optimization is Wrong

2015-09-10 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-10539: Summary: Intersection Optimization is Wrong Key: SPARK-10539 URL: https://issues.apache.org/jira/browse/SPARK-10539 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

2015-09-10 Thread Aliaksei Belablotski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739292#comment-14739292 ] Aliaksei Belablotski commented on SPARK-10528: -- Yes, Spark Shell is working - I'm

<    1   2   3