[jira] [Resolved] (SPARK-11286) Make Outbox stopped exception singleton

2015-10-23 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-11286. Resolution: Won't Fix > Make Outbox stopped exception singleton > ---

[jira] [Commented] (SPARK-11162) Allow enabling debug logging from the command line

2015-10-23 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971520#comment-14971520 ] Ryan Williams commented on SPARK-11162: --- Makes sense, some googling has left me with the impression

[jira] [Updated] (SPARK-11289) Substitute code examples in ML features with include_example

2015-10-23 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-11289: -- Description: Substitute code examples with include_example. (was: [~mengxr] I have one question,

[jira] [Commented] (SPARK-11289) Substitute code examples in ML features with include_example

2015-10-23 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971536#comment-14971536 ] Xusen Yin commented on SPARK-11289: --- [~mengxr] I have one question, there are some code examples in the

[jira] [Commented] (SPARK-5210) Support log rolling in EventLogger

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970670#comment-14970670 ] Apache Spark commented on SPARK-5210: - User 'XuTingjun' has created a pull request for this issue:

[jira] [Created] (SPARK-11289) Substitute code examples in ML features with include_example

2015-10-23 Thread Xusen Yin (JIRA)
Xusen Yin created SPARK-11289: - Summary: Substitute code examples in ML features with include_example Key: SPARK-11289 URL: https://issues.apache.org/jira/browse/SPARK-11289 Project: Spark

[jira] [Assigned] (SPARK-11277) sort_array throws exception scala.MatchError

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11277: Assignee: Apache Spark > sort_array throws exception scala.MatchError >

[jira] [Created] (SPARK-11279) Add DataFrame#toDF in PySpark

2015-10-23 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-11279: -- Summary: Add DataFrame#toDF in PySpark Key: SPARK-11279 URL: https://issues.apache.org/jira/browse/SPARK-11279 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970836#comment-14970836 ] Sean Owen commented on SPARK-10857: --- Rick you're saying that this code path only comes up when the

[jira] [Resolved] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11229. --- Resolution: Cannot Reproduce Fix Version/s: (was: 1.6.0) > NPE in JoinedRow.isNullAt when

[jira] [Commented] (SPARK-11167) Incorrect type resolution on heterogeneous data structures

2015-10-23 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970839#comment-14970839 ] Maciej Szymkiewicz commented on SPARK-11167: spark-csv has a much simpler job to do and

[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971188#comment-14971188 ] Steve Loughran commented on SPARK-11265: Pull request is :

[jira] [Resolved] (SPARK-10277) Add @since annotation to pyspark.mllib.regression

2015-10-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10277. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8684

[jira] [Commented] (SPARK-7970) Optimize code for SQL queries fired on Union of RDDs (closure cleaner)

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971217#comment-14971217 ] Apache Spark commented on SPARK-7970: - User 'nitin2goyal' has created a pull request for this issue:

[jira] [Assigned] (SPARK-7970) Optimize code for SQL queries fired on Union of RDDs (closure cleaner)

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7970: --- Assignee: (was: Apache Spark) > Optimize code for SQL queries fired on Union of RDDs

[jira] [Assigned] (SPARK-7970) Optimize code for SQL queries fired on Union of RDDs (closure cleaner)

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7970: --- Assignee: Apache Spark > Optimize code for SQL queries fired on Union of RDDs (closure

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-10-23 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:01 PM: I just want to

[jira] [Assigned] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11265: Assignee: Apache Spark > YarnClient can't get tokens to talk to Hive in a secure cluster

[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971190#comment-14971190 ] Apache Spark commented on SPARK-11265: -- User 'steveloughran' has created a pull request for this

[jira] [Assigned] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11265: Assignee: (was: Apache Spark) > YarnClient can't get tokens to talk to Hive in a

[jira] [Resolved] (SPARK-6723) Model import/export for ChiSqSelector

2015-10-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6723. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 6785

[jira] [Updated] (SPARK-10610) Using AppName instead of AppId in the name of all metrics

2015-10-23 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-10610: Summary: Using AppName instead of AppId in the name of all metrics (was: Using AppName instead AppId in

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-10-23 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:02 PM: I just want to

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-10-23 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:15 PM: I just want to

[jira] [Comment Edited] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-10-23 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971228#comment-14971228 ] Jerry Lam edited comment on SPARK-4940 at 10/23/15 4:16 PM: I just want to

[jira] [Updated] (SPARK-6723) Model import/export for ChiSqSelector

2015-10-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6723: - Assignee: Jayant Shekhar > Model import/export for ChiSqSelector >

[jira] [Commented] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-10-23 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971228#comment-14971228 ] Jerry Lam commented on SPARK-4940: -- I just want to weight in the importance of this issue. My observation

[jira] [Commented] (SPARK-10975) Shuffle files left behind on Mesos without dynamic allocation

2015-10-23 Thread Chris Bannister (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971265#comment-14971265 ] Chris Bannister commented on SPARK-10975: - Spark will use MESOS_DIRECTORY sandbox when not using

[jira] [Updated] (SPARK-11280) Mesos cluster deployment using only one node

2015-10-23 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-11280: -- Attachment: Screen Shot 2015-10-23 at 11.37.43.png > Mesos cluster deployment using only one

[jira] [Updated] (SPARK-7021) JUnit output for Python tests

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7021: - Assignee: Gabor Liptak > JUnit output for Python tests > - > >

[jira] [Comment Edited] (SPARK-9325) Support `collect` on DataFrame columns

2015-10-23 Thread Russell Pierce (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970917#comment-14970917 ] Russell Pierce edited comment on SPARK-9325 at 10/23/15 12:53 PM: -- You're

[jira] [Updated] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-11282: --- Description: Hi, I found very strange broadcast join behaviour. According to this Jira

[jira] [Resolved] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11282. --- Resolution: Duplicate [~maver1ck] this could use a better title, and there is no code attached. I

[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970930#comment-14970930 ] Steve Loughran commented on SPARK-11265: I can trigger a failure in a unit test now, once you get

[jira] [Commented] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970931#comment-14970931 ] Maciej Bryński commented on SPARK-11282: We had race condition here. I was attaching file when

[jira] [Commented] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970952#comment-14970952 ] Maciej Bryński commented on SPARK-11282: UPDATE: Looks like: -XX:-UseCompressedOops solve the

[jira] [Commented] (SPARK-6847) Stack overflow on updateStateByKey which followed by a dstream with checkpoint set

2015-10-23 Thread Glyton Camilleri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971004#comment-14971004 ] Glyton Camilleri commented on SPARK-6847: - Hi, we managed to actually get rid of the overflow

[jira] [Created] (SPARK-11283) List column gets additional level of nesting when converted to Spark DataFrame

2015-10-23 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-11283: -- Summary: List column gets additional level of nesting when converted to Spark DataFrame Key: SPARK-11283 URL: https://issues.apache.org/jira/browse/SPARK-11283

[jira] [Commented] (SPARK-9325) Support `collect` on DataFrame columns

2015-10-23 Thread Russell Pierce (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970917#comment-14970917 ] Russell Pierce commented on SPARK-9325: --- You're right, Spark had been producing an error because the

[jira] [Commented] (SPARK-11167) Incorrect type resolution on heterogeneous data structures

2015-10-23 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970922#comment-14970922 ] Maciej Szymkiewicz commented on SPARK-11167: Related problem:

[jira] [Commented] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

2015-10-23 Thread Jim Haughwout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970915#comment-14970915 ] Jim Haughwout commented on SPARK-6270: -- [~tdas]: Can the team update this issue to reflect that this

[jira] [Updated] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-11282: --- Description: Hi, I found very strange broadcast join behaviour. According to this Jira

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971027#comment-14971027 ] Apache Spark commented on SPARK-11016: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11016: Assignee: (was: Apache Spark) > Spark fails when running with a task that requires a

[jira] [Updated] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-11282: --- Description: Hi, I found very strange broadcast join behaviour. According to this Jira

[jira] [Updated] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-11282: --- Attachment: SPARK-11282.py > Very strange broadcast join behaviour >

[jira] [Comment Edited] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970931#comment-14970931 ] Maciej Bryński edited comment on SPARK-11282 at 10/23/15 1:07 PM: -- We

[jira] [Comment Edited] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970931#comment-14970931 ] Maciej Bryński edited comment on SPARK-11282 at 10/23/15 1:07 PM: -- We

[jira] [Commented] (SPARK-11258) Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame

2015-10-23 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971025#comment-14971025 ] Frank Rosner commented on SPARK-11258: -- Actually I am pretty confused now. Thinking about it, having

[jira] [Created] (SPARK-11282) Very strange broadcast join behaviour

2015-10-23 Thread JIRA
Maciej Bryński created SPARK-11282: -- Summary: Very strange broadcast join behaviour Key: SPARK-11282 URL: https://issues.apache.org/jira/browse/SPARK-11282 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-11281) Issue with creating and collecting DataFrame using environments

2015-10-23 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-11281: -- Summary: Issue with creating and collecting DataFrame using environments Key: SPARK-11281 URL: https://issues.apache.org/jira/browse/SPARK-11281

[jira] [Issue Comment Deleted] (SPARK-11167) Incorrect type resolution on heterogeneous data structures

2015-10-23 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-11167: --- Comment: was deleted (was: Related problem:

[jira] [Updated] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

2015-10-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-6270: -- Affects Version/s: 1.5.1 > Standalone Master hangs when streaming job completes and event

[jira] [Created] (SPARK-11284) ALS produces predictions as floats and should be double

2015-10-23 Thread Dominik Dahlem (JIRA)
Dominik Dahlem created SPARK-11284: -- Summary: ALS produces predictions as floats and should be double Key: SPARK-11284 URL: https://issues.apache.org/jira/browse/SPARK-11284 Project: Spark

[jira] [Commented] (SPARK-10562) .partitionBy() creates the metastore partition columns in all lowercase, but persists the data path as MixedCase resulting in an error when the data is later attempted

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971042#comment-14971042 ] Apache Spark commented on SPARK-10562: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-10947) With schema inference from JSON into a Dataframe, add option to infer all primitive object types as strings

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970777#comment-14970777 ] Apache Spark commented on SPARK-10947: -- User 'stephend-realitymine' has created a pull request for

[jira] [Commented] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-10-23 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970870#comment-14970870 ] Nick Pentreath commented on SPARK-7008: --- Is this now going in 1.6 (as per SPARK-10324)? If so is

[jira] [Created] (SPARK-11278) PageRank fails with unified memory manager

2015-10-23 Thread Nishkam Ravi (JIRA)
Nishkam Ravi created SPARK-11278: Summary: PageRank fails with unified memory manager Key: SPARK-11278 URL: https://issues.apache.org/jira/browse/SPARK-11278 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11277) sort_array throws exception scala.MatchError

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970694#comment-14970694 ] Apache Spark commented on SPARK-11277: -- User 'jliwork' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11277) sort_array throws exception scala.MatchError

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11277: Assignee: (was: Apache Spark) > sort_array throws exception scala.MatchError >

[jira] [Closed] (SPARK-11087) spark.sql.orc.filterPushdown does not work, No ORC pushdown predicate

2015-10-23 Thread patcharee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] patcharee closed SPARK-11087. - Resolution: Not A Problem The predicate is indeed generated and can be found in the executor log >

[jira] [Commented] (SPARK-9265) Dataframe.limit joined with another dataframe can be non-deterministic

2015-10-23 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970801#comment-14970801 ] Yanbo Liang commented on SPARK-9265: I'm working on it. > Dataframe.limit joined with another

[jira] [Updated] (SPARK-11262) Unit test for gradient, loss layers, memory management for multilayer perceptron

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11262: -- Target Version/s: (was: 1.6.0) Fix Version/s: (was: 1.5.1) [~avulanov] don't set

[jira] [Updated] (SPARK-11267) NettyRpcEnv and sparkDriver services report the same port in the logs

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11267: -- Component/s: Spark Core > NettyRpcEnv and sparkDriver services report the same port in the logs >

[jira] [Updated] (SPARK-11270) Add improved equality testing for TopicAndPartition from the Kafka Streaming API

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11270: -- Target Version/s: (was: 1.5.1) Fix Version/s: (was: 1.5.1) [~manygrams] have a look at

[jira] [Commented] (SPARK-11087) spark.sql.orc.filterPushdown does not work, No ORC pushdown predicate

2015-10-23 Thread patcharee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970786#comment-14970786 ] patcharee commented on SPARK-11087: --- [~zzhan] I found the predicate generated in the executor log for

[jira] [Commented] (SPARK-11279) Add DataFrame#toDF in PySpark

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970705#comment-14970705 ] Apache Spark commented on SPARK-11279: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Updated] (SPARK-11259) Params.validateParams() should be called automatically

2015-10-23 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-11259: Description: Params.validateParams() can not be called automatically currently. Such as the

[jira] [Commented] (SPARK-6724) Model import/export for FPGrowth

2015-10-23 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970887#comment-14970887 ] Yanbo Liang commented on SPARK-6724: [~MeethuMathew] I will take over this task and send a PR, welcome

[jira] [Assigned] (SPARK-11279) Add DataFrame#toDF in PySpark

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11279: Assignee: (was: Apache Spark) > Add DataFrame#toDF in PySpark >

[jira] [Assigned] (SPARK-11279) Add DataFrame#toDF in PySpark

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11279: Assignee: Apache Spark > Add DataFrame#toDF in PySpark > - >

[jira] [Created] (SPARK-11280) Mesos cluster deployment using only one node

2015-10-23 Thread Iulian Dragos (JIRA)
Iulian Dragos created SPARK-11280: - Summary: Mesos cluster deployment using only one node Key: SPARK-11280 URL: https://issues.apache.org/jira/browse/SPARK-11280 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970783#comment-14970783 ] Sean Owen commented on SPARK-11016: --- NB: the resolution here may be to simply remove usage of

[jira] [Reopened] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-11229: --- "Fixed" implies there was a change attached to this JIRA that resolved the issue, and we don't have

[jira] [Commented] (SPARK-4940) Support more evenly distributing cores for Mesos mode

2015-10-23 Thread Martin Tapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971416#comment-14971416 ] Martin Tapp commented on SPARK-4940: No real workaround for now as we need the round-robin strategy.

[jira] [Assigned] (SPARK-11286) Make Outbox stopped exception singleton

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11286: Assignee: (was: Apache Spark) > Make Outbox stopped exception singleton >

[jira] [Assigned] (SPARK-11286) Make Outbox stopped exception singleton

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11286: Assignee: Apache Spark > Make Outbox stopped exception singleton >

[jira] [Commented] (SPARK-11286) Make Outbox stopped exception singleton

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971307#comment-14971307 ] Apache Spark commented on SPARK-11286: -- User 'tedyu' has created a pull request for this issue:

[jira] [Commented] (SPARK-11162) Allow enabling debug logging from the command line

2015-10-23 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971361#comment-14971361 ] Ryan Williams commented on SPARK-11162: --- Do you know how I might enable debug logging with -D

[jira] [Updated] (SPARK-11278) PageRank fails with unified memory manager

2015-10-23 Thread Michael Malak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Malak updated SPARK-11278: -- Component/s: GraphX > PageRank fails with unified memory manager >

[jira] [Created] (SPARK-11287) Executing deploy.client TestClient fails with bad class name

2015-10-23 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-11287: Summary: Executing deploy.client TestClient fails with bad class name Key: SPARK-11287 URL: https://issues.apache.org/jira/browse/SPARK-11287 Project: Spark

[jira] [Assigned] (SPARK-11287) Executing deploy.client TestClient fails with bad class name

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11287: Assignee: (was: Apache Spark) > Executing deploy.client TestClient fails with bad

[jira] [Commented] (SPARK-11287) Executing deploy.client TestClient fails with bad class name

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971454#comment-14971454 ] Apache Spark commented on SPARK-11287: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-11287) Executing deploy.client TestClient fails with bad class name

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11287: Assignee: Apache Spark > Executing deploy.client TestClient fails with bad class name >

[jira] [Created] (SPARK-11286) Make Outbox stopped exception singleton

2015-10-23 Thread Ted Yu (JIRA)
Ted Yu created SPARK-11286: -- Summary: Make Outbox stopped exception singleton Key: SPARK-11286 URL: https://issues.apache.org/jira/browse/SPARK-11286 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-10975) Shuffle files left behind on Mesos without dynamic allocation

2015-10-23 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971268#comment-14971268 ] Iulian Dragos commented on SPARK-10975: --- No, it's not a duplicate, but fixed by the same PR :) >

[jira] [Updated] (SPARK-11258) Converting a Spark DataFrame into an R data.frame is slow / requires a lot of memory

2015-10-23 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Rosner updated SPARK-11258: - Description: h4. Problem We tried to collect a DataFrame with > 1 million rows and a few

[jira] [Commented] (SPARK-11162) Allow enabling debug logging from the command line

2015-10-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971438#comment-14971438 ] Sean Owen commented on SPARK-11162: --- Hm, can you set logger levels with syntax like

[jira] [Commented] (SPARK-11278) PageRank fails with unified memory manager

2015-10-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971442#comment-14971442 ] Andrew Or commented on SPARK-11278: --- are there any exceptions in the executor logs? Does the problem go

[jira] [Created] (SPARK-11288) Specify the return type for UDF in Scala

2015-10-23 Thread Davies Liu (JIRA)
Davies Liu created SPARK-11288: -- Summary: Specify the return type for UDF in Scala Key: SPARK-11288 URL: https://issues.apache.org/jira/browse/SPARK-11288 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-11294) Improve R doc for read.df, write.df, saveAsTable

2015-10-23 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-11294: -- Assignee: Felix Cheung > Improve R doc for read.df, write.df, saveAsTable >

[jira] [Resolved] (SPARK-11294) Improve R doc for read.df, write.df, saveAsTable

2015-10-23 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-11294. --- Resolution: Fixed Fix Version/s: 1.5.2 1.6.0 Issue

[jira] [Commented] (SPARK-11289) Substitute code examples in ML features with include_example

2015-10-23 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972345#comment-14972345 ] Xusen Yin commented on SPARK-11289: --- A feasible way to do it is create new example files in

[jira] [Created] (SPARK-11294) Improve R doc for read.df, write.df, saveAsTable

2015-10-23 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11294: Summary: Improve R doc for read.df, write.df, saveAsTable Key: SPARK-11294 URL: https://issues.apache.org/jira/browse/SPARK-11294 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11255) R Test build should run on R 3.1.1

2015-10-23 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972418#comment-14972418 ] Sun Rui commented on SPARK-11255: - +1 for this request. Or we can update the supported R version to a

[jira] [Assigned] (SPARK-11294) Improve R doc for read.df, write.df, saveAsTable

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11294: Assignee: (was: Apache Spark) > Improve R doc for read.df, write.df, saveAsTable >

[jira] [Commented] (SPARK-11294) Improve R doc for read.df, write.df, saveAsTable

2015-10-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972330#comment-14972330 ] Apache Spark commented on SPARK-11294: -- User 'felixcheung' has created a pull request for this

[jira] [Resolved] (SPARK-11125) Unreadable exception when running spark-sql without building with -Phive-thriftserver and SPARK_PREPEND_CLASSES is set

2015-10-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-11125. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9134

[jira] [Updated] (SPARK-11125) Unreadable exception when running spark-sql without building with -Phive-thriftserver and SPARK_PREPEND_CLASSES is set

2015-10-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-11125: - Assignee: Jeff Zhang > Unreadable exception when running spark-sql without building with >

[jira] [Resolved] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-23 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-10971. --- Resolution: Fixed Fix Version/s: 1.5.2 1.6.0 Issue

  1   2   >