[jira] [Assigned] (SPARK-13296) Move UserDefinedFunction into sql.expressions package

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13296: Assignee: Reynold Xin (was: Apache Spark) > Move UserDefinedFunction into

[jira] [Assigned] (SPARK-13296) Move UserDefinedFunction into sql.expressions package

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13296: Assignee: Apache Spark (was: Reynold Xin) > Move UserDefinedFunction into

[jira] [Commented] (SPARK-13296) Move UserDefinedFunction into sql.expressions package

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144289#comment-15144289 ] Apache Spark commented on SPARK-13296: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-13298) DAG visualization does not render correctly for jobs

2016-02-12 Thread Lucas Woltmann (JIRA)
Lucas Woltmann created SPARK-13298: -- Summary: DAG visualization does not render correctly for jobs Key: SPARK-13298 URL: https://issues.apache.org/jira/browse/SPARK-13298 Project: Spark

[jira] [Commented] (SPARK-13289) Word2Vec generate infinite distances when numIterations>5

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144383#comment-15144383 ] Sean Owen commented on SPARK-13289: --- THere isn't enough info to reproduce anything here. We don't have

[jira] [Resolved] (SPARK-13290) wholeTextFile and binaryFiles are really slow

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13290. --- Resolution: Invalid I don't think this is actionable. "Slow" relative to what? slow reading can be

[jira] [Updated] (SPARK-12974) Add Python API for spark.ml bisecting k-means

2016-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-12974: -- Fix Version/s: 2.0.0 > Add Python API for spark.ml bisecting k-means >

[jira] [Updated] (SPARK-12974) Add Python API for spark.ml bisecting k-means

2016-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-12974: -- Assignee: Yanbo Liang > Add Python API for spark.ml bisecting k-means >

[jira] [Resolved] (SPARK-12974) Add Python API for spark.ml bisecting k-means

2016-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12974. --- Resolution: Fixed > Add Python API for spark.ml bisecting k-means >

[jira] [Created] (SPARK-13297) [SQL] Backticks cannot be escaped in column names

2016-02-12 Thread Grzegorz Chilkiewicz (JIRA)
Grzegorz Chilkiewicz created SPARK-13297: Summary: [SQL] Backticks cannot be escaped in column names Key: SPARK-13297 URL: https://issues.apache.org/jira/browse/SPARK-13297 Project: Spark

[jira] [Created] (SPARK-13296) Move UserDefinedFunction into sql.expressions package

2016-02-12 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-13296: --- Summary: Move UserDefinedFunction into sql.expressions package Key: SPARK-13296 URL: https://issues.apache.org/jira/browse/SPARK-13296 Project: Spark Issue

[jira] [Commented] (SPARK-8000) SQLContext.read.load() should be able to auto-detect input data

2016-02-12 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144285#comment-15144285 ] Yanbo Liang commented on SPARK-8000: [~hyukjin.kwon] I'm not working on this issue now, please feel

[jira] [Updated] (SPARK-13154) Add pydoc lint for docs

2016-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13154: -- Assignee: holdenk > Add pydoc lint for docs > --- > > Key:

[jira] [Updated] (SPARK-13154) Add pydoc lint for docs

2016-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13154: -- Target Version/s: 2.0.0 > Add pydoc lint for docs > --- > >

[jira] [Resolved] (SPARK-13154) Add pydoc lint for docs

2016-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13154. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11109

[jira] [Commented] (SPARK-13286) JDBC driver doesn't report full exception

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144398#comment-15144398 ] Sean Owen commented on SPARK-13286: --- Agree it would be better, though I don't see a good clean way to

[jira] [Assigned] (SPARK-13172) Stop using RichException.getStackTrace it is deprecated

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13172: Assignee: (was: Apache Spark) > Stop using RichException.getStackTrace it is

[jira] [Commented] (SPARK-5925) YARN - Spark progress bar stucks at 10% but after finishing shows 100%

2016-02-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144460#comment-15144460 ] Steve Loughran commented on SPARK-5925: --- I think this should be fixed by SPARK-7889; linking to it.

[jira] [Updated] (SPARK-13299) DataFrame limit operation is not consistent

2016-02-12 Thread Nazarii Balkovskyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nazarii Balkovskyi updated SPARK-13299: --- Attachment: SparkLimitIssue.png > DataFrame limit operation is not consistent >

[jira] [Commented] (SPARK-13172) Stop using RichException.getStackTrace it is deprecated

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144404#comment-15144404 ] Apache Spark commented on SPARK-13172: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13172) Stop using RichException.getStackTrace it is deprecated

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13172: Assignee: Apache Spark > Stop using RichException.getStackTrace it is deprecated >

[jira] [Updated] (SPARK-13299) DataFrame limit operation is not consistent

2016-02-12 Thread Nazarii Balkovskyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nazarii Balkovskyi updated SPARK-13299: --- Description: I faced to a problem with using limit method from DataFrame API. I try

[jira] [Updated] (SPARK-13299) DataFrame limit operation is not consistent

2016-02-12 Thread Nazarii Balkovskyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nazarii Balkovskyi updated SPARK-13299: --- Description: I faced to a problem with using limit method from DataFrame API. I try

[jira] [Commented] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-12 Thread Fabiano Francesconi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144540#comment-15144540 ] Fabiano Francesconi commented on SPARK-12583: - +1 is there a follow-up on this issue? Looks

[jira] [Resolved] (SPARK-13299) DataFrame limit operation is not consistent

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13299. --- Resolution: Not A Problem Unless your DataFrame has a defined ordering, I don't think you'd expect

[jira] [Commented] (SPARK-13287) Standalone REST API throttling?

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144386#comment-15144386 ] Sean Owen commented on SPARK-13287: --- What's in the second output, an error? > Standalone REST API

[jira] [Commented] (SPARK-13288) [1.6.0] Memory leak in Spark streaming

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144384#comment-15144384 ] Sean Owen commented on SPARK-13288: --- Why do you think this is a leak though? what's leaking? > [1.6.0]

[jira] [Commented] (SPARK-12646) Support _HOST in kerberos principal for connecting to secure cluster

2016-02-12 Thread Hari Krishna Dara (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144465#comment-15144465 ] Hari Krishna Dara commented on SPARK-12646: --- Sorry for the delay in getting back, but I had to

[jira] [Updated] (SPARK-13294) Don't build assembly in dev/run-tests

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13294: -- Component/s: Build > Don't build assembly in dev/run-tests > - > >

[jira] [Updated] (SPARK-13299) DataFrame limit operation is not consistent

2016-02-12 Thread Nazarii Balkovskyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nazarii Balkovskyi updated SPARK-13299: --- Description: I faced to a problem with using limit method from DataFrame API. I try

[jira] [Updated] (SPARK-13299) DataFrame limit operation is not consistent

2016-02-12 Thread Nazarii Balkovskyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nazarii Balkovskyi updated SPARK-13299: --- Description: I faced to a problem with using limit method from DataFrame API. I try

[jira] [Commented] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-12 Thread Adrian Bridgett (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144624#comment-15144624 ] Adrian Bridgett commented on SPARK-12583: - Phew - thought maybe it was a bit odd no-one else had

[jira] [Created] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread stefan (JIRA)
stefan created SPARK-13300: -- Summary: Spark examples page gives errors : Liquid error: pygments Key: SPARK-13300 URL: https://issues.apache.org/jira/browse/SPARK-13300 Project: Spark Issue Type:

[jira] [Commented] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144626#comment-15144626 ] Sean Owen commented on SPARK-13300: --- Hm, I see that on all browsers on OS X too. I wonder if this is

[jira] [Reopened] (SPARK-13290) wholeTextFile and binaryFiles are really slow

2016-02-12 Thread mathieu longtin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mathieu longtin reopened SPARK-13290: - Slow relative to reading the exact same file on a local disk on the same machine. Python

[jira] [Commented] (SPARK-13289) Word2Vec generate infinite distances when numIterations>5

2016-02-12 Thread Qi Dai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144685#comment-15144685 ] Qi Dai commented on SPARK-13289: I'm using the "One Billion Words Language Modeling" dataset available

[jira] [Commented] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread stefan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144621#comment-15144621 ] stefan commented on SPARK-13300: jekyll has never been installed on this machine. is it needed to render

[jira] [Created] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-12 Thread Simone (JIRA)
Simone created SPARK-13301: -- Summary: PySpark Dataframe return wrong results with custom UDF Key: SPARK-13301 URL: https://issues.apache.org/jira/browse/SPARK-13301 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13298) DAG visualization does not render correctly for jobs

2016-02-12 Thread Lucas Woltmann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Woltmann updated SPARK-13298: --- Attachment: dag_viz.png > DAG visualization does not render correctly for jobs >

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Christopher Bourez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144844#comment-15144844 ] Christopher Bourez commented on SPARK-12261: Sean Owen, do you reconsider the status as a

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Christopher Bourez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144873#comment-15144873 ] Christopher Bourez commented on SPARK-12261: Here is what i see when i activate the logs :

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144880#comment-15144880 ] Sean Owen commented on SPARK-12261: --- This is still just the driver log. > pyspark crash for large

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Christopher Bourez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144901#comment-15144901 ] Christopher Bourez commented on SPARK-12261: Sean, how can I get the executor log in local

[jira] [Resolved] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-02-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12705. Resolution: Fixed Issue resolved by pull request 11153

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144857#comment-15144857 ] Sean Owen commented on SPARK-12261: --- The change above is definitely not correct in general, right? it's

[jira] [Resolved] (SPARK-13290) wholeTextFile and binaryFiles are really slow

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13290. --- Resolution: Not A Problem Yes, just reading a file length locally is going to be much much faster

[jira] [Closed] (SPARK-13290) wholeTextFile and binaryFiles are really slow

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-13290. - > wholeTextFile and binaryFiles are really slow > - > >

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Christopher Bourez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144793#comment-15144793 ] Christopher Bourez commented on SPARK-12261: Dear Niall Your solution works very well :)

[jira] [Commented] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread stefan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144717#comment-15144717 ] stefan commented on SPARK-13300: Happening on Windows 7 now. Chrome. Internet Explorer 11. > Spark

[jira] [Updated] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-12 Thread Simone (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simone updated SPARK-13301: --- Description: Using a User Defined Function in PySpark inside the withColumn() method of Dataframe, gives

[jira] [Updated] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-12 Thread Simone (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simone updated SPARK-13301: --- Environment: PySpark in yarn-client mode - CDH 5.5.1 (was: PySpark - CDH 5.5.1) > PySpark Dataframe return

[jira] [Commented] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144723#comment-15144723 ] Sean Owen commented on SPARK-13300: --- Yes, it's irrelevant -- you can see this in the source HTML as

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Niall McCarroll (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144734#comment-15144734 ] Niall McCarroll commented on SPARK-12261: - As a workaround you might try the following change to

[jira] [Comment Edited] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Niall McCarroll (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144734#comment-15144734 ] Niall McCarroll edited comment on SPARK-12261 at 2/12/16 3:53 PM: -- As a

[jira] [Updated] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-12 Thread Simone (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simone updated SPARK-13301: --- Description: Using a User Defined Function in PySpark inside the withColumn() method of Dataframe, gives

[jira] [Updated] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-12 Thread Simone (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simone updated SPARK-13301: --- Description: Using a User Defined Function in PySpark inside the withColumn() method of Dataframe, gives

[jira] [Updated] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-12 Thread Simone (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simone updated SPARK-13301: --- Description: Using a User Defined Function in PySpark inside the withColumn() method of Dataframe, gives

[jira] [Created] (SPARK-13302) Cleanup Docstests in ml/clustering.py

2016-02-12 Thread holdenk (JIRA)
holdenk created SPARK-13302: --- Summary: Cleanup Docstests in ml/clustering.py Key: SPARK-13302 URL: https://issues.apache.org/jira/browse/SPARK-13302 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-13303) Spark fails with pandas import error when pandas is not explicitly imported by user

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144974#comment-15144974 ] Sean Owen commented on SPARK-13303: --- Agree, I think this is one of those big "known issues", that

[jira] [Resolved] (SPARK-13282) LogicalPlan toSql should just return a String rather than Option[String]

2016-02-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13282. - Resolution: Fixed Fix Version/s: 2.0.0 > LogicalPlan toSql should just return a String

[jira] [Commented] (SPARK-12251) Document Spark 1.6's off-heap memory configurations and add config validation

2016-02-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144988#comment-15144988 ] Josh Rosen commented on SPARK-12251: It's off by default; the 1.6.0 documentation was wrong and this

[jira] [Commented] (SPARK-13287) Standalone REST API throttling?

2016-02-12 Thread Rares Vernica (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144995#comment-15144995 ] Rares Vernica commented on SPARK-13287: --- See the description: {quote} The response from the second

[jira] [Created] (SPARK-13303) Spark fails with pandas import error when pandas is not explicitly imported by user

2016-02-12 Thread Juliet Hougland (JIRA)
Juliet Hougland created SPARK-13303: --- Summary: Spark fails with pandas import error when pandas is not explicitly imported by user Key: SPARK-13303 URL: https://issues.apache.org/jira/browse/SPARK-13303

[jira] [Commented] (SPARK-12630) Make Parameter Descriptions Consistent for PySpark MLlib Classification

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144981#comment-15144981 ] Apache Spark commented on SPARK-12630: -- User 'BryanCutler' has created a pull request for this

[jira] [Commented] (SPARK-12251) Document Spark 1.6's off-heap memory configurations and add config validation

2016-02-12 Thread Ovidiu Marcu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144986#comment-15144986 ] Ovidiu Marcu commented on SPARK-12251: -- Reading though the latest documentation for Memory

[jira] [Commented] (SPARK-12261) pyspark crash for large dataset

2016-02-12 Thread Niall McCarroll (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145077#comment-15145077 ] Niall McCarroll commented on SPARK-12261: - In various windows environments I've tried, I see

[jira] [Commented] (SPARK-10777) order by fails when column is aliased and projection includes windowed aggregate

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145084#comment-15145084 ] Apache Spark commented on SPARK-10777: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13253) Error aliasing array columns.

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13253: Assignee: Apache Spark > Error aliasing array columns. > - >

[jira] [Commented] (SPARK-13253) Error aliasing array columns.

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145088#comment-15145088 ] Apache Spark commented on SPARK-13253: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Commented] (SPARK-12917) Add DML support to Spark SQL for HIVE

2016-02-12 Thread Hemang Nagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145057#comment-15145057 ] Hemang Nagar commented on SPARK-12917: -- Yes it is a transaction table feature, and since Hive

[jira] [Assigned] (SPARK-13253) Error aliasing array columns.

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13253: Assignee: (was: Apache Spark) > Error aliasing array columns. >

[jira] [Commented] (SPARK-13288) [1.6.0] Memory leak in Spark streaming

2016-02-12 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145148#comment-15145148 ] JESSE CHEN commented on SPARK-13288: Maybe "heap exhaustion" a better term to call this. jstat

[jira] [Commented] (SPARK-13288) [1.6.0] Memory leak in Spark streaming

2016-02-12 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145174#comment-15145174 ] JESSE CHEN commented on SPARK-13288: I have the heapdumps from 1.5 and 1.6. They are large so let me

[jira] [Commented] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-12 Thread Xiu (Joe) Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145173#comment-15145173 ] Xiu (Joe) Guo commented on SPARK-13301: --- Hi Simone: How long is the string length for each row in

[jira] [Resolved] (SPARK-13260) count(*) does not work with CSV data source

2016-02-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13260. - Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.0.0 > count(*) does

[jira] [Commented] (SPARK-12632) Make Parameter Descriptions Consistent for PySpark MLlib FPM and Recommendation

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145207#comment-15145207 ] Apache Spark commented on SPARK-12632: -- User 'BryanCutler' has created a pull request for this

[jira] [Closed] (SPARK-6763) CountMinSketch

2016-02-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-6763. -- Resolution: Duplicate Assignee: Reynold Xin (was: Liang-Chi Hsieh) Fix Version/s: 2.0.0

[jira] [Updated] (SPARK-12772) Better error message for syntax error in the SQL parser

2016-02-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12772: Summary: Better error message for syntax error in the SQL parser (was: Better error message for

[jira] [Issue Comment Deleted] (SPARK-12772) Better error message for parsing failure?

2016-02-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12772: Comment: was deleted (was: cc [~hvanhovell] / [~viirya] any idea about this one? ) > Better

[jira] [Updated] (SPARK-12772) Better error message for syntax error in the parser

2016-02-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12772: Summary: Better error message for syntax error in the parser (was: Better error message for

[jira] [Assigned] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13300: Assignee: (was: Apache Spark) > Spark examples page gives errors : Liquid error:

[jira] [Commented] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145857#comment-15145857 ] Apache Spark commented on SPARK-13300: -- User 'amitdev' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13300) Spark examples page gives errors : Liquid error: pygments

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13300: Assignee: Apache Spark > Spark examples page gives errors : Liquid error: pygments >

[jira] [Commented] (SPARK-13307) TPCDS query 66 degraded by 30% in 1.6.0 compared to 1.4.1

2016-02-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145829#comment-15145829 ] Xiao Li commented on SPARK-13307: - Please use explain(true). It will be much easier to analyze when

[jira] [Updated] (SPARK-13307) TPCDS query 66 degraded by 30% in 1.6.0 compared to 1.4.1

2016-02-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13307: Labels: (was: spark, sql,) > TPCDS query 66 degraded by 30% in 1.6.0 compared to 1.4.1 >

[jira] [Commented] (SPARK-13080) Implementation of the internal catalog API using Hive

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145431#comment-15145431 ] Apache Spark commented on SPARK-13080: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Created] (SPARK-13305) With SPARK_WORKER_WEBUI_PORT and --webui-port set for start-slave.sh script, --webui-port is used twice

2016-02-12 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-13305: --- Summary: With SPARK_WORKER_WEBUI_PORT and --webui-port set for start-slave.sh script, --webui-port is used twice Key: SPARK-13305 URL:

[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145239#comment-15145239 ] Apache Spark commented on SPARK-5095: - User 'mgummelt' has created a pull request for this issue:

[jira] [Created] (SPARK-13304) Broadcast join with two ints could be very slow

2016-02-12 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13304: -- Summary: Broadcast join with two ints could be very slow Key: SPARK-13304 URL: https://issues.apache.org/jira/browse/SPARK-13304 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-13306) Uncorrelated scalar subquery

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13306: Assignee: Apache Spark (was: Davies Liu) > Uncorrelated scalar subquery >

[jira] [Assigned] (SPARK-13306) Uncorrelated scalar subquery

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13306: Assignee: Davies Liu (was: Apache Spark) > Uncorrelated scalar subquery >

[jira] [Resolved] (SPARK-12630) Make Parameter Descriptions Consistent for PySpark MLlib Classification

2016-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12630. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11183

[jira] [Created] (SPARK-13306) Uncorrelated scalar subquery

2016-02-12 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13306: -- Summary: Uncorrelated scalar subquery Key: SPARK-13306 URL: https://issues.apache.org/jira/browse/SPARK-13306 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-13306) Uncorrelated scalar subquery

2016-02-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145476#comment-15145476 ] Apache Spark commented on SPARK-13306: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-12917) Add DML support to Spark SQL for HIVE

2016-02-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145479#comment-15145479 ] Herman van Hovell commented on SPARK-12917: --- It is currently not planned, and will require

[jira] [Commented] (SPARK-13287) Standalone REST API throttling?

2016-02-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145254#comment-15145254 ] Sean Owen commented on SPARK-13287: --- Yeah, I was guessing/hoping that wasn't quite right, and that the

[jira] [Resolved] (SPARK-12962) PySpark support covar_samp and covar_pop

2016-02-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12962. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10876

[jira] [Resolved] (SPARK-12544) Support window functions in SQLContext

2016-02-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12544. Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.0.0 Since we

[jira] [Commented] (SPARK-12544) Support window functions in SQLContext

2016-02-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145467#comment-15145467 ] Herman van Hovell commented on SPARK-12544: --- [~davies] No, they only require a SQLContext.

[jira] [Updated] (SPARK-13304) Broadcast join with two ints could be very slow

2016-02-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13304: --- Component/s: SQL > Broadcast join with two ints could be very slow >

  1   2   >