[jira] [Assigned] (SPARK-17101) Provide format identifier for TextFileFormat

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17101: Assignee: Apache Spark > Provide format identifier for TextFileFormat > --

[jira] [Commented] (SPARK-17101) Provide format identifier for TextFileFormat

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424001#comment-15424001 ] Apache Spark commented on SPARK-17101: -- User 'jaceklaskowski' has created a pull req

[jira] [Updated] (SPARK-16391) KeyValueGroupedDataset.reduceGroups should support partial aggregation

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16391: Target Version/s: 2.0.1, 2.1.0 (was: 2.1.0) > KeyValueGroupedDataset.reduceGroups should support p

[jira] [Assigned] (SPARK-17102) bypass UserDefinedGenerator for json format check

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17102: Assignee: Apache Spark (was: Wenchen Fan) > bypass UserDefinedGenerator for json format c

[jira] [Commented] (SPARK-17102) bypass UserDefinedGenerator for json format check

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423989#comment-15423989 ] Apache Spark commented on SPARK-17102: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-17102) bypass UserDefinedGenerator for json format check

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17102: Assignee: Wenchen Fan (was: Apache Spark) > bypass UserDefinedGenerator for json format c

[jira] [Updated] (SPARK-17101) Provide format identifier for TextFileFormat

2016-08-16 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-17101: Description: Define the format identifier that is used in {{Optimized Logical Plan}} in {{

[jira] [Created] (SPARK-17102) bypass UserDefinedGenerator for json format check

2016-08-16 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-17102: --- Summary: bypass UserDefinedGenerator for json format check Key: SPARK-17102 URL: https://issues.apache.org/jira/browse/SPARK-17102 Project: Spark Issue Type: T

[jira] [Created] (SPARK-17101) Provide format identifier for TextFileFormat

2016-08-16 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-17101: --- Summary: Provide format identifier for TextFileFormat Key: SPARK-17101 URL: https://issues.apache.org/jira/browse/SPARK-17101 Project: Spark Issue Type

[jira] [Resolved] (SPARK-17068) Retain view visibility information through out Analysis

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17068. - Resolution: Fixed Fix Version/s: 2.0.1 > Retain view visibility information through out An

[jira] [Updated] (SPARK-17068) Retain view visibility information through out Analysis

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17068: Fix Version/s: (was: 2.0.1) > Retain view visibility information through out Analysis > ---

[jira] [Commented] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

2016-08-16 Thread Tim Sell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423917#comment-15423917 ] Tim Sell commented on SPARK-17100: -- I don't know why, but using `dataframe.cache()` befo

[jira] [Updated] (SPARK-17084) Rename ParserUtils.assert to validate

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17084: Summary: Rename ParserUtils.assert to validate (was: Rename ParserUtils.assert to require) > Rena

[jira] [Updated] (SPARK-17084) Rename ParserUtils.assert to validate

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17084: Description: We currently have an assert method in ParserUtils. This is, however, not used as an as

[jira] [Resolved] (SPARK-17084) Rename ParserUtils.assert to require

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17084. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Rename ParserUtils.asser

[jira] [Updated] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

2016-08-16 Thread Tim Sell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Sell updated SPARK-17100: - Attachment: test_bug.py > pyspark filter on a udf column after join gives > java.lang.UnsupportedOperati

[jira] [Updated] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

2016-08-16 Thread Tim Sell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Sell updated SPARK-17100: - Attachment: bug.py > pyspark filter on a udf column after join gives > java.lang.UnsupportedOperationExc

[jira] [Created] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

2016-08-16 Thread Tim Sell (JIRA)
Tim Sell created SPARK-17100: Summary: pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException Key: SPARK-17100 URL: https://issues.apache.org/jira/browse/SPARK-17100 Proje

[jira] [Updated] (SPARK-17082) Replace ByteBuffer with ChunkedByteBuffer

2016-08-16 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-17082: Description: The size of ByteBuffers can not be greater than 2G, should be replaced by ChunkedByteB

[jira] [Updated] (SPARK-17082) Replace ByteBuffer with ChunkedByteBuffer

2016-08-16 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-17082: Description: The size of ByteBuffers can not be greater than 2G, should be replaced by Java (was:

[jira] [Commented] (SPARK-17093) Roundtrip encoding of array> fields is wrong when whole-stage codegen is disabled

2016-08-16 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423777#comment-15423777 ] Liwei Lin commented on SPARK-17093: --- Oh the interpreted evaluation codepath indeed forg

[jira] [Updated] (SPARK-17099) Incorrect result when HAVING clause is added to group by query

2016-08-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17099: --- Description: Random query generation uncovered the following query which returns incorrect results w

[jira] [Created] (SPARK-17099) Incorrect result when complex HAVING clause is added to query

2016-08-16 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17099: -- Summary: Incorrect result when complex HAVING clause is added to query Key: SPARK-17099 URL: https://issues.apache.org/jira/browse/SPARK-17099 Project: Spark Is

[jira] [Updated] (SPARK-17099) Incorrect result when HAVING clause is added to group by query

2016-08-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17099: --- Summary: Incorrect result when HAVING clause is added to group by query (was: Incorrect result when

[jira] [Commented] (SPARK-17054) SparkR can not run in yarn-cluster mode on mac os

2016-08-16 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423745#comment-15423745 ] Jeff Zhang commented on SPARK-17054: I push another commit to disable downloading spa

[jira] [Commented] (SPARK-16578) Configurable hostname for RBackend

2016-08-16 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423741#comment-15423741 ] Jeff Zhang commented on SPARK-16578: Another scenario I'd like to clarify is that. Sa

[jira] [Assigned] (SPARK-17054) SparkR can not run in yarn-cluster mode on mac os

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17054: Assignee: Apache Spark > SparkR can not run in yarn-cluster mode on mac os > -

[jira] [Assigned] (SPARK-17054) SparkR can not run in yarn-cluster mode on mac os

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17054: Assignee: (was: Apache Spark) > SparkR can not run in yarn-cluster mode on mac os > --

[jira] [Commented] (SPARK-16757) Set up caller context to HDFS

2016-08-16 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423730#comment-15423730 ] Weiqing Yang commented on SPARK-16757: -- Hi, [~srowen] Could you help review this PR

[jira] [Commented] (SPARK-16947) Support type coercion and foldable expression for inline tables

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423678#comment-15423678 ] Apache Spark commented on SPARK-16947: -- User 'petermaxlee' has created a pull reques

[jira] [Updated] (SPARK-16947) Support type coercion and foldable expression for inline tables

2016-08-16 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Lee updated SPARK-16947: -- Summary: Support type coercion and foldable expression for inline tables (was: Improve type coercion a

[jira] [Updated] (SPARK-16947) Improve type coercion and support foldable expression for inline tables

2016-08-16 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Lee updated SPARK-16947: -- Summary: Improve type coercion and support foldable expression for inline tables (was: Improve type co

[jira] [Updated] (SPARK-17096) Fix StreamingQueryListener to return message and stacktrace of actual exception

2016-08-16 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-17096: -- Description: Currently only the stacktrace of StreamingQueryException is returned through Stre

[jira] [Updated] (SPARK-17096) Fix StreamingQueryListener to return message and stacktrace of actual exception

2016-08-16 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-17096: -- Description: Currently only the stacktrace of StreamingQueryException is returned through Stre

[jira] [Assigned] (SPARK-17096) Fix StreamingQueryListener to return message and stacktrace of actual exception

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17096: Assignee: Apache Spark (was: Tathagata Das) > Fix StreamingQueryListener to return messag

[jira] [Commented] (SPARK-17096) Fix StreamingQueryListener to return message and stacktrace of actual exception

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423650#comment-15423650 ] Apache Spark commented on SPARK-17096: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-17096) Fix StreamingQueryListener to return message and stacktrace of actual exception

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17096: Assignee: Tathagata Das (was: Apache Spark) > Fix StreamingQueryListener to return messag

[jira] [Commented] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423634#comment-15423634 ] Josh Rosen commented on SPARK-17098: Actually, given the error here I think that the

[jira] [Updated] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17098: --- Description: Running {code} SELECT COUNT(NULL) OVER () {code} throws an UnsupportedOperationExcepti

[jira] [Created] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-16 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17098: -- Summary: "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis Key: SPARK-17098 URL: https://issues.apache.org/jira/browse/SPARK-17098 Project

[jira] [Commented] (SPARK-17095) Latex and Scala doc do not play nicely

2016-08-16 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423602#comment-15423602 ] Jakob Odersky commented on SPARK-17095: --- Since this bug also occurs when there are

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16700: --- Labels: releasenotes (was: ) > StructType doesn't accept Python dicts anymore >

[jira] [Created] (SPARK-17097) Pregel does not keep vertex state properly; fails to terminate

2016-08-16 Thread Seth Bromberger (JIRA)
Seth Bromberger created SPARK-17097: --- Summary: Pregel does not keep vertex state properly; fails to terminate Key: SPARK-17097 URL: https://issues.apache.org/jira/browse/SPARK-17097 Project: Spark

[jira] [Assigned] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17002: Assignee: Apache Spark > Document that spark.ssl.protocol. is required for SSL > -

[jira] [Commented] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423564#comment-15423564 ] Apache Spark commented on SPARK-17002: -- User 'wangmiao1981' has created a pull reque

[jira] [Assigned] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17002: Assignee: (was: Apache Spark) > Document that spark.ssl.protocol. is required for SSL

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423559#comment-15423559 ] Marcelo Vanzin commented on SPARK-16725: Don't get me wrong, I feel your pain, an

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423555#comment-15423555 ] Russell Spitzer commented on SPARK-16725: - I'm well aware as we've been dealing w

[jira] [Created] (SPARK-17096) Fix StreamingQueryListener to return message and stacktrace of actual exception

2016-08-16 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-17096: - Summary: Fix StreamingQueryListener to return message and stacktrace of actual exception Key: SPARK-17096 URL: https://issues.apache.org/jira/browse/SPARK-17096 Pro

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423544#comment-15423544 ] Marcelo Vanzin commented on SPARK-16725: Welcome to dependency hell. See my comme

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423542#comment-15423542 ] Marcelo Vanzin commented on SPARK-16725: Spark's use of Guava is shaded. That com

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423539#comment-15423539 ] Russell Spitzer commented on SPARK-16725: - In our case it's exposing a library wh

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Brian Hess (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423534#comment-15423534 ] Brian Hess commented on SPARK-16725: - But it looks like it is not being shaded in 2.

[jira] [Commented] (SPARK-17095) Latex and Scala doc do not play nicely

2016-08-16 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423514#comment-15423514 ] Seth Hendrickson commented on SPARK-17095: -- cc [~lins05] [~srowen] [~jodersky]

[jira] [Created] (SPARK-17095) Latex and Scala doc do not play nicely

2016-08-16 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17095: Summary: Latex and Scala doc do not play nicely Key: SPARK-17095 URL: https://issues.apache.org/jira/browse/SPARK-17095 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423505#comment-15423505 ] Marcelo Vanzin commented on SPARK-16725: Exposing 3rd-party libraries in an API s

[jira] [Commented] (SPARK-16725) Migrate Guava to 16+?

2016-08-16 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423498#comment-15423498 ] Russell Spitzer commented on SPARK-16725: - I think *But it works* is a bit of an

[jira] [Created] (SPARK-17094) provide simplified API for ML pipeline

2016-08-16 Thread yuhao yang (JIRA)
yuhao yang created SPARK-17094: -- Summary: provide simplified API for ML pipeline Key: SPARK-17094 URL: https://issues.apache.org/jira/browse/SPARK-17094 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-17093) Roundtrip encoding of array> fields is wrong when whole-stage codegen is disabled

2016-08-16 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17093: -- Summary: Roundtrip encoding of array> fields is wrong when whole-stage codegen is disabled Key: SPARK-17093 URL: https://issues.apache.org/jira/browse/SPARK-17093 Project

[jira] [Commented] (SPARK-15083) History Server would OOM due to unlimited TaskUIData in some stages

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423393#comment-15423393 ] Apache Spark commented on SPARK-15083: -- User 'ajbozarth' has created a pull request

[jira] [Created] (SPARK-17092) DataFrame with large number of columns causing code generation error

2016-08-16 Thread Aris V (JIRA)
Aris V created SPARK-17092: -- Summary: DataFrame with large number of columns causing code generation error Key: SPARK-17092 URL: https://issues.apache.org/jira/browse/SPARK-17092 Project: Spark Iss

[jira] [Commented] (SPARK-17034) Ordinal in ORDER BY or GROUP BY should be treated as an unresolved expression

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423376#comment-15423376 ] Apache Spark commented on SPARK-17034: -- User 'petermaxlee' has created a pull reques

[jira] [Updated] (SPARK-16656) CreateTableAsSelectSuite is flaky

2016-08-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16656: - Fix Version/s: 1.6.3 > CreateTableAsSelectSuite is flaky > - > >

[jira] [Resolved] (SPARK-17089) Remove link of api doc for mapReduceTriplets because its removed from api.

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17089. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Remove link of api doc f

[jira] [Updated] (SPARK-17089) Remove link of api doc for mapReduceTriplets because its removed from api.

2016-08-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17089: Assignee: sandeep purohit > Remove link of api doc for mapReduceTriplets because its removed from a

[jira] [Commented] (SPARK-17091) ParquetFilters rewrite IN to OR of Eq

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423245#comment-15423245 ] Apache Spark commented on SPARK-17091: -- User 'andreweduffy' has created a pull reque

[jira] [Assigned] (SPARK-17091) ParquetFilters rewrite IN to OR of Eq

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17091: Assignee: Apache Spark > ParquetFilters rewrite IN to OR of Eq > -

[jira] [Assigned] (SPARK-17091) ParquetFilters rewrite IN to OR of Eq

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17091: Assignee: (was: Apache Spark) > ParquetFilters rewrite IN to OR of Eq > --

[jira] [Commented] (SPARK-16654) UI Should show blacklisted executors & nodes

2016-08-16 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423240#comment-15423240 ] Alex Bozarth commented on SPARK-16654: -- I don't have time right now to tackle this s

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423235#comment-15423235 ] Apache Spark commented on SPARK-15285: -- User 'kiszk' has created a pull request for

[jira] [Assigned] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15285: Assignee: Apache Spark (was: Kazuaki Ishizaki) > Generated SpecificSafeProjection.apply m

[jira] [Assigned] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15285: Assignee: Kazuaki Ishizaki (was: Apache Spark) > Generated SpecificSafeProjection.apply m

[jira] [Updated] (SPARK-16519) Handle SparkR RDD generics that create warnings in R CMD check

2016-08-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-16519: -- Assignee: Felix Cheung > Handle SparkR RDD generics that create warnings in R C

[jira] [Resolved] (SPARK-16519) Handle SparkR RDD generics that create warnings in R CMD check

2016-08-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-16519. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue

[jira] [Created] (SPARK-17091) ParquetFilters rewrite IN to OR of Eq

2016-08-16 Thread Andrew Duffy (JIRA)
Andrew Duffy created SPARK-17091: Summary: ParquetFilters rewrite IN to OR of Eq Key: SPARK-17091 URL: https://issues.apache.org/jira/browse/SPARK-17091 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

2016-08-16 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423156#comment-15423156 ] Miao Wang commented on SPARK-17002: --- I think we can add a require() statement and remov

[jira] [Commented] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-16 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423151#comment-15423151 ] Seth Hendrickson commented on SPARK-17090: -- cc [~dbtsai] > Make tree aggregatio

[jira] [Created] (SPARK-17090) Make tree aggregation level in linear/logistic regression configurable

2016-08-16 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17090: Summary: Make tree aggregation level in linear/logistic regression configurable Key: SPARK-17090 URL: https://issues.apache.org/jira/browse/SPARK-17090 Projec

[jira] [Commented] (SPARK-17054) SparkR can not run in yarn-cluster mode on mac os

2016-08-16 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423139#comment-15423139 ] Miao Wang commented on SPARK-17054: --- Maybe, I can try it out. > SparkR can not run in

[jira] [Commented] (SPARK-17089) Remove link of api doc for mapReduceTriplets because its removed from api.

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423135#comment-15423135 ] Apache Spark commented on SPARK-17089: -- User 'phalodi' has created a pull request fo

[jira] [Assigned] (SPARK-17089) Remove link of api doc for mapReduceTriplets because its removed from api.

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17089: Assignee: Apache Spark > Remove link of api doc for mapReduceTriplets because its removed

[jira] [Assigned] (SPARK-17089) Remove link of api doc for mapReduceTriplets because its removed from api.

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17089: Assignee: (was: Apache Spark) > Remove link of api doc for mapReduceTriplets because i

[jira] [Created] (SPARK-17089) Remove link of api doc for mapReduceTriplets because its removed from api.

2016-08-16 Thread sandeep purohit (JIRA)
sandeep purohit created SPARK-17089: --- Summary: Remove link of api doc for mapReduceTriplets because its removed from api. Key: SPARK-17089 URL: https://issues.apache.org/jira/browse/SPARK-17089 Pro

[jira] [Created] (SPARK-17088) IsolatedClientLoader fails to load Hive client when sharesHadoopClasses is false

2016-08-16 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-17088: -- Summary: IsolatedClientLoader fails to load Hive client when sharesHadoopClasses is false Key: SPARK-17088 URL: https://issues.apache.org/jira/browse/SPARK-17088

[jira] [Resolved] (SPARK-17035) Conversion of datetime.max to microseconds produces incorrect value

2016-08-16 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17035. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14631 [https://github.

[jira] [Commented] (SPARK-16656) CreateTableAsSelectSuite is flaky

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423051#comment-15423051 ] Apache Spark commented on SPARK-16656: -- User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-17087) Make Spark on Mesos honor port restrictions - Documentation

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423007#comment-15423007 ] Apache Spark commented on SPARK-17087: -- User 'skonto' has created a pull request for

[jira] [Assigned] (SPARK-17087) Make Spark on Mesos honor port restrictions - Documentation

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17087: Assignee: (was: Apache Spark) > Make Spark on Mesos honor port restrictions - Document

[jira] [Assigned] (SPARK-17087) Make Spark on Mesos honor port restrictions - Documentation

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17087: Assignee: Apache Spark > Make Spark on Mesos honor port restrictions - Documentation > ---

[jira] [Updated] (SPARK-17087) Make Spark on Mesos honor port restrictions - Documentation

2016-08-16 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-17087: Description: Need to add the documentation missing from: https://issues.apache.org/

[jira] [Commented] (SPARK-17087) Make Spark on Mesos honor port restrictions - Documentation

2016-08-16 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422996#comment-15422996 ] Stavros Kontopoulos commented on SPARK-17087: - WIP > Make Spark on Mesos hon

[jira] [Created] (SPARK-17087) Make Spark on Mesos honor port restrictions - Documentation

2016-08-16 Thread Stavros Kontopoulos (JIRA)
Stavros Kontopoulos created SPARK-17087: --- Summary: Make Spark on Mesos honor port restrictions - Documentation Key: SPARK-17087 URL: https://issues.apache.org/jira/browse/SPARK-17087 Project: Sp

[jira] [Created] (SPARK-17086) QuantileDiscretizer throws InvalidArgumentException (parameter splits given invalid value) on valid data

2016-08-16 Thread Barry Becker (JIRA)
Barry Becker created SPARK-17086: Summary: QuantileDiscretizer throws InvalidArgumentException (parameter splits given invalid value) on valid data Key: SPARK-17086 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Assigned] (SPARK-16578) Configurable hostname for RBackend

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16578: Assignee: (was: Apache Spark) > Configurable hostname for RBackend > -

[jira] [Commented] (SPARK-16578) Configurable hostname for RBackend

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422978#comment-15422978 ] Apache Spark commented on SPARK-16578: -- User 'junyangq' has created a pull request f

[jira] [Assigned] (SPARK-16578) Configurable hostname for RBackend

2016-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16578: Assignee: Apache Spark > Configurable hostname for RBackend >

[jira] [Commented] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

2016-08-16 Thread Yongjia Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422887#comment-15422887 ] Yongjia Wang commented on SPARK-16484: -- Here is my solution using Spark UDAF and UDT

[jira] [Commented] (SPARK-17085) Documentation and actual code differs - Unsupported Operations

2016-08-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422864#comment-15422864 ] Sean Owen commented on SPARK-17085: --- Yes I think the first doc link is wrong. Go ahead

[jira] [Commented] (SPARK-11714) Make Spark on Mesos honor port restrictions

2016-08-16 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422856#comment-15422856 ] Stavros Kontopoulos commented on SPARK-11714: - I will create another one for

[jira] [Updated] (SPARK-17085) Documentation and actual code differs - Unsupported Operations

2016-08-16 Thread Samritti (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samritti updated SPARK-17085: - Priority: Minor (was: Major) > Documentation and actual code differs - Unsupported Operations >

  1   2   >