[jira] [Created] (SPARK-17987) ML Evaluator fails to handle null values in the dataset

2016-10-17 Thread bo song (JIRA)
bo song created SPARK-17987: --- Summary: ML Evaluator fails to handle null values in the dataset Key: SPARK-17987 URL: https://issues.apache.org/jira/browse/SPARK-17987 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-17899) add a debug mode to keep raw table properties in HiveExternalCatalog

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584611#comment-15584611 ] Apache Spark commented on SPARK-17899: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-17813) Maximum data per trigger

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17813: Assignee: (was: Apache Spark) > Maximum data per trigger > >

[jira] [Assigned] (SPARK-17813) Maximum data per trigger

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17813: Assignee: Apache Spark > Maximum data per trigger > > >

[jira] [Commented] (SPARK-17813) Maximum data per trigger

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584515#comment-15584515 ] Apache Spark commented on SPARK-17813: -- User 'koeninger' has created a pull request

[jira] [Assigned] (SPARK-17986) SQLTransformer leaks temporary tables

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17986: Assignee: Apache Spark > SQLTransformer leaks temporary tables > -

[jira] [Assigned] (SPARK-17986) SQLTransformer leaks temporary tables

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17986: Assignee: (was: Apache Spark) > SQLTransformer leaks temporary tables > --

[jira] [Commented] (SPARK-17986) SQLTransformer leaks temporary tables

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584466#comment-15584466 ] Apache Spark commented on SPARK-17986: -- User 'drewrobb' has created a pull request f

[jira] [Updated] (SPARK-17986) SQLTransformer leaks temporary tables

2016-10-17 Thread Drew Robb (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Drew Robb updated SPARK-17986: -- Description: The SQLTransformer creates a temporary table when called, and does not delete this tempor

[jira] [Closed] (SPARK-17956) ProjectExec has incorrect outputOrdering property

2016-10-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-17956. --- Resolution: Won't Fix > ProjectExec has incorrect outputOrdering property > -

[jira] [Assigned] (SPARK-17974) Refactor FileCatalog classes to simplify the inheritance tree

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17974: Assignee: Apache Spark (was: Eric Liang) > Refactor FileCatalog classes to simplify the i

[jira] [Assigned] (SPARK-17974) Refactor FileCatalog classes to simplify the inheritance tree

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17974: Assignee: Eric Liang (was: Apache Spark) > Refactor FileCatalog classes to simplify the i

[jira] [Reopened] (SPARK-17974) Refactor FileCatalog classes to simplify the inheritance tree

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-17974: - Reopening since the previous commit was not tested by Jenkins (failed Scala linter). > Refactor Fi

[jira] [Commented] (SPARK-17862) Feature flag SPARK-16980

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584374#comment-15584374 ] Reynold Xin commented on SPARK-17862: - cc [~ekhliang] this was done right? Can you pu

[jira] [Updated] (SPARK-17970) store partition spec in metastore for data source table

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17970: Issue Type: Sub-task (was: New Feature) Parent: SPARK-17861 > store partition spec in meta

[jira] [Updated] (SPARK-17974) Refactor FileCatalog classes to simplify the inheritance tree

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17974: Issue Type: Sub-task (was: Improvement) Parent: SPARK-17861 > Refactor FileCatalog classes

[jira] [Resolved] (SPARK-17974) Refactor FileCatalog classes to simplify the inheritance tree

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17974. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0 > Refactor FileCatalo

[jira] [Comment Edited] (SPARK-14212) Add configuration element for --packages option

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584142#comment-15584142 ] Marcelo Vanzin edited comment on SPARK-14212 at 10/18/16 3:50 AM: -

[jira] [Resolved] (SPARK-17620) hive.default.fileformat=orc does not set OrcSerde

2016-10-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-17620. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15495 [https://github.com/ap

[jira] [Created] (SPARK-17986) SQLTransformer leaks temporary tables

2016-10-17 Thread Drew Robb (JIRA)
Drew Robb created SPARK-17986: - Summary: SQLTransformer leaks temporary tables Key: SPARK-17986 URL: https://issues.apache.org/jira/browse/SPARK-17986 Project: Spark Issue Type: Bug Com

[jira] [Created] (SPARK-17985) Bump commons-lang3 version to 3.5.

2016-10-17 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-17985: - Summary: Bump commons-lang3 version to 3.5. Key: SPARK-17985 URL: https://issues.apache.org/jira/browse/SPARK-17985 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-17985) Bump commons-lang3 version to 3.5.

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17985: Assignee: Apache Spark > Bump commons-lang3 version to 3.5. >

[jira] [Assigned] (SPARK-17985) Bump commons-lang3 version to 3.5.

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17985: Assignee: (was: Apache Spark) > Bump commons-lang3 version to 3.5. > -

[jira] [Commented] (SPARK-17985) Bump commons-lang3 version to 3.5.

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584263#comment-15584263 ] Apache Spark commented on SPARK-17985: -- User 'ueshin' has created a pull request for

[jira] [Updated] (SPARK-17984) Add support for numa aware

2016-10-17 Thread quanfuwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] quanfuwang updated SPARK-17984: --- Description: This Jira is target to add support numa aware feature which can help improve performanc

[jira] [Updated] (SPARK-17984) Add support for numa aware feature

2016-10-17 Thread quanfuwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] quanfuwang updated SPARK-17984: --- Summary: Add support for numa aware feature (was: Add support for numa aware) > Add support for num

[jira] [Resolved] (SPARK-5230) Print usage for spark-submit and spark-class in Windows

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-5230. --- Resolution: Done Pretty sure I implemented this somewhere in the 1.x line. > Print usage for

[jira] [Resolved] (SPARK-5925) YARN - Spark progress bar stucks at 10% but after finishing shows 100%

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-5925. --- Resolution: Won't Fix I don't think this can be fixed in Spark at all. There's no way to know

[jira] [Resolved] (SPARK-6108) No application number limit in spark history server

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6108. --- Resolution: Won't Fix There are many ways currently to control how many applications are kept

[jira] [Resolved] (SPARK-7882) HBase Input Format Example does not allow passing ZK parent node

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-7882. --- Resolution: Not A Problem HBase examples are not included anymore. > HBase Input Format Examp

[jira] [Updated] (SPARK-17984) Add support for numa aware

2016-10-17 Thread quanfuwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] quanfuwang updated SPARK-17984: --- Issue Type: New Feature (was: Task) > Add support for numa aware > -- > >

[jira] [Updated] (SPARK-17984) Add support for numa aware

2016-10-17 Thread quanfuwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] quanfuwang updated SPARK-17984: --- Shepherd: (was: quanfuwang) > Add support for numa aware > -- > >

[jira] [Resolved] (SPARK-8122) ParquetRelation.enableLogForwarding() may fail to configure loggers

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-8122. --- Resolution: Won't Fix This code doesn't exist anymore in 2.x at least, so I'll assume this won

[jira] [Commented] (SPARK-17984) Add support for numa aware

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584206#comment-15584206 ] Apache Spark commented on SPARK-17984: -- User 'quanfuw' has created a pull request fo

[jira] [Assigned] (SPARK-17984) Add support for numa aware

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17984: Assignee: (was: Apache Spark) > Add support for numa aware > -

[jira] [Assigned] (SPARK-17984) Add support for numa aware

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17984: Assignee: Apache Spark > Add support for numa aware > -- > >

[jira] [Resolved] (SPARK-12280) "--packages" command doesn't work in "spark-submit"

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-12280. Resolution: Cannot Reproduce Please reopen with more info if you're still running into issu

[jira] [Updated] (SPARK-17984) Add support for numa aware

2016-10-17 Thread quanfuwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] quanfuwang updated SPARK-17984: --- Description: This Jira is target to add support numa aware feature which can help improve performanc

[jira] [Updated] (SPARK-17984) Add support for numa aware

2016-10-17 Thread quanfuwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] quanfuwang updated SPARK-17984: --- Description: This Jira is target to add support numa aware feature which can help improve performanc

[jira] [Commented] (SPARK-17982) Spark 2.0.0 CREATE VIEW statement fails when select statement contains limit clause

2016-10-17 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584178#comment-15584178 ] Franck Tago commented on SPARK-17982: - == SQL == SELECT `gen_attr_0` AS `WHERE_ID`, `

[jira] [Created] (SPARK-17984) Add support for numa aware

2016-10-17 Thread quanfuwang (JIRA)
quanfuwang created SPARK-17984: -- Summary: Add support for numa aware Key: SPARK-17984 URL: https://issues.apache.org/jira/browse/SPARK-17984 Project: Spark Issue Type: Task Components:

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-10-17 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584172#comment-15584172 ] Cody Koeninger commented on SPARK-17147: Well, are you using compacted topics? >

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-10-17 Thread Justin Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584169#comment-15584169 ] Justin Miller commented on SPARK-17147: --- Could this possibly be related to why I'm

[jira] [Resolved] (SPARK-17504) Spark App Handle from SparkLauncher always returns UNKNOWN app state when used with Mesos in Client Mode

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17504. Resolution: Duplicate > Spark App Handle from SparkLauncher always returns UNKNOWN app stat

[jira] [Commented] (SPARK-14212) Add configuration element for --packages option

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584142#comment-15584142 ] Marcelo Vanzin commented on SPARK-14212: SPARK-15760 added the docs to 2.0 only,

[jira] [Commented] (SPARK-4160) Standalone cluster mode does not upload all needed jars to driver node

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584137#comment-15584137 ] Marcelo Vanzin commented on SPARK-4160: --- You don't need to ask for permission to wor

[jira] [Created] (SPARK-17983) Can't filter over mixed case parquet columns of converted Hive tables

2016-10-17 Thread Eric Liang (JIRA)
Eric Liang created SPARK-17983: -- Summary: Can't filter over mixed case parquet columns of converted Hive tables Key: SPARK-17983 URL: https://issues.apache.org/jira/browse/SPARK-17983 Project: Spark

[jira] [Updated] (SPARK-17983) Can't filter over mixed case parquet columns of converted Hive tables

2016-10-17 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-17983: --- Description: We should probably revive https://github.com/apache/spark/pull/14750 in order to fix th

[jira] [Created] (SPARK-17982) Spark 2.0.0 CREATE VIEW statement fails when select statement contains limit clause

2016-10-17 Thread Franck Tago (JIRA)
Franck Tago created SPARK-17982: --- Summary: Spark 2.0.0 CREATE VIEW statement fails when select statement contains limit clause Key: SPARK-17982 URL: https://issues.apache.org/jira/browse/SPARK-17982 Pr

[jira] [Commented] (SPARK-10872) Derby error (XSDB6) when creating new HiveContext after restarting SparkContext

2016-10-17 Thread Angus Gerry (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584052#comment-15584052 ] Angus Gerry commented on SPARK-10872: - Hi [~srowen], I'm chasing down something in o

[jira] [Comment Edited] (SPARK-17950) Match SparseVector behavior with DenseVector

2016-10-17 Thread AbderRahman Sobh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583877#comment-15583877 ] AbderRahman Sobh edited comment on SPARK-17950 at 10/18/16 12:07 AM: --

[jira] [Comment Edited] (SPARK-17950) Match SparseVector behavior with DenseVector

2016-10-17 Thread AbderRahman Sobh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583877#comment-15583877 ] AbderRahman Sobh edited comment on SPARK-17950 at 10/18/16 12:07 AM: --

[jira] [Comment Edited] (SPARK-17950) Match SparseVector behavior with DenseVector

2016-10-17 Thread AbderRahman Sobh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583877#comment-15583877 ] AbderRahman Sobh edited comment on SPARK-17950 at 10/18/16 12:05 AM: --

[jira] [Updated] (SPARK-17731) Metrics for Structured Streaming

2016-10-17 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-17731: -- Fix Version/s: 2.0.2 > Metrics for Structured Streaming > > >

[jira] [Commented] (SPARK-17950) Match SparseVector behavior with DenseVector

2016-10-17 Thread AbderRahman Sobh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583877#comment-15583877 ] AbderRahman Sobh commented on SPARK-17950: -- Yes, the full array needs to be expa

[jira] [Assigned] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17981: Assignee: Apache Spark (was: Xiao Li) > Incorrectly Set Nullability to False in FilterExe

[jira] [Assigned] (SPARK-17957) Calling outer join and na.fill(0) and then inner join will miss rows

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17957: Assignee: Apache Spark (was: Xiao Li) > Calling outer join and na.fill(0) and then inner

[jira] [Commented] (SPARK-17957) Calling outer join and na.fill(0) and then inner join will miss rows

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583874#comment-15583874 ] Apache Spark commented on SPARK-17957: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-17368) Scala value classes create encoder problems and break at runtime

2016-10-17 Thread Aris Vlasakakis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583873#comment-15583873 ] Aris Vlasakakis commented on SPARK-17368: - That is great, thank you for the help

[jira] [Assigned] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17981: Assignee: Xiao Li (was: Apache Spark) > Incorrectly Set Nullability to False in FilterExe

[jira] [Commented] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583872#comment-15583872 ] Apache Spark commented on SPARK-17981: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-17957) Calling outer join and na.fill(0) and then inner join will miss rows

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17957: Assignee: Xiao Li (was: Apache Spark) > Calling outer join and na.fill(0) and then inner

[jira] [Created] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

2016-10-17 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17981: --- Summary: Incorrectly Set Nullability to False in FilterExec Key: SPARK-17981 URL: https://issues.apache.org/jira/browse/SPARK-17981 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15708) Tasks table in Detailed Stage page shows ip instead of hostname under Executor ID/Host

2016-10-17 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583751#comment-15583751 ] Alex Bozarth commented on SPARK-15708: -- I'm not sure closing this as cannot reproduc

[jira] [Updated] (SPARK-17979) Remove deprecated support for config SPARK_YARN_USER_ENV

2016-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17979: -- Priority: Trivial (was: Major) Issue Type: Improvement (was: Bug) (Please set fields appropriat

[jira] [Commented] (SPARK-17971) Unix timestamp handling in Spark SQL not allowing calculations on UTC times

2016-10-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583676#comment-15583676 ] Sean Owen commented on SPARK-17971: --- I'll say that I find the semantics of the Hive QL

[jira] [Assigned] (SPARK-17980) Fix refreshByPath for converted Hive tables

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17980: Assignee: Apache Spark > Fix refreshByPath for converted Hive tables > ---

[jira] [Commented] (SPARK-17980) Fix refreshByPath for converted Hive tables

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583652#comment-15583652 ] Apache Spark commented on SPARK-17980: -- User 'ericl' has created a pull request for

[jira] [Assigned] (SPARK-17980) Fix refreshByPath for converted Hive tables

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17980: Assignee: (was: Apache Spark) > Fix refreshByPath for converted Hive tables >

[jira] [Created] (SPARK-17980) Fix refreshByPath for converted Hive tables

2016-10-17 Thread Eric Liang (JIRA)
Eric Liang created SPARK-17980: -- Summary: Fix refreshByPath for converted Hive tables Key: SPARK-17980 URL: https://issues.apache.org/jira/browse/SPARK-17980 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-7721) Generate test coverage report from Python

2016-10-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583642#comment-15583642 ] Josh Rosen commented on SPARK-7721: --- IIRC when I looked into this I hit problems with th

[jira] [Commented] (SPARK-10915) Add support for UDAFs in Python

2016-10-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583638#comment-15583638 ] Davies Liu commented on SPARK-10915: Currently all the aggregate functions are implem

[jira] [Created] (SPARK-17979) Remove deprecated support for config SPARK_YARN_USER_ENV

2016-10-17 Thread Kishor Patil (JIRA)
Kishor Patil created SPARK-17979: Summary: Remove deprecated support for config SPARK_YARN_USER_ENV Key: SPARK-17979 URL: https://issues.apache.org/jira/browse/SPARK-17979 Project: Spark Iss

[jira] [Closed] (SPARK-3132) Avoid serialization for Array[Byte] in TorrentBroadcast

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-3132. -- Resolution: Not A Problem Marking it as not-a-problem for now given Josh's comment. > Avoid serializat

[jira] [Commented] (SPARK-3132) Avoid serialization for Array[Byte] in TorrentBroadcast

2016-10-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583605#comment-15583605 ] Josh Rosen commented on SPARK-3132: --- I don't think that this is being actively worked on

[jira] [Updated] (SPARK-3132) Avoid serialization for Array[Byte] in TorrentBroadcast

2016-10-17 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3132: -- Assignee: (was: Davies Liu) > Avoid serialization for Array[Byte] in TorrentBroadcast >

[jira] [Commented] (SPARK-4160) Standalone cluster mode does not upload all needed jars to driver node

2016-10-17 Thread Amit Assudani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583531#comment-15583531 ] Amit Assudani commented on SPARK-4160: -- I can work on fixing this. Let me know. > St

[jira] [Closed] (SPARK-17976) Global options to spark-submit should not be position-sensitive

2016-10-17 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas closed SPARK-17976. Resolution: Not A Problem Ah, makes perfect sense. Would have realized that myself if I had

[jira] [Updated] (SPARK-17977) DataFrameReader and DataStreamReader should have an ancestor class

2016-10-17 Thread Amit Assudani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Assudani updated SPARK-17977: -- Affects Version/s: 2.0.0 > DataFrameReader and DataStreamReader should have an ancestor class >

[jira] [Resolved] (SPARK-17978) --jars option in spark-submit does not load jars for driver in spark - standalone mode

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17978. Resolution: Duplicate > --jars option in spark-submit does not load jars for driver in spar

[jira] [Commented] (SPARK-17978) --jars option in spark-submit does not load jars for driver in spark - standalone mode

2016-10-17 Thread Amit Assudani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583508#comment-15583508 ] Amit Assudani commented on SPARK-17978: --- I can fix this and send a PR. Let me know.

[jira] [Commented] (SPARK-17976) Global options to spark-submit should not be position-sensitive

2016-10-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583506#comment-15583506 ] Marcelo Vanzin commented on SPARK-17976: They are not being ignored. They are bei

[jira] [Created] (SPARK-17978) --jars option in spark-submit does not load jars for driver in spark - standalone mode

2016-10-17 Thread Amit Assudani (JIRA)
Amit Assudani created SPARK-17978: - Summary: --jars option in spark-submit does not load jars for driver in spark - standalone mode Key: SPARK-17978 URL: https://issues.apache.org/jira/browse/SPARK-17978

[jira] [Created] (SPARK-17977) DataFrameReader and DataStreamReader should have an ancestor class

2016-10-17 Thread Amit Assudani (JIRA)
Amit Assudani created SPARK-17977: - Summary: DataFrameReader and DataStreamReader should have an ancestor class Key: SPARK-17977 URL: https://issues.apache.org/jira/browse/SPARK-17977 Project: Spark

[jira] [Created] (SPARK-17976) Global options to spark-submit should not be position-sensitive

2016-10-17 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-17976: Summary: Global options to spark-submit should not be position-sensitive Key: SPARK-17976 URL: https://issues.apache.org/jira/browse/SPARK-17976 Project: Spar

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583453#comment-15583453 ] Apache Spark commented on SPARK-13747: -- User 'zsxwing' has created a pull request fo

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-10-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583454#comment-15583454 ] Shixiong Zhu commented on SPARK-13747: -- [~chinwei] Could you test https://github.com

[jira] [Assigned] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13747: Assignee: Apache Spark (was: Shixiong Zhu) > Concurrent execution in SQL doesn't work wit

[jira] [Assigned] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-10-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13747: Assignee: Shixiong Zhu (was: Apache Spark) > Concurrent execution in SQL doesn't work wit

[jira] [Reopened] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-10-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-13747: -- Assignee: Shixiong Zhu (was: Andrew Or) There are other places need to be fixed. > Concurre

[jira] [Updated] (SPARK-17944) sbin/start-* scripts use of `hostname -f` fail with Solaris

2016-10-17 Thread Erik O'Shaughnessy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik O'Shaughnessy updated SPARK-17944: --- Component/s: Deploy > sbin/start-* scripts use of `hostname -f` fail with Solaris >

[jira] [Updated] (SPARK-15689) Data source API v2

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15689: Description: This ticket tracks progress in creating the v2 of data source API. This new API shoul

[jira] [Updated] (SPARK-15689) Data source API v2

2016-10-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15689: Description: This ticket tracks progress in creating the v2 of data source API. This new API shoul

[jira] [Commented] (SPARK-17911) Scheduler does not need messageScheduler for ResubmitFailedStages

2016-10-17 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583334#comment-15583334 ] Mark Hamstra commented on SPARK-17911: -- I think we're pretty much on the same page w

[jira] [Updated] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2016-10-17 Thread Jeff Stein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Stein updated SPARK-17975: --- Description: I'm able to reproduce the error consistently with a 2000 record text file with each rec

[jira] [Commented] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2016-10-17 Thread Jeff Stein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583284#comment-15583284 ] Jeff Stein commented on SPARK-17975: Another issue that seems to be related to EdgeRD

[jira] [Comment Edited] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2016-10-17 Thread Jeff Stein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583284#comment-15583284 ] Jeff Stein edited comment on SPARK-17975 at 10/17/16 8:04 PM: -

[jira] [Commented] (SPARK-17971) Unix timestamp handling in Spark SQL not allowing calculations on UTC times

2016-10-17 Thread Gabriele Del Prete (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583274#comment-15583274 ] Gabriele Del Prete commented on SPARK-17971: Already tried, and I could not m

[jira] [Commented] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2016-10-17 Thread Jeff Stein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583268#comment-15583268 ] Jeff Stein commented on SPARK-17975: This change resolves the issue for me: https://

[jira] [Created] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2016-10-17 Thread Jeff Stein (JIRA)
Jeff Stein created SPARK-17975: -- Summary: EMLDAOptimizer fails with ClassCastException on YARN Key: SPARK-17975 URL: https://issues.apache.org/jira/browse/SPARK-17975 Project: Spark Issue Type:

  1   2   >