[jira] [Resolved] (SPARK-8393) JavaStreamingContext#awaitTermination() throws non-declared InterruptedException

[ https://issues.apache.org/jira/browse/SPARK-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8393. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12418

[jira] [Updated] (SPARK-8393) JavaStreamingContext#awaitTermination() throws non-declared InterruptedException

[ https://issues.apache.org/jira/browse/SPARK-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8393: - Priority: Minor (was: Trivial) Issue Type: Improvement (was: Bug) >

[jira] [Assigned] (SPARK-1989) Exit executors faster if they get into a cycle of heavy GC

[ https://issues.apache.org/jira/browse/SPARK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1989: --- Assignee: Apache Spark > Exit executors faster if they get into a cycle of heavy GC >

[jira] [Commented] (SPARK-1989) Exit executors faster if they get into a cycle of heavy GC

[ https://issues.apache.org/jira/browse/SPARK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251613#comment-15251613 ] Apache Spark commented on SPARK-1989: - User 'devaraj-kavali' has created a pull request for this

[jira] [Assigned] (SPARK-1989) Exit executors faster if they get into a cycle of heavy GC

[ https://issues.apache.org/jira/browse/SPARK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1989: --- Assignee: (was: Apache Spark) > Exit executors faster if they get into a cycle of heavy

[jira] [Issue Comment Deleted] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuYan updated SPARK-14750: -- Comment: was deleted (was: # but there's not a reason to expect they also remain wherever they were logged

[jira] [Commented] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251608#comment-15251608 ] SuYan commented on SPARK-14750: --- # but there's not a reason to expect they also remain wherever they were

[jira] [Commented] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251607#comment-15251607 ] SuYan commented on SPARK-14750: --- # but there's not a reason to expect they also remain wherever they were

[jira] [Assigned] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14750: Assignee: (was: Apache Spark) > Make historyServer refer application log in hdfs >

[jira] [Commented] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251592#comment-15251592 ] Apache Spark commented on SPARK-14750: -- User 'suyanNone' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14750: Assignee: Apache Spark > Make historyServer refer application log in hdfs >

[jira] [Commented] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251578#comment-15251578 ] Sean Owen commented on SPARK-14750: --- Isn't that the point of the history server, to (in a different

[jira] [Comment Edited] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251558#comment-15251558 ] SuYan edited comment on SPARK-14750 at 4/21/16 8:37 AM: historyServer for spark

[jira] [Assigned] (SPARK-14731) Revert SPARK-12130 to make 2.0 shuffle service compatible with 1.x

[ https://issues.apache.org/jira/browse/SPARK-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14731: Assignee: Apache Spark > Revert SPARK-12130 to make 2.0 shuffle service compatible with

[jira] [Commented] (SPARK-12130) Replace shuffleManagerClass with shortShuffleMgrNames in ExternalShuffleBlockResolver

[ https://issues.apache.org/jira/browse/SPARK-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251564#comment-15251564 ] Apache Spark commented on SPARK-12130: -- User 'lianhuiwang' has created a pull request for this

[jira] [Commented] (SPARK-14731) Revert SPARK-12130 to make 2.0 shuffle service compatible with 1.x

[ https://issues.apache.org/jira/browse/SPARK-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251563#comment-15251563 ] Apache Spark commented on SPARK-14731: -- User 'lianhuiwang' has created a pull request for this

[jira] [Assigned] (SPARK-14731) Revert SPARK-12130 to make 2.0 shuffle service compatible with 1.x

[ https://issues.apache.org/jira/browse/SPARK-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14731: Assignee: (was: Apache Spark) > Revert SPARK-12130 to make 2.0 shuffle service

[jira] [Commented] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251558#comment-15251558 ] SuYan commented on SPARK-14750: --- historyServer for spark on yarn, the logUrl was something like:

[jira] [Comment Edited] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251201#comment-15251201 ] Zhongshuai Pei edited comment on SPARK-4105 at 4/21/16 8:08 AM: I find

[jira] [Resolved] (SPARK-14753) remove internal flag in Accumulable

[ https://issues.apache.org/jira/browse/SPARK-14753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14753. - Resolution: Fixed Fix Version/s: 2.0.0 > remove internal flag in Accumulable >

[jira] [Assigned] (SPARK-14801) Move MetastoreRelation to its own file

[ https://issues.apache.org/jira/browse/SPARK-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14801: Assignee: Apache Spark (was: Reynold Xin) > Move MetastoreRelation to its own file >

[jira] [Commented] (SPARK-14801) Move MetastoreRelation to its own file

[ https://issues.apache.org/jira/browse/SPARK-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251519#comment-15251519 ] Apache Spark commented on SPARK-14801: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14801) Move MetastoreRelation to its own file

[ https://issues.apache.org/jira/browse/SPARK-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14801: Assignee: Reynold Xin (was: Apache Spark) > Move MetastoreRelation to its own file >

[jira] [Created] (SPARK-14801) Move MetastoreRelation to its own file

Reynold Xin created SPARK-14801: --- Summary: Move MetastoreRelation to its own file Key: SPARK-14801 URL: https://issues.apache.org/jira/browse/SPARK-14801 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-14800) Dealing with null as a value in options for each internal data source

[ https://issues.apache.org/jira/browse/SPARK-14800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251509#comment-15251509 ] Hyukjin Kwon commented on SPARK-14800: -- Let me work on this if

[jira] [Created] (SPARK-14800) Dealing with null as a value in options for each internal data source

Hyukjin Kwon created SPARK-14800: Summary: Dealing with null as a value in options for each internal data source Key: SPARK-14800 URL: https://issues.apache.org/jira/browse/SPARK-14800 Project: Spark

[jira] [Assigned] (SPARK-14799) Remove MetastoreRelation dependency from AnalyzeTable

[ https://issues.apache.org/jira/browse/SPARK-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14799: Assignee: Apache Spark > Remove MetastoreRelation dependency from AnalyzeTable >

[jira] [Assigned] (SPARK-14799) Remove MetastoreRelation dependency from AnalyzeTable

[ https://issues.apache.org/jira/browse/SPARK-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14799: Assignee: (was: Apache Spark) > Remove MetastoreRelation dependency from AnalyzeTable

[jira] [Commented] (SPARK-14799) Remove MetastoreRelation dependency from AnalyzeTable

[ https://issues.apache.org/jira/browse/SPARK-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251508#comment-15251508 ] Apache Spark commented on SPARK-14799: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-14594) Improve error messages for RDD API

[ https://issues.apache.org/jira/browse/SPARK-14594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251507#comment-15251507 ] Marco Gaido commented on SPARK-14594: - The code is quite simple, what I can't give you is the data

[jira] [Created] (SPARK-14799) Remove MetastoreRelation dependency from AnalyzeTable

Reynold Xin created SPARK-14799: --- Summary: Remove MetastoreRelation dependency from AnalyzeTable Key: SPARK-14799 URL: https://issues.apache.org/jira/browse/SPARK-14799 Project: Spark Issue

[jira] [Resolved] (SPARK-14794) Don't pass analyze command into Hive

[ https://issues.apache.org/jira/browse/SPARK-14794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14794. - Resolution: Fixed Fix Version/s: 2.0.0 > Don't pass analyze command into Hive >

[jira] [Resolved] (SPARK-14792) Move as many parsing rules as possible into SQL parser

[ https://issues.apache.org/jira/browse/SPARK-14792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14792. - Resolution: Fixed Fix Version/s: 2.0.0 > Move as many parsing rules as possible into SQL

[jira] [Commented] (SPARK-13289) Word2Vec generate infinite distances when numIterations>5

[ https://issues.apache.org/jira/browse/SPARK-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251415#comment-15251415 ] Junyang Shen commented on SPARK-13289: -- The latest PR should have fixed the issue, and is ready to

[jira] [Commented] (SPARK-14790) Scalastyle should run on compile in sbt

[ https://issues.apache.org/jira/browse/SPARK-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251402#comment-15251402 ] Sean Owen commented on SPARK-14790: --- It's not standard in Maven to run verification checks in a compile

[jira] [Commented] (SPARK-14750) Make historyServer refer application log in hdfs

[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251401#comment-15251401 ] Sean Owen commented on SPARK-14750: --- Can you give a specific example of what function (in the UI?) you

[jira] [Commented] (SPARK-14798) Move native command and script transformation parsing into SparkSqlAstBuilder

[ https://issues.apache.org/jira/browse/SPARK-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251397#comment-15251397 ] Apache Spark commented on SPARK-14798: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14798) Move native command and script transformation parsing into SparkSqlAstBuilder

[ https://issues.apache.org/jira/browse/SPARK-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14798: Assignee: Apache Spark (was: Reynold Xin) > Move native command and script

[jira] [Assigned] (SPARK-14798) Move native command and script transformation parsing into SparkSqlAstBuilder

[ https://issues.apache.org/jira/browse/SPARK-14798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14798: Assignee: Reynold Xin (was: Apache Spark) > Move native command and script

[jira] [Created] (SPARK-14798) Move native command and script transformation parsing into SparkSqlAstBuilder

Reynold Xin created SPARK-14798: --- Summary: Move native command and script transformation parsing into SparkSqlAstBuilder Key: SPARK-14798 URL: https://issues.apache.org/jira/browse/SPARK-14798 Project:

[jira] [Commented] (SPARK-14760) Feature transformers should always invoke transformSchema in transform or fit

[ https://issues.apache.org/jira/browse/SPARK-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251391#comment-15251391 ] Nick Pentreath commented on SPARK-14760: In general, given the name {{transformSchema}}, one

[jira] [Commented] (SPARK-14797) Spark SQL should not hardcode dependency on spark-sketch_2.11

[ https://issues.apache.org/jira/browse/SPARK-14797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251389#comment-15251389 ] Apache Spark commented on SPARK-14797: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14797) Spark SQL should not hardcode dependency on spark-sketch_2.11

[ https://issues.apache.org/jira/browse/SPARK-14797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14797: Assignee: Apache Spark (was: Josh Rosen) > Spark SQL should not hardcode dependency on

[jira] [Assigned] (SPARK-14797) Spark SQL should not hardcode dependency on spark-sketch_2.11

[ https://issues.apache.org/jira/browse/SPARK-14797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14797: Assignee: Josh Rosen (was: Apache Spark) > Spark SQL should not hardcode dependency on

[jira] [Created] (SPARK-14797) Spark SQL should not hardcode dependency on spark-sketch_2.11

Josh Rosen created SPARK-14797: -- Summary: Spark SQL should not hardcode dependency on spark-sketch_2.11 Key: SPARK-14797 URL: https://issues.apache.org/jira/browse/SPARK-14797 Project: Spark

[jira] [Commented] (SPARK-14760) Feature transformers should always invoke transformSchema in transform or fit

[ https://issues.apache.org/jira/browse/SPARK-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251372#comment-15251372 ] Nick Pentreath commented on SPARK-14760: It seems it is there for validation now, but then the

[jira] [Assigned] (SPARK-14796) Add spark.sql.optimizer.minSetSize config option

[ https://issues.apache.org/jira/browse/SPARK-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14796: Assignee: (was: Apache Spark) > Add spark.sql.optimizer.minSetSize config option >

[jira] [Assigned] (SPARK-14796) Add spark.sql.optimizer.minSetSize config option

[ https://issues.apache.org/jira/browse/SPARK-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14796: Assignee: Apache Spark > Add spark.sql.optimizer.minSetSize config option >

[jira] [Resolved] (SPARK-14658) when executor lost DagScheduer may submit one stage twice even if the first running taskset for this stage is not finished

[ https://issues.apache.org/jira/browse/SPARK-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14658. --- Resolution: Duplicate This looks like a duplicate; at least the other JIRA already had a PR and some

[jira] [Commented] (SPARK-14796) Add spark.sql.optimizer.minSetSize config option

[ https://issues.apache.org/jira/browse/SPARK-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251368#comment-15251368 ] Apache Spark commented on SPARK-14796: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-14796) Add spark.sql.optimizer.minSetSize config option

Dongjoon Hyun created SPARK-14796: - Summary: Add spark.sql.optimizer.minSetSize config option Key: SPARK-14796 URL: https://issues.apache.org/jira/browse/SPARK-14796 Project: Spark Issue

[jira] [Commented] (SPARK-14693) Spark Streaming Context Hangs on Start

[ https://issues.apache.org/jira/browse/SPARK-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251354#comment-15251354 ] Sean Owen commented on SPARK-14693: --- The question is what the state of the application is, since all

[jira] [Commented] (SPARK-14745) CEP support in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251353#comment-15251353 ] Mario Briggs commented on SPARK-14745: -- Hi Sean, thanks for 'fixing' the target. To answer you

[jira] [Comment Edited] (SPARK-14525) DataFrameWriter's save method should delegate to jdbc for jdbc datasource

[ https://issues.apache.org/jira/browse/SPARK-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251339#comment-15251339 ] Hyukjin Kwon edited comment on SPARK-14525 at 4/21/16 6:17 AM: --- I think it

[jira] [Assigned] (SPARK-14795) Remove the use of Hive's variable substitution

[ https://issues.apache.org/jira/browse/SPARK-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14795: Assignee: Apache Spark (was: Reynold Xin) > Remove the use of Hive's variable

[jira] [Commented] (SPARK-14795) Remove the use of Hive's variable substitution

[ https://issues.apache.org/jira/browse/SPARK-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251345#comment-15251345 ] Apache Spark commented on SPARK-14795: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14795) Remove the use of Hive's variable substitution

[ https://issues.apache.org/jira/browse/SPARK-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14795: Assignee: Reynold Xin (was: Apache Spark) > Remove the use of Hive's variable

[jira] [Created] (SPARK-14795) Remove the use of Hive's variable substitution

Reynold Xin created SPARK-14795: --- Summary: Remove the use of Hive's variable substitution Key: SPARK-14795 URL: https://issues.apache.org/jira/browse/SPARK-14795 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14525) DataFrameWriter's save method should delegate to jdbc for jdbc datasource

[ https://issues.apache.org/jira/browse/SPARK-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251339#comment-15251339 ] Hyukjin Kwon commented on SPARK-14525: -- I think it might be great if we can cc who knows this part

[jira] [Assigned] (SPARK-14794) Don't pass analyze command into Hive

[ https://issues.apache.org/jira/browse/SPARK-14794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14794: Assignee: Apache Spark (was: Reynold Xin) > Don't pass analyze command into Hive >

[jira] [Assigned] (SPARK-14794) Don't pass analyze command into Hive

[ https://issues.apache.org/jira/browse/SPARK-14794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14794: Assignee: Reynold Xin (was: Apache Spark) > Don't pass analyze command into Hive >

[jira] [Commented] (SPARK-14794) Don't pass analyze command into Hive

[ https://issues.apache.org/jira/browse/SPARK-14794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251329#comment-15251329 ] Apache Spark commented on SPARK-14794: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-14794) Don't pass analyze command into Hive

Reynold Xin created SPARK-14794: --- Summary: Don't pass analyze command into Hive Key: SPARK-14794 URL: https://issues.apache.org/jira/browse/SPARK-14794 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-14760) Feature transformers should always invoke transformSchema in transform or fit

[ https://issues.apache.org/jira/browse/SPARK-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251324#comment-15251324 ] yuhao yang commented on SPARK-14760: [~holdenkarau] also shared some thoughts in the PR. In my

<    1   2   3   4