[jira] [Created] (SPARK-15274) CSV default column names should be consistent

2016-05-11 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15274: - Summary: CSV default column names should be consistent Key: SPARK-15274 URL: https://issues.apache.org/jira/browse/SPARK-15274 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-15274) CSV default column names should be consistent

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15274: -- Assignee: Bill Chambers > CSV default column names should be consistent > -

[jira] [Updated] (SPARK-15264) Spark 2.0 CSV Reader: NPE on Blank Column Names

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15264: -- Summary: Spark 2.0 CSV Reader: NPE on Blank Column Names (was: Spark 2.0 CSV Reader: Error on Blank Co

[jira] [Updated] (SPARK-15264) Spark 2.0 CSV Reader: Error on Blank Column Names

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15264: -- Target Version/s: 2.0.0 > Spark 2.0 CSV Reader: Error on Blank Column Names > -

[jira] [Updated] (SPARK-15264) Spark 2.0 CSV Reader: Error on Blank Column Names

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15264: -- Assignee: Bill Chambers > Spark 2.0 CSV Reader: Error on Blank Column Names > -

[jira] [Created] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2016-05-11 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15276: - Summary: CREATE TABLE with LOCATION should imply EXTERNAL Key: SPARK-15276 URL: https://issues.apache.org/jira/browse/SPARK-15276 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Affects Version/s: (was: 1.6.1) 2.0.0 > CatalogTable should store sort order

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Assignee: Tejas Patil > CatalogTable should store sort ordering for sorted columns > --

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Priority: Major (was: Trivial) > CatalogTable should store sort ordering for sorted columns >

[jira] [Updated] (SPARK-15275) CatalogTable should store sort ordering for sorted columns

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15275: -- Summary: CatalogTable should store sort ordering for sorted columns (was: [SQL] CatalogTable should st

[jira] [Created] (SPARK-15279) Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.)

2016-05-11 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15279: - Summary: Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.) Key: SPARK-15279 URL: https://issues.apache.org/jira/browse/SPARK-15279 Project: Spark Issu

[jira] [Resolved] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15276. --- Resolution: Fixed Fix Version/s: 2.0.0 > CREATE TABLE with LOCATION should imply EXTERNAL > --

[jira] [Resolved] (SPARK-15274) CSV default column names should be consistent

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15274. --- Resolution: Fixed Fix Version/s: 2.0.0 > CSV default column names should be consistent > -

[jira] [Resolved] (SPARK-15264) Spark 2.0 CSV Reader: NPE on Blank Column Names

2016-05-11 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15264. --- Resolution: Fixed Fix Version/s: 2.0.0 > Spark 2.0 CSV Reader: NPE on Blank Column Names > ---

[jira] [Resolved] (SPARK-15289) SQL test compilation error from merge conflict

2016-05-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15289. --- Resolution: Fixed Fix Version/s: 2.0.0 > SQL test compilation error from merge conflict >

[jira] [Commented] (SPARK-15289) SQL test compilation error from merge conflict

2016-05-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281678#comment-15281678 ] Andrew Or commented on SPARK-15289: --- Done, thanks for the ping. > SQL test compilation

[jira] [Updated] (SPARK-14684) Verification of partition specs in SessionCatalog

2016-05-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14684: -- Description: When attempting to drop partitions of a table, if the user provides an unknown column, Hi

[jira] [Resolved] (SPARK-15277) Checking Partition Spec Existence Before Dropping

2016-05-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15277. --- Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.0.0 Target Version/s

[jira] [Resolved] (SPARK-14684) Verification of partition specs in SessionCatalog

2016-05-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14684. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Verification of par

[jira] [Created] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-16 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15357: - Summary: Cooperative spilling should check consumer memory mode Key: SPARK-15357 URL: https://issues.apache.org/jira/browse/SPARK-15357 Project: Spark Issue Type:

[jira] [Updated] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15357: -- Description: In TaskMemoryManager.java: {code} for (MemoryConsumer c: consumers) { if (c != consumer

[jira] [Resolved] (SPARK-15300) Can't remove a block if it's under evicting

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15300. --- Resolution: Fixed Fix Version/s: 2.0.0 > Can't remove a block if it's under evicting > ---

[jira] [Resolved] (SPARK-15387) SessionCatalog in SimpleAnalyzer does not need to make database directory.

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15387. --- Resolution: Fixed Assignee: Kousuke Saruta Fix Version/s: 2.0.0 Target Ve

[jira] [Resolved] (SPARK-15317) JobProgressListener takes a huge amount of memory with iterative DataFrame program in local, standalone

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15317. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 Target Vers

[jira] [Resolved] (SPARK-15392) The default value of size estimation is not good

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15392. --- Resolution: Fixed Target Version/s: 2.0.0 > The default value of size estimation is not goo

[jira] [Assigned] (SPARK-15417) Failed to enable HiveSupport in PySpark

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-15417: - Assignee: Andrew Or > Failed to enable HiveSupport in PySpark >

[jira] [Commented] (SPARK-15417) Failed to enable HiveSupport in PySpark

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292159#comment-15292159 ] Andrew Or commented on SPARK-15417: --- Good catch, I have a patch to fix this. > Failed

[jira] [Created] (SPARK-15421) Table and Database property values need to be validated

2016-05-19 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15421: - Summary: Table and Database property values need to be validated Key: SPARK-15421 URL: https://issues.apache.org/jira/browse/SPARK-15421 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-15421) Table and Database property values need to be validated

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15421. --- Resolution: Fixed Fix Version/s: 2.0.0 > Table and Database property values need to be validat

[jira] [Resolved] (SPARK-15417) Failed to enable HiveSupport in PySpark

2016-05-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15417. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Failed to enable Hi

[jira] [Updated] (SPARK-15417) Failed to enable hive support in PySpark shell

2016-05-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15417: -- Summary: Failed to enable hive support in PySpark shell (was: Failed to enable HiveSupport in PySpark)

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293899#comment-15293899 ] Andrew Or commented on SPARK-15345: --- The python part should be resolved by SPARK-15417,

[jira] [Created] (SPARK-15450) Clean up SparkSession builder for python

2016-05-20 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15450: - Summary: Clean up SparkSession builder for python Key: SPARK-15450 URL: https://issues.apache.org/jira/browse/SPARK-15450 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-15456) PySpark Shell fails to create SparkContext if HiveConf not found

2016-05-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15456. --- Resolution: Fixed Assignee: Bryan Cutler Fix Version/s: 2.0.0 Target Vers

[jira] [Commented] (SPARK-15477) HiveContext is private[hive] and not accessible to users.

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296763#comment-15296763 ] Andrew Or commented on SPARK-15477: --- What part of the code makes you think it's private

[jira] [Resolved] (SPARK-15477) HiveContext is private[hive] and not accessible to users.

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15477. --- Resolution: Not A Bug > HiveContext is private[hive] and not accessible to users. >

[jira] [Updated] (SPARK-15397) 'locate' UDF got different result with boundary value case compared to Hive engine

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15397: -- Assignee: Adrian Wang > 'locate' UDF got different result with boundary value case compared to Hive >

[jira] [Updated] (SPARK-15397) 'locate' UDF got different result with boundary value case compared to Hive engine

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15397: -- Summary: 'locate' UDF got different result with boundary value case compared to Hive engine (was: [Spa

[jira] [Resolved] (SPARK-15279) Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.)

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15279. --- Resolution: Fixed Fix Version/s: 2.0.0 > Disallow ROW FORMAT and STORED AS (parquet | orc | av

[jira] [Resolved] (SPARK-15488) Possible Accumulator bug causing OneVsRestSuite to be flaky

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15488. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.0.0 Target V

[jira] [Resolved] (SPARK-15311) Disallow DML on Non-temporary Tables when Using In-Memory Catalog

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15311. --- Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.0.0 Target Version/s

[jira] [Resolved] (SPARK-15464) Replace SQLContext and SparkContext with SparkSession using builder pattern in python testsuites

2016-05-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15464. --- Resolution: Fixed Assignee: Weichen Xu Fix Version/s: 2.0.0 Target Versio

[jira] [Commented] (SPARK-15450) Clean up SparkSession builder for python

2016-05-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297800#comment-15297800 ] Andrew Or commented on SPARK-15450: --- Actually I already have some ideas for this one.

[jira] [Updated] (SPARK-15388) spark sql "CREATE FUNCTION" throws exception with hive 1.2.1

2016-05-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15388: -- Assignee: Yang Wang > spark sql "CREATE FUNCTION" throws exception with hive 1.2.1 > --

[jira] [Resolved] (SPARK-15388) spark sql "CREATE FUNCTION" throws exception with hive 1.2.1

2016-05-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15388. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > spark sql "CREATE F

[jira] [Created] (SPARK-15511) Dropping data source table succeeds but throws exception

2016-05-24 Thread Andrew Or (JIRA)
Andrew Or created SPARK-15511: - Summary: Dropping data source table succeeds but throws exception Key: SPARK-15511 URL: https://issues.apache.org/jira/browse/SPARK-15511 Project: Spark Issue Type

[jira] [Updated] (SPARK-15511) Dropping data source table succeeds but throws exception

2016-05-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15511: -- Description: If the catalog is backed by Hive: {code} scala> sql("CREATE TABLE boxes (width INT, lengt

[jira] [Resolved] (SPARK-15511) Dropping data source table succeeds but throws exception

2016-05-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-15511. --- Resolution: Not A Problem Assignee: Andrew Or If you run into this issue again, just delete $SP

[jira] [Created] (SPARK-19522) --executor-memory flag doesn't work in local-cluster mode

2017-02-08 Thread Andrew Or (JIRA)
Andrew Or created SPARK-19522: - Summary: --executor-memory flag doesn't work in local-cluster mode Key: SPARK-19522 URL: https://issues.apache.org/jira/browse/SPARK-19522 Project: Spark Issue Typ

[jira] [Updated] (SPARK-19522) --executor-memory flag doesn't work in local-cluster mode

2017-02-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-19522: -- Description: {code} bin/spark-shell --master local-cluster[2,1,2048] {code} doesn't do what you think

[jira] [Commented] (SPARK-19326) Speculated task attempts do not get launched in few scenarios

2017-02-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869020#comment-15869020 ] Andrew Or commented on SPARK-19326: --- Sorry for slipping on this. When I was implementin

[jira] [Commented] (SPARK-19326) Speculated task attempts do not get launched in few scenarios

2017-02-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15869244#comment-15869244 ] Andrew Or commented on SPARK-19326: --- I would say it's a bad idea to make ExecutorAlloca

[jira] [Commented] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

2016-09-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490654#comment-15490654 ] Andrew Or commented on SPARK-15917: --- Hi Jonathan, I think your proposal makes sense. T

[jira] [Commented] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

2016-09-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490663#comment-15490663 ] Andrew Or commented on SPARK-15917: --- By the way is there a pull request? > Define the

[jira] [Resolved] (SPARK-17438) Master UI should show the correct core limit when `ApplicationInfo.executorLimit` is set

2016-09-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17438. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Master UI should show the co

[jira] [Resolved] (SPARK-17623) Failed tasks end reason is always a TaskFailedReason, types should reflect this

2016-09-21 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17623. --- Resolution: Fixed Fix Version/s: 2.1.0 > Failed tasks end reason is always a TaskFailedReason,

[jira] [Resolved] (SPARK-17648) TaskSchedulerImpl.resourceOffers should take an IndexedSeq, not a Seq

2016-09-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17648. --- Resolution: Fixed Fix Version/s: 2.1.0 Target Version/s: 2.1.0 > TaskSchedulerImpl.r

[jira] [Resolved] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

2016-09-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17672. --- Resolution: Fixed Fix Version/s: 2.0.1 Target Version/s: 2.0.1 > Spark 2.0 history s

[jira] [Updated] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

2016-09-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-17672: -- Assignee: Gang Wu > Spark 2.0 history server web Ui takes too long for a single application > -

[jira] [Resolved] (SPARK-17715) Log INFO per task launch creates a large driver log

2016-09-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17715. --- Resolution: Fixed Assignee: Brian Cho Fix Version/s: 2.1.0 Target Version

[jira] [Resolved] (SPARK-11272) Support importing and exporting event logs from HistoryServer web portal

2016-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-11272. --- Resolution: Fixed Assignee: Alex Bozarth Fix Version/s: 2.1.0 Target Vers

[jira] [Resolved] (SPARK-17899) add a debug mode to keep raw table properties in HiveExternalCatalog

2016-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17899. --- Resolution: Fixed Fix Version/s: 2.1.0 > add a debug mode to keep raw table properties in Hive

[jira] [Resolved] (SPARK-17686) Propose to print Scala version in "spark-submit --version" command

2016-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17686. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.1.0 Target Versi

[jira] [Updated] (SPARK-13344) Tests have many "accumulator not found" exceptions

2016-02-17 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13344: -- Description: This is because SparkFunSuite clears all accumulators after every single test. This suite

[jira] [Updated] (SPARK-13344) Tests have many "accumulator not found" exceptions

2016-02-17 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13344: -- Summary: Tests have many "accumulator not found" exceptions (was: SaveLoadSuite has many accumulator e

[jira] [Resolved] (SPARK-13371) TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly.

2016-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13371. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 Target Version/s:

[jira] [Updated] (SPARK-13371) TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly.

2016-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13371: -- Assignee: Sean Owen > TaskSetManager.dequeueSpeculativeTask compares Option[String] and String > direc

[jira] [Issue Comment Deleted] (SPARK-13054) Always post TaskEnd event for tasks in cancelled stages

2016-02-18 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13054: -- Comment: was deleted (was: User 'andrewor14' has created a pull request for this issue: https://github.

[jira] [Updated] (SPARK-12864) Fetch failure from AM restart

2016-02-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12864: -- Summary: Fetch failure from AM restart (was: initialize executorIdCounter after ApplicationMaster kil

[jira] [Updated] (SPARK-13329) Considering output for statistics of logical plan

2016-02-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13329: -- Summary: Considering output for statistics of logical plan (was: Considering output for statistics of

[jira] [Resolved] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

2016-02-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10749. --- Resolution: Fixed Fix Version/s: 2.0.0 > Support multiple roles with Spark Mesos dispatcher >

[jira] [Updated] (SPARK-12583) spark shuffle fails with mesos after 2mins

2016-02-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12583: -- Target Version/s: 2.0.0 (was: 1.6.1) > spark shuffle fails with mesos after 2mins > --

[jira] [Updated] (SPARK-12757) Use reference counting to prevent blocks from being evicted during reads

2016-02-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12757: -- Target Version/s: 2.0.0 > Use reference counting to prevent blocks from being evicted during reads > --

[jira] [Updated] (SPARK-13298) DAG visualization does not render correctly for jobs

2016-02-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13298: -- Target Version/s: 1.6.1, 2.0.0 > DAG visualization does not render correctly for jobs > ---

[jira] [Resolved] (SPARK-13298) DAG visualization does not render correctly for jobs

2016-02-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13298. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > DAG visualization does not r

[jira] [Resolved] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2016-02-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-7729. -- Resolution: Fixed Assignee: Lianhui Wang Fix Version/s: 2.0.0 Target Version

[jira] [Resolved] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

2016-02-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12009. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Avoid re-allocate yarn conta

[jira] [Updated] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

2016-02-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12009: -- Assignee: SuYan > Avoid re-allocate yarn container while driver want to stop all Executors > --

[jira] [Updated] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

2016-02-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12009: -- Fix Version/s: (was: 1.6.1) > Avoid re-allocate yarn container while driver want to stop all Execut

[jira] [Resolved] (SPARK-13501) Remove use of Guava Stopwatch class

2016-02-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13501. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove use of Guava Stopwatch class > ---

[jira] [Resolved] (SPARK-13387) Add support for SPARK_DAEMON_JAVA_OPTS with MesosClusterDispatcher.

2016-02-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13387. --- Resolution: Fixed Assignee: Timothy Chen Fix Version/s: 2.0.0 Target Vers

[jira] [Resolved] (SPARK-12757) Use reference counting to prevent blocks from being evicted during reads

2016-02-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12757. --- Resolution: Fixed Fix Version/s: 2.0.0 > Use reference counting to prevent blocks from being e

[jira] [Created] (SPARK-13524) Remove BroadcastManager

2016-02-26 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13524: - Summary: Remove BroadcastManager Key: SPARK-13524 URL: https://issues.apache.org/jira/browse/SPARK-13524 Project: Spark Issue Type: Improvement Component

[jira] [Resolved] (SPARK-13519) Driver should tell Executor to stop itself when cleaning executor's state

2016-02-26 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13519. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 Target Vers

[jira] [Created] (SPARK-13526) Refactor: Move SQLContext/HiveContext per-session state to separate class

2016-02-26 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13526: - Summary: Refactor: Move SQLContext/HiveContext per-session state to separate class Key: SPARK-13526 URL: https://issues.apache.org/jira/browse/SPARK-13526 Project: Spark

[jira] [Resolved] (SPARK-13522) Executor should kill itself when it's unable to heartbeat to the driver more than N times

2016-02-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13522. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 Target Vers

[jira] [Created] (SPARK-13571) Track current database in SQL/HiveContext

2016-02-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13571: - Summary: Track current database in SQL/HiveContext Key: SPARK-13571 URL: https://issues.apache.org/jira/browse/SPARK-13571 Project: Spark Issue Type: Improvement

[jira] [Issue Comment Deleted] (SPARK-13571) Track current database in SQL/HiveContext

2016-02-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13571: -- Comment: was deleted (was: User 'andrewor14' has created a pull request for this issue: https://github.

[jira] [Resolved] (SPARK-12817) Remove CacheManager and replace it with new BlockManager.getOrElseUpdate method

2016-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12817. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove CacheManager and replace it with new BlockMana

[jira] [Created] (SPARK-13632) Create new o.a.s.sql.execution.commands package

2016-03-02 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13632: - Summary: Create new o.a.s.sql.execution.commands package Key: SPARK-13632 URL: https://issues.apache.org/jira/browse/SPARK-13632 Project: Spark Issue Type: Sub-tas

[jira] [Created] (SPARK-13633) Create new o.a.s.sql.catalyst.parser package

2016-03-02 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13633: - Summary: Create new o.a.s.sql.catalyst.parser package Key: SPARK-13633 URL: https://issues.apache.org/jira/browse/SPARK-13633 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-13633) Move parser classes to o.a.s.sql.catalyst.parser package

2016-03-02 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13633: -- Summary: Move parser classes to o.a.s.sql.catalyst.parser package (was: Create new o.a.s.sql.catalyst.

[jira] [Resolved] (SPARK-13633) Move parser classes to o.a.s.sql.catalyst.parser package

2016-03-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13633. --- Resolution: Fixed Fix Version/s: 2.0.0 > Move parser classes to o.a.s.sql.catalyst.parser pack

[jira] [Created] (SPARK-13685) Rename catalog.Catalog to ExternalCatalog

2016-03-04 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13685: - Summary: Rename catalog.Catalog to ExternalCatalog Key: SPARK-13685 URL: https://issues.apache.org/jira/browse/SPARK-13685 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13689) Move some methods in CatalystQl to a util object

2016-03-04 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13689: - Summary: Move some methods in CatalystQl to a util object Key: SPARK-13689 URL: https://issues.apache.org/jira/browse/SPARK-13689 Project: Spark Issue Type: Sub-ta

[jira] [Resolved] (SPARK-13689) Move some methods in CatalystQl to a util object

2016-03-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13689. --- Resolution: Fixed Fix Version/s: 2.0.0 > Move some methods in CatalystQl to a util object > --

[jira] [Resolved] (SPARK-13659) Remove returnValues from BlockStore APIs

2016-03-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13659. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove returnValues from BlockStore APIs > --

[jira] [Commented] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

2016-03-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185511#comment-15185511 ] Andrew Or commented on SPARK-11857: --- [~dragos][~tnachen] any thoughts on at least depre

[jira] [Resolved] (SPARK-13778) Master's ApplicationPage displays wrong application executor state when a worker is lost

2016-03-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13778. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Master's Applicatio

[jira] [Resolved] (SPARK-13775) history server sort by completed time by default

2016-03-09 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13775. --- Resolution: Fixed Assignee: Zhuo Liu Fix Version/s: 2.0.0 Target Version/

<    1   2   3   4   5   6   7   8   9   10   >