[jira] [Commented] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048731#comment-16048731 ] Apache Spark commented on SPARK-21090: -- User '10110346' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21090: Assignee: (was: Apache Spark) > Optimize the unified memory manager code >

[jira] [Assigned] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21090: Assignee: Apache Spark > Optimize the unified memory manager code >

[jira] [Updated] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-21090: Description: 1.In *acquireStorageMemory*, when the MemoryMode is OFF_HEAP ,the *maxMemory* should be

[jira] [Created] (SPARK-21090) Optimize the unified memory manager code

liuxian created SPARK-21090: --- Summary: Optimize the unified memory manager code Key: SPARK-21090 URL: https://issues.apache.org/jira/browse/SPARK-21090 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048723#comment-16048723 ] yuhao yang commented on SPARK-21087: I'd like to work on this if my

[jira] [Comment Edited] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048647#comment-16048647 ] yuhao yang edited comment on SPARK-21086 at 6/14/17 5:22 AM: - Sounds good.

[jira] [Commented] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048720#comment-16048720 ] Apache Spark commented on SPARK-21085: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21085: Assignee: Apache Spark (was: Xiao Li) > Failed to read the partitioned table created by

[jira] [Assigned] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21085: Assignee: Xiao Li (was: Apache Spark) > Failed to read the partitioned table created by

[jira] [Comment Edited] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048647#comment-16048647 ] yuhao yang edited comment on SPARK-21086 at 6/14/17 5:12 AM: - Sounds good.

[jira] [Assigned] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21089: Assignee: Apache Spark (was: Xiao Li) > Table properties are not shown in DESC

[jira] [Assigned] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21089: Assignee: Xiao Li (was: Apache Spark) > Table properties are not shown in DESC

[jira] [Commented] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048717#comment-16048717 ] Apache Spark commented on SPARK-21089: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

Xiao Li created SPARK-21089: --- Summary: Table properties are not shown in DESC EXTENDED/FORMATTED Key: SPARK-21089 URL: https://issues.apache.org/jira/browse/SPARK-21089 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21089: Target Version/s: 2.2.0 > Table properties are not shown in DESC EXTENDED/FORMATTED >

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048698#comment-16048698 ] yuhao yang commented on SPARK-20988: Eh.. I was trying to add the squared_hinge loss to LinearSVC and

[jira] [Assigned] (SPARK-19753) Remove all shuffle files on a host in case of slave lost of fetch failure

[ https://issues.apache.org/jira/browse/SPARK-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19753: --- Assignee: Sital Kedia > Remove all shuffle files on a host in case of slave lost of fetch

[jira] [Resolved] (SPARK-19753) Remove all shuffle files on a host in case of slave lost of fetch failure

[ https://issues.apache.org/jira/browse/SPARK-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19753. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18150

[jira] [Resolved] (SPARK-20348) Support squared hinge loss (L2 loss) for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang resolved SPARK-20348. Resolution: Duplicate Combine it with SPARK-20602 and resolve this as duplicate. > Support

[jira] [Commented] (SPARK-20602) Adding LBFGS optimizer and Squared_hinge loss for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048663#comment-16048663 ] yuhao yang commented on SPARK-20602: Combining this with SPARK-20348. Support squared hinge loss (L2

[jira] [Updated] (SPARK-20602) Adding LBFGS optimizer and Squared_hinge loss for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-20602: --- Summary: Adding LBFGS optimizer and Squared_hinge loss for LinearSVC (was: Adding LBFGS as

[jira] [Comment Edited] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048653#comment-16048653 ] DjvuLee edited comment on SPARK-21082 at 6/14/17 3:15 AM: -- [~srowen] This

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048654#comment-16048654 ] DjvuLee commented on SPARK-21082: - My idea is try to consider the BlockManger information when scheduling

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048653#comment-16048653 ] DjvuLee commented on SPARK-21082: - [~srowen] This situation occurred when the partition number is larger

[jira] [Commented] (SPARK-21078) JobHistory applications synchronized is invalid

[ https://issues.apache.org/jira/browse/SPARK-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048650#comment-16048650 ] fangfengbin commented on SPARK-21078: - [~sowen], this actually cause a problem in practice, when

[jira] [Commented] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048647#comment-16048647 ] yuhao yang commented on SPARK-21086: Sounds good. About the default path for saving different models,

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048612#comment-16048612 ] 吴志龙 commented on SPARK-21075: - ok,thanks > spark 2.2 mvn [error] javac: invalid source release: 1.8 >

[jira] [Assigned] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20986: --- Assignee: Lianhui Wang > Reset table's statistics after PruneFileSourcePartitions rule. >

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048593#comment-16048593 ] Vincent commented on SPARK-20988: - opps. I have finished the conversion part, but there are still other

[jira] [Resolved] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20986. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18205

[jira] [Created] (SPARK-21088) CrossValidator, TrainValidationSplit should preserve all models after fitting: Python

Joseph K. Bradley created SPARK-21088: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting: Python Key: SPARK-21088 URL: https://issues.apache.org/jira/browse/SPARK-21088

[jira] [Updated] (SPARK-21088) CrossValidator, TrainValidationSplit should preserve all models after fitting: Python

[ https://issues.apache.org/jira/browse/SPARK-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21088: -- Component/s: PySpark > CrossValidator, TrainValidationSplit should preserve all models

[jira] [Created] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

Joseph K. Bradley created SPARK-21087: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala Key: SPARK-21087 URL: https://issues.apache.org/jira/browse/SPARK-21087

[jira] [Created] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

Joseph K. Bradley created SPARK-21086: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting Key: SPARK-21086 URL: https://issues.apache.org/jira/browse/SPARK-21086

[jira] [Assigned] (SPARK-12552) Recovered driver's resource is not counted in the Master

[ https://issues.apache.org/jira/browse/SPARK-12552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-12552: --- Assignee: Saisai Shao (was: Apache Spark) > Recovered driver's resource is not counted in

[jira] [Resolved] (SPARK-12552) Recovered driver's resource is not counted in the Master

[ https://issues.apache.org/jira/browse/SPARK-12552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-12552. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Updated] (SPARK-20979) Add a rate source to generate values for tests and benchmark

[ https://issues.apache.org/jira/browse/SPARK-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20979: - Fix Version/s: 2.2.0 > Add a rate source to generate values for tests and benchmark >

[jira] [Updated] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21085: Summary: Failed to read the partitioned table created by Spark 2.1 (was: Failed to read the table created

[jira] [Updated] (SPARK-21085) Failed to read the table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21085: Description: Spark 2.2 is unable to read the partitioned table created by Spark 2.1 when the table schema

[jira] [Created] (SPARK-21085) Failed to read the table created by Spark 2.1

Xiao Li created SPARK-21085: --- Summary: Failed to read the table created by Spark 2.1 Key: SPARK-21085 URL: https://issues.apache.org/jira/browse/SPARK-21085 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048489#comment-16048489 ] Seth Hendrickson commented on SPARK-20988: -- I've already started it a bit. Would you mind doing

[jira] [Updated] (SPARK-20602) Adding LBFGS as optimizer for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-20602: --- Description: Currently LinearSVC in Spark only supports OWLQN as the optimizer ( check

[jira] [Updated] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-21084: Component/s: YARN Scheduler Block Manager > Improvements to dynamic

[jira] [Updated] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-21084: Affects Version/s: 2.3.0 > Improvements to dynamic allocation for notebook use cases >

[jira] [Updated] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frederick Reiss updated SPARK-21084: Description: One important application of Spark is to support many notebook users with a

[jira] [Created] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

Frederick Reiss created SPARK-21084: --- Summary: Improvements to dynamic allocation for notebook use cases Key: SPARK-21084 URL: https://issues.apache.org/jira/browse/SPARK-21084 Project: Spark

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048477#comment-16048477 ] Joseph K. Bradley commented on SPARK-1: --- Thanks for explaining! I just rediscovered this

[jira] [Commented] (SPARK-21077) Cannot access public files over S3 protocol

[ https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048470#comment-16048470 ] Hyukjin Kwon commented on SPARK-21077: -- I also think it is not a Spark issue at least and it looks

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048441#comment-16048441 ] Zhenhua Wang commented on SPARK-21079: -- [~mbasmanova] Great~ > ANALYZE TABLE fails to calculate

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048432#comment-16048432 ] Maria commented on SPARK-21079: --- [~ZenWzh], yes, I have a fix and will try to submit a PR. > ANALYZE TABLE

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048427#comment-16048427 ] Zhenhua Wang commented on SPARK-21079: -- [~tejasp] Thanks for the explanation! [~mbasmanova] Would

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: (was: Apache Spark) > Allow setting SSL-related passwords through env

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: Apache Spark > Allow setting SSL-related passwords through env variables >

[jira] [Commented] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048424#comment-16048424 ] Apache Spark commented on SPARK-20379: -- User 'vanzin' has created a pull request for this issue:

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA, because there are many tasks found and needed to be done here.

[jira] [Updated] (SPARK-15616) CatalogRelation should fallback to HDFS size of partitions that are involved in Query if statistics are not available.

[ https://issues.apache.org/jira/browse/SPARK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-15616: - Affects Version/s: 2.3.0 Issue Type: Sub-task (was: Improvement)

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA, because there are a few tasks found and needed to be done

[jira] [Updated] (SPARK-16669) Partition pruning for metastore relation size estimates for better join selection.

[ https://issues.apache.org/jira/browse/SPARK-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-16669: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > Partition pruning for metastore

[jira] [Updated] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20986: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > Reset table's statistics after

[jira] [Updated] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21079: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > ANALYZE TABLE fails to calculate

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Affects Version/s: 2.3.0 Issue Type: Improvement (was: Sub-task)

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA as an umbrella ticket, because there are a few tasks found and

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Summary: Support statistics collection and cardinality estimation for partitioned tables (was:

[jira] [Comment Edited] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048384#comment-16048384 ] Dongjoon Hyun edited comment on SPARK-21075 at 6/13/17 8:46 PM: Please do

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048384#comment-16048384 ] Dongjoon Hyun commented on SPARK-21075: --- Please do `jps` and check whether `Zinc` is running or

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048381#comment-16048381 ] Tejas Patil commented on SPARK-21079: - [~ZenWzh] The reason why unit tests won't catch this is

[jira] [Updated] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Ricard updated SPARK-21067: --- Description: After upgrading our Thrift cluster to 2.1.1, we ran into an issue where CTAS

[jira] [Issue Comment Deleted] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maria updated SPARK-21079: -- Comment: was deleted (was: [~ZenWzh], I'm using partitioned table created by Hive. The data is stored in DWRF

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048270#comment-16048270 ] Sean Owen commented on SPARK-21082: --- I don't see how this would interact with, for example, data

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048246#comment-16048246 ] Maria commented on SPARK-21079: --- [~ZenWzh], I'm using partitioned table created by Hive. The data is stored

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048182#comment-16048182 ] Zhenhua Wang commented on SPARK-21079: -- Can you post your usage here? We have tests for partitioned

[jira] [Commented] (SPARK-18294) Implement commit protocol to support `mapred` package's committer

[ https://issues.apache.org/jira/browse/SPARK-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048164#comment-16048164 ] Aarati Khobare commented on SPARK-18294: Hi Jiang I am new to spark and hive, so please let me

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: Spark Scheduler do not consider the memory usage during dispatch tasks, this can lead to

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048146#comment-16048146 ] DjvuLee commented on SPARK-21082: - If this feature is a good suggestion(we encounter this problem in

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: Spark Scheduler do not consider the memory usage during dispatch tasks, this can lead to

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: Spark Scheduler do not consider the memory usage during dispatch tasks, this can lead to

[jira] [Assigned] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21083: Assignee: Apache Spark > Consider staleness when collecting column stats >

[jira] [Assigned] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21083: Assignee: (was: Apache Spark) > Consider staleness when collecting column stats >

[jira] [Commented] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048139#comment-16048139 ] Apache Spark commented on SPARK-21083: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Updated] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Description: Suppose we already collected column stats for some columns before, then, when we

[jira] [Updated] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Description: Suppose we already collected column stats for some columns before, then, when we

[jira] [Created] (SPARK-21083) Consider staleness when collecting column stats

Zhenhua Wang created SPARK-21083: Summary: Consider staleness when collecting column stats Key: SPARK-21083 URL: https://issues.apache.org/jira/browse/SPARK-21083 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: When we cache the > Consider Executor's memory usage when scheduling task >

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Component/s: Scheduler > Consider Executor's memory usage when scheduling task >

[jira] [Created] (SPARK-21082) Consider the Executor's Memory usage when scheduling task

DjvuLee created SPARK-21082: --- Summary: Consider the Executor's Memory usage when scheduling task Key: SPARK-21082 URL: https://issues.apache.org/jira/browse/SPARK-21082 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Summary: Consider Executor's memory usage when scheduling task (was: Consider the Executor's Memory

[jira] [Resolved] (SPARK-21051) Add hash map metrics to aggregate

[ https://issues.apache.org/jira/browse/SPARK-21051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21051. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.3.0 > Add hash map metrics

[jira] [Updated] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

[ https://issues.apache.org/jira/browse/SPARK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-20812: Description: Mesos 1.4 will support secrets. In order to support sending keytabs through

[jira] [Commented] (SPARK-20892) Add SQL trunc function to SparkR

[ https://issues.apache.org/jira/browse/SPARK-20892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048111#comment-16048111 ] Apache Spark commented on SPARK-20892: -- User 'actuaryzhang' has created a pull request for this

[jira] [Commented] (SPARK-21081) Throw specific IllegalStateException subtype when asserting that SparkContext not stopped

[ https://issues.apache.org/jira/browse/SPARK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048104#comment-16048104 ] Sean Owen commented on SPARK-21081: --- Where would you want to catch and handle that separately? > Throw

[jira] [Updated] (SPARK-21081) Throw specific IllegalStateException subtype when asserting that SparkContext not stopped

[ https://issues.apache.org/jira/browse/SPARK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Filipp Zhinkin updated SPARK-21081: --- Summary: Throw specific IllegalStateException subtype when asserting that SparkContext not

[jira] [Created] (SPARK-21081) Throw specific IllegalStateException subtype when that asserting SparkContext not stopped

Filipp Zhinkin created SPARK-21081: -- Summary: Throw specific IllegalStateException subtype when that asserting SparkContext not stopped Key: SPARK-21081 URL: https://issues.apache.org/jira/browse/SPARK-21081

[jira] [Assigned] (SPARK-20989) Fail to start multiple workers on one host if external shuffle service is enabled in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20989: Assignee: Apache Spark > Fail to start multiple workers on one host if external shuffle

[jira] [Assigned] (SPARK-20989) Fail to start multiple workers on one host if external shuffle service is enabled in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20989: Assignee: (was: Apache Spark) > Fail to start multiple workers on one host if

[jira] [Commented] (SPARK-20989) Fail to start multiple workers on one host if external shuffle service is enabled in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048028#comment-16048028 ] Apache Spark commented on SPARK-20989: -- User 'jiangxb1987' has created a pull request for this

[jira] [Created] (SPARK-21080) Workaround for HDFS delegation token expiry broken with some Hadoop versions

Lukasz Raszka created SPARK-21080: - Summary: Workaround for HDFS delegation token expiry broken with some Hadoop versions Key: SPARK-21080 URL: https://issues.apache.org/jira/browse/SPARK-21080

[jira] [Resolved] (SPARK-21064) Fix the default value bug in NettyBlockTransferServiceSuite

[ https://issues.apache.org/jira/browse/SPARK-21064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21064. --- Resolution: Fixed Fix Version/s: 2.1.2 2.3.0 2.2.1

[jira] [Resolved] (SPARK-21060) Css style about paging function is error in the executor page.

[ https://issues.apache.org/jira/browse/SPARK-21060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21060. --- Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Resolved by

[jira] [Created] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

Maria created SPARK-21079: - Summary: ANALYZE TABLE fails to calculate totalSize for a partitioned table Key: SPARK-21079 URL: https://issues.apache.org/jira/browse/SPARK-21079 Project: Spark Issue

  1   2   >