[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048769#comment-16048769 ] Vincent commented on SPARK-20988: - okay, no problem :) > Convert logistic regression to

[jira] [Commented] (SPARK-21091) Move constraint code into QueryPlanConstraints

[ https://issues.apache.org/jira/browse/SPARK-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048759#comment-16048759 ] Apache Spark commented on SPARK-21091: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-21091) Move constraint code into QueryPlanConstraints

[ https://issues.apache.org/jira/browse/SPARK-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21091: Assignee: Reynold Xin (was: Apache Spark) > Move constraint code into QueryPlanConstraint

[jira] [Assigned] (SPARK-21091) Move constraint code into QueryPlanConstraints

[ https://issues.apache.org/jira/browse/SPARK-21091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21091: Assignee: Apache Spark (was: Reynold Xin) > Move constraint code into QueryPlanConstraint

[jira] [Created] (SPARK-21091) Move constraint code into QueryPlanConstraints

Reynold Xin created SPARK-21091: --- Summary: Move constraint code into QueryPlanConstraints Key: SPARK-21091 URL: https://issues.apache.org/jira/browse/SPARK-21091 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-20211) `1 > 0.0001` throws Decimal scale (0) cannot be greater than precision (-2) exception

[ https://issues.apache.org/jira/browse/SPARK-20211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048743#comment-16048743 ] Apache Spark commented on SPARK-20211: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048731#comment-16048731 ] Apache Spark commented on SPARK-21090: -- User '10110346' has created a pull request f

[jira] [Assigned] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21090: Assignee: (was: Apache Spark) > Optimize the unified memory manager code > --

[jira] [Assigned] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21090: Assignee: Apache Spark > Optimize the unified memory manager code > -

[jira] [Updated] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-21090: Description: 1.In *acquireStorageMemory*, when the MemoryMode is OFF_HEAP ,the *maxMemory* should be modif

[jira] [Created] (SPARK-21090) Optimize the unified memory manager code

liuxian created SPARK-21090: --- Summary: Optimize the unified memory manager code Key: SPARK-21090 URL: https://issues.apache.org/jira/browse/SPARK-21090 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048723#comment-16048723 ] yuhao yang commented on SPARK-21087: I'd like to work on this if my [comment|https:/

[jira] [Comment Edited] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048647#comment-16048647 ] yuhao yang edited comment on SPARK-21086 at 6/14/17 5:22 AM: -

[jira] [Commented] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048720#comment-16048720 ] Apache Spark commented on SPARK-21085: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21085: Assignee: Apache Spark (was: Xiao Li) > Failed to read the partitioned table created by S

[jira] [Assigned] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21085: Assignee: Xiao Li (was: Apache Spark) > Failed to read the partitioned table created by S

[jira] [Comment Edited] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048647#comment-16048647 ] yuhao yang edited comment on SPARK-21086 at 6/14/17 5:12 AM: -

[jira] [Assigned] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21089: Assignee: Apache Spark (was: Xiao Li) > Table properties are not shown in DESC EXTENDED/F

[jira] [Assigned] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21089: Assignee: Xiao Li (was: Apache Spark) > Table properties are not shown in DESC EXTENDED/F

[jira] [Commented] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048717#comment-16048717 ] Apache Spark commented on SPARK-21089: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

Xiao Li created SPARK-21089: --- Summary: Table properties are not shown in DESC EXTENDED/FORMATTED Key: SPARK-21089 URL: https://issues.apache.org/jira/browse/SPARK-21089 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21089: Target Version/s: 2.2.0 > Table properties are not shown in DESC EXTENDED/FORMATTED > -

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048698#comment-16048698 ] yuhao yang commented on SPARK-20988: Eh.. I was trying to add the squared_hinge loss

[jira] [Assigned] (SPARK-19753) Remove all shuffle files on a host in case of slave lost of fetch failure

[ https://issues.apache.org/jira/browse/SPARK-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19753: --- Assignee: Sital Kedia > Remove all shuffle files on a host in case of slave lost of fetch fa

[jira] [Resolved] (SPARK-19753) Remove all shuffle files on a host in case of slave lost of fetch failure

[ https://issues.apache.org/jira/browse/SPARK-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19753. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18150 [https://githu

[jira] [Resolved] (SPARK-20348) Support squared hinge loss (L2 loss) for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang resolved SPARK-20348. Resolution: Duplicate Combine it with SPARK-20602 and resolve this as duplicate. > Support squared

[jira] [Commented] (SPARK-20602) Adding LBFGS optimizer and Squared_hinge loss for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048663#comment-16048663 ] yuhao yang commented on SPARK-20602: Combining this with SPARK-20348. Support squared

[jira] [Updated] (SPARK-20602) Adding LBFGS optimizer and Squared_hinge loss for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-20602: --- Summary: Adding LBFGS optimizer and Squared_hinge loss for LinearSVC (was: Adding LBFGS as optimizer

[jira] [Comment Edited] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048653#comment-16048653 ] DjvuLee edited comment on SPARK-21082 at 6/14/17 3:15 AM: -- [~sro

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048654#comment-16048654 ] DjvuLee commented on SPARK-21082: - My idea is try to consider the BlockManger information

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048653#comment-16048653 ] DjvuLee commented on SPARK-21082: - [~srowen] This situation occurred when the partition n

[jira] [Commented] (SPARK-21078) JobHistory applications synchronized is invalid

[ https://issues.apache.org/jira/browse/SPARK-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048650#comment-16048650 ] fangfengbin commented on SPARK-21078: - [~sowen], this actually cause a problem in pra

[jira] [Commented] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

[ https://issues.apache.org/jira/browse/SPARK-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048647#comment-16048647 ] yuhao yang commented on SPARK-21086: Sounds good. About the default path for saving d

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048612#comment-16048612 ] 吴志龙 commented on SPARK-21075: - ok,thanks > spark 2.2 mvn [error] javac: invalid source relea

[jira] [Assigned] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20986: --- Assignee: Lianhui Wang > Reset table's statistics after PruneFileSourcePartitions rule. > --

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048593#comment-16048593 ] Vincent commented on SPARK-20988: - opps. I have finished the conversion part, but there a

[jira] [Resolved] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20986. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18205 [https://githu

[jira] [Created] (SPARK-21088) CrossValidator, TrainValidationSplit should preserve all models after fitting: Python

Joseph K. Bradley created SPARK-21088: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting: Python Key: SPARK-21088 URL: https://issues.apache.org/jira/browse/SPARK-21088

[jira] [Updated] (SPARK-21088) CrossValidator, TrainValidationSplit should preserve all models after fitting: Python

[ https://issues.apache.org/jira/browse/SPARK-21088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21088: -- Component/s: PySpark > CrossValidator, TrainValidationSplit should preserve all models

[jira] [Created] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

Joseph K. Bradley created SPARK-21087: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala Key: SPARK-21087 URL: https://issues.apache.org/jira/browse/SPARK-21087

[jira] [Created] (SPARK-21086) CrossValidator, TrainValidationSplit should preserve all models after fitting

Joseph K. Bradley created SPARK-21086: - Summary: CrossValidator, TrainValidationSplit should preserve all models after fitting Key: SPARK-21086 URL: https://issues.apache.org/jira/browse/SPARK-21086

[jira] [Assigned] (SPARK-12552) Recovered driver's resource is not counted in the Master

[ https://issues.apache.org/jira/browse/SPARK-12552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-12552: --- Assignee: Saisai Shao (was: Apache Spark) > Recovered driver's resource is not counted in t

[jira] [Resolved] (SPARK-12552) Recovered driver's resource is not counted in the Master

[ https://issues.apache.org/jira/browse/SPARK-12552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-12552. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull req

[jira] [Updated] (SPARK-20979) Add a rate source to generate values for tests and benchmark

[ https://issues.apache.org/jira/browse/SPARK-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20979: - Fix Version/s: 2.2.0 > Add a rate source to generate values for tests and benchmark > ---

[jira] [Updated] (SPARK-21085) Failed to read the partitioned table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21085: Summary: Failed to read the partitioned table created by Spark 2.1 (was: Failed to read the table created

[jira] [Updated] (SPARK-21085) Failed to read the table created by Spark 2.1

[ https://issues.apache.org/jira/browse/SPARK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21085: Description: Spark 2.2 is unable to read the partitioned table created by Spark 2.1 when the table schema

[jira] [Created] (SPARK-21085) Failed to read the table created by Spark 2.1

Xiao Li created SPARK-21085: --- Summary: Failed to read the table created by Spark 2.1 Key: SPARK-21085 URL: https://issues.apache.org/jira/browse/SPARK-21085 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-20988) Convert logistic regression to new aggregator framework

[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048489#comment-16048489 ] Seth Hendrickson commented on SPARK-20988: -- I've already started it a bit. Would

[jira] [Updated] (SPARK-20602) Adding LBFGS as optimizer for LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-20602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-20602: --- Description: Currently LinearSVC in Spark only supports OWLQN as the optimizer ( check https://issue

[jira] [Updated] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-21084: Component/s: YARN Scheduler Block Manager > Improvements to dynamic alloc

[jira] [Updated] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-21084: Affects Version/s: 2.3.0 > Improvements to dynamic allocation for notebook use cases >

[jira] [Updated] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frederick Reiss updated SPARK-21084: Description: One important application of Spark is to support many notebook users with a s

[jira] [Created] (SPARK-21084) Improvements to dynamic allocation for notebook use cases

Frederick Reiss created SPARK-21084: --- Summary: Improvements to dynamic allocation for notebook use cases Key: SPARK-21084 URL: https://issues.apache.org/jira/browse/SPARK-21084 Project: Spark

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048477#comment-16048477 ] Joseph K. Bradley commented on SPARK-1: --- Thanks for explaining! I just red

[jira] [Commented] (SPARK-21077) Cannot access public files over S3 protocol

[ https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048470#comment-16048470 ] Hyukjin Kwon commented on SPARK-21077: -- I also think it is not a Spark issue at leas

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048441#comment-16048441 ] Zhenhua Wang commented on SPARK-21079: -- [~mbasmanova] Great~ > ANALYZE TABLE fails

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048432#comment-16048432 ] Maria commented on SPARK-21079: --- [~ZenWzh], yes, I have a fix and will try to submit a PR.

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048427#comment-16048427 ] Zhenhua Wang commented on SPARK-21079: -- [~tejasp] Thanks for the explanation! [~mba

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: (was: Apache Spark) > Allow setting SSL-related passwords through env variab

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: Apache Spark > Allow setting SSL-related passwords through env variables > -

[jira] [Commented] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048424#comment-16048424 ] Apache Spark commented on SPARK-20379: -- User 'vanzin' has created a pull request for

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA, because there are many tasks found and needed to be done here.

[jira] [Updated] (SPARK-15616) CatalogRelation should fallback to HDFS size of partitions that are involved in Query if statistics are not available.

[ https://issues.apache.org/jira/browse/SPARK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-15616: - Affects Version/s: 2.3.0 Issue Type: Sub-task (was: Improvement) Parent

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA, because there are a few tasks found and needed to be done here.

[jira] [Updated] (SPARK-16669) Partition pruning for metastore relation size estimates for better join selection.

[ https://issues.apache.org/jira/browse/SPARK-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-16669: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > Partition pruning for metastore re

[jira] [Updated] (SPARK-20986) Reset table's statistics after PruneFileSourcePartitions rule.

[ https://issues.apache.org/jira/browse/SPARK-20986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20986: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > Reset table's statistics after Pru

[jira] [Updated] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21079: - Issue Type: Sub-task (was: Bug) Parent: SPARK-17129 > ANALYZE TABLE fails to calculate t

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Affects Version/s: 2.3.0 Issue Type: Improvement (was: Sub-task) Parent

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Description: I upgrade this JIRA as an umbrella ticket, because there are a few tasks found and n

[jira] [Updated] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17129: - Summary: Support statistics collection and cardinality estimation for partitioned tables (was: s

[jira] [Comment Edited] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048384#comment-16048384 ] Dongjoon Hyun edited comment on SPARK-21075 at 6/13/17 8:46 PM: ---

[jira] [Commented] (SPARK-21075) spark 2.2 mvn [error] javac: invalid source release: 1.8

[ https://issues.apache.org/jira/browse/SPARK-21075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048384#comment-16048384 ] Dongjoon Hyun commented on SPARK-21075: --- Please do `jps` and check whether `Zinc` i

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048381#comment-16048381 ] Tejas Patil commented on SPARK-21079: - [~ZenWzh] The reason why unit tests won't catc

[jira] [Updated] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominic Ricard updated SPARK-21067: --- Description: After upgrading our Thrift cluster to 2.1.1, we ran into an issue where CTAS wo

[jira] [Issue Comment Deleted] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maria updated SPARK-21079: -- Comment: was deleted (was: [~ZenWzh], I'm using partitioned table created by Hive. The data is stored in DWRF

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048270#comment-16048270 ] Sean Owen commented on SPARK-21082: --- I don't see how this would interact with, for exam

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048246#comment-16048246 ] Maria commented on SPARK-21079: --- [~ZenWzh], I'm using partitioned table created by Hive. Th

[jira] [Commented] (SPARK-21079) ANALYZE TABLE fails to calculate totalSize for a partitioned table

[ https://issues.apache.org/jira/browse/SPARK-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048182#comment-16048182 ] Zhenhua Wang commented on SPARK-21079: -- Can you post your usage here? We have tests

[jira] [Commented] (SPARK-18294) Implement commit protocol to support `mapred` package's committer

[ https://issues.apache.org/jira/browse/SPARK-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048164#comment-16048164 ] Aarati Khobare commented on SPARK-18294: Hi Jiang I am new to spark and hive, s

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: Spark Scheduler do not consider the memory usage during dispatch tasks, this can lead to Exe

[jira] [Commented] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048146#comment-16048146 ] DjvuLee commented on SPARK-21082: - If this feature is a good suggestion(we encounter this

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: Spark Scheduler do not consider the memory usage during dispatch tasks, this can lead to Exe

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: Spark Scheduler do not consider the memory usage during dispatch tasks, this can lead to Exec

[jira] [Assigned] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21083: Assignee: Apache Spark > Consider staleness when collecting column stats > ---

[jira] [Commented] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048139#comment-16048139 ] Apache Spark commented on SPARK-21083: -- User 'wzhfy' has created a pull request for

[jira] [Assigned] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21083: Assignee: (was: Apache Spark) > Consider staleness when collecting column stats >

[jira] [Updated] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Description: Suppose we already collected column stats for some columns before, then, when we co

[jira] [Updated] (SPARK-21083) Consider staleness when collecting column stats

[ https://issues.apache.org/jira/browse/SPARK-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-21083: - Description: Suppose we already collected column stats for some columns before, then, when we co

[jira] [Created] (SPARK-21083) Consider staleness when collecting column stats

Zhenhua Wang created SPARK-21083: Summary: Consider staleness when collecting column stats Key: SPARK-21083 URL: https://issues.apache.org/jira/browse/SPARK-21083 Project: Spark Issue Type: S

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Description: When we cache the > Consider Executor's memory usage when scheduling task >

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Component/s: Scheduler > Consider Executor's memory usage when scheduling task > -

[jira] [Created] (SPARK-21082) Consider the Executor's Memory usage when scheduling task

DjvuLee created SPARK-21082: --- Summary: Consider the Executor's Memory usage when scheduling task Key: SPARK-21082 URL: https://issues.apache.org/jira/browse/SPARK-21082 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21082) Consider Executor's memory usage when scheduling task

[ https://issues.apache.org/jira/browse/SPARK-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-21082: Summary: Consider Executor's memory usage when scheduling task (was: Consider the Executor's Memory usage

[jira] [Resolved] (SPARK-21051) Add hash map metrics to aggregate

[ https://issues.apache.org/jira/browse/SPARK-21051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21051. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.3.0 > Add hash map metrics t

[jira] [Updated] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

[ https://issues.apache.org/jira/browse/SPARK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-20812: Description: Mesos 1.4 will support secrets. In order to support sending keytabs through

[jira] [Commented] (SPARK-20892) Add SQL trunc function to SparkR

[ https://issues.apache.org/jira/browse/SPARK-20892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048111#comment-16048111 ] Apache Spark commented on SPARK-20892: -- User 'actuaryzhang' has created a pull reque

[jira] [Commented] (SPARK-21081) Throw specific IllegalStateException subtype when asserting that SparkContext not stopped

[ https://issues.apache.org/jira/browse/SPARK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048104#comment-16048104 ] Sean Owen commented on SPARK-21081: --- Where would you want to catch and handle that sepa

[jira] [Updated] (SPARK-21081) Throw specific IllegalStateException subtype when asserting that SparkContext not stopped

[ https://issues.apache.org/jira/browse/SPARK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Filipp Zhinkin updated SPARK-21081: --- Summary: Throw specific IllegalStateException subtype when asserting that SparkContext not st

[jira] [Created] (SPARK-21081) Throw specific IllegalStateException subtype when that asserting SparkContext not stopped

Filipp Zhinkin created SPARK-21081: -- Summary: Throw specific IllegalStateException subtype when that asserting SparkContext not stopped Key: SPARK-21081 URL: https://issues.apache.org/jira/browse/SPARK-21081

[jira] [Assigned] (SPARK-20989) Fail to start multiple workers on one host if external shuffle service is enabled in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20989: Assignee: Apache Spark > Fail to start multiple workers on one host if external shuffle se

  1   2   >