[ https://issues.apache.org/jira/browse/SPARK-17073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15755367#comment-15755367 ]
Ioana Delaney commented on SPARK-17073: --------------------------------------- [~mikewzh] FYI, I ran the ANALYZE COMPUTE STATISTICS FOR COLUMNS command on tables defined in TPCDS 1TB db. On November builds, the command worked well for a reasonable number of columns, but it failed as the number of columns increased. The workaround that I used was to run the command on subsets of columns, and that worked fine. I retried the command lately, using a December build, and this time it seems to complete successfully on a large range of columns + data (store_sales table with 2.88e+9 and 23 columns). Great work! > generate basic stats for column > ------------------------------- > > Key: SPARK-17073 > URL: https://issues.apache.org/jira/browse/SPARK-17073 > Project: Spark > Issue Type: Sub-task > Components: Optimizer > Affects Versions: 2.0.0 > Reporter: Ron Hu > Assignee: Zhenhua Wang > Fix For: 2.1.0 > > > For a specified column, we need to generate basic stats including max, min, > number of nulls, number of distinct values, max column length, average column > length. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org