[ https://issues.apache.org/jira/browse/SPARK-31735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17109148#comment-17109148 ]
Apache Spark commented on SPARK-31735: -------------------------------------- User 'Fokko' has created a pull request for this issue: https://github.com/apache/spark/pull/28554 > Include all columns in the summary report > ----------------------------------------- > > Key: SPARK-31735 > URL: https://issues.apache.org/jira/browse/SPARK-31735 > Project: Spark > Issue Type: Improvement > Components: Spark Core, SQL > Affects Versions: 2.4.5 > Reporter: Fokko Driesprong > Priority: Major > > Dates and other columns are excluded: > > {{from datetime import datetime, timedelta, timezone}} > {{from pyspark.sql import types as T}} > {{from pyspark.sql import Row}} > {{from pyspark.sql import functions as F}}{{START = datetime(2014, 1, 1, > tzinfo=timezone.utc)}}{{n_days = 22}}{{date_range = [Row(date=(START + > timedelta(days=n))) for n in range(0, n_days)]}}{{schema = > T.StructType([T.StructField(name="date", dataType=T.DateType(), > nullable=False)])}} > {{rdd = spark.sparkContext.parallelize(date_range)}}{{df = > spark.createDataFrame(data=rdd, schema=schema)}} > {{df.agg(F.max("date")).show()}}{{df.summary().show()}} > {{+-------+}} > {{|summary|}} > {{+-------+}} > {{| count |}} > {{| mean |}} > {{| stddev|}} > {{| min |}} > {{| 25% |}} > {{| 50% |}} > {{| 75% |}} > {{| max |}} > {{+-------+}} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org