[ https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-23604: ------------------------------------ Assignee: Apache Spark > ParquetInteroperabilityTest timestamp test should use > Statistics.hasNonNullValue > -------------------------------------------------------------------------------- > > Key: SPARK-23604 > URL: https://issues.apache.org/jira/browse/SPARK-23604 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.3.0 > Reporter: Henry Robinson > Assignee: Apache Spark > Priority: Minor > > We ran into an issue with a downstream build of Spark running against a > custom Parquet build where {{ParquetInteroperabilityTestSuite}} started > failing because {{Statistics.isEmpty}} changed its behavior as of > PARQUET-1217. ({{isEmpty() now considers whether there are 0 or more nulls, > and by default {{num_nulls}} is 0 for 'empty' stats objects). > The test really cares about whether the statistics object has values, so a > very simple fix to use {{hasNonNullValue}} instead corrects the issue. Filing > it now because it's a backwards-compatible fix to the current Parquet version > so we can fix it right now before we hit the issue in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org