[ https://issues.apache.org/jira/browse/PARQUET-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035485#comment-16035485 ]
Deepak Majeti commented on PARQUET-1012: ---------------------------------------- I could not reproduce this with parquet-mr 1.8.2. The error seems to originate from {{shouldIgnoreStatistics}} in {{parquet-column/src/main/java/org/apache/parquet/CorruptStatistics.java:52}}. I added a test in {{parquet-column/src/test/java/org/apache/parquet/CorruptStatisticsTest.java}} with the {{parquet-cpp version 1.0.0}} string and it works fine. Can you share the parquet-cpp file? > parquet-cpp and parquet-mr version parse inconsistent > ----------------------------------------------------- > > Key: PARQUET-1012 > URL: https://issues.apache.org/jira/browse/PARQUET-1012 > Project: Parquet > Issue Type: Improvement > Components: parquet-cpp > Reporter: Xianjin YE > Assignee: Deepak Majeti > > Spark 2.1 uses parquet-mr(common) 1.8.2 which requires created_by to match > certain pattern. I found the following exception when using spark to read > parquet file generated by parquet-cpp. > 17/05/31 16:33:53 WARN CorruptStatistics: Ignoring statistics because > created_by could not be parsed (see PARQUET-251): parquet-cpp version 1.0.0 > org.apache.parquet.VersionParser$VersionParseException: Could not parse > created_by: parquet-cpp version 1.0.0 using format: (.+) version ((.*) > )?\(build ?(.*)\) > Proposal to fix this issue: set created_by to the certain pattern. -- This message was sent by Atlassian JIRA (v6.3.15#6346)