Xianjin YE created PARQUET-1012: ----------------------------------- Summary: parquet-cpp and parquet-mr version parse inconsistent Key: PARQUET-1012 URL: https://issues.apache.org/jira/browse/PARQUET-1012 Project: Parquet Issue Type: Improvement Reporter: Xianjin YE
Spark 2.1 uses parquet-mr(common) 1.8.2 which requires created_by to match certain pattern. I found the following exception when using spark to read parquet file generated by parquet-cpp. 17/05/31 16:33:53 WARN CorruptStatistics: Ignoring statistics because created_by could not be parsed (see PARQUET-251): parquet-cpp version 1.0.0 org.apache.parquet.VersionParser$VersionParseException: Could not parse created_by: parquet-cpp version 1.0.0 using format: (.+) version ((.*) )?\(build ?(.*)\) Proposal to fix this issue: set created_by to the certain pattern. -- This message was sent by Atlassian JIRA (v6.3.15#6346)