gszadovszky commented on PR #184: URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1232491783
I've came up with this ordering thing because we specify it for every logical types. (Unfortunately we don't do this for primitive types.) Therefore, I would expect to have the order specified for this new logical type as well which is not trivial and requires to solve [PARQUET-1222](https://issues.apache.org/jira/browse/PARQUET-1222). At least we should add a note about this issue. > It seems this would require parquet implementations to null out statistics for logical types that they don't support, does parquet-mr do that today? I do not have the proper environment to test it but based on the code we do not handle unknown logical types well in parquet-mr. I think it handles unknown logical types as if they were not there at all which is fine from the data point of view but we would blindly use the statistics which may cause data loss. Created [PARQUET-2182](https://issues.apache.org/jira/browse/PARQUET-2182) to track this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
