Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r129486097
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java
---
@@ -1851,9 +1922,73 @@ private static String relativize(String baseDir,
String childPath) {
.relativize(fullPathWithoutSchemeAndAuthority.toUri()));
if (relativeFilePath.isAbsolute()) {
throw new IllegalStateException(String.format("Path %s is not a
subpath of %s.",
- basePathWithoutSchemeAndAuthority.toUri().toString(),
fullPathWithoutSchemeAndAuthority.toUri().toString()));
+ basePathWithoutSchemeAndAuthority.toUri().getPath(),
fullPathWithoutSchemeAndAuthority.toUri().getPath()));
+ }
+ return relativeFilePath.toUri().getPath();
+ }
+ }
+
+ /**
+ * Supported metadata versions.
+ * <p>
+ * Note: keep them synchronized with {@link ParquetTableMetadataBase}
versions
+ */
+ public static class MetadataVersion {
+
+ /**
+ * Version 1: Introduces parquet file metadata caching.<br>
+ * See DRILL-2743
+ */
+ public static final String V1 = "v1";
+ /**
+ * Version 2: Metadata cache file size is reduced.<br>
+ * See DRILL-4053
+ */
+ public static final String V2 = "v2";
+ /**
+ * Version 3: Difference between v3 and v2 : min/max, type_length,
precision, scale, repetitionLevel, definitionLevel.<br>
+ * Filter pushdown for Parquet is implemented. <br>
+ * See DRILL-1950
+ */
+ public static final String V3 = "v3";
+ /**
+ * Version 3.1: Absolute paths of files and directories are replaced
with relative ones.<br>
+ * See DRILL-3867
+ */
+ public static final String V3_1 = "v3.1";
+
+
+ /**
+ * All historical versions of the Drill metadata cache files
+ */
+ public static final List<String> SUPPORTED_VERSIONS =
Lists.newArrayList(V1, V2, V3, V3_1);
+
+ /**
+ * @param metadataVersion parquet metadata version
+ * @return true if metadata version is supported, false otherwise
+ */
+ public static boolean isVersionSupported(String metadataVersion) {
+ return SUPPORTED_VERSIONS.contains(metadataVersion);
+ }
+
+ /**
+ * Helper compare method similar to {@link
java.util.Comparator#compare}
+ *
+ * @param metadataVersion1 the first metadata version to be compared
+ * @param metadataVersion2 the second metadata version to be compared
+ * @return a negative integer, zero, or a positive integer as the
+ * first argument is less than, equal to, or greater than the
+ * second.
+ */
+ public static int compare(String metadataVersion1, String
metadataVersion2) {
--- End diff --
Added new `VersionParser` class, which allow to parse parquet metadata
version string. Added test cases for different correct and wrong parquet
metadata versions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
