[
https://issues.apache.org/jira/browse/DRILL-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058071#comment-16058071
]
ASF GitHub Bot commented on DRILL-3867:
---------------------------------------
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/824#discussion_r123331277
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java
---
@@ -748,6 +771,22 @@ public ParquetTableMetadataDirs(List<String>
directories) {
return directories;
}
+ /** If directories list contains relative paths, update it to absolute
ones
--- End diff --
Yes, we do, internally we use absolute paths (for the FileSelection,
FileStatus, ReadEntryWithPath).
By the way it is possible to convert paths to absolute ones just before
retrieving, but converting immediately after deserializing has advantages:
avoiding of the keeping the metadata with appropriate `baseDir`, avoiding of
the over number of checking the type of the path and avoiding an extra
converting paths (when the data is retrieved several times from one metadata
object).
> Store relative paths in metadata file
> -------------------------------------
>
> Key: DRILL-3867
> URL: https://issues.apache.org/jira/browse/DRILL-3867
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
> Affects Versions: 1.2.0
> Reporter: Rahul Challapalli
> Assignee: Vitalii Diravka
> Fix For: Future
>
>
> git.commit.id.abbrev=cf4f745
> git.commit.time=29.09.2015 @ 23\:19\:52 UTC
> The below sequence of steps reproduces the issue
> 1. Create the cache file
> {code}
> 0: jdbc:drill:zk=10.10.103.60:5181> refresh table metadata
> dfs.`/drill/testdata/metadata_caching/lineitem`;
> +-------+-------------------------------------------------------------------------------------+
> | ok | summary
> |
> +-------+-------------------------------------------------------------------------------------+
> | true | Successfully updated metadata for table
> /drill/testdata/metadata_caching/lineitem. |
> +-------+-------------------------------------------------------------------------------------+
> 1 row selected (1.558 seconds)
> {code}
> 2. Move the directory
> {code}
> hadoop fs -mv /drill/testdata/metadata_caching/lineitem /drill/
> {code}
> 3. Now run a query on top of it
> {code}
> 0: jdbc:drill:zk=10.10.103.60:5181> select * from dfs.`/drill/lineitem` limit
> 1;
> Error: SYSTEM ERROR: FileNotFoundException: Requested file
> maprfs:///drill/testdata/metadata_caching/lineitem/2006/1 does not exist.
> [Error Id: b456d912-57a0-4690-a44b-140d4964903e on pssc-66.qa.lab:31010]
> (state=,code=0)
> {code}
> This is obvious given the fact that we are storing absolute file paths in the
> cache file
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
