soumyakanti3578 commented on code in PR #6851:
URL: https://github.com/apache/hive/pull/6851#discussion_r4596339588
##########
ql/src/java/org/apache/hadoop/hive/ql/io/SymbolicInputFormat.java:
##########
@@ -98,4 +102,58 @@ public void rework(HiveConf job, MapredWork work) throws
IOException {
work.getMapWork().removePathToPartitionInfo(toRemove);
}
}
+
+ /**
+ * Expands a symlink file line, accepting only matches under the symlink
file's directory or an allowed root.
+ */
+ static List<Path> resolveTargets(Configuration conf, Path symlinkFile,
List<Path> allowedRoots, String line)
+ throws IOException {
+ // Qualified against the default filesystem, so no filesystem is created
for an arbitrary scheme.
+ FileSystem defaultFs = FileSystem.get(conf);
+ List<Path> roots = new ArrayList<>(allowedRoots);
+ roots.add(qualify(defaultFs, symlinkFile.getParent()));
+ Path pattern = qualify(defaultFs, new Path(line));
+ // Matches are checked too, as a glob can climb out of the root.
+ checkAllowed(pattern, roots);
Review Comment:
With allowed root `/data/a`:
| Line | Check before `globStatus` | `globStatus` result | Check in the loop
| Result |
|---|---|---|---|---|
| `/data/a/*.txt` | pass | `/data/a/1.txt` | pass | read |
| `/other/*` | fail | not called | - | rejected, `/other` is never listed |
| `/data/{a,b}/x` | fail | not called | - | rejected |
| `/data/a/{../b/x}` | pass | `/data/b/x` | fail | rejected |
1. Without the first check `/other/*` would be listed before the loop
rejects it. The first check is an inexpensive way to prevent any unnecessary
I/O, or unnecessary connections to s3a for example.
2. Yes, `/data/{a,b}/x` would match `/data/a/x`, but it is rejected up
front. I think that's fine, the pattern has to be written under an allowed root.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]