[
https://issues.apache.org/jira/browse/DRILL-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694499#comment-16694499
]
ASF GitHub Bot commented on DRILL-6858:
---------------------------------------
vvysotskyi commented on a change in pull request #1547: DRILL-6858: Add
functionality to list directories / files with exceptions suppression
URL: https://github.com/apache/drill/pull/1547#discussion_r235316095
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/util/FileSystemUtil.java
##########
@@ -55,7 +53,28 @@ public boolean accept(Path path) {
*/
public static List<FileStatus> listDirectories(final FileSystem fs, Path
path, boolean recursive, PathFilter... filters) throws IOException {
List<FileStatus> statuses = new ArrayList<>();
- listDirectories(fs, path, recursive, statuses, mergeFilters(filters));
+ listDirectories(fs, path, recursive, false, statuses,
mergeFilters(filters));
+ return statuses;
+ }
+
+ /**
+ * Returns statuses of all directories present in given path applying custom
filters if present.
+ * Will also include nested directories if recursive flag is set to true.
+ * Will ignore all exceptions during listing if any.
+ *
+ * @param fs current file system
+ * @param path path to directory
+ * @param recursive true if nested directories should be included
+ * @param filters list of custom filters (optional)
+ * @return list of matching directory statuses
+ */
+ public static List<FileStatus> listDirectoriesSafe(final FileSystem fs, Path
path, boolean recursive, PathFilter... filters) {
+ List<FileStatus> statuses = new ArrayList<>();
+ try {
+ listDirectories(fs, path, recursive, false, statuses,
mergeFilters(filters));
Review comment:
Looks like `suppressExceptions` should be `true` here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> INFORMATION_SCHEMA.`FILES` table not show any files if storage contains
> "write only" folder
> -------------------------------------------------------------------------------------------
>
> Key: DRILL-6858
> URL: https://issues.apache.org/jira/browse/DRILL-6858
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.15.0
> Reporter: Denys Ordynskiy
> Assignee: Arina Ielchiieva
> Priority: Major
> Fix For: 1.15.0
>
>
> *Steps to reproduce bug:*
> * storage contains folder with "write only" permission;
> * Drill option "storage.list_files_recursively" is false;
> * query result for "SELECT * FROM INFORMATION_SCHEMA.`FILES`;" shows the
> list of the files and folders for every workspace for each storage plugin;
> * set Drill option "storage.list_files_recursively" to true;
> * run query "SELECT * FROM INFORMATION_SCHEMA.`FILES`;"
> *Actual result:*
> Drill returns empty results table if workspace has a folder with "write only"
> permission. Even if this workspace contains files and folders, that have
> "read" access right.
> *Expected result:*
> Drill returns list of files for each folder with "read" permission and hide
> files list for "write only" folders. Showing only folders name when
> permission is "write only".
>
> This bug can be reproduced on "Amazon S3" and "File System" Drill storage
> plugins.
>
> In "drillbit.log" for "*File System*" Drill storage plugin:
> {code:java}
> 2018-11-15 20:20:19,760 [2412471b-a4f5-d247-0db6-5d50cdcef634:frag:0:0] WARN
> o.a.d.e.s.i.InfoSchemaRecordGenerator - Failure while trying to list files
> java.io.FileNotFoundException: file:///home/mapr/test/writeonly: null file
> list{code}
>
> In "drillbit.log" for "*Amazon S3*" Drill storage plugin:
> {code:java}
> 2018-11-15 16:50:04,735 [24127862-a0e3-02ad-6b66-d649a62e8b8a:frag:0:0] WARN
> o.a.d.e.s.i.InfoSchemaRecordGenerator - Failure while trying to list files
> java.io.FileNotFoundException: No such file or directory:
> s3a://bucket/writeonly{code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
