[
https://issues.apache.org/jira/browse/FALCON-81?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated FALCON-81:
------------------------------
Description:
{noformat}
If the data path is /data/${YEAR}/${MONTH}/${DAY}/${HOUR}/SomeData, feed
eviction checks for pattern /data/*/*/*/*/SomeData which should be directory.
Glob status on this fails if there is a file /data/2013/01/01/01/file:
Failing Oozie Launcher, Main class [org.apache.ivory.retention.FeedEvictor],
main() threw exception, org.apache.hadoop.security.AccessControlException:
Permission denied: user=<user>, access=EXECUTE,
inode="/data/2013/01/01/01/file":<user>:supergroup:-rw-r--r--
org.apache.hadoop.security.AccessControlException:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=dataload, access=EXECUTE,
inode="/data/2013/01/01/01/file":<user>:supergroup:-rw-r--r--
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:914)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:523)
at org.apache.hadoop.fs.FileSystem.getFileStatus(FileSystem.java:1534)
at
org.apache.hadoop.fs.FileSystem.globStatusInternal(FileSystem.java:1096)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1038)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1016)
at
org.apache.ivory.retention.FeedEvictor.findFilesForFeed(FeedEvictor.java:204)
at
org.apache.ivory.retention.FeedEvictor.discoverInstanceToDelete(FeedEvictor.java:166)
at org.apache.ivory.retention.FeedEvictor.run(FeedEvictor.java:112)
This is because of an issue in glob status. Feed evictor should search with
pattern /data/*/*/*/*/*/{SomeData}
{noformat}
was:
If the data path is /data/${YEAR}/${MONTH}/${DAY}/${HOUR}/SomeData, feed
eviction checks for pattern /data/*/*/*/*/SomeData which should be directory.
Glob status on this fails if there is a file /data/2013/01/01/01/file:
Failing Oozie Launcher, Main class [org.apache.ivory.retention.FeedEvictor],
main() threw exception, org.apache.hadoop.security.AccessControlException:
Permission denied: user=<user>, access=EXECUTE,
inode="/data/2013/01/01/01/file":<user>:supergroup:-rw-r--r--
org.apache.hadoop.security.AccessControlException:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=dataload, access=EXECUTE,
inode="/data/meta/2013/06/19/23/dataseller_gen.csv":dataload:supergroup:-rw-r--r--
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:914)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:523)
at org.apache.hadoop.fs.FileSystem.getFileStatus(FileSystem.java:1534)
at
org.apache.hadoop.fs.FileSystem.globStatusInternal(FileSystem.java:1096)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1038)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1016)
at
org.apache.ivory.retention.FeedEvictor.findFilesForFeed(FeedEvictor.java:204)
at
org.apache.ivory.retention.FeedEvictor.discoverInstanceToDelete(FeedEvictor.java:166)
at org.apache.ivory.retention.FeedEvictor.run(FeedEvictor.java:112)
This is because of an issue in glob status. Feed evictor should search with
pattern /data/*/*/*/*/*/{SomeData}
> globStatus in feed retention fails in some cases
> ------------------------------------------------
>
> Key: FALCON-81
> URL: https://issues.apache.org/jira/browse/FALCON-81
> Project: Falcon
> Issue Type: Bug
> Reporter: Shwetha G S
>
> {noformat}
> If the data path is /data/${YEAR}/${MONTH}/${DAY}/${HOUR}/SomeData, feed
> eviction checks for pattern /data/*/*/*/*/SomeData which should be directory.
> Glob status on this fails if there is a file /data/2013/01/01/01/file:
> Failing Oozie Launcher, Main class [org.apache.ivory.retention.FeedEvictor],
> main() threw exception, org.apache.hadoop.security.AccessControlException:
> Permission denied: user=<user>, access=EXECUTE,
> inode="/data/2013/01/01/01/file":<user>:supergroup:-rw-r--r--
> org.apache.hadoop.security.AccessControlException:
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=dataload, access=EXECUTE,
> inode="/data/2013/01/01/01/file":<user>:supergroup:-rw-r--r--
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:914)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:523)
> at org.apache.hadoop.fs.FileSystem.getFileStatus(FileSystem.java:1534)
> at
> org.apache.hadoop.fs.FileSystem.globStatusInternal(FileSystem.java:1096)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1038)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1016)
> at
> org.apache.ivory.retention.FeedEvictor.findFilesForFeed(FeedEvictor.java:204)
> at
> org.apache.ivory.retention.FeedEvictor.discoverInstanceToDelete(FeedEvictor.java:166)
> at org.apache.ivory.retention.FeedEvictor.run(FeedEvictor.java:112)
> This is because of an issue in glob status. Feed evictor should search with
> pattern /data/*/*/*/*/*/{SomeData}
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
