[
https://issues.apache.org/jira/browse/FALCON-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14506434#comment-14506434
]
Venkat Ramachandran commented on FALCON-1162:
---------------------------------------------
Looking at the code:
checkPathOwnerAndPermission()
- when exactPerms = true, exact check for the HDFS folder permission against
the requested permission
- when exactPerms = false, checks if HDFS folder permission is greater than
request permission for (U/G/O)
When passing in requested perms 777, the else block should trigger and throw
exception.
Now the question is what is the purpose of exactPerms option here.
With the patch applied, If the staging dir does not have 777, it is caught
during cluster entity creation:
{code}
falcon entity -submit -type cluster -file primaryCluster.xml
ERROR: Bad Request;Path /apps/falcon/primaryCluster/staging has permissions:
rwxr-xr-x, should be rwxrwxrwx at least
{code}
Without the patch, the process entity schedule fails with the following error
in falcon.application.log
{code}
2015-04-21 21:24:24,195 ERROR - [1129226136@qtp-1808415473-119 -
adfffb6b-123d-4127-a636-48c456db0c79:hrt_qa:POST//entities/schedule/process/SampleProcess3]
~ Unable to schedule workflow (AbstractSchedulableEntityManager:69)
org.apache.falcon.FalconException: Entity schedule failed for process:
SampleProcess3
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:89)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:66)
at
org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:122)
at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:49)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$9.doExecute(SchedulableEntityManagerProxy.java:379)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$EntityProxy.execute(SchedulableEntityManagerProxy.java:550)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule_aroundBody12(SchedulableEntityManagerProxy.java:381)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy$AjcClosure13.run(SchedulableEntityManagerProxy.java:1)
at
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at
org.apache.falcon.aspect.AbstractFalconAspect.logAroundMonitored(AbstractFalconAspect.java:51)
at
org.apache.falcon.resource.proxy.SchedulableEntityManagerProxy.schedule(SchedulableEntityManagerProxy.java:365)
at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.falcon.security.FalconAuthorizationFilter.doFilter(FalconAuthorizationFilter.java:106)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.falcon.security.FalconAuthenticationFilter$2.doFilter(FalconAuthenticationFilter.java:184)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:585)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:548)
at
org.apache.falcon.security.FalconAuthenticationFilter.doFilter(FalconAuthenticationFilter.java:193)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.falcon.security.FalconAuditFilter.doFilter(FalconAuditFilter.java:64)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: org.apache.falcon.FalconException: Error preparing base staging
dirs:
/apps/falcon/primaryCluster/staging/falcon/workflows/process/SampleProcess3
at
org.apache.falcon.workflow.engine.OozieWorkflowEngine.prepareEntityBuildPath(OozieWorkflowEngine.java:184)
at
org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:152)
at
org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:87)
... 57 more
Caused by: org.apache.hadoop.security.AccessControlException: Permission
denied: user=hrt_qa, access=WRITE,
inode="/apps/falcon/primaryCluster/staging/falcon/workflows/process/SampleProcess3":falcon:users:drwxr-xr-x
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1698)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1682)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1665)
at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3888)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:977)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043){code}
> Cluster submit succeeds when staging HDFS dir does not have 777 (ALL)
> permission
> ---------------------------------------------------------------------------------
>
> Key: FALCON-1162
> URL: https://issues.apache.org/jira/browse/FALCON-1162
> Project: Falcon
> Issue Type: Bug
> Components: common
> Affects Versions: 0.6
> Reporter: Venkat Ramachandran
> Assignee: Venkat Ramachandran
> Priority: Blocker
> Fix For: 0.6.1
>
> Attachments: patch-1162.1
>
>
> Staging HDFS dir specified in the cluster definition should have WORLD
> WRITABLE permission. It seems Cluster entity submit used to validate this
> condition in order to avoid further runtime failures.
> But, this check had been reverted as part of the FALCON-910 commits (only
> checks for 755) as below:
> ClusterEntityParser.java
> {code}
> checkPathOwnerAndPermission(cluster.getName(),
> stagingLocation.getPath(), fs, HadoopClientFactory.READ_EXECUTE_PERMISSION,
> false);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
