[
https://issues.apache.org/jira/browse/FALCON-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498786#comment-14498786
]
Balu Vellanki edited comment on FALCON-1153 at 4/16/15 11:12 PM:
-----------------------------------------------------------------
[~sriksun] I increased the default count to 30, and will pick up the count
from runtime.properties "scheduler.action.status.retry.count" if this property
is defined. The delay is kept at 100ms. Since there is no good location to
document this property, the documentation for this property will be added when
https://issues.apache.org/jira/browse/FALCON-778 is completed.
[~sriksun] [~ajayydv] Attached is the new patch, please review.
was (Author: bvellanki):
[~sriksun] I increased the default count to 30, and will pick up the count
from runtime.properties "oozie.action.retry.count" if this property is defined.
The delay is kept at 100ms. Since there is no good location to document this
property, the documentation for this property will be added when
https://issues.apache.org/jira/browse/FALCON-778 is completed.
[~sriksun] [~ajayydv] Attached is the new patch, please review.
> Falcon kill on /instance/kill/feed fails intermittently
> -------------------------------------------------------
>
> Key: FALCON-1153
> URL: https://issues.apache.org/jira/browse/FALCON-1153
> Project: Falcon
> Issue Type: Bug
> Components: webapp
> Affects Versions: 0.6
> Reporter: Balu Vellanki
> Assignee: Balu Vellanki
> Priority: Critical
> Fix For: 0.6.1
>
> Attachments: FALCON-1153-v1.patch, FALCON-1153.patch
>
>
> Kill operation fails with response:
> {code:xml}
> <?xml version="1.0" encoding="UTF-8"?>
> <instancesResult>
> <status>FAILED</status>
> <message>default/KILL</message>
> <requestId>default/null</requestId>
> <instances>
> <instance>2015-03-12T15:38Z</instance>
> <status>ERROR</status>
>
> <logFile>http://MAINT22-FCN12:11000/oozie?job=0000135-150312134223103-oozie-hado-W</logFile>
> <cluster>corp-34ce5f65</cluster>
> <sourceCluster>corp-1b70e962</sourceCluster>
> <startTime>2015-03-12T16:26:10Z</startTime>
> <details />
> </instances>
> </instancesResult>
> {code}
> Stacktrace from falcon.application.log:
> {code}
> 2015-03-12 16:26:19,506 WARN -
> [941235624@qtp-2073066709-135:d88a0d21-00c0-4646-8f05-ee606c6170a5
> hadoopqa:POST//instance/kill/feed/raaw-logs16-ac0c9860/] ~ Unable to perform
> action KILL on cluster (OozieWorkflowEngine:577)
> org.apache.falcon.FalconException: For
> Job0000135-150312134223103-oozie-hado-W, actual statuses: RUNNING, expected
> statuses: [KILLED, SUCCEEDED, FAILED]
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.assertStatus(OozieWorkflowEngine.java:1341)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.kill(OozieWorkflowEngine.java:1424)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.performAction(OozieWorkflowEngine.java:757)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.doJobAction(OozieWorkflowEngine.java:575)
> at
> org.apache.falcon.workflow.engine.OozieWorkflowEngine.killInstances(OozieWorkflowEngine.java:495)
> at
> org.apache.falcon.resource.AbstractInstanceManager.killInstance(AbstractInstanceManager.java:392)
> at
> org.apache.falcon.resource.InstanceManager.killInstance(InstanceManager.java:180)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.falcon.resource.channel.IPCChannel.invoke(IPCChannel.java:49)
> at
> org.apache.falcon.resource.proxy.InstanceManagerProxy$8.doExecute(InstanceManagerProxy.java:265)
> at
> org.apache.falcon.resource.proxy.InstanceManagerProxy$8.doExecute(InstanceManagerProxy.java:1)
> at
> org.apache.falcon.resource.proxy.InstanceManagerProxy$InstanceProxy.execute(InstanceManagerProxy.java:357)
> at
> org.apache.falcon.resource.proxy.InstanceManagerProxy.killInstance_aroundBody14(InstanceManagerProxy.java:268)
> at
> org.apache.falcon.resource.proxy.InstanceManagerProxy$AjcClosure15.run(InstanceManagerProxy.java:1)
> at
> org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> at
> org.apache.falcon.aspect.AbstractFalconAspect.logAroundMonitored(AbstractFalconAspect.java:51)
> at
> org.apache.falcon.resource.proxy.InstanceManagerProxy.killInstance(InstanceManagerProxy.java:252)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.falcon.security.FalconAuthorizationFilter.doFilter(FalconAuthorizationFilter.java:102)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.falcon.security.FalconAuthenticationFilter$2.doFilter(FalconAuthenticationFilter.java:184)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:572)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:542)
> at
> org.apache.falcon.security.FalconAuthenticationFilter.doFilter(FalconAuthenticationFilter.java:193)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.falcon.security.FalconAuditFilter.doFilter(FalconAuditFilter.java:65)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
