[
https://issues.apache.org/jira/browse/FALCON-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108139#comment-15108139
]
Pallavi Rao commented on FALCON-1750:
-------------------------------------
Had an offline conversation with [~pragya.mittal] and learnt that the errors
occurred when the Oozie DB was cleaned up. That explains the error. When
running instances are removed from Oozie DB, and Falcon tries to contact Oozie
for job info, it is bound to fail. And hence, the context is not updated with
information required by downstream listeners.
The exception is warranted. One change we can make is log the original
(currently it is not) error, so it gives a context on the downstream error.
> Null Pointer Exception while listening to Workflow Notifications
> -----------------------------------------------------------------
>
> Key: FALCON-1750
> URL: https://issues.apache.org/jira/browse/FALCON-1750
> Project: Falcon
> Issue Type: Bug
> Components: general
> Affects Versions: 0.9
> Reporter: Pragya Mittal
>
> The following stacktrace is there in server logs when falcon post-processing
> is enabled :
> {noformat}
> 2016-01-14 15:20:31,728 INFO - [ActiveMQ Session Task-82:] ~ Created context
> from Oozie JMS message WorkflowExecutionContext{{workflowUser=pragya,
> entityType=PROCESS, entityName=processMerlinOozie,
> workflowEndTime=1452764965085, contextType=WORKFLOW_JOB, operation=GENERATE,
> workflowId=0000816-160113111840703-oozie-oozi-W, status=KILLED,
> workflowStartTime=1452764938610}} (JMSMessageConsumer:120)
> 2016-01-14 15:20:31,728 INFO - [ActiveMQ Session Task-82:] ~ Logging in
> pragya (CurrentUser:65)
> 2016-01-14 15:20:31,728 DEBUG - [ActiveMQ Session Task-82:] ~ Sending
> workflow end notification to listeners with context :
> WorkflowExecutionContext{{workflowUser=pragya, entityType=PROCESS,
> entityName=processMerlinOozie, workflowEndTime=1452764965085,
> contextType=WORKFLOW_JOB, operation=GENERATE,
> workflowId=0000816-160113111840703-oozie-oozi-W, status=KILLED,
> workflowStartTime=1452764938610}} (WorkflowJobEndNotificationService:230)
> 2016-01-14 15:20:31,728 ERROR - [ActiveMQ Session Task-82:] ~ Error in
> listener org.apache.falcon.handler.SLAMonitoringHandler
> (WorkflowJobEndNotificationService:245)
> java.lang.NullPointerException
> at
> org.apache.falcon.entity.v0.SchemaHelper.formatDateUTCToISO8601(SchemaHelper.java:64)
> at
> org.apache.falcon.workflow.WorkflowExecutionContext.getNominalTimeAsISO8601(WorkflowExecutionContext.java:182)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.instrumentAlert(WorkflowJobEndNotificationService.java:281)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:240)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyFailure(WorkflowJobEndNotificationService.java:101)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:215)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:121)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-14 15:20:31,729 ERROR - [ActiveMQ Session Task-82:] ~ Got Error in
> getting error codes from actions: org.apache.falcon.FalconException:
> java.lang.NullPointerException (WorkflowExecutionContext:144)
> 2016-01-14 15:20:31,729 ERROR - [ActiveMQ Session Task-82:] ~ Error in
> listener org.apache.falcon.rerun.handler.RetryHandler
> (WorkflowJobEndNotificationService:245)
> java.lang.NullPointerException
> at
> org.apache.falcon.entity.v0.SchemaHelper.formatDateUTCToISO8601(SchemaHelper.java:64)
> at
> org.apache.falcon.workflow.WorkflowExecutionContext.getNominalTimeAsISO8601(WorkflowExecutionContext.java:182)
> at
> org.apache.falcon.rerun.handler.RetryHandler.onFailure(RetryHandler.java:113)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:238)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyFailure(WorkflowJobEndNotificationService.java:101)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:215)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:121)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-14 15:20:31,729 ERROR - [ActiveMQ Session Task-82:] ~ Error in
> listener org.apache.falcon.rerun.handler.LateRerunHandler
> (WorkflowJobEndNotificationService:245)
> java.lang.NullPointerException
> at
> org.apache.falcon.entity.v0.SchemaHelper.formatDateUTCToISO8601(SchemaHelper.java:64)
> at
> org.apache.falcon.workflow.WorkflowExecutionContext.getNominalTimeAsISO8601(WorkflowExecutionContext.java:182)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.instrumentAlert(WorkflowJobEndNotificationService.java:281)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:240)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyFailure(WorkflowJobEndNotificationService.java:101)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:215)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:121)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-14 15:20:31,729 ERROR - [ActiveMQ Session Task-82:] ~ Error in
> listener org.apache.falcon.metadata.MetadataMappingService
> (WorkflowJobEndNotificationService:245)
> java.lang.NullPointerException
> at
> org.apache.falcon.entity.v0.SchemaHelper.formatDateUTCToISO8601(SchemaHelper.java:64)
> at
> org.apache.falcon.workflow.WorkflowExecutionContext.getNominalTimeAsISO8601(WorkflowExecutionContext.java:182)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.instrumentAlert(WorkflowJobEndNotificationService.java:281)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:240)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyFailure(WorkflowJobEndNotificationService.java:101)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:215)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:121)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-01-14 15:20:31,730 ERROR - [ActiveMQ Session Task-82:] ~ Error in
> listener org.apache.falcon.notification.service.impl.JobCompletionService
> (WorkflowJobEndNotificationService:245)
> java.lang.NullPointerException
> at
> org.apache.falcon.entity.v0.SchemaHelper.formatDateUTCToISO8601(SchemaHelper.java:64)
> at
> org.apache.falcon.workflow.WorkflowExecutionContext.getNominalTimeAsISO8601(WorkflowExecutionContext.java:182)
> at
> org.apache.falcon.notification.service.impl.JobCompletionService.constructCallbackID(JobCompletionService.java:171)
> at
> org.apache.falcon.notification.service.impl.JobCompletionService.onEnd(JobCompletionService.java:150)
> at
> org.apache.falcon.notification.service.impl.JobCompletionService.onFailure(JobCompletionService.java:131)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:238)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyFailure(WorkflowJobEndNotificationService.java:101)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:215)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:121)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1393)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
