[
https://issues.apache.org/jira/browse/OOZIE-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293213#comment-15293213
]
Sergey commented on OOZIE-2392:
-------------------------------
Assumptions:
h1. Definitions
*event* - a message with well defined structure.
*event type*, **TBD in details later**. event could be related to some entity:
action/workflow/coordinator. Event could be related to specific status:
something Failed, Succeeded. Event could be related to specific scope: Oozie
wide: server started/restarted/stopped. Coordinator wide: instance
materialized, coordinator suspended. There are many dimensions to consider.
*Event describes* something happening inside Oozie:
*Event provider* tool that integrates Oozie with outer system for event
publishing
*Oozie admin*, a guy who can modify oozie-site.xml and restart Oozie server
*Oozie user*, a guy who submits workflows/coordinators to Oozie
h1. Event Provider examples
*Kafka provider*, provider that can publish events to Kafka topic
*Redis provider*, it can push events to Redis list for example.
h1. Event Examples
* Workflow with id XXX has been started.
* Coordinator with id YYY has been started.
* Action with id ZZZ from workflow XXX has been finished successfully.
h1. Configuration
h2. Oozie-wide configuration.
* Oozie admin can configure providers in oozie-site.xml These providers could
be used by Oozie users. Oozie user can setup event provider for specific
workflow or coordinator.
* Ooize admin can enable events publishing for all running
workflows/coordinators
* Oozie admin can enable/disable publishing for specific event type/scope.
h2. Entity specific configuration
* Oozie user should be able to configure provider before firing
workflow/coordinator execution
* Oozie user should be able to override oozie-wide configuration before firing
workflow/coordinator execution
* Oozie user should be able to refer pre-configured providers and use them
* Oozie user should be able to enable/disable publishing for specific event
type/scope.
h1. Configuration example
h2. Server-wide
{code:xml}
<!-- define kafka in oozie-site.xml-->
<!-- It's server-side configuration. Workflow/Coordinator users can refer
provider by their names. -->
<configuration>
<property>
<name>oozie.publishing.event.provider.names</name>
<value>MyKafkaProvider,MyRedisProvider</value>
<description>
List names (aliases) for providers.
I'm going to define two providers that can work with Redis and Kafka
</description>
</property>
<property>
<name>oozie.publishing.event.provider.class.MyKafkaProvider</name>
<value>org.apache.oozie.publishing.event.provider.KafkaProvider</value>
<description>
class that implements integration with Kafka.
software design is TBD
</description>
</property>
<property>
<name>oozie.publishing.event.provider.configuration.MyKafkaProvider</name>
<value>
{
zookeeper: "xxx"
topic: "zzz"
}
</value>
<description>
Configuration could be complex, I would like to pass it as json as
one piece
</description>
</property>
<!-- Enable event publishing globally. Everything is turned off by default. -->
<property>
<name>oozie.publishing.event.provider.publishActionEvents.MyKafkaProvider</name>
<value>false</value>
<description>
false is default value.
Disable publishing oozie workflow action events using
MyKafkaProvider.
Oozie admin forced Oozie to publish all action events to
MyKafkaProvider
</description>
</property>
<property>
<name>oozie.publishing.event.provider.publishWorkflowEvents.MyKafkaProvider</name>
<value>true</value>
<description>
Enable event publishing for workflow events
</description>
</property>
<property>
<name>oozie.publishing.event.provider.publishCoordinatorEvents.MyKafkaProvider</name>
<value>true</value>
<description>
See above, for coordinator events
</description>
</property>
</configuration>
{code}
h2. workflow/coordinator configuration
{code:xml}
<configuration>
<property>
<name>oozie.publishing.event.provider.names</name>
<value>MyKafkaProvider</value>
<description>
You can pass the same configuration for event providers like in
oozie-site.xml
That provider would used for paerticualr workflow/coordinator run.
</description>
</property>
<property>
<name>oozie.publishing.event.provider.configuration.MyKafkaProvider.topic</name>
<value>other_topic</value>
<description>
It's possible to override configuration settings for provider.
Provider should be responsible for handling overrider configuration.
</description>
</property>
<property>
<name>oozie.publishing.event.provider.publishActionEvents.MyKafkaProvider</name>
<value>true</value>
<description>
Override global settings, enable event publishing for workflow
events for this particular workflow/coordinator run
</description>
</property>
</configuration>
{code}
> Add Kafka support for notifications
> -----------------------------------
>
> Key: OOZIE-2392
> URL: https://issues.apache.org/jira/browse/OOZIE-2392
> Project: Oozie
> Issue Type: New Feature
> Components: core
> Reporter: Robert Kanter
>
> Oozie currently lets you send JMS and email notifications. It could be
> useful to add support for Kafka as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
