What do you think of this idea for a simple adaptor: - Adaptor extends App - Adaptor can send events but not receive (for now) - Adaptor is deployed as a regular App to the S4 cluster and as an Adaptor type in a host (separate from the S4 cluster). - Adaptor, unlike regular apps, can accept event data (in any format) directly, not via comm layer. - Input data is transformed into S4 events using a modular approach and by providing standard modules such as JSON. - Output events are exposed using EventSource and consumed by other apps without even knowing that they are Adaptors (only the App type is exposed in the cluster). - S4 events can be processed locally using PEs and Streams as usual. (We kind of need to get a local Sender for the local PEs and a standard cluster Sender for the EventSource object.)
So why this approach? The GOOD: - Seems to be the least disruptive way to inject external events - Apps can easily consume the events in a modular way without any dependencies. Getting events from an adaptor or from another app is identical. - The adaptor would be packaged and deployed to the cluster as if it was an App (no incremental cost) - The adaptor can do preprocessing using the same programming model and can reuse PEs. The CHALLENGE: - We need to also deploy the Adaptor in a separate host. On the other hand, this is inevitable. At least we use the same approach instead of creating a different system. - The Adaptor will need to be integrated with ZK to get the physical addresses. - We need to deal with two senders. for later: two-way communication and adapter clusters. thoughts? -- -leo
