It is a great idea. It can accumulate external events and the reuse of PE will be helpful.
-- Mohit On Thu, Oct 20, 2011 at 6:25 AM, Leo Neumeyer <[email protected]> wrote: > What do you think of this idea for a simple adaptor: > > - Adaptor extends App > - Adaptor can send events but not receive (for now) > - Adaptor is deployed as a regular App to the S4 cluster and as an > Adaptor type in a host (separate from the S4 cluster). > - Adaptor, unlike regular apps, can accept event data (in any format) > directly, not via comm layer. > - Input data is transformed into S4 events using a modular approach > and by providing standard modules such as JSON. > - Output events are exposed using EventSource and consumed by other > apps without even knowing that they are Adaptors (only the App type is > exposed in the cluster). > - S4 events can be processed locally using PEs and Streams as usual. > (We kind of need to get a local Sender for the local PEs and a > standard cluster Sender for the EventSource object.) > > So why this approach? > > The GOOD: > - Seems to be the least disruptive way to inject external events > - Apps can easily consume the events in a modular way without any > dependencies. Getting events from an adaptor or from another app is > identical. > - The adaptor would be packaged and deployed to the cluster as if it > was an App (no incremental cost) > - The adaptor can do preprocessing using the same programming model > and can reuse PEs. > > The CHALLENGE: > - We need to also deploy the Adaptor in a separate host. On the other > hand, this is inevitable. At least we use the same approach instead of > creating a different system. > - The Adaptor will need to be integrated with ZK to get the physical > addresses. > - We need to deal with two senders. > > for later: two-way communication and adapter clusters. > > thoughts? > > -- > > -leo >
