We need to maintain a single location to give publisher credential details. At the moment we are maintaining same details in a different location that is a bit inconvenient. As an example if we change BAM url then we have to change BAM server profiles as well as mediation stat publish UI as well. Also, this password should support secureVault.
On Fri, Oct 24, 2014 at 12:25 PM, Maninda Edirisooriya <mani...@wso2.com> wrote: > Hi all, > > Today morning we had a discussion where Anjana, Lahiru, Vijitha, Kasun and > I participated. > > There we decided to develop a common Data Publisher / Data Sink axis2 > artifact which is generic throughout the platform. This will enable any > other carbon products also for finding data publishing connection / > credential details when they want to publish data to CEP / BAM. > > And specific to the ESB we are going to develop a mediator (it's name is > not confirmed yet) which will be similar to the current BAM mediator which > will refer to the above mentioned artifact for credentials and connection > parameters. As also we should include the transport we are referring in > that particular data sink / data publisher. For the first cut we can use > only the Thrift and hope to extend for HTTP and JMS in future. > > In new implementation of the mediator, there will be no configuration for > Stream configuration like the BAM mediator. Instead there will be inline > configurations in the mediator XML which consists of > 1. a stream ID (or the combination of Stream Name and Stream Version) > 2. and mappings from message XPath to the stream field. (i.e. maps from > message properties to the stream fields.) > > For the convenience to the user, we can prompt the stream fields for > mapping from the UI, (once stream ID is provided) if the stream is already > there in the stream definition store. Anyway there (if want to get UI help) > we have to use the existing streams that were already created from CEP or > BAM. But still you can add a mapping for a new stream which can > subsequently be added to the stream definition store from CEP or BAM. > > We have decided to go for a mediator instead of a connector as this has > nothing to do for accessing an external API kind of operation but to > publish data for our own products according to the connection information > given by the internal Data Publisher / Data Sink artifact. > > WDYT about the cases when we use other transports like HTTP, JMS, MQTT > etc. ? > > Feedback is welcome. > > Thanks. > > > *Maninda Edirisooriya* > Senior Software Engineer > > *WSO2, Inc.*lean.enterprise.middleware. > > *Blog* : http://maninda.blogspot.com/ > *E-mail* : mani...@wso2.com > *Skype* : @manindae > *Twitter* : @maninda > > On Mon, Oct 20, 2014 at 8:00 PM, Sriskandarajah Suhothayan <s...@wso2.com> > wrote: > >> >> >> On Mon, Oct 20, 2014 at 4:06 AM, Mohanadarshan Vivekanandalingam < >> mo...@wso2.com> wrote: >> >>> Hi All, >>> >>> We had a discussion sometime back (last year) on this.. Please check the >>> mail thread [1] for more info.. >>> >>> >> Yes, I believe we all agree that we need to improve the current BAM >> Mediator. >> When I had a chat with kasun on this he suggested writing it as a >> connector will be much cleaner. >> >> Where we can have the connection information and stream definition in >> <init> and <defineStream> then then pass the data via <publish>. >> >> I believe its better to have a f2f meeting with kasun on this. >> >> Regards >> Suho >> >> [1] [Architecture] Improving Data Publisher Mediator to replace BAM >>> Mediator >>> >>> Thanks, >>> Mohan >>> >>> >>> On Mon, Oct 20, 2014 at 12:55 PM, Anjana Fernando <anj...@wso2.com> >>> wrote: >>> >>>> Hi Sanjiva, >>>> >>>> On Sat, Oct 18, 2014 at 9:44 PM, Sanjiva Weerawarana <sanj...@wso2.com> >>>> wrote: >>>> >>>>> On Fri, Oct 17, 2014 at 6:47 PM, Lahiru Chandima <lahi...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> We (Vijitha<vijit...@wso2.com> and myself) are going to develop an >>>>>> ESB Connector which can publish events to BAM and CEP. >>>>>> >>>>> >>>>> We're currently using the words "ESB Connector" to mean something that >>>>> becomes a set of mediators to access (primarily) HTTP APIs. This is not >>>>> that right?? If so lets not use the same words please. >>>>> >>>> >>>> Yeah, I guess, this is simply a mediator, just like what we have now. >>>> It was just Kasun has mentioned to Suho, to create this as an ESB >>>> Connector, where we haven't actually gone into what it means to create a >>>> connector. @Kasun, maybe you can shed some light on this, on what you meant >>>> earlier. >>>> >>>> >>>>> >>>>> Currently, BAM mediator is used for sending events from ESB to BAM. >>>>>> BAM Mediator needs to be configured with server details and streams need >>>>>> to >>>>>> be defined prior to using them in sequences of ESB Synapse configuration. >>>>>> >>>>>> With new ESB connector, it will be possible to add all configurations >>>>>> needed to publish events to BAM or CEP in the ESB Synapse configuration >>>>>> itself. This brings the configuration to a single place unlike when using >>>>>> the BAM mediator. >>>>>> >>>>> >>>>> Is this a real requirement? Do we expect different ESB configs >>>>> (running on the same server) to want to send data to different BAM/CEP >>>>> endpoints? I understand the multitenancy usecase - is it only for that? >>>>> >>>> >>>> Actually, the idea here is, what we have now for defining the target >>>> for events, "BAM Profiles", to replace that. Basically in a BAM Profile, we >>>> put all the information such as the event receiver details, and the stream >>>> definition, and how the properties are mapped using properties from a >>>> sequence. We want to clean this up by, simply having something equivalent >>>> to a message store we have now, where we named is "data sink" for now, to >>>> be the target for events, and the actual stream mapping will be done by the >>>> mediator itself, in the sequence. And the motivation for having this as an >>>> artifact in the synapse configuration is, where from a place like DevStudio >>>> or any development environment, we can create these deployment artifacts >>>> can directly deploy them, without going to the admin console and adding >>>> them through the UI. >>>> >>>> >>>>> >>>>> *High level architecture - components.* >>>>>> >>>>>> 1) New Synapse construct will be introduced to configure details >>>>>> related to the data sink (ie. CEP or BAM). >>>>>> >>>>>> Eg. >>>>>> >>>>>> <dataSink name=”bam_thrift_server”> >>>>>> <url>tcp://localhost:7611<url> >>>>>> <username>admin</username> >>>>>> <password>admin</password> >>>>>> </dataSink> >>>>>> >>>>> >>>>> What do we call this now? The term "dataSink" doesn't work for me but >>>>> that could be just me. >>>>> >>>> >>>> Yeah, we had a meeting sometime back to discuss this, and that day >>>> also, we couldn't decide on the name :) .. basically something related to a >>>> dataSink, any suggestions? :) .. >>>> >>>> >>>>> >>>>> 2) An ESB connector will be developed which can be used to publish >>>>>> events to a data sink. Connector will use data-bridge API (Thrift) for >>>>>> publishing events (It will be possible to introduce new transports other >>>>>> than Thrift later). >>>>>> >>>>> >>>>> Once again, what are the operations? Why do we want to write a >>>>> connector instead of using / improving the current "BAM mediator"? >>>>> >>>> >>>> So yeah, it will basically be improving/re-writing the current >>>> mediator, with a name like "dataPublish" or something, without any "BAM" >>>> part, and with the stream property mapping part added there itself, rather >>>> than having it in another place. >>>> >>>> >>>>> >>>>> Thrift stream related data (stream name, version and stream >>>>>> properties) will be configured in the ESB connector configuration (in >>>>>> Synapse config), which is more logical compared to configuring it >>>>>> separately as done in the BAM mediator. >>>>>> >>>>> >>>>> Can you give a specific example to illustrate the difference please? >>>>> >>>> >>>> So basically in the current BAM mediator, we do not give the direct >>>> mapping to the stream properties there, we simply populate some message >>>> context properties and call the mediator. And the mediator simply looks up >>>> the information provided by the BAM Profile to do the actual mapping, where >>>> in our profile, we have given like Xpath expressions to say, what are the >>>> values in a specific target stream event. >>>> >>>> Cheers, >>>> Anjana. >>>> >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Sanjiva. >>>>> -- >>>>> Sanjiva Weerawarana, Ph.D. >>>>> Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ >>>>> email: sanj...@wso2.com; office: (+1 650 745 4499 | +94 11 214 5345) >>>>> x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 265 8311 >>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>> Lean . Enterprise . Middleware >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> Architecture@wso2.org >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Anjana Fernando* >>>> Senior Technical Lead >>>> WSO2 Inc. | http://wso2.com >>>> lean . enterprise . middleware >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *V. Mohanadarshan* >>> *Software Engineer,* >>> *Data Technologies Team,* >>> *WSO2, Inc. http://wso2.com <http://wso2.com> * >>> *lean.enterprise.middleware.* >>> >>> email: mo...@wso2.com >>> phone:(+94) 771117673 >>> >> >> >> >> -- >> >> *S. Suhothayan* >> Technical Lead & Team Lead of WSO2 Complex Event Processor >> *WSO2 Inc. *http://wso2.com >> * <http://wso2.com/>* >> lean . enterprise . middleware >> >> >> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >> > > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
