Re: [Carbon-dev] Data Services Mediator for ESB
Hi Amila, On Sun, Jan 22, 2012 at 3:19 PM, Amila Suriarachchi wrote: > > yes I do understand that. That is why in real case users have to write a > .dbs file which is not shown here. What I am saying is that that may not be > the best way to do that when thinking > in a mediator perspective. > I see, the justification there is, you will get all the features of data services, when you use the existing .dbs file. To mirror all that functionality in-line in the mediator will be a bit tedious. Cheers, Anjana. > > thanks, > Amila. > >> >> Cheers, >> Anjana. >> >> >>> >>> But I also see some use case here as you have mentioned. Where users do >>> need to enrich some xml messages with additional data. (for an example if >>> ESB receives a customer Id this can be used to add all the customer >>> formation with his last orders etc.. which will be send to another >>> service). In this example[1] instead of using a data service we should have >>> used the data service mediator to do that. >>> >>> thanks, >>> Amila. >>> >>> [1] >>> http://wso2.org/library/articles/2012/01/integrating-different-systems-with-wso2-esb >>> >>> So the proposal was actually, not to necessarily ship this mediator by default with the ESB, but to make it a separate feature and to install it on-demand for anyone interested. Cheers, Anjana. > Thanks, > Hiranya > > > On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi > wrote: > >> >> >> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka > > wrote: >> >>> Hi Amila, >>> >>> Thanks for the clarifications and feedbacks. >>> >>> On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi >>> wrote: >>> I had a chat with Dinush and suggested to have some thing like this as in rule service. This is also similar to callout mediator calling a data service. Difference is not using the soap/Axis2/tcp layers. soapBody|soapHeader|$foo|$bar >>> resultXpath="">soapBody|soapHeader|$foo|$bar In this model data service is looked as some thing take an xml and outputs an xml. source is the way mediator finds the xml to invoke DS and target is the place it stores the result xml. >>> >>> Earlier what we have suggested is to map the input parameters one by >>> one using xpath or provide them as inline or map the whole set of >>> parameters @ once using xpath. In new suggested way, we have only above >>> last option and need to have the xml that can be directly used to call >>> operation or do some xslt transformation and make the incoming soap >>> body to >>> have that xml. >>> >> >> When you say using xpath then we need to think against which >> OMElement? that can be soapBody, soapHeader or an xml stored in a >> property. >> >> yes it is better to have parameter xpath support. I thought Data >> Serivice support that at the .dbs level. If not it may be more meaning >> full >> to add that the data service level. >> >> Can you please send a sample mediator configuration with the .dbs >> file for paramter xpath case? >> >> thanks, >> Amila. >> >> >>> The source can be soapBody, soapHeader or a property and we can get a portion of that using xpath. Then again when the result came back it can be saved in any place and a part of the result can be selected using resultXpath and can be stored in a location given by xpath. >>> >>> >>> Regards, >>> Dinusha. >>> thanks, Amila. On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi >>> > wrote: > > > On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka < > dinu...@wso2.com> wrote: > >> Hi All, >> >> We are going to develop a ESB mediator which can be shipped as a >> feature and once this feature is installed within ESB, the DS >> mediator can >> be used to make data services calls in-line, without making actual >> SOAP >> requests, but it will use in-memory calls to invoke data service >> operations. >> > > I think first you need to think the features this mediator going > provide for an ESB user and hence what type of interface expect from > the > mediator configuration. > For an example I am not seeing much relevance of service/operation > resource here. > Then think about the mediator configuration according to that and > implement. > > For an example Rules also has to support both as a web service and > a mediator. >
Re: [Carbon-dev] Data Services Mediator for ESB
On Sun, Jan 22, 2012 at 11:18 AM, Hiranya Jayathilaka wrote: > Hi, > > On Sat, Jan 21, 2012 at 11:54 PM, Anjana Fernando wrote: > >> Hi Hiranya, >> >> On Sat, Jan 21, 2012 at 6:37 PM, Hiranya Jayathilaka wrote: >> >>> I'm not entirely on board with the idea of running ESB functionality and >>> DSS functionality on the same box/JVM. From SOA perspective it is best to >>> keep these components separate as there could be several consumers who are >>> interested in accessing the data exposed by the data service. Also >>> performance wise the existing model is better because we can take advantage >>> of the non-blocking model of the ESB to invoke data services without much >>> overhead on the ESB. >>> >>> What is the exact problem you're trying to solve here? >>> >> >> I guess, maybe we didn't describe this properly :) .. The idea is not to >> deploy a data service as a web service in the ESB. But it's just a way to >> have something like extended DBReport/DBLookup functionality. >> > > My argument was from SOA perspective it's better to have data services as > a separate component exposed as 'web services' :) Also from a performance > point of view that relieves the ESB from making expensive JDBC calls. With > the current model server worker threads are released as soon as they invoke > the data services. But with the suggested mediator approach server worker > threads will do lot more work thus adding up to decreased throughput. > > >> It works in the same way, where the mediator execution comprises simply >> of in-VM calls, basically calling directly the data services API. The idea >> was to give more richer functionality, than the DB-* mediators we have now. >> In the current way, even to do some simple operations such as, reading >> multiple records from the database, you can't do that with DBLookup, but >> you'll have to call an external DSS sever to get it done, which may not be >> the ideal solution all the time, specially, if it makes no sense to have >> the data related operations as a service. So the proposal was actually, not >> to necessarily ship this mediator by default with the ESB, but to make it a >> separate feature and to install it on-demand for anyone interested. >> > > In a way this proposal is good because it provides some middle ground > between using data services and DB mediators. But OTOH it provides a third > option to access databases adding up to the existing confusion with two > options. > > I guess this is one of those things that we can argue forever pointing out > various pros and cons without coming to a conclusion :) So I'm ok with > having this as an optional feature. As Amila pointed out there will always > be some users who find this useful. But lets make this a point to properly > document the 3 approaches pointing out strengths, weaknesses and potential > use cases of each technique. > With the current ESB there are two ways to invoke data services. Using call out mediator or end point. If you use call out mediator then that will block the threads. So we can think of new approach as way of alternative for using call out mediator in terms of reducing system latency. Further the over head ESB have to do for send an XML message I think greater than to do a JDBC call. thanks, Amila. > Thanks, > Hiranya > > >> >> Cheers, >> Anjana. >> >> >>> Thanks, >>> Hiranya >>> >>> >>> On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: >>> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: > Hi Amila, > > Thanks for the clarifications and feedbacks. > > On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > >> I had a chat with Dinush and suggested to have some thing like this >> as in rule service. This is also similar to callout mediator >> calling a data service. Difference is not using the soap/Axis2/tcp >> layers. >> >> >> soapBody|soapHeader|$foo|$bar >> > resultXpath="">soapBody|soapHeader|$foo|$bar >> >> >> >> >> >> In this model data service is looked as some thing take an xml and >> outputs an xml. source is the way mediator finds the xml to >> invoke DS and target is the place it stores the result xml. >> > > Earlier what we have suggested is to map the input parameters one by > one using xpath or provide them as inline or map the whole set of > parameters @ once using xpath. In new suggested way, we have only above > last option and need to have the xml that can be directly used to call > operation or do some xslt transformation and make the incoming soap body > to > have that xml. > When you say using xpath then we need to think against which OMElement? that can be soapBody, soapHeader or an xml stored in a property. yes it is better to have parameter xpath support. I thought Data Serivice support that at the .dbs level. If not it may be mo
Re: [Carbon-dev] Data Services Mediator for ESB
On Sun, Jan 22, 2012 at 1:52 PM, Anjana Fernando wrote: > Hi Amila, > > On Sun, Jan 22, 2012 at 8:59 AM, Amila Suriarachchi wrote: > >> >> >> I think this is not a problem of data related operations. But for a >> mediator there is non concept of service and operations. It just take an >> xml and output an xml. So it is like invoking an operation directly. If you >> look at the rules service and mediator configuration this service and >> operations are there only with web service part. Rule mediator does not >> have them (Event architecturly there is no relationship with web service >> and mediator). >> > > I see the confusion there. The idea on using "operations" in the mediator > was to keep the original data service format intact. As in, to re-use the > same data service configuration format, so we won't have to create a new > separate configuration for that part. And in our current data service > configuration, what we have are operations/resources. So we can look it at > as an API, rather than a set of operations in a service. And what the > mediator does is, pick a certain operations from a set of operations in an > API, and call it. > yes I do understand that. That is why in real case users have to write a .dbs file which is not shown here. What I am saying is that that may not be the best way to do that when thinking in a mediator perspective. thanks, Amila. > > Cheers, > Anjana. > > >> >> But I also see some use case here as you have mentioned. Where users do >> need to enrich some xml messages with additional data. (for an example if >> ESB receives a customer Id this can be used to add all the customer >> formation with his last orders etc.. which will be send to another >> service). In this example[1] instead of using a data service we should have >> used the data service mediator to do that. >> >> thanks, >> Amila. >> >> [1] >> http://wso2.org/library/articles/2012/01/integrating-different-systems-with-wso2-esb >> >> >>> So the proposal was actually, not to necessarily ship this mediator by >>> default with the ESB, but to make it a separate feature and to install it >>> on-demand for anyone interested. >>> >>> Cheers, >>> Anjana. >>> >>> Thanks, Hiranya On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: > > > On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka > wrote: > >> Hi Amila, >> >> Thanks for the clarifications and feedbacks. >> >> On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi >> wrote: >> >>> I had a chat with Dinush and suggested to have some thing like this >>> as in rule service. This is also similar to callout mediator >>> calling a data service. Difference is not using the soap/Axis2/tcp >>> layers. >>> >>> >>> soapBody|soapHeader|$foo|$bar >>> >> resultXpath="">soapBody|soapHeader|$foo|$bar >>> >>> >>> >>> >>> >>> In this model data service is looked as some thing take an xml and >>> outputs an xml. source is the way mediator finds the xml to >>> invoke DS and target is the place it stores the result xml. >>> >> >> Earlier what we have suggested is to map the input parameters one by >> one using xpath or provide them as inline or map the whole set of >> parameters @ once using xpath. In new suggested way, we have only above >> last option and need to have the xml that can be directly used to call >> operation or do some xslt transformation and make the incoming soap body >> to >> have that xml. >> > > When you say using xpath then we need to think against which > OMElement? that can be soapBody, soapHeader or an xml stored in a > property. > > yes it is better to have parameter xpath support. I thought Data > Serivice support that at the .dbs level. If not it may be more meaning > full > to add that the data service level. > > Can you please send a sample mediator configuration with the .dbs file > for paramter xpath case? > > thanks, > Amila. > > >> >>> The source can be soapBody, soapHeader or a property and we can get >>> a portion of that using xpath. >>> Then again when the result came back it can be saved in any place >>> and a part of the result can be selected using resultXpath and can be >>> stored in a location given by xpath. >>> >> >> >> Regards, >> Dinusha. >> >>> >>> thanks, >>> Amila. >>> >>> >>> On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi >>> wrote: >>> On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka < dinu...@wso2.com> wrote: > Hi All, > > We are going to develop a ESB mediator which can be shipped as a > feature and once this feature is installed within ESB, the DS > mediator can > be
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Hiranya, On Sun, Jan 22, 2012 at 11:18 AM, Hiranya Jayathilaka wrote: > Hi, > > On Sat, Jan 21, 2012 at 11:54 PM, Anjana Fernando wrote: > >> Hi Hiranya, >> >> On Sat, Jan 21, 2012 at 6:37 PM, Hiranya Jayathilaka wrote: >> >>> I'm not entirely on board with the idea of running ESB functionality and >>> DSS functionality on the same box/JVM. From SOA perspective it is best to >>> keep these components separate as there could be several consumers who are >>> interested in accessing the data exposed by the data service. Also >>> performance wise the existing model is better because we can take advantage >>> of the non-blocking model of the ESB to invoke data services without much >>> overhead on the ESB. >>> >>> What is the exact problem you're trying to solve here? >>> >> >> I guess, maybe we didn't describe this properly :) .. The idea is not to >> deploy a data service as a web service in the ESB. But it's just a way to >> have something like extended DBReport/DBLookup functionality. >> > > My argument was from SOA perspective it's better to have data services as > a separate component exposed as 'web services' :) Also from a performance > point of view that relieves the ESB from making expensive JDBC calls. With > the current model server worker threads are released as soon as they invoke > the data services. But with the suggested mediator approach server worker > threads will do lot more work thus adding up to decreased throughput. > IMHO, the JDBC calls may not necessarily be as expensive as waiting for a remote web service call to go to a server and come back. The actual database operations will anyway be happening in a separate server. As I see, the overall latency of a service call to the ESB maybe cut down, if we didn't have a remote data service request. I know, the service based approach is much cleaner, looking from an SOA architecture perspective :) .. as you also concluded, we can just have this as an optional feature. Cheers, Anjana. > > >> It works in the same way, where the mediator execution comprises simply >> of in-VM calls, basically calling directly the data services API. The idea >> was to give more richer functionality, than the DB-* mediators we have now. >> In the current way, even to do some simple operations such as, reading >> multiple records from the database, you can't do that with DBLookup, but >> you'll have to call an external DSS sever to get it done, which may not be >> the ideal solution all the time, specially, if it makes no sense to have >> the data related operations as a service. So the proposal was actually, not >> to necessarily ship this mediator by default with the ESB, but to make it a >> separate feature and to install it on-demand for anyone interested. >> > > In a way this proposal is good because it provides some middle ground > between using data services and DB mediators. But OTOH it provides a third > option to access databases adding up to the existing confusion with two > options. > > I guess this is one of those things that we can argue forever pointing out > various pros and cons without coming to a conclusion :) So I'm ok with > having this as an optional feature. As Amila pointed out there will always > be some users who find this useful. But lets make this a point to properly > document the 3 approaches pointing out strengths, weaknesses and potential > use cases of each technique. > > Thanks, > Hiranya > > >> >> Cheers, >> Anjana. >> >> >>> Thanks, >>> Hiranya >>> >>> >>> On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: >>> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: > Hi Amila, > > Thanks for the clarifications and feedbacks. > > On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > >> I had a chat with Dinush and suggested to have some thing like this >> as in rule service. This is also similar to callout mediator >> calling a data service. Difference is not using the soap/Axis2/tcp >> layers. >> >> >> soapBody|soapHeader|$foo|$bar >> > resultXpath="">soapBody|soapHeader|$foo|$bar >> >> >> >> >> >> In this model data service is looked as some thing take an xml and >> outputs an xml. source is the way mediator finds the xml to >> invoke DS and target is the place it stores the result xml. >> > > Earlier what we have suggested is to map the input parameters one by > one using xpath or provide them as inline or map the whole set of > parameters @ once using xpath. In new suggested way, we have only above > last option and need to have the xml that can be directly used to call > operation or do some xslt transformation and make the incoming soap body > to > have that xml. > When you say using xpath then we need to think against which OMElement? that can be soapBody, soapHeader or an xml stored in a property. >
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Amila, On Sun, Jan 22, 2012 at 8:59 AM, Amila Suriarachchi wrote: > > > I think this is not a problem of data related operations. But for a > mediator there is non concept of service and operations. It just take an > xml and output an xml. So it is like invoking an operation directly. If you > look at the rules service and mediator configuration this service and > operations are there only with web service part. Rule mediator does not > have them (Event architecturly there is no relationship with web service > and mediator). > I see the confusion there. The idea on using "operations" in the mediator was to keep the original data service format intact. As in, to re-use the same data service configuration format, so we won't have to create a new separate configuration for that part. And in our current data service configuration, what we have are operations/resources. So we can look it at as an API, rather than a set of operations in a service. And what the mediator does is, pick a certain operations from a set of operations in an API, and call it. Cheers, Anjana. > > But I also see some use case here as you have mentioned. Where users do > need to enrich some xml messages with additional data. (for an example if > ESB receives a customer Id this can be used to add all the customer > formation with his last orders etc.. which will be send to another > service). In this example[1] instead of using a data service we should have > used the data service mediator to do that. > > thanks, > Amila. > > [1] > http://wso2.org/library/articles/2012/01/integrating-different-systems-with-wso2-esb > > >> So the proposal was actually, not to necessarily ship this mediator by >> default with the ESB, but to make it a separate feature and to install it >> on-demand for anyone interested. >> >> Cheers, >> Anjana. >> >> >>> Thanks, >>> Hiranya >>> >>> >>> On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: >>> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: > Hi Amila, > > Thanks for the clarifications and feedbacks. > > On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > >> I had a chat with Dinush and suggested to have some thing like this >> as in rule service. This is also similar to callout mediator >> calling a data service. Difference is not using the soap/Axis2/tcp >> layers. >> >> >> soapBody|soapHeader|$foo|$bar >> > resultXpath="">soapBody|soapHeader|$foo|$bar >> >> >> >> >> >> In this model data service is looked as some thing take an xml and >> outputs an xml. source is the way mediator finds the xml to >> invoke DS and target is the place it stores the result xml. >> > > Earlier what we have suggested is to map the input parameters one by > one using xpath or provide them as inline or map the whole set of > parameters @ once using xpath. In new suggested way, we have only above > last option and need to have the xml that can be directly used to call > operation or do some xslt transformation and make the incoming soap body > to > have that xml. > When you say using xpath then we need to think against which OMElement? that can be soapBody, soapHeader or an xml stored in a property. yes it is better to have parameter xpath support. I thought Data Serivice support that at the .dbs level. If not it may be more meaning full to add that the data service level. Can you please send a sample mediator configuration with the .dbs file for paramter xpath case? thanks, Amila. > >> The source can be soapBody, soapHeader or a property and we can get a >> portion of that using xpath. >> Then again when the result came back it can be saved in any place and >> a part of the result can be selected using resultXpath and can be stored >> in >> a location given by xpath. >> > > > Regards, > Dinusha. > >> >> thanks, >> Amila. >> >> >> On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi >> wrote: >> >>> >>> >>> On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka < >>> dinu...@wso2.com> wrote: >>> Hi All, We are going to develop a ESB mediator which can be shipped as a feature and once this feature is installed within ESB, the DS mediator can be used to make data services calls in-line, without making actual SOAP requests, but it will use in-memory calls to invoke data service operations. >>> >>> I think first you need to think the features this mediator going >>> provide for an ESB user and hence what type of interface expect from the >>> mediator configuration. >>> For an example I am not seeing much relevance of service/operation >>> resour
Re: [Carbon-dev] Data Services Mediator for ESB
On Sun, Jan 22, 2012 at 11:24 AM, Hiranya Jayathilaka wrote: > > > On Sun, Jan 22, 2012 at 8:59 AM, Amila Suriarachchi wrote: > >> >> >> On Sat, Jan 21, 2012 at 11:54 PM, Anjana Fernando wrote: >> >>> Hi Hiranya, >>> >>> On Sat, Jan 21, 2012 at 6:37 PM, Hiranya Jayathilaka >>> wrote: >>> I'm not entirely on board with the idea of running ESB functionality and DSS functionality on the same box/JVM. From SOA perspective it is best to keep these components separate as there could be several consumers who are interested in accessing the data exposed by the data service. Also performance wise the existing model is better because we can take advantage of the non-blocking model of the ESB to invoke data services without much overhead on the ESB. What is the exact problem you're trying to solve here? >>> >>> I guess, maybe we didn't describe this properly :) .. The idea is not to >>> deploy a data service as a web service in the ESB. But it's just a way to >>> have something like extended DBReport/DBLookup functionality. It works in >>> the same way, where the mediator execution comprises simply of in-VM calls, >>> basically calling directly the data services API. The idea was to give more >>> richer functionality, than the DB-* mediators we have now. In the current >>> way, even to do some simple operations such as, reading multiple records >>> from the database, you can't do that with DBLookup, but you'll have to call >>> an external DSS sever to get it done, which may not be the ideal solution >>> all the time, specially, if it makes no sense to have the data related >>> operations as a service. >>> >> >> I think this is not a problem of data related operations. But for a >> mediator there is non concept of service and operations. It just take an >> xml and output an xml. So it is like invoking an operation directly. If you >> look at the rules service and mediator configuration this service and >> operations are there only with web service part. Rule mediator does not >> have them (Event architecturly there is no relationship with web service >> and mediator). >> >> But I also see some use case here as you have mentioned. Where users do >> need to enrich some xml messages with additional data. (for an example if >> ESB receives a customer Id this can be used to add all the customer >> formation with his last orders etc.. which will be send to another >> service). In this example[1] instead of using a data service we should have >> used the data service mediator to do that. >> > > I prefer data services in this scenario for 2 reasons: > > 1. Batch operations are really expensive - Better to offload that overhead > from the ESB > 2. There could be other applications pumping orders into the same DB > you can think of reducing ESB over head vs Overall system over head (hence the system response time). As you have said usefulness depends on the use case and overall requirements. thanks, Amila. > > Thanks, > Hiranya > > >> >> thanks, >> Amila. >> >> [1] >> http://wso2.org/library/articles/2012/01/integrating-different-systems-with-wso2-esb >> >> >>> So the proposal was actually, not to necessarily ship this mediator by >>> default with the ESB, but to make it a separate feature and to install it >>> on-demand for anyone interested. >>> >>> Cheers, >>> Anjana. >>> >>> Thanks, Hiranya On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: > > > On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka > wrote: > >> Hi Amila, >> >> Thanks for the clarifications and feedbacks. >> >> On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi >> wrote: >> >>> I had a chat with Dinush and suggested to have some thing like this >>> as in rule service. This is also similar to callout mediator >>> calling a data service. Difference is not using the soap/Axis2/tcp >>> layers. >>> >>> >>> soapBody|soapHeader|$foo|$bar >>> >> resultXpath="">soapBody|soapHeader|$foo|$bar >>> >>> >>> >>> >>> >>> In this model data service is looked as some thing take an xml and >>> outputs an xml. source is the way mediator finds the xml to >>> invoke DS and target is the place it stores the result xml. >>> >> >> Earlier what we have suggested is to map the input parameters one by >> one using xpath or provide them as inline or map the whole set of >> parameters @ once using xpath. In new suggested way, we have only above >> last option and need to have the xml that can be directly used to call >> operation or do some xslt transformation and make the incoming soap body >> to >> have that xml. >> > > When you say using xpath then we need to think against which > OMElement? that can be soapBody, soapHeader or an xml stored in a > property. > > yes it is better to have para
Re: [Carbon-dev] Data Services Mediator for ESB
On Sun, Jan 22, 2012 at 8:59 AM, Amila Suriarachchi wrote: > > > On Sat, Jan 21, 2012 at 11:54 PM, Anjana Fernando wrote: > >> Hi Hiranya, >> >> On Sat, Jan 21, 2012 at 6:37 PM, Hiranya Jayathilaka wrote: >> >>> I'm not entirely on board with the idea of running ESB functionality and >>> DSS functionality on the same box/JVM. From SOA perspective it is best to >>> keep these components separate as there could be several consumers who are >>> interested in accessing the data exposed by the data service. Also >>> performance wise the existing model is better because we can take advantage >>> of the non-blocking model of the ESB to invoke data services without much >>> overhead on the ESB. >>> >>> What is the exact problem you're trying to solve here? >>> >> >> I guess, maybe we didn't describe this properly :) .. The idea is not to >> deploy a data service as a web service in the ESB. But it's just a way to >> have something like extended DBReport/DBLookup functionality. It works in >> the same way, where the mediator execution comprises simply of in-VM calls, >> basically calling directly the data services API. The idea was to give more >> richer functionality, than the DB-* mediators we have now. In the current >> way, even to do some simple operations such as, reading multiple records >> from the database, you can't do that with DBLookup, but you'll have to call >> an external DSS sever to get it done, which may not be the ideal solution >> all the time, specially, if it makes no sense to have the data related >> operations as a service. >> > > I think this is not a problem of data related operations. But for a > mediator there is non concept of service and operations. It just take an > xml and output an xml. So it is like invoking an operation directly. If you > look at the rules service and mediator configuration this service and > operations are there only with web service part. Rule mediator does not > have them (Event architecturly there is no relationship with web service > and mediator). > > But I also see some use case here as you have mentioned. Where users do > need to enrich some xml messages with additional data. (for an example if > ESB receives a customer Id this can be used to add all the customer > formation with his last orders etc.. which will be send to another > service). In this example[1] instead of using a data service we should have > used the data service mediator to do that. > I prefer data services in this scenario for 2 reasons: 1. Batch operations are really expensive - Better to offload that overhead from the ESB 2. There could be other applications pumping orders into the same DB Thanks, Hiranya > > thanks, > Amila. > > [1] > http://wso2.org/library/articles/2012/01/integrating-different-systems-with-wso2-esb > > >> So the proposal was actually, not to necessarily ship this mediator by >> default with the ESB, but to make it a separate feature and to install it >> on-demand for anyone interested. >> >> Cheers, >> Anjana. >> >> >>> Thanks, >>> Hiranya >>> >>> >>> On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: >>> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: > Hi Amila, > > Thanks for the clarifications and feedbacks. > > On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > >> I had a chat with Dinush and suggested to have some thing like this >> as in rule service. This is also similar to callout mediator >> calling a data service. Difference is not using the soap/Axis2/tcp >> layers. >> >> >> soapBody|soapHeader|$foo|$bar >> > resultXpath="">soapBody|soapHeader|$foo|$bar >> >> >> >> >> >> In this model data service is looked as some thing take an xml and >> outputs an xml. source is the way mediator finds the xml to >> invoke DS and target is the place it stores the result xml. >> > > Earlier what we have suggested is to map the input parameters one by > one using xpath or provide them as inline or map the whole set of > parameters @ once using xpath. In new suggested way, we have only above > last option and need to have the xml that can be directly used to call > operation or do some xslt transformation and make the incoming soap body > to > have that xml. > When you say using xpath then we need to think against which OMElement? that can be soapBody, soapHeader or an xml stored in a property. yes it is better to have parameter xpath support. I thought Data Serivice support that at the .dbs level. If not it may be more meaning full to add that the data service level. Can you please send a sample mediator configuration with the .dbs file for paramter xpath case? thanks, Amila. > >> The source can be soapBody, soapHeader or a property and we can get a >> portion of that us
Re: [Carbon-dev] Data Services Mediator for ESB
Hi, On Sat, Jan 21, 2012 at 11:54 PM, Anjana Fernando wrote: > Hi Hiranya, > > On Sat, Jan 21, 2012 at 6:37 PM, Hiranya Jayathilaka wrote: > >> I'm not entirely on board with the idea of running ESB functionality and >> DSS functionality on the same box/JVM. From SOA perspective it is best to >> keep these components separate as there could be several consumers who are >> interested in accessing the data exposed by the data service. Also >> performance wise the existing model is better because we can take advantage >> of the non-blocking model of the ESB to invoke data services without much >> overhead on the ESB. >> >> What is the exact problem you're trying to solve here? >> > > I guess, maybe we didn't describe this properly :) .. The idea is not to > deploy a data service as a web service in the ESB. But it's just a way to > have something like extended DBReport/DBLookup functionality. > My argument was from SOA perspective it's better to have data services as a separate component exposed as 'web services' :) Also from a performance point of view that relieves the ESB from making expensive JDBC calls. With the current model server worker threads are released as soon as they invoke the data services. But with the suggested mediator approach server worker threads will do lot more work thus adding up to decreased throughput. > It works in the same way, where the mediator execution comprises simply of > in-VM calls, basically calling directly the data services API. The idea was > to give more richer functionality, than the DB-* mediators we have now. In > the current way, even to do some simple operations such as, reading > multiple records from the database, you can't do that with DBLookup, but > you'll have to call an external DSS sever to get it done, which may not be > the ideal solution all the time, specially, if it makes no sense to have > the data related operations as a service. So the proposal was actually, not > to necessarily ship this mediator by default with the ESB, but to make it a > separate feature and to install it on-demand for anyone interested. > In a way this proposal is good because it provides some middle ground between using data services and DB mediators. But OTOH it provides a third option to access databases adding up to the existing confusion with two options. I guess this is one of those things that we can argue forever pointing out various pros and cons without coming to a conclusion :) So I'm ok with having this as an optional feature. As Amila pointed out there will always be some users who find this useful. But lets make this a point to properly document the 3 approaches pointing out strengths, weaknesses and potential use cases of each technique. Thanks, Hiranya > > Cheers, > Anjana. > > >> Thanks, >> Hiranya >> >> >> On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: >> >>> >>> >>> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: >>> Hi Amila, Thanks for the clarifications and feedbacks. On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > I had a chat with Dinush and suggested to have some thing like this as > in rule service. This is also similar to callout mediator > calling a data service. Difference is not using the soap/Axis2/tcp > layers. > > > soapBody|soapHeader|$foo|$bar > resultXpath="">soapBody|soapHeader|$foo|$bar > > > > > > In this model data service is looked as some thing take an xml and > outputs an xml. source is the way mediator finds the xml to > invoke DS and target is the place it stores the result xml. > Earlier what we have suggested is to map the input parameters one by one using xpath or provide them as inline or map the whole set of parameters @ once using xpath. In new suggested way, we have only above last option and need to have the xml that can be directly used to call operation or do some xslt transformation and make the incoming soap body to have that xml. >>> >>> When you say using xpath then we need to think against which OMElement? >>> that can be soapBody, soapHeader or an xml stored in a property. >>> >>> yes it is better to have parameter xpath support. I thought Data >>> Serivice support that at the .dbs level. If not it may be more meaning full >>> to add that the data service level. >>> >>> Can you please send a sample mediator configuration with the .dbs file >>> for paramter xpath case? >>> >>> thanks, >>> Amila. >>> >>> > The source can be soapBody, soapHeader or a property and we can get a > portion of that using xpath. > Then again when the result came back it can be saved in any place and > a part of the result can be selected using resultXpath and can be stored > in > a location given by xpath. > Regards, Dinusha. > > thanks, > Amila. > > > O
Re: [Carbon-dev] Data Services Mediator for ESB
On Sat, Jan 21, 2012 at 11:54 PM, Anjana Fernando wrote: > Hi Hiranya, > > On Sat, Jan 21, 2012 at 6:37 PM, Hiranya Jayathilaka wrote: > >> I'm not entirely on board with the idea of running ESB functionality and >> DSS functionality on the same box/JVM. From SOA perspective it is best to >> keep these components separate as there could be several consumers who are >> interested in accessing the data exposed by the data service. Also >> performance wise the existing model is better because we can take advantage >> of the non-blocking model of the ESB to invoke data services without much >> overhead on the ESB. >> >> What is the exact problem you're trying to solve here? >> > > I guess, maybe we didn't describe this properly :) .. The idea is not to > deploy a data service as a web service in the ESB. But it's just a way to > have something like extended DBReport/DBLookup functionality. It works in > the same way, where the mediator execution comprises simply of in-VM calls, > basically calling directly the data services API. The idea was to give more > richer functionality, than the DB-* mediators we have now. In the current > way, even to do some simple operations such as, reading multiple records > from the database, you can't do that with DBLookup, but you'll have to call > an external DSS sever to get it done, which may not be the ideal solution > all the time, specially, if it makes no sense to have the data related > operations as a service. > I think this is not a problem of data related operations. But for a mediator there is non concept of service and operations. It just take an xml and output an xml. So it is like invoking an operation directly. If you look at the rules service and mediator configuration this service and operations are there only with web service part. Rule mediator does not have them (Event architecturly there is no relationship with web service and mediator). But I also see some use case here as you have mentioned. Where users do need to enrich some xml messages with additional data. (for an example if ESB receives a customer Id this can be used to add all the customer formation with his last orders etc.. which will be send to another service). In this example[1] instead of using a data service we should have used the data service mediator to do that. thanks, Amila. [1] http://wso2.org/library/articles/2012/01/integrating-different-systems-with-wso2-esb > So the proposal was actually, not to necessarily ship this mediator by > default with the ESB, but to make it a separate feature and to install it > on-demand for anyone interested. > > Cheers, > Anjana. > > >> Thanks, >> Hiranya >> >> >> On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: >> >>> >>> >>> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: >>> Hi Amila, Thanks for the clarifications and feedbacks. On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > I had a chat with Dinush and suggested to have some thing like this as > in rule service. This is also similar to callout mediator > calling a data service. Difference is not using the soap/Axis2/tcp > layers. > > > soapBody|soapHeader|$foo|$bar > resultXpath="">soapBody|soapHeader|$foo|$bar > > > > > > In this model data service is looked as some thing take an xml and > outputs an xml. source is the way mediator finds the xml to > invoke DS and target is the place it stores the result xml. > Earlier what we have suggested is to map the input parameters one by one using xpath or provide them as inline or map the whole set of parameters @ once using xpath. In new suggested way, we have only above last option and need to have the xml that can be directly used to call operation or do some xslt transformation and make the incoming soap body to have that xml. >>> >>> When you say using xpath then we need to think against which OMElement? >>> that can be soapBody, soapHeader or an xml stored in a property. >>> >>> yes it is better to have parameter xpath support. I thought Data >>> Serivice support that at the .dbs level. If not it may be more meaning full >>> to add that the data service level. >>> >>> Can you please send a sample mediator configuration with the .dbs file >>> for paramter xpath case? >>> >>> thanks, >>> Amila. >>> >>> > The source can be soapBody, soapHeader or a property and we can get a > portion of that using xpath. > Then again when the result came back it can be saved in any place and > a part of the result can be selected using resultXpath and can be stored > in > a location given by xpath. > Regards, Dinusha. > > thanks, > Amila. > > > On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi wrote: > >> >> >> On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Hiranya, On Sat, Jan 21, 2012 at 6:37 PM, Hiranya Jayathilaka wrote: > I'm not entirely on board with the idea of running ESB functionality and > DSS functionality on the same box/JVM. From SOA perspective it is best to > keep these components separate as there could be several consumers who are > interested in accessing the data exposed by the data service. Also > performance wise the existing model is better because we can take advantage > of the non-blocking model of the ESB to invoke data services without much > overhead on the ESB. > > What is the exact problem you're trying to solve here? > I guess, maybe we didn't describe this properly :) .. The idea is not to deploy a data service as a web service in the ESB. But it's just a way to have something like extended DBReport/DBLookup functionality. It works in the same way, where the mediator execution comprises simply of in-VM calls, basically calling directly the data services API. The idea was to give more richer functionality, than the DB-* mediators we have now. In the current way, even to do some simple operations such as, reading multiple records from the database, you can't do that with DBLookup, but you'll have to call an external DSS sever to get it done, which may not be the ideal solution all the time, specially, if it makes no sense to have the data related operations as a service. So the proposal was actually, not to necessarily ship this mediator by default with the ESB, but to make it a separate feature and to install it on-demand for anyone interested. Cheers, Anjana. > Thanks, > Hiranya > > > On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: > >> >> >> On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: >> >>> Hi Amila, >>> >>> Thanks for the clarifications and feedbacks. >>> >>> On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: >>> I had a chat with Dinush and suggested to have some thing like this as in rule service. This is also similar to callout mediator calling a data service. Difference is not using the soap/Axis2/tcp layers. soapBody|soapHeader|$foo|$bar >>> resultXpath="">soapBody|soapHeader|$foo|$bar In this model data service is looked as some thing take an xml and outputs an xml. source is the way mediator finds the xml to invoke DS and target is the place it stores the result xml. >>> >>> Earlier what we have suggested is to map the input parameters one by one >>> using xpath or provide them as inline or map the whole set of parameters @ >>> once using xpath. In new suggested way, we have only above last option and >>> need to have the xml that can be directly used to call operation or do some >>> xslt transformation and make the incoming soap body to have that xml. >>> >> >> When you say using xpath then we need to think against which OMElement? >> that can be soapBody, soapHeader or an xml stored in a property. >> >> yes it is better to have parameter xpath support. I thought Data Serivice >> support that at the .dbs level. If not it may be more meaning full to add >> that the data service level. >> >> Can you please send a sample mediator configuration with the .dbs file >> for paramter xpath case? >> >> thanks, >> Amila. >> >> >>> The source can be soapBody, soapHeader or a property and we can get a portion of that using xpath. Then again when the result came back it can be saved in any place and a part of the result can be selected using resultXpath and can be stored in a location given by xpath. >>> >>> >>> Regards, >>> Dinusha. >>> thanks, Amila. On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi wrote: > > > On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka > wrote: > >> Hi All, >> >> We are going to develop a ESB mediator which can be shipped as a >> feature and once this feature is installed within ESB, the DS mediator >> can >> be used to make data services calls in-line, without making actual SOAP >> requests, but it will use in-memory calls to invoke data service >> operations. >> > > I think first you need to think the features this mediator going > provide for an ESB user and hence what type of interface expect from the > mediator configuration. > For an example I am not seeing much relevance of service/operation > resource here. > Then think about the mediator configuration according to that and > implement. > > For an example Rules also has to support both as a web service and a > mediator. > > The idea of rule mediator is to use it as > > 1. Transformer > 2. Rule based content routing > > If we take the mediation aspect there is not need to have the WS- > aspects. > For an example ws component only shift with BRS server and Mediation > component only shift with
Re: [Carbon-dev] Data Services Mediator for ESB
I'm not entirely on board with the idea of running ESB functionality and DSS functionality on the same box/JVM. From SOA perspective it is best to keep these components separate as there could be several consumers who are interested in accessing the data exposed by the data service. Also performance wise the existing model is better because we can take advantage of the non-blocking model of the ESB to invoke data services without much overhead on the ESB. What is the exact problem you're trying to solve here? Thanks, Hiranya On Sat, Jan 21, 2012 at 11:52 AM, Amila Suriarachchi wrote: > > > On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: > >> Hi Amila, >> >> Thanks for the clarifications and feedbacks. >> >> On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: >> >>> I had a chat with Dinush and suggested to have some thing like this as >>> in rule service. This is also similar to callout mediator >>> calling a data service. Difference is not using the soap/Axis2/tcp >>> layers. >>> >>> >>> soapBody|soapHeader|$foo|$bar >>> >> resultXpath="">soapBody|soapHeader|$foo|$bar >>> >>> >>> >>> >>> >>> In this model data service is looked as some thing take an xml and >>> outputs an xml. source is the way mediator finds the xml to >>> invoke DS and target is the place it stores the result xml. >>> >> >> Earlier what we have suggested is to map the input parameters one by one >> using xpath or provide them as inline or map the whole set of parameters @ >> once using xpath. In new suggested way, we have only above last option and >> need to have the xml that can be directly used to call operation or do some >> xslt transformation and make the incoming soap body to have that xml. >> > > When you say using xpath then we need to think against which OMElement? > that can be soapBody, soapHeader or an xml stored in a property. > > yes it is better to have parameter xpath support. I thought Data Serivice > support that at the .dbs level. If not it may be more meaning full to add > that the data service level. > > Can you please send a sample mediator configuration with the .dbs file for > paramter xpath case? > > thanks, > Amila. > > >> >>> The source can be soapBody, soapHeader or a property and we can get a >>> portion of that using xpath. >>> Then again when the result came back it can be saved in any place and a >>> part of the result can be selected using resultXpath and can be stored in a >>> location given by xpath. >>> >> >> >> Regards, >> Dinusha. >> >>> >>> thanks, >>> Amila. >>> >>> >>> On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi wrote: >>> On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka wrote: > Hi All, > > We are going to develop a ESB mediator which can be shipped as a > feature and once this feature is installed within ESB, the DS mediator can > be used to make data services calls in-line, without making actual SOAP > requests, but it will use in-memory calls to invoke data service > operations. > I think first you need to think the features this mediator going provide for an ESB user and hence what type of interface expect from the mediator configuration. For an example I am not seeing much relevance of service/operation resource here. Then think about the mediator configuration according to that and implement. For an example Rules also has to support both as a web service and a mediator. The idea of rule mediator is to use it as 1. Transformer 2. Rule based content routing If we take the mediation aspect there is not need to have the WS- aspects. For an example ws component only shift with BRS server and Mediation component only shift with ESB. For an instance in the transformation case it can either get the soapBody apply rules and send the result to soapBody (or some part given as an Xpath) thanks, Amila. > > So this will add the capability to have .dbs file in registry or some > other file location and invoke the data-service operations without > deploying the .dbs as a data-service and process the response within the > ESB. > > Possible mediator configuration will look as follows: > > > > > > > > > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > >>
Re: [Carbon-dev] Data Services Mediator for ESB
On Fri, Jan 20, 2012 at 3:00 PM, Dinusha Senanayaka wrote: > Hi Amila, > > Thanks for the clarifications and feedbacks. > > On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > >> I had a chat with Dinush and suggested to have some thing like this as in >> rule service. This is also similar to callout mediator >> calling a data service. Difference is not using the soap/Axis2/tcp layers. >> >> >> soapBody|soapHeader|$foo|$bar >> > resultXpath="">soapBody|soapHeader|$foo|$bar >> >> >> >> >> >> In this model data service is looked as some thing take an xml and >> outputs an xml. source is the way mediator finds the xml to >> invoke DS and target is the place it stores the result xml. >> > > Earlier what we have suggested is to map the input parameters one by one > using xpath or provide them as inline or map the whole set of parameters @ > once using xpath. In new suggested way, we have only above last option and > need to have the xml that can be directly used to call operation or do some > xslt transformation and make the incoming soap body to have that xml. > When you say using xpath then we need to think against which OMElement? that can be soapBody, soapHeader or an xml stored in a property. yes it is better to have parameter xpath support. I thought Data Serivice support that at the .dbs level. If not it may be more meaning full to add that the data service level. Can you please send a sample mediator configuration with the .dbs file for paramter xpath case? thanks, Amila. > >> The source can be soapBody, soapHeader or a property and we can get a >> portion of that using xpath. >> Then again when the result came back it can be saved in any place and a >> part of the result can be selected using resultXpath and can be stored in a >> location given by xpath. >> > > > Regards, > Dinusha. > >> >> thanks, >> Amila. >> >> >> On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi wrote: >> >>> >>> >>> On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka wrote: >>> Hi All, We are going to develop a ESB mediator which can be shipped as a feature and once this feature is installed within ESB, the DS mediator can be used to make data services calls in-line, without making actual SOAP requests, but it will use in-memory calls to invoke data service operations. >>> >>> I think first you need to think the features this mediator going provide >>> for an ESB user and hence what type of interface expect from the mediator >>> configuration. >>> For an example I am not seeing much relevance of service/operation >>> resource here. >>> Then think about the mediator configuration according to that and >>> implement. >>> >>> For an example Rules also has to support both as a web service and a >>> mediator. >>> >>> The idea of rule mediator is to use it as >>> >>> 1. Transformer >>> 2. Rule based content routing >>> >>> If we take the mediation aspect there is not need to have the WS- >>> aspects. >>> For an example ws component only shift with BRS server and Mediation >>> component only shift with ESB. >>> >>> For an instance in the transformation case it can either get the >>> soapBody apply rules and send the result to soapBody (or some part given as >>> an Xpath) >>> >>> thanks, >>> Amila. >>> >>> >>> >>> So this will add the capability to have .dbs file in registry or some other file location and invoke the data-service operations without deploying the .dbs as a data-service and process the response within the ESB. Possible mediator configuration will look as follows: ... Appreciate any feedback and ideas. Regards, Dinusha. ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>> >>> >>> -- >>> *Amila Suriarachchi* >>> >>> Software Architect >>> >>> WSO2 Inc. ; http://wso2.com >>> lean . enterprise . middleware >>> >>> phone : +94 71 3082805 >>> >>> >> >> >> -- >> *Amila Suriarachchi* >> >> Software Architect >> WSO2 Inc. ; http://wso2.com >> lean . enterprise . middleware >> >> phone : +94 71 3082805 >> >> >> ___ >> Carbon-dev mailing list >> Carbon-dev@wso2.org >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >> >> > >
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Amila, Thanks for the clarifications and feedbacks. On Fri, Jan 20, 2012 at 5:16 PM, Amila Suriarachchi wrote: > I had a chat with Dinush and suggested to have some thing like this as in > rule service. This is also similar to callout mediator > calling a data service. Difference is not using the soap/Axis2/tcp layers. > > > soapBody|soapHeader|$foo|$bar > resultXpath="">soapBody|soapHeader|$foo|$bar > > > > > > In this model data service is looked as some thing take an xml and outputs > an xml. source is the way mediator finds the xml to > invoke DS and target is the place it stores the result xml. > Earlier what we have suggested is to map the input parameters one by one using xpath or provide them as inline or map the whole set of parameters @ once using xpath. In new suggested way, we have only above last option and need to have the xml that can be directly used to call operation or do some xslt transformation and make the incoming soap body to have that xml. > > The source can be soapBody, soapHeader or a property and we can get a > portion of that using xpath. > Then again when the result came back it can be saved in any place and a > part of the result can be selected using resultXpath and can be stored in a > location given by xpath. > Regards, Dinusha. > > thanks, > Amila. > > > On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi wrote: > >> >> >> On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka wrote: >> >>> Hi All, >>> >>> We are going to develop a ESB mediator which can be shipped as a feature >>> and once this feature is installed within ESB, the DS mediator can be used >>> to make data services calls in-line, without making actual SOAP requests, >>> but it will use in-memory calls to invoke data service operations. >>> >> >> I think first you need to think the features this mediator going provide >> for an ESB user and hence what type of interface expect from the mediator >> configuration. >> For an example I am not seeing much relevance of service/operation >> resource here. >> Then think about the mediator configuration according to that and >> implement. >> >> For an example Rules also has to support both as a web service and a >> mediator. >> >> The idea of rule mediator is to use it as >> >> 1. Transformer >> 2. Rule based content routing >> >> If we take the mediation aspect there is not need to have the WS- aspects. >> For an example ws component only shift with BRS server and Mediation >> component only shift with ESB. >> >> For an instance in the transformation case it can either get the soapBody >> apply rules and send the result to soapBody (or some part given as an Xpath) >> >> thanks, >> Amila. >> >> >> >> >>> >>> So this will add the capability to have .dbs file in registry or some >>> other file location and invoke the data-service operations without >>> deploying the .dbs as a data-service and process the response within the >>> ESB. >>> >>> Possible mediator configuration will look as follows: >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> ... >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Appreciate any feedback and ideas. >>> >>> Regards, >>> Dinusha. >>> >>> ___ >>> Carbon-dev mailing list >>> Carbon-dev@wso2.org >>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>> >>> >> >> >> -- >> *Amila Suriarachchi* >> >> Software Architect >> >> WSO2 Inc. ; http://wso2.com >> lean . enterprise . middleware >> >> phone : +94 71 3082805 >> >> > > > -- > *Amila Suriarachchi* > > Software Architect > WSO2 Inc. ; http://wso2.com > lean . enterprise . middleware > > phone : +94 71 3082805 > > > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
I had a chat with Dinush and suggested to have some thing like this as in rule service. This is also similar to callout mediator calling a data service. Difference is not using the soap/Axis2/tcp layers. soapBody|soapHeader|$foo|$bar soapBody|soapHeader|$foo|$bar In this model data service is looked as some thing take an xml and outputs an xml. source is the way mediator finds the xml to invoke DS and target is the place it stores the result xml. The source can be soapBody, soapHeader or a property and we can get a portion of that using xpath. Then again when the result came back it can be saved in any place and a part of the result can be selected using resultXpath and can be stored in a location given by xpath. thanks, Amila. On Thu, Jan 19, 2012 at 7:25 PM, Amila Suriarachchi wrote: > > > On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka wrote: > >> Hi All, >> >> We are going to develop a ESB mediator which can be shipped as a feature >> and once this feature is installed within ESB, the DS mediator can be used >> to make data services calls in-line, without making actual SOAP requests, >> but it will use in-memory calls to invoke data service operations. >> > > I think first you need to think the features this mediator going provide > for an ESB user and hence what type of interface expect from the mediator > configuration. > For an example I am not seeing much relevance of service/operation > resource here. > Then think about the mediator configuration according to that and > implement. > > For an example Rules also has to support both as a web service and a > mediator. > > The idea of rule mediator is to use it as > > 1. Transformer > 2. Rule based content routing > > If we take the mediation aspect there is not need to have the WS- aspects. > For an example ws component only shift with BRS server and Mediation > component only shift with ESB. > > For an instance in the transformation case it can either get the soapBody > apply rules and send the result to soapBody (or some part given as an Xpath) > > thanks, > Amila. > > > > >> >> So this will add the capability to have .dbs file in registry or some >> other file location and invoke the data-service operations without >> deploying the .dbs as a data-service and process the response within the >> ESB. >> >> Possible mediator configuration will look as follows: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ... >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Appreciate any feedback and ideas. >> >> Regards, >> Dinusha. >> >> ___ >> Carbon-dev mailing list >> Carbon-dev@wso2.org >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >> >> > > > -- > *Amila Suriarachchi* > > Software Architect > > WSO2 Inc. ; http://wso2.com > lean . enterprise . middleware > > phone : +94 71 3082805 > > -- *Amila Suriarachchi* Software Architect WSO2 Inc. ; http://wso2.com lean . enterprise . middleware phone : +94 71 3082805 ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka wrote: > Hi All, > > We are going to develop a ESB mediator which can be shipped as a feature > and once this feature is installed within ESB, the DS mediator can be used > to make data services calls in-line, without making actual SOAP requests, > but it will use in-memory calls to invoke data service operations. > I think first you need to think the features this mediator going provide for an ESB user and hence what type of interface expect from the mediator configuration. For an example I am not seeing much relevance of service/operation resource here. Then think about the mediator configuration according to that and implement. For an example Rules also has to support both as a web service and a mediator. The idea of rule mediator is to use it as 1. Transformer 2. Rule based content routing If we take the mediation aspect there is not need to have the WS- aspects. For an example ws component only shift with BRS server and Mediation component only shift with ESB. For an instance in the transformation case it can either get the soapBody apply rules and send the result to soapBody (or some part given as an Xpath) thanks, Amila. > > So this will add the capability to have .dbs file in registry or some > other file location and invoke the data-service operations without > deploying the .dbs as a data-service and process the response within the > ESB. > > Possible mediator configuration will look as follows: > > > > > > > > > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > > > > > Appreciate any feedback and ideas. > > Regards, > Dinusha. > > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > -- *Amila Suriarachchi* Software Architect WSO2 Inc. ; http://wso2.com lean . enterprise . middleware phone : +94 71 3082805 placeOrder.rsl Description: Binary data http://wso2.org/carbon/rules"; backendRuntimeFactory="org.wso2.carbon.rule.backend.drools.DroolsBackendRuntimeFactory"> soapBody soapBody http://com.test/placeorder";> http://com.test/placeorder"; type="samples.userguide.PlaceOrder"> http://com.test/placeorder";> http://com.test/placeorder"; type="samples.userguide.OrderAccept"> http://com.test/placeorder"; type="samples.userguide.OrderReject"> ruledesign.pdf Description: Adobe PDF document ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Srinath, On Thu, Jan 19, 2012 at 9:01 AM, Srinath Perera wrote: > > I disagree as that forces the user to write two sets of codes when DS > and ESB are in two nodes. In generally, we try most of our constructs > work such a way that moving parts to a different node is a > configuration change rather than code change (e.g. Registry, IS etc). > IMO we should handle this case in the same manner. > Well .. the idea was to strictly not to use it in a manner to do remote calls, and do in-memory calls to data service operations. Anyways, if for some reason, a person would switch between the two, yeah, we can certainly just add that feature too. Cheers, Anjana. > > --Srinath > > > > > Cheers, > > Anjana. > >> > >> thanks, > >> waruna > >> > >> sent from my mobile. > >> > >> On Jan 18, 2012 5:04 PM, "Dinusha Senanayaka" wrote: > >>> > >>> Hi All, > >>> > >>> We are going to develop a ESB mediator which can be shipped as a > feature > >>> and once this feature is installed within ESB, the DS mediator can be > used > >>> to make data services calls in-line, without making actual SOAP > requests, > >>> but it will use in-memory calls to invoke data service operations. > >>> > >>> So this will add the capability to have .dbs file in registry or some > >>> other file location and invoke the data-service operations without > deploying > >>> the .dbs as a data-service and process the response within the ESB. > >>> > >>> Possible mediator configuration will look as follows: > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> ... > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> Appreciate any feedback and ideas. > >>> > >>> Regards, > >>> Dinusha. > >>> > >>> ___ > >>> Carbon-dev mailing list > >>> Carbon-dev@wso2.org > >>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > >>> > >> > >> ___ > >> Carbon-dev mailing list > >> Carbon-dev@wso2.org > >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > >> > > > > > > > > -- > > Anjana Fernando > > Senior Software Engineer > > WSO2 Inc. | http://wso2.com > > lean . enterprise . middleware > > > > ___ > > Carbon-dev mailing list > > Carbon-dev@wso2.org > > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > > > > > -- > > Srinath Perera, Ph.D. > Senior Software Architect, WSO2 Inc. > Visiting Faculty, University of Moratuwa > Member, Apache Software Foundation > Research Scientist, Lanka Software Foundation > Blog: http://srinathsview.blogspot.com/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > -- *Anjana Fernando* Senior Software Engineer WSO2 Inc. | http://wso2.com lean . enterprise . middleware ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Anjana, > This is suppose to be a possible replacement for DB-* mediators. Those > mediators also basically worked in the same way. The proposed data services > mediator is only suppose to work in in-memory calls. It won't make any > remote calls, basically it will not do any SOAP calls. What we do is, > internally create the data service representation and call it directly, same > in the case if the data service is already deployed as an Axis2 service, we > can get the required data service object from that service and invoke our > operations directly. Also as for the question made by Srinath, whether if > till do a SOAP call if the data service is in another node, no it won't. It > won't make sense to use this, then you can simply make an usual SOAP call, > using "send" or "callout" mediators. I disagree as that forces the user to write two sets of codes when DS and ESB are in two nodes. In generally, we try most of our constructs work such a way that moving parts to a different node is a configuration change rather than code change (e.g. Registry, IS etc). IMO we should handle this case in the same manner. --Srinath > > Cheers, > Anjana. >> >> thanks, >> waruna >> >> sent from my mobile. >> >> On Jan 18, 2012 5:04 PM, "Dinusha Senanayaka" wrote: >>> >>> Hi All, >>> >>> We are going to develop a ESB mediator which can be shipped as a feature >>> and once this feature is installed within ESB, the DS mediator can be used >>> to make data services calls in-line, without making actual SOAP requests, >>> but it will use in-memory calls to invoke data service operations. >>> >>> So this will add the capability to have .dbs file in registry or some >>> other file location and invoke the data-service operations without deploying >>> the .dbs as a data-service and process the response within the ESB. >>> >>> Possible mediator configuration will look as follows: >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> ... >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Appreciate any feedback and ideas. >>> >>> Regards, >>> Dinusha. >>> >>> ___ >>> Carbon-dev mailing list >>> Carbon-dev@wso2.org >>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>> >> >> ___ >> Carbon-dev mailing list >> Carbon-dev@wso2.org >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >> > > > > -- > Anjana Fernando > Senior Software Engineer > WSO2 Inc. | http://wso2.com > lean . enterprise . middleware > > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > -- Srinath Perera, Ph.D. Senior Software Architect, WSO2 Inc. Visiting Faculty, University of Moratuwa Member, Apache Software Foundation Research Scientist, Lanka Software Foundation Blog: http://srinathsview.blogspot.com/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902 ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Waruna, On Thu, Jan 19, 2012 at 8:47 AM, Waruna Ranasinghe wrote: > Now we have two methods to consume data services. What is the best > practice? > This is still not yet implemented/finalized btw :) .. > Won't this affect the non-blocking manner of Synspse, given that the > thread get blocked until response received from dataservice? > This is suppose to be a possible replacement for DB-* mediators. Those mediators also basically worked in the same way. The proposed data services mediator is only suppose to work in in-memory calls. It won't make any remote calls, basically it will not do any SOAP calls. What we do is, internally create the data service representation and call it directly, same in the case if the data service is already deployed as an Axis2 service, we can get the required data service object from that service and invoke our operations directly. Also as for the question made by Srinath, whether if till do a SOAP call if the data service is in another node, no it won't. It won't make sense to use this, then you can simply make an usual SOAP call, using "send" or "callout" mediators. Cheers, Anjana. > thanks, > waruna > > sent from my mobile. > On Jan 18, 2012 5:04 PM, "Dinusha Senanayaka" wrote: > >> Hi All, >> >> We are going to develop a ESB mediator which can be shipped as a feature >> and once this feature is installed within ESB, the DS mediator can be used >> to make data services calls in-line, without making actual SOAP requests, >> but it will use in-memory calls to invoke data service operations. >> >> So this will add the capability to have .dbs file in registry or some >> other file location and invoke the data-service operations without >> deploying the .dbs as a data-service and process the response within the >> ESB. >> >> Possible mediator configuration will look as follows: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ... >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Appreciate any feedback and ideas. >> >> Regards, >> Dinusha. >> >> ___ >> Carbon-dev mailing list >> Carbon-dev@wso2.org >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >> >> > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > -- *Anjana Fernando* Senior Software Engineer WSO2 Inc. | http://wso2.com lean . enterprise . middleware ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
Hi Srinath, Basically, for data services input parameters, what we support are scalar variables and arrays. In the future, we are going to support providing complex types as input (shown in the WSDL), but that will also be converted to a set of scalar variables to be mapped into data services parameters using xpath. So at the end, for the actual service call, we won't have complex values as such. So as you mentioned, actually array values can be given in the way as you suggested, where all the child elements from the xpath expression can be treated as array elements. Cheers, Anjana. On Thu, Jan 19, 2012 at 6:03 AM, Srinath Perera wrote: > Also if I am giving the value inline, how can I express the complex types? > > --Srinath > > On Thu, Jan 19, 2012 at 6:02 AM, Srinath Perera wrote: > > Couple of Qs > > > > if ESB and DS are two nodes, I assume this will do a SOAP call and still > work? > > > > if the parameter is a complex type in the ESB message, and if I refer > > to it though a XPath, will that be copied completely? > > > > for example, if the ESB message is > > > > > > > > > > > > > > ... > >. > > > > > > > > > > > > > > > > Now if I refer to this as .. will that > > copy all its subelements as well? > > > > --Srinath > > > > On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka > wrote: > >> Hi All, > >> > >> We are going to develop a ESB mediator which can be shipped as a > feature and > >> once this feature is installed within ESB, the DS mediator can be used > to > >> make data services calls in-line, without making actual SOAP requests, > but > >> it will use in-memory calls to invoke data service operations. > >> > >> So this will add the capability to have .dbs file in registry or some > other > >> file location and invoke the data-service operations without deploying > the > >> .dbs as a data-service and process the response within the ESB. > >> > >> Possible mediator configuration will look as follows: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> ... > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> Appreciate any feedback and ideas. > >> > >> Regards, > >> Dinusha. > >> > >> ___ > >> Carbon-dev mailing list > >> Carbon-dev@wso2.org > >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > >> > > > > > > > > -- > > > > Srinath Perera, Ph.D. > >http://www.cs.indiana.edu/~hperera/ > >http://srinathsview.blogspot.com/ > > > > -- > > Srinath Perera, Ph.D. >http://www.cs.indiana.edu/~hperera/ >http://srinathsview.blogspot.com/ > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > -- *Anjana Fernando* Senior Software Engineer WSO2 Inc. | http://wso2.com lean . enterprise . middleware ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
Now we have two methods to consume data services. What is the best practice? Won't this affect the non-blocking manner of Synspse, given that the thread get blocked until response received from dataservice? thanks, waruna sent from my mobile. On Jan 18, 2012 5:04 PM, "Dinusha Senanayaka" wrote: > Hi All, > > We are going to develop a ESB mediator which can be shipped as a feature > and once this feature is installed within ESB, the DS mediator can be used > to make data services calls in-line, without making actual SOAP requests, > but it will use in-memory calls to invoke data service operations. > > So this will add the capability to have .dbs file in registry or some > other file location and invoke the data-service operations without > deploying the .dbs as a data-service and process the response within the > ESB. > > Possible mediator configuration will look as follows: > > > > > > > > > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > > > > > Appreciate any feedback and ideas. > > Regards, > Dinusha. > > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
Also if I am giving the value inline, how can I express the complex types? --Srinath On Thu, Jan 19, 2012 at 6:02 AM, Srinath Perera wrote: > Couple of Qs > > if ESB and DS are two nodes, I assume this will do a SOAP call and still work? > > if the parameter is a complex type in the ESB message, and if I refer > to it though a XPath, will that be copied completely? > > for example, if the ESB message is > > > > > > > ... > . > > > > > > > > Now if I refer to this as .. will that > copy all its subelements as well? > > --Srinath > > On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka wrote: >> Hi All, >> >> We are going to develop a ESB mediator which can be shipped as a feature and >> once this feature is installed within ESB, the DS mediator can be used to >> make data services calls in-line, without making actual SOAP requests, but >> it will use in-memory calls to invoke data service operations. >> >> So this will add the capability to have .dbs file in registry or some other >> file location and invoke the data-service operations without deploying the >> .dbs as a data-service and process the response within the ESB. >> >> Possible mediator configuration will look as follows: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ... >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Appreciate any feedback and ideas. >> >> Regards, >> Dinusha. >> >> ___ >> Carbon-dev mailing list >> Carbon-dev@wso2.org >> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >> > > > > -- > > Srinath Perera, Ph.D. > http://www.cs.indiana.edu/~hperera/ > http://srinathsview.blogspot.com/ -- Srinath Perera, Ph.D. http://www.cs.indiana.edu/~hperera/ http://srinathsview.blogspot.com/ ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
Re: [Carbon-dev] Data Services Mediator for ESB
Couple of Qs if ESB and DS are two nodes, I assume this will do a SOAP call and still work? if the parameter is a complex type in the ESB message, and if I refer to it though a XPath, will that be copied completely? for example, if the ESB message is ... . Now if I refer to this as .. will that copy all its subelements as well? --Srinath On Wed, Jan 18, 2012 at 5:03 PM, Dinusha Senanayaka wrote: > Hi All, > > We are going to develop a ESB mediator which can be shipped as a feature and > once this feature is installed within ESB, the DS mediator can be used to > make data services calls in-line, without making actual SOAP requests, but > it will use in-memory calls to invoke data service operations. > > So this will add the capability to have .dbs file in registry or some other > file location and invoke the data-service operations without deploying the > .dbs as a data-service and process the response within the ESB. > > Possible mediator configuration will look as follows: > > > > > > > > > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > > > > > Appreciate any feedback and ideas. > > Regards, > Dinusha. > > ___ > Carbon-dev mailing list > Carbon-dev@wso2.org > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > -- Srinath Perera, Ph.D. http://www.cs.indiana.edu/~hperera/ http://srinathsview.blogspot.com/ ___ Carbon-dev mailing list Carbon-dev@wso2.org http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
