HI all, it would be just great to have these options available:
Option 1: JMS based triggering of updates Option 2: Gateway pull mode with Option 2.1: cron-like timer definition within Gateway to pull updates Option 2.2: transient update -> Gateway exposes some API by itself to get triggered to pull updates (triggered some 3rd party) Why Option 2: there are setups with strict security policies in place which make it very hard or impossible to establish a JMS-connection. This can be found in "security zone" architectures with some Gateways being deployed in a higher secure zone than Publisher. Thanks, Jochen 2017-04-28 11:51 GMT+02:00 Nuwan Dias <nuw...@wso2.com>: > > > On Fri, Apr 28, 2017 at 2:10 PM, Manoj Gunawardena <man...@wso2.com> > wrote: > >> This is a suggestion, >> >> How about a REST API instead of a JMS service. >> >> GW nodes pull changes from APIMCore via REST service in APIMCore with >> configurable intervals. >> > >> The GW nodes fetch changes based on a timestamp (last fetch time) and api >> labels.(apis which hosted in that gw) >> The REST api will response meta data of changes as JSON payloads. >> >> The worry of connectivity breaking between core and GW is not with this >> mechanism. >> > > Well, I don't think intervals work. If an update is not reflected soon > enough, it becomes a frustrating user experience. So we have to keep this > interval to a small value. But in practice, API updates are very rare > (almost never happens in prod). So its a waste to have to keep running a > process on such a small interval. > >> >> WDYT? >> >> >> >> On Fri, Apr 28, 2017 at 1:29 PM, Lakmal Warusawithana <lak...@wso2.com> >> wrote: >> >>> Hi Kasun, >>> >>> On Fri, Apr 28, 2017 at 4:47 AM, Kasun Indrasiri <ka...@wso2.com> wrote: >>> >>>> Hi, >>>> >>>> Few questions related to the above design. >>>> >>>> >>>> - Here we are using pub-sub messaging to receive notification of the >>>> event of different API lifecycle states and use that to apply those changes >>>> (deploy, redeploy etc.) at the Ballerina service level. This is inherently >>>> subjective to lost the event notifications in case of a communication >>>> failure between the broker and GWs. For instance, if there is a >>>> communication failure, some API may not deploy/update on a given GW node (I >>>> doubt that we will face the same set of issues as we have in the old >>>> dep-sync approach). >>>> I'd rather think a push mechanism from the APIMCore to GW would be more >>>> solid compared to pub-sub. >>>> >>>> WDYT? >>>> >>> >>> Even push model APIMCore to some of gateway communication can be broken. >>> Then we need to replay that push messages otherwise they will not get them >>> at all. But why we want to have MB and the pub-sub is for reliable message >>> delivery. Second, this is very loosely couple and we can scale GWs without >>> updating the topology in the APIMCore. >>> >>> But, yes you have a point. Like NuwanD said, we may need to use durable >>> topics to handle the communication issues. I think we can have unique >>> identifiers for GWs and use in durable topic subscriptions. Only catch is >>> in container scenario, GWs can come and go very frequently and we need to >>> have some mechanism to clean these dead subscriptions. Hope from MB we may >>> have some way to clean them up. (or else we may need to have some clean up >>> process in the core) >>> >>> >>> >>>> >>>> From the Ballerina perspective: >>>> >>>> - A pre-built JMS service can subscribe to the respective topics >>>> (during the deployment time) and as the event updates are received it has >>>> to deploy (create/update new Ballerina service file?) the respective >>>> Ballerina service. However, I feel we need to clarify the above issues >>>> prior to the implementation. >>>> >>>> On Wed, Apr 26, 2017 at 9:58 PM, Thilini Shanika <thili...@wso2.com> >>>> wrote: >>>> >>>>> Hi Manoj, >>>>> >>>>> Please find my comments inline. >>>>> >>>>> Fetching new apis from APIM core do via database call or REST api? >>>>> >>>>> We are providing a set of REST apis (in API core) for ballerina >>>>> gateways to access and fetch API conf, subscription details etc. Following >>>>> diagram depicts how the communication happens in between the APIM core, >>>>> the >>>>> gateway, and the broker. >>>>> >>>>> >>>>> >>>>> I think, we have to think about time to fetch all apis and starting >>>>> time of a node. If num of apis increases, the startup time of a node may >>>>> increase. >>>>> This may critical for a containerized environments. >>>>> We found some production instances which has around 1000 apis. >>>>> >>>>> Yes. As per the new design, there is a plan of deploying APIs on >>>>> demand and this will eliminate the overhead of API deployment delays at >>>>> server startup. >>>>> >>>>> On Thu, Apr 27, 2017 at 9:58 AM, Manoj Gunawardena <man...@wso2.com> >>>>> wrote: >>>>> >>>>>> Fetching new apis from APIM core do via database call or REST api? >>>>>> I think, we have to think about time to fetch all apis and starting >>>>>> time of a node. If num of apis increases, the startup time of a node may >>>>>> increase. >>>>>> This may critical for a containerized environments. >>>>>> We found some production instances which has around 1000 apis. >>>>>> >>>>>> On Wed, Apr 26, 2017 at 1:56 PM, Sanjeewa Malalgoda < >>>>>> sanje...@wso2.com> wrote: >>>>>> >>>>>>> In fully automated containerized environment can we assume we can >>>>>>> see node restarts. As i understood nodes come and go(each time we >>>>>>> require >>>>>>> we start new instance and when its not in use it will terminate). In >>>>>>> such >>>>>>> scenario having durable subscription or persisting data to local node do >>>>>>> not make much difference. >>>>>>> >>>>>>> But if we are thinking of traditional deployment then we may be able >>>>>>> to restart instances and reuse them. >>>>>>> @Lakmal, Pubudu any thoughts? >>>>>>> >>>>>>> Thanks, >>>>>>> sanjeewa. >>>>>>> >>>>>>> On Wed, Apr 26, 2017 at 11:34 AM, Pubudu Gunatilaka < >>>>>>> pubu...@wso2.com> wrote: >>>>>>> >>>>>>>> On Wed, Apr 26, 2017 at 11:02 AM, Lakmal Warusawithana < >>>>>>>> lak...@wso2.com> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Apr 26, 2017 at 10:51 AM, Manoj Gunawardena < >>>>>>>>> man...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> Are we going to use durable subscription here? When ever gw node >>>>>>>>>> down and re start, >>>>>>>>>> it should fetch messages which add to the topic during the down >>>>>>>>>> time. >>>>>>>>>> In case durable subscription what is the mechanism to create >>>>>>>>>> client id for each gw.Is that discussed and confirm? >>>>>>>>>> >>>>>>>>> >>>>>>>>> No, we are not going to use durable topic. New gateways (scaling) >>>>>>>>> or if we restart a gateway, existing APIs will fetch via API Manager >>>>>>>>> core. >>>>>>>>> Only new API (after boot up) notifications are getting via the topic >>>>>>>>> and >>>>>>>>> actual API will fetch from the core. ( we can try what sanjeewa >>>>>>>>> mention - >>>>>>>>> getting API from the topic - but previous APIs need to fetch from the >>>>>>>>> core) >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> When we restart a gateway node, it would be better to fetch only >>>>>>>> the updated APIs as well as the new APIs. Restarted gateway has the >>>>>>>> already >>>>>>>> fetched APIs and it is not worth to fetch all the APIs again. We will >>>>>>>> have >>>>>>>> to use durable subscription for this. >>>>>>>> >>>>>>>> If we consider the container scenario, are we persisting gateway >>>>>>>> artifacts? If we are not persisting gateway artifacts and restart the >>>>>>>> gateway container, then we need to fetch all the APIs from the core. >>>>>>>> IMHO >>>>>>>> it would better to persist data and load that data at restart as it >>>>>>>> would >>>>>>>> take time to fetch all the APIs from the core. >>>>>>>> >>>>>>>> Thank you! >>>>>>>> -- >>>>>>>> *Pubudu Gunatilaka* >>>>>>>> Committer and PMC Member - Apache Stratos >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc.: http://wso2.com >>>>>>>> mobile : +94774078049 <%2B94772207163> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> Architecture@wso2.org >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *Sanjeewa Malalgoda* >>>>>>> WSO2 Inc. >>>>>>> Mobile : +94713068779 <+94%2071%20306%208779> >>>>>>> >>>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> Architecture@wso2.org >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Manoj Gunawardena >>>>>> Tech Lead >>>>>> WSO2, Inc.: http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> Mobile : +94 77 2291643 >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> Architecture@wso2.org >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thilini Shanika >>>>> Senior Software Engineer >>>>> WSO2, Inc.; http://wso2.com >>>>> 20, Palmgrove Avenue, Colombo 3 >>>>> >>>>> E-mail: tgtshan...@gmail.com >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> Architecture@wso2.org >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Kasun Indrasiri >>>> Director - Integration Architecture >>>> WSO2, Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> cell: +1 650 450 2293 <(650)%20450-2293> >>>> Blog : http://kasunpanorama.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> Lakmal Warusawithana >>> Director - Cloud Architecture; WSO2 Inc. >>> Mobile : +94714289692 <+94%2071%20428%209692> >>> Blogs : https://medium.com/@lakwarus/ >>> http://lakmalsview.blogspot.com/ >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Manoj Gunawardena >> Tech Lead >> WSO2, Inc.: http://wso2.com >> lean.enterprise.middleware >> Mobile : +94 77 2291643 >> > > > > -- > Nuwan Dias > > Software Architect - WSO2, Inc. http://wso2.com > email : nuw...@wso2.com > Phone : +94 777 775 729 <+94%2077%20777%205729> > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Gruss / regards Jochen Traunecker mailto: jochen.traunec...@gmail.com
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
