These are the Improvements identified during the Design Review; - UI for specifying throttling limits will be moved to Manage Page. - Rather than providing two text fields to specify Unit Time and MaxRequest count, there'd be single filed to specify Transactions Per Second. (This is done by defaulting Unit Time to one second). But users would still be able to change the Unit Time by directly editing the synapse config. - Hard Throttling will be last to apply while engaging throttling. - Throttled Out message will be modified to make a distinction between requests that have been throttled out by User-level throttling and requests that have been rejected as a result of exceeding the hard limit. HTTP Status Code 429 will be sent for requests throttled out by User-Throttling and 503 for those throttled out by Hard Throttling. - While publishing details about throttled out requests, throttled out reason will also be published. A new version of the throttled out stream will be created.
On Wed, Sep 16, 2015 at 8:03 PM, Amila De Silva <ami...@wso2.com> wrote: > For displaying at Publisher, throttling limits have to persisted on a DB > in some format (either in the registry artifact, in the swagger definition > or in AM_API table). But at the runtime, values will be picked from the > synapse definition. > > Plan is to enforce the hard limit on a particular version of the API. For > example if there are two APIs having two versions v1 and v2, the hard limit > defined on v1 only ensures requests flowing through v1 version doesn't > exceed the specified limit. > > On Wed, Sep 16, 2015 at 5:11 PM, Amila De Silva <ami...@wso2.com> wrote: > >> >> >> On Wed, Sep 16, 2015 at 5:08 PM, Sanjeewa Malalgoda <sanje...@wso2.com> >> wrote: >> >>> Yes what you said have value actually. >>> But wont it make things complicated? For the initial implementation we >>> may go with configuration per API/Endpoint. >>> May be later we need to implement something to have multiple back end >>> services for same API(endpoint per resource). >>> And at that point we need to have something like you mentioned. >>> >>> And did we checked DMZ/MZ problem here. >>> What do you mean by having max count inside API. Is it meant in synapse >>> configuration? >>> >> Yes, as two properties of APIThrottleHandler. >> >>> >>> Thanks, >>> sanjeewa. >>> >>> On Wed, Sep 16, 2015 at 2:11 PM, Amila De Silva <ami...@wso2.com> wrote: >>> >>>> >>>> >>>> On Wed, Sep 16, 2015 at 1:42 PM, Amila De Silva <ami...@wso2.com> >>>> wrote: >>>> >>>>> >>>>> On Wed, Sep 9, 2015 at 11:47 AM, Sanjeewa Malalgoda <sanje...@wso2.com >>>>> > wrote: >>>>> >>>>>> I also believe we should use existing handler without writing new one. >>>>>> >>>>>> And regarding registry decoupling, publisher may be able to push API >>>>>> to gateway as we do now. >>>>>> And just before we call rest API admin service we may call registry >>>>>> admin service(may be we need to write simple service for this as flow >>>>>> continue with super admin - super tenant deployed multi tenanted service) >>>>>> and push tiers.xml file. >>>>>> With that we may keep registry separation between gateway and >>>>>> publisher. >>>>>> That would be valuable when we have MZ and DMZ pattern and gateway in >>>>>> DMZ. >>>>>> >>>>>> Thanks, >>>>>> sanjeewa. >>>>>> >>>>>> >>>>>> On Tue, Sep 8, 2015 at 11:11 PM, Amila De Silva <ami...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Nuwan, >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 8, 2015 at 9:58 PM, Nuwan Dias <nuw...@wso2.com> wrote: >>>>>>> >>>>>>>> This design brings on a hard (mandatory) dependency to the registry >>>>>>>> from the API Gateway right? If each API is to have its own policy >>>>>>>> definition, the publisher and gateway must connect to a single >>>>>>>> registry. >>>>>>>> This will cause problems in some deployment patterns which have the >>>>>>>> Gateway >>>>>>>> in DMZ and publisher in the LAN. Can our throttling engine work if we >>>>>>>> just >>>>>>>> feed it the request count and unit time without feeding it a policy >>>>>>>> definition? >>>>>>>> >>>>>>> >>>>>>> Not in a very straightforward way. The API exposed by throttle.core >>>>>>> only allows passing ThrottleContext, a key and the Tier name. When going >>>>>>> through the usages of ThrottleContext it seems that it can only be >>>>>>> created >>>>>>> out from a policy file. But need to check if it's possible to create >>>>>>> ThrottleContext only using unit time and MaxRequest count. >>>>>>> >>>>>>>> If that works then we can survive without the registry for this. >>>>>>>> >>>>>>> We can keep max request count and unit time in the API itself. While >>>>> initialising the APIThrottleHandler, a policy.xml has to be created out of >>>>> these values and the generated policy will be used to create the >>>>> ThrottleContext. Dynamic policy won't be stored anywhere, so when moving >>>>> the API from one environment to another, there won't be a risk of losing >>>>> anything. >>>>> >>>>> For the scope of this feature, should be consider about providing >>>>> capability to specify hard throttling limits per each resource verb >>>>> combination. >>>>> >>>> >>>> Accidentally sent the mail halfway :) >>>> >>>> What I was going to highlight was need of having different hard limits >>>> for each resource + verb combination. >>>> >>>> Not every resource of an API, consume the same amount of server >>>> resources. For example, when sending response to an OPTIONS call, the >>>> backend might be sending a cached (or a static) response, whilst when >>>> responding to a POST, it might be doing several DB calls back and forth. >>>> Actually, it might be an overflow of POST requests that can take the >>>> back-end down. So while specifying a quota, heavy weight resources should >>>> be given less quota, and lightweight (but more frequently accessed >>>> resources) should be given a higher quota. >>>> >>>> Another option is to use the same tiers.xml that is being used for >>>>>>> other throttling cases and refer to a role defined there. If needed to >>>>>>> update/add a new tier it will have to be done through tier edit UI. Even >>>>>>> with this approach, the problem won't be completely solved. >>>>>>> >>>>>>>> >>>>>>>> Why would we need a new handler? What would be the drawbacks of >>>>>>>> using the existing handler? >>>>>>>> >>>>>>> We can use the existing one. >>>>>>> >>>>>>>> >>>>>>>> The location to get the limit would be on the API Implement section >>>>>>>> IMO. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> NuwanD. >>>>>>>> >>>>>>>> On Tue, Sep 8, 2015 at 9:40 PM, Amila De Silva <ami...@wso2.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Current Throttling capabilities of API Manager only allows >>>>>>>>> defining user wise and Application wise Access Quotas. >>>>>>>>> >>>>>>>>> >>>>>>>>> For example when considering an Application and a set of APIs >>>>>>>>> Subscribed, like below (tier limit is shown next to the API) >>>>>>>>> >>>>>>>>> Application-1 (1000 Req/min) >>>>>>>>> | >>>>>>>>> +-+API-1 (10 Req/min) >>>>>>>>> | >>>>>>>>> +-+API-2 (1 Req/min) >>>>>>>>> | >>>>>>>>> +-+API-3 (5 Req/min) >>>>>>>>> >>>>>>>>> each new token would allow end-users to make the number of >>>>>>>>> requests defined by the tier. Using a token generated for >>>>>>>>> Application-1, >>>>>>>>> API1 can be invoked at a rate of 10 Req/min, API-2 - 1Req/min and >>>>>>>>> likewise. >>>>>>>>> So when a new user signs in, there'd be a potential increase in the >>>>>>>>> traffic >>>>>>>>> on the API. >>>>>>>>> >>>>>>>>> As of now there isn't a way to limit the total number of calls >>>>>>>>> made on an API. Tiers defined at the API Level, doesn't reflect the >>>>>>>>> global >>>>>>>>> limit of the backend; which means that, as an API keeps gathering >>>>>>>>> users, >>>>>>>>> hits on the Backend would also keep increasing without being >>>>>>>>> controlled. >>>>>>>>> >>>>>>>>> With API Manager 1.10.0, the plan is to provide the capability to >>>>>>>>> define Hard Throttling limits for APIs. The limit will be defined per >>>>>>>>> API >>>>>>>>> basis, and this usually will reflect the number of requests the actual >>>>>>>>> backend can handle. >>>>>>>>> >>>>>>>>> This feature warrants several changes on API Designing UI, and >>>>>>>>> those can be discussed in detail in mails to follow. >>>>>>>>> >>>>>>>>> If giving a high level idea about the flow. >>>>>>>>> 1. API creator logs into the publisher and creates an API. >>>>>>>>> 2. API Creator enables Hard Limit throttling for the API. >>>>>>>>> 3. Number of requests allowed and Unit Time is specified. >>>>>>>>> 4. Changes are saved and Published to the Gateway. >>>>>>>>> >>>>>>>>> When saving the API, a throttling policy specific to the API will >>>>>>>>> be created and saved in the registry. >>>>>>>>> >>>>>>>>> For enforcing throttling limit, a new handler will be written, >>>>>>>>> which would only appear for those APIs on which Hard limit is >>>>>>>>> enabled. The >>>>>>>>> handler would refer to the policy saved to the registry and would >>>>>>>>> apply the >>>>>>>>> limit defined. >>>>>>>>> >>>>>>>>> Please share your thoughts on this. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Amila De Silva* >>>>>>>>> >>>>>>>>> WSO2 Inc. >>>>>>>>> mobile :(+94) 775119302 >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Nuwan Dias >>>>>>>> >>>>>>>> Technical Lead - WSO2, Inc. http://wso2.com >>>>>>>> email : nuw...@wso2.com >>>>>>>> Phone : +94 777 775 729 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Amila De Silva* >>>>>>> >>>>>>> WSO2 Inc. >>>>>>> mobile :(+94) 775119302 >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *Sanjeewa Malalgoda* >>>>>> WSO2 Inc. >>>>>> Mobile : +94713068779 >>>>>> >>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Amila De Silva* >>>>> >>>>> WSO2 Inc. >>>>> mobile :(+94) 775119302 >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Amila De Silva* >>>> >>>> WSO2 Inc. >>>> mobile :(+94) 775119302 >>>> >>>> >>> >>> >>> -- >>> >>> *Sanjeewa Malalgoda* >>> WSO2 Inc. >>> Mobile : +94713068779 >>> >>> <http://sanjeewamalalgoda.blogspot.com/>blog >>> :http://sanjeewamalalgoda.blogspot.com/ >>> <http://sanjeewamalalgoda.blogspot.com/> >>> >>> >>> >> >> >> -- >> *Amila De Silva* >> >> WSO2 Inc. >> mobile :(+94) 775119302 >> >> > > > -- > *Amila De Silva* > > WSO2 Inc. > mobile :(+94) 775119302 > > -- *Amila De Silva* WSO2 Inc. mobile :(+94) 775119302
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
