Re: [Architecture] [APIM] Hard Throttling limit for APIs

[Amila De Silva]

On Wed, Sep 9, 2015 at 11:47 AM, Sanjeewa Malalgoda 
wrote:

> I also believe we should use existing handler without writing new one.
>
> And regarding registry decoupling, publisher may be able to push API to
> gateway as we do now.
> And just before we call rest API admin service we may call registry admin
> service(may be we need to write simple service for this as flow continue
> with super admin - super tenant deployed multi tenanted service) and push
> tiers.xml file.
> With that we may keep registry separation between gateway and publisher.
> That would be valuable when we have MZ and DMZ pattern and gateway in DMZ.
>
> Thanks,
> sanjeewa.
>
>
> On Tue, Sep 8, 2015 at 11:11 PM, Amila De Silva  wrote:
>
>> Hi Nuwan,
>>
>>
>>
>> On Tue, Sep 8, 2015 at 9:58 PM, Nuwan Dias  wrote:
>>
>>> This design brings on a hard (mandatory) dependency to the registry from
>>> the API Gateway right? If each API is to have its own policy definition,
>>> the publisher and gateway must connect to a single registry. This will
>>> cause problems in some deployment patterns which have the Gateway in DMZ
>>> and publisher in the LAN. Can our throttling engine work if we just feed it
>>> the request count and unit time without feeding it a policy definition?
>>>
>>
>> Not in a very straightforward way. The API exposed by throttle.core only
>> allows passing ThrottleContext, a key and the Tier name. When going through
>> the usages of ThrottleContext it seems that it can only be created out from
>> a policy file. But need to check if it's possible to create ThrottleContext
>> only using unit time and MaxRequest count.
>>
>>> If that works then we can survive without the registry for this.
>>>
>> We can keep max request count and unit time in the API itself. While
initialising the APIThrottleHandler, a policy.xml has to be created out of
these values and the generated policy will be used to create the
ThrottleContext. Dynamic policy won't be stored anywhere, so when moving
the API from one environment to another, there won't be a risk of losing
anything.

For the scope of this feature, should be consider about providing
capability to specify hard throttling limits per each resource verb
combination.

>
>> Another option is to use the same tiers.xml that is being used for other
>> throttling cases and refer to a role defined there. If needed to update/add
>> a new tier it will have to be done through tier edit UI. Even with this
>> approach, the problem won't be completely solved.
>>
>>>
>>> Why would we need a new handler? What would be the drawbacks of using
>>> the existing handler?
>>>
>> We can use the existing one.
>>
>>>
>>> The location to get the limit would be on the API Implement section IMO.
>>>
>>> Thanks,
>>> NuwanD.
>>>
>>> On Tue, Sep 8, 2015 at 9:40 PM, Amila De Silva  wrote:
>>>
 Hi,

 Current Throttling capabilities of API Manager only allows defining
 user wise and Application wise Access Quotas.


 For example when considering an Application and a set of APIs
 Subscribed, like below (tier limit is shown next to the API)

 Application-1 (1000 Req/min)
 |
 +-+API-1 (10 Req/min)
 |
 +-+API-2 (1 Req/min)
 |
 +-+API-3 (5 Req/min)

 each new token would allow end-users to make the number of requests
 defined by the tier. Using a token generated for Application-1, API1 can be
 invoked at a rate of 10 Req/min, API-2 - 1Req/min and likewise. So when a
 new user signs in, there'd be a potential increase in the traffic on the
 API.

 As of now there isn't a way to limit the total number of calls made on
 an API. Tiers defined at the API Level, doesn't reflect the global limit of
 the backend; which means that, as an API keeps gathering users, hits on the
 Backend would also keep increasing without being controlled.

 With API Manager 1.10.0, the plan is to provide the capability to
 define Hard Throttling limits for APIs. The limit will be defined per API
 basis, and this usually will reflect the number of requests the actual
 backend can handle.

 This feature warrants several changes on API Designing UI, and those
 can be discussed in detail in mails to follow.

 If giving a high level idea about the flow.
 1. API creator logs into the publisher and creates an API.
 2. API Creator enables Hard Limit throttling for the API.
 3. Number of requests allowed and Unit Time is specified.
 4. Changes are saved and Published to the Gateway.

 When saving the API, a throttling policy specific to the API will be
 created and saved in the registry.

 For enforcing throttling limit, a new handler will be written, which
 would only appear for those APIs on which Hard limit is enabled. The
 handler would refer to the policy saved to the registry and would apply the
 

Re: [Architecture] [APIM] Hard Throttling limit for APIs

[Amila De Silva]

On Wed, Sep 16, 2015 at 1:42 PM, Amila De Silva  wrote:

>
> On Wed, Sep 9, 2015 at 11:47 AM, Sanjeewa Malalgoda 
> wrote:
>
>> I also believe we should use existing handler without writing new one.
>>
>> And regarding registry decoupling, publisher may be able to push API to
>> gateway as we do now.
>> And just before we call rest API admin service we may call registry admin
>> service(may be we need to write simple service for this as flow continue
>> with super admin - super tenant deployed multi tenanted service) and push
>> tiers.xml file.
>> With that we may keep registry separation between gateway and publisher.
>> That would be valuable when we have MZ and DMZ pattern and gateway in DMZ.
>>
>> Thanks,
>> sanjeewa.
>>
>>
>> On Tue, Sep 8, 2015 at 11:11 PM, Amila De Silva  wrote:
>>
>>> Hi Nuwan,
>>>
>>>
>>>
>>> On Tue, Sep 8, 2015 at 9:58 PM, Nuwan Dias  wrote:
>>>
 This design brings on a hard (mandatory) dependency to the registry
 from the API Gateway right? If each API is to have its own policy
 definition, the publisher and gateway must connect to a single registry.
 This will cause problems in some deployment patterns which have the Gateway
 in DMZ and publisher in the LAN. Can our throttling engine work if we just
 feed it the request count and unit time without feeding it a policy
 definition?

>>>
>>> Not in a very straightforward way. The API exposed by throttle.core only
>>> allows passing ThrottleContext, a key and the Tier name. When going through
>>> the usages of ThrottleContext it seems that it can only be created out from
>>> a policy file. But need to check if it's possible to create ThrottleContext
>>> only using unit time and MaxRequest count.
>>>
 If that works then we can survive without the registry for this.

>>> We can keep max request count and unit time in the API itself. While
> initialising the APIThrottleHandler, a policy.xml has to be created out of
> these values and the generated policy will be used to create the
> ThrottleContext. Dynamic policy won't be stored anywhere, so when moving
> the API from one environment to another, there won't be a risk of losing
> anything.
>
> For the scope of this feature, should be consider about providing
> capability to specify hard throttling limits per each resource verb
> combination.
>

Accidentally sent the mail halfway :)

What I was going to highlight was need of having different hard limits for
each resource + verb combination.

Not every resource of an API, consume the same amount of server resources.
For example, when sending response to an OPTIONS call, the backend might be
sending a cached (or a static) response, whilst when responding to a POST,
it might be doing several DB calls back and forth. Actually, it might be an
overflow of POST requests that can take the back-end down. So while
specifying a quota, heavy weight resources should be given less quota, and
lightweight (but more frequently accessed resources) should be given a
higher quota.

Another option is to use the same tiers.xml that is being used for other
>>> throttling cases and refer to a role defined there. If needed to update/add
>>> a new tier it will have to be done through tier edit UI. Even with this
>>> approach, the problem won't be completely solved.
>>>

 Why would we need a new handler? What would be the drawbacks of using
 the existing handler?

>>> We can use the existing one.
>>>

 The location to get the limit would be on the API Implement section IMO.

 Thanks,
 NuwanD.

 On Tue, Sep 8, 2015 at 9:40 PM, Amila De Silva  wrote:

> Hi,
>
> Current Throttling capabilities of API Manager only allows defining
> user wise and Application wise Access Quotas.
>
>
> For example when considering an Application and a set of APIs
> Subscribed, like below (tier limit is shown next to the API)
>
> Application-1 (1000 Req/min)
> |
> +-+API-1 (10 Req/min)
> |
> +-+API-2 (1 Req/min)
> |
> +-+API-3 (5 Req/min)
>
> each new token would allow end-users to make the number of requests
> defined by the tier. Using a token generated for Application-1, API1 can 
> be
> invoked at a rate of 10 Req/min, API-2 - 1Req/min and likewise. So when a
> new user signs in, there'd be a potential increase in the traffic on the
> API.
>
> As of now there isn't a way to limit the total number of calls made on
> an API. Tiers defined at the API Level, doesn't reflect the global limit 
> of
> the backend; which means that, as an API keeps gathering users, hits on 
> the
> Backend would also keep increasing without being controlled.
>
> With API Manager 1.10.0, the plan is to provide the capability to
> define Hard Throttling limits for APIs. The limit will be defined per API

Re: [Architecture] [APIM] Hard Throttling limit for APIs

[Amila De Silva]

On Wed, Sep 16, 2015 at 5:08 PM, Sanjeewa Malalgoda 
wrote:

> Yes what you said have value actually.
> But wont it make things complicated? For the initial implementation we may
> go with configuration per API/Endpoint.
> May be later we need to implement something to have multiple back end
> services for same API(endpoint per resource).
> And at that point we need to have something like you mentioned.
>
> And did we checked DMZ/MZ problem here.
> What do you mean by having max count inside API. Is it meant in synapse
> configuration?
>
Yes, as two properties of APIThrottleHandler.

>
> Thanks,
> sanjeewa.
>
> On Wed, Sep 16, 2015 at 2:11 PM, Amila De Silva  wrote:
>
>>
>>
>> On Wed, Sep 16, 2015 at 1:42 PM, Amila De Silva  wrote:
>>
>>>
>>> On Wed, Sep 9, 2015 at 11:47 AM, Sanjeewa Malalgoda 
>>> wrote:
>>>
 I also believe we should use existing handler without writing new one.

 And regarding registry decoupling, publisher may be able to push API to
 gateway as we do now.
 And just before we call rest API admin service we may call registry
 admin service(may be we need to write simple service for this as flow
 continue with super admin - super tenant deployed multi tenanted service)
 and push tiers.xml file.
 With that we may keep registry separation between gateway and publisher.
 That would be valuable when we have MZ and DMZ pattern and gateway in
 DMZ.

 Thanks,
 sanjeewa.


 On Tue, Sep 8, 2015 at 11:11 PM, Amila De Silva 
 wrote:

> Hi Nuwan,
>
>
>
> On Tue, Sep 8, 2015 at 9:58 PM, Nuwan Dias  wrote:
>
>> This design brings on a hard (mandatory) dependency to the registry
>> from the API Gateway right? If each API is to have its own policy
>> definition, the publisher and gateway must connect to a single registry.
>> This will cause problems in some deployment patterns which have the 
>> Gateway
>> in DMZ and publisher in the LAN. Can our throttling engine work if we 
>> just
>> feed it the request count and unit time without feeding it a policy
>> definition?
>>
>
> Not in a very straightforward way. The API exposed by throttle.core
> only allows passing ThrottleContext, a key and the Tier name. When going
> through the usages of ThrottleContext it seems that it can only be created
> out from a policy file. But need to check if it's possible to create
> ThrottleContext only using unit time and MaxRequest count.
>
>> If that works then we can survive without the registry for this.
>>
> We can keep max request count and unit time in the API itself. While
>>> initialising the APIThrottleHandler, a policy.xml has to be created out of
>>> these values and the generated policy will be used to create the
>>> ThrottleContext. Dynamic policy won't be stored anywhere, so when moving
>>> the API from one environment to another, there won't be a risk of losing
>>> anything.
>>>
>>> For the scope of this feature, should be consider about providing
>>> capability to specify hard throttling limits per each resource verb
>>> combination.
>>>
>>
>> Accidentally sent the mail halfway :)
>>
>> What I was going to highlight was need of having different hard limits
>> for each resource + verb combination.
>>
>> Not every resource of an API, consume the same amount of server
>> resources. For example, when sending response to an OPTIONS call, the
>> backend might be sending a cached (or a static) response, whilst when
>> responding to a POST, it might be doing several DB calls back and forth.
>> Actually, it might be an overflow of POST requests that can take the
>> back-end down. So while specifying a quota, heavy weight resources should
>> be given less quota, and lightweight (but more frequently accessed
>> resources) should be given a higher quota.
>>
>> Another option is to use the same tiers.xml that is being used for other
> throttling cases and refer to a role defined there. If needed to 
> update/add
> a new tier it will have to be done through tier edit UI. Even with this
> approach, the problem won't be completely solved.
>
>>
>> Why would we need a new handler? What would be the drawbacks of using
>> the existing handler?
>>
> We can use the existing one.
>
>>
>> The location to get the limit would be on the API Implement section
>> IMO.
>>
>> Thanks,
>> NuwanD.
>>
>> On Tue, Sep 8, 2015 at 9:40 PM, Amila De Silva 
>> wrote:
>>
>>> Hi,
>>>
>>> Current Throttling capabilities of API Manager only allows defining
>>> user wise and Application wise Access Quotas.
>>>
>>>
>>> For example when considering an Application and a set of APIs
>>> Subscribed, like below (tier limit is shown next to the 

Re: [Architecture] [APIM] Hard Throttling limit for APIs

[Amila De Silva]

For displaying at Publisher, throttling limits have to persisted on a DB in
some format (either in the registry artifact, in the swagger definition or
in AM_API table). But at the runtime, values will be picked from the
synapse definition.

Plan is to enforce the hard limit on a particular version of the API. For
example if there are two APIs having two versions v1 and v2, the hard limit
defined on v1 only ensures requests flowing through v1 version doesn't
exceed the specified limit.

On Wed, Sep 16, 2015 at 5:11 PM, Amila De Silva  wrote:

>
>
> On Wed, Sep 16, 2015 at 5:08 PM, Sanjeewa Malalgoda 
> wrote:
>
>> Yes what you said have value actually.
>> But wont it make things complicated? For the initial implementation we
>> may go with configuration per API/Endpoint.
>> May be later we need to implement something to have multiple back end
>> services for same API(endpoint per resource).
>> And at that point we need to have something like you mentioned.
>>
>> And did we checked DMZ/MZ problem here.
>> What do you mean by having max count inside API. Is it meant in synapse
>> configuration?
>>
> Yes, as two properties of APIThrottleHandler.
>
>>
>> Thanks,
>> sanjeewa.
>>
>> On Wed, Sep 16, 2015 at 2:11 PM, Amila De Silva  wrote:
>>
>>>
>>>
>>> On Wed, Sep 16, 2015 at 1:42 PM, Amila De Silva  wrote:
>>>

 On Wed, Sep 9, 2015 at 11:47 AM, Sanjeewa Malalgoda 
 wrote:

> I also believe we should use existing handler without writing new one.
>
> And regarding registry decoupling, publisher may be able to push API
> to gateway as we do now.
> And just before we call rest API admin service we may call registry
> admin service(may be we need to write simple service for this as flow
> continue with super admin - super tenant deployed multi tenanted service)
> and push tiers.xml file.
> With that we may keep registry separation between gateway and
> publisher.
> That would be valuable when we have MZ and DMZ pattern and gateway in
> DMZ.
>
> Thanks,
> sanjeewa.
>
>
> On Tue, Sep 8, 2015 at 11:11 PM, Amila De Silva 
> wrote:
>
>> Hi Nuwan,
>>
>>
>>
>> On Tue, Sep 8, 2015 at 9:58 PM, Nuwan Dias  wrote:
>>
>>> This design brings on a hard (mandatory) dependency to the registry
>>> from the API Gateway right? If each API is to have its own policy
>>> definition, the publisher and gateway must connect to a single registry.
>>> This will cause problems in some deployment patterns which have the 
>>> Gateway
>>> in DMZ and publisher in the LAN. Can our throttling engine work if we 
>>> just
>>> feed it the request count and unit time without feeding it a policy
>>> definition?
>>>
>>
>> Not in a very straightforward way. The API exposed by throttle.core
>> only allows passing ThrottleContext, a key and the Tier name. When going
>> through the usages of ThrottleContext it seems that it can only be 
>> created
>> out from a policy file. But need to check if it's possible to create
>> ThrottleContext only using unit time and MaxRequest count.
>>
>>> If that works then we can survive without the registry for this.
>>>
>> We can keep max request count and unit time in the API itself. While
 initialising the APIThrottleHandler, a policy.xml has to be created out of
 these values and the generated policy will be used to create the
 ThrottleContext. Dynamic policy won't be stored anywhere, so when moving
 the API from one environment to another, there won't be a risk of losing
 anything.

 For the scope of this feature, should be consider about providing
 capability to specify hard throttling limits per each resource verb
 combination.

>>>
>>> Accidentally sent the mail halfway :)
>>>
>>> What I was going to highlight was need of having different hard limits
>>> for each resource + verb combination.
>>>
>>> Not every resource of an API, consume the same amount of server
>>> resources. For example, when sending response to an OPTIONS call, the
>>> backend might be sending a cached (or a static) response, whilst when
>>> responding to a POST, it might be doing several DB calls back and forth.
>>> Actually, it might be an overflow of POST requests that can take the
>>> back-end down. So while specifying a quota, heavy weight resources should
>>> be given less quota, and lightweight (but more frequently accessed
>>> resources) should be given a higher quota.
>>>
>>> Another option is to use the same tiers.xml that is being used for other
>> throttling cases and refer to a role defined there. If needed to 
>> update/add
>> a new tier it will have to be done through tier edit UI. Even with this
>> approach, the problem won't be completely solved.
>>

Re: [Architecture] [APIM] Hard Throttling limit for APIs

[Sanjeewa Malalgoda]

I also believe we should use existing handler without writing new one.

And regarding registry decoupling, publisher may be able to push API to
gateway as we do now.
And just before we call rest API admin service we may call registry admin
service(may be we need to write simple service for this as flow continue
with super admin - super tenant deployed multi tenanted service) and push
tiers.xml file.
With that we may keep registry separation between gateway and publisher.
That would be valuable when we have MZ and DMZ pattern and gateway in DMZ.

Thanks,
sanjeewa.


On Tue, Sep 8, 2015 at 11:11 PM, Amila De Silva  wrote:

> Hi Nuwan,
>
>
>
> On Tue, Sep 8, 2015 at 9:58 PM, Nuwan Dias  wrote:
>
>> This design brings on a hard (mandatory) dependency to the registry from
>> the API Gateway right? If each API is to have its own policy definition,
>> the publisher and gateway must connect to a single registry. This will
>> cause problems in some deployment patterns which have the Gateway in DMZ
>> and publisher in the LAN. Can our throttling engine work if we just feed it
>> the request count and unit time without feeding it a policy definition?
>>
>
> Not in a very straightforward way. The API exposed by throttle.core only
> allows passing ThrottleContext, a key and the Tier name. When going through
> the usages of ThrottleContext it seems that it can only be created out from
> a policy file. But need to check if it's possible to create ThrottleContext
> only using unit time and MaxRequest count.
>
>> If that works then we can survive without the registry for this.
>>
>
> Another option is to use the same tiers.xml that is being used for other
> throttling cases and refer to a role defined there. If needed to update/add
> a new tier it will have to be done through tier edit UI. Even with this
> approach, the problem won't be completely solved.
>
>>
>> Why would we need a new handler? What would be the drawbacks of using the
>> existing handler?
>>
> We can use the existing one.
>
>>
>> The location to get the limit would be on the API Implement section IMO.
>>
>> Thanks,
>> NuwanD.
>>
>> On Tue, Sep 8, 2015 at 9:40 PM, Amila De Silva  wrote:
>>
>>> Hi,
>>>
>>> Current Throttling capabilities of API Manager only allows defining user
>>> wise and Application wise Access Quotas.
>>>
>>>
>>> For example when considering an Application and a set of APIs
>>> Subscribed, like below (tier limit is shown next to the API)
>>>
>>> Application-1 (1000 Req/min)
>>> |
>>> +-+API-1 (10 Req/min)
>>> |
>>> +-+API-2 (1 Req/min)
>>> |
>>> +-+API-3 (5 Req/min)
>>>
>>> each new token would allow end-users to make the number of requests
>>> defined by the tier. Using a token generated for Application-1, API1 can be
>>> invoked at a rate of 10 Req/min, API-2 - 1Req/min and likewise. So when a
>>> new user signs in, there'd be a potential increase in the traffic on the
>>> API.
>>>
>>> As of now there isn't a way to limit the total number of calls made on
>>> an API. Tiers defined at the API Level, doesn't reflect the global limit of
>>> the backend; which means that, as an API keeps gathering users, hits on the
>>> Backend would also keep increasing without being controlled.
>>>
>>> With API Manager 1.10.0, the plan is to provide the capability to define
>>> Hard Throttling limits for APIs. The limit will be defined per API basis,
>>> and this usually will reflect the number of requests the actual backend can
>>> handle.
>>>
>>> This feature warrants several changes on API Designing UI, and those can
>>> be discussed in detail in mails to follow.
>>>
>>> If giving a high level idea about the flow.
>>> 1. API creator logs into the publisher and creates an API.
>>> 2. API Creator enables Hard Limit throttling for the API.
>>> 3. Number of requests allowed and Unit Time is specified.
>>> 4. Changes are saved and Published to the Gateway.
>>>
>>> When saving the API, a throttling policy specific to the API will be
>>> created and saved in the registry.
>>>
>>> For enforcing throttling limit, a new handler will be written, which
>>> would only appear for those APIs on which Hard limit is enabled. The
>>> handler would refer to the policy saved to the registry and would apply the
>>> limit defined.
>>>
>>> Please share your thoughts on this.
>>>
>>> --
>>> *Amila De Silva*
>>>
>>> WSO2 Inc.
>>> mobile :(+94) 775119302
>>>
>>>
>>
>>
>> --
>> Nuwan Dias
>>
>> Technical Lead - WSO2, Inc. http://wso2.com
>> email : nuw...@wso2.com
>> Phone : +94 777 775 729
>>
>
>
>
> --
> *Amila De Silva*
>
> WSO2 Inc.
> mobile :(+94) 775119302
>
>


-- 

*Sanjeewa Malalgoda*
WSO2 Inc.
Mobile : +94713068779

blog
:http://sanjeewamalalgoda.blogspot.com/

___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture