Hi, According to the new throttling architecture in API Manager, the throttling decision will be taken by a global throttling engine (CEP). The Gateway nodes publishes events to the throttling engine and the throttling engine evaluates data within those events to decide whether a particular 'key' is throttled or not. Note: Each event will have a 'key' against which throttling counters are maintained. When the throttling engine decides a particular key is to be throttled out, it will persist this data to a Database. It will later clean this record from the DB when the state of key is reverted later.
The next problem is on how this decision that is made by the throttling engine is communicated to the Gateway cluster. We evaluated two options as below. 1. Each Gateway node keeps polling the throttling engine periodically (every 5 secs or so) and gets a snapshot of the DB into local memory. 2. Using a pub-sub mechanism where the throttling engine publishes a state change event of a key to a topic and each Gateway node will have subscribed to that topic and hence receive updates as and when state changes occur. This pattern is illustrated in a diagram below. The first approach has some drawbacks such as a) Each Gateway has to poll the engine even if no events are throttled out. b) The amount of data being transferred over the network can be large. c) The load on the throttling engine increases as more Gateways start polling it. d) The load on the DB reads can be high (can use a cache to avoid this). The second approach looks better than the first but still have a few other drawbacks such as a) We will anyway need to get a snapshot from the DB the first time a Gateway starts to get the current state. b) Deployment complexities when scaling out. We would probably need to have a separate message broker cluster in larger deployments. c) Need to maintain a consistent http connection between the Gateway and the message broker. d) If a Gateway node disconnects, the messages intended for that node will pile up on the message broker. We have currently opted for option 2. Please share your thoughts, suggestions on this approach. [image: Inline image 1] Thanks, NuwanD. -- Nuwan Dias Technical Lead - WSO2, Inc. http://wso2.com email : nuw...@wso2.com Phone : +94 777 775 729
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
