tarun-google commented on code in PR #37218:
URL: https://github.com/apache/beam/pull/37218#discussion_r2668922301
##########
sdks/python/apache_beam/ml/inference/base_test.py:
##########
@@ -2071,6 +2071,70 @@ def run_inference(self,
responses.append(model.predict(example))
return responses
+ def test_run_inference_with_rate_limiter(self):
+ class FakeRateLimiter(base.RateLimiter):
+ def __init__(self):
+ super().__init__(namespace='test_namespace')
+
+ def throttle(self, hits_added=1):
+ self.requests_counter.inc()
+ return True
Review Comment:
Context is different in AdaptiveThrottler and RateLimiter. In
AdaptiveThrottler throttling is always possible throttle_request is giving
caller info to either apply throttle delay or allow. In RateLimiter throttling
might not be always possible, and if possible, delay is already applied by
RateLimiter. So true indicates that it is throttled and false indicates its
not.
If the naming of throttle() is confusing for people coming from
AdaptiveThrottler context we can change it to allow_request/acquire()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
