Andrea Di Stefano created NIFI-14355:
----------------------------------------
Summary: Authenticated GitHub Registry Client hitting rate limit
(5000 reqs per hour)
Key: NIFI-14355
URL: https://issues.apache.org/jira/browse/NIFI-14355
Project: Apache NiFi
Issue Type: Improvement
Components: Flow Versioning, NiFi Registry
Affects Versions: 2.2.0
Environment: Not relevant
Reporter: Andrea Di Stefano
Attachments: image-2025-03-11-16-47-39-522.png
Hello.
We are using the GitHub Registry Client with a Personal Authentication Token
(PAT) and constantly hitting the API rate limit. The rate limit, [as stated in
the GitHub
docs|https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-authenticated-users],
is 5000 requests per hour.
We have a pipeline with 53 instances of 3 versioned process groups, which are
part of a bigger pipeline (~1.1k components).
I uploaded a related stack trace [here|https://pastebin.com/R0tTw4bP].
Calling the GitHub API with Postman, we saw that the response headers include
data about the rate limit, current usage, and reset time in Unix epoch format,
see the attached screenshot for an example:
!image-2025-03-11-16-26-09-253.png|width=578,height=153!
Is NiFi periodically polling the GitHub API in background?
Can we somehow limit the number of calls?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
