stefanherdy opened a new issue, #67336: URL: https://github.com/apache/airflow/issues/67336
### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.2.1 ### What happened and how to reproduce it? The Dashboard's Historical Metrics section displays running, succeeded, waiting, and failed task instances / Dag runs with a percentage bar. These percentages are calculated in the frontend by dividing each state's count by the sum of all state counts (total): <img width="1711" height="525" alt="Image" src="https://github.com/user-attachments/assets/b48d1905-13a0-40f7-9ad7-6d75fe32684c"; /> The API endpoint "/ui/dashboard/historical_metrics_data" internally limits/caps the state count at STATE_COUNT_CAP = 1000 to limit the SQL query row count for performance reasons, so a maximum count of 1000 is returned. The bug: The total used to calculate percentages is computed from the capped API values. So if success has 2500 and failure has 7 entries but the API returns 1000 for success, the computed total is 1007 instead of 2507. This wrong total results in wrong percentages. <img width="1698" height="505" alt="Image" src="https://github.com/user-attachments/assets/43696636-02c3-49ce-9d70-1a9dc9afa61b"; /> ### What you think should happen instead? The UI should show the correct percentage or no percentage for capped values, but not the wrong percentages. There are two possible ways to fix this: 1. Return real counts from the API 2. Hide percentages in the frontend when any state is capped Returning the real counts from the API with a huge amount of task instances can be significantly slower than the current approach, which uses the limit per state for a good reason: to ensure a fast response even on huge datasets. The downside of keeping the cap is that the frontend cannot compute correct percentages when any state exceeds it, since the total cannot be computed correctly. I would be happy to file a PR about this issue, but want to discuss those tradeoffs first with the community. Best, Stefan ### Operating System Ubuntu 24.04.4 LTS ### Deployment Virtualenv installation ### Apache Airflow Provider(s) _No response_ ### Versions of Apache Airflow Providers _No response_ ### Official Helm Chart version Not Applicable ### Kubernetes Version _No response_ ### Helm Chart configuration _No response_ ### Docker Image customizations _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
