SulimanAbdulrazzaq opened a new pull request, #44652: URL: https://github.com/apache/superset/pull/44652
### SUMMARY With both **Cumulative** and **Normalize** checked, the histogram post-processing accumulated the bin counts first and then divided by the sum of the running totals, not by the number of data points. For ten evenly spread values in five bins, the bins read `0.067, 0.133, 0.2, 0.267, 0.333` instead of `0.2, 0.4, 0.6, 0.8, 1.0`. The Normalize control describes the values as each bin's count divided by the total count of data points. This change normalizes the raw bin counts first and then accumulates them, so each bin shows the share of data points up to that bin. With a group by, the values are still shares of all data points, as they are for a normalized histogram that is not cumulative. Histograms with only one of the two options return the same values as before. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF Not applicable (backend post-processing). Output of `histogram(df, "a", [], 5, cumulative=True, normalize=True)` for `a = 1..10`: - Before: `[[0.0667, 0.1333, 0.2, 0.2667, 0.3333]]` - After: `[[0.2, 0.4, 0.6, 0.8, 1.0]]` ### TESTING INSTRUCTIONS `pytest tests/unit_tests/pandas_postprocessing/test_histogram.py` - `test_histogram_cumulative_and_normalize_ends_at_one` is new. - `test_histogram_with_groupby_and_cumulative_and_normalize` pinned the old values and now expects the running shares (`A: 0.2, 0.2, 0.4, 0.4, 0.6`, `B: 0.0, 0.2, 0.2, 0.4, 0.4`). Without the fix both tests fail. With the fix, `tests/unit_tests/pandas_postprocessing` passes (165 passed), and the full `tests/common tests/unit_tests` run has 17945 passed. Its only 3 failures read `.github/workflows` files that my test fork had removed, and they failed the same way without the fix. `ruff format --check`, `ruff check` and `mypy --check-untyped-defs` are clean on the two touched files. To check in the UI: build a Histogram chart, check Cumulative and Normalize, and confirm that the last bar reaches 1 when there is no dimension. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
