Archita-kale opened a new pull request, #43693:
URL: https://github.com/apache/superset/pull/43693
@kokhlo I’ve implemented the follow-up fix discussed here, preserving NULL
grouping values through the pivot post-processing path. I’ve also added
regression tests for flat and MultiIndex pivots. Would appreciate your review!
### SUMMARY
Follow-up fix for #43547 to preserve `NULL`/`NaN` grouping values through
the pandas post-processing `pivot()` operator.
Problem
In `superset/utils/pandas_postprocessing/pivot.py`, `NULL` values in the
`columns` parameter were already filled with `NULL_STRING` (`"<NULL>"`) before
`pivot_table()`. However, `index` columns lacked this handling. Because pandas
`df.pivot_table()` drops rows with `NaN` in index/grouping keys regardless of
the `dropna=` setting, grouping rows with `NULL` values that survived the
`aggregate` step were silently dropped during pivoting.
Solution
- Mirrored the existing `columns` fill pattern: filled `index` columns with
`NULL_STRING` (`"<NULL>"`) prior to calling `df.pivot_table()`.
- Preserved existing `drop_missing_columns` behavior without modifying
`dropna=drop_missing_columns`.
- Reused the existing `NULL_STRING` constant without introducing any new
constants or modifying PR #43682.
- Added comprehensive unit tests for both flat pivots and `columns`
MultiIndex pivots with NULL index values.
### TESTING INSTRUCTIONS
Run the pandas postprocessing unit tests:
```bash
pytest tests/unit_tests/pandas_postprocessing/test_pivot.py -v
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
