SkinnyPigeon opened a new pull request, #35481: URL: https://github.com/apache/superset/pull/35481
<!--- Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/ Example: fix(dashboard): load charts correctly --> ### SUMMARY We have encountered a case wherein a user's surname is being excluded from a report as the name is `NA`. Pandas' exclusion list for `null` values includes the value `NA`. I propose that we add a new feature flag to complement the existing one in place for uploads to allow users to customise which values are or are not allowed, `CSV_DEFAULT_NA_NAMES`. For instance, in our use case, we would set the configuration as follows: ```python CSV_DEFAULT_NA_NAMES = [value for value in list(STR_NA_VALUES) if value != 'NA'] REPORTS_CSV_NA_NAMES = CSV_DEFAULT_NA_NAMES ``` The video below shows the before and after effect of this in action. To start, we see the config using the default value of ```python from pandas._libs.parsers import STR_NA_VALUES ``` Afterwards, I reload the config with the above example, where we filter out `NA` from the list, allowing our user to have their data sent in a report. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF https://github.com/user-attachments/assets/76d406d0-ec85-41d9-9a15-d989936c7ab7 ### TESTING INSTRUCTIONS I have added unit tests as well as deployed the changes locally. However, to test: 1. Build and launch this branch 2. Create a new Dataset with the following query: `select 'NANANANA' as first_name, 'JEFF' as middle_name, 'NA' as last_name` 3. Create a report with a Chart based on this query 4. Update the config to filter out `NA` from the list 5. Rerun report ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [ ] Has associated issue: - [x] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [x] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
