rusackas opened a new pull request, #41644: URL: https://github.com/apache/superset/pull/41644
### SUMMARY Improvements to `scripts/translations/backfill_po.py` (the AI translation backfill tool), split into two focused commits. This is the tooling companion to the per-language backfill PRs (#41608 de, #41609 es, #41612 lv, #41613 fi, #41640 sk, #41641 th, …); it will accumulate further hardening as the sweep surfaces issues. **1. Complete the `LANGUAGE_NAMES` map** The prompt falls back to the raw ISO code when a language is absent from `LANGUAGE_NAMES` (e.g. *"Translate … into lv (lv)"*), a weaker signal than a real language name. Adds the six present catalogs that were missing: `cs` (Czech), `fi` (Finnish), `lv` (Latvian), `ro` (Romanian), `sr_Latn` (Serbian, Latin script), `th` (Thai). The map now covers every locale directory under `superset/translations`. **2. Resilient batch translation (bisect + plain-text fallback)** A source `msgid` containing a literal double-quote (e.g. `"Exact match"`) can make the model echo unescaped quotes into its JSON response, so `json.loads` fails and the **entire batch** is discarded — losing up to `--batch-size` good translations per offending entry. (Observed live on the Slovak catalog: 3 such strings sank ~100 translations across batches.) `_resilient_translate` wraps `translate_batch`: on a JSON parse failure it **bisects the batch and recurses**, isolating the offenders; a lone entry that still fails falls back to a **plain-text prompt** (`_translate_single_plaintext`) that avoids the JSON envelope entirely. CLI failures (`RuntimeError`) still propagate to the existing per-batch handler. Verified live on the Thai catalog — the same 3 quote-bearing entries were recovered automatically with zero batch loss. Adds 4 unit tests: passthrough, bisect-and-fallback with correct index remapping, dropped-entry-on-fallback-failure, and RuntimeError propagation. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF N/A — internal tooling. ### TESTING INSTRUCTIONS ```bash pytest tests/unit_tests/scripts/translations/backfill_po_test.py ``` (31 tests pass.) ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [x] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
