zyratlo opened a new pull request, #8911: URL: https://github.com/apache/texera/pull/8911
### What changes were proposed in this PR? > [!NOTE] > Stacked on #8910. Until that merges, this diff also shows its commit. This is the second of three PRs adding R support to the migration tool (#8009). It adds the R side of the conversion. Nothing is user visible yet: the import modal option arrives in the next PR, and the new flag defaults to off. **Flag.** `gui.workflow-workspace.r-notebook-migration-enabled`, default false, overridable with `GUI_WORKFLOW_WORKSPACE_R_NOTEBOOK_MIGRATION_ENABLED` and exposed to the frontend as `rNotebookMigrationEnabled`. R UDFs only run where the computing unit has the texera-rudf plugin installed (https://github.com/apache/texera/discussions/4155), and no Texera image ships it, so R stays off unless a deployment opts in. It sits under the existing migration flag rather than replacing it. **R prompts.** A new `migration-prompts-r.ts` holds an R version of each prompt, notebook and script, with the same JSON reply contract so parsing does not change. The design follows a spike against texera-rudf that ran chained R UDFs in a real workflow: * Every UDF uses the Table API: one `function(table, port)` that takes and returns a data.frame. `RUDF` needs `useTupleAPI` set explicitly, and teaching one API keeps generated code and the flag consistent. * Python passes objects between UDFs pickled in `binary` columns. R has no pickle, so a UDF whose output another UDF reads returns one row of `I(list(serialize(x, NULL)))` columns, and the reader calls `unserialize(table$x[[1]])`. The spike passed data.frames and fitted `lm` models this way between real workers. * A UDF no other UDF reads returns character columns. Texera's schema check rejects a number in a column `buildWorkflow` declares as `string`, and does not cast it. * Plots are converted with `ggplotly()` into a page that loads Plotly from a CDN, returned as `html-content`. `htmlwidgets::saveWidget` is ruled out because it needs pandoc. * Each UDF reads from at most one other UDF. Results a later step needs are carried along one chain instead of merged from branches, which avoids R UDFs having to union tables. The worked examples are the diabetes pipeline in R. Their line numbers, mapping ranges and JSON escaping were generated rather than typed, and all five example UDFs were run through texera-rudf on `diabetes.csv`, chained the way `buildWorkflow` wires them. Unlike the Python examples, they are valid JSON. **Wiring.** `MigrationLanguage` widens to `"python" | "r"`. The R prompt set builds `RUDF` operators with `useTupleAPI: false` through a new `properties` overlay (empty for Python, so Python operators are unchanged) and derives notebooks with the `ir` kernelspec. A `promptsFor` guard refuses an R conversion while the R flag is off, before any model call. `NotebookMigrationService` passes the language through. ### Any related issues, documentation, discussions? Part of #8009 Parent-issue #4301 The plan is in https://github.com/apache/texera/discussions/8866. The R option in the import modal follows in the next PR. ### How was this PR tested? * `migration-llm.spec.ts`, six new tests: the exact R prelude and prompts for notebooks and scripts, the full `RUDF` property set with binary and string outputs, the `ir` kernelspec on a derived notebook, the R flag blocking R without a model call, and Python still converting with the R flag off. The Python prompt tests from #8910 pass unchanged. * `migration-prompts-r.spec.ts`, new, ten tests guarding the worked examples: both example blocks parse as JSON, edges only reference defined UDFs, every UDF is a numbered Table API function, only UDFs another UDF reads call `serialize`, no UDF has two inputs, and the mapping cells and line ranges point at real, non blank lines. * `notebook-migration.service.spec.ts`, two new tests that the language reaches both conversions. * `GuiConfigSpec` checks the flag defaults to false. Coverage of `migration-llm.ts` and `notebook-migration.service.ts` stays at 100% of lines. The full frontend suite passes. End to end with a live model, through the modal from the next PR: an R script and an R notebook each converted to a workflow of Table API `RUDF` operators. Run on the source dataset, the script's workflow reported the same model accuracies as running the original script in `Rscript`, and its plot rendered through an HTML Visualizer. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
