GitHub user carloea2 created a discussion: Discuss workflow macro design and lifecycle
### Feature Summary Workflow macros are an important feature for Texera, but the current discussion in #5179 mixes several design choices with implementation review. Based on the feedback from @Yicong-Huang and @chenlica, we should move the macro design discussion into an issue, hold a focused live discussion, and record the decisions here before continuing. References: - #5179: frontend-oriented workflow macro implementation - #5178: original feature proposal for importing/grouping/collapsing reusable sub-workflows - #5115: hackathon macro-operator prototype with a different UX - #754: previous operator grouping/collapse behavior ### Proposed Solution or Design Use this issue to align on the macro design before reviewing code in detail. The discussion should decide: 1. Macro creation UX - Should users create a macro by selecting existing operators and choosing `Create Macro` from the context menu? - Should users create a macro by importing an existing workflow? - Should both flows be supported? 2. Macro representation and reuse - Should a macro look and behave like a normal logical operator? - Should macro input/output ports be derived from the internal sub-DAG? - Should reusable macros appear directly in the operator menu, for example under `My Macros`? - Should the long-term design also be able to represent operators such as Hash Join as macros? 3. Macro internal view and editing - Should users edit macro internals in-place on the current canvas, or jump into a separate/drill-down workflow view? - Are edits local to one workflow snapshot, or shared across all uses of the macro? - Should macro internals expose explicit input/output ports instead of dummy source/sink operators? 4. Runtime and statistics behavior - Should macro expansion remain frontend-only, with the backend executing the flattened operator graph? - Should the backend be unaware of macros? - When collapsed, should a macro show aggregated statistics, internal leaf-operator statistics, or allow users to choose? 5. Macro lifecycle and persistence - How should macro definitions and macro instances be stored? - How should users remove, rename, disable, search, share, or version macros? - What documentation, tutorials, and tests are needed before enabling the feature by default? ### Impact / Priority ? ### Affected Area - Workflow UI - Workflow Engine (Amber) - Storage / Metadata - Documentation / Tutorials GitHub link: https://github.com/apache/texera/discussions/5248 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
