bobbai00 opened a new issue, #5012:
URL: https://github.com/apache/texera/issues/5012
### Task Summary
Sub-issue of #5011.
Move every direct `SqlServer` call reachable from CU Master / CU Worker
that touches the **execution metadata tables** (i.e. `workflow_executions`
and the operator/port URI registry) behind an HTTP service that owns
the credentials. The executor forwards the originating user's JWT.
Code paths in scope (per the Current Usage table in #5011):
- Execution lifecycle — `web/service/WorkflowService.scala` (INSERT new
row, UPDATE on every status / log_location / runtime_stats_uri / result
change).
- State transitions — `web/storage/ExecutionStateStore.scala`
(`updateWorkflowState` from many sites in `WorkflowService` /
`WorkflowExecutionService`).
- Operator/port URI registry —
`web/resource/.../WorkflowExecutionsResource.scala`
(`insertOperatorPortResultUri`, `insertOperatorConsoleUri`,
`getResultUriByLogicalPortId`, `getLatestExecutionID`, …). Called by
engine code (e.g. `RegionExecutionCoordinator`) and by
`SyncExecutionResource`.
- Result export — `web/resource/.../WorkflowExecutionsResource.scala`
(`exportResultToDataset`, `exportResultToLocal`) →
`web/service/ResultExportService.scala`. Endpoints today are hosted on
CU Master; their DB lookups go through the same registry calls above.
- Result/log cleanup — `web/ComputingUnitMaster.scala` (`cleanExecutions`,
`recurringCheckExpiredResults`).
- Cost-based scheduling —
`engine/architecture/scheduling/CostEstimator.scala`
(`getOperatorExecutionTimeInSeconds` reads the latest successful
`workflow_executions.runtime_stats_uri` for a `wid`).
Done when no engine or service code reachable from CU Master / Worker
calls `SqlServer` for execution-metadata reads or writes, every former
DB call has an HTTP endpoint on web-app with `@Auth`-checked JWT
authorization, and the existing test suite plus an end-to-end run still
pass.
### Task Type
- [x] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
