Yicong-Huang opened a new pull request, #5961:
URL: https://github.com/apache/texera/pull/5961
Closes #5960
## What
A single-script manager (`bin/local-dev.sh`) and a Textual dashboard
(`bin/local-dev-tui.py`) for the steady-state local dev workflow — bringing up
docker infra, JVM services, agent-service, and the frontend; checking status;
bouncing one service after a code change; tailing logs.
<!--
TODO: attach a demo GIF here.
Recommended capture:
asciinema rec /tmp/local-dev.cast \
-c "bin/local-dev.sh" \
--idle-time-limit 1
agg /tmp/local-dev.cast /tmp/local-dev.gif
Show: launching the TUI → live state → editing a Scala file (★ lights up)
→ typing `<svc>` to rebuild+bounce → ★ clears → double-clicking a row for
logs.
-->
## CLI (`bin/local-dev.sh`)
| Command | What it does
|
| ------------------------------ |
---------------------------------------------------------------------------- |
| `bin/local-dev.sh` | Launch the Textual TUI (default).
|
| `bin/local-dev.sh status` | One-shot text dashboard.
|
| `bin/local-dev.sh up` | `sbt dist` (incremental, content-hash
skip), `yarn install` / `bun install` if locks moved, `docker compose up -d`
for infra, kick off every native service. |
| `bin/local-dev.sh up --build` | Force the build even if hashes match.
|
| `bin/local-dev.sh up --fresh` | `sbt clean dist`.
|
| `bin/local-dev.sh up --no-build` / `--skip=svc1,svc2` | Skip build /
individual services. |
| `bin/local-dev.sh down` | `docker compose down` + SIGTERM the
native PIDs (reverse order). |
| `bin/local-dev.sh start <svc>` / `stop <svc>` | Single-service start /
stop. |
| `bin/local-dev.sh <svc>` | Rebuild that service (sbt
`<Project>/dist`, unzip) and bounce it. |
| `bin/local-dev.sh logs <svc>` | `docker compose logs -f <svc>` for
docker, `tail -f` for native logs. |
| `bin/local-dev.sh watch` | Hands-off monitor (text dashboard
auto-refreshing every 2 s). |
## TUI (`bin/local-dev-tui.py`)
A Textual app that takes the dashboard live without flickering or appending
frames to the scrollback (Textual's diff renderer). It shells out to
`bin/local-dev.sh` for every action so behaviour matches the CLI.
| Column | What it shows |
|------------|---------------|
| `●` | State symbol (running / starting / unhealthy / stopped). |
| `SERVICE` | Service name. |
| `PORT` | Port the service listens on. |
| `PID` | Host PID (for native services) or — for docker. |
| `ARTIFACT` | Last build / lock mtime. |
| `BUILD` | `★` source changed since last build (rebuild needed); `↻`
service auto-rebuilds on save (ng serve / bun --watch); blank: up-to-date. |
| `STATE` | Coloured state word. |
**Inputs & shortcuts**
- Type any CLI verb (`u`, `d`, `b`, `<svc>`, `s <svc>`, `l <svc>`…) in the
prompt — same set as the CLI.
- `↑` / `↓` — shell-style history (persisted under
`${TEXERA_LOCAL_DEV_DIR:-/tmp/texera-local-dev}/tui-history`).
- Double-click a service row → tail its log. `Enter` on a focused row does
the same.
- `ESC` — exit the log view, give the dashboard the screen back.
- `Ctrl-C` — cancel the active command; idle, a second tap within 2 s quits.
- `Ctrl-L` — clear / hide the log pane.
- `clear` / `log` / `h` — clear, toggle log visibility, help.
**Banner**
`worktree: <name>` if we're in a non-canonical worktree, `checkout: <name>`
if main checkout. Branch + short SHA + clock alongside.
## Dirty-source detection
`BUILD` ★ uses SHA-1 of all `.scala` / `.java` / `.proto` content under the
service's `src/` plus `common/*/src`, compared against a per-service stamp file
written at the end of each successful build. mtime fast-filter avoids hashing
on every tick. This is robust to `git checkout` touching mtimes without
changing content — the old mtime-only check used to flash ★ after every branch
switch.
## Docker
5 infra containers — postgres, minio, lakefs, lakekeeper, litellm — each
surfaced as its own row, sourced from `docker compose ps -a` cached for 1 s.
The compose stack is the upstream `bin/single-node/docker-compose.yml` plus a
small overlay (`bin/local-dev/docker-compose.override.yml`) that publishes the
infra ports to the host (single-node compose intentionally keeps them
docker-network-only).
## What's intentionally out of scope
- Linux portability: uses `stat -f` (BSD), BSD `lsof` flags, and assumes
Docker Desktop / colima. Linux support is a follow-up — happy to file the issue
and take it.
- CI integration: this is a dev-loop tool. CI continues to use existing
scripts.
## How was this PR tested?
- `bin/local-dev.sh up` from a cold start brings up all 13 services; `down`
tears them down.
- Editing a Scala file flashes ★ in `BUILD`; running `<svc>` clears it; `git
checkout` between branches no longer false-flags ★ (content hash, not mtime).
- TUI tested in iTerm2 on macOS 14. Double-click → logs, ↑/↓ → history, ESC
→ exits log, Ctrl-C twice → quit.
- `bin/local-dev.sh status` runs without a TTY (used in scripting contexts).
## Files
- `bin/local-dev.sh` (new, executable)
- `bin/local-dev-tui.py` (new)
- `bin/local-dev/docker-compose.override.yml` (new — publishes infra ports
for the host-native JVM services to reach)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
