Yicong-Huang opened a new pull request, #7079:
URL: https://github.com/apache/texera/pull/7079
### What changes were proposed in this PR?
The dirty-source fast filter treated "source mtime **equal** to the build
stamp's" as clean, so an edit landing in the same filesystem timestamp tick
as
the stamp write was never rebuilt — `auto` reported `everything up-to-date`
and
bounced nothing.
The window is one tick wide, which sounds negligible but isn't: the stamp is
written at the end of a build, and the natural next action is editing the
file
you were just building. How wide the tick is depends on the filesystem and
kernel clock granularity, not on how fast anyone types.
Both implementations had it, and the shell one is the consequential half — it
gates the rebuild, while `tui.py` only colours the `SRC` column:
| Where | Was | Now |
| --- | --- | --- |
| `main.sh` `svc_src_changed` | `find … -newer "$stamp"` (strictly newer) |
compares against a throwaway marker one second behind the stamp |
| `tui.py` `_newest_mtime_after` | `st_mtime > stamp_mtime` | `>=` |
```
Before: build -> edit within the same tick -> auto: "everything up-to-date"
After: build -> edit within the same tick -> hash compared -> rebuilt
```
The two fixes differ because the two languages can express different things:
`find` has no portable "not older than", so the shell borrows one second of
slack via a marker; Python can say exactly what it means, so it uses `>=`.
The
marker is a throwaway — the stamp itself keeps its real mtime, so the mtime
refresh at the end of the slow path converges exactly as before.
Both directions stay conservative rather than wrong. Widening the filter only
enlarges the *candidate* set; the content hash underneath — which was always
correct — still makes the decision. And it self-heals: the first tick after a
build takes the hash path once, finds the content unchanged, and bumps the
stamp
past the sources, so later ticks are cheap again. A source that is genuinely
older than the stamp is not dragged in by the slack (asserted below).
### Any related issues, documentation, discussions?
Closes #7075
### How was this PR tested?
The bug's own symptom was an existing test that failed only on some
filesystems. This PR adds a deterministic version of it —
`os.utime` forces the colliding mtime instead of racing for it — so it holds
on
every platform:
```
$ python -m pytest bin/local-dev/tests/ -q
44 passed
```
That count matters: before this change the suite was `1 failed, 42 passed` on
this machine, the failure being `test_is_dirty_after_seed_then_edit`, which
had
been hitting the same bug by accident wherever the filesystem granularity was
coarser than its two consecutive writes. It now passes for the right reason
rather than by platform luck.
The shell side is covered by the mechanism it actually uses, with the
colliding
mtime forced by `touch -r`:
```
$ bash bin/local-dev/tests/test_local_dev_sh.sh
...
✓ stamp backdate: bare `-newer $stamp` misses an equal mtime (the bug)
✓ stamp backdate: backdated marker sees the equal-mtime edit
✓ stamp backdate: a clearly older source stays clean
✓ stamp backdate: missing file is a quiet no-op
✓ svc_src_changed compares against the backdated marker
55 passed, 0 failed
```
The first of those characterises the bug itself, so it will start failing if
a
future `find` learns to include equal mtimes — at which point the marker can
go.
Reproduction outside the suite, for the record:
```
$ D=$(mktemp -d); mkdir -p $D/src; : > $D/stamp; : > $D/src/A.scala
$ touch -r $D/stamp $D/src/A.scala # identical mtimes
$ find $D/src -name '*.scala' -newer $D/stamp -print
# empty — A.scala is invisible
```
Not covered: an end-to-end `auto` that rebuilds off a same-tick edit. Forcing
that on the real stack means winning the same race the test now sidesteps, so
the suite's deterministic version is the check that carries the weight here.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
