On Wed, Jul 29, 2026 at 11:27:40AM -0400, Richard Kimberly Heck wrote:
> If it weren't too difficult, a preference setting for word-only diffs might
> be welcome. That would speed things up a ton.
Right, at the same time, while chasing the littering in the result can be
written as small surgical touch - the heavy lifting is really to get the
heuristics right conceptually, not the code - the shift to word-based diff
will be somewhat massive :/
Anyway, we are not there yet, as Vincent rightly pointed out in the bug, the
same problem re-occurs on the word-level, I spent some time playing with
examples to nail down the rules for word-based spillover. Below is the attempt
for word-level heuristics.
Pavel
-----
IVb. Detector for "this phrase's word diff is the scattered-noise"
The scatter which IVa removes inside a word is not special to characters.
Myers' algorithm
fragments the diff wherever old and new coincide by chance rather than by
meaning, and that
recurs one level up: coincidental common words scatter phrase just as
coincidental common
characters scattered word. Compare:
characters (IVa): unique -> single {u}[si]n{iqu}[gl]e
("n" by chance)
words (IVb): buy a car -> sell a truck {buy}[sell] a {car}[truck]
("a" by chance)
Both split one conceptual replacement around a meaningless survivor. IVa
collapsed the
character survivor; IVb collapses the word survivor - same idea, the next
level up.
Now IVa and IVb aren't symmetric because word-level survivors are harder to
classify than
char-level crumbs - a single absolute threshold 3 suffices for characters,
but words need
"small relative to the surrounding edits" and "not too large in absolute
terms".
The good news is that we can stop at this level. The probability of
coincidentally shared word
is still decent, but probability of such paragraph is nil - that's our 50
character cap.
* Terminology (the same objects as IVa, one level up):
"Continuous run" - one edit as a whole: at most one deleted block {...}
then one inserted
block [...] (ie "{...}", "[...]", or "{...}[...]"),
exactly as in IVa.
Its length is max(deleted chars, inserted chars) -
how much it replaced.
E.g. {understanding}[memorizing] is one run, length
max(13,10)=13.
We use max, not sum: the two sides are two versions
of the same replaced
chunk.
"Survivor" - an unchanged stretch (whole words + spaces) between two
continuous runs;
the word-level version of IVa's interior survivor.
* Now the heuristic:
Collapse a survivor into one combined {old}[new] run when it is both
1) short RELATIVELY: (survivor <= left_run) && (survivor <= right_run)
eg1: {eat}[cook] a {pie}[cake] -> {eat a pie}[cook a cake]
(survivor " a "=3 <= runs 4,4 -> short ->
collapse)
eg2: {buy}[rent] a house near {work}[town]
(survivor " a house near "=14 > runs 4,4
-> not short -> keep)
2) short ABSOLUTELY: survivor <= 50 chars (the only knob of IVb - max
collapsed survivor size)
eg: an identical clause "the meeting is confirmed for next
Tuesday afternoon"
(51 chars between two edits exceeds 50 -> keep)
Otherwise keep the survivor shown as unchanged.
Applied repeatedly until nothing more collapses (a merge grows the run
and may then swallow
the next survivor).
Insets and paragraph breaks are hard anchors: never collapsed or crossed
(so a footnote or
graphic is never duplicated, and every rewrite stays within one
paragraph).
* Consequences
1. A short accidental survivor between two larger rewrites is collapsed
into one clean
run - the relative test.
2. Meaningful common text between small edits is kept: the relative test
self-scales,
so " a house near " survives when only the words around it changed.
3. A long identical span is always kept, even when flanked by
still-larger rewrites:
the absolute cap wins, because a long span cannot be an accidental
coincidence.
4. The trigger is the survivor's length against its neighbours and the
cap, never the
amount of surrounding change.
--
lyx-devel mailing list
[email protected]
https://lists.lyx.org/mailman/listinfo/lyx-devel
