On Sunday, 22 May 2016 at 19:21:13 UTC, Walter Bright wrote:
On 5/22/2016 3:32 AM, Era Scarecrow wrote:
Curiously the extra 95 symbols actually is just enough to keep
compression up and bring speed to about 37% speed gain, with
it looks like no disadvantages.
You're doing the gods' work, Era!
Maybe... I have to wonder if LZ4 from the looks of things will
take over instead. Have to see how it performs (and if we can
integrate it); Meanwhile the main advantage of id_compress (as
I'm tinkering with it) is it makes no changes to the
signature/behavior so taking advantage of it's boost in speed can
be immediate (although it will no doubt have a bit weaker
compression, and demangling needs to be re-written for it).
Honestly the majority of speed gains with id_compress are simply
from having a much smaller window(1023 to 220), also resulting in
a smaller footprint for the compressed output (3 bytes to 2
bytes).
Compare to the speed gains from my new algorithm (at 20% faster)
is pretty constant, and retains a HUGE window (pre-scans 8k, has
a 2k window) that it can find matches from, so it will win in
compression.
If LZ4 is half as good as the initial results are then I'd go
with it instead.
