On Tue, Mar 30, 2021 at 02:25:14PM +0500, ???? ??????? wrote: > I would wait for feedback from some other high load projects. From my > observation it is significant benefit.
Originally slz was written for use in haproxy to solve the huge memory consumption that comes with zlib (~288kB of context per stream IIRC). SLZ needs much less and uses ~40 bytes or so per stream, and by keeping less context, has less opportunities for high compression ratios. This simplified its lookup algorithms, and as a byproduct made it significantly faster. The only sane way I've seen to use zlib is to limit its CPU usage. When it reaches a target ratio (typically 80%), compression automatically disables itself, so overall the savings-to-cpu usage makes it less interesting for zlib in our use cases. The only case where zlib is more interesting is for those having a small line and who prefer to spend more CPU cycles to get more savings. One reasonable approach we could think of would consist in importing SLZ directly into haproxy now. I didn't want to do it initially because the code was young and I wanted it to live its own life and get its own fixes. Now the burn-in test is long finished, and the only updates over the last 4 years were for performance improvements or fixes for use cases outside of haproxy. That's only 1500 lines of code to integrate and it would certainly simplify a lot of things. We could even imagine to always build with compression enabled, SLZ when not specified otherwise zlib when asked for it. Just a few ideas. Willy
