On 10/24/19 12:58 AM, Leyne, Sean wrote:
-----Original Message-----
From: Alex Peshkoff <peshk...@mail.ru>
Sent: Wednesday, October 23, 2019 3:38 AM
On 10/22/19 6:19 PM, Leyne, Sean wrote:
It's faster but that makes no difference for our use cases. When
working with LAN passing uncompressed data is anyway the best option.
Compression makes sense only for very slow and/or very long (many
hops) WAN. Under that conditions compression's performance does not
matter.
If you are using a metered connection to access a database, the amount of data
exchanged does matter.
Further, given the obvious performance benefit of zstd over other
compression algorithms (https://community.centminmod.com/threads/round-3-
compression-comparison-benchmarks-zstd-vs-brotli-vs-pigz-vs-bzip2-vs-xz-
etc.17259/), the project should replace zlib with zstd.
Personally, I would like to see the project adopt "library" type for compression
approach which would allow me to use the 7zip library with the LZMA2 algorithm
for creating gbaks and/or nbackups, it is multi-threaded (CPU intensive) but
generates the best compression.
In that case looks like we need one more type of plugin?
Certainly, it can be added in next protocol version, in fb5.
I think there several issues/tickets/changes here:
1- Replace zlib with zstd for over-the-wire compression client communication
Not replace! Do you really want to break connection from/to old
clients/servers?
2- Replace zlib with zstd in gbak (really nice to have in v4.0)
Here replace might be possible but it's not good idea to delay fb4 any
more. Next, libz is installed all over the world. With zstd we are
lickely to have one more installation error issue (i.e. even greater delay).
3- Add zstd compression to nbackup (really nice to have in v4.0)
This makes no sense. Internal compression in gbak became required
feature after adding encrypted backup support because compressing gbak's
stdout using pipeline does not work with encrypted backup. Nbackup of
encrypted database (btw, still not supported in all cases) is always
encrypted and can't be compressed. Decompression of multiple .nbak files
needed for restore works in posix for a long time (Paul's clients who
needed it use linux on servers). All decompression utilities are
supported with one requirement - utility should be able to write
decompressed data to stdout. What is sooner of all good idea is to have
windows port. BTW, nobody cares about it, which probably demonstrates
how "much" people need nbackup compression...
4- Adding support for compression libraries/plugins for over-the-wire
compression
5- Adding support for compression libraries/plugins to gbak
When I was adding wire compression it was decided to wait and see do we
really need plugin here. And yes - looks like a moment to have that
plugin came, and this should be done. Same for gbak. Regarding this 2
points I 100% agree with your suggestion.
What makes no sense is to add any algorithm before adding plugins. This
will greatly complicate further wire protocol and gbak file with a need
to support pre-plugins formats. I already had that type of issue with
windows trusted authentication support in plugins-enabled environment
and I do not want to repeat it.
6- Adding support for compression libraries/plugins to nbackup
As I've already mentioned this makes no sense.
I am happy to create Tracker entries, if there is agreement.
I think we need single item: add compression plugins. Same plugin can be
used for both wire and backup, the only usage-specific thing we need is
two entries in firebird.conf:
BackupCompressor = Plug1, Plug2, ...
WireCompressor = PlugA, PlugB, ...
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
