On 3/3/2016 9:10 AM, Slavomir Skopalik wrote:
Who cares how effect of smaller space occupied by the record is called,
important thing is a result - size and certainly time for compression
and specially decompression.
Elekt Labs RLE is faster then current one, because works on larger
blocks (up ~4MB).
Excuse me, but faster than the current one is not the question. The
question is what is the best one.
And, I dare say, performance on a single pathological case is not very
interesting. A better architectural solution for your case is for
Firebird to treat large fields as blobs stored separately from the
record for the same reasons I invented blobs in the first place:
* There expensive to fetch and frequently aren't even referenced
* They're updated infrequently and can be shared among record versions
* Storage as part of the record has a huge effect on the cost of
exhaustive retrievals.
So whether compression scheme is better or worse than alternatives, it's
a very poor solution to the problem you have posited.
See performance results here:
http://elektlabs.cz/fbrle/
I had new version with little better compression and speed, but it is
marginal.
But back to original question.
Is it right time to change record/fragment compression/encoding?
We are going to have soon TTG solution re. list of v.4 features.
Can you provide more info about this, I miss this :(
If we decide not to change ODS major in v.4 commit of LZ4 appears
suspicious. I.e. I think you should wait a bit.
LZ4 has only marginal effect on my DBs.
I'm using short records (after RLE about 30 bytes).
But on large record, mainly texts, the benefits is significant.
What is also interesting - how does your compression work with records
other than containing single varachar(8000)? Some mix of
integrs/floats/strings?
Yes. As I wrote, Elekt Labs RLE works well up 4MB record size.
It is mean, 4MB of zeroes is packed into 3 bytes.
Personally I (after my own experiments with LZ4 for wire compression)
believe that it should fit well for records compression, but it's always
good to provide more facts. What is looking strange for me is that LZ4
Because LZ4 has limit 1:255, current RLE has limit 1:64.
VARCHAR(8000) in UTF8 has 32k.
When is null, Elekt Labs RLE will pack into 3 bytes, LZ4 into ~125 +
overhead, current RLE into 500.
works better when used over RLE. As far as I understand it's algorithm
is should do it's job fine without RLE. Results of comparisons for data
mix is very interesting.
Last but not least - did you run our test suites with your patch?
Only our internal tests + we was using in my company for a while without
problems.
We stops, because I need testing 2.5.4 and 2.5.5 before deploy to customers.
May be Pavel Cisar did a tests, good time to ask him.
Slavek
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
