Re: [Tigervnc-devel] Simplifying compression levels in the TigerVNC GUI
On 1/25/14 3:04 PM, DRC wrote: > -- If someone decides that they want to jack it up to CL 9, this will > never have a negative effect on compression ratio relative to CL 6, but > whether it has a significant benefit will depend on the workload. On > average, 2D datasets will compress 10% better with a 10% loss in > performance, and on average, 3D datasets will compress 3% better with a > 10% loss in performance. That's a lot nicer of a trade-off than the > current CL 9 provides (basically no compression benefit with 4-5x more > CPU usage.) I should clarify that that 10% gain with the new proposed CL 9 is generally only realized if the JPEG quality is medium or higher. With low-quality JPEG, the existing TigerVNC CL 5 already performs almost on par with TightVNC in most cases (except that pesky photos dataset again.) But anyhow, the new CL 9 offers a real option, whereas the existing CL 9 is not a real option. -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Tigervnc-devel mailing list Tigervnc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tigervnc-devel
Re: [Tigervnc-devel] Simplifying compression levels in the TigerVNC GUI
Sorry about that previous bit of noise. I thought I had stumbled upon an even simpler way of accomplishing this with TigerVNC, but it was because of a spreadsheet error. Anyhow, the original analysis is all very buried in the libvncserver-common mailing list in multiple threads, with lots of hemming and hawing, and the spreadsheet is an eye chart. Thus, I'm going to repeat the analysis here, so you can have the information all in one place. Basically, this all started with convincing the libvncserver developers to support the TurboVNC encoder as a build option, mainly because I wanted to be able to use that encoder with x11vnc without patching the libvncserver code. They agreed, but this led to discussions about why it was then necessary to keep supporting the TightVNC 1.3.x encoder, so ultimately we agreed that replacing the TightVNC encoder with the TurboVNC encoder was the cleanest solution. However, this led to questions about whether the TurboVNC encoder could compress as "tightly" in all cases, which prompted me to do this research. Basically, the tests I conducted were similar to the ones I performed when designing the TurboVNC codec, using the same canonical datasets (8 2D datasets from Constantin and 12 3D datasets of my own design. Refer to http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf for dataset descriptions.) NOTE: the 8-bit test (freshmeat-8) is not particularly realistic anymore, since it doesn't use JPEG, so it was excluded from this analysis, but I also verified that including it would not have changed any of the conclusions (that is, I verified that that dataset was never an outlier.) Since libvncserver has to support both libjpeg and libjpeg-turbo, the libvncserver developers also needed reassurance that the performance with the new TurboVNC encoder was not solely dependent on the use of libjpeg-turbo. Thus, I tested the Turbo encoder against both libjpeg and libjpeg-turbo, but that aspect of the research is largely irrelevant for TigerVNC. Further, I tested the encoder with 4:2:2 Q80 and Q37 JPEG images, so as to get a true apples-to-apples comparison with low-quality JPEG (JPEG Quality Level 4) and medium-quality JPEG (JPEG Quality Level 9) in TightVNC 1.3.x (since TightVNC 1.3.x only supports 4:2:2 subsampling.) In all cases, I compared aggregate 2D and 3D compression ratio and performance across the series of datasets as well as looked for any outliers that diverged significantly from the average. I started by finding the "maximum useful" compression level in TightVNC, which I defined as the level beyond which it was impossible to get more than a 5% improvement in compression ratio by going any higher. For almost all datasets, this was CL 5, but a few were able to get some incremental benefit (no more than 10%) by moving from CL 5 to CL 6. No datasets saw more than a 5% improvement in compression ratio by moving from CL 6 to CL 9. TightVNC CL 9 vs. TightVNC CL 6: -- Compression Ratio for 2D datasets: -0.58% to +3.4% (avg +1.0% for Q80, +2.2% for Q37) -- Compression Ratio for 3D datasets: -2.3% to +5.0% (avg +0.82% for Q80, +1.3% for Q37) -- Speedup for 2D datasets:-83% to -46%(avg -74% for Q80, -75% for Q37) -- Speedup for 3D datasets:-87% to -4.7% (avg -78% for Q80, -78% for Q37) Thus, CL 6 in TightVNC is the maximum useful compression level and served as the compression ratio target for the TurboVNC encoder. Note that switching from CL 6 to CL 9 in the TightVNC encoder increased CPU time by, on average, 4-5x without providing any significant increase in compression ratio for any dataset I tested. Why does CPU time matter, you ask? Because using CL 9 in TightVNC produces a situation in which the CPU usage on the server is so high that it can become the primary bottleneck, even on low-bandwidth connections. For some datasets, TightVNC CL 9 would have been unable to fill even a 2-megabit pipe. CL 6, in contrast, could generally fill a 10-20 megabit pipe without any significant reduction in compression ratio relative to CL 9. NOTE: I ran the same tests without JPEG enabled as well (using Raw subrects in place of JPEG, not using gradient encoding), just as a sanity check, and CL 6 proved to be the maximum useful compression ratio in that case as well, with similar statistics to the above. I then tested TurboVNC's CL 2 (which is similar to TigerVNC's CL 2, and at the time this was the maximum compression level available in TurboVNC) against TightVNC CL 6: -- Compression Ratio for 2D datasets:-29% to +1.1% (avg -16% for Q80, -12% for Q37) -- Compression Ratio for 3D datasets:-14% to +131% (avg +51% for Q80, +4.5% for Q37) -- Speedup for 2D datasets (libjpeg):-4.9% to +115% (avg +53% for Q80, +50% for Q37) -- Speedup for 3D datasets (libjpeg):-12% to +210% (avg +91% for Q80, +52% for Q37) -- Speedup for 2D data
Re: [Tigervnc-devel] Simplifying compression levels in the TigerVNC GUI
Ignore previous message. I spoke too soon. I will send a complete analysis shortly. On 1/23/14 4:01 AM, Pierre Ossman wrote: > On Wed, 22 Jan 2014 17:30:25 -0600, > DRC wrote: > >> >> My proposal is for TigerVNC to adopt four compression modes: CL 0, CL >> 1, CL 2, and CL 5. CL 3 and 4 would map to 2, and CL 6-9 would map to >> 5, and the GUI could be restructured so that it sets the compression >> level to "low, high, and very high", with a warning that "very high" is >> only better than "high" in some rare cases. >> >> Just a suggestion. I can provide a server-side patch for this if requested. >> > > Interesting stuff. I'd definitely like to know more about the > discussion and the testing that was done. > > I'm shuffling around all of that code at the moment though, so please > hold off on that patch for a while. :) > > Rgds > -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk ___ Tigervnc-devel mailing list Tigervnc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tigervnc-devel
Re: [Tigervnc-devel] Simplifying compression levels in the TigerVNC GUI
Further analysis reveals that TigerVNC's Compression Level 5 and 6
(which are basically identical when JPEG is enabled) can already produce
roughly the same compression ratio (within +/- 5%) as the maximum useful
mode in TightVNC 1.3.x, so actually, I think the only thing that might
need addressing here is the fact that the higher Zlib levels are not
needed for Compression Levels 7-9. I can find no dataset for which
moving from CL 6 to CL 9 in TigerVNC produces more than a 5% increase in
compression ratio, and generally this move increases CPU time by a
factor of 4-5. Thus, I would suggest the following modification, which
extends the Zlib levels from CL 6 through CL 7-9 as well:
--- a/common/rfb/TightEncoder.cxx
+++ b/common/rfb/TightEncoder.cxx
@@ -70,9 +70,9 @@ const TIGHT_CONF TightEncoder::conf[10] = {
{ 65536, 2048, 12, 6, 7, 3, 32, 96, 62, SUBSAMP_422 }, // 4
{ 65536, 2048, 12, 7, 8, 4, 32, 96, 77, SUBSAMP_422 }, // 5
{ 65536, 2048, 16, 7, 8, 5, 32, 96, 79, SUBSAMP_NONE }, // 6
- { 65536, 2048, 16, 8, 9, 6, 64, 96, 86, SUBSAMP_NONE }, // 7
- { 65536, 2048, 24, 9, 9, 7, 64, 96, 92, SUBSAMP_NONE }, // 8
- { 65536, 2048, 32, 9, 9, 9, 96, 96,100, SUBSAMP_NONE } // 9
+ { 65536, 2048, 16, 7, 8, 5, 64, 96, 86, SUBSAMP_NONE }, // 7
+ { 65536, 2048, 24, 7, 8, 5, 64, 96, 92, SUBSAMP_NONE }, // 8
+ { 65536, 2048, 32, 7, 8, 5, 96, 96,100, SUBSAMP_NONE } // 9
};
On 1/23/14 4:01 AM, Pierre Ossman wrote:
> On Wed, 22 Jan 2014 17:30:25 -0600,
> DRC wrote:
>
>>
>> My proposal is for TigerVNC to adopt four compression modes: CL 0, CL
>> 1, CL 2, and CL 5. CL 3 and 4 would map to 2, and CL 6-9 would map to
>> 5, and the GUI could be restructured so that it sets the compression
>> level to "low, high, and very high", with a warning that "very high" is
>> only better than "high" in some rare cases.
>>
>> Just a suggestion. I can provide a server-side patch for this if requested.
>>
>
> Interesting stuff. I'd definitely like to know more about the
> discussion and the testing that was done.
>
> I'm shuffling around all of that code at the moment though, so please
> hold off on that patch for a while. :)
>
> Rgds
>
--
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
___
Tigervnc-devel mailing list
Tigervnc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/tigervnc-devel
Re: [Tigervnc-devel] Simplifying compression levels in the TigerVNC GUI
On Wed, 22 Jan 2014 17:30:25 -0600, DRC wrote: > > My proposal is for TigerVNC to adopt four compression modes: CL 0, CL > 1, CL 2, and CL 5. CL 3 and 4 would map to 2, and CL 6-9 would map to > 5, and the GUI could be restructured so that it sets the compression > level to "low, high, and very high", with a warning that "very high" is > only better than "high" in some rare cases. > > Just a suggestion. I can provide a server-side patch for this if requested. > Interesting stuff. I'd definitely like to know more about the discussion and the testing that was done. I'm shuffling around all of that code at the moment though, so please hold off on that patch for a while. :) Rgds -- Pierre Ossman Software Development Cendio AB http://cendio.com Teknikringen 8 http://twitter.com/ThinLinc 583 30 Linköpinghttp://facebook.com/ThinLinc Phone: +46-13-214600http://plus.google.com/+CendioThinLinc A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? signature.asc Description: PGP signature -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk___ Tigervnc-devel mailing list Tigervnc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tigervnc-devel
