Years ago we did some testing with HTTP compression for search results with the Ultraseek search engine. It wasn’t faster. It was sometimes slower.
Once you have enough RAM, search is a CPU-limited problem. HTTP compression uses more CPU to save network bandwidth. But search isn’t limited by network bandwidth, so this uses more of the bottleneck resource (CPU) to reduce usage of a plentiful resource (network bandwidth). Look at the amount of data going in and out of your nodes. I bet it is far below the maximum. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Feb 21, 2019, at 6:07 AM, Jörn Franke <jornfra...@gmail.com> wrote: > > You could also change the responsewriter from json to javabin to improve > performance. > Or increase network bandwidth. Then often people fetch more from solr than > they need. There is a huge saving potential. Increasing the cores for https > encryption can sometimes help. > > Compression also leads to other issues (performance but potentially also > security wise). > >> Am 21.02.2019 um 12:03 schrieb Luthien Dulk <maike.d...@europeana.eu>: >> >> hi all, >> >> I was wondering if anyone could point me in the right direction. >> >> I am looking into whether enabling Gzip HTTP compression for our Solr >> clusters (all running Solr 6.6.5) would help performance; my problem is that >> I can’t figure out how to do that. >> >> Our infrastructure setup is like this: our applications are running on a >> Cloud Foundry PAAS environment, but our Solr clusters run elsewhere. >> Communication between applications and Solr clusters is secured by firewalls >> on every Solr machine (we do have a Socks Proxy set up in the CF >> environment, but unfortunately we can't use that for Solr because of the >> incompatibility between Zookeeper and Java Nio I/O - much to the chagrin of >> our sysadmin). >> >> We think that HTTP compression might be very interesting for us because of >> the hight volume of traffic between two separate environments. >> >> Here’s what I found out so far: >> >> (re. config changes in Solr’s embedded Jetty) >> - I’m aware that this is mostly a matter of configuring Jetty; >> - it seems that this should preferably be set in the solr-jetty-context.xml >> file; >> - this seems to relate to enabling Jetty's “GzipHandler” >> >> (re. gzip ‘module’ activation ..?) >> - it puzzles me that >> https://aroratimus.blogspot.com/2017/08/jettyserver-9.html mentions that >> Jetty’s GzipHandler should be enabled using two files not found in Solr's >> embedded Jetty: server/etc/jetty-gzip.xml and server/modules/gzip.mod (they >> are available when installing Jetty separately though); >> - apparently, Jetty's Gzip module should be activated by adding >> —add-to-start=gzip to the server startup command. For the embedded Jetty in >> Solr, it seems that this would require changing the solr startup script >> >> (re. changes in Solr client) >> - the calling application should add the HTTP Accept-Encoding: gzip, deflate >> ( according to >> https://menelic.com/2015/12/04/deploying-solrcloud-across-multiple-data-centers-dc-performance/ >> ) >> >> >> I wonder, has anyone ever got this working? In particular: >> >> - is that gzip ‘module’ activation necessary? That would seem a bit >> far-fetched, because it involves files not found in the Solr installation >> and possibly hacking the Solr startup script; >> - what did you add to solr-jetty-context.xml in order to enable gzip >> compression? >> >> >> I suppose that situations with high volumes of external network traffic >> between Solr and Client must be quite rare. Otherwise I’d think that a >> feature that potentially offers such obvious benefits (one of the pages >> above mentions a drop of 75% of network traffic and a 60% faster response >> time) would have been turned into an “enable http compression yes/no” >> setting by now :) >> >> Anyhow, we’re stuck with it … I hope I can get it working. >> >> >> Thank in advance for any advice! >> >> Luthien >> Api developer >> Europeana.eu >> >> >> >> >> >> >> -- >> Disclaimer: This email and any files transmitted with it are confidential >> and intended solely for the use of the individual or entity to whom they >> are >> addressed. If you have received this email in error please notify the >> system manager. If you are not the named addressee you should not >> disseminate, >> distribute or copy this email. Please notify the sender >> immediately by email if you have received this email by mistake and delete >> this email from your >> system.
