CPU was at 100%, it was not IO bound. --wunder On 10/30/08 8:58 AM, "christophe" <[EMAIL PROTECTED]> wrote:
> Gziping on disk requires quite some I/O. I guess that on the fly zipping > should be faster. > > C. > > Walter Underwood wrote: >> About a factor of 2 on a small, optimized index. Gzipping took 20 seconds, >> so it isn't free. >> >> $ cd index-copy >> $ du -sk >> 134336 . >> $ gzip * >> $ du -sk >> 62084 . >> >> wunder >> >> On 10/30/08 8:20 AM, "Otis Gospodnetic" <[EMAIL PROTECTED]> wrote: >> >> >>> Yeah. I'm just not sure how much benefit in terms of data transfer this >>> will >>> save. Has anyone tested this to see if this is even worth it? >>> >>> >>> Otis >>> -- >>> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >>> >>> >>> >>> ----- Original Message ---- >>> >>>> From: Erik Hatcher <[EMAIL PROTECTED]> >>>> To: solr-user@lucene.apache.org >>>> Sent: Thursday, October 30, 2008 9:54:28 AM >>>> Subject: Re: replication handler - compression >>>> >>>> +1 - the GzipServletFilter is the way to go. >>>> >>>> Regarding request handlers reading HTTP headers, yeah,... this will >>>> improve, >>>> for >>>> sure. >>>> >>>> Erik >>>> >>>> On Oct 30, 2008, at 12:18 AM, Chris Hostetter wrote: >>>> >>>> >>>>> : You are partially right. Instead of the HTTP header , we use a request >>>>> : parameter. (RequestHandlers cannot read HTP headers). If the param is >>>>> >>>>> hmmm, i'm with walter: we shouldn't invent new mechanisms for >>>>> clients to request compression over HTTP from servers. >>>>> >>>>> replicatoin is both special enough and important enough that if we had to >>>>> add special support to make that information available to the handler on >>>>> the master we could. >>>>> >>>>> but frankly i don't think that's neccessary: the logic to turn on >>>>> compression if the client requests it using "Accept-Encoding: gzip" is >>>>> generic enough that there is no reason for it to be in a handler. we >>>>> could easily put it in the SolrDispatchFilter, or even in a new >>>>> ServletFilte (i'm guessing iv'e seen about 74 different implementations of >>>>> a GzipServletFilter in the wild that could be used as is. >>>>> >>>>> then we'd have double wins: compression for replication, and compression >>>>> of all responses generated by Solr if hte client requests it. >>>>> >>>>> -Hoss >>>>> >> >>
