> Which brings to another question, what about persistent
> connections? Are they possible?
Yes, you can hold HTTP/1.1 connections open to the REST server. If the client
uses HTTP/1.1 you can also use request pipelining to do more work on a given
connection, though you will need to be careful if pipelining non-idempotent
changes. The 1.1 spec also requires a server must send its responses to
requests in the same order that the requests were received, so a long running
request mid pipeline will stall it.
> (Compression from client side would be really nice, we
> could write our own wrapper I guess to make transmission
> more efficient).
Let me look in to a filter that wraps both output and input side processing. A
quick search did not turn up anything with available source so probably will
have to implement from scratch. But at least we could borrow code from Jetty's
GzipFilter to handle half of it.
- Andy
--- On Thu, 11/25/10, Jack Levin <magn...@gmail.com> wrote:
> From: Jack Levin <magn...@gmail.com>
> Subject: Re: REST compression support (was Re: question about meta data query
> intensity)
> To: user@hbase.apache.org, apurt...@apache.org
> Date: Thursday, November 25, 2010, 9:56 AM
> Yes, we will... issue is, our product
> collects data from various
> sources and then stuffs it en'mass into HBASE. It
> makes no sense to
> do one PUT at a time because we would be opening 1000 tcp
> connections.
> Which brings to another question, what about persistent
> connections?
> Are they possible? (Compression from client side
> would be really
> nice, we could write our own wrapper I guess to make
> transmission more
> efficient).
>
> -Jack
>
> On Wed, Nov 24, 2010 at 10:46 PM, Andrew Purtell <apurt...@apache.org>
> wrote:
> >> Btw, does it mean, I can send in a compressed
> query? Or only receive
> >> compressed data from REST or both?
> >
> > Jetty's GzipFilter only wraps response handling.
> >
> > I tested to see if Jetty has some built in support for
> Content-Encoding: gzip for PUT or POST and it appears not:
> >
> > Error 415 Unsupported Media Type
> >
> > You going to be posting large amounts of compressible
> data?
> >
> > - Andy
> >
> > --- On Wed, 11/24/10, Jack Levin <magn...@gmail.com>
> wrote:
> >
> >> From: Jack Levin <magn...@gmail.com>
> >> Subject: Re: REST compression support (was Re:
> question about meta data query intensity)
> >> To: user@hbase.apache.org,
> apurt...@apache.org
> >> Date: Wednesday, November 24, 2010, 2:21 PM
> >> Btw, does it mean, I can send in a
> >> compressed query? Or only receive
> >> compressed data from REST or both?
> >>
> >> -Jack
> >>
> >> On Wed, Nov 24, 2010 at 10:15 AM, Andrew Purtell
> <apurt...@apache.org>
> >> wrote:
> >> > Regards compressing the HTTP transactions
> between the
> >> REST server and REST client we punted on this back
> when
> >> Stargate had a WAR target so we could push that
> off to the
> >> servlet container configuration. Thanks for the
> question,
> >> which reminded me... I have just committed
> HBASE-3275, which
> >> is a trivial patch to support Accept-Encoding:
> gzip,deflate
> >> >
> >> > Index:
> >>
> src/main/java/org/apache/hadoop/hbase/rest/Main.java
> >> >
> >>
> ===================================================================
> >> > ---
> >>
> src/main/java/org/apache/hadoop/hbase/rest/Main.java
> >> (revision 1038732)
> >> > +++
> >>
> src/main/java/org/apache/hadoop/hbase/rest/Main.java
> >> (working copy)
> >> > @@ -37,6 +37,7 @@
> >> > import org.mortbay.jetty.Server;
> >> > import org.mortbay.jetty.servlet.Context;
> >> > import
> org.mortbay.jetty.servlet.ServletHolder;
> >> > +import org.mortbay.servlet.GzipFilter;
> >> >
> >> > import
> >>
> com.sun.jersey.spi.container.servlet.ServletContainer;
> >> >
> >> > @@ -132,6 +133,7 @@
> >> > // set up context
> >> > Context context = new Context(server,
> "/",
> >> Context.SESSIONS);
> >> > context.addServlet(sh, "/*");
> >> > + context.addFilter(GzipFilter.class,
> "/*", 0);
> >> >
> >> > server.start();
> >> > server.join();
> >> >
> >> > Regards interactions between HBase client and
> server,
> >> there is no option available for compressing
> Hadoop RPC.
> >> >
> >> > - Andy
> >> >
> >> >
> >> > --- On Wed, 11/24/10, Jack Levin <magn...@gmail.com>
> >> wrote:
> >> >
> >> >> From: Jack Levin <magn...@gmail.com>
> >> >> Subject: Re: question about meta data
> query
> >> intensity
> >> >> To: user@hbase.apache.org,
> >> apurt...@apache.org
> >> >> Date: Wednesday, November 24, 2010, 9:25
> AM
> >> >>
> >> >> Yes, but that does not alleviate CPU
> contention
> >> should there be too
> >> >> many queries to a single region
> server. On a
> >> separate topic, is
> >> >> 'compression' in the works for REST
> >> gateway? Similar to
> >> >> mysql_client_compression? We plan to
> drop in
> >> 500K or
> >> >> more queries at a time into the REST, and
> it would
> >> be interesting
> >> >> to see the performance gain against
> uncompressed
> >> data.
> >> >>
> >> >> Thanks.
> >> >>
> >> >> -Jack
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> >
> >
>
