Re: Zipping HTTP Request
On Mon, 2018-05-07 at 09:24 -0600, Gary Gregory wrote:
> What could be more useful is a more general solution in a new module
> based
> on Apache Commons Compress that calls
> org.apache.commons.compress.compressors.CompressorStreamFactory.creat
> eCompressorOutputStream(String,
> OutputStream).
>
> That would give you support for:
>
> GZIP
> BZIP2
> XZ
> PACK200
> LZMA
> DEFLATE
> SNAPPY_FRAMED
> LZ4_BLOCK
> LZ4_FRAMED
> ZSTANDARD
>
> Gary
>
Hi Philippe
Yes, this sounds like a useful feature. It would likely be too big for
4.5.x but could go into 4.6.x or 5.x.
It would also be super if you could look into other compression feature
requests at the same time.
Cheers
Oleg
> On Mon, May 7, 2018 at 9:15 AM, Philippe Mouawad <
> p.moua...@ubik-ingenierie.com> wrote:
>
> > Hello,
> > There are sometimes a requirement to automatically gzip the request
> > body
> > whenever Content-Encoding is set to gzip.
> >
> > Would you find it interesting to add this feature to HttpClient ?
> >
> > The implementation would look like:
> >
> > private static final class GzipHttpRequestInterceptor
> > implements
> > HttpRequestInterceptor {
> > @Override
> > public void process(HttpRequest request, HttpContext
> > context)
> > throws HttpException, IOException {
> > if(request instanceof HttpEntityEnclosingRequest) {
> > Header header = request.getFirstHeader("
> > Content-Encoding");
> > if(header != null &&
> > "gzip".equals(header.getValue())) {
> > HttpEntityEnclosingRequest enclosingRequest =
> > (HttpEntityEnclosingRequest) request;
> > HttpEntity entity =
> > enclosingRequest.getEntity();
> > ByteArrayOutputStream out = new
> > ByteArrayOutputStream();
> > try (GZIPOutputStream gzipOS = new
> > GZIPOutputStream(out)) {
> > entity.writeTo(gzipOS);
> > }
> > enclosingRequest.setEntity(new
> > ByteArrayEntity(out.toByteArray()));
> > }
> > }
> > }
> > }
> >
> >
> > Use could add it to processing like this:
> >
> > - builder.addInterceptorFirst(new GzipHttpRequestInterceptor()
> > );
> >
> > Or it could be enabled by default.
> >
> >
> > Questions:
> >
> > - Would you accept it as a PR ?
> > - Do you see immediate improvements in the provided code ?
> >
> > Thanks
> > --
> > Regards
> > Philippe Mouawad.
> > Ubik-Ingénierie
> >
-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Zipping HTTP Request
Hi,
Brotli is supported in read-only mode:
https://commons.apache.org/proper/commons-compress/limitations.html
This is a limitation of the underlying library org.brotli.dec.
We welcome patches ;-)
Gary
On Mon, May 7, 2018 at 10:50 AM, Christopher Schultz <
ch...@christopherschultz.net> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Gary,
>
> On 5/7/18 11:24 AM, Gary Gregory wrote:
> > What could be more useful is a more general solution in a new
> > module based on Apache Commons Compress that calls
> > org.apache.commons.compress.compressors.CompressorStreamFactory.create
> CompressorOutputStream(String,
> >
> >
> OutputStream).
> >
> > That would give you support for:
> >
> > GZIP BZIP2 XZ PACK200 LZMA DEFLATE SNAPPY_FRAMED LZ4_BLOCK
> > LZ4_FRAMED ZSTANDARD
>
> What, no Brotli[1]? ;)
>
> I'm almost entirely kidding, but most of the above compression methods
> are not supported by web servers. RFC 7231 only defined
> content-encoding tokens for gzip, deflate, and compress/x-compress
> (and "identity"). Mozilla extends that list to include 'br' (for Brotli)
> .
>
> So, while using a generalized method for compression, you might want
> to limit it to actually-useful compression types.
>
> - -chris
>
> [1] https://en.wikipedia.org/wiki/Brotli
>
> > On Mon, May 7, 2018 at 9:15 AM, Philippe Mouawad <
> > p.moua...@ubik-ingenierie.com> wrote:
> >
> >> Hello, There are sometimes a requirement to automatically gzip
> >> the request body whenever Content-Encoding is set to gzip.
> >>
> >> Would you find it interesting to add this feature to HttpClient
> >> ?
> >>
> >> The implementation would look like:
> >>
> >> private static final class GzipHttpRequestInterceptor implements
> >> HttpRequestInterceptor { @Override public void
> >> process(HttpRequest request, HttpContext context) throws
> >> HttpException, IOException { if(request instanceof
> >> HttpEntityEnclosingRequest) { Header header =
> >> request.getFirstHeader(" Content-Encoding"); if(header != null &&
> >> "gzip".equals(header.getValue())) { HttpEntityEnclosingRequest
> >> enclosingRequest = (HttpEntityEnclosingRequest) request;
> >> HttpEntity entity = enclosingRequest.getEntity();
> >> ByteArrayOutputStream out = new ByteArrayOutputStream(); try
> >> (GZIPOutputStream gzipOS = new GZIPOutputStream(out)) {
> >> entity.writeTo(gzipOS); } enclosingRequest.setEntity(new
> >> ByteArrayEntity(out.toByteArray())); } } } }
> >>
> >>
> >> Use could add it to processing like this:
> >>
> >> - builder.addInterceptorFirst(new GzipHttpRequestInterceptor()
> >> );
> >>
> >> Or it could be enabled by default.
> >>
> >>
> >> Questions:
> >>
> >> - Would you accept it as a PR ? - Do you see immediate
> >> improvements in the provided code ?
> >>
> >> Thanks -- Regards Philippe Mouawad. Ubik-Ingénierie
> >>
> >
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlrwg+wACgkQHPApP6U8
> pFhYtQ//f55PE47LkRrtD3xp0HiAoT1uTRVnHmlvNKRjueO9c/HdZLmO5R2kMHY0
> yeFyrAGwlYwNA3d78htHRUgB9QPA8ErIxZF1/z96xI8fsIyB5fa/sz5fCWINX73F
> paF9oqle/jJ7brbO13cvmylqbHVPlHCBzHsqf5jYsCwMURfjpwjaBv5E5TYruENY
> Vk90y1EUcmyz2ZLf56nGzQ/1+3sewapzrjS/FoOvF4Jj5gUOwYm9hKMRobX5WGt6
> +netnDXy8lZZUcY/DB9iM2SYwze1cKJzDlT2mxQDmmeiUKaPvoCA6NwG+/xw/z5f
> ZQL9GJRToTLTY+Agqd7pQ7I3BE+DTM/4+EbueO9t3O9J3hLE9HYhvUvTKcs0+Z8c
> MTDF9UreQhO679Nf+FYbwJIlZ9DCI4JxLNAn5HI+re8bdX9YTXA+j+O80Cfmki2K
> 5ocO5B9kejzPXyVJM2rIoOZvlTs7jje5XdbJN68P0Pa+gWLFb1+9Own0y8tJ8Ipq
> N7k9gClYreL3Em+mYeEL7BBKT3/hiIUrVo/d25tDBJ77sEQsFQw8sH2jFOzZkdA9
> /5tnMs8Fn7rIRkc0si3zQ8xovdgJjlRuKwxMZLDg4fGZRX7ZzAt3j4GoAvhHIiD+
> H+prHme2L4ymblBy7Meyd9C4342m2sTjS0y3ZZAfZyaOuNQpOmY=
> =LM2i
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>
Re: Zipping HTTP Request
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Gary,
On 5/7/18 11:24 AM, Gary Gregory wrote:
> What could be more useful is a more general solution in a new
> module based on Apache Commons Compress that calls
> org.apache.commons.compress.compressors.CompressorStreamFactory.create
CompressorOutputStream(String,
>
>
OutputStream).
>
> That would give you support for:
>
> GZIP BZIP2 XZ PACK200 LZMA DEFLATE SNAPPY_FRAMED LZ4_BLOCK
> LZ4_FRAMED ZSTANDARD
What, no Brotli[1]? ;)
I'm almost entirely kidding, but most of the above compression methods
are not supported by web servers. RFC 7231 only defined
content-encoding tokens for gzip, deflate, and compress/x-compress
(and "identity"). Mozilla extends that list to include 'br' (for Brotli)
.
So, while using a generalized method for compression, you might want
to limit it to actually-useful compression types.
- -chris
[1] https://en.wikipedia.org/wiki/Brotli
> On Mon, May 7, 2018 at 9:15 AM, Philippe Mouawad <
> p.moua...@ubik-ingenierie.com> wrote:
>
>> Hello, There are sometimes a requirement to automatically gzip
>> the request body whenever Content-Encoding is set to gzip.
>>
>> Would you find it interesting to add this feature to HttpClient
>> ?
>>
>> The implementation would look like:
>>
>> private static final class GzipHttpRequestInterceptor implements
>> HttpRequestInterceptor { @Override public void
>> process(HttpRequest request, HttpContext context) throws
>> HttpException, IOException { if(request instanceof
>> HttpEntityEnclosingRequest) { Header header =
>> request.getFirstHeader(" Content-Encoding"); if(header != null &&
>> "gzip".equals(header.getValue())) { HttpEntityEnclosingRequest
>> enclosingRequest = (HttpEntityEnclosingRequest) request;
>> HttpEntity entity = enclosingRequest.getEntity();
>> ByteArrayOutputStream out = new ByteArrayOutputStream(); try
>> (GZIPOutputStream gzipOS = new GZIPOutputStream(out)) {
>> entity.writeTo(gzipOS); } enclosingRequest.setEntity(new
>> ByteArrayEntity(out.toByteArray())); } } } }
>>
>>
>> Use could add it to processing like this:
>>
>> - builder.addInterceptorFirst(new GzipHttpRequestInterceptor()
>> );
>>
>> Or it could be enabled by default.
>>
>>
>> Questions:
>>
>> - Would you accept it as a PR ? - Do you see immediate
>> improvements in the provided code ?
>>
>> Thanks -- Regards Philippe Mouawad. Ubik-Ingénierie
>>
>
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlrwg+wACgkQHPApP6U8
pFhYtQ//f55PE47LkRrtD3xp0HiAoT1uTRVnHmlvNKRjueO9c/HdZLmO5R2kMHY0
yeFyrAGwlYwNA3d78htHRUgB9QPA8ErIxZF1/z96xI8fsIyB5fa/sz5fCWINX73F
paF9oqle/jJ7brbO13cvmylqbHVPlHCBzHsqf5jYsCwMURfjpwjaBv5E5TYruENY
Vk90y1EUcmyz2ZLf56nGzQ/1+3sewapzrjS/FoOvF4Jj5gUOwYm9hKMRobX5WGt6
+netnDXy8lZZUcY/DB9iM2SYwze1cKJzDlT2mxQDmmeiUKaPvoCA6NwG+/xw/z5f
ZQL9GJRToTLTY+Agqd7pQ7I3BE+DTM/4+EbueO9t3O9J3hLE9HYhvUvTKcs0+Z8c
MTDF9UreQhO679Nf+FYbwJIlZ9DCI4JxLNAn5HI+re8bdX9YTXA+j+O80Cfmki2K
5ocO5B9kejzPXyVJM2rIoOZvlTs7jje5XdbJN68P0Pa+gWLFb1+9Own0y8tJ8Ipq
N7k9gClYreL3Em+mYeEL7BBKT3/hiIUrVo/d25tDBJ77sEQsFQw8sH2jFOzZkdA9
/5tnMs8Fn7rIRkc0si3zQ8xovdgJjlRuKwxMZLDg4fGZRX7ZzAt3j4GoAvhHIiD+
H+prHme2L4ymblBy7Meyd9C4342m2sTjS0y3ZZAfZyaOuNQpOmY=
=LM2i
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Zipping HTTP Request
What could be more useful is a more general solution in a new module based
on Apache Commons Compress that calls
org.apache.commons.compress.compressors.CompressorStreamFactory.createCompressorOutputStream(String,
OutputStream).
That would give you support for:
GZIP
BZIP2
XZ
PACK200
LZMA
DEFLATE
SNAPPY_FRAMED
LZ4_BLOCK
LZ4_FRAMED
ZSTANDARD
Gary
On Mon, May 7, 2018 at 9:15 AM, Philippe Mouawad <
p.moua...@ubik-ingenierie.com> wrote:
> Hello,
> There are sometimes a requirement to automatically gzip the request body
> whenever Content-Encoding is set to gzip.
>
> Would you find it interesting to add this feature to HttpClient ?
>
> The implementation would look like:
>
> private static final class GzipHttpRequestInterceptor implements
> HttpRequestInterceptor {
> @Override
> public void process(HttpRequest request, HttpContext context)
> throws HttpException, IOException {
> if(request instanceof HttpEntityEnclosingRequest) {
> Header header = request.getFirstHeader("
> Content-Encoding");
> if(header != null && "gzip".equals(header.getValue())) {
> HttpEntityEnclosingRequest enclosingRequest =
> (HttpEntityEnclosingRequest) request;
> HttpEntity entity = enclosingRequest.getEntity();
> ByteArrayOutputStream out = new
> ByteArrayOutputStream();
> try (GZIPOutputStream gzipOS = new
> GZIPOutputStream(out)) {
> entity.writeTo(gzipOS);
> }
> enclosingRequest.setEntity(new
> ByteArrayEntity(out.toByteArray()));
> }
> }
> }
> }
>
>
> Use could add it to processing like this:
>
>- builder.addInterceptorFirst(new GzipHttpRequestInterceptor() );
>
> Or it could be enabled by default.
>
>
> Questions:
>
>- Would you accept it as a PR ?
>- Do you see immediate improvements in the provided code ?
>
> Thanks
> --
> Regards
> Philippe Mouawad.
> Ubik-Ingénierie
>
