Oh sorry, I forgot the curl parameter --compressed Now added to the gist.
Jörg On Wed, Mar 25, 2015 at 9:02 PM, Marcel Matus <matusmar...@gmail.com> wrote: > Hi Jorg, > thank you for your help - mainly your practical example is great. The > header Accept-Encoding isn't really necessary. I was almost there, but was > missing either the id of document or <at> before the binary file. > > Thanks. > Marcel > > > On Wed, Mar 25, 2015 at 2:15 PM, joergpra...@gmail.com < > joergpra...@gmail.com> wrote: > >> Yes, I mean pushing compressed data. >> >> You have several wrong assumptions: use PUT instead of POST, mark file >> with a preceding '@', set 'Accept-Encoding' header to 'gzip,deflate', and >> use URL for a document creation 'localhost:9200/test/docs/1' for example. >> >> Here is my full demo of a correct curl command >> >> https://gist.github.com/jprante/15f570be70504285a678 >> >> Jörg >> >> >> On Wed, Mar 25, 2015 at 8:03 AM, Marcel Matus <matusmar...@gmail.com> >> wrote: >> >>> Actually, I am more interested about pushing compressed data into >>> elasticsearch than asking for compressed data. >>> So far, I tried it manually, using Google chrome plugin - postman, or >>> curl, without success: >>> >>> curl -sS -X POST --data-binary txn001.json.gz >>> http://localhost:9200/gzip/new -H "Content-Type: application/json" -H >>> "Content-Encoding: gzip" >>> Response: curl: (52) Empty reply from server >>> >>> Marcel >>> >>> >>> >>> On Tue, Mar 24, 2015 at 10:18 PM, joergpra...@gmail.com < >>> joergpra...@gmail.com> wrote: >>> >>>> Logstash has both Java and HTTP output, but I assume you want to use >>>> HTTP. >>>> >>>> Set http.compression parameter to true in ES configuration, then you >>>> can use gzip-compressed HTTP traffic using "Accept-Encoding" header. >>>> >>>> >>>> http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html >>>> >>>> Jörg >>>> >>>> On Tue, Mar 24, 2015 at 2:30 PM, Marcel Matus <matusmar...@gmail.com> >>>> wrote: >>>> >>>>> Hi Joe, >>>>> no, we post data into elasticsearch using Logstash. Actually, the >>>>> complete way is: APP - Logstash - RMQ - Logstash - ES. >>>>> And when I include F5 VIP's, there are 6 services on the way to >>>>> process these data... >>>>> >>>>> Marcel >>>>> >>>>> >>>>> On Tue, Mar 24, 2015 at 1:50 PM, joergpra...@gmail.com < >>>>> joergpra...@gmail.com> wrote: >>>>> >>>>>> Are you using Java API? >>>>>> >>>>>> Jörg >>>>>> >>>>>> On Tue, Mar 24, 2015 at 11:59 AM, Marcel Matus <matusmar...@gmail.com >>>>>> > wrote: >>>>>> >>>>>>> Hi, >>>>>>> some of our data are big ones (1 - 10 MB), and if there are milions >>>>>>> of those, it causes us trouble in our internal network. >>>>>>> We would like to compress these data in generation time, and send >>>>>>> over the network to elasticsearch in compressed state. I tried to find >>>>>>> some >>>>>>> solution, but usually I found only retrieving gzipped data. Is there a >>>>>>> way, >>>>>>> how to put compressed data into elasticsearch? >>>>>>> >>>>>>> Thanks, >>>>>>> Marcel >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "elasticsearch" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to elasticsearch+unsubscr...@googlegroups.com. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/elasticsearch/bce57798-0b51-4c9e-a0ca-06e41b2ca160%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/elasticsearch/bce57798-0b51-4c9e-a0ca-06e41b2ca160%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "elasticsearch" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/elasticsearch/iWT19g3_Viw/unsubscribe >>>>>> . >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> elasticsearch+unsubscr...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFagGZYQ1wCZe%2BCMrUSUJ9gE7erjA7fk1aSOLfsdZkXjg%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFagGZYQ1wCZe%2BCMrUSUJ9gE7erjA7fk1aSOLfsdZkXjg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Marcel Matus >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to elasticsearch+unsubscr...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elasticsearch/CAHUcNyj4cAi0oDGqgLmeSxLwVwtCssF72pyhufKV8ML911auHA%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/elasticsearch/CAHUcNyj4cAi0oDGqgLmeSxLwVwtCssF72pyhufKV8ML911auHA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "elasticsearch" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/elasticsearch/iWT19g3_Viw/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> elasticsearch+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG%3DzuBnV4LWr7ySj6arirFzjxgCARzq%2Bvh05mFqyJ7%2Bpg%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG%3DzuBnV4LWr7ySj6arirFzjxgCARzq%2Bvh05mFqyJ7%2Bpg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Marcel Matus >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elasticsearch+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/CAHUcNyhFUHHgwQwubaqPed0O6oWrHJaeD_nousLF%3DJW69p_1zg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/elasticsearch/CAHUcNyhFUHHgwQwubaqPed0O6oWrHJaeD_nousLF%3DJW69p_1zg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/iWT19g3_Viw/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> elasticsearch+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEiKwvCWtRrLZ1OrorZp-oJPpQvqbBNAmOJPA-%3D%2Bba5zg%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEiKwvCWtRrLZ1OrorZp-oJPpQvqbBNAmOJPA-%3D%2Bba5zg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Marcel Matus > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAHUcNyg6iuaNa88D-koRb9j37nYvKh_8j29cSUt_%2ByMViz%2Brbg%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CAHUcNyg6iuaNa88D-koRb9j37nYvKh_8j29cSUt_%2ByMViz%2Brbg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFUEkTprkFb0ieNE0Q807rnHwXU7ukroGwW_9gr0BuE5Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
