The Elasticsearch file size does not only contain compressed fields, but much more. For example, term vectors, norms, etc. You would have to disable field attributes you do not want. Also note, Elasticsearch has replica enabled by default, and segment count is not optimized automatically.
Jörg On Sun, Jun 8, 2014 at 7:09 PM, sri <1.fr....@gmail.com> wrote: > Okay i will make the changes and upload the new stats. > > I am just curious, could you explain how the results were making sense, i > just want to get a proper idea of what ES is actually doing to the data. > > Thanks and Regards > Sri > > On Sunday, June 8, 2014 12:56:55 PM UTC-4, David Pilato wrote: > >> Well. Think that you index all field individualy, that you are storing >> source (compressed) and that you are indexing _all field as well. >> >> So with defaults, this results make sense to me. >> >> Try disable _all field and see what gain you can get. >> >> -- >> David ;-) >> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs >> >> >> Le 8 juin 2014 à 18:50, sri <1.fr...@gmail.com> a écrit : >> >> Hi David, >> >> Thank you very much for the prompt reply. >> >> Below are the stats that i got when i was testing the ES cluster: >> >> Number of Nodes :2 >> Input format : rsyslog >> >> input file size(Mb) ES file size per node(Mb) 1 1.8 2 3.6 3 5.3 4 >> 6.8 5 8.5 6 10.1 7 11.7 8 13 9 14.1 10 16 >> I am sorry to ask like this, but i wasn't understanding how the >> compression was taking place. >> >> Thanks and Regards >> Sri >> >> On Sunday, June 8, 2014 12:41:35 PM UTC-4, David Pilato wrote: >>> >>> It's compressed by default now. >>> >>> -- >>> David ;-) >>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs >>> >>> >>> Le 8 juin 2014 à 18:01, sri <1.fr...@gmail.com> a écrit : >>> >>> Hello everyone, >>> >>> I have read posts and blogs on how elasticsearch compression can be >>> enabled in the previous versions(0.17 - 0.19). >>> >>> I am currently using ES 1.2.1, i wasn't able to find out how to enable >>> compression in this version or if at all there is any such option for it. >>> >>> I know that i can reduce the storage amount by disabling the source >>> using the mapping api >>> <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-source-field.html>, >>> but what i was interested is the compression of data storage. >>> >>> Thanks and Regards >>> Sri >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elasticsearc...@googlegroups.com. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/209b1832-6924-4794-833e-489917962211% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/209b1832-6924-4794-833e-489917962211%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elasticsearc...@googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/elasticsearch/2e95acf2-1658-40ff-adfe-2be2e2031add% >> 40googlegroups.com >> <https://groups.google.com/d/msgid/elasticsearch/2e95acf2-1658-40ff-adfe-2be2e2031add%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/b2cac83a-777a-4876-bf07-5cf093a92c1c%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/b2cac83a-777a-4876-bf07-5cf093a92c1c%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG5TAS08k2Wtqe647reMKHUkNkvyepfnp7Sz7u9YqyDag%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
