Re: Some interesting storage numbers for people interested.

2015-02-23 Thread Mark Walkom
Thanks John, this is a really interesting test. If you have no _source you cannot reindex or view the actual raw content that was sent to ES, only the analysed portions you keep. No _all means you have to know the exact field you want to search on or else you may get no results, as ES will

Re: Some interesting storage numbers for people interested.

2015-02-23 Thread Jack Park
What is lost (the tradeoff) when _source is disabled? What is lost when _all is disabled? This is interesting! Thanks Jack On Mon, Feb 23, 2015 at 12:10 PM, John Smith java.dev@gmail.com wrote: I don't run a blog but I thought I would share some results with the community. Using

Some interesting storage numbers for people interested.

2015-02-23 Thread John Smith
I don't run a blog but I thought I would share some results with the community. Using Elasticsearch 1.4.3 I wanted to test the various ways we could save some storage on our ES index and here are some numbers Created 6 different indexes with the various mapping settings. Each index containing

Re: Some interesting storage numbers for people interested.

2015-02-23 Thread Jack Park
Thank you very much, Mark. On Mon, Feb 23, 2015 at 12:54 PM, Mark Walkom markwal...@gmail.com wrote: Thanks John, this is a really interesting test. If you have no _source you cannot reindex or view the actual raw content that was sent to ES, only the analysed portions you keep. No _all

Re: Some interesting storage numbers for people interested.

2015-02-23 Thread John Smith
Mark when you say you cannot re-index the document you mean re-index within the cluster? But if we resubmit the document using the index API it will get re-indexed and updated version 2 right? So Elastic search will mark the document to be deleted from the segment and eventually merge the

Re: Some interesting storage numbers for people interested.

2015-02-23 Thread John Smith
Yeah sorry should have mentioned the tradeoffs. I think there some interesting use-cases here for instance if you are building a pure analytics dashboard where it's 100% aggregations then you can save allot of space with _source: false, _all: false In my case I'm opting for _source: true,

Re: Some interesting storage numbers for people interested.

2015-02-23 Thread Mark Walkom
Yes, sorry. You can definitely reindex from an external source :) On 24 February 2015 at 09:33, John Smith java.dev@gmail.com wrote: Mark when you say you cannot re-index the document you mean re-index within the cluster? But if we resubmit the document using the index API it will get