RE: best practice wanted for huge number of index time serial data

2014-07-16 Thread Wang Yong
Thank you Mark, if I use daily index, I have to specify multiple indexes based 
on the time range. That will make my service a little more complicate. 

So I am wondering, even if I put all data in one huge index, as long as I limit 
the time range in my query, it looks like es will locate the data as quickly as 
I do it in a much smaller daily index, cause es will not need to search through 
the whole index, just need to locate the data first by the time range specified 
in the query. Is that true?

 

Alan

 

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On 
Behalf Of Mark Walkom
Sent: 2014年7月14日 10:03
To: elasticsearch@googlegroups.com
Subject: Re: best practice wanted for huge number of index time serial data

 

This is pretty standard for logstash type data.

 

Use daily indexes, don't use TTL.




Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com mailto:ma...@campaignmonitor.com 
web: www.campaignmonitor.com http://www.campaignmonitor.com 

 

On 14 July 2014 11:40, LiMac cnwangy...@gmail.com 
mailto:cnwangy...@gmail.com  wrote:

Hi folks,

 

I am trying to index a huge number of time serial data. The total number will 
be 5k docs for one second which will continue for several months. I also need 
to search these data, but only inside a very small time rage, maybe one hour. 
Is there any best practice for this kind of use case?

 

Thanks!

 

Alan

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com 
mailto:elasticsearch+unsubscr...@googlegroups.com .
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com
 
https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com?utm_medium=emailutm_source=footer
 .
For more options, visit https://groups.google.com/d/optout.

 

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com 
mailto:elasticsearch+unsubscr...@googlegroups.com .
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt_oY3ZSyqNZoBpzw%40mail.gmail.com
 
https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt_oY3ZSyqNZoBpzw%40mail.gmail.com?utm_medium=emailutm_source=footer
 .
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/010601cfa155%24e55579b0%24b0006d10%24%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: best practice wanted for huge number of index time serial data

2014-07-13 Thread Mark Walkom
This is pretty standard for logstash type data.

Use daily indexes, don't use TTL.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 14 July 2014 11:40, LiMac cnwangy...@gmail.com wrote:

 Hi folks,

 I am trying to index a huge number of time serial data. The total number
 will be 5k docs for one second which will continue for several months. I
 also need to search these data, but only inside a very small time rage,
 maybe one hour. Is there any best practice for this kind of use case?

 Thanks!

 Alan

 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com
 https://groups.google.com/d/msgid/elasticsearch/7b659b5f-7f50-483d-a2d5-de9c2e4b650c%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEM624Y4GSxMc39SJf%3DCk5MwANMt7PRpnXt_oY3ZSyqNZoBpzw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.