Hi,
How many indices you create is really up to you. Typically you would make a
snapshot of an index once and then schedule incremental snapshots at regular
intervals.
Andrew
On Apr 21, 2014, at 2:24 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
> Thanks for the information. To keep the indexes in near real time sync I'll
> end up creating thousands of indexes every day alone with this method,
> correct?
>
> On Mon, Apr 21, 2014 at 1:37 PM, Andrew Selden
> <andrew.sel...@elasticsearch.com> wrote:
> Hi,
>
> I can see how the documentation may not be as clear as it should be on this
> point. Let me try to explain. If you refer back to [1], you will see an
> example restore command:
> $ curl -XPOST "localhost:9200/_snapshot/my_backup/snapshot_1/_restore" -d '{
>
> "indices": "index_1,index_2",
>
> "ignore_unavailable": "true",
>
> "include_global_state": false,
>
> "rename_pattern": "index_(.+)",
>
> "rename_replacement": "restored_index_$1"
>
> }'
> The important parts to notice are: rename_pattern and rename_replacement.
> What this means is that, for any index in the repository named with the
> prefix 'index_', it will be restored to the cluster under the new name
> 'restored_index_$1', where $1 is the name of the original index. So "index_1"
> gets restored to "restored_index_1", meanwhile you can keep index_1 open and
> servicing requests.
>
> The basic operational workflow is that you would perform the restore
> operation with a rename. Once the restore is complete, do some sanity
> checking to confirm that you have in fact restored what you intended to.
> Finally, to make "restored_index_1" live and take "index_1" offline without
> impacting production service, you would use an atomic rename alias command
> [2].
>
> This assumes you have been using aliases to manage your indices. For example,
> typically you would set up an alias to point to a 'logical' index to mask one
> or more 'physical' indices. This is highly recommended as it gives you the
> flexibility to do things like hot-swap restored/live indices as in the above
> example.
>
> Hope that helps.
>
> Andrew
>
> 1.
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html
> 2.
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-aliases.html
>
> On Apr 21, 2014, at 12:36 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
>
>> This document seem to suggest that to do the restore elasticsearch need to
>> close the index, which means index is not available during that time while
>> restore is occurring?
>>
>> On Mon, Apr 21, 2014 at 12:31 PM, Andrew Selden
>> <andrew.sel...@elasticsearch.com> wrote:
>> Hi,
>>
>> Snapshot/restore is fine to run on an active, production system. The indices
>> are still available for normal indexing and searching operations. You can
>> throttle the snapshot operation if you are concerned about impacting
>> production quality of service [1].
>>
>> I would suggest avoiding using scroll as a backup strategy. Keeping a scroll
>> open for a long time can be taxing on an actively used production cluster.
>> You are much better off using the snapshot/restore functionality.
>>
>> Andrew
>>
>> 1.
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html
>>
>>
>> On Apr 21, 2014, at 9:23 AM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
>>
>>> Our data centers are a actively used and need to be highly available. Does
>>> snapshot and restore happen in the background and indexes are still
>>> available to the clients for read/write?
>>>
>>> Can Scroll be used to read indexes continuously and insert/update them?
>>>
>>> On Mon, Apr 21, 2014 at 5:57 AM, Alexander Reelsen <a...@spinscale.de>
>>> wrote:
>>> hey,
>>>
>>> snapshot and restore might be an option, if you are willing to accept a lag
>>> a few minutes behind. another option might be to index/update/delete all
>>> your data into two different clusters, so you dont need replication at all?
>>>
>>>
>>> --Alex
>>>
>>>
>>> On Fri, Apr 18, 2014 at 10:19 PM, Mohit Anchlia <mohitanch...@gmail.com>
>>> wrote:
>>> As I understand there is currently no feature that does async replication
>>> between 2 clusters or even within the same cluster, but we have a need to
>>> write one. What would be the best way to do it in elasticsearch? I was
>>> thinking of leveraging Scroll for this.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrUdDdiqy62yHaaS6bJJ08_txDCNNXR8rGr%3DRGY8gAv-Q%40mail.gmail.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAGCwEM9G8539fUmrzLWgkXoSYAEyUYkjUbyjeBiNVhBdarWTFA%40mail.gmail.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpqVdoSWv552vh3ZOL7siBiMUCGRUV5NcBwAFoWPisVLQ%40mail.gmail.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/B852C0E8-A2BE-4C23-B14A-EE2593A4D7ED%40elasticsearch.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWr14cNv05GAjoxtXVtUY4-uPzaxt200tp_spDNXvO_tyg%40mail.gmail.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/C9E4589A-FBCB-4FC3-9492-35B8A722EB20%40elasticsearch.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9AC0A79F-B695-4F7D-80FC-D39C9FD2F15E%40elasticsearch.com.
For more options, visit https://groups.google.com/d/optout.
