Elastic Search and consistency

[Anand Somani]

Hi,

We are trying to use ES as a system of record, so I trying to understand
its consistency and durability related guarantees and knobs.

For this scenario assume I will have a 2 copies ( 1 replica). From what I
have read and gleaned (
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-consistency)
=

   - write.consistency is a check to see if the operation can be executed.
   It offers no guarantees in terms of data durability or operation guarantee
- replication type = Is this for durability guarantees?
  - sync all copies of the data are saved/indexed on nodes before it
  returns.
  - async primary node will be synchronous the others will be async.

Failure scenario question

Scenario is

= PUT with write.consistency=quorum, but different replications =
sync/async.
= What happens if there is a network glitch after the quorum check
succeeds, such that one of the non-primary shard machines is not reachable
or down?


   1. Put with write.consistency=quorum and replication=sync =
  1. Would the operation fail from the callers perspective? Seems like
  it would after some retries?
  2. I have read that if the primary succeeds, and if it has problems
  with replica then it will try to create a new shard, is that correct
  understanding? I assume that should be preventable to handle network
  failures/partitions?
   2. Put with write.consistency=quorum and replication=async =
  1. The operation succeeds, since primary shard is up.
  2. Now in sometime the unreachable node is back, how does it get that
  copy of data that it did not get, is there some steps/process
that will do
  that automatically?


If these have been asked/answered or documents, please point me to a source
and my apologies.


Thanks
Anand

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CANN9sNpFY39T2t2uy51Zh0m1p%3DBpqtqb7f9ho1iWTV%3DeUab23Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Elastic search for durable data

[Anand Somani]

Hi

If I have data (that needs indexing) that I cannot lose, would it advisable
to store it only in Elastic Search as the source of truth with 3 copies (2
replicas) ? Reading a little about index corruption and such, my gut
feeling is no, but I was wondering if people had any experience with it.

Thanks
Anand

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CANN9sNrVjG70inBwQZrq%2BXQOMivERE8eZekOD-6Co3RJDE8v5A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.