Re: Can i elastic search as my primary store?

[shikhar]

Depends on how valuable your data is.

I wouldn't say Elasticsearch is quite there yet to be considered a reliable
primary data store, even with 1.4. Lots of work to be done on hardening
replication.

On Sat, Oct 25, 2014 at 4:08 AM, Nikolas Everett  wrote:

> I'd wait for 1.4 before considering it.  There are lots of stability
> improvements there.  One thing to consider is that updates are quite costly
> compared to Mongo/MySQL whatever.
>
> Nik
>
> On Fri, Oct 24, 2014 at 6:34 PM, Zennet Wheatcroft  > wrote:
>
>> I have heard from the source, Do not use Elasticsearch as a data store.
>> But some people do and it works ok. I would recommend that you use the
>> snapshot and restore features. And back up your json file data so you can
>> re-index in case your index gets corrupted. And be careful upgrading,
>> especially between breaking versions.
>>
>>
>> On Friday, October 24, 2014 2:32:56 PM UTC-7, Akram Hussein wrote:
>>>
>>> Is it a use case today to use elastic search as a primary store?
>>> basically using it similar to mongodb? is that a use case the product is
>>> moving towards or it is mostly just for search?
>>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/a6124515-fea1-47ab-9b0a-6718e4123164%40googlegroups.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2R85TSSyQxfdpwU_ucUdCezCgP536xDb3UojBgt25PMg%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DO-y8NwQDedz-Rvu1L_F%3DwO%3D8Bvz9MMUxRFA6J%2BsU-%3DSw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: partial index replication causes data loss?

[shikhar]

Yes, this is the 2nd issue I mentioned, where ES will pick basically any
replica as primary without consideration to which one might be more
'up-to-date'

On Fri, Oct 24, 2014 at 3:57 AM, Evan Tahler  wrote:

> Interesting!
>
> However, the write *may not* be the cause of the data loss here.  Even if
> there was no write while A and B are down, would the recovery process have
> happened the same way?  In some further tests, it still looks like C would
> have overwritten all the data in A and B when they rebooted.
>
> This type of error is easily triggered by garbage collection with large
> data sets, and a server becoming unresponsive for too long. (perhaps the
> cluster kicks out the unresponsive node, or a supervisor restarts the
> application)
>
> On Thursday, October 23, 2014 12:59:00 PM UTC-7, Shikhar Bhushan wrote:
>>
>> Very interesting. The default 'write consistency level' with
>> Elasticsearch is QUORUM, i.e. verify a quorum of replicas for a shard are
>> available before processing a write for it. In this case you were just left
>> with 1 replica, C, and a write happened. So you would think that it should
>> not go through since 2 replicas would be required for quorum. However:
>> https://github.com/elasticsearch/elasticsearch/issues/6482. I think this
>> goes to show this is a real, not a hypothetical problem!
>>
>> But guess what? *Even if this were fixed, and a write to C never
>> happened: *it is still possible that once A & B were back, C could be
>> picked as primary and clobber data. See: https://github.com/
>> elasticsearch/elasticsearch/issues/7572#issuecomment-59983759
>>
>> On Thu, Oct 23, 2014 at 7:48 PM, Evan Tahler  wrote:
>>
>>> Bump?  I would love to hear some thoughts on this flow, and if there are
>>> any suggestions on how to mitigate it (other than replicating all data to
>>> all nodes).
>>>
>>> Thanks!
>>>
>>>
>>> On Tuesday, October 14, 2014 3:52:31 PM UTC-7, Evan Tahler wrote:
>>>>
>>>> Hi Mailing List!  I'm a first-time poster, and a long time reader.
>>>>
>>>> We recently had a crash in our ES (1.3.1 on Ubuntu) cluster which
>>>> caused us to loose a significant volume of data.  I have a "theory" on what
>>>> happened to cause this, and I would love to hear your opinions on this, and
>>>> if you have any suggestions to mitigate it.
>>>>
>>>> Here is a simplified play-by-play:
>>>>
>>>>
>>>>1. Cluster has 3 data nodes, A, B, and C.  The index has 10
>>>>shards.  The index has a replica count of 1, so A is the master and B 
>>>> is a
>>>>replica.  C is doing nothing.  Re-allocation of indexes/shards is 
>>>> enabled.
>>>>2. A crashes.  B takes over as master, and then starts transferring
>>>>data to C as a new replica.
>>>>3. B crashes.  C is now master with an impartial dataset.
>>>>4. There is a write to the index.
>>>>5. A and B finally reboot, and they are told that they are now
>>>>stale (as C had a write while they were away).  Both A and B delete 
>>>> their
>>>>local data.  A is chosen to be the new replica and re-sync from C.
>>>>6. ... all the data A and B had which C never got is lost forever.
>>>>
>>>>
>>>> Is the above situation scenario possible?  If it is, it seems like the
>>>> default behavior of ES might be better to not reallocate in this scenario?
>>>> This would have caused the write in step #4 to fail, but in our use case,
>>>> that is preferable to data loss.
>>>>
>>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/elasticsearch/58e98223-c036-41e2-b53c-265343fa3173%
>>> 40googlegroups.com
>>> <https://groups.google.com/d/msgid/elasticsearch/58e98223-c036-41e2-b53c-265343fa3173%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubsc

Re: partial index replication causes data loss?

[shikhar]

Very interesting. The default 'write consistency level' with Elasticsearch
is QUORUM, i.e. verify a quorum of replicas for a shard are available
before processing a write for it. In this case you were just left with 1
replica, C, and a write happened. So you would think that it should not go
through since 2 replicas would be required for quorum. However:
https://github.com/elasticsearch/elasticsearch/issues/6482. I think this
goes to show this is a real, not a hypothetical problem!

But guess what? *Even if this were fixed, and a write to C never happened: *it
is still possible that once A & B were back, C could be picked as primary
and clobber data. See:
https://github.com/elasticsearch/elasticsearch/issues/7572#issuecomment-59983759

On Thu, Oct 23, 2014 at 7:48 PM, Evan Tahler  wrote:

> Bump?  I would love to hear some thoughts on this flow, and if there are
> any suggestions on how to mitigate it (other than replicating all data to
> all nodes).
>
> Thanks!
>
>
> On Tuesday, October 14, 2014 3:52:31 PM UTC-7, Evan Tahler wrote:
>>
>> Hi Mailing List!  I'm a first-time poster, and a long time reader.
>>
>> We recently had a crash in our ES (1.3.1 on Ubuntu) cluster which caused
>> us to loose a significant volume of data.  I have a "theory" on what
>> happened to cause this, and I would love to hear your opinions on this, and
>> if you have any suggestions to mitigate it.
>>
>> Here is a simplified play-by-play:
>>
>>
>>1. Cluster has 3 data nodes, A, B, and C.  The index has 10 shards.
>>The index has a replica count of 1, so A is the master and B is a replica.
>>C is doing nothing.  Re-allocation of indexes/shards is enabled.
>>2. A crashes.  B takes over as master, and then starts transferring
>>data to C as a new replica.
>>3. B crashes.  C is now master with an impartial dataset.
>>4. There is a write to the index.
>>5. A and B finally reboot, and they are told that they are now stale
>>(as C had a write while they were away).  Both A and B delete their local
>>data.  A is chosen to be the new replica and re-sync from C.
>>6. ... all the data A and B had which C never got is lost forever.
>>
>>
>> Is the above situation scenario possible?  If it is, it seems like the
>> default behavior of ES might be better to not reallocate in this scenario?
>> This would have caused the write in step #4 to fail, but in our use case,
>> that is preferable to data loss.
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/58e98223-c036-41e2-b53c-265343fa3173%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DP848XunJ8_pQKYi36uF2Df1UghZVOwS%2BuzABaocmKKJw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: bulk indexing - optimal refresh_interval

[shikhar]

Thanks for the explanation! I'll switch over for the next time I need to
reindex.


On Tue, Jul 29, 2014 at 6:35 PM, Michael McCandless 
wrote:

> Disabling refresh (-1) is a good choice if you are fully maximizing your
> cluster's CPU/IO resources (using enough bulk client threads or async
> requests).  In that case it should give faster indexing throughput than 30s
> refresh.
>
> But if you are not saturating the cluster's resources, then a refresh
> interval of 30s may in fact get you faster indexing throughput because
> refreshes are done with a background thread in ES, so you effectively get
> one more thread working for you than if you disable refresh which causes
> the bulk indexing threads to do the flushing.
>
> Try both and see and then report back!
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Jul 29, 2014 at 3:11 AM, joergpra...@gmail.com <
> joergpra...@gmail.com> wrote:
>
>> There is no more a massive hit when opening an index for read once than
>> at every 30 seconds.
>>
>> The only explanation I can think of is that users perform searches while
>> indexing and somehow want up-to-date results while they search along.
>>
>> This is not the case when I do bulk indexing, search is disabled
>> completely. So I still recommend disabling refresh_interval in the case
>> when there is no search activity while bulk indexing. And before search is
>> opened again, the index is flushed, optimized, and extended to replica
>> levels as well.
>>
>> Jörg
>>
>>
>> On Tue, Jul 29, 2014 at 8:23 AM, Mark Walkom 
>> wrote:
>>
>>> I'd say because if you are inserting a lot of data, you will have a
>>> massive hit at the end when you need to index, as opposed to smaller ones
>>> along the way.
>>>
>>> Regards,
>>> Mark Walkom
>>>
>>> Infrastructure Engineer
>>> Campaign Monitor
>>> email: ma...@campaignmonitor.com
>>> web: www.campaignmonitor.com
>>>
>>>
>>> On 29 July 2014 16:20, shikhar  wrote:
>>>
>>>> The 1.3.0 release notes state:
>>>>
>>>>
>>>>>- Increase the refresh_interval
>>>>>
>>>>> <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html#bulk>
>>>>>  if
>>>>>you are doing heavy bulk indexing, or you are happy with your search
>>>>>results being refreshed less frequently than once every second. *It
>>>>>is better to use a refresh interval like “30s” rather than disabling it
>>>>>completely with “-1″.*
>>>>>
>>>>> I'd love to get an explanation on why 30s is better than -1, which is
>>>> the setting we are using right now when reindexing.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/CAHWG4DOaGphdmA%3DbQqV-0ic8HXxLM3ZmBzbW9YFDtZ_zWG8BHA%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/elasticsearch/CAHWG4DOaGphdmA%3DbQqV-0ic8HXxLM3ZmBzbW9YFDtZ_zWG8BHA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAEM624Y6qPWwG5rtqh7V2WaqTTqcWJoRP6%2Bct5CYhGR2xUXPgQ%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/elasticsearch/CAEM624Y6qPWwG5rtqh7V2WaqTTqcWJoRP6%2Bct5CYhGR2xUXPgQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elastic

bulk indexing - optimal refresh_interval

[shikhar]

The 1.3.0 release notes state:


>- Increase the refresh_interval
>
> 
>  if
>you are doing heavy bulk indexing, or you are happy with your search
>results being refreshed less frequently than once every second. *It is
>better to use a refresh interval like “30s” rather than disabling it
>completely with “-1″.*
>
> I'd love to get an explanation on why 30s is better than -1, which is the
setting we are using right now when reindexing.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DOaGphdmA%3DbQqV-0ic8HXxLM3ZmBzbW9YFDtZ_zWG8BHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elastic Search and consistency

[shikhar]

On Fri, Jun 13, 2014 at 12:11 PM, shikhar  wrote:

> I take this back, I understand the ES model better now. So although the
> write-consistency-level check is only applied before the write is about
> to be issued, with sync replication the client can only get an ack if it
> succeded on the primary shard as well as all replicas (as per the same
> cluster state as the check is performed on). In case it fails on some
> replica(s), the operation would be retried (together with the write-
> consistency-level check using a possibly-updated cluster state).
>

FWIW I'm really not sure anymore. TransportShardReplicationOperationAction
where this stuff is happening has a bunch of logic in performReplicas(..)
<https://github.com/elasticsearch/elasticsearch/blob/a06fd46a72193a387024b00e226241511a3851d0/src/main/java/org/elasticsearch/action/support/replication/TransportShardReplicationOperationAction.java#L557-L676>
where it decides to take into account updated cluster state, and there seem
to be exceptions for certain kinds of failures being tolerated.

Seems like this would be so much more straightforward if a write were to be
fanned-out and then block uptil max of timeout for checking that the
requried number of replicas succeeded (with success on primary being
required).

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DMM_-0ySs3iCFkVKxL3DrNW2vT9daUAT9eWfXHpUrN2wQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elastic Search and consistency

[shikhar]

On Thu, Jun 12, 2014 at 8:52 PM, shikhar  wrote:

> ES currently does not seem to provide any guarantee that an acknowledged
> write (from the caller's perspective) succeeded on a quorum of replicas.


I take this back, I understand the ES model better now. So although the
write-consistency-level check is only applied before the write is about to
be issued, with sync replication the client can only get an ack if it
succeded on the primary shard as well as all replicas (as per the same
cluster state as the check is performed on). In case it fails on some
replica(s), the operation would be retried (together with the
write-consistency-level check using a possibly-updated cluster state).


> This makes it unsuitable for a primary data store, given you can see data
> loss despite having replicas!


If using ES as a primary store, you should really be running it with
*  index.gateway.local.sync: 0*
to make sure the translog fsync's on every write operation



A follow-up question: what if there is a failure on one of the replicas
that prevents writes (e.g. disk full) but this is not preventing the node
from dropping out of discovery state due to being healthy otherwise? Does
it not make that node a SPOF? This is something we have run into with
SolrCloud <https://issues.apache.org/jira/browse/SOLR-5805>.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DOi0sTu5_mwv%3DNJL5SH7%3D1Z5CG2iULSbF0_P7ZDULY-qw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elastic Search and consistency

[shikhar]

On Thu, Jun 12, 2014 at 8:42 PM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> 1.1 Not sure what you mean "after the quorum check". Write consistency is
> a model where ES makes sure there are enough recipients (nodes) before
> writes are executed. consistency=quorum fails if you have too few nodes to
> match the actual replica count of the index.
>

This is the part that I am also after. ES currently does not seem to
provide any guarantee that an acknowledged write (from the caller's
perspective) succeeded on a quorum of replicas. This makes it unsuitable
for a primary data store, given you can see data loss despite having
replicas!

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DPDXoRzF1wXOK44MWKihR2R0A_JVZnZ1L1T6dxq7JVySg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elastic Search and consistency

[shikhar]

Hi Anand,

I would love to know as well, and asked in this thread today: "quorum write
+ sync replication: guarantees".

It is pretty important information given ElasticSearch is being touted as a
reliable data store... which the presence of a "write consistency level"
setting would tempt one to believe, but there doesn't seem to be evidence
to support the guarantees that implies.

Cheers,

Shikhar

On Sat, Apr 5, 2014 at 12:06 AM, Anand Somani  wrote:

> Hi,
>
> We are trying to use ES as a system of record, so I trying to understand
> its consistency and durability related guarantees and knobs.
>
> For this scenario assume I will have a 2 copies ( 1 replica). From what I
> have read and gleaned (
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-consistency)
> =>
>
>- write.consistency is a check to see if the operation can be
>executed. It offers no guarantees in terms of data durability or operation
>guarantee
> - replication type => Is this for durability guarantees?
>   - sync all copies of the data are saved/indexed on nodes before it
>   returns.
>   - async primary node will be synchronous the others will be async.
>
> Failure scenario question
>
> Scenario is
>
> => PUT with write.consistency=quorum, but different replications =
> sync/async.
> => What happens if there is a network glitch after the quorum check
> succeeds, such that one of the non-primary shard machines is not reachable
> or down?
>
>
>1. Put with write.consistency=quorum and replication=sync =>
>   1. Would the operation fail from the callers perspective? Seems
>   like it would after some retries?
>   2. I have read that if the primary succeeds, and if it has problems
>   with replica then it will try to create a new shard, is that correct
>   understanding? I assume that should be preventable to handle network
>   failures/partitions?
>2. Put with write.consistency=quorum and replication=async =>
>   1. The operation succeeds, since primary shard is up.
>   2. Now in sometime the unreachable node is back, how does it get
>   that copy of data that it did not get, is there some steps/process that
>   will do that automatically?
>
>
> If these have been asked/answered or documents, please point me to a
> source and my apologies.
>
>
> Thanks
> Anand
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CANN9sNpFY39T2t2uy51Zh0m1p%3DBpqtqb7f9ho1iWTV%3DeUab23Q%40mail.gmail.com
> <https://groups.google.com/d/msgid/elasticsearch/CANN9sNpFY39T2t2uy51Zh0m1p%3DBpqtqb7f9ho1iWTV%3DeUab23Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DPhhY20_PPDOC%2BDTCsDv%2BddhkNDd-Q5pejeAS-SK3qREg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: quorum write + sync replication: guarantees

[shikhar]

On Thu, Jun 12, 2014 at 6:09 PM, shikhar  wrote:

> It seems like the only time we look at how many replicas the write
> succeeded on is prior to issuing the write
> <https://github.com/elasticsearch/elasticsearch/blob/20278f491ac3e62be30544a862344a9b8b1f647c/src/main/java/org/elasticsearch/action/support/replication/TransportShardReplicationOperationAction.java#L398-L418>
>  [1].
>

I meant "how many replicas the write could succeed on"... of course how
many it actually succeded on can only happen after the fact :)

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DMkhJ59ALK4ZRS%2BQkDYpwhNPZ%3D_AXTNgf6prGnyHLGwDQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

quorum write + sync replication: guarantees

[shikhar]

I have been digging into the consistency guarantees provided by
ElasticSearch and I am a bit confused by looking at the code. It seems like
the only time we look at how many replicas the write succeeded on is prior
to issuing the write
<https://github.com/elasticsearch/elasticsearch/blob/20278f491ac3e62be30544a862344a9b8b1f647c/src/main/java/org/elasticsearch/action/support/replication/TransportShardReplicationOperationAction.java#L398-L418>
 [1].

So I want to ask a question which should hopefully have a clear answer.

* write consistency level is quorum
* replication type is sync
* a write is issued. it succeeds on primary but not on a quorum of all the
shards (primary shard & replicas). will this result in a failure response?

If yes, I would love to see some corroboration!

Cheers,

Shikhar

[1] related issue: definition of quorum in ES code is off. github issue
<https://github.com/elasticsearch/elasticsearch/issues/6482>.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DP-mMc-UoQ-RYMB1iA9770v-3QE3FxzTdEUoFxKz_N-sQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: new elasticsearch discovery plugin - eskka

[shikhar]

Thanks for the feedback Otis!

On Fri, May 9, 2014 at 12:33 AM, Otis Gospodnetic <
otis.gospodne...@gmail.com> wrote:

> At Sematext we use both ES and Akka (in SPM <http://sematext.com/spm/>),
> so this is interesting for me to see...  Would it make sense to add a bit
> more to the README. things like:
> * why? is something wrong with Zen?
> * pros and cons of this vs. Zen vs. ZK
>

Makes sense to add this context, I'll add to the docs soon.

The major con for eskka is of course that it is new and unproven.

*vs zookeeper*

 You don't need to setup or admin ZooKeeper.

In Shay's resiliency blog post <http://resiliency and elasticsearch> he
writes "By having the discovery module in Elasticsearch using its own
infrastructure, specifically the communication layer, it means that we can
use the “liveness” of the cluster to assess its health." This is an
interesting point which partly holds for eskka as well, since it also lives
in the same JVM. However it doesn't use the same communication layer (the
ES internal transport), the akka cluster runs on a different port.

*vs zen*

Essentially: eskka is built to be split-brain resistant. But then so is Zen
in theory ;-) So I'd like to maybe do some testing with Jepsen before
making that tall claim.

Both Zen and Akka Cluster are implementing a gossip protocol for discovery
& fault-detection, however to me the latter feels better specified and
tested. eskka itself does not have any tests at the application-level which
I intend to fix.

I'd like to think the use of actors -- the main ones being
Master<https://github.com/shikhar/eskka/blob/master/src/main/scala/eskka/Master.scala>
, 
Follower<https://github.com/shikhar/eskka/blob/master/src/main/scala/eskka/Follower.scala>
 and 
QuorumBasedPartitionMonitor<https://github.com/shikhar/eskka/blob/master/src/main/scala/eskka/QuorumBasedPartitionMonitor.scala>
--
makes it easier to reason about state in a distributed system and maintain
invariants.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DMMPZMhmi7w74vsUqJJria%2BtVj5p7sn-v%3DVeXNUqaXziw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: new elasticsearch discovery plugin - eskka

[shikhar]

On Thu, May 8, 2014 at 1:02 PM, shikhar  wrote:

> Worth noting that besides being initial contact points for when the
> cluster is starting up, with eskka they are also used for resolving
> partitions. Given this requirement, you would ideally have 3 or more
> specified. It is perfectly ok to have all of your nodes listed, if you know
> their addresses before startup.
>

Another idea for what nodes to use as seed: if you are using master-only
nodes, make them seed nodes.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DPFje-U8LriQgDMefhUzxOr8CuH1jqKghqMDO3bLYgpvg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: new elasticsearch discovery plugin - eskka

[shikhar]

All Elasticsearch nodes will end up being part of the Akka cluster :) I
think you're really asking how many seed nodes you should specify. The seed
node list is probably going to be similar to what you might use for
zen.unicast.hosts.

Worth noting that besides being initial contact points for when the cluster
is starting up, with eskka they are also used for resolving partitions.
Given this requirement, you would ideally have 3 or more specified. It is
perfectly ok to have all of your nodes listed, if you know their addresses
before startup.

https://github.com/shikhar/eskka#configuration


On Wed, May 7, 2014 at 9:31 PM, Ivan Brusic  wrote:

> Extremely interesting! What is the recommended size of the Akka cluster
> compared to the Elasticsearch cluster?
>
> --
> Ivan
>
>
> On Tue, May 6, 2014 at 8:42 PM, shikhar  wrote:
>
>>  Just released 0.1.1<https://github.com/shikhar/eskka/releases/tag/v0.1.1>
>>
>> This version is working well in my manual testing. Automated testing is
>> on the roadmap...
>>
>>
>> On Mon, May 5, 2014 at 10:49 AM, shikhar  wrote:
>>
>>> See README <https://github.com/shikhar/eskka/blob/release-0.1/README.md>
>>>
>>> I'd love to have feedback on this first release!
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>>  To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAHWG4DNiwCxbZakzogFfqFxYxabcQ_ysG2_OMd06%3D%2BfDqFEQdA%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAHWG4DNiwCxbZakzogFfqFxYxabcQ_ysG2_OMd06%3D%2BfDqFEQdA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDjg7Uaw4tTVr2-8cGM7w2sutH6F2XzCVj3JjKK-RCt5g%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDjg7Uaw4tTVr2-8cGM7w2sutH6F2XzCVj3JjKK-RCt5g%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DO-nGqKuv9i_wFpR0nomTY4mk%3DmkVxvgai3bjO8sSyRxw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: new elasticsearch discovery plugin - eskka

[shikhar]

Just released 0.1.1 <https://github.com/shikhar/eskka/releases/tag/v0.1.1>

This version is working well in my manual testing. Automated testing is on
the roadmap...


On Mon, May 5, 2014 at 10:49 AM, shikhar  wrote:

> See README <https://github.com/shikhar/eskka/blob/release-0.1/README.md>
>
> I'd love to have feedback on this first release!
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DNiwCxbZakzogFfqFxYxabcQ_ysG2_OMd06%3D%2BfDqFEQdA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

ANN: new elasticsearch discovery plugin - eskka

[shikhar]

See README <https://github.com/shikhar/eskka/blob/release-0.1/README.md>

I'd love to have feedback on this first release!

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHWG4DMdPDCSpm0QJOznZuMDgi%2BTdFGx8dBTXvsACLuTvFmgxg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.