Re: Can i elastic search as my primary store?

2014-10-25 Thread shikhar
Depends on how valuable your data is. I wouldn't say Elasticsearch is quite there yet to be considered a reliable primary data store, even with 1.4. Lots of work to be done on hardening replication. On Sat, Oct 25, 2014 at 4:08 AM, Nikolas Everett nik9...@gmail.com wrote: I'd wait for 1.4

Re: partial index replication causes data loss?

2014-10-24 Thread shikhar
with large data sets, and a server becoming unresponsive for too long. (perhaps the cluster kicks out the unresponsive node, or a supervisor restarts the application) On Thursday, October 23, 2014 12:59:00 PM UTC-7, Shikhar Bhushan wrote: Very interesting. The default 'write consistency level

Re: partial index replication causes data loss?

2014-10-23 Thread shikhar
Very interesting. The default 'write consistency level' with Elasticsearch is QUORUM, i.e. verify a quorum of replicas for a shard are available before processing a write for it. In this case you were just left with 1 replica, C, and a write happened. So you would think that it should not go

Re: bulk indexing - optimal refresh_interval

2014-07-30 Thread shikhar
...@campaignmonitor.com web: www.campaignmonitor.com On 29 July 2014 16:20, shikhar shik...@schmizz.net wrote: The 1.3.0 release notes state: - Increase the refresh_interval http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html#bulk

bulk indexing - optimal refresh_interval

2014-07-29 Thread shikhar
The 1.3.0 release notes state: - Increase the refresh_interval http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html#bulk if you are doing heavy bulk indexing, or you are happy with your search results being refreshed less

Re: Elastic Search and consistency

2014-06-17 Thread shikhar
On Fri, Jun 13, 2014 at 12:11 PM, shikhar shik...@schmizz.net wrote: I take this back, I understand the ES model better now. So although the write-consistency-level check is only applied before the write is about to be issued, with sync replication the client can only get an ack

Re: Elastic Search and consistency

2014-06-13 Thread shikhar
On Thu, Jun 12, 2014 at 8:52 PM, shikhar shik...@schmizz.net wrote: ES currently does not seem to provide any guarantee that an acknowledged write (from the caller's perspective) succeeded on a quorum of replicas. I take this back, I understand the ES model better now. So although the write

quorum write + sync replication: guarantees

2014-06-12 Thread shikhar
is issued. it succeeds on primary but not on a quorum of all the shards (primary shard replicas). will this result in a failure response? If yes, I would love to see some corroboration! Cheers, Shikhar [1] related issue: definition of quorum in ES code is off. github issue https://github.com

Re: quorum write + sync replication: guarantees

2014-06-12 Thread shikhar
On Thu, Jun 12, 2014 at 6:09 PM, shikhar shik...@schmizz.net wrote: It seems like the only time we look at how many replicas the write succeeded on is prior to issuing the write https://github.com/elasticsearch/elasticsearch/blob/20278f491ac3e62be30544a862344a9b8b1f647c/src/main/java/org

Re: Elastic Search and consistency

2014-06-12 Thread shikhar
to believe, but there doesn't seem to be evidence to support the guarantees that implies. Cheers, Shikhar On Sat, Apr 5, 2014 at 12:06 AM, Anand Somani meatfor...@gmail.com wrote: Hi, We are trying to use ES as a system of record, so I trying to understand its consistency and durability related

Re: Elastic Search and consistency

2014-06-12 Thread shikhar
On Thu, Jun 12, 2014 at 8:42 PM, joergpra...@gmail.com joergpra...@gmail.com wrote: 1.1 Not sure what you mean after the quorum check. Write consistency is a model where ES makes sure there are enough recipients (nodes) before writes are executed. consistency=quorum fails if you have too few

Re: ANN: new elasticsearch discovery plugin - eskka

2014-05-09 Thread shikhar
to think the use of actors -- the main ones being Masterhttps://github.com/shikhar/eskka/blob/master/src/main/scala/eskka/Master.scala , Followerhttps://github.com/shikhar/eskka/blob/master/src/main/scala/eskka/Follower.scala and QuorumBasedPartitionMonitorhttps://github.com/shikhar/eskka/blob

Re: ANN: new elasticsearch discovery plugin - eskka

2014-05-08 Thread shikhar
for when the cluster is starting up, with eskka they are also used for resolving partitions. Given this requirement, you would ideally have 3 or more specified. It is perfectly ok to have all of your nodes listed, if you know their addresses before startup. https://github.com/shikhar/eskka#configuration

Re: ANN: new elasticsearch discovery plugin - eskka

2014-05-08 Thread shikhar
On Thu, May 8, 2014 at 1:02 PM, shikhar shik...@schmizz.net wrote: Worth noting that besides being initial contact points for when the cluster is starting up, with eskka they are also used for resolving partitions. Given this requirement, you would ideally have 3 or more specified

ANN: new elasticsearch discovery plugin - eskka

2014-05-04 Thread shikhar
See README https://github.com/shikhar/eskka/blob/release-0.1/README.md I'd love to have feedback on this first release! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send