RE: N = 3 and RW = 2 not finding some keys

2018-05-18 Thread Nicholas Adams
Sorry for being late to the party but have you tried repairing all partitions? https://docs.basho.com/riak/kv/2.1.4/using/repair-recovery/repairs/#repairing-all-partitions-on-a-node This forces every single piece of information to be read and thus causes read repair to kick in for any partitions

Re: N = 3 and RW = 2 not finding some keys

2018-05-18 Thread Guido Medina
Yes, but we haven't set back R = 2, we are currently with R = 3 until we resolve the issue. I'm in the middle of writing something that will read every single key for the cluster using our relation DB IDs (each key has a counterpart on DB) I'm guessing that 2i streaming won't help? we need to m

Re: N = 3 and RW = 2 not finding some keys

2018-05-18 Thread Bryan Hunt
Russell, Good question. I’m guessing they are iterating, and requesting a different object for each request? Guido, given the behaviour you initially described, before applying the configuration I suggested - did you receive a successful response upon subsequent requests for the same object ?

Re: N = 3 and RW = 2 not finding some keys

2018-05-18 Thread Guido Medina
A force replace done wrong causing some partitions to only have one copy. And the cluster running extremely slow have forced us to disable AAE for now. Guido. On 18/05/18 13:13, Russell Brown wrote: But why isn’t read repair “working”? On 18 May 2018, at 11:07, Bryan Hunt wrote: Of course

Re: N = 3 and RW = 2 not finding some keys

2018-05-18 Thread Russell Brown
But why isn’t read repair “working”? > On 18 May 2018, at 11:07, Bryan Hunt wrote: > > Of course, AAE will eventually repair the missing object replicas but it > seems like you need something more immediate. > >> On 18 May 2018, at 11:00, Bryan Hunt wrote: >> >> Hi Guido, >> >> You should

Re: N = 3 and RW = 2 not finding some keys

2018-05-18 Thread Bryan Hunt
Of course, AAE will eventually repair the missing object replicas but it seems like you need something more immediate. > On 18 May 2018, at 11:00, Bryan Hunt wrote: > > Hi Guido, > > You should attempt to change the bucket property ‘notfound_ok’ from the > default of ‘true' to ‘false'. > >

Re: N = 3 and RW = 2 not finding some keys

2018-05-18 Thread Bryan Hunt
Hi Guido, You should attempt to change the bucket property ‘notfound_ok’ from the default of ‘true' to ‘false'. I.e curl -XPUT 127.0.0.1:10018/buckets/foo/props -H "Content-Type: application/json" -d '{"props":{"notfound_ok": false}}' This makes GET operations for non-existent keys slower a