Re: Search Crashes
is this still an issues on 2.0?
On Sat, Nov 23, 2013 at 12:07 PM, John Daily wrote:
> allow_mult=true is the recommended approach for addressing conflict
> resolution; when multiple writes to the same object are received and vector
> clocks cannot determine a causal history, all copies of the writes are
> retained for future merging/selection by the application. We are planning to
> make this the default setting for Riak 2.0.
> A couple of blog posts (pardon me for referencing my own content):
> https://basho.com/understanding-riaks-configurable-behaviors-part-1/
> https://basho.com/clocks-are-bad-or-welcome-to-distributed-systems/
> I imagine that the presence of these multiple copies (aka “siblings”) is what
> is breaking legacy search, but that’s purely guesswork on my part.
> -John
> On Nov 23, 2013, at 12:00 PM, Kartik Thakore wrote:
>> what does allow_mult do and why is it breaking the search?
>>
>> just curious
>>
>>
>> On Sat, Nov 23, 2013 at 11:00 AM, Mark Phillips wrote:
>>
>> Please send all rock star praise payments to m...@basho.com. I'll make sure
>> Ryan gets it. ;)
>>
>> On Friday, November 22, 2013, Gabriel Littman wrote:
>>
>> PS: The searching that should include this element that failed before now
>> succeeds as well.
>>
>> >>> r.fulltext_search('ctv_tvdata', "searchable_key:monstersvsaliens")
>> {'max_score': 0.353553, 'num_found': 1, 'docs': [{'date': '2013-11-23',
>> 'expires': '99', 'subject_name': 'Monsters vs. Aliens', 'sub_type':
>> 'Series', 'topic':
>> '__ref--/data/v2/topic/TMS.Show.9838380:r1384276501.854346', 'person':
>> '__None__', 'searchable_key': 'aliens vs monstersvsaliens monsters',
>> 'genre': 'Children', 'sport': '__None__', 'type': 'show', u'id':
>> '/data/v2/search_show/TMS.Show.9838380'}]}
>>
>>
>> On Fri, Nov 22, 2013 at 8:07 PM, Gabriel Littman wrote:
>> Hi Ryan,
>>
>> You are a rock star. I owe you $5.
>>
>> The bucket had allow_mult set to true:
>> $ curl 'http://10.1.2.95:8098/buckets/_rsid_ctv_tvdata/props'
>> {"props":{"name":"_rsid_ctv_tvdata","allow_mult":true,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":1,"small_vclock":50,"w":1,"young_vclock":20}}
>>
>> I fixed it in the python console since that is where I'm more comfortable.
>> And just to be sure I attempted the same riak PUT both before and after.
>>
>> >>> import riak
>> >>> r = riak.RiakClient()
>> >>> b = r.bucket('ctv_tvdata')
>> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
>> >>> o.store()
>> Traceback (most recent call last):
>> File "", line 1, in
>> File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
>> 281, in store
>> timeout=timeout)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
>> line 127, in wrapper
>> return self._with_retries(pool, thunk)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
>> line 69, in _with_retries
>> return fn(transport)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
>> line 125, in thunk
>> return fn(self, transport, *args, **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
>> line 289, in put
>> timeout=timeout)
>> File
>> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
>> line 144, in put
>> return self._parse_body(robj, response, [200, 201, 204, 300])
>> File
>> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/codec.py", line
>> 64, in _parse_body
>> self.check_http_code(status, expected_statuses)
>> File
>> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
>> line 446, in check_http_code
>> (expected_statuses, status))
>> Exception: Expected status [200, 201, 204, 300], received 500
>> >>> b2 = r.bucket('_rsid_ctv_tvdata')
>> >>> b2.get_properties()
>> {'old_vclock': 86400, 'pr': 0, 'allow_mult': True, 'big_vclock': 50, 'name':
>> '_rsid_ctv_tvdata', 'chash_keyfun': {'fun': 'chash_std_keyfun', 'mod':
>> 'riak_core_util'}, 'n_val': 3, 'notfound_ok': False, 'linkfun': {'fun':
>> 'mapreduce_linkfun', 'mod': 'riak_kv_wm_link_walker'}, 'pw': 0,
>> 'last_write_wins': False, 'r': 1, 'small_vclock': 50, 'rw': 1,
>> 'basic_quorum': False, 'postcommit': [], 'dw': 0, 'w': 1, 'youn
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
allow_mult=true is the recommended approach for addressing conflict resolution;
when multiple writes to the same object are received and vector clocks cannot
determine a causal history, all copies of the writes are retained for future
merging/selection by the application. We are planning to make this the default
setting for Riak 2.0.
A couple of blog posts (pardon me for referencing my own content):
https://basho.com/understanding-riaks-configurable-behaviors-part-1/
https://basho.com/clocks-are-bad-or-welcome-to-distributed-systems/
I imagine that the presence of these multiple copies (aka “siblings”) is what
is breaking legacy search, but that’s purely guesswork on my part.
-John
On Nov 23, 2013, at 12:00 PM, Kartik Thakore wrote:
> what does allow_mult do and why is it breaking the search?
>
> just curious
>
>
> On Sat, Nov 23, 2013 at 11:00 AM, Mark Phillips wrote:
>
> Please send all rock star praise payments to m...@basho.com. I'll make sure
> Ryan gets it. ;)
>
> On Friday, November 22, 2013, Gabriel Littman wrote:
>
> PS: The searching that should include this element that failed before now
> succeeds as well.
>
> >>> r.fulltext_search('ctv_tvdata', "searchable_key:monstersvsaliens")
> {'max_score': 0.353553, 'num_found': 1, 'docs': [{'date': '2013-11-23',
> 'expires': '99', 'subject_name': 'Monsters vs. Aliens', 'sub_type':
> 'Series', 'topic':
> '__ref--/data/v2/topic/TMS.Show.9838380:r1384276501.854346', 'person':
> '__None__', 'searchable_key': 'aliens vs monstersvsaliens monsters', 'genre':
> 'Children', 'sport': '__None__', 'type': 'show', u'id':
> '/data/v2/search_show/TMS.Show.9838380'}]}
>
>
> On Fri, Nov 22, 2013 at 8:07 PM, Gabriel Littman wrote:
> Hi Ryan,
>
> You are a rock star. I owe you $5.
>
> The bucket had allow_mult set to true:
> $ curl 'http://10.1.2.95:8098/buckets/_rsid_ctv_tvdata/props'
> {"props":{"name":"_rsid_ctv_tvdata","allow_mult":true,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":1,"small_vclock":50,"w":1,"young_vclock":20}}
>
> I fixed it in the python console since that is where I'm more comfortable.
> And just to be sure I attempted the same riak PUT both before and after.
>
> >>> import riak
> >>> r = riak.RiakClient()
> >>> b = r.bucket('ctv_tvdata')
> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
> >>> o.store()
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
> 281, in store
> timeout=timeout)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 127, in wrapper
> return self._with_retries(pool, thunk)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 69, in _with_retries
> return fn(transport)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 125, in thunk
> return fn(self, transport, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
> line 289, in put
> timeout=timeout)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 144, in put
> return self._parse_body(robj, response, [200, 201, 204, 300])
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/codec.py", line
> 64, in _parse_body
> self.check_http_code(status, expected_statuses)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 446, in check_http_code
> (expected_statuses, status))
> Exception: Expected status [200, 201, 204, 300], received 500
> >>> b2 = r.bucket('_rsid_ctv_tvdata')
> >>> b2.get_properties()
> {'old_vclock': 86400, 'pr': 0, 'allow_mult': True, 'big_vclock': 50, 'name':
> '_rsid_ctv_tvdata', 'chash_keyfun': {'fun': 'chash_std_keyfun', 'mod':
> 'riak_core_util'}, 'n_val': 3, 'notfound_ok': False, 'linkfun': {'fun':
> 'mapreduce_linkfun', 'mod': 'riak_kv_wm_link_walker'}, 'pw': 0,
> 'last_write_wins': False, 'r': 1, 'small_vclock': 50, 'rw': 1,
> 'basic_quorum': False, 'postcommit': [], 'dw': 0, 'w': 1, 'youn
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
what does allow_mult do and why is it breaking the search?
just curious
On Sat, Nov 23, 2013 at 11:00 AM, Mark Phillips wrote:
> Please send all rock star praise payments to m...@basho.com. I'll make sure
> Ryan gets it. ;)
> On Friday, November 22, 2013, Gabriel Littman wrote:
>> PS: The searching that should include this element that failed before now
>> succeeds as well.
>>
>> >>> r.fulltext_search('ctv_tvdata', "searchable_key:monstersvsaliens")
>> {'max_score': 0.353553, 'num_found': 1, 'docs': [{'date': '2013-11-23',
>> 'expires': '99', 'subject_name': 'Monsters vs. Aliens', 'sub_type':
>> 'Series', 'topic':
>> '__ref--/data/v2/topic/TMS.Show.9838380:r1384276501.854346', 'person':
>> '__None__', 'searchable_key': 'aliens vs monstersvsaliens monsters',
>> 'genre': 'Children', 'sport': '__None__', 'type': 'show', u'id':
>> '/data/v2/search_show/TMS.Show.9838380'}]}
>>
>>
>> On Fri, Nov 22, 2013 at 8:07 PM, Gabriel Littman wrote:
>>
>> Hi Ryan,
>>
>> You are a rock star. I owe you $5.
>>
>> The bucket had allow_mult set to true:
>> $ curl 'http://10.1.2.95:8098/buckets/_rsid_ctv_tvdata/props'
>>
>> {"props":{"name":"_rsid_ctv_tvdata","allow_mult":true,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":1,"small_vclock":50,"w":1,"young_vclock":20}}
>>
>> I fixed it in the python console since that is where I'm more comfortable.
>> And just to be sure I attempted the same riak PUT both before and after.
>>
>> >>> import riak
>> >>> r = riak.RiakClient()
>> >>> b = r.bucket('ctv_tvdata')
>> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
>> >>> o.store()
>> Traceback (most recent call last):
>>File "", line 1, in
>> File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
>> 281, in store
>> timeout=timeout)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
>> line 127, in wrapper
>> return self._with_retries(pool, thunk)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
>> line 69, in _with_retries
>> return fn(transport)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
>> line 125, in thunk
>> return fn(self, transport, *args, **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
>> line 289, in put
>> timeout=timeout)
>> File
>> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
>> line 144, in put
>> return self._parse_body(robj, response, [200, 201, 204, 300])
>> File
>> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/codec.py",
>> line 64, in _parse_body
>> self.check_http_code(status, expected_statuses)
>> File
>> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
>> line 446, in check_http_code
>> (expected_statuses, status))
>> Exception: Expected status [200, 201, 204, 300], received 500
>> >>> b2 = r.bucket('_rsid_ctv_tvdata')
>> >>> b2.get_properties()
>> {'old_vclock': 86400, 'pr': 0, 'allow_mult': True, 'big_vclock': 50,
>> 'name': '_rsid_ctv_tvdata', 'chash_keyfun': {'fun': 'chash_std_keyfun',
>> 'mod': 'riak_core_util'}, 'n_val': 3, 'notfound_ok': False, 'linkfun':
>> {'fun': 'mapreduce_linkfun', 'mod': 'riak_kv_wm_link_walker'}, 'pw': 0,
>> 'last_write_wins': False, 'r': 1, 'small_vclock': 50, 'rw': 1,
>> 'basic_quorum': False, 'postcommit': [], 'dw': 0, 'w': 1, 'youn
>>
>>___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
Please send all rock star praise payments to m...@basho.com. I'll make sure
Ryan gets it. ;)
On Friday, November 22, 2013, Gabriel Littman wrote:
> PS: The searching that should include this element that failed before now
> succeeds as well.
>
> >>> r.fulltext_search('ctv_tvdata', "searchable_key:monstersvsaliens")
> {'max_score': 0.353553, 'num_found': 1, 'docs': [{'date': '2013-11-23',
> 'expires': '99', 'subject_name': 'Monsters vs. Aliens', 'sub_type':
> 'Series', 'topic':
> '__ref--/data/v2/topic/TMS.Show.9838380:r1384276501.854346', 'person':
> '__None__', 'searchable_key': 'aliens vs monstersvsaliens monsters',
> 'genre': 'Children', 'sport': '__None__', 'type': 'show', u'id':
> '/data/v2/search_show/TMS.Show.9838380'}]}
>
>
> On Fri, Nov 22, 2013 at 8:07 PM, Gabriel Littman wrote:
>
> Hi Ryan,
>
> You are a rock star. I owe you $5.
>
> The bucket had allow_mult set to true:
> $ curl 'http://10.1.2.95:8098/buckets/_rsid_ctv_tvdata/props'
>
> {"props":{"name":"_rsid_ctv_tvdata","allow_mult":true,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":1,"small_vclock":50,"w":1,"young_vclock":20}}
>
> I fixed it in the python console since that is where I'm more comfortable.
> And just to be sure I attempted the same riak PUT both before and after.
>
> >>> import riak
> >>> r = riak.RiakClient()
> >>> b = r.bucket('ctv_tvdata')
> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
> >>> o.store()
> Traceback (most recent call last):
>File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
> 281, in store
> timeout=timeout)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 127, in wrapper
> return self._with_retries(pool, thunk)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 69, in _with_retries
> return fn(transport)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 125, in thunk
> return fn(self, transport, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
> line 289, in put
> timeout=timeout)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 144, in put
> return self._parse_body(robj, response, [200, 201, 204, 300])
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/codec.py",
> line 64, in _parse_body
> self.check_http_code(status, expected_statuses)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 446, in check_http_code
> (expected_statuses, status))
> Exception: Expected status [200, 201, 204, 300], received 500
> >>> b2 = r.bucket('_rsid_ctv_tvdata')
> >>> b2.get_properties()
> {'old_vclock': 86400, 'pr': 0, 'allow_mult': True, 'big_vclock': 50,
> 'name': '_rsid_ctv_tvdata', 'chash_keyfun': {'fun': 'chash_std_keyfun',
> 'mod': 'riak_core_util'}, 'n_val': 3, 'notfound_ok': False, 'linkfun':
> {'fun': 'mapreduce_linkfun', 'mod': 'riak_kv_wm_link_walker'}, 'pw': 0,
> 'last_write_wins': False, 'r': 1, 'small_vclock': 50, 'rw': 1,
> 'basic_quorum': False, 'postcommit': [], 'dw': 0, 'w': 1, 'youn
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
PS: The searching that should include this element that failed before now
succeeds as well.
>>> r.fulltext_search('ctv_tvdata', "searchable_key:monstersvsaliens")
{'max_score': 0.353553, 'num_found': 1, 'docs': [{'date': '2013-11-23',
'expires': '99', 'subject_name': 'Monsters vs. Aliens', 'sub_type':
'Series', 'topic':
'__ref--/data/v2/topic/TMS.Show.9838380:r1384276501.854346', 'person':
'__None__', 'searchable_key': 'aliens vs monstersvsaliens monsters',
'genre': 'Children', 'sport': '__None__', 'type': 'show', u'id':
'/data/v2/search_show/TMS.Show.9838380'}]}
On Fri, Nov 22, 2013 at 8:07 PM, Gabriel Littman wrote:
> Hi Ryan,
>
> You are a rock star. I owe you $5.
>
> The bucket had allow_mult set to true:
> $ curl 'http://10.1.2.95:8098/buckets/_rsid_ctv_tvdata/props'
>
> {"props":{"name":"_rsid_ctv_tvdata","allow_mult":true,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":1,"small_vclock":50,"w":1,"young_vclock":20}}
>
> I fixed it in the python console since that is where I'm more comfortable.
> And just to be sure I attempted the same riak PUT both before and after.
>
> >>> import riak
> >>> r = riak.RiakClient()
> >>> b = r.bucket('ctv_tvdata')
> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
> >>> o.store()
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
> 281, in store
> timeout=timeout)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 127, in wrapper
> return self._with_retries(pool, thunk)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 69, in _with_retries
> return fn(transport)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 125, in thunk
> return fn(self, transport, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
> line 289, in put
> timeout=timeout)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 144, in put
> return self._parse_body(robj, response, [200, 201, 204, 300])
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/codec.py",
> line 64, in _parse_body
> self.check_http_code(status, expected_statuses)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 446, in check_http_code
> (expected_statuses, status))
> Exception: Expected status [200, 201, 204, 300], received 500
> >>> b2 = r.bucket('_rsid_ctv_tvdata')
> >>> b2.get_properties()
> {'old_vclock': 86400, 'pr': 0, 'allow_mult': True, 'big_vclock': 50,
> 'name': '_rsid_ctv_tvdata', 'chash_keyfun': {'fun': 'chash_std_keyfun',
> 'mod': 'riak_core_util'}, 'n_val': 3, 'notfound_ok': False, 'linkfun':
> {'fun': 'mapreduce_linkfun', 'mod': 'riak_kv_wm_link_walker'}, 'pw': 0,
> 'last_write_wins': False, 'r': 1, 'small_vclock': 50, 'rw': 1,
> 'basic_quorum': False, 'postcommit': [], 'dw': 0, 'w': 1, 'young_vclock':
> 20, 'precommit': []}
> >>> b2.set_property('allow_mult', False)
> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
> >>> o.store()
>
>
> As you can see it worked like a charm. Thank you so much for your help.
> I never would have figured this out on my own.
>
> Once again, you rock,
>
> Gabe
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
Hi Ryan,
You are a rock star. I owe you $5.
The bucket had allow_mult set to true:
$ curl 'http://10.1.2.95:8098/buckets/_rsid_ctv_tvdata/props'
{"props":{"name":"_rsid_ctv_tvdata","allow_mult":true,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":1,"small_vclock":50,"w":1,"young_vclock":20}}
I fixed it in the python console since that is where I'm more comfortable.
And just to be sure I attempted the same riak PUT both before and after.
>>> import riak
>>> r = riak.RiakClient()
>>> b = r.bucket('ctv_tvdata')
>>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
>>> o.store()
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
281, in store
timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
line 127, in wrapper
return self._with_retries(pool, thunk)
File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
line 69, in _with_retries
return fn(transport)
File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
line 125, in thunk
return fn(self, transport, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
line 289, in put
timeout=timeout)
File
"/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
line 144, in put
return self._parse_body(robj, response, [200, 201, 204, 300])
File
"/usr/local/lib/python2.7/dist-packages/riak/transports/http/codec.py",
line 64, in _parse_body
self.check_http_code(status, expected_statuses)
File
"/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
line 446, in check_http_code
(expected_statuses, status))
Exception: Expected status [200, 201, 204, 300], received 500
>>> b2 = r.bucket('_rsid_ctv_tvdata')
>>> b2.get_properties()
{'old_vclock': 86400, 'pr': 0, 'allow_mult': True, 'big_vclock': 50,
'name': '_rsid_ctv_tvdata', 'chash_keyfun': {'fun': 'chash_std_keyfun',
'mod': 'riak_core_util'}, 'n_val': 3, 'notfound_ok': False, 'linkfun':
{'fun': 'mapreduce_linkfun', 'mod': 'riak_kv_wm_link_walker'}, 'pw': 0,
'last_write_wins': False, 'r': 1, 'small_vclock': 50, 'rw': 1,
'basic_quorum': False, 'postcommit': [], 'dw': 0, 'w': 1, 'young_vclock':
20, 'precommit': []}
>>> b2.set_property('allow_mult', False)
>>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
>>> o.store()
As you can see it worked like a charm. Thank you so much for your help.
I never would have figured this out on my own.
Once again, you rock,
Gabe
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
On Wed, Nov 20, 2013 at 2:38 PM, Gabriel Littman wrote:
>
>
> 1) We are installed via deb package
> ii riak 1.4.1-1
> Riak is a distributed data store
>
There's a 1.4.2 out but your issue doesn't seem to have anything do with a
specific 1.4.1 bug.
>
> 2) We did recently upgrade our to riak python library 2.0 but I also have
> a cluster still on the 1.4 client that has similar problems.
>
Okay, so for now we assume the client upgrade didn't cause the issues
either.
>
> 3) We less recently upgraded riak itself from 1.2.x to 1.4. We ended up
> starting with an empty riak store in the processes. Honestly we've had
> many problems with search index even under 1.2. Mostly riak would get into
> a state where it would continuously crash after startup until we
> deleted /var/lib/riak/merge_index on the node and then rebuilt the search
> index via read/write. The particular problems I'm having now I cannot
> confirm if they were happening under riak 1.2 or not.
>
The 1.2 issues may very well have been caused by a corruption bug that was
fixed in 1.4.0 [1].
>
> looks like allow_mult is false, but I just confirmed with my colleague
> that *it was previously set to true* so it could be that we have a hold
> over issue from that.
> $ curl 'http://10.1.2.95:8098/buckets/ctv_tvdata/props'
>
> {"props":{"allow_mult":false,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"name":"ctv_tvdata","notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[{"fun":"precommit","mod":"riak_search_kv_hook"},{"mod":"riak_search_kv_hook","fun":"precommit"}],"pw":0,"r":1,"rw":1,"search":true,"small_vclock":50,"w":1,"young_vclock":20}}
>
So after setting allow_mult back to false you'd have to make sure to
resolve any siblings but that should be done automatically for you now that
allow_mult is false again. However, the commit hook will also crash if you
have allow_mult set to true on Riak Search's special "proxy object" bucket.
Looking at your original insert crash message I notice the problem is
actually with the proxy objets stored in this bucket [2]. What does the
following curl show you:
curl 'http://host:post/buckets/_rsid_ctv_tvdata/props'
I bet $5 it has allow_mult set to true. Try setting that to false and see
what happens.
>
> Since it is now set to false now would you have a suggestion on how to
> clear the problem? (Delete merge_index?)
>
You shouldn't have to delete merge index files unless they are corrupted.
Let's see if we can fix your insert/index problem first. Then we can work
on search if it is still broken.
-Z
[1]: https://github.com/basho/merge_index/pull/30
[2]: It's not easy to see by there is the atom 'riak_idx_doc' which
indicates this is a "proxy object" created by Riak Search. If you squint
hard enough you can see the analyzed fields as well. I should have looked
more closely the first time. This is not an obvious error. I wouldn't
expect many people to pick up on it.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
Resending without entire thread:
On Wed, Nov 20, 2013 at 11:38 AM, Gabriel Littman wrote:
> Hi Ryan,
>
> Thanks for your response, let me get right into it.
>
> 1) We are installed via deb package
> ii riak 1.4.1-1
> Riak is a distributed data store
>
> 2) We did recently upgrade our to riak python library 2.0 but I also have
> a cluster still on the 1.4 client that has similar problems.
>
> 3) We less recently upgraded riak itself from 1.2.x to 1.4. We ended up
> starting with an empty riak store in the processes. Honestly we've had
> many problems with search index even under 1.2. Mostly riak would get into
> a state where it would continuously crash after startup until we
> deleted /var/lib/riak/merge_index on the node and then rebuilt the search
> index via read/write. The particular problems I'm having now I cannot
> confirm if they were happening under riak 1.2 or not.
>
> looks like allow_mult is false, but I just confirmed with my colleague
> that *it was previously set to true* so it could be that we have a hold
> over issue from that.
> $ curl 'http://10.1.2.95:8098/buckets/ctv_tvdata/props'
>
> {"props":{"allow_mult":false,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":0,"last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"name":"ctv_tvdata","notfound_ok":false,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[{"fun":"precommit","mod":"riak_search_kv_hook"},{"mod":"riak_search_kv_hook","fun":"precommit"}],"pw":0,"r":1,"rw":1,"search":true,"small_vclock":50,"w":1,"young_vclock":20}}
>
> Since it is now set to false now would you have a suggestion on how to
> clear the problem? (Delete merge_index?)
>
> I can package up my logs and send them to you directly if we still feel
> it's needed.
>
> Thank you so much for your help,
>
> Gabe
>
>
>
> On Wed, Nov 20, 2013 at 9:39 AM, Ryan Zezeski wrote:
>
>> Hi Gabriel,
>>
>> First, let me verify a few things.
>>
>> 1. You are on Riak 1.4? Which patch version? 1.4.2?
>>
>> 2. You recently upgraded you client? Did you have any of these failures
>> before upgrading the client?
>>
>> 3. Have you made any other changes between the time your system was
>> working and the time it started exhibiting these failures? For example, set
>> allow_mult=true?
>>
>> Given that you are having 'badmatch' hook crashes during insert I have
>> the suspicion that allow_mult was recently changed to true as the Riak
>> Search hook cannot deal with siblings. What does the following curl show:
>>
>> curl 'http://host:port/buckets/ctv_tvdata/props'
>>
>> If that has 'allow_mult: true' then that is your issue.
>>
>> As for your search operations. I'm not sure why they are failing. If you
>> want you could tar.gz all the logs for each node and email that to me.
>>
>> -Z
>>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Search Crashes
Hi Gabriel,
First, let me verify a few things.
1. You are on Riak 1.4? Which patch version? 1.4.2?
2. You recently upgraded you client? Did you have any of these failures
before upgrading the client?
3. Have you made any other changes between the time your system was working
and the time it started exhibiting these failures? For example, set
allow_mult=true?
Given that you are having 'badmatch' hook crashes during insert I have the
suspicion that allow_mult was recently changed to true as the Riak Search
hook cannot deal with siblings. What does the following curl show:
curl 'http://host:port/buckets/ctv_tvdata/props'
If that has 'allow_mult: true' then that is your issue.
As for your search operations. I'm not sure why they are failing. If you
want you could tar.gz all the logs for each node and email that to me.
-Z
On Mon, Nov 18, 2013 at 7:00 PM, Gabriel Littman wrote:
> Hi All,
>
> We've been working with a search enabled bucket in riak for a while now
> and off and on it has been giving us trouble. In the past it has been
> solved by reindexing all the data by just reading and writing the data back
> into riak. But even this is failing now on some input data. Any
> help/insite would be greatly appreciated.
>
> We are on riak 1.4
> We have recently switched to riak python api 2.0
>
> smrtv@fre-prod-svr15:~$ python
> Python 2.7.3 (default, Aug 1 2012, 05:14:39)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import riak
> >>> r = riak.RiakClient()
> >>> b = r.bucket('ctv_tvdata')
> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
> >>> o.data
> {'type': 'show', 'expires': '99', 'subject_name': 'Monsters vs.
> Aliens', 'sub_type': 'Series', 'topic':
> '__ref--/data/v2/topic/TMS.Show.9838380:r1384276501.854346', 'person':
> '__None__', 'searchable_key': 'aliens vs monstersvsaliens monsters',
> 'date': '2013-11-23', 'sport': '__None__', 'genre': 'Children', 'id':
> '/data/v2/search_show/TMS.Show.9838380'}
> >>> o.store()
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
> 281, in store
> timeout=timeout)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 127, in wrapper
> return self._with_retries(pool, thunk)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 69, in _with_retries
> return fn(transport)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 125, in thunk
> return fn(self, transport, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
> line 289, in put
> timeout=timeout)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 144, in put
> return self._parse_body(robj, response, [200, 201, 204, 300])
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/codec.py",
> line 64, in _parse_body
> self.check_http_code(status, expected_statuses)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/http/transport.py",
> line 446, in check_http_code
> (expected_statuses, status))
> Exception: Expected status [200, 201, 204, 300], received 500
>
> Using protocol buffs gives an erlang riak_search_kv_hook,precommit,error:
>
> >>> r = riak.RiakClent(protocol='pcb')
> Traceback (most recent call last):
> File "", line 1, in
> AttributeError: 'module' object has no attribute 'RiakClent'
> >>> r = riak.RiakClient(protocol='pcb')
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/riak/client/__init__.py",
> line 99, in __init__
> self.protocol = protocol or 'http'
> File "/usr/local/lib/python2.7/dist-packages/riak/client/__init__.py",
> line 118, in _set_protocol
> repr(self.PROTOCOLS))
> ValueError: protocol option is invalid, must be one of ['http', 'https',
> 'pbc']
> >>> r = riak.RiakClient(protocol='pbc')
> >>> b = r.bucket('ctv_tvdata')
> >>> o = b.get('/data/v2/search_show/TMS.Show.9838380')
> >>> o.store()
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/local/lib/python2.7/dist-packages/riak/riak_object.py", line
> 281, in store
> timeout=timeout)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 127, in wrapper
> return self._with_retries(pool, thunk)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 69, in _with_retries
> return fn(transport)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/transport.py",
> line 125, in thunk
> return fn(self, transport, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/riak/client/operations.py",
> line 289, in put
> timeout=timeout)
> File
> "/usr/local/lib/python2.7/dist-packages/riak/transports/pbc/transport.py",
> line 194, in put
> MSG_
