Re: Unexpected stored value for multivalued field in riak search

[Zeeshan Lakhani]

Hello Wendy,

This may be an issue with the Python client. Can you open this up as a bug at 
https://github.com/basho/riak-python-client 
<https://github.com/basho/riak-python-client>, and our client team can take a 
look? I was not able to reproduce via http in an example (i.e. *interests_set*) 
I tried:

```
"docs" : [
 {
"_yz_rt" : "maps",
"name_register" : “3 Names",
"interests_set" : [
   "thing"
],
"_yz_id" : "1*maps*testbucket*3 Names*2",
    "_yz_rk" : “3 Names",
"_yz_rb" : "testbucket"
 }
  ],...
```

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jul 1, 2015, at 11:47 AM, Wendy Liu  wrote:
> 
> Hi all,
> 
> I'm having an issue with stored multiValued fields in Riak search not being 
> stored the way I would expect.
> 
> If I use the default schema, and store the following JSON document (from the 
> documentation <http://docs.basho.com/riak/2.1.1/dev/using/search/#Extractors>)
> 
> {"people_ss": ["Ryan", "Eric", "Brett"]}
> 
> and then search with the query "people_ss:Ryan", the document that I get back 
> contains "people_ss": ["Ryan", "Eric", "Brett"], as I would expect.
> 
> However, if I instead create the document
> 
> {"people_ss": ["Ryan"]}
> 
> then searching with the query "people_ss:Ryan" results in a document with 
> "people_ss": "Ryan", where the value associated with "people_ss" is a string, 
> not a list of strings as I would expect.
> 
> I couldn't find anything about this in any of the Github issue trackers or 
> the mailing list. Is this a bug, or desired behaviour? Is there a way to 
> ensure that lists with a single element are stored in Solr as lists, instead 
> of as strings?
> 
> I'm using the official Riak Python client with Riak 2.1.1 and search enabled.
> 
> Thanks in advance!
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Unexpected stored value for multivalued field in riak search

[Zeeshan Lakhani]

Hi Wendy, 

I realized that I first read this as crdt sets and not multivalued fields, 
haha. 

Nonetheless, via http again (*groups_s*), I’ve confirmed that it’s on the 
client:

```
  "response" : {
  "maxScore" : 0.30685282,
  "numFound" : 1,
  "docs" : [
 {
"_yz_rt" : "multifunarray",
"_yz_fpn" : "49",
"groups_s" : [
   "3304cf79"
],
"_yz_rk" : "multivalued_array",
"_yz_id" : "1*multifunarray*b2*multivalued_array*49",
"_yz_ed" : "2 49 bXVsdGlmdW5hcnJheQ== YjI= bXVsdGl2YWx1ZWRfYXJyYXk= 
g2IBnehH",
"_yz_rb" : "b2",
"_yz_pn" : "49"
 }
  ],
  "start" : 0
```

I’ll update the client ticket with this example, accordingly :). Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jul 7, 2015, at 1:56 PM, Zeeshan Lakhani  wrote:
> 
> Hello Wendy,
> 
> This may be an issue with the Python client. Can you open this up as a bug at 
> https://github.com/basho/riak-python-client 
> <https://github.com/basho/riak-python-client>, and our client team can take a 
> look? I was not able to reproduce via http in an example (i.e. 
> *interests_set*) I tried:
> 
> ```
> "docs" : [
>  {
> "_yz_rt" : "maps",
>     "name_register" : “3 Names",
> "interests_set" : [
>"thing"
> ],
> "_yz_id" : "1*maps*testbucket*3 Names*2",
> "_yz_rk" : “3 Names",
> "_yz_rb" : "testbucket"
>  }
>   ],...
> ```
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Jul 1, 2015, at 11:47 AM, Wendy Liu > <mailto:ilostwa...@gmail.com>> wrote:
>> 
>> Hi all,
>> 
>> I'm having an issue with stored multiValued fields in Riak search not being 
>> stored the way I would expect.
>> 
>> If I use the default schema, and store the following JSON document (from the 
>> documentation 
>> <http://docs.basho.com/riak/2.1.1/dev/using/search/#Extractors>)
>> 
>> {"people_ss": ["Ryan", "Eric", "Brett"]}
>> 
>> and then search with the query "people_ss:Ryan", the document that I get 
>> back contains "people_ss": ["Ryan", "Eric", "Brett"], as I would expect.
>> 
>> However, if I instead create the document
>> 
>> {"people_ss": ["Ryan"]}
>> 
>> then searching with the query "people_ss:Ryan" results in a document with 
>> "people_ss": "Ryan", where the value associated with "people_ss" is a 
>> string, not a list of strings as I would expect.
>> 
>> I couldn't find anything about this in any of the Github issue trackers or 
>> the mailing list. Is this a bug, or desired behaviour? Is there a way to 
>> ensure that lists with a single element are stored in Solr as lists, instead 
>> of as strings?
>> 
>> I'm using the official Riak Python client with Riak 2.1.1 and search enabled.
>> 
>> Thanks in advance!
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna question

[Zeeshan Lakhani]

Hey Nick,

What version of Riak are you using? Did you see any errors in your Solr log? It 
may take a little bit to eventually sync up (and remove everything).

Thanks.

> On Jul 10, 2015, at 3:22 AM, Nick Golovin  wrote:
> 
> Hey, guys, could you please help me with this problem: after removing all 
> items from bucket i still can see some of them if using riak search. Why it 
> could happened and how to fix it?
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak 2.1.1 not starting with "search = on"

[Zeeshan Lakhani]

Hello,

We’ve done upgrades and have tests that upgrade from 2.0.* to 2.1.1 without 
issue.

Are you seeing any errors in the solr logs themselves? That may help us debug 
this. Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jul 17, 2015, at 1:11 PM, SodaHanker  wrote:
> 
> I am new working with Riak, so it might be a simple oversight, but working
> with Riak 2.0.5 I had no problems with Riak starting, when "search = on",
> when Riak was fully installed. But recently attempting to work with Riak
> 2.1.1, following all the same instructions as 2.0.5, when I attempt to start
> a node with "search = on" in the riak.conf the node times out and fails to
> start. My crash.log reports 
> 
> 2015-07-17 12:05:32 =ERROR REPORT
> ** Generic server yz_solr_proc terminating
> ** Last message in was {#Port<0.9597>,{exit_status,1}}
> ** When Server state == {state,"/var/lib/riak/yz",#Port<0.9597>,8093,8985}
> ** Reason for termination ==
> ** {"solr OS process exited",1}
> 2015-07-17 12:05:32 =CRASH REPORT
>  crasher:
>initial call: yz_solr_proc:init/1
>pid: <0.606.0>
>registered_name: yz_solr_proc
>exception exit: {{"solr OS process
> exited",1},[{gen_server,terminate,6,[{file,"gen_server.erl"},{line,744}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}
>ancestors: [yz_solr_sup,yz_sup,<0.603.0>]
>messages: [{'EXIT',#Port<0.9597>,normal}]
>links: [<0.605.0>]
>dictionary: []
>trap_exit: true
>status: running
>heap_size: 376
>stack_size: 27
>reductions: 6996
>  neighbours:
> 
> 2015-07-17 12:05:32 =SUPERVISOR REPORT
> Supervisor: {local,yz_solr_sup}
> Context:shutdown
> Reason: reached_max_restart_intensity
> Offender:  
> [{pid,<0.647.0>},{name,yz_solr_proc},{mfargs,{yz_solr_proc,start_link,["/var/lib/riak/yz",8093,8985]}},{restart_type,permanent},{shutdown,5000},{child_type,worker}]
> 
> 2015-07-17 12:05:32 =SUPERVISOR REPORT
> Supervisor: {local,yz_sup}
> Context:child_terminated
> Reason: shutdown
> Offender:  
> [{pid,<0.605.0>},{name,yz_solr_sup},{mfargs,{yz_solr_sup,start_link,[]}},{restart_type,permanent},{shutdown,5000},{child_type,supervisor}]
> 
> I am wondering if there is something I have configured wrong. If someone
> knows what is wrong or thinks they could solve it but requires more
> information I would love help.
> 
> 
> 
> --
> View this message in context: 
> http://riak-users.197444.n3.nabble.com/Riak-2-1-1-not-starting-with-search-on-tp4033223.html
> Sent from the Riak Users mailing list archive at Nabble.com.
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Selective indexing of Riak Data Types

[Zeeshan Lakhani]

Hello Marius, 

Yes, you can by just specifying those fields in the schema, e.g. 
field_1_register. For example, I created similar fields in this test (using a 
custom schema, inlined): 
https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR75
 
<https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR75>,
 and then set them in the map like so: 
https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR178
 
<https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR178>.
 Obviously, you then don’t need the default, embedded dynamic *_register fields 
and such, which captures all of them.

Hope that helps!

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jul 31, 2015, at 10:59 AM, Marius Cobzarenco  wrote:
> 
> Hi all,
> 
> I was curios if there's a way to enable Solr indexing on only some of the 
> fields of a Riak CRDT Map (not all of them). E.g. for a map:
> [{"field_1", "register"}, {"field_2", "register"}] I would like to _only_ 
> create the Solar index field_1_register .
> 
> By default if I enable search on a bucket with maps, Riak creates Solr 
> indexes for all fields in the map.
> https://docs.basho.com/riak/latest/dev/search/search-data-types/ 
> <https://docs.basho.com/riak/latest/dev/search/search-data-types/>
> 
> Many thanks,
> Marius
> 
> 
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Selective indexing of Riak Data Types

[Zeeshan Lakhani]

No problem :).

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jul 31, 2015, at 11:17 AM, Marius Cobzarenco  wrote:
> 
> Hi Zeeshan,
> 
> It makes sense, thanks loads for the pointer & code examples, it's exactly 
> what I was looking for.
> 
> Cheers,
> Marius
> 
> On 31 July 2015 at 16:14, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hello Marius, 
> 
> Yes, you can by just specifying those fields in the schema, e.g. 
> field_1_register. For example, I created similar fields in this test (using a 
> custom schema, inlined): 
> https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR75
>  
> <https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR75>,
>  and then set them in the map like so: 
> https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR178
>  
> <https://github.com/basho/riak_test/pull/800/files#diff-ab08303805d0b8fe5d7fe683273b2fbfR178>.
>  Obviously, you then don’t need the default, embedded dynamic *_register 
> fields and such, which captures all of them.
> 
> Hope that helps!
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Jul 31, 2015, at 10:59 AM, Marius Cobzarenco > <mailto:mar...@reinfer.io>> wrote:
>> 
>> Hi all,
>> 
>> I was curios if there's a way to enable Solr indexing on only some of the 
>> fields of a Riak CRDT Map (not all of them). E.g. for a map:
>> [{"field_1", "register"}, {"field_2", "register"}] I would like to _only_ 
>> create the Solar index field_1_register .
>> 
>> By default if I enable search on a bucket with maps, Riak creates Solr 
>> indexes for all fields in the map.
>> https://docs.basho.com/riak/latest/dev/search/search-data-types/ 
>> <https://docs.basho.com/riak/latest/dev/search/search-data-types/>
>> 
>> Many thanks,
>> Marius
>> 
>> 
>> 
>> 
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 
> 
> 
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: SolrException: Error CREATEing SolrCore after deleting index

[Zeeshan Lakhani]

Hello Hao,

I’m guessing that `todoriak_main_movie_idx` is the index you removed? Can you 
check to see if there are any buckets or bucket_types associated with that 
search_index? Also, what version of Riak are you using? 

We have a test here w/ the pb client to test a delete: 
https://github.com/basho/yokozuna/blob/3e749512d2df07c81def3c9c592615fd8d2d1234/riak_test/yz_pb.erl#L175
 
.
 I’m guessing that there may have been a race condition set in when deleting 
the index and the delete registering on all the nodes, as the operation happens 
asynchronously over cluster metadata. 

Is the error occurring on many nodes or just one?

Thanks. 

> On Aug 10, 2015, at 4:54 AM, Hao  wrote:
> 
> Error opening new searcher

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search in bucket with write_once = true

[Zeeshan Lakhani]

Hey Marius,

To put it simply, yes, 2.1.1 does not support write_once with search. Some 
objects will get indexed, but it will take a very long time and there will be 
issues with the cluster, etc…

There should have been a note about this. Our apologies. We have solved the 
issue, and it will be supported in 2.1.2, which is our next release. You can 
read more about it here: https://github.com/basho/yokozuna/issues/512 
<https://github.com/basho/yokozuna/issues/512>. 

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Aug 10, 2015, at 4:22 PM, Marius Cobzarenco  wrote:
> 
> Hi everyone,
> 
> Riak Search does not seem to work in a bucket whose bucket type has 
> write_once = true. I could not see this mentioned anywhere in the 
> documentation, but I was curious if it is to be expected? It seems like the 
> objects never get indexed. 
> 
> My bucket type is:
> 
> riak-admin bucket-type create raw_tweets \
> '{"props": {"search": true, "search_index": "index_raw_tweets", "n_val": 
> 2, "allow_mult": false, "write_once": true}}' && \
> riak-admin bucket-type activate raw_tweets
> 
> 
> If I remove "write_once":true, the objects are indexed.
> 
> Many thanks
> Marius
> 
> 
> 
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: X-Riak-Meta-* and Solr

[Zeeshan Lakhani]

Hey Joe,

Yes, you are able to index with meta/tag fields. Here’s an example in one of 
our erlang tests using the riak_erlang pb client: 
https://github.com/basho/yokozuna/blob/32d71135644a579943799ff9a70b67dee0201f80/riak_test/yokozuna_essential.erl#L306
 
<https://github.com/basho/yokozuna/blob/32d71135644a579943799ff9a70b67dee0201f80/riak_test/yokozuna_essential.erl#L306>.
 

`user_s` is indexed, for example, b/c of 
https://github.com/basho/yokozuna/blob/develop/priv/default_schema.xml#L36 
<https://github.com/basho/yokozuna/blob/develop/priv/default_schema.xml#L36> in 
the default schema.

Hope this helps. Thanks.


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Aug 11, 2015, at 5:34 PM, Joe Olson  wrote:
> 
> 
> Is it possible to automatically index custom X-Riak-Meta-* fields with Solr? 
> Do I have to create a custom extractor or modify the default search schema as 
> outlined at http://docs.basho.com/riak/latest/dev/search/custom-extractors/ ?
> 
> Here is my python code I am using to test:
> 
> # Create search index
> client = RiakClient(protocol='pbc',nodes=[{ 'host': 'xxx.yyy.zzz.www', 
> 'pb_port': 8087 }])
> client.create_search_index('testindex1', '_yz_default')
> 
> # On cluster: create bucket type, assign index 
> # riak-admin bucket-type create solrindexed 
> '{"props":{"search_index":"testindex1"}}'
> # riak-admin bucket-type activate solrindexed
> 
> # Create instance of bucket using bucket type
> bucket = client.bucket_type('solrindexed').bucket('myBucket')
> 
> # Create new key in bucket
> obj = RiakObject(client, bucket, 'myKey')
> 
> # Create metadata in headers, integers not allowed here???
> obj.usermeta = {"startTime":"144350640", "endTime":"144350700"}
> 
> # Jam some random bytes in there
> obj.encoded_data = open("/dev/urandom","rb").read(int(1258291))
> obj.content_type = 'application/octet-stream'
> 
> # Write it
> obj.store()
> 
> # Now see if we can find it..what is the syntax here? Always returns 
> 'num_found': 0
> results = client.fulltext_search('testindex1', 
> 'X-Riak-Meta-startTime:144350640')
> print results
> 
> Also, according to the documentation at 
> http://basho.github.io/riak-python-client/object.html, 'usermeta' can only 
> have strings on the value side. Is there an explanation why integers aren't 
> allowed?
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search Pagination

[Zeeshan Lakhani]

The coverage plan can change per query. Are you providing a sort on the query? 
If not or if by score, does each  item return the same score?

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Dec 21, 2015, at 18:34, Garrido  wrote:
> 
> Hello, 
> 
> Recently we migrated our Riak nodes to another network, so we backup the data 
> and then regenerate the ring, all is well, but there is a strange behaviour 
> in a riak search, for example if we execute a query using the 
> riak_erlang_client, returns the objects in the order:
> 
> A, B, C
> 
> And then if we execute again the same query the result is:
> 
> B, A, C, 
> 
> So, in other order, do you know what is causing this?, before to change our 
> riak ring to another network, it was working perfectly.
> 
> Thank you
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search Pagination

[Zeeshan Lakhani]

Best to provide a specific sort ordering on a field if you can. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Dec 21, 2015, at 21:54, Garrido  wrote:
> 
> No, we don’t provide a sort on the query, let us check and we can tell you if 
> its the same score, but, in case of  search returns the same score, which one 
> will be the solution?
> 
>> On Dec 21, 2015, at 8:21 PM, Zeeshan Lakhani  wrote:
>> 
>> The coverage plan can change per query. Are you providing a sort on the 
>> query? If not or if by score, does each  item return the same score?
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love |
>> twitter => @zeeshanlakhani
>> 
>>> On Dec 21, 2015, at 18:34, Garrido  wrote:
>>> 
>>> Hello, 
>>> 
>>> Recently we migrated our Riak nodes to another network, so we backup the 
>>> data and then regenerate the ring, all is well, but there is a strange 
>>> behaviour in a riak search, for example if we execute a query using the 
>>> riak_erlang_client, returns the objects in the order:
>>> 
>>> A, B, C
>>> 
>>> And then if we execute again the same query the result is:
>>> 
>>> B, A, C, 
>>> 
>>> So, in other order, do you know what is causing this?, before to change our 
>>> riak ring to another network, it was working perfectly.
>>> 
>>> Thank you
>>> ___
>>> riak-users mailing list
>>> riak-users@lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search Pagination

[Zeeshan Lakhani]

Yes. Score is the default ordering. For  "ties," insertion order is the 
default. That order could change based on which node is replicated to first and 
the different coverage plan may showcase that. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Dec 21, 2015, at 22:34, Jorge Garrido  wrote:
> 
> Great!! But can you explain why this issue? 
> 
>> On Monday, December 21, 2015, Zeeshan Lakhani  wrote:
>> Best to provide a specific sort ordering on a field if you can.
>> 
>> Zeeshan Lakhani
>> programmer |
>> software engineer at @basho |
>> org. member/founder of @papers_we_love |
>> twitter => @zeeshanlakhani
>> 
>> > On Dec 21, 2015, at 21:54, Garrido  wrote:
>> >
>> > No, we don’t provide a sort on the query, let us check and we can tell you 
>> > if its the same score, but, in case of  search returns the same score, 
>> > which one will be the solution?
>> >
>> >> On Dec 21, 2015, at 8:21 PM, Zeeshan Lakhani  wrote:
>> >>
>> >> The coverage plan can change per query. Are you providing a sort on the 
>> >> query? If not or if by score, does each  item return the same score?
>> >>
>> >> Zeeshan Lakhani
>> >> programmer |
>> >> software engineer at @basho |
>> >> org. member/founder of @papers_we_love |
>> >> twitter => @zeeshanlakhani
>> >>
>> >>> On Dec 21, 2015, at 18:34, Garrido  wrote:
>> >>>
>> >>> Hello,
>> >>>
>> >>> Recently we migrated our Riak nodes to another network, so we backup the 
>> >>> data and then regenerate the ring, all is well, but there is a strange 
>> >>> behaviour in a riak search, for example if we execute a query using the 
>> >>> riak_erlang_client, returns the objects in the order:
>> >>>
>> >>> A, B, C
>> >>>
>> >>> And then if we execute again the same query the result is:
>> >>>
>> >>> B, A, C,
>> >>>
>> >>> So, in other order, do you know what is causing this?, before to change 
>> >>> our riak ring to another network, it was working perfectly.
>> >>>
>> >>> Thank you
>> >>> ___
>> >>> riak-users mailing list
>> >>> riak-users@lists.basho.com
>> >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search 2.0 - CRDT Sets within Maps not returned in search results

[Zeeshan Lakhani]

Weixi,

Are you using 2.0.0 or another 2.0.* version?

If you look at 
https://github.com/basho/yokozuna/blob/2.0.0/priv/default_schema.xml#L104 
<https://github.com/basho/yokozuna/blob/2.0.0/priv/default_schema.xml#L104>, 
sets are not stored in the default schema, as they can get large. They’re 
indexed for search, but will not return in the result-set. In 2.0.7 and in 
2.1.3, the default schema was changed to store them. If you modify the existing 
default schema you have (stored=true), reload the schema, and reindex, you’ll 
see the results.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jan 27, 2016, at 2:34 PM, Weixi Yen  wrote:
> 
> I have a CRDT Map bucket that I'm running searches on in Yokozuna (using 
> _yz_default index for now)
> 
> However, the search results do not return any fields of the map that are CRDT 
> sets.  Is this intended behavior?
> 
> Thanks in advance!
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search 2.0 - CRDT Sets within Maps not returned in search results

[Zeeshan Lakhani]

Yep, it was changed in latest versions of 2.0 and 2.1. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jan 27, 2016, at 3:19 PM, Weixi Yen  wrote:
> 
> Just FYI, I'm actually running 2.1.0 version of Riak KV and the default 
> schema is showing stored=false.  2.1.0-0-g6e88b24 to be specific on ubuntu.
> 
> On OSX it's the same issue as well for 2.1.1
> http://s3.amazonaws.com/downloads.basho.com/riak/2.1/2.1.1/osx/10.8/riak-2.1.1-OSX-x86_64.tar.gz
>  
> <http://s3.amazonaws.com/downloads.basho.com/riak/2.1/2.1.1/osx/10.8/riak-2.1.1-OSX-x86_64.tar.gz>
> 
> It seems that 2.1.3 has stored=true though.
> 
> 
> On Wed, Jan 27, 2016 at 11:47 AM, Weixi Yen  <mailto:we...@spacechatapp.com>> wrote:
> Awesome, that's what I was looking for, thank you!
> 
> On Wed, Jan 27, 2016 at 11:40 AM, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Weixi,
> 
> Are you using 2.0.0 or another 2.0.* version?
> 
> If you look at 
> https://github.com/basho/yokozuna/blob/2.0.0/priv/default_schema.xml#L104 
> <https://github.com/basho/yokozuna/blob/2.0.0/priv/default_schema.xml#L104>, 
> sets are not stored in the default schema, as they can get large. They’re 
> indexed for search, but will not return in the result-set. In 2.0.7 and in 
> 2.1.3, the default schema was changed to store them. If you modify the 
> existing default schema you have (stored=true), reload the schema, and 
> reindex, you’ll see the results.
> 
> Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Jan 27, 2016, at 2:34 PM, Weixi Yen > <mailto:we...@spacechatapp.com>> wrote:
>> 
>> I have a CRDT Map bucket that I'm running searches on in Yokozuna (using 
>> _yz_default index for now)
>> 
>> However, the search results do not return any fields of the map that are 
>> CRDT sets.  Is this intended behavior?
>> 
>> Thanks in advance!
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: A Solr join query in Riak KV?

[Zeeshan Lakhani]

Hey Joe,

Riak Search operates via Distributed Solr, for which “join” is *still* not 
supported: https://issues.apache.org/jira/browse/LUCENE-3759 
<https://issues.apache.org/jira/browse/LUCENE-3759>.

I’d recommend reading this LucidWorks post as well, related to “joins” and 
SolrCloud - 
https://support.lucidworks.com/hc/en-us/articles/201298257-Sharding-w-joins-SolrCloud-question
 
<https://support.lucidworks.com/hc/en-us/articles/201298257-Sharding-w-joins-SolrCloud-question>.
 

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Feb 8, 2016, at 2:36 PM, Joe Olson  wrote:
> 
> I'm trying to get a Solr join query to work on our Riak KV cluster. 
> 
> The Solr join query is documented here:
> 
> https://wiki.apache.org/solr/Join
> 
> Using the example under the "Compared to SQL" heading, I am formatting my 
> http request to Riak as:
> 
> curl "http:// IP>:8098/search/query/?wt=json&df=_yz_rk&fl=xxx,yyy&q={%21join+from=+to=}zzz:vvv"
>  | jsonpp
> 
> The Riak listener will accept this query, but will return no documents. I've 
> verified each half of the query returns data when queried standalone using 
> the non-join notation.
> 
> Is anyone currently using Solr localparms notation successfully in Riak KV?
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search indexing json

[Zeeshan Lakhani]

Hey Colin,

Riak Search definitely indexes json documents. My guess is that your 
json-object keys are not indexed via the schema?

We have an example in the docs page (w/ the Java client as well): 
http://docs.basho.com/riak/latest/dev/using/search/#Indexing-Values 
<http://docs.basho.com/riak/latest/dev/using/search/#Indexing-Values> (& 
http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Creating-a-Custom-Schema
 
<http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Creating-a-Custom-Schema>
 may be helpful too).

Zeeshan Lakhani
programmer | 
software engineer at @basho

> On Feb 10, 2016, at 10:40 AM, Colin Walker  wrote:
> 
> Hello everyone,
> 
> Sorry for the basic question but I couldn't find the answer in the 
> documentation. Does Riak search currently index documents sent as JSON? It 
> doesn't look like it from the basic tests that I've run. I am using the new 
> Solr search on Riak 2.1.3.
> 
> When I send data to the bucket using a Map type in the java client everything 
> is indexed and I can search for my data as expected. However, when I send the 
> same Java map as JSON I cannot find a way to search within it. I am using the 
> default index schema.
> 
> Thanks everyone!
> 
> Colin Walker
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search indexing json

[Zeeshan Lakhani]

Good point Vitaly, also possible. 


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Feb 10, 2016, at 10:53 AM, Vitaly E <13vitam...@gmail.com> wrote:
> 
> Hi Colin,
> 
> The problem is that in your case JSON is interpreted as a String, as opposed 
> to a Map/POJO with fields.
> 
> Is there any reason you don't rely on the JSON serialization built into Riak 
> Java client?
> 
> Regards,
> Vitaly
> 
> 
> 
> On Wed, Feb 10, 2016 at 5:40 PM, Colin Walker  <mailto:co...@musefind.com>> wrote:
> Hello everyone,
> 
> Sorry for the basic question but I couldn't find the answer in the 
> documentation. Does Riak search currently index documents sent as JSON? It 
> doesn't look like it from the basic tests that I've run. I am using the new 
> Solr search on Riak 2.1.3.
> 
> When I send data to the bucket using a Map type in the java client everything 
> is indexed and I can search for my data as expected. However, when I send the 
> same Java map as JSON I cannot find a way to search within it. I am using the 
> default index schema.
> 
> Thanks everyone!
> 
> Colin Walker
> 
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak search 2 returning multiple stale results for single record

[Zeeshan Lakhani]

Hello Peter,

The cleanup for delete, as a call to SOLR, occurs after the reap happens in the 
backend, depending on delete_mode of course. Did you notice any Solr errors in 
the logs related to “failed to index” or something similar? Maybe the delete 
request never hit the solr_core/search_index, and, therefore, has stayed around 
as a sibling. The logs would help there. Have you set delete_mode to something 
specific (i.e. 
http://docs.basho.com/riak/latest/ops/advanced/deletion/#Configuring-Object-Deletion
 
<http://docs.basho.com/riak/latest/ops/advanced/deletion/#Configuring-Object-Deletion>)?
 

I can tell you that the soon-to-be-released 2.0.7 and 2.2 releases fixes this 
with more aggressive tactics to handle deletes, especially around datatypes (I 
see that you’re using maps), instead of waiting for the call after the backend 
reap/removal. 

Looking at previous threads, your W quorum value can also matter - 
http://riak-users.197444.n3.nabble.com/Deleted-keys-come-back-td4033536.html#a4033576
 
<http://riak-users.197444.n3.nabble.com/Deleted-keys-come-back-td4033536.html#a4033576>.
 

For you current situation, an updated re-PUT for those with siblings would 
resolve them, or via an internal solr cleanup call, or through a reindexing 
process. If you want to find me on IRC, we can discuss a few others tools we 
have that may help you get most of the way there until the new release, 
depending on the questions I asked in the first paragraph.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho

> On Feb 15, 2016, at 5:33 AM, Peter Roberts  wrote:
> 
> We’re currently evaluating whether Riak is suitable for our system and have 
> an issue with multiple/stale results being returned from Riak search. We’re 
> reliably seeing this occur when a record is  deleted and a new one created 
> under the same key shortly afterwards – which, based on the _yz_id, causes 
> siblings to be created. 
> 
> Accessing the record through the map datatype only gives us the expected 
> record.  We’ve considered de-duplicating the result by ID but this could 
> still result in search results where the query matches the old but not the 
> new data. The obsolete records in Riak search/Solr never get cleaned up. 
> 
> Is this a known issue? Are we missing something on inserting/querying the 
> data? Is it possible to tie the version of data in the Riak search result to 
> the version retrieved from Riak by key?
> 
> Below is an example of the Riak search query/result (using Node 
> basho-riak-client) and datatype data. The date.value is set to the 
> creation/update time. 
> 
>  var searchOptions = {
> indexName: 'minimals_index',
> q: 'name_register:foo',
> presort: ‘score'
> }
> 
> riakClient.search(searchOptions, function(err, result) {
>   console.log('Search result', err, result);
>   callback(err, result);
> }
> 
> Results:
> { numFound: 6,
>   maxScore: 0.35434481501579285,
>   docs: 
>[ { score: 0.35434482,
>_yz_rb: 'minimals',
>_yz_rt: 'maps',
>_yz_rk: 'test:foo',
>_yz_id: '1*maps*minimals*test:foo*13*2EB6xtIHCicwmARKDJ81tS',
>'date_map._type_register': '_date',
>'date_map._value_register': '2016-02-12T11:39:22.942Z',
>name_register: 'foo',
>owner_register: 'test' },
> …
>  { score: 0.35434482,
>_yz_rb: 'minimals',
>_yz_rt: 'maps',
>_yz_rk: 'test:foo',
>_yz_id: '1*maps*minimals*test:foo*13*4hNPnHzeJpTc7S6nEW8vNb',
>'date_map._type_register': '_date',
>'date_map._value_register': '2016-02-12T13:28:11.785Z',
>name_register: 'foo',
>owner_register: 'test' } ] 
> }
> 
> Riak get by ID:
> $ curl -XGET 
> "http://localhost:11098/types/maps/buckets/minimals/datatypes/test:foo";
> {"type":"map","value":{"date_map":{"_type_register":"_date","_value_register":"2016-02-12T13:28:11.785Z"},"name_register":"foo","owner_register":"test"},"context":"g2wBaAJtDMHSXwlPAZAxQmEDag==“}
> 
> 
> Thanks,
> Peter
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak search 2 returning multiple stale results for single record

[Zeeshan Lakhani]

Good to hear. 

The upcoming releases with the changes are soon on the agenda. As a user, you 
will not have to do anything special with the delete, and it will work across 
all delete_mode(s). The change is that riak_search (yokozuna) now handles 
tombstones on cleanup instead of waiting for the backend to reap. 


Zeeshan Lakhani
programmer | 
software engineer at @basho 

> On Feb 17, 2016, at 7:59 AM, Peter Roberts  wrote:
> 
> Hi Zeeshan,
> 
> Configuring the delete_mode to immediate (or a very low millisecond value) 
> does prevent this occurring for me, I hadn’t spotted this setting before.  
> This is a suitable solution for us at the moment.  There were no Solr errors 
> in the logs so I think it wasn’t being called by Riak. 
> 
> Are you able to provide me with any more details on how the delete will 
> change in the upcoming releases and when they might be available?
> 
> Thanks,
> Peter
> 
> From: Zeeshan Lakhani mailto:zlakh...@basho.com>>
> Date: Monday, 15 February 2016 20:27
> To: Peter Roberts mailto:peter.robe...@piksel.com>>
> Cc: "riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>" 
> mailto:riak-users@lists.basho.com>>
> Subject: Re: Riak search 2 returning multiple stale results for single record
> 
> Hello Peter,
> 
> The cleanup for delete, as a call to SOLR, occurs after the reap happens in 
> the backend, depending on delete_mode of course. Did you notice any Solr 
> errors in the logs related to “failed to index” or something similar? Maybe 
> the delete request never hit the solr_core/search_index, and, therefore, has 
> stayed around as a sibling. The logs would help there. Have you set 
> delete_mode to something specific (i.e. 
> http://docs.basho.com/riak/latest/ops/advanced/deletion/#Configuring-Object-Deletion
>  
> <http://docs.basho.com/riak/latest/ops/advanced/deletion/#Configuring-Object-Deletion>)?
>  
> 
> I can tell you that the soon-to-be-released 2.0.7 and 2.2 releases fixes this 
> with more aggressive tactics to handle deletes, especially around datatypes 
> (I see that you’re using maps), instead of waiting for the call after the 
> backend reap/removal. 
> 
> Looking at previous threads, your W quorum value can also matter - 
> http://riak-users.197444.n3.nabble.com/Deleted-keys-come-back-td4033536.html#a4033576
>  
> <http://riak-users.197444.n3.nabble.com/Deleted-keys-come-back-td4033536.html#a4033576>.
>  
> 
> For you current situation, an updated re-PUT for those with siblings would 
> resolve them, or via an internal solr cleanup call, or through a reindexing 
> process. If you want to find me on IRC, we can discuss a few others tools we 
> have that may help you get most of the way there until the new release, 
> depending on the questions I asked in the first paragraph.
> 
> Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho
> 
>> On Feb 15, 2016, at 5:33 AM, Peter Roberts > <mailto:peter.robe...@piksel.com>> wrote:
>> 
>> We’re currently evaluating whether Riak is suitable for our system and have 
>> an issue with multiple/stale results being returned from Riak search. We’re 
>> reliably seeing this occur when a record is  deleted and a new one created 
>> under the same key shortly afterwards – which, based on the _yz_id, causes 
>> siblings to be created. 
>> 
>> Accessing the record through the map datatype only gives us the expected 
>> record.  We’ve considered de-duplicating the result by ID but this could 
>> still result in search results where the query matches the old but not the 
>> new data. The obsolete records in Riak search/Solr never get cleaned up. 
>> 
>> Is this a known issue? Are we missing something on inserting/querying the 
>> data? Is it possible to tie the version of data in the Riak search result to 
>> the version retrieved from Riak by key?
>> 
>> Below is an example of the Riak search query/result (using Node 
>> basho-riak-client) and datatype data. The date.value is set to the 
>> creation/update time. 
>> 
>>  var searchOptions = {
>> indexName: 'minimals_index',
>> q: 'name_register:foo',
>> presort: ‘score'
>> }
>> 
>> riakClient.search(searchOptions, function(err, result) {
>>   console.log('Search result', err, result);
>>   callback(err, result);
>> }
>> 
>> Results:
>> { numFound: 6,
>>   maxScore: 0.35434481501579285,
>>   docs: 
>>[ { score: 0.35434482,
>>_yz_rb: 'minimals',
>>_yz_rt: 'maps',
>>_yz_rk: 'test

Re: Riak Search: cursormark support

[Zeeshan Lakhani]

Hello Guillaume,

We’re definitely working on it for a future release and there is still an open 
ticket regarding it: https://github.com/basho/yokozuna/issues/309 
<https://github.com/basho/yokozuna/issues/309>.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Jul 25, 2016, at 11:03 AM, Guillaume Boddaert 
>  wrote:
> 
> Hi,
> 
> I'm currently in great pain trying to walk through many records in my Riak 
> yokozuna system. I need to scan data through many pages in Solr for some 
> operation and I really feel the need for cursormark support.
> 
> I've read that Yokozuna closed a bug on that matter in may. What is the 
> status of this feature in Riak ?
> 
> Is there any deep paging solution you guys from Riak suggest ?
> 
> Guillaume
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Search reindexing during a handoff?

[Zeeshan Lakhani]

Search does reindex objects during a handoff.


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 17, 2015, at 11:55 AM, Miroslav Urbanek  
> wrote:
> 
> Hello,
> 
> I have a four node cluster running Riak 2.0.4 with n_val = 2 and
> Search turned on. I removed (riak-admin cluster leave) one node. The
> operation took a long time, but eventually completed. However, I could
> see the java process consuming fair amount of CPU time during the
> operation.
> 
> This leads me to a question: does Riak Search reindex transferred
> objects during a handoff?
> 
> Thanks,
> Miro
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak 2.0.4 Search AAE

[Zeeshan Lakhani]

Hey Steve, I’ve run AAE manually by running `riak attach` on a running node and 
calling 
https://github.com/basho/yokozuna/blob/develop/src/yz_entropy_mgr.erl#L142 
<https://github.com/basho/yokozuna/blob/develop/src/yz_entropy_mgr.erl#L142> 
(yz_entropy_mgr:init([])).

Nonetheless, to take a closer look, is there any data within search’s 
data-directory? You can find out by looking in ./data/yz_anti_entropy (on any 
of nodes). You can find more info in our docs, 
http://docs.basho.com/riak/latest/ops/advanced/configs/search/ 
<http://docs.basho.com/riak/latest/ops/advanced/configs/search/>, about 
search’s config settings.
 
Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 18, 2015, at 9:03 AM, Steve Garon  wrote:
> 
> Hi all, 
> 
> I seem to have trouble with aae on riak 2.0.4. Specifically the search aae. 
> When I run the command 'riak-admin aae-status' everything seems fine but if I 
> run the same command for search (riak-admin search aae-status) it seem like 
> the search entropy trees have never been built and never do. All I get on 
> every field is --. 
> 
> Any command I can run to force the trees to be built so I get consistent 
> search results?  
> 
> Steve
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak 2.0.4 Search AAE

[Zeeshan Lakhani]

Well, that means that it is storing AAE data. Are you have consistency issues 
between what’s in KV (the canonical source) and Search/Solr? 

You can remove the data on every node by using 
`riak_core_util:rpc_every_member_ann(yz_entropy_mgr, clear_trees, [], 
infinity)` as well and then try to initialize AAE again. As you can see in this 
part of Ryan’s vid, https://www.youtube.com/watch?v=ETJqu5SmwOc#t=2290 
<https://www.youtube.com/watch?v=ETJqu5SmwOc#t=2290> (and our constants in 
https://github.com/basho/yokozuna/blob/47214a66e07fdfd04bbaae6bdd18580f87693edf/include/yokozuna.hrl#L187
 
<https://github.com/basho/yokozuna/blob/47214a66e07fdfd04bbaae6bdd18580f87693edf/include/yokozuna.hrl#L187>),
 you can, via `riak attach`, update the  environment vars of entropy_expire, 
tick, and concurrency to speed up AAE in your testing of this bug.


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 18, 2015, at 1:21 PM, Steve Garon  wrote:
> 
> Hey Zeeshan, 
> 
> There is tons of data in yz_anti_entropy. Should I delete it before running 
> your command? 
> 
> 
> 
> 
> Steve
> 
> On 18 February 2015 at 09:49, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hey Steve, I’ve run AAE manually by running `riak attach` on a running node 
> and calling 
> https://github.com/basho/yokozuna/blob/develop/src/yz_entropy_mgr.erl#L142 
> <https://github.com/basho/yokozuna/blob/develop/src/yz_entropy_mgr.erl#L142> 
> (yz_entropy_mgr:init([])).
> 
> Nonetheless, to take a closer look, is there any data within search’s 
> data-directory? You can find out by looking in ./data/yz_anti_entropy (on any 
> of nodes). You can find more info in our docs, 
> http://docs.basho.com/riak/latest/ops/advanced/configs/search/ 
> <http://docs.basho.com/riak/latest/ops/advanced/configs/search/>, about 
> search’s config settings.
>  
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love |
> twitter => @zeeshanlakhani
> 
>> On Feb 18, 2015, at 9:03 AM, Steve Garon > <mailto:steve.ga...@gmail.com>> wrote:
>> 
>> Hi all, 
>> 
>> I seem to have trouble with aae on riak 2.0.4. Specifically the search aae. 
>> When I run the command 'riak-admin aae-status' everything seems fine but if 
>> I run the same command for search (riak-admin search aae-status) it seem 
>> like the search entropy trees have never been built and never do. All I get 
>> on every field is --. 
>> 
>> Any command I can run to force the trees to be built so I get consistent 
>> search results?  
>> 
>> Steve
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna unpredictable result count

[Zeeshan Lakhani]

Hello Cezary,

Firstly, are you able to retrieve your search result consistently when not 
using doing a mapreduce job? 

To better help out, can you send a gist of the mapreduce code you’re running? 
Thanks.


> On Feb 18, 2015, at 9:13 PM, Cezary Kosko  wrote:
> 
> Hi,
> 
> I've got a search index, and I'd like to run mapred job against that index. 
> The thing is, for a search query that should return exactly one result, I 
> sometimes (not always, yet not rarely) get none, i.e. the mapred job returns 
> an empty list instead of, say, a list containing a single object. Did this 
> only happen some time after uploading the data and then was consistently 
> giving the right results, I wouldn't object. However, it's kind of an 
> on-and-off situation - I get proper results, but then for a brief period of 
> time I don't and so on.
> 
> I've read on a solr doc page that specifying a timeAllowed parameter in the 
> query can give it longer to gather results and help, but that can't be 
> specified in a mapred definition, or can it?
> 
> Is there anything else I can look for?
> 
> The data I'm querying is of the CRDT map-of-sets type, should that be of any 
> relevance.
> 
> Kind regards,
> Cezary
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna unpredictable result count

[Zeeshan Lakhani]

Thanks Cezary. 

Have you deleted this object at some point in your runs? Please make sure AAE 
is running by checking search’s AAE status, `riak-admin search aae-status`, and 
that data exists in the correct directory, `./data/yz_anti_entropy` 
(http://docs.basho.com/riak/latest/ops/advanced/configs/search/ 
<http://docs.basho.com/riak/latest/ops/advanced/configs/search/>). You may just 
need to perform a read-repair by performing a fetch of the object itself first, 
before performing search queries again.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 19, 2015, at 10:35 AM, Cezary Kosko  wrote:
> 
> I have the exact same issue with regular http search queries, so I guess I'll 
> just describe that part.
> 
> I've got a bucket of maps-of-sets, 2 of them are entityId_set and 
> timestamps_set. Its search index is called 'job' and it's only this bucket 
> that's indexed.
> 
> When I run
> curl 
> "localhost:8098/search/query/job?wt=json&q=entityId_set:10%20AND%20timestamps_set:%5B1419721530%20TO%201419721539%5D"
> 
> (that's a query that is supposed to return exactly one result), the numFound 
> field is either 0 or 1, it seems that I get both kinds of result in 10 
> consecutive requests (and the timeAllowed parameter I wrote about, it doesn't 
> really help).
> 
> That's the way they're handled in the schema:
> 
> stored="false" multiValued="true" />
> stored="false" multiValued="true" />
> 
> 
> Kind regards,
> Cezary
> 
> 2015-02-19 16:04 GMT+01:00 Zeeshan Lakhani  <mailto:zlakh...@basho.com>>:
> Hello Cezary,
> 
> Firstly, are you able to retrieve your search result consistently when not 
> using doing a mapreduce job? 
> 
> To better help out, can you send a gist of the mapreduce code you’re running? 
> Thanks.
> 
> 
>> On Feb 18, 2015, at 9:13 PM, Cezary Kosko > <mailto:koskoceza...@gmail.com>> wrote:
>> 
>> Hi,
>> 
>> I've got a search index, and I'd like to run mapred job against that index. 
>> The thing is, for a search query that should return exactly one result, I 
>> sometimes (not always, yet not rarely) get none, i.e. the mapred job returns 
>> an empty list instead of, say, a list containing a single object. Did this 
>> only happen some time after uploading the data and then was consistently 
>> giving the right results, I wouldn't object. However, it's kind of an 
>> on-and-off situation - I get proper results, but then for a brief period of 
>> time I don't and so on.
>> 
>> I've read on a solr doc page that specifying a timeAllowed parameter in the 
>> query can give it longer to gather results and help, but that can't be 
>> specified in a mapred definition, or can it?
>> 
>> Is there anything else I can look for?
>> 
>> The data I'm querying is of the CRDT map-of-sets type, should that be of any 
>> relevance.
>> 
>> Kind regards,
>> Cezary
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna unpredictable result count

[Zeeshan Lakhani]

So, AAE is running.

Again, did you delete the single object at some point? Trying to see if this is 
related to you hitting a tombstone on queries. Also, when you added the object, 
did you add it and later leave (drop) a node from your cluster? 

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 19, 2015, at 11:53 AM, Cezary Kosko  wrote:
> 
> By './data/yz_anti_entropy' do you mean '/var/lib/riak/yz_anti_entropy' by 
> default or './data/yz_anti_entropy' inside each index's directory? If the 
> former - it's there, the latter - not. riak-admin search aae-status says 
> there's been some AAE activity in the past few hours.
> 
> Also I called yz_entropy_mgr:init([]) inside an attached erlang shell and 
> curl-ed the object, it's still the same.
> 
> Kind regards,
> Cezary
> 
> 2015-02-19 17:27 GMT+01:00 Zeeshan Lakhani  <mailto:zlakh...@basho.com>>:
> Thanks Cezary. 
> 
> Have you deleted this object at some point in your runs? Please make sure AAE 
> is running by checking search’s AAE status, `riak-admin search aae-status`, 
> and that data exists in the correct directory, `./data/yz_anti_entropy` 
> (http://docs.basho.com/riak/latest/ops/advanced/configs/search/ 
> <http://docs.basho.com/riak/latest/ops/advanced/configs/search/>). You may 
> just need to perform a read-repair by performing a fetch of the object itself 
> first, before performing search queries again.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love |
> twitter => @zeeshanlakhani
> 
>> On Feb 19, 2015, at 10:35 AM, Cezary Kosko > <mailto:koskoceza...@gmail.com>> wrote:
>> 
>> I have the exact same issue with regular http search queries, so I guess 
>> I'll just describe that part.
>> 
>> I've got a bucket of maps-of-sets, 2 of them are entityId_set and 
>> timestamps_set. Its search index is called 'job' and it's only this bucket 
>> that's indexed.
>> 
>> When I run
>> curl 
>> "localhost:8098/search/query/job?wt=json&q=entityId_set:10%20AND%20timestamps_set:%5B1419721530%20TO%201419721539%5D"
>> 
>> (that's a query that is supposed to return exactly one result), the numFound 
>> field is either 0 or 1, it seems that I get both kinds of result in 10 
>> consecutive requests (and the timeAllowed parameter I wrote about, it 
>> doesn't really help).
>> 
>> That's the way they're handled in the schema:
>> 
>>> stored="false" multiValued="true" />
>>> stored="false" multiValued="true" />
>> 
>> 
>> Kind regards,
>> Cezary
>> 
>> 2015-02-19 16:04 GMT+01:00 Zeeshan Lakhani > <mailto:zlakh...@basho.com>>:
>> Hello Cezary,
>> 
>> Firstly, are you able to retrieve your search result consistently when not 
>> using doing a mapreduce job? 
>> 
>> To better help out, can you send a gist of the mapreduce code you’re 
>> running? Thanks.
>> 
>> 
>>> On Feb 18, 2015, at 9:13 PM, Cezary Kosko >> <mailto:koskoceza...@gmail.com>> wrote:
>>> 
>>> Hi,
>>> 
>>> I've got a search index, and I'd like to run mapred job against that index. 
>>> The thing is, for a search query that should return exactly one result, I 
>>> sometimes (not always, yet not rarely) get none, i.e. the mapred job 
>>> returns an empty list instead of, say, a list containing a single object. 
>>> Did this only happen some time after uploading the data and then was 
>>> consistently giving the right results, I wouldn't object. However, it's 
>>> kind of an on-and-off situation - I get proper results, but then for a 
>>> brief period of time I don't and so on.
>>> 
>>> I've read on a solr doc page that specifying a timeAllowed parameter in the 
>>> query can give it longer to gather results and help, but that can't be 
>>> specified in a mapred definition, or can it?
>>> 
>>> Is there anything else I can look for?
>>> 
>>> The data I'm querying is of the CRDT map-of-sets type, should that be of 
>>> any relevance.
>>> 
>>> Kind regards,
>>> Cezary
>>> ___
>>> riak-users mailing list
>>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
>> 
>> 
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna unpredictable result count

[Zeeshan Lakhani]

Ok. I’m guessing that you’re running a default cluster on one node. 

Did you attempt the read-repair I suggested earlier in the thread?

Also, you should try and get each node’s value for the bucket/key and see if 
they’re consistent. You can use 
https://github.com/basho/riak_kv/blob/d17409fdb934c076cca9e65a02818ad860fe310b/src/riak_kv_vnode.erl#L248
 
<https://github.com/basho/riak_kv/blob/d17409fdb934c076cca9e65a02818ad860fe310b/src/riak_kv_vnode.erl#L248>
 via `riak attach` on each vnode to check each value. You can get active 
preflist for that bucket/key to help you get the index via 
https://github.com/binarytemple/riak_utils/blob/d59bd38e34b697b2b48b853b39ab1a51ab04111a/sparrows.preflist.getter.markdown
 
<https://github.com/binarytemple/riak_utils/blob/d59bd38e34b697b2b48b853b39ab1a51ab04111a/sparrows.preflist.getter.markdown>.
 That Preflist fun would take in an input like (<<“maps”>>, <<“albums”>>, 
<<“REM”>>).

Thanks.


> On Feb 19, 2015, at 12:09 PM, Cezary Kosko  wrote:
> 
> I can't recall right now, but it's safe to assume I did delete it.
> 
> As for nodes, I have added one, nothing more.
> 
> Kind regards,
> Cezary
> 
> 2015-02-19 18:01 GMT+01:00 Zeeshan Lakhani  <mailto:zlakh...@basho.com>>:
> So, AAE is running.
> 
> Again, did you delete the single object at some point? Trying to see if this 
> is related to you hitting a tombstone on queries. Also, when you added the 
> object, did you add it and later leave (drop) a node from your cluster? 
> 
> Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love |
> twitter => @zeeshanlakhani
> 
>> On Feb 19, 2015, at 11:53 AM, Cezary Kosko > <mailto:koskoceza...@gmail.com>> wrote:
>> 
>> By './data/yz_anti_entropy' do you mean '/var/lib/riak/yz_anti_entropy' by 
>> default or './data/yz_anti_entropy' inside each index's directory? If the 
>> former - it's there, the latter - not. riak-admin search aae-status says 
>> there's been some AAE activity in the past few hours.
>> 
>> Also I called yz_entropy_mgr:init([]) inside an attached erlang shell and 
>> curl-ed the object, it's still the same.
>> 
>> Kind regards,
>> Cezary
>> 
>> 2015-02-19 17:27 GMT+01:00 Zeeshan Lakhani > <mailto:zlakh...@basho.com>>:
>> Thanks Cezary. 
>> 
>> Have you deleted this object at some point in your runs? Please make sure 
>> AAE is running by checking search’s AAE status, `riak-admin search 
>> aae-status`, and that data exists in the correct directory, 
>> `./data/yz_anti_entropy` 
>> (http://docs.basho.com/riak/latest/ops/advanced/configs/search/ 
>> <http://docs.basho.com/riak/latest/ops/advanced/configs/search/>). You may 
>> just need to perform a read-repair by performing a fetch of the object 
>> itself first, before performing search queries again.
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love |
>> twitter => @zeeshanlakhani
>> 
>>> On Feb 19, 2015, at 10:35 AM, Cezary Kosko >> <mailto:koskoceza...@gmail.com>> wrote:
>>> 
>>> I have the exact same issue with regular http search queries, so I guess 
>>> I'll just describe that part.
>>> 
>>> I've got a bucket of maps-of-sets, 2 of them are entityId_set and 
>>> timestamps_set. Its search index is called 'job' and it's only this bucket 
>>> that's indexed.
>>> 
>>> When I run
>>> curl 
>>> "localhost:8098/search/query/job?wt=json&q=entityId_set:100000%20AND%20timestamps_set:%5B1419721530%20TO%201419721539%5D"
>>> 
>>> (that's a query that is supposed to return exactly one result), the 
>>> numFound field is either 0 or 1, it seems that I get both kinds of result 
>>> in 10 consecutive requests (and the timeAllowed parameter I wrote about, it 
>>> doesn't really help).
>>> 
>>> That's the way they're handled in the schema:
>>> 
>>>>> stored="false" multiValued="true" />
>>>>> stored="false" multiValued="true" />
>>> 
>>> 
>>> Kind regards,
>>> Cezary
>>> 
>>> 2015-02-19 16:04 GMT+01:00 Zeeshan Lakhani >> <mailto:zlakh...@basho.com>>:
>>> Hello Cezary,
>>> 
>>> Firstly, are you able to retrieve your search result consistently when

Re: YZ search schema question

[Zeeshan Lakhani]

Hello Michael,

You should be able to then set a field to be multiValued in your schema that 
would capture your use-case. You can find more info here -> 
http://docs.basho.com/riak/latest/dev/advanced/search-schema/ 
<http://docs.basho.com/riak/latest/dev/advanced/search-schema/> and 
http://stackoverflow.com/a/5800830 <http://stackoverflow.com/a/5800830>. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 24, 2015, at 9:35 AM, Michael Martin  wrote:
> 
> Hi all,
> 
> I have a need to index on two separate fields in a json document - rather, I 
> need to concatenate the two and index on that.
> 
> Short of duplicating both into a single new item in my json document, how 
> would I go about doing that?
> 
> Example:
> 
> Given this JSON:
> 
> { "parent": "/path/to/parent",
>   "self": "myname"
> }
> 
> How would I build a schema that would search on "/path/to/parent/myname" 
> without doing something like:
> 
> { "parent": "/path/to/parent",
>   "self": "myname",
>   "fullpath": "/path/to/parent/myname"
> }
> 
> Thanks,
> Michael Martin
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: YZ search schema question

[Zeeshan Lakhani]

Just for the query itself, then yep :). 


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 24, 2015, at 10:11 AM, Michael Martin  wrote:
> 
> Hi Zeeshan,
> 
> Actually, I was trying to make this harder than it really is. The easiest way 
> to do this appears to be simply ANDing the two values in the query parms. 
> 
> Thanks,
> Michael Martin
> 
> 
> 
> On Tue, Feb 24, 2015 at 9:59 AM, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hello Michael,
> 
> You should be able to then set a field to be multiValued in your schema that 
> would capture your use-case. You can find more info here -> 
> http://docs.basho.com/riak/latest/dev/advanced/search-schema/ 
> <http://docs.basho.com/riak/latest/dev/advanced/search-schema/> and 
> http://stackoverflow.com/a/5800830 <http://stackoverflow.com/a/5800830>. 
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love |
> twitter => @zeeshanlakhani
> 
>> On Feb 24, 2015, at 9:35 AM, Michael Martin > <mailto:mmartin4...@gmail.com>> wrote:
>> 
>> Hi all,
>> 
>> I have a need to index on two separate fields in a json document - rather, I 
>> need to concatenate the two and index on that.
>> 
>> Short of duplicating both into a single new item in my json document, how 
>> would I go about doing that?
>> 
>> Example:
>> 
>> Given this JSON:
>> 
>> { "parent": "/path/to/parent",
>>   "self": "myname"
>> }
>> 
>> How would I build a schema that would search on "/path/to/parent/myname" 
>> without doing something like:
>> 
>> { "parent": "/path/to/parent",
>>   "self": "myname",
>>   "fullpath": "/path/to/parent/myname"
>> }
>> 
>> Thanks,
>> Michael Martin
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: 2.0.5 compiled from source SOLR failed to start

[Zeeshan Lakhani]

Hello Alexander,

Can you tell me which operating system you’re on, so we can test this out 
specifically? I wasn’t able to reproduce on my local setup.

Also, can you list (ls) out that `../lib/yokozuna-2.0.0-34-g122659d/priv` 
directory (mine is located at `../rel/riak/lib/yokozuna-2.0.0-34-g122659d/priv` 
for example)?

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Feb 27, 2015, at 5:46 AM, Alexander Popov  wrote:
> 
> 2015-02-27 10:38:27.960 [info] <0.563.0>@yz_solr_proc:ensure_data_dir:219 No 
> solr config found, creating a new one
> 2015-02-27 10:38:27.961 [info] <0.563.0>@yz_solr_proc:init:96 Starting solr: 
> "/usr/bin/java" 
> ["-Djava.awt.headless=true","-Djetty.home=/var/riak-dist/riak1/bin/../lib/yokozuna-2.0.0-34-g122659d/priv/solr","
> -Djetty.port=18093","-Dsolr.solr.home=/var/lib/riak1/yz","-DhostContext=/internal_solr","-cp","/var/riak-dist/riak1/bin/../lib/yokozuna-2.0.0-34-g122659d/priv/solr/start.jar","-Dlog4j.configuration=file:///
> var/riak-dist/riak1/etc/solr-log4j.properties","-Dyz.lib.dir=/var/riak-dist/riak1/bin/../lib/yokozuna-2.0.0-34-g122659d/priv/java_lib","-d64","-Xms1g","-Xmx3g","-XX:+UseStringCache","-XX:+UseCompressedOops"
> ,"-Dcom.sun.management.jmxremote.port=18985","-Dcom.sun.management.jmxremote.authenticate=false","-Dcom.sun.management.jmxremote.ssl=false","org.eclipse.jetty.start.Main"]
> 2015-02-27 10:38:28.004 [info] <0.7.0> Application yokozuna started on node 
> 'riak1@10.0.0.133 <mailto:riak1@10.0.0.133>'
> 2015-02-27 10:38:28.007 [info] <0.7.0> Application cluster_info started on 
> node 'riak1@10.0.0.133 <mailto:riak1@10.0.0.133>'
> 2015-02-27 10:38:28.033 [info] 
> <0.198.0>@riak_core_capability:process_capability_changes:555 New capability: 
> {riak_control,member_info_version} = v1
> 2015-02-27 10:38:28.035 [info] <0.7.0> Application riak_control started on 
> node 'riak1@10.0.0.133 <mailto:riak1@10.0.0.133>'
> 2015-02-27 10:38:28.035 [info] <0.7.0> Application erlydtl started on node 
> 'riak1@10.0.0.133 <mailto:riak1@10.0.0.133>'
> 2015-02-27 10:38:28.043 [info] <0.7.0> Application riak_auth_mods started on 
> node 'riak1@10.0.0.133 <mailto:riak1@10.0.0.133>'
> 2015-02-27 10:38:28.307 [info] <0.563.0>@yz_solr_proc:handle_info:135 solr 
> stdout/err: java.io.FileNotFoundException: No XML configuration files 
> specified in start.config or command line.
> 
> 2015-02-27 10:38:28.308 [info] <0.563.0>@yz_solr_proc:handle_info:135 solr 
> stdout/err:  at org.eclipse.jetty.start.Main.start(Main.java:502)
> at org.eclipse.jetty.start.Main.main(Main.java:96)
>
> Usage: java -jar start.jar [options] [properties] [configs]
>java -jar start.jar --help  # for more information
> 2015-02-27 10:38:28.625 [error] <0.563.0> gen_server yz_solr_proc terminated 
> with reason: {"solr OS process exited",251}
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak 2.0 Search Issues

[Zeeshan Lakhani]

Hey Steve,

Sorry to see you’re having new issues.

We’ll have the fix for “space in the key” out soon; it’s currently under 
review. And, I know that this issue is unrelated.

I have a few different routes/thoughts for you, but are you seeing anything 
related to `yz_events,handle_info` in your crash logs? Also, can you 
gist/pastebin me your solr logs? I’d like to seem if it correlates with 
something we’re currently looking at. 

Thanks. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 4, 2015, at 10:39 AM, Steve Garon  wrote:
> 
> Hey all, 
> 
> We were having the "space in the key" bug in our cluster so we went through 
> the whole dataset, backing it up to json file and removing the spaces in the 
> keys. Then we trashed our whole cluster and restart from scratch reimporting 
> the whole data. Everything worked like a charm for two weeks but this 
> weekend, not sure what happened but AAE died again. 
> 
> I have two issues now: 
> 1. AAE is trying to recreate an index that already exists and crashes with an 
> "Already exists" error ... I get this in my error log every 15 seconds
> 2. AAE crashes while iterating through entropy data with a request timeout 
> error every hour followed by tons of failed to index objects with request 
> timeout as well. Stack trace looks like this:
> 
> [error] emulator Error in process <025931.7166> on node 'riak@IP' with exit 
> value: {function_clause,[{yz_entropy,iterate_entropy_data,[<<11 
> bytes>>,[{continuation,<<159 
> bytes>>},{limit,100},{partition,12}],#Fun  (TRUNCATED)
> 
> [error] <0.1371.0>@yz_kv:index:215 failed to index object 
> {{<<"TYPE">>,<<"BUCKET">>},<<"KEY">>} with error {"Failed to index 
> docs",{error,req_timeout}} because ... (REPEATED MULTIPLE TIME FOR DIFFERENT 
> KEYS)
> 
> I tried clearing the yz anti-entropy tree and reinitialising the 
> yz_entropy_mgr with no luck. Anything I can do to fix this? 
> 
> Oh FYI, I cannot insert data with spaces in the key anymore because we are 
> using a wrapper on top of riak that will prevent us to do so therefore my 
> issues are not related to this for sure.
> 
> These are some config changes that may be good to know for more context.
> We added this to ibrowse.conf:
> {dest, "localhost", 8093, 100, 1000, []}.
> 
> Jetty is set with minthread 80, acceptors 80, and is using the NIO connector.
> 
> All our solr buckets have filterCache disabled with softcommits set to 10s 
> instead of 1.
> 
> Our riak.conf has background_manager turned on with AAE and handoff using it.
> 
> Thanks,
> 
> Steve
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime

[Zeeshan Lakhani]

Hello Santi, 

Have you deleted an object in that bucket/index at some point? 

Please make sure AAE is running by checking search’s AAE status, `riak-admin 
search aae-status`, and that data exists in the correct directory, 
`./data/yz_anti_entropy` 
(http://docs.basho.com/riak/latest/ops/advanced/configs/search/). 

You may just need to perform a read-repair by performing a fetch of the object 
itself first, before performing search queries again.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 4, 2015, at 9:06 PM, Santi Kumar  wrote:
> 
> But in our case we didnt have spaces in keys. All our keys are UUID's so I 
> wouldn't suspect that. AAE I haven't verified .
> 
> On Thu, Mar 5, 2015 at 7:16 AM, John O'Brien  <mailto:boar...@gmail.com>> wrote:
> I'd lean towards AAE issues on Yokozuna... Same problems we were having with 
> our 'spaces-in-keys' issue... Once we cleaned those up, things were great 
> again.
> 
> On Wed, Mar 4, 2015 at 8:36 PM, Santi Kumar  <mailto:sa...@veradocs.com>> wrote:
> Raik 2.0.0
> 
> On Mar 5, 2015 12:31 AM, "Christopher Meiklejohn"  <mailto:cmeiklej...@basho.com>> wrote:
> 
> > On Mar 4, 2015, at 1:15 PM, Santi Kumar  > <mailto:sa...@veradocs.com>> wrote:
> >
> > Hi,
> > We are running into a strange issue with Riak Search. Our setup is with 3 
> > nodes of Riak (with search enabled) in 3 different  ec2 instances behind 
> > ELB. App server talks to the cluster through ELB. We are querying for list 
> > of objects through *:* query instead of list bucket keys and every time the 
> > result is different. We have only 4 objects in that bucket / index, only 
> > few times it's giving all 4 but ofther it returns 2 or 3.
> >
> > This wasn't happening when we have one instance of Riak. Any insight?
> 
> Hi Santi,
> 
> Can you provide information regarding what versions of Riak you are running?
> 
> - Chris
> 
> Christopher Meiklejohn
> Senior Software Engineer
> Basho Technologies, Inc.
> cmeiklej...@basho.com <mailto:cmeiklej...@basho.com>
> ___
> riak-users mailing list
> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak 2.0 Search Issues

[Zeeshan Lakhani]

Hey Steve,

We’re currently tracking this issue, 
https://github.com/basho/yokozuna/issues/442 
<https://github.com/basho/yokozuna/issues/442>, and are working on testing out 
a patch internally. I will update you as soon as we get clarity there.

As a possible workaround, I would attempt to delete the `core.properties` file 
located in the search root_directory/index of each node 
(./data/yz/<>). This file should then be recreated on the next 
attempt to creating the core on the Solr side, 
https://github.com/basho/yokozuna/blob/92ca14cc35b46c8e7ac86cad6d92547e68e8d917/src/yz_index.erl#L171
 
<https://github.com/basho/yokozuna/blob/92ca14cc35b46c8e7ac86cad6d92547e68e8d917/src/yz_index.erl#L171>.

If that doesn’t work, you can then delete or `mv` that search-root/index 
directory out of the way, for which it will then get recreated and AAE will 
sync the data.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 5, 2015, at 9:02 AM, Steve Garon  wrote:
> 
> In solr.log, the kind of exceptions that I'm getting right now is the 
> following:
> 
> 1. IO Error while trying to get the size of the 
> Directory:java.io.FileNotFoundException: SOME_RAMDOM_FILE (.doc, .pos, .fnm, 
> .si, .nv, .gen extensions)
> 2. SolrException.java:120 null:org.apache.sorl.common.SolrException: Core 
> with name 'BUCKET NAME' already exists. 
> 3. SolrException.java:120 null:org.eclipse.jetty.io.EofException
> 4. Server refused connection 
> 5. IOException occured when talking to server
> 
> 
> 
> 
> Steve
> 
> On 5 March 2015 at 08:50, Steve Garon  <mailto:steve.ga...@gmail.com>> wrote:
> Yes I do have yz_events,handle_info in crash log. Tons of them actually and 
> they have a big ass strack trace attached to each of them.
> 
> It would be hard for me to provide you with logs. If you have specific 
> questions you want answers to I'd be happy to help though.
> 
> 
> Steve
> 
> On 4 March 2015 at 14:20, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hey Steve,
> 
> Sorry to see you’re having new issues.
> 
> We’ll have the fix for “space in the key” out soon; it’s currently under 
> review. And, I know that this issue is unrelated.
> 
> I have a few different routes/thoughts for you, but are you seeing anything 
> related to `yz_events,handle_info` in your crash logs? Also, can you 
> gist/pastebin me your solr logs? I’d like to seem if it correlates with 
> something we’re currently looking at. 
> 
> Thanks. 
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 4, 2015, at 10:39 AM, Steve Garon > <mailto:steve.ga...@gmail.com>> wrote:
>> 
>> Hey all, 
>> 
>> We were having the "space in the key" bug in our cluster so we went through 
>> the whole dataset, backing it up to json file and removing the spaces in the 
>> keys. Then we trashed our whole cluster and restart from scratch reimporting 
>> the whole data. Everything worked like a charm for two weeks but this 
>> weekend, not sure what happened but AAE died again. 
>> 
>> I have two issues now: 
>> 1. AAE is trying to recreate an index that already exists and crashes with 
>> an "Already exists" error ... I get this in my error log every 15 seconds
>> 2. AAE crashes while iterating through entropy data with a request timeout 
>> error every hour followed by tons of failed to index objects with request 
>> timeout as well. Stack trace looks like this:
>> 
>> [error] emulator Error in process <025931.7166> on node 'riak@IP' with exit 
>> value: {function_clause,[{yz_entropy,iterate_entropy_data,[<<11 
>> bytes>>,[{continuation,<<159 
>> bytes>>},{limit,100},{partition,12}],#Fun>  (TRUNCATED)
>> 
>> [error] <0.1371.0>@yz_kv:index:215 failed to index object 
>> {{<<"TYPE">>,<<"BUCKET">>},<<"KEY">>} with error {"Failed to index 
>> docs",{error,req_timeout}} because ... (REPEATED MULTIPLE TIME FOR DIFFERENT 
>> KEYS)
>> 
>> I tried clearing the yz anti-entropy tree and reinitialising the 
>> yz_entropy_mgr with no luck. Anything I can do to fix this? 
>> 
>> Oh FYI, I cannot insert data with spaces in the key anymore because we are 
>> using a wrapper on top of riak that will prevent us to do so therefore my 
>> issues are not related to thi

Fwd: Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime

[Zeeshan Lakhani]

Hello Baskar, Santi.

I’d definitely like to know what your AAE status is like, as per my last email. 
The fix here may just be clearing the AAE trees and rebuilding them.

Yes, index creation is currently async and you have to wait for it to be 
created before sending data. You can poll to check for the index, similar to 
this function used in the search java-client tests, 
https://github.com/basho/riak-java-client/blob/89778ae5a2274ce8693709bcc4c5ea3558d72971/src/test/java/com/basho/riak/client/core/operations/itest/ITestBase.java#L264
 
<https://github.com/basho/riak-java-client/blob/89778ae5a2274ce8693709bcc4c5ea3558d72971/src/test/java/com/basho/riak/client/core/operations/itest/ITestBase.java#L264>.
 We do something similarly in our erlang tests for search. 

I’m not aware of a better way right now. Currently, I’m actually working on 
making index creation synchronous before returning a response (within a 
timeout), which could solve your issues.

Thanks.


> Begin forwarded message:
> 
> Date: March 5, 2015 at 11:21:59 AM EST
> Subject: Re: Query on Riak Search in a cluster of 3 nodes behind ELB is 
> giving different result everytime
> From: Baskar Srinivasan 
> To: Santi Kumar 
> Cc: Zeeshan Lakhani 
> 
> Hello Zeeshan,
> 
> I work with Santi on the same server backend component. Our primary issue 
> seems to be that Index creation and Bucket association with the indices is 
> being done via load balancer. After we do that, we move onto actually 
> creating data. When data is created is getting into Riak fine. However not 
> all data make it to the indices. 
> 
> It appears that depending when a node gets the Riak data persist call it may 
> not have completed index creation and bucket to index association for that 
> node which in turns means that particular data doesn't get indexed anywhere.
> 
> Is it possible to ensure that the call to create indices and associate 
> buckets with indices guarantees that this action has been completed on all 
> nodes of a cluster?
> 
> The API we are using are as follows:
> 
> 1. Index creation
> YokozunaSchema yschema = getSchema();
> 
> YokozunaIndex vdIndex = new YokozunaIndex(indexName,yschema.getName());
> 
>   
> StoreIndex storeIndex =new StoreIndex.Builder(vdIndex).build();
> 
> RiakFuture storeIndexFuture = 
> client.executeAsync(storeIndex);
> 
> storeIndexFuture.await();
> 
>  if(!storeIndexFuture.isSuccess()){
> 
> throw new Exception("");
> 
> 
>   }
> 
> 
> 
> 2. Bucket association
> 
> 
> 
> StoreBucketProperties storeBucketProps = new 
> StoreBucketProperties.Builder(bucket)
> 
> .withSearchIndex(indexName)
> 
> .build();
> 
>   
> RiakFuture storeBucketPropsFuture = 
> client.executeAsync(storeBucketProps);
> 
> try{
> 
> storeBucketPropsFuture.await();
> 
> if(storeBucketPropsFuture.isDone() && 
> storeBucketPropsFuture.isSuccess()){
> 
> logger.info(" Associated index "+indexName+" With bucket 
> "+bucketName);
> 
> }else{
> 
> logger.warn(" Unable to associate bucket "+bucketName);
> 
> }
> 
> }catch(Exception e){
> 
> logger.warn("Issues with ");
> 
> 
>   }
> 
> 
> 
> Regards,
> 
> Baskar
> 
> 
> On Thu, Mar 5, 2015 at 8:06 AM, Santi Kumar  <mailto:sa...@veradocs.com>> wrote:
> This issue is happening for *:* query only. It doesn't have lot of objects so 
> we are doing *:* query. Ever time it fetches different number when I switched 
> to list all keys and do a multi get its returning all objects
> 
> Another thing what we do is we create indexes on the fly and wait for index 
> creation by checking Listindex operation. Once it found the index then we are 
> associating the Bucket with index. After that we write some entries into that 
> bucket.
> 
> I fetched the key and tried searching again but it's the same again. So read 
> repair would have happened so that wouldn't be the issue I can check AAE
> 
> Can you suggest any better way for these index creations and bucket 
> association? I suspect that might be the reason
> Is there any way to check what's the data each shard has? I tried solr query 
> with shards param 
> 
> Thanks 
> Santi
> 
> On Mar 5, 2015 8:05 PM, "Zeeshan Lakhani"  <mailto:zlakh...@basho.com>> wrote:
> Hello Santi, 
> 
> Have you deleted an object in that bucket/index at some point? 
> 
> Please make sure AAE is running by checking search’s AAE status, `riak-admin 

Re: Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime

[Zeeshan Lakhani]

Hello Santi, Baskar. Please keep your messages on the user group mailing list, 
btw. Thanks.

Here’s an example of our testing harness’s wait_for_index function, 
https://github.com/basho/yokozuna/blob/develop/riak_test/yz_rt.erl#L420 
. We 
check for the index on each of the nodes, which is an approach you can take. 

And, as I mentioned, I’m currently working on making Index creation synchronous 
to make this easier.

If your logs are not pointing to any errors and being that your bucket, index 
contains so few objects, I’d delete or mv the search-root/index directory 
(./data/yz/<>) and let AAE resync the data, which should then give 
you consistent results.

Thanks.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak 2.0 Search Issues

[Zeeshan Lakhani]

Hey Steve,

Your data will remain in Riak, but Search will then use AAE to rebuild the data 
on the Solr side. This is only to fix Solr’s core data issues. If you’re 
worried about an interruption on the SOLR side, as there would be one to 
reindex the data, you could then wait until things are sured-up in our next 
release/patch-release to help solve these issues.

As I mentioned, we’re working on a fix for this with an internal patch that 
we’re testing at this moment.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 5, 2015, at 12:13 PM, Steve Garon  wrote:
> 
> Does deleting core.properties or moving search-root/index deletes any of my 
> index data? Will that cause any interruption to SOLR? Cause I'm having 
> problems on a production cluster and I really don't want to have to backup 
> 300 millions keys again or cause any interruptions to the system ...
> 
> 
> Steve
> 
> On 5 March 2015 at 10:11, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hey Steve,
> 
> We’re currently tracking this issue, 
> https://github.com/basho/yokozuna/issues/442 
> <https://github.com/basho/yokozuna/issues/442>, and are working on testing 
> out a patch internally. I will update you as soon as we get clarity there.
> 
> As a possible workaround, I would attempt to delete the `core.properties` 
> file located in the search root_directory/index of each node 
> (./data/yz/<>). This file should then be recreated on the next 
> attempt to creating the core on the Solr side, 
> https://github.com/basho/yokozuna/blob/92ca14cc35b46c8e7ac86cad6d92547e68e8d917/src/yz_index.erl#L171
>  
> <https://github.com/basho/yokozuna/blob/92ca14cc35b46c8e7ac86cad6d92547e68e8d917/src/yz_index.erl#L171>.
> 
> If that doesn’t work, you can then delete or `mv` that search-root/index 
> directory out of the way, for which it will then get recreated and AAE will 
> sync the data.
> 
> Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 5, 2015, at 9:02 AM, Steve Garon > <mailto:steve.ga...@gmail.com>> wrote:
>> 
>> In solr.log, the kind of exceptions that I'm getting right now is the 
>> following:
>> 
>> 1. IO Error while trying to get the size of the 
>> Directory:java.io.FileNotFoundException: SOME_RAMDOM_FILE (.doc, .pos, .fnm, 
>> .si, .nv, .gen extensions)
>> 2. SolrException.java:120 null:org.apache.sorl.common.SolrException: Core 
>> with name 'BUCKET NAME' already exists. 
>> 3. SolrException.java:120 null:org.eclipse.jetty.io.EofException
>> 4. Server refused connection 
>> 5. IOException occured when talking to server
>> 
>> 
>> 
>> 
>> Steve
>> 
>> On 5 March 2015 at 08:50, Steve Garon > <mailto:steve.ga...@gmail.com>> wrote:
>> Yes I do have yz_events,handle_info in crash log. Tons of them actually and 
>> they have a big ass strack trace attached to each of them.
>> 
>> It would be hard for me to provide you with logs. If you have specific 
>> questions you want answers to I'd be happy to help though.
>> 
>> 
>> Steve
>> 
>> On 4 March 2015 at 14:20, Zeeshan Lakhani > <mailto:zlakh...@basho.com>> wrote:
>> Hey Steve,
>> 
>> Sorry to see you’re having new issues.
>> 
>> We’ll have the fix for “space in the key” out soon; it’s currently under 
>> review. And, I know that this issue is unrelated.
>> 
>> I have a few different routes/thoughts for you, but are you seeing anything 
>> related to `yz_events,handle_info` in your crash logs? Also, can you 
>> gist/pastebin me your solr logs? I’d like to seem if it correlates with 
>> something we’re currently looking at. 
>> 
>> Thanks. 
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love | paperswelove.org 
>> <http://paperswelove.org/>
>> twitter => @zeeshanlakhani
>> 
>>> On Mar 4, 2015, at 10:39 AM, Steve Garon >> <mailto:steve.ga...@gmail.com>> wrote:
>>> 
>>> Hey all, 
>>> 
>>> We were having the "space in the key" bug in our cluster so we went through 
>>> the whole dataset, backing it up to json file and removing the spaces in 
>>> the keys. Then we trashed our whole cluster and restart from scratch 
>>> r

Re: Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime

[Zeeshan Lakhani]

Hello Baskar, Santi,

2-15 minutes is a long while, and we’ve not seen index creation/propagation be 
so slow. I’d definitely take a closer look at how you’re creating these indexes 
dynamically on the fly, as index creation is typically a more straightforward 
admin task.

We’ve added defaults to solrconfig.xml to handle most typical use-cases. You 
can read more about solrconfig.xml at 
http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section 
<http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section>. You may want to 
take another look and optimize/improve your schema design to prevent such 
issues. You can read more about Solr’s performance factors here -> 
http://wiki.apache.org/solr/SolrPerformanceFactors 
<http://wiki.apache.org/solr/SolrPerformanceFactors>. 

Thanks.


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 5, 2015, at 3:00 PM, Baskar Srinivasan  wrote:
> 
> Hello Zeeshan,
> 
> Thanks for the pointer regarding waiting for index creation in each node in 
> the cluster.
> 
> Presently, when the indices get created on one node, it takes a full 2-15 
> minutes for it to get created on other nodes in the cluster. Following are 
> the timestamps on 3 nodes for a single index:
> 
> #Create index request from our server via load balancer
> 11:16:52.999 [http-bio-8080-exec-3] INFO  c.v.s.u.RiakClientUtil - Created 
> index for bsr-test-fromlocal-1-Access_index
> 
> #1st node, immediate creation (12 secs) once call is issued from our server
> 2015-03-05 19:17:04.135 [info] <0.17388.104>@yz_index:local_create:189 
> Created index bsr-test-fromlocal-1-Access_index with schema
> 
> #2nd node, takes another 4 minutes for creation request to propagate
> 
> 
> 2015-03-05 19:21:17.879 [info] <0.20606.449>@yz_index:local_create:189 
> Created index bsr-test-fromlocal-1-Access_index
> 
> #3rd node, takes 15 minutes for creation request to propagate
> 
> 
> 2015-03-05 19:32:32.172 [info] <0.14715.94>@yz_index:local_create:189 Created 
> index bsr-test-fromlocal-1-Access_index
> 
> Is there a solr config we can tune to make the 2nd and 3rd node propagation 
> more immediate in the order of < 60 seconds?
> 
> Thanks,
> 
> Baskar
> 
> 
> On Thu, Mar 5, 2015 at 9:11 AM, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hello Santi, Baskar. Please keep your messages on the user group mailing 
> list, btw. Thanks.
> 
> Here’s an example of our testing harness’s wait_for_index function, 
> https://github.com/basho/yokozuna/blob/develop/riak_test/yz_rt.erl#L420 
> <https://github.com/basho/yokozuna/blob/develop/riak_test/yz_rt.erl#L420>. We 
> check for the index on each of the nodes, which is an approach you can take. 
> 
> And, as I mentioned, I’m currently working on making Index creation 
> synchronous to make this easier.
> 
> If your logs are not pointing to any errors and being that your bucket, index 
> contains so few objects, I’d delete or mv the search-root/index directory 
> (./data/yz/<>) and let AAE resync the data, which should then 
> give you consistent results.
> 
> Thanks.
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Still getting peaks on solr queries

[Zeeshan Lakhani]

Hello Alexander,

I would definitely take a look at how much RAM each node has. You can adjust 
the max heap size in riak.conf by changing search.solr.jvm_options max heap 
size values from -Xmx1g to -Xmx2g or more. I’d also take a look at 
http://docs.basho.com/riak/latest/ops/advanced/configs/search/#Solr-for-Operators
 
<http://docs.basho.com/riak/latest/ops/advanced/configs/search/#Solr-for-Operators>,
 which has links to Solr performance issues/factors and ways to monitor Solr.

You can get more statistical information for search by using `riak-admin stat 
show `. Our documentation of these statistics is still in progress, 
but you can see what’s available for search here: 
https://github.com/basho/yokozuna/blob/develop/src/yz_stat.erl#L92 
<https://github.com/basho/yokozuna/blob/develop/src/yz_stat.erl#L92>.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 6, 2015, at 4:36 PM, Alexander Popov  wrote:
> 
> I not have stable reproduce but peaks happens several times in day and takes 
> 10-50 sec.
> some times it happens on very low traffic, so I think it not related on 
> traffic.
> Looks like query is locked somehow.  
> What can I do for troubleshoot peaks? Enable addition logs or Monitoring ? 
> There is no errors in logs at time of the peaks
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Searching

[Zeeshan Lakhani]

Hello Jason,

I initially was thinking that you’re issues were similar to what we’ve found 
here, https://github.com/basho/yokozuna/issues/442#issuecomment-77233636 
<https://github.com/basho/yokozuna/issues/442#issuecomment-77233636>, but 
seeing {error, retry_later} and the 500s seem to place the problems more on the 
Solr end. Just to make sure, are you seeing `yz_events` crashes in your logs?

Can you tell me how much RAM each node has and have you adjusted/increased the 
search.sol.jmv_options max heap size at all (more info on issues/factors w/ 
Solr performance here -> 
http://docs.basho.com/riak/latest/ops/advanced/configs/search/#Solr-for-Operators
 
<http://docs.basho.com/riak/latest/ops/advanced/configs/search/#Solr-for-Operators>)?
 

What kind of issues are you getting in your solr logs? Anything related to 
bad/corrupted data (which will index into _yz_err fields when dealing with 
failed extractions) or possible sibling explosion (duplicates of the same 
object with different _yz_vtag fields)? 

Thanks.


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 7, 2015, at 5:43 AM, Jason Ryan  wrote:
> 
> Hi all,
> 
> We're having real trouble with Riak Search.
> We are seeing an awful lot of errors, which leads to alot of logging for solr 
> and the disk IO reaching 95% + which is causing lots of issues.
> 
> - We consistently see errors around Riak trying to create indexes which 
> already exist - only a restart of each Riak node stops this for a period of 
> time - indexes are only ever created manually, not by software.
> 
> - We see lots of errors around failing to index objects - details of what 
> appears in the log are:
> 
> 2015-03-07 10:30:26.871 [error] <0.2538.0>@yz_kv:index:215 failed to index 
> object 
> {{<<"Production">>,<<"Grains.Domain.Case">>},<<"455203890918dfc6fd3c7da49dd6adb0030043a46a51">>}
>  with error {"Failed to index docs",{error,retry_later}} because 
> [{yz_solr,index,3,[{file,"src/yz_solr.erl"},{line,192}]},{yz_kv,index,7,[{file,"src/yz_kv.erl"},{line,267}]},{yz_kv,index,3,[{file,"src/yz_kv.erl"},{line,202}]},{riak_kv_vnode,actual_put,6,[{file,"src/riak_kv_vnode.erl"},{line,1418}]},{riak_kv_vnode,perform_put,3,[{file,"src/riak_kv_vnode.erl"},{line,1406}]},{riak_kv_vnode,do_put,7,[{file,"src/riak_kv_vnode.erl"},{line,1201}]},{riak_kv_vnode,handle_command,3,[{file,"src/riak_kv_vnode.erl"},{line,486}]},{riak_core_vnode,vnode_command,3,[{file,"src/riak_core_vnode.erl"},{line,345}]}]
> 
> 
> - We are also starting to see 500's being returned for search queries - the 
> response looks like this:
> 
> 500 Internal Server ErrorInternal 
> Server ErrorThe server encountered an error while processing this 
> request:{error,
> 
> {throw,
> 
> {"Failed to search",
> 
>  "http://localhost:8093/internal_solr/sessions/select 
> <http://localhost:8093/internal_solr/sessions/select>",
> 
>  {error,retry_later}},
> 
> [{yz_solr,search,3,[{file,"src/yz_solr.erl"},{line,278}]},
> 
>  {yz_wm_search,search,2,[{file,"src/yz_wm_search.erl"},{line,129}]},
> 
>  {webmachine_resource,resource_call,3,
> 
>  [{file,"src/webmachine_resource.erl"},{line,186}]},
> 
>  {webmachine_resource,do,3,
> 
>  [{file,"src/webmachine_resource.erl"},{line,142}]},
> 
>  {webmachine_decision_core,resource_call,1,
> 
>  [{file,"src/webmachine_decision_core.erl"},{line,48}]},
> 
>  {webmachine_decision_core,decision,1,
> 
>  [{file,"src/webmachine_decision_core.erl"},{line,558}]},
> 
>  {webmachine_decision_core,handle_request,2,
> 
>  [{file,"src/webmachine_decision_core.erl"},{line,33}]},
> 
>  {webmachine_mochiweb,loop,2,
> 
> * Connection #0 to host 10.0.5.6 left intact
> 
>  
> [{file,"src/webmachine_mochiweb.erl"},{line,74}]}]}}mochiweb+webmachine
>  web server
> 
> 
> 
> Anyone that could point us in the right direction of where to look and debug, 
> as this is becoming a huge issue for us.
> 
> 
> 
> Thanks,
> 
> Jason
> 
> 
> 
> 
> 
> 
> This message is for the named person's use only. If you received this message 
> in error, please immediately delete it and all copies and notify the sender. 
> You must not, directly or indirectly, use, disclose, distribute, prin

Re: Searching

[Zeeshan Lakhani]

Saw your more recent email as well.

We’re currently working on attempting patch up the issues with 
_yz_events:handle_info and what that crash was causing. Are you seeing "Core 
with name ‘' already exists” in your solr logs? If so, that’s means 
it’s a similar issue, and, again, we’re working on testing patches internally.


Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 7, 2015, at 11:02 AM, Jason Ryan  wrote:
> 
> We have seen crash _yz_events yes, none today for example, but there was 
> quite a bit yesterday.
> 
> It's a 5 node cluster - 14GB of RAM in each, solr jvm is set at 8GB on each.
> 
> I've not seen any corrupted data, but we could be looking in the wrong place? 
> Our buckets are set to allow_mult false and last_write_wins true, so we don't 
> expect any siblings.
> 
> We had a health done by yourselves also, the main change, which we have not 
> yet applied is switching AAE from active to passive and setting erlang buffer 
> to 128MB
> 
> 
>   
> Jason Ryan
> VP Engineering
> 
> Trustev
> Real Time, Online Identity Verification
>  
> email: jason.r...@trustev.com <mailto:jason.r...@trustev.com> 
> skype: jason_j_ryan
> web: www.trustev.com <http://www.trustev.com/>
> 
> Trustev Ltd, 2100 Cork Airport Business Park, Cork, Ireland. 
> 
> On 7 March 2015 at 15:54, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hello Jason,
> 
> I initially was thinking that you’re issues were similar to what we’ve found 
> here, https://github.com/basho/yokozuna/issues/442#issuecomment-77233636 
> <https://github.com/basho/yokozuna/issues/442#issuecomment-77233636>, but 
> seeing {error, retry_later} and the 500s seem to place the problems more on 
> the Solr end. Just to make sure, are you seeing `yz_events` crashes in your 
> logs?
> 
> Can you tell me how much RAM each node has and have you adjusted/increased 
> the search.sol.jmv_options max heap size at all (more info on issues/factors 
> w/ Solr performance here -> 
> http://docs.basho.com/riak/latest/ops/advanced/configs/search/#Solr-for-Operators
>  
> <http://docs.basho.com/riak/latest/ops/advanced/configs/search/#Solr-for-Operators>)?
>  
> 
> What kind of issues are you getting in your solr logs? Anything related to 
> bad/corrupted data (which will index into _yz_err fields when dealing with 
> failed extractions) or possible sibling explosion (duplicates of the same 
> object with different _yz_vtag fields)? 
> 
> Thanks.
> 
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 7, 2015, at 5:43 AM, Jason Ryan > <mailto:jason.r...@trustev.com>> wrote:
>> 
>> Hi all,
>> 
>> We're having real trouble with Riak Search.
>> We are seeing an awful lot of errors, which leads to alot of logging for 
>> solr and the disk IO reaching 95% + which is causing lots of issues.
>> 
>> - We consistently see errors around Riak trying to create indexes which 
>> already exist - only a restart of each Riak node stops this for a period of 
>> time - indexes are only ever created manually, not by software.
>> 
>> - We see lots of errors around failing to index objects - details of what 
>> appears in the log are:
>> 
>> 2015-03-07 10:30:26.871 [error] <0.2538.0>@yz_kv:index:215 failed to index 
>> object 
>> {{<<"Production">>,<<"Grains.Domain.Case">>},<<"455203890918dfc6fd3c7da49dd6adb0030043a46a51">>}
>>  with error {"Failed to index docs",{error,retry_later}} because 
>> [{yz_solr,index,3,[{file,"src/yz_solr.erl"},{line,192}]},{yz_kv,index,7,[{file,"src/yz_kv.erl"},{line,267}]},{yz_kv,index,3,[{file,"src/yz_kv.erl"},{line,202}]},{riak_kv_vnode,actual_put,6,[{file,"src/riak_kv_vnode.erl"},{line,1418}]},{riak_kv_vnode,perform_put,3,[{file,"src/riak_kv_vnode.erl"},{line,1406}]},{riak_kv_vnode,do_put,7,[{file,"src/riak_kv_vnode.erl"},{line,1201}]},{riak_kv_vnode,handle_command,3,[{file,"src/riak_kv_vnode.erl"},{line,486}]},{riak_core_vnode,vnode_command,3,[{file,"src/riak_core_vnode.erl"},{line,345}]}]
>> 
>> 
>> - We are also starting to see 500's being returned for search queries - the 
>> response looks like this:
>> 
>> 500 Internal Server 
>> ErrorInternal Server ErrorThe server 
>

Re: Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime

[Zeeshan Lakhani]

Hey Santi, Baskar,

Are you noticing increased CPU load as you create more and more indexes? 
Running `riak-admin top -interval 2` a few times may bring sometime to light.

I’d see how you could increase resources or think more critically on how you’re 
indexing data for Solr. Does the data share most fields? Can you reuse indexes 
for some of the data and filter certain queries?

You may also wanted to look at this thread, 
https://groups.google.com/forum/#!topic/nosql-databases/9ECQpVS0QjE 
<https://groups.google.com/forum/#!topic/nosql-databases/9ECQpVS0QjE>, which 
discusses modeling Riak Search data and the issues you’ll have with the 
overhead with gossiping so much metadata and the what Solr can handle.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 9, 2015, at 8:25 AM, Santi Kumar  wrote:
> 
> Hi Zeeshan,
> 
> We have typically seen this issue when we have lots of indexes created in 
> that instance. On a t2.medium machine we already have around 512+ indexes 
> created in data folder. In such case, if we trying to create any new indexes 
> it's taking time. Association of Index to Bucket is failing even after  the 
> FetchIndex operation returning sucess as shown in the below code.
> 
> is there any limitation of the number of Indexes? Any thing related to 
> FileSystem handlers causing this issue?
> 
> while(!isCreated){
> 
> FetchIndex fetchIndex = new FetchIndex.Builder(indexName).build();
> 
> 
> RiakFuture  String> fetchIndexFuture = client.executeAsync(fetchIndex);
> 
> try{
> 
> fetchIndexFuture.await();
> 
> com.basho.riak.client.core.operations.YzFetchIndexOperation.Response 
> response = fetchIndexFuture.get();
> 
> List indexes = response.getIndexes();
> 
> for(YokozunaIndex index:indexes){
> 
> if(indexName.equals(index.getName())){
> 
> isCreated=true;
> 
> logger.info("Index "+indexName+" created ");
> 
> continue;
> 
> }
> 
> }
> 
> }catch(Exception e){
> 
> logger.warn("Unable to get "+indexName+" Still trying");
> 
> isCreated=false;
> 
> }
> 
> }
> 
> 
> On Fri, Mar 6, 2015 at 2:11 AM, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hello Baskar, Santi,
> 
> 2-15 minutes is a long while, and we’ve not seen index creation/propagation 
> be so slow. I’d definitely take a closer look at how you’re creating these 
> indexes dynamically on the fly, as index creation is typically a more 
> straightforward admin task.
> 
> We’ve added defaults to solrconfig.xml to handle most typical use-cases. You 
> can read more about solrconfig.xml at 
> http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section 
> <http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section>. You may want 
> to take another look and optimize/improve your schema design to prevent such 
> issues. You can read more about Solr’s performance factors here -> 
> http://wiki.apache.org/solr/SolrPerformanceFactors 
> <http://wiki.apache.org/solr/SolrPerformanceFactors>. 
> 
> Thanks.
> 
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 5, 2015, at 3:00 PM, Baskar Srinivasan > <mailto:bas...@veradocs.com>> wrote:
>> 
>> Hello Zeeshan,
>> 
>> Thanks for the pointer regarding waiting for index creation in each node in 
>> the cluster.
>> 
>> Presently, when the indices get created on one node, it takes a full 2-15 
>> minutes for it to get created on other nodes in the cluster. Following are 
>> the timestamps on 3 nodes for a single index:
>> 
>> #Create index request from our server via load balancer
>> 11:16:52.999 [http-bio-8080-exec-3] INFO  c.v.s.u.RiakClientUtil - Created 
>> index for bsr-test-fromlocal-1-Access_index
>> 
>> #1st node, immediate creation (12 secs) once call is issued from our server
>> 2015-03-05 19:17:04.135 [info] <0.17388.104>@yz_index:local_create:189 
>> Created index bsr-test-fromlocal-1-Access_index with schema
>> 
>> #2nd node, takes another 4 minutes for creation request to propagate
>> 
>> 
>> 2015-03-05 19:21:17.879 [info] <0.20606.449>@yz_index:local_create:189 
>> Created index bsr-test-fromlocal-1-Access_index
>> 
>> #3rd node, takes 15 minutes for creation request to propagate
>> 
>> 
>> 2015-03-05 19:32:32.172 [info] &

Re: Query on Riak Search in a cluster of 3 nodes behind ELB is giving different result everytime

[Zeeshan Lakhani]

The second approach would most probably cut down on index creation time. 
However, you should definitely spend a little time testing it out and 
benchmarking accordingly. And, as I mentioned, please take a look at CPU load  
as indexes are created, as well as experiment with solrconfig and increasing 
jvm heap memory settings for your use-case.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 9, 2015, at 10:13 AM, Baskar Srinivasan  wrote:
> 
> Hello Zeeshan,
> 
> We create a new set of buckets/indices when a new tenant is created in a 
> multi-tenancy environment. Alternate approach for us is to use single set of 
> index/buckets and filter by a tenant identifier. Before moving to the second 
> approach we want to confirm if we expect to see significant delays (several 
> minutes) with index propagation as the number of indices in the system grows.
> 
> Regards,
> Baskar
> 
> On Mon, Mar 9, 2015 at 7:02 AM, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hey Santi, Baskar,
> 
> Are you noticing increased CPU load as you create more and more indexes? 
> Running `riak-admin top -interval 2` a few times may bring sometime to light.
> 
> I’d see how you could increase resources or think more critically on how 
> you’re indexing data for Solr. Does the data share most fields? Can you reuse 
> indexes for some of the data and filter certain queries?
> 
> You may also wanted to look at this thread, 
> https://groups.google.com/forum/#!topic/nosql-databases/9ECQpVS0QjE 
> <https://groups.google.com/forum/#!topic/nosql-databases/9ECQpVS0QjE>, which 
> discusses modeling Riak Search data and the issues you’ll have with the 
> overhead with gossiping so much metadata and the what Solr can handle.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 9, 2015, at 8:25 AM, Santi Kumar > <mailto:sa...@veradocs.com>> wrote:
>> 
>> Hi Zeeshan,
>> 
>> We have typically seen this issue when we have lots of indexes created in 
>> that instance. On a t2.medium machine we already have around 512+ indexes 
>> created in data folder. In such case, if we trying to create any new indexes 
>> it's taking time. Association of Index to Bucket is failing even after  the 
>> FetchIndex operation returning sucess as shown in the below code.
>> 
>> is there any limitation of the number of Indexes? Any thing related to 
>> FileSystem handlers causing this issue?
>> 
>> while(!isCreated){
>> 
>> FetchIndex fetchIndex = new FetchIndex.Builder(indexName).build();
>> 
>> 
>> RiakFuture>  String> fetchIndexFuture = client.executeAsync(fetchIndex);
>> 
>> try{
>> 
>> fetchIndexFuture.await();
>> 
>> com.basho.riak.client.core.operations.YzFetchIndexOperation.Response 
>> response = fetchIndexFuture.get();
>> 
>> List indexes = response.getIndexes();
>> 
>> for(YokozunaIndex index:indexes){
>> 
>> if(indexName.equals(index.getName())){
>> 
>> isCreated=true;
>> 
>> logger.info("Index "+indexName+" created ");
>> 
>> continue;
>> 
>> }
>> 
>> }
>> 
>> }catch(Exception e){
>> 
>> logger.warn("Unable to get "+indexName+" Still trying");
>> 
>> isCreated=false;
>> 
>> }
>> 
>> }
>> 
>> 
>> On Fri, Mar 6, 2015 at 2:11 AM, Zeeshan Lakhani > <mailto:zlakh...@basho.com>> wrote:
>> Hello Baskar, Santi,
>> 
>> 2-15 minutes is a long while, and we’ve not seen index creation/propagation 
>> be so slow. I’d definitely take a closer look at how you’re creating these 
>> indexes dynamically on the fly, as index creation is typically a more 
>> straightforward admin task.
>> 
>> We’ve added defaults to solrconfig.xml to handle most typical use-cases. You 
>> can read more about solrconfig.xml at 
>> http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section 
>> <http://wiki.apache.org/solr/SolrConfigXml#mainIndex_Section>. You may want 
>> to take another look and optimize/improve your schema design to prevent such 
>> issues. You can read more about Solr’s performance factors here -> 
>> http://wiki.apache.org/solr/SolrPerformanceFactors 
>> <http://wiki.apache.org/solr/SolrPerformanceFact

Re: Yokozuna - Solr won't start, keeps throwing OutOfMemoryErrors

[Zeeshan Lakhani]

Hello Cezary,

I’d monitor your system's memory as well as the JVM’s memory usage 
(http://visualvm.java.net/ <http://visualvm.java.net/>). Solr’s documentation 
on OutofMemoryErrors is here -> 
http://wiki.apache.org/solr/SolrPerformanceFactors#OutOfMemoryErrors 
<http://wiki.apache.org/solr/SolrPerformanceFactors#OutOfMemoryErrors>. 
Updating search.solr.jvm_options should alleviate most issues however.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 10, 2015, at 4:32 PM, Cezary Kosko  wrote:
> 
> All,
> 
> I've been working on a Riak setup with Search (version 2.0.0), 2 custom 
> schemas for 3 datatypes (maps of sets, wanted to query against values of 
> these sets, one of the schemas stores one of these sets, but there's max 2 
> values in that particular one per record).
> 
> Everything seemed to be running smoothly up to one day, when all of a sudden 
> Solr started throwing OutOfMemoryErrors. I have increased the max allocated 
> memory from the default 1g to 2g, then 3g, but that did not help.
> 
> Is there a routine to be done in such cases?
> 
> Kind regards,
> Cezary
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Different numFound request to riak search

[Zeeshan Lakhani]

Hello Roma, 

Have you deleted this object at some point in your runs? Please make sure AAE 
is running by checking search’s AAE status, `riak-admin search aae-status`, and 
that data exists in the correct directory, `./data/yz_anti_entropy` 
(http://docs.basho.com/riak/latest/ops/advanced/configs/search/). You may just 
need to perform a read-repair by performing a fetch of the object itself first, 
before performing search queries again.

Also, have you left or added nodes? I’m guessing that  even your 1 node 
instance is still running a cluster on that one node, right?

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 12, 2015, at 5:59 AM, Roma Lakotko  wrote:
> 
> Each request to riak search return different results. It's return different 
> numFound.
> 
> I use request like this:
> 
> http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc 
> <http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc>
> 
> If add start offset it can return:
> 
> http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc&start=1247
>  
> <http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc&start=1247>
> "response": {
> "numFound": 1248,
> "start": 1247,
> "docs": [
> {
> "_yz_id": 
> "1*default*assets*fff63ecf-a0c4-4ecf-b24d-c493ca3a302f*44",
> "_yz_rk": "fff63ecf-a0c4-4ecf-b24d-c493ca3a302f",
> "_yz_rt": "default",
> "_yz_rb": "assets"
> }
> ]
> }
> 
> On next request it return something like this
> 
> "numFound": 1224,
> "start": 1247,
> "docs": []
> 
> I have 1 node installation, and no process write to Riak.
> I have same problem this production cluster with 7 nodes.
> 
> Scheme for document
> 
> 
> 
>  
> stored="false" />
>
> stored="false" />
>
> stored="false" />
> stored="false" multiValued="true" />
>
> stored="false" />
> stored="false" />
>
>
> 
>
> multiValued="false" required="true"/>
> multiValued="false"/>
> multiValued="false"/>
> multiValued="false"/>
> multiValued="false"/>
> multiValued="false"/>
> multiValued="false"/>
> multiValued="false"/>
> multiValued="false"/>
> 
>
>  
> 
>  _yz_id
> 
>  
>   
>omitNorms="true"/>
>sortMissingLast="true" omitNorms="true"/>
>omitNorms="true"/>
> 
>omitNorms="true"/>
>   
>class="solr.StrField" />
>omitNorms="true">
> 
> 
> 
>  replacement='е' replace='all'/>
> 
>  
>   
> 
> 
> Best regards,
> Roman
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Different numFound request to riak search

[Zeeshan Lakhani]

Are you noticing any Solr errors in the logs? 

For your container instance, you can attempt to clear the AAE trees and force a 
rebuild by removing the entropy directories in `./data/yz_anti_entropy` and 
running `yz_entropy_mgr:init([])` via `riak attach`.  Or, you can let AAE occur 
naturally (after removing the entropy data) and up the 
concurrency/build_limit/tick (using set_env). You can see what you’re current 
settings are by calling...

```
riak_core_util:rpc_every_member_ann(application, get_env, [riak_kv, 
anti_entropy_build_limit],infinity).
riak_core_util:rpc_every_member_ann(application, get_env, [riak_kv, 
anti_entropy_concurrency],infinity).
riak_core_util:rpc_every_member_ann(application, get_env, [yokozuna, 
anti_entropy_tick],infinity).
```

… on any of the nodes.  Query coverage is R=1, but the values should be 
replicated across.

Thanks.


> On Mar 12, 2015, at 9:51 AM, Roma Lakotko  wrote:
> 
> Hello Zeeshan.
> 
> While i run queries no delete object is occurs. 
> 
> Stats on production and developer nodes output something like this: 
> https://gist.github.com/romulka/d0254aa193a9dbb52b67 
> <https://gist.github.com/romulka/d0254aa193a9dbb52b67>
> 
> On dev container:
> 
> /etc/riak# grep anti_entropy *
> riak.conf:anti_entropy = active
> riak.conf.dpkg-dist:anti_entropy = active
> 
> ll -h /var/lib/riak/yz_anti_entropy/
> total 264K
> drwxrwxr-x 66 riak riak 4.0K Sep 25 12:08 ./
> drwxr-xr-x 12 riak riak 4.0K Dec  9 12:19 ../
> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:01 0/
> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:01 
> 1004782375664995756265033322492444576013453623296/
> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:01 
> 1027618338748291114361965898003636498195577569280/
> 
> 
> On prod:
> 
> grep anti_entropy * /etc/riak/ -> empty
> 
> root@riak-21:/var/lib/riak/yz_anti_entropy# ll -h
> total 64K
> drwxrwxr-x 16 riak riak 4.0K Dec  4 03:44 ./
> drwxr-xr-x 14 riak riak 4.0K Dec  9 12:10 ../
> drwxr-xr-x  9 riak riak 4.0K Dec  4 03:44 0/
> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:57 
> 1027618338748291114361965898003636498195577569280/
> 
> 
> I'm already try re-save all keys, it doesn't helps.
> 
> Production cluster have 7 node, start from 3. So yes, nodes was added/delete 
> sometimes.
> 
> On dev, i have 1 instance in docker container, never added to cluster. But 
> data in that riak is imported from production cluster a while ago.
> 
> I can give you a copy of container, if you need to.
> 
> Thanks,
> Ronan Lakotko
> 
> 
> 
> 2015-03-12 16:36 GMT+03:00 Zeeshan Lakhani  <mailto:zlakh...@basho.com>>:
> Hello Roma, 
> 
> Have you deleted this object at some point in your runs? Please make sure AAE 
> is running by checking search’s AAE status, `riak-admin search aae-status`, 
> and that data exists in the correct directory, `./data/yz_anti_entropy` 
> (http://docs.basho.com/riak/latest/ops/advanced/configs/search/ 
> <http://docs.basho.com/riak/latest/ops/advanced/configs/search/>). You may 
> just need to perform a read-repair by performing a fetch of the object itself 
> first, before performing search queries again.
> 
> Also, have you left or added nodes? I’m guessing that  even your 1 node 
> instance is still running a cluster on that one node, right?
> 
> Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 12, 2015, at 5:59 AM, Roma Lakotko > <mailto:ro...@lakotko.ru>> wrote:
>> 
>> Each request to riak search return different results. It's return different 
>> numFound.
>> 
>> I use request like this:
>> 
>> http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc 
>> <http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc>
>> 
>> If add start offset it can return:
>> 
>> http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc&start=1247
>>  
>> <http://localhost:8098/search/query/assets?wt=json&q=type:*&sort=_yz_rk%20asc&start=1247>
>> "response": {
>> "numFound": 1248,
>> "start": 1247,
>> "docs": [
>> {
>> "_yz_id": 
>> "1*default*assets*fff63ecf-a0c4-4ecf-b24d-c493ca3a302f*44",
>> "_yz_rk": "fff63ecf-a0c4-4ecf-b24d-c493ca3a302f",
>> "_yz_rt": "default",
>> "

Re: Different numFound request to riak search

[Zeeshan Lakhani]

Are you running mapreduce with Solr queries? 


> On Mar 12, 2015, at 11:50 AM, Roma Lakotko  wrote:
> 
> I don't see any solr errors. But each 10-20 minutes on prod and once a day on 
> dev i see strange errors:
> 
> 2015-03-11 09:18:10.668 [error] <0.234.0> Supervisor riak_pipe_fitting_sup 
> had child undefined started with riak_pipe_fitting:start_link() at 
> <0.12060.2> exit with reason noproc in context shutdown_error
> 2015-03-12 13:12:05.200 [error] <0.379.0> Supervisor riak_kv_mrc_sink_sup had 
> child undefined started with riak_kv_mrc_sink:start_link() at <0.6601.1> exit 
> with reason noproc in context shutdown_error
> 
> For both prod and dev instance values are:
> 
> anti_entropy_build_limit  -> {ok,{1,360}}
> anti_entropy_concurrency -> {ok,2}
> anti_entropy_tick - > undefined
> 
> I delete data folder and run init method, i'll  results after it rebuild 
> trees.
> 
> 2015-03-12 18:22 GMT+03:00 Zeeshan Lakhani  <mailto:zlakh...@basho.com>>:
> Are you noticing any Solr errors in the logs? 
> 
> For your container instance, you can attempt to clear the AAE trees and force 
> a rebuild by removing the entropy directories in `./data/yz_anti_entropy` and 
> running `yz_entropy_mgr:init([])` via `riak attach`.  Or, you can let AAE 
> occur naturally (after removing the entropy data) and up the 
> concurrency/build_limit/tick (using set_env). You can see what you’re current 
> settings are by calling...
> 
> ```
> riak_core_util:rpc_every_member_ann(application, get_env, [riak_kv, 
> anti_entropy_build_limit],infinity).
> riak_core_util:rpc_every_member_ann(application, get_env, [riak_kv, 
> anti_entropy_concurrency],infinity).
> riak_core_util:rpc_every_member_ann(application, get_env, [yokozuna, 
> anti_entropy_tick],infinity).
> ```
> 
> … on any of the nodes.  Query coverage is R=1, but the values should be 
> replicated across.
> 
> Thanks.
> 
> 
>> On Mar 12, 2015, at 9:51 AM, Roma Lakotko > <mailto:ro...@lakotko.ru>> wrote:
>> 
>> Hello Zeeshan.
>> 
>> While i run queries no delete object is occurs. 
>> 
>> Stats on production and developer nodes output something like this: 
>> https://gist.github.com/romulka/d0254aa193a9dbb52b67 
>> <https://gist.github.com/romulka/d0254aa193a9dbb52b67>
>> 
>> On dev container:
>> 
>> /etc/riak# grep anti_entropy *
>> riak.conf:anti_entropy = active
>> riak.conf.dpkg-dist:anti_entropy = active
>> 
>> ll -h /var/lib/riak/yz_anti_entropy/
>> total 264K
>> drwxrwxr-x 66 riak riak 4.0K Sep 25 12:08 ./
>> drwxr-xr-x 12 riak riak 4.0K Dec  9 12:19 ../
>> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:01 0/
>> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:01 
>> 1004782375664995756265033322492444576013453623296/
>> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:01 
>> 1027618338748291114361965898003636498195577569280/
>> 
>> 
>> On prod:
>> 
>> grep anti_entropy * /etc/riak/ -> empty
>> 
>> root@riak-21:/var/lib/riak/yz_anti_entropy# ll -h
>> total 64K
>> drwxrwxr-x 16 riak riak 4.0K Dec  4 03:44 ./
>> drwxr-xr-x 14 riak riak 4.0K Dec  9 12:10 ../
>> drwxr-xr-x  9 riak riak 4.0K Dec  4 03:44 0/
>> drwxr-xr-x  9 riak riak 4.0K Mar 12 12:57 
>> 1027618338748291114361965898003636498195577569280/
>> 
>> 
>> I'm already try re-save all keys, it doesn't helps.
>> 
>> Production cluster have 7 node, start from 3. So yes, nodes was added/delete 
>> sometimes.
>> 
>> On dev, i have 1 instance in docker container, never added to cluster. But 
>> data in that riak is imported from production cluster a while ago.
>> 
>> I can give you a copy of container, if you need to.
>> 
>> Thanks,
>> Ronan Lakotko
>> 
>> 
>> 
>> 2015-03-12 16:36 GMT+03:00 Zeeshan Lakhani > <mailto:zlakh...@basho.com>>:
>> Hello Roma, 
>> 
>> Have you deleted this object at some point in your runs? Please make sure 
>> AAE is running by checking search’s AAE status, `riak-admin search 
>> aae-status`, and that data exists in the correct directory, 
>> `./data/yz_anti_entropy` 
>> (http://docs.basho.com/riak/latest/ops/advanced/configs/search/ 
>> <http://docs.basho.com/riak/latest/ops/advanced/configs/search/>). You may 
>> just need to perform a read-repair by performing a fetch of the object 
>> itself first, before performing search queries again.
>> 
>> Also, have you left or added nodes? I’m guessing that  even your 1 node 
>> instance is still running a cluster on that

Re: Trouble with Riak Search JSON Extractor

[Zeeshan Lakhani]

Hello Matt,

Are you receiving any indexing errors in your logs (check the Solr logs as 
well)? To be sure, I just wrote a test (using our erlang pb api, 
https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251
 
<https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251>)
 using a custom schema, and I didn’t run into any troubles; I also checked via 
the http api. If you have the required _yz* fields, you should get those back 
with the document. If you want the groups items themselves, you’d have to make 
`stored` true.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 22, 2015, at 4:13 PM, Matt Brooks  wrote:
> 
> Hello,
> 
> I have a quick question about a search schema that would index an array of 
> strings in a JSON object. I am storing user data in JSON that looks something 
> like this:
> 
> {
>"name" : "John Smith",
>"email" : "jsm...@gmail.com <mailto:jsm...@gmail.com>",
>"groups" : [
>   "3304cf79",
>   "abe155cf"
>]
> }
> The custom schema I use for users includes this field entry: 
> 
>  multiValued="true"/>
> I also have the following field type entry: 
> 
> 
> I assumed that these entries would allow me to search for users using a query 
> like: 
> 
> curl $RIAK/search/query/user?wt=json&q=groups:3304cf79
> But no docs are returned. I'm almost certain I have an issue with my schema. 
> How can I query users by string entries in the "groups" array?
> 
> Thank you, 
> Matt.
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Trouble with Riak Search JSON Extractor

[Zeeshan Lakhani]

Hey Matt,

Thanks for clarifying the situation. Yes, I’m working on auto-reloading and 
re-syncing data along with changes to schemas; it’ll be in a future release. 
Currently, you can add fields to the schema, for example, for new sets of data, 
and reload the index if you `riak attach` into one of the running nodes and 
reload the index across the cluster (based on 
https://github.com/basho/yokozuna/pull/260 
<https://github.com/basho/yokozuna/pull/260>), e.g. 
https://github.com/basho/yokozuna/issues/130#issuecomment-43802826 
<https://github.com/basho/yokozuna/issues/130#issuecomment-43802826>.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 22, 2015, at 9:37 PM, Matt Brooks  wrote:
> 
> Just discovered this on GitHub https://github.com/basho/yokozuna/issues/130 
> <https://github.com/basho/yokozuna/issues/130>. Looks like auto-reloading is 
> a work in progress.
> 
> On Sun, Mar 22, 2015 at 9:24 PM Matt Brooks  <mailto:mtbrook...@gmail.com>> wrote:
> Not getting any errors... but I did notice something interesting after 
> deleting the nodes on my system and then reconfiguring them. 
> 
> If the first time I set up a schema I add the multivalued field in question, 
> everything works. If I then change the schema and then update it over HTTP, 
> the changes do not take affect. 
> 
> For example, when I reconfigured the nodes on my system I added kept the 
> multivalued "groups" field I mentioned above. I then added some users and was 
> able to successfully retrieve them using q=groups:* or something more 
> specific. I then changed the schema from "groups" to "groups.id 
> <http://groups.id/>" and PUT it on the server successfully (a subsequent GET 
> returned the new schema). I also updated the users so that the "groups" array 
> was changed to an array of JSON objects, each with an "id" field that would 
> match the "groups.id <http://groups.id/>" field in the schema. I then tried 
> to query the users using q=groups.id:*. This failed. 
> 
> Do changed to the schema not cause the solr indexes to update? 
> 
> PS: I dug around in $RIAK/data/yz/user/conf and noticed a user.xml file. The 
> file held the ORIGINAL schema, not the updated one.
> 
> 
> On Sun, Mar 22, 2015 at 6:45 PM Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hello Matt,
> 
> Are you receiving any indexing errors in your logs (check the Solr logs as 
> well)? To be sure, I just wrote a test (using our erlang pb api, 
> https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251
>  
> <https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251>)
>  using a custom schema, and I didn’t run into any troubles; I also checked 
> via the http api. If you have the required _yz* fields, you should get those 
> back with the document. If you want the groups items themselves, you’d have 
> to make `stored` true.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
> 
>> On Mar 22, 2015, at 4:13 PM, Matt Brooks > <mailto:mtbrook...@gmail.com>> wrote:
>> 
> 
>> Hello,
>> 
>> I have a quick question about a search schema that would index an array of 
>> strings in a JSON object. I am storing user data in JSON that looks 
>> something like this:
>> 
>> {
>>"name" : "John Smith",
>>"email" : "jsm...@gmail.com <mailto:jsm...@gmail.com>",
>>"groups" : [
>>   "3304cf79",
>>   "abe155cf"
>>]
>> }
>> The custom schema I use for users includes this field entry: 
>> 
>> > multiValued="true"/>
>> I also have the following field type entry: 
>> 
>> 
>> I assumed that these entries would allow me to search for users using a 
>> query like: 
>> 
>> curl $RIAK/search/query/user?wt=json&q=groups:3304cf79
>> But no docs are returned. I'm almost certain I have an issue with my schema. 
>> How can I query users by string entries in the "groups" array?
>> 
>> Thank you, 
>> Matt.
>> 
> 
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Trouble with Riak Search JSON Extractor

[Zeeshan Lakhani]

Hey Matt, 

Remember, with the reload, it’s only new objects in *new* fields that will take 
the updates. Depending on how big your dataset is, the quick fix may be to 
upload the *updated* schema under a new name and tie the index to that. The K/V 
pairs should then work correctly when PUT.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 23, 2015, at 12:07 PM, Matt Brooks  wrote:
> 
> Zeeshan, 
> 
> Is there a way to reload the index manually? I tried running 
> "rp(yz_index:reload(<<"index_name">>))." on each node as suggested on a 
> GitHub issue, and though I got an "ok" response back listing each nodes in 
> the cluster, I still could not query using the updated schema. I even deleted 
> all of my K/V pairs and re-added them with changed keys. 
> 
> Thank you, 
> Matt.
> 
> On Sun, Mar 22, 2015 at 10:21 PM Matt Brooks  <mailto:mtbrook...@gmail.com>> wrote:
> I see. I'll be looking forward to the future release! Thank you, Zeeshan.
> 
> Matt.
> 
> On Sun, Mar 22, 2015 at 9:57 PM Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Hey Matt,
> 
> Thanks for clarifying the situation. Yes, I’m working on auto-reloading and 
> re-syncing data along with changes to schemas; it’ll be in a future release. 
> Currently, you can add fields to the schema, for example, for new sets of 
> data, and reload the index if you `riak attach` into one of the running nodes 
> and reload the index across the cluster (based on 
> https://github.com/basho/yokozuna/pull/260 
> <https://github.com/basho/yokozuna/pull/260>), e.g. 
> https://github.com/basho/yokozuna/issues/130#issuecomment-43802826 
> <https://github.com/basho/yokozuna/issues/130#issuecomment-43802826>.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 22, 2015, at 9:37 PM, Matt Brooks > <mailto:mtbrook...@gmail.com>> wrote:
>> 
>> Just discovered this on GitHub https://github.com/basho/yokozuna/issues/130 
>> <https://github.com/basho/yokozuna/issues/130>. Looks like auto-reloading is 
>> a work in progress.
>> 
>> On Sun, Mar 22, 2015 at 9:24 PM Matt Brooks > <mailto:mtbrook...@gmail.com>> wrote:
>> Not getting any errors... but I did notice something interesting after 
>> deleting the nodes on my system and then reconfiguring them. 
>> 
>> If the first time I set up a schema I add the multivalued field in question, 
>> everything works. If I then change the schema and then update it over HTTP, 
>> the changes do not take affect. 
>> 
>> For example, when I reconfigured the nodes on my system I added kept the 
>> multivalued "groups" field I mentioned above. I then added some users and 
>> was able to successfully retrieve them using q=groups:* or something more 
>> specific. I then changed the schema from "groups" to "groups.id 
>> <http://groups.id/>" and PUT it on the server successfully (a subsequent GET 
>> returned the new schema). I also updated the users so that the "groups" 
>> array was changed to an array of JSON objects, each with an "id" field that 
>> would match the "groups.id <http://groups.id/>" field in the schema. I then 
>> tried to query the users using q=groups.id:*. This failed. 
>> 
>> Do changed to the schema not cause the solr indexes to update? 
>> 
>> PS: I dug around in $RIAK/data/yz/user/conf and noticed a user.xml file. The 
>> file held the ORIGINAL schema, not the updated one.
>> 
>> 
>> On Sun, Mar 22, 2015 at 6:45 PM Zeeshan Lakhani > <mailto:zlakh...@basho.com>> wrote:
>> Hello Matt,
>> 
>> Are you receiving any indexing errors in your logs (check the Solr logs as 
>> well)? To be sure, I just wrote a test (using our erlang pb api, 
>> https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251
>>  
>> <https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251>)
>>  using a custom schema, and I didn’t run into any troubles; I also checked 
>> via the http api. If you have the required _yz* fields, you should get those 
>> back with the document. If you want the groups items themselves, you’d have 
>> to make `stored` true.
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. me

Re: Trouble with Riak Search JSON Extractor

[Zeeshan Lakhani]

And Matt, please choose one platform to discuss the matter, preferably here :). 
It makes things easier. 

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 23, 2015, at 12:14 PM, Zeeshan Lakhani  wrote:
> 
> Remember, with the reload, it’s only new objects in *new* fields that will 
> take the updates. Depending on how big your dataset is, the quick fix may be 
> to upload the *updated* schema under a new name and tie the index to that. 
> The K/V pairs should then work correctly when PUT.

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak2.0 with Solr Search: index on one node contains not all entries

[Zeeshan Lakhani]

Hello Michael,

First off, that section of the docs related to search index repair that you 
linked to is not referring to Riak Search 2.0 with Solr, and the link itself is 
actually pointing to Riak 1.4.7. We’ll update them accordingly for the current 
incarnation of Riak Search (for the 2.0 series).

Just to be sure, does your custom schema include the required fields, as 
mentioned in the docs: 
http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Custom-Schemas? 
<http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Custom-Schemas?> 
Are these Riak nodes joined? What’s your ring size, n_val value?

How are you querying the Solr nodes to know which node has the data and which 
one doesn't? Coverage is R=1, so you would be getting a different number on 
some search queries (using the standard /search/query/?...) if its 
only on one of the Solr cores.

Can you also post me a screenshot of your search AAE exchanges, e.g. 
`riak-admin search aae-status`? You could look at this thread, 
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html
 
<http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html>,
 for answers on how to perform read-repair/repair the AAE tree.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 25, 2015, at 6:42 AM, Michael Weibel  wrote:
> 
> Hi all,
> 
> I have on a test environment two riak nodes and each of them has solr 
> activated which index 3 buckets using a custom schema.
> After testing a bit back and forth, I have the case that on one solr node, an 
> entry is not in the index (I know in which node though). 
> Fetching the specific key in the bucket works however, both nodes have the 
> respective entry.
> 
> 1) How can this happen? I don't see any error/warning in the logs (neither 
> solr nor riak logs). 
> 2) Is there a possibility to fix this without having to do e.g. a PUT on the 
> specific key with the same content in order to update it?
> 
> I tried to run a repair on the failing node using the guide: 
> http://docs.basho.com/riak/1.4.7/ops/running/recovery/repairing-indexes/#Repairing-Search-Indexes
>  
> <http://docs.basho.com/riak/1.4.7/ops/running/recovery/repairing-indexes/#Repairing-Search-Indexes>
> When running the repair command on the partitions I received, it gave me 
> however a [{, down}, {...}] response, which gives me an 
> uncomfortable feeling, but I didn't really figure out yet what this means 
> exactly. 
> 
> Thanks a lot for your help :)
> 
> Best,
> Michael
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak2.0 with Solr Search: index on one node contains not all entries

[Zeeshan Lakhani]

Hey Michael,

Ideally, for this “testing" setup, n_val=2 would be the effective choice. I’d 
create a new bucket_type/bucket and re-PUT your data in and test search again 
to be sure.

Let me know. Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 25, 2015, at 11:02 AM, Michael Weibel  wrote:
> 
> Hi Zeeshan,
> 
> Thanks for your answer.
> Just to be sure, does your custom schema include the required fields, as 
> mentioned in the docs: 
> http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Custom-Schemas? 
> <http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Custom-Schemas?>
> 
> Yes, I double checked that now to make sure, and the schema includes the 
> required fields.
>  
>  Are these Riak nodes joined? What’s your ring size, n_val value?
> 
> They run in a cluster, yes. Output of "riak-admin status":
> 
> ring_creation_size : 64
> ring_members : ['riak@IPADDRESS','riak@IPADDRESS']
> ring_num_partitions : 64
> ring_ownership : <<"[{'riak@IPADDRESS',32},{'riak@IPADDRESS',32}]">>
> rings_reconciled : 0
> rings_reconciled_total : 33
> 
> n_val is 3, the initial one. According to the docs we should probably either 
> add another node or reduce it to "2" though..correct? (Sorry, newbie in riak 
> here :D)
>  
> How are you querying the Solr nodes to know which node has the data and which 
> one doesn't? Coverage is R=1, so you would be getting a different number on 
> some search queries (using the standard /search/query/?...) if 
> its only on one of the Solr cores.
> 
> Yes exactly. That's how I figured out that there's a difference, later on I 
> queried the separate Solr instances using the solr admin interface itself. I 
> also then fetched the missing key on both the riak nodes (without going to 
> solr, just fetching it directly using the riak HTTP API) and they exist on 
> both nodes. 
>  
> Can you also post me a screenshot of your search AAE exchanges, e.g. 
> `riak-admin search aae-status`? You could look at this thread, 
> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html
>  
> <http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html>,
>  for answers on how to perform read-repair/repair the AAE tree.
> 
> aae-status is in the .log file attached.
> 
> So you'd propose to perform a read-repair on the AAE tree?
> 
> Best,
> Michael
> 
>  
> 
> Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 25, 2015, at 6:42 AM, Michael Weibel > <mailto:michael.wei...@gmail.com>> wrote:
>> 
>> Hi all,
>> 
>> I have on a test environment two riak nodes and each of them has solr 
>> activated which index 3 buckets using a custom schema.
>> After testing a bit back and forth, I have the case that on one solr node, 
>> an entry is not in the index (I know in which node though). 
>> Fetching the specific key in the bucket works however, both nodes have the 
>> respective entry.
>> 
>> 1) How can this happen? I don't see any error/warning in the logs (neither 
>> solr nor riak logs). 
>> 2) Is there a possibility to fix this without having to do e.g. a PUT on the 
>> specific key with the same content in order to update it?
>> 
>> I tried to run a repair on the failing node using the guide: 
>> http://docs.basho.com/riak/1.4.7/ops/running/recovery/repairing-indexes/#Repairing-Search-Indexes
>>  
>> <http://docs.basho.com/riak/1.4.7/ops/running/recovery/repairing-indexes/#Repairing-Search-Indexes>
>> When running the repair command on the partitions I received, it gave me 
>> however a [{, down}, {...}] response, which gives me an 
>> uncomfortable feeling, but I didn't really figure out yet what this means 
>> exactly. 
>> 
>> Thanks a lot for your help :)
>> 
>> Best,
>> Michael
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
>> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
> 
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak2.0 with Solr Search: index on one node contains not all entries

[Zeeshan Lakhani]

Ok, no worries.

Also, sometimes, AAE may take a little time to complete the exchanges. 
Definitely read through my responses in 
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html
 
<http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html>
 if you continue to see issues with Riak Search being out-of-sync.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 25, 2015, at 12:27 PM, Michael Weibel  wrote:
> 
> Hi Zeeshan,
> 
> Ok, will do that and report back as soon as I have it.. Might take a while 
> though because I first also have to figure out whether I have still the same 
> issue or not.. ;) 
> 
> Thanks!
> Michael
> 
> 
> 2015-03-25 16:56 GMT+01:00 Zeeshan Lakhani  <mailto:zlakh...@basho.com>>:
> Hey Michael,
> 
> Ideally, for this “testing" setup, n_val=2 would be the effective choice. I’d 
> create a new bucket_type/bucket and re-PUT your data in and test search again 
> to be sure.
> 
> Let me know. Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 25, 2015, at 11:02 AM, Michael Weibel > <mailto:michael.wei...@gmail.com>> wrote:
>> 
>> Hi Zeeshan,
>> 
>> Thanks for your answer.
>> Just to be sure, does your custom schema include the required fields, as 
>> mentioned in the docs: 
>> http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Custom-Schemas?
>>  
>> <http://docs.basho.com/riak/latest/dev/advanced/search-schema/#Custom-Schemas?>
>> 
>> Yes, I double checked that now to make sure, and the schema includes the 
>> required fields.
>>  
>>  Are these Riak nodes joined? What’s your ring size, n_val value?
>> 
>> They run in a cluster, yes. Output of "riak-admin status":
>> 
>> ring_creation_size : 64
>> ring_members : ['riak@IPADDRESS','riak@IPADDRESS']
>> ring_num_partitions : 64
>> ring_ownership : <<"[{'riak@IPADDRESS',32},{'riak@IPADDRESS',32}]">>
>> rings_reconciled : 0
>> rings_reconciled_total : 33
>> 
>> n_val is 3, the initial one. According to the docs we should probably either 
>> add another node or reduce it to "2" though..correct? (Sorry, newbie in riak 
>> here :D)
>>  
>> How are you querying the Solr nodes to know which node has the data and 
>> which one doesn't? Coverage is R=1, so you would be getting a different 
>> number on some search queries (using the standard 
>> /search/query/?...) if its only on one of the Solr cores.
>> 
>> Yes exactly. That's how I figured out that there's a difference, later on I 
>> queried the separate Solr instances using the solr admin interface itself. I 
>> also then fetched the missing key on both the riak nodes (without going to 
>> solr, just fetching it directly using the riak HTTP API) and they exist on 
>> both nodes. 
>>  
>> Can you also post me a screenshot of your search AAE exchanges, e.g. 
>> `riak-admin search aae-status`? You could look at this thread, 
>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html
>>  
>> <http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016926.html>,
>>  for answers on how to perform read-repair/repair the AAE tree.
>> 
>> aae-status is in the .log file attached.
>> 
>> So you'd propose to perform a read-repair on the AAE tree?
>> 
>> Best,
>> Michael
>> 
>>  
>> 
>> Thanks.
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love | paperswelove.org 
>> <http://paperswelove.org/>
>> twitter => @zeeshanlakhani
>> 
>>> On Mar 25, 2015, at 6:42 AM, Michael Weibel >> <mailto:michael.wei...@gmail.com>> wrote:
>>> 
>>> Hi all,
>>> 
>>> I have on a test environment two riak nodes and each of them has solr 
>>> activated which index 3 buckets using a custom schema.
>>> After testing a bit back and forth, I have the case that on one solr node, 
>>> an entry is not in the index (I know in which node though). 
>>> Fetching the specific key in the bucket works however, both nodes have the 
>>> respective entry.
>>> 
>>

Re: Delete Objects SOLR

[Zeeshan Lakhani]

On riak 2.0 with Yokozuna, you can delete search indexes (async) via http/pb, 
e.g. http://docs.basho.com/riak/latest/dev/references/http/delete-search-index/ 
<http://docs.basho.com/riak/latest/dev/references/http/delete-search-index/>. 
There is no issue with data base corruption when removing indexes or setting 
indexes to _dont_index_.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 30, 2015, at 3:39 PM, Jorge Garrido gomez  wrote:
> 
> Hello, 
> 
> On riak version 1.x when the search indexes are deleted then the data base is 
> corrupted, is the issue present on riak 2.x with YOKOZUNA and SOLR?
> 
> The question is because we want delete the objects indexed into Riak Search 
> 2.0
> 
> Thank you!
> Jorge Garrido 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search 2.0 & Tagging

[Zeeshan Lakhani]

Hello Antonio, 

You can insert an object and tag in the same operation, and you can query on 
that tag via Riak Search.

Before writing the object, just apply/set the metadata accordingly. Here’s an 
example in the Erlang client: 
https://github.com/basho/riak-erlang-client/blob/7487c90275c88dbe8ef4c2fed6540864364ca3d4/src/riakc_pb_socket.erl#L3348
 
<https://github.com/basho/riak-erlang-client/blob/7487c90275c88dbe8ef4c2fed6540864364ca3d4/src/riakc_pb_socket.erl#L3348>.
 

Also, this can be done via http: 
https://github.com/basho/yokozuna/blob/5868266b11f131d14c85495e50f899f3fe8158ba/riak_test/yokozuna_essential.erl#L281
 
<https://github.com/basho/yokozuna/blob/5868266b11f131d14c85495e50f899f3fe8158ba/riak_test/yokozuna_essential.erl#L281>.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 2, 2015, at 5:24 AM, Antonio Teixeira  wrote:
> 
> I've been using Riak as my main database for a few months, now I've been 
> experimenting with Riak Search 2.0 and for what I read in your documentation 
> there is no way to insert a object and tag it in the same operation.
> Right now we have an opaque object and we query them through secondary 
> indexes, this is becoming unbearable.
>  What we need is to store an object and 
> tag(https://github.com/basho/yokozuna/blob/develop/docs/TAGGING.md 
> <https://github.com/basho/yokozuna/blob/develop/docs/TAGGING.md>) it all in 
> one step.
> Our objects consist of erlang dictionaries and it would be a bit expensive 
> (performance wise) to convert the Dictionary to a list and then to json in 
> every database related operation.
> We are using Riak 2.0.5, Erlang 17.0 and the PB Driver.

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak Search 2.0 & Tagging

[Zeeshan Lakhani]

Hello Antonio,

Firstly, please always reply to the list and not via personal email.

In regards to your question, I wrote a test that showcases how to write objects 
with tags in pb for search: 
https://github.com/basho/yokozuna/pull/479/files#diff-f9f0e102b2a5208f41b2f304ada0ee5cR306
 
<https://github.com/basho/yokozuna/pull/479/files#diff-f9f0e102b2a5208f41b2f304ada0ee5cR306>.
 It’s a bit of a workaround, needing the ‘x-riak-meta*’, which gets stripped in 
preparation for query, but it does work.

Please make sure you’ve also created an index and associated it with a 
bucket_type/bucket, i.e. 
http://docs.basho.com/riak/latest/dev/using/search/#Simple-Setup 
<http://docs.basho.com/riak/latest/dev/using/search/#Simple-Setup>. To query, 
it’d be the same as in the docs (field:*query*): 
http://docs.basho.com/riak/latest/dev/using/search/#Querying 
<http://docs.basho.com/riak/latest/dev/using/search/#Querying>.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 7, 2015, at 4:28 AM, Antonio Teixeira  wrote:
> 
> Hi ,
> 
> I've been experimenting with with Riak Search and Tags but I've reached a 
> dead-end. I can't find any documentation on Querying Tags, I have followed 
> your example at: 
> https://github.com/basho/riak-erlang-client/blob/7487c90275c88dbe8ef4c2fed6540864364ca3d4/src/riakc_...
>  
> <https://github.com/basho/riak-erlang-client/blob/7487c90275c88dbe8ef4c2fed6540864364ca3d4/src/riakc_pb_socket.erl#L3348>
> Here is my Code:
> 
> O0 = riakc_obj:new(<<"test">>, <<"key0">>, <<"value0">>),
> MD0 = riakc_obj:get_update_metadata(O0),
> MD1 = riakc_obj:set_user_metadata_entry(MD0, {<<"ola_s">>,<<"nuno">>}),
> O1 = riakc_obj:update_metadata(O0, MD1),
> ok = riakc_pb_socket:put(Pid, O1).
> 
> My question is, how do I query this data without knowing the Key of the 
> Object.
> 
> Regards
> Antonio
> 
> 2015-04-02 16:10 GMT+01:00 Zeeshan Lakhani  <mailto:zlakh...@basho.com>>:
> Hello Antonio, 
> 
> You can insert an object and tag in the same operation, and you can query on 
> that tag via Riak Search.
> 
> Before writing the object, just apply/set the metadata accordingly. Here’s an 
> example in the Erlang client: 
> https://github.com/basho/riak-erlang-client/blob/7487c90275c88dbe8ef4c2fed6540864364ca3d4/src/riakc_pb_socket.erl#L3348
>  
> <https://github.com/basho/riak-erlang-client/blob/7487c90275c88dbe8ef4c2fed6540864364ca3d4/src/riakc_pb_socket.erl#L3348>.
>  
> 
> Also, this can be done via http: 
> https://github.com/basho/yokozuna/blob/5868266b11f131d14c85495e50f899f3fe8158ba/riak_test/yokozuna_essential.erl#L281
>  
> <https://github.com/basho/yokozuna/blob/5868266b11f131d14c85495e50f899f3fe8158ba/riak_test/yokozuna_essential.erl#L281>.
> 
> Thanks.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Apr 2, 2015, at 5:24 AM, Antonio Teixeira > <mailto:eagle.anto...@gmail.com>> wrote:
>> 
>> I've been using Riak as my main database for a few months, now I've been 
>> experimenting with Riak Search 2.0 and for what I read in your documentation 
>> there is no way to insert a object and tag it in the same operation.
>> Right now we have an opaque object and we query them through secondary 
>> indexes, this is becoming unbearable.
>>  What we need is to store an object and 
>> tag(https://github.com/basho/yokozuna/blob/develop/docs/TAGGING.md 
>> <https://github.com/basho/yokozuna/blob/develop/docs/TAGGING.md>) it all in 
>> one step.
>> Our objects consist of erlang dictionaries and it would be a bit expensive 
>> (performance wise) to convert the Dictionary to a list and then to json in 
>> every database related operation.
>> We are using Riak 2.0.5, Erlang 17.0 and the PB Driver.
> 
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Distributed joins and Riak search

[Zeeshan Lakhani]

Hello Karri,

That link is not describing SolrCloud, and Yokozuna does *not* use SolrCloud.

And, yes, those limitations do apply because we use Distributed Solr. For 
example, the join ticket is still open: 
https://issues.apache.org/jira/browse/LUCENE-3759 
<https://issues.apache.org/jira/browse/LUCENE-3759>.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 7, 2015, at 4:26 PM, Karri Niemelä  wrote:
> 
> Do same limitations apply for yokozuna and SolrCloud?
> (http://wiki.apache.org/solr/DistributedSearch#Distributed_Searching_Limitations
>  
> <http://wiki.apache.org/solr/DistributedSearch#Distributed_Searching_Limitations>,
>  meaning no joins …)
> 
> ?
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna queries slow

[Zeeshan Lakhani]

Hey Jason,

We’re working on performance issues with YZ filter queries, e.g. 
https://github.com/basho/yokozuna/issues/392 
<https://github.com/basho/yokozuna/issues/392>, and coverage plan 
generation/caching, and our CliServ team has started doing a ton of benchmarks 
as well.

You can bypass YZ, but then you’d have to create a way to generate your own 
coverage plans and other things involving distributed solr that YZ gives you. 
Nonetheless, we’re actively working on improving these issues you’ve 
encountered. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 21, 2015, at 1:06 AM, Jason Campbell  wrote:
> 
> Hello,
> 
> I'm currently trying to debug slow YZ queries, and I've narrowed down the 
> issue, but not sure how to solve it.
> 
> First off, we have about 80 million records in Riak (and YZ), but the queries 
> return relatively few (a thousand or so at most).  Our query times are 
> anywhere from 800ms to 1.5s.
> 
> I have been experimenting with queries directly on the Solr node, and it 
> seems to be a problem with YZ and the way it does vnode filters.
> 
> Here is the same query, emulating YZ first:
> 
> {
>  "responseHeader":{
>"status":0,
>"QTime":958,
>"params":{
>  "q":"timestamp:[1429579919010 TO 1429579921010]",
>  "indent":"true",
>  "fq":"_yz_pn:55 OR _yz_pn:40 OR _yz_pn:25 OR _yz_pn:10",
>  "rows":"0",
>  "wt":"json"}},
>  "response":{"numFound":80,"start":0,"docs":[]
>  }}
> 
> And the same query, but including the vnode filter in the main body instead 
> of using a filter query:
> 
> {
>  "responseHeader":{
>"status":0,
>"QTime":1,
>"params":{
>  "q":"timestamp:[1429579919010 TO 1429579921010] AND (_yz_pn:55 OR 
> _yz_pn:40 OR _yz_pn:25 OR _yz_pn:10)",
>  "indent":"true",
>  "rows":"0",
>  "wt":"json"}},
>  "response":{"numFound":80,"start":0,"docs":[]
>  }}
> 
> I understand there is a caching benefit to using filter queries, but a 
> performance difference of 100x or greater doesn't seem worth it, especially 
> with a constant data stream.
> 
> Is there a way to make YZ do this, or is the only way to query Solr directly, 
> bypassing YZ?  Does anyone have any other suggestions of how to make this 
> faster?
> 
> The timestamp field is a SolrTrieLongField with default settings if anyone is 
> curious.
> 
> Thanks,
> Jason
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna queries slow

[Zeeshan Lakhani]

No real workaround other than what you described or looking into 
config/fq-no-cache settings as mentioned in 
http://lucidworks.com/blog/advanced-filter-caching-in-solr/ 
<http://lucidworks.com/blog/advanced-filter-caching-in-solr/> and playing 
around with those.

Riak is now at 2.1.0. I hope that one of the next few point releases will see 
the fix. 


Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 21, 2015, at 4:11 AM, Jason Campbell  wrote:
> 
> Thanks Zeeshan for the info.
> 
> Is there a workaround in the mean time, or is the only option to handle 
> queries to the individual nodes ourselves?
> 
> Is there a planned timeframe for the 2.0.1 release?
> 
> Thanks,
> Jason
> 
>> On 21 Apr 2015, at 16:13, Zeeshan Lakhani  wrote:
>> 
>> Hey Jason,
>> 
>> We’re working on performance issues with YZ filter queries, e.g. 
>> https://github.com/basho/yokozuna/issues/392, and coverage plan 
>> generation/caching, and our CliServ team has started doing a ton of 
>> benchmarks as well.
>> 
>> You can bypass YZ, but then you’d have to create a way to generate your own 
>> coverage plans and other things involving distributed solr that YZ gives 
>> you. Nonetheless, we’re actively working on improving these issues you’ve 
>> encountered. 
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love | paperswelove.org
>> twitter => @zeeshanlakhani
>> 
>>> On Apr 21, 2015, at 1:06 AM, Jason Campbell  wrote:
>>> 
>>> Hello,
>>> 
>>> I'm currently trying to debug slow YZ queries, and I've narrowed down the 
>>> issue, but not sure how to solve it.
>>> 
>>> First off, we have about 80 million records in Riak (and YZ), but the 
>>> queries return relatively few (a thousand or so at most).  Our query times 
>>> are anywhere from 800ms to 1.5s.
>>> 
>>> I have been experimenting with queries directly on the Solr node, and it 
>>> seems to be a problem with YZ and the way it does vnode filters.
>>> 
>>> Here is the same query, emulating YZ first:
>>> 
>>> {
>>> "responseHeader":{
>>>   "status":0,
>>>   "QTime":958,
>>>   "params":{
>>> "q":"timestamp:[1429579919010 TO 1429579921010]",
>>> "indent":"true",
>>> "fq":"_yz_pn:55 OR _yz_pn:40 OR _yz_pn:25 OR _yz_pn:10",
>>> "rows":"0",
>>> "wt":"json"}},
>>> "response":{"numFound":80,"start":0,"docs":[]
>>> }}
>>> 
>>> And the same query, but including the vnode filter in the main body instead 
>>> of using a filter query:
>>> 
>>> {
>>> "responseHeader":{
>>>   "status":0,
>>>   "QTime":1,
>>>   "params":{
>>> "q":"timestamp:[1429579919010 TO 1429579921010] AND (_yz_pn:55 OR 
>>> _yz_pn:40 OR _yz_pn:25 OR _yz_pn:10)",
>>> "indent":"true",
>>> "rows":"0",
>>> "wt":"json"}},
>>> "response":{"numFound":80,"start":0,"docs":[]
>>> }}
>>> 
>>> I understand there is a caching benefit to using filter queries, but a 
>>> performance difference of 100x or greater doesn't seem worth it, especially 
>>> with a constant data stream.
>>> 
>>> Is there a way to make YZ do this, or is the only way to query Solr 
>>> directly, bypassing YZ?  Does anyone have any other suggestions of how to 
>>> make this faster?
>>> 
>>> The timestamp field is a SolrTrieLongField with default settings if anyone 
>>> is curious.
>>> 
>>> Thanks,
>>> Jason
>>> ___
>>> riak-users mailing list
>>> riak-users@lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> 
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Yokozuna queries slow

[Zeeshan Lakhani]

Nice Steve.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 21, 2015, at 3:57 PM, Steve Garon  wrote:
> 
> Jason, 
> 
> Comment the  section in the bucket's solrconfig.xml and 
> restart riak. Now your queries will be fast again :-)
> 
> 
> Steve
> 
> On 21 April 2015 at 04:24, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> No real workaround other than what you described or looking into 
> config/fq-no-cache settings as mentioned in 
> http://lucidworks.com/blog/advanced-filter-caching-in-solr/ 
> <http://lucidworks.com/blog/advanced-filter-caching-in-solr/> and playing 
> around with those.
> 
> Riak is now at 2.1.0. I hope that one of the next few point releases will see 
> the fix. 
> 
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Apr 21, 2015, at 4:11 AM, Jason Campbell > <mailto:xia...@xiaclo.net>> wrote:
>> 
>> Thanks Zeeshan for the info.
>> 
>> Is there a workaround in the mean time, or is the only option to handle 
>> queries to the individual nodes ourselves?
>> 
>> Is there a planned timeframe for the 2.0.1 release?
>> 
>> Thanks,
>> Jason
>> 
>>> On 21 Apr 2015, at 16:13, Zeeshan Lakhani >> <mailto:zlakh...@basho.com>> wrote:
>>> 
>>> Hey Jason,
>>> 
>>> We’re working on performance issues with YZ filter queries, e.g. 
>>> https://github.com/basho/yokozuna/issues/392 
>>> <https://github.com/basho/yokozuna/issues/392>, and coverage plan 
>>> generation/caching, and our CliServ team has started doing a ton of 
>>> benchmarks as well.
>>> 
>>> You can bypass YZ, but then you’d have to create a way to generate your own 
>>> coverage plans and other things involving distributed solr that YZ gives 
>>> you. Nonetheless, we’re actively working on improving these issues you’ve 
>>> encountered. 
>>> 
>>> Zeeshan Lakhani
>>> programmer | 
>>> software engineer at @basho | 
>>> org. member/founder of @papers_we_love | paperswelove.org 
>>> <http://paperswelove.org/>
>>> twitter => @zeeshanlakhani
>>> 
>>>> On Apr 21, 2015, at 1:06 AM, Jason Campbell >>> <mailto:xia...@xiaclo.net>> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> I'm currently trying to debug slow YZ queries, and I've narrowed down the 
>>>> issue, but not sure how to solve it.
>>>> 
>>>> First off, we have about 80 million records in Riak (and YZ), but the 
>>>> queries return relatively few (a thousand or so at most).  Our query times 
>>>> are anywhere from 800ms to 1.5s.
>>>> 
>>>> I have been experimenting with queries directly on the Solr node, and it 
>>>> seems to be a problem with YZ and the way it does vnode filters.
>>>> 
>>>> Here is the same query, emulating YZ first:
>>>> 
>>>> {
>>>> "responseHeader":{
>>>>   "status":0,
>>>>   "QTime":958,
>>>>   "params":{
>>>> "q":"timestamp:[1429579919010 TO 1429579921010]",
>>>> "indent":"true",
>>>> "fq":"_yz_pn:55 OR _yz_pn:40 OR _yz_pn:25 OR _yz_pn:10",
>>>> "rows":"0",
>>>> "wt":"json"}},
>>>> "response":{"numFound":80,"start":0,"docs":[]
>>>> }}
>>>> 
>>>> And the same query, but including the vnode filter in the main body 
>>>> instead of using a filter query:
>>>> 
>>>> {
>>>> "responseHeader":{
>>>>   "status":0,
>>>>   "QTime":1,
>>>>   "params":{
>>>> "q":"timestamp:[1429579919010 TO 1429579921010] AND (_yz_pn:55 OR 
>>>> _yz_pn:40 OR _yz_pn:25 OR _yz_pn:10)",
>>>> "indent":"true",
>>>> "rows":"0",
>>>> "wt":"json"}},
>>>> "response":{"numFound":80,"start":0,"docs":[]
>>>> }}
>&

Re: Yokozuna queries slow

[Zeeshan Lakhani]

Hey Steve,

Yep, that’s the specific reference point I was referring to in the link I 
shared. 

We’re working on cleaning-up some stuff to help perf for large indexes within 
the yz code itself. We’ve also discussed how to improve configuring specific 
solrconfig tunable items as a property of the index (not bucket); so, it’s 
something we’re looking into.

Thanks for helping out :).

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 21, 2015, at 5:41 PM, Steve Garon  wrote:
> 
> Zeeshan, 
> 
> For that specific case, you guys should add {!cache=false} in front on your 
> query plan. Therefore, queries on large index won't be slowed down. I'd 
> really like to see some of the solrconfig.xml config to be exported to the 
> riak bucket properties. The caching flag could be a property on the bucket. 
> Same for soft commit timeouts. We had to increase soft commit timeouts to 
> 10sec instead of the 1sec default. 
> 
> 
> Steve
> 
> On 21 April 2015 at 16:02, Zeeshan Lakhani  <mailto:zlakh...@basho.com>> wrote:
> Nice Steve.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org 
> <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Apr 21, 2015, at 3:57 PM, Steve Garon > <mailto:steve.ga...@gmail.com>> wrote:
>> 
>> Jason, 
>> 
>> Comment the  section in the bucket's solrconfig.xml and 
>> restart riak. Now your queries will be fast again :-)
>> 
>> 
>> Steve
>> 
>> On 21 April 2015 at 04:24, Zeeshan Lakhani > <mailto:zlakh...@basho.com>> wrote:
>> No real workaround other than what you described or looking into 
>> config/fq-no-cache settings as mentioned in 
>> http://lucidworks.com/blog/advanced-filter-caching-in-solr/ 
>> <http://lucidworks.com/blog/advanced-filter-caching-in-solr/> and playing 
>> around with those.
>> 
>> Riak is now at 2.1.0. I hope that one of the next few point releases will 
>> see the fix. 
>> 
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love | paperswelove.org 
>> <http://paperswelove.org/>
>> twitter => @zeeshanlakhani
>> 
>>> On Apr 21, 2015, at 4:11 AM, Jason Campbell >> <mailto:xia...@xiaclo.net>> wrote:
>>> 
>>> Thanks Zeeshan for the info.
>>> 
>>> Is there a workaround in the mean time, or is the only option to handle 
>>> queries to the individual nodes ourselves?
>>> 
>>> Is there a planned timeframe for the 2.0.1 release?
>>> 
>>> Thanks,
>>> Jason
>>> 
>>>> On 21 Apr 2015, at 16:13, Zeeshan Lakhani >>> <mailto:zlakh...@basho.com>> wrote:
>>>> 
>>>> Hey Jason,
>>>> 
>>>> We’re working on performance issues with YZ filter queries, e.g. 
>>>> https://github.com/basho/yokozuna/issues/392 
>>>> <https://github.com/basho/yokozuna/issues/392>, and coverage plan 
>>>> generation/caching, and our CliServ team has started doing a ton of 
>>>> benchmarks as well.
>>>> 
>>>> You can bypass YZ, but then you’d have to create a way to generate your 
>>>> own coverage plans and other things involving distributed solr that YZ 
>>>> gives you. Nonetheless, we’re actively working on improving these issues 
>>>> you’ve encountered. 
>>>> 
>>>> Zeeshan Lakhani
>>>> programmer | 
>>>> software engineer at @basho | 
>>>> org. member/founder of @papers_we_love | paperswelove.org 
>>>> <http://paperswelove.org/>
>>>> twitter => @zeeshanlakhani
>>>> 
>>>>> On Apr 21, 2015, at 1:06 AM, Jason Campbell >>>> <mailto:xia...@xiaclo.net>> wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> I'm currently trying to debug slow YZ queries, and I've narrowed down the 
>>>>> issue, but not sure how to solve it.
>>>>> 
>>>>> First off, we have about 80 million records in Riak (and YZ), but the 
>>>>> queries return relatively few (a thousand or so at most).  Our query 
>>>>> times are anywhere from 800ms to 1.5s.
>>>>> 
>>>>> I have been experimenting with queries directly on the Solr node, and it 
>>>>> seems to be a

Re: Reindexing solr after backup restore

[Zeeshan Lakhani]

Hey Jason,

Here’s a little more discussion on Yokozuna backup strategies: 
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2014-January/014514.html
 
<http://lists.basho.com/pipermail/riak-users_lists.basho.com/2014-January/014514.html>.

Nonetheless, I wouldn’t say the behavior’s expected, but we’re going to be 
adding more to the docs on how to rebuild indexes.

To do so, you could just remove the yz_anti_entropy directory, and make AAE 
more aggressive, via

```
rpc:multicall([node() | nodes()], application, set_env, [yokozuna, 
anti_entropy_build_limit, {100, 1000}]).
rpc:multicall([node() | nodes()], application, set_env, [yokozuna, 
anti_entropy_concurrency, 4])
```

and the indexes will rebuild. You can try to initialize the building of trees 
with `yz_entropy_mgr:init([])` via `riak attach`, but a restart would also kick 
AAE into gear. There’s a bit more related info on this thread: 
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016929.html
 
<http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016929.html>.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 24, 2015, at 1:34 AM, Jason Campbell  wrote:
> 
> I think I figured it out.
> 
> I followed this guide: 
> http://docs.basho.com/riak/latest/ops/running/nodes/renaming/#Clusters-from-Backups
> 
> The first Riak node (changed with riak-admin reip) kept it's Solr index.  
> However, the other nodes when joined via riak-admin cluster force-replace, 
> dropped their Solr indexes.
> 
> Is this expected?  If so, it should really be in the docs, and there should 
> be another way to restore a cluster keeping Solr intact.
> 
> Also, is there a way to rebuild a Solr index?
> 
> Thanks,
> Jason
> 
>> On 24 Apr 2015, at 15:16, Jason Campbell  wrote:
>> 
>> I've just done a backup and restore of our production Riak cluster, and 
>> Yokozuna has dropped from around 125 million records to 25million.  
>> Obviously the IPs have changed, and although the Riak cluster is stable, I'm 
>> not sure Solr handled the transition as nicely.
>> 
>> Is there a way to force Solr to rebuild the indexes, or at least get back to 
>> the state it was in before the backup?
>> 
>> Also, is this expected behaviour?
>> 
>> Thanks,
>> Jason
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Reindexing solr after backup restore

[Zeeshan Lakhani]

Hey Jason,

Yeah, nodes can normally be joined without a cluster dropping its Solr Index 
and AAE normally rebuilds the missing KV bits.

In the case of restoring from a backup and having missing data, we can only 
recommend a reindex (the indexes that have the issue) with aggressive AAE 
settings to speed things up. It can be pretty fast. Recreating indexes are 
cheap in Yokozuna, but are the `data/yz` directories missing from the nodes 
that were force-replaced? Unless someone else wants to chime in, I’ll gather 
more info on what occurred from the reip vs the force-replace. 

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Apr 24, 2015, at 7:02 PM, Jason Campbell  wrote:
> 
> Is there a way to do a restore without rebuilding these indexes though?  
> Obviously this could take a long time depending on the amount of indexed data 
> in the cluster.  It's a fairly big gotcha to say that Yokozuna fixes a lot of 
> the data access issues that Riak has, but if you restore from a backup, it 
> could be useless for days or weeks.
> 
> As far as disk consistency, the nodes were stopped during the snapshot, so 
> I'm assuming on-disk it would be consistent within a single node.  And 
> cluster wide, I would expect the overall data to fall somewhere between the 
> first and last node snapshot.  AAE should still repair the bits left over, 
> but it shouldn't have to rebuild the entire Solr index.
> 
> So the heart of the question can I join a node to a cluster without dropping 
> it's Solr index?  force-replace obviously doesn't work, what is the harm in 
> running reip on every node instead of just the first?
> 
> Thanks for the help,
> Jason
> 
>> On 25 Apr 2015, at 00:36, Zeeshan Lakhani  wrote:
>> 
>> Hey Jason,
>> 
>> Here’s a little more discussion on Yokozuna backup strategies: 
>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2014-January/014514.html.
>> 
>> Nonetheless, I wouldn’t say the behavior’s expected, but we’re going to be 
>> adding more to the docs on how to rebuild indexes.
>> 
>> To do so, you could just remove the yz_anti_entropy directory, and make AAE 
>> more aggressive, via
>> 
>> ```
>> rpc:multicall([node() | nodes()], application, set_env, [yokozuna, 
>> anti_entropy_build_limit, {100, 1000}]).
>> rpc:multicall([node() | nodes()], application, set_env, [yokozuna, 
>> anti_entropy_concurrency, 4])
>> ```
>> 
>> and the indexes will rebuild. You can try to initialize the building of 
>> trees with `yz_entropy_mgr:init([])` via `riak attach`, but a restart would 
>> also kick AAE into gear. There’s a bit more related info on this thread: 
>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016929.html.
>> 
>> Thanks.
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love | paperswelove.org
>> twitter => @zeeshanlakhani
>> 
>>> On Apr 24, 2015, at 1:34 AM, Jason Campbell  wrote:
>>> 
>>> I think I figured it out.
>>> 
>>> I followed this guide: 
>>> http://docs.basho.com/riak/latest/ops/running/nodes/renaming/#Clusters-from-Backups
>>> 
>>> The first Riak node (changed with riak-admin reip) kept it's Solr index.  
>>> However, the other nodes when joined via riak-admin cluster force-replace, 
>>> dropped their Solr indexes.
>>> 
>>> Is this expected?  If so, it should really be in the docs, and there should 
>>> be another way to restore a cluster keeping Solr intact.
>>> 
>>> Also, is there a way to rebuild a Solr index?
>>> 
>>> Thanks,
>>> Jason
>>> 
>>>> On 24 Apr 2015, at 15:16, Jason Campbell  wrote:
>>>> 
>>>> I've just done a backup and restore of our production Riak cluster, and 
>>>> Yokozuna has dropped from around 125 million records to 25million.  
>>>> Obviously the IPs have changed, and although the Riak cluster is stable, 
>>>> I'm not sure Solr handled the transition as nicely.
>>>> 
>>>> Is there a way to force Solr to rebuild the indexes, or at least get back 
>>>> to the state it was in before the backup?
>>>> 
>>>> Also, is this expected behaviour?
>>>> 
>>>> Thanks,
>>>> Jason
>>>> ___
>>>> riak-users mailing list
>>>> riak-users@lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> 
>>> 
>>> ___
>>> riak-users mailing list
>>> riak-users@lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> 
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Reindexing solr after backup restore

[Zeeshan Lakhani]

Awesome. Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love |
twitter => @zeeshanlakhani

> On Apr 24, 2015, at 8:56 PM, Jason Campbell  wrote:
> 
> This may be a case of force-replace vs replace vs reip.  I'm happy to see if 
> I can get new cluster from backup to keep the Solr indexes.
> 
> The disk backup was all of /var/lib/riak, so definitely included the YZ 
> indexes before the force-replace, and they were kept on the first node that 
> was changed with reip.  I stopped each node before the snapshot to ensure 
> consistency.  So I would expect the final restored cluster to be somewhere 
> between the first and last node snapshot in terms of data, and AAE to repair 
> things to a consistent state for that few minute gap.
> 
> I'll experiment with different methods of rebuilding the cluster on Monday 
> and see if I can get it to keep the Solr indexes.  Maybe moving the YZ 
> indexes out of the way during the force-replace, then stopping the node and 
> putting them back could help as well.  I'll let you know the results of the 
> experiments either way.
> 
> Thanks,
> Jason
> 
>> On 25 Apr 2015, at 09:25, Zeeshan Lakhani  wrote:
>> 
>> Hey Jason,
>> 
>> Yeah, nodes can normally be joined without a cluster dropping its Solr Index 
>> and AAE normally rebuilds the missing KV bits.
>> 
>> In the case of restoring from a backup and having missing data, we can only 
>> recommend a reindex (the indexes that have the issue) with aggressive AAE 
>> settings to speed things up. It can be pretty fast. Recreating indexes are 
>> cheap in Yokozuna, but are the `data/yz` directories missing from the nodes 
>> that were force-replaced? Unless someone else wants to chime in, I’ll gather 
>> more info on what occurred from the reip vs the force-replace. 
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love | paperswelove.org
>> twitter => @zeeshanlakhani
>> 
>>> On Apr 24, 2015, at 7:02 PM, Jason Campbell  wrote:
>>> 
>>> Is there a way to do a restore without rebuilding these indexes though?  
>>> Obviously this could take a long time depending on the amount of indexed 
>>> data in the cluster.  It's a fairly big gotcha to say that Yokozuna fixes a 
>>> lot of the data access issues that Riak has, but if you restore from a 
>>> backup, it could be useless for days or weeks.
>>> 
>>> As far as disk consistency, the nodes were stopped during the snapshot, so 
>>> I'm assuming on-disk it would be consistent within a single node.  And 
>>> cluster wide, I would expect the overall data to fall somewhere between the 
>>> first and last node snapshot.  AAE should still repair the bits left over, 
>>> but it shouldn't have to rebuild the entire Solr index.
>>> 
>>> So the heart of the question can I join a node to a cluster without 
>>> dropping it's Solr index?  force-replace obviously doesn't work, what is 
>>> the harm in running reip on every node instead of just the first?
>>> 
>>> Thanks for the help,
>>> Jason
>>> 
>>>> On 25 Apr 2015, at 00:36, Zeeshan Lakhani  wrote:
>>>> 
>>>> Hey Jason,
>>>> 
>>>> Here’s a little more discussion on Yokozuna backup strategies: 
>>>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2014-January/014514.html.
>>>> 
>>>> Nonetheless, I wouldn’t say the behavior’s expected, but we’re going to be 
>>>> adding more to the docs on how to rebuild indexes.
>>>> 
>>>> To do so, you could just remove the yz_anti_entropy directory, and make 
>>>> AAE more aggressive, via
>>>> 
>>>> ```
>>>> rpc:multicall([node() | nodes()], application, set_env, [yokozuna, 
>>>> anti_entropy_build_limit, {100, 1000}]).
>>>> rpc:multicall([node() | nodes()], application, set_env, [yokozuna, 
>>>> anti_entropy_concurrency, 4])
>>>> ```
>>>> 
>>>> and the indexes will rebuild. You can try to initialize the building of 
>>>> trees with `yz_entropy_mgr:init([])` via `riak attach`, but a restart 
>>>> would also kick AAE into gear. There’s a bit more related info on this 
>>>> thread: 
>>>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-March/016929.html.
>>>> 
>>>> Thanks.
>>>> 
>>>> Zeeshan Lakhani
>>>>

Re: [Riak-Users]Why riak search only return 1 result every query, how to return all results at once

[Zeeshan Lakhani]

To include with Chris’s questions, are you using a custom or the “default” 
bucket type?

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On May 5, 2015, at 4:22 AM, Christopher Meiklejohn  
> wrote:
> 
>> 
>> On Apr 21, 2015, at 4:07 PM, 寂寂 <44290...@qq.com> wrote:
>> 
>> And I create the bucket type by myself:
>> 
>> riak-admin bucket-type create video '{"props":{"search_index":"videos"}}'
>> riak-admin bucket-type activate vide
>> 
>> 
>> -- Original --
>> From:  "寂寂 ";<44290...@qq.com>;
>> Send time: Tuesday, Apr 21, 2015 9:49 PM
>> To: "riak-users";
>> Subject:  [Riak-Users]Why riak search only return 1 result every query,how 
>> to return all results at once
>> 
>> Dear All,
>> 
>> I have many objects with the same "tag:hot" stored in Riak cluster with two 
>> nodes, all with "leveldb" backend.
>> 
>> When I try to search with the same query condition, I just got one result 
>> returned, and everytime with different result:
>> curl -v "http://192.168.1.102:10028/search/query/videos?wt=json&q=tag:hot";
>> {"responseHeader":{"status":0,"QTime":18,"params":{"shards":"127.0.0.1:10014/internal_solr/videos,127.0.0.1:10024/internal_solr/videos","q":"tag:hot","127.0.0.1:10014":"_yz_pn:64
>>  OR (_yz_pn:61 AND (_yz_fpn:61)) OR _yz_pn:60 OR _yz_pn:57 OR _yz_pn:48 OR 
>> _yz_pn:45 OR _yz_pn:36 OR _yz_pn:33 OR _yz_pn:24 OR _yz_pn:21 OR _yz_pn:12 
>> OR _yz_pn:9","wt":"json","127.0.0.1:10024":"_yz_pn:54 OR _yz_pn:51 OR 
>> _yz_pn:42 OR _yz_pn:39 OR _yz_pn:30 OR _yz_pn:27 OR _yz_pn:18 OR _yz_pn:15 
>> OR _yz_pn:6 OR 
>> _yz_pn:3"}},"response":{"numFound":1,"start":0,"maxScore":0.71231794,"docs":[{"title":"Love
>>  
>> Story","code":"8vBcw6rJCS6D6til2u","_yz_id":"1*video*movie*8vBcw6rJCS6D6til2u*48","_yz_rk":"8vBcw6rJCS6D6til2u","_yz_rt":"video","_yz_rb":"movie"}]}}
>> 
>> 
>> curl -v "http://192.168.1.102:10028/search/query/videos?wt=json&q=tag:hot";
>> {"responseHeader":{"status":0,"QTime":14,"params":{"shards":"127.0.0.1:10014/internal_solr/videos,127.0.0.1:10024/internal_solr/videos","q":"tag:hot","127.0.0.1:10014":"(_yz_pn:60
>>  AND (_yz_fpn:60)) OR _yz_pn:56 OR _yz_pn:53 OR _yz_pn:44 OR _yz_pn:41 OR 
>> _yz_pn:32 OR _yz_pn:29 OR _yz_pn:20 OR _yz_pn:17 OR _yz_pn:8 OR 
>> _yz_pn:5","wt":"json","127.0.0.1:10024":"_yz_pn:63 OR _yz_pn:59 OR _yz_pn:50 
>> OR _yz_pn:47 OR _yz_pn:38 OR _yz_pn:35 OR _yz_pn:26 OR _yz_pn:23 OR 
>> _yz_pn:14 OR _yz_pn:11 OR 
>> _yz_pn:2"}},"response":{"numFound":1,"start":0,"maxScore":0.71231794,"docs":[{"title":"Forest
>>  
>> Gan","code":"8vBhSeP2Q2hCEEGmsC","_yz_id":"1*video*movie*8vBhSeP2Q2hCEEGmsC*20","_yz_rk":"8vBhSeP2Q2hCEEGmsC","_yz_rt":"video","_yz_rb":"movie"}]}}
>> 
>> How can I get all at once query?
> 
> Hi there,
> 
> Can you clarify how you are writing the values?  Are you writing with a w=1 
> or using the defaults?  Given the index delay on Riak Search, subsequent 
> queries may return different values given on when you query and if you are 
> using majority write quorums.
> 
> When are you writing the values and how are you performing the writes?
> 
> Thanks,
> Chris
> 
> Christopher Meiklejohn
> Senior Software Engineer
> Basho Technologies, Inc.
> cmeiklej...@basho.com <mailto:cmeiklej...@basho.com>
> 
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com 
> <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Solr indexes becomes broken after node replacement

[Zeeshan Lakhani]

Hello Alexander,

Can you send us the logs? I’d like to see the errors that occurred after the 
time of the node name change.

In terms of getting your data back into the indexes, I’d recommend letting AAE 
handle the situation by removing the `yz_anti_entropy` directory and letting 
the keys re-index. You can even remove the search index directories, e.g. 
https://github.com/basho/riak_test/blob/master/tests/yz_core_properties_create_unload.erl#L94
 
<https://github.com/basho/riak_test/blob/master/tests/yz_core_properties_create_unload.erl#L94>,
  and let the them get recreated, which will still re-index via AAE. 

Nonetheless, we’d like to see the logs and try and reproduce the issue in a 
test when the node name changes.

Thanks.

Zeeshan Lakhani
programmer | 
software engineer at @basho

> On Jun 8, 2015, at 8:30 AM, Alexander Popov  wrote:
> 
> I was perform all nodes upgrade  in x6 nodes cluster
> and at same time  4 of them was prooceed with node name changed by steps:
> http://docs.basho.com/riak/latest/ops/running/recovery/failed-node/#Node-Name-Changed
>  
> <http://docs.basho.com/riak/latest/ops/running/recovery/failed-node/#Node-Name-Changed>
> 
> after that search indexes becomes completely broken - many data missed,  
> returns different results on every requests.
> 
> Does it have any options now except re-save  entire data in bucket to fix it?
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com