Hi All,
I would like to store ephemeral objects in Riak, while retaining the
ability to search among them before they expire. So far, I had no
luck.
When I query the cache with the ephemeral objects, Riak Search returns
the expired entries among the matches. AFAICT, it correctly indexes
the entries at pre-commit time but don't update the index at key
expiration time.
I've tried rebuilding the index but, sadly, it didn't helped and, even
if it would, the rebuild frequency would make the workaround too slow
for any short lived data.
Am I missing something? Probable. Any clues?
Here is what I have done, so you can try to reproduce this behaviour
-- running Riak 1.2.0 on Ubuntu 12.04.1 LTS:
1. Configured Riak to use multi_backend to have both expiring and
normal persistent data
{storage_backend, riak_kv_multi_backend},
{multi_backend_default, <<"default">>},
{multi_backend, [
{<<"default">>, riak_kv_bitcask_backend, []},
{<<"cache">>, riak_kv_bitcask_backend,
[{expiry_secs, 60}]}
]},
2. Enabled search functionality, setting riak_search to 'true'
3. Set up index for the cache bucket
$ search-cmd install cache
4. Cached some JSON encoded data
$ curl -v -X PUT -H "Content-Type: application/json" --data-binary
@./data http://localhost:8098/riak/cache/key1
$ curl -v -X PUT -H "Content-Type: application/json" --data-binary
@./data http://localhost:8098/riak/cache/key2
5. Waited the expiration of the first key
6. Searched
$ sudo search-cmd search cache "name:\"Alyssa\""
Attempting to restart script through sudo -H -u riak
:: Searching for 'name:"Alyssa"' / '' in cache...
------------------------------
index/id: cache/key1
p -> [0]
score -> 0.35355339059327373
------------------------------
index/id: cache/key2
p -> [0]
score -> 0.35355339059327373
------------------------------
:: Found 2 results.
7. Retrieved the keys
$ curl http://localhost:8098/riak/cache/key1
not found
$ curl http://localhost:8098/riak/cache/key2
{
"name":"Alyssa P. Hacker",
"bio":"I'm an engineer, making awesome things.",
"favorites":{
"book":"The Moon is a Harsh Mistress",
"album":"Magical Mystery Tour"
}
}
8. Tried the Solr-like interface -- it reports the presence of two
matching keys and return (of course) only the one not expired yet
$ curl
"http://localhost:8098/solr/cache/select?start=0&rows=10000&q=name:\"Alyssa\"";
...
<result name="response" numFound="2" start="0" maxScore="0.353553">
<doc>
<str name="id">key2
</str>
<str name="bio">I'm an engineer, making awesome things.
</str>
<str name="favorites_album">Magical Mystery Tour
</str>
<str name="favorites_book">The Moon is a Harsh Mistress
</str>
<str name="name">Alyssa P. Hacker
</str>
</doc>
</result>
9. Rebuild the index
$ riak attach
{ok, Ring} = riak_core_ring_manager:get_my_ring().
Partitions = [P || {P, '[email protected]'} <-
riak_core_ring:all_owners(Ring)].
[riak_kv_vnode:repair(P) || P <- Partitions].
10. Searched again, getting the same results as above
Thank you.
Cheers,
