I apologize for the confusion and defer to Luke on all the technical Riak things.
On Fri, Mar 10, 2017 at 11:16 Luke Bakken <lbak...@basho.com> wrote: > Just to clarify ... > > What Alexander is suggesting is what Daniel is currently using, and > what I suspect may be causing Daniel's issues. > > If you wish to run a leveldb-only Riak CS cluster, you still *must* > use the advanced.config file and the riak_cs_kv_multi_backend, and the > other settings that I mention in my response and in the docs. Notice > the multi_backend_prefix_list setting, for one thing. > > Daniel - > > The storage_backend setting in advanced.config will *override* > storage_backend in riak.conf. If you wish to ensure the riak.conf > setting is overridden, you may comment it out in that file. > > -- > Luke Bakken > Engineer > lbak...@basho.com > > On Fri, Mar 10, 2017 at 9:08 AM, Alexander Sicular <sicul...@basho.com> > wrote: > > > > Hi Daniel, > > > > Riak CS uses multi by default. By default the manifests are stored in > leveldb and the blobs/chunks are stored in bitcask. If you're looking to > force everything to level you should remove multi and use level as the > backend setting. As Luke noted elsewhere, this configuration hasn't been > fully tested and is not supported. > > > > Off the top of my head, take a look at the email Martin (?) sent about > his modified level backend a few weeks ago for reasons why using level for > data chunks may not be the best idea at this time. > > > > Thanks, > > Alexander > > > > @siculars > > http://siculars.posthaven.com > > > > Sent from my iRotaryPhone > > > > On Mar 10, 2017, at 10:50, Daniel Miller <dmil...@dimagi.com> wrote: > > > > Hi Luke, > > > > Again, thanks for your help. We are currently preparing to move all > objects into a new cluster using the S3 API. One question on configuration: > currently I have "storage_backend = leveldb" in my riak.conf. I assume that > on the new cluster, in addition to using the advanced.config you provided, > I also need to set "storage_backend = multi" in riak.conf – is that correct? > > > > Referring back to the subject of this thread for a bit, I'm assuming > your current theory for why the (most recent) object went missing is > because we have a bad backend configuration. Note that that object went > missing weeks after it was originally written into riak, and it was > successfully retrieved many times before it went missing. Is there a way I > can query riak to verify your theory that the manifest was overwritten? > Russel Brown suggested: "I wonder if you can get the manifest and then see > if any/all of the chunks are present?" Would that help to answer the > question about why the object went missing? Can you provide any hints on > how to do that? > > > > While bad configuration may be the cause of this most recent object > going missing, it does not explain the original two objects that went > missing immediately after they were PUT. Those original incidents happened > when our cluster was still using bitcask/mutli backend, so should not have > been affected by bad configuration. > > > > ~ Daniel > > > > On Tue, Mar 7, 2017 at 3:58 PM, Luke Bakken <lbak...@basho.com> wrote: > >> > >> Hi Daniel, > >> > >> Thanks for providing all of that information. > >> > >> You are missing important configuration for riak_kv that can only be > provided in an /etc/riak/advanced.config file. Please see the following > document, especially the section to which I link here: > >> > >> > http://docs.basho.com/riak/cs/2.1.1/cookbooks/configuration/riak-for-cs/#setting-up-the-proper-riak-backend > >> > >> [ > >> {riak_kv, [ > >> % NOTE: double-check this path for your environment: > >> {add_paths, ["/usr/lib/riak-cs/lib/riak_cs-2.1.1/ebin"]}, > >> {storage_backend, riak_cs_kv_multi_backend}, > >> {multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]}, > >> {multi_backend_default, be_default}, > >> {multi_backend, [ > >> {be_default, riak_kv_eleveldb_backend, [ > >> {data_root, "/opt/data/ecryptfs/riak"} > >> ]}, > >> {be_blocks, riak_kv_eleveldb_backend, [ > >> {data_root, "/opt/data/ecryptfs/riak_blocks"} > >> ]} > >> ]} > >> ]} > >> ]. > >> > >> Your configuration will look like the above. The contents of this file > are merged with the contents of /etc/riak/riak.conf to produce the > configuration that Riak uses. > >> > >> Notice that I chose riak_kv_eleveldb_backend twice because of the > discussion you had previously about RAM usage and bitcask ( > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-November/018801.html > ) > >> > >> In your current configuration, you are not using the expected prefix > for the block data. My guess is that on very rare occasions your data > happens to overwrite the manifest for a file. You may also have corrupted > files at this point without noticing it at all. > >> > >> IMPORTANT: you can't switch from your current configuration to this new > one without re-saving all of your data. > >> > >> -- > >> Luke Bakken > >> Engineer > >> lbak...@basho.com > >> > >> -- > >> Luke Bakken > >> Engineer > >> lbak...@basho.com > >> > >> On Tue, Mar 7, 2017 at 6:47 AM, Daniel Miller <dmil...@dimagi.com> > wrote: > >>> > >>> Responses inline. > >>> > >>> On Mon, Mar 6, 2017 at 3:04 PM, Luke Bakken <lbak...@basho.com> wrote: > >>>> > >>>> Hi Daniel, > >>>> > >>>> Two questions: > >>>> > >>>> * Do you happen to have an /etc/riak/app.config file present? > >>> > >>> > >>> No. > >>> > >>> Not sure if relevant, but I did notice that > /etc/riak-cs/advanced.config does exist, which contradicts with what I said > earlier. This is surprising to me because I did not create this file. Maybe > it was created by the riak installer? Anyway, the content is: > >>> > >>> $ cat /etc/riak-cs/advanced.config > >>> [ > >>> {riak_cs, > >>> [ > >>> ]} > >>> ]. > >>> > >>>> > >>>> > >>>> * On one of your Riak nodes, could you please execute the following > commands: > >>>> > >>>> riak attach > >>>> rp(application:get_all_env(riak_kv)). > >>>> > >>>> Copy the output of the previous command and attach as a separate file > >>>> to your response. Please note that the period is significant. Use > >>>> CTRL-C CTRL-C to exit the "riak attach" session. > >>> > >>> > >>> Attached. > >>> > >>> > >> > > > > _______________________________________________ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > -- Alexander Sicular Solutions Architect Basho Technologies 9175130679 @siculars
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
