Re: Erlang Driver Error
Hi, Satish. OS X's implementation of poll(3) has a known limitation of 1024 file descriptors, if I recall correctly. If you don't have "+K true" in vm.args (or cuttlefish equivalent, sorry, I don't have it handy right now), then please use that flag to bypass some libc (?) brokenness. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: my cluster spontaneously loses a node after ~48hrs
Hi, Jason. Have you tried using the system inspection utilities bundled with Riak? http://docs.basho.com/riak/latest/ops/running/tools/riak-admin/#top http://docs.basho.com/riak/latest/ops/running/tools/riak-admin/#cluster-info http://docs.basho.com/riak/latest/ops/upgrading/production-checklist/#Confirming-Configuration-with-Riaknostic The top utility can show very quickly the most active processes within the virtual machine. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Erlang crash dump viewer with Riak crash dumps
Hi, Bryan, sorry to jump in so late. Have you tried this? webtool:start(). Webtool has a non-native-GUI version of the CrashDumpViewer: load http://127.0.0.1:/ in a local web browser, then start the CrashDumpViewer, then load the dump file, then browse. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: bitcask io mode
Oleksiy Krivoshey oleks...@gmail.com wrote: ok Thanks! What kind of negative impact can the NIF mode cause on ok Bitcask? Please see the Long-running NIFs section of http://www.erlang.org/doc/man/erl_nif.html ... and NIF execution blocks the Erlang scheduler thread until the NIF is finished. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: bitcask io mode
Oops, sorry, I overlooked your question. The erlang/nif I/O setting affect all Bitcask file I/O, see https://github.com/basho/bitcask/blob/develop/src/bitcask_io.erl -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Reclaim disk space occupied by deleted keys
raghwani sohil sohil4...@gmail.com wrote: rs what is meaning of keys that are no longer the latest value ? Keys that have been overwritten with a newer value. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Reclaim disk space occupied by deleted keys
raghwani sohil sohil4...@gmail.com wrote: rs It means key k1 and k3 becomes dead keys Right ? Correct. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak errors after node left the cluster
Sargun Dhillon sar...@sargun.me wrote: sd Can you run: [...] Hi, Sargun and Oleksiy. Those commands and a lot more are run as part of the suite of info-gathering done by the riak-debug utility. I recommend using it instead of managing a hodge-podge of separate commands. The output from riak-admin cluster-info is also exceptionally helpful, especially because it contains even more diagnostic information, especially about Erlang process mailbox contents. I recommend running it during overload conditions to see what's going on internally. Also, riak-admin top -sort msg_q can give a real-time view of Erlang mailbox sizes, sorted by mailbox size. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Merge error in bitcask data store
Tom Santero tsant...@gmail.com wrote: ts Can we expect a 2.0.1 release forthcoming? I was planning on ts upgrading Riak today to 2.0, but since I use Bticask I'm affected by ts this. I'd prefer not to upgrade to 2.0 in production until this is ts patched and packaged in a proper release. Hiya, Tom. I'm not sure of the exact dates, but Riak 2.0.x releases will be running on a roughly monthly schedule until it reaches LTS (long term support) status and then patched on a critical-as-needed basis afterward. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Merge error in bitcask data store
Tom Santero tsant...@gmail.com wrote: ts Can we expect a 2.0.1 release forthcoming? I was planning on ts upgrading Riak today to 2.0, but since I use Bticask I'm affected by ts this. I'd prefer not to upgrade to 2.0 in production until this is ts patched and packaged in a proper release. Hiya, Tom. I'm not sure of the exact dates, but Riak 2.0.x releases will be running on a roughly monthly schedule until it reaches LTS (long term support) status and then patched on a critical-as-needed basis afterward. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Cleaning up bucket after basho_bench run
Toby Corkindale t...@dryft.net wrote: tc So basho_bench creates keys that tc can't be manipulated by anything except erlang :/ You can delete tc them if you write some erlang though.. -Toby You're correct, though you can also configure basho_bench to string-ify the integers that are used as keys. Use either the 'int_to_str' or 'base64' key generators. https://github.com/basho/basho_bench/blob/master/src/basho_bench_keygen.erl#L50 https://github.com/basho/basho_bench/blob/master/src/basho_bench_keygen.erl#L56 -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: erlang gen_server connecting to riak
Hi, Mahesh. I didn't see a reply to your question, but my email server was down for a while, so perhaps I missed it. There are several ways to manipulate Erlang/OTP's code server's configuration (similar in spirit to the Java CLASSPATH). See the -pz (recommended) and -pa flags in http://www.erlang.org/doc/man/erl.html -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Update to 1.4.8
Edgar Veiga edgarmve...@gmail.com wrote: ev Is this normal? Yes. One or more of your vnodes can't keep up with the workload generated by AAE repair or a vnode can't keep up for another reason, and AAE repair shouldn't actively make things worse. The logging is done by: https://github.com/basho/riak_kv/blob/1.4/src/riak_kv_entropy_manager.erl#L821 -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Caching in RIAK CS
Andre Lohmann lohmann.an...@gmail.com wrote: al When Hitting Server 1 again for the same file, will it be fetched al again piece by piece from the other servers, or is there some kind al of caching (prefered in memory) available, to deliver this file from al cache the next time? Riak CS does not have such a caching feature at this time. Today, the cache resides in the OS page cache of the Riak servers. Adding a feature to cache extremely hot objects is intended but has not yet been scheduled on Basho's roadmap. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: basho_bench's http raw client constantly crashes
Denis Zhdanov denis.zhda...@gmail.com wrote: dz Hi All, Could someone answer is basho_bench's dz basho_bench_driver_http_raw driver working or not now? Hi. Yes, it works fine ... but does not support the 'delete' operation that you're trying to use in your config's 'operation' list. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak benchmarking
Kevin Burton rkevinbur...@charter.net wrote: kb I replaced the 127.0.0.1 with the address for my cluster (I have 4 kb nodes in the cluster). I get an error: `[info] can't set long node kb name Please check configuration' followed by a fatal error [error] kb Failed to start net kernel. Kevin, what arguments did you use when running basho_bench? The -N argument requires a long node name: a short name like foo@bar is not sufficient. http://www.snookles.com/erlang-docs/R15B03/doc/reference_manual/distributed.html Translation: the RHS of the node name must have a dot in it. :-) -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: what about millions of buckets
Wu Ray mjollnir@gmail.com wrote: wr Thanks. what's more, I want to know if there is best practice for wr performance in this special situation, such as avoiding to do wr something, etc. With Riak right now, the best choice is to avoid arbitrary bucket names and use a small (or one) bucket that can support the bucket properties that you need (e.g., secondary indexes). If you don't mind writing a little Erlang code, there's nothing that says that bucket properties must be specified by exact bucket name. A modest amount of modfied code could apply certain properties to all buckets that had FOO2I as a prefix. The simplest would be to hardcode those properties, but it would be quite easily to use the FOO2I bucket's properties in the ring (using the existing Riak code) and apply those props to all FOO2I-prefixed keys. Sorry that prefixing scheme isn't baked-in today. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: How to tell when all riak_core vnodes are up?
Will Hutchinson w...@trueex.com wrote: wh Hi, we are using riak_core and our own custom vnode type (call it wh foo) that implements the riak_core_vnode behaviour. [...] Will, have you tried riak_core:wait_for_service(Service)? Note that that function will only wait for vnodes on the local node and not vnodes cluster-wide. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak cluster unresponsive after single node failure
ar == Armon Dadgar armon.dad...@gmail.com wrote: ar All the nodes appeared to have been blocked trying to talk to riak ar 001 which was the ring claimant at the time. Doing this seems to ar have cleared the state enough for the cluster to make progress ar again. Armon, it's quite unlikely that the ring claimant was doing anything special because the claimant only acts when cluster membership changes. Instead, it's quite likely that riak001 was busy doing a set of LevelDB compactions. There have been a number of changes recently to reduce the amount of time that we've seen worst-case LevelDB compaction blocking Erlang process schedulers which blocks *everything*, including the keep-alives that are sent between Erlang nodes. The longest LevelDB-related stoppage that I've seen was 7.5 minutes. :-( When that happens on a node X, then all other nodes will complain (almost simultaneously) that node X is down. It's not *down*, it's just reallyreallyreally slow to respond to messages ... which is effectively the same as being down. Checking for big LevelDB compaction storms is pretty easy using DTrace or SystemTap, but you're probably not using a kernel that has user-space SystemTap available. There are compaction messages in the LOG file of each LevelDB data directory. The hassle is the need to look at all of them in parallel. A secondary effect is watching write ops via iostat -x 1: the amount of data written spikes much higher than writes triggered only by Riak client operations. (Read ops would go higher too, except that many files input to a compaction are already cached by the OS.) Your primary keys look UID'ish. If they are not lexigraphically adjacent to other keys inserted at the same time, you will cause many more LevelDB compaction events than if your keys were adjacent (e.g. prefixing them with a wall-clock timestamp). -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: problem with rolling upgrade 0.14 - 1.0
Tomer Naor to...@conduit.com wrote: tn riak start tn Attempting to restart script through sudo -u riak tn pthread/ethr_event.c:98: Fatal error in wait__(): Function not tn implemented (38) tn pthread/ethr_event.c:98: Fatal error in wait__(): Function not tn implemented (38) Tomer, those are new to me. You didn't mention whether you were using Red Hat Enterprise Linux, CentOS, Fedora Core, or some other RPM-based Linux distribution (or its version/release number). It could also be hopeful to know if you were using the 32 or 64 bit Riak package. tn When installing the riak-1.0 rpm I'm getting the following output: tn chcon: couldn't compute security context from unlabeled Those are release to SELinux. If you've got SELinux enforcement disabled, then I believe that you can ignore those messages. tn in addition, I noticed that the app.config did not update as it tn should on the installation process. tn warning: /etc/riak/app.config created as /etc/riak/app.config.rpmnew tn warning: /etc/riak/vm.args created as /etc/riak/vm.args.rpmnew That's RPM's behavior: if the config file already exists, then the package's new version is saved using a different file name. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Segregated data storage for different bucket
Elias Levy fearsome.lucid...@gmail.com wrote: el Please correct me if I am wrong, but after el reading the documentation and looking at a test system, it appears el that by default Riak will store data from different buckets within el the same backend. That it merely treats the bucket as a prefix to el the key and as a index to the per bucket configuration data. Correct. el Looking at the multiple backend example config at el [1]https://help.basho.com/entries/20186031-how-can-i-run-bitcask-and el -innostore-on-the-same-cluster it would appear that even if one el enables multi backend support one only gains the ability to select el different backend types, not different backend instances of the same el type storing to different locations. Correct. el Even then it would appear that such instances of the same backend el type would have to be predefined in app.config rather than el programmatic created using the bucket properties API. Correct. That limitation can be removed by writing your own backend implementation. (Hey, it's just software. :-) -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: innostore performance tuning
David Koblas da...@koblas.com wrote:
dk * innostore section - {flush_method, O_DIRECT}
David, what happens when you remove that option and allow the default to
be used?
-Scott
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Achieving 100% consitency
Jonathan Langevin jlange...@loomlearning.com wrote: jl Even for development-purposes only? Otherwise it seems data would be jl written n times to the same machine, which is needless in a dev jl environment with low storage specs... ... which would be true until the 1-node cluster grows to a 2-node or N-node cluster. Changing the N value of a key that already exists, especially where N is increasing relative to its old value, creates many different kinds of unpleasantness. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Bulk loading data and Could not contact Riak Server error
gtuhl r...@uhls.com wrote: With Riak, my main challenge is getting this data loaded. Using the PHP library I am able to push 100-200 documents/sec. A quick grep through the PHP client source suggests that that client doesn't support the Protocol Buffers interface to Riak. Depending on the workload, a PB-based client is anywhere from 20% to several hundred percent faster than an HTTP-based client. You'll definitely want to run multiple clients in parallel, especially if/when your cluster is larger than a single box: pointing those clients at different cluster members will get you different throughput than pointing all clients at a single cluster member. Your message hadn't included any messages from the Riak server logs that might give hints to why the HTTP service becomes unavailable ... but it's likely that there's useful info there. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak_err starts writing to stdout/stderr
Anthony, I shouldn't reply to email in the morning before coffee, but here's to risking looking silly The SASL PROGRESS REPORT messages are SASL messages and thus are written to where the sasl_error_logger attributes says to write them. Messages from a generic error_logger:error_msg/info_msg/warning_msg and friends aren't SASL messages and thus aren't necessarily written to where SASL messages are. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Inconsistent results from Riak Search
James Linder james.h.lin...@gmail.com wrote: jl I am getting inconsistent results using the Riak Search Solr jl interface. Specifically, the number of results and ordering of jl results is changing from one query to the next. jl [...] jl - We are running riak-search-0.14.0-1.el5 installed via the Basho jl provided RPM James, does the same thing happen with the 0.14.2 release? Earlier 0.14 releases used a sort that wasn't stable and so result ordering could be (and frequently were) different. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Duplicate bucket props r/w/dw/rw if set in app.config
Nicolas Fouché nico...@silentale.com wrote: nf [...] But when I look at the props of a bucket, r,w,dw and rw are present nf twice : nf [...] nf I use riak search 0.14.0. Bug ? Feature. Erlang property lists take the first element in the list that matches the property name. On the other hand, I don't know what JavaScript will do with an object that's got the same member name twice, or if different JS interpreters handle that differently. To avoid hassles there, when setting your bucket property, you could delete the old values of 'r', 'w', et al. before prepending your new values onto the list. -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Storing large collections in Riak (or any distributed store)
Nathan Sobo ns...@pivotallabs.com wrote: ns Is a key-value store actually inappropriate for this problem? No. One way to do it is to use a single KV key to store multiple addresses worth of info. Pick a relatively big number, 50K subscribers/key, though it may vary. Use a key naming scheme so that you can pre-calculate all keys for a given list, e.g. bucket = list-subscribers, key = name + range index #, or perhaps list name + start-of-hash-range + end-of-hash-range. How do you know the range index # or start ends of range? One method would be hashing, MD5 or SHA1 or whatever. If you store all addresses for a list with a fixed number of hash hunks, e.g. 100, then each hash hunk will have roughly 20K entries for a 2M subscriber list. To find all subscribers, fetch 100 known keys. If you want to keep addresses in sorted order, it's more work but also doable. A naive plan is to make your hash function F(addr) = first letter of 'addr'. Keys get clumpy that way, but only slightly more creativity can get around it. To find a particular subscriber, hash that subscriber's address and fetch 1 key. You're also getting a lot of uninteresting data with that key's value, but if event is uncommon, it's not a problem. (If that event actually is common, consider moving the commonly-queried data elsewhere. Or duplicate that info in another smaller key somewhere else.) Similar logic for list maintenance events (add subscriber, delete subscriber). -Scott ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
