Le 04/09/2012 16:54, Erik Søe Sørensen a écrit :
On 04-09-2012 13:56, Bernard Fouché wrote:
Maybe this proposal is already an error by itself (comments welcome!)
Possibly :-) - see below for things you should at the very least be
aware of.
It triggers the following questions (from what I've understood from Erlang
and Riak at this point):
- When multiple rings are needed, it seems that each ring needs a new
Riak instance on each node. But all examples I found are about running
an Erlang VM per Riak instance, instead of having more than one Riak
instance in a single VM. I'd like to know if there is a solution to run
many Riak instance in a single VM since Erlang seems to be designed to
sustain large quantities of processes in a single VM instead of having
many VM on a single host (for better resource management/monitoring).
I'd like to know what is really possible with Riak regarding multiple
instances ?
You can run multiple apps in a single Erlang VM.
But you can not run multiple instances of the same app - for one
thing, there are singleton processes which are registered with a
certain name.
A close match to what you're asking - *in case* you really want to run
multiple rings and cannot just use a backend with auto-expiry - is to
run multi_backend with appropriate configurations (i.e., data dir
locations).
However, that would leave the problem that the individual storage
backends would presumably be confused when all of their files are
deleted under their feet.
That can probably be solved by making a storage backend wrapper for
such "droppable" buckets, which can shut down the actual backend prior
to deletion. (The deletion functionality should then be put in that
wrapper).
- For the same reason, I'd like to have the server that process the
traffic coming from the sensors in the same VM, to have just one VM per
physical machine. Is there some doc about bundling an Erlang user made
app with riak in the same .beam (a 'release'?)?
- Or if a user can tell if having many different Erlang VM running Riak
on a single node works efficiently?
At least with LevelDB, there would be a potential problem with the
number of simultaneously open FDs.
Note that that's also the case with one Riak instance which has many
LevelDB backends under a multi_backend.
All in all... if you can use auto expiry, it'll be less trouble.
/Erik
Hello Erik, many thanks for your answer.
Is there some document describing how auto expiry deletion is handled in
HanoiDb? I guess there is some kind of index built with timestamp
values, if this index is shared between processes than can add new
records and processes that discard old ones all in all in falls down to
per-record deletion my guess is that this system has the same
performance issue than massively erasing records in any kind of storage
engine.
Another advantage of having multiple rings that get obsoleted (at least
at the beginning of the deployment) is the possibility to use a set of
parameters for a given ring and use different values when a new ring
comes in. Migrating to a new datacenter is also easier: when a new
current ring is being built, you can do the change at that time since
the application has multi-ring awareness since the beginning, it can
continue to perform read only queries to older ring in a previous
datacenter while writing only in the new one.
Bernard
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
