We're in the middle of building out a Riak cluster with OmniOS and ZFS
snapshot backups offsite with the help some friends that have deployed the
same basic solution (not Riak, but ZFS snapshots for hot-backup) at a
tremendous scale.
You get some other nice bits along with "hot" backups - ZFS snap
I consider that the main use case ;p
On Thu, Oct 3, 2013 at 8:38 PM, Mike Oxford wrote:
> One more use-case for backups: If you're running a big cluster and UserX
> makes a bad code deploy which horks a bunch of data ... restore may be the
> only option.
>
> It happens.
>
> -mox
>
>
> On Wed,
One more use-case for backups: If you're running a big cluster and UserX
makes a bad code deploy which horks a bunch of data ... restore may be the
only option.
It happens.
-mox
On Wed, Oct 2, 2013 at 12:12 PM, John E. Vincent <
lusis.org+riak-us...@gmail.com> wrote:
> I'm going to take a com
Chiming in with the completely anecdotal statement that we have customers
who run large Riak clusters on ZFS. As far as I know, we haven't gotten
any complaints due to ZFS. The only cautionary tale would be to not let
your zpool completely fill up because deletes take up storage due to the
append
Some users might avoid the ZFS overheads, remember we are on a KV world
where to read/write many keys you will have to do so concurrently, say
there is less than 1% chances for things things going wrong with 1 a
server belonging to a Riak cluster, if building a Riak server is cheap,
would you p
Hi Guido,
I don’t see how snappy compression renders ZFS useless, you might do some
things twice like crcing but it also protects on different layers. While the
ZFS crc protects data on the disks the in app crc could protect the data ‘all’
the way up, compression wise you might not even turn on
If using LevelDB backend, LevelDB has a nice compression (snappy),
including CRC checks and all sort of data corruption checks, I have read
on this mail list people that has required to disable snappy compression
because it renders ZFS useless (not much to compress after that)
Hence, it is kin
Not sure what ZFS has to do with snappy compression, as it's a file system
not a compression algorithm..
feature wise, ZFS is quite possibly the most enterprise file system around,
including advanced data corruption prevention and remote backing up..
This would be a viable option in BSD/Solaris en
And for ZFS? I wouldn't recommend it, after Riak 1.4 snappy LevelDB
compression does a nice job, why take the risk of yet another not so
enterprise ready compression algorithms.
I could be wrong though,
Guido.
On 03/10/13 12:11, Guido Medina wrote:
I have heard some SAN's horrors stories too,
I have heard some SAN's horrors stories too, Riak nodes are so cheap
that I don't see the point in even having any mirror on the node, here
my points:
1. Erlang interprocess communication brings some network usage, why yet
another network usage on replicating the data? If the whole idea of
So, call me naive, but couldn't ZFS be used as Heinze suggested?
I have some SAN horror stories - both operationally and from an economic
perspective.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-use
To inject a Basho-colored perspective briefly (opinions/thoughts are my
own, disclaimer blah blah):
We have had customers deploy with SANs backing Riak. We recommend against
it for all the reasons discussed in this thread: lack of isolation (storage
and bus), extensive cost, etc. Nevertheless, the
I think this is just an area that hasn't been addressed unlike things like
a "hot" copy in the rdbms world..
I think at least a plan for how a hotcopy tool for a riak core semantics
database system
could be made would be a good step in the right direction, ie.. maybe a
combination of
specifying a t
Man I go away for a few hours for family time and off things go ;)
So this lead to some interesting convos on twitter and here. Others have
addressed some things. I figure it helps to explain the sad lonely world I
live in - It's called "Enterprise".
A few folks are somewhat aware that our produc
Responses inline.
TL;DR - I actually agree with John, SANs make management of storage
stupidly easy, but you pay more money for it. Make the right decision for
your org, but make sure you can monitor and backup that decision. The SAN
isn't a magic box. And a Drobo b1200i [2] is definitely not a S
October 02, 2013 3:12 PM
To: riak-users
Subject: Re: Riak
on SAN
I'm going to take a compet
ime on node re-setup, but redundancy on cluster-level seems
> reliable enough.
>
> *From:* riak-users [mailto:riak -users-boun...@lists.basho.com] *On
> Behalf Of *John E. Vincent
> *Sent:* Wednesday, October 02, 2013 3:12 PM
> *To:* riak-users
> *Subject:* Re: Riak on SAN
>
>
M
To: riak-users
Subject: Re:
Riak on SAN
I'm going to take a competing view here.
, and not any harder then SAN snapshot.
Sounds like win-win situation for me.
Victor P.
From: Alexander Sicular [mailto:sicul...@gmail.com]
Sent: Wednesday, October 02, 2013 4:12 PM
To: Victor
Cc: 'John E. Vincent'; 'riak-users'
Subject: Re: Riak on SAN
Cluste
t; Of *John E. Vincent
> *Sent:* Wednesday, October 02, 2013 3:12 PM
> *To:* riak-users
> *Subject:* Re: Riak on SAN
> ** **
> I'm going to take a competing view here.
> ** **
> SAN is a bit overloaded of a term at this point. Nothing precludes a SAN
> from
n cluster-level seems
> reliable enough.
>
> From: riak-users [mailto:riak-users-boun...@lists.basho.com] On Behalf Of
> John E. Vincent
> Sent: Wednesday, October 02, 2013 3:12 PM
> To: riak-users
> Subject: Re: Riak on SAN
>
> I'm going to take a competing view
-setup, but redundancy on cluster-level seems reliable
enough.
From: riak-users [mailto:riak-users-boun...@lists.basho.com] On Behalf Of John
E. Vincent
Sent: Wednesday, October 02, 2013 3:12 PM
To: riak-users
Subject: Re: Riak on SAN
I'm going to take a competing view here.
SAN
zfs snapshot -> zfs send -> zfs receive on big ass storage box.
On 02 Oct 2013, at 15:12, John E. Vincent
wrote:
> I love riak and other distributed stores but backing them up is NOT a solved
> problem. Walking all keys, coordinating the take down of all your nodes in a
> given order or whatev
I'm going to take a competing view here.
SAN is a bit overloaded of a term at this point. Nothing precludes a SAN
from being performant or having SSDs. Yes the cost is overkill for fiber
but iSCSI is much more realistic. Alternately you can even do ATAoE.
>From a hardware perspective, if I have 5
Seconded.
Leave the SAN. Take the SSDs.
@siculars
http://siculars.posthaven.com
Sent from my iRotaryPhone
> On Oct 2, 2013, at 2:18, Jeremiah Peschka wrote:
>
> Could you do it? Sure.
>
> Should you do it? No.
>
> An advantage of Riak is that you can avoid the cost of SAN storage by gett
Thanks, that's what I figured.
--
Guy Morton
Web Development Manager
Brüel & Kjær EMS
From: Jeremiah Peschka [jeremiah.pesc...@gmail.com]
Sent: Wednesday, 2 October 2013 4:18 PM
To: Guy Morton
Cc: riak-users
Subject: Re: Riak on SAN
Could you do
Could you do it? Sure.
Should you do it? No.
An advantage of Riak is that you can avoid the cost of SAN storage by
getting duplication at the machine level rather than rely on your storage
vendor to provide it.
Running Riak on a SAN also exposes you to the SAN becoming your bottleneck;
you only
Does this make sense?
--
Guy Morton
Web Development Manager
Brüel & Kjær EMS
This e-mail is confidential and may be read, copied and used only by the
intended recipient. If you have received it in error, please contact the sender
immediately by return e-mail. Please then delete the e-mail and d
28 matches
Mail list logo
