That *does* sound like an interesting way to do it. Kinda best-of-both-worlds, depending on your backup schemes and whatnot. I'm definitely curious to hear about how it works out for you.
-B. On Tue, Aug 13, 2013 at 4:03 PM, Dave Martorana <d...@flyclops.com> wrote: > An interesting hybrid that I'm coming around to seems to be using a Unix > release - OmniOS has an AMI, for instance - and ZFS. With a large-enough > store, I can run without EBS on my nodes, and have a single ZFS backup > instance with a huge amount of slow-EBS storage for accepting ZFS snapshots. > > I'm still learning all the pieces, but luckily I have a company upstairs > from me that does a very similar thing with > 300TB and is willing to help > me set up my ZFS backup infrastructure. > > Dave > > > On Mon, Aug 12, 2013 at 10:00 PM, Brady Wetherington <br...@bespincorp.com > > wrote: > >> I will probably stick with EBS-store for now. I don't know how >> comfortable I can get with a replica that could disappear with simply an >> unintended reboot (one of my nodes just did that randomly today, for >> example). Sure, I would immediately start rebuilding it as soon as that >> were to happen, but we could be talking a pretty huge chunk of data that >> would have to get rebuilt out of the cluster. And that sounds scary. Even >> though, logically, I understand that it should not be. >> >> I will get there; I'm just a little cautious. As I learn Riak better and >> get more comfortable with it, maybe I would be able to start to move in a >> direction like that. And certainly as the performance characteristics of >> EBS-volumes start to bite me in the butt; that might force me to get >> comfortable with instance-store real quick. I would at least hope to be >> serving a decent-sized chunk of my data from memory, however. >> >> As for throwing my instances in one AZ - I don't feel comfortable with >> that either. I'll try out the way I'm saying and will report back - do I >> end up with crazy latencies all over the map, or does it seem to "just >> work?" We'll see. >> >> In the meantime, I still feel funny about "breaking the rules" on the >> 5-node cluster policy. Given my other choices as having been kinda >> nailed-down for now, what do you guys think of that? >> >> E.g. - should I take the risk of putting a 5th instance up in the same AZ >> as one of the others, or should I just "be ok" with having 4? Or should I >> do something weird like changing my 'n' value to be one fewer or something >> like that? (I think, as I understand it so far, I'm really liking "n=3, >> w=2, r=2" - but I could change it if it made more sense with the topology >> I've selected.) >> >> -B. >> >> >> Date: Sun, 11 Aug 2013 18:57:11 -0600 >>> From: Jared Morrow <ja...@basho.com> >>> To: Jeremiah Peschka <jeremiah.pesc...@gmail.com> >>> Cc: riak-users <riak-users@lists.basho.com> >>> Subject: Re: Practical Riak cluster choices in AWS (number of nodes? >>> AZ's?) >>> Message-ID: >>> < >>> cacusovelpu8yfcivykexm9ztkhq-kdnowk1afvpflcsip2h...@mail.gmail.com> >>> Content-Type: text/plain; charset="iso-8859-1" >>> >>> >>> +1 to what Jeremiah said, putting a 4 or 5 node cluster in each US West >>> and >>> US East using MDC between them would be the optimum solution. I'm also >>> not >>> buying consistent latencies between AZ's, but I've also not tested it >>> personally in a production environment. We have many riak-users members >>> on >>> AWS, so hopefully more experienced people will chime in. >>> >>> If you haven't seen them already, here's what I have in my "Riak on AWS" >>> bookmark folder: >>> >>> http://media.amazonwebservices.com/AWS_NoSQL_Riak.pdf >>> http://docs.basho.com/riak/latest/ops/tuning/aws/ >>> http://basho.com/riak-on-aws-deployment-options/ >>> >>> -Jared >>> >>> >>> >>> >>> On Sun, Aug 11, 2013 at 6:11 PM, Jeremiah Peschka < >>> jeremiah.pesc...@gmail.com> wrote: >>> >>> > I'd be wary of using EBS backed nodes for Riak - with only a single >>> > ethernet connection, it wil be very easy to saturate the max of >>> 1000mbps >>> > available in a single AWS NIC (unless you're using cluster compute >>> > instances). I'd be more worried about temporarily losing contact with a >>> > node through network saturation than through AZ failure, truthfully. >>> > >>> > The beauty of Riak is that a node can drop and you can replace it with >>> > minimal fuss. Use that to your advantage and make every node in the >>> cluster >>> > disposable. >>> > >>> > As far as doubling up in one AZ goes - if you're worried about AZ >>> failure, >>> > you should treat each AZ as a separate data center and design your >>> failure >>> > scenarios accordingly. Yes, Amazon say you should put one Riak node in >>> each >>> > AZ; I'm not buying that. With no guarantee around latency, and no >>> control >>> > around between DCs, you need to be very careful how much of that >>> latency >>> > you're willing to introduce into your application. >>> > >>> > Were I in your position, I'd stand up a 5 node cluster in US-WEST-2 >>> and be >>> > done with it. I'd consider Riak EE for my HA/DR solution once the >>> business >>> > decides that off-site HA/DR is something it wants/needs. >>> > >>> > >>> > --- >>> > Jeremiah Peschka - Founder, Brent Ozar Unlimited >>> > MCITP: SQL Server 2008, MVP >>> > Cloudera Certified Developer for Apache Hadoop >>> > >>> > >>> > On Sun, Aug 11, 2013 at 1:52 PM, Brady Wetherington < >>> br...@bespincorp.com>wrote: >>> > >>> >> Hi all - >>> >> >>> >> I have some questions about how I want my Riak stuff to work - I've >>> >> already asked these questions of some Basho people and gotten some >>> answers, >>> >> but thought I would toss it out into the wider world to see what you >>> all >>> >> have to say, too: >>> >> >>> >> First off - I know 5 instances is the "magic number" of instances to >>> >> have. If I understand the thinking here, it's that at the default >>> >> redundancy level ('n'?) of 3, it is most likely to start getting me >>> some >>> >> scaling (e.g., performance > just that of a single node), and yet >>> also have >>> >> redundancy; whereby I can lose one box and not start to take a >>> performance >>> >> hit. >>> >> >>> >> My question is - I think I can only do 4 in a way that makes sense. I >>> >> only have 4 AZ's that I can use right now; AWS won't let me boot >>> instances >>> >> in 1a. My concern is if I try to do 5, I will be "doubling up" in one >>> AZ - >>> >> and in AWS you're almost as likely to lose an entire AZ as you are a >>> single >>> >> instance. And so, if I have instances doubled-up in one AZ (let's say >>> >> us-east-1e), and then I lose 1e, I've now lost two instances. What >>> are the >>> >> chances that all three of my replicas of some chunk of my data are on >>> those >>> >> two instances? I know that it's not guaranteed that all replicas are >>> on >>> >> separate nodes. >>> >> >>> >> So is it better for me to ignore the recommendation of 5 nodes, and >>> just >>> >> do 4? Or to ignore the fact that I might be doubling-up in one AZ? >>> Also, >>> >> another note. These are designed to be 'durable' nodes, so if one >>> should go >>> >> down I would expect to bring it back up *with* its data - or, if I >>> >> couldn't, I would do a force-replace or replace and rebuild it from >>> the >>> >> other replicas. I'm definitely not doing instance-store. So I don't >>> know if >>> >> that mitigates my need for a full 5 nodes. I would also consider >>> losing one >>> >> node to be "degraded" and would probably seek to fix that problem as >>> soon >>> >> as possible, so I wouldn't expect to be in that situation for long. I >>> would >>> >> probably tolerate a drop in performance during that time, too. (Not a >>> >> super-severe one, but 20-30 percent? Sure.) >>> >> >>> >> What do you folks think? >>> >> >>> >> -B. >>> >> >>> >> >>> >> _______________________________________________ >>> >> riak-users mailing list >>> >> riak-users@lists.basho.com >>> >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >> >>> >> >>> > >>> > _______________________________________________ >>> > riak-users mailing list >>> > riak-users@lists.basho.com >>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> > >>> > >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: < >>> http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130811/d350b1f1/attachment-0001.html >>> > >>> >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
