Re: Riak on SAN
We're in the middle of building out a Riak cluster with OmniOS and ZFS snapshot backups offsite with the help some friends that have deployed the same basic solution (not Riak, but ZFS snapshots for hot-backup) at a tremendous scale. You get some other nice bits along with "hot" backups - ZFS snapshots are basically diffs, so you can snapshot as often as once-per-second if you're so inclined. Vertical scaling is also interesting in that you can launch a beefier replacement machine for node and restore the older machine's ZFS snapshot. The node won't be wise to the fact it's not the same machine, which makes vertical scaling much faster than the traditional replace method. (This is where someone at Basho can scream "NOOO! DON'T DO THAT!") You can continue down that line of thinking to plenty of advantages of using ZFS if backup is of ultimate importance - including being able to restore a cluster even if the building hosting your physical boxes are hit by a daisy cutter. :) Here's a bit of information on how we're going about it, so far: http://tech.flyclops.com/building-riak-on-omnios-360 Dave On Thu, Oct 3, 2013 at 8:45 PM, Pedram Nimreezi wrote: > I consider that the main use case ;p > > > On Thu, Oct 3, 2013 at 8:38 PM, Mike Oxford wrote: > >> One more use-case for backups: If you're running a big cluster and UserX >> makes a bad code deploy which horks a bunch of data ... restore may be the >> only option. >> >> It happens. >> >> -mox >> >> >> On Wed, Oct 2, 2013 at 12:12 PM, John E. Vincent < >> lusis.org+riak-us...@gmail.com> wrote: >> >>> I'm going to take a competing view here. >>> >>> SAN is a bit overloaded of a term at this point. Nothing precludes a SAN >>> from being performant or having SSDs. Yes the cost is overkill for fiber >>> but iSCSI is much more realistic. Alternately you can even do ATAoE. >>> >>> From a hardware perspective, if I have 5 pizza boxes as riak nodes, I >>> can only fit so many disks in them. Meanwhile I can add another shelf to my >>> SAN and expand as needed. Additionally backup of a SAN is MUCH easier than >>> backup of a riak node itself. It's a snapshot and you're done. Mind you >>> nothing precludes you from doing LVM snapshots in the OS but you still need >>> to get the data OFF that system for it to be truly backed up. >>> >>> I love riak and other distributed stores but backing them up is NOT a >>> solved problem. Walking all keys, coordinating the take down of all your >>> nodes in a given order or whatever your strategy is a serious pain point. >>> >>> Using a SAN or local disk also doesn't excuse you from watching I/O >>> performance. With a SAN I get multiple redundant paths to a block device >>> and I don't get that necessarily with local storage. >>> >>> Just my two bits. >>> >>> >>> >>> On Wed, Oct 2, 2013 at 2:18 AM, Jeremiah Peschka < >>> jeremiah.pesc...@gmail.com> wrote: >>> Could you do it? Sure. Should you do it? No. An advantage of Riak is that you can avoid the cost of SAN storage by getting duplication at the machine level rather than rely on your storage vendor to provide it. Running Riak on a SAN also exposes you to the SAN becoming your bottleneck; you only have so many fiber/iSCSI ports and a fixed number of disks. The risk of storage contention is high, too, so you can run into latency issues that are difficult to diagnose without looking into both Riak as well as the storage system. Keeping cost in mind, too, SAN storage is about 10x the cost of consumer grade SSDs. Not to mention feature licensing and support... The cost comparison isn't favorable. Please note: Even though your vendor calls it a SAN, that doesn't mean it's a SAN. On Oct 1, 2013 11:08 PM, "Guy Morton" wrote: > Does this make sense? > > -- > Guy Morton > Web Development Manager > Brüel & Kjær EMS > > This e-mail is confidential and may be read, copied and used only by > the intended recipient. If you have received it in error, please contact > the sender immediately by return e-mail. Please then delete the e-mail and > do not disclose its contents to any other person. > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> ___ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/ri
Re: Riak Java client not returning deleted sibling
Daniel -
I'll get 1.1.3 out today; I'll post to the list. There's actually a
couple other small things I need to squeeze in since we're going to do
a release.
Re: Setting the vclock on the tombstone in JSONConverter, you're
right. It would only be an issue if you only had a tombstone, but it
should be there.
Thanks,
- Roach
On Fri, Oct 4, 2013 at 2:58 AM, Daniel Iwan wrote:
> Thanks Brian for putting fix together so quickly.
>
> I think I found something else though.
> In JSONConverter I don't see vclock being set in toDomain() when converting
> deleted sibling?
> That vclock should be used for following delete if I understood it
> correctly?
>
> Also where can I download latest build? I tried
> http://riak-java-client.s3.amazonaws.com/riak-client-1.1.3-jar-with-dependencies.jar
> but access is denied
>
> Cheers
> Daniel
>
>
> On 3 October 2013 19:36, Brian Roach wrote:
>>
>> On Thu, Oct 3, 2013 at 10:32 AM, Daniel Iwan
>> wrote:
>> > Thanks Brian for quick response.
>> >
>> > As a side question, what is the best way to delete such an object i.e.
>> > once
>> > I know one of the siblings has 'deleted' flag true because I fetched it?
>> > Should I just use DomainBucket.delete(key) without providing any vclock?
>> > Would it wipe it from Riak or create yet another sibling?
>>
>> You should always use vclocks when possible, which in the case it is.
>> There are additional issues around doing the delete without a vclock
>> and if there's concurrently a store operation occurring.
>>
>> Ideally you should look at why you're getting that tombstone sibling.
>> If it's simply a case of high write concurrency and you're using
>> vclocks with your writes, then there's not much you can do except
>> resolve it later (without changing how you're using the DB)... but
>> usually these things are caused by writes without a vclock.
>>
>> Thanks,
>> - Roach
>>
>>
>>
>>
>> >
>> > Regards
>> > Daniel
>> >
>> >
>> > On 3 October 2013 17:20, Brian Roach wrote:
>> >>
>> >> Daniel -
>> >>
>> >> Yeah, that is the case. When the ability to pass fetch/store/delete
>> >> meta was added to DomainBucket way back when it appears that was
>> >> missed.
>> >>
>> >> I'll add it and forward-port to 1.4.x as well and cut new jars. Should
>> >> be avail by tomorrow morning at the latest.
>> >>
>> >> Thanks!
>> >> - Roach
>> >>
>> >> On Thu, Oct 3, 2013 at 9:38 AM, Daniel Iwan
>> >> wrote:
>> >> > Hi I'm using Riak 1.3.1 and Java client 1.1.2
>> >> >
>> >> > Using http and curl I see 4 siblings for an object one of which has
>> >> > X-Riak-Deleted: true
>> >> > but when I'm using Java client with DomainBucket my Converter's
>> >> > method
>> >> > toDomain is called only 3 times.
>> >> >
>> >> > I have set the property
>> >> >
>> >> > builder.returnDeletedVClock(true);
>> >> >
>> >> > on my DomainBuilder which I keep reusing for all queries and store
>> >> > operations (I guess that's good practice btw.?)
>> >> >
>> >> >
>> >> > I run that under debugger and it seems raw client sees 4 siblings but
>> >> > passes
>> >> > over only 3 due to bug (I think) in DomainBucket.fetch() method which
>> >> > should
>> >> > have
>> >> >
>> >> > if (fetchMeta.hasReturnDeletedVClock()) {
>> >> >
>> >> >
>> >> > so.returnDeletedVClock(fetchMeta.getReturnDeletedVClock());
>> >> >
>> >> > }
>> >> >
>> >> > at the end, as store() method has.
>> >> >
>> >> > Could you confirm or I'm I completely wrong?
>> >> >
>> >> >
>> >> > Regards
>> >> >
>> >> > Daniel
>> >> >
>> >> >
>> >> > ___
>> >> > riak-users mailing list
>> >> > riak-users@lists.basho.com
>> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >> >
>> >
>> >
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak Java client not returning deleted sibling
Thanks Brian for putting fix together so quickly.
I think I found something else though.
In JSONConverter I don't see vclock being set in toDomain() when converting
deleted sibling?
That vclock should be used for following delete if I understood it
correctly?
Also where can I download latest build? I tried
http://riak-java-client.s3.amazonaws.com/riak-client-1.1.3-jar-with-dependencies.jar
but access is denied
Cheers
Daniel
On 3 October 2013 19:36, Brian Roach wrote:
> On Thu, Oct 3, 2013 at 10:32 AM, Daniel Iwan
> wrote:
> > Thanks Brian for quick response.
> >
> > As a side question, what is the best way to delete such an object i.e.
> once
> > I know one of the siblings has 'deleted' flag true because I fetched it?
> > Should I just use DomainBucket.delete(key) without providing any vclock?
> > Would it wipe it from Riak or create yet another sibling?
>
> You should always use vclocks when possible, which in the case it is.
> There are additional issues around doing the delete without a vclock
> and if there's concurrently a store operation occurring.
>
> Ideally you should look at why you're getting that tombstone sibling.
> If it's simply a case of high write concurrency and you're using
> vclocks with your writes, then there's not much you can do except
> resolve it later (without changing how you're using the DB)... but
> usually these things are caused by writes without a vclock.
>
> Thanks,
> - Roach
>
>
>
>
> >
> > Regards
> > Daniel
> >
> >
> > On 3 October 2013 17:20, Brian Roach wrote:
> >>
> >> Daniel -
> >>
> >> Yeah, that is the case. When the ability to pass fetch/store/delete
> >> meta was added to DomainBucket way back when it appears that was
> >> missed.
> >>
> >> I'll add it and forward-port to 1.4.x as well and cut new jars. Should
> >> be avail by tomorrow morning at the latest.
> >>
> >> Thanks!
> >> - Roach
> >>
> >> On Thu, Oct 3, 2013 at 9:38 AM, Daniel Iwan
> wrote:
> >> > Hi I'm using Riak 1.3.1 and Java client 1.1.2
> >> >
> >> > Using http and curl I see 4 siblings for an object one of which has
> >> > X-Riak-Deleted: true
> >> > but when I'm using Java client with DomainBucket my Converter's method
> >> > toDomain is called only 3 times.
> >> >
> >> > I have set the property
> >> >
> >> > builder.returnDeletedVClock(true);
> >> >
> >> > on my DomainBuilder which I keep reusing for all queries and store
> >> > operations (I guess that's good practice btw.?)
> >> >
> >> >
> >> > I run that under debugger and it seems raw client sees 4 siblings but
> >> > passes
> >> > over only 3 due to bug (I think) in DomainBucket.fetch() method which
> >> > should
> >> > have
> >> >
> >> > if (fetchMeta.hasReturnDeletedVClock()) {
> >> >
> >> >
> so.returnDeletedVClock(fetchMeta.getReturnDeletedVClock());
> >> >
> >> > }
> >> >
> >> > at the end, as store() method has.
> >> >
> >> > Could you confirm or I'm I completely wrong?
> >> >
> >> >
> >> > Regards
> >> >
> >> > Daniel
> >> >
> >> >
> >> > ___
> >> > riak-users mailing list
> >> > riak-users@lists.basho.com
> >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >> >
> >
> >
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
