Re: [akka-user] Re: ANNOUNCE: Akka Multi-DC Persistence

[Igor Baltiyskiy]

Hi Patrik,

Thanks for your reply! I think I'll go with implementing the dynamic 
solution for my project: we're fine with causal eventual consistency, and 
we think being available during intra-DC netsplits will be beneficial. 
Another reason I didn't mention is that we need a particular schema for our 
storage. As far as I understand, there's no way currently to have a custom 
schema for the Cassandra event journal in multi-DC Akka Persistence, right?

Thanks
Igor

On Thursday, February 1, 2018 at 7:09:31 PM UTC+3, Patrik Nordwall wrote:
>
> Hi Igor,
>
> There are several reasons, such as:
>
>- we believe that it's useful to have the strong consistency with 
>single-writer within one DC, and "only" have the asynchronous replication 
>between DCs
>- failures and partitions within one DC can be handled by SBR, fits 
>the model of Akka Multi-DC Clustering where each DC is autonomous
>- easier to implement, and therefore faster to market, we could 
>perhaps add dynamic later if requested
>- number of DCs are typically static if it's used for availability 
>zones or regions
>- note that our DC concept is only a logical grouping of nodes, so it 
>doesn't have to be physical DC
>- performance and storage costs increase with number of replicas (DCs) 
>so we recommend keeping it rather low anyway
>- we could build on top of existing infrastructure, Akka Persistence, 
>and the Cassandra plugin
>
>
> On Thu, Feb 1, 2018 at 2:01 PM, Igor Baltiyskiy  > wrote:
>
>> Hello Patrik,
>>
>> excellent news!
>>
>> May I ask why you chose to have static topology over dynamic one? With 
>> the latter one, I mean lifting the "only one writer" requirement locally as 
>> well, thus enabling always-available operation --- even during netsplit, 
>> where you could have multiple writers. CRDT-wise, this is the same, the 
>> difference is that replicas may appear and disappear at any time. I 
>> understand that this is more work than supporting only the static topology 
>> (with replica scope (DC) predefined), but seemingly not super-much, and it 
>> would cover the static case too. 
>>
>> Is it because you feel that keeping one-writer-per-DC + using split brain 
>> resolver is the right thing to do? Or is it because you considered the cost 
>> of this solution to be prohibitively high? I'm asking because I was 
>> considering implementing a similar thing (causal reliable broadcast over 
>> Cassandra) with dynamic topology to handle the intra-DC netsplit scenario, 
>> so I'd be glad to hear your thoughts.
>>
>> Thanks
>> Igor
>>
>>
>> On Thursday, January 18, 2018 at 6:47:03 PM UTC+3, Patrik Nordwall wrote:
>>>
>>> Several exciting capabilities for multiple data centers have recently 
>>> been added to Akka. That includes Multi-DC Clustering 
>>>  that was added 
>>> in Akka a few months ago. Now we are thrilled to announce that Akka 
>>> Multi-DC Persistence has also been released. It's a commercial feature 
>>> available to Lightbend subscribers 
>>> .
>>>
>>> The screencast  and blog 
>>> post  should 
>>> give you a good overview of the feature, and more details can be found in 
>>> the documentation 
>>> 
>>> .
>>>
>>> -- 
>>>
>>> Patrik Nordwall
>>> Akka Tech Lead
>>> Lightbend  -  Reactive apps on the JVM
>>> Twitter: @patriknw
>>>
>>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
>
> Patrik Nordwall
> Akka Tech Lead
> Lightbend  -  Reactive apps on the JVM
> Twitter: @patriknw
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at

Re: [akka-user] Re: ANNOUNCE: Akka Multi-DC Persistence

[Patrik Nordwall]

Hi Igor,

There are several reasons, such as:

   - we believe that it's useful to have the strong consistency with
   single-writer within one DC, and "only" have the asynchronous replication
   between DCs
   - failures and partitions within one DC can be handled by SBR, fits the
   model of Akka Multi-DC Clustering where each DC is autonomous
   - easier to implement, and therefore faster to market, we could perhaps
   add dynamic later if requested
   - number of DCs are typically static if it's used for availability zones
   or regions
   - note that our DC concept is only a logical grouping of nodes, so it
   doesn't have to be physical DC
   - performance and storage costs increase with number of replicas (DCs)
   so we recommend keeping it rather low anyway
   - we could build on top of existing infrastructure, Akka Persistence,
   and the Cassandra plugin


On Thu, Feb 1, 2018 at 2:01 PM, Igor Baltiyskiy  wrote:

> Hello Patrik,
>
> excellent news!
>
> May I ask why you chose to have static topology over dynamic one? With the
> latter one, I mean lifting the "only one writer" requirement locally as
> well, thus enabling always-available operation --- even during netsplit,
> where you could have multiple writers. CRDT-wise, this is the same, the
> difference is that replicas may appear and disappear at any time. I
> understand that this is more work than supporting only the static topology
> (with replica scope (DC) predefined), but seemingly not super-much, and it
> would cover the static case too.
>
> Is it because you feel that keeping one-writer-per-DC + using split brain
> resolver is the right thing to do? Or is it because you considered the cost
> of this solution to be prohibitively high? I'm asking because I was
> considering implementing a similar thing (causal reliable broadcast over
> Cassandra) with dynamic topology to handle the intra-DC netsplit scenario,
> so I'd be glad to hear your thoughts.
>
> Thanks
> Igor
>
>
> On Thursday, January 18, 2018 at 6:47:03 PM UTC+3, Patrik Nordwall wrote:
>>
>> Several exciting capabilities for multiple data centers have recently
>> been added to Akka. That includes Multi-DC Clustering
>>  that was added
>> in Akka a few months ago. Now we are thrilled to announce that Akka
>> Multi-DC Persistence has also been released. It's a commercial feature
>> available to Lightbend subscribers
>> .
>>
>> The screencast  and blog
>> post  should
>> give you a good overview of the feature, and more details can be found in
>> the documentation
>> 
>> .
>>
>> --
>>
>> Patrik Nordwall
>> Akka Tech Lead
>> Lightbend  -  Reactive apps on the JVM
>> Twitter: @patriknw
>>
>> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

Patrik Nordwall
Akka Tech Lead
Lightbend  -  Reactive apps on the JVM
Twitter: @patriknw

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: ANNOUNCE: Akka Multi-DC Persistence

[Igor Baltiyskiy]

Hello Patrik,

excellent news!

May I ask why you chose to have static topology over dynamic one? With the 
latter one, I mean lifting the "only one writer" requirement locally as 
well, thus enabling always-available operation --- even during netsplit, 
where you could have multiple writers. CRDT-wise, this is the same, the 
difference is that replicas may appear and disappear at any time. I 
understand that this is more work than supporting only the static topology 
(with replica scope (DC) predefined), but seemingly not super-much, and it 
would cover the static case too. 

Is it because you feel that keeping one-writer-per-DC + using split brain 
resolver is the right thing to do? Or is it because you considered the cost 
of this solution to be prohibitively high? I'm asking because I was 
considering implementing a similar thing (causal reliable broadcast over 
Cassandra) with dynamic topology to handle the intra-DC netsplit scenario, 
so I'd be glad to hear your thoughts.

Thanks
Igor

On Thursday, January 18, 2018 at 6:47:03 PM UTC+3, Patrik Nordwall wrote:
>
> Several exciting capabilities for multiple data centers have recently been 
> added to Akka. That includes Multi-DC Clustering 
>  that was added in 
> Akka a few months ago. Now we are thrilled to announce that Akka Multi-DC 
> Persistence has also been released. It's a commercial feature available to 
> Lightbend 
> subscribers .
>
> The screencast  and blog post 
>  should 
> give you a good overview of the feature, and more details can be found in 
> the documentation 
> 
> .
>
> -- 
>
> Patrik Nordwall
> Akka Tech Lead
> Lightbend  -  Reactive apps on the JVM
> Twitter: @patriknw
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.