[akka-user] When to use thread-pool-executor
Hi, In most cases, fork-join-executor is better than thread-pool-executor, however I'm wondering what's the best scenario to use thread-pool-executor. Thanks Leon -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] [akka-stream 0.9] Create Flow[A,B] from Publisher[A] and Subscriber[B]
Id love to see an example of two actors that are both publisher and subscriber connected. I've been trying to get that going and have been having a bunch of trouble getting it working. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Broadcast messages are not working in a cluster?
Not sure if it relative to the problem you want to solve, but you can do
following stuff to get all routees
def guardianReceive: Actor.Receive = {
case DiscoverChildren ⇒
router ! GetRoutees
case Routees(routees) =
routees.foreach {
case ActorRefRoutee(ref) = addWorker(ref)
ref ! GuardianRegistration
}
}
On Wednesday, December 24, 2014 5:12:39 PM UTC-5, Eugene Dzhurinsky wrote:
I'm trying to create a simple proof of concept for working with a group of
actors in a cluster.
I've created the following example
https://gist.github.com/jdevelop/b57c9b1b0e0063b4ccc1
the problem is - sending a *Broadcast* message to a
*consistent-hashing-group* router, obtained from the cluster
configuration - seems to send a message only to a single routee - not all
of them.
The sample output from the code above:
Started akka://Test/user/routeeA/$h
Started akka://Test/user/routeeA/$i
Started akka://Test/user/routeeA/$f
Started akka://Test/user/routeeA/$j
Started akka://Test/user/routeeA/$a
Started akka://Test/user/routeeA/$g
Started akka://Test/user/routeeA/$b
Started akka://Test/user/routeeA/$c
Started akka://Test/user/routeeA/$e
Started akka://Test/user/routeeA/$d
Started akka://Test/user/splitter/$b
Started akka://Test/user/splitter/$a
Received data to process
Broadcasting work for akka://Test/user/routerA
Received broadcast from akka.tcp://Test@127.0.0.1:2552/user/splitter/$a
Worker Actor[akka.tcp://Test@127.0.0.1:2551/user/routeeA/$a#410953742]
wants some data
Process to respond akka.tcp://Test@127.0.0.1:2552/user/splitter/$a
Worker Actor[akka.tcp://Test@127.0.0.1:2551/user/routeeA/$a#410953742]
completes its job
As it is seen - the only one actor receives the broadcast message - not
10, which is weird.
Can you please advice how to fix that (without using
DistributedPubSubMediator)?
Thank you!
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] Event Sourcing with eventual consistency guarantees
Hi guys, I am currently prototyping a new app which involves transferring money between various accounts. On a very high level I have a system account (which is used to collect fees), one account for each member in the platform (many of them), and group accounts. Money needs to be moved on a frequent basis from member accounts (M) to group accounts (G) and the system account (S). Traditionally a transaction like this would be accomplished atomically: tx start - read M account to check for sufficient funds - deduct money from M account - add money to S account - add money to G account tx end I know already that the system account S will be involved in most of these transactions and eventually become a bottleneck in the platform. Another requirement in the platform is to have all money movements fully auditable, which is a very common requirement. So using event sourcing and CQRS comes to mind to solve this problem. Initially my thinking is to have a single persistent actor for each member account (M), a persistent actor for the system account (S) and a single persistent actor for each group account (G). Each will store events related to their respective accounts and offer different views (to keep the balance, monitor fraudulent behaviour, statistics, etc). In addition I would like to have a persistent actor to persist the overarching transaction events (lets call it TX actor), mostly for bookkeeping statistics via its views. The idea is that a 'transaction' starts with this TX actor which then issues money transfer commands to all account actors involved (M, G, S), monitors their responses and either persists his own event of the successful transfer or issues a compensation commands in case something goes wrong). Because there will be a large number of members in the platform I would like to use Akka clustering where the persistent actors may live on different nodes. I have played with hash based routing and cluster sharding to address the single writers per account. My problem at the moment is to figure out how each transaction can become eventually consistent (say within a few seconds) in a clustered environment like this where there are multiple points of failure. Obviously I need to ensure that a transaction cannot leave the system in an inconsistent state and potential manual compensations are also subject to their own errors. I guess my question is if anyone has used Akka persistence / event sourcing / CQRS for handling financial transactions in a clustered environment? If so can you share some experiences or ideas, especially around ensuring (eventual) consistency? It seems like event sourcing is a good solution to overcome some of the bottlenecks which a SQL database will create (especially where there is one very contentious resource (account S)) but there is are not many reports out there where people have used ES specifically for money handling (perhaps for good reason ;)). Thanks in advance. -Stefan -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Event Sourcing with eventual consistency guarantees
Hi Stefan, Systems like you describe definitely *are* implemented in terms of persisting events. I would highly recommend these following videos / papers as a holiday-read”: * Eric Evans (Dad of DDD) about modeling to such constraints WITH thinking of time and WITHOUT transactions http://www.ustream.tv/recorded/46744749 -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io On 25 December 2014 at 21:39:24, Stefan Schmidt (stsme...@gmail.com) wrote: Hi guys, I am currently prototyping a new app which involves transferring money between various accounts. On a very high level I have a system account (which is used to collect fees), one account for each member in the platform (many of them), and group accounts. Money needs to be moved on a frequent basis from member accounts (M) to group accounts (G) and the system account (S). Traditionally a transaction like this would be accomplished atomically: tx start - read M account to check for sufficient funds - deduct money from M account - add money to S account - add money to G account tx end I know already that the system account S will be involved in most of these transactions and eventually become a bottleneck in the platform. Another requirement in the platform is to have all money movements fully auditable, which is a very common requirement. So using event sourcing and CQRS comes to mind to solve this problem. Initially my thinking is to have a single persistent actor for each member account (M), a persistent actor for the system account (S) and a single persistent actor for each group account (G). Each will store events related to their respective accounts and offer different views (to keep the balance, monitor fraudulent behaviour, statistics, etc). In addition I would like to have a persistent actor to persist the overarching transaction events (lets call it TX actor), mostly for bookkeeping statistics via its views. The idea is that a 'transaction' starts with this TX actor which then issues money transfer commands to all account actors involved (M, G, S), monitors their responses and either persists his own event of the successful transfer or issues a compensation commands in case something goes wrong). Because there will be a large number of members in the platform I would like to use Akka clustering where the persistent actors may live on different nodes. I have played with hash based routing and cluster sharding to address the single writers per account. My problem at the moment is to figure out how each transaction can become eventually consistent (say within a few seconds) in a clustered environment like this where there are multiple points of failure. Obviously I need to ensure that a transaction cannot leave the system in an inconsistent state and potential manual compensations are also subject to their own errors. I guess my question is if anyone has used Akka persistence / event sourcing / CQRS for handling financial transactions in a clustered environment? If so can you share some experiences or ideas, especially around ensuring (eventual) consistency? It seems like event sourcing is a good solution to overcome some of the bottlenecks which a SQL database will create (especially where there is one very contentious resource (account S)) but there is are not many reports out there where people have used ES specifically for money handling (perhaps for good reason ;)). Thanks in advance. -Stefan -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Event Sourcing with eventual consistency guarantees
Hit command-enter too soon by accident before finishing my email - sorry! Here’s the rest. To read: And also Pet Helland’s (at that time at Amazon) legendary Life Beyond Distributed Transactions” In general though. What you described as a transaction is in fact a series of events _plus_ a so called “Process Manager”. This is DDD terminology, you’ll find more about thise in Books around Domain Driven Design or some of Vaughn’s talks ( I did not fully watch this one, but topic wise seems to be just what you’re asking https://vimeo.com/104021785 ). So in essence, there is a persistent entity, which “takes care to drive the operations to their end”, for example it sees that this and that “transaction” (or simply “process”) did not succeed, maybe some message was lost, or maybe some server was down and we couldn’t proceed etc. So for furher reading on process managers I’ll refer to The “CQRS Journey” which I think you should go over while thinking about your business, esp. this chapter on “Sagas http://msdn.microsoft.com/en-us/library/jj591569 While we’re discussing this, please remember that money transactions don’t always offer full guarantees anyway. Typical examples here being ATMs which can be not connected at all times, and *may* give out 50 bucks without checking if you really can because it’s *low risk* and they can block your account afterwards (hello eventual consistency!) anyway if you did in fact overdraw. However the same logic does not apply if you’re trying to get a few k out of an ATM, that won’t be as “low risk” of course, so the checks will have to be run before handing out the monies. So instead of transactions, we can (just like real people) keep a process running and react to things happening around it, and if we failed - take compensating actions - by always appending data + making sure *global* invariants are perserved (check Eric Evan’s talk I liked for awesame examples of this). I hope this helps! Merry x-mas hakking! -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io On 25 December 2014 at 21:39:24, Stefan Schmidt (stsme...@gmail.com) wrote: Hi guys, I am currently prototyping a new app which involves transferring money between various accounts. On a very high level I have a system account (which is used to collect fees), one account for each member in the platform (many of them), and group accounts. Money needs to be moved on a frequent basis from member accounts (M) to group accounts (G) and the system account (S). Traditionally a transaction like this would be accomplished atomically: tx start - read M account to check for sufficient funds - deduct money from M account - add money to S account - add money to G account tx end I know already that the system account S will be involved in most of these transactions and eventually become a bottleneck in the platform. Another requirement in the platform is to have all money movements fully auditable, which is a very common requirement. So using event sourcing and CQRS comes to mind to solve this problem. Initially my thinking is to have a single persistent actor for each member account (M), a persistent actor for the system account (S) and a single persistent actor for each group account (G). Each will store events related to their respective accounts and offer different views (to keep the balance, monitor fraudulent behaviour, statistics, etc). In addition I would like to have a persistent actor to persist the overarching transaction events (lets call it TX actor), mostly for bookkeeping statistics via its views. The idea is that a 'transaction' starts with this TX actor which then issues money transfer commands to all account actors involved (M, G, S), monitors their responses and either persists his own event of the successful transfer or issues a compensation commands in case something goes wrong). Because there will be a large number of members in the platform I would like to use Akka clustering where the persistent actors may live on different nodes. I have played with hash based routing and cluster sharding to address the single writers per account. My problem at the moment is to figure out how each transaction can become eventually consistent (say within a few seconds) in a clustered environment like this where there are multiple points of failure. Obviously I need to ensure that a transaction cannot leave the system in an inconsistent state and potential manual compensations are also subject to their own errors. I guess my question is if anyone has used Akka persistence / event sourcing / CQRS for handling financial transactions in a clustered environment? If so can you share some experiences or ideas, especially around ensuring (eventual) consistency? It seems like event sourcing is a good solution to overcome some of the bottlenecks which a SQL database will create (especially where there is one very contentious resource (account S)) but
[akka-user] Running a stream inside an Actor ?
My understanding is that a running stream is a bunch of actors underneath executing the flow. Assuming this to be true, is there any restriction or concerns of running a stream inside a normal Akka actor ? Thanks -Soumya -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Recording latency of operations that return a Future inside an akka stream
This is related to this
https://groups.google.com/forum/#!topic/akka-user/NrSkEwMrS3s thread but
sufficiently different that I decided to create new thread. Hope that's
okay.
I would like to create a histogram of latency of a large number of set
operations ( set returns a Future[Boolean]) using LatencyUtils
https://github.com/LatencyUtils/LatencyUtils
Basically I need to start recording the time before the set operation
(inside mapAsyncUnordered(k = redis.set(k + rnd, message))) and then
somehow record the end time in a map operation( .map( //record the end time
here) after this. I'm having a hard time trying to figure this out.
My understanding is that the even though the mapAsyncUnordered doesn't
maintain the order of operations the map following the mapAsynUnordered
will maintain the order from the previous stage because of TCP maintaining
the order. Is this correct?
val redis = RedisClient(localhost)
val random1 = UUID.randomUUID().toString
def insertValues(rnd: String): Flow[Int, Boolean] = {
Flow[Int].mapAsyncUnordered(k = redis.set(k + rnd, message)).map(
//record the end time here)
}
val blackhole = BlackholeSink
val maxSeq = 500
val seqSource = Source( () = (1 to maxSeq).iterator )
*val streamWithSeqSource =
seqSource.via(insertValues(random1)).runWith(blackhole)*
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] How to convert a 'process' into a set of actors? Or what is the correct way of adding actors to this code?
I've written some scala code to simulate a stock exchange (actually just the order book) at https://gist.github.com/falconair/d669cac75ecab0010e9a I wrote this just to learn scala so please don't expect production code here. I'm having trouble understanding how to convert this to use actors. Currently, OrderBook is an object with only three public methods: processOrderBookRequest() listenForEvents() listenForMarketData() The first one, processOrderBookRequest accepts events, such as create new order, cancel order, etc. listenForEvents() is used to subscribe for responses to those events, such as new order accepted, order canceled, order filled, order rejected, etc. listenForMarketData() is also used to subscribe to updates published by the OrderBook object. Any number of 'clients' can send requests or subscribe to updates. Note that events are being sent to OrderBook or sent by OrderBook to whoever subscribes to them. No one is supposed to directly call a method on the object to get its internal state. In other words, this is already supposed to operate as a 'process.' I can naturally turn OrderBook into an actor and publish events to it. I suppose those who are only interested in subscribing to market data can send a 'subscribe to market data' event to OrderBook where the object will add the sender to its list of subscribers...similar to how it is done now. Same is true for non-market data events. I'm not sure if what I'm thinking is the correct way of using actors. I'd love someone to take a look at the code (about 250 lines) and tell me if I'm headed in the wrong direction. If this is the wrong place to ask such a question, please let me know. Thanks -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
