Be careful with Flink. IME it's got a long way to go. The core model is
fantastic but there's a lot of low-hanging fruit that needs to be fixed.
On Tuesday, May 2, 2017 at 4:03:12 PM UTC-6, Evan Chan wrote:
>
> Hi Shiva,
>
> Spark will likely be too high latency for you. Practical minimal batc
Two possibilities:
1. Go out-of-band and drop down to Http using, say, Netty. Flink does this
(it uses Akka remoting for command-and-control but transmits data using
Netty).
2. Stick the files on a blob store like S3 and just send messages telling
the other actor a file is available (or whate
Leaving aside the tone of this post...
Have you looked at the longer example of the DSL?
http://doc.akka.io/docs/akka-http/current/scala/http/routing-dsl/index.html#longer-example
It has examples of how to delegate an endpoint's business logic off to
actors or futures.
On Sunday, April 2, 2017
One option, which may not be feasible depending on your use case, is to
simply replay *all* events from scratch, rebuilding current state on the
fly when needed. If you can configure enough retention on your Kafka
cluster, it's an option.
Another is to keep periodic snapshots so that you have
I've used it quite heavily in the past
(https://www.conspire.com/blog/2013/10/akka-at-conspire-part-5-the-importance-of/).
I would argue that if you still need to use actors, it's still a
best-practice. Excellent way to add backpressure to a process that
involves multiple actors across multi
+1 Guido.
We use Kafka for this purpose.
On Wednesday, September 7, 2016 at 5:34:09 AM UTC-6, Guido Medina wrote:
>
> Short answer is no, no, no, you need to interconnect the DCs using some
> recoverable mechanism otherwise your cluster will suffer.
> In other words, don't form Akka clusters amo
Akka is generally not recommended for spanning data centers.
I would consider the cluster client, though personally I'd probably choose
Kafka for this role, with separate clusters in each DC. IME cross-DC
communication is generally a scenario where you really want to make sure
messages are per
Did you add the dependency?
"com.typesafe.akka" %% "akka-persistence-query-experimental" % "2.4.8"
On Sunday, July 10, 2016 at 3:03:15 PM UTC-6, Jakub Hajto wrote:
>
> I'm on version 2.4.8
>
> W dniu niedziela, 10 lipca 2016 22:21:41 UTC+2 użytkownik Jakub Hajto
> napisał:
>>
>> I have encounte
You're probably on an old version.
http://doc.akka.io/api/akka/2.4/#akka.persistence.query.PersistenceQuery
On Sunday, July 10, 2016 at 2:21:41 PM UTC-6, Jakub Hajto wrote:
>
> I have encountered weird info. I tried to use PersistenceView but i saw
> message that it's deprecated and told me to
The mention of VPN has me tingling. Are you trying to join to a cluster
across a WAN?
On Sunday, June 12, 2016 at 12:20:08 PM UTC-6, kraythe wrote:
>
> cluster {
>> // FIXME cant use static seed nodes config!
>> seed-nodes = [
>> *"akka.tcp://application@**w.x.y.z*
>> *:2551"*
Unless you've profiled and found it to be a problem, worry about it later.
On Wednesday, June 8, 2016 at 10:10:37 AM UTC-6, kraythe wrote:
>
> Sorry but i have another newbie question. I have an actor that uses
> cluster sharding. There are potentially millions of these in the sytem
> concurrent
Might be best to send those over Kafka. Millions of updates per minute is
a trivial load for a properly tuned Kafka cluster, doesn't even require
more than 2-3 machines.
On Thursday, June 2, 2016 at 3:09:31 PM UTC-6, kraythe wrote:
>
> Good to know. I think I could batch the updates if need be.
How are you coordinating state between logically-equal actors on different
physical nodes?
On Wednesday, June 1, 2016 at 3:24:57 PM UTC-6, kraythe wrote:
>
> So the reason I didn't think this was cluster sharding is that I actually
> want these supervisor actors (and their supervised children) t
Great post Matthew!
On Sunday, May 22, 2016 at 4:51:39 PM UTC-6, Matthew Howard wrote:
>
> ugh double-post... I left out the Pat Helland link
> https://www.youtube.com/watch?v=EWSRbRUH7pg
>
> On Thursday, May 12, 2016 at 11:41:24 PM UTC-4, kraythe wrote:
>>
>> I have a system that is a tradition
I used to run an Akka cluster on top of etcd. I just had a wrapper script
that grabbed seed hosts from etcd and passed them as flags to my java
command.
We were using fleetd to deploy our Akka cluster nodes as Docker instances.
Each node had a sidekick job in fleetd that updated etcd with the
I don't want to rain on anyone's parade, but I think there's a reason that
products like Solr and Elasticsearch are complicated: search is hard!
Getting it right at scale is *really tough*.
Personally, I would gladly trade the operational complexity of ES/Solr over
reimplementing them myself.
Don't pass auth info into the *system, *pass with with each message into
the *actor*. When a request comes in and its identity is authenticated,
pass that authentication along with your messages. Then, if needed, let
your actors figure out authorization based on that authenticated identity
wi
Congrats Roland!
On Monday, March 7, 2016 at 10:42:32 AM UTC-7, rkuhn wrote:
>
> Dear fellow hakkers,
>
> as of today I am passing on the baton of Akka Tech Lead @ Lightbend to
> Patrik Nordwall. Don’t worry, I will stay around and keep working on Akka,
> but it will no longer be my day job come
Yes.
But without knowing more there's no way we can know if it's an appropriate
choice or not.
On Wednesday, March 2, 2016 at 9:15:50 AM UTC-7, Oscar Gabriel Reyes Pupo
wrote:
>
> Hello,
>
> I'm new in Akka. After reading some examples, I have a simple question, It
> is suitable to use Akka fo
That sounds like it may have the potential to violate actor
encapsulation—actors should only be created by the ActorSystem itself. It
would depend on the exact implementation but that sounds worrisome to me.
Why not just use composition? Let your actors contain and manage business
service obj
Actors aren't necessarily stopped due to user-land messages. If you use
context.stop(ref), what would be the message in postStop? Though I suppose
it could be an Option.
That said, to answer your question, it is definitely intentional.
If an exception is thrown while a message is being proc
Don't create the replacement actor until you've received a Terminated
message for the original actor.
On Friday, October 2, 2015 at 8:58:54 AM UTC-6, Kostas kougios wrote:
>
> I create a named actor, stop it and recreate it with the same name.
> Because stop is async, I end up with exceptions li
I'm not quite sure but it sounds like work pulling pattern + cluster
sharding is what you're looking for.
On Wednesday, August 26, 2015 at 2:22:32 PM UTC-6, kraythe wrote:
>
> Interesting ideas. Actor per request is something i didn't know about. I
> will research that. That would of course prev
aturday, August 22, 2015 at 11:28:01 AM UTC-5, Ryan Tanner wrote:
>>
>> case class SmartMessage(data: String, fn: String => Unit)
>>
>>
>> var mutableState: String = "this can change"
>>
>>
>> remoteActor ! SmartMessage("some data&q
case class SmartMessage(data: String, fn: String => Unit)
var mutableState: String = "this can change"
remoteActor ! SmartMessage("some data", x => println(x + mutableState))
On Saturday, August 22, 2015 at 8:12:02 AM UTC-6, kraythe wrote:
>
> I don't know what you mean by that. Callable or
Functions aren't serializable, they can't be sent across the network so
this would break location transparency.
Perhaps with spores/SIP-21 you could shoehorn something in but there still
seem to be a lot of pitfalls. And of course I don't think there's anything
similar for the Java API.
To
ReceivePipeline looks really great. When do you think it'll be released?
On Wednesday, March 25, 2015 at 3:38:47 AM UTC-6, Patrik Nordwall wrote:
>
> Hi Eric,
>
> sender() has caused a lot of trouble in practise, and we would do it
> differently if we would have known. We don't want to add a sim
Indeed. One thing that is especially tricky is how the lifecycle of your
own actor system integrates with the tests. If you don't clean up properly
you can wind up inadvertently reusing actors across test suites which can
get quite confusing if they're at all stateful.
On Thursday, February 1
Well that was a little harsh.
OP (and others): I've definitely run in to what I can only call "weirdness"
when trying to rely too heavily on Play's internal actor system for my own
purposes. I've found it's much more reliable to create my own (I use a
plugin for this).
On Tuesday, February
>From the documentation on stopping actors:
Since stopping an actor is asynchronous, you cannot immediately reuse the
> name of the child you just stopped; this will result in an
> InvalidActorNameException. Instead, watch the terminating actor and
> create its replacement in response to the Te
If you want distributed pub/sub, I would use an actual pub/sub system.
Akka can certainly do it, but Kafka or RabbitMQ are built *specifically
for that purpose, *especially if you want distributed pub/sub. Of course
the publishers and consumers on either end can be Akka-based.
Though there is
y is a little
> lacking. I'm super glad that business like Conspire are giving back to the
> community like this - your 5 part blog series has been absolutely
> invaluable to us.
>
> Thanks again, Ryan,
>
> Kane
>
>
> On Wednesday, 7 January 2015 02:39:57 UTC+
We're also deploying an Akka cluster on CoreOS with Docker. We deploy the
same fat JAR for every node (in fact, the exact same docker image) and then
change their behavior by setting the role via environment variables.
In our case, each role has a "bootstrap" class which sets up whatever
role-
Rather than simply sending all messages to the child actor, you could only
send messages in response to a "request for work" message from the child,
holding all other messages in an internal queue until ready. Then, if the
child stops itself, it *won't* request work and the parent will correctl
As you can see, futures have their own form of error handling. The
behavior you're seeing is exactly what's supposed to happen. Futures can
be used completely independently of actors so if you want failed futures to
trigger Akka's supervision, you'll need to do so yourself. In this case,
if
I think it really depends on how you've designed your cluster. If you make
use of cluster roles, you could just redeploy specific parts of your
cluster (e.g., we have an "analytics" role in our cluster and I could just
redeploy those nodes *if* their inter-node classes haven't changed). Doing
Just to be clear, these are all *Akka* actors, right? Parts of your post
make it sound as if some other Java actor library might be involed.
If it's just Akka actors written in Java, there's no problem creating Java
actors in Scala and vice-versa. We have a mixed language codebase and that
w
is
> experimenting on:
> https://github.com/rkuhn/akka/tree/wip-g%C3%A5lbma-step1/akka-typed
> Not sure if that's the latest and greatest (branch), probably it is.
> Please treat this as "random draft and ideas", it's not even an early
> preview *yet*.
>
> In
I saw on Twitter that Roland promised type safety in 3.0 during ReactConf
today. I recall a very preliminary preview from a few months ago but the
link no longer works:
https://github.com/rkuhn/akka/blob/wip-g%C3%A5lbma-step1/akka-actor/src/main/scala/akka/typed/Sample.scala
Is there anything
> There are two ways to run current akka in docker containers. If you run
all containers on the same host, then akka containers can talk with each
other using container local addresses. Another way is to start containers
with --net=host flag, which breaks network containerization but allows
con
seen! Tons of help
there if anyone else considers going this same path.
On Saturday, October 4, 2014 3:09:24 PM UTC-6, Rafał Krzewski wrote:
>
> Sounds great! I'm looking forward to the blog post :)
>
> Cheers,
> Rafał
>
> W dniu czwartek, 2 października 2014 16:46:04 UTC+2
I've been working on this the past few weeks. Here's what I'm currently
doing.
Background:
* CoreOS as the host OS
* Using etcd/fleet for service discovery and scheduling
* Using SkyDNS in order to address nodes by their logical name, also allows
nodes to bind to that name rather than the IP of
still registers itself with etcd?
>
> On Mon, Sep 29, 2014 at 8:34 AM, Ryan Tanner > wrote:
>
>> We're in the process of moving our Akka cluster over to CoreOS using etcd
>> + skydns for service discovery. Our dev environment has been running on
>> that for
We're in the process of moving our Akka cluster over to CoreOS using etcd +
skydns for service discovery. Our dev environment has been running on that
for about two weeks now. Akka nodes publish their network location into
etcd and then get their seed nodes out of the same, with skydns mapping
I've never used it in production but wy back I built a proof-of-concept
based on one of the Play Framework sample apps using Redis pub/sub to feed
a WebSocket based chat room.
https://github.com/ryantanner/websocketchat-redis
It does extremely little, really just round-trips messages across
I've been toying with this idea lately. Not so much using Neo4j as the
storage for Akka Persistence but rather using Akka Persistence to store the
events that cause changes to Neo4j so that the graph can be rebuilt later
on without going all the way back to source data (which takes a long time
When those large messages are received, what's supposed to happen? It's
possible that whatever processing is triggered by that message is so CPU
intensive that Akka's internal remoting can't get any time for its own
messages.
On Wednesday, August 6, 2014 12:08:20 PM UTC-6, Syed Ahmed wrote:
>
I think that's significantly simplifying the issue. Notably, if you have
long-running, CPU-intensive code and you blindly throw it into futures
within an actor, you can easily overwhelm your system. I've had major
problems with this. Really, this comes down to your specific requirements.
Wi
n,
>
>
> On Tue, Jul 22, 2014 at 8:06 PM, Ryan Tanner > wrote:
>
>> This is an issue I've been dealing with for a while now. I was hoping
>> that the fix for log-remote-lifecycle-events = off in 2.2.4 would've fixed
>> it but it hasn't.
>>
&g
This is an issue I've been dealing with for a while now. I was hoping that
the fix for log-remote-lifecycle-events = off in 2.2.4 would've fixed it
but it hasn't.
During multi-JVM cluster testing, we remove and re-add a node to test
failure handling:
testConductor.exit(BackendConfig.analytics
You could call context.unwatch(worker) and then context.stop(worker) from
your supervisor actor. Then you won't get the Terminated message, though
you also won't find out about a crash if the actor were the die for
whatever reason before being stopped.
For the reason I might lean towards just
Elsewhere you've said that there are performance downsides to
context.become, specifically that this is the reason sender() is a function
rather than doing something like def receive(sender: ActorRef): Receive = {
... }, which would eliminate the problem of closing over sender() in
futures with
Akka offers only two guarantees:
* At-most-once delivery. Each message will be delivered either 0 or 1
times. There is no guarantee of exactly-once delivery.
* Messages will be received in order for a given pair of actors.
http://doc.akka.io/docs/akka/2.3.3/general/message-delivery-reliability
Unrelated to the OP's issue, but so long as we're discussing logback, a
problem we've always had with logback is it's own debug output during
multi-JVM cluster tests. Logback outputs its status info for every JVM in
the test. Multiple times. No one on our team has been able to get it to
stop
This is just a stab in the dark, but does the actor that receives the ask()
request use futures in it's receive function? I'm wondering if maybe
you're closing over sender() improperly, causing some replies to go to the
wrong actor.
On Saturday, May 10, 2014 2:35:43 AM UTC-6, Giovanni Ruggiero
>
> 1) which router/deployed I need to use in order to start actors of certain
> type on nodes with an appropriate role?
http://doc.akka.io/docs/akka/2.3.2/scala/cluster-usage.html#Cluster_Aware_Routers
2) I want to share a "job state" with all actors involved in the job
> processing, so even
On Saturday, May 10, 2014 10:34:04 AM UTC-6, Ryan Tanner wrote:
>
> Glad to help!
>
> You're understanding of (1) is mostly correct, but the facades don't
> subscribe to DeathWatch on the supervisor actors, just cluster event
> notifications.
>
> 1a: All work mes
Glad to help!
You're understanding of (1) is mostly correct, but the facades don't
subscribe to DeathWatch on the supervisor actors, just cluster event
notifications.
1a: All work messages (start, acknowledge, complete, error) are tagged with
a UUID for that chunk of work and that's used to ro
I think you misunderstood what cluster events are for.
MemberUp/Down aren't fired when your actors come up, they're fired when a
new member joins the cluster. If you want an actor to know when another
actor is ready, you need to explicitly send a message. You can however use
DeathWatch to kno
Hi Luis,
1. First just to make sure I understood what you guys ended up doing,
> whenever your supervisor crashes, an event is sent to some sort of event
> bus to which workers your are subscribed to which let's them know that
> their supervisor is down. When this happens, each worker that was
d correct that to make post fully readable again :-).
>
> Chris
>
> On Tuesday, May 6, 2014 8:38:19 AM UTC-7, Ryan Tanner wrote:
>>
>> I'm the author of the blog post from Conspire you referenced. In our
>> case, losing the supervisor isn't a problem because all work
I'm the author of the blog post from Conspire you referenced. In our case,
losing the supervisor isn't a problem because all work is generated from a
SQL database. If the supervisor crashes, we can just start over. Our
worker nodes subscribe to cluster event notifications and will queue their
That's pretty much what we do. We give one of our nodes the "supervisor"
role and start a ClusterSingletonActor on it which spawns a control FSM (we
call it a Pipeline) which in turn spawns worker FSMs (PipelineWorkers).
Those actors then marshall data and delegate work to stateless actors
el
We like to isolate side effects (DB, S3 stuff, etc) in their own actors and
include them in the constructor of the actor doing the actual work. Then
we can just use TestProbes when we construct the worker actor in unit tests.
In our integration tests, we use concrete actors for everything and l
It was discussed last week ;)
https://groups.google.com/forum/#!topic/akka-user/zPJOLe0V760
On Monday, April 7, 2014 10:36:11 AM UTC-6, Scott Clasen wrote:
>
> From my testing I see that FSMs dont fire transitions on stay() or
> goto(SameState).
>
> I haven't found any docs that say that why tha
I should've just looked at the source to begin with, that answered all my
questions.
On Tuesday, March 4, 2014 12:15:35 PM UTC-7, Ryan Tanner wrote:
>
> Perhaps I'm misunderstanding the use of durable mailboxes. Here's what
> I'm trying to accomplish:
>
>
Perhaps I'm misunderstanding the use of durable mailboxes. Here's what I'm
trying to accomplish:
We have a long-running background process on one of our servers (Play
Framework) which fetches data from a URL and eventually persists it in
Elasticsearch based on other events in the system. Esse
We just started getting that same error on 2.2.3. The only major change
we've made lately is to move from m1.large EC2 instances to m3.large, no
other changes to config or how we use our cluster since December. For now
we've upped akka.actor.creation-timeout to 60s, I'll have to check with the
>From the docs:
If the current actor behavior does not match a received message, unhandled is
called, which by default publishes an akka.actor.UnhandledMessage(message,
sender, recipient) on the actor system’s event stream (set configuration
item akka.actor.debug.unhandled to on to have them co
I'm not terribly familiar with them yet but a lot of that sounds similar to
the reasoning behind Play's iteratees. Is that the sort of abstraction
you're talking about or am I completely off-base?
On Saturday, January 18, 2014 5:01:55 AM UTC-6, drewhk wrote:
>
> One of the important concepts in
over the basic clustering
> membership functionality.
>
> Matt - we are also using a variation of "Balancing Workload Across Nodes"
> (pre Akka Cluster) and it works great.
>
> Thanks,
>
> On Friday, January 10, 2014 9:15:20 AM UTC-5, Ryan Tanner wrote:
>>
>
We use that pattern in conjunction with clustering. We use a cluster-aware
router to deal with node membership within a worker role but we never
actually send messages to the router's ActorRef, we let the routees
register themselves with the "role leader" as we call it when they start.
On Frid
We have an Akka cluster that might be relevant to your requirements.
Sounds like Akka clustering might suit your requirements very well. We
have an analytics process that can take up to 10 minutes per user depending
on how much data they have. We have a "supervisor" node in the cluster
which
Check on /var/log/syslog, your process could be getting killed by the Linux
out-of-memory killer. We had the same problem for a while: random crashes
under load with no error in the logs.
On Tuesday, December 31, 2013 4:05:40 AM UTC-6, SS wrote:
>
> Hi,
>
> I'm running a small Akka cluster (2 n
There are a number of options:
1) Route 53 DNS
2) Have your config management tool (Chef, Puppet, etc) write out the IPs
of your newly-provisioned seed nodes to your application.conf files.
3) Zookeeper
We've moved to (2) and as much as I don't like Chef it does work but I
think Chef and Vagran
8, drewhk wrote:
>
> Hi Ryan,
>
> Is there any reason why you have not used the Amazon EC2 API? At one point
> we used simply the Java API to discover nodes.
>
> -Endre
>
>
> On Mon, Dec 23, 2013 at 1:49 AM, Ryan Tanner
> > wrote:
>
>> We use Vagran
We use Vagrant and Chef to deploy our Akka cluster on AWS. At the moment
we're using a quite stupid method of service discovery: I wrote a bash
script that uses "vagrant ssh" to get the IP of each EC2 instance after
it's been created (curl'ing the AWS magic metadata IP) and then update the
/et
No. For one, actor A may have a bunch of messages in its mailbox while
actor B has none, meaning A will have to process all those before
processing 'x' but B will process 'y' immediately.
On Friday, December 20, 2013 9:08:34 AM UTC-7, Tim Pigden wrote:
>
> Hi in the following
>
> a ! x
> b ! y
78 matches
Mail list logo
