> 21 juli 2016 kl. 18:39 skrev Mark Kaberman <mkaber...@gmail.com>:
>
> I was able to write a test app which reproduces the problem (attached). Based
> on the application.conf I would expect 65 instances of the actors to be
> created. When I start my actors (by executing POST request to
> http://localhost:8080/start <http://localhost:8080/start> with no body) I see
> 385 instances created. Which each sequential request extra 385 instances are
> added until the system runs out of memory.
I don’t know Spring that well, but could it be that you create one ActorSystem
per POST request? (you can verify that by looking at the number of scheduler
threads)
Another thing that I’m extremely curious about is why you are making all these
actors routers—this just creates a huge number of actors, very likely many more
than you have processors in your system. A tree of routers almost never makes
sense.
Regards,
Roland
>
> On Tuesday, June 14, 2016 at 10:47:55 AM UTC-4, rkuhn wrote:
>
>> 14 juni 2016 kl. 16:28 skrev Mark Kaberman <mkab...@gmail.com <>>:
>>
>> Hi Roland,
>>
>> As I wrote in my original post my application traverses the tree. The depth
>> of the three is small and the paths are known in advance, but the breath is
>> very large. So I have relatively small number of actors (and actorRouterName
>> ) arguments
>
> This is only consistent if I assume that the breadth of the tree is small and
> the depth is very large (because otherwise there would need to be a very
> large number of actorRouterNames). In this case I guess that you are simply
> creating way too many actors—and since you never stop them you will
> eventually drown in them. Using routers of size 10 at every level is going to
> explode exponentially as you are certainly aware …
>
> In any case: without seeing the full code (which I wouldn’t have time to
> read) the most sensible hint is that you should use a memory analyzer to
> verify my guess or find a different explanation (i.e. where all these beans
> are coming from). Akka for sure does not create anything when calling
> context().child(...).get() (which is easy to prove since no Props are given).
>
> Regards,
>
> Roland
>
>>
>> My Akka config looks approximately like:
>>
>> /rootActor {
>> ...
>> }
>>
>> /rootActor/myActor1 {
>> ...
>> }
>>
>> /rootActor/myActor1/myActor2 {
>> ...
>> }
>>
>> /rootActor/myActor1/myActor2/myActor1 {
>> ...
>> }
>>
>>
>> - The actors are never stopped. The idea was to reuse the actors. They were
>> only supposed to be stopped as result of supervision strategy triggered by
>> traversal failures.
>>
>> - My actors are implemented as Spring beans. I am using Akka/Spring
>> integration from here
>> <https://github.com/typesafehub/activator-akka-java-spring>
>>
>> - sendAkkaMessage called from the context of the parent actor: as part of
>> the vertex discovery I determine if a vertex has children and if yes, of
>> what type. Then I call sendAkkaMessage with the proper parameters from the
>> context of an actor which is currently processing the vertex.
>>
>>
>> On Tuesday, June 14, 2016 at 9:29:37 AM UTC-4, rkuhn wrote:
>> Hi Mark,
>>
>> it is not easy to help you due to insufficient information:
>>
>> How many different actorRouterName arguments can there be? Will the
>> resulting number of actors fit into memory?
>> Do you or do you not stop all created actors eventually? Stopping an actor
>> requires a call to context().stop(self()) on every routee.
>> You talk about beans, but actors are not beans: where do you create beans
>> and are you certain that you do not retain references that prevent them from
>> being collected? (You should log from postStop() instead of from the
>> finalizer to see whether an actor gets stopped.)
>> What is the context in which the sendAkkaMessage method is called? How many
>> of these contexts exist?
>>
>> Regards,
>>
>> Roland
>>
>>> 14 juni 2016 kl. 14:39 skrev Mark Kaberman <mkab...@ <>gmail.com
>>> <http://gmail.com/>>:
>>>
>>> It does not.
>>>
>>> On Tuesday, June 14, 2016 at 8:14:20 AM UTC-4, √ wrote:
>>> Is supervision triggering? (i.e. restarts, possibly creating new beans)?
>>>
>>> On Tue, Jun 14, 2016 at 2:03 PM, Mark Kaberman <mkab...@ <>gmail.com
>>> <http://gmail.com/>> wrote:
>>> I added the static instantiation counter to my actor Spring bean (see my
>>> original post). When an actor is created via getContext().ActorOf(... I see
>>> number of logging entries coming from the bean's constructor indicating
>>> that n instances of the actor beans has been created where n matches
>>> nr-of-instances from my Akka config. When context().child.(..).get() is
>>> called, I see a single logging entry coming from bean's constructor
>>> indicating that a singe actor bean has been created. My logging indicates
>>> that getContext().ActorOf(.. is only called once per actor, but
>>> context().child.(..).get() is called constantly (as it should). So I see
>>> ever increasing counter logging indicating that more and more Spring beans
>>> are created.
>>>
>>>
>>> On Tuesday, June 14, 2016 at 7:38:37 AM UTC-4, Patrik Nordwall wrote:
>>> I'm 100% sure that the child.get method here doesn't create an actor. How
>>> do you see that?
>>>
>>> On Fri, Jun 10, 2016 at 10:43 PM, Mark Kaberman <mkab...@ <>gmail.com
>>> <http://gmail.com/>> wrote:
>>> I debugged my application and it seems to be a bug in Akka:
>>>
>>> My routers are defined similarly to each other as
>>> /myActor/ {
>>> dispatcher = my-pinned-dispatcher
>>> router = round-robin
>>> nr-of-instances = 10
>>> }
>>>
>>> When I debug into my actor creation and messaging method
>>>
>>> public sendAkkaMessage(String actorBeanName, String actorRouterName, Object
>>> message) {
>>> ActorRef actor = null;
>>> final scala.Option<ActorRef> child = context().child(actorRouterName);
>>> if (child != null && child.isDefined()) {
>>> actor = child.get();
>>> } else {
>>> actor =
>>> getContext().actorOf(SpringExtProvider.get(system).props(actorBeanName).withRouter(new
>>> FromConfig()), actorRouterName);
>>> }
>>> actor.tell(message, self());
>>> }
>>>
>>> I see that when a new actor is created by calling getContext().actorOf()...
>>> I see 10 new Spring beans being created (as per Akka config). When an
>>> actor is created via child.get() I see one Spring bean is created. So it
>>> seems that in the case of a get() Akka disregards already created Spring
>>> beans and creates one more.
>>>
>>>
>>>
>>> On Friday, June 10, 2016 at 1:24:21 PM UTC-4, √ wrote:
>>> I don't know how your app works or what router config you are using so it
>>> is impossible for me to know what's happening.
>>>
>>> Perhaps you have Restarts happening which will create new MyActor instances
>>> but the old instances are still reachable by something else. Spring perhaps?
>>>
>>> Use a memory debugger and trace the reachability of those MyActor instances.
>>>
>>> --
>>> Cheers,
>>> √
>>>
>>> On Jun 10, 2016 6:03 PM, "Mark Kaberman" <mkab...@ <>gmail.com
>>> <http://gmail.com/>> wrote:
>>> Isn't the code:
>>>
>>> if (child != null && child.isDefined()) {
>>> actor = child.get();
>>> }
>>>
>>> supposed to fetch existing actor actor reference instead of creating a new
>>> one?
>>>
>>> If I call stop() at the end of onReceive() what happens to other instances
>>> of the same actor which could be processing different vertices? Will they
>>> be shut down as well?
>>>
>>>
>>>
>>> On Friday, June 10, 2016 at 10:48:48 AM UTC-4, √ wrote:
>>> If you create new actors continually and never stop any of them then you
>>> have by design got a leak.
>>>
>>> --
>>> Cheers,
>>> √
>>>
>>> On Jun 10, 2016 4:35 PM, "Mark Kaberman" <mkab...@ <>gmail.com
>>> <http://gmail.com/>> wrote:
>>> Hi Viktor,
>>>
>>> I never stop my actors explicitly (except as reaction to failure in
>>> supervision strategy). All actors process the vertex data in onReceive()
>>> method, determine if a vertex has children, get the children actor by
>>> calling my createActorRef, send the message to a child via tell (never ask)
>>> and exit onReceive.
>>>
>>> Regards,
>>>
>>> Mark
>>>
>>>
>>> On Friday, June 10, 2016 at 10:27:25 AM UTC-4, √ wrote:
>>> Hi Mark,
>>>
>>> Where are you stopping your actors?
>>>
>>> --
>>> Cheers,
>>> √
>>>
>>> On Jun 10, 2016 2:55 PM, "Mark Kaberman" <mkab...@ <>gmail.com
>>> <http://gmail.com/>> wrote:
>>> I have Akka application which is essentially traverses a very large tree
>>> where each vertex processing is done by an individual actor. Different
>>> kinds of vertices are processed by different actors. The actors are
>>> implemented as prototype Spring beans (they are derived from the same
>>> abstract class) and I am using Akka/Spring integration from Akka Spring
>>> integration <https://github.com/typesafehub/activator-akka-java-spring>.
>>> The only two differences between the githib example and my code is that I
>>> use configuration file to configure the routers and the way I get actor
>>> references.
>>> Since the example only uses one actor it creates it like
>>>
>>> system.actorOf(SpringExtProvider.get(system).props("CountingActor"),
>>> "counter");
>>>
>>> My actor system is hierarchical and I create my actors differently
>>>
>>> public ActorRef createActorRef(String actorBeanName, String
>>> actorRouterName) {
>>> ActorRef actor = null;
>>> final scala.Option<ActorRef> child = context().child(actorRouterName);
>>> if (child != null && child.isDefined()) {
>>> actor = child.get();
>>> } else {
>>> actor =
>>> getContext().actorOf(SpringExtProvider.get(system).props(actorBeanName).withRouter(new
>>> FromConfig()), actorRouterName);
>>> }
>>> }
>>>
>>>
>>> When my system is running for few days it runs out of memory. I ran the
>>> profiler and discovered that there is a huge number of actor's Spring beans
>>> being instantiated. So I instrumented my actors with the instance counters:
>>>
>>> public abstract class MyActor extends UntypedActor {
>>>
>>>
>>> private static AtomicInteger instantiationCount = new AtomicInteger(0);
>>>
>>> public MyActor() {
>>> logger.info <http://logger.info/>("ACTOR CREATED. Instantiation
>>> count {}", instantiationCount.getAndIncrement());
>>>
>>> }
>>>
>>> @Override
>>> protected void finalize() {
>>> logger.info <http://logger.info/>("ACTOR FINALIZED. Instantiation
>>> count {}", instantiationCount.getAndDecrement());
>>> }
>>> }
>>>
>>>
>>> When I run my application I see constant flow of "ACTOR CREATED" log
>>> entries with ever incrementing counter with the finalize() method never
>>> called. Eventually after few days of running the system runs out of memory.
>>> I am wondering if I am doing something wrong with obtaining the actor
>>> references or there is some issues with Akka/Spring integration
>>>
>>>
>>>
>>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
>>> >>>>>>>>>> Check the FAQ:
>>> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
>>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>>> >>>>>>>>>> <https://groups.google.com/group/akka-user>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to akka-user+...@ <>googlegroups.com <http://googlegroups.com/>.
>>> To post to this group, send email to akka...@ <>googlegroups.com
>>> <http://googlegroups.com/>.
>>> Visit this group at https://groups.google.com/group/akka-user
>>> <https://groups.google.com/group/akka-user>.
>>> For more options, visit https://groups.google.com/d/optout
>>> <https://groups.google.com/d/optout>.
>>>
>>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
>>> >>>>>>>>>> Check the FAQ:
>>> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
>>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>>> >>>>>>>>>> <https://groups.google.com/group/akka-user>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to akka-user+...@ <>googlegroups.com <http://googlegroups.com/>.
>>> To post to this group, send email to akka...@ <>googlegroups.com
>>> <http://googlegroups.com/>.
>>> Visit this group at https://groups.google.com/group/akka-user
>>> <https://groups.google.com/group/akka-user>.
>>> For more options, visit https://groups.google.com/d/optout
>>> <https://groups.google.com/d/optout>.
>>>
>>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
>>> >>>>>>>>>> Check the FAQ:
>>> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
>>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>>> >>>>>>>>>> <https://groups.google.com/group/akka-user>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to akka-user+...@ <>googlegroups.com <http://googlegroups.com/>.
>>> To post to this group, send email to akka...@ <>googlegroups.com
>>> <http://googlegroups.com/>.
>>> Visit this group at https://groups.google.com/group/akka-user
>>> <https://groups.google.com/group/akka-user>.
>>> For more options, visit https://groups.google.com/d/optout
>>> <https://groups.google.com/d/optout>.
>>>
>>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
>>> >>>>>>>>>> Check the FAQ:
>>> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
>>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>>> >>>>>>>>>> <https://groups.google.com/group/akka-user>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to akka-user+...@ <>googlegroups.com <http://googlegroups.com/>.
>>> To post to this group, send email to akka...@ <>googlegroups.com
>>> <http://googlegroups.com/>.
>>> Visit this group at https://groups.google.com/group/akka-user
>>> <https://groups.google.com/group/akka-user>.
>>> For more options, visit https://groups.google.com/d/optout
>>> <https://groups.google.com/d/optout>.
>>>
>>>
>>>
>>> --
>>>
>>> Patrik Nordwall
>>> Akka Tech Lead
>>> Lightbend <http://www.lightbend.com/> - Reactive apps on the JVM
>>> Twitter: @patriknw
>>>
>>>
>>>
>>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
>>> >>>>>>>>>> Check the FAQ:
>>> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
>>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>>> >>>>>>>>>> <https://groups.google.com/group/akka-user>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to akka-user+...@ <>googlegroups.com <http://googlegroups.com/>.
>>> To post to this group, send email to akka...@ <>googlegroups.com
>>> <http://googlegroups.com/>.
>>> Visit this group at https://groups.google.com/group/akka-user
>>> <https://groups.google.com/group/akka-user>.
>>> For more options, visit https://groups.google.com/d/optout
>>> <https://groups.google.com/d/optout>.
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> √
>>>
>>> --
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
>>> >>>>>>>>>> Check the FAQ:
>>> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
>>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>>> >>>>>>>>>> <https://groups.google.com/group/akka-user>
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to akka-user+...@ <>googlegroups.com <http://googlegroups.com/>.
>>> To post to this group, send email to akka...@ <>googlegroups.com
>>> <http://googlegroups.com/>.
>>> Visit this group at https://groups.google.com/group/akka-user
>>> <https://groups.google.com/group/akka-user>.
>>> For more options, visit https://groups.google.com/d/optout
>>> <https://groups.google.com/d/optout>.
>>
>>
>> --
>> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
>> >>>>>>>>>> Check the FAQ:
>> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> >>>>>>>>>> <https://groups.google.com/group/akka-user>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com <>.
>> To post to this group, send email to akka...@googlegroups.com <>.
>> Visit this group at https://groups.google.com/group/akka-user
>> <https://groups.google.com/group/akka-user>.
>> For more options, visit https://groups.google.com/d/optout
>> <https://groups.google.com/d/optout>.
>
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
> >>>>>>>>>> Check the FAQ:
> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> >>>>>>>>>> <https://groups.google.com/group/akka-user>
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com
> <mailto:akka-user+unsubscr...@googlegroups.com>.
> To post to this group, send email to akka-user@googlegroups.com
> <mailto:akka-user@googlegroups.com>.
> Visit this group at https://groups.google.com/group/akka-user
> <https://groups.google.com/group/akka-user>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
> <AkkaDemo.zip>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
