Re: [akka-user] DistributedPubSub - topic actor liveness race condition

2015-11-25 Thread Andrzej Dębski 
I created an issue: https://github.com/akka/akka/issues/19017

As for the contributing I can try to come up with an implementation 
solution for this problem but I will have to think how to handle messages 
that come during the gap between sending message to kill topic actor and 
receiving confirmation from death watch. 

W dniu piątek, 20 listopada 2015 16:33:48 UTC+1 użytkownik Patrik Nordwall 
napisał:
>
> Ah, now I see what you mean. You are right. Thanks for reporting. Please 
> create 
> an issue . If you are interested 
> in contributing a pull request would be great.
>
> /Patrik
>
> On Fri, Nov 20, 2015 at 2:25 PM, Andrzej Dębski  > wrote:
>
>>  I think it just postpones the problem - you could have the same sequence 
>> of events: Subscribe to mediator in the same time as scheduler puts Prune 
>> message in topic actor mailbox, mediator doesn't see topic actor as dead so 
>> Subscribe message is forwarded, Prune is processed and context.stop(self) 
>> is invoked in topic actor stopping it immidietly, also in docs of 
>> context.stop(self) it says that "If this method is applied to the `self` 
>> reference from inside an Actor then that Actor is guaranteed to not process 
>> any further messages after this call;"
>>
>> W dniu piątek, 20 listopada 2015 13:48:37 UTC+1 użytkownik Patrik 
>> Nordwall napisał:
>>>
>>> The topic actors don’t stop themselves immediately. For efficiency 
>>> reasons they stay around for a while even when there is no subscribers. See 
>>> pruneDeadline in the code.
>>>
>>> Doesn’t that take care of this also?
>>>
>>> Cheers,
>>> Patrik
>>> ​
>>>
>>> On Fri, Nov 20, 2015 at 12:36 PM, Andrzej Dębski  
>>> wrote:
>>>
  I was looking through DistributedPubSub code and I was wondering about 
 possible problem due to how topic/children actors are managed.

 When new subscriber arrive for given topic the child/topic actor is 
 looked up using  context.child(encTopic), if actor exists message is 
 forwarded, if not the child actor is created, registered in "registry" and 
 then message is forwarded. Topic actors stop themselves when all their 
 subscribers are either terminated (death watch by topic actor on 
 subscriber) or they just unsubscribe themselves from the topic.

 Could there be a situation where

1. Subscribe message is sent to mediator from SubscriberA
2. At the same time SubscriberB is terminated and this message 
arrives to topic actor.
3. Topic actor stops itself using context.stop(self)
4. At the same time parent/mediator actor performs lookup of child 
and decided that topic actor is alive and forwards him Subscribe message
5. Subscribe message goes to dead letters because topic actor is 
already dead, mediator does not inform subscriber that something went 
 wrong 
and/or do not try to retry sending Subscribe message to new version of 
topic actor.

 According to 
 http://stackoverflow.com/questions/23629159/get-or-create-child-akka-actor-and-ensure-liveness
  
 the solution would be to send kind-of Terminated message to mediator so 
 Subscribe and Terminated messages are processed serially, also watching 
 out 
 for the fact that stopping actor is async operation so even if we invoke 
 context.stop(child) from context of parent there is no guarantee that path 
 is immediately. Maybe the semantics of 
 termination/death-watch/child-lookup 
 are a bit different from general ones due to the fact that here we are 
 discussing about actors that reside in the same JVM but I would like to 
 know if this is genuine problem or I just missed something when reading 
 the 
 code. 

 -- 
 >> Read the docs: http://akka.io/docs/
 >> Check the FAQ: 
 http://doc.akka.io/docs/akka/current/additional/faq.html
 >> Search the archives: 
 https://groups.google.com/group/akka-user
 --- 
 You received this message because you are subscribed to the Google 
 Groups "Akka User List" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to akka-user+...@googlegroups.com.
 To post to this group, send email to akka...@googlegroups.com.
 Visit this group at http://groups.google.com/group/akka-user.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> -- 
>>>
>>> Patrik Nordwall
>>> Typesafe  -  Reactive apps on the JVM
>>> Twitter: @patriknw
>>>
>>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google

[akka-user] DistributedPubSub - topic actor liveness race condition

2015-11-20 Thread Andrzej Dębski 
 I was looking through DistributedPubSub code and I was wondering about 
possible problem due to how topic/children actors are managed.

When new subscriber arrive for given topic the child/topic actor is looked 
up using  context.child(encTopic), if actor exists message is forwarded, if 
not the child actor is created, registered in "registry" and then message 
is forwarded. Topic actors stop themselves when all their subscribers are 
either terminated (death watch by topic actor on subscriber) or they just 
unsubscribe themselves from the topic.

Could there be a situation where

   1. Subscribe message is sent to mediator from SubscriberA
   2. At the same time SubscriberB is terminated and this message arrives 
   to topic actor.
   3. Topic actor stops itself using context.stop(self)
   4. At the same time parent/mediator actor performs lookup of child and 
   decided that topic actor is alive and forwards him Subscribe message
   5. Subscribe message goes to dead letters because topic actor is already 
   dead, mediator does not inform subscriber that something went wrong and/or 
   do not try to retry sending Subscribe message to new version of topic actor.

According to 
http://stackoverflow.com/questions/23629159/get-or-create-child-akka-actor-and-ensure-liveness
 
the solution would be to send kind-of Terminated message to mediator so 
Subscribe and Terminated messages are processed serially, also watching out 
for the fact that stopping actor is async operation so even if we invoke 
context.stop(child) from context of parent there is no guarantee that path 
is immediately. Maybe the semantics of termination/death-watch/child-lookup 
are a bit different from general ones due to the fact that here we are 
discussing about actors that reside in the same JVM but I would like to 
know if this is genuine problem or I just missed something when reading the 
code. 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] DistributedPubSub - topic actor liveness race condition

2015-11-20 Thread Andrzej Dębski 
 I think it just postpones the problem - you could have the same sequence 
of events: Subscribe to mediator in the same time as scheduler puts Prune 
message in topic actor mailbox, mediator doesn't see topic actor as dead so 
Subscribe message is forwarded, Prune is processed and context.stop(self) 
is invoked in topic actor stopping it immidietly, also in docs of 
context.stop(self) it says that "If this method is applied to the `self` 
reference from inside an Actor then that Actor is guaranteed to not process 
any further messages after this call;"

W dniu piątek, 20 listopada 2015 13:48:37 UTC+1 użytkownik Patrik Nordwall 
napisał:
>
> The topic actors don’t stop themselves immediately. For efficiency reasons 
> they stay around for a while even when there is no subscribers. See 
> pruneDeadline in the code.
>
> Doesn’t that take care of this also?
>
> Cheers,
> Patrik
> ​
>
> On Fri, Nov 20, 2015 at 12:36 PM, Andrzej Dębski  > wrote:
>
>>  I was looking through DistributedPubSub code and I was wondering about 
>> possible problem due to how topic/children actors are managed.
>>
>> When new subscriber arrive for given topic the child/topic actor is 
>> looked up using  context.child(encTopic), if actor exists message is 
>> forwarded, if not the child actor is created, registered in "registry" and 
>> then message is forwarded. Topic actors stop themselves when all their 
>> subscribers are either terminated (death watch by topic actor on 
>> subscriber) or they just unsubscribe themselves from the topic.
>>
>> Could there be a situation where
>>
>>1. Subscribe message is sent to mediator from SubscriberA
>>2. At the same time SubscriberB is terminated and this message 
>>arrives to topic actor.
>>3. Topic actor stops itself using context.stop(self)
>>4. At the same time parent/mediator actor performs lookup of child 
>>and decided that topic actor is alive and forwards him Subscribe message
>>5. Subscribe message goes to dead letters because topic actor is 
>>already dead, mediator does not inform subscriber that something went 
>> wrong 
>>and/or do not try to retry sending Subscribe message to new version of 
>>topic actor.
>>
>> According to 
>> http://stackoverflow.com/questions/23629159/get-or-create-child-akka-actor-and-ensure-liveness
>>  
>> the solution would be to send kind-of Terminated message to mediator so 
>> Subscribe and Terminated messages are processed serially, also watching out 
>> for the fact that stopping actor is async operation so even if we invoke 
>> context.stop(child) from context of parent there is no guarantee that path 
>> is immediately. Maybe the semantics of termination/death-watch/child-lookup 
>> are a bit different from general ones due to the fact that here we are 
>> discussing about actors that reside in the same JVM but I would like to 
>> know if this is genuine problem or I just missed something when reading the 
>> code. 
>>
>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
>
> Patrik Nordwall
> Typesafe  -  Reactive apps on the JVM
> Twitter: @patriknw
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] DistributedPubSub - topic actor liveness race condition

2015-11-20 Thread Patrik Nordwall 
The topic actors don’t stop themselves immediately. For efficiency reasons
they stay around for a while even when there is no subscribers. See
pruneDeadline in the code.

Doesn’t that take care of this also?

Cheers,
Patrik
​

On Fri, Nov 20, 2015 at 12:36 PM, Andrzej Dębski 
wrote:

>  I was looking through DistributedPubSub code and I was wondering about
> possible problem due to how topic/children actors are managed.
>
> When new subscriber arrive for given topic the child/topic actor is looked
> up using  context.child(encTopic), if actor exists message is forwarded, if
> not the child actor is created, registered in "registry" and then message
> is forwarded. Topic actors stop themselves when all their subscribers are
> either terminated (death watch by topic actor on subscriber) or they just
> unsubscribe themselves from the topic.
>
> Could there be a situation where
>
>1. Subscribe message is sent to mediator from SubscriberA
>2. At the same time SubscriberB is terminated and this message arrives
>to topic actor.
>3. Topic actor stops itself using context.stop(self)
>4. At the same time parent/mediator actor performs lookup of child and
>decided that topic actor is alive and forwards him Subscribe message
>5. Subscribe message goes to dead letters because topic actor is
>already dead, mediator does not inform subscriber that something went wrong
>and/or do not try to retry sending Subscribe message to new version of
>topic actor.
>
> According to
> http://stackoverflow.com/questions/23629159/get-or-create-child-akka-actor-and-ensure-liveness
> the solution would be to send kind-of Terminated message to mediator so
> Subscribe and Terminated messages are processed serially, also watching out
> for the fact that stopping actor is async operation so even if we invoke
> context.stop(child) from context of parent there is no guarantee that path
> is immediately. Maybe the semantics of termination/death-watch/child-lookup
> are a bit different from general ones due to the fact that here we are
> discussing about actors that reside in the same JVM but I would like to
> know if this is genuine problem or I just missed something when reading the
> code.
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

Patrik Nordwall
Typesafe  -  Reactive apps on the JVM
Twitter: @patriknw

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] DistributedPubSub - topic actor liveness race condition

2015-11-20 Thread Patrik Nordwall 
Ah, now I see what you mean. You are right. Thanks for reporting. Please create
an issue . If you are interested
in contributing a pull request would be great.

/Patrik

On Fri, Nov 20, 2015 at 2:25 PM, Andrzej Dębski 
wrote:

>  I think it just postpones the problem - you could have the same sequence
> of events: Subscribe to mediator in the same time as scheduler puts Prune
> message in topic actor mailbox, mediator doesn't see topic actor as dead so
> Subscribe message is forwarded, Prune is processed and context.stop(self)
> is invoked in topic actor stopping it immidietly, also in docs of
> context.stop(self) it says that "If this method is applied to the `self`
> reference from inside an Actor then that Actor is guaranteed to not process
> any further messages after this call;"
>
> W dniu piątek, 20 listopada 2015 13:48:37 UTC+1 użytkownik Patrik Nordwall
> napisał:
>>
>> The topic actors don’t stop themselves immediately. For efficiency
>> reasons they stay around for a while even when there is no subscribers. See
>> pruneDeadline in the code.
>>
>> Doesn’t that take care of this also?
>>
>> Cheers,
>> Patrik
>> ​
>>
>> On Fri, Nov 20, 2015 at 12:36 PM, Andrzej Dębski 
>> wrote:
>>
>>>  I was looking through DistributedPubSub code and I was wondering about
>>> possible problem due to how topic/children actors are managed.
>>>
>>> When new subscriber arrive for given topic the child/topic actor is
>>> looked up using  context.child(encTopic), if actor exists message is
>>> forwarded, if not the child actor is created, registered in "registry" and
>>> then message is forwarded. Topic actors stop themselves when all their
>>> subscribers are either terminated (death watch by topic actor on
>>> subscriber) or they just unsubscribe themselves from the topic.
>>>
>>> Could there be a situation where
>>>
>>>1. Subscribe message is sent to mediator from SubscriberA
>>>2. At the same time SubscriberB is terminated and this message
>>>arrives to topic actor.
>>>3. Topic actor stops itself using context.stop(self)
>>>4. At the same time parent/mediator actor performs lookup of child
>>>and decided that topic actor is alive and forwards him Subscribe message
>>>5. Subscribe message goes to dead letters because topic actor is
>>>already dead, mediator does not inform subscriber that something went 
>>> wrong
>>>and/or do not try to retry sending Subscribe message to new version of
>>>topic actor.
>>>
>>> According to
>>> http://stackoverflow.com/questions/23629159/get-or-create-child-akka-actor-and-ensure-liveness
>>> the solution would be to send kind-of Terminated message to mediator so
>>> Subscribe and Terminated messages are processed serially, also watching out
>>> for the fact that stopping actor is async operation so even if we invoke
>>> context.stop(child) from context of parent there is no guarantee that path
>>> is immediately. Maybe the semantics of termination/death-watch/child-lookup
>>> are a bit different from general ones due to the fact that here we are
>>> discussing about actors that reside in the same JVM but I would like to
>>> know if this is genuine problem or I just missed something when reading the
>>> code.
>>>
>>> --
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ:
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >> Search the archives:
>>> https://groups.google.com/group/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>>
>> Patrik Nordwall
>> Typesafe  -  Reactive apps on the JVM
>> Twitter: @patriknw
>>
>> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

Patrik Nordwall
Typesafe  -  Reactive apps on the JVM
Twitter: @patriknw

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>>