Hi Guido, Thanks for the pointer to the chopping head mailbox. I will take a look, sounds really interesting. I am already using different bounded mailboxes between the Source and the Processing actor depending on the semanitcs of the computation: if the source is shared between multiple processing actors (in the cases where getting the results on-time is more important than processing all the data) I use non-blocking mailbox. If the source is not shared and I care about processing all the input data I use blocking mailbox to slow-down the source.
Regards, luben On Tue, Feb 16, 2016 at 10:13 AM, Guido Medina <oxyg...@gmail.com> wrote: > Hi Luben, > > There are other types of mailboxes good for that (depending on your > requirements) you have bounded non-blocking and I'm not 100% sure but I > think I read somewhere that in Akka streams you have a bounded chopping > head sink, say you don't care about head messages that have taken too long > to process and only the last newest 10k, etc. > > HTH, > > Guido. > > On Monday, February 15, 2016 at 6:47:33 PM UTC, Luben Karavelov wrote: >> >> Hi Michael, >> >> I have taken another approach. I already had a Send/Ack flow control over >> the network. So I extended it to use the 'ask' and wait for the Ack from >> the collecting actor in order to exersice back-pressure to the source. >> Using blocking and bounded queue will also block the source after the queue >> is full. >> >> Thanks for the advice, >> luben >> >> On Fri, Feb 12, 2016 at 6:59 PM, Michael Frank <syntax...@gmail.com> >> wrote: >> >>> i think you will need to implement your own acking protocol. >>> >>> SourceActor sends something like PushMessage to ProcessingActor, which >>> enriches the message with some processing, then sends to the remote >>> CollectingActor. CollectingActor then responds to ProcessingActor with >>> AckMessage. ProcessingActor forwards the AckMessage to SourceActor. Once >>> SourceActor receives AckMessage it sends the next PushMessage. >>> >>> I don't think bounded mailboxes are a good solution, because they have a >>> fixed capacity, and that capacity is not related to your network capacity. >>> an acking protocol will get you better network utilization, and be more >>> tolerant/performant with respect to changing network conditions. >>> >>> -Michael >>> >>> On 02/10/16 17:37, Luben Karavelov wrote: >>> >>> Hi, >>> >>> I have the following push dataflow using remote actors (netty.tcp >>> transport) over the network: >>> >>> SourceActor -> ProcessingActor ~~ network ~~> CollectingActor >>> >>> The problem is that if the network is relatively slow the processing >>> actor successfully sends the messages but they get all buffered inside the >>> same JVM until it blows with OOM. I am looking how to slow down the >>> SourceActor if the upstream network is slower than the production rate. I >>> have put bounded mailbox between the Source and Processing actor but it >>> does not help because the ProcessingActor is not slowed by the network >>> transport. >>> >>> I looked at the code and it looks the problem arises because the >>> EndpointWriter actor (that is kind of a proxy between the local actors and >>> the transport as far as I understand) buffers the data to be sent in itself. >>> >>> So here are my questions: >>> >>> 1. Is there a way to create a back-pressure between the network >>> transport and the sender? >>> >>> 2. If there is no way and I have to create an application level >>> flow-control over the network (between the Processing and Collecting >>> actors) what will be the best way to slow down the SourceActor? Should I >>> extend the flow control down to it? Or is there a way to slow down the >>> reception of messages coming from the SourceActor so that the bounded >>> mailbox will slow it down (I will still need the other messages to be >>> delivered for flow-control signalization)? >>> >>> Thanks in advance for any advises and suggestions, >>> luben >>> >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: <http://akka.io/docs/>http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> <http://doc.akka.io/docs/akka/current/additional/faq.html> >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> <https://groups.google.com/group/akka-user> >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Akka User List" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/akka-user/UYojosSS298/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
