[akka-user] Re: How to synchronize/notify multiple parallel Actors on completion?

[Sean Zhong]

>
> I'd be interested to know if there are other examples which might solve
> this problem without explicitly maintaining state?


@Denis

You can create a only-use-once temporary actor in the router to track all 
acking. After it receive all acking from workers, send a message to router, 
and kill itself.


On Friday, October 16, 2015 at 7:50:58 AM UTC+8, Denis Papathanasiou wrote:
>
> Answering my own question, I found this example[1] which was similar to
> what I needed.
>
> So I wind up maintaining the file contents within the distribution actor,
> and have the workers report back how many lines they've processed
> (analogous to the total number of pints the Bartender tracks in the
> example).
>
> When the distributor gets a total equivalent to the number of lines in
> the first file, it starts processing the contents of the second file.
>
> It's a simple solution which solves my problem, but the idea of keeping
> state in the distribution actor is not as clean/elegant as I would like.
>
> I'd be interested to know if there are other examples which might solve
> this problem without explicitly maintaining state?
>
> [1] 
> http://www.reactive.io/tips/2014/03/28/getting-started-with-actor-based-programming-using-scala-and-akka/
>
> On Wednesday, October 14, 2015 at 8:21:35 PM UTC-4, Denis Papathanasiou 
> wrote:
>>
>> I've written a simple actor system, based on the "calculate Pi"
>> example[1].
>>
>> In my case, I am processing two types of (large) csv files.
>>
>> I load and parse the first one into a list of lines, and then split
>> the lines into equal chunks, for multiple worker actors to process
>> in parallel:
>>
>> lines.grouped(chunks).toList.par.map( lineGroup => router ! Process(
>> lineGroup) )
>>
>> Next, I want to do the exact same thing on all the lines contained in
>> the second file.
>>
>> But, before I issue any new requests to any worker actors, I want to
>> make sure that all the actors who received the initial Process message
>> are finished.
>>
>> I know there is a mechanism to send replies back to the actor which made
>> the request, but I'm not sure how to wait for all the replies to arrive
>> withoout blocking.
>>
>> Are there any good examples of this?
>>
>> [1] 
>> http://doc.akka.io/docs/akka/2.0/intro/getting-started-first-scala.html
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] What is the best practice enforce Type Check for Actor during compilation

[Sean Zhong]

Thanks,

The  akka-typed 
<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fakka%2Fakka%2Fpull%2F16665&sa=D&sntz=1&usg=AFQjCNGA-XEDn6qAZsuXDu7o-8ayLGXjYw>
 PR 
is exactly what I want.

On Friday, January 23, 2015 at 11:44:25 PM UTC+8, Martynas Mickevičius 
wrote:
>
> Hi Sean,
>
> a good practice to define messages that actor handles in actor's companion 
> object. This is of course not enforced by compiler in anyway, but it is 
> good code convention to follow.
>
> If you want more compiler help, you may want to check out Typed Actors 
> <http://doc.akka.io/docs/akka/2.3.9/scala/typed-actors.html> or if you 
> want something fresh and exciting you can take a look at akka-typed 
> <https://github.com/akka/akka/pull/16665> which is still in PR form but 
> is going to address the issue you are facing.
>
>
>
> On Fri, Jan 23, 2015 at 5:21 AM, Sean Zhong  > wrote:
>
>> Suppose we have a servie actor:
>>
>> class Service extends Actor {
>>   def receive: Receive = {
>> case RequestA(args) => doSomething()
>> case RequestB(args) => doAnotherThing()
>>   }
>> }
>>
>> // Some client is using RequestA indirectly, like this:
>>
>>
>> class Source(request: RequestA)
>>
>>
>> A client is trying to sending RequestA to service
>> class Client(source: Source) {
>>   def query: Unit = {
>> service ? source.request
>>   }
>> }
>>
>> Then one day, Service decide to NO longer support RequestA. 
>> The above code will still compile as the client doesn't know the Service 
>> has changed the interface, and will still send the invalid command. 
>>
>> Do you have recommended coding practice to follow, or tools to use, so 
>> that it is easir to identity and track this kind of errors?
>>
>> -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Martynas Mickevičius
> Typesafe <http://typesafe.com/> – Reactive 
> <http://www.reactivemanifesto.org/> Apps on the JVM
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] What is the best practice enforce Type Check for Actor during compilation

[Sean Zhong]

Suppose we have a servie actor:

class Service extends Actor {
  def receive: Receive = {
case RequestA(args) => doSomething()
case RequestB(args) => doAnotherThing()
  }
}

// Some client is using RequestA indirectly, like this:


class Source(request: RequestA)


A client is trying to sending RequestA to service
class Client(source: Source) {
  def query: Unit = {
service ? source.request
  }
}

Then one day, Service decide to NO longer support RequestA. 
The above code will still compile as the client doesn't know the Service 
has changed the interface, and will still send the invalid command. 

Do you have recommended coding practice to follow, or tools to use, so that 
it is easir to identity and track this kind of errors?

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] how to recursive restart remote actors

[Sean Zhong]

The hierachy may looks like this:

grandpa -> parent -> child

When I restart grandpa(grandpa throws a exception, and its supervisor 
handle with Restart), I want the whole tree is restarted.  

parent actor and child actor are remote actors, and their life cycle is not 
binded to their parents. 


-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

In case, someone is also looking for the workaround, here is the hacked 
implementation 

https://github.com/clockfly/akka_ShortActorRefProvider


On Saturday, August 9, 2014 5:03:19 PM UTC+8, Sean Zhong wrote:
>
> Thanks, I hacked the RemoteActorRefProvider, and now be able to reduce to 
> per message overhead to 34 byes(26 bytes are fixed overhead of wire format, 
> not sure what it is).
>
>
>
> On Saturday, August 9, 2014 5:26:16 AM UTC+8, Kam Kasravi wrote:
>
> What was Endre's idea - is it related to URL shortening as discussed here? 
> http://stackoverflow.com/questions/742013/how-to-code-a-url-shortener - 
> ie using a bijective function?
>
> On Friday, August 8, 2014 6:18:24 AM UTC-7, Konrad Malawski wrote:
>
> Hi Sean,
> to summarise our thoughts about the on-the-wire overhead:
>
> * Viktor hinted at this technique: 
> http://en.wikipedia.org/wiki/TCP_acceleration which requires special 
> hardware AFAIK, but is able to transparently compress data sent over TCP 
> connections. It's probably costly but can be applied without app 
> modifications.
> * We are aware of the large overhead and want to fix it soon – this will 
> require changes in the remoting protocol, so it won't happen sooner than 
> 2.4.x. The biggest part of the overhead is the ActorPath and Endre has 
> suggested a neat "actor path shortening" trick which will save a lot of 
> space on the wire for popular senders / receivers.
> * You could implement your own Transport (like our NettyTransport) and 
> hook additional Gzip stages to the netty pipeline in there... This is quite 
> some work but depending on your use-case perhaps it's worth it? In order to 
> gzip to give anything here it would have to span multiple messages.
>
> Hope this helps!
> If your case is not strong enough for implementing this yourself you'll 
> have to wait for our remoting updates which will address this problem (we 
> totally agree the overhead is too big and will bring it down).
>
>
> On Fri, Aug 8, 2014 at 1:29 AM, Sean Zhong  wrote:
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme which 
> allows the sender to abbreviate the most common paths used to a single 
> number, but this needs a new protocol.
>
>
> Hi, Endre and Viktor, 
>
> suppose I want to hack the wire format transport, which source file I 
> should change? 
>
> On Thursday, August 7, 2014 5:11:17 PM UTC+8, √ wrote:
>
> That would be completely outside of Akka.
>
>
> On Thu, Aug 7, 2014 at 11:01 AM, Sean Zhong  wrote:
>
> compressed link/interface
>
>
> Is this configuration inside Akka conf? I cannot find the document, do you 
> have pointer to this?
>  
>
> On Thursday, August 7, 2014 4:58:05 PM UTC+8, √ wrote:
>
> Hi Sean,
>
>
> On Thu, Aug 7, 2014 at 10:49 AM, Sean Zhong  wrote:
>
> Hi Viktor,
>
> About wire-compression, do you mean this?
>
> akka {
>  remote {
>  compression-scheme = "zlib" # Options: "zlib" (lzf to come), leave out 
> for no compression
>  zlib-compression-level = 6 # Options: 0-9 (1 being fastest and 9 being 
> the most compressed), default is 6
>  }
> }
>
>
>
> No, that's from a legacy version of Akka. I mean using a compressed 
> link/interface.
>  
>
> Will it do compression at message level? or at a batch level(share same 
> source machine and target machine)? 
>
>  
>
> Hi Endre,
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>
>
> Which part of source code I can look at to write a new transport?  
>
>
>
>
>
>
> On Thursday, August 7, 2014 4:22:16 PM UTC+8, √ wrote:
>
> You can do wire-level compression.
>
>
> On Thu, Aug 7, 2014 at 10:09 AM, Endre Varga  
> wrote:
>
>
>
>
> On Thu, Aug 7, 2014 at 10:05 AM, √iktor Ҡlang  wrote:
>
> Or add compression.
>
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>  
> -Endre
>  
>
> On Aug 7, 2014 9:52 AM, "Endre Varga"  wrote:
>
>  Hi Sean,
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme whic

Re: [akka-user] Re: dispatcher performance

[Sean Zhong]

I think there is a difference.

The benchmark you showed have no flow control. the receiver actor mailbox 
will always have message.  So, Mailbox.run() will only resubmit itself when 
throughput is met(1024 in the test case).

So, take the throughput number from the benchmark "8,189,290 ops/s", there 
will be only around 8000 executorService.execute(mbox)

For a system with flow control, Mailbox.run() is more likely to finish 
before throughput setting is met. And there will be far more 
executorService.execute(mbox) be called. 


On Monday, August 18, 2014 10:51:38 PM UTC+8, √ wrote:
>
> registerForExecution has to be called for every message.
>
> See the link I sent for both benchmark numbers and configuration for 
> throughput.
>
>
> On Mon, Aug 18, 2014 at 4:25 PM, Sean Zhong  > wrote:
>
>> Revise the typo.
>>
>>
>> When I send 300, 000 message from one actor to another (all local 
>> actors), more than 90% time of the source actor is spent in 
>> Dispatcher.registerForExecution(), 
>> inside it, there is call "executorService.execute(mbox)"
>>
>> The Mailbox.run() logic looks like this: It will first try to process all 
>> pending messages in mailbox, when there is no pending messages, then 
>> current runnable will exit. So for the worst case, one call to 
>> Mailbox.run() can only process 1 message, and then exit. 
>>
>> So 300, 000 messages dispatch will have to call 
>> Dispatcher.registerForExecution() 
>> --> executorService.execute(mbox) for 300, 000 times, which will have 
>> performance problems.
>>
>> What we can do for this to improve performance? 300, 000 message per pair 
>> of actors is not good enough.
>>
>>
>>  -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Cheers,
> √
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: dispatcher performance

[Sean Zhong]

Revise the typo.


When I send 300, 000 message from one actor to another (all local actors), 
more than 90% time of the source actor is spent in 
Dispatcher.registerForExecution(), inside it, there is call 
"executorService.execute(mbox)"

The Mailbox.run() logic looks like this: It will first try to process all 
pending messages in mailbox, when there is no pending messages, then 
current runnable will exit. So for the worst case, one call to 
Mailbox.run() can only process 1 message, and then exit. 

So 300, 000 messages dispatch will have to call 
Dispatcher.registerForExecution() --> executorService.execute(mbox) for 
300, 000 times, which will have performance problems.

What we can do for this to improve performance? 300, 000 message per pair 
of actors is not good enough.


-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] dispatcher performance

[Sean Zhong]

When I send 300, 000 message from one actor to another (all local), more 
than 90% time of first the source actor is spent in 
Dispatcher.registerForExecution(), inside it, there is
executorService.execute(mbox)

The Mailbox.run will first try to process pending messages, when there is 
no pending messages, when there is no messages, then current runnable will 
exit. So for the worst case, 300, 000 messages dispatch to mailbox, which 
also have 300, 000 call for executorService.execute(mbox), which will cause 
performance issue.

What we can do for this? 300, 000 message per pair of actors is not good 
enough.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] how to tune akka remoting performance

[Sean Zhong]

I have not tried to investigate why akka remote is slow, however I wrote a 
out of band netty transport(not in framework of akka remote) as simple as 
possible which 1) do batching send and receive 2) remove sender and 
receiver address overhead 3) simplified wire format. 4) try to avoid to 
dispatch to single intermediate layer queue(mailbox). Now we are able to 
reduce the mean latency from 300ms to 30ms, the throughput about is 3x 
speedup. 

However, it cannot be implemented with akka remote transport interface, as 
it has many assumptions(like only support one way message sending, and the 
sender and recevier address is not in format of ActorRef). so it is purely 
application level optimization.

On Wednesday, August 13, 2014 4:14:27 PM UTC+8, Patrik Nordwall wrote:
>
>
>
>
> On Wed, Aug 13, 2014 at 3:18 AM, Sean Zhong  > wrote:
>
>> Thanks
>>
>> with extrem high latency 727ms,
>>>> If I remember the test correctly, that is not the latency of one 
>>>> message, but the total latency of all 1 messages in one burst.
>>>
>>>
>>
>> If little's law applies here, the estimated latency is about 0.3s. 
>> (Assume pattern: burst -> wait for ack back -> burst -> wait for ack 
>> back)
>> The latency is consistent with my test.
>>
>> It was tested on GCE, with two machines in same zone.
>>
>> My test shows there is no difference between the two cases, two machine, 
>> or two jvm on same machine. It should not be a network bandwidth problem.
>>
>> After some performance profile, the bottleneck seems to be caused by 
>> netty transport, and it is not caused by the backoff logic. So, it will be 
>> interesting why netty performance here is so bad.
>>
>
> Thank you for investigating. Keep us updated.
> /Patrik
>  
>
>>
>>
>> On Tuesday, August 12, 2014 11:36:03 PM UTC+8, Patrik Nordwall wrote:
>>
>>>
>>>
>>>
>>> On Tue, Aug 12, 2014 at 3:36 PM, Sean Zhong  wrote:
>>>
>>>> Hi Patrik,
>>>>
>>>> In your report, looks like only be able to send 3.8MB/s data
>>>>
>>>
>>> Note that payload size (100 bytes) is not equal to message size. 
>>> Additional overhead of the sender and receiver actor refs are also part of 
>>> the full message size, and that is probably larger than the payload size in 
>>> this case. The GCE console reported 23 MBytes/s as max during the test.
>>>  
>>>
>>>> with extrem high latency 727ms,
>>>>
>>>
>>> If I remember the test correctly, that is not the latency of one 
>>> message, but the total latency of all 1 messages in one burst.
>>>  
>>>
>>>>  did you test this with sender and receiver on same machine or on 
>>>> different machine?
>>>>
>>>
>>> It was tested on GCE, with two machines in same zone.
>>> Ping rtt min/avg/max/mdev = 0.516/0.683/0.895/0.102 ms
>>>  
>>>
>>>> The performance is terrible. 
>>>>
>>>
>>> The test was focused on testing something else than looking for maximum 
>>> throughput, so I have no comment on that.
>>>  
>>>
>>>>
>>>> (payload size) (burst size) (new: msg/s) (old: msg/s) (new latency ms) 
>>>> (old latency ms)
>>>> 100 1 42668 38058 727 744
>>>>
>>>>
>>>>
>>>>
>>>> On Monday, August 11, 2014 9:41:17 PM UTC+8, Patrik Nordwall wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Unfortunately, the only test I'm aware of is: 
>>>>> https://github.com/akka/akka/tree/v2.3.4/akka-samples/akka-s
>>>>> ample-remote-scala/src/main/scala/sample/remote/benchmark
>>>>>
>>>>> We have only used it for add-hoc testing. You can find some results of 
>>>>> running it here: https://groups.google.com/d/msg/akka-dev/mFvz_d737t4/
>>>>> pZSmbFRLAV8J
>>>>>
>>>>> Regards,
>>>>> Patrik 
>>>>>
>>>>>
>>>>> On Sat, Aug 9, 2014 at 4:08 PM, Sean Zhong  wrote:
>>>>>
>>>>>> For Akka 2.3.4, 2 machines, GbE network, one machine contains a 
>>>>>> sender actor, the other contains a reciver actor. Sender actor will 
>>>>>> continuously send message (100 bytes each, pre-created in memory) to 
>>>>>> receiver actor with simple flow control based on sliding window and 
>>>>

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Hi Michael,

 does 140MB/s include TCP, IP, ethernet header data?

Yes, it includes. when I use wireshark to capture the traffic, I found 
netty will assemble multiple message to use single tcp packet. So, the 
amortized tcp/ip header cost is not that big.

are you communicating across local network or across the internet?

I use lan. there are several machines in a private cluster. 140MB/s 
represents real network traffic, which means local loopback traffic is not 
counted. 

the greater the distance your packets have to travel (specifically the 
> number of hops), the higher chance that they will get dropped and 
> retransmitted

The test is in LAN, it is not a problem. The travel latency from machine to 
machine should be less than 1 ms.

On Thursday, August 7, 2014 1:34:42 AM UTC+8, Michael Frank wrote:
>
>  sean- how are you measuring your network thoroughput?  does 140MB/s 
> include TCP, IP, ethernet header data?  are you communicating across local 
> network or across the internet?  the greater the distance your packets have 
> to travel (specifically the number of hops), the higher chance that they 
> will get dropped and retransmitted, or fragmented.  a tool like Wireshark, 
> tcpdump, or ScaPy could help you differentiate utilization at different 
> protocol layers and also identify network instability.
>
> -Michael
>
> On 08/06/14 08:46, Sean Zhong wrote:
>  
> Konrad, thanks. 
>
>  After enabling the debug flag, 
>
>  I saw the system message like Terminate are using javaSerialiaer, is 
> this expected?
>
>  [DEBUG] [08/06/2014 22:19:11.836] 
>> [0e6fb647-7893-4328-a335-5e26e2ab080c-akka.actor.default-dispatcher-4] 
>> [akka.serialization.Serialization(akka://0e6fb647-7893-4328-a335-5e26e2ab080c)]
>>  
>> Using serializer[*akka.serialization.JavaSerializer*] for message 
>> [akka.dispatch.sysmsg.Terminate]
>>
>
>  Besides, as my message is String, I cannot find related serialization 
> log for type java.lang.String. How to know for sure protobuf is used for 
> String?
>
>  Are you sure you’re not CPU or something else -bound? And you should be 
>> able to stuff the network?
>>
>
>  What I means is that 140MB/s network are occupied, but source message 
> throughput is only 60MB/s, so there are 80MB/s bandwidth I cannot explain.
>
>  
> On Wednesday, August 6, 2014 11:30:49 PM UTC+8, Konrad Malawski wrote: 
>>
>>   Hi Sean,
>>
>> On the wire:
>> You can look into 
>> https://github.com/akka/akka/tree/master/akka-remote/src/main/protobuf 
>> for what exactly we pass around on the wire.
>>
>> Which serializer is used:
>> enable debug logging, we log this info in Serialization.scala 
>> log.debug("Using 
>> serializer[{}] for message [{}]", ser.getClass.getName, clazz.getName)
>>
>> Other hints:
>> Are you sure you’re not CPU or something else -bound? And you should be 
>> able to stuff the network?
>> Have you played around with the number of threads etc?
>> What hardware are you benchmarking on?
>> ​
>>  
>>
>> On Wed, Aug 6, 2014 at 5:19 PM, Sean Zhong  wrote:
>>
>>>  Thanks, Konrad,
>>>
>>>
>>> Are there other akka data/attributes attached to a remote sending 
>>> message?  Or just?
>>> serialized(msg) + actorRef
>>>
>>>  Which part of akka source code I can check for answer?
>>>
>>>  Besides, is there LOG flags I can enable so that I can check which 
>>> serilization framework is in effect?
>>>
>>>  In my experiment, I cannot explain half network bandwitdh usage with 
>>> akka remote messaging. My message throughput is 60MB/s, but the network 
>>> bandwidth is 140MB/s. How can I trouble shooting this.
>>>
>>>  
>>> On Wednesday, August 6, 2014 7:53:39 PM UTC+8, Konrad Malawski wrote:
>>>
>>>>  Hello Sean, 
>>>> actual overhead in terms of how many bytes – depends on your serialiser.
>>>>
>>>>  1) Yes, a message includes the sender. Not much optimisations in 
>>>> there currently, although we have nice ideas on what we can do within a 
>>>> cluster (short "handles" instead of full addresses),
>>>> 2) Refer to: 
>>>> http://doc.akka.io/docs/akka/snapshot/scala/serialization.html Strings 
>>>> are a byte array, the envelope is protobuf (you can find it in 
>>>> `akka-remote`), other kinds of payload is whatever you configured – 
>>>> defaults to java serialisation.
>>>> 3) Of the entire message? It depends on your serialiser.
>>>>  
>>>&g

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Hi Prakhyat,

I created an test benchmark which have two layers, source actors shuffle 
data to target actors. I add metrics in the source actors to determine how 
many messages are passed to downstream target actors. That is for message 
throughput. I then place the source actors and target actors at my will to 
different machines, that I know whehther the link from one selected source 
actor to a selected target actor requires network transfer traffic cross 
machine, I then use this to compute how many network bandwidth is required. 
I use ganglia to gather the overall network traffic bandwidth.

Thanks

Sean



On Sunday, August 10, 2014 8:58:58 PM UTC+8, Prakhyat Mallikarjun wrote:
>
> Hi Sean,
>
> I was just reading through your discussions. I was quiet curious on how 
> did you measure message throughput in your app?
>
> -prakhyat m m
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] how to tune akka remoting performance

[Sean Zhong]

Thanks

with extrem high latency 727ms,
>> If I remember the test correctly, that is not the latency of one message, 
>> but the total latency of all 1 messages in one burst.
>
>

If little's law applies here, the estimated latency is about 0.3s. (Assume 
pattern: burst -> wait for ack back -> burst -> wait for ack back)
The latency is consistent with my test.

It was tested on GCE, with two machines in same zone.

My test shows there is no difference between the two cases, two machine, or 
two jvm on same machine. It should not be a network bandwidth problem.

After some performance profile, the bottleneck seems to be caused by netty 
transport, and it is not caused by the backoff logic. So, it will be 
interesting why netty performance here is so bad.


On Tuesday, August 12, 2014 11:36:03 PM UTC+8, Patrik Nordwall wrote:
>
>
>
>
> On Tue, Aug 12, 2014 at 3:36 PM, Sean Zhong  > wrote:
>
>> Hi Patrik,
>>
>> In your report, looks like only be able to send 3.8MB/s data
>>
>
> Note that payload size (100 bytes) is not equal to message size. 
> Additional overhead of the sender and receiver actor refs are also part of 
> the full message size, and that is probably larger than the payload size in 
> this case. The GCE console reported 23 MBytes/s as max during the test.
>  
>
>> with extrem high latency 727ms,
>>
>
> If I remember the test correctly, that is not the latency of one message, 
> but the total latency of all 1 messages in one burst.
>  
>
>>  did you test this with sender and receiver on same machine or on 
>> different machine?
>>
>
> It was tested on GCE, with two machines in same zone.
> Ping rtt min/avg/max/mdev = 0.516/0.683/0.895/0.102 ms
>  
>
>> The performance is terrible. 
>>
>
> The test was focused on testing something else than looking for maximum 
> throughput, so I have no comment on that.
>  
>
>>
>> (payload size) (burst size) (new: msg/s) (old: msg/s) (new latency ms) 
>> (old latency ms)
>> 100 1 42668 38058 727 744
>>
>>
>>
>>
>> On Monday, August 11, 2014 9:41:17 PM UTC+8, Patrik Nordwall wrote:
>>
>>> Hi,
>>>
>>> Unfortunately, the only test I'm aware of is: 
>>> https://github.com/akka/akka/tree/v2.3.4/akka-samples/akka-
>>> sample-remote-scala/src/main/scala/sample/remote/benchmark
>>>
>>> We have only used it for add-hoc testing. You can find some results of 
>>> running it here: https://groups.google.com/d/msg/akka-dev/mFvz_
>>> d737t4/pZSmbFRLAV8J
>>>
>>> Regards,
>>> Patrik 
>>>
>>>
>>> On Sat, Aug 9, 2014 at 4:08 PM, Sean Zhong  wrote:
>>>
>>>> For Akka 2.3.4, 2 machines, GbE network, one machine contains a sender 
>>>> actor, the other contains a reciver actor. Sender actor will continuously 
>>>> send message (100 bytes each, pre-created in memory) to receiver actor 
>>>> with 
>>>> simple flow control based on sliding window and acking(one ack per 100 
>>>> messages).
>>>>
>>>> I can only get throughput about 60K messages/second , which is much 
>>>> slower than my expectation. The CPU usage is almost idle(3%). Network 
>>>> usage 
>>>> is about 20% of the bandwidth.  I use kryo serialization, and 
>>>> singleConsumerUnboundedMailBox, 4 local dispatcher, 4 remote 
>>>> dispatcher, default setting for netty. 
>>>>
>>>> Is there performance baseline  number for akka remoting in your smoke 
>>>> test? 
>>>>
>>>>
>>>>
>>>>
>>>>  -- 
>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/
>>>> current/additional/faq.html
>>>> >>>>>>>>>> Search the archives: https://groups.google.com/
>>>> group/akka-user
>>>> --- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to akka-user+...@googlegroups.com.
>>>> To post to this group, send email to akka...@googlegroups.com.
>>>>
>>>> Visit this group at http://groups.google.com/group/akka-user.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>&

Re: [akka-user] how to tune akka remoting performance

[Sean Zhong]

Hi Patrik,

In your report, looks like only be able to send 3.8MB/s data with extrem 
high latency 727ms, did you test this with sender and receiver on same 
machine or on different machine?
The performance is terrible. 

(payload size) (burst size) (new: msg/s) (old: msg/s) (new latency ms) (old 
latency ms)
100 1 42668 38058 727 744




On Monday, August 11, 2014 9:41:17 PM UTC+8, Patrik Nordwall wrote:
>
> Hi,
>
> Unfortunately, the only test I'm aware of is: 
> https://github.com/akka/akka/tree/v2.3.4/akka-samples/akka-sample-remote-scala/src/main/scala/sample/remote/benchmark
>
> We have only used it for add-hoc testing. You can find some results of 
> running it here: 
> https://groups.google.com/d/msg/akka-dev/mFvz_d737t4/pZSmbFRLAV8J
>
> Regards,
> Patrik 
>
>
> On Sat, Aug 9, 2014 at 4:08 PM, Sean Zhong  > wrote:
>
>> For Akka 2.3.4, 2 machines, GbE network, one machine contains a sender 
>> actor, the other contains a reciver actor. Sender actor will continuously 
>> send message (100 bytes each, pre-created in memory) to receiver actor with 
>> simple flow control based on sliding window and acking(one ack per 100 
>> messages).
>>
>> I can only get throughput about 60K messages/second , which is much 
>> slower than my expectation. The CPU usage is almost idle(3%). Network usage 
>> is about 20% of the bandwidth.  I use kryo serialization, and 
>> singleConsumerUnboundedMailBox, 4 local dispatcher, 4 remote dispatcher, 
>> default setting for netty. 
>>
>> Is there performance baseline  number for akka remoting in your smoke 
>> test? 
>>
>>
>>
>>
>>  -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
>
> Patrik Nordwall
> Typesafe <http://typesafe.com/> -  Reactive apps on the JVM
> Twitter: @patriknw
>
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] how to tune akka remoting performance

[Sean Zhong]

Thanks, Patrik,

That may points to the right direction.

After adding some metrics, the latency of akka remoting is extremely high, 
on average, it tooks *0.3s* to travel from one JVM to another JVM!!!


message receive latency (ms):  
 count = 2743521
   min = 237
   max = 1247
  mean = 291.87
stddev = 56.80
median = 287.00
  75% <= 303.00
  95% <= 327.55
  98% <= 338.42
  99% <= 355.26
99.9% <= 1240.91


On Monday, August 11, 2014 9:41:17 PM UTC+8, Patrik Nordwall wrote:
>
> Hi,
>
> Unfortunately, the only test I'm aware of is: 
> https://github.com/akka/akka/tree/v2.3.4/akka-samples/akka-sample-remote-scala/src/main/scala/sample/remote/benchmark
>
> We have only used it for add-hoc testing. You can find some results of 
> running it here: 
> https://groups.google.com/d/msg/akka-dev/mFvz_d737t4/pZSmbFRLAV8J
>
> Regards,
> Patrik 
>
>
> On Sat, Aug 9, 2014 at 4:08 PM, Sean Zhong  > wrote:
>
>> For Akka 2.3.4, 2 machines, GbE network, one machine contains a sender 
>> actor, the other contains a reciver actor. Sender actor will continuously 
>> send message (100 bytes each, pre-created in memory) to receiver actor with 
>> simple flow control based on sliding window and acking(one ack per 100 
>> messages).
>>
>> I can only get throughput about 60K messages/second , which is much 
>> slower than my expectation. The CPU usage is almost idle(3%). Network usage 
>> is about 20% of the bandwidth.  I use kryo serialization, and 
>> singleConsumerUnboundedMailBox, 4 local dispatcher, 4 remote dispatcher, 
>> default setting for netty. 
>>
>> Is there performance baseline  number for akka remoting in your smoke 
>> test? 
>>
>>
>>
>>
>>  -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
>
> Patrik Nordwall
> Typesafe <http://typesafe.com/> -  Reactive apps on the JVM
> Twitter: @patriknw
>
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] how to tune akka remoting performance

[Sean Zhong]

For Akka 2.3.4, 2 machines, GbE network, one machine contains a sender 
actor, the other contains a reciver actor. Sender actor will continuously 
send message (100 bytes each, pre-created in memory) to receiver actor with 
simple flow control based on sliding window and acking(one ack per 100 
messages).

I can only get throughput about 60K messages/second , which is much slower 
than my expectation. The CPU usage is almost idle(3%). Network usage is 
about 20% of the bandwidth.  I use kryo serialization, and 
singleConsumerUnboundedMailBox, 4 local dispatcher, 4 remote dispatcher, 
default setting for netty. 

Is there performance baseline  number for akka remoting in your smoke test? 




-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Besides the actorPath, mesage payload, there are also 21 bytes overhead in 
the wire format, which is also not trivival. You may want to consider this 
when changing the wire format in future.


On Saturday, August 9, 2014 5:03:19 PM UTC+8, Sean Zhong wrote:
>
> Thanks, I hacked the RemoteActorRefProvider, and now be able to reduce to 
> per message overhead to 34 byes(26 bytes are fixed overhead of wire format, 
> not sure what it is).
>
>
>
> On Saturday, August 9, 2014 5:26:16 AM UTC+8, Kam Kasravi wrote:
>
> What was Endre's idea - is it related to URL shortening as discussed here? 
> http://stackoverflow.com/questions/742013/how-to-code-a-url-shortener - 
> ie using a bijective function?
>
> On Friday, August 8, 2014 6:18:24 AM UTC-7, Konrad Malawski wrote:
>
> Hi Sean,
> to summarise our thoughts about the on-the-wire overhead:
>
> * Viktor hinted at this technique: 
> http://en.wikipedia.org/wiki/TCP_acceleration which requires special 
> hardware AFAIK, but is able to transparently compress data sent over TCP 
> connections. It's probably costly but can be applied without app 
> modifications.
> * We are aware of the large overhead and want to fix it soon – this will 
> require changes in the remoting protocol, so it won't happen sooner than 
> 2.4.x. The biggest part of the overhead is the ActorPath and Endre has 
> suggested a neat "actor path shortening" trick which will save a lot of 
> space on the wire for popular senders / receivers.
> * You could implement your own Transport (like our NettyTransport) and 
> hook additional Gzip stages to the netty pipeline in there... This is quite 
> some work but depending on your use-case perhaps it's worth it? In order to 
> gzip to give anything here it would have to span multiple messages.
>
> Hope this helps!
> If your case is not strong enough for implementing this yourself you'll 
> have to wait for our remoting updates which will address this problem (we 
> totally agree the overhead is too big and will bring it down).
>
>
> On Fri, Aug 8, 2014 at 1:29 AM, Sean Zhong  wrote:
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme which 
> allows the sender to abbreviate the most common paths used to a single 
> number, but this needs a new protocol.
>
>
> Hi, Endre and Viktor, 
>
> suppose I want to hack the wire format transport, which source file I 
> should change? 
>
> On Thursday, August 7, 2014 5:11:17 PM UTC+8, √ wrote:
>
> That would be completely outside of Akka.
>
>
> On Thu, Aug 7, 2014 at 11:01 AM, Sean Zhong  wrote:
>
> compressed link/interface
>
>
> Is this configuration inside Akka conf? I cannot find the document, do you 
> have pointer to this?
>  
>
> On Thursday, August 7, 2014 4:58:05 PM UTC+8, √ wrote:
>
> Hi Sean,
>
>
> On Thu, Aug 7, 2014 at 10:49 AM, Sean Zhong  wrote:
>
> Hi Viktor,
>
> About wire-compression, do you mean this?
>
> akka {
>  remote {
>  compression-scheme = "zlib" # Options: "zlib" (lzf to come), leave out 
> for no compression
>  zlib-compression-level = 6 # Options: 0-9 (1 being fastest and 9 being 
> the most compressed), default is 6
>  }
> }
>
>
>
> No, that's from a legacy version of Akka. I mean using a compressed 
> link/interface.
>  
>
> Will it do compression at message level? or at a batch level(share same 
> source machine and target machine)? 
>
>  
>
> Hi Endre,
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>
>
> Which part of source code I can look at to write a new transport?  
>
>
>
>
>
>
> On Thursday, August 7, 2014 4:22:16 PM UTC+8, √ wrote:
>
> You can do wire-level compression.
>
>
> On Thu, Aug 7, 2014 at 10:09 AM, Endre Varga  
> wrote:
>
>
>
>
> On Thu, Aug 7, 2014 at 10:05 AM, √iktor Ҡlang  wrote:
>
> Or add compression.
>
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>  
> -Endre
>  
>
> On Aug 7, 2014 9:52 AM, "Endre Varga"  wrote:
>
>  Hi Sean,
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Thanks, I hacked the RemoteActorRefProvider, and now be able to reduce to 
per message overhead to 34 byes(26 bytes are fixed overhead of wire format, 
not sure what it is).



On Saturday, August 9, 2014 5:26:16 AM UTC+8, Kam Kasravi wrote:
>
> What was Endre's idea - is it related to URL shortening as discussed here? 
> http://stackoverflow.com/questions/742013/how-to-code-a-url-shortener - 
> ie using a bijective function?
>
> On Friday, August 8, 2014 6:18:24 AM UTC-7, Konrad Malawski wrote:
>
> Hi Sean,
> to summarise our thoughts about the on-the-wire overhead:
>
> * Viktor hinted at this technique: 
> http://en.wikipedia.org/wiki/TCP_acceleration which requires special 
> hardware AFAIK, but is able to transparently compress data sent over TCP 
> connections. It's probably costly but can be applied without app 
> modifications.
> * We are aware of the large overhead and want to fix it soon – this will 
> require changes in the remoting protocol, so it won't happen sooner than 
> 2.4.x. The biggest part of the overhead is the ActorPath and Endre has 
> suggested a neat "actor path shortening" trick which will save a lot of 
> space on the wire for popular senders / receivers.
> * You could implement your own Transport (like our NettyTransport) and 
> hook additional Gzip stages to the netty pipeline in there... This is quite 
> some work but depending on your use-case perhaps it's worth it? In order to 
> gzip to give anything here it would have to span multiple messages.
>
> Hope this helps!
> If your case is not strong enough for implementing this yourself you'll 
> have to wait for our remoting updates which will address this problem (we 
> totally agree the overhead is too big and will bring it down).
>
>
> On Fri, Aug 8, 2014 at 1:29 AM, Sean Zhong  wrote:
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme which 
> allows the sender to abbreviate the most common paths used to a single 
> number, but this needs a new protocol.
>
>
> Hi, Endre and Viktor, 
>
> suppose I want to hack the wire format transport, which source file I 
> should change? 
>
> On Thursday, August 7, 2014 5:11:17 PM UTC+8, √ wrote:
>
> That would be completely outside of Akka.
>
>
> On Thu, Aug 7, 2014 at 11:01 AM, Sean Zhong  wrote:
>
> compressed link/interface
>
>
> Is this configuration inside Akka conf? I cannot find the document, do you 
> have pointer to this?
>  
>
> On Thursday, August 7, 2014 4:58:05 PM UTC+8, √ wrote:
>
> Hi Sean,
>
>
> On Thu, Aug 7, 2014 at 10:49 AM, Sean Zhong  wrote:
>
> Hi Viktor,
>
> About wire-compression, do you mean this?
>
> akka {
>  remote {
>  compression-scheme = "zlib" # Options: "zlib" (lzf to come), leave out 
> for no compression
>  zlib-compression-level = 6 # Options: 0-9 (1 being fastest and 9 being 
> the most compressed), default is 6
>  }
> }
>
>
>
> No, that's from a legacy version of Akka. I mean using a compressed 
> link/interface.
>  
>
> Will it do compression at message level? or at a batch level(share same 
> source machine and target machine)? 
>
>  
>
> Hi Endre,
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>
>
> Which part of source code I can look at to write a new transport?  
>
>
>
>
>
>
> On Thursday, August 7, 2014 4:22:16 PM UTC+8, √ wrote:
>
> You can do wire-level compression.
>
>
> On Thu, Aug 7, 2014 at 10:09 AM, Endre Varga  
> wrote:
>
>
>
>
> On Thu, Aug 7, 2014 at 10:05 AM, √iktor Ҡlang  wrote:
>
> Or add compression.
>
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>  
> -Endre
>  
>
> On Aug 7, 2014 9:52 AM, "Endre Varga"  wrote:
>
>  Hi Sean,
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme which 
> allows the sender to abbreviate the most common paths used to a single 
> number, but this needs a new protocol.
>
> So the answer currently is that you cannot reduce this overhead without 
> introducing some batching scheme

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme which 
> allows the sender to abbreviate the most common paths used to a single 
> number, but this needs a new protocol.


Hi, Endre and Viktor, 

suppose I want to hack the wire format transport, which source file I 
should change? 

On Thursday, August 7, 2014 5:11:17 PM UTC+8, √ wrote:
>
> That would be completely outside of Akka.
>
>
> On Thu, Aug 7, 2014 at 11:01 AM, Sean Zhong  > wrote:
>
> compressed link/interface
>
>
> Is this configuration inside Akka conf? I cannot find the document, do you 
> have pointer to this?
>  
>
> On Thursday, August 7, 2014 4:58:05 PM UTC+8, √ wrote:
>
> Hi Sean,
>
>
> On Thu, Aug 7, 2014 at 10:49 AM, Sean Zhong  wrote:
>
> Hi Viktor,
>
> About wire-compression, do you mean this?
>
> akka {
>  remote {
>  compression-scheme = "zlib" # Options: "zlib" (lzf to come), leave out 
> for no compression
>  zlib-compression-level = 6 # Options: 0-9 (1 being fastest and 9 being 
> the most compressed), default is 6
>  }
> }
>
>
>
> No, that's from a legacy version of Akka. I mean using a compressed 
> link/interface.
>  
>
> Will it do compression at message level? or at a batch level(share same 
> source machine and target machine)? 
>
>  
>
> Hi Endre,
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>
>
> Which part of source code I can look at to write a new transport?  
>
>
>
>
>
>
> On Thursday, August 7, 2014 4:22:16 PM UTC+8, √ wrote:
>
> You can do wire-level compression.
>
>
> On Thu, Aug 7, 2014 at 10:09 AM, Endre Varga  
> wrote:
>
>
>
>
> On Thu, Aug 7, 2014 at 10:05 AM, √iktor Ҡlang  wrote:
>
> Or add compression.
>
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>  
> -Endre
>  
>
> On Aug 7, 2014 9:52 AM, "Endre Varga"  wrote:
>
>  Hi Sean,
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme which 
> allows the sender to abbreviate the most common paths used to a single 
> number, but this needs a new protocol.
>
> So the answer currently is that you cannot reduce this overhead without 
> introducing some batching scheme yourself: instead of sending MyMessage you 
> can send Array[MyMessage], so the cost of the recipient path is only 
> suffered once for the batch, but not for the individual messages -- i.e. 
> you can amortize the overhead.
>
> -Endre
>
>
> On Thu, Aug 7, 2014 at 8:11 AM, Sean Zhong  wrote:
>
> Is it possible to reduce the average message overhead?
>
> 200 bytes extra cost per remote message doesn't looks good...
>
>
> On Thursday, August 7, 2014 1:45:12 PM UTC+8, Sean Zhong wrote:
>
> Hi Michael,
>
> I used wireshark to capture the traffic. I found for each message sent(the 
> message is sent with option noSender), there is an extra cost of ActorPath. 
>
> For example, for the following msg example, message payload length length 
> is 100 bytes(bold), but there is also a target actor path for 221 
> bytes(red), which is much bigger than message itself.
> Can the actorPath overhead cost be reduced?
> . 
>
> akka.tcp://app0executor0@192.168.1.53:51582/remote/akka.tcp/
> 2120193a-e10b-474e-bccb-8ebc4b3a0247@192.168.1.53:48948/remo
> te/akka.tcp/cluster@192.168.1.54:43676/user/master/Worker1/app_0_executor_
> 0/group_1_task_0#-768886794o
> h
> *1880512348131407402383073127833013356174562750285568666448502416582566533241241053122856142164774120*
> :
>
>
>
> On Thursday, August 7, 2014 1:34:42 AM UTC+8, Michael Frank wrote:
>
>  sean- how are you measuring your network thoroughput?  does 140MB/s 
> include TCP, IP, ethernet header data?  are you communicating across local 
> network or across the internet?  the greater the distance your packets have 
> to travel (specifically the number of hops), the higher chance that they 
> will get dropped and retransmitted, or fragmented.  a tool like Wireshark, 
> tcpdump, or ScaPy could help you differentiate utilization at different 

Re: [akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

I finally find out it is all out remote dispatcher parallism setting!  when 
I change "parallelism-max" to 10, the performance is greatly improved. 

  "default-remote-dispatcher": {
"executor": "fork-join-executor",
"fork-join-executor": {
  "parallelism-max": 10, 
  "parallelism-factor" : 2.0
  "parallelism-min": 2
},
   use-dispatcher: "akka.remote.default-remote-dispatcher",
   

Sorry for taking so long to find out the answer, thanks for your help

On Thursday, August 7, 2014 8:47:26 PM UTC+8, Sean Zhong wrote:
>
> The default-remote-dispatcher config is:
> ### Default dispatcher for the remoting subsystem
>
> default-remote-dispatcher {
>   type = Dispatcher
>   executor = "fork-join-executor"
>   fork-join-executor {
> # Min number of threads to cap factor-based parallelism number to
> parallelism-min = 2
> parallelism-max = 2
>   }
> }
>
 

>
> Is it possible that the value setting here are too Conservative which 
> impact performance. I will tune this and see what happens...
>
>
>
> On Thursday, August 7, 2014 8:35:50 PM UTC+8, Sean Zhong wrote:
>>
>> I finally narrowed down the config item: akka.actor.remote.use-dispatcher
>>
>> The default setting for akka 2.3.4 remote is 
>> akka.actor.remote.use-dispatcher = "akka.remote.default-remote-dispatcher", 
>> when I change it to akka.actor.remote.use-dispatcher = "", then the 
>> performance is same or better with akka 2.2.3.
>>
>>
>>
>>
>> On Thursday, August 7, 2014 7:57:28 PM UTC+8, √ wrote:
>>>
>>> I hope you didn't try to run the config verbatim as it works differently 
>>> between the versions, but what happened when you updated to use the 2.2.3 
>>> values for the options that were still there in 2.3.4?
>>>
>>>
>>> On Thu, Aug 7, 2014 at 1:55 PM, Sean Zhong  wrote:
>>>
>>>> I made a diff, and try to use old akka 2.2.3 config when running with 
>>>> akka 2.3.4
>>>>
>>>> Here is the diff:
>>>>
>>>> --- akka.2.3.4.conf.json Thu Aug  7 19:33:37 2014
>>>>> +++ akka2.2.3.conf.json Thu Aug  7 19:32:35 2014
>>>>> @@ -13,13 +13,10 @@
>>>>>},
>>>>>"default-dispatcher": {
>>>>>  "attempt-teamwork": "on",
>>>>> -"default-executor": {
>>>>> -  "fallback": "fork-join-executor"
>>>>> -},
>>>>> -"executor": "default-executor",
>>>>> +"executor": "fork-join-executor",
>>>>>  "fork-join-executor": {
>>>>>"parallelism-factor": 3,
>>>>> -  "parallelism-max": 4,
>>>>> +  "parallelism-max": 64,
>>>>>"parallelism-min": 8
>>>>>  },
>>>>>  "mailbox-capacity": -1,
>>>>> @@ -40,14 +37,14 @@
>>>>>"task-queue-size": -1,
>>>>>"task-queue-type": "linked"
>>>>>  },
>>>>> -"throughput": 1024,
>>>>> +"throughput": 5,
>>>>>  "throughput-deadline-time": "0ms",
>>>>>  "type": "Dispatcher"
>>>>>},
>>>>>"default-mailbox": {
>>>>>  "mailbox-capacity": 1000,
>>>>>  "mailbox-push-timeout-time": "10s",
>>>>> -"mailbox-type": 
>>>>> "akka.dispatch.SingleConsumerOnlyUnboundedMailbox",
>>>>> +"mailbox-type": "akka.dispatch.UnboundedMailbox",
>>>>>  "stash-capacity": -1
>>>>>},
>>>>>"default-stash-dispatcher": {
>>>>> @@ -62,7 +59,6 @@
>>>>>"resizer": {
>>>>>  "backoff-rate": 0.1,
>>>>>  "backoff-threshold": 0.3,
>>>>> -"enabled": "off",
>>>>>  "lower-bound"

Re: [akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

The default-remote-dispatcher config is:
### Default dispatcher for the remoting subsystem

default-remote-dispatcher {
  type = Dispatcher
  executor = "fork-join-executor"
  fork-join-executor {
# Min number of threads to cap factor-based parallelism number to
parallelism-min = 2
parallelism-max = 2
  }
}

Is it possible that the value setting here are too Conservative which 
impact performance. I will tune this and see what happens...



On Thursday, August 7, 2014 8:35:50 PM UTC+8, Sean Zhong wrote:
>
> I finally narrowed down the config item: akka.actor.remote.use-dispatcher
>
> The default setting for akka 2.3.4 remote is 
> akka.actor.remote.use-dispatcher = "akka.remote.default-remote-dispatcher", 
> when I change it to akka.actor.remote.use-dispatcher = "", then the 
> performance is same or better with akka 2.2.3.
>
>
>
>
> On Thursday, August 7, 2014 7:57:28 PM UTC+8, √ wrote:
>>
>> I hope you didn't try to run the config verbatim as it works differently 
>> between the versions, but what happened when you updated to use the 2.2.3 
>> values for the options that were still there in 2.3.4?
>>
>>
>> On Thu, Aug 7, 2014 at 1:55 PM, Sean Zhong  wrote:
>>
>>> I made a diff, and try to use old akka 2.2.3 config when running with 
>>> akka 2.3.4
>>>
>>> Here is the diff:
>>>
>>> --- akka.2.3.4.conf.json Thu Aug  7 19:33:37 2014
>>>> +++ akka2.2.3.conf.json Thu Aug  7 19:32:35 2014
>>>> @@ -13,13 +13,10 @@
>>>>},
>>>>"default-dispatcher": {
>>>>  "attempt-teamwork": "on",
>>>> -"default-executor": {
>>>> -  "fallback": "fork-join-executor"
>>>> -},
>>>> -"executor": "default-executor",
>>>> +"executor": "fork-join-executor",
>>>>  "fork-join-executor": {
>>>>"parallelism-factor": 3,
>>>> -  "parallelism-max": 4,
>>>> +  "parallelism-max": 64,
>>>>"parallelism-min": 8
>>>>  },
>>>>  "mailbox-capacity": -1,
>>>> @@ -40,14 +37,14 @@
>>>>"task-queue-size": -1,
>>>>"task-queue-type": "linked"
>>>>  },
>>>> -"throughput": 1024,
>>>> +"throughput": 5,
>>>>  "throughput-deadline-time": "0ms",
>>>>  "type": "Dispatcher"
>>>>},
>>>>"default-mailbox": {
>>>>  "mailbox-capacity": 1000,
>>>>  "mailbox-push-timeout-time": "10s",
>>>> -"mailbox-type": 
>>>> "akka.dispatch.SingleConsumerOnlyUnboundedMailbox",
>>>> +"mailbox-type": "akka.dispatch.UnboundedMailbox",
>>>>  "stash-capacity": -1
>>>>},
>>>>"default-stash-dispatcher": {
>>>> @@ -62,7 +59,6 @@
>>>>"resizer": {
>>>>  "backoff-rate": 0.1,
>>>>  "backoff-threshold": 0.3,
>>>> -"enabled": "off",
>>>>  "lower-bound": 1,
>>>>  "messages-per-resize": 10,
>>>>  "pressure-threshold": 1,
>>>> @@ -110,29 +106,12 @@
>>>>},
>>>>"provider": "akka.remote.RemoteActorRefProvider",
>>>>"reaper-interval": "5s",
>>>> -  "router": {
>>>> -"type-mapping": {
>>>> -  "balancing-pool": "akka.routing.BalancingPool",
>>>> -  "broadcast-group": "akka.routing.BroadcastGroup",
>>>> -  "broadcast-pool": "akka.routing.BroadcastPool",
>>>> -  "consistent-hashing-group": 
>>>> "akka.routing.ConsistentHashingGroup",
>>>> -  "consistent-hashing-pool": 
>>>> "akka.routing.ConsistentHashingPool",
>>>> -   

Re: [akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

I finally narrowed down the config item: akka.actor.remote.use-dispatcher

The default setting for akka 2.3.4 remote is 
akka.actor.remote.use-dispatcher = "akka.remote.default-remote-dispatcher", 
when I change it to akka.actor.remote.use-dispatcher = "", then the 
performance is same or better with akka 2.2.3.




On Thursday, August 7, 2014 7:57:28 PM UTC+8, √ wrote:
>
> I hope you didn't try to run the config verbatim as it works differently 
> between the versions, but what happened when you updated to use the 2.2.3 
> values for the options that were still there in 2.3.4?
>
>
> On Thu, Aug 7, 2014 at 1:55 PM, Sean Zhong  > wrote:
>
>> I made a diff, and try to use old akka 2.2.3 config when running with 
>> akka 2.3.4
>>
>> Here is the diff:
>>
>> --- akka.2.3.4.conf.json Thu Aug  7 19:33:37 2014
>>> +++ akka2.2.3.conf.json Thu Aug  7 19:32:35 2014
>>> @@ -13,13 +13,10 @@
>>>},
>>>"default-dispatcher": {
>>>  "attempt-teamwork": "on",
>>> -"default-executor": {
>>> -  "fallback": "fork-join-executor"
>>> -},
>>> -"executor": "default-executor",
>>> +"executor": "fork-join-executor",
>>>  "fork-join-executor": {
>>>"parallelism-factor": 3,
>>> -  "parallelism-max": 4,
>>> +  "parallelism-max": 64,
>>>"parallelism-min": 8
>>>  },
>>>  "mailbox-capacity": -1,
>>> @@ -40,14 +37,14 @@
>>>"task-queue-size": -1,
>>>"task-queue-type": "linked"
>>>  },
>>> -"throughput": 1024,
>>> +"throughput": 5,
>>>  "throughput-deadline-time": "0ms",
>>>  "type": "Dispatcher"
>>>},
>>>"default-mailbox": {
>>>  "mailbox-capacity": 1000,
>>>  "mailbox-push-timeout-time": "10s",
>>> -"mailbox-type": 
>>> "akka.dispatch.SingleConsumerOnlyUnboundedMailbox",
>>> +"mailbox-type": "akka.dispatch.UnboundedMailbox",
>>>  "stash-capacity": -1
>>>},
>>>"default-stash-dispatcher": {
>>> @@ -62,7 +59,6 @@
>>>"resizer": {
>>>  "backoff-rate": 0.1,
>>>  "backoff-threshold": 0.3,
>>> -"enabled": "off",
>>>  "lower-bound": 1,
>>>  "messages-per-resize": 10,
>>>  "pressure-threshold": 1,
>>> @@ -110,29 +106,12 @@
>>>},
>>>"provider": "akka.remote.RemoteActorRefProvider",
>>>"reaper-interval": "5s",
>>> -  "router": {
>>> -"type-mapping": {
>>> -  "balancing-pool": "akka.routing.BalancingPool",
>>> -  "broadcast-group": "akka.routing.BroadcastGroup",
>>> -  "broadcast-pool": "akka.routing.BroadcastPool",
>>> -  "consistent-hashing-group": 
>>> "akka.routing.ConsistentHashingGroup",
>>> -  "consistent-hashing-pool": 
>>> "akka.routing.ConsistentHashingPool",
>>> -  "from-code": "akka.routing.NoRouter",
>>> -  "random-group": "akka.routing.RandomGroup",
>>> -  "random-pool": "akka.routing.RandomPool",
>>> -  "round-robin-group": "akka.routing.RoundRobinGroup",
>>> -  "round-robin-pool": "akka.routing.RoundRobinPool",
>>> -  "scatter-gather-group": 
>>> "akka.routing.ScatterGatherFirstCompletedGroup",
>>> -  "scatter-gather-pool": 
>>> "akka.routing.ScatterGatherFirstCompletedPool",
>>> -  "smallest-mailbox-pool": "akka.routing.SmallestMailboxPool"
>>> -}
>>> -  },
>>>"serialization-bindings": {
>>

Re: [akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

],
> +"extensions": [],
>  "home": "",
>  "io": {
>"default-backlog": 1000,
> @@ -228,37 +203,18 @@
>  "gremlin": "akka.remote.transport.FailureInjectorProvider",
>  "trttl": "akka.remote.transport.ThrottlerProvider"
>},
> -  "backoff-interval": "5 ms",
> -  "backoff-remote-dispatcher": {
> -"executor": "fork-join-executor",
> -"fork-join-executor": {
> -  "parallelism-max": 2,
> -  "parallelism-min": 2
> -},
> -"type": "Dispatcher"
> -  },
> +  "backoff-interval": "0.01 s",
>"command-ack-timeout": "30 s",
> -  "default-remote-dispatcher": {
> -"executor": "fork-join-executor",
> -"fork-join-executor": {
> -  "parallelism-max": 2,
> -  "parallelism-min": 2
> -},
> -"type": "Dispatcher"
> -  },
>"enabled-transports": [
>  "akka.remote.netty.tcp"
>],
>"flush-wait-on-shutdown": "2 s",
> -  "gremlin": {
> -"debug": "off"
> -  },
> -  "initial-system-message-delivery-timeout": "3 m",
> -  "log-buffer-size-exceeding": 5,
> +  "gate-invalid-addresses-for": "60 s",
>"log-frame-size-exceeding": "off",
>"log-received-messages": "off",
>"log-remote-lifecycle-events": "on",
>"log-sent-messages": "off",
> +  "maximum-retries-in-window": 3,
>"netty": {
>  "ssl": {
>"applied-adapters": [],
> @@ -360,26 +316,29 @@
>"write-buffer-low-water-mark": "0b"
>  }
>},
> -  "prune-quarantine-marker-after": "5 d",
> +  "quarantine-systems-for": "60s",
>"require-cookie": "off",
> -  "resend-interval": "2 s",
> -  "retry-gate-closed-for": "5 s",
> +  "resend-interval": "1 s",
> +  "retry-gate-closed-for": "0 s",
> +  "retry-window": "60 s",
>"secure-cookie": "",
>"shutdown-timeout": "10 s",
>"startup-timeout": "10 s",
> -  "system-message-ack-piggyback-timeout": "0.3 s",
> +  "system-message-ack-piggyback-timeout": "1 s",
>"system-message-buffer-size": 1000,
>"transport-failure-detector": {
> -"acceptable-heartbeat-pause": "20 s",
> -"heartbeat-interval": "4 s",
> -"implementation-class": "akka.remote.DeadlineFailureDetector"
> +"acceptable-heartbeat-pause": "3 s",
> +"heartbeat-interval": "1 s",
> +"implementation-class": "akka.remote.PhiAccrualFailureDetector",
> +"max-sample-size": 100,
> +"min-std-deviation": "100 ms",
> +"threshold": 7
>},
> -  "trusted-selection-paths": [],
>"untrusted-mode": "off",
> -  "use-dispatcher": "akka.remote.default-remote-dispatcher",
> +  "use-dispatcher": "",
>"use-passive-connections": "on",
>"watch-failure-detector": {
> -"acceptable-heartbeat-pause": "10 s",
> +"acceptable-heartbeat-pause": "4 s",
>  "expected-response-after": "3 s",
>  "heartbeat-interval": "1 s",
>  "implementation-class": "akka.remote.PhiAccrualFailureDetector",
> @@ -396,6 +355,6 @@
>"ticks-per-wheel": 512
>  },
>  "stdout-loglevel": "WARNING",
> -"version": "2.3.4"
> +"version": "2.2.3"
>}
>  }




On Thursday, August 7, 2014 7:36:27 PM UTC+8, Sean Zhong wrote:
>
>
>
>
>
> <https://lh5.googleu

Re: [akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

<https://lh5.googleusercontent.com/-HoimNVHLEVs/U-NbcKNJOYI/D7Q/kESkTGWcAdQ/s1600/001.png>

Cluster: 4 machines, each machine has 1 source actor and 1 target actor.. 
all actor started remotely by a master.
test scenario:   Each source actor will randomly deliver a 100 bytes messge 
at a time to any target actor.

Test result：

akka 2.2.3: 500K 100bytes message /second 
akka 2.3.4: 320K 100bytes message / second

the network bandwidth occupation reflects the message throughput. network 
bandwidth is 3.2x message throughput. (as the actorPath overhead is 221 
bytes, aka 2.2 times the message size in my test)

Network bandwidth usage ratio:  akka2.2.3/akka 2.3.4 ~= 1.5

But the CPU ratio akka2.2.3/akka 2.3.4 = 40/15 = 2.6.  

akka 2.2.3 use much more CPU for same throughput, does this mean akka 2.3.4 
is more efficient? But why the message throughput drops on akka 2.3.4.

I dumped the effective akka conf for 2.2.3 and 2.3.4 in the attachment.


On Thursday, August 7, 2014 6:05:54 PM UTC+8, Sean Zhong wrote:
>
> Can it be the case that you have a lot of system message traffic between 
>> your systems? Do you have lots of remote deployed actors maybe?
>
>
> All actors(4 source, 4 target) are created using remote actors.
>
>
> On Thursday, August 7, 2014 5:45:34 PM UTC+8, drewhk wrote:
>>
>> Hi Sean,
>>
>> This is interesting, we actually measured increase with 2.3.4, in fact 
>> the relevant changes in that version were directly targeted to increasing 
>> performance somewhat. One important difference is that 2.3.4 prioritizes 
>> internal Akka messages over user messages to improve stability. Can it be 
>> the case that you have a lot of system message traffic between your 
>> systems? Do you have lots of remote deployed actors maybe?
>>
>> -Endre
>>
>>
>> On Thu, Aug 7, 2014 at 10:53 AM, Sean Zhong  wrote:
>>
>>> Hi, 
>>>
>>> When I upgrade from akka 2.2.3 to akka 2.3.4, I found the message 
>>> throughput drops about 30%.
>>>
>>> My benchmark looks like this:
>>> 4 machines, each machine has 1 source actor and 1 target actor. Each 
>>> source actor will randomly deliver a 100 bytes messge at a time to any 
>>> target actor.
>>>
>>> I use default configuration. Are there default configuration changes in 
>>> akka reference.conf which result in this behavior?
>>>
>>>  -- 
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>> >>>>>>>>>> Check the FAQ: 
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> Search the archives: 
>>> https://groups.google.com/group/akka-user
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.


akka.2.3.4.conf.json
Description: Binary data


akka2.2.3.conf.json
Description: Binary data

Re: [akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

>
> Can it be the case that you have a lot of system message traffic between 
> your systems? Do you have lots of remote deployed actors maybe?


All actors(4 source, 4 target) are created using remote actors.


On Thursday, August 7, 2014 5:45:34 PM UTC+8, drewhk wrote:
>
> Hi Sean,
>
> This is interesting, we actually measured increase with 2.3.4, in fact the 
> relevant changes in that version were directly targeted to increasing 
> performance somewhat. One important difference is that 2.3.4 prioritizes 
> internal Akka messages over user messages to improve stability. Can it be 
> the case that you have a lot of system message traffic between your 
> systems? Do you have lots of remote deployed actors maybe?
>
> -Endre
>
>
> On Thu, Aug 7, 2014 at 10:53 AM, Sean Zhong  > wrote:
>
>> Hi, 
>>
>> When I upgrade from akka 2.2.3 to akka 2.3.4, I found the message 
>> throughput drops about 30%.
>>
>> My benchmark looks like this:
>> 4 machines, each machine has 1 source actor and 1 target actor. Each 
>> source actor will randomly deliver a 100 bytes messge at a time to any 
>> target actor.
>>
>> I use default configuration. Are there default configuration changes in 
>> akka reference.conf which result in this behavior?
>>
>>  -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

I will try to get a min code set to reproduce this. I will post updates 
here.

On Thursday, August 7, 2014 5:47:51 PM UTC+8, Patrik Nordwall wrote:
>
> Could you please share the benchmark source code?
> /Patrik
>
>
> On Thu, Aug 7, 2014 at 11:45 AM, Endre Varga  > wrote:
>
>> Hi Sean,
>>
>> This is interesting, we actually measured increase with 2.3.4, in fact 
>> the relevant changes in that version were directly targeted to increasing 
>> performance somewhat. One important difference is that 2.3.4 prioritizes 
>> internal Akka messages over user messages to improve stability. Can it be 
>> the case that you have a lot of system message traffic between your 
>> systems? Do you have lots of remote deployed actors maybe?
>>  
>> -Endre
>>
>>
>> On Thu, Aug 7, 2014 at 10:53 AM, Sean Zhong > > wrote:
>>
>>> Hi, 
>>>
>>> When I upgrade from akka 2.2.3 to akka 2.3.4, I found the message 
>>> throughput drops about 30%.
>>>
>>> My benchmark looks like this:
>>> 4 machines, each machine has 1 source actor and 1 target actor. Each 
>>> source actor will randomly deliver a 100 bytes messge at a time to any 
>>> target actor.
>>>
>>> I use default configuration. Are there default configuration changes in 
>>> akka reference.conf which result in this behavior?
>>>
>>>  -- 
>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>> >>>>>>>>>> Check the FAQ: 
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >>>>>>>>>> Search the archives: 
>>> https://groups.google.com/group/akka-user
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to akka-user+...@googlegroups.com .
>>> To post to this group, send email to akka...@googlegroups.com 
>>> .
>>> Visit this group at http://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
>
> Patrik Nordwall
> Typesafe <http://typesafe.com/> -  Reactive apps on the JVM
> Twitter: @patriknw
>
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

>
> compressed link/interface


Is this configuration inside Akka conf? I cannot find the document, do you 
have pointer to this?
 

On Thursday, August 7, 2014 4:58:05 PM UTC+8, √ wrote:
>
> Hi Sean,
>
>
> On Thu, Aug 7, 2014 at 10:49 AM, Sean Zhong  > wrote:
>
> Hi Viktor,
>
> About wire-compression, do you mean this?
>
> akka {
>  remote {
>  compression-scheme = "zlib" # Options: "zlib" (lzf to come), leave out 
> for no compression
>  zlib-compression-level = 6 # Options: 0-9 (1 being fastest and 9 being 
> the most compressed), default is 6
>  }
> }
>
>
>
> No, that's from a legacy version of Akka. I mean using a compressed 
> link/interface.
>  
>
> Will it do compression at message level? or at a batch level(share same 
> source machine and target machine)? 
>
>
>
> Hi Endre,
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>
>
> Which part of source code I can look at to write a new transport?  
>
>
>
>
>
>
> On Thursday, August 7, 2014 4:22:16 PM UTC+8, √ wrote:
>
> You can do wire-level compression.
>
>
> On Thu, Aug 7, 2014 at 10:09 AM, Endre Varga  
> wrote:
>
>
>
>
> On Thu, Aug 7, 2014 at 10:05 AM, √iktor Ҡlang  wrote:
>
> Or add compression.
>
>
> This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).
>  
> -Endre
>  
>
> On Aug 7, 2014 9:52 AM, "Endre Varga"  wrote:
>
>  Hi Sean,
>
> Unfortunately there is no way to reduce this overhead without changing the 
> wire layer format, which we cannot do now. As you correctly see, 
> practically all the overhead comes from the path of the destination and 
> sender actor. In the future we have plans to implement a scheme which 
> allows the sender to abbreviate the most common paths used to a single 
> number, but this needs a new protocol.
>
> So the answer currently is that you cannot reduce this overhead without 
> introducing some batching scheme yourself: instead of sending MyMessage you 
> can send Array[MyMessage], so the cost of the recipient path is only 
> suffered once for the batch, but not for the individual messages -- i.e. 
> you can amortize the overhead.
>
> -Endre
>
>
> On Thu, Aug 7, 2014 at 8:11 AM, Sean Zhong  wrote:
>
> Is it possible to reduce the average message overhead?
>
> 200 bytes extra cost per remote message doesn't looks good...
>
>
> On Thursday, August 7, 2014 1:45:12 PM UTC+8, Sean Zhong wrote:
>
> Hi Michael,
>
> I used wireshark to capture the traffic. I found for each message sent(the 
> message is sent with option noSender), there is an extra cost of ActorPath. 
>
> For example, for the following msg example, message payload length length 
> is 100 bytes(bold), but there is also a target actor path for 221 
> bytes(red), which is much bigger than message itself.
> Can the actorPath overhead cost be reduced?
> . 
>
> akka.tcp://app0executor0@192.168.1.53:51582/remote/akka.tcp/
> 2120193a-e10b-474e-bccb-8ebc4b3a0247@192.168.1.53:48948/
> remote/akka.tcp/cluster@192.168.1.54:43676/user/master/
> Worker1/app_0_executor_0/group_1_task_0#-768886794o
> h
> *1880512348131407402383073127833013356174562750285568666448502416582566533241241053122856142164774120*
> :
>
>
>
> On Thursday, August 7, 2014 1:34:42 AM UTC+8, Michael Frank wrote:
>
>  sean- how are you measuring your network thoroughput?  does 140MB/s 
> include TCP, IP, ethernet header data?  are you communicating across local 
> network or across the internet?  the greater the distance your packets have 
> to travel (specifically the number of hops), the higher chance that they 
> will get dropped and retransmitted, or fragmented.  a tool like Wireshark, 
> tcpdump, or ScaPy could help you differentiate utilization at different 
> protocol layers and also identify network instability.
>
> -Michael
>
> On 08/06/14 08:46, Sean Zhong wrote:
>  
> Konrad, thanks. 
>
>  After enabling the debug flag, 
>
>  I saw the system message like Terminate are using javaSerialiaer, is 
> this expected?
>
>  [DEBUG] [08/06/2014 22:19:11.836] [0e6fb647-7893-4328-a335-5e26e
> 2ab080c-akka.actor.default-dispatcher-4] [akka.serialization.Serializat
> ion(akka://0e6fb647-7893-4328-a335-5e26e2ab080c)] Using serializer[
> *akka.serialization.JavaSerializer*] for message [akka.dispatch.sysmsg.
> Terminate]
>
>
>  Besides, as my message is String, I cannot find related serialization 
> log for type java.lang.String. How to know for sure protob

[akka-user] performance drops when upgrade from akka 2.2.3 to 2.3.4 with default config

[Sean Zhong]

Hi, 

When I upgrade from akka 2.2.3 to akka 2.3.4, I found the message 
throughput drops about 30%.

My benchmark looks like this:
4 machines, each machine has 1 source actor and 1 target actor. Each source 
actor will randomly deliver a 100 bytes messge at a time to any target 
actor.

I use default configuration. Are there default configuration changes in 
akka reference.conf which result in this behavior?

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Hi Viktor,

About wire-compression, do you mean this?

akka {
> remote {
> compression-scheme = "zlib" # Options: "zlib" (lzf to come), leave out 
> for no compression
> zlib-compression-level = 6 # Options: 0-9 (1 being fastest and 9 being 
> the most compressed), default is 6
> }
> }


Will it do compression at message level? or at a batch level(share same 
source machine and target machine)?


Hi Endre,

This is the Akka wire level envelope, cannot be directly controlled by 
> users (unless someone writes a new transport of course).


Which part of source code I can look at to write a new transport?  






On Thursday, August 7, 2014 4:22:16 PM UTC+8, √ wrote:
>
> You can do wire-level compression.
>
>
> On Thu, Aug 7, 2014 at 10:09 AM, Endre Varga  > wrote:
>
>>
>>
>>
>> On Thu, Aug 7, 2014 at 10:05 AM, √iktor Ҡlang > > wrote:
>>
>>> Or add compression.
>>>
>>
>> This is the Akka wire level envelope, cannot be directly controlled by 
>> users (unless someone writes a new transport of course).
>>  
>> -Endre
>>  
>>
>>> On Aug 7, 2014 9:52 AM, "Endre Varga" >> > wrote:
>>>
>>>> Hi Sean,
>>>>
>>>> Unfortunately there is no way to reduce this overhead without changing 
>>>> the wire layer format, which we cannot do now. As you correctly see, 
>>>> practically all the overhead comes from the path of the destination and 
>>>> sender actor. In the future we have plans to implement a scheme which 
>>>> allows the sender to abbreviate the most common paths used to a single 
>>>> number, but this needs a new protocol.
>>>>
>>>> So the answer currently is that you cannot reduce this overhead without 
>>>> introducing some batching scheme yourself: instead of sending MyMessage 
>>>> you 
>>>> can send Array[MyMessage], so the cost of the recipient path is only 
>>>> suffered once for the batch, but not for the individual messages -- i.e. 
>>>> you can amortize the overhead.
>>>>
>>>> -Endre
>>>>
>>>>
>>>> On Thu, Aug 7, 2014 at 8:11 AM, Sean Zhong >>> > wrote:
>>>>
>>>>> Is it possible to reduce the average message overhead?
>>>>>
>>>>> 200 bytes extra cost per remote message doesn't looks good...
>>>>>
>>>>>
>>>>> On Thursday, August 7, 2014 1:45:12 PM UTC+8, Sean Zhong wrote:
>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> I used wireshark to capture the traffic. I found for each message 
>>>>>> sent(the message is sent with option noSender), there is an extra cost 
>>>>>> of 
>>>>>> ActorPath. 
>>>>>>
>>>>>> For example, for the following msg example, message payload length 
>>>>>> length is 100 bytes(bold), but there is also a target actor path for 221 
>>>>>> bytes(red), which is much bigger than message itself.
>>>>>> Can the actorPath overhead cost be reduced?
>>>>>> . 
>>>>>>
>>>>>> akka.tcp://app0executor0@192.168.1.53:51582/remote/akka.
>>>>>>> tcp/2120193a-e10b-474e-bccb-8ebc4b3a0247@192.168.1.53:
>>>>>>> 48948/remote/akka.tcp/cluster@192.168.1.54:43676/user/
>>>>>>> master/Worker1/app_0_executor_0/group_1_task_0#-768886794o
>>>>>>> h
>>>>>>> *1880512348131407402383073127833013356174562750285568666448502416582566533241241053122856142164774120*
>>>>>>> :
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thursday, August 7, 2014 1:34:42 AM UTC+8, Michael Frank wrote:
>>>>>>>
>>>>>>>  sean- how are you measuring your network thoroughput?  does 
>>>>>>> 140MB/s include TCP, IP, ethernet header data?  are you communicating 
>>>>>>> across local network or across the internet?  the greater the distance 
>>>>>>> your 
>>>>>>> packets have to travel (specifically the number of hops), the higher 
>>>>>>> chance 
>>>>>>> that they will get dropped and retransmitted, or fragmented.  a tool 
>>>>>>> like 
>>>>>>> Wireshark, tcpdump, or ScaPy could help you differentiate utilization 
>>>>>>> at 
>>>>

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Is it possible to reduce the average message overhead?

200 bytes extra cost per remote message doesn't looks good...

On Thursday, August 7, 2014 1:45:12 PM UTC+8, Sean Zhong wrote:
>
> Hi Michael,
>
> I used wireshark to capture the traffic. I found for each message sent(the 
> message is sent with option noSender), there is an extra cost of ActorPath. 
>
> For example, for the following msg example, message payload length length 
> is 100 bytes(bold), but there is also a target actor path for 221 
> bytes(red), which is much bigger than message itself.
> Can the actorPath overhead cost be reduced?
> . 
>
> akka.tcp://
>> app0executor0@192.168.1.53:51582/remote/akka.tcp/2120193a-e10b-474e-bccb-8ebc4b3a0247@192.168.1.53:48948/remote/akka.tcp/cluster@192.168.1.54:43676/user/master/Worker1/app_0_executor_0/group_1_task_0#-768886794
>> o
>> h
>> *1880512348131407402383073127833013356174562750285568666448502416582566533241241053122856142164774120*
>> :
>
>
>
> On Thursday, August 7, 2014 1:34:42 AM UTC+8, Michael Frank wrote:
>>
>>  sean- how are you measuring your network thoroughput?  does 140MB/s 
>> include TCP, IP, ethernet header data?  are you communicating across local 
>> network or across the internet?  the greater the distance your packets have 
>> to travel (specifically the number of hops), the higher chance that they 
>> will get dropped and retransmitted, or fragmented.  a tool like Wireshark, 
>> tcpdump, or ScaPy could help you differentiate utilization at different 
>> protocol layers and also identify network instability.
>>
>> -Michael
>>
>> On 08/06/14 08:46, Sean Zhong wrote:
>>  
>> Konrad, thanks. 
>>
>>  After enabling the debug flag, 
>>
>>  I saw the system message like Terminate are using javaSerialiaer, is 
>> this expected?
>>
>>  [DEBUG] [08/06/2014 22:19:11.836] 
>>> [0e6fb647-7893-4328-a335-5e26e2ab080c-akka.actor.default-dispatcher-4] 
>>> [akka.serialization.Serialization(akka://0e6fb647-7893-4328-a335-5e26e2ab080c)]
>>>  
>>> Using serializer[*akka.serialization.JavaSerializer*] for message 
>>> [akka.dispatch.sysmsg.Terminate]
>>>
>>
>>  Besides, as my message is String, I cannot find related serialization 
>> log for type java.lang.String. How to know for sure protobuf is used for 
>> String?
>>
>>  Are you sure you’re not CPU or something else -bound? And you should be 
>>> able to stuff the network?
>>>
>>
>>  What I means is that 140MB/s network are occupied, but source message 
>> throughput is only 60MB/s, so there are 80MB/s bandwidth I cannot explain.
>>
>>  
>> On Wednesday, August 6, 2014 11:30:49 PM UTC+8, Konrad Malawski wrote: 
>>>
>>>   Hi Sean,
>>>
>>> On the wire:
>>> You can look into 
>>> https://github.com/akka/akka/tree/master/akka-remote/src/main/protobuf 
>>> for what exactly we pass around on the wire.
>>>
>>> Which serializer is used:
>>> enable debug logging, we log this info in Serialization.scala 
>>> log.debug("Using 
>>> serializer[{}] for message [{}]", ser.getClass.getName, clazz.getName)
>>>
>>> Other hints:
>>> Are you sure you’re not CPU or something else -bound? And you should be 
>>> able to stuff the network?
>>> Have you played around with the number of threads etc?
>>> What hardware are you benchmarking on?
>>> ​
>>>  
>>>
>>> On Wed, Aug 6, 2014 at 5:19 PM, Sean Zhong  wrote:
>>>
>>>>  Thanks, Konrad,
>>>>
>>>>
>>>> Are there other akka data/attributes attached to a remote sending 
>>>> message?  Or just?
>>>> serialized(msg) + actorRef
>>>>
>>>>  Which part of akka source code I can check for answer?
>>>>
>>>>  Besides, is there LOG flags I can enable so that I can check which 
>>>> serilization framework is in effect?
>>>>
>>>>  In my experiment, I cannot explain half network bandwitdh usage with 
>>>> akka remote messaging. My message throughput is 60MB/s, but the network 
>>>> bandwidth is 140MB/s. How can I trouble shooting this.
>>>>
>>>>  
>>>> On Wednesday, August 6, 2014 7:53:39 PM UTC+8, Konrad Malawski wrote:
>>>>
>>>>>  Hello Sean, 
>>>>> actual overhead in terms of how many bytes – depends on your 
>>>>> serialiser.
>>>>>
>>>>>  1) Yes, a message includes the 

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

My application.conf

akka {
  loglevel = "INFO"

  extensions = 
["com.romix.akka.serialization.kryo.KryoSerializationExtension$"]

  actor {
provider = "akka.remote.RemoteActorRefProvider"

serializers {
  kryo = "com.romix.akka.serialization.kryo.KryoSerializer"
}

serialization-bindings {
   "java.lang.String" = kryo
}

default-dispatcher {
  throughput = 1024
  fork-join-executor {
parallelism-max = 4
  }
}
   default-mailbox {
  mailbox-type = "akka.dispatch.SingleConsumerOnlyUnboundedMailbox"
}
  }
  remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
  port = 0
    }
  }
}



On Thursday, August 7, 2014 1:45:12 PM UTC+8, Sean Zhong wrote:
>
> Hi Michael,
>
> I used wireshark to capture the traffic. I found for each message sent(the 
> message is sent with option noSender), there is an extra cost of ActorPath. 
>
> For example, for the following msg example, message payload length length 
> is 100 bytes(bold), but there is also a target actor path for 221 
> bytes(red), which is much bigger than message itself.
> Can the actorPath overhead cost be reduced?
> . 
>
> akka.tcp://
>> app0executor0@192.168.1.53:51582/remote/akka.tcp/2120193a-e10b-474e-bccb-8ebc4b3a0247@192.168.1.53:48948/remote/akka.tcp/cluster@192.168.1.54:43676/user/master/Worker1/app_0_executor_0/group_1_task_0#-768886794
>> o
>> h
>> *1880512348131407402383073127833013356174562750285568666448502416582566533241241053122856142164774120*
>> :
>
>
>
> On Thursday, August 7, 2014 1:34:42 AM UTC+8, Michael Frank wrote:
>>
>>  sean- how are you measuring your network thoroughput?  does 140MB/s 
>> include TCP, IP, ethernet header data?  are you communicating across local 
>> network or across the internet?  the greater the distance your packets have 
>> to travel (specifically the number of hops), the higher chance that they 
>> will get dropped and retransmitted, or fragmented.  a tool like Wireshark, 
>> tcpdump, or ScaPy could help you differentiate utilization at different 
>> protocol layers and also identify network instability.
>>
>> -Michael
>>
>> On 08/06/14 08:46, Sean Zhong wrote:
>>  
>> Konrad, thanks. 
>>
>>  After enabling the debug flag, 
>>
>>  I saw the system message like Terminate are using javaSerialiaer, is 
>> this expected?
>>
>>  [DEBUG] [08/06/2014 22:19:11.836] 
>>> [0e6fb647-7893-4328-a335-5e26e2ab080c-akka.actor.default-dispatcher-4] 
>>> [akka.serialization.Serialization(akka://0e6fb647-7893-4328-a335-5e26e2ab080c)]
>>>  
>>> Using serializer[*akka.serialization.JavaSerializer*] for message 
>>> [akka.dispatch.sysmsg.Terminate]
>>>
>>
>>  Besides, as my message is String, I cannot find related serialization 
>> log for type java.lang.String. How to know for sure protobuf is used for 
>> String?
>>
>>  Are you sure you’re not CPU or something else -bound? And you should be 
>>> able to stuff the network?
>>>
>>
>>  What I means is that 140MB/s network are occupied, but source message 
>> throughput is only 60MB/s, so there are 80MB/s bandwidth I cannot explain.
>>
>>  
>> On Wednesday, August 6, 2014 11:30:49 PM UTC+8, Konrad Malawski wrote: 
>>>
>>>   Hi Sean,
>>>
>>> On the wire:
>>> You can look into 
>>> https://github.com/akka/akka/tree/master/akka-remote/src/main/protobuf 
>>> for what exactly we pass around on the wire.
>>>
>>> Which serializer is used:
>>> enable debug logging, we log this info in Serialization.scala 
>>> log.debug("Using 
>>> serializer[{}] for message [{}]", ser.getClass.getName, clazz.getName)
>>>
>>> Other hints:
>>> Are you sure you’re not CPU or something else -bound? And you should be 
>>> able to stuff the network?
>>> Have you played around with the number of threads etc?
>>> What hardware are you benchmarking on?
>>> ​
>>>  
>>>
>>> On Wed, Aug 6, 2014 at 5:19 PM, Sean Zhong  wrote:
>>>
>>>>  Thanks, Konrad,
>>>>
>>>>
>>>> Are there other akka data/attributes attached to a remote sending 
>>>> message?  Or just?
>>>> serialized(msg) + actorRef
>>>>
>>>>  Which part of akka source code I can check for answer?
>>>>
>>>>  Besides, is there LOG flags I can enable so that I can check which 
>>>> serilization framework is in effect?
&

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Hi Michael,

I used wireshark to capture the traffic. I found for each message sent(the 
message is sent with option noSender), there is an extra cost of ActorPath. 

For example, for the following msg example, message payload length length 
is 100 bytes(bold), but there is also a target actor path for 221 
bytes(red), which is much bigger than message itself.
Can the actorPath overhead cost be reduced?
. 

akka.tcp://app0executor0@192.168.1.53:51582/remote/akka.tcp/2120193a-e10b-474e-bccb-8ebc4b3a0247@192.168.1.53:48948/remote/akka.tcp/cluster@192.168.1.54:43676/user/master/Worker1/app_0_executor_0/group_1_task_0#-768886794
> o
> h
> *1880512348131407402383073127833013356174562750285568666448502416582566533241241053122856142164774120*
> :



On Thursday, August 7, 2014 1:34:42 AM UTC+8, Michael Frank wrote:
>
>  sean- how are you measuring your network thoroughput?  does 140MB/s 
> include TCP, IP, ethernet header data?  are you communicating across local 
> network or across the internet?  the greater the distance your packets have 
> to travel (specifically the number of hops), the higher chance that they 
> will get dropped and retransmitted, or fragmented.  a tool like Wireshark, 
> tcpdump, or ScaPy could help you differentiate utilization at different 
> protocol layers and also identify network instability.
>
> -Michael
>
> On 08/06/14 08:46, Sean Zhong wrote:
>  
> Konrad, thanks. 
>
>  After enabling the debug flag, 
>
>  I saw the system message like Terminate are using javaSerialiaer, is 
> this expected?
>
>  [DEBUG] [08/06/2014 22:19:11.836] 
>> [0e6fb647-7893-4328-a335-5e26e2ab080c-akka.actor.default-dispatcher-4] 
>> [akka.serialization.Serialization(akka://0e6fb647-7893-4328-a335-5e26e2ab080c)]
>>  
>> Using serializer[*akka.serialization.JavaSerializer*] for message 
>> [akka.dispatch.sysmsg.Terminate]
>>
>
>  Besides, as my message is String, I cannot find related serialization 
> log for type java.lang.String. How to know for sure protobuf is used for 
> String?
>
>  Are you sure you’re not CPU or something else -bound? And you should be 
>> able to stuff the network?
>>
>
>  What I means is that 140MB/s network are occupied, but source message 
> throughput is only 60MB/s, so there are 80MB/s bandwidth I cannot explain.
>
>  
> On Wednesday, August 6, 2014 11:30:49 PM UTC+8, Konrad Malawski wrote: 
>>
>>   Hi Sean,
>>
>> On the wire:
>> You can look into 
>> https://github.com/akka/akka/tree/master/akka-remote/src/main/protobuf 
>> for what exactly we pass around on the wire.
>>
>> Which serializer is used:
>> enable debug logging, we log this info in Serialization.scala 
>> log.debug("Using 
>> serializer[{}] for message [{}]", ser.getClass.getName, clazz.getName)
>>
>> Other hints:
>> Are you sure you’re not CPU or something else -bound? And you should be 
>> able to stuff the network?
>> Have you played around with the number of threads etc?
>> What hardware are you benchmarking on?
>> ​
>>  
>>
>> On Wed, Aug 6, 2014 at 5:19 PM, Sean Zhong  wrote:
>>
>>>  Thanks, Konrad,
>>>
>>>
>>> Are there other akka data/attributes attached to a remote sending 
>>> message?  Or just?
>>> serialized(msg) + actorRef
>>>
>>>  Which part of akka source code I can check for answer?
>>>
>>>  Besides, is there LOG flags I can enable so that I can check which 
>>> serilization framework is in effect?
>>>
>>>  In my experiment, I cannot explain half network bandwitdh usage with 
>>> akka remote messaging. My message throughput is 60MB/s, but the network 
>>> bandwidth is 140MB/s. How can I trouble shooting this.
>>>
>>>  
>>> On Wednesday, August 6, 2014 7:53:39 PM UTC+8, Konrad Malawski wrote:
>>>
>>>>  Hello Sean, 
>>>> actual overhead in terms of how many bytes – depends on your serialiser.
>>>>
>>>>  1) Yes, a message includes the sender. Not much optimisations in 
>>>> there currently, although we have nice ideas on what we can do within a 
>>>> cluster (short "handles" instead of full addresses),
>>>> 2) Refer to: 
>>>> http://doc.akka.io/docs/akka/snapshot/scala/serialization.html Strings 
>>>> are a byte array, the envelope is protobuf (you can find it in 
>>>> `akka-remote`), other kinds of payload is whatever you configured – 
>>>> defaults to java serialisation.
>>>> 3) Of the entire message? It depends on your serialiser.
>>>>  
>>

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Konrad, thanks.

After enabling the debug flag, 

I saw the system message like Terminate are using javaSerialiaer, is this 
expected?

[DEBUG] [08/06/2014 22:19:11.836] 
> [0e6fb647-7893-4328-a335-5e26e2ab080c-akka.actor.default-dispatcher-4] 
> [akka.serialization.Serialization(akka://0e6fb647-7893-4328-a335-5e26e2ab080c)]
>  
> Using serializer[*akka.serialization.JavaSerializer*] for message 
> [akka.dispatch.sysmsg.Terminate]
>

Besides, as my message is String, I cannot find related serialization log 
for type java.lang.String. How to know for sure protobuf is used for String?

Are you sure you’re not CPU or something else -bound? And you should be 
> able to stuff the network?
>

What I means is that 140MB/s network are occupied, but source message 
throughput is only 60MB/s, so there are 80MB/s bandwidth I cannot explain.


On Wednesday, August 6, 2014 11:30:49 PM UTC+8, Konrad Malawski wrote:
>
> Hi Sean,
>
> On the wire:
> You can look into 
> https://github.com/akka/akka/tree/master/akka-remote/src/main/protobuf 
> for what exactly we pass around on the wire.
>
> Which serializer is used:
> enable debug logging, we log this info in Serialization.scala 
> log.debug("Using 
> serializer[{}] for message [{}]", ser.getClass.getName, clazz.getName)
>
> Other hints:
> Are you sure you’re not CPU or something else -bound? And you should be 
> able to stuff the network?
> Have you played around with the number of threads etc?
> What hardware are you benchmarking on?
> ​
>
>
> On Wed, Aug 6, 2014 at 5:19 PM, Sean Zhong  > wrote:
>
>> Thanks, Konrad,
>>
>>
>> Are there other akka data/attributes attached to a remote sending 
>> message?  Or just?
>> serialized(msg) + actorRef
>>
>> Which part of akka source code I can check for answer?
>>
>> Besides, is there LOG flags I can enable so that I can check which 
>> serilization framework is in effect?
>>
>> In my experiment, I cannot explain half network bandwitdh usage with akka 
>> remote messaging. My message throughput is 60MB/s, but the network 
>> bandwidth is 140MB/s. How can I trouble shooting this.
>>
>>
>> On Wednesday, August 6, 2014 7:53:39 PM UTC+8, Konrad Malawski wrote:
>>
>>> Hello Sean,
>>> actual overhead in terms of how many bytes – depends on your serialiser.
>>>
>>> 1) Yes, a message includes the sender. Not much optimisations in there 
>>> currently, although we have nice ideas on what we can do within a cluster 
>>> (short "handles" instead of full addresses),
>>> 2) Refer to: http://doc.akka.io/docs/akka/snapshot/scala/
>>> serialization.html Strings are a byte array, the envelope is protobuf 
>>> (you can find it in `akka-remote`), other kinds of payload is whatever you 
>>> configured – defaults to java serialisation.
>>> 3) Of the entire message? It depends on your serialiser.
>>>
>>> Hope this helps, happy hakking :-)
>>>
>>>
>>> On Wed, Aug 6, 2014 at 1:43 PM, Sean Zhong  wrote:
>>>
>>>> Suppose I want to transfer a msg to another machine 
>>>>
>>>> otherActor ! “hello" 
>>>>
>>>> what is the wire message overhead?
>>>>
>>>> 1. the wire messge need to have information about ActorRef, will the 
>>>> ActorRef be attached for every message, or will it be cached on target 
>>>> machine, so the sender only need to attach an Id?
>>>> 2. How string "hello" is serialized on the wire, with java 
>>>> serialization? or protobuf? Can this be configured?
>>>> 3. What is other overhead for this message?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  -- 
>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/
>>>> current/additional/faq.html
>>>> >>>>>>>>>> Search the archives: https://groups.google.com/
>>>> group/akka-user
>>>> --- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to akka-user+...@googlegroups.com.
>>>> To post to this group, send email to akka...@googlegroups.com.
>>>>
>>>> Visit this group at http://groups.google.com/group/akka-user.
>>>> For more options, visit https:/

Re: [akka-user] What is akka message network overhead?

[Sean Zhong]

Thanks, Konrad,


Are there other akka data/attributes attached to a remote sending message? 
 Or just?
serialized(msg) + actorRef

Which part of akka source code I can check for answer?

Besides, is there LOG flags I can enable so that I can check which 
serilization framework is in effect?

In my experiment, I cannot explain half network bandwitdh usage with akka 
remote messaging. My message throughput is 60MB/s, but the network 
bandwidth is 140MB/s. How can I trouble shooting this.


On Wednesday, August 6, 2014 7:53:39 PM UTC+8, Konrad Malawski wrote:
>
> Hello Sean,
> actual overhead in terms of how many bytes – depends on your serialiser.
>
> 1) Yes, a message includes the sender. Not much optimisations in there 
> currently, although we have nice ideas on what we can do within a cluster 
> (short "handles" instead of full addresses),
> 2) Refer to: 
> http://doc.akka.io/docs/akka/snapshot/scala/serialization.html Strings 
> are a byte array, the envelope is protobuf (you can find it in 
> `akka-remote`), other kinds of payload is whatever you configured – 
> defaults to java serialisation.
> 3) Of the entire message? It depends on your serialiser.
>
> Hope this helps, happy hakking :-)
>
>
> On Wed, Aug 6, 2014 at 1:43 PM, Sean Zhong  > wrote:
>
>> Suppose I want to transfer a msg to another machine 
>>
>> otherActor ! “hello" 
>>
>> what is the wire message overhead?
>>
>> 1. the wire messge need to have information about ActorRef, will the 
>> ActorRef be attached for every message, or will it be cached on target 
>> machine, so the sender only need to attach an Id?
>> 2. How string "hello" is serialized on the wire, with java serialization? 
>> or protobuf? Can this be configured?
>> 3. What is other overhead for this message?
>>
>>
>>
>>
>>
>>  -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Cheers,
> Konrad 'ktoso' Malawski
> hAkker @ Typesafe
>
> <http://typesafe.com>
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] What is akka message network overhead?

[Sean Zhong]

Suppose I want to transfer a msg to another machine 

otherActor ! “hello" 

what is the wire message overhead?

1. the wire messge need to have information about ActorRef, will the 
ActorRef be attached for every message, or will it be cached on target 
machine, so the sender only need to attach an Id?
2. How string "hello" is serialized on the wire, with java serialization? 
or protobuf? Can this be configured?
3. What is other overhead for this message?





-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: How I can watch for actor child change

[Sean Zhong]

Thanks, Martynas.

On Tuesday, July 29, 2014 9:38:48 PM UTC+8, Martynas Mickevičius wrote:
>
> Hi Sean,
>
> I can not think of a clean way of doing that. One idea that popped into my 
> mind was to write a custom Mailbox implementation where it would be 
> possible to track mailbox owners. This is surely and definitely a hack. 
> Maybe it will give you some nicer ideas. :) 
>
> Writing custom ActorRef provider would be another hacky idea, because for 
> that you would need to go to private[akka] code.
>
>
> On Tue, Jul 29, 2014 at 3:51 PM, Sean Zhong  > wrote:
>
>> The child's path is not known. 
>>
>> I want get notified when a new child come up no matter what the path is. 
>>
>>
>> On Monday, July 28, 2014 2:48:28 PM UTC+8, 09goral wrote:
>>>
>>> use context.watch(ActorRef) to register for chaning of actor's life 
>>> cycle.
>>>
>>> W dniu niedziela, 27 lipca 2014 09:23:14 UTC+2 użytkownik Sean Zhong 
>>> napisał:
>>>>
>>>> I cannot change the code of this actor, I want to get notified when 
>>>> there is a child created or stopped, how can this be done?
>>>>
>>>> There was ActorRegistry, but it was removed in akka 2. (
>>>> http://doc.akka.io/docs/akka/1.3.1/java/actor-registry.
>>>> html#ActorRegistry__Finding_Actors)
>>>>
>>>>
>>>>
>>>>  -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Martynas Mickevičius
> Typesafe <http://typesafe.com/> – Reactive 
> <http://www.reactivemanifesto.org/> Apps on the JVM
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] child doesn't get notified or killed when remote parent is dead

[Sean Zhong]

Test case:
parent actor: in a local machine, child actor, in a remote machine. child 
actor is created by parent actor by akka remote deployment.

Behavior observed:
when the parent JVM is killed, the child actor is *NOT* stopped. I have not 
use child context.watch(parent) for this case.  I oringally thought when 
the parent actor *cannot* connected, the child actor should 
*suicide itself.*

Is this expected? I use akka 2.2.3


Sean



-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] how to know supervised actor is stopped normally or abnormally?

[Sean Zhong]

Hi Roland,

Yes, you are right, the task need to send a explicit "TASK_SUCESS_FINISH" 
message to notify the subscribe that task finish normally, all other cases 
is abnormal. The only pain point in this case is that I have to write the 
subscription twice. First the subscriber register it self as a subscriber 
so that it can receive "TASK_SUCESS_FINISH" message, second it also need to 
context.watch to watch for abnormal termination.

Another good use case of  addressTerminated=true is that I can notify the 
system user probably the network is down, and need to be checked.



On Sunday, July 27, 2014 5:17:23 PM UTC+8, rkuhn wrote:
>
> The question then is: which meaning do you attribute to that flag in the 
> context of your application? Because receiving a Terminated with 
> addressTerminated=true does not preclude that the actor meanwhile 
> terminated normally on the partitioned remote node—you simply cannot know.
>
> What I am getting at is that you can choose to optimize those two cases 
> differently, but as soon as your program depends on this boolean for 
> correctness you are in trouble.
>
> Regards,
>
> Roland
>
> 26 jul 2014 kl. 21:21 skrev Sean Zhong >:
>
> Ah, I find out!
>
> addressTerminated seems to server this purpose. 
>
> addressTerminated the Terminated message was derived from that the remote 
> node hosting the watched actor was detected as unreachable
>
> Terminated private[akka] (@BeanProperty actor: ActorRef)(
>   @BeanProperty val existenceConfirmed: Boolean,
>   @BeanProperty val addressTerminated: Boolean)
>
> On Sunday, July 27, 2014 2:55:38 AM UTC+8, Sean Zhong wrote:
>>
>> fix typo
>>
>> A remote supervise B, A watch B 
>>
>> There are two scenarios:
>> 1. normal: inside B, call context.stop(self) when B finish its work
>> 2. abnormal: the network connection between A and B is down
>>
>> A will receive a Terminated message, how can A know the Terminated 
>> message is caused by "1" or "2"?
>>
>
> -- 
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ: 
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to akka-user+...@googlegroups.com .
> To post to this group, send email to akka...@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> *Dr. Roland Kuhn*
> *Akka Tech Lead*
> Typesafe <http://typesafe.com/> – Reactive apps on the JVM.
> twitter: @rolandkuhn
> <http://twitter.com/#!/rolandkuhn>
>  
>

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: How I can watch for actor child change

[Sean Zhong]

The child's path is not known. 

I want get notified when a new child come up no matter what the path is. 

On Monday, July 28, 2014 2:48:28 PM UTC+8, 09goral wrote:
>
> use context.watch(ActorRef) to register for chaning of actor's life cycle.
>
> W dniu niedziela, 27 lipca 2014 09:23:14 UTC+2 użytkownik Sean Zhong 
> napisał:
>>
>> I cannot change the code of this actor, I want to get notified when there 
>> is a child created or stopped, how can this be done?
>>
>> There was ActorRegistry, but it was removed in akka 2. (
>> http://doc.akka.io/docs/akka/1.3.1/java/actor-registry.html#ActorRegistry__Finding_Actors
>> )
>>
>>
>>
>>

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] How I can watch for actor child change

[Sean Zhong]

I cannot change the code of this actor, I want to get notified when there 
is a child created or stopped, how can this be done?

There was ActorRegistry, but it was removed in akka 2. 
(http://doc.akka.io/docs/akka/1.3.1/java/actor-registry.html#ActorRegistry__Finding_Actors)



-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: how to know supervised actor is stopped normally or abnormally?

[Sean Zhong]

Ah, I find out!

addressTerminated seems to server this purpose. 

addressTerminated the Terminated message was derived from that the remote 
node hosting the watched actor was detected as unreachable

Terminated private[akka] (@BeanProperty actor: ActorRef)(
  @BeanProperty val existenceConfirmed: Boolean,
  @BeanProperty val addressTerminated: Boolean)

On Sunday, July 27, 2014 2:55:38 AM UTC+8, Sean Zhong wrote:
>
> fix typo
>
> A remote supervise B, A watch B 
>
> There are two scenarios:
> 1. normal: inside B, call context.stop(self) when B finish its work
> 2. abnormal: the network connection between A and B is down
>
> A will receive a Terminated message, how can A know the Terminated message 
> is caused by "1" or "2"?
>

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: how to know supervised actor is stopped normally or abnormally?

[Sean Zhong]

fix typo

A remote supervise B, A watch B 

There are two scenarios:
1. normal: inside B, call context.stop(self) when B finish its work
2. abnormal: the network connection between A and B is down

A will receive a Terminated message, how can A know the Terminated message 
is caused by "1" or "2"?

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] how to know supervised actor is stopped normally or abnormally?

[Sean Zhong]

A remote supervise B, 

There are two scenarios:
1. normal: inside B, call context.stop(self) when B finish its work
2. abnormal: the network connection between A and B is down.


How can to B is stopped in scenario 1 or scenario 2?


-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] How to run akka stream distributedly in a cluster?

[Sean Zhong]

Thank you both for your reply.

I believe there lacks some combinators, I will comment later after I check 
the github todo list.

On Monday, July 14, 2014 4:55:59 PM UTC+8, Konrad Malawski wrote:
>
> As Endre already highlighted, we have a different focus than the 
> specialised-for-big-data frameworks.
>
> Our focus is to get the back-pressure semantics and common API among 
> library implementors right (the reactive streams API).
> There’s a lot of discussions around these still happening at 
> https://github.com/reactive-streams/reactive-streams if you’re interested.
> The spec is still evolving actively.
>
> APIs will continue to be improved of course and we are well aware of 
> Spark/Storm/Scalding (with the last one I’ve spent a lot of time ;-)).
>
> If you think we’re missing any fundamental combinators please check if 
> they’re already in our todo’s or open a ticket: 
> https://github.com/akka/akka/issues?labels=t%3Astream&page=1&state=open
>
> 
> -- 
> Konrad 'ktoso' Malawski
> hAkker @ typesafe
> http://akka.io
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] How to run akka stream distributedly in a cluster?

[Sean Zhong]

Thanks,

I come from bigdata background.
In my opinion, the Flow DSL is not expressive for many cases, like data 
shuffle. Have you considered using DSL like the one used in cascading, or 
spark, or storm trident? 

How can I help in this process?

Sean

On Monday, July 14, 2014 4:26:58 PM UTC+8, Konrad Malawski wrote:
>
> That’s currently not supported.
> Streams are a very fresh module, and we’re still working to get the in-jvm 
> semantics and APIs *right* before we go distributed.
>
>
> -- 
> Konrad 'ktoso' Malawski
> hAkker @ typesafe
> http://akka.io
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] How to run akka stream distributedly in a cluster?

[Sean Zhong]

How to run akka stream distributedly in a cluster?

Suppose a Flow DSL will be translated to 100 actors. How to configure these 
actor in start in different machine of the cluster?



-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] How to do shuffle in reactive stream

[Sean Zhong]

Hi Martynas,

I am trying to do a data shuffle between two layers.
On the upstream, there are a layer with multiple actors.
On the downstream, there are another layer with multiple actors.

Each upstream actor will distribute the message flowed by it to all 
downstream actors, maybe groupped the message by a key hash function. And 
the whole flow speed is controlled by backpressure.

Is it possible to express this in Flow DSL?


On Friday, July 11, 2014 11:59:15 PM UTC+8, Martynas Mickevičius wrote:
>
> Hi Sean,
>
> could you elaborate a bit mote what are you trying to do?
>
>
> On Thu, Jul 10, 2014 at 4:41 AM, Sean Zhong  > wrote:
>
>>
>>  
>> <https://lh3.googleusercontent.com/-CknzCjr2mNo/U73vEEZlsDI/D3Y/Sd-exFRr3aQ/s1600/akka_shuffle.png>
>>
>>
>> Take wordcount as an example, S is a list of files, 
>>
>> Flow(S).foreach(fileName => 
>>
>>   val file = Source.fromFile(fileName)
>>
>>   Flow(file.getLines())
>>
>> .flatMap(splitWords)
>>
>> .groupBy(_)
>>
>>  ...
>>
>> Ideas?
>>
>>
>> Sean
>>
>> -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Martynas Mickevičius
> Typesafe <http://typesafe.com/> – Reactive 
> <http://www.reactivemanifesto.org/> Apps on the JVM
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] How to do shuffle in reactive stream

[Sean Zhong]

Take wordcount as an example, S is a list of files, 

Flow(S).foreach(fileName => 

  val file = Source.fromFile(fileName)

  Flow(file.getLines())

.flatMap(splitWords)

.groupBy(_)

 ...

Ideas?


Sean

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] What is the difference of Flow.produceTo(mat, consumer) with Flow.toProducer(mat).producerTo(consumer)?

[Sean Zhong]

Thank you!

On Thursday, July 10, 2014 12:43:22 AM UTC+8, √ wrote:
>
> I believe this code answers it :)
>
>
> https://github.com/akka/akka/blob/release-2.3-dev/akka-stream/src/main/scala/akka/stream/impl/FlowImpl.scala#L71
>
>
> On Wed, Jul 9, 2014 at 6:23 PM, Sean Zhong  > wrote:
>
>> When checking examples from https://github.com/adamw/reactmq.git, we saw 
>> two similar usage of Flow:
>>
>> One is:
>>
>>  Flow(conn.inputStream)
>>   .mapConcat(reconcileFrames.apply)
>>   .produceTo(materializer, sendToQueueConsumer)
>>
>>  The other is:
>>
>> Flow(receiveFromQueueProducer)
>>   .map(_.encodeAsString)
>>   .map(createFrame)
>>   .toProducer(materializer)
>>   .produceTo(conn.outputStream)
>>
>>
>> What is the difference of Flow.produceTo(mat, consumer) with 
>> Flow.toProducer(mat).producerTo(consumer)?
>>
>> -- 
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Cheers,
> √
>  

-- 
>>>>>>>>>>  Read the docs: http://akka.io/docs/
>>>>>>>>>>  Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] What is the difference of Flow.produceTo(mat, consumer) with Flow.toProducer(mat).producerTo(consumer)?

[Sean Zhong]

When checking examples from https://github.com/adamw/reactmq.git, we saw 
two similar usage of Flow:

One is:

 Flow(conn.inputStream)
  .mapConcat(reconcileFrames.apply)
  .produceTo(materializer, sendToQueueConsumer)

 The other is:

Flow(receiveFromQueueProducer)
  .map(_.encodeAsString)
  .map(createFrame)
  .toProducer(materializer)
  .produceTo(conn.outputStream)


What is the difference of Flow.produceTo(mat, consumer) with 
Flow.toProducer(mat).producerTo(consumer)?

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.