Re: [akka-user] Re: Strange behavior with Akka Tcp

2014-04-26 Thread Patrik Nordwall 
Sounds great!

/Patrik

> 25 apr 2014 kl. 18:11 skrev Chris Baxter :
> 
> Hi Patrick.  I will try and boil this down to something much simpler before 
> taking out a ticket.  Thanks for looking into it for me.
> 
> Yes, I'm just starting to familiarize myself with the new Akka reactive 
> streams api.  This is just a prototype to get myself familiar with the inner 
> workings of the core IO stuff.  I'm really waiting for reactive streams to 
> come out and then I plan on building on top of that as it will be much 
> cleaner and simpler.
> 
>> On Friday, April 11, 2014 2:32:37 PM UTC-4, Chris Baxter wrote:
>> I'm using the latest release of Akka (2.3.2) and I've been playing around 
>> with the Tcp Api a bit, trying to get familiar with how it works.  In doing 
>> so, I've been writing a little Memcached binary client.  This usage of Tcp 
>> is a little different then the examples in the docs in that the connection 
>> is kept alive and never closed (unless the peer closes the connection in 
>> which a reconnect will happen).  In my little prototype, I'm using the Ack 
>> based back-pressure solution.  I also recently switched to pullMode for 
>> reads because I found that when I didn't, so many reads were coming when I 
>> was hammering it with load that it took forever to receive the write ack 
>> thus delaying the next write and slowing throughput down.  When I switched 
>> to pullMode, things sped up, but now I'm running into a strange issue where 
>> I eventually do not receive an Ack for one of the writes that I made which 
>> pretty much kills the flow as it's the acks that keep data flowing from my 
>> memcached node actor into the connection actor.  I enabled trace logging and 
>> more often then not, when this happens, I see this is the log as the last 
>> log message from the selection handler:
>> 
>> [DEBUG] [04/11/2014 14:11:02.281] [couch-akka.actor.default-dispatcher-22] 
>> [akka://couch/system/IO-TCP/selectors/$a/0] Wrote [0] bytes to channel
>> 
>> I can post my code if need be, but I just wanted to first see if anyone else 
>> has ever seen this behavior.  
>> 
> 
> -- 
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> >> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Strange behavior with Akka Tcp

2014-04-25 Thread Chris Baxter 
Hi Patrick.  I will try and boil this down to something much simpler before 
taking out a ticket.  Thanks for looking into it for me.

Yes, I'm just starting to familiarize myself with the new Akka reactive 
streams api.  This is just a prototype to get myself familiar with the 
inner workings of the core IO stuff.  I'm really waiting for reactive 
streams to come out and then I plan on building on top of that as it will 
be much cleaner and simpler.

On Friday, April 11, 2014 2:32:37 PM UTC-4, Chris Baxter wrote:
>
> I'm using the latest release of Akka (2.3.2) and I've been playing around 
> with the Tcp Api a bit, trying to get familiar with how it works.  In doing 
> so, I've been writing a little Memcached binary client.  This usage of Tcp 
> is a little different then the examples in the docs in that the connection 
> is kept alive and never closed (unless the peer closes the connection in 
> which a reconnect will happen).  In my little prototype, I'm using the Ack 
> based back-pressure solution.  I also recently switched to pullMode for 
> reads because I found that when I didn't, so many reads were coming when I 
> was hammering it with load that it took forever to receive the write ack 
> thus delaying the next write and slowing throughput down.  When I switched 
> to pullMode, things sped up, but now I'm running into a strange issue where 
> I eventually do not receive an Ack for one of the writes that I made which 
> pretty much kills the flow as it's the acks that keep data flowing from my 
> memcached node actor into the connection actor.  I enabled trace logging 
> and more often then not, when this happens, I see this is the log as the 
> last log message from the selection handler:
>
> [DEBUG] [04/11/2014 14:11:02.281] [couch-akka.actor.default-dispatcher-22] 
> [akka://couch/system/IO-TCP/selectors/$a/0] Wrote [0] bytes to channel
>
> I can post my code if need be, but I just wanted to first see if anyone 
> else has ever seen this behavior.  
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Strange behavior with Akka Tcp

2014-04-25 Thread Patrik Nordwall 
I have looked at your code and I can't see anything wrong. The "Wrote [0]
bytes" is interesting and it is done here:
https://github.com/akka/akka/blob/v2.3.2/akka-actor/src/main/scala/akka/io/TcpConnection.scala#L366

You have to open a ticket  for further
investigation. If you can mimimize it to something without external
products it lowers the bar for us a lot.

By the way, have you looked at the Preview of Akka
Streamswith
stream based api on top of Akka I/O?

Regards,
Patrik


On Thu, Apr 24, 2014 at 5:18 PM, Chris Baxter  wrote:

> Okay, my code is available here
> https://github.com/cmbaxter/slipstream-proto.  The code of interest is in
> TcpMemcachedNode.scala.  One thing to note is that I am "pipelining"
> requests into memcached.  That is, I am sending more than one request per
> Write that I am doing against it.  In this current load test I am sending
> 10 gets in one ByteString.  So you will notice code in the TcpMemcachedNode
> class around building up pipelined requests first (in a ByteStringBuilder)
> before putting them into the queue of messages to be sent.
>
> So when I run the LoadTest.scala App, It stops after a while and the log
> results generally look like this:
>
> [INFO] [04/24/2014 11:18:54.438] [couch-akka.actor.default-dispatcher-25]
> [akka://couch/user/$a/$a] writing 250 bytes, 31091 messages remaining in
> queue
>
> [DEBUG] [04/24/2014 11:18:54.438] [couch-akka.actor.default-dispatcher-19]
> [akka://couch/system/IO-TCP/selectors/$a/0] Wrote [0] bytes to channel
>
> Please let me know if there is something I am doing wrong in my code that
> I can fix and continue on with this prototype.
>
> On Friday, April 11, 2014 2:32:37 PM UTC-4, Chris Baxter wrote:
>>
>> I'm using the latest release of Akka (2.3.2) and I've been playing around
>> with the Tcp Api a bit, trying to get familiar with how it works.  In doing
>> so, I've been writing a little Memcached binary client.  This usage of Tcp
>> is a little different then the examples in the docs in that the connection
>> is kept alive and never closed (unless the peer closes the connection in
>> which a reconnect will happen).  In my little prototype, I'm using the Ack
>> based back-pressure solution.  I also recently switched to pullMode for
>> reads because I found that when I didn't, so many reads were coming when I
>> was hammering it with load that it took forever to receive the write ack
>> thus delaying the next write and slowing throughput down.  When I switched
>> to pullMode, things sped up, but now I'm running into a strange issue where
>> I eventually do not receive an Ack for one of the writes that I made which
>> pretty much kills the flow as it's the acks that keep data flowing from my
>> memcached node actor into the connection actor.  I enabled trace logging
>> and more often then not, when this happens, I see this is the log as the
>> last log message from the selection handler:
>>
>> [DEBUG] [04/11/2014 14:11:02.281] [couch-akka.actor.default-dispatcher-22]
>> [akka://couch/system/IO-TCP/selectors/$a/0] Wrote [0] bytes to channel
>>
>> I can post my code if need be, but I just wanted to first see if anyone
>> else has ever seen this behavior.
>>
>  --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

Patrik Nordwall
Typesafe  -  Reactive apps on the JVM
Twitter: @patriknw
JOIN US. REGISTER TODAY! 
Scala 
Days 
June 16th-18th, 
Berlin 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Strange behavior with Akka Tcp

2014-04-24 Thread Chris Baxter 
Okay, my code is available 
here https://github.com/cmbaxter/slipstream-proto.  The code of interest is 
in TcpMemcachedNode.scala.  One thing to note is that I am "pipelining" 
requests into memcached.  That is, I am sending more than one request per 
Write that I am doing against it.  In this current load test I am sending 
10 gets in one ByteString.  So you will notice code in the TcpMemcachedNode 
class around building up pipelined requests first (in a ByteStringBuilder) 
before putting them into the queue of messages to be sent.  

So when I run the LoadTest.scala App, It stops after a while and the log 
results generally look like this:

[INFO] [04/24/2014 11:18:54.438] [couch-akka.actor.default-dispatcher-25] 
[akka://couch/user/$a/$a] writing 250 bytes, 31091 messages remaining in 
queue

[DEBUG] [04/24/2014 11:18:54.438] [couch-akka.actor.default-dispatcher-19] 
[akka://couch/system/IO-TCP/selectors/$a/0] Wrote [0] bytes to channel

Please let me know if there is something I am doing wrong in my code that I 
can fix and continue on with this prototype.

On Friday, April 11, 2014 2:32:37 PM UTC-4, Chris Baxter wrote:
>
> I'm using the latest release of Akka (2.3.2) and I've been playing around 
> with the Tcp Api a bit, trying to get familiar with how it works.  In doing 
> so, I've been writing a little Memcached binary client.  This usage of Tcp 
> is a little different then the examples in the docs in that the connection 
> is kept alive and never closed (unless the peer closes the connection in 
> which a reconnect will happen).  In my little prototype, I'm using the Ack 
> based back-pressure solution.  I also recently switched to pullMode for 
> reads because I found that when I didn't, so many reads were coming when I 
> was hammering it with load that it took forever to receive the write ack 
> thus delaying the next write and slowing throughput down.  When I switched 
> to pullMode, things sped up, but now I'm running into a strange issue where 
> I eventually do not receive an Ack for one of the writes that I made which 
> pretty much kills the flow as it's the acks that keep data flowing from my 
> memcached node actor into the connection actor.  I enabled trace logging 
> and more often then not, when this happens, I see this is the log as the 
> last log message from the selection handler:
>
> [DEBUG] [04/11/2014 14:11:02.281] [couch-akka.actor.default-dispatcher-22] 
> [akka://couch/system/IO-TCP/selectors/$a/0] Wrote [0] bytes to channel
>
> I can post my code if need be, but I just wanted to first see if anyone 
> else has ever seen this behavior.  
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.