Re: [akka-user] Akka HTTP Streams vs Pulse Queues

2015-07-06 Thread Viktor Klang
Hi Eric,

I'm not sure I can answer your very broad question, would you mind/be able
to formulate a couple of more specific ones?

On Fri, Jul 3, 2015 at 6:34 PM, Eric Kolotyluk eric.koloty...@gmail.com
wrote:

 I have prototyped an application that basically multiplexes messages,
 assembles them into blocks, and then segments them. The application tries
 to do as much concurrently as possible.

 I have used something I call a 'pulse queue' which is based on a
 non-blocking concurrent queue.


 http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html

 Basically, I create a separate future to handle the processing of each
 message, and the futures complete their work by adding their results to the
 non-blocking queue. The pulse part is that periodically (i.e. each second)
 the queue is drained to create a block or segment. There are other reasons
 to drain a queue, i.e. size, but primarily it is the pulse that keeps data
 moving through the system.

 I have found this to be simple to code, and effective at handling large
 numbers of Futures concurrently.

 As I am trying to understand Akka Streams, and Akka HTTP, I am wondering
 if Streams would be a better or equivalent solution, but I do not really
 understand how streams work under the hood to answer my curiosity. For
 example, in the Streams environment is it possible to create a separate
 future to handle each message, the way the pulse queue does? My sense is
 that each element in the stream is an Actor under the hood, so that the
 messages (while non-blocking) would get serialized.

 Cheers, Eric

 --

  Read the docs: http://akka.io/docs/
  Check the FAQ:
 http://doc.akka.io/docs/akka/current/additional/faq.html
  Search the archives:
 https://groups.google.com/group/akka-user

 --- You received this message because you are subscribed to the
 Google Groups Akka User List group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to akka-user+unsubscr...@googlegroups.com.
 To post to this group, send email to akka-user@googlegroups.com.
 Visit this group at http://groups.google.com/group/akka-user.
 For more options, visit https://groups.google.com/d/optout.




-- 
Cheers,
√

-- 
  Read the docs: http://akka.io/docs/
  Check the FAQ: 
 http://doc.akka.io/docs/akka/current/additional/faq.html
  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups Akka 
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.


[akka-user] Akka HTTP Streams vs Pulse Queues

2015-07-03 Thread Eric Kolotyluk
I have prototyped an application that basically multiplexes messages, 
assembles them into blocks, and then segments them. The application 
tries to do as much concurrently as possible.


I have used something I call a 'pulse queue' which is based on a 
non-blocking concurrent queue.


http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html

Basically, I create a separate future to handle the processing of each 
message, and the futures complete their work by adding their results to 
the non-blocking queue. The pulse part is that periodically (i.e. each 
second) the queue is drained to create a block or segment. There are 
other reasons to drain a queue, i.e. size, but primarily it is the pulse 
that keeps data moving through the system.


I have found this to be simple to code, and effective at handling large 
numbers of Futures concurrently.


As I am trying to understand Akka Streams, and Akka HTTP, I am wondering 
if Streams would be a better or equivalent solution, but I do not really 
understand how streams work under the hood to answer my curiosity. For 
example, in the Streams environment is it possible to create a separate 
future to handle each message, the way the pulse queue does? My sense is 
that each element in the stream is an Actor under the hood, so that the 
messages (while non-blocking) would get serialized.


Cheers, Eric

--

 Read the docs: http://akka.io/docs/
 Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
 Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups Akka User List group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.