Re: [akka-user] [Akka stream] Sink/source materialization
On Wednesday, February 8, 2017 at 12:25:25 AM UTC-8, √ wrote: > > You mean like: Source.single(()).flatMapConcat(_ => yourFunction())? > Cool, that works. I also see Source.unfoldResource, which I missed before and seems super useful. I was more struggling with the sink-side: is there a primitive with which I can can create a sink on materialization of the graph? It sounds as if lazyInit would do the job but is seems to be designed especially for the situation where you want to avoid a materialization unless there really is an element in the stream; delaying the materialization may not always be desired. Hbf > -- > Cheers, > √ > > On Feb 8, 2017 03:33, "hbf" > wrote: > >> Heya everybody! >> >> I know that Akka Stream elegantly distinguishes *describing* a graph >> from *materializing* a graph. >> >> When working on custom Akka Stream sink/sources, I often find myself >> looking for a simple way to create a source (or sink) from a function that >> gets called at materialization time. However, I don't see this in the >> Source/Sink companion object. >> >> Examples: >> >>- Create a source that returns the results from a single HTTP call >>when materialized. >>- Create a sink that when materializes opens a connection to Redis >>and writes to it. >> >> In both cases, I'd like the some action (make the HTTP request, open the >> connection) to only take part when we materialize, and each time we >> materialize the flow. >> >> Am I missing something? >> >> Thanks for any feedback! >> Hbf >> >> P.S. I know I can use the GraphStage API to achieve this (and that's what >> I'm doing) but I think it might be nice to have this functionality in a >> more visible API – if what I say makes sense. >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+...@googlegroups.com . >> To post to this group, send email to akka...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka stream] Sink/source materialization
Heya everybody! I know that Akka Stream elegantly distinguishes *describing* a graph from *materializing* a graph. When working on custom Akka Stream sink/sources, I often find myself looking for a simple way to create a source (or sink) from a function that gets called at materialization time. However, I don't see this in the Source/Sink companion object. Examples: - Create a source that returns the results from a single HTTP call when materialized. - Create a sink that when materializes opens a connection to Redis and writes to it. In both cases, I'd like the some action (make the HTTP request, open the connection) to only take part when we materialize, and each time we materialize the flow. Am I missing something? Thanks for any feedback! Hbf P.S. I know I can use the GraphStage API to achieve this (and that's what I'm doing) but I think it might be nice to have this functionality in a more visible API – if what I say makes sense. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Akka Stream] Happens-before relation and runForEach
Thanks, Roland and √. My example was indeed made up (there are better
constructs in place) but thanks for clarifying that it's indeed the JVM
happens-before relations that applies.
– Kaspar
On Wednesday, September 21, 2016 at 12:56:41 AM UTC-7, √ wrote:
>
> Technically I think the solution holds water, there are appropriate
> barriers in place. (multi-materialization aside)
>
> However, as mentioned, fold() would be a much cleaner, safer and
> maintainable solution.
>
> --
> Cheers,
> √
>
> On Sep 21, 2016 07:05, "Roland Kuhn" >
> wrote:
>
>> Technically I'm not sure this is correct: since f is only called from
>> within a single operator (and assuming only a single materialization) the
>> shown code should work just fine. Of course it is necessary to know what
>> you're doing when coding like this, and as Konrad says there are better
>> alternatives available, but guaranteeing ordering for example for db writes
>> would certainly be desirable.
>>
>> Regards, Roland
>>
>> Sent from my iPhone
>>
>> On 21 Sep 2016, at 01:33, Konrad Malawski > > wrote:
>>
>>
>> final Source source = // ...
>> final MutableInt max = new MutableInt(Integer.MIN_VALUE);
>> final Procedure f = i -> {
>> if (i > max.intValue()) {
>> max.setValue(i);
>> }
>> };
>> final CompletionStage result = source
>> .runForeach(f, materializer)
>> .thenApply(__ -> max.intValue());
>>
>> Is the above code correct in the sense that it always commutes the
>> maximum (or MIN_VALUE in case the stream is empty)?
>>
>> According to Akka Stream documentation
>>
>> on
>> Stream Ordering, there is a Java happens-before relation between
>> invocations of f. Do I still need to use AtomicInteger so that the effects
>> are seen by all threads?
>>
>> You're side-effecting outside the stream, so nothing is guaranteed.
>>
>> Yes, it would have to be an atomic integer.
>>
>>
>> Instead consider using a better operator for this kind of thing than
>> foreach - like fold (runFold) which is enough for such operation you're
>> doing here.
>>
>>
>> Happy hakking.
>>
>> -- konrad
>>
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] Akka Stream buffer size monitoring
Hey Akka Stream'ers,
I'm trying to monitor the current size of the buffer of a Akka Stream
buffer() stage to something like Grafana.
Is there a way to hook a call to my metrics framework into the buffer? I
was hoping for a second parameter to buffer, maybe
flow.buffer(1000, size => Metrics.gauge("Buffer size", size))
(where Metrics.gauge() is just an example call to a metrics framework).
Any ideas?
Thanks,
Hbf
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Akka Stream] Happens-before relation and runForEach
Thanks for clarifying, Konrad.
– Hbf
On Tuesday, September 20, 2016 at 4:33:53 PM UTC-7, Konrad Malawski wrote:
>
>
> final Source source = // ...
> final MutableInt max = new MutableInt(Integer.MIN_VALUE);
> final Procedure f = i -> {
> if (i > max.intValue()) {
> max.setValue(i);
> }
> };
> final CompletionStage result = source
> .runForeach(f, materializer)
> .thenApply(__ -> max.intValue());
>
> Is the above code correct in the sense that it always commutes the maximum
> (or MIN_VALUE in case the stream is empty)?
>
> According to Akka Stream documentation
> <http://doc.akka.io/docs/akka/2.4.10/scala/stream/stream-flows-and-basics.html#Stream_ordering>
> on
> Stream Ordering, there is a Java happens-before relation between
> invocations of f. Do I still need to use AtomicInteger so that the effects
> are seen by all threads?
>
> You're side-effecting outside the stream, so nothing is guaranteed.
>
> Yes, it would have to be an atomic integer.
>
>
> Instead consider using a better operator for this kind of thing than
> foreach - like fold (runFold) which is enough for such operation you're
> doing here.
>
>
> Happy hakking.
>
> -- konrad
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka Stream] Happens-before relation and runForEach
Hey everybody,
Akka Stream may execute the function given to Source.runForEach() from
different threads. Therefore, if this function has state, it may need some
synchronization.
For example:
final Source source = // ...
final MutableInt max = new MutableInt(Integer.MIN_VALUE);
final Procedure f = i -> {
if (i > max.intValue()) {
max.setValue(i);
}
};
final CompletionStage result = source
.runForeach(f, materializer)
.thenApply(__ -> max.intValue());
Is the above code correct in the sense that it always commutes the maximum
(or MIN_VALUE in case the stream is empty)?
According to Akka Stream documentation
<http://doc.akka.io/docs/akka/2.4.10/scala/stream/stream-flows-and-basics.html#Stream_ordering>
on
Stream Ordering, there is a Java happens-before relation between
invocations of f. Do I still need to use AtomicInteger so that the effects
are seen by all threads?
Thanks!
Hbf
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Stream] Connecting arbitrary number of graphs with materialized values
Can you do it recursively? Write method that takes two and combines their materialized values. Then recurse to do *n*. Hbf. On Friday, 26 February 2016 05:02:34 UTC-8, Tal Pressman wrote: > > This isn't actually a matter of size - I don't expect that I'll have too > many sinks, I just don't know how many I'll have ahead of time. > > Tal > > On Friday, February 26, 2016 at 2:49:16 PM UTC+2, drewhk wrote: >> >> >> >> On Fri, Feb 26, 2016 at 1:45 PM, Tal Pressman wrote: >> >>> Hi, >>> >>> So in my case all the sinks return a Future (well, CompletionStage) and >>> I want to sequence them. >>> I understand the HLists might be a little too complicated (and I'm not >>> sure if it can be used from Java...) but I still think something that works >>> on a Seq of the least common ancestor could be useful. >>> >> >> I don't necessarily agree. Graphs should not grow too large, that is a >> sign of mismatch of abstractions to me. While in general we don't recommend >> using stream-of-streams this might be one of the cases where it is the best >> option. >> >> -Endre >> >> >>> >>> Tal >>> >>> >>> >>> >>> On Friday, February 26, 2016 at 12:47:33 PM UTC+2, Akka Team wrote: >>>> >>>> Hi Tal, >>>> >>>> The issue why you can't convert in the GraphDSL an arbitrary number of >>>> materialized values is because that would need HLists (lists if >>>> heterogeneous types, a generalization of tuples). What is exactly what you >>>> want to achieve? >>>> >>>> -Endre >>>> >>>> On Thu, Feb 25, 2016 at 3:50 PM, Tal Pressman wrote: >>>> >>>>> Hi, >>>>> >>>>> I find myself needing a way to send messages to several sinks, while >>>>> aggregating their materialized values. Trying to use the GraphDSL, I can >>>>> either: >>>>> >>>>>- Pass a static number of graphs to GraphDSL.create and use their >>>>>materialized values, or >>>>>- Call builder.add for each graph, but lose their materialized >>>>>value, or >>>>> >>>>> I could also implement some custom sink that calls the appropriate >>>>> GraphDSL.create method based on the number of sinks I need connected at >>>>> runtime, but that seems ridiculous... >>>>> Ideally, I would like to have something like: >>>>> def create[S <: Shape, M, Mat](graphs:Seq[Graph[Shape, _ <: M]])(matF: >>>>> Seq[M] => Mat)(buildBlock: Builder[Mat] => Seq[Shape] => S): Graph[S, >>>>> Seq[Mat]] >>>>> >>>>> Is there any way of achieving this? >>>>> >>>>> Thanks, >>>>> Tal >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: >>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>> >>>>>>>>>> Search the archives: >>>>> https://groups.google.com/group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to akka-user+...@googlegroups.com. >>>>> To post to this group, send email to akka...@googlegroups.com. >>>>> Visit this group at https://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> Akka Team >>>> Typesafe - Reactive apps on the JVM >>>> Blog: letitcrash.com >>>> Twitter: @akkateam >>>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Thread interrupt during message send
On Friday, February 19, 2016 at 12:25:36 AM UTC-8, rkuhn wrote: > > Hi Hbf, > > (which is a bit funny because I always read «Hauptbahnhof» :-) ) > I can imagine :) > there are no guarantees, we only promise at-most-once messaging. That > said, local message sends are hard to drop without killing the JVM in the > process. But interrupting threads may result in undefined behavior (in > particular the interrupt and/or the message may get lost, depending on the > mailbox and dispatcher implementations). > Thanks for explaining in detail! > May I ask why you intend to interrupt Actor threads? > I am callling a legacy API which is blocking and can only get stopped via interruption. The exception I posted happened during a unit test. There's hope I can abandon that API soon, so I think it's all fine. Thanks! Hbf > 19 feb 2016 kl. 06:45 skrev hbf >: > > Hi everybody, > > If a thread gets interrupted during a actor.tell(msg, sender), is it > guaranteed that the message still gets sent? > > In my test case, the actor probe confirms that the message is indeed sent > but having read that there are minor differences between the dispatcher > used for tests and the "real" one, I'd like to confirm this. > > Thanks! > Hbf > > [ERROR] [02/18/2016 21:39:03.400] [Thread-3] [ > akka://default/system/testProbe-2] interrupted during message send > java.lang.InterruptedException > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1245) > at java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:442) > at > akka.testkit.CallingThreadDispatcher.runQueue(CallingThreadDispatcher.scala:275) > at > akka.testkit.CallingThreadDispatcher.dispatch(CallingThreadDispatcher.scala:208) > at akka.actor.dungeon.Dispatch$class.sendMessage(Dispatch.scala:132) > at akka.actor.ActorCell.sendMessage(ActorCell.scala:374) > at akka.actor.Cell$class.sendMessage(ActorCell.scala:295) > at akka.actor.ActorCell.sendMessage(ActorCell.scala:374) > at akka.actor.RepointableActorRef.$bang(RepointableActorRef.scala:169) > at akka.actor.ActorRef.tell(ActorRef.scala:128) > at org.foo.lambda$run$3(Bar.java:62) > at java.util.Optional.ifPresent(Optional.java:159) > at org.foo.run(Bar.java:57) > > > > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+...@googlegroups.com . > To post to this group, send email to akka...@googlegroups.com > . > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > > > > *Dr. Roland Kuhn* > *Akka Tech Lead* > Typesafe <http://typesafe.com/> – Reactive apps on the JVM. > twitter: @rolandkuhn > <http://twitter.com/#!/rolandkuhn> > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Akka Stream] Exceptions in AbstractInHandler and AbstractOutHandler
Thanks, √! On Friday, February 12, 2016 at 12:19:23 AM UTC-8, √ wrote: > > They will fail the stage if they throw > > -- > Cheers, > √ > On Feb 12, 2016 8:54 AM, "hbf" > wrote: > >> Hey everybody, >> >> The documentation doesn't say how exceptions in a GraphStage's handlers ( >> AbstractInHandler and AbstractOutHandler) are treated. Do they cause >> undefined behavior or will they implicitly call failStage()? >> >> Thanks! >> Hbf >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+...@googlegroups.com . >> To post to this group, send email to akka...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Thread interrupt during message send
Hi everybody, If a thread gets interrupted during a actor.tell(msg, sender), is it guaranteed that the message still gets sent? In my test case, the actor probe confirms that the message is indeed sent but having read that there are minor differences between the dispatcher used for tests and the "real" one, I'd like to confirm this. Thanks! Hbf [ERROR] [02/18/2016 21:39:03.400] [Thread-3] [akka://default/system/testProbe-2] interrupted during message send java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1245) at java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:442) at akka.testkit.CallingThreadDispatcher.runQueue(CallingThreadDispatcher.scala:275) at akka.testkit.CallingThreadDispatcher.dispatch(CallingThreadDispatcher.scala:208) at akka.actor.dungeon.Dispatch$class.sendMessage(Dispatch.scala:132) at akka.actor.ActorCell.sendMessage(ActorCell.scala:374) at akka.actor.Cell$class.sendMessage(ActorCell.scala:295) at akka.actor.ActorCell.sendMessage(ActorCell.scala:374) at akka.actor.RepointableActorRef.$bang(RepointableActorRef.scala:169) at akka.actor.ActorRef.tell(ActorRef.scala:128) at org.foo.lambda$run$3(Bar.java:62) at java.util.Optional.ifPresent(Optional.java:159) at org.foo.run(Bar.java:57) -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: [ANNOUNCE] Akka 2.4.2 including Streams and HTTP Released!
Thanks a lot! – A "humongous" thanks, I want to say :) One question: will there be a release for Scala 2.10 (i.e., 'com.typesafe.akka:akka-actor_2.10:2.4.2')? – Hbf On Wednesday, February 17, 2016 at 7:42:18 AM UTC-8, Konrad Malawski wrote: > > *Dear hakkers,* > > we—the Akka committers—are proud to announce the FINAL RELEASE of Akka > 2.4.2. The main change in this release is that it includes Streams & HTTP. > Some of these new modules are still marked experimental due to impending > API changes that we could not yet finish, in particular these are akka-http > (the DSLs) and the HTTP marshaller modules for spray-json, jackson, and > scala-xml. In comparison to Streams & HTTP 2.0 the main changes are: > >- significant performance improvement for HTTP handling, now reaching >roughly 75% of Spray’s performance for long-lived HTTP connections—this is >not the end of the performance work, we have only just begun (in > particular >the number of connections per second needs more work) >- replacement of all uses of the Unit type (represented as BoxedUnit >in Java) with the more descriptive typesakka.Done (for signaling >successful completion) and akka.NotUsed (for materialization results >of stages that do not produce a value) >- usage of Java 8 types in the Java DSLs: java.util.Optional instead of > scala.Option andjava.util.concurrent.CompletionStage instead of >scala.concurrent.Future > > Especially the second and third point mean that porting code from Streams > & HTTP 2.0 to Akka 2.4.2 will require some mechanical source code changes, > please refer to the migration guide(for Java > <http://doc.akka.io/docs/akka/2.4.2/java/stream/migration-guide-2.0-2.4-java.html> > and Scala > <http://doc.akka.io/docs/akka/2.4.2/scala/stream/migration-guide-2.0-2.4-scala.html>) > > for the details. > > But also in the rest of Akka much work was done, with highlights being: > >- fixed a possible replay consistency issue ><https://github.com/akka/akka/issues/19694> in Akka Persistence and a too >tight specification <https://github.com/akka/akka/issues/19728> in the >TCK >- silence heartbeat logging <https://github.com/akka/akka/issues/19381> > in ClusterClient and add a notification when it cannot reconnect ><https://github.com/akka/akka/issues/18577> >- provide an API for ClusterSharding state retrieval ><https://github.com/akka/akka/issues/17695> >- improvements to BackoffSupervisor ><https://github.com/akka/akka/issues/19246> >- fix documentation for max-pool-size settings ><https://github.com/akka/akka/issues/19201> (this was a little >unintuitive, makes a good trivia question!) >- add Java API for ByteString builders ><https://github.com/akka/akka/issues/19085> >- fix a race condition that could lead to lost messages in >DistributedPubSub <https://github.com/akka/akka/issues/19017> >- add an ask() variant that can be used without sender(), i.e. by putting >the promise reference into the message ><https://github.com/akka/akka/issues/15819> >- added Java patterns for CompletionStage, see PatternsCS ><http://doc.akka.io/japi/akka/2.4.2/akka/pattern/PatternsCS.html> > > Closed issues since version 2.4.1 can be found here > <https://github.com/akka/akka/issues?q=milestone%3A2.4.2+is%3Aclosed>. > > > *Binary Compatibility* > > Akka 2.4.2 is backwards binary compatible with previous 2.4.x and 2.3.x > versions (exceptions listed below). This means that the new JARs are a > drop-in replacement for the old one (but not the other way around) as long > as your build does not enable the inliner (Scala-only restriction). It > should be noted that Scala 2.11.x is is not binary compatible with Scala > 2.10.x, which means that Akka’s binary compatibility property only holds > between versions that were built for a given Scala > version—akka-actor_2.11-2.4.2-RC1.jar is compatible with > akka-actor_2.11-2.3.14.jar but not with akka-actor_2.10-2.3.14.jar. > > Binary compatibility is *not* maintained for the following: > >- testkits: > - akka-testkit > - akka-multi-node-testkit > - akka-persistence-tck > - akka-stream-testkit > - akka-http-testkit >- experimental modules: > - akka-persistence-query-experimental > - akka-distributed-data-experimental > - akka-typed-experimental > - akka-http-experimental > - akka-http-spray-json-experimental > - akka-http-xml-experimental > - akka-http-jacks
[akka-user] [Akka Stream] Exceptions in AbstractInHandler and AbstractOutHandler
Hey everybody, The documentation doesn't say how exceptions in a GraphStage's handlers ( AbstractInHandler and AbstractOutHandler) are treated. Do they cause undefined behavior or will they implicitly call failStage()? Thanks! Hbf -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Akka Stream] Waiting for stream completion
Thanks, Victor, for asking the right question! Makes sense. – Kaspar On Tuesday, 9 February 2016 00:32:50 UTC-8, √ wrote: > > If the Sink doesn't report when it is done, how would you know it's done? > > On Tue, Feb 9, 2016 at 6:11 AM, hbf > wrote: > >> Hey Akka Stream'ers, >> >> Is there a simple way to await the completion of stream? >> >> If I have a source that is not yet connected, I can pipe it throw a >> Sink.fold() to materialize it to Future. This allows me to wait >> for the stream to complete. But what if I already have a sink and it >> doesn't materialize to a future? >> >> Thanks! >> Kaspar >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+...@googlegroups.com . >> To post to this group, send email to akka...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Cheers, > √ > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka Stream] Waiting for stream completion
Hey Akka Stream'ers, Is there a simple way to await the completion of stream? If I have a source that is not yet connected, I can pipe it throw a Sink.fold() to materialize it to Future. This allows me to wait for the stream to complete. But what if I already have a sink and it doesn't materialize to a future? Thanks! Kaspar -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-stream] Wait until two sinks are completed
Thanks a lot for your reply, Konrad – I have overlooked that overload of
GraphDSL.create() and your suggestion works great!
As a note, my user case is actually about *n *sinks that I need to combine
in this way. I am doing this via recursion right now, which again works
great.
– Kaspar
On Wednesday, December 23, 2015 at 1:25:50 AM UTC-8, Konrad Malawski wrote:
>
> You need the s1 and s2 values to "contribute to the materialized value of
> graph".
> Please read this section of the docs:
> http://doc.akka.io/docs/akka-stream-and-http-experimental/2.0/java/stream-composition.html
> then:
> http://doc.akka.io/docs/akka-stream-and-http-experimental/2.0/java/stream-graphs.html#Constructing_Graphs
>
> You'll want something among the lines of (did not compile, so sorry if
> there's some silly mistake):
>
> final Graph, Future>> graph =
> GraphDSL.create(s1, s2, // these two will be imported, and
> (m1, m2) -> Futures.sequence(Arrays.asList(m1, m2),
> system.dispatcher()), // combine the materialized values, result is Future
> that completes when both complete
> (builder, first, second) -> { // will be automatically imported
> final UniformFanOutShape bcast =
> builder.add(Broadcast.create(Integer.class, 2));
>
> builder.from(bcast).to(first);
> builder.from(bcast).to(second);
> return SinkShape.of(bcast.in());
> });
>
> Hope this helps, happy hakking!
>
> --
> Cheers,
> Konrad 'ktoso’ Malawski
> Akka <http://akka.io> @ Typesafe <http://typesafe.com>
>
> On 23 December 2015 at 05:23:29, hbf (kaspar@gmail.com )
> wrote:
>
> Hey everybody,
>
> I can create a sink that broadcasts incoming messages to a given list of
> sinks. How can I make that sink materialize a future that completes when
> the downstream sinks have completed?
>
> For example,
>
> final Source in = Source.from(Arrays.asList(1,
> 2, 3, 4, 5));
> final Sink> s1 = Sink.foreach(t ->
> log.info("first: {}", t));
> final Sink> s2 = Sink.foreach(t ->
> log.info("second: {}", t));
>
> // Construct a sink that broadcasts to the sinks s1, s2:
> final Graph, *BoxedUnit*> graph =
> GraphDSL.> create(
> builder -> {
> final UniformFanOutShape bcast =
> builder.add(Broadcast.create(2));
> final SinkShape first = builder.add(s1);
> final SinkShape second = builder.add(s2);
>
> builder.from(bcast).to(first);
> builder.from(bcast).to(second);
> return SinkShape.of(bcast.in());
> });
>
> I'd like graph to be of shape SinkShape*>. Is
> that possible?
>
> – Kaspar
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+...@googlegroups.com .
> To post to this group, send email to akka...@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] [akka-stream] Wait until two sinks are completed
Hey everybody,
I can create a sink that broadcasts incoming messages to a given list of
sinks. How can I make that sink materialize a future that completes when
the downstream sinks have completed?
For example,
final Source in = Source.from(Arrays.asList(1,
2, 3, 4, 5));
final Sink> s1 = Sink.foreach(t ->
log.info("first: {}", t));
final Sink> s2 = Sink.foreach(t ->
log.info("second: {}", t));
// Construct a sink that broadcasts to the sinks s1, s2:
final Graph, *BoxedUnit*> graph =
GraphDSL.> create(
builder -> {
final UniformFanOutShape bcast =
builder.add(Broadcast.create(2));
final SinkShape first = builder.add(s1);
final SinkShape second = builder.add(s2);
builder.from(bcast).to(first);
builder.from(bcast).to(second);
return SinkShape.of(bcast.in());
});
I'd like graph to be of shape SinkShape*>. Is
that possible?
– Kaspar
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] Deadlock in a flow with a cycle?
Hey everybody, I'm trying to convince myself that a flow I'm building with Akka Streams is deadlock-free. Here's what I'm trying to do: - I have an infinite source *s* of some kind of requests *r1, r2, ... *that I need "execute". - In case such an execution fails, I'd like to wait a bit (1s, say) and try again. - If a request cannot be executed for 10 times, it will be dropped. To keep resource consumption bounded, I thought I'd limit the number of requests (to 100, say). So when there are around 100 requests in the pipeline, the pipeline should not ask the source *s* for new elements but just continue retrying until we have again only 99 or fewer requests. I implemented this pseudo code: s ~> PreferredMerge~> Delay ~> buffer(100) ~> ExecuteRequest ~> Broadcast ~> Report PreferredMerge.pref() <~ RetriedOnceMore<~ FilterFail<~ Broadcast Here, the source *s* emits Retry(request, retryCount) objects. These enter a PreferredMerge stage on the un-preferred port, get delayed (using a mapAsync) and then buffered. ExecuteRequest executes the request. The result of this goes to a broadcast, one of whose outputs emits results (succeeded or failed). In FilterFail, requests that failed for less than 10 times are kept and get their retryCount incremented in RetriedOnceMore before they enter the PreferredMerge stage on the preferred port. This works. But could it deadlock? – Kaspar -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: [Akka Streams] Sink vs Flow (or: committing offsets after writing to Kafka)
On Monday, 12 October 2015 10:03:06 UTC-7, Julian Howarth wrote: > > Have you looked at reactive-kafka: > https://github.com/softwaremill/reactive-kafka ? > I have looked at reactive-kaka and am following some discussions on Gitter on the subject. > We use the method documented in the Manual Commit section of that page > which works well for us. Basically, it automatically commits some > configurable period of time after your processing of the message is > complete. So as long as your handling of the incoming messages is > idempotent, you can never lose messages. > Can you elaborate how that works exactly? My main question is this: an Akka Stream flow does message processing in parallel. So what can happen is that the Kafka consumer has read two messages and both of them are still in some flow and have *not* reached the sink (= the Kafka writer who writes to an output topic) yet. Suppose the auto-commit happens now and we crash. In this case, I will have lost two messages. Correct? If so, then auto-commit doesn't work. So we need to commit messages *after* they been written out. Looking at reactive-kafka, I think they use approach two from my original mail: val consumerWithOffsetSink = kafka.consumeWithOffsetSink(consumerProperties)Source(consumerWithOffsetSink.publisher) .map(processMessage(_)) // your message processing .to(consumerWithOffsetSink.offsetCommitSink) // stream back for commit .run() You'd write your message out in processMessage. – Kaspar > HTH, > > Julian > > On Sunday, October 11, 2015 at 11:55:21 PM UTC+1, hbf wrote: >> >> Hi, >> >> I using Akka streams to read (= consume) messages from a Kafka tropic, >> transform them, and write them to another Kafka topic. I am looking for a >> way to commit the consumer offset of a message after it was written. >> >> Example: if I've read message *m*, I'd like to first process it and >> write it out to the destination topic. Only then do I want to tell Kafka >> "ok, I've read *m; *if i crash and restart, position me after *m*, >> please!" >> >> Here are a few ways to realize this: >> >>- Make the writer a Sink and give it knowledge about the consumer so >>it can commit the latter's offset after writing. Not nice: the sink >>shouldn't have to know about a consumer. >>- Make the writer a Flow that as a side effect writes to Kafka. Then >>connect this flow to a CommitSink that commits the offsets. That doesn't >>sound nice either, as conceptually, both are sinks. >> >> Any suggestions how to do this in The Akka Streams Way®? >> >> Thanks, >> K >> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka Streams] Sink vs Flow (or: committing offsets after writing to Kafka)
Hi, I using Akka streams to read (= consume) messages from a Kafka tropic, transform them, and write them to another Kafka topic. I am looking for a way to commit the consumer offset of a message after it was written. Example: if I've read message *m*, I'd like to first process it and write it out to the destination topic. Only then do I want to tell Kafka "ok, I've read *m; *if i crash and restart, position me after *m*, please!" Here are a few ways to realize this: - Make the writer a Sink and give it knowledge about the consumer so it can commit the latter's offset after writing. Not nice: the sink shouldn't have to know about a consumer. - Make the writer a Flow that as a side effect writes to Kafka. Then connect this flow to a CommitSink that commits the offsets. That doesn't sound nice either, as conceptually, both are sinks. Any suggestions how to do this in The Akka Streams Way®? Thanks, K -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
