Re: [akka-user] Memory Bounding Akka Streams

[Dagny Taggart]

LATEST updated understanding I have then is the following; and if someone
could PLEASE correct my (Newbie) understanding with a link to a clear Blog
Post or GitHub repo illustrating the concept!

1) The Akka Stream Graph DSL API's abstraction-level hides the details of
handling Backpressure between Source to Sink.

i.e. A Developer never has to call the Publisher or Subscriber APIs to
initially request or signal Sink capacity for bytes of data; nor to call
'onNext' to get the next batch of data after processing the first batch.
Akka Streams handles this 'under-the-covers'.

2) Buffers (of specified Byte size) and with various drop strategies (e.g.
drop oldest event, etc) can be specified on each Graph DSL API's Flow
Stage.  This would delay the signal back to prior Stage to Source to 'slow
down' when the (default-sized) Akka Byte memory buffer is pressured.

3) At the moment; I understand that a backpressured Source will just appear
to 'slow down'.  In which case, I suspect memory on the Source would be
pressured with the backpressure signal if Sink is slower in processing
Source data.

SUMMARY QUESTION:
I just don't know how to pickup that backpressure signal on an Http
Websocket Javascript client;
so that I can show some kind of Error-Message to the mobile or webapp User
that they have to slow down on their event activity due to system overload.
i.e. What's an example of using a high-level Akka API method would o do
this!

THANKS in advance!
D





On Wed, Sep 21, 2016 at 11:01 AM, Dagny T  wrote:

>
> Just wanted to check with folks if I had the correct implementation for
> how to protect from blowing up memory when working with Akka Streams.
>
> I've merged a Lightbend Blog post's code, with the latest API changes for
> Akka v2.4.9, and the latest documentation about buffered streams in the
> v2.4.9 API Docs.
>
> However, none of those explain these questions I have.  Please see
> question comments, regarding the code snippet below it!  THANKS in advance
> for any insights!
>
> // TODO 3:  MODIFIED to calling buffered API within Graph mapping -- check 
> assumptions!
> //  - where INTERNAL Akka implementation calls onNext() to get next 
> BUFFERED batch,
> //so you don't have to worry about it as a DEV?
> //  - NUMERIC bound of 10 refers to NUMBER of elements (of possibly 
> complex types) on a
> //UNIFORM-ELEMENT-TYPED stream, rather than Bytes, right?
> //  - if source produces N < BUFFER_MAX elements; then those are 
> simply passed through the pipeline without
> //waiting to accumulate BUFFER_MAX elements
> //
>
>
> inputSource.buffer(10, OverflowStrategy.dropHead) ~> f1 ~> ...
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Akka User List" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/akka-user/bolqHjF_dvc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Akka and Streams Concurrency Fundamentals

[Dagny Taggart]

Hi Johan and Patrik,

Thanks very much for your responses; and I'm just getting back to this.

Here's a cheeky request:
* Any chance you guys could Blog and put Links to GitHub for "Newbie
Reference Implementations" for putting together focused learning code
samples for these kinds of fundamentals, and with samples of using your
Testkits?
* Otherwise, could you get someone who is good at teaching Newbies; publish
a coherent learning progression project with Manning on the latest and
greatest Streams-Microservices-Architecture techniques?

That kind of insight on project-build-approach is hard to put together for
a Newbie from even top-notch API documentation.
And, I certainly did want to express my gratitude for the excellent
searchable Akka 2.4.8 Documentation!

THANK YOU so much!
D


On Wed, Sep 7, 2016 at 5:00 AM, Akka Team  wrote:

> Hi Dagny,
>
> The Akka Streams tests for the built in stages could perhaps be useful to
> see various strategies for testing. You can find those sources here:
> https://github.com/akka/akka/tree/master/akka-stream-tests/
> src/test/scala/akka/stream/scaladsl
>
> --
> Johan
> Akka Team
>
> On Fri, Sep 2, 2016 at 8:08 PM, Dagny T  wrote:
>
>>
>> Yes, well; that link is rather generic, no?  ;0)
>>
>> What I (and likely the 19 other Viewers of this post) are looking for is
>> baseline-thorough coverage of test-cases to be aware of when testing Akka
>> Actor and Streams operations.
>> I'm a newbie, and I learn best through running idiomatic code examples
>> with test cases; hence my interest.
>>
>> I didn't find that in the docs; nor am I finding it in any of the Blogs
>> that I've been Googling around for.
>> That's why I'm asking for help with finding a current and reliable
>> information source.
>>
>> I found these specific docs do provide a scratch-the-surface start:
>> http://doc.akka.io/docs/akka/2.4.9/scala/stream/stream-testkit.html
>>
>> However, I'm looking for Tutorial-level code covering things like:
>> - testing backpressure operation with Bounded memory settings when
>> operating simple Graphs of Flows
>> - testing for connection drops, resend of duplicate message,  and Client
>> retries and wait-timeouts
>> - testing for system exception-handling occurring in a mid-Flow Stage.
>>
>> Would be darn grateful if you had any specific insights on where to find
>> such learning resources; and were willing to share them with this Forum!
>>
>> Thanks in advance for continuing to support a welcoming community for
>> Akka Newbs!  You'll make the World a Better Place with Akka Everywhere!  ;0)
>> Dagny T
>>
>>
>> On Thursday, September 1, 2016 at 2:43:46 PM UTC-7, Dagny T wrote:
>>>
>>>
>>> I'd like to find reliable and current information for learning this
>>> using v2.4.9.
>>>
>>> I've posted a reply to someone else's related message; but wanted to
>>> call Forum attention to please help refer us to a useful Blog or GitHub
>>> repo on this topic; AND
>>> using the latest and greatest TestKit tools!
>>>
>>> It's been rather hit-or-miss with Googling samples lately; as the APIs
>>> have been moving rapidly, and I'm still too new to pick out what's
>>> essentially most important!
>>>
>>> THANK you so much in advance for any Best-Practice info on these
>>> fundamentals!
>>>
>>> Here was my reply to that earlier related post:
>>> https://groups.google.com/forum/#!topic/akka-user/4k4nXVn-wWw
>>>
>>> THANKS in advance to hAKKArs!
>>>
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: http://doc.akka.io/docs/akka/c
>> urrent/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+unsubscr...@googlegroups.com.
>> To post to this group, send email to akka-user@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Akka Team
> Lightbend  - Reactive apps on the JVM
> Twitter: @akkateam
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Akka User List" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/akka-user/FN3690Il9Kw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit 

Re: [akka-user] Re: How to Test WebSocket Streams

[Dagny Taggart]

Hi!

Just getting back to this; and THANKS for the sbt dependencies info -- I'll
have to try that out.
It looks like I already had the http testkit; but you're using an
additional "test" specifier ...;
and then your ScalaTest note is helpful too.

Would be awesome for Scala/Akka to have the equivalent of Java's
"Jarfinder" website to lookup SBT Maven Artifacts by Classname!

Here's a cheeky request:
* Any chance you guys could Blog and put Links to GitHub for "Newbie
Reference Implementations" for putting together focused learning code
samples like this?
* Otherwise, could you get someone who is good at teaching Newbies; publish
a coherent learning progression project with Manning on the latest and
greatest Streams-Microservices-Architecture techniques?

Thanks so much again!
D







On Thu, Sep 8, 2016 at 6:03 AM, Akka Team  wrote:

> Hi Dagny,
>
> The dependency you need is akka-http-testkit, so in sbt an entry like
> this: "com.typesafe.akka" %% "akka-http-testkit" % akkaV % "test"
> It also depends on scalatest, we use version 2.2.1 and I see you have put
> a dependency on a snapshot of Scalatest 3, this may cause some problems,
> you should notice those as it probably wont compile if there is an
> incompability.
>
> The complete example file used in the docs can be found here, and should
> help you see what imports are needed:
> https://github.com/akka/akka/blob/4acc1cca6a27be0ff80f801de3640f
> 91343dce94/akka-docs/rst/scala/code/docs/http/scaladsl/server/directives/
> WebSocketDirectivesExamplesSpec.scala
>
> --
> Johan
> Akka Team
>
> On Wed, Aug 31, 2016 at 10:32 PM, Dagny T  wrote:
>
>>
>> TYPO!  My latest WebSocketServer experiment is ACTUALLY in:
>> - streamsWebSocket.scala
>>
>>
>> On Wednesday, August 31, 2016 at 1:23:36 PM UTC-7, Dagny T wrote:
>>>
>>> Newbie Testing WebSocket on Stream.
>>>
>>> Needed to verify I'm following best-practices on the basics for this!
>>>
>>> Please refer to my Repo:
>>> https://github.com/DagnyTagg2013/ScalaReactive/tree/master/newbie
>>>
>>> 1) Tried initiating Web Socket Connection JS Client: basicWebSockets.js
>>> then handling it on an Akka Streams Server with an Http Request Handler:
>>> firstWebSocketServer.scala
>>>
>>> 2) Tried to test Server with Akka TestKit; but missing build.sbt
>>> dependencies for WS(...) and RoutingSpec(...)
>>> - Test code here: testWebSocketServer2.scala
>>> - Dependencies here:  build.sbt
>>>
>>> If someone could please comment on how to:
>>> - successfully get a WebSocket connection from (1) -- am I setting up
>>> the JS and Scala connection correctly?
>>> - successfully get the test dependencies to build from (2) -- am I using
>>> the correct version of Akka Test libraries to work the the code in
>>> testWebSocketServer2?
>>>
>>> THANKS a lot!
>>> D
>>>
>>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: http://doc.akka.io/docs/akka/c
>> urrent/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+unsubscr...@googlegroups.com.
>> To post to this group, send email to akka-user@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Akka Team
> Lightbend  - Reactive apps on the JVM
> Twitter: @akkateam
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Akka User List" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/akka-user/aC19F2J-QSY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit