[akka-user] How to manage internal state of the system

[mandeep gandhi]

Hi,

I am building a system where I am receiving a request from the client and 
putting it into messaging queue. I only send the response to the client 
once I have an acknowledgment from the messaging queue that the message was 
placed into. 

The way I am doing it is using Akka-HTTP by passing the request to an 
internal actor form a directive and then the internal actor ( say 
MyLordActor) puts it in a queue.  MyLordActor, being a publisher,  has to 
maintain certain internal state intermittently ( until acknowledgment comes 
from the queue). 

Similarly, there is a queue consumer actor ( QConsumerActor) which is a 
queue consumer which de-queues the message and does not acknowledge the 
message (that the message is consumed) to the queue until the processing of 
the message is done. Processing of the message takes a sequence of 
non-trivial steps and passing ( and processing) messages between a bunch of 
actors.   This means that would have message processing would need the 
message to be processed by other services which may be either REST based or 
queue based. Finally, after the message is processed by the actors, a 
message is sent to QConsumerActor, which acknowledges the queue that the 
message is consumed. Hence the consumer actor, QConsumerActor,  also 
maintains some state to track the message in the system. 


Addtional fact -  this is deployed as a container on top of Apache Mesos ( 
DC/OS).  If I want to scale system - I scale containers. 

Here comes the bunch of question I wanted to ask - 


a. If I scale the system by scaling containers (say I want multiple 
consumers), in my case at least, I have the problem that my message can end 
up with the wrong consumer ( not the one who started consuming the message)
Now if the actor does not have any information that it consumed the 
message, it even won't be able to acknowledge it to the queue. 

How do people deal with this internal state nuisance? Can I avoid 
Akka clustering and remoting and still deal with the problem?


b.  How do people deal with concurrency ( DB writes) which you have 
multiple containers of the same actor system? In other words, if I have a 
DB actor which updates the DB and if I have multiple such containers ( I am 
horizontally scaling the system), my DB actor has to be extra careful while 
updating message with same message id because the other DB actor also might 
be updating it.

Do their DB actor have to deal with comparing custom logic such as 
comparing last modified time of the message etc. or people generally follow 
Akka remoting for this case - maintaining a single actor which writes to 
the DB. 


Thanks,
Mandeep Gandhi

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] HTTP server performance

[Mirek Suk]

Hi everyone.

AKKA 2.5.9 / HTTP 10.0.11

As part of prototyping architecture, I ran some benchmarks (including http) 
few months ago.
A simple hello world (request handler is completed with result immediately) 
had nice performance ~2.7m requests/minute. with 1,6m r/m when response 
included ask to sharded actor.

Recently I re-did those benchmarks to find out around ~9x performance drop 
on original test system.
Rolled prototype back to point of original test with similar result.
Built scala http sample with similar result.
Reinstalled JVM with similar result.
Randomly got performance back, until system restart (yeah like what ?)

On EC2-T2.medium I can do ~7k req/s to "hello world" page with 100% CPU 
usage (2 cores).
No akka.http configuration is changed and system does no background 
processing.
Default dispatcher is unchanged (only due to clustering throughput = 10).

Is this kind of performance expected ?
Is there something I can try ?

Thanks,
M.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] [Akka Streams] Difference between KillSwitch and Materializer.shutdown()

[Konrad “ktoso” Malawski]

First warning sign: Why would you have one stream per materializer?

shutting down the materializer while things are running is very brutal.
It’s like forcefully pulling the carpet from under someone’s feet,
and the laughing as they spill their coffee upon themselves — don’t do this
as the go-to solution.
It’s better than leaving resource leaks, but don’t do this as the “clean”
shutdown.


KillSwitches actually signal termination properly using streams signals —
cancelation and completion/error.
Use them when you want to externally stop things.

-- 
Cheers,
Konrad 'ktoso ' Malawski
Akka  @ Lightbend 

On February 19, 2018 at 21:31:58, dol...@thoughtworks.com (
dol...@thoughtworks.com) wrote:

I came across 2 ways to terminate a stream

   1. KillSwitch
   2. Materializer.shutdown()

I see one difference which is

   - Materializer.shutdown() is used to kill all streams materialized by
   that materializer whereas KillSwitch can be used to terminate one
   particular stream.


In a scenario where I have one materializer per stream, is there any
difference between KillSwitch and Materializer.shutdow()? Which one should
be used? And When?
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] [Akka Streams] Difference between KillSwitch and Materializer.shutdown()

[dollyg]

I came across 2 ways to terminate a stream

   1. KillSwitch
   2. Materializer.shutdown()

I see one difference which is 

   - Materializer.shutdown() is used to kill all streams materialized by 
   that materializer whereas KillSwitch can be used to terminate one 
   particular stream.
   

In a scenario where I have one materializer per stream, is there any 
difference between KillSwitch and Materializer.shutdow()? Which one should 
be used? And When?

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Getting a timeout error when testing a stream with mapAsync stage

[Jakub Kahovec]

Thank you. So I'll add the request(1) before the expectComplete(). I
actually use the .runWih(TesSink.probe) in my test, I used the
TestSubscriber.manualProbe just in the demonstration example to be able to
print the intermediate results.

Jakub

On 19 February 2018 at 12:18, Patrik Nordwall 
wrote:

> btw, probably easier to use .runWith(TestSink.probe) in that example, or
> .runWith(Sink.seq).futureValue
>
> On Mon, Feb 19, 2018 at 12:12 PM, Patrik Nordwall <
> patrik.nordw...@gmail.com> wrote:
>
>> Doc improvement is welcome. If you add request(1) before the
>> expectComplete it’s not undefined, then you are supposed to get it.
>>
>> /Patrik
>>
>> mån 19 feb. 2018 kl. 07:40 skrev Jakub Kahovec :
>>
>>> I had another thought about this behaviour and two things came to my
>>> mind. First, it might be worth mentioning this somewhere in the
>>> documentation about testing as other users might also experience this and I
>>> think didn't see this mentioned anywhere in the docs. And second, is it
>>> safe to use *expectComplete *at all if the behaviour is undefined and
>>> any such change might break the tests unexpectedly ?
>>>
>>> Jakub
>>>
>>> On Friday, February 16, 2018 at 9:55:45 PM UTC+1, Patrik Nordwall wrote:
>>>
 It is allowed to ”delay” delivery of the complete signal when there is
 no demand. This behavior is undefined, i.e. some stages deliver it
 immediately, some only when demand is requested. This is as expected and in
 such test you have to request enough to be sure to get the completed 
 signal.

 /Patrik

>>> fre 16 feb. 2018 kl. 09:02 skrev Jakub Kahovec :

>>> Hi,
>
>
> when using Akka Streams (2.5.9)  I've recently bumped into a problem
> when testing a stream with a mapAsync stage. It unexpectedly ends with an
> assertion failing on *"timeout (3 seconds) during expectMsg while
> waiting for OnComplete"*
>
>
> To demonstrate it I've created a simple example.
>
>
> The code below works as expected.
>
>
> val sourceUnderTest = Source(1 to 2).mapAsync(2)(i => Future.
> successful(i * 2))
>
> val c = TestSubscriber.manualProbe[Int]()
> val p = sourceUnderTest.to(Sink.fromSubscriber(c)).run()
> val sub = c.expectSubscription()
>
> sub.request(2)
> c.expectNextN(2)   // List(2,4)
> c.expectComplete()   // akka.stream.testkit.TestSubscr
> iber$ManualProbe@fc258b1
>
>
> However, when I add another mapping function (bold code), the test
> fails with a timeout. When I call additional c.request(1) (commented code)
> it ends correctly. So it looks like the mapping function adds an
> additional item into the stream, which seems strange.
>
>
> val sourceUnderTest = Source(1 to 2).mapAsync(2)(i => Future.
> successful(i * 2)*.**map(identity**)*)
>
> val c = TestSubscriber.manualProbe[Int]()
> val p = sourceUnderTest.to(Sink.fromSubscriber(c)).run()
> val sub = c.expectSubscription()
>
> sub.request(2)
> c.expectNextN(2)   // List(2,4)
> // c.request(1)
> c.expectComplete()   // ends with java.lang.AssertionError: assertion
> failed: timeout (3 seconds) during expectMsg while waiting for OnComplete
>
>
> Can anyone explain this strange behaviour ?
>
>
> Thanks
>
> Jakub
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/c
> urrent/additional/faq.html
> >> Search the archives: https://groups.google.com/grou
> p/akka-user
> ---
> You received this message because you are subscribed to the Google
> Groups "Akka User List" group.
>
 To unsubscribe from this group and stop receiving emails from it, send
> an email to akka-user+...@googlegroups.com.
> To post to this group, send email to akka...@googlegroups.com.


> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
 --
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ: http://doc.akka.io/docs/akka/c
>>> urrent/additional/faq.html
>>> >> Search the archives: https://groups.google.com/grou
>>> p/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to akka-user+unsubscr...@googlegroups.com.
>>> To post to this group, send email to akka-user@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>
>
> --
>
> Patrik Nordwall
> Akka Tech Lead
> Lightbend  -  Reactive apps on the JVM
> Twitter: @patriknw
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the

Re: [akka-user] Getting a timeout error when testing a stream with mapAsync stage

[Patrik Nordwall]

btw, probably easier to use .runWith(TestSink.probe) in that example, or
.runWith(Sink.seq).futureValue

On Mon, Feb 19, 2018 at 12:12 PM, Patrik Nordwall  wrote:

> Doc improvement is welcome. If you add request(1) before the
> expectComplete it’s not undefined, then you are supposed to get it.
>
> /Patrik
>
> mån 19 feb. 2018 kl. 07:40 skrev Jakub Kahovec :
>
>> I had another thought about this behaviour and two things came to my
>> mind. First, it might be worth mentioning this somewhere in the
>> documentation about testing as other users might also experience this and I
>> think didn't see this mentioned anywhere in the docs. And second, is it
>> safe to use *expectComplete *at all if the behaviour is undefined and
>> any such change might break the tests unexpectedly ?
>>
>> Jakub
>>
>> On Friday, February 16, 2018 at 9:55:45 PM UTC+1, Patrik Nordwall wrote:
>>
>>> It is allowed to ”delay” delivery of the complete signal when there is
>>> no demand. This behavior is undefined, i.e. some stages deliver it
>>> immediately, some only when demand is requested. This is as expected and in
>>> such test you have to request enough to be sure to get the completed signal.
>>>
>>> /Patrik
>>>
>> fre 16 feb. 2018 kl. 09:02 skrev Jakub Kahovec :
>>>
>> Hi,


 when using Akka Streams (2.5.9)  I've recently bumped into a problem
 when testing a stream with a mapAsync stage. It unexpectedly ends with an
 assertion failing on *"timeout (3 seconds) during expectMsg while
 waiting for OnComplete"*


 To demonstrate it I've created a simple example.


 The code below works as expected.


 val sourceUnderTest = Source(1 to 2).mapAsync(2)(i => Future.successful
 (i * 2))

 val c = TestSubscriber.manualProbe[Int]()
 val p = sourceUnderTest.to(Sink.fromSubscriber(c)).run()
 val sub = c.expectSubscription()

 sub.request(2)
 c.expectNextN(2)   // List(2,4)
 c.expectComplete()   // akka.stream.testkit.TestSubscriber$ManualProbe@
 fc258b1


 However, when I add another mapping function (bold code), the test
 fails with a timeout. When I call additional c.request(1) (commented code)
 it ends correctly. So it looks like the mapping function adds an
 additional item into the stream, which seems strange.


 val sourceUnderTest = Source(1 to 2).mapAsync(2)(i => Future.successful
 (i * 2)*.**map(identity**)*)

 val c = TestSubscriber.manualProbe[Int]()
 val p = sourceUnderTest.to(Sink.fromSubscriber(c)).run()
 val sub = c.expectSubscription()

 sub.request(2)
 c.expectNextN(2)   // List(2,4)
 // c.request(1)
 c.expectComplete()   // ends with java.lang.AssertionError: assertion
 failed: timeout (3 seconds) during expectMsg while waiting for OnComplete


 Can anyone explain this strange behaviour ?


 Thanks

 Jakub

 --
 >> Read the docs: http://akka.io/docs/
 >> Check the FAQ: http://doc.akka.io/docs/akka/
 current/additional/faq.html
 >> Search the archives: https://groups.google.com/
 group/akka-user
 ---
 You received this message because you are subscribed to the Google
 Groups "Akka User List" group.

>>> To unsubscribe from this group and stop receiving emails from it, send
 an email to akka-user+...@googlegroups.com.
 To post to this group, send email to akka...@googlegroups.com.
>>>
>>>
 Visit this group at https://groups.google.com/group/akka-user.
 For more options, visit https://groups.google.com/d/optout.

>>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: http://doc.akka.io/docs/akka/
>> current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+unsubscr...@googlegroups.com.
>> To post to this group, send email to akka-user@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>


-- 

Patrik Nordwall
Akka Tech Lead
Lightbend  -  Reactive apps on the JVM
Twitter: @patriknw

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https:/

Re: [akka-user] Getting a timeout error when testing a stream with mapAsync stage

[Patrik Nordwall]

Doc improvement is welcome. If you add request(1) before the expectComplete
it’s not undefined, then you are supposed to get it.

/Patrik
mån 19 feb. 2018 kl. 07:40 skrev Jakub Kahovec :

> I had another thought about this behaviour and two things came to my mind.
> First, it might be worth mentioning this somewhere in the documentation
> about testing as other users might also experience this and I think didn't
> see this mentioned anywhere in the docs. And second, is it safe to use 
> *expectComplete
> *at all if the behaviour is undefined and any such change might break the
> tests unexpectedly ?
>
> Jakub
>
> On Friday, February 16, 2018 at 9:55:45 PM UTC+1, Patrik Nordwall wrote:
>
>> It is allowed to ”delay” delivery of the complete signal when there is no
>> demand. This behavior is undefined, i.e. some stages deliver it
>> immediately, some only when demand is requested. This is as expected and in
>> such test you have to request enough to be sure to get the completed signal.
>>
>> /Patrik
>>
> fre 16 feb. 2018 kl. 09:02 skrev Jakub Kahovec :
>>
> Hi,
>>>
>>>
>>> when using Akka Streams (2.5.9)  I've recently bumped into a problem
>>> when testing a stream with a mapAsync stage. It unexpectedly ends with an
>>> assertion failing on *"timeout (3 seconds) during expectMsg while
>>> waiting for OnComplete"*
>>>
>>>
>>> To demonstrate it I've created a simple example.
>>>
>>>
>>> The code below works as expected.
>>>
>>>
>>> val sourceUnderTest = Source(1 to 2).mapAsync(2)(i => Future.successful(i
>>> * 2))
>>>
>>> val c = TestSubscriber.manualProbe[Int]()
>>> val p = sourceUnderTest.to(Sink.fromSubscriber(c)).run()
>>> val sub = c.expectSubscription()
>>>
>>> sub.request(2)
>>> c.expectNextN(2)   // List(2,4)
>>> c.expectComplete()   //
>>> akka.stream.testkit.TestSubscriber$ManualProbe@fc258b1
>>>
>>>
>>> However, when I add another mapping function (bold code), the test fails
>>> with a timeout. When I call additional c.request(1) (commented code) it
>>> ends correctly. So it looks like the mapping function adds an
>>> additional item into the stream, which seems strange.
>>>
>>>
>>> val sourceUnderTest = Source(1 to 2).mapAsync(2)(i => Future.successful(i
>>> * 2)*.**map(identity**)*)
>>>
>>> val c = TestSubscriber.manualProbe[Int]()
>>> val p = sourceUnderTest.to(Sink.fromSubscriber(c)).run()
>>> val sub = c.expectSubscription()
>>>
>>> sub.request(2)
>>> c.expectNextN(2)   // List(2,4)
>>> // c.request(1)
>>> c.expectComplete()   // ends with java.lang.AssertionError: assertion
>>> failed: timeout (3 seconds) during expectMsg while waiting for OnComplete
>>>
>>>
>>> Can anyone explain this strange behaviour ?
>>>
>>>
>>> Thanks
>>>
>>> Jakub
>>>
>>> --
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ:
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >> Search the archives:
>>> https://groups.google.com/group/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Akka User List" group.
>>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>
>>
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] How to build the Akka API documentation for offline use?

[Martynas Mickevičius]

Run the unidoc task in sbt. It comes from sbt-unidoc and aggregates the API
docs from all sub-projects to ./target/unidoc/

On Mon, Feb 19, 2018 at 9:45 AM Thai  wrote:

> Hi All,
>
> I noticed that the Akka API doc is unavailable to download like Scala, so
> I git-cloned the akka and ran `doc` in sbt. However the API docs were
> separated like this:
>
> ./akka/akka-cluster-sharding/target/api
> ./akka/akka-contrib/target/api
> ./akka/akka-cluster-tools/target/api
> ./akka/akka-cluster/target/api
> ./akka/akka-agent/target/api
> ./akka/akka-camel/target/api
> ./akka/akka-persistence-tck/target/api
> ./akka/akka-multi-node-testkit/target/api
> ./akka/akka-distributed-data/target/api
> ./akka/akka-cluster-sharding-typed/target/api
> ./akka/akka-stream-testkit/target/api
> ./akka/akka-remote/target/api
> ./akka/akka-cluster-metrics/target/api
> ./akka/akka-protobuf/target/api
> ./akka/akka-persistence-query/target/api
> ./akka/akka-persistence/target/api
> ./akka/akka-testkit-typed/target/api
> ./akka/akka-cluster-typed/target/api
> ./akka/akka-actor-typed/target/api
> ./akka/akka-stream/target/api
> ./akka/akka-osgi/target/api
> ./akka/akka-testkit/target/api
> ./akka/akka-persistence-typed/target/api
>
> Does anybody know how to put them together so that I can have something
> similar to https://doc.akka.io/api/akka/current/akka/actor/index.html?
>
> Regards.
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Actor usage in Apache Spark and Apache Flink

[Konrad Malawski]

Another pretty big Akka based project (as in, really using Akka a lot) is
Apache OpenWhisk, the open source "amazon lambda";
Check it out as well.

https://github.com/apache?q=openwhisk

On Mon, Feb 19, 2018 at 2:32 PM, Konrad “ktoso” Malawski <
konrad.malaw...@lightbend.com> wrote:

> Best to ask on a Flink forum ;-)
>
> --
> Cheers,
> Konrad 'ktoso ' Malawski
> Akka  @ Lightbend 
>
> On February 19, 2018 at 14:32:15, Unmesh Joshi (unmeshjo...@gmail.com)
> wrote:
>
> Thanks Konrad.
> What I was interested in was what use cases both Spark and Flink used
> Actors for specifically. Are there any notes/docs around those that anyone
> knows of?
>
>
>
> On Monday, 19 February 2018 07:33:43 UTC+5:30, Konrad Malawski wrote:
>>
>> Hi there,
>> There's been some confusion about how Spark used Akka, so let's clear
>> that up first:
>> It wasn't really core to any of the operations or core of Spark, and was
>> used only to orchestrate some things,
>> it was using raw Akka Remoting (which is discouraged, use Cluster instead
>> due to the resilience benefits it gives).
>>
>> Users would sometimes want to combine Akka based things with their Spark
>> jobs, but since Spark did not have Akka as a focus-on
>> thing, but merely a small piece of code they tagged along, and they had
>> internal mechanisms to do similar things as they used Akka for (just a few
>> simple rpc-style calls),
>> it was better for everyone to remove the dependency -- as Spark was using
>> a rather old version, and people wanted to use recent versions of Akka --
>> the usual dependency mess you get with big projects in other words.
>>
>> Thus Akka was removed from Spark, allowing users to bring their own Akka
>> based libraries into Spark apps at-will, which is better for both projects
>> and users as well.
>> Shedding dependencies is generally a good thing, and I think Spark did
>> the right thing there. You'll also notice how Akka tries to avoid any kind
>> of external dependency that we're able to avoid -- for the same reasons,
>> avoiding dependency hell for end users.
>>
>> Hope this helps,
>>
>> On Sun, Feb 18, 2018 at 8:58 PM, Unmesh Joshi  wrote:
>>
>>> Hi,
>>>
>>> Apache Spark used to use Akka Actors (which is replaced by Netty now)
>>> and Apache Flink uses Actors. I am trying to understand the use case of
>>> Actor usage in both these frameworks. Any pointers for help?
>>>
>>> Thanks,
>>> Unmesh
>>> --
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ: http://doc.akka.io/docs/akka/c
>>> urrent/additional/faq.html
>>> >> Search the archives: https://groups.google.com/grou
>>> p/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Cheers,
>> Konrad 'ktoso' Malawski
>> Akka  @ Typesafe 
>>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>


-- 
Cheers,
Konrad 'ktoso' Malawski
Akka  @ Lightbend 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Actor usage in Apache Spark and Apache Flink

[Konrad “ktoso” Malawski]

Best to ask on a Flink forum ;-)

-- 
Cheers,
Konrad 'ktoso ' Malawski
Akka  @ Lightbend 

On February 19, 2018 at 14:32:15, Unmesh Joshi (unmeshjo...@gmail.com)
wrote:

Thanks Konrad.
What I was interested in was what use cases both Spark and Flink used
Actors for specifically. Are there any notes/docs around those that anyone
knows of?



On Monday, 19 February 2018 07:33:43 UTC+5:30, Konrad Malawski wrote:
>
> Hi there,
> There's been some confusion about how Spark used Akka, so let's clear that
> up first:
> It wasn't really core to any of the operations or core of Spark, and was
> used only to orchestrate some things,
> it was using raw Akka Remoting (which is discouraged, use Cluster instead
> due to the resilience benefits it gives).
>
> Users would sometimes want to combine Akka based things with their Spark
> jobs, but since Spark did not have Akka as a focus-on
> thing, but merely a small piece of code they tagged along, and they had
> internal mechanisms to do similar things as they used Akka for (just a few
> simple rpc-style calls),
> it was better for everyone to remove the dependency -- as Spark was using
> a rather old version, and people wanted to use recent versions of Akka --
> the usual dependency mess you get with big projects in other words.
>
> Thus Akka was removed from Spark, allowing users to bring their own Akka
> based libraries into Spark apps at-will, which is better for both projects
> and users as well.
> Shedding dependencies is generally a good thing, and I think Spark did the
> right thing there. You'll also notice how Akka tries to avoid any kind of
> external dependency that we're able to avoid -- for the same reasons,
> avoiding dependency hell for end users.
>
> Hope this helps,
>
> On Sun, Feb 18, 2018 at 8:58 PM, Unmesh Joshi  wrote:
>
>> Hi,
>>
>> Apache Spark used to use Akka Actors (which is replaced by Netty now) and
>> Apache Flink uses Actors. I am trying to understand the use case of Actor
>> usage in both these frameworks. Any pointers for help?
>>
>> Thanks,
>> Unmesh
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: http://doc.akka.io/docs/akka/
>> current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com.
>> To post to this group, send email to akka...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Cheers,
> Konrad 'ktoso' Malawski
> Akka  @ Typesafe 
>
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Alpakka - Contribution ideas

[Arun Manivannan]

Hi,

I've always known that the Akka community is very welcoming to the
contributors. And Enno proved that to me again.

I just made a small contribution towards the self-contained example (789
) in Alpakka and I am looking for
ideas where I could learn and contribute more. TBH, I am just familiar with
plain old Actors and everything else is a learning opportunity.

Here's the list that I have in mind.  Appreciate if you could let me know
if these are not suitable for Alpakka contributions.  If you have any other
ideas else that involves reading and contributing to the Akka code base,
please help.

*Needs some homework but I can take a stab *
Doobie connector (I believe it must be similar to Slick)
HDFS connector (Already taken - Connector for HDFS #557)

*Self-contained examples (Definitely possible)*
Consume from RDBMS and route it to Solr (just like ES)
Consume from Kafka and route it to RDBMS - Slick sink
Consume from Kafka and push it to HBase
Consume from Kafka and push it to Cassandra

Thanks a ton, all.
Cheers,
Arun

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.