[akka-user] ActorPublisher GraphStage alternative?

2017-05-26 Thread Curt Siffert 

Hi, I see in the docs for 2.5.2 that ActorPublisher/ActorSubscriber will be 
deprecated.

In my (still beginning) experiments with akka streams I used ActorPublisher 
as a way to help create some back pressure controls while consuming 
messages from an external queue. This worked just by consuming the queue 
like normal and then for each message consumed, sending a message to 
ActorPublisher.

Without using ActorPublisher, I can use a Source.actorRef, but that doesn't 
have back pressure controls.

I know the recommended alternative to ActorPublisher is to use a custom 
graph stage and I have started experimenting with that but so far I don't 
see how to meet the ActorPublisher use case with it. So far it doesn't seem 
like a custom Source has an ActorRef type signature like Source.actorRef 
does. Once the custom stage is created, can I send a "tell" message to it 
the way I did to ActorPublisher? Or am I supposed to use Source.actorRef 
and then funnel it through the custom stage to get the back pressure 
controls?

Sorry if my question is muddled, I am still making my way through this. :-) 
I recognize this is a bit weird since ideally the back pressure controls 
would be applied to the queueing tech itself.

Curt

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka stream - Source of http response.

2017-05-26 Thread Konrad Malawski 
Hi Alejandro,
I don't get the question - could you rephrase it a bit (more details)?

An HttpRequest simply contains a Source[ByteString, _] and that "just
works"™.

-- 
Konrad `ktoso` Malawski
Akka  @ Lightbend 

On 26 May 2017 at 00:21:38, Alejandro Merchan (alejandromercha...@gmail.com)
wrote:

Hi everybody.

One question, how is possible create a continuous Source from http client
response?

thanks !!
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Dispatcher configuration ?

2017-05-26 Thread Patrik Nordwall 
The most important is that you put the blocking actors/task on a dedicated
dispatcher, and don't run them on the default dispatcher.
http://doc.akka.io/docs/akka/current/scala/general/actor-systems.html#blocking-needs-careful-management

/Patrik

On Thu, May 25, 2017 at 5:52 PM, Vishnu Vardhan  wrote:

> Hi,
>
> I posted earlier here https://groups.google.com/forum/#!topic/akka-user/
> 7jwQC7vuYQs but i wanted to reframe what i think is going on and seek
> some advice.
>
> The code is available on GitHub if anybody is interested. https://github.
> com/vardhanv/cosbench_ng
>
> Essentially, i am writing an S3 performance tester. Because it is a
> performance tester it needs to scale..
>
> My architecture is
> - An akka cluster, with a singleton consistent hashing router
> - Many slaves running on other servers (default is 2 slave workers per
> host)
> - An akka streams implementation that is routing jobs to the slaves
> (gets/PUTs) etc
> - Slave workers that receive the commands and forward them to
>- a stateless S3 scala object that does blocking S3 PUTs - but is
> protected by a custom thread pool executor with 40 threads
>
> My default test setup is "one server that also has two slaves, and one
> agent that hosts two more slaves - for a total of 4 slaves".
>
> All this in docker containers, so that deployment is simple.
>
> I believe that the inconsistent behavior i am seeing with startup, message
> passing, unexpected shutdowns are to do with how these various pieces and
> their threads are getting scheduled.
>
> I am wondering if my hypothesis is true, and what can i do to protect
> these components from each other (cluster message passing, from stream
> messages, from the blocking calls)
>
> Best Regards
> Vishnu
>
>
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

Patrik Nordwall
Akka Tech Lead
Lightbend  -  Reactive apps on the JVM
Twitter: @patriknw

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Issue in Sending messages through akka cluster.

2017-05-26 Thread Patrik Nordwall 
Work pulling (see my previous link) might be a good choice.

Note that it's not only about that the end consumers might not consume fast
enough, it's also anything on the way there, such as serialization and
deserialization (which I think is your bottleneck).

On Thu, May 25, 2017 at 1:16 PM, chaitanya  wrote:

> We used Single Producer and Multiple Consumer pattern
>
>
> On Thursday, 25 May 2017 15:56:15 UTC+5:30, chaitanya wrote:
>>
>> " The buffers in Artery are bounded and will start dropping messages if
>> overwhelmed " -
>>
>> which pattern would be best to resolve this issue and faster?
>>
>>
>> On Wednesday, 24 May 2017 17:54:11 UTC+5:30, Patrik Nordwall wrote:
>>>
>>> You have to implement some flow control, such as as using the "Work
>>> Pulling" pattern
>>> ,
>>> throttling, or send acknowledgment messages back from the consumer. If
>>> producer is sending messages at a higher rate than they can be consumed
>>> those messages must be buffered somewhere and that can eventually result in
>>> out of memory. The buffers in Artery are bounded and will start dropping
>>> messages if overwhelmed.
>>>
>>> Cheers,
>>> Patrik
>>>
>>> On Wed, May 24, 2017 at 9:57 AM, chaitanya  wrote:
>>>
 Following is the console output of my program it says Dropping messages
 due to overflow of send queue. How to solve this?

  GET Messages   >>> Content
  GET Messages   >>> Content
 12:49:35.091 [ClusterSystem-akka.actor.default-dispatcher-3] DEBUG
 a.r.a.Association(akka://ClusterSystem) - Dropping message
 [ActorSelectionMessage(java.lang.String)] from
 [Actor[akka://ClusterSystem/user/master_1#-495216506]] to
 [Actor[akka://ClusterSystem@127.0.0.1:2551/]] due to overflow of send
 queue, size [3072]
 12:49:35.092 [ClusterSystem-akka.actor.default-dispatcher-3] DEBUG
 a.r.a.Association(akka://ClusterSystem) - Dropping message
 [ActorSelectionMessage(java.lang.String)] from
 [Actor[akka://ClusterSystem/user/master_1#-495216506]] to
 [Actor[akka://ClusterSystem@127.0.0.1:2551/]] due to overflow of send
 queue, size [3072]
 12:49:35.092 [ClusterSystem-akka.actor.default-dispatcher-3] DEBUG
 a.r.a.Association(akka://ClusterSystem) - Dropping message
 [ActorSelectionMessage(java.lang.String)] from
 [Actor[akka://ClusterSystem/user/master_1#-495216506]] to
 [Actor[akka://ClusterSystem@127.0.0.1:2551/]] due to overflow of send
 queue, size [3072]
 12:49:35.092 [ClusterSystem-akka.actor.default-dispatcher-3] INFO
  a.r.RemoteActorRefProvider$RemoteDeadLetterActorRef - Message
 [java.lang.String] from 
 Actor[akka://ClusterSystem/user/master_1#-495216506]
 to Actor[akka://ClusterSystem/deadLetters] was not delivered. [1] dead
 letters encountered. This logging can be turned off or adjusted with
 configuration settings 'akka.log-dead-letters' and
 'akka.log-dead-letters-during-shutdown'.
 12:49:35.093 [ClusterSystem-akka.actor.default-dispatcher-3] INFO
  a.r.RemoteActorRefProvider$RemoteDeadLetterActorRef - Message
 [java.lang.String] from 
 Actor[akka://ClusterSystem/user/master_1#-495216506]
 to Actor[akka://ClusterSystem/deadLetters] was not delivered. [2] dead
 letters encountered. This logging can be turned off or adjusted with
 configuration settings 'akka.log-dead-letters' and
 'akka.log-dead-letters-during-shutdown'.
 12:49:35.093 [ClusterSystem-akka.actor.default-dispatcher-2] DEBUG
 a.r.a.Association(akka://ClusterSystem) - Dropping message
 [ActorSelectionMessage(java.lang.String)] from
 [Actor[akka://ClusterSystem/user/master_1#-495216506]] to
 [Actor[akka://ClusterSystem@127.0.0.1:2551/]] due to overflow of send
 queue, size [3072]
 12:49:35.095 [ClusterSystem-akka.actor.default-dispatcher-2] DEBUG
 a.r.a.Association(akka://ClusterSystem) - Dropping message
 [ActorSelectionMessage(java.lang.String)] from
 [Actor[akka://ClusterSystem/user/master_1#-495216506]] to
 [Actor[akka://ClusterSystem@127.0.0.1:2551/]] due to overflow of send
 queue, size [3072]
 12:49:35.097 [ClusterSystem-akka.actor.default-dispatcher-2] DEBUG
 a.r.a.Association(akka://ClusterSystem) - Dropping message
 [ActorSelectionMessage(java.lang.String)] from
 [Actor[akka://ClusterSystem/user/master_1#-495216506]] to
 [Actor[akka://ClusterSystem@127.0.0.1:2551/]] due to overflow of send
 queue, size [3072]

 How can

 On Wednesday, 24 May 2017 00:05:42 UTC+5:30, chaitanya wrote:
>
> Hi guys,'
> I am a newbie to akka cluster  and  i have created a sample program. I
> am using producer consumer pattern. My producer sends messages to consumer
> which are 2 seed nodes.
> The program works fine when no of messages are less like in

Re: [akka-user] ANNOUNCE: New Akka documentation, website and Akka 2.5.2 released

2017-05-26 Thread Lee_T 


On Thursday, May 25, 2017 at 4:23:02 AM UTC-4, Konrad Malawski wrote:
>
> Hi Lee,
> RSS works for me, do you use this feed: http://akka.io/rss.xml ?
>
>
Konrad,

Thank you for checking.  For the sake of discussion & common
reference, I use the Lightbend site as reference.  Yes, I understand
that Akka is a distinct subset of Lightbend...

By the Lightbend (and many other website) reference, RSS on the
new Akka.io site is definitely broken, as lest for some subset of people.


My existing bookmark had been working for months/years. With
the new site I get (loosely) "bookmark would not load".

I deleted my old bookmark, went to the  https://akka.io/rss.xml you kindly 
provided, and re-subscribed.  Things got better. I now get a pull-down
list of topics.  When I follow any one of those topics, I get an unformatted
list of all topics, not raw xml, but only lightly processed.  On the 
Lightbend
site, I get a nicely formatted single topic.

Seem like it is a thing these days in web design to not put the, admittedly 
ugly,
RSS subscription button anywhere it can be easily found.  Guess I will have
to roll with changing times on that one.

I appreciate the time & effort people put into providing RSS.  It makes it 
_way_
easier to keep up with software changes & news, without having to go to 
a million web sites..

Thank you

Lee
 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] cluster sharding and auto-down

2017-05-26 Thread Justin du coeur 
Clarifying question: are you doing downing somewhere in this?

The naive auto-down feature shouldn't be used because it is *too* naive,
and leads to split-brain when you get transient network failures.  But you
do still have to down nodes *somehow* when they crash, so that the system
knows to readjust accordingly...

On Fri, May 26, 2017 at 4:52 AM, Dai Yinhua  wrote:

> I am using *akka cluster sharding *to build a long running calculation
> service, and I have set auto-down to false.
> And we have mechanism that *when one jvm get crashed it will be restarted
> immediately* by some external monitoring tools.
>
> I would like to confirm below behaviors when some jvm get crashed:
>
> Two cases:
>
> 1. *If the crashed jvm is seed node*, and the jvm is restarted
> immediately
> Is this the an expected result:
> 1.1) Other nodes see seed node as unreachable
> 1.2) One from other node become the new leader, but it can not perform
> any duty
> 1.3) Seed node restarted, and join it self
> *Seems in this case the cluster was partitioned to two?*
> How should I deal with this situation?
>
> 2. *If the crashed jvm is not seed node but a shard region node*, and the
> jvm is restarted immediately
> Before crash, the jvm contains some shards and have some entity actors
> running.
> Is this the an expected result:
> 2.1) The crashed node marked as unreachable
> 2.2) Leader can not performance any operation from now
> 2.3) A new instance is trying to join the cluster, and succeed
> 2.4) The jvm marked as up
> 2.5) Leader become available again
> Question:
> Will the *entities *that running in the crashed jvm be *moved to
> other available node automatically *after the crash happened?
> What if the *shard coordinator* happened to be running in the crashed
> jvm?
>
> Thank you.
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] cluster sharding and auto-down

2017-05-26 Thread Dai Yinhua 
I am using *akka cluster sharding *to build a long running calculation 
service, and I have set auto-down to false.
And we have mechanism that *when one jvm get crashed it will be restarted 
immediately* by some external monitoring tools.

I would like to confirm below behaviors when some jvm get crashed:

Two cases:

1. *If the crashed jvm is seed node*, and the jvm is restarted immediately 
Is this the an expected result:
1.1) Other nodes see seed node as unreachable
1.2) One from other node become the new leader, but it can not perform 
any duty
1.3) Seed node restarted, and join it self
*Seems in this case the cluster was partitioned to two?*
How should I deal with this situation?

2. *If the crashed jvm is not seed node but a shard region node*, and the 
jvm is restarted immediately 
Before crash, the jvm contains some shards and have some entity actors 
running.
Is this the an expected result:
2.1) The crashed node marked as unreachable
2.2) Leader can not performance any operation from now
2.3) A new instance is trying to join the cluster, and succeed
2.4) The jvm marked as up
2.5) Leader become available again
Question:
Will the *entities *that running in the crashed jvm be *moved to other 
available node automatically *after the crash happened?
What if the *shard coordinator* happened to be running in the crashed 
jvm?

Thank you.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.