Re: [akka-user] Apply different functions on SubFlows

2017-04-26 Thread Vishal John 

Thanks a lot!!


On Tuesday, 25 April 2017 18:39:58 UTC+7, Akka Team wrote:
>
> Partition allows you to look at a message and decide which of its outlets 
> to pass the element to, this way you can design the logic of each sub 
> stream independently. Something like this:
>
> val flow: Flow[Int, Int, NotUsed] = Flow.fromGraph(GraphDSL.create() { 
> implicit builder =>
> import GraphDSL.Implicits._
>
> val partition = builder.add(Partition[Int](2, _ % 2))
> val merge = builder.add(Merge[Int](2))
>
> val processEven = builder.add(Flow.fromFunction((n: Int)  => n + 100))
> val processOdd = builder.add(Flow.fromFunction((n: Int) => n - 1))
>
> partition.out(0) ~> processEven ~> merge.in(0)
> partition.out(1) ~> processOdd ~> merge.in(1)
>
> FlowShape(partition.in, merge.out)
>   })
>
>
> -- 
> Johan
> Akka Team
>
> On Tue, Mar 21, 2017 at 1:18 PM, Vishal John  > wrote:
>
>>
>> Hello all,
>>
>>
>> I have defined my application flow like this,
>>
>> *val flow = Flow[StringConsumerRecord].*
>> *  map(tup => new Ticket(tup.value.toLong)).*
>> *  mapConcat(applyBusinessLogic(_)).*
>> *  groupBy(4, _._1).*
>> *  groupedWithin(100, 5.second).*
>> *  mergeSubstreams.*
>> *  throttle(1, 1.second, 1, ThrottleMode.shaping)*
>>
>>
>> Basically I don't want to apply `*groupedWithin(100, 5.second)*` for all 
>> the sub flows.
>>
>> Based on the type of the grouped data, for one sub stream I want to do 
>> *groupedWithin(10, 
>> 10.second) *and for another *groupedWithin(1000, 1.second) etc... *and 
>> finally merge all. Basically I am trying to do some sort to pattern 
>> matching on the sub-flows and take different actions.
>>
>> Can someone give any pointers on how to achieve this.
>>
>>
>> -thanks
>> Vishal
>>
>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Is Akka Multi Node Testing available in Java

2017-04-26 Thread Dai Yinhua 
OK, thanks.

On Tuesday, 25 April 2017 16:59:10 UTC+8, Akka Team wrote:
>
> Hi Dai,
>
> I'm afraid it is currently only possible to run the multi-node tests with 
> Scala + SBT
>
> -- 
> Johan
> Akka Team
>
> On Wed, Mar 22, 2017 at 3:11 AM, Dai Yinhua  > wrote:
>
>> Anybody knows how to run akka multi node testing in java?
>> Is there any example code?
>>
>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka HTTP/JSON: Rejecting requests with unknown fields

2017-04-26 Thread Henrik Nordvik 
On Wednesday, March 29, 2017 at 7:13:28 AM UTC-4, Konrad Malawski wrote:

> In principle sure, though not on anyones roadmap nowadays.
> You could contribute such a thing or find a json parser whihc has such 
> failure mode - I'm unaware of any parser which has such failure mode though.
>
 
Jackson has a failure mode to fail on unknown fields, and actually it's the 
default.
You need to explicitly disable it if you don't want it. Either globally
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, 
false);
or per class:
@JsonIgnoreProperties(ignoreUnknown = true)

It looks like there's some support for using jackson in 
akka-http: http://doc.akka.io/docs/akka/2.4.6/java/http/common/json-support.html
so maybe that's one way to go.


On Wednesday, March 29, 2017 at 7:54:57 AM UTC-4, Alan Burlison wrote:
>
> On 29/03/2017 12:13, Konrad Malawski wrote: 
>
> I can't think of any way of doing it that doesn't require reflection to 
> enumerate the fields in the case class that the JSON is being 
> unmarshalled in to. And although I've used Spray/JSON quite a bit I have 
> no idea how the case class/tuple marshalling/unmarshalling stuff works. 



Would it be possible to override the JsonReader and do something like:

def read(value: JsValue) = {
  val names = Set(fieldName1, fieldName2, fieldName3)
  value.asJsObject.fields.keys.filterNot(names.contains).foreach { field =>
 throw new UnknownFieldException(field)
  }
  val p1V = fromField[P1](value, fieldName1)
  val p2V = fromField[P2](value, fieldName2)
  val p3V = fromField[P3](value, fieldName3)
  construct(p1V, p2V, p3V)
}

It's using reflection but only once per case class.
If you need it for a lot of case classes you can probably generate some 
code by using the template:
https://github.com/spray/spray-json/blob/master/src/main/boilerplate/spray/json/ProductFormatsInstances.scala.template

-
Henrik

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka HTTP client - Advice for executor context

2017-04-26 Thread Albert 
Great explanation Arne - thank you!

I think I have all pieces except one:

I call web services in parallel, creating Futures. Futures are then 
converted to one Future (like Future.sequence). 
Do I reach backpressure in case of "too many requests" when router respond 
with Source (OverflowStrategy.backpressure) with flow with business logic 
producing future?
In other words if my Future will be assembled as Flow and router respond 
with Source?

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka HTTP client - Advice for executor context

2017-04-26 Thread Albert Gorski 
Great explanation Arne - thank you!

I think I have all pieces except one:

I call web services in parallel, creating Futures. Futures are then
converted to one Future (like *Future.sequence*).
When, in router, I create response with Source and
set OverflowStrategy.backpressure


2017-04-26 10:15 GMT+02:00 Arno Haase :

>
> > /_Question_:/
> > - does it make really sense to create dispatcher pro web service client
> > or one for all? Do you've any experience with it?
> > - is it better to use Dispatcher with fork-join or fixed-size one?
>
> Using the default pool fully utilizes the CPU, and using a separate
> dispatcher will not provide "more" throughput. As Konrad pointed out,
> this relies on your code *never* blocking threads - if you really want
> to do that, you need a separate dispatcher.
>
> But since you ask for "better", avoiding blocking code (or isolating it
> in separate actors with a separate dispatcher, if they cannot be
> avoided) is typically the "better" approach in Akka.
>
> There are basically two situations where using separate dispatchers
> provides benefits: Encapsulating blocking operations, and managing /
> limiting load. For HTTP clients, usually neither fits: Blocking code is
> typically better off in separate actors (if it can not be avoided), and
> load limitation is usually easier to manage using connection configuration.
>
> > *2. Connection pool(s)*
> > Currently I use one /cachedHostConnectionPool/ connection pool for all
> > materializations for given host/port. I use also /Source.queue /to
> > handle more requests, if necessary.
> > Size of pool is of course very specific to the running environment - I
> > set max to 256.
> >
> > /_Questions_:/
> > - how "heavy" is queuing? Are "just" request objects kept in queue?
> > - theoretically if connection pool is bigger then dispatcher threads
> > should be also bigger -> to handle responses fast enough. In other
> > words, make it sense to limit connection pool if dispatcher has less
> > threads? Am I right / make it sense?
>
> This is about back pressure: Queueing requests can buffer load peaks,
> but if the incoming load is more than the server can handle, the queue
> will overflow sooner or later, resulting in rejected requests - which is
> a good thing, the alternative being some kind of crash of the server.
>
> If your application overloads the called server, there is nothing you
> can do about it. If processing the responses overloads your application,
> the approach depends on which resource is the limiting factor.
>
> You should go for implementing some kind of explicit back pressure all
> the way back to the code triggering the web service calls - trying to
> solve that using thread pool sizes is tricky and will usually not solve
> the problem in a robust way.
>
> - Arno
>
> --
> >>  Read the docs: http://akka.io/docs/
> >>  Check the FAQ: http://doc.akka.io/docs/akka/c
> urrent/additional/faq.html
> >>  Search the archives: https://groups.google.com/grou
> p/akka-user
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Akka User List" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/to
> pic/akka-user/6XMGUhB8caE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka HTTP/JSON: Rejecting requests with unknown fields

2017-04-26 Thread Konrad Malawski 
That Jackson support is for JavaDSL though - do not mix the DSLs.
It is absolutely possible and supported to use Jackson in ScalaDSL though:
It's implemented by Heiko's library here:
https://github.com/hseeberger/akka-http-json

Hope this helps -

-- 
Konrad `ktoso` Malawski
Akka  @ Lightbend 

On 27 April 2017 at 01:59:42, Henrik Nordvik (henri...@gmail.com) wrote:

On Wednesday, March 29, 2017 at 7:13:28 AM UTC-4, Konrad Malawski wrote:

> In principle sure, though not on anyones roadmap nowadays.
> You could contribute such a thing or find a json parser whihc has such
> failure mode - I'm unaware of any parser which has such failure mode though.
>

Jackson has a failure mode to fail on unknown fields, and actually it's the
default.
You need to explicitly disable it if you don't want it. Either globally
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES,
false);
or per class:
@JsonIgnoreProperties(ignoreUnknown = true)

It looks like there's some support for using jackson in akka-http:
http://doc.akka.io/docs/akka/2.4.6/java/http/common/json-support.html
so maybe that's one way to go.


On Wednesday, March 29, 2017 at 7:54:57 AM UTC-4, Alan Burlison wrote:
>
> On 29/03/2017 12:13, Konrad Malawski wrote:
>
> I can't think of any way of doing it that doesn't require reflection to
> enumerate the fields in the case class that the JSON is being
> unmarshalled in to. And although I've used Spray/JSON quite a bit I have
> no idea how the case class/tuple marshalling/unmarshalling stuff works.



Would it be possible to override the JsonReader and do something like:

def read(value: JsValue) = {
  val names = Set(fieldName1, fieldName2, fieldName3)
  value.asJsObject.fields.keys.filterNot(names.contains).foreach { field =>
 throw new UnknownFieldException(field)
  }
  val p1V = fromField[P1](value, fieldName1)
  val p2V = fromField[P2](value, fieldName2)
  val p3V = fromField[P3](value, fieldName3)
  construct(p1V, p2V, p3V)
}

It's using reflection but only once per case class.
If you need it for a lot of case classes you can probably generate some
code by using the template:
https://github.com/spray/spray-json/blob/master/src/main/boilerplate/spray/json/ProductFormatsInstances.scala.template

-
Henrik

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Akka http compilation error due to _marshaller

2017-04-26 Thread nikhlesh 
Hi,

I am using akka http for graphql sangria subscription. 
It is working fine in example application but while integrating graphql 
subscription code with this 
 simple query 
mutation code it is giving compilation error.
Below is the code i'm trying to execute: 

complete(
  executor.prepare(queryAst, ctx, (), operation, variables)
.map { preparedQuery ⇒
  ToResponseMarshallable(preparedQuery.execute()
.map(result ⇒ ServerSentEvent(result.compactPrint))
.recover { case NonFatal(error) ⇒
  logger.error(error, "Unexpected error during event stream 
processing.")
  ServerSentEvent(error.getMessage)
})
}
   
It is showing error inline: 


ToResponseMarshallable(preparedQuery.execute()


and i'm getting: 


could not find implicit value for parameter _marshaller: 
akka.http.scaladsl.marshalling.ToResponseMarshaller[akka.stream.scaladsl.Source[de.heikoseeberger.akkasse.ServerSentEvent,akka.NotUsed]]


Can some one help me to resolve this error


Thanks


 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka HTTP client - Advice for executor context

2017-04-26 Thread Arno Haase 

> /_Question_:/ 
> - does it make really sense to create dispatcher pro web service client
> or one for all? Do you've any experience with it? 
> - is it better to use Dispatcher with fork-join or fixed-size one?

Using the default pool fully utilizes the CPU, and using a separate
dispatcher will not provide "more" throughput. As Konrad pointed out,
this relies on your code *never* blocking threads - if you really want
to do that, you need a separate dispatcher.

But since you ask for "better", avoiding blocking code (or isolating it
in separate actors with a separate dispatcher, if they cannot be
avoided) is typically the "better" approach in Akka.

There are basically two situations where using separate dispatchers
provides benefits: Encapsulating blocking operations, and managing /
limiting load. For HTTP clients, usually neither fits: Blocking code is
typically better off in separate actors (if it can not be avoided), and
load limitation is usually easier to manage using connection configuration.

> *2. Connection pool(s)*
> Currently I use one /cachedHostConnectionPool/ connection pool for all
> materializations for given host/port. I use also /Source.queue /to
> handle more requests, if necessary.
> Size of pool is of course very specific to the running environment - I
> set max to 256.
> 
> /_Questions_:/ 
> - how "heavy" is queuing? Are "just" request objects kept in queue?
> - theoretically if connection pool is bigger then dispatcher threads
> should be also bigger -> to handle responses fast enough. In other
> words, make it sense to limit connection pool if dispatcher has less
> threads? Am I right / make it sense?

This is about back pressure: Queueing requests can buffer load peaks,
but if the incoming load is more than the server can handle, the queue
will overflow sooner or later, resulting in rejected requests - which is
a good thing, the alternative being some kind of crash of the server.

If your application overloads the called server, there is nothing you
can do about it. If processing the responses overloads your application,
the approach depends on which resource is the limiting factor.

You should go for implementing some kind of explicit back pressure all
the way back to the code triggering the web service calls - trying to
solve that using thread pool sizes is tricky and will usually not solve
the problem in a robust way.

- Arno

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Multi Host Master with Remote Worker

2017-04-26 Thread Akka Team 
These two sample projects shows an example of this:
https://github.com/typesafehub/activator-akka-distributed-workers
https://github.com/typesafehub/activator-akka-distributed-workers-java

--
Johan
Akka Team

On Fri, Mar 3, 2017 at 9:52 PM,  wrote:

> How do i create Master on Node 1 and deploy worker on different nodes ,
> Any help for configuration or example will be a great help
>
> Thanks,
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/c
> urrent/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka HTTP client - Advice for executor context

2017-04-26 Thread Albert 
Thank you Konrad for really fast response!

I read the docs again and if I correct understood my configuration 
can/should be like this one:
*1. Dispatcher(s)*
One dispatcher for all web service clients with fixed size. Size of the 
pool correspond to CPU amount (16 for 16 cores)
*or*
dispatcher pro web service client with fixed size. Size of the pool = 
number of cores / dispatcher amount

*Question:* 
- does it make really sense to create dispatcher pro web service client or 
one for all? Do you've any experience with it? 
- is it better to use Dispatcher with fork-join or fixed-size one?

*2. Connection pool(s)*
Currently I use one *cachedHostConnectionPool* connection pool for all 
materializations for given host/port. I use also *Source.queue *to handle 
more requests, if necessary.
Size of pool is of course very specific to the running environment - I set 
max to 256.

*Questions:* 
- how "heavy" is queuing? Are "just" request objects kept in queue?
- theoretically if connection pool is bigger then dispatcher threads should 
be also bigger -> to handle responses fast enough. In other words, make it 
sense to limit connection pool if dispatcher has less threads? Am I right / 
make it sense?

Cheers,
Albert

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.