subject:"\[akka\-user\] akka http client only excecute the `max\-connections` times \~"

Re: [akka-user] akka http client only excecute the `max-connections` times ~

2016-07-22 Thread zhenchuan


i found it's depends on the `client-idle-timeout = 120s` ;
after that time, the rest requests will hang up

[ERROR] [07/22/2016 14:34:09.641] 
[httpClient-akka.actor.default-dispatcher-10] 
[akka.actor.ActorSystemImpl(httpClient)] Outgoing request stream error
java.util.concurrent.TimeoutException: No elements passed in the last 2 
minutes.
at akka.stream.impl.Timers$IdleTimeoutBidi$$anon$7.onTimer(Timers.scala:160)
at 
akka.stream.stage.TimerGraphStageLogic.akka$stream$stage$TimerGraphStageLogic$$onInternalTimer(GraphStage.scala:1125)
at 
akka.stream.stage.TimerGraphStageLogic$$anonfun$akka$stream$stage$TimerGraphStageLogic$$getTimerAsyncCallback$1.apply(GraphStage.scala:1114)
at 
akka.stream.stage.TimerGraphStageLogic$$anonfun$akka$stream$stage$TimerGraphStageLogic$$getTimerAsyncCallback$1.apply(GraphStage.scala:1114)
at 
akka.stream.impl.fusing.GraphInterpreter.runAsyncInput(GraphInterpreter.scala:572)
at 
akka.stream.impl.fusing.GraphInterpreterShell.receive(ActorGraphInterpreter.scala:419)
at 
akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:603)
at 
akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:618)
at akka.actor.Actor$class.aroundReceive(Actor.scala:484)
at 
akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:529)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at 
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at 
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[ERROR] [07/22/2016 14:34:09.641] 
[httpClient-akka.actor.default-dispatcher-17] 
[akka.actor.ActorSystemImpl(httpClient)] Outgoing request stream error
java.util.concurrent.TimeoutException: No elements passed in the last 2 
minutes.
at akka.stream.impl.Timers$IdleTimeoutBidi$$anon$7.onTimer(Timers.scala:160)
at 
akka.stream.stage.TimerGraphStageLogic.akka$stream$stage$TimerGraphStageLogic$$onInternalTimer(GraphStage.scala:1125)
at 
akka.stream.stage.TimerGraphStageLogic$$anonfun$akka$stream$stage$TimerGraphStageLogic$$getTimerAsyncCallback$1.apply(GraphStage.scala:1114)
at 
akka.stream.stage.TimerGraphStageLogic$$anonfun$akka$stream$stage$TimerGraphStageLogic$$getTimerAsyncCallback$1.apply(GraphStage.scala:1114)
at 
akka.stream.impl.fusing.GraphInterpreter.runAsyncInput(GraphInterpreter.scala:572)
at 
akka.stream.impl.fusing.GraphInterpreterShell.receive(ActorGraphInterpreter.scala:419)
at 
akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:603)
at 
akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:618)
at akka.actor.Actor$class.aroundReceive(Actor.scala:484)
at 
akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:529)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at 
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at 
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)


在 2016年7月22日星期五 UTC+8上午10:38:36，zhenchuan写道：
>
> thanks for your reply ~
> i changed my code like this , it works well when i set the `max-connections 
> = 10`
>
> ```
>
> (1 to 500).foreach{idx=>
>   val futureResponse = 
> Source.single(httpRequest->1).via(poolFlow).runWith(Sink.head).map(_._1).flatMap(Future.fromTry)
>   futureResponse.andThen{
> case Success(response) =>
>   *response.entity.dataBytes.runWith(Sink.ignore)*
>   println(s"${atomicLong.incrementAndGet()} -> 
> ${Thread.currentThread().getName} ")
> case Failure(e) =>
>   e.printStackTrace()
>   println(s"error ${e.getMessage}")
>   }
>
> ```
>
> but when i enlarge the `max-connections = 100` ,  the errors occured 
> again, only 2/3 of the request was excuted ~
>
> the debug exception info as belows:
>
> ```
> [DEBUG] [07/22/2016 10:21:31.279] 
> [httpClient-akka.actor.default-dispatcher-15] 
> [akka://httpClient/user/SlotProcessor-53] Slot 53 disconnected after 
> java.util.concurrent.TimeoutException: No elements passed in the last 1 
> minute.
> [DEBUG] [07/22/2016

Re: [akka-user] akka http client only excecute the `max-connections` times ~

2016-07-21 Thread lzc6

thanks for your reply ~
i changed my code like this , it works well when i set the `max-connections 
= 10`

```

(1 to 500).foreach{idx=>
  val futureResponse = 
Source.single(httpRequest->1).via(poolFlow).runWith(Sink.head).map(_._1).flatMap(Future.fromTry)
  futureResponse.andThen{
case Success(response) =>
  *response.entity.dataBytes.runWith(Sink.ignore)*
  println(s"${atomicLong.incrementAndGet()} -> 
${Thread.currentThread().getName} ")
case Failure(e) =>
  e.printStackTrace()
  println(s"error ${e.getMessage}")
  }

```

but when i enlarge the `max-connections = 100` ,  the errors occured again, 
only 2/3 of the request was excuted ~

the debug exception info as belows:

```
[DEBUG] [07/22/2016 10:21:31.279] 
[httpClient-akka.actor.default-dispatcher-15] 
[akka://httpClient/user/SlotProcessor-53] Slot 53 disconnected after 
java.util.concurrent.TimeoutException: No elements passed in the last 1 
minute.
[DEBUG] [07/22/2016 10:21:31.279] 
[httpClient-akka.actor.default-dispatcher-7] 
[akka://httpClient/user/StreamSupervisor-0/flow-78-0-unknown-operation] 
Aborting tcp connection because of upstream failure: No elements passed in 
the last 1 minute.
akka.stream.impl.Timers$IdleTimeoutBidi$$anon$7.onTimer(Timers.scala:160)
akka.stream.stage.TimerGraphStageLogic.akka$stream$stage$TimerGraphStageLogic$$onInternalTimer(GraphStage.scala:1125)
akka.stream.stage.TimerGraphStageLogic$$anonfun$akka$stream$stage$TimerGraphStageLogic$$getTimerAsyncCallback$1.apply(GraphStage.scala:1114)
akka.stream.stage.TimerGraphStageLogic$$anonfun$akka$stream$stage$TimerGraphStageLogic$$getTimerAsyncCallback$1.apply(GraphStage.scala:1114)
akka.stream.impl.fusing.GraphInterpreter.runAsyncInput(GraphInterpreter.scala:572)
akka.stream.impl.fusing.GraphInterpreterShell.receive(ActorGraphInterpreter.scala:419)
akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:603)
akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:618)
akka.actor.Actor$class.aroundReceive(Actor.scala:484)
akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:529)
akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
akka.actor.ActorCell.invoke(ActorCell.scala:495)
akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
akka.dispatch.Mailbox.run(Mailbox.scala:224)
akka.dispatch.Mailbox.exec(Mailbox.scala:234)
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
```

Another info to be mentioned , as begining,everything works well as 
expected, 100 request was excuted at once . 
but after that ,  the output is more and more slower, the following diagram 
show that the threads was blocked ~
i want to know what cause this ~

thanks again ~

在 2016年7月21日星期四 UTC+8下午4:10:58，Konrad Malawski写道：
>
> You're not using the data you've requested, thus stalling the connections. 
> This is by design. Akka HTTP is a *streaming* HTTP Client.
>
> Please read the docs about this: 
> http://doc.akka.io/docs/akka/2.4/scala/http/implications-of-streaming-http-entity.html
>
>
> -- 
> Konrad `ktoso` Malawski
> Akka  @ Lightbend 
>
> On 21 July 2016 at 10:09:56, lz...@vip.qq.com  (
> lz...@vip.qq.com ) wrote:
>
>
> *When I use the pool-client to issue some requests, *
>
> *the program seems only execute only 10 times ,just the value of 
> `max-connections` ,*
>
> *the remaining 90 requests didn't execute at all;*
>
> *And after a while, *
>
> *the connection in the connection pool are closed ,no more connection alive ~*
>
>
> *i don't know what happended, thansk for your help ~*
>
>
>
> ```the scala code ~
>
> val poolFlow = Http().newHostConnectionPool[Int]("www.baidu.com")
>
> (1 to 100).foreach{idx=>
>   val futureResponse = 
> Source.single(httpRequest->1).via(poolFlow).runWith(Sink.head).map(_._1).flatMap(Future.fromTry)
>   futureResponse.andThen{
> case Success(_) =>
>   println(s"${atomicLong.incrementAndGet()} -> 
> ${Thread.currentThread().getName} ")
> case Failure(e) =>
>   e.printStackTrace()
>   println(s"error ${e.getMessage}")
>   }
>
> }
>
> ```
>
>
> the application.conf
>
> ```
>
> http {
>   host-connection-pool {
> max-connections = 10
> min-connections = 10
> max-retries = 0
> max-open-requests = 1048576
>   }
> }
>
> ```
>
>
> 
>
> the output :
>
> ```
>
> after pool ~
> after execute !
> 1 -> httpClient-akka.actor.default-dispatcher-2  
> 4 ->

Re: [akka-user] akka http client only excecute the `max-connections` times ~

2016-07-21 Thread Konrad Malawski

You're not using the data you've requested, thus stalling the connections.
This is by design. Akka HTTP is a *streaming* HTTP Client.

Please read the docs about this:
http://doc.akka.io/docs/akka/2.4/scala/http/implications-of-streaming-http-entity.html


-- 
Konrad `ktoso` Malawski
Akka  @ Lightbend 

On 21 July 2016 at 10:09:56, l...@vip.qq.com (l...@vip.qq.com) wrote:


*When I use the pool-client to issue some requests, *

*the program seems only execute only 10 times ,just the value of
`max-connections` ,*

*the remaining 90 requests didn't execute at all;*

*And after a while, *

*the connection in the connection pool are closed ,no more connection alive ~*


*i don't know what happended, thansk for your help ~*



```the scala code ~

val poolFlow = Http().newHostConnectionPool[Int]("www.baidu.com")

(1 to 100).foreach{idx=>
  val futureResponse =
Source.single(httpRequest->1).via(poolFlow).runWith(Sink.head).map(_._1).flatMap(Future.fromTry)
  futureResponse.andThen{
case Success(_) =>
  println(s"${atomicLong.incrementAndGet()} ->
${Thread.currentThread().getName} ")
case Failure(e) =>
  e.printStackTrace()
  println(s"error ${e.getMessage}")
  }

}

```


the application.conf

```

http {
  host-connection-pool {
max-connections = 10
min-connections = 10
max-retries = 0
max-open-requests = 1048576
  }
}

```




the output :

```

after pool ~
after execute !
1 -> httpClient-akka.actor.default-dispatcher-2
4 -> httpClient-akka.actor.default-dispatcher-19
5 -> httpClient-akka.actor.default-dispatcher-12
2 -> httpClient-akka.actor.default-dispatcher-13
6 -> httpClient-akka.actor.default-dispatcher-10
3 -> httpClient-akka.actor.default-dispatcher-8
7 -> httpClient-akka.actor.default-dispatcher-13
8 -> httpClient-akka.actor.default-dispatcher-4
9 -> httpClient-akka.actor.default-dispatcher-4
10 -> httpClient-akka.actor.default-dispatcher-15

```


--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] akka http client only excecute the `max-connections` times ~

Re: [akka-user] akka http client only excecute the `max-connections` times ~

Re: [akka-user] akka http client only excecute the `max-connections` times ~

3 matches

Site Navigation

Mail list logo

Footer information