[akka-user] Re: Completeing a stream will terminate http connection pool?

2017-02-27 Thread Donovan Levinson 
Will the pool shut itself down even if "min-connections" is set greater 
than zero?

On Sunday, February 26, 2017 at 9:17:23 AM UTC-8, Johannes Rudolph wrote:
>
> Hi Johnson,
>
> if you use a `cachedHostConnectionPool`, the pool itself is a shared 
> resource that is managed by the infrastructure. If you don't do anything 
> else, the pool will shutdown itself after 
> the akka.http.host-connection-pool.idle-timeout has passed. If you want 
> manual control, the `HostConnectionPool` object materialized by 
> `cachedHostConnectionPool` has a `shutdown` method you can call to shutdown 
> this pool at any time.
>
> Use
>
> val (queue, pool) = 
> Source.queue(...).viaMat(pool)(Keep.right).toMat(...).run
>
> to get access to the pool.
>
> HTH
> Johannes
>
> On Sunday, February 26, 2017 at 5:08:31 PM UTC+1, Johnson Liu wrote:
>>
>> Hello,
>>
>> I am using akka http connection pool like this:
>>
>> val pool = Http().cachedHostConnectionPool[Promise[HttpResponse]](host, 
>> port)
>>   
>> val source = Source.queue[(HttpRequest, 
>> Promise[HttpResponse])](connectionOverflowBufferSize, 
>> OverflowStrategy.dropNew)
>>   .via(pool)
>>   .toMat(Sink.foreach {
>> case ((Success(resp), p)) => p.success(resp)
>> case ((Failure(e), p)) => p.failure(e)
>>   })(Keep.left)
>>   .run
>>
>> When I am done with the http flow, if I do source.complete(), does it 
>> mean that the connection pool is automatically shut down or I have to do it 
>> manually? I couldn't find it in the documentation. Thanks.
>>
>> Johnson
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] How to multi-thread publish to Source.queue with backpressure?

2017-02-27 Thread Marcin Milewski 


Hi

I have a case where I want to use an akka stream with backpressure and 
insert messages to the stream from multiple threads.

Here is the code to explain better

import akka.actor.ActorSystemimport akka.stream.scaladsl._import 
akka.stream.{ActorMaterializer, OverflowStrategy, QueueOfferResult}
import scala.concurrent.duration._import scala.concurrent.{Await, Future}
object PublishToSourceQueueFromManyThreads {

  def main(args: Array[String]): Unit = {
implicit val system = ActorSystem("QuickStart")
implicit val materializer = ActorMaterializer()

// build the processing pipeline with queue as entry point
val queue = Source.queue[Int](bufferSize = 2, OverflowStrategy.backpressure)
  .groupedWithin(2, 2.seconds)
  .mapAsyncUnordered(2) { elem =>
Future {
  println(s"${Thread.currentThread().getName} simulating delay, $elem")
  Thread.sleep(1000L)
  elem
}(scala.concurrent.ExecutionContext.global)
  }.to(Sink.ignore)
  .run

// here we start few threads that push events to the queue in parallel
new Thread(() => {
  while (true) {
val offerResult: Future[QueueOfferResult] = queue.offer(1)
Await.ready(offerResult, 10.seconds)
println(s"${Thread.currentThread().getName} Emitted 1 $offerResult")
  }
}).start()

new Thread(() => {
  while (true) {
val offerResult: Future[QueueOfferResult] = queue.offer(2)
Await.ready(offerResult, 10.seconds)
println(s"${Thread.currentThread().getName} Emitted 2 $offerResult")
  }
}).start()

println("done")

  }
}

What I get when I execute the code above is

done
Thread-1 Emitted 2 Future(Success(Enqueued))
Thread-0 Emitted 1 Future(Success(Enqueued))
scala-execution-context-global-18 simulating delay, Vector(1, 2)
Thread-1 Emitted 2 Future(Success(Enqueued))
Thread-0 Emitted 1 Future(Success(Enqueued))
Thread-1 Emitted 2 Future(Success(Enqueued))
scala-execution-context-global-19 simulating delay, Vector(2, 1)
Thread-0 Emitted 1 Future(Success(Enqueued))
Thread-1 Emitted 2 Future(Success(Enqueued))
Thread-0 Emitted 1 Future(Success(Enqueued))
Thread-0 Emitted 1 Future(Failure(java.lang.IllegalStateException: You have to 
wait for previous offer to be resolved to send another request))
Thread-0 Emitted 1 Future(Failure(java.lang.IllegalStateException: You have to 
wait for previous offer to be resolved to send another request))
Thread-0 Emitted 1 Future(Failure(java.lang.IllegalStateException: You have to 
wait for previous offer to be resolved to send another request))
Thread-0 Emitted 1 Future(Failure(java.lang.IllegalStateException: You have to 
wait for previous offer to be resolved to send another request))
Thread-0 Emitted 1 Future(Failure(java.lang.IllegalStateException: You have to 
wait for previous offer to be resolved to send another request))

I checked with the source code of Queue that this is the expected behaviour 
https://github.com/akka/akka/blob/master/akka-stream/src/main/scala/akka/stream/scaladsl/Queue.scala#L21-L22

   * - fails when stream is completed or you cannot call offer in this moment 
because of implementation rules
   * (like for backpressure mode and full buffer you need to wait for last 
offer call Future completion)

What I did for now is I wrapped the SourceQueue, used synchronized block 
and I'm Awaiting result before returning from offerBlocking.

import akka.actor.ActorSystemimport akka.stream.scaladsl._import 
akka.stream.{ActorMaterializer, OverflowStrategy, QueueOfferResult}
import scala.concurrent.duration._import scala.concurrent.{Await, Future, 
TimeoutException}
class SyncQueue[T](q: SourceQueue[T]) {
  /*** @throws TimeoutException if it couldn't get the value within 
`maxWait` time*/
  def offerBlocking(elem: T, maxWait: Duration = 10.seconds): 
Future[QueueOfferResult] = 
synchronized {
  val result = q.offer(elem)
  Await.ready(result, maxWait)
  result
}
}
object PublishToSourceQueueFromManyThreads {

  def main(args: Array[String]): Unit = {
implicit val system = ActorSystem("QuickStart")
implicit val materializer = ActorMaterializer()

// build the queue processing pipeline
val queue = Source.queue[Int](bufferSize = 2, OverflowStrategy.backpressure)
  .groupedWithin(2, 2.seconds)
  .mapAsyncUnordered(2) { elem =>
Future {
  println(s"${Thread.currentThread().getName} simulating delay, $elem")
  Thread.sleep(1000L)
  elem
}(scala.concurrent.ExecutionContext.global)
  }.to(Sink.ignore)
  .run

val queue2 = new SyncQueue(queue)

// here we start few threads that would push events to the queue
new Thread(() => {
  while (true) {
val offerResult: Future[QueueOfferResult] = queue2.offerBlocking(1)
// Await.ready(offerResult, 10.seconds)
println(s"${Thread.currentThread().getName} Emitted 1 $offerResult")
  }
}).start()

[akka-user] CircuitBreaker and ExponentialBackOff in Streams

2017-02-27 Thread Richard Rodseth 
What's the current thinking on this? This ticket is open:

https://github.com/akka/akka/issues/15307

I have an infinite stream communicating with an external service. If the
external service goes down or some threshhold of errors is reached, it may
be appropriate to stop the actor that runs the stream and restart it after
some period.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Sharding and unreachable node

2017-02-27 Thread Justin du coeur 
Well, scaling down is straightforward: in that case, you *are* taking down
the node (or at least, the ActorSystem), so you should down it.  As soon as
you down it, the shards should migrate properly to other nodes.  That's
normal stuff.  Keep in mind that "unreachable but not down" should be a
*temporary* situation -- your system should be built to properly down the
unreachable node before long.  (With the appropriate precautions against
split-brain.)

Rolling out a new version of your application without stopping *is*
tricky.  At the moment, I'm accepting very brief downtime (a few seconds)
to shut down the old cluster and boot the new one.  Doing a proper rolling
release seems plausible, but you have to think carefully about wire
compatibility between the nodes.  I don't know offhand of a simple way to
do a rolling release if the wire protocols aren't compatible -- at that
point, I think you have to have application complicity in the process, and
it probably has to be handled with a lot of care...

On Mon, Feb 27, 2017 at 2:24 PM, Andrey Ilinykh  wrote:

>
>
> On Monday, February 27, 2017 at 10:11:59 AM UTC-8, Justin du coeur wrote:
>>
>> There's no silver-bullet option for "fault tolerance" in situations like
>> this.  You absolutely do *not* want to start up alternate versions of the
>> sharded entities in this situation -- that's classic split-brain, and is
>> the surest route to data corruption.
>>
> yes, this is a dangerous route. But what is the a way to manage cluster? I
> mean roll out of a new version or scaling down. In such situations one has
> to stop akka (or whole node).
>
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Sharding and unreachable node

2017-02-27 Thread Andrey Ilinykh 


On Monday, February 27, 2017 at 10:11:59 AM UTC-8, Justin du coeur wrote:
>
> There's no silver-bullet option for "fault tolerance" in situations like 
> this.  You absolutely do *not* want to start up alternate versions of the 
> sharded entities in this situation -- that's classic split-brain, and is 
> the surest route to data corruption.  
>
yes, this is a dangerous route. But what is the a way to manage cluster? I 
mean roll out of a new version or scaling down. In such situations one has 
to stop akka (or whole node).


-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Sharding and unreachable node

2017-02-27 Thread Justin du coeur 
There's no silver-bullet option for "fault tolerance" in situations like
this.  You absolutely do *not* want to start up alternate versions of the
sharded entities in this situation -- that's classic split-brain, and is
the surest route to data corruption.  You could create some sort of
reliable-ish messaging to the entities in this situation, so that the
messages will get retried after the entities are back, but as with any
reliable-messaging solution, getting it right is a considerable amount of
effort, and involves somewhere between "some" and "tons" of overhead.
 (Depending on how tolerant you're trying to be.)

This is pretty classic Akka: messaging *is* unreliable, due to situations
like this and others like it.  Success with Akka requires internalizing
that.  Any fault tolerance is typically up to the higher-level code -- if
it needs reliability, it is up to the higher-level protocol (particularly
at the sending end) to implement that.

On Mon, Feb 27, 2017 at 12:53 PM, Andrey Ilinykh  wrote:

> Hello everybody!
> I do some research about akka sharding. One moment is not clear for me
> right now.
> If some node becomes unreachable (crashed but not down, for example) all
> shards which belong to this node become unreachable. Which means all
> messages to these actors are dropped.
> Is it correct?
> If so, is it possible to implement some fault tolerance mechanism here?
>
> Thank you,
>   Andrey
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Sharding and unreachable node

2017-02-27 Thread Andrey Ilinykh 
Hello everybody!
I do some research about akka sharding. One moment is not clear for me 
right now.
If some node becomes unreachable (crashed but not down, for example) all 
shards which belong to this node become unreachable. Which means all 
messages to these actors are dropped.
Is it correct?
If so, is it possible to implement some fault tolerance mechanism here? 

Thank you,
  Andrey

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Detail: akka.stream.StreamTcpException: Connection failed.

2017-02-27 Thread Chaitanya Mahipath 
Yes. Status code, headers & the content using entity.toStrict.

Thanks

On Monday, February 27, 2017 at 5:06:53 PM UTC+5:30, √ wrote:
>
> Are you consuming your response bodies?
>
> -- 
> Cheers,
> √
>
> On Feb 27, 2017 12:27 PM, "Chaitanya Mahipath"  > wrote:
>
>> Hi Johannes,
>>
>> I tried with Http.SingleRequest as suggested but could see the following 
>> exception
>>
>> akka.stream.BufferOverflowException: Exceeded configured 
>> max-open-requests value of [32]
>>
>> This time the test has lasted only for 40 odd minutes(as compared to 
>> previous test result which lasted for over 2hrs 30mins.). Could you please 
>> let me know what configuration changes should be made?
>>
>> On Monday, February 27, 2017 at 1:59:10 PM UTC+5:30, Johannes Rudolph 
>> wrote:
>>>
>>> Hello,
>>>
>>> the error is thrown here: 
>>> https://github.com/akka/akka/blob/master/akka-stream/src/main/scala/akka/stream/impl/io/TcpStages.scala#L303.
>>>  
>>> I agree it's hard to see why that error would happen. One thing that could 
>>> have happened is that a connection attempt was very slow and so the 
>>> idle-timeout kicked in and killed the TCP stage before a connection was 
>>> established. Either there was a real connection problem, or the 
>>> infrastructure didn't make progress in due time e.g. because of a thread 
>>> starvation issue.
>>>
>>> Btw. using `Source.single().via(connectionFlow).runWith(Sink.head)` is 
>>> discouraged. Please use `Http.singleRequest` instead. See also the note at 
>>> http://doc.akka.io/docs/akka-http/current/scala/http/client-side/host-level.html#host-level-api
>>> .
>>>
>>> HTH
>>> Johannes
>>>
>>>
>>>
>>> On Monday, February 27, 2017 at 8:25:37 AM UTC+1, Chaitanya Mahipath 
>>> wrote:

 Hi,

 I am using the akka http scaladsl library to do a HTTP calls against a 
 server. I am running a test where I have 15 threads running concurrently 
 each making around 250 GET calls(15*250). Trying to run this test for 
 three 
 hours but I see that after two and half hours of execution I could see
 Detail: akka.stream.StreamTcpException: Connection failed. exception 
 being thrown.

 Initially I thought the servers isn't accepting the incoming requests. 
 But, ran the same test directly against the server and could see no issues.
 Code snippet to connect to the server -

 connectionFlow= 
 Http(context.system).outgoingConnectionTls(req.uri.authority.host.address(),
  
 port)
 Source.single(req.withUri(req.uri.toRelative)).via(connectionFlow).runWith(Sink.head)
  
 - Here, req is the HttpRequest.

 Could someone please help me out why is that StreamTcpException 
 exception is thrown?

>>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Detail: akka.stream.StreamTcpException: Connection failed.

2017-02-27 Thread Viktor Klang 
Are you consuming your response bodies?

-- 
Cheers,
√

On Feb 27, 2017 12:27 PM, "Chaitanya Mahipath" 
wrote:

> Hi Johannes,
>
> I tried with Http.SingleRequest as suggested but could see the following
> exception
>
> akka.stream.BufferOverflowException: Exceeded configured
> max-open-requests value of [32]
>
> This time the test has lasted only for 40 odd minutes(as compared to
> previous test result which lasted for over 2hrs 30mins.). Could you please
> let me know what configuration changes should be made?
>
> On Monday, February 27, 2017 at 1:59:10 PM UTC+5:30, Johannes Rudolph
> wrote:
>>
>> Hello,
>>
>> the error is thrown here: https://github.com/akka/
>> akka/blob/master/akka-stream/src/main/scala/akka/stream/impl
>> /io/TcpStages.scala#L303. I agree it's hard to see why that error would
>> happen. One thing that could have happened is that a connection attempt was
>> very slow and so the idle-timeout kicked in and killed the TCP stage before
>> a connection was established. Either there was a real connection problem,
>> or the infrastructure didn't make progress in due time e.g. because of a
>> thread starvation issue.
>>
>> Btw. using `Source.single().via(connectionFlow).runWith(Sink.head)` is
>> discouraged. Please use `Http.singleRequest` instead. See also the note at
>> http://doc.akka.io/docs/akka-http/current/scala/http/clie
>> nt-side/host-level.html#host-level-api.
>>
>> HTH
>> Johannes
>>
>>
>>
>> On Monday, February 27, 2017 at 8:25:37 AM UTC+1, Chaitanya Mahipath
>> wrote:
>>>
>>> Hi,
>>>
>>> I am using the akka http scaladsl library to do a HTTP calls against a
>>> server. I am running a test where I have 15 threads running concurrently
>>> each making around 250 GET calls(15*250). Trying to run this test for three
>>> hours but I see that after two and half hours of execution I could see
>>> Detail: akka.stream.StreamTcpException: Connection failed. exception
>>> being thrown.
>>>
>>> Initially I thought the servers isn't accepting the incoming requests.
>>> But, ran the same test directly against the server and could see no issues.
>>> Code snippet to connect to the server -
>>>
>>> connectionFlow= Http(context.system).outgoingC
>>> onnectionTls(req.uri.authority.host.address(), port)
>>> Source.single(req.withUri(req.uri.toRelative)).via(connectionFlow).runWith(Sink.head)
>>> - Here, req is the HttpRequest.
>>>
>>> Could someone please help me out why is that StreamTcpException
>>> exception is thrown?
>>>
>> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Detail: akka.stream.StreamTcpException: Connection failed.

2017-02-27 Thread Chaitanya Mahipath 
Hi Johannes,

I tried with Http.SingleRequest as suggested but could see the following 
exception

akka.stream.BufferOverflowException: Exceeded configured max-open-requests 
value of [32]

This time the test has lasted only for 40 odd minutes(as compared to 
previous test result which lasted for over 2hrs 30mins.). Could you please 
let me know what configuration changes should be made?

On Monday, February 27, 2017 at 1:59:10 PM UTC+5:30, Johannes Rudolph wrote:
>
> Hello,
>
> the error is thrown here: 
> https://github.com/akka/akka/blob/master/akka-stream/src/main/scala/akka/stream/impl/io/TcpStages.scala#L303.
>  
> I agree it's hard to see why that error would happen. One thing that could 
> have happened is that a connection attempt was very slow and so the 
> idle-timeout kicked in and killed the TCP stage before a connection was 
> established. Either there was a real connection problem, or the 
> infrastructure didn't make progress in due time e.g. because of a thread 
> starvation issue.
>
> Btw. using `Source.single().via(connectionFlow).runWith(Sink.head)` is 
> discouraged. Please use `Http.singleRequest` instead. See also the note at 
> http://doc.akka.io/docs/akka-http/current/scala/http/client-side/host-level.html#host-level-api
> .
>
> HTH
> Johannes
>
>
>
> On Monday, February 27, 2017 at 8:25:37 AM UTC+1, Chaitanya Mahipath wrote:
>>
>> Hi,
>>
>> I am using the akka http scaladsl library to do a HTTP calls against a 
>> server. I am running a test where I have 15 threads running concurrently 
>> each making around 250 GET calls(15*250). Trying to run this test for three 
>> hours but I see that after two and half hours of execution I could see
>> Detail: akka.stream.StreamTcpException: Connection failed. exception 
>> being thrown.
>>
>> Initially I thought the servers isn't accepting the incoming requests. 
>> But, ran the same test directly against the server and could see no issues.
>> Code snippet to connect to the server -
>>
>> connectionFlow= 
>> Http(context.system).outgoingConnectionTls(req.uri.authority.host.address(), 
>> port)
>> Source.single(req.withUri(req.uri.toRelative)).via(connectionFlow).runWith(Sink.head)
>>  
>> - Here, req is the HttpRequest.
>>
>> Could someone please help me out why is that StreamTcpException exception 
>> is thrown?
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: illegal-rawheaders

2017-02-27 Thread 'Johannes Rudolph' via Akka User List 
Hi Dimitry,

that message is a different one. It means that your code is issuing 
requests with an explicit header `RawHeader("User-Agent", ...)` somewhere.

Johannes

On Thursday, February 23, 2017 at 8:10:03 PM UTC+1, 
dmitriy...@alisagaming.com wrote:
>
> Hi everyone.
>
> I've been using akka with spray and, later, akka-http.
>
> Right now I'm confused with configuration and *illegalHeaderWarnings* in 
> particular.
>
> First, I added this to my *application.conf*:
>
>
> akka.http {
>
>  host-connection-pool {
>  parsing.illegal-header-warnings = off
>  parsing.error-logging-verbosity = off
>  }
>
>  parsing.illegal-header-warnings = off
>
>  server {
>  idle-timeout=600s
>  request-timeout=600s
>  remote-address-header=true
>  }
>
> }
>
>
>
> With no result. I've overridden the values programmatically:
>
> private val settings = ConnectionPoolSettings(config = 
> Application.system.settings.config)
> private val cs = settings.connectionSettings
> private val ps = settings.connectionSettings.parserSettings
> private val _settings = 
> settings.withConnectionSettings(cs.withParserSettings(ps.withIllegalHeaderWarnings(false)))
> lazy val pool = Http().cachedHostConnectionPoolHttps[Int](
>  host = ""
>  , port = 443
>  , settings = _settings
>  , connectionContext = SSLSupport.sslContext
> )
>
> Same here. I keep receiving thousands of beautiful messages like this:
>
>
> [WARN] [02/23/2017 18:51:24.596] [default-akka.actor.default-dispatcher-9] 
> [akka.actor.ActorSystemImpl(default)] Explicitly set HTTP header 'User-Agent: 
> akka-http/10.0.3' is ignored, illegal RawHeader
>
> What can I do now?
>
> It seems that the related bugs were closed in 2015.
>
> --
> WBR
> Dmitry
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Detail: akka.stream.StreamTcpException: Connection failed.

2017-02-27 Thread 'Johannes Rudolph' via Akka User List 
Hello,

the error is thrown 
here: 
https://github.com/akka/akka/blob/master/akka-stream/src/main/scala/akka/stream/impl/io/TcpStages.scala#L303.
 
I agree it's hard to see why that error would happen. One thing that could 
have happened is that a connection attempt was very slow and so the 
idle-timeout kicked in and killed the TCP stage before a connection was 
established. Either there was a real connection problem, or the 
infrastructure didn't make progress in due time e.g. because of a thread 
starvation issue.

Btw. using `Source.single().via(connectionFlow).runWith(Sink.head)` is 
discouraged. Please use `Http.singleRequest` instead. See also the note 
at 
http://doc.akka.io/docs/akka-http/current/scala/http/client-side/host-level.html#host-level-api.

HTH
Johannes



On Monday, February 27, 2017 at 8:25:37 AM UTC+1, Chaitanya Mahipath wrote:
>
> Hi,
>
> I am using the akka http scaladsl library to do a HTTP calls against a 
> server. I am running a test where I have 15 threads running concurrently 
> each making around 250 GET calls(15*250). Trying to run this test for three 
> hours but I see that after two and half hours of execution I could see
> Detail: akka.stream.StreamTcpException: Connection failed. exception being 
> thrown.
>
> Initially I thought the servers isn't accepting the incoming requests. 
> But, ran the same test directly against the server and could see no issues.
> Code snippet to connect to the server -
>
> connectionFlow= 
> Http(context.system).outgoingConnectionTls(req.uri.authority.host.address(), 
> port)
> Source.single(req.withUri(req.uri.toRelative)).via(connectionFlow).runWith(Sink.head)
>  
> - Here, req is the HttpRequest.
>
> Could someone please help me out why is that StreamTcpException exception 
> is thrown?
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.