Re: Concurrency in invoker http client

2018-07-15 Thread Rodric Rabbah
> On Jul 13, 2018, at 8:21 PM, Markus Thoemmes > wrote: > > As a more general, existential question: Do we even need the truncation path? > Could we just deny the response straight away if the user's action returns a > bigger value than allowed? Thanks for asking the question. We could chan

Re: Concurrency in invoker http client

2018-07-14 Thread Tyson Norris
Right, got it. Got an initial impl working here: https://github.com/apache/incubator-openwhisk/pull/3812/files#diff-3fca67626fe5f92ce431902dd2318a10R170 So now in that PR, the entity is either consumed via Unmarshal().to[String], when within the allowed response size, or else via the truncated()

Re: Concurrency in invoker http client

2018-07-14 Thread Markus Thoemmes
Hi Tyson, I think that is (kinda) expected. The ByteStrings are coming through in chunks, usually in sizes like 4k or 8k depending on the underlying connection. You won't know beforehand how big these chunks are. Have you tried with a big response (like 1M?). The chunk should not be 1M in size

Re: Concurrency in invoker http client

2018-07-13 Thread Tyson Norris
BTW, trying prefixAndTail, this doesn’t work for me e.g. response.entity.dataBytes .prefixAndTail(1) .runWith(Sink.head) .map({ case (Seq(r), _) => Right(ContainerResponse(response.status.intValue, r.utf8String, Some(contentLength.B, maxResponse))) }) Does not result with r.size

Re: Concurrency in invoker http client

2018-07-13 Thread Tyson Norris
Thanks Currently HttpUtils does return (almost) the same response when it is truncated.. so that the truncated version can later be included in the ErrorResponse… I would be OK with changing the error message to not include ANY of the response, since either way it is an error message. Then th

Re: Concurrency in invoker http client

2018-07-13 Thread Markus Thoemmes
Hi Tyson, Chetan solved a similar problem in "inlineOrAttach" in "AttachmentSupport.scala". He did this by continuously running a stream through "prefixAndTail", where he'd pick just one element and then run the stream to check whether he already crossed the limit. This case now is a lot more

Re: Concurrency in invoker http client

2018-07-13 Thread Tyson Norris
Getting back to this, I’m close, but not sure how to get akka http client to truncate a response? There is response.entity.withSizeLimit() - but this throws an exception, not really what I want. I think what I want is a variant of the HttpEntity.Limitable GraphStage that just stops emitting c

Re: Concurrency in invoker http client

2018-06-25 Thread Rodric Rabbah
the retires are only on failure to establish a connection - no other retries should be happening iirc. -r On Mon, Jun 25, 2018 at 1:29 PM Tyson Norris wrote: > Thanks Markus - one other question: > > Assuming retry is the current missing piece to using PoolingRestClient (or > akka http directly

Re: Concurrency in invoker http client

2018-06-25 Thread Tyson Norris
Thanks Markus - one other question: Assuming retry is the current missing piece to using PoolingRestClient (or akka http directly), I’m also wondering if “retry” is the proper approach here? It may be worthwhile to initiate a port connection (with its own timeout/retry behavior) before the /ini

Re: Concurrency in invoker http client

2018-06-25 Thread Markus Thoemmes
Hi Tyson, Ha, I was thinking about moving back to akka the other day. A few comments: 1. Travis build environments have 1.5 CPU cores which might explain the "strange" behavior you get from the apache client? Maybe it adjusts its thread pool based on the number of cores available? 2. To impleme

Concurrency in invoker http client

2018-06-25 Thread Tyson Norris
Hi - As part of the support for concurrency in action containers, I have this PR open in the nodejs action container: https://github.com/apache/incubator-openwhisk-runtime-nodejs/pull/41 I’ve been having real trouble getting concurrency tests to operate properly in travis env. The behavior app