Re: [akka-user] Long-running HTTP requests not marking the TCP slot as busy
I will file an issue.
The problem would exist and even be amplified by long-polling GET requests,
as they would be seen as idempotent. Moreover, in Akka it looks like
disabling pipelining (setting it to 1) might not be enough if the
connection is seen as idle, at least it didn't work for me. Maybe we should
have a way of marking a request as potentially long running, which would
make the slot busy as long as the connection has not terminated, and
non-idempotent methods could set this flag by default.
2015-05-27 19:58 GMT+02:00 Roland Kuhn goo...@rkuhn.info:
Hi Samuel,
what you describe sounds like a bug, and I think I know how it arises as
well. Would you please file an issue with this log and explanation? Thanks!
Independently, it seems that you are pointing out an overall problematic
relation between pipelining and long-running HTTP/1.1 requests: would this
very same problem not exist for long-polling GET requests? If a user wants
to schedule a long-running request of this kind, it seems to me that manual
care would need to be taken to disable pipelining on the connection that is
being used. How is this usually handled?
Regards,
Roland
27 maj 2015 kl. 18:59 skrev Samuel Tardieu s...@rfc1149.net:
Hi.
Using Akka HTTP Streams 1.0-RC3 in Scala, I have the impression that a
connection can be used to pipeline another request even if a non-idempotent
request is being executed (vs. a non-idempotent request being completed). I
had the same problem in 1.0-RC2 but took no time to investigate.
The context: I’m using an in-house library to access CouchDB databases
over HTTP named “canape”. I create a database and wait for the result, I
then open a long-running connection to the “_changes” stream which returns
live database changes and send them to a “fold” sink in the background, and
I immediately create 5 documents named “docid“ and wait for 100
milliseconds after every document creation.
In the example below, we have a single TCP stream dumped with wireshark.
Note how the request to “_changes” (which represents a live stream of
changes in CouchDB) returns a chunked response, and the chunked response is
obviously not over when the PUT to create the second document (“docid2”)
is sent over the same connection. Even though I took care of using POST
to ensure that Akka does see the “_changes” request as non-idempotent.
Note that the first document (“docid1”) has been created on another TCP
connection (and thus not shown here), probably because it has been sent
right after the request to the “_changes” stream, which means that at this
time Akka probably considered the non-idempotent request to be running.
However, the second document is being created on the busy original TCP
connection, as if as soon as the headers were sent back the connection is
considered idle again, although its entity is still being transmitted and
may be for a long time. Since the PUT for “docid2” is still blocked on
this connection, we can also see that “docid3”, “docid4” and “docid5” have
been properly created using another TCP connection.
Right now, I work around this by creating a new host connection pool for
every long-running connection, but it is a waste or resources and causes
some code duplication. The problem shown below happens when I try to use a
common pool for all requests (which should work fine).
Any idea of what might be wrong here?
PUT /canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/ HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Length: 0
HTTP/1.1 201 Created
Server: CouchDB/1.6.1 (Erlang OTP/17)
Location:
http://localhost:5984/canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312
Date: Wed, 27 May 2015 16:43:24 GMT
Content-Type: application/json
Content-Length: 12
Cache-Control: must-revalidate
{ok:true}
POST
/canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/_changes?feed=continuous
HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Type: application/json
Content-Length: 2
{}
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: CouchDB/1.6.1 (Erlang OTP/17)
Date: Wed, 27 May 2015 16:43:25 GMT
Content-Type: application/json
Cache-Control: must-revalidate
51
{seq:1,id:docid1,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
PUT /canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/docid2 HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Type: application/json
Content-Length: 2
{}
51
{seq:2,id:docid3,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
51
{seq:3,id:docid4,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
51
{seq:4,id:docid5,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
(hangs there, as the connection is still busy sending the long-running
_changes stream, the creation of “docid2” will timeout)
--
Re: [akka-user] Long-running HTTP requests not marking the TCP slot as busy
Hi Samuel,
what you describe sounds like a bug, and I think I know how it arises as well.
Would you please file an issue with this log and explanation? Thanks!
Independently, it seems that you are pointing out an overall problematic
relation between pipelining and long-running HTTP/1.1 requests: would this very
same problem not exist for long-polling GET requests? If a user wants to
schedule a long-running request of this kind, it seems to me that manual care
would need to be taken to disable pipelining on the connection that is being
used. How is this usually handled?
Regards,
Roland
27 maj 2015 kl. 18:59 skrev Samuel Tardieu s...@rfc1149.net:
Hi.
Using Akka HTTP Streams 1.0-RC3 in Scala, I have the impression that a
connection can be used to pipeline another request even if a non-idempotent
request is being executed (vs. a non-idempotent request being completed). I
had the same problem in 1.0-RC2 but took no time to investigate.
The context: I’m using an in-house library to access CouchDB databases over
HTTP named “canape”. I create a database and wait for the result, I then open
a long-running connection to the “_changes” stream which returns live
database changes and send them to a “fold” sink in the background, and I
immediately create 5 documents named “docid“ and wait for 100 milliseconds
after every document creation.
In the example below, we have a single TCP stream dumped with wireshark. Note
how the request to “_changes” (which represents a live stream of changes in
CouchDB) returns a chunked response, and the chunked response is obviously
not over when the PUT to create the second document (“docid2”) is sent over
the same connection. Even though I took care of using POST to ensure that
Akka does see the “_changes” request as non-idempotent.
Note that the first document (“docid1”) has been created on another TCP
connection (and thus not shown here), probably because it has been sent right
after the request to the “_changes” stream, which means that at this time
Akka probably considered the non-idempotent request to be running. However,
the second document is being created on the busy original TCP connection, as
if as soon as the headers were sent back the connection is considered idle
again, although its entity is still being transmitted and may be for a long
time. Since the PUT for “docid2” is still blocked on this connection, we can
also see that “docid3”, “docid4” and “docid5” have been properly created
using another TCP connection.
Right now, I work around this by creating a new host connection pool for
every long-running connection, but it is a waste or resources and causes some
code duplication. The problem shown below happens when I try to use a common
pool for all requests (which should work fine).
Any idea of what might be wrong here?
PUT /canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/ HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Length: 0
HTTP/1.1 201 Created
Server: CouchDB/1.6.1 (Erlang OTP/17)
Location:
http://localhost:5984/canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312
http://localhost:5984/canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312
Date: Wed, 27 May 2015 16:43:24 GMT
Content-Type: application/json
Content-Length: 12
Cache-Control: must-revalidate
{ok:true}
POST
/canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/_changes?feed=continuous
HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Type: application/json
Content-Length: 2
{}
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: CouchDB/1.6.1 (Erlang OTP/17)
Date: Wed, 27 May 2015 16:43:25 GMT
Content-Type: application/json
Cache-Control: must-revalidate
51
{seq:1,id:docid1,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
PUT /canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/docid2 HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Type: application/json
Content-Length: 2
{}
51
{seq:2,id:docid3,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
51
{seq:3,id:docid4,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
51
{seq:4,id:docid5,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
(hangs there, as the connection is still busy sending the long-running
_changes stream, the creation of “docid2” will timeout)
--
Read the docs: http://akka.io/docs/ http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
Akka User List group.
[akka-user] Long-running HTTP requests not marking the TCP slot as busy
Hi.
Using Akka HTTP Streams 1.0-RC3 in Scala, I have the impression that a
connection can be used to pipeline another request even if a non-idempotent
request is being executed (vs. a non-idempotent request being completed). I
had the same problem in 1.0-RC2 but took no time to investigate.
The context: I’m using an in-house library to access CouchDB databases over
HTTP named “canape”. I create a database and wait for the result, I then
open a long-running connection to the “_changes” stream which returns live
database changes and send them to a “fold” sink in the background, and I
immediately create 5 documents named “docid“ and wait for 100 milliseconds
after every document creation.
In the example below, we have a single TCP stream dumped with wireshark.
Note how the request to “_changes” (which represents a live stream of
changes in CouchDB) returns a chunked response, and the chunked response is
obviously not over when the PUT to create the second document (“docid2”) is
sent over the same connection. Even though I took care of using POST to
ensure that Akka does see the “_changes” request as non-idempotent.
Note that the first document (“docid1”) has been created on another TCP
connection (and thus not shown here), probably because it has been sent
right after the request to the “_changes” stream, which means that at this
time Akka probably considered the non-idempotent request to be running.
However, the second document is being created on the busy original TCP
connection, as if as soon as the headers were sent back the connection is
considered idle again, although its entity is still being transmitted and
may be for a long time. Since the PUT for “docid2” is still blocked on this
connection, we can also see that “docid3”, “docid4” and “docid5” have been
properly created using another TCP connection.
Right now, I work around this by creating a new host connection pool for
every long-running connection, but it is a waste or resources and causes
some code duplication. The problem shown below happens when I try to use a
common pool for all requests (which should work fine).
Any idea of what might be wrong here?
PUT /canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/ HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Length: 0
HTTP/1.1 201 Created
Server: CouchDB/1.6.1 (Erlang OTP/17)
Location:
http://localhost:5984/canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312
Date: Wed, 27 May 2015 16:43:24 GMT
Content-Type: application/json
Content-Length: 12
Cache-Control: must-revalidate
{ok:true}
POST
/canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/_changes?feed=continuous
HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Type: application/json
Content-Length: 2
{}
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: CouchDB/1.6.1 (Erlang OTP/17)
Date: Wed, 27 May 2015 16:43:25 GMT
Content-Type: application/json
Cache-Control: must-revalidate
51
{seq:1,id:docid1,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
PUT /canape-test-db-d45ad182-e2ec-4387-b5a1-a51d016c0312/docid2 HTTP/1.1
Host: localhost:5984
User-Agent: canape for Scala
Accept: application/json
Content-Type: application/json
Content-Length: 2
{}
51
{seq:2,id:docid3,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
51
{seq:3,id:docid4,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
51
{seq:4,id:docid5,changes:[{rev:1-967a00dff5e02add41819138abb3284d}]}
(hangs there, as the connection is still busy sending the long-running
_changes stream, the creation of “docid2” will timeout)
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
