[jira] [Commented] (TINKERPOP-2546) Change transport layer to use AIOHTTP instead of Tornado
[ https://issues.apache.org/jira/browse/TINKERPOP-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321290#comment-17321290 ] ASF GitHub Bot commented on TINKERPOP-2546: --- lyndonb-bq commented on pull request #1416: URL: https://github.com/apache/tinkerpop/pull/1416#issuecomment-819794303 I updated the changes to allow keyword arg passthrough directly. This way a user can just directly pass through anything available in https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientSession.ws_connect I also mapped ssl_options to ssl and max_content_length to max_msg_size for backwards compatibility if users already have those being set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Change transport layer to use AIOHTTP instead of Tornado > > > Key: TINKERPOP-2546 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2546 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.5.0 >Reporter: Lyndon Bauto >Priority: Major > Fix For: 3.5.0 > > > Following discussion here > https://lists.apache.org/thread.html/rc4d047ba0245fde3261bad0ea156a0f35db1ab0f92c8e80d4fb1b815%40%3Cdev.tinkerpop.apache.org%3E, > this ticket will track rewriting the tranport layer of gremlin-python to use > AIOHTTP instead of Tornado. I am currently working on this and will have a > pull request shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2546) Change transport layer to use AIOHTTP instead of Tornado
[ https://issues.apache.org/jira/browse/TINKERPOP-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321214#comment-17321214 ] ASF GitHub Bot commented on TINKERPOP-2546: --- lyndonb-bq commented on pull request #1416: URL: https://github.com/apache/tinkerpop/pull/1416#issuecomment-819726446 > i recall you saying that the compression dict map was removed as there was not quite an analogous configuration for aiohttp but i though that there was a compression option in aiohttp that could be enabled as a flag still but i didn't see that here. am i missing where that is somehow? @spmallette There is a compression you can set for it but from what I have read it is a different type than Tornado's. I tried setting it up and all the tests failed whenever I used it so I don't think it is compatible with the current Gremlin server. I looked at aiogremlin's driver and found they didn't have it setup there either so I thought it might just not work for this use case unfortunately. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Change transport layer to use AIOHTTP instead of Tornado > > > Key: TINKERPOP-2546 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2546 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.5.0 >Reporter: Lyndon Bauto >Priority: Major > Fix For: 3.5.0 > > > Following discussion here > https://lists.apache.org/thread.html/rc4d047ba0245fde3261bad0ea156a0f35db1ab0f92c8e80d4fb1b815%40%3Cdev.tinkerpop.apache.org%3E, > this ticket will track rewriting the tranport layer of gremlin-python to use > AIOHTTP instead of Tornado. I am currently working on this and will have a > pull request shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2390) Connections not released when closed abruptly in the server side
[ https://issues.apache.org/jira/browse/TINKERPOP-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321134#comment-17321134 ] Carlos commented on TINKERPOP-2390: --- Hi Florian, we moved our implementation to the Java driver. However, the scenario where we saw this behaviour was not exactly due to timeout, but due to overwhelming the server with petitions (not so many in fact). When the underlying Tinkerpop provider was saturated (janusgraph in this case), seemed to start closing connections and the Tinkerpop server was not notifying properly the client, that's why we saw a lot of dead connections. I'm afraid I cannot give more details rather than it was a continuous flow of requests that provoked the behaviour. Best, > Connections not released when closed abruptly in the server side > > > Key: TINKERPOP-2390 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2390 > Project: TinkerPop > Issue Type: Bug > Components: dotnet >Affects Versions: 3.4.7 > Environment: Tinkerpop 3.4.7 + Janusgraph 0.5.1 (optional opencypher > 1.0.0) >Reporter: Carlos >Priority: Major > > We have developed a WService to query a gremlin-server (JanusGraph 0.5.1) > using the .net driver. Using the opencypher plugin has allowed us to see a > behaviour where the server gets completely blocked after a timeout on the > server side. We thought this might be related to issue > https://issues.apache.org/jira/browse/TINKERPOP-2288, so we have moved our > driver version to the master one (3.4-dev, which includes the PR solving this > issue). However, when facing a timeout (server side always, it is the one > launching the exception), quite a lot of connections get stalled at > CLOSE_WAIT status, and the server becomes unusable. > I've been digging around other bugs and issues, and from what I've read, some > similar behaviour happened to CosmoDB (although it might be caused in that > situation due to the some connection leaks, in this case is the timeout). We > have traced down the problem to the driver itself after isolating all the > components involved (optimizing the cypher query results in a non-timeout > situation where everything is ok; forcing the timeout from pure gremlin > replicates the behaviour). > We have set up the connection pool params to 16 / 4096 (we are expecting > quite a high concurrency load). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2390) Connections not released when closed abruptly in the server side
[
https://issues.apache.org/jira/browse/TINKERPOP-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321093#comment-17321093
]
Florian Hockmann commented on TINKERPOP-2390:
-
I just tried to reproduce the scenario but I don't see anything wrong. Here is
what I did:
# Start the server with a {{gremlinPool}} of 1 as described above
(_TinkerpopServer configured not to provide any concurrent service (i.e., all
the queries were processed sequentially_).
# Connect from Gremlin.Net (I used the version from current {{master}} and
also tried it with the version from {{3.4-dev}}) with default settings
({{PoolSize}} of 4 and {{MaxInProcessPerConnection}}: 32)
# Send 10 requests with a custom evaluation timeout of 1 ms that simply sleep
for 3 seconds.
# Result:
## All requests get a {{ResponseException}} with a timeout on the server side.
## 4 connections in state {{ESTABLISHED}} on the server side.
# Send 1 request to verify that both the driver and the server are still in a
valid state. -> Receive the expected result.
# Dispose the {{GremlinClient}} instance.
# Result:
## All 4 connections in state {{TIME_WAIT}} on the server
## After 1 min: connections completely closed
The server is still responsive after this. The {{TIME_WAIT}} is expected from
my limited knowledge about TCP as connections are not completely closed
immediately in case a packet is received out of order. But they are closed
after a timeout which seems to be one minute on my machine.
What I really don't understand here is why the server should close the
connection just because one request ran into a timeout. That doesn't make much
sense as multiple requests can be processed on the same connection. So, the
connection shouldn't be affected by a failing request (failing here in the
sense of timing out).
[~Bobed] Could you please provide more information on this, ideally a setup to
reproduce the problem deterministically? Otherwise, I'm inclined to close this
issue as we cannot reproduce it.
> Connections not released when closed abruptly in the server side
>
>
> Key: TINKERPOP-2390
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2390
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.4.7
> Environment: Tinkerpop 3.4.7 + Janusgraph 0.5.1 (optional opencypher
> 1.0.0)
>Reporter: Carlos
>Priority: Major
>
> We have developed a WService to query a gremlin-server (JanusGraph 0.5.1)
> using the .net driver. Using the opencypher plugin has allowed us to see a
> behaviour where the server gets completely blocked after a timeout on the
> server side. We thought this might be related to issue
> https://issues.apache.org/jira/browse/TINKERPOP-2288, so we have moved our
> driver version to the master one (3.4-dev, which includes the PR solving this
> issue). However, when facing a timeout (server side always, it is the one
> launching the exception), quite a lot of connections get stalled at
> CLOSE_WAIT status, and the server becomes unusable.
> I've been digging around other bugs and issues, and from what I've read, some
> similar behaviour happened to CosmoDB (although it might be caused in that
> situation due to the some connection leaks, in this case is the timeout). We
> have traced down the problem to the driver itself after isolating all the
> components involved (optimizing the cypher query results in a non-timeout
> situation where everything is ok; forcing the timeout from pure gremlin
> replicates the behaviour).
> We have set up the connection pool params to 16 / 4096 (we are expecting
> quite a high concurrency load).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321052#comment-17321052
]
Florian Hockmann commented on TINKERPOP-2019:
-
I implemented that option today
([commit|https://github.com/apache/tinkerpop/commit/93ab05c8d53d9573822b9012d18410c24ed469c7]).
The {{Connection}} now cancels all outstanding Websocket operations before
closing the underlying connection. [~dzmitry.lahoda] Would you be available to
test out that change with your stresser tool and CosmosDB?
I think the change makes sense in general so we could also merge it like this,
but it would definitely be good to know whether that actually fixes this
problem.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[DISCUSS] 3.4.11/3.5.0 readiness
I think we're closing in on a code freeze date for 3.4.11 and 3.5.0. The things I'd considered blockers for 3.5.0 are on their way to merging: + https://github.com/apache/tinkerpop/pull/1414 - UnifiedChannelizer + https://github.com/apache/tinkerpop/pull/1416 - gremlin-python and aiohttp I'm aware of one other change that I'd like to see in there with the java driver which I think will arrive next week: https://issues.apache.org/jira/browse/TINKERPOP-2547 As such, I think we could firm up for code freeze on the Friday after this one, April 23. If anyone has other important things to get in, please reply here so that we can make sure this date makes sense.
[jira] [Created] (TINKERPOP-2547) Provide an option to supply a callback before request submission
Stephen Mallette created TINKERPOP-2547: --- Summary: Provide an option to supply a callback before request submission Key: TINKERPOP-2547 URL: https://issues.apache.org/jira/browse/TINKERPOP-2547 Project: TinkerPop Issue Type: Improvement Components: driver Affects Versions: 3.4.10 Reporter: Stephen Mallette Provide a callback that allows the user to modify a request before it gets sent to the server. This change would allow support for custom authentication methods and similar functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2546) Change transport layer to use AIOHTTP instead of Tornado
[ https://issues.apache.org/jira/browse/TINKERPOP-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17320900#comment-17320900 ] ASF GitHub Bot commented on TINKERPOP-2546: --- spmallette commented on pull request #1416: URL: https://github.com/apache/tinkerpop/pull/1416#issuecomment-819423801 i recall you saying that the compression dict map was removed as there was not quite an analogous configuration for aiohttp but i though that there was a compression option in aiohttp that could be enabled as a flag still but i didn't see that here. am i missing where that is somehow? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Change transport layer to use AIOHTTP instead of Tornado > > > Key: TINKERPOP-2546 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2546 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.5.0 >Reporter: Lyndon Bauto >Priority: Major > Fix For: 3.5.0 > > > Following discussion here > https://lists.apache.org/thread.html/rc4d047ba0245fde3261bad0ea156a0f35db1ab0f92c8e80d4fb1b815%40%3Cdev.tinkerpop.apache.org%3E, > this ticket will track rewriting the tranport layer of gremlin-python to use > AIOHTTP instead of Tornado. I am currently working on this and will have a > pull request shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
