[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321052#comment-17321052
]
Florian Hockmann commented on TINKERPOP-2019:
-
I implemented that option today
([commit|https://github.com/apache/tinkerpop/commit/93ab05c8d53d9573822b9012d18410c24ed469c7]).
The {{Connection}} now cancels all outstanding Websocket operations before
closing the underlying connection. [~dzmitry.lahoda] Would you be available to
test out that change with your stresser tool and CosmosDB?
I think the change makes sense in general so we could also merge it like this,
but it would definitely be good to know whether that actually fixes this
problem.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17078402#comment-17078402
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
Sound very resonable to cancel out receive before close. Feels very promising
fix. Will touch as soon as will do AAA stuff with gremlin.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17078396#comment-17078396
]
Florian Hockmann commented on TINKERPOP-2019:
-
I looked into this some more as I wanted to create an issue in the {{dotnet}}
repo. After looking into [the source code of the {{ManagedWebSocket}}
class|https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/System/Net/WebSockets/ManagedWebSocket.cs],
I think that this could actually not be a problem of calling {{ReceiveAsync}}
two times in parallel, but maybe that we call {{CloseAsync}} while we still
have a {{ReceiveAsync}} that hasn't completed yet or vice versa. {{CloseAsync}}
also calls internally {{ReceiveAsyncPrivate}} to receive a close response from
the server.
{{ManagedWebSocket}} shouldn't have a problem in general when {{CloseAsync}}
is called when there is still an unfinished {{ReceiveAsync}} operation in
progress as that [was an issue that has been fixed in 2016
already|https://github.com/dotnet/runtime/issues/17819].
But I wonder whether we could see some kind of race condition here where the
checks that were added for the mentioned issue succeed although the task is not
fully completed yet or something like that. The fact that you needed to perform
4,5 M calls under bad network conditions, if I understood that correctly,
[~dzmitry.lahoda], until the exception occurred, could also be seen as a hint
that we have a rare race condition here.
I see two options to proceed further here. We could either:
* try whether we can simply fix this by cancelling any pending operations on
the {{ClientWebSocket}} in our {{WebSocketConnection}} class with a
{{CancellationTokenSource}} before calling {{CloseAsync}} or
* try to reproduce this problem with a minimalistic example outside of
Gremlin.Net and then use that example to create an issue in the {{dotnet}} repo.
I think that we need a minimalistic example that still reproduces this
exception to get meaningful help from the {{dotnet}} team as they otherwise
cannot be sure that the error is not in our usage of {{ClientWebSocket}} and it
would of course also rule out that possibility for us.
Using a {{CancellationTokenSource}} to cancel any operations before calling
{{CloseAsync}} is probably a good idea in general, but it would of course be
good to know whether that already solves the problem.
I'll try to follow up on this and try out one or both approaches, but I'm not
sure yet when I'll have the time for this. So, if anyone wants to take this up,
then that would be greatly appreciated.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
>
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043549#comment-17043549
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
Please write it together. I will like it and add more details.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043525#comment-17043525
]
Florian Hockmann commented on TINKERPOP-2019:
-
{quote}Looks like receive error .NET Core bug. I have validated that it is kind
of impossible in Gremlin Driver.
{quote}
That's what I'm also wondering about here. Our code should make it impossible
for two read or two write operations to occur on the same connection at the
same time. It also looks like we can't really make any progress on this without
more information about the underlying {{System.Net.WebSockets}} implementation.
I think we should create an issue in the GitHub .NET repo for this. If you want
to do that as you already investigated that, then feel free to go ahead,
[~dzmitry.lahoda]. Otherwise, I will do it when I find some time to write it
together.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042206#comment-17042206
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
Looks like receive error .NET Core bug. I have validated that it is kind of
impossible in Gremlin Driver.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041483#comment-17041483
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
Attached logs (unfortunately without timestamps and thread ids and message ids
or statuses...). These seems happened after ResponseException of rate limiting.
Will check what was in our production for that case.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png, invalid.txt
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041472#comment-17041472
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
I got received error locally (so it is in 14GB text file - need find a way to
open it):
!image-2020-02-21-05-33-27-246.png!
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
> Attachments: image-2020-02-21-05-32-58-730.png,
> image-2020-02-21-05-33-27-246.png
>
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041333#comment-17041333
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
See my comment on github PR for null reference.
As for `There is already one outstanding..` I do not have reproduction yet. I
guess we should increase query rate (so local Gremlin Server could help, kind
of increase possibility to interleave queries) and use some form of container
based HTTP/WS based chaos engineering (not TCP level, like to reorder HTTP/WS
messages, not TCP chunks; and we run in container).
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040993#comment-17040993
]
Florian Hockmann commented on TINKERPOP-2019:
-
Great to hear that you seem to be able to reproduce this issue,
[~dzmitry.lahoda]! Are you using Cosmos DB for that or can you also reproduce
it with Gremlin Server?
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040866#comment-17040866
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
My current plan. As we recreate client on web socket exception, recreate also
on invalid operation exception to handle this and TINKERPOP-2192 (after and if
my fix merged).
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040838#comment-17040838
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
I was able to repro #2192 once (so no stack logged). I have used next
[https://github.com/dzmitry-lahoda-forks/tinkerpop/blob/feature/gremlin-net-stability/gremlin-dotnet/test/Gremlin.Net.StressTest/Main.fs]
Today I will put it on run for the night. I used
[https://jagt.github.io/clumsy/index.html] to to worsen network conditions.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039816#comment-17039816
]
Dzmitry.Lahoda commented on TINKERPOP-2019:
---
Please reopen issue.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036355#comment-17036355
]
Dzmitry L commented on TINKERPOP-2019:
--
Yes, I'm using CosmosDb. Reproduction steps are not clear. The bug appeared
twice on simple query like `g.V('123-123-123')`.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036224#comment-17036224
]
Stephen Mallette commented on TINKERPOP-2019:
-
Are you using CosmosDB [~idzmitry]? If so, I think you can see from the
comments that [~Florian Hockmann] struggled for a long time to reproduce this.
If we were to reopen this issue I think we'd need support from CosmosDB folks
for anything at all to happen with it. If you have reproduction steps for
making this happen with Gremlin Server by itself then we might have chance at
fixing it independently.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035493#comment-17035493
]
Dzmitry L commented on TINKERPOP-2019:
--
Hi. I've got same exception using v 3.4.4 on .net core 3.1. App spins in a
docker container based on
mcr.microsoft.com{color:#ff}/{color}dotnet{color:#ff}/{color}core{color:#ff}/{color}sdk:3.1
image.
{code:java}
System.InvalidOperationException: There is already one outstanding
'ReceiveAsync' call for this WebSocket instance. ReceiveAsync and SendAsync can
be called simultaneously, but at most one outstanding operation for each of
them is allowed at the same time. at
System.Net.WebSockets.ManagedWebSocket.ThrowOperationInProgress(String
methodName) at
System.Net.WebSockets.ManagedWebSocket.ReceiveAsync(ArraySegment`1 buffer,
CancellationToken cancellationToken) --- End of stack trace from previous
location where exception was thrown --- {code}
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768644#comment-16768644
]
stephen mallette commented on TINKERPOP-2019:
-
Is there any new information that can be provided on this one given the last
comment from [~Florian Hockmann]? if not, i think we will look to close this
one. thanks.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741245#comment-16741245
]
Florian Hockmann commented on TINKERPOP-2019:
-
We released [version 3.4.0|https://www.nuget.org/packages/Gremlin.Net/3.4.0]
this week which contains an improved connection pool and which now also
supports request pipelining. This means that a lot of the code in the driver
changed. (Those changes weren't yet in 3.4.0-rc2.) [~samimajed] could you
please try out whether the issue still happens with that version? Or could you
already fix this by upgrading to .NET Core 2.1?
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724309#comment-16724309
]
stephen mallette commented on TINKERPOP-2019:
-
Just a note that we're closing up development on 3.4.0 this week in preparation
for release at the very start of 2019.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689058#comment-16689058
]
Sami commented on TINKERPOP-2019:
-
No, not resolved yet, Stephen. The underlying issue here is that the library
encounters a WinHttpException when a web socket connection is terminated
abnormally, and that happens very seldom. When this occurs, however, the web
socket state is still recognized as "Open" and stays in the pool. And when the
'bad' connection gets reused (i.e. reissued by the pool), then the library
throws the InvalidOperationException. That still occurs in 3.4-rc2.
We've been working with Microsoft (O.Towers) to identify why the web socket
state isn't recognized correctly and have been using a fork which temporarily
fixes the state. Our solution may be to use .NET Core 2.1 (which, according to
MS and my understanding, should have a rewrite of the underlying web socket
implementation). We're working on a proof of concept to verify and will update
this thread when we're sure.
-Sami
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676882#comment-16676882
]
stephen mallette commented on TINKERPOP-2019:
-
[~samimajed] 3.2.10 and 3.3.4 were officially released (as was 3.4.0-rc2 for
.NET) - did those versions resolve this problem? is there still an issue here?
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580916#comment-16580916
]
Florian Hockmann commented on TINKERPOP-2019:
-
Ok, now we're talking about two problems here, namely the
{{InvalidOperationException}} mentioned in the ticket description and now the
new problem you mention with closed connections you encounter with your retry
policy. I concentrate in this comment on the latter as I still have no idea
where the {{InvalidOperationException}} is coming from.
{quote}with a retry and now we are getting the following error; basically we
run out of pooled connections and Gremlin.Net shuts down. (Yikes!) : )
{quote}
Normally, the {{ConnectionPool}} should close all connections when it thinks
that the remote server isn't reachable but afterwards new connections should be
established when new requests arive for the driver. However, it looks like you
encountered another exception where the driver tries to close the connection
although it was already closed (probably by the server). I created
TINKERPOP-2026 for this issue.
As a possible workaround for, couldn't you simply add the
{{WebSocketException}} to your retry policy?
{quote}Also, I'm not sure if I can abstract out Cosmosdb and only work with the
Gremlin server since I don't have experience with it.
{quote}
The problem right now is that it's very hard to reason about this error you're
seeing without being able to reproduce it. When you could reproduce it with the
Gremlin Server, then we could much easier investigate this. You can simply
start the Gremlin Server [as describes in the
docs|http://tinkerpop.apache.org/docs/current/reference/#starting-gremlin-server]
and you can download it directly from [TinkerPop's
homepage|http://tinkerpop.apache.org/]. Another option would be the [new Docker
image we provide|https://hub.docker.com/r/tinkerpop/gremlin-server/]. Just note
that it's currently only available as a release candidate for version 3.2.10
(the tag is 3.2.10-rc1) which means that you have to configure Gremlin.Net for
GraphSON 2.
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the WebSocketConnection class
> to keep the single pending "send" or a single pending "receive" rule may be
> the cause.
>
> In our .NET Core web api application, we create the GremlinConnection as a
> singleton in Startup.cs and then have one central call that makes Gremlin
> calls; i.e. it's a very straightforward implementation.
>
> Startup.cs:
> {code:c#}
> public void ConfigureServices(IServiceCollection services)
> {
> //...other stuff removed for brevity
> services.AddSingleton();
> }{code}
>
> Reader.cs:
> {code:c#}
> public async Task> ExecuteGremlinQuery(string
> query)
> {
> try
> {
> return await _gremlinConnection.Client.SubmitAsync(query);
> }
> catch (Gremlin.Net.Driver.Exceptions.ResponseException responseException)
> {
> //our error handling removed for brevity!
> }
> }{code}
>
> We use the Gremlin.Net version 3.3.3 nuget package and the
> Microsoft.NETCore.App SDK
>
> Would it be possible to identify if this is indeed a bug on Gremlin.NET?
> And if it is, any thoughts on a best-practice (temporary) work-around that
> we can implement?
--
[jira] [Commented] (TINKERPOP-2019) Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
[
https://issues.apache.org/jira/browse/TINKERPOP-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571716#comment-16571716
]
Florian Hockmann commented on TINKERPOP-2019:
-
Thanks for bringing this to our attention, [~samimajed]. I just formatted the
issue description a bit.
{quote}Would it be possible to identify if this is indeed a bug on Gremlin.NET?
{quote}
Sure, this might be a bug in Gremlin.Net but I'm afraid it could be hard to
track down. If you were able to create a minimal code example that reproduces
the problem, ideally with just Gremlin Server and not CosmosDB, that would be
extremely helpful for us to find and understand the problem better and then
hopefully also fix it.
Regarding work-arounds, without knowing what causes the problem I don't know
how you could try to avoid it. You could of course simply implement a retry
policy for the {{InvalidOperationException}}.
{quote}After looking at GitHub's WebSocketConnection class, I don't see any
orchestration between SendMessageAsync's {{_client.SendAsync}} (currently line
54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line 66).
{quote}
You're right, there is no orchestration directly in that class but the
{{WebSocketConnection}} class is an {{internal}} class and the only usage of
these two methods is [in the {{Connection}}
class|https://github.com/apache/tinkerpop/blob/dd09c059566a0e4d48845ac1dcc789ae075ff10b/gremlin-dotnet/src/Gremlin.Net/Driver/Connection.cs#L56]:
{code:java}
public async Task> SubmitAsync(RequestMessage
requestMessage)
{
await SendAsync(requestMessage).ConfigureAwait(false);
return await ReceiveAsync().ConfigureAwait(false);
}
{code}
As this small snippet already shows, {{ReceiveAsync}} will only be called after
{{SendAsync}} was awaited. So, it should not be possible for these two
functions to be executed at the same time. [Each {{Connection}} also has its
own
{{WebSocketConnection}}|https://github.com/apache/tinkerpop/blob/dd09c059566a0e4d48845ac1dcc789ae075ff10b/gremlin-dotnet/src/Gremlin.Net/Driver/Connection.cs#L41]:
{code:java}
private readonly WebSocketConnection _webSocketConnection = new
WebSocketConnection();
{code}
However, your error message sounds more like {{SendAsync}} gets called two
times simultaneously:
{quote}There is already one outstanding '*SendAsync*' call for this WebSocket
instance.
[...]
System.InvalidOperationException at
Gremlin.Net.Driver.WebSocketConnection+<*SendMessageAsync*>d__5.MoveNext
{quote}
which should also not be possible as the aforementioned method {{SubmitAsync}}
is also only called in one place and the {{Connection}} object gets only
returned to the {{ConnectionPool}} [after being
awaited|https://github.com/apache/tinkerpop/blob/dd09c059566a0e4d48845ac1dcc789ae075ff10b/gremlin-dotnet/src/Gremlin.Net/Driver/GremlinClient.cs#L70]:
{code:java}
using (var connection = await
_connectionPool.GetAvailableConnectionAsync().ConfigureAwait(false))
{
return await
connection.SubmitAsync(requestMessage).ConfigureAwait(false);
}
{code}
So, I have no idea right now how this exception could have occurred.
Did you see anything unusual before this exception occurred? Was there maybe
another exception before?
> Gremlin.Net.Driver.WebSocketConnection throws System.InvalidOperationException
> --
>
> Key: TINKERPOP-2019
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2019
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.3.3
> Environment: Azure App Service
>Reporter: Sami
>Priority: Critical
>
> We're getting the following {{System.InvalidOperationException}} error
> message:
> {code:c#}
> "There is already one outstanding 'SendAsync' call for this WebSocket
> instance. ReceiveAsync and SendAsync can be called simultaneously, but at
> most one outstanding operation for each of them is allowed at the same time.
> Problem Id:
> System.InvalidOperationException at
> Gremlin.Net.Driver.WebSocketConnection+d__5.MoveNext"{code}
>
> We get this exception sporadically and only a few times out of thousands.
> Unfortunately we have not been able to reproduce it.
>
> I understand that when dealing with web sockets, it is allowed to have only
> a single pending "send" or a single pending "receive".
>
> After looking at GitHub's WebSocketConnection class, I don't see any
> orchestration between SendMessageAsync's {{_client.SendAsync}} (currently
> line 54) and ReceiveMessageAsync's {{_client.ReceiveAsync}} (currently line
> 66).
>
> Reference Link:
>
> [https://github.com/apache/tinkerpop/blob/master/gremlin-dotnet/src/Gremlin.Net/Driver/WebSocketConnection.cs]
>
> I'm wondering if not having orchestration in the
