Hello, Is it possible that the huge increase in the reponse times comes from contention when multiple gRPC threads are using the same thin client (thus, the same ClientSocket) to communicate with the cluster?
Yes, that’s correct. Threads will share the same TCP connection by default. But there is no such warning in the .NET thin client documentation (https://apacheignite-net.readme.io/docs/thin-client). I think we need to update the docs to include that warning. In the mean time we will use a thin client pool as recommended in the Java documentation to see if it improves the performance. Well, in general yes, it should help you increase the performance. Also it’s worth to know how many server nodes do you have in a cluster? Is your data well-collocated? A thin client utilizes a single connection to a single node, but the requested data could be located at a different one, and thus could cause an additional network overhead. Please, refer to the affinity awareness wiki page [1]. [1] - https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients From: Eduard Llull Sent: Friday, August 23, 2019 9:49 AM To: user@ignite.apache.org Subject: .NET thin client multithreaded? Hello everyone, We have just developed a gRPC service in .NET core that performs a bunch of cache gets for every RPC. We've been using the Apache.NET nuGet starting the Ignite node in client mode (thick client) but we just changed it to use the thin client and we see much much worse response times: from 4ms average and 15ms 95th percentile, to 21ms average and 86ms 95th percentile, but the times get event worse under load: peaks of 115ms average, 1s 95th percentile. We were expecting some level of degradation in the response times when changeing from the thick to the thin client but not as much. In fact, trying to reduce the impact, we've deployed a Ignite node in client mode on every host where we have our gRPC service deployed and the gRPC service connects to the local Ignite node. The gRPC service receives several tens of concurrent requests when under load, but we instantiate one single ApacheClient (Ignition.StartClient()) shared by all the threads that are serving the RPC requests. I've seen in the Java Thin Client documentation (https://apacheignite.readme.io/docs/java-thin-client-initialization-and-configuration#section-multithreading) the following: Thin client is single-threaded and thread-safe. The only shared resource is the underlying communication channel, and only one thread reads/writes to the channel while other threads are waiting for the operation to complete. Use multiple threads with thin client connection pool to improve performance Presently thin client has no feature to create multiple threads to improve throughput. You can create multiple threads by getting thin client connection from a pool in your application to improve throughput. But there is no such warning in the .NET thin client documentation (https://apacheignite-net.readme.io/docs/thin-client). Is it possible that the huge increase in the reponse times comes from contention when multiple gRPC threads are using the same thin client (thus, the same ClientSocket) to communicate with the cluster? In the mean time we will use a thin client pool as recommended in the Java documentation to see if it improves the performance. Thank you very much.
