Alexandr, If .WithExecute is not planned to be made available in the C# client, what is the plan to support custom thread pools from the C# side of things?
Thanks, Raymond. On Thu, Jul 18, 2019 at 9:28 AM Raymond Wilson <raymond_wil...@trimble.com> wrote: > The source of inbound requests into Server A is from client applications. > > Server B is really a cluster of servers that are performing clustered > transformations and computations across a data set. > > I originally used IComputeJob and similar functions which work very well > but have the restriction that they return the entire result set from a > Server B node in a single response. These result sets can be large (100's > of megabytes and larger), which makes life pretty hard for Server A if it > has to field multiple incoming responses of this size. So, these types of > requests progressively send responses back (using Ignite messaging) to > Server A using the Ignite messaging fabric. As Server A receives each part > of the overall response it processes it according the business rules > relevant to the request. > > The cluster config and numbers of nodes are not really material to this. > > Raymond. > > On Thu, Jul 18, 2019 at 12:26 AM Alexandr Shapkin <lexw...@gmail.com> > wrote: > >> Hi, >> >> >> >> Can you share a more detailed use case, please? >> >> >> >> Right now it’s not clear why do you need a messaging fabric. >> >> If you are interesting in a progress tracking, then you could try a >> CacheAPI or QueryContinious, for example. >> >> >> >> What are the sources of inbound requests? Is it a client requests? >> >> >> >> What is your cluster config? How many nodes do you have for your >> distributed computations? >> >> >> >> *From: *Raymond Wilson <raymond_wil...@trimble.com> >> *Sent: *Wednesday, July 17, 2019 1:49 PM >> *To: *user <user@ignite.apache.org> >> *Subject: *Re: Threadpools and .WithExecute() for C# clients >> >> >> >> Hi Alexandr, >> >> >> >> To summarise from the original thread, say I have server A that accepts >> requests. It contacts server B in order to help processing those requests. >> Server B sends in-progress results to server A using the Ignite messaging >> fabric. If the thread pool in server A is saturated with inbound requests, >> then there are no available threads to service the messaging fabric traffic >> from server B to server A resulting in a deadlock condition. >> >> >> >> In the original discussion it was suggested creating a custom thread pool >> to handle the Server B to Server A traffic would resolve it. >> >> >> >> Thanks, >> >> Raymond. >> >> >> >> On Wed, Jul 17, 2019 at 9:48 PM Alexandr Shapkin <lexw...@gmail.com> >> wrote: >> >> Hi, Raymond! >> >> >> >> As far as I can see, there are no plans for porting custom executors >> configuration in .NET client right now [1]. >> >> >> >> Please, remind, why do you need a separate pool instead of a default >> PublicPool? >> >> >> >> [1] - https://issues.apache.org/jira/browse/IGNITE-6566 >> >> >> >> >> >> >> >> *From: *Raymond Wilson <raymond_wil...@trimble.com> >> *Sent: *Wednesday, July 17, 2019 10:58 AM >> *To: *user <user@ignite.apache.org> >> *Subject: *Threadpools and .WithExecute() for C# clients >> >> >> >> Some time ago I ran into and issue with thread pool exhaustion and >> deadlocking in AI 2.2. >> >> >> >> This is the original thread: >> http://apache-ignite-users.70518.x6.nabble.com/Possible-dead-lock-when-number-of-jobs-exceeds-thread-pool-td17262.html >> >> >> >> >> At the time .WithExecutor() was not implemented in the C# client so there >> was little option but to expand the size of the public thread pool >> sufficiently to prevent the deadlocking. >> >> >> >> We have been revisiting this issue and see that .WithExecutor() is not >> supported in the AI 2.7.5 client. >> >> >> >> Can this be supported in the C# client, or is there a workaround in the >> .Net environment? that does not require this capability? >> >> >> >> Thanks, >> >> Raymond. >> >> >> >> >> >> >> >
