Ilya, When will an official announcement and support-drop schedule be made about dropping IGFS?
Thank you, Chris On Tue, Aug 27, 2019 at 1:47 PM Chris Software <softwarechri...@gmail.com> wrote: > I see. Thank you. > > On Tue, Aug 27, 2019 at 12:30 PM Ilya Kasnacheev < > ilya.kasnach...@gmail.com> wrote: > >> Hello! >> >> This looks like a mistake. However, we're going to drop IGFS so the fix >> is unlikely to be expected. >> >> The recommended practical approach is to increase number of threads in >> system thread pool to large value. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> вт, 27 авг. 2019 г. в 00:34, Chris Software <softwarechri...@gmail.com>: >> >>> Hello, >>> >>> I am working on a project and we have run into two related problems >>> while doing Map_Reduce on Ignite Filesystem Cache. >>> >>> We were originally on Ignite 2.6 but upgraded to 2.7.5 in an >>> unsuccessful bid to resolve the problem. >>> >>> We have a deadlock in our map-reduce process and have reproduced it at >>> https://github.com/csteppp/ignite-deadlock-issue in >>> https://github.com/csteppp/ignite-deadlock-issue/blob/master/src/test/java/testignite/MapReduceIgniteTest.java. >>> >>> >>> Basically, if you run the test (mvn test) it will deadlock and hang. We >>> have two IgfsTasks created and have set the SYS threadpool to size 2 for >>> demonstration purposes. Each IgfsTask sleeps and then writes to a file. >>> This causes a deadlock because: >>> 1. The IgfsTask is run in the SYS pool. >>> 2. The Igfs write action uses a separate thread in the SYS pool >>> 3. Then if there are no empty threads available, the whole system >>> hangs. >>> >>> First, shouldn't executeAsync execute the task in the PUBLIC pool? >>> Using the SYS pool seems unnecessarily risky, as we found it actually *locks >>> up an entire cluster of many ignite nodes *when it deadlocks. How do I >>> get it to use the PUBLIC pool? Also, since it is using the SYS pool, it >>> actually seems to execute this on the client. This is not obvious in this >>> test, but in my real cluster of 30 nodes, the client seems to be doing this >>> work, which is a problem. >>> >>> Second, is it bad form to open a file within a map-reduce? Even using >>> the public pool will not solve the inherent deadlock here--that one thread >>> is depending on another thread in the same thread pool. That's an inherent >>> risk. In our real process we open the file because we are performing file >>> transformations in the IgfsTask, and then writing the results out to temp >>> files in the cluster. In the end, we collate all the temp files. Is there >>> a better approach, or a safe way to open a file and write to it from within >>> a reduce? >>> >>> Thank you for your time! >>> >>> Chris >>> >>> >>>
