Ilya Kasnacheev created IGNITE-12751: ----------------------------------------
Summary: callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows Key: IGNITE-12751 URL: https://issues.apache.org/jira/browse/IGNITE-12751 Project: Ignite Issue Type: Bug Components: compute Affects Versions: 2.7.6 Reporter: Ilya Kasnacheev Attachments: word-count-reproducer.zip Please consider attached reproducer and linked report. {{compute.callAsync(jobs, reducer); Result [res=33, tookMs=81, jobs=5] //warm up Result [res=99, tookMs=21, jobs=15] Result [res=330, tookMs=22, jobs=50] Result [res=990, tookMs=57, jobs=150] Result [res=3300, tookMs=146, jobs=500] Result [res=9900, tookMs=231, jobs=1500] Result [res=33000, tookMs=840, jobs=5000] Result [res=99000, tookMs=6965, jobs=15000] Result [res=330000, tookMs=118394, jobs=50000]}} As soon jobs.size() grows past 5000, performance begins to degrade quadratically. I don't expect that it will be completely linear, but I would assume that it should stay linear-ish until size() hits at least 100000, given that we see clusters of 100 nodes and it's not unthinkable to expect 1000 jobs to be run on each node. 5000 jobs (which still give OK performance) / 100 nodes is just 50 jobs per node, which becomes limiting factor. Linked question also mentions OOM event, which may be caused of intermediate storage of (N^2) data. -- This message was sent by Atlassian Jira (v8.3.4#803005)