Vladimir Ozerov created IGNITE-6699: ---------------------------------------
Summary: Optimize client-side data streamer performance Key: IGNITE-6699 URL: https://issues.apache.org/jira/browse/IGNITE-6699 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Components: streaming Affects Versions: 2.3 Reporter: Vladimir Ozerov Fix For: 2.4 Currently if a user has several server nodes and a single client node with single thread pushing data to streamer, he will not be able to load data at maximum speed. On the other hand, if he start several data loading threads, throughput will increase. One of root causes of this is bad data streamer design. Method {{IgniteDataStreamer.addData(K, V)}} returns new feature for every operation, this is too fine grained approach. Also it generates a lot of garbage and causes contention on streamer internals. Proposed implementation flow: 1) Compare performance of {{addData(K, V)}} vs {{addData(Collection)}} methods from one thread in distributed environment. The latter should show considerably higher throughput. 2) Users should receive per-batch features, rather than per-key. 3) Try caching thread data in some collection until it is large enough to avoid contention and unnecessary allocations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)