+1 with Ivan, let`s store it in core product just because it looks like inalienable functionality and release cycle of extensions a little bit different.
>Anton, I disagree. > >1. This should be released with main distro. >2. This should not be abandoned. >3. There is not any release process in ignite-extensions. >4. Everything is working now and working good. > > >So lets do not do this :) > >пн, 6 дек. 2021 г. в 14:49, Anton Vinogradov < a...@apache.org >: > >> Let's move all GCC-related parts to ignite-extensions, release, and use >> them as a maven dependency. >> >> >> On Fri, Dec 3, 2021 at 1:08 PM Ivan Daschinsky < ivanda...@gmail.com > >> wrote: >> >> > Ok, TC suite is ready [1]. >> > If there is no objections, I will merge it soon. >> > >> > Possible concerns -- now it is required to install build-essentials and >> > libnuma-dev in order to build ignite on 64 bit linux. >> > I suppose that this is not a big deal, but maybe someone will contradict? >> > >> > >> > [1] -- >> > >> > >> >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_NumaAllocator/?mode=builds >> > >> > чт, 2 дек. 2021 г. в 12:03, Ivan Daschinsky < ivanda...@gmail.com >: >> > >> > > >> Our runs show about 7-10 speedup, >> > > Sorry, typo 7-10% speedup >> > > >> > > чт, 2 дек. 2021 г. в 12:01, Ivan Daschinsky < ivanda...@gmail.com >: >> > > >> > >> Andrey, thanks! >> > >> >> > >> This allocator can be tested on every NUMA system. >> > >> Our runs show about 7-10 speedup, if we use allocattor with >> interleaved >> > >> strategy + -XX:+UseNUMA. >> > >> But unfortunately our yardstick benches doesn't use offheap a lot, >> > >> usually above one Gb. >> > >> We trying to do more benches with real data and share them, possibly >> in >> > >> meetup. >> > >> >> > >> AFAIK, GG lab servers are two-sockets machines, aren't they? So it is >> > >> worth to run benches with a lot data on them, using >> > >> allocator with interleaved strategy (you can skip specifying numa >> nodes, >> > >> by default it will use all available) and use -XX:+UseNUMA jvm >> > >> flag. >> > >> >> > >> >> > >> >> > >> чт, 2 дек. 2021 г. в 11:48, Andrey Mashenkov < >> > andrey.mashen...@gmail.com >> > >> >: >> > >> >> > >>> Ivan, >> > >>> >> > >>> Great job. PR looks good. >> > >>> >> > >>> This allocator in interleaved mode and passing `-XX:+UseNUMA` flag to >> > jvm >> > >>> > show promising results on yardstick benches. Technically, G1 is >> not a >> > >>> numa >> > >>> > aware collector for java versions less than 14, but allocation of >> > heap >> > >>> in >> > >>> > interleaved mode shows good results even on java 11. >> > >>> >> > >>> Can you share benchmark results? >> > >>> I'm not sure I'll have an Optane on my notebook in a reasonable time >> ;) >> > >>> >> > >>> >> > >>> On Thu, Dec 2, 2021 at 10:41 AM Ivan Daschinsky < ivanda...@gmail.com >> > >> > >>> wrote: >> > >>> >> > >>> > Semyon D. and Maks T. -- thanks a lot for review. >> > >>> > >> > >>> > BTW, Igniters, I will appreciate all opinions and feedback. >> > >>> > >> > >>> > пн, 29 нояб. 2021 г. в 10:13, Ivan Daschinsky < >> ivanda...@apache.org >> > >: >> > >>> > >> > >>> > > Hi, igniters! >> > >>> > > >> > >>> > > There is not a big secret that nowadays NUMA is quite common in >> > >>> > > multiprocessor systems. >> > >>> > > And this memory architecture should be treated in specific ways. >> > >>> > > >> > >>> > > Support for NUMA is present in many commercial and open-source >> > >>> products. >> > >>> > > >> > >>> > > I've implemented a NUMA aware allocator for Apache Ignite [1] >> > >>> > > It is a JNI wrapper around `libnuma` and supports different >> > >>> allocation >> > >>> > > options. >> > >>> > > I.e. interleaved, local, interleved_mask and so on. For more >> > >>> information, >> > >>> > > see >> > >>> > > [2], [3]. >> > >>> > > This allocator in interleaved mode and passing `-XX:+UseNUMA` >> flag >> > >>> to jvm >> > >>> > > show promising results on yardstick benches. Technically, G1 is >> > not a >> > >>> > numa >> > >>> > > aware collector for java versions less than 14, but allocation of >> > >>> heap in >> > >>> > > interleaved mode shows good results even on java 11. >> > >>> > > >> > >>> > > Currently, all needed libraries and tools for building this >> module >> > >>> are >> > >>> > > available on TC agents >> > >>> > > setup of specific test suite is in progress [4] >> > >>> > > >> > >>> > > So I am asking for a review of my patch. >> > >>> > > >> > >>> > > [1] -- https://issues.apache.org/jira/browse/IGNITE-15922 >> > >>> > > [2] -- https://man7.org/linux/man-pages/man3/numa.3.html >> > >>> > > [3] -- https://man7.org/linux/man-pages/man2/mbind.2.html >> > >>> > > [4] -- https://issues.apache.org/jira/browse/IGNITE-15994 >> > >>> > > >> > >>> > >> > >>> > >> > >>> > -- >> > >>> > Sincerely yours, Ivan Daschinskiy >> > >>> > >> > >>> >> > >>> >> > >>> -- >> > >>> Best regards, >> > >>> Andrey V. Mashenkov >> > >>> >> > >> >> > >> >> > >> -- >> > >> Sincerely yours, Ivan Daschinskiy >> > >> >> > > >> > > >> > > -- >> > > Sincerely yours, Ivan Daschinskiy >> > > >> > >> > >> > -- >> > Sincerely yours, Ivan Daschinskiy >> > >> > >-- >Sincerely yours, Ivan Daschinskiy
