No, there still will be a case when BinaryObjectOffHeapImpl is put into intermediate result set in a query, but concurrently removed and deallocated from cache and index. Probably we could apply some fancy technics like reference counting here, but I'm not sure.
Sergi 2016-04-21 22:59 GMT+03:00 Denis Magda <dma...@gridgain.com>: > Sergi, > > However if we implement non snapshotable data structure to hold indexes > (skip list or b-tree) then it will be feasible to optimize point 1) - > working with off-heap data with BinaryObjectOffHeapImpl. Am I right? > > — > Denis > > On Apr 21, 2016, at 5:27 PM, Sergi Vladykin <sergi.vlady...@gmail.com> > wrote: > > 1. In general it is not possible to keep BinaryObject in "attached" > state, because at the query end we release snapshots and the entry can be > concurrently removed with offheap memory deallocation, while result row is > still needed. > > 2. Of course there are multiple alternatives (skiplist or b+tree), but > we'll need to implement them over offheap memory, this will take some > time... > > Sergi > > > Sergi > > 2016-04-21 16:48 GMT+03:00 Denis Magda <dma...@gridgain.com>: > >> Sergi, Igniters, >> >> Big deployments store enormous amount of data in off-heap. If to keep >> using such deployments under high load executing SQL queries and basic >> cache operations in parallel then it’s easy to see that Java heap pressure >> grows due to the bigger number of short live objects creation. >> >> I’ve been investigating Ignite code and noted the following >> >> 1) During SQL query execution off-heap data is unswapped and stored in >> BinaryObjectImpl wrapper for columns comparison. Also as I see the same >> happens when an index is updated. >> >> 2) Off-heap index tree is snapshotable. Taking into account hight update >> and query rates, Java heap will be populated with much more short snapshots. >> >> Actually I think that we can try to lower the impact caused by 1) by >> trying reusing BinaryObjectOffHeapImpl that allows to access fields >> directly working with off-heap. What do you think, any obstacles? >> >> In regards to the second. Is there any alternatives to snapshotable tree >> in general? >> >> — >> Denis > > > >
