Re: Page Locking vs Entry-level Locking
Hi John, Entry level locks still exist. I've described locks coming from page memory, page memory and its locks is lower-level abstraction than entries level. Segment is part of region consist from number of pages. 1 region - * segments, 1 segment - * pages. Segment lock is performed when set of pages in this segment is changed (page rotated with disk - this process named page replacement, or new additional page is created - allocatePage). Segments intended to decrease contention to page sets, by default number of segments is coming from CPU count. Some info can be found here https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Regionandsegmentstructure Unfortunately there is no picture there about segments, but I hope some day it will be created. Sincerely, Dmitriy Pavlov вт, 13 февр. 2018 г. в 16:51, John Wilson : > Hi Pavlov, > > Thanks for your explanation. However, I still have these questions: > > >1. If locking is per page-level and there are no entry-level locks, >then why does the documentation here talk about having entry-level >transaction locks? https://apacheignite.readme.io/docs/transactions >2. I'm not clear with what "locking a durable memory region segment " >means, what is a segment? Does a page contain multiple segments or a >segment contains multiple pages? The mental picture I have is: a memory >region is divided into pages (which can be meta data pages, index, or data >pages). Index pages hold links to data page id and offset for key-value >pairs while data pages contain the actual key-value pairs. So, what really >is a segment and what does locking a memory segment means? I understanding >page-locks, what is segment lock? > > Appreciate your response!!! > > On Tue, Feb 13, 2018 at 2:32 AM, Dmitry Pavlov > wrote: > >> Hi John, >> >> 1. No, content modification require lock holding to page to provide >> consitency in multithreaded environment. >> 2. Page is locked for read before reading its content, and unlocked after. >> Same for lock for write for writting. 1 writer or N readers allowed for >> page. On unlock write lock, dirty flag for page may be set if there data >> was actually modified. >> 3. Lock has per-page basis, additional striping by any offest within page >> is not required accoring to tests. >> >> Only one contention is observed sometimes in high load test, it is >> contention of threads to lock to durable memory region segment. But this >> situation can be handled by setting concurrenclyLevel in >> DataStorageConfiguration. >> >> Sincerely, >> Dmitriy Pavlov >> >> >> вт, 13 февр. 2018 г. в 9:56, John Wilson : >> >> > Hi, >> > >> > Ignite documentation talks about entry-level locks and the page >> structure >> > has a LOCK_OFFSET that I assume is used to store tag info. I have these >> > questions. >> > >> >1. Does Ignite use a lock-free implementation to lock pages and/or >> >entries? >> >2. When is a page locked and when is it released? >> >3. When an entry is inserted/modified in a page, is the page locked >> >(forbidding other threads from inserting entries in the page)? or >> only >> > the >> >entry's offset is locked? (allowing other threads to insert/modify >> other >> >items) >> > >> > Thanks! >> > >> > >
Re: Page Locking vs Entry-level Locking
Hi Pavlov, Thanks for your explanation. However, I still have these questions: 1. If locking is per page-level and there are no entry-level locks, then why does the documentation here talk about having entry-level transaction locks? https://apacheignite.readme.io/docs/transactions 2. I'm not clear with what "locking a durable memory region segment " means, what is a segment? Does a page contain multiple segments or a segment contains multiple pages? The mental picture I have is: a memory region is divided into pages (which can be meta data pages, index, or data pages). Index pages hold links to data page id and offset for key-value pairs while data pages contain the actual key-value pairs. So, what really is a segment and what does locking a memory segment means? I understanding page-locks, what is segment lock? Appreciate your response!!! On Tue, Feb 13, 2018 at 2:32 AM, Dmitry Pavlov wrote: > Hi John, > > 1. No, content modification require lock holding to page to provide > consitency in multithreaded environment. > 2. Page is locked for read before reading its content, and unlocked after. > Same for lock for write for writting. 1 writer or N readers allowed for > page. On unlock write lock, dirty flag for page may be set if there data > was actually modified. > 3. Lock has per-page basis, additional striping by any offest within page > is not required accoring to tests. > > Only one contention is observed sometimes in high load test, it is > contention of threads to lock to durable memory region segment. But this > situation can be handled by setting concurrenclyLevel in > DataStorageConfiguration. > > Sincerely, > Dmitriy Pavlov > > > вт, 13 февр. 2018 г. в 9:56, John Wilson : > > > Hi, > > > > Ignite documentation talks about entry-level locks and the page structure > > has a LOCK_OFFSET that I assume is used to store tag info. I have these > > questions. > > > >1. Does Ignite use a lock-free implementation to lock pages and/or > >entries? > >2. When is a page locked and when is it released? > >3. When an entry is inserted/modified in a page, is the page locked > >(forbidding other threads from inserting entries in the page)? or only > > the > >entry's offset is locked? (allowing other threads to insert/modify > other > >items) > > > > Thanks! > > >
Re: Page Locking vs Entry-level Locking
Hi John, 1. No, content modification require lock holding to page to provide consitency in multithreaded environment. 2. Page is locked for read before reading its content, and unlocked after. Same for lock for write for writting. 1 writer or N readers allowed for page. On unlock write lock, dirty flag for page may be set if there data was actually modified. 3. Lock has per-page basis, additional striping by any offest within page is not required accoring to tests. Only one contention is observed sometimes in high load test, it is contention of threads to lock to durable memory region segment. But this situation can be handled by setting concurrenclyLevel in DataStorageConfiguration. Sincerely, Dmitriy Pavlov вт, 13 февр. 2018 г. в 9:56, John Wilson : > Hi, > > Ignite documentation talks about entry-level locks and the page structure > has a LOCK_OFFSET that I assume is used to store tag info. I have these > questions. > >1. Does Ignite use a lock-free implementation to lock pages and/or >entries? >2. When is a page locked and when is it released? >3. When an entry is inserted/modified in a page, is the page locked >(forbidding other threads from inserting entries in the page)? or only > the >entry's offset is locked? (allowing other threads to insert/modify other >items) > > Thanks! >
Page Locking vs Entry-level Locking
Hi, Ignite documentation talks about entry-level locks and the page structure has a LOCK_OFFSET that I assume is used to store tag info. I have these questions. 1. Does Ignite use a lock-free implementation to lock pages and/or entries? 2. When is a page locked and when is it released? 3. When an entry is inserted/modified in a page, is the page locked (forbidding other threads from inserting entries in the page)? or only the entry's offset is locked? (allowing other threads to insert/modify other items) Thanks!
