[jira] [Commented] (IGNITE-11998) Fix DataPageScan for fragmented pages.
[
https://issues.apache.org/jira/browse/IGNITE-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17542075#comment-17542075
]
Maxim Muzafarov commented on IGNITE-11998:
--
The following notes must also be considered:
- the {{cacheId}} is not always been written to the data pages the
investigation required here;
- we can create a new FragmentedDataPageIO layer to identify the pages with
tails (a pageIo migration required);
> Fix DataPageScan for fragmented pages.
> --
>
> Key: IGNITE-11998
> URL: https://issues.apache.org/jira/browse/IGNITE-11998
> Project: Ignite
> Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Maxim Muzafarov
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Fragmented pages crash JVM when accessed by DataPageScan scanner/query
> optimized scanner. It happens when scanner accesses data in later chunk in
> fragmented entry but treats it like the first one, expecting length of the
> payload, which is absent and replaced with raw entry data.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (IGNITE-11998) Fix DataPageScan for fragmented pages.
[
https://issues.apache.org/jira/browse/IGNITE-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541980#comment-17541980
]
Maxim Muzafarov commented on IGNITE-11998:
--
h4. The inital proposal
Currently, during a full scan of a cache group partition (SqlQuery or
ScanQuery) all the data is read though the partition B-Tree and this in turn
leads to the _n(log n)_ complexity. For such a queries it may be necessary to
read all the data by sequential pages read directly from the partition file
which has the _n_ complexity and also the sequential file reads has some
benefits instead of random access file reads.
h4. The main issue
Accoring to the [Ignite Multi-Tier Storage - under the
hood|https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Multi-Tier+Storage+-+under+the+hood#IgniteMultiTierStorageunderthehood-Longobjects]
long objects are splitted on the several pages. For the pages which are
contain an entry tail there is no any dedicated page attribute or page header
flag to identify such a pages, however, such a pages have a link to an other
fragment or a entry head. These pages may only be accessed from the page which
contain the entry head.
h4. Current solution and benchmarks
_The double loop over the all partition pages. _
During the first loop we are reading all the pages and collecting references to
the other pages (reading entries are performed from the head to tail, writing
entries are preformed from the tail to head). On the second loop we are
building the list of pages that doesn't have a references on itself - and these
are the pages that containing the entries headers to be read.
||Data Page Scan||true||false||
|IgniteDataPageScanBenchmark|148848|179228|
|IgniteDataPageScanBenchmark|186917|166980|
|IgniteDataPageScanBenchmark|197114|175667|
h4. Possible solutions
An additional analysis and investigation required to perform the full partition
scan using only the one loop. We need to identify the fragmented pages with
entries tails:
- for such a pages we can write the {{freeSpace}}, {{directCounter}},
{{indirectCounter}} e.g. {{-1}} value (currently it's zero) and here we need
check the pds compatibility.
- almost the same issue with identifying fragmented pages are here -
IGNITE-12510
> Fix DataPageScan for fragmented pages.
> --
>
> Key: IGNITE-11998
> URL: https://issues.apache.org/jira/browse/IGNITE-11998
> Project: Ignite
> Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Maxim Muzafarov
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Fragmented pages crash JVM when accessed by DataPageScan scanner/query
> optimized scanner. It happens when scanner accesses data in later chunk in
> fragmented entry but treats it like the first one, expecting length of the
> payload, which is absent and replaced with raw entry data.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (IGNITE-11998) Fix DataPageScan for fragmented pages.
[ https://issues.apache.org/jira/browse/IGNITE-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242443#comment-17242443 ] Maxim Muzafarov commented on IGNITE-11998: -- [~ibessonov] Thank you! Yes, I'm currently working on it. > Fix DataPageScan for fragmented pages. > -- > > Key: IGNITE-11998 > URL: https://issues.apache.org/jira/browse/IGNITE-11998 > Project: Ignite > Issue Type: Bug >Reporter: Ivan Bessonov >Assignee: Maxim Muzafarov >Priority: Critical > Fix For: 2.10 > > > Fragmented pages crash JVM when accessed by DataPageScan scanner/query > optimized scanner. It happens when scanner accesses data in later chunk in > fragmented entry but treats it like the first one, expecting length of the > payload, which is absent and replaced with raw entry data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11998) Fix DataPageScan for fragmented pages.
[ https://issues.apache.org/jira/browse/IGNITE-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242332#comment-17242332 ] Ivan Bessonov commented on IGNITE-11998: Hi [~mmuzaf], just found out that this issue is on you now. DataPageScan optimization was disabled as a part of https://issues.apache.org/jira/browse/IGNITE-11982. It comes with a bunch of other fixes, which sucks. But you can still use it as a reference, many of those changes must be reapplied back. BTW, what's your plan? How are you going to fix it? > Fix DataPageScan for fragmented pages. > -- > > Key: IGNITE-11998 > URL: https://issues.apache.org/jira/browse/IGNITE-11998 > Project: Ignite > Issue Type: Bug >Reporter: Ivan Bessonov >Assignee: Maxim Muzafarov >Priority: Critical > Fix For: 2.10 > > > Fragmented pages crash JVM when accessed by DataPageScan scanner/query > optimized scanner. It happens when scanner accesses data in later chunk in > fragmented entry but treats it like the first one, expecting length of the > payload, which is absent and replaced with raw entry data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11998) Fix DataPageScan for fragmented pages.
[ https://issues.apache.org/jira/browse/IGNITE-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889420#comment-16889420 ] Dmitriy Govorukhin commented on IGNITE-11998: - Muted tests 5b2948bfc2d78eff04eb4d30425c8d18d6e6e26b > Fix DataPageScan for fragmented pages. > -- > > Key: IGNITE-11998 > URL: https://issues.apache.org/jira/browse/IGNITE-11998 > Project: Ignite > Issue Type: Bug >Reporter: Ivan Bessonov >Priority: Major > Fix For: 2.8 > > > Fragmented pages crash JVM when accessed by DataPageScan scanner/query > optimized scanner. It happens when scanner accesses data in later chunk in > fragmented entry but treats it like the first one, expecting length of the > payload, which is absent and replaced with raw entry data. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
