[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881252#comment-16881252
]
Stefan Egli commented on OAK-8351:
--
Note that this was primarily caused by
[SERVER-20616|https://jira.mongodb.org/browse/SERVER-20616] and that
[SERVER-42090|https://jira.mongodb.org/browse/SERVER-42090] and
[SERVER-24396|https://jira.mongodb.org/browse/SERVER-24396] would also have
been helpful preventing this.
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Fix For: 1.10.3, 1.16.0, 1.8.15
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch,
> OAK-8351-merged-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873346#comment-16873346
]
Stefan Egli commented on OAK-8351:
--
backport to 1.8 ready in appropriate svn merge format :
[^OAK-8351-merged-1.8.patch]
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_8
> Fix For: 1.10.3, 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch,
> OAK-8351-merged-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873151#comment-16873151
]
Julian Reschke commented on OAK-8351:
-
This should be set to "resolved" as it is fixed in trunk.
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873113#comment-16873113
]
Stefan Egli commented on OAK-8351:
--
Also a big dependent change missing in 1.8 is OAK-7360 where the new mongo java
API was made use of and eg switched from
{noformat}
return start()
.and(
start(SD_TYPE).in(gcTypeCodes).get(),
orClause.get(),
start(NodeDocument.SD_MAX_REV_TIME_IN_SECS)
.lessThan(NodeDocument.getModifiedInSecs(oldestRevTimeStamp))
.get()
).get();
{noformat}
to
{noformat}
return Filters.and(
Filters.in(SD_TYPE, gcTypeCodes),
Filters.or(orClauses),
Filters.lt(SD_MAX_REV_TIME_IN_SECS,
getModifiedInSecs(oldestRevTimeStamp))
);
{noformat}
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : Numbe
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873047#comment-16873047
]
Stefan Egli commented on OAK-8351:
--
right, 1.10 is straight forward but 1.8 was manual work (see
[^OAK-8351-1.8.patch] )
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873017#comment-16873017
]
Julian Reschke commented on OAK-8351:
-
I just checked: r1862044 has one conflict in 1.10, and that's just around
import statements and can be easily resolved.
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873016#comment-16873016
]
Julian Reschke commented on OAK-8351:
-
trunk: [r1862044|http://svn.apache.org/r1862044]
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872975#comment-16872975
]
Marcel Reutegger commented on OAK-8351:
---
bq. ported to 1.10 in this branch with the svn patch being attached as
OAK-8351-1.10.patch
Does this mean the changes do not merge cleanly to the 1.10 (and 1.8) branch?
If they don't, what are the changes they depend on?
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872432#comment-16872432
]
Stefan Egli commented on OAK-8351:
--
* leaving ticket open for couple days for potential backport feedback from the
list
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872430#comment-16872430
]
Stefan Egli commented on OAK-8351:
--
* ported to 1.8 in [this
branch|https://github.com/stefan-egli/jackrabbit-oak/tree/1.8] with the svn
patch being attached as [^OAK-8351-1.8.patch]
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Fix For: 1.16.0
>
> Attachments: OAK-8351-1.10.patch, OAK-8351-1.8.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872308#comment-16872308
]
Stefan Egli commented on OAK-8351:
--
* [announced|https://oak.markmail.org/thread/hp3ekgz3cdr54sad] intent to
backport on the list
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Attachments: OAK-8351-1.10.patch, OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872298#comment-16872298
]
Stefan Egli commented on OAK-8351:
--
* ported to 1.10 in [this
branch|https://github.com/stefan-egli/jackrabbit-oak/tree/1.10]
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Attachments: OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872197#comment-16872197
]
Stefan Egli commented on OAK-8351:
--
thx [~mreutegg]
* committed to trunk
[here|http://svn.apache.org/viewvc?view=revision&revision=1862044]
* pending: backports
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Labels: candidate_oak_1_10, candidate_oak_1_8
> Attachments: OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872052#comment-16872052
]
Marcel Reutegger commented on OAK-8351:
---
Looks good to me. Just one minor comment about the {{gc()}} method in the new
test. There's a {{Thread.sleep(100)}} before the garbage is collected. Is this
really necessary? The test is using a virtual clock and the tests pass on my
machine when I comment out the sleep.
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Attachments: OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866764#comment-16866764
]
Stefan Egli commented on OAK-8351:
--
Updated the
[branch|https://github.com/stefan-egli/jackrabbit-oak/tree/OAK-8351] and
attached the full resulting change set as an SVN [^OAK-8351.patch] for a
possible final review.
/cc [~mreutegg]
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Attachments: OAK-8351.patch
>
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866565#comment-16866565
]
Stefan Egli commented on OAK-8351:
--
Thx for the review [~mreutegg] ! I commented and will work through the changes
next
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
> },
>
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862813#comment-16862813
]
Marcel Reutegger commented on OAK-8351:
---
I reviewed the changes on the GitHub branch. See my comments on the two commits.
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
> },
>
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862175#comment-16862175
]
Stefan Egli commented on OAK-8351:
--
+1, agree, I'll look into that next
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-2\/0/
>
> },
>
> {
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862165#comment-16862165
]
Marcel Reutegger commented on OAK-8351:
---
Regarding tests that cover MongoVersionGCSupport. There are indeed only a
limited number of unit tests for this class. E.g. VersionGCSupportTest. The
class is tested additionally with VersionGarbageCollectorIT when the
DOCUMENT_NS fixture is enabled (or none at all).
The GitHub branch introduces a new test class MongoVersionGCTest, which is
MongoDB specific. Most of the new test code however is independent of the
underlying DocumentStore implementation. What do you think about moving the new
tests to VersionGarbageCollectorIT? This would allows us to run the tests also
on RDB.
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861190#comment-16861190
]
Stefan Egli commented on OAK-8351:
--
{quote}pending: test that covers MongoVersionGCSupport as there are currently
no tests that do verify if deleteSplitDocuments does anything
{quote}
done in mentioned
[branch|https://github.com/stefan-egli/jackrabbit-oak/tree/OAK-8351] now.
[~mreutegg], [~reschke], would you like to review the change? it's somewhat in
a critical area...
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16857811#comment-16857811
]
Stefan Egli commented on OAK-8351:
--
* working in [this
branch|https://github.com/stefan-egli/jackrabbit-oak/tree/OAK-8351]
* split existing (problematic) query into 2 simpler ones (done
[here|https://github.com/stefan-egli/jackrabbit-oak/commit/1ad89945c7ac0305d0d226ae1f3e1f091018a496])
* pending: test that covers MongoVersionGCSupport as there are currently no
tests that do verify if deleteSplitDocuments does anything
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
> }
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
[jira] [Commented] (OAK-8351) Long running RGC remove and getmore operations
[
https://issues.apache.org/jira/browse/OAK-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855496#comment-16855496
]
Stefan Egli commented on OAK-8351:
--
The reason as to why the unfortunate plan was generated in the first place is
still unclear.
However once it was generated the reason why it won is likely related to
https://jira.mongodb.org/browse/SERVER-20616
Potential solutions include:
* splitting up the query into 2 (one for the simple splitDocTypes
Default_Leaf(50 and Commit_Root_Only(60), and one for the more complex
Default_No_Branch(70))
* providing an index filter before running the find/deleteMany: the problem
here is that Oak might not know when mongod did a restart, thus would have to
set the index filter regularly and even then there might be a time window where
no index filter would be set yet)
* providing a hint: the problem here is that MongoDB doesn't take a hint for
deleteMany due to https://jira.mongodb.org/browse/SERVER-1599 - and we'd need
it for both find() and deleteMany().
> Long running RGC remove and getmore operations
> --
>
> Key: OAK-8351
> URL: https://issues.apache.org/jira/browse/OAK-8351
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Affects Versions: 1.12.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> On a mongodb setup a long running revision garbage collection operation has
> been witnessed. The query was running for hours. Doing a
> {{planCacheSetFilter}}, which hinted mongodb to use a specific index,
> together with killing the running command resolved the situation.
> The problem was that mongodb generated a query plan which scored high
> (2.0003) but included an index scan through the {{\_id_}} index (and the
> collection contained millions of documents). It also generated other, better,
> plans, but they all "only" had the same high score, so it seemed legitimate
> that mongodb would choose this one.
> The reason why this, problematic, query plan resulted in a high score seems
> to be that it does indeed find 101 documents after entering the first "or" -
> but during query execution it would also enter the other "or" parts where it
> has chosen to do a {{\_id_}} index scan.
> The query involved was:
> {noformat}
> {
> "_sdType" : {
> "$in" : [
>
> 50,
>
> 60,
>
> 70
> ]
> },
> "$or" : [
> {
>
> "_sdType" : 50
> },
> {
>
> "_sdType" : 60
> },
> {
>
> "_sdType" : 70,
>
> "$or" : [
>
> {
>
> "_id" : /.*-1\/0/
>
> },
>
> {
>
> "_id" : /[^-]*/,
>
> "_path" : /.*-1\/0/
>
> }
>
> ],
>
> "_sdMaxRevTime" : {
>
> "$lt" : NumberLong(1551843365)
>
