[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #14 from Markus --- OK, opened new one here: https://bugs.openldap.org/show_bug.cgi?id=10024 -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #13 from Howard Chu --- You'll have to provide more context for what you're trying to do. Note that the PREVSNAPSHOT flag only takes effect once and then is cleared. And probably you should open a new ticket since this one is already closed. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #12 from Markus --- I was just able to debug into this and thus gathered new info. Inside mdb_txn_renew0, I made some "odd" observations that I wanted to check with you. "Odd" in the sense that it seems that the meta page selection does not consider MDB_PREVSNAPSHOT. It entered mdb_txn_renew0 with MDB_TXN_RDONLY and ti (MDB_txninfo; env->me_txns) being non-NULL. However, ti->mti_txnid was 0 and thus txn->mt_txnid was set to 0. That's the reason for always selecting the first (index 0) meta page in this code line: meta = env->me_metas[txn->mt_txnid & 1]; This seems wrong but maybe I missed something? -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #11 from Quanah Gibson-Mount --- (In reply to opensou...@gmx-topmail.de from comment #10) > For clarification: we've seen the behavior without thread sanitizer too, > just not as frequently. Also, the code in question does not use multiple > threads. LMDB write ops are 100% deterministic. Having flaky results on some runs and not others cannot be due to LMDB code. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #10 from opensou...@gmx-topmail.de --- For clarification: we've seen the behavior without thread sanitizer too, just not as frequently. Also, the code in question does not use multiple threads. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #9 from Howard Chu --- (In reply to opensou...@gmx-topmail.de from comment #8) > After applying this patch, I observed some conditions in which > MDB_PREVSNAPSHOT does not work anymore: instead of the previous commit, the > current commit is taken. > > While the exact conditions when this happens is not 100% clear, these are > the observations: > > - Having the DB on a RAM disk has impact. On macOS, we observed that it is > always reproducible for one of our test case (straight forward test for > MDB_PREVSNAPSHOT but too much context required to share here). RAM disk was > created like that: > diskutil partitionDisk $(hdiutil attach -nomount ram://2048000) 1 GPTFormat > APFS 'ramdisk' '100%' > > - When building with thread sanitizer the issue was also observed on Linux. > It was reproducible in about half of the runs (flaky). Note that all writes in LMDB are fully serialized, which means there cannot possibly be any threading bugs in LMDB. Any thread sanitizer issues showing up indicates bugs in your calling app. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #8 from opensou...@gmx-topmail.de --- After applying this patch, I observed some conditions in which MDB_PREVSNAPSHOT does not work anymore: instead of the previous commit, the current commit is taken. While the exact conditions when this happens is not 100% clear, these are the observations: - Having the DB on a RAM disk has impact. On macOS, we observed that it is always reproducible for one of our test case (straight forward test for MDB_PREVSNAPSHOT but too much context required to share here). RAM disk was created like that: diskutil partitionDisk $(hdiutil attach -nomount ram://2048000) 1 GPTFormat APFS 'ramdisk' '100%' - When building with thread sanitizer the issue was also observed on Linux. It was reproducible in about half of the runs (flaky). - It happened also in regular builds, but very infrequent. I'm sorry that I cannot dig deeper at this moment and e.g. provide a patch. Still wanted to put these observations down in the hope that they might be useful for someone. Best, Markus -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 Quanah Gibson-Mount changed: What|Removed |Added Target Milestone|--- |1.0.0 -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #7 from Quanah Gibson-Mount --- mdb.master: Commits: • 4b615434 by Howard Chu at 2021-04-09T14:06:33+01:00 ITS#9496 fix mdb_env_open bug from #8704 mdb.master3: Commits: • 557ab606 by Howard Chu at 2021-04-09T14:12:41+01:00 ITS#9496 fix mdb_env_open bug from #8704 -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #6 from Howard Chu --- fixed in mdb.master, mdb.master3. not present in mdb.RE/0.9. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 Howard Chu changed: What|Removed |Added Resolution|--- |TEST Status|UNCONFIRMED |RESOLVED --- Comment #5 from Howard Chu --- (In reply to Nate Pierce from comment #4) > Created attachment 814 [details] > backs out a portion of af2f8cc814fabe2814cacb573be3338292f47c0d > > I don't know if this has any adverse effects - I don't know the lmdb > internals, but the two spots look related. After applying, all of the > existing tests still pass, and your supplied test does now too. Thanks for investigating. Looks like this is the right fix. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 Nate Pierce changed: What|Removed |Added CC||nwpie...@gmail.com --- Comment #4 from Nate Pierce --- Created attachment 814 --> https://bugs.openldap.org/attachment.cgi?id=814&action=edit backs out a portion of af2f8cc814fabe2814cacb573be3338292f47c0d I don't know if this has any adverse effects - I don't know the lmdb internals, but the two spots look related. After applying, all of the existing tests still pass, and your supplied test does now too. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #3 from Nate Pierce --- Anyone look into this yet? I made a script to back up through the commit history, rebuild mdb_dump and your test, and run it 10 times - as long as I was able to get it to lose a commit on at least one run, I kept rewinding. I cannot get your code to fail as of commit ce834559041747a8ae29884d2b82e144adc7600f. Everything at af2f8cc814fabe2814cacb573be3338292f47c0d or later will trigger it. Tested on Debian 10 (4.0.19-11) and MacOS 11.2.3. -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 ig...@github.com changed: What|Removed |Added Hardware|All |x86_64 OS|All |Linux -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #2 from ig...@github.com --- Created attachment 806 --> https://bugs.openldap.org/attachment.cgi?id=806&action=edit Script -- You are receiving this mail because: You are on the CC list for the issue.
[Issue 9496] Some writes missing from database
https://bugs.openldap.org/show_bug.cgi?id=9496 --- Comment #1 from ig...@github.com --- Created attachment 805 --> https://bugs.openldap.org/attachment.cgi?id=805&action=edit Test case -- You are receiving this mail because: You are on the CC list for the issue.
