jonasplaum opened a new pull request, #6182:
URL: https://github.com/apache/couchdb/pull/6182
## Overview
When a database is deleted, CouchDB tells Nouveau to delete the shard
directory of that database (`<shard>/<db>.<timestamp>`). The actual indexes
live one level deeper (`<shard>/<db>.<timestamp>/<sig>`).
`IndexManager.deleteAll` only closed an index whose name was exactly the
deleted path, so it deleted the files on disk, but the loaded indexes below it
stayed open in the cache.
Every deleted database whose index was loaded at that time keeps its index
writer open on deleted files. The disk space and file handles are only freed
when Nouveau restarts or the index is evicted from the cache, and for each of
these indexes that still had uncommitted changes Nouveau logs `I/O exception
while committing` with a `NoSuchFileException` every 30 seconds.
We now close every loaded index below a deleted path before deleting it.
## Testing recommendations
The change is covered by tests.
<details>
<summary>Manual Testing</summary>
**Start a dev cluster with Nouveau:**
```bash
dev/run -n 1 --with-nouveau -a adm:pass
```
Create a database with an index and some docs, and search it once so the
index is loaded:
```bash
curl -sS -X PUT \
"http://adm:[email protected]:15984/recreate_test?q=1";
curl -sS -X PUT \
"http://adm:[email protected]:15984/recreate_test/_design/foo"; \
-H "Content-Type: application/json" \
-d '{
"nouveau": {
"bar": {
"index": "function (doc) { if (doc.foo) { index(\"string\", \"foo\",
doc.foo, {store: true}); } }"
}
}
}'
curl -sS -X POST \
"http://adm:[email protected]:15984/recreate_test/_bulk_docs"; \
-H "Content-Type: application/json" \
-d '{"docs": [{"_id": "old1", "foo": "old"}, {"_id": "old2", "foo":
"old"}]}'
curl -sS
"http://adm:[email protected]:15984/recreate_test/_design/foo/_nouveau/bar?q=*:*";
```
Delete the database right after the search (within 30 s):
```bash
curl -sS -X DELETE "http://adm:[email protected]:15984/recreate_test";
```
Wait 30 seconds and look at the Nouveau log:
```bash
grep "I/O exception while committing" dev/logs/nouveau.log
```
Without the fix there is a new warning for the deleted index every 30
seconds.
Check which file handles and memory mapped files Nouveau still has open:
```bash
PID=$(pgrep -u "$(id -u)" -f '^[^ ]*java .*NouveauApplication')
ls -l /proc/$PID/fd | grep deleted
grep deleted /proc/$PID/maps | awk '{print $6, $7}' | sort -u
```
Without the fix this shows the write.lock and the segment files of the
deleted index. With the fix both commands print nothing.
</details>
## Checklist
- [x] This is my own work, I did not use AI, LLM's or similar technology
- [x] Code is written and works correctly
- [x] Changes are covered by tests
- [ ] Any new configurable parameters are documented in
`rel/overlay/etc/default.ini`
- [ ] Documentation changes were made in the `src/docs` folder
- [ ] Documentation changes were backported (separated PR) to affected
branches
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
