nickva opened a new pull request, #5672:
URL: https://github.com/apache/couchdb/pull/5672
Previously, shard properties were duplicated across all Q*N shards in the
cache. If we needed to access them we loaded all the shards (from ets or disk),
and then immediately threw them all away except the first one.
To optimise and clean up properties put them in their own ?OPTS ets table.
Item lookup, updating, and cleanup mirrors the behavior of ?SHARDS.
There are a few other related optimisations and cleanups:
* In the `for_docid` function we calculated the hash twice: once, when we
calculated the `HashKey` for the ets selector, then again, in the
`load_shards_from_disk(DbName, DocId)` if we loaded shards from disk. To
optimise it, calculate the `HashKey` once and pass it on as
`load_shards_from_disk(DbName, HashKey)`.
* Previously, we didn't cache the properties for the shards dbs itself, so
add a way to do that. If shards db changes the changes feed will restart ,and
then the shards dbs properties will update again. These properties may be used
used in the `_all_docs` call for instance, so having it cached would help not
having to load it from disk.
* Remove functions which were not used anywhere, and stop exporting
functions which are used locally only: `mem3:engine/1`, `find_dirty_shards/0`,
`get_engine_opt/1`, `get_props_opt/1`, `get_shard_props/1`.
* For mem3 shards and opts ets tables, since there could be multiple
pending writers in different processes trying to update different entries, it
makes sense to also enable `{write_concurrency, auto}` for those public tables.
See: https://www.erlang.org/doc/apps/stdlib/ets#new_2_write_concurrency
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
