================
@@ -123,6 +156,107 @@ class IndexActionFactory : public
tooling::FrontendActionFactory {
RelationSlab::Builder Relations;
};
+// Action factory that writes per-file shards (for sharded index format).
+// Each TU's index data is sharded independently — no merging across TUs.
+// Header shards are deduplicated: if a header's content hasn't changed since
+// the last time we wrote its shard, we skip writing it again.
+class ShardedIndexActionFactory : public tooling::FrontendActionFactory {
+public:
+ ShardedIndexActionFactory(BackgroundIndexStorage &Storage)
+ : Storage(Storage) {}
+
+ std::unique_ptr<FrontendAction> create() override {
+ // Snapshot the current shard versions so the callback can check staleness
+ // without holding the lock during indexing.
+ llvm::StringMap<ShardVersion> Snapshot;
+ {
+ std::lock_guard<std::mutex> Lock(ShardVersionsMu);
+ Snapshot = ShardVersions;
+ }
+
+ SymbolCollector::Options Opts;
+ Opts.CountReferences = true;
----------------
HighCommander4 wrote:
`CountReferences` should be false for a sharded index, because loading a
sharded index goes through `FileSymbols` which [does its
own](https://searchfox.org/llvm/rev/aff1ba103ab3006242c72af33420a3437f4ff1db/clang-tools-extra/clangd/index/FileIndex.cpp#320)
bookkeeping of reference counts, and indeed
[asserts](https://searchfox.org/llvm/rev/aff1ba103ab3006242c72af33420a3437f4ff1db/clang-tools-extra/clangd/index/FileIndex.cpp#307-308)
that the incoming shards have reference counts of zero.
On the other hand, there are two options we **should** be specifying here:
* `CollectMainFileRefs` should be set to `true`, to match the [background
indexer](https://searchfox.org/llvm/rev/aff1ba103ab3006242c72af33420a3437f4ff1db/clang-tools-extra/clangd/index/Background.cpp#308).
(The fact that the monolithic index doesn't use it seems like an oversight to
me; feel free to add it there too.)
* Like the background indexer, we should use a `FileFilter` which [skips
files](https://searchfox.org/llvm/rev/aff1ba103ab3006242c72af33420a3437f4ff1db/clang-tools-extra/clangd/index/Background.cpp#289-307)
whose contents haven't changed, using the `ShardVersions` machinery we already
have.
https://github.com/llvm/llvm-project/pull/175209
_______________________________________________
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
