https://github.com/naveen-seth created https://github.com/llvm/llvm-project/pull/169962
This patch is the first of two in refactoring Clang's dependency scanning tooling to remove its dependency on `clangDriver`. It separates `Tooling/DependencyScanningTool.cpp` from the rest of `clangDependencyScanning` and moves `clangDependencyScanning` out of `clangTooling` into its own library. No functional changes are introduced. The follow-up patch will restrict `clangDependencyScanning` to handling only `-cc1` command line inputs and will move all functionality related to handling driver commands into `clangTooling`. (`Tooling/DependencyScanningTool.cpp`). This is part of a broader effort to support driver-managed builds for compilations using C++ named modules and/or Clang modules. It is required for linking the dependency scanning tooling against the driver without introducing cyclic dependencies, which would otherwise cause build failures when dynamic linking is enabled. The RFC for this change can be found here: https://discourse.llvm.org/t/rfc-new-clangoptions-library-remove-dependency-on-clangdriver-from-clangfrontend-and-flangfrontend/88773?u=naveen-seth In particular, this change was motivated by [this](https://github.com/llvm/llvm-project/pull/152770#discussion_r2395998210) review comment. >From 65063e8f4927bd408b3f8356da9d2bfd5740c867 Mon Sep 17 00:00:00 2001 From: Naveen Seth Hanig <[email protected]> Date: Fri, 28 Nov 2025 21:21:48 +0100 Subject: [PATCH] [clang][deps] Separate clangDependencyScanning and DependencyScanningTool (NFC) This patch is the first of two in refactoring Clang's dependency scanning tooling to remove its dependency on clangDriver. It separates Tooling/DependencyScanningTool.cpp from the rest of clangDependencyScanning and moves clangDependencyScanning out of clangTooling into its own library. No functional changes are introduced. The follow-up patch will restrict clangDependencyScanning to handling only -cc1 command line inputs and move functionality related to handling driver commands into clangTooling (DependencyScanningTool.cpp). This is part of a broader effort to support driver-managed builds for compilations using C++ named modules and/or Clang modules. It is required for linking the dependency scanning tooling against the driver without introducing cyclic dependencies, which would otherwise cause build failures when dynamic linking is enabled. The RFC for this change can be found here: https://discourse.llvm.org/t/rfc-new-clangoptions-library-remove-dependency-on-clangdriver-from-clangfrontend-and-flangfrontend/88773?u=naveen-seth --- .../clangd/ScanningProjectModules.cpp | 10 +- .../DependencyScannerImpl.h | 7 +- .../DependencyScanningFilesystem.h | 10 +- .../DependencyScanningService.h | 14 +- .../DependencyScanningUtils.h | 166 ++++++++++++++++ .../DependencyScanningWorker.h | 14 +- .../DependencyScanning/InProcessModuleCache.h | 12 +- .../DependencyScanning/ModuleDepCollector.h | 23 ++- .../DependencyScanningTool.h | 187 +++--------------- clang/lib/CMakeLists.txt | 1 + .../DependencyScanning/CMakeLists.txt | 2 +- .../DependencyScannerImpl.cpp | 5 +- .../DependencyScanningFilesystem.cpp | 5 +- .../DependencyScanningService.cpp | 5 +- .../DependencyScanningUtils.cpp | 38 ++++ .../DependencyScanningWorker.cpp | 7 +- .../InProcessModuleCache.cpp | 5 +- .../DependencyScanning/ModuleDepCollector.cpp | 5 +- clang/lib/Tooling/CMakeLists.txt | 3 +- .../DependencyScanningTool.cpp | 31 +-- clang/tools/clang-scan-deps/ClangScanDeps.cpp | 10 +- clang/unittests/CMakeLists.txt | 1 + .../DependencyScanning/CMakeLists.txt | 11 ++ .../DependencyScanningFilesystemTest.cpp | 4 +- .../DependencyScanningWorkerTest.cpp | 97 +++++++++ clang/unittests/Tooling/CMakeLists.txt | 3 +- .../DependencyScannerTest.cpp | 88 +-------- 27 files changed, 417 insertions(+), 347 deletions(-) rename clang/{lib/Tooling => include/clang}/DependencyScanning/DependencyScannerImpl.h (97%) rename clang/include/clang/{Tooling => }/DependencyScanning/DependencyScanningFilesystem.h (98%) rename clang/include/clang/{Tooling => }/DependencyScanning/DependencyScanningService.h (89%) create mode 100644 clang/include/clang/DependencyScanning/DependencyScanningUtils.h rename clang/include/clang/{Tooling => }/DependencyScanning/DependencyScanningWorker.h (94%) rename clang/include/clang/{Tooling => }/DependencyScanning/InProcessModuleCache.h (75%) rename clang/include/clang/{Tooling => }/DependencyScanning/ModuleDepCollector.h (95%) rename clang/include/clang/Tooling/{DependencyScanning => }/DependencyScanningTool.h (51%) rename clang/lib/{Tooling => }/DependencyScanning/CMakeLists.txt (93%) rename clang/lib/{Tooling => }/DependencyScanning/DependencyScannerImpl.cpp (99%) rename clang/lib/{Tooling => }/DependencyScanning/DependencyScanningFilesystem.cpp (99%) rename clang/lib/{Tooling => }/DependencyScanning/DependencyScanningService.cpp (82%) create mode 100644 clang/lib/DependencyScanning/DependencyScanningUtils.cpp rename clang/lib/{Tooling => }/DependencyScanning/DependencyScanningWorker.cpp (97%) rename clang/lib/{Tooling => }/DependencyScanning/InProcessModuleCache.cpp (95%) rename clang/lib/{Tooling => }/DependencyScanning/ModuleDepCollector.cpp (99%) rename clang/lib/Tooling/{DependencyScanning => }/DependencyScanningTool.cpp (88%) create mode 100644 clang/unittests/DependencyScanning/CMakeLists.txt rename clang/unittests/{Tooling => }/DependencyScanning/DependencyScanningFilesystemTest.cpp (98%) create mode 100644 clang/unittests/DependencyScanning/DependencyScanningWorkerTest.cpp rename clang/unittests/Tooling/{DependencyScanning => }/DependencyScannerTest.cpp (78%) diff --git a/clang-tools-extra/clangd/ScanningProjectModules.cpp b/clang-tools-extra/clangd/ScanningProjectModules.cpp index 672e99632019d..6a21ad2920764 100644 --- a/clang-tools-extra/clangd/ScanningProjectModules.cpp +++ b/clang-tools-extra/clangd/ScanningProjectModules.cpp @@ -8,8 +8,8 @@ #include "ProjectModules.h" #include "support/Logger.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningService.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h" +#include "clang/DependencyScanning/DependencyScanningService.h" +#include "clang/Tooling/DependencyScanningTool.h" namespace clang::clangd { namespace { @@ -36,8 +36,8 @@ class ModuleDependencyScanner { std::shared_ptr<const clang::tooling::CompilationDatabase> CDB, const ThreadsafeFS &TFS) : CDB(CDB), TFS(TFS), - Service(tooling::dependencies::ScanningMode::CanonicalPreprocessing, - tooling::dependencies::ScanningOutputFormat::P1689) {} + Service(dependencies::ScanningMode::CanonicalPreprocessing, + dependencies::ScanningOutputFormat::P1689) {} /// The scanned modules dependency information for a specific source file. struct ModuleDependencyInfo { @@ -81,7 +81,7 @@ class ModuleDependencyScanner { // Whether the scanner has scanned the project globally. bool GlobalScanned = false; - clang::tooling::dependencies::DependencyScanningService Service; + clang::dependencies::DependencyScanningService Service; // TODO: Add a scanning cache. diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScannerImpl.h b/clang/include/clang/DependencyScanning/DependencyScannerImpl.h similarity index 97% rename from clang/lib/Tooling/DependencyScanning/DependencyScannerImpl.h rename to clang/include/clang/DependencyScanning/DependencyScannerImpl.h index b94d1b472f920..0a0808dd9b93e 100644 --- a/clang/lib/Tooling/DependencyScanning/DependencyScannerImpl.h +++ b/clang/include/clang/DependencyScanning/DependencyScannerImpl.h @@ -9,18 +9,18 @@ #ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNER_H #define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNER_H +#include "clang/DependencyScanning/DependencyScanningFilesystem.h" +#include "clang/DependencyScanning/ModuleDepCollector.h" #include "clang/Driver/Compilation.h" +#include "clang/Driver/Driver.h" #include "clang/Frontend/CompilerInstance.h" #include "clang/Frontend/CompilerInvocation.h" #include "clang/Frontend/TextDiagnosticPrinter.h" #include "clang/Serialization/ObjectFilePCHContainerReader.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h" -#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h" namespace clang { class DiagnosticConsumer; -namespace tooling { namespace dependencies { class DependencyScanningService; class DependencyScanningWorker; @@ -206,7 +206,6 @@ class CompilerInstanceWithContext { llvm::Error handleReturnStatus(bool Success); }; } // namespace dependencies -} // namespace tooling } // namespace clang #endif diff --git a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h b/clang/include/clang/DependencyScanning/DependencyScanningFilesystem.h similarity index 98% rename from clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h rename to clang/include/clang/DependencyScanning/DependencyScanningFilesystem.h index 2b21be7712693..a4516ff77509d 100644 --- a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h +++ b/clang/include/clang/DependencyScanning/DependencyScanningFilesystem.h @@ -1,4 +1,4 @@ -//===- DependencyScanningFilesystem.h - clang-scan-deps fs ===---*- C++ -*-===// +//===- DependencyScanningFilesystem.h - Optimized Scanning FS ---*- C++ -*-===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,8 +6,8 @@ // //===----------------------------------------------------------------------===// -#ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGFILESYSTEM_H -#define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGFILESYSTEM_H +#ifndef LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGFILESYSTEM_H +#define LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGFILESYSTEM_H #include "clang/Basic/LLVM.h" #include "clang/Lex/DependencyDirectivesScanner.h" @@ -21,7 +21,6 @@ #include <variant> namespace clang { -namespace tooling { namespace dependencies { using DependencyDirectivesTy = @@ -521,7 +520,6 @@ class DependencyScanningWorkerFilesystem }; } // end namespace dependencies -} // end namespace tooling } // end namespace clang -#endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGFILESYSTEM_H +#endif // LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGFILESYSTEM_H diff --git a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h b/clang/include/clang/DependencyScanning/DependencyScanningService.h similarity index 89% rename from clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h rename to clang/include/clang/DependencyScanning/DependencyScanningService.h index 4e97c7bc9f36e..96dd33c28cf5a 100644 --- a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h +++ b/clang/include/clang/DependencyScanning/DependencyScanningService.h @@ -1,4 +1,4 @@ -//===- DependencyScanningService.h - clang-scan-deps service ===-*- C++ -*-===// +//===- DependencyScanningService.h - Scanning Service -----------*- C++ -*-===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,16 +6,15 @@ // //===----------------------------------------------------------------------===// -#ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGSERVICE_H -#define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGSERVICE_H +#ifndef LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGSERVICE_H +#define LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGSERVICE_H -#include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h" -#include "clang/Tooling/DependencyScanning/InProcessModuleCache.h" +#include "clang/DependencyScanning/DependencyScanningFilesystem.h" +#include "clang/DependencyScanning/InProcessModuleCache.h" #include "llvm/ADT/BitmaskEnum.h" #include "llvm/Support/Chrono.h" namespace clang { -namespace tooling { namespace dependencies { /// The mode in which the dependency scanner will operate to find the @@ -125,7 +124,6 @@ class DependencyScanningService { }; } // end namespace dependencies -} // end namespace tooling } // end namespace clang -#endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGSERVICE_H +#endif // LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGSERVICE_H diff --git a/clang/include/clang/DependencyScanning/DependencyScanningUtils.h b/clang/include/clang/DependencyScanning/DependencyScanningUtils.h new file mode 100644 index 0000000000000..e2fb5ad3e5cf3 --- /dev/null +++ b/clang/include/clang/DependencyScanning/DependencyScanningUtils.h @@ -0,0 +1,166 @@ +//===- DependencyScanningUtils.h - Common Scanning Utilities ----*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGUTILS_H +#define LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGUTILS_H + +#include "clang/DependencyScanning/DependencyScannerImpl.h" +#include "clang/DependencyScanning/DependencyScanningWorker.h" +#include "clang/DependencyScanning/ModuleDepCollector.h" + +namespace clang { +namespace dependencies { + +/// Graph of modular dependencies. +using ModuleDepsGraph = std::vector<clang::dependencies::ModuleDeps>; + +/// The full dependencies and module graph for a specific input. +struct TranslationUnitDeps { + /// The graph of direct and transitive modular dependencies. + ModuleDepsGraph ModuleGraph; + + /// The identifier of the C++20 module this translation unit exports. + /// + /// If the translation unit is not a module then \c ID.ModuleName is empty. + clang::dependencies::ModuleID ID; + + /// A collection of absolute paths to files that this translation unit + /// directly depends on, not including transitive dependencies. + std::vector<std::string> FileDeps; + + /// A collection of prebuilt modules this translation unit directly depends + /// on, not including transitive dependencies. + std::vector<clang::dependencies::PrebuiltModuleDep> PrebuiltModuleDeps; + + /// A list of modules this translation unit directly depends on, not including + /// transitive dependencies. + /// + /// This may include modules with a different context hash when it can be + /// determined that the differences are benign for this compilation. + std::vector<clang::dependencies::ModuleID> ClangModuleDeps; + + /// A list of module names that are visible to this translation unit. This + /// includes both direct and transitive module dependencies. + std::vector<std::string> VisibleModules; + + /// A list of the C++20 named modules this translation unit depends on. + std::vector<std::string> NamedModuleDeps; + + /// The sequence of commands required to build the translation unit. Commands + /// should be executed in order. + /// + /// FIXME: If we add support for multi-arch builds in clang-scan-deps, we + /// should make the dependencies between commands explicit to enable parallel + /// builds of each architecture. + std::vector<clang::dependencies::Command> Commands; + + /// Deprecated driver command-line. This will be removed in a future version. + std::vector<std::string> DriverCommandLine; +}; + +class FullDependencyConsumer : public clang::dependencies::DependencyConsumer { +public: + FullDependencyConsumer( + const llvm::DenseSet<clang::dependencies::ModuleID> &AlreadySeen) + : AlreadySeen(AlreadySeen) {} + + void handleBuildCommand(clang::dependencies::Command Cmd) override { + Commands.push_back(std::move(Cmd)); + } + + void handleDependencyOutputOpts(const DependencyOutputOptions &) override {} + + void handleFileDependency(StringRef File) override { + Dependencies.push_back(std::string(File)); + } + + void handlePrebuiltModuleDependency( + clang::dependencies::PrebuiltModuleDep PMD) override { + PrebuiltModuleDeps.emplace_back(std::move(PMD)); + } + + void handleModuleDependency(clang::dependencies::ModuleDeps MD) override { + ClangModuleDeps[MD.ID] = std::move(MD); + } + + void handleDirectModuleDependency(clang::dependencies::ModuleID ID) override { + DirectModuleDeps.push_back(ID); + } + + void handleVisibleModule(std::string ModuleName) override { + VisibleModules.push_back(ModuleName); + } + + void handleContextHash(std::string Hash) override { + ContextHash = std::move(Hash); + } + + void handleProvidedAndRequiredStdCXXModules( + std::optional<clang::dependencies::P1689ModuleInfo> Provided, + std::vector<clang::dependencies::P1689ModuleInfo> Requires) override { + ModuleName = Provided ? Provided->ModuleName : ""; + llvm::transform(Requires, std::back_inserter(NamedModuleDeps), + [](const auto &Module) { return Module.ModuleName; }); + } + + TranslationUnitDeps takeTranslationUnitDeps(); + +private: + std::vector<std::string> Dependencies; + std::vector<clang::dependencies::PrebuiltModuleDep> PrebuiltModuleDeps; + llvm::MapVector<clang::dependencies::ModuleID, + clang::dependencies::ModuleDeps> + ClangModuleDeps; + std::string ModuleName; + std::vector<std::string> NamedModuleDeps; + std::vector<clang::dependencies::ModuleID> DirectModuleDeps; + std::vector<std::string> VisibleModules; + std::vector<clang::dependencies::Command> Commands; + std::string ContextHash; + const llvm::DenseSet<clang::dependencies::ModuleID> &AlreadySeen; +}; + +/// A callback to lookup module outputs for "-fmodule-file=", "-o" etc. +using LookupModuleOutputCallback = + llvm::function_ref<std::string(const clang::dependencies::ModuleDeps &, + clang::dependencies::ModuleOutputKind)>; + +/// A simple dependency action controller that uses a callback. If no callback +/// is provided, it is assumed that looking up module outputs is unreachable. +class CallbackActionController + : public clang::dependencies::DependencyActionController { +public: + virtual ~CallbackActionController(); + + static std::string + lookupUnreachableModuleOutput(const clang::dependencies::ModuleDeps &MD, + clang::dependencies::ModuleOutputKind Kind) { + llvm::report_fatal_error("unexpected call to lookupModuleOutput"); + }; + + CallbackActionController(LookupModuleOutputCallback LMO) + : LookupModuleOutput(std::move(LMO)) { + if (!LookupModuleOutput) { + LookupModuleOutput = lookupUnreachableModuleOutput; + } + } + + std::string + lookupModuleOutput(const clang::dependencies::ModuleDeps &MD, + clang::dependencies::ModuleOutputKind Kind) override { + return LookupModuleOutput(MD, Kind); + } + +private: + LookupModuleOutputCallback LookupModuleOutput; +}; + +} // end namespace dependencies +} // end namespace clang + +#endif // LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGUTILS_H diff --git a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h b/clang/include/clang/DependencyScanning/DependencyScanningWorker.h similarity index 94% rename from clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h rename to clang/include/clang/DependencyScanning/DependencyScanningWorker.h index e2c353a254bf3..9d3966c25414a 100644 --- a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h +++ b/clang/include/clang/DependencyScanning/DependencyScanningWorker.h @@ -1,4 +1,4 @@ -//===- DependencyScanningWorker.h - clang-scan-deps worker ===---*- C++ -*-===// +//===- DependencyScanningWorker.h - Thread-Safe Scanning Worker -*- C++ -*-===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,15 +6,15 @@ // //===----------------------------------------------------------------------===// -#ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGWORKER_H -#define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGWORKER_H +#ifndef LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGWORKER_H +#define LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGWORKER_H #include "clang/Basic/DiagnosticOptions.h" #include "clang/Basic/FileManager.h" #include "clang/Basic/LLVM.h" +#include "clang/DependencyScanning/DependencyScanningService.h" +#include "clang/DependencyScanning/ModuleDepCollector.h" #include "clang/Frontend/PCHContainerOperations.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningService.h" -#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h" #include "llvm/Support/Error.h" #include "llvm/Support/FileSystem.h" #include "llvm/Support/MemoryBufferRef.h" @@ -25,7 +25,6 @@ namespace clang { class DependencyOutputOptions; -namespace tooling { namespace dependencies { class DependencyScanningWorkerFilesystem; @@ -185,7 +184,6 @@ class DependencyScanningWorker { }; } // end namespace dependencies -} // end namespace tooling } // end namespace clang -#endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGWORKER_H +#endif // LLVM_CLANG_DEPENDENCYSCANNING_DEPENDENCYSCANNINGWORKER_H diff --git a/clang/include/clang/Tooling/DependencyScanning/InProcessModuleCache.h b/clang/include/clang/DependencyScanning/InProcessModuleCache.h similarity index 75% rename from clang/include/clang/Tooling/DependencyScanning/InProcessModuleCache.h rename to clang/include/clang/DependencyScanning/InProcessModuleCache.h index 213e60b39c199..c0e8f00b7fb59 100644 --- a/clang/include/clang/Tooling/DependencyScanning/InProcessModuleCache.h +++ b/clang/include/clang/DependencyScanning/InProcessModuleCache.h @@ -1,4 +1,4 @@ -//===----------------------------------------------------------------------===// +//===- InProcessModuleCache.h - Implicit Module Cache -----------*- C++ -*-===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,8 +6,8 @@ // //===----------------------------------------------------------------------===// -#ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_INPROCESSMODULECACHE_H -#define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_INPROCESSMODULECACHE_H +#ifndef LLVM_CLANG_DEPENDENCYSCANNING_INPROCESSMODULECACHE_H +#define LLVM_CLANG_DEPENDENCYSCANNING_INPROCESSMODULECACHE_H #include "clang/Serialization/ModuleCache.h" #include "llvm/ADT/StringMap.h" @@ -16,8 +16,8 @@ #include <shared_mutex> namespace clang { -namespace tooling { namespace dependencies { + struct ModuleCacheEntry { std::shared_mutex CompilationMutex; std::atomic<std::time_t> Timestamp = 0; @@ -30,8 +30,8 @@ struct ModuleCacheEntries { IntrusiveRefCntPtr<ModuleCache> makeInProcessModuleCache(ModuleCacheEntries &Entries); + } // namespace dependencies -} // namespace tooling } // namespace clang -#endif +#endif // LLVM_CLANG_DEPENDENCYSCANNING_INPROCESSMODULECACHE_H diff --git a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h b/clang/include/clang/DependencyScanning/ModuleDepCollector.h similarity index 95% rename from clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h rename to clang/include/clang/DependencyScanning/ModuleDepCollector.h index b0a91b60ff6da..0243f7abcbe10 100644 --- a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h +++ b/clang/include/clang/DependencyScanning/ModuleDepCollector.h @@ -6,18 +6,18 @@ // //===----------------------------------------------------------------------===// -#ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H -#define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H +#ifndef LLVM_CLANG_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H +#define LLVM_CLANG_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H #include "clang/Basic/LLVM.h" #include "clang/Basic/Module.h" #include "clang/Basic/SourceManager.h" +#include "clang/DependencyScanning/DependencyScanningService.h" #include "clang/Frontend/CompilerInvocation.h" #include "clang/Frontend/Utils.h" #include "clang/Lex/HeaderSearch.h" #include "clang/Lex/PPCallbacks.h" #include "clang/Serialization/ASTReader.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningService.h" #include "llvm/ADT/DenseMap.h" #include "llvm/ADT/Hashing.h" #include "llvm/ADT/StringSet.h" @@ -28,7 +28,6 @@ #include <variant> namespace clang { -namespace tooling { namespace dependencies { class DependencyActionController; @@ -109,7 +108,7 @@ struct ModuleID { std::tie(Other.ModuleName, Other.ContextHash); } - bool operator<(const ModuleID& Other) const { + bool operator<(const ModuleID &Other) const { return std::tie(ModuleName, ContextHash) < std::tie(Other.ModuleName, Other.ContextHash); } @@ -264,10 +263,11 @@ class ModuleDepCollectorPP final : public PPCallbacks { /// Traverses the affecting modules and updates \c MD with references to the /// parent \c ModuleDepCollector info. - void addAllAffectingClangModules(const Module *M, ModuleDeps &MD, + void + addAllAffectingClangModules(const Module *M, ModuleDeps &MD, llvm::DenseSet<const Module *> &AddedModules); void addAffectingClangModule(const Module *M, ModuleDeps &MD, - llvm::DenseSet<const Module *> &AddedModules); + llvm::DenseSet<const Module *> &AddedModules); /// Add discovered module dependency for the given module. void addOneModuleDep(const Module *M, const ModuleID ID, ModuleDeps &MD); @@ -406,16 +406,15 @@ bool areOptionsInStableDir(const ArrayRef<StringRef> Directories, const HeaderSearchOptions &HSOpts); } // end namespace dependencies -} // end namespace tooling } // end namespace clang namespace llvm { -inline hash_code hash_value(const clang::tooling::dependencies::ModuleID &ID) { +inline hash_code hash_value(const clang::dependencies::ModuleID &ID) { return hash_combine(ID.ModuleName, ID.ContextHash); } -template <> struct DenseMapInfo<clang::tooling::dependencies::ModuleID> { - using ModuleID = clang::tooling::dependencies::ModuleID; +template <> struct DenseMapInfo<clang::dependencies::ModuleID> { + using ModuleID = clang::dependencies::ModuleID; static inline ModuleID getEmptyKey() { return ModuleID{"", ""}; } static inline ModuleID getTombstoneKey() { return ModuleID{"~", "~"}; // ~ is not a valid module name or context hash @@ -427,4 +426,4 @@ template <> struct DenseMapInfo<clang::tooling::dependencies::ModuleID> { }; } // namespace llvm -#endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H +#endif // LLVM_CLANG_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H diff --git a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h b/clang/include/clang/Tooling/DependencyScanningTool.h similarity index 51% rename from clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h rename to clang/include/clang/Tooling/DependencyScanningTool.h index ed562f46cfdaa..8e03f6e949689 100644 --- a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h +++ b/clang/include/clang/Tooling/DependencyScanningTool.h @@ -6,12 +6,13 @@ // //===----------------------------------------------------------------------===// -#ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGTOOL_H -#define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGTOOL_H +#ifndef LLVM_CLANG_TOOLING_DEPENDENCYSCANNINGTOOL_H +#define LLVM_CLANG_TOOLING_DEPENDENCYSCANNINGTOOL_H -#include "clang/Tooling/DependencyScanning/DependencyScanningService.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h" -#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h" +#include "clang/DependencyScanning/DependencyScanningService.h" +#include "clang/DependencyScanning/DependencyScanningUtils.h" +#include "clang/DependencyScanning/DependencyScanningWorker.h" +#include "clang/DependencyScanning/ModuleDepCollector.h" #include "clang/Tooling/JSONCompilationDatabase.h" #include "llvm/ADT/DenseSet.h" #include "llvm/ADT/MapVector.h" @@ -25,61 +26,10 @@ namespace clang { namespace tooling { namespace dependencies { -/// A callback to lookup module outputs for "-fmodule-file=", "-o" etc. -using LookupModuleOutputCallback = - llvm::function_ref<std::string(const ModuleDeps &, ModuleOutputKind)>; - -/// Graph of modular dependencies. -using ModuleDepsGraph = std::vector<ModuleDeps>; - -/// The full dependencies and module graph for a specific input. -struct TranslationUnitDeps { - /// The graph of direct and transitive modular dependencies. - ModuleDepsGraph ModuleGraph; - - /// The identifier of the C++20 module this translation unit exports. - /// - /// If the translation unit is not a module then \c ID.ModuleName is empty. - ModuleID ID; - - /// A collection of absolute paths to files that this translation unit - /// directly depends on, not including transitive dependencies. - std::vector<std::string> FileDeps; - - /// A collection of prebuilt modules this translation unit directly depends - /// on, not including transitive dependencies. - std::vector<PrebuiltModuleDep> PrebuiltModuleDeps; - - /// A list of modules this translation unit directly depends on, not including - /// transitive dependencies. - /// - /// This may include modules with a different context hash when it can be - /// determined that the differences are benign for this compilation. - std::vector<ModuleID> ClangModuleDeps; - - /// A list of module names that are visible to this translation unit. This - /// includes both direct and transitive module dependencies. - std::vector<std::string> VisibleModules; - - /// A list of the C++20 named modules this translation unit depends on. - std::vector<std::string> NamedModuleDeps; - - /// The sequence of commands required to build the translation unit. Commands - /// should be executed in order. - /// - /// FIXME: If we add support for multi-arch builds in clang-scan-deps, we - /// should make the dependencies between commands explicit to enable parallel - /// builds of each architecture. - std::vector<Command> Commands; - - /// Deprecated driver command-line. This will be removed in a future version. - std::vector<std::string> DriverCommandLine; -}; - struct P1689Rule { std::string PrimaryOutput; - std::optional<P1689ModuleInfo> Provides; - std::vector<P1689ModuleInfo> Requires; + std::optional<clang::dependencies::P1689ModuleInfo> Provides; + std::vector<clang::dependencies::P1689ModuleInfo> Requires; }; /// The high-level implementation of the dependency discovery tool that runs on @@ -90,9 +40,10 @@ class DependencyScanningTool { /// /// @param Service The parent service. Must outlive the tool. /// @param FS The filesystem for the tool to use. Defaults to the physical FS. - DependencyScanningTool(DependencyScanningService &Service, - llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS = - llvm::vfs::createPhysicalFileSystem()); + DependencyScanningTool( + clang::dependencies::DependencyScanningService &Service, + llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS = + llvm::vfs::createPhysicalFileSystem()); /// Print out the dependency information into a string using the dependency /// file format that is specified in the options (-MD is the default) and @@ -145,10 +96,11 @@ class DependencyScanningTool { /// /// \returns a \c StringError with the diagnostic output if clang errors /// occurred, \c TranslationUnitDeps otherwise. - llvm::Expected<TranslationUnitDeps> getTranslationUnitDependencies( + llvm::Expected<clang::dependencies::TranslationUnitDeps> + getTranslationUnitDependencies( const std::vector<std::string> &CommandLine, StringRef CWD, - const llvm::DenseSet<ModuleID> &AlreadySeen, - LookupModuleOutputCallback LookupModuleOutput, + const llvm::DenseSet<clang::dependencies::ModuleID> &AlreadySeen, + clang::dependencies::LookupModuleOutputCallback LookupModuleOutput, std::optional<llvm::MemoryBufferRef> TUBuffer = std::nullopt); /// Given a compilation context specified via the Clang driver command-line, @@ -157,10 +109,12 @@ class DependencyScanningTool { /// TODO: this method should be removed as soon as Swift and our C-APIs adopt /// CompilerInstanceWithContext. We are keeping it here so that it is easier /// to coordinate with Swift and C-API changes. - llvm::Expected<TranslationUnitDeps> getModuleDependencies( + llvm::Expected<clang::dependencies::TranslationUnitDeps> + getModuleDependencies( StringRef ModuleName, const std::vector<std::string> &CommandLine, - StringRef CWD, const llvm::DenseSet<ModuleID> &AlreadySeen, - LookupModuleOutputCallback LookupModuleOutput); + StringRef CWD, + const llvm::DenseSet<clang::dependencies::ModuleID> &AlreadySeen, + clang::dependencies::LookupModuleOutputCallback LookupModuleOutput); /// The following three methods provide a new interface to perform /// by name dependency scan. The new interface's intention is to improve @@ -190,9 +144,11 @@ class DependencyScanningTool { /// arguments for dependencies. /// @return An instance of \c TranslationUnitDeps if the scan is successful. /// Otherwise it returns an error. - llvm::Expected<TranslationUnitDeps> computeDependenciesByNameWithContext( - StringRef ModuleName, const llvm::DenseSet<ModuleID> &AlreadySeen, - LookupModuleOutputCallback LookupModuleOutput); + llvm::Expected<clang::dependencies::TranslationUnitDeps> + computeDependenciesByNameWithContext( + StringRef ModuleName, + const llvm::DenseSet<clang::dependencies::ModuleID> &AlreadySeen, + clang::dependencies::LookupModuleOutputCallback LookupModuleOutput); /// @brief This method finializes the compiler instance. It finalizes the /// diagnostics and deletes the compiler instance. Call this method @@ -203,96 +159,13 @@ class DependencyScanningTool { llvm::vfs::FileSystem &getWorkerVFS() const { return Worker.getVFS(); } private: - DependencyScanningWorker Worker; -}; - -class FullDependencyConsumer : public DependencyConsumer { -public: - FullDependencyConsumer(const llvm::DenseSet<ModuleID> &AlreadySeen) - : AlreadySeen(AlreadySeen) {} - - void handleBuildCommand(Command Cmd) override { - Commands.push_back(std::move(Cmd)); - } - - void handleDependencyOutputOpts(const DependencyOutputOptions &) override {} - - void handleFileDependency(StringRef File) override { - Dependencies.push_back(std::string(File)); - } - - void handlePrebuiltModuleDependency(PrebuiltModuleDep PMD) override { - PrebuiltModuleDeps.emplace_back(std::move(PMD)); - } - - void handleModuleDependency(ModuleDeps MD) override { - ClangModuleDeps[MD.ID] = std::move(MD); - } - - void handleDirectModuleDependency(ModuleID ID) override { - DirectModuleDeps.push_back(ID); - } - - void handleVisibleModule(std::string ModuleName) override { - VisibleModules.push_back(ModuleName); - } - - void handleContextHash(std::string Hash) override { - ContextHash = std::move(Hash); - } - - void handleProvidedAndRequiredStdCXXModules( - std::optional<P1689ModuleInfo> Provided, - std::vector<P1689ModuleInfo> Requires) override { - ModuleName = Provided ? Provided->ModuleName : ""; - llvm::transform(Requires, std::back_inserter(NamedModuleDeps), - [](const auto &Module) { return Module.ModuleName; }); - } - - TranslationUnitDeps takeTranslationUnitDeps(); - -private: - std::vector<std::string> Dependencies; - std::vector<PrebuiltModuleDep> PrebuiltModuleDeps; - llvm::MapVector<ModuleID, ModuleDeps> ClangModuleDeps; - std::string ModuleName; - std::vector<std::string> NamedModuleDeps; - std::vector<ModuleID> DirectModuleDeps; - std::vector<std::string> VisibleModules; - std::vector<Command> Commands; - std::string ContextHash; - const llvm::DenseSet<ModuleID> &AlreadySeen; -}; - -/// A simple dependency action controller that uses a callback. If no callback -/// is provided, it is assumed that looking up module outputs is unreachable. -class CallbackActionController : public DependencyActionController { -public: - virtual ~CallbackActionController(); - - static std::string lookupUnreachableModuleOutput(const ModuleDeps &MD, - ModuleOutputKind Kind) { - llvm::report_fatal_error("unexpected call to lookupModuleOutput"); - }; - - CallbackActionController(LookupModuleOutputCallback LMO) - : LookupModuleOutput(std::move(LMO)) { - if (!LookupModuleOutput) { - LookupModuleOutput = lookupUnreachableModuleOutput; - } - } - - std::string lookupModuleOutput(const ModuleDeps &MD, - ModuleOutputKind Kind) override { - return LookupModuleOutput(MD, Kind); - } - -private: - LookupModuleOutputCallback LookupModuleOutput; + clang::dependencies::DependencyScanningWorker Worker; + std::unique_ptr<clang::dependencies::TextDiagnosticsPrinterWithOutput> + DiagPrinterWithOS; }; } // end namespace dependencies } // end namespace tooling } // end namespace clang -#endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGTOOL_H +#endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNINGTOOL_H diff --git a/clang/lib/CMakeLists.txt b/clang/lib/CMakeLists.txt index e90b009da606a..2fc69e4e4fa6f 100644 --- a/clang/lib/CMakeLists.txt +++ b/clang/lib/CMakeLists.txt @@ -18,6 +18,7 @@ add_subdirectory(Serialization) add_subdirectory(Frontend) add_subdirectory(FrontendTool) add_subdirectory(Tooling) +add_subdirectory(DependencyScanning) add_subdirectory(DirectoryWatcher) add_subdirectory(Index) add_subdirectory(IndexSerialization) diff --git a/clang/lib/Tooling/DependencyScanning/CMakeLists.txt b/clang/lib/DependencyScanning/CMakeLists.txt similarity index 93% rename from clang/lib/Tooling/DependencyScanning/CMakeLists.txt rename to clang/lib/DependencyScanning/CMakeLists.txt index 76bdc50097fff..2976f7c236f2e 100644 --- a/clang/lib/Tooling/DependencyScanning/CMakeLists.txt +++ b/clang/lib/DependencyScanning/CMakeLists.txt @@ -9,7 +9,7 @@ add_clang_library(clangDependencyScanning DependencyScanningFilesystem.cpp DependencyScanningService.cpp DependencyScanningWorker.cpp - DependencyScanningTool.cpp + DependencyScanningUtils.cpp DependencyScannerImpl.cpp InProcessModuleCache.cpp ModuleDepCollector.cpp diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScannerImpl.cpp b/clang/lib/DependencyScanning/DependencyScannerImpl.cpp similarity index 99% rename from clang/lib/Tooling/DependencyScanning/DependencyScannerImpl.cpp rename to clang/lib/DependencyScanning/DependencyScannerImpl.cpp index 657547d299abd..b17d6aec7263e 100644 --- a/clang/lib/Tooling/DependencyScanning/DependencyScannerImpl.cpp +++ b/clang/lib/DependencyScanning/DependencyScannerImpl.cpp @@ -6,17 +6,16 @@ // //===----------------------------------------------------------------------===// -#include "DependencyScannerImpl.h" +#include "clang/DependencyScanning/DependencyScannerImpl.h" #include "clang/Basic/DiagnosticFrontend.h" #include "clang/Basic/DiagnosticSerialization.h" +#include "clang/DependencyScanning/DependencyScanningWorker.h" #include "clang/Driver/Driver.h" #include "clang/Frontend/FrontendActions.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h" #include "llvm/ADT/ScopeExit.h" #include "llvm/TargetParser/Host.h" using namespace clang; -using namespace tooling; using namespace dependencies; namespace { diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp b/clang/lib/DependencyScanning/DependencyScanningFilesystem.cpp similarity index 99% rename from clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp rename to clang/lib/DependencyScanning/DependencyScanningFilesystem.cpp index 266944ee730cb..24a794e4a6a22 100644 --- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp +++ b/clang/lib/DependencyScanning/DependencyScanningFilesystem.cpp @@ -1,4 +1,4 @@ -//===- DependencyScanningFilesystem.cpp - clang-scan-deps fs --------------===// +//===- DependencyScanningFilesystem.cpp - Optimized Scanning FS -----------===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,13 +6,12 @@ // //===----------------------------------------------------------------------===// -#include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h" +#include "clang/DependencyScanning/DependencyScanningFilesystem.h" #include "llvm/Support/MemoryBuffer.h" #include "llvm/Support/Threading.h" #include <optional> using namespace clang; -using namespace tooling; using namespace dependencies; llvm::ErrorOr<DependencyScanningWorkerFilesystem::TentativeEntry> diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp b/clang/lib/DependencyScanning/DependencyScanningService.cpp similarity index 82% rename from clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp rename to clang/lib/DependencyScanning/DependencyScanningService.cpp index 7f40c99f07287..72f359e56d116 100644 --- a/clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp +++ b/clang/lib/DependencyScanning/DependencyScanningService.cpp @@ -1,4 +1,4 @@ -//===- DependencyScanningService.cpp - clang-scan-deps service ------------===// +//===- DependencyScanningService.cpp - Scanning Service -------------------===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,10 +6,9 @@ // //===----------------------------------------------------------------------===// -#include "clang/Tooling/DependencyScanning/DependencyScanningService.h" +#include "clang/DependencyScanning/DependencyScanningService.h" using namespace clang; -using namespace tooling; using namespace dependencies; DependencyScanningService::DependencyScanningService( diff --git a/clang/lib/DependencyScanning/DependencyScanningUtils.cpp b/clang/lib/DependencyScanning/DependencyScanningUtils.cpp new file mode 100644 index 0000000000000..e27c597a14fcc --- /dev/null +++ b/clang/lib/DependencyScanning/DependencyScanningUtils.cpp @@ -0,0 +1,38 @@ +//===- DependencyScanningUtils.cpp - Common Scanning Utilities ------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "clang/DependencyScanning/DependencyScanningUtils.h" + +using namespace clang; +using namespace dependencies; + +TranslationUnitDeps FullDependencyConsumer::takeTranslationUnitDeps() { + TranslationUnitDeps TU; + + TU.ID.ContextHash = std::move(ContextHash); + TU.ID.ModuleName = std::move(ModuleName); + TU.NamedModuleDeps = std::move(NamedModuleDeps); + TU.FileDeps = std::move(Dependencies); + TU.PrebuiltModuleDeps = std::move(PrebuiltModuleDeps); + TU.VisibleModules = std::move(VisibleModules); + TU.Commands = std::move(Commands); + + for (auto &&M : ClangModuleDeps) { + auto &MD = M.second; + // TODO: Avoid handleModuleDependency even being called for modules + // we've already seen. + if (AlreadySeen.count(M.first)) + continue; + TU.ModuleGraph.push_back(std::move(MD)); + } + TU.ClangModuleDeps = std::move(DirectModuleDeps); + + return TU; +} + +CallbackActionController::~CallbackActionController() {} diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp b/clang/lib/DependencyScanning/DependencyScanningWorker.cpp similarity index 97% rename from clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp rename to clang/lib/DependencyScanning/DependencyScanningWorker.cpp index 0bc17f9c80605..b22b0f456fd5c 100644 --- a/clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp +++ b/clang/lib/DependencyScanning/DependencyScanningWorker.cpp @@ -1,4 +1,4 @@ -//===- DependencyScanningWorker.cpp - clang-scan-deps worker --------------===// +//===- DependencyScanningWorker.cpp - Thread-Safe Scanning Worker ---------===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,14 +6,13 @@ // //===----------------------------------------------------------------------===// -#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h" -#include "DependencyScannerImpl.h" +#include "clang/DependencyScanning/DependencyScanningWorker.h" #include "clang/Basic/DiagnosticFrontend.h" +#include "clang/DependencyScanning/DependencyScannerImpl.h" #include "clang/Driver/Driver.h" #include "clang/Driver/Tool.h" using namespace clang; -using namespace tooling; using namespace dependencies; DependencyScanningWorker::DependencyScanningWorker( diff --git a/clang/lib/Tooling/DependencyScanning/InProcessModuleCache.cpp b/clang/lib/DependencyScanning/InProcessModuleCache.cpp similarity index 95% rename from clang/lib/Tooling/DependencyScanning/InProcessModuleCache.cpp rename to clang/lib/DependencyScanning/InProcessModuleCache.cpp index d1e543b438225..1dd2d34032a96 100644 --- a/clang/lib/Tooling/DependencyScanning/InProcessModuleCache.cpp +++ b/clang/lib/DependencyScanning/InProcessModuleCache.cpp @@ -1,4 +1,4 @@ -//===----------------------------------------------------------------------===// +//===- InProcessModuleCache.cpp - Implicit Module Cache ---------*- C++ -*-===// // // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. @@ -6,7 +6,7 @@ // //===----------------------------------------------------------------------===// -#include "clang/Tooling/DependencyScanning/InProcessModuleCache.h" +#include "clang/DependencyScanning/InProcessModuleCache.h" #include "clang/Serialization/InMemoryModuleCache.h" #include "llvm/Support/AdvisoryLock.h" @@ -15,7 +15,6 @@ #include <mutex> using namespace clang; -using namespace tooling; using namespace dependencies; namespace { diff --git a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp b/clang/lib/DependencyScanning/ModuleDepCollector.cpp similarity index 99% rename from clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp rename to clang/lib/DependencyScanning/ModuleDepCollector.cpp index 3a99f8c882b8f..39bd2e2ab0032 100644 --- a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp +++ b/clang/lib/DependencyScanning/ModuleDepCollector.cpp @@ -6,18 +6,17 @@ // //===----------------------------------------------------------------------===// -#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h" +#include "clang/DependencyScanning/ModuleDepCollector.h" #include "clang/Basic/MakeSupport.h" +#include "clang/DependencyScanning/DependencyScanningWorker.h" #include "clang/Frontend/CompilerInstance.h" #include "clang/Lex/Preprocessor.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h" #include "llvm/ADT/STLExtras.h" #include "llvm/Support/BLAKE3.h" #include <optional> using namespace clang; -using namespace tooling; using namespace dependencies; void ModuleDeps::forEachFileDep(llvm::function_ref<void(StringRef)> Cb) const { diff --git a/clang/lib/Tooling/CMakeLists.txt b/clang/lib/Tooling/CMakeLists.txt index faaa53276d0e6..0972ecb08437f 100644 --- a/clang/lib/Tooling/CMakeLists.txt +++ b/clang/lib/Tooling/CMakeLists.txt @@ -10,7 +10,6 @@ add_subdirectory(Inclusions) add_subdirectory(Refactoring) add_subdirectory(ASTDiff) add_subdirectory(Syntax) -add_subdirectory(DependencyScanning) add_subdirectory(Transformer) add_clang_library(clangTooling @@ -18,6 +17,7 @@ add_clang_library(clangTooling ArgumentsAdjusters.cpp CommonOptionsParser.cpp CompilationDatabase.cpp + DependencyScanningTool.cpp Execution.cpp ExpandResponseFilesCompilationDatabase.cpp FileMatchTrie.cpp @@ -39,6 +39,7 @@ add_clang_library(clangTooling clangAST clangASTMatchers clangBasic + clangDependencyScanning clangDriver clangOptions clangFormat diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp b/clang/lib/Tooling/DependencyScanningTool.cpp similarity index 88% rename from clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp rename to clang/lib/Tooling/DependencyScanningTool.cpp index a1f2db7a471be..e037420f4fcf2 100644 --- a/clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp +++ b/clang/lib/Tooling/DependencyScanningTool.cpp @@ -6,13 +6,14 @@ // //===----------------------------------------------------------------------===// -#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h" +#include "clang/Tooling/DependencyScanningTool.h" #include "clang/Frontend/Utils.h" #include <optional> using namespace clang; using namespace tooling; -using namespace dependencies; +using namespace clang::dependencies; +using namespace clang::tooling::dependencies; DependencyScanningTool::DependencyScanningTool( DependencyScanningService &Service, @@ -200,29 +201,3 @@ DependencyScanningTool::computeDependenciesByNameWithContext( llvm::Error DependencyScanningTool::finalizeCompilerInstanceWithContext() { return Worker.finalizeCompilerInstanceWithContextOrError(); } - -TranslationUnitDeps FullDependencyConsumer::takeTranslationUnitDeps() { - TranslationUnitDeps TU; - - TU.ID.ContextHash = std::move(ContextHash); - TU.ID.ModuleName = std::move(ModuleName); - TU.NamedModuleDeps = std::move(NamedModuleDeps); - TU.FileDeps = std::move(Dependencies); - TU.PrebuiltModuleDeps = std::move(PrebuiltModuleDeps); - TU.VisibleModules = std::move(VisibleModules); - TU.Commands = std::move(Commands); - - for (auto &&M : ClangModuleDeps) { - auto &MD = M.second; - // TODO: Avoid handleModuleDependency even being called for modules - // we've already seen. - if (AlreadySeen.count(M.first)) - continue; - TU.ModuleGraph.push_back(std::move(MD)); - } - TU.ClangModuleDeps = std::move(DirectModuleDeps); - - return TU; -} - -CallbackActionController::~CallbackActionController() {} diff --git a/clang/tools/clang-scan-deps/ClangScanDeps.cpp b/clang/tools/clang-scan-deps/ClangScanDeps.cpp index 5f5bf42df5e6b..984a51c915f45 100644 --- a/clang/tools/clang-scan-deps/ClangScanDeps.cpp +++ b/clang/tools/clang-scan-deps/ClangScanDeps.cpp @@ -6,14 +6,14 @@ // //===----------------------------------------------------------------------===// +#include "clang/DependencyScanning/DependencyScanningService.h" +#include "clang/DependencyScanning/DependencyScanningWorker.h" #include "clang/Driver/Compilation.h" #include "clang/Driver/Driver.h" #include "clang/Frontend/CompilerInstance.h" #include "clang/Frontend/TextDiagnosticPrinter.h" #include "clang/Tooling/CommonOptionsParser.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningService.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h" +#include "clang/Tooling/DependencyScanningTool.h" #include "clang/Tooling/JSONCompilationDatabase.h" #include "clang/Tooling/Tooling.h" #include "llvm/ADT/STLExtras.h" @@ -40,7 +40,9 @@ #include "Opts.inc" using namespace clang; -using namespace tooling::dependencies; +using namespace tooling; +using namespace clang::dependencies; +using namespace clang::tooling::dependencies; namespace { diff --git a/clang/unittests/CMakeLists.txt b/clang/unittests/CMakeLists.txt index 54c781a35c20c..438a5c4c2e711 100644 --- a/clang/unittests/CMakeLists.txt +++ b/clang/unittests/CMakeLists.txt @@ -79,6 +79,7 @@ add_subdirectory(Basic) add_subdirectory(Lex) add_subdirectory(Parse) add_subdirectory(Driver) +add_subdirectory(DependencyScanning) if(CLANG_ENABLE_STATIC_ANALYZER) add_subdirectory(Analysis) add_subdirectory(StaticAnalyzer) diff --git a/clang/unittests/DependencyScanning/CMakeLists.txt b/clang/unittests/DependencyScanning/CMakeLists.txt new file mode 100644 index 0000000000000..40425820d4d08 --- /dev/null +++ b/clang/unittests/DependencyScanning/CMakeLists.txt @@ -0,0 +1,11 @@ +add_clang_unittest(ClangDependencyScanningTests + DependencyScanningFilesystemTest.cpp + DependencyScanningWorkerTest.cpp + CLANG_LIBS + clangDependencyScanning + clangFrontend # For TextDiagnosticPrinter. + LLVM_COMPONENTS + ${LLVM_TARGETS_TO_BUILD} + Option + Support + ) diff --git a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp b/clang/unittests/DependencyScanning/DependencyScanningFilesystemTest.cpp similarity index 98% rename from clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp rename to clang/unittests/DependencyScanning/DependencyScanningFilesystemTest.cpp index cdb0ce2100d60..0e195411915aa 100644 --- a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp +++ b/clang/unittests/DependencyScanning/DependencyScanningFilesystemTest.cpp @@ -6,12 +6,12 @@ // //===----------------------------------------------------------------------===// -#include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h" +#include "clang/DependencyScanning/DependencyScanningFilesystem.h" #include "llvm/ADT/SmallString.h" #include "llvm/Support/VirtualFileSystem.h" #include "gtest/gtest.h" -using namespace clang::tooling::dependencies; +using namespace clang::dependencies; TEST(DependencyScanningFilesystem, OpenFileAndGetBufferRepeatedly) { auto InMemoryFS = llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>(); diff --git a/clang/unittests/DependencyScanning/DependencyScanningWorkerTest.cpp b/clang/unittests/DependencyScanning/DependencyScanningWorkerTest.cpp new file mode 100644 index 0000000000000..e6a5684b10cc9 --- /dev/null +++ b/clang/unittests/DependencyScanning/DependencyScanningWorkerTest.cpp @@ -0,0 +1,97 @@ +//===- DependencyScanningWorkerTest.cpp -----------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "clang/DependencyScanning/DependencyScanningWorker.h" +#include "clang/DependencyScanning/DependencyScanningUtils.h" +#include "llvm/Support/FormatVariadic.h" +#include "gtest/gtest.h" +#include <string> + +using namespace clang; +using namespace dependencies; + +TEST(DependencyScanner, ScanDepsWithDiagConsumer) { + StringRef CWD = "/root"; + + auto VFS = llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>(); + VFS->setCurrentWorkingDirectory(CWD); + auto Sept = llvm::sys::path::get_separator(); + std::string HeaderPath = + std::string(llvm::formatv("{0}root{0}header.h", Sept)); + std::string TestPath = std::string(llvm::formatv("{0}root{0}test.cpp", Sept)); + std::string AsmPath = std::string(llvm::formatv("{0}root{0}test.s", Sept)); + + VFS->addFile(HeaderPath, 0, llvm::MemoryBuffer::getMemBuffer("\n")); + VFS->addFile(TestPath, 0, + llvm::MemoryBuffer::getMemBuffer("#include \"header.h\"\n")); + VFS->addFile(AsmPath, 0, llvm::MemoryBuffer::getMemBuffer("")); + + DependencyScanningService Service(ScanningMode::DependencyDirectivesScan, + ScanningOutputFormat::Make); + DependencyScanningWorker Worker(Service, VFS); + + llvm::DenseSet<ModuleID> AlreadySeen; + FullDependencyConsumer DC(AlreadySeen); + CallbackActionController AC(nullptr); + + struct EnsureFinishedConsumer : public DiagnosticConsumer { + bool Finished = false; + void finish() override { Finished = true; } + }; + + { + // Check that a successful scan calls DiagConsumer.finish(). + std::vector<std::string> Args = {"clang", + "-target", + "x86_64-apple-macosx10.7", + "-c", + "test.cpp", + "-o" + "test.cpp.o"}; + + EnsureFinishedConsumer DiagConsumer; + bool Success = Worker.computeDependencies(CWD, Args, DC, AC, DiagConsumer); + + EXPECT_TRUE(Success); + EXPECT_EQ(DiagConsumer.getNumErrors(), 0u); + EXPECT_TRUE(DiagConsumer.Finished); + } + + { + // Check that an invalid command-line, which never enters the scanning + // action calls DiagConsumer.finish(). + std::vector<std::string> Args = {"clang", "-invalid-arg"}; + EnsureFinishedConsumer DiagConsumer; + bool Success = Worker.computeDependencies(CWD, Args, DC, AC, DiagConsumer); + + EXPECT_FALSE(Success); + EXPECT_GE(DiagConsumer.getNumErrors(), 1u); + EXPECT_TRUE(DiagConsumer.Finished); + } + + { + // Check that a valid command line that produces no scanning jobs calls + // DiagConsumer.finish(). + std::vector<std::string> Args = {"clang", + "-target", + "x86_64-apple-macosx10.7", + "-c", + "-x", + "assembler", + "test.s", + "-o" + "test.cpp.o"}; + + EnsureFinishedConsumer DiagConsumer; + bool Success = Worker.computeDependencies(CWD, Args, DC, AC, DiagConsumer); + + EXPECT_FALSE(Success); + EXPECT_EQ(DiagConsumer.getNumErrors(), 1u); + EXPECT_TRUE(DiagConsumer.Finished); + } +} diff --git a/clang/unittests/Tooling/CMakeLists.txt b/clang/unittests/Tooling/CMakeLists.txt index 106c6b9dc38bd..8c8b22250cd83 100644 --- a/clang/unittests/Tooling/CMakeLists.txt +++ b/clang/unittests/Tooling/CMakeLists.txt @@ -13,8 +13,7 @@ add_clang_unittest(ToolingTests LookupTest.cpp QualTypeNamesTest.cpp RangeSelectorTest.cpp - DependencyScanning/DependencyScannerTest.cpp - DependencyScanning/DependencyScanningFilesystemTest.cpp + DependencyScannerTest.cpp RecursiveASTVisitorTests/Attr.cpp RecursiveASTVisitorTests/BitfieldInitializer.cpp RecursiveASTVisitorTests/CallbacksLeaf.cpp diff --git a/clang/unittests/Tooling/DependencyScanning/DependencyScannerTest.cpp b/clang/unittests/Tooling/DependencyScannerTest.cpp similarity index 78% rename from clang/unittests/Tooling/DependencyScanning/DependencyScannerTest.cpp rename to clang/unittests/Tooling/DependencyScannerTest.cpp index 4523af33e3c28..9fcd0545b17fa 100644 --- a/clang/unittests/Tooling/DependencyScanning/DependencyScannerTest.cpp +++ b/clang/unittests/Tooling/DependencyScannerTest.cpp @@ -9,13 +9,13 @@ #include "clang/AST/ASTConsumer.h" #include "clang/AST/DeclCXX.h" #include "clang/AST/DeclGroup.h" +#include "clang/DependencyScanning/DependencyScanningWorker.h" #include "clang/Frontend/ASTUnit.h" #include "clang/Frontend/CompilerInstance.h" #include "clang/Frontend/FrontendAction.h" #include "clang/Frontend/FrontendActions.h" #include "clang/Tooling/CompilationDatabase.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h" -#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h" +#include "clang/Tooling/DependencyScanningTool.h" #include "clang/Tooling/Tooling.h" #include "llvm/ADT/STLExtras.h" #include "llvm/MC/TargetRegistry.h" @@ -29,7 +29,8 @@ using namespace clang; using namespace tooling; -using namespace dependencies; +using namespace clang::dependencies; +using namespace tooling::dependencies; namespace { @@ -304,84 +305,3 @@ TEST(DependencyScanner, ScanDepsWithModuleLookup) { EXPECT_TRUE(!llvm::is_contained(InterceptFS->StatPaths, OtherPath)); EXPECT_EQ(InterceptFS->ReadFiles, std::vector<std::string>{"test.m"}); } - -TEST(DependencyScanner, ScanDepsWithDiagConsumer) { - StringRef CWD = "/root"; - - auto VFS = llvm::makeIntrusiveRefCnt<llvm::vfs::InMemoryFileSystem>(); - VFS->setCurrentWorkingDirectory(CWD); - auto Sept = llvm::sys::path::get_separator(); - std::string HeaderPath = - std::string(llvm::formatv("{0}root{0}header.h", Sept)); - std::string TestPath = std::string(llvm::formatv("{0}root{0}test.cpp", Sept)); - std::string AsmPath = std::string(llvm::formatv("{0}root{0}test.s", Sept)); - - VFS->addFile(HeaderPath, 0, llvm::MemoryBuffer::getMemBuffer("\n")); - VFS->addFile(TestPath, 0, - llvm::MemoryBuffer::getMemBuffer("#include \"header.h\"\n")); - VFS->addFile(AsmPath, 0, llvm::MemoryBuffer::getMemBuffer("")); - - DependencyScanningService Service(ScanningMode::DependencyDirectivesScan, - ScanningOutputFormat::Make); - DependencyScanningWorker Worker(Service, VFS); - - llvm::DenseSet<ModuleID> AlreadySeen; - FullDependencyConsumer DC(AlreadySeen); - CallbackActionController AC(nullptr); - - struct EnsureFinishedConsumer : public DiagnosticConsumer { - bool Finished = false; - void finish() override { Finished = true; } - }; - - { - // Check that a successful scan calls DiagConsumer.finish(). - std::vector<std::string> Args = {"clang", - "-target", - "x86_64-apple-macosx10.7", - "-c", - "test.cpp", - "-o" - "test.cpp.o"}; - - EnsureFinishedConsumer DiagConsumer; - bool Success = Worker.computeDependencies(CWD, Args, DC, AC, DiagConsumer); - - EXPECT_TRUE(Success); - EXPECT_EQ(DiagConsumer.getNumErrors(), 0u); - EXPECT_TRUE(DiagConsumer.Finished); - } - - { - // Check that an invalid command-line, which never enters the scanning - // action calls DiagConsumer.finish(). - std::vector<std::string> Args = {"clang", "-invalid-arg"}; - EnsureFinishedConsumer DiagConsumer; - bool Success = Worker.computeDependencies(CWD, Args, DC, AC, DiagConsumer); - - EXPECT_FALSE(Success); - EXPECT_GE(DiagConsumer.getNumErrors(), 1u); - EXPECT_TRUE(DiagConsumer.Finished); - } - - { - // Check that a valid command line that produces no scanning jobs calls - // DiagConsumer.finish(). - std::vector<std::string> Args = {"clang", - "-target", - "x86_64-apple-macosx10.7", - "-c", - "-x", - "assembler", - "test.s", - "-o" - "test.cpp.o"}; - - EnsureFinishedConsumer DiagConsumer; - bool Success = Worker.computeDependencies(CWD, Args, DC, AC, DiagConsumer); - - EXPECT_FALSE(Success); - EXPECT_EQ(DiagConsumer.getNumErrors(), 1u); - EXPECT_TRUE(DiagConsumer.Finished); - } -} _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
