================
@@ -3647,6 +3702,153 @@ Examples matching error: "variable has incomplete type
'struct s'"
// expected-error-re {{variable has type 'struct {{(.*)}}'}}
// expected-error-re {{variable has type 'struct{{[[:space:]](.*)}}'}}
+Verifying the Directives
+~~~~~~~~~~~~~~~~~~~~~~~~
+Clang ``-cc1`` also has ``-verify-directives`` mode, which can be enabled
+alongside ``-verify`` to place additional restrictions on directives,
+and strives to improve readability of the expected compiler output
+for reviewers and future readers, but makes it a bit harder to write the test:
+
+- Diagnostic text specified in a directive has to be a full match.
+- Lexical order of directives has to match order in which diagnostics are
+ emitted.
+- Each directive can match exactly one diagnostic.
+- Wildcards (``*``) are not allowed in diagnostic location.
+
+It is recommended to write new tests with this mode enabled, but for some tests
+it's not possible. Typical problems and their solutions are listed below.
+
+- **Diagnostic is issued only sometimes.**
+ The cause of variance needs to be identified and captured. Typically it is
+ either a compiler option like language mode, in which case additional custom
+ prefix should be sufficient, or platform, in which case triple has to be
+ explicitly passed in command-line arguments.
+- **Template instantiations at the end of translation unit.**
+ Instantiations at the end of the TU cause associated diagnostics to appear
+ too late. Instantiations need to happen before the next directive, and when
+ it is possible, it still takes some creativity to achieve, especially in
+ C++98 mode. Typical solutions include explicit template instantiations,
+ manifestly constant-evaluated expressions, ``enable_if``, and removing
+ template parameters altogether.
+- **Analysis-based warnings.** Some warnings, like ``-Wunused``, are based
+ on CFG analysis and are emitted later than e.g. parser would emit them,
+ because some lookahead is required to collect all the necessary information
+ for such analysis. Not much can be done in this case.
+- **Lambdas.** When lambda is mentioned in diagnostic message, it is identified
+ by its source location, which is typically not salient for the test,
+ and would make it overspecified. In this case regex match can be used to skip
+ over source location.
+
+C++ Defect Report Tests
+^^^^^^^^^^^^^^^^^^^^^^^
+C++ Defect Report tests are placed in ``clang/test/CXX/drs`` directory, which
+consists of two types of files:
+
+- files where tests are grouped by 100 by issue number, e.g. ``cwg15xx.cpp``
+ ("big files");
+- files with individual tests or smaller groups of tests, e.g. ``cwg177x.cpp``
+ or ``cwg787.cpp``.
+
+C++ defect report tests are run in all language modes available via ``-std=``,
+except for modes with GNU extensions and ``c++03``, as it is synonymous to
+``c++98``. Exceptions, pedantic errors, and ``-verify-directives`` are enabled.
+
+Big files are where most of the tests end up being placed to save on process
+startup cost during test runs. Big files also serve as an index of tests:
+even if test is placed in its individual file, it's still mentioned in the
+big file where this test would be placed otherwise. Big files need all tests
+to work with the same set of compiler options specified in RUN lines, but
+sometimes tests need special provisions, in which case they should be placed
+in their own file. Typical reasons for that include:
+
+- **Test is checking AST dump or LLVM IR.** We don't yet know how
+ to concatenate FileCheck directives from multiple tests into a single file
+ without avoiding test interference.
+- **Test is not compatible with ``-verify-directives``.** For instance, some
+ tests can't be rewritten to prevent instantiations at the end of TU.
+- **Test needs C++20 modules.** Such tests require special RUN lines to compile
+ modules in the right order.
+- **Test prevents compiler from checking subsequent tests.** In some cases
+ that involve templates Clang refuses to recover from expected errors and will
+ skip over expected errors in subsequent tests.
+
+C++ defect report tests make heavy use of markers. Marker names need to
+be prefixed with ``cwgNNN-``, where NNN is number of the core issue being
+tested. This is especially needed in big files, because all markers share
+the same namespace.
+
+Some diagnostics are expected only in certain language modes. This in handled
+in two parts. First, C++ defect report tests share a common pool of custom
+prefixes:
+
+- ``expected`` for diagnostics issued in all language modes;
+- ``cxxNN`` (e.g. ``cxx98``) for diagnostics issued only in one language mode;
+- ``since-cxxNN`` (e.g. ``since-cxx11``) for diagnostics that are issued
+ in a certain language mode and all newer language modes;
+- ``cxxNN-MM`` (e.g. ``cxx98-14``) for diagnostics that appear in all language
+ modes within a certain range.
+
+Second, parts of tests that test features not univerally available in all
----------------
erichkeane wrote:
```suggestion
Second, parts of tests that test features not universally available in all
```
https://github.com/llvm/llvm-project/pull/179835
_______________________________________________
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
