On Fri, Aug 31, 2012 at 11:53 AM, Daniel Dunbar <[email protected]> wrote: > On Fri, Aug 31, 2012 at 11:23 AM, David Blaikie <[email protected]> wrote: >> Daniel (& anyone else), >> >> Do you know if the documentation here ( >> http://llvm.org/docs/TestingGuide.html#rtcustom / >> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?view=diff&r1=36059&r2=36060 >> ) is still valid? It doesn't appear to be, since my change below >> doesn't do particularly interesting quoting/escaping gymnastics & >> seems to work fine. >> >> Any ideas? Should we update the documentation? Does anyone know >> what/when/how/why this changed? > > What is actually going on here is that the LLVM test suite and the > Clang test suite use a different style of test. > > The LLVM test suite is technically written using Tcl syntax, and the > Clang test suite uses shell syntax. The document in question is > referring to the LLVM test suite. > > The fact that the test suites use different styles is an unfortunate > and poorly documented historical accident. Ideally we would kill off > the Tcl style and only use the shell style.
I see - I think I'd heard some of that in various discussions. Where is the option for shell V tcl specified? > Also, what is a "lesser lit-like" test runner? In this case, it's a shell script that attempts to transform RUN lines into a shell script to run a lit test - it works for the basic cases. - David > > - Daniel > >> >> Thanks, >> - David >> >> On Fri, Aug 31, 2012 at 10:49 AM, David Blaikie <[email protected]> wrote: >>> Author: dblaikie >>> Date: Fri Aug 31 12:49:33 2012 >>> New Revision: 163009 >>> >>> URL: http://llvm.org/viewvc/llvm-project?rev=163009&view=rev >>> Log: >>> Fix tests to be more robust (to older versions of grep, lesser lit-like >>> test runners, etc) >>> >>> Seems I had a problem with my version of grep, when run by lit, not >>> supporting >>> the \s escape. This seems to fix it for me & I'll be getting the buildbots >>> to >>> run these tests too to keep an eye on them (actually loop-convert tests >>> still >>> fail when run via a make build, so that'll be addressed in a future >>> commit). I >>> could use [[:space:]] to generalize over other whitespace but that seemed >>> unnecessarily verbose when the flexibility wasn't actually required by the >>> current text of the tests. >>> >>> Also I just simplified a lot of the loop-convert tests (removing the >>> unecessary temp file deletion at the start, removing the unnecessary && for >>> FileCheck, etc). >>> >>> The remove-cstr-calls/basic.cpp changes were necessitated by an out of tree >>> lit-like test runner that's a bit less fantastic about escaping. They were >>> modeled on existing tooling test cases in Clang, with thanks to Manuel >>> Klimek >>> for the pointers. >>> >>> Modified: >>> clang-tools-extra/trunk/test/loop-convert/array.cpp >>> clang-tools-extra/trunk/test/loop-convert/confidence.cpp >>> clang-tools-extra/trunk/test/loop-convert/dependency.cpp >>> clang-tools-extra/trunk/test/loop-convert/iterator.cpp >>> clang-tools-extra/trunk/test/loop-convert/naming.cpp >>> clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp >>> clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >>> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >>> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp >>> clang-tools-extra/trunk/test/loop-convert/negative.cpp >>> clang-tools-extra/trunk/test/loop-convert/nesting.cpp >>> clang-tools-extra/trunk/test/loop-convert/nocompile.cpp >>> clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp >>> clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp >>> clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/array.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/array.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/array.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/array.cpp Fri Aug 31 12:49:33 >>> 2012 >>> @@ -1,12 +1,11 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> // RUN: cp %t.cpp %t.base >>> -// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out \ >>> -// RUN: && FileCheck -check-prefix=COUNTONLY -input-file=%T/out %s >>> \ >>> -// RUN: && diff %t.cpp %t.base >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> +// RUN: cp %t.base %t.cpp >>> +// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out >>> +// RUN: FileCheck -check-prefix=COUNTONLY -input-file=%T/out %s >>> +// RUN: diff %t.cpp %t.base >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/confidence.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/confidence.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/confidence.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/confidence.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,9 +1,8 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> -// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs \ >>> -// RUN: && FileCheck -check-prefix=RISKY -input-file=%t.cpp %s >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> +// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs >>> +// RUN: FileCheck -check-prefix=RISKY -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/dependency.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/dependency.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/dependency.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/dependency.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,5 +1,4 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> // RUN: loop-convert . %t.cpp -- && FileCheck -input-file=%t.cpp %s >>> >>> void f() { >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/iterator.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/iterator.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/iterator.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,9 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: rm -rf %t.cpp >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/naming.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/naming.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/naming.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/naming.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,7 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp >>> (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp Fri Aug >>> 31 12:49:33 2012 >>> @@ -1,7 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: >>> clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >>> (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >>> Fri Aug 31 12:49:33 2012 >>> @@ -1,7 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: >>> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- >>> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >>> (original) >>> +++ >>> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >>> Fri Aug 31 12:49:33 2012 >>> @@ -1,7 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp >>> (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp Fri >>> Aug 31 12:49:33 2012 >>> @@ -1,7 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/negative.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/negative.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/negative.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,11 +1,9 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %S/Inputs/negative-header.h > \ >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: grep -Ev "// *[A-Z-]+:" %S/Inputs/negative-header.h > \ >>> // RUN: %T/negative-header.h >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs/ \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s \ >>> -// RUN: && FileCheck -input-file=%T/negative-header.h \ >>> -// RUN: %S/Inputs/negative-header.h >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs/ >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> +// RUN: FileCheck -input-file=%T/negative-header.h >>> %S/Inputs/negative-header.h >>> >>> #include "negative-header.h" >>> #include "structures.h" >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/nesting.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nesting.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/nesting.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/nesting.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,7 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/nocompile.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nocompile.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/nocompile.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/nocompile.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,8 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: || FileCheck -input-file=%t.cpp %s >>> -// Note that this test expects the compilation to fail! >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: not loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> void valid() { >>> const int arr[5]; >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,8 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> -// RUN: rm -rf %t.cpp >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> #include "structures.h" >>> >>> const int N = 6; >>> >>> Modified: clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp (original) >>> +++ clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp Fri Aug >>> 31 12:49:33 2012 >>> @@ -1,9 +1,6 @@ >>> -// RUN: rm -rf %t.cpp >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >>> -// RUN: && FileCheck -input-file=%t.cpp %s >>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >>> -// RUN: rm -rf %t.cpp >>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >>> +// RUN: FileCheck -input-file=%t.cpp %s >>> >>> #include "structures.h" >>> >>> >>> Modified: clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp >>> URL: >>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp?rev=163009&r1=163008&r2=163009&view=diff >>> ============================================================================== >>> --- clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp (original) >>> +++ clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp Fri Aug 31 >>> 12:49:33 2012 >>> @@ -1,7 +1,9 @@ >>> -// RUN: echo '[{"directory":".","command":"clang++ -c >>> %T/test.cpp","file":"%T/test.cpp"}]' > %T/compile_commands.json >>> -// RUN: cp "%s" "%T/test.cpp" >>> -// RUN: remove-cstr-calls "%T" "%T/test.cpp" >>> -// RUN: cat "%T/test.cpp" | FileCheck %s >>> +// RUN: rm -rf %t >>> +// RUN: mkdir %t >>> +// RUN: echo "[{\"directory\":\".\",\"command\":\"clang++ -c >>> %t/test.cpp\",\"file\":\"%t/test.cpp\"}]" | sed -e 's/\\/\//g' > >>> %t/compile_commands.json >>> +// RUN: cp "%s" "%t/test.cpp" >>> +// RUN: remove-cstr-calls "%t" "%t/test.cpp" >>> +// RUN: cat "%t/test.cpp" | FileCheck %s >>> // REQUIRES: shell >>> // FIXME: implement a mode for refactoring tools that takes input from >>> stdin >>> // and writes output to stdout for easier testing of tools. >>> >>> >>> _______________________________________________ >>> cfe-commits mailing list >>> [email protected] >>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits _______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
