Thanks for reporting this. I think may be we can make reduce tests into 1/3. For example: We have: * gcc.target/riscv/rvv/base/vmand_mm-1.c: New test. * gcc.target/riscv/rvv/base/vmand_mm-2.c: New test. * gcc.target/riscv/rvv/base/vmand_mm-3.c: New test.
Maybe we can reduce it into one test: vmand_mm.c only. I will improve and reduce all intrinsic tests like this soon (I almost done all intrinsic in this week, next week I will do this soon). RVV intrinsics are really huge, this is the document: https://github.com/riscv-non-isa/rvv-intrinsic-doc/tree/master/auto-generated The testcases are directly come from LLVM (We just add assembler check into the test), they also have this amount of testcases and the just recently change them: https://reviews.llvm.org/D142697 https://reviews.llvm.org/D142644 Take a look at the changing LLVM patch, I am aggree with you ,the LLVM patch is quite huge and not easy to maintain. So...... I think I can reduce the tests into 1/3 of them in the next. But it's still very big (you can take a look at LLVM). Let's see whether kito has more comments about it. juzhe.zh...@rivai.ai From: Jakub Jelinek Date: 2023-02-16 17:38 To: juzhe.zhong CC: gcc-patches; kito.cheng; Jeff Law Subject: Re: [PATCH] RISC-V: Add vm* mask C api tests Hi! I see in the past few weeks you've added huge amounts of these tests du -shc *.target/riscv/*/ 34M gcc.target/riscv/rvv/ 28M g++.target/riscv/rvv/ 61M total and new are coming (nothing at all at this year's start). This is far larger than tests of any other architecture (i386 has 35M total, aarch64 31M total, arm 17M total, powerpc 12M total, everything else is even much smaller) but for the other architectures it has been decades of testsuite coverage for features added over the years. Rather than looking purely at size, I'm more worried about the content of the tests. Usually target testsuites include runtime tests whether particular intrinsics etc. behave correctly at runtime, plus some compile tests that they can be compiled with occassional scan-assembler* to mention a particular instruction appears, but in these cases the scan-assembler* covers the entire (albeit small) functions, which makes it IMHO a maintainance nightmare whenever one wants to change something important in the compiler. Take e.g. the recent Andreas Schwab's change to make -fasynchronous-unwind-tables the default on riscv, even that change required quite a few changes. My worry is that with these kind of tests changes like that will become much harder and some people will simply decide not to do such changes because having to adjust tens of thousands of tests even with some scripting would be a nightmare. Can't we do better than this? E.g. what is the difference between gcc.target/riscv/rvv/ and g++.target/riscv/rvv/ tests? Are the <riscv_vector.h> APIs so different between C and C++ that it needs to be tested twice? Even if so, we have the concept of c-c++-common tests, we could add c-c++-common.target and make riscv.exp handle it similarly to how e.g. C and C++ dg.exp handles those. How do you create these tests? If you use some generator for them, wouldn't it be better to include the generator in the testsuite and generate them on the fly? We already have a precedent for that, e.g. the gcc/testsuite/g*.dg/compat/struct-layout-1.exp testsuite has a generator program written in C that creates tests on the fly. Now, using something like that would have 2 advantages, it would be much easier for maintainance, if you do some global change in the compiler that affects those tests, just adjust a few spots in the generator instead of tweaking currently 6000 tests and counting. Even if you aren't using a generator to write these tests (that would be a lot of work then!), a question is if it couldn't be done by one, have say some file like gcc has *.def files all around to describe what you want to test and something that generates those. Just wanted to chime in before we have 10 times more of such tests and it will be too late to adjust... On Thu, Feb 16, 2023 at 11:36:19AM +0800, juzhe.zh...@rivai.ai wrote: > From: Ju-Zhe Zhong <juzhe.zh...@rivai.ai> > > gcc/testsuite/ChangeLog: > > * gcc.target/riscv/rvv/base/vmand_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmand_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmand_mm-3.c: New test. > * gcc.target/riscv/rvv/base/vmandn_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmandn_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmandn_mm-3.c: New test. > * gcc.target/riscv/rvv/base/vmclr_m_m-1.c: New test. > * gcc.target/riscv/rvv/base/vmclr_m_m-2.c: New test. > * gcc.target/riscv/rvv/base/vmclr_m_m-3.c: New test. > * gcc.target/riscv/rvv/base/vmmv_m_m-1.c: New test. > * gcc.target/riscv/rvv/base/vmmv_m_m-2.c: New test. > * gcc.target/riscv/rvv/base/vmmv_m_m-3.c: New test. > * gcc.target/riscv/rvv/base/vmnand_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmnand_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmnand_mm-3.c: New test. > * gcc.target/riscv/rvv/base/vmnor_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmnor_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmnor_mm-3.c: New test. > * gcc.target/riscv/rvv/base/vmnot_m_m-1.c: New test. > * gcc.target/riscv/rvv/base/vmnot_m_m-2.c: New test. > * gcc.target/riscv/rvv/base/vmnot_m_m-3.c: New test. > * gcc.target/riscv/rvv/base/vmor_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmor_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmor_mm-3.c: New test. > * gcc.target/riscv/rvv/base/vmorn_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmorn_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmorn_mm-3.c: New test. > * gcc.target/riscv/rvv/base/vmsbf_m_m-1.c: New test. > * gcc.target/riscv/rvv/base/vmsbf_m_m-2.c: New test. > * gcc.target/riscv/rvv/base/vmsbf_m_m-3.c: New test. > * gcc.target/riscv/rvv/base/vmsbf_m_mu-1.c: New test. > * gcc.target/riscv/rvv/base/vmsbf_m_mu-2.c: New test. > * gcc.target/riscv/rvv/base/vmsbf_m_mu-3.c: New test. > * gcc.target/riscv/rvv/base/vmset_m_m-1.c: New test. > * gcc.target/riscv/rvv/base/vmset_m_m-2.c: New test. > * gcc.target/riscv/rvv/base/vmset_m_m-3.c: New test. > * gcc.target/riscv/rvv/base/vmsif_m_m-1.c: New test. > * gcc.target/riscv/rvv/base/vmsif_m_m-2.c: New test. > * gcc.target/riscv/rvv/base/vmsif_m_m-3.c: New test. > * gcc.target/riscv/rvv/base/vmsif_m_mu-1.c: New test. > * gcc.target/riscv/rvv/base/vmsif_m_mu-2.c: New test. > * gcc.target/riscv/rvv/base/vmsif_m_mu-3.c: New test. > * gcc.target/riscv/rvv/base/vmsof_m_m-1.c: New test. > * gcc.target/riscv/rvv/base/vmsof_m_m-2.c: New test. > * gcc.target/riscv/rvv/base/vmsof_m_m-3.c: New test. > * gcc.target/riscv/rvv/base/vmsof_m_mu-1.c: New test. > * gcc.target/riscv/rvv/base/vmsof_m_mu-2.c: New test. > * gcc.target/riscv/rvv/base/vmsof_m_mu-3.c: New test. > * gcc.target/riscv/rvv/base/vmxnor_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmxnor_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmxnor_mm-3.c: New test. > * gcc.target/riscv/rvv/base/vmxor_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmxor_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmxor_mm-3.c: New test. Jakub
