Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Control: severity 873508 normal Control: retitle 873508 Fix FTBFS for m68k, hurd, x32 and ppc64 On Thu, Sep 21, 2017 at 08:58:23PM +0200, Uwe Kleine-König wrote: > I still expect some platforms to fail with the wrapper, too, because > cgcc doesn't know about all platforms yet. But I intend to upload a new > sparse package soon that includes a build time check for that, and the > respective fixes are easy. I did that now, and at least all official Debian ports pass that new check. I downgraded the severity accordingly to normal. Current build failures can be seen at https://buildd.debian.org/status/package.php?p=sparse&suite=unstable hurd fails because it doesn't define PATH_MAX and NAME_MAX. m68k fails with sparse: ptrlist.c:125: __add_ptr_list: Assertion `(3 & (unsigned long)ptr) == 0' failed. (Didn't look into this one yet.) And ppc64 and x32 need the respective cpp defines added I think. If noone beats me to it, I will look into the latter at least during the next few days. Best regards Uwe signature.asc Description: PGP signature
Processed: Re: Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Processing control commands: > severity 873508 normal Bug #873508 [src:sparse] parsing horst source code fails on s390x and ppc64el Severity set to 'normal' from 'serious' > retitle 873508 Fix FTBFS for m68k, hurd, x32 and ppc64 Bug #873508 [src:sparse] parsing horst source code fails on s390x and ppc64el Changed Bug title to 'Fix FTBFS for m68k, hurd, x32 and ppc64' from 'parsing horst source code fails on s390x and ppc64el'. -- 873508: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=873508 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Control: clone 873508 -1 Control: retitle -1 Please use cgcc to check hosted C code instead of sparse Control: severity -1 normal Control: reassign -1 horst On Fri, Sep 01, 2017 at 09:46:44AM +0200, Uwe Kleine-König wrote: > @anarcat: Given that cgcc seems to work, would you agree to apply the > following patch to horst: > > diff --git a/Makefile b/Makefile > index 4f924fa..d563652 100644 > --- a/Makefile > +++ b/Makefile > @@ -110,7 +110,7 @@ $(NAME): $(OBJS) > $(OBJS): .buildflags > > check: > - sparse $(CFLAGS) *.[ch] > + cgcc -no-compile $(CFLAGS) *.[ch] > > clean: > -rm -f *.o radiotap/*.o *~ > In the meantime I learned from upstream that sparse is not expected to grok arbitrary hosted code. For that it is needed to use the cgcc wrapper to handle the required cpp symbols. That it works on some architectures with plain sparse is mostly luck. I still expect some platforms to fail with the wrapper, too, because cgcc doesn't know about all platforms yet. But I intend to upload a new sparse package soon that includes a build time check for that, and the respective fixes are easy. > and downgrade the bug to "important"? That would be a compromise that > buys us a bit of time. I'd say sparse failing on hosted code isn't "important", but cgcc should have all necessary definitions for Debian platforms. So I'm keeping this bug at important and intend to close it once all platforms are known to it. Best regards Uwe signature.asc Description: PGP signature
Processed: Re: Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Processing control commands: > clone 873508 -1 Bug #873508 [src:sparse] parsing horst source code fails on s390x and ppc64el Bug 873508 cloned as bug 876402 > retitle -1 Please use cgcc to check hosted C code instead of sparse Bug #876402 [src:sparse] parsing horst source code fails on s390x and ppc64el Changed Bug title to 'Please use cgcc to check hosted C code instead of sparse' from 'parsing horst source code fails on s390x and ppc64el'. > severity -1 normal Bug #876402 [src:sparse] Please use cgcc to check hosted C code instead of sparse Severity set to 'normal' from 'serious' > reassign -1 horst Bug #876402 [src:sparse] Please use cgcc to check hosted C code instead of sparse Bug reassigned from package 'src:sparse' to 'horst'. No longer marked as found in versions sparse/0.5.0-4. Ignoring request to alter fixed versions of bug #876402 to the same values previously set -- 873508: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=873508 876402: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876402 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Tue, Sep 12, 2017 at 2:27 AM, Uwe Kleine-König wrote: >> BTW, if I want to get a PPC64 machine for Linux testing purpose, is the >> used apple G5 a good place to start? > > Honestly I don't know. https://wiki.debian.org/ppc64el tells > > Debian/ppc64el requires, at minimum, a POWER8 processor machine. > Although Debian was initially bootstrapped on a POWER7 set of > servers. this class of server is not supported anymore, and you > are not able to run Debian/ppc64el on a POWER7 processor without > hitting an illegal instruction fault. > > Hm https://en.wikipedia.org/wiki/POWER8 tells: > > Systems based on POWER8 became available from IBM in June > 2014. Systems and POWER8 processor designs made by other > OpenPOWER members was available in early 2015. > > So I think this rules out a G5. Thanks for the tip. I am glad I asked before I pull the trigger. > https://wiki.debian.org/ppc64el/Installation mentions you can run this > under qemu however. Good idea. I will give that a try too. Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Tue, Sep 12, 2017 at 01:59:47AM -0400, Christopher Li wrote: > On Wed, Sep 6, 2017 at 11:36 AM, Uwe Kleine-König > wrote: > > There is https://dsa.debian.org/doc/guest-account/ which would give you > > the possibility to access some Debian machines. Other than that I intend > > Sorry for the delay. Thanks for the pointer of the guest account. Tell me if you want to do that. And plan for a delay because there is some "paper work" to be done before; mainly by other people, so it might (or might not) take a moment. > > to upload 0.5.1 to Debian soon and then can provide you links to build > > failures in the build server farm :-) And if you have a patch, I can > > volunteer to make the test monkey for you. > > BTW, if I want to get a PPC64 machine for Linux testing purpose, is the > used apple G5 a good place to start? Honestly I don't know. https://wiki.debian.org/ppc64el tells Debian/ppc64el requires, at minimum, a POWER8 processor machine. Although Debian was initially bootstrapped on a POWER7 set of servers. this class of server is not supported anymore, and you are not able to run Debian/ppc64el on a POWER7 processor without hitting an illegal instruction fault. Hm https://en.wikipedia.org/wiki/POWER8 tells: Systems based on POWER8 became available from IBM in June 2014. Systems and POWER8 processor designs made by other OpenPOWER members was available in early 2015. So I think this rules out a G5. https://wiki.debian.org/ppc64el/Installation mentions you can run this under qemu however. Best regards Uwe signature.asc Description: PGP signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Wed, Sep 6, 2017 at 11:36 AM, Uwe Kleine-König wrote: > There is https://dsa.debian.org/doc/guest-account/ which would give you > the possibility to access some Debian machines. Other than that I intend Sorry for the delay. Thanks for the pointer of the guest account. > to upload 0.5.1 to Debian soon and then can provide you links to build > failures in the build server farm :-) And if you have a patch, I can > volunteer to make the test monkey for you. BTW, if I want to get a PPC64 machine for Linux testing purpose, is the used apple G5 a good place to start? Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Sun, Sep 10, 2017 at 10:43 AM, Uwe Kleine-König wrote: > >> Meanwhile, is it possible to have the build logs but with 'make V=1 ...' ? >> It would also be useful to have: >> - the output of 'uname -a' >> - the details about the version of LLVM you're using > > Sure, can do. Attached is a build from the ppc64el machine with Chris' > patch applied. Tell me if it contains everything you need. I've taken a look at it what happens. The problem is easy to identify and very annoying to solve: in sparsec (a wrapper for sparse-llvm + llc + as [+ ld]) there is a discrepancy between the defaults for llc and as. 'llc' seems to default to the sub-architecture of the build machine (possibly including the most modern features) while 'as' defaults to the minimal features for the build machine architecture. The problem can be solved (on the machine I have access to) by either: - using the option "-mgeneric" for llc - using the option "-mpower8" for as Something smarter would be better. -- Luc
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Sun, Sep 10, 2017 at 10:43 AM, Uwe Kleine-König wrote: > On 09/10/2017 03:22 AM, Luc Van Oostenryck wrote: >> >> I fully test on x86, x86-64, arm & ARM64 (with LLVM 3.9 or 4.0). >> I also test on ppc64 but not the LLVM part because the machines I have >> access to have not LLVM installed and I never bothered to install it myself. >> >> Would it be possible to have access to a machine with the architectures >> you care about? > > Debian provides access to porter boxes for such problems. See > https://dsa.debian.org/doc/guest-account/. OK. I'll first try to install LLVM on what I have already access, it should be faster. >> Meanwhile, is it possible to have the build logs but with 'make V=1 ...' ? >> It would also be useful to have: >> - the output of 'uname -a' >> - the details about the version of LLVM you're using > > Sure, can do. Attached is a build from the ppc64el machine with Chris' > patch applied. Tell me if it contains everything you need. Yes, enough to investigate the problem. Thanks. >> On the other hand, you/us should disable the sparse-llvm part since: >> - it's something that is bundled and build by default but absolutely not >> needed (or even useful) to use sparse. >> - it hasn't been written for anything else than x86/x86-64 (no 'layout' >> for anything else than those architectures. > > With your patch applied I get (independent of having Chris' patch > applied or not): > > Out of 265 tests, 255 passed, 10 failed (10 of them are known to fail) Perfect. With this, you should be unblocked. @Chris, can you apply the patch, please? Best regards, -- Luc Van Oostenryck
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On 09/10/2017 03:22 AM, Luc Van Oostenryck wrote: > On Sat, Sep 9, 2017 at 11:02 PM, Uwe Kleine-König > wrote: >> >> I tried this on ppc64le and it fixes 2 tests, so were at >> >> Out of 287 tests, 273 passed, 14 failed (10 of them are known to >> fail) >> >> The repaired tests are: >> >> backend/hello.c >> backend/sum.c >> >> unexpected failures are: >> >> backend/arithmetic-ops.c >> backend/cmp-ops.c >> backend/int-cond.c >> backend/logical-ops.c >> >> These are not about missing preprocessor tokens as there are no system >> includes used, but the error there is >> >> Error: unrecognized opcode: `...` >> >> . I didn't look into what the problem is there, but attached the test >> log. > > It clearly looks as the code generated by LLVM (the machine code/assembly > not LLVM's bytecode) is not understood by the assembler (or at least some > instructions). Probably a mismatch with the architecture version or something > like that. > >> I did a build test on a few other Debian machines, arm64 was fine, mips >> and mipx64el had 15 failures, ppc64 (i.e. big endian) had 12. I didn't >> look in more detail and suggest to tackle one after the other :-) > > I fully test on x86, x86-64, arm & ARM64 (with LLVM 3.9 or 4.0). > I also test on ppc64 but not the LLVM part because the machines I have > access to have not LLVM installed and I never bothered to install it myself. > > Would it be possible to have access to a machine with the architectures > you care about? Debian provides access to porter boxes for such problems. See https://dsa.debian.org/doc/guest-account/. > Meanwhile, is it possible to have the build logs but with 'make V=1 ...' ? > It would also be useful to have: > - the output of 'uname -a' > - the details about the version of LLVM you're using Sure, can do. Attached is a build from the ppc64el machine with Chris' patch applied. Tell me if it contains everything you need. > On the other hand, you/us should disable the sparse-llvm part since: > - it's something that is bundled and build by default but absolutely not > needed (or even useful) to use sparse. > - it hasn't been written for anything else than x86/x86-64 (no 'layout' > for anything else than those architectures. With your patch applied I get (independent of having Chris' patch applied or not): Out of 265 tests, 255 passed, 10 failed (10 of them are known to fail) Best regards Uwe buildlog-23a393b1cd48ea50bff94fa4a1e2c02a5d78d9cb+ Description: Binary data signature.asc Description: OpenPGP digital signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Sat, Sep 9, 2017 at 11:02 PM, Uwe Kleine-König wrote: > > I tried this on ppc64le and it fixes 2 tests, so were at > > Out of 287 tests, 273 passed, 14 failed (10 of them are known to fail) > > The repaired tests are: > > backend/hello.c > backend/sum.c > > unexpected failures are: > > backend/arithmetic-ops.c > backend/cmp-ops.c > backend/int-cond.c > backend/logical-ops.c > > These are not about missing preprocessor tokens as there are no system > includes used, but the error there is > > Error: unrecognized opcode: `...` > > . I didn't look into what the problem is there, but attached the test > log. It clearly looks as the code generated by LLVM (the machine code/assembly not LLVM's bytecode) is not understood by the assembler (or at least some instructions). Probably a mismatch with the architecture version or something like that. > I did a build test on a few other Debian machines, arm64 was fine, mips > and mipx64el had 15 failures, ppc64 (i.e. big endian) had 12. I didn't > look in more detail and suggest to tackle one after the other :-) I fully test on x86, x86-64, arm & ARM64 (with LLVM 3.9 or 4.0). I also test on ppc64 but not the LLVM part because the machines I have access to have not LLVM installed and I never bothered to install it myself. Would it be possible to have access to a machine with the architectures you care about? Meanwhile, is it possible to have the build logs but with 'make V=1 ...' ? It would also be useful to have: - the output of 'uname -a' - the details about the version of LLVM you're using On the other hand, you/us should disable the sparse-llvm part since: - it's something that is bundled and build by default but absolutely not needed (or even useful) to use sparse. - it hasn't been written for anything else than x86/x86-64 (no 'layout' for anything else than those architectures. Best regards, -- Luc Van Ooostenryck
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Thu, Aug 31, 2017 at 08:47:55PM -0400, Christopher Li wrote: > On Thu, Aug 31, 2017 at 4:55 PM, Uwe Kleine-König Yes > that works. So to address the Debian bug I can do: > > > > - move sparse to /usr/lib > > - teach cgcc about the move of sparse > > - make /usr/bin/sparse call cgcc -no-compile "$@" > > I don't like that. It means the user can't invoke sparse directly. > > > > > or is it easier to teach sparse about the architecture stuff? > > First of all. It is not very trivial to teach sparse about the architecture > stuff. To my mind, we need to move all the cgcc logic into sparse. > > For this case, I think it is easier to teach the sparse validation > code to use cgcc on those back end testing. Most validation don't > need to include system header file at all so it does not have > this problem. > > How about this patch? > I know my patch is white space damaged in email. > Git branch is at: > https://git.kernel.org/pub/scm/devel/sparse/chrisl/sparse.git/log/?h=llvm-cgcc > > Please let me know if that fix your problem. It pass check > on my local machine running x86_64. I don't have ppc64 to > test with. > > Chris > > diff --git a/sparsec b/sparsec > index 9dc96c9..2990d26 100755 > --- a/sparsec > +++ b/sparsec > @@ -32,7 +32,8 @@ done > TMPLLVM=`mktemp -t tmp.XX`".llvm" > TMPFILE=`mktemp -t tmp.XX`".o" > > -$DIRNAME/sparse-llvm $SPARSEOPTS > $TMPLLVM > +env CHECK=$DIRNAME/sparse-llvm $DIRNAME/cgcc -no-compile \ > + $SPARSEOPTS > $TMPLLVM > > LLC=`"${LLVM_CONFIG:-llvm-config}" --bindir`/llc > > diff --git a/sparsei b/sparsei > index 3431a9f..3abd00f 100755 > --- a/sparsei > +++ b/sparsei > @@ -10,4 +10,4 @@ if [ $# -eq 0 ]; then >exit 1 > fi > > -$DIRNAME/sparse-llvm $@ | $LLI > +env CHECK=$DIRNAME/sparse-llvm $DIRNAME/cgcc -no-compile $@ | $LLI I tried this on ppc64le and it fixes 2 tests, so were at Out of 287 tests, 273 passed, 14 failed (10 of them are known to fail) The repaired tests are: backend/hello.c backend/sum.c unexpected failures are: backend/arithmetic-ops.c backend/cmp-ops.c backend/int-cond.c backend/logical-ops.c These are not about missing preprocessor tokens as there are no system includes used, but the error there is Error: unrecognized opcode: `...` . I didn't look into what the problem is there, but attached the test log. I did a build test on a few other Debian machines, arm64 was fine, mips and mipx64el had 15 failures, ppc64 (i.e. big endian) had 12. I didn't look in more detail and suggest to tackle one after the other :-) Best regards Uwe I: Started sh -c make && make check O: make: Nothing to be done for 'all'. O: TESTWoverride-init-def (Woverride-init-def.c) O: TESTWoverride-init-no (Woverride-init-no.c) O: TESTWoverride-init-yes (Woverride-init-yes.c) O: TESTwarn-unknown-attribute (Wunknown-attribute-def.c) O: TESTwarn-unknown-attribute-no (Wunknown-attribute-no.c) O: TESTwarn-unknown-attribute-yes (Wunknown-attribute-yes.c) O: TEST__func__ (__func__.c) O: TESTabstract array declarator static (abstract-array-declarator-static.c) O: TESTaddress_space attribute (address_space.c) O: TESTalias distinct symbols (alias-distinct.c) O: TESTalias symbol/pointer (alias-mixed.c) O: TESTalias same symbols (alias-same.c) O: TESTattribute __alloc_align__ (alloc-align.c) O: TESTalternate keywords (alternate-keywords.c) O: TESTtest anonymous union initializer (anon-union.c) O: TESTAsm with goto labels. (asm-empty-clobber.c) O: TESTAsm with goto labels. (asm-goto-lables.c) O: TESTasm-toplevel.c (asm-toplevel.c) O: TESTinline attributes (attr-inline.c) O: TESTattribute no_sanitize_address (attr-no_sanitize_address.c) O: TESTattribute noclone (attr-noclone.c) O: TESToptimize attributes (attr-optimize.c) O: TESTattribute warning (attr-warning.c) O: TESTattribute assume_aligned (attr_aligned.c) O: TESTattribute after ( in direct-declarator (attr_in_parameter.c) O: TESTattribute vector_size (attr_vector_size.c) O: TESTArithmetic operator code generation (backend/arithmetic-ops.c) O: error: actual error text does not match expected error text. O: error: see backend/arithmetic-ops.c.error.* for further investigation. O: --- backend/arithmetic-ops.c.error.expected 2017-09-09 20:44:47.964306005 + O: +++ backend/arithmetic-ops.c.error.got 2017-09-09 20:44:47.960305943 + O: @@ -0,0 +1,10 @@ O: +{standard input}: Assembler messages: O: +{standard input}:38: Error: unrecognized opcode: `xsaddsp' O: +{standard input}:52: Error: unrecognized opcode: `xsadddp' O: +{standard input}:94: Error: unrecognized opcode: `xssubsp' O: +{standard input}:108: Error: unrecognized opcode: `xssubdp' O: +{standard input}:150: Error: unrecognized opcod
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Hello Christopher, On Wed, Sep 06, 2017 at 11:18:04AM -0400, Christopher Li wrote: > On Wed, Sep 6, 2017 at 10:44 AM, Uwe Kleine-König > wrote: > > and while it's ok to test the core stuff and not wanting the system > > includes to interfere, there should also be tests that check "ordinary" > > userspace programs which naturally depend on the system headers. > > There is one. The "selfcheck" target was checking sparse on its own > source file. That will definitively use the system header file. However, > there are some warning trigger in the system header file can't be fixed > in the sparse source code. It need to change the system header to make > some warning go away, or disable that warning. > > > > > No, it's not solved. But given that it is somehow known that sparse > > (without cgcc) fails to work well on userspace stuff, I think the > > following would be fine for the Debian side: > > > > a) let horst use cgcc instead of sparse > > b) downgrade bug to important (or even normal) pointing out that > > cgcc should be used for userspace programs > > That seems to be the right thing to do for now. That is until > sparse are smarter on user space header regarding architecture stuff. ok, Antoine, can you talk to the horst people and ask them to switch to cgcc then? > > For sparse it would be cool to: > > > > c) drop the #weak_define of __amd64__ to make this "problem" more > > apparent. (Assuming this doesn't break e.g. a kernel build.) > > You mean remove define of "__x86_64__". ack. > It will likely break some other stuff. For the record the "selfcheck" target > already using cgcc. We still need to fix the breakage. > > Any suggestion how to test sparse running on other platform headfile > without have to get access to ppc64 for example? I think that is the biggest > obstacle right now. I can make some changes, but I don't have a good way > to test it other than x86 platform. There is https://dsa.debian.org/doc/guest-account/ which would give you the possibility to access some Debian machines. Other than that I intend to upload 0.5.1 to Debian soon and then can provide you links to build failures in the build server farm :-) And if you have a patch, I can volunteer to make the test monkey for you. > > d) fix the test suite, at least mark the tests that are expected to > > fail on !x86 as such to make $(make check) succeed. (Otherwise I'd > > have to disable or ignore the testsuite which isn't that great.) > > We can make the test-suite not depend on system header files. > That seems to be the right think to do. I also send out a patch > to let the llvm back end test-suite use cgcc last week. Removing > system header usage in test suite is better. Testing that patch is on my todo list. Best regards Uwe signature.asc Description: PGP signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Wed, Sep 6, 2017 at 10:44 AM, Uwe Kleine-König wrote: > > and while it's ok to test the core stuff and not wanting the system > includes to interfere, there should also be tests that check "ordinary" > userspace programs which naturally depend on the system headers. > There is one. The "selfcheck" target was checking sparse on its own source file. That will definitively use the system header file. However, there are some warning trigger in the system header file can't be fixed in the sparse source code. It need to change the system header to make some warning go away, or disable that warning. > > No, it's not solved. But given that it is somehow known that sparse > (without cgcc) fails to work well on userspace stuff, I think the > following would be fine for the Debian side: > > a) let horst use cgcc instead of sparse > b) downgrade bug to important or even normal pointing out that cgcc > should be used for userspace programs That seems to be the right thing to do for now. That is until sparse are smarter on user space header regarding architecture stuff. > For sparse it would be cool to: > > c) drop the #weak_define of __amd64__ to make this "problem" more > apparent. (Assuming this doesn't break e.g. a kernel build.) You mean remove define of "__x86_64__". It will likely break some other stuff. For the record the "selfcheck" target already using cgcc. We still need to fix the breakage. Any suggestion how to test sparse running on other platform headfile without have to get access to ppc64 for example? I think that is the biggest obstacle right now. I can make some changes, but I don't have a good way to test it other than x86 platform. > d) fix the test suite, at least mark the tests that are expected to > fail on !x86 as such to make $(make check) succeed. (Otherwise I'd > have to disable or ignore the testsuite which isn't that great.) We can make the test-suite not depend on system header files. That seems to be the right think to do. I also send out a patch to let the llvm back end test-suite use cgcc last week. Removing system header usage in test suite is better. Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
[readding people to Cc assuming that's ok] Hello Luc, On Mon, Sep 04, 2017 at 10:36:47PM +0200, Luc Van Oostenryck wrote: > On Mon, Sep 4, 2017 at 10:57 AM, Uwe Kleine-König > wrote: > > On 09/03/2017 11:14 PM, Luc Van Oostenryck wrote: > >> On Fri, Sep 1, 2017 at 9:02 AM, Josh Triplett > >> wrote: > >>> Related to that: while it would mean we couldn't necessarily just rely > >>> entirely on GCC's definitions for a target platform, I think in an ideal > >>> world we could have a sparse binary that understood *all* target > >>> platforms at once, such that you could ask Sparse on x86_64 to "compile" > >>> as though targeting any arbitrary architecture. That would also have the > >>> major advantage of making it easy to run the Sparse testsuite for > >>> *every* target architecture without needing compilers for every such > >>> architecture. > >> > >> I really think that the testsuite should not depend on system or library > >> header. > > > > Assuming it's intended that sparse should be able to check userspace > > programs, I don't agree here. > > I understand this. > I'll explain a bit better my point of view. > First, I make a distinction between 'sparse core functionalities' and > general usage. > I was talking about this core usage and the testsuite is currently for this > core > usage too. and while it's ok to test the core stuff and not wanting the system includes to interfere, there should also be tests that check "ordinary" userspace programs which naturally depend on the system headers. > Asking for the testsuite to not depends on system or library header is exactly > the same as GCC people asking bug reports to be done on pre-processed file > (so that they focus on the core problem and not some problem with an header). > This, of course, doesn't mean that GCC should only be used on standalone > source files nor that GCC shouldn't be tested on real code, using > system headers. > It's just something different. > > So to answer to your objection: yes, you're right but it should be done in > some > specific tests, not the core ones. ah, we agree. Fine. > Secondly, about "intended to check userspace programs": > It's clear that sparse's main use is for the kernel, but it's also > clear that it can > and is used on other (userspace) projects. > However, as you have seen yourself, you can't use sparse as is and expect > to work on any environment, on any architecture. Even for the kernel it > doesn't: > each architecture has to specify a few flags (like -m32/-m64) and a few > defines > (-D__arm__, ...). For userspace, cgcc can do a part of this job for you. > > Josh proposal to have what I called a 'universal' sparse, won't solve this, > on the contrary. > Compilers eschew part of this problem by having to configure the build > > I'm all in favor to move cgcc logic to sparse and/or it's build system so that > *for a native build* it can be used as-is in most cases. > This would solve your problem, I think. > > BTW, sorry I didn't follow last week but is your problem solved now? No, it's not solved. But given that it is somehow known that sparse (without cgcc) fails to work well on userspace stuff, I think the following would be fine for the Debian side: a) let horst use cgcc instead of sparse b) downgrade bug to important or even normal pointing out that cgcc should be used for userspace programs For sparse it would be cool to: c) drop the #weak_define of __amd64__ to make this "problem" more apparent. (Assuming this doesn't break e.g. a kernel build.) d) fix the test suite, at least mark the tests that are expected to fail on !x86 as such to make $(make check) succeed. (Otherwise I'd have to disable or ignore the testsuite which isn't that great.) Best regards Uwe signature.asc Description: PGP signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Sun, Sep 3, 2017 at 5:14 PM, Luc Van Oostenryck wrote: > > I really think that the testsuite should not depend on system or library > header. I think that is a good point. We can start cleaning up the system header file dependency in the existing test suite. See how it goes. > > Otherwise, I'm not at all opposed to sparse being universal but I would like > to note that things can become very quickly very very messy. > For example, for the current problem here I understood that it was > at least partially based on the lack of a definition of _CALL_ELF > but do we need to define it to 1 or to 2, in other words, do we need > to support the ELFv1 ABI or the ELFv2? GCC has some flags for this > (-mabi=elfv[12]) but what default value do we want? ELFv1 is the default I think we can just sparse default to as late as the latest release version of gcc. > for big-endian platform and ELFv2 for little-endian platform, so yes, > we need a flag for the endianness but which endianness we want as default? I am tempting to make the endianness the same as the host gcc by default. Then it can be overwrite by architecture flags. > > Things become even more fun when taking in account the difference > between GCC version. Do we want to be universal there too (and thus > have some flags for to specify which gcc's version we want to mimick)? > What about other compilers? I purpose just sync to the latest gcc version (or a late enough version we can agree on. e.g. the one that supported by kernel compile.) Sparse current try to sync to the latest gcc attributes already. > I think that part of the needed info can be auto-extracted from GCC > when doing a native build. Using some sort of spec file or a .sparserc Is there a way to do auto-extract? That would be a good starting point. > I also note that currently, sparse is already largely universal *because* > it *doesn't* need those platform details (or only the very minimal: word > size). Sparse is not universal, it just support a very small sub set of the C source file that haven't expose to those platform detail macros. Adding those architecture macro support will not make it any less universal. Might slow things down, that is some thing we need to watch out for. Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Fri, Sep 1, 2017 at 9:02 AM, Josh Triplett wrote: > On Thu, Aug 31, 2017 at 08:47:55PM -0400, Christopher Li wrote: >> On Thu, Aug 31, 2017 at 4:55 PM, Uwe Kleine-König Yes >> that works. So to address the Debian bug I can do: >> > >> > - move sparse to /usr/lib >> > - teach cgcc about the move of sparse >> > - make /usr/bin/sparse call cgcc -no-compile "$@" >> >> I don't like that. It means the user can't invoke sparse directly. >> >> > >> > or is it easier to teach sparse about the architecture stuff? >> >> First of all. It is not very trivial to teach sparse about the architecture >> stuff. To my mind, we need to move all the cgcc logic into sparse. > > Related to that: while it would mean we couldn't necessarily just rely > entirely on GCC's definitions for a target platform, I think in an ideal > world we could have a sparse binary that understood *all* target > platforms at once, such that you could ask Sparse on x86_64 to "compile" > as though targeting any arbitrary architecture. That would also have the > major advantage of making it easy to run the Sparse testsuite for > *every* target architecture without needing compilers for every such > architecture. I really think that the testsuite should not depend on system or library header. Otherwise, I'm not at all opposed to sparse being universal but I would like to note that things can become very quickly very very messy. For example, for the current problem here I understood that it was at least partially based on the lack of a definition of _CALL_ELF but do we need to define it to 1 or to 2, in other words, do we need to support the ELFv1 ABI or the ELFv2? GCC has some flags for this (-mabi=elfv[12]) but what default value do we want? ELFv1 is the default for big-endian platform and ELFv2 for little-endian platform, so yes, we need a flag for the endianness but which endianness we want as default? And so on. Things become even more fun when taking in account the difference between GCC version. Do we want to be universal there too (and thus have some flags for to specify which gcc's version we want to mimick)? What about other compilers? I think that part of the needed info can be auto-extracted from GCC when doing a native build. Using some sort of spec file or a .sparserc can help too. I also note that currently, sparse is already largely universal *because* it *doesn't* need those platform details (or only the very minimal: word size). -- Luc Van Oostenryck
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Fri, Sep 01, 2017 at 09:57:09AM +0200, Uwe Kleine-König wrote: > On Fri, Sep 01, 2017 at 12:02:12AM -0700, Josh Triplett wrote: > > On Thu, Aug 31, 2017 at 08:47:55PM -0400, Christopher Li wrote: > > > On Thu, Aug 31, 2017 at 4:55 PM, Uwe Kleine-König Yes > > > that works. So to address the Debian bug I can do: > > > > > > > > - move sparse to /usr/lib > > > > - teach cgcc about the move of sparse > > > > - make /usr/bin/sparse call cgcc -no-compile "$@" > > > > > > I don't like that. It means the user can't invoke sparse directly. > > > > > > > > > > > or is it easier to teach sparse about the architecture stuff? > > > > > > First of all. It is not very trivial to teach sparse about the > > > architecture > > > stuff. To my mind, we need to move all the cgcc logic into sparse. > > > > Related to that: while it would mean we couldn't necessarily just rely > > entirely on GCC's definitions for a target platform, I think in an ideal > > world we could have a sparse binary that understood *all* target > > platforms at once, such that you could ask Sparse on x86_64 to "compile" > > as though targeting any arbitrary architecture. That would also have the > > major advantage of making it easy to run the Sparse testsuite for > > *every* target architecture without needing compilers for every such > > architecture. > > You'd need the target arch's system headers though. Only for building userspace code, not for building standalone/kernel code, or the Sparse testsuite. - Josh Triplett
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Fri, Sep 1, 2017 at 3:02 AM, Josh Triplett wrote: >> First of all. It is not very trivial to teach sparse about the architecture >> stuff. To my mind, we need to move all the cgcc logic into sparse. > > Related to that: while it would mean we couldn't necessarily just rely > entirely on GCC's definitions for a target platform, I think in an ideal > world we could have a sparse binary that understood *all* target > platforms at once, such that you could ask Sparse on x86_64 to "compile" Yes, that is what I want to have. It is list as one of the project in project idea document as well. I have a related question. How do we test the different architecture handling without actually run sparse on different platform? I am thinking maybe using gcc cross platform compiler and compare some macro against the sparse one. > as though targeting any arbitrary architecture. That would also have the > major advantage of making it easy to run the Sparse testsuite for > *every* target architecture without needing compilers for every such > architecture. Another way to fix the test suite for now would be let testsuite specify using cgcc instead of sparse directly, for the test source that needs it. That will buy us some time. Fixing sparse properly in the long term obvious would be better. Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Fri, Sep 1, 2017 at 3:46 AM, Uwe Kleine-König wrote: \> > Nearly right. I'm responsible for the sparse Debian package and the > problem at hand is https://bugs.debian.org/873508. This bug report has > "Severity: serious" wihch might eventually result in the removal of > sparse from the Debian archive. > > @anarcat: Given that cgcc seems to work, would you agree to apply the > following patch to horst: > > diff --git a/Makefile b/Makefile > index 4f924fa..d563652 100644 > --- a/Makefile > +++ b/Makefile > @@ -110,7 +110,7 @@ $(NAME): $(OBJS) > $(OBJS): .buildflags > > check: > - sparse $(CFLAGS) *.[ch] > + cgcc -no-compile $(CFLAGS) *.[ch] You patch definitely works. I think it is the better way to fix it. Sparse "selfcheck" target was doing the same thing. It is using "cgcc -no-compile" already. I suggest in your patch you can do some thing similar to "selfcheck" target: CHECKER = cgcc -no-compile $(CHECKER) $(CFLAGS) *.[ch] Later when we update sparse to handle architecture properly. We can just invoke make with "CHECKER=" to test. > and downgrade the bug to "important"? That would be a compromise that > buys us a bit of time. Agree. > > This only solves a part of the problem because the bug report is about > sparse itself, not it's llvm part. I agree with that too. Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On 2017-09-01 09:46:44, Uwe Kleine-König wrote: > Hello, > > On Thu, Aug 31, 2017 at 11:43:53PM +0100, Ramsay Jones wrote: >> On 31/08/17 21:55, Uwe Kleine-König wrote: >> > On Wed, Aug 30, 2017 at 08:11:49PM -0400, Christopher Li wrote: >> >> That is very much like on x86_64 missing define "#weak_define __x86_64__ >> >> 1" >> >> >> >> Does cgcc work for you? In the future we do want to move the archetecture >> >> related define from cgcc into sparse by itself. For now you can set >> >> "sparse" as "cgcc -no-compile" >> > >> > Yes that works. So to address the Debian bug I can do: >> > >> > - move sparse to /usr/lib >> > - teach cgcc about the move of sparse >> > - make /usr/bin/sparse call cgcc -no-compile "$@" >> >> Hmm, I don't think that would be a good idea ... >> >> > or is it easier to teach sparse about the architecture stuff? >> >> I now understand (I think!) that you are building a sparse >> package (presumably a .deb) and you are concerned that sparse >> does not pass it's own testsuite on those platforms. > > Nearly right. I'm responsible for the sparse Debian package and the > problem at hand is https://bugs.debian.org/873508. This bug report has > "Severity: serious" wihch might eventually result in the removal of > sparse from the Debian archive. > > @anarcat: Given that cgcc seems to work, would you agree to apply the > following patch to horst: > > diff --git a/Makefile b/Makefile > index 4f924fa..d563652 100644 > --- a/Makefile > +++ b/Makefile > @@ -110,7 +110,7 @@ $(NAME): $(OBJS) > $(OBJS): .buildflags > > check: > - sparse $(CFLAGS) *.[ch] > + cgcc -no-compile $(CFLAGS) *.[ch] > > clean: > -rm -f *.o radiotap/*.o *~ > > and downgrade the bug to "important"? That would be a compromise that > buys us a bit of time. Well right now I have simply disabled the checks for those broken architectures, so sparse isn't as affected anymore. Frankly, I don't quite know what to do with this - but I'd be happy to happly that patch to sparse if it fixes the issue better. A. -- Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety. - Benjamin Franklin, 1755
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Fri, Sep 01, 2017 at 12:02:12AM -0700, Josh Triplett wrote: > On Thu, Aug 31, 2017 at 08:47:55PM -0400, Christopher Li wrote: > > On Thu, Aug 31, 2017 at 4:55 PM, Uwe Kleine-König Yes > > that works. So to address the Debian bug I can do: > > > > > > - move sparse to /usr/lib > > > - teach cgcc about the move of sparse > > > - make /usr/bin/sparse call cgcc -no-compile "$@" > > > > I don't like that. It means the user can't invoke sparse directly. > > > > > > > > or is it easier to teach sparse about the architecture stuff? > > > > First of all. It is not very trivial to teach sparse about the architecture > > stuff. To my mind, we need to move all the cgcc logic into sparse. > > Related to that: while it would mean we couldn't necessarily just rely > entirely on GCC's definitions for a target platform, I think in an ideal > world we could have a sparse binary that understood *all* target > platforms at once, such that you could ask Sparse on x86_64 to "compile" > as though targeting any arbitrary architecture. That would also have the > major advantage of making it easy to run the Sparse testsuite for > *every* target architecture without needing compilers for every such > architecture. You'd need the target arch's system headers though. But still it would be great. In a first attempt something like: #ifdef __powerpc__ add_pre_buffer("#weak_define __powerpc__ " __powerpc__ "\n"); #ifdef _CALL_ELF add_pre_buffer("#weak_define _CALL_ELF " _CALL_ELF "\n"); #endif #endif would be helpful already. Best regards Uwe signature.asc Description: PGP signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Hello, On Thu, Aug 31, 2017 at 11:43:53PM +0100, Ramsay Jones wrote: > On 31/08/17 21:55, Uwe Kleine-König wrote: > > On Wed, Aug 30, 2017 at 08:11:49PM -0400, Christopher Li wrote: > >> That is very much like on x86_64 missing define "#weak_define __x86_64__ 1" > >> > >> Does cgcc work for you? In the future we do want to move the archetecture > >> related define from cgcc into sparse by itself. For now you can set > >> "sparse" as "cgcc -no-compile" > > > > Yes that works. So to address the Debian bug I can do: > > > > - move sparse to /usr/lib > > - teach cgcc about the move of sparse > > - make /usr/bin/sparse call cgcc -no-compile "$@" > > Hmm, I don't think that would be a good idea ... > > > or is it easier to teach sparse about the architecture stuff? > > I now understand (I think!) that you are building a sparse > package (presumably a .deb) and you are concerned that sparse > does not pass it's own testsuite on those platforms. Nearly right. I'm responsible for the sparse Debian package and the problem at hand is https://bugs.debian.org/873508. This bug report has "Severity: serious" wihch might eventually result in the removal of sparse from the Debian archive. @anarcat: Given that cgcc seems to work, would you agree to apply the following patch to horst: diff --git a/Makefile b/Makefile index 4f924fa..d563652 100644 --- a/Makefile +++ b/Makefile @@ -110,7 +110,7 @@ $(NAME): $(OBJS) $(OBJS): .buildflags check: - sparse $(CFLAGS) *.[ch] + cgcc -no-compile $(CFLAGS) *.[ch] clean: -rm -f *.o radiotap/*.o *~ and downgrade the bug to "important"? That would be a compromise that buys us a bit of time. > As I said before, the additional failures you are seeing are > in the 'llvm backend' code (which, as far as I know, only passes > on x86_64 Linux), and in my opinion the llvm-backend programs should > not be installed. (The Makefile will build them automatically if > you have llvm installed, likewise for c2xml/libxml and test-inspect/gtk). Currently the sparse package contains /usr/include/sparse/, c2xml, cgcc, sparse, sparse-llvm, sparsec and a separate package sparse-test-inspect includes test-inspect. (That's how I inherited the package some time ago.) > [I would like to see build variable(s) to allow the user to suppress > the build (or installation) of the other 'non-primary' sparse programs.] > > Anyway, if you were to un-install llvm, sparse-llvm etc., would not > be built, and the tests would not be run ... ;-) > > Christopher, as the project maintainer, has the joy of making these > kinds of decisions! :-D This only solves a part of the problem because the bug report is about sparse itself, not it's llvm part. Best regards Uwe signature.asc Description: PGP signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Thu, Aug 31, 2017 at 08:47:55PM -0400, Christopher Li wrote: > On Thu, Aug 31, 2017 at 4:55 PM, Uwe Kleine-König Yes > that works. So to address the Debian bug I can do: > > > > - move sparse to /usr/lib > > - teach cgcc about the move of sparse > > - make /usr/bin/sparse call cgcc -no-compile "$@" > > I don't like that. It means the user can't invoke sparse directly. > > > > > or is it easier to teach sparse about the architecture stuff? > > First of all. It is not very trivial to teach sparse about the architecture > stuff. To my mind, we need to move all the cgcc logic into sparse. Related to that: while it would mean we couldn't necessarily just rely entirely on GCC's definitions for a target platform, I think in an ideal world we could have a sparse binary that understood *all* target platforms at once, such that you could ask Sparse on x86_64 to "compile" as though targeting any arbitrary architecture. That would also have the major advantage of making it easy to run the Sparse testsuite for *every* target architecture without needing compilers for every such architecture.
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Thu, Aug 31, 2017 at 6:43 PM, Ramsay Jones >> - move sparse to /usr/lib >> - teach cgcc about the move of sparse >> - make /usr/bin/sparse call cgcc -no-compile "$@" > > Hmm, I don't think that would be a good idea ... > Agree. > > Anyway, if you were to un-install llvm, sparse-llvm etc., would not > be built, and the tests would not be run ... ;-) I think Uwe already confirm using cgcc to invoke sparse will fix the problem. In this test, we just need to use cgcc to invoke sparse-llvm. I have a patch to fix the usage in test-suite in the other email. Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Thu, Aug 31, 2017 at 4:55 PM, Uwe Kleine-König Yes that works. So to address the Debian bug I can do: > > - move sparse to /usr/lib > - teach cgcc about the move of sparse > - make /usr/bin/sparse call cgcc -no-compile "$@" I don't like that. It means the user can't invoke sparse directly. > > or is it easier to teach sparse about the architecture stuff? First of all. It is not very trivial to teach sparse about the architecture stuff. To my mind, we need to move all the cgcc logic into sparse. For this case, I think it is easier to teach the sparse validation code to use cgcc on those back end testing. Most validation don't need to include system header file at all so it does not have this problem. How about this patch? I know my patch is white space damaged in email. Git branch is at: https://git.kernel.org/pub/scm/devel/sparse/chrisl/sparse.git/log/?h=llvm-cgcc Please let me know if that fix your problem. It pass check on my local machine running x86_64. I don't have ppc64 to test with. Chris diff --git a/sparsec b/sparsec index 9dc96c9..2990d26 100755 --- a/sparsec +++ b/sparsec @@ -32,7 +32,8 @@ done TMPLLVM=`mktemp -t tmp.XX`".llvm" TMPFILE=`mktemp -t tmp.XX`".o" -$DIRNAME/sparse-llvm $SPARSEOPTS > $TMPLLVM +env CHECK=$DIRNAME/sparse-llvm $DIRNAME/cgcc -no-compile \ + $SPARSEOPTS > $TMPLLVM LLC=`"${LLVM_CONFIG:-llvm-config}" --bindir`/llc diff --git a/sparsei b/sparsei index 3431a9f..3abd00f 100755 --- a/sparsei +++ b/sparsei @@ -10,4 +10,4 @@ if [ $# -eq 0 ]; then exit 1 fi -$DIRNAME/sparse-llvm $@ | $LLI +env CHECK=$DIRNAME/sparse-llvm $DIRNAME/cgcc -no-compile $@ | $LLI
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On 31/08/17 21:55, Uwe Kleine-König wrote: > On Wed, Aug 30, 2017 at 08:11:49PM -0400, Christopher Li wrote: >> That is very much like on x86_64 missing define "#weak_define __x86_64__ 1" >> >> Does cgcc work for you? In the future we do want to move the archetecture >> related define from cgcc into sparse by itself. For now you can set >> "sparse" as "cgcc -no-compile" > > Yes that works. So to address the Debian bug I can do: > > - move sparse to /usr/lib > - teach cgcc about the move of sparse > - make /usr/bin/sparse call cgcc -no-compile "$@" Hmm, I don't think that would be a good idea ... > or is it easier to teach sparse about the architecture stuff? I now understand (I think!) that you are building a sparse package (presumably a .deb) and you are concerned that sparse does not pass it's own testsuite on those platforms. As I said before, the additional failures you are seeing are in the 'llvm backend' code (which, as far as I know, only passes on x86_64 Linux), and in my opinion the llvm-backend programs should not be installed. (The Makefile will build them automatically if you have llvm installed, likewise for c2xml/libxml and test-inspect/gtk). [I would like to see build variable(s) to allow the user to suppress the build (or installation) of the other 'non-primary' sparse programs.] Anyway, if you were to un-install llvm, sparse-llvm etc., would not be built, and the tests would not be run ... ;-) Christopher, as the project maintainer, has the joy of making these kinds of decisions! :-D ATB, Ramsay Jones
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Hello Christopher, On Wed, Aug 30, 2017 at 08:11:49PM -0400, Christopher Li wrote: > On Wed, Aug 30, 2017 at 1:36 PM, Uwe Kleine-König > wrote: > >> > > >> > diff --git a/validation/backend/sum.c b/validation/backend/sum.c > >> > index 0604299..d0be8dd 100644 > >> > --- a/validation/backend/sum.c > >> > +++ b/validation/backend/sum.c > >> > @@ -1,3 +1,5 @@ > >> > +#define __powerpc64__ > >> > +#define _CALL_ELF 2 > >> > #include > >> > #include > >> > > >> > > >> > >> Yep, sparse/sparsec do not define various macros that gcc/clang define > >> by default on a given architecture. This is a known problem (that I have > >> been meaning to fix ...). The 'workaround' for the time being is to use > >> the cgcc front-end to sparse. (for example 'make CC=cgcc', or perhaps > >> 'cgcc -no-compile'). > >> > >> [You didn't mention your usage - is this for a kernel build?] > > > > This problem became visible during the make check phase when creating > > packaged > > on the listed archs for horst[1]. You can see a build logs at > > > > > > https://buildd.debian.org/status/fetch.php?pkg=horst&arch=s390x&ver=5.0-1&stamp=1503905687&raw=0 > > > > https://buildd.debian.org/status/fetch.php?pkg=horst&arch=ppc64el&ver=5.0-1&stamp=1503906226&raw=0 > > > > The error message looks identical (I checked the ppc64el log) to the > > problem with backend/sum.c: > > > > sparse -g -O2 -fdebug-prefix-map=/<>=. > > -fstack-protector-strong -Wformat -Werror=format-security -std=gnu99 -Wall > > -Wextra -g -I. -DDO_DEBUG -I/usr/include/libnl3 *.[ch] > > /usr/include/powerpc64le-linux-gnu/gnu/stubs.h:8:12: error: unable > > to open 'gnu/stubs-32.h' > > That is very much like on x86_64 missing define "#weak_define __x86_64__ 1" > > Does cgcc work for you? In the future we do want to move the archetecture > related define from cgcc into sparse by itself. For now you can set > "sparse" as "cgcc -no-compile" Yes that works. So to address the Debian bug I can do: - move sparse to /usr/lib - teach cgcc about the move of sparse - make /usr/bin/sparse call cgcc -no-compile "$@" or is it easier to teach sparse about the architecture stuff? Best regards Uwe signature.asc Description: PGP signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On Wed, Aug 30, 2017 at 1:36 PM, Uwe Kleine-König wrote: >> > >> > diff --git a/validation/backend/sum.c b/validation/backend/sum.c >> > index 0604299..d0be8dd 100644 >> > --- a/validation/backend/sum.c >> > +++ b/validation/backend/sum.c >> > @@ -1,3 +1,5 @@ >> > +#define __powerpc64__ >> > +#define _CALL_ELF 2 >> > #include >> > #include >> > >> > >> >> Yep, sparse/sparsec do not define various macros that gcc/clang define >> by default on a given architecture. This is a known problem (that I have >> been meaning to fix ...). The 'workaround' for the time being is to use >> the cgcc front-end to sparse. (for example 'make CC=cgcc', or perhaps >> 'cgcc -no-compile'). >> >> [You didn't mention your usage - is this for a kernel build?] > > This problem became visible during the make check phase when creating packaged > on the listed archs for horst[1]. You can see a build logs at > > > https://buildd.debian.org/status/fetch.php?pkg=horst&arch=s390x&ver=5.0-1&stamp=1503905687&raw=0 > > https://buildd.debian.org/status/fetch.php?pkg=horst&arch=ppc64el&ver=5.0-1&stamp=1503906226&raw=0 > > The error message looks identical (I checked the ppc64el log) to the > problem with backend/sum.c: > > sparse -g -O2 -fdebug-prefix-map=/<>=. > -fstack-protector-strong -Wformat -Werror=format-security -std=gnu99 -Wall > -Wextra -g -I. -DDO_DEBUG -I/usr/include/libnl3 *.[ch] > /usr/include/powerpc64le-linux-gnu/gnu/stubs.h:8:12: error: unable to > open 'gnu/stubs-32.h' That is very much like on x86_64 missing define "#weak_define __x86_64__ 1" Does cgcc work for you? In the future we do want to move the archetecture related define from cgcc into sparse by itself. For now you can set "sparse" as "cgcc -no-compile" Chris
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Hello, On Wed, Aug 30, 2017 at 05:55:00PM +0100, Ramsay Jones wrote: > On 30/08/17 17:14, Uwe Kleine-König wrote: > > ukleinek@plummer:~/sparse$ make check V=1 > > It would be easier to see the results if you _didn't_ add V=1. ;-) noted for the next time. > [snip] > > Out of 287 tests, 272 passed, 15 failed (10 of them are known to fail) > > Makefile:232: recipe for target 'check' failed > > make: *** [check] Error 1 > > ukleinek@plummer:~/sparse$ > > The additional five failures are all in the llvm backend (sparsec), > which you do not need to use sparse as a 'checker'. > > > The problem is that some cpp symbols are not defined in sparse that are > > expected to exist. So I can "fix" backend/sum.c with the following > > patch: > > > > diff --git a/validation/backend/sum.c b/validation/backend/sum.c > > index 0604299..d0be8dd 100644 > > --- a/validation/backend/sum.c > > +++ b/validation/backend/sum.c > > @@ -1,3 +1,5 @@ > > +#define __powerpc64__ > > +#define _CALL_ELF 2 > > #include > > #include > > > > > > Yep, sparse/sparsec do not define various macros that gcc/clang define > by default on a given architecture. This is a known problem (that I have > been meaning to fix ...). The 'workaround' for the time being is to use > the cgcc front-end to sparse. (for example 'make CC=cgcc', or perhaps > 'cgcc -no-compile'). > > [You didn't mention your usage - is this for a kernel build?] This problem became visible during the make check phase when creating packaged on the listed archs for horst[1]. You can see a build logs at https://buildd.debian.org/status/fetch.php?pkg=horst&arch=s390x&ver=5.0-1&stamp=1503905687&raw=0 https://buildd.debian.org/status/fetch.php?pkg=horst&arch=ppc64el&ver=5.0-1&stamp=1503906226&raw=0 The error message looks identical (I checked the ppc64el log) to the problem with backend/sum.c: sparse -g -O2 -fdebug-prefix-map=/<>=. -fstack-protector-strong -Wformat -Werror=format-security -std=gnu99 -Wall -Wextra -g -I. -DDO_DEBUG -I/usr/include/libnl3 *.[ch] /usr/include/powerpc64le-linux-gnu/gnu/stubs.h:8:12: error: unable to open 'gnu/stubs-32.h' Makefile:113: recipe for target 'check' failed make[1]: *** [check] Error 1 Best regards Uwe [1] https://packages.debian.org/sid/horst signature.asc Description: PGP signature
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
On 30/08/17 17:14, Uwe Kleine-König wrote: > Hello, > > Antoine Beaupre (on Cc:) noticed that sparse doesn't work on some not so > common architectures like ppc32le, s390x, ppc64 and sparc64[1]. This is > nicely catched by the testsuite, e.g.: The only architecture, from the above list, that is not supported by cgcc seems to be ppc32le. > ukleinek@plummer:~/sparse$ git rev-parse HEAD > 958c11c35d98417eb6b948bffe2dffed14eb3320 > ukleinek@plummer:~/sparse$ uname -a > Linux plummer 4.9.0-3-powerpc64le #1 SMP Debian 4.9.30-2+deb9u3 > (2017-08-06) ppc64le GNU/Linux > ukleinek@plummer:~/sparse$ make check V=1 It would be easier to see the results if you _didn't_ add V=1. ;-) [snip] > Out of 287 tests, 272 passed, 15 failed (10 of them are known to fail) > Makefile:232: recipe for target 'check' failed > make: *** [check] Error 1 > ukleinek@plummer:~/sparse$ The additional five failures are all in the llvm backend (sparsec), which you do not need to use sparse as a 'checker'. > The problem is that some cpp symbols are not defined in sparse that are > expected to exist. So I can "fix" backend/sum.c with the following > patch: > > diff --git a/validation/backend/sum.c b/validation/backend/sum.c > index 0604299..d0be8dd 100644 > --- a/validation/backend/sum.c > +++ b/validation/backend/sum.c > @@ -1,3 +1,5 @@ > +#define __powerpc64__ > +#define _CALL_ELF 2 > #include > #include > > Yep, sparse/sparsec do not define various macros that gcc/clang define by default on a given architecture. This is a known problem (that I have been meaning to fix ...). The 'workaround' for the time being is to use the cgcc front-end to sparse. (for example 'make CC=cgcc', or perhaps 'cgcc -no-compile'). [You didn't mention your usage - is this for a kernel build?] ATB, Ramsay Jones
Bug#873508: sparse test failures on ppc32le (and other not so common archs)
Hello, Antoine Beaupre (on Cc:) noticed that sparse doesn't work on some not so common architectures like ppc32le, s390x, ppc64 and sparc64[1]. This is nicely catched by the testsuite, e.g.: ukleinek@plummer:~/sparse$ git rev-parse HEAD 958c11c35d98417eb6b948bffe2dffed14eb3320 ukleinek@plummer:~/sparse$ uname -a Linux plummer 4.9.0-3-powerpc64le #1 SMP Debian 4.9.30-2+deb9u3 (2017-08-06) ppc64le GNU/Linux ukleinek@plummer:~/sparse$ make check V=1 cd validation && ./test-suite TESTWoverride-init-def (Woverride-init-def.c) Using command : ../sparse Woverride-init-def.c Expecting exit value: 0 TESTWoverride-init-no (Woverride-init-no.c) Using command : ../sparse -Wno-override-init Woverride-init-no.c Expecting exit value: 0 TESTWoverride-init-yes (Woverride-init-yes.c) Using command : ../sparse -Woverride-init Woverride-init-yes.c Expecting exit value: 0 TESTwarn-unknown-attribute (Wunknown-attribute-def.c) Using command : ../sparse Wunknown-attribute-def.c Expecting exit value: 0 TESTwarn-unknown-attribute-no (Wunknown-attribute-no.c) Using command : ../sparse -Wno-unknown-attribute Wunknown-attribute-no.c Expecting exit value: 0 TESTwarn-unknown-attribute-yes (Wunknown-attribute-yes.c) Using command : ../sparse -Wunknown-attribute Wunknown-attribute-yes.c Expecting exit value: 0 TEST__func__ (__func__.c) Using command : ../sparse -Wall __func__.c Expecting exit value: 0 TESTabstract array declarator static (abstract-array-declarator-static.c) Using command : ../sparse abstract-array-declarator-static.c Expecting exit value: 0 TESTaddress_space attribute (address_space.c) Using command : ../sparse address_space.c Expecting exit value: 0 TESTalias distinct symbols (alias-distinct.c) Using command : ../test-linearize alias-distinct.c Expecting exit value: 0 TESTalias symbol/pointer (alias-mixed.c) Using command : ../test-linearize alias-mixed.c Expecting exit value: 0 TESTalias same symbols (alias-same.c) Using command : ../test-linearize alias-same.c Expecting exit value: 0 TESTattribute __alloc_align__ (alloc-align.c) Using command : ../sparse alloc-align.c Expecting exit value: 0 TESTalternate keywords (alternate-keywords.c) Using command : ../sparse alternate-keywords.c Expecting exit value: 0 TESTtest anonymous union initializer (anon-union.c) Using command : ../sparse anon-union.c Expecting exit value: 0 TESTAsm with goto labels. (asm-empty-clobber.c) Using command : ../sparse asm-empty-clobber.c Expecting exit value: 0 TESTAsm with goto labels. (asm-goto-lables.c) Using command : ../sparse asm-goto-lables.c Expecting exit value: 0 TESTasm-toplevel.c (asm-toplevel.c) Using command : ../test-linearize asm-toplevel.c Expecting exit value: 0 TESTinline attributes (attr-inline.c) Using command : ../sparse attr-inline.c Expecting exit value: 0 TESTattribute no_sanitize_address (attr-no_sanitize_address.c) Using command : ../sparse attr-no_sanitize_address.c Expecting exit value: 0 TESTattribute noclone (attr-noclone.c) Using command : ../sparse attr-noclone.c Expecting exit value: 0 TESToptimize attributes (attr-optimize.c) Using command : ../sparse attr-optimize.c Expecting exit value: 0 TESTattribute warning (attr-warning.c) Using command : ../sparse attr-warning.c Expecting exit value: 0 TESTattribute assume_aligned (attr_aligned.c) Using command : ../sparse attr_aligned.c Expecting exit value: 0 TESTattribute after ( in direct-declarator (attr_in_parameter.c) Using command : ../sparse attr_in_parameter.c Expecting exit value: 0 TESTattribute vector_size (attr_vector_size.c)