Re: Update and Questions on CPython Extension Module -fanalyzer plugin development
Hi Dave, Tests related to our plugin which depend on Python-specific definitions have been run by including /* { dg-options "-fanalyzer -I/usr/include/python3.9" } */. This is undoubtedly not ideal; is it best to approach this problem by adapting a subset of relevant definitions like in gil.h? Best, Eric On Tue, Aug 1, 2023 at 1:06 PM David Malcolm wrote: > > On Tue, 2023-08-01 at 09:57 -0400, Eric Feng wrote: > > > > > > My guess is that you were trying to do it from the > > > PLUGIN_ANALYZER_INIT > > > hook rather than from the plugin_init function, but it's hard to be > > > sure without seeing the code. > > > > > > > Thanks Dave, you are entirely right — I made the mistake of trying to > > do it from PLUGIN_ANALYZER_INIT hook and not from the plugin_init > > function. After following your suggestion, the callbacks are getting > > registered as expected. > > Ah, good. > > > I submitted a patch to review for this feature > > on gcc-patches; please let me know if it looks OK. > > Thanks Eric; I've posted a reply to your email there, so let's discuss > the details there. > > Dave >
Re: Update and Questions on CPython Extension Module -fanalyzer plugin development
On Fri, 2023-08-04 at 11:02 -0400, Eric Feng wrote: > Hi Dave, > > Tests related to our plugin which depend on Python-specific > definitions have been run by including /* { dg-options "-fanalyzer > -I/usr/include/python3.9" } */. This is undoubtedly not ideal; is it > best to approach this problem by adapting a subset of relevant > definitions like in gil.h? That might be acceptable in the very short-term, but to create a plugin that's useful to end-user (authors of CPython extension modules) we want to be testing against real Python headers. As I understand it, https://peps.python.org/pep-0394/ allows for distributors of Python to symlink "python3-config" in the PATH to a python3.X-config script (for some X). So on such systems running: python3-config --includes should emit the correct -I option. On my box it emits: -I/usr/include/python3.8 -I/usr/include/python3.8 It's more complicated, but I believe: python3-config --cflags should emit the build flags that C/C++ extensions ought to use when building. On my box this emits: -I/usr/include/python3.8 -I/usr/include/python3.8 -Wno-unused-result - Wsign-compare -O2 -g -pipe -Wall -Werror=format-security -Wp,- D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack- protector-strong -grecord-gcc-switches -m64 -mtune=generic - fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection - D_GNU_SOURCE -fPIC -fwrapv -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG - O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,- D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord- gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables - fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv and it's likely going to vary from distribution to distribution. Some of those options *are* going to affect the gimple that -fanalyzer "sees". Does your installation of Python have such a script? So in the short term you could hack in a minimal subset of the decls/defns from Python.h, but I'd prefer it if target-supports.exp gained a DejaGnu directive that invokes python3-config, captures the result (or fails with UNSUPPORTED for systems without python3 development headers), and then adds the result to the build flags of the file being tested. The .exp files are implemented in Tcl, alas; let me know if you want help with that. Dave > > Best, > Eric > > On Tue, Aug 1, 2023 at 1:06 PM David Malcolm > wrote: > > > > On Tue, 2023-08-01 at 09:57 -0400, Eric Feng wrote: > > > > > > > > My guess is that you were trying to do it from the > > > > PLUGIN_ANALYZER_INIT > > > > hook rather than from the plugin_init function, but it's hard > > > > to be > > > > sure without seeing the code. > > > > > > > > > > Thanks Dave, you are entirely right — I made the mistake of > > > trying to > > > do it from PLUGIN_ANALYZER_INIT hook and not from the plugin_init > > > function. After following your suggestion, the callbacks are > > > getting > > > registered as expected. > > > > Ah, good. > > > > > I submitted a patch to review for this feature > > > on gcc-patches; please let me know if it looks OK. > > > > Thanks Eric; I've posted a reply to your email there, so let's > > discuss > > the details there. > > > > Dave > > >
The gcc version used for compiling qt4.
Did you ever solve this problem? I'm getting the exact same error.
Re: Update and Questions on CPython Extension Module -fanalyzer plugin development
On Fri, Aug 4, 2023 at 11:39 AM David Malcolm wrote: > > On Fri, 2023-08-04 at 11:02 -0400, Eric Feng wrote: > > Hi Dave, > > > > Tests related to our plugin which depend on Python-specific > > definitions have been run by including /* { dg-options "-fanalyzer > > -I/usr/include/python3.9" } */. This is undoubtedly not ideal; is it > > best to approach this problem by adapting a subset of relevant > > definitions like in gil.h? > > That might be acceptable in the very short-term, but to create a plugin > that's useful to end-user (authors of CPython extension modules) we > want to be testing against real Python headers. > > As I understand it, https://peps.python.org/pep-0394/ allows for > distributors of Python to symlink "python3-config" in the PATH to a > python3.X-config script (for some X). > > So on such systems running: > python3-config --includes > should emit the correct -I option. On my box it emits: > > -I/usr/include/python3.8 -I/usr/include/python3.8 > > > It's more complicated, but I believe: > python3-config --cflags > should emit the build flags that C/C++ extensions ought to use when > building. On my box this emits: > > -I/usr/include/python3.8 -I/usr/include/python3.8 -Wno-unused-result - > Wsign-compare -O2 -g -pipe -Wall -Werror=format-security -Wp,- > D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack- > protector-strong -grecord-gcc-switches -m64 -mtune=generic - > fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection - > D_GNU_SOURCE -fPIC -fwrapv -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG - > O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,- > D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord- > gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables - > fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv > > and it's likely going to vary from distribution to distribution. Some > of those options *are* going to affect the gimple that -fanalyzer > "sees". > > Does your installation of Python have such a script? > > So in the short term you could hack in a minimal subset of the > decls/defns from Python.h, but I'd prefer it if target-supports.exp > gained a DejaGnu directive that invokes python3-config, captures the > result (or fails with UNSUPPORTED for systems without python3 > development headers), and then adds the result to the build flags of > the file being tested. The .exp files are implemented in Tcl, alas; > let me know if you want help with that. > > Dave Sounds good; thanks! Following existing examples in target-supports.exp, the following works as expected in terms of extracting the build flags we are interested in. In target-supports.exp: proc check_python_flags { } { set result [remote_exec host "python3-config --cflags"] set status [lindex $result 0] if { $status == 0 } { return [lindex $result 1] } else { return "UNSUPPORTED" } } However, I'm having some trouble figuring out the specifics as to how we may add the build flags to our test cases. My intuition looks like something like the following: In plugin.exp: foreach plugin_test $plugin_test_list { if {[lindex $plugin_test 0] eq "analyzer_cpython_plugin.c"} { set python_flags [check_python_flags] if { $python_flags ne "UNSUPPORTED" } { // append $python_flags to build flags here } } } How might we do so? > > > > > > Best, > > Eric > > > > On Tue, Aug 1, 2023 at 1:06 PM David Malcolm > > wrote: > > > > > > On Tue, 2023-08-01 at 09:57 -0400, Eric Feng wrote: > > > > > > > > > > My guess is that you were trying to do it from the > > > > > PLUGIN_ANALYZER_INIT > > > > > hook rather than from the plugin_init function, but it's hard > > > > > to be > > > > > sure without seeing the code. > > > > > > > > > > > > > Thanks Dave, you are entirely right — I made the mistake of > > > > trying to > > > > do it from PLUGIN_ANALYZER_INIT hook and not from the plugin_init > > > > function. After following your suggestion, the callbacks are > > > > getting > > > > registered as expected. > > > > > > Ah, good. > > > > > > > I submitted a patch to review for this feature > > > > on gcc-patches; please let me know if it looks OK. > > > > > > Thanks Eric; I've posted a reply to your email there, so let's > > > discuss > > > the details there. > > > > > > Dave > > > > > >
David Faust appointed BPF backend reviewer
I am pleased to announce that the GCC Steering Committee has appointed David Faust as BPF backend reviewer. Please join me in congratulating David on his new role. David, please update your listings in the MAINTAINERS file. Happy hacking! David
gcc-12-20230804 is now available
Snapshot gcc-12-20230804 is now available on https://gcc.gnu.org/pub/gcc/snapshots/12-20230804/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 12 git branch with the following options: git://gcc.gnu.org/git/gcc.git branch releases/gcc-12 revision 913dc14bc63dd5d508ee7a2def1a1b3f035bc778 You'll find: gcc-12-20230804.tar.xz Complete GCC SHA256=0289ed9f7e0b10abaae3bb99e97b702c426f05a5bc7e48d3906f1a36cc3f86a2 SHA1=f45fecd15e488bf0209e68600943e7d288578043 Diffs from 12-20230728 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-12 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
Re: Update and Questions on CPython Extension Module -fanalyzer plugin development
On Fri, 2023-08-04 at 16:48 -0400, Eric Feng wrote: > On Fri, Aug 4, 2023 at 11:39 AM David Malcolm > wrote: > > > > On Fri, 2023-08-04 at 11:02 -0400, Eric Feng wrote: > > > Hi Dave, > > > > > > Tests related to our plugin which depend on Python-specific > > > definitions have been run by including /* { dg-options "- > > > fanalyzer > > > -I/usr/include/python3.9" } */. This is undoubtedly not ideal; is > > > it > > > best to approach this problem by adapting a subset of relevant > > > definitions like in gil.h? > > > > That might be acceptable in the very short-term, but to create a > > plugin > > that's useful to end-user (authors of CPython extension modules) we > > want to be testing against real Python headers. > > > > As I understand it, https://peps.python.org/pep-0394/ allows for > > distributors of Python to symlink "python3-config" in the PATH to a > > python3.X-config script (for some X). > > > > So on such systems running: > > python3-config --includes > > should emit the correct -I option. On my box it emits: > > > > -I/usr/include/python3.8 -I/usr/include/python3.8 > > > > > > It's more complicated, but I believe: > > python3-config --cflags > > should emit the build flags that C/C++ extensions ought to use when > > building. On my box this emits: > > > > -I/usr/include/python3.8 -I/usr/include/python3.8 -Wno-unused- > > result - > > Wsign-compare -O2 -g -pipe -Wall -Werror=format-security -Wp,- > > D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack- > > protector-strong -grecord-gcc-switches -m64 -mtune=generic - > > fasynchronous-unwind-tables -fstack-clash-protection -fcf- > > protection - > > D_GNU_SOURCE -fPIC -fwrapv -DDYNAMIC_ANNOTATIONS_ENABLED=1 - > > DNDEBUG - > > O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 - > > Wp,- > > D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong - > > grecord- > > gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables - > > fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv > > > > and it's likely going to vary from distribution to distribution. > > Some > > of those options *are* going to affect the gimple that -fanalyzer > > "sees". > > > > Does your installation of Python have such a script? > > > > So in the short term you could hack in a minimal subset of the > > decls/defns from Python.h, but I'd prefer it if target-supports.exp > > gained a DejaGnu directive that invokes python3-config, captures > > the > > result (or fails with UNSUPPORTED for systems without python3 > > development headers), and then adds the result to the build flags > > of > > the file being tested. The .exp files are implemented in Tcl, > > alas; > > let me know if you want help with that. > > > > Dave > Sounds good; thanks! Following existing examples in > target-supports.exp, the following works as expected in terms of > extracting the build flags we are interested in. > > In target-supports.exp: > proc check_python_flags { } { > set result [remote_exec host "python3-config --cflags"] > set status [lindex $result 0] > if { $status == 0 } { > return [lindex $result 1] > } else { > return "UNSUPPORTED" > } > } > > However, I'm having some trouble figuring out the specifics as to how > we may add the build flags to our test cases. My intuition looks like > something like the following: > > In plugin.exp: > foreach plugin_test $plugin_test_list { > if {[lindex $plugin_test 0] eq "analyzer_cpython_plugin.c"} { > set python_flags [check_python_flags] > if { $python_flags ne "UNSUPPORTED" } { > // append $python_flags to build flags here > } > } > > } > > How might we do so? Good question. Looking at plugin.exp I see it uses plugin-test-execute, which is defined in gcc/testsuite/lib/plugin-support.exp. Looking there, I see it attempts to build the plugin, and then if it succeeds, it calls dg-runtest $plugin_tests $plugin_enabling_flags $default_flags where $plugin_tests is the list of source files to be compiled using the plugin. So one way to do this would be to modify that code from plugin.exp to pass in a different value, rather than $default_flags. Though it seems hackish to special-case this. As another way, that avoids adding special-casing to plugin.exp, there's an existing directive: dg-additional-options implemented in gcc/testsuite/lib/gcc-defs.exp which appends options to the default options. Unfortunately, it works via: upvar dg-extra-tool-flags extra-tool-flags which is a nasty Tcl hack meaning access the local variable named "dg- extra-tool-flags" in *the frame above*, referring to it as "extra-tool- flags". (this is why I don't like Tcl) So I think what could be done is to invoke your "check_python_flags" test as a directive from the test case, so that in target-supports.exp you'd have something like: proc dg-require-python-h {} { which could do the invocation/
Re: Update and Questions on CPython Extension Module -fanalyzer plugin development
On Fri, 2023-08-04 at 18:42 -0400, David Malcolm wrote: > On Fri, 2023-08-04 at 16:48 -0400, Eric Feng wrote: > > On Fri, Aug 4, 2023 at 11:39 AM David Malcolm > > wrote: > > > > > > On Fri, 2023-08-04 at 11:02 -0400, Eric Feng wrote: > > > > Hi Dave, > > > > > > > > Tests related to our plugin which depend on Python-specific > > > > definitions have been run by including /* { dg-options "- > > > > fanalyzer > > > > -I/usr/include/python3.9" } */. This is undoubtedly not ideal; > > > > is > > > > it > > > > best to approach this problem by adapting a subset of relevant > > > > definitions like in gil.h? > > > > > > That might be acceptable in the very short-term, but to create a > > > plugin > > > that's useful to end-user (authors of CPython extension modules) > > > we > > > want to be testing against real Python headers. > > > > > > As I understand it, https://peps.python.org/pep-0394/ allows for > > > distributors of Python to symlink "python3-config" in the PATH to > > > a > > > python3.X-config script (for some X). > > > > > > So on such systems running: > > > python3-config --includes > > > should emit the correct -I option. On my box it emits: > > > > > > -I/usr/include/python3.8 -I/usr/include/python3.8 > > > > > > > > > It's more complicated, but I believe: > > > python3-config --cflags > > > should emit the build flags that C/C++ extensions ought to use > > > when > > > building. On my box this emits: > > > > > > -I/usr/include/python3.8 -I/usr/include/python3.8 -Wno-unused- > > > result - > > > Wsign-compare -O2 -g -pipe -Wall -Werror=format-security -Wp,- > > > D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions - > > > fstack- > > > protector-strong -grecord-gcc-switches -m64 -mtune=generic - > > > fasynchronous-unwind-tables -fstack-clash-protection -fcf- > > > protection - > > > D_GNU_SOURCE -fPIC -fwrapv -DDYNAMIC_ANNOTATIONS_ENABLED=1 - > > > DNDEBUG - > > > O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 > > > - > > > Wp,- > > > D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong - > > > grecord- > > > gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables - > > > fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC - > > > fwrapv > > > > > > and it's likely going to vary from distribution to distribution. > > > Some > > > of those options *are* going to affect the gimple that -fanalyzer > > > "sees". > > > > > > Does your installation of Python have such a script? > > > > > > So in the short term you could hack in a minimal subset of the > > > decls/defns from Python.h, but I'd prefer it if target- > > > supports.exp > > > gained a DejaGnu directive that invokes python3-config, captures > > > the > > > result (or fails with UNSUPPORTED for systems without python3 > > > development headers), and then adds the result to the build flags > > > of > > > the file being tested. The .exp files are implemented in Tcl, > > > alas; > > > let me know if you want help with that. > > > > > > Dave > > Sounds good; thanks! Following existing examples in > > target-supports.exp, the following works as expected in terms of > > extracting the build flags we are interested in. > > > > In target-supports.exp: > > proc check_python_flags { } { > > set result [remote_exec host "python3-config --cflags"] > > set status [lindex $result 0] > > if { $status == 0 } { > > return [lindex $result 1] > > } else { > > return "UNSUPPORTED" > > } > > } > > > > However, I'm having some trouble figuring out the specifics as to > > how > > we may add the build flags to our test cases. My intuition looks > > like > > something like the following: > > > > In plugin.exp: > > foreach plugin_test $plugin_test_list { > > if {[lindex $plugin_test 0] eq "analyzer_cpython_plugin.c"} { > > set python_flags [check_python_flags] > > if { $python_flags ne "UNSUPPORTED" } { > > // append $python_flags to build flags here > > } > > } > > > > } > > > > How might we do so? > > Good question. > > Looking at plugin.exp I see it uses plugin-test-execute, which is > defined in gcc/testsuite/lib/plugin-support.exp. > > Looking there, I see it attempts to build the plugin, and then if it > succeeds, it calls > dg-runtest $plugin_tests $plugin_enabling_flags $default_flags > where $plugin_tests is the list of source files to be compiled using > the plugin. So one way to do this would be to modify that code from > plugin.exp to pass in a different value, rather than $default_flags. > Though it seems hackish to special-case this. Sorry, I think I misspoke here; that line that uses $default_flags is from plugin-support.exp, not from plugin.exp; $default_flags is a global variable. So I think my 2nd approach below may be the one to try: > > As another way, that avoids adding special-casing to plugin.exp, > there's an existing directive: > dg-additional-options > implemented in