Re: RFC: Monitoring old PRs, new dg directives [v2]

2020-11-10 Thread Marek Polacek via Gcc-patches
On Tue, Nov 10, 2020 at 03:15:00PM +0100, Thomas Schwinge wrote: > Hi! > > On 2020-08-10T17:30:21-0400, Marek Polacek via Gcc-patches > wrote: > > This patch adds a new DejaGNU directive, dg-ice, as outlined in the > > proposal here: > >

Re: RFC: Monitoring old PRs, new dg directives [v2]

2020-11-10 Thread Thomas Schwinge
Hi! On 2020-08-10T17:30:21-0400, Marek Polacek via Gcc-patches wrote: > This patch adds a new DejaGNU directive, dg-ice, as outlined in the > proposal here: > https://gcc.gnu.org/pipermail/gcc-patches/2020-July/550913.html > > It means that it's expected that the compiler crashes with an

Re: RFC: Monitoring old PRs, new dg directives [v2]

2020-08-10 Thread Mike Stump via Gcc-patches
On Aug 10, 2020, at 2:30 PM, Marek Polacek via Gcc-patches wrote: > > Thanks a lot. Here's the latest version of my patch, which only adds dg-ice > at this point. > > So, um, OK? Ok.

Re: RFC: Monitoring old PRs, new dg directives

2020-08-10 Thread Mike Stump via Gcc-patches
On Aug 7, 2020, at 6:16 AM, Nathan Sidwell wrote: > > On 8/6/20 8:01 PM, Mike Stump wrote: >> On Aug 6, 2020, at 7:01 AM, Nathan Sidwell wrote: >>> XFAIL: g++.dg/foo.C -std=c++17 (internal compiler error) PASS: g++.dg/foo.C -std=c++17 (test for excess errors) Which one of

Re: RFC: Monitoring old PRs, new dg directives

2020-08-10 Thread Marek Polacek via Gcc-patches
On Mon, Aug 10, 2020 at 08:58:54AM -0400, Nathan Sidwell wrote: > On 8/10/20 4:48 AM, Richard Sandiford wrote: > > Marek Polacek via Gcc-patches writes: > > > > > > sure. > > > > > > * develop patch > > > > > > * run testsuite > > > > > > * observe unexpected ICEs > > > > > > * load g++.log into

Re: RFC: Monitoring old PRs, new dg directives [v2]

2020-08-10 Thread Marek Polacek via Gcc-patches
On Mon, Aug 10, 2020 at 09:48:06AM +0100, Richard Sandiford wrote: > Marek Polacek via Gcc-patches writes: > >> > > sure. > >> > > * develop patch > >> > > * run testsuite > >> > > * observe unexpected ICEs > >> > > * load g++.log into editor > >> > > * ^sinternal comp > >> > > * gets to first

Re: RFC: Monitoring old PRs, new dg directives

2020-08-10 Thread Nathan Sidwell
On 8/10/20 4:48 AM, Richard Sandiford wrote: Marek Polacek via Gcc-patches writes: sure. * develop patch * run testsuite * observe unexpected ICEs * load g++.log into editor * ^sinternal comp * gets to first unexpected ICE * debug it. What does '^sinternal comp' become? As there could be

Re: RFC: Monitoring old PRs, new dg directives

2020-08-10 Thread Richard Sandiford
Marek Polacek via Gcc-patches writes: >> > > sure. >> > > * develop patch >> > > * run testsuite >> > > * observe unexpected ICEs >> > > * load g++.log into editor >> > > * ^sinternal comp >> > > * gets to first unexpected ICE >> > > * debug it. >> > > >> > > What does '^sinternal comp' become?

Re: RFC: Monitoring old PRs, new dg directives

2020-08-09 Thread Martin Liška
On 8/5/20 12:22 AM, Marek Polacek wrote: On Thu, Jul 30, 2020 at 11:08:03AM +0200, Martin Liška wrote: Hello. I support the initiative! What would be nice to add leading 'PR component/12345' to a git commit so that these test additions are linked to bugzilla issues. Thanks! Yes, it should

Re: RFC: Monitoring old PRs, new dg directives

2020-08-07 Thread Marek Polacek via Gcc-patches
On Fri, Aug 07, 2020 at 09:21:27AM -0400, Nathan Sidwell wrote: > On 8/6/20 6:55 PM, Marek Polacek wrote: > > On Thu, Aug 06, 2020 at 10:01:37AM -0400, Nathan Sidwell wrote: > > > On 8/5/20 7:29 PM, Marek Polacek wrote: > > > > On Wed, Aug 05, 2020 at 11:03:08AM -0400, Nathan Sidwell wrote: > > >

Re: RFC: Monitoring old PRs, new dg directives

2020-08-07 Thread Nathan Sidwell
On 8/6/20 6:55 PM, Marek Polacek wrote: On Thu, Aug 06, 2020 at 10:01:37AM -0400, Nathan Sidwell wrote: On 8/5/20 7:29 PM, Marek Polacek wrote: On Wed, Aug 05, 2020 at 11:03:08AM -0400, Nathan Sidwell wrote: On 8/4/20 8:54 PM, Marek Polacek via Gcc-patches wrote: On Tue, Aug 04, 2020 at

Re: RFC: Monitoring old PRs, new dg directives

2020-08-07 Thread Nathan Sidwell
On 8/6/20 8:01 PM, Mike Stump wrote: On Aug 6, 2020, at 7:01 AM, Nathan Sidwell wrote: XFAIL: g++.dg/foo.C -std=c++17 (internal compiler error) PASS: g++.dg/foo.C -std=c++17 (test for excess errors) Which one of these would you not like to see? Neither of these is solving the issue. How

Re: RFC: Monitoring old PRs, new dg directives

2020-08-06 Thread Mike Stump via Gcc-patches
On Aug 6, 2020, at 7:01 AM, Nathan Sidwell wrote: > >> XFAIL: g++.dg/foo.C -std=c++17 (internal compiler error) >> PASS: g++.dg/foo.C -std=c++17 (test for excess errors) >> Which one of these would you not like to see? > > Neither of these is solving the issue. How do I find the ICES that

Re: RFC: Monitoring old PRs, new dg directives

2020-08-06 Thread Marek Polacek via Gcc-patches
On Thu, Aug 06, 2020 at 10:01:37AM -0400, Nathan Sidwell wrote: > On 8/5/20 7:29 PM, Marek Polacek wrote: > > On Wed, Aug 05, 2020 at 11:03:08AM -0400, Nathan Sidwell wrote: > > > On 8/4/20 8:54 PM, Marek Polacek via Gcc-patches wrote: > > > > On Tue, Aug 04, 2020 at 03:33:23PM -0700, Mike Stump

Re: RFC: Monitoring old PRs, new dg directives

2020-08-06 Thread Nathan Sidwell
On 8/5/20 7:29 PM, Marek Polacek wrote: On Wed, Aug 05, 2020 at 11:03:08AM -0400, Nathan Sidwell wrote: On 8/4/20 8:54 PM, Marek Polacek via Gcc-patches wrote: On Tue, Aug 04, 2020 at 03:33:23PM -0700, Mike Stump wrote: I think the read of the room is that people think it would be generally

Re: RFC: Monitoring old PRs, new dg directives

2020-08-06 Thread Marek Polacek via Gcc-patches
On Thu, Aug 06, 2020 at 02:30:11PM +0200, Jakub Jelinek wrote: > On Thu, Aug 06, 2020 at 08:27:05AM -0400, Marek Polacek via Gcc-patches wrote: > > // PR c++/88003 > > // { dg-do compile { target c++14 } } > > // { dg-prune-output ".*internal compiler error.*" } > > // { dg-xfail-if "" { *-*-* } }

Re: RFC: Monitoring old PRs, new dg directives

2020-08-06 Thread Jakub Jelinek via Gcc-patches
On Thu, Aug 06, 2020 at 08:27:05AM -0400, Marek Polacek via Gcc-patches wrote: > // PR c++/88003 > // { dg-do compile { target c++14 } } > // { dg-prune-output ".*internal compiler error.*" } > // { dg-xfail-if "" { *-*-* } } Would that XPASS if the ICE is fixed though? Jakub

Re: RFC: Monitoring old PRs, new dg directives

2020-08-06 Thread Marek Polacek via Gcc-patches
On Wed, Aug 05, 2020 at 01:01:32PM -0700, Mike Stump wrote: > On Aug 4, 2020, at 5:54 PM, Marek Polacek wrote: > >> As you find it difficult to express a test using the existing mechanisms, > >> let's talk about those and see if anyone has a good idea on how to express > >> it. I think ICEs

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Marek Polacek via Gcc-patches
On Wed, Aug 05, 2020 at 11:03:08AM -0400, Nathan Sidwell wrote: > On 8/4/20 8:54 PM, Marek Polacek via Gcc-patches wrote: > > On Tue, Aug 04, 2020 at 03:33:23PM -0700, Mike Stump wrote: > > > I think the read of the room is that people think it would be generally > > > useful, so let approve the

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Mike Stump via Gcc-patches
On Aug 4, 2020, at 5:54 PM, Marek Polacek wrote: >> As you find it difficult to express a test using the existing mechanisms, >> let's talk about those and see if anyone has a good idea on how to express >> it. I think ICEs are the most annoying to manage, but, I think excess and >> prune

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Marek Polacek via Gcc-patches
On Tue, Aug 04, 2020 at 08:54:32PM -0400, Marek Polacek via Gcc-patches wrote: > On Tue, Aug 04, 2020 at 03:33:23PM -0700, Mike Stump wrote: > > A word of caution, if we produce core files, before you add tons of core > > file producing test cases, you'll want to submit a, ulimit -c 0 patch that

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Mike Stump via Gcc-patches
On Aug 5, 2020, at 5:56 AM, Marek Polacek wrote: > > Sorry for being unclear. Say you have > > // PR c++/55408 > > struct foo { >template >void bar(); > }; > > template > void foo::bar() {} > > int main() > { >extern int i; >foo {}.bar<>(); > } > > which we wrongly accept.

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Nathan Sidwell
On 8/4/20 8:54 PM, Marek Polacek via Gcc-patches wrote: On Tue, Aug 04, 2020 at 03:33:23PM -0700, Mike Stump wrote: I think the read of the room is that people think it would be generally useful, so let approve the general plan. Cool. So, now we are down to the fine details. Please do see

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Marek Polacek via Gcc-patches
On Wed, Aug 05, 2020 at 08:58:40AM +0100, Richard Sandiford wrote: > Marek Polacek via Gcc-patches writes: > > On Thu, Jul 30, 2020 at 11:54:23AM +0200, Jakub Jelinek via Gcc-patches > > wrote: > >> On Tue, Jul 28, 2020 at 05:44:47PM -0400, Marek Polacek via Gcc-patches > >> wrote: > >> > We

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Marek Polacek via Gcc-patches
On Wed, Aug 05, 2020 at 09:04:53AM +0100, Richard Sandiford wrote: > Marek Polacek writes: > > On Wed, Jul 29, 2020 at 09:40:35AM +0100, Richard Sandiford wrote: > >> I guess there's a possibility that some tests happen to pass already > >> on some targets. That's more likely with middle-end and

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Marek Polacek via Gcc-patches
On Tue, Aug 04, 2020 at 03:45:11PM -0700, Mike Stump wrote: > On Aug 4, 2020, at 3:08 PM, Marek Polacek via Gcc-patches > wrote: > > > > That works well if you know where to expect an error. But if you don't, > > it's > > worse. E.g., > > > > // { dg-xfail-if "" { *-*-* } } > > int i =

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Richard Sandiford
Marek Polacek writes: > On Wed, Jul 29, 2020 at 09:40:35AM +0100, Richard Sandiford wrote: >> I guess there's a possibility that some tests happen to pass already >> on some targets. That's more likely with middle-end and backend bugs >> rather than frontend stuff though. Perhaps for those it

Re: RFC: Monitoring old PRs, new dg directives

2020-08-05 Thread Richard Sandiford
Marek Polacek via Gcc-patches writes: > On Thu, Jul 30, 2020 at 11:54:23AM +0200, Jakub Jelinek via Gcc-patches wrote: >> On Tue, Jul 28, 2020 at 05:44:47PM -0400, Marek Polacek via Gcc-patches >> wrote: >> > We will still have a surfeit of bugs that we've given short shrift to, but >> > let's

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Tue, Aug 04, 2020 at 03:53:50PM -0700, Mike Stump wrote: > On Aug 4, 2020, at 3:16 PM, Marek Polacek via Gcc-patches > wrote: > > > > The benefit of dg-accepts-invalid was that you would > > get an XPASS even for a test that should not be accepted, but you didn't > > know > > what line to

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Tue, Aug 04, 2020 at 03:33:23PM -0700, Mike Stump wrote: > I think the read of the room is that people think it would be generally > useful, so let approve the general plan. Cool. > So, now we are down to the fine details. Please do see just how far you can > stretch the existing

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Mike Stump via Gcc-patches
On Aug 4, 2020, at 3:16 PM, Marek Polacek via Gcc-patches wrote: > > The benefit of dg-accepts-invalid was that you would > get an XPASS even for a test that should not be accepted, but you didn't know > what line to expect an error on, so you put a dg-error at the end of the test. I think for

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Mike Stump via Gcc-patches
On Aug 4, 2020, at 3:08 PM, Marek Polacek via Gcc-patches wrote: > > That works well if you know where to expect an error. But if you don't, it's > worse. E.g., > > // { dg-xfail-if "" { *-*-* } } > int i = nothere; // demonstrates something that errors out > // { dg-error "" } didn't know

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Thu, Jul 30, 2020 at 11:54:23AM +0200, Jakub Jelinek via Gcc-patches wrote: > On Tue, Jul 28, 2020 at 05:44:47PM -0400, Marek Polacek via Gcc-patches wrote: > > We will still have a surfeit of bugs that we've given short shrift to, but > > let's at least automate what we can. The initial

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Mike Stump via Gcc-patches
I think the read of the room is that people think it would be generally useful, so let approve the general plan. So, now we are down to the fine details. Please do see just how far you can stretch the existing mechanisms to cover what you need to do. I think the existing mechanisms should be

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Thu, Jul 30, 2020 at 11:08:03AM +0200, Martin Liška wrote: > Hello. > > I support the initiative! > What would be nice to add leading 'PR component/12345' > to a git commit so that these test additions are linked to bugzilla issues. Thanks! Yes, it should be clear which test tests a PR that

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Wed, Jul 29, 2020 at 04:00:27PM -0600, Martin Sebor wrote: > I've created a much more rudimentary setup for myself to deal > with the same problem. I copy tests from Bugzilla, sometimes > with tweaks, and compile them from time to time as I revisit > unresolved bugs. I've also thought about

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Wed, Jul 29, 2020 at 04:37:03PM -0400, Jason Merrill wrote: > On Tue, Jul 28, 2020 at 5:45 PM Marek Polacek via Gcc-patches < > gcc-patches@gcc.gnu.org> wrote: > > > In Bugzilla, for the c++ component, we currently have over 3200 open > > bugs. In > > my experience, a good amount of them have

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Wed, Jul 29, 2020 at 09:40:35AM +0100, Richard Sandiford wrote: > Thanks for doing this. +1 for the best fix being to add XFAILing tests > to the main testsute, enabled by default. I don't see any other realistic > way of ensuring that fixes are matched with PRs at the time that the fix > is

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
On Tue, Jul 28, 2020 at 09:02:17PM -0600, Jeff Law wrote: > On Tue, 2020-07-28 at 17:44 -0400, Marek Polacek via Gcc-patches wrote: > > In Bugzilla, for the c++ component, we currently have over 3200 open bugs. > > In > > my experience, a good amount of them have already been fixed; my

Re: RFC: Monitoring old PRs, new dg directives

2020-08-04 Thread Marek Polacek via Gcc-patches
Hi Mike, thanks for your comments. On Tue, Jul 28, 2020 at 06:37:26PM -0700, Mike Stump via Gcc-patches wrote: > I'll punt to the the C++ front-end folks to chime in. Usually we only check > in bugs that are fixed, as they are fixed, this is what makes it a regression > suite. Doing this

Re: RFC: Monitoring old PRs, new dg directives

2020-07-30 Thread Jakub Jelinek via Gcc-patches
On Tue, Jul 28, 2020 at 05:44:47PM -0400, Marek Polacek via Gcc-patches wrote: > We will still have a surfeit of bugs that we've given short shrift to, but > let's at least automate what we can. The initial addition of the relevant > old(-ish) tests won't of course happen automagically, but it's

Re: RFC: Monitoring old PRs, new dg directives

2020-07-30 Thread Martin Liška
Hello. I support the initiative! What would be nice to add leading 'PR component/12345' to a git commit so that these test additions are linked to bugzilla issues. Martin

Re: RFC: Monitoring old PRs, new dg directives

2020-07-29 Thread Martin Sebor via Gcc-patches
I've created a much more rudimentary setup for myself to deal with the same problem. I copy tests from Bugzilla, sometimes with tweaks, and compile them from time to time as I revisit unresolved bugs. I've also thought about adding those to the test suite and marking them XFAIL but I don't

Re: RFC: Monitoring old PRs, new dg directives

2020-07-29 Thread Jason Merrill via Gcc-patches
On Tue, Jul 28, 2020 at 5:45 PM Marek Polacek via Gcc-patches < gcc-patches@gcc.gnu.org> wrote: > In Bugzilla, for the c++ component, we currently have over 3200 open > bugs. In > my experience, a good amount of them have already been fixed; my periodical > sweeps always turn up a bunch of PRs

Re: RFC: Monitoring old PRs, new dg directives

2020-07-29 Thread Richard Sandiford
Thanks for doing this. +1 for the best fix being to add XFAILing tests to the main testsute, enabled by default. I don't see any other realistic way of ensuring that fixes are matched with PRs at the time that the fix is made (rather than some time after the fact). Marek Polacek via Gcc-patches

Re: RFC: Monitoring old PRs, new dg directives

2020-07-28 Thread Jeff Law via Gcc-patches
On Tue, 2020-07-28 at 17:44 -0400, Marek Polacek via Gcc-patches wrote: > In Bugzilla, for the c++ component, we currently have over 3200 open bugs. In > my experience, a good amount of them have already been fixed; my periodical > sweeps always turn up a bunch of PRs that had already been fixed

Re: RFC: Monitoring old PRs, new dg directives

2020-07-28 Thread Mike Stump via Gcc-patches
I'll punt to the the C++ front-end folks to chime in. Usually we only check in bugs that are fixed, as they are fixed, this is what makes it a regression suite. Doing this does have advantages, like, the testsuite is small and doesn't have duplicates and doesn't test anything that is known to

RFC: Monitoring old PRs, new dg directives

2020-07-28 Thread Marek Polacek via Gcc-patches
In Bugzilla, for the c++ component, we currently have over 3200 open bugs. In my experience, a good amount of them have already been fixed; my periodical sweeps always turn up a bunch of PRs that had already been fixed previously. Sometimes my sweeps are more or less random, but more often than