GCC Buildbot

2017-09-20 Thread Paulo Matos
-buildbot.linki.tools With brief documentation in: https://linkitools.github.io/gcc-buildbot and configuration in: https://github.com/LinkiTools/gcc-buildbot Now, this is still pretty raw but it: * Configures a fedora x86_64 for C, C++ and ObjectiveC (./configure --disable-multilib) * Does an incremental build

gcc buildbot?

2014-01-06 Thread Philippe Baril Lecavalier
Hi, Is anyone working on an implementation of buildbot for GCC? I have been experimenting with buildbot lately, and I would be glad to help in providing it. If there is interest, I could have a prototype and a detailed proposal ready in a few days. It could serve GCC, bin

Re: GCC Buildbot

2017-09-20 Thread Jeff Law
nine (which is numerically 0.9.x). > > To make a long story short: https://gcc-buildbot.linki.tools > With brief documentation in: https://linkitools.github.io/gcc-buildbot > and configuration in: https://github.com/LinkiTools/gcc-buildbot > > Now, this is still pretty raw but i

Re: GCC Buildbot

2017-09-20 Thread David Edelsohn
gt;> with a few private additions and ported it to the most recent buildbot >> version nine (which is numerically 0.9.x). >> >> To make a long story short: https://gcc-buildbot.linki.tools >> With brief documentation in: https://linkitools.github.io/gcc-buildbot >> an

Re: GCC Buildbot

2017-09-20 Thread Joseph Myers
On Wed, 20 Sep 2017, Paulo Matos wrote: > - buildbot can notify people if the build fails or if there's a test > regression. Notification can be sent to IRC and email for example. What > would people prefer to have as the settings for notifications? It's very useful if someone reviews regressions

Re: GCC Buildbot

2017-09-20 Thread R0b0t1
bot configuration for GDB. I then >>> stripped out his configuration and transformed it into one from GCC, >>> with a few private additions and ported it to the most recent buildbot >>> version nine (which is numerically 0.9.x). >>> >>> To make a long sto

Re: GCC Buildbot

2017-09-20 Thread Segher Boessenkool
Hi! On Wed, Sep 20, 2017 at 05:01:55PM +0200, Paulo Matos wrote: > This mail's intention is to gauge the interest of having a buildbot for > GCC. +1. Or no, +100. > - which machines we can use as workers: we certainly need more worker > (previously known as slave) machines to test GCC in differ

Re: GCC Buildbot

2017-09-20 Thread Joseph Myers
On Wed, 20 Sep 2017, Segher Boessenkool wrote: > > - buildbot can notify people if the build fails or if there's a test > > regression. Notification can be sent to IRC and email for example. What > > would people prefer to have as the settings for notifications? > > Just try it! IRC is most usef

Re: GCC Buildbot

2017-09-20 Thread Markus Trippelsdorf
On 2017.09.20 at 18:01 -0500, Segher Boessenkool wrote: > Hi! > > On Wed, Sep 20, 2017 at 05:01:55PM +0200, Paulo Matos wrote: > > This mail's intention is to gauge the interest of having a buildbot for > > GCC. > > +1. Or no, +100. > > > - which machines we can use as workers: we certainly nee

Re: GCC Buildbot

2017-09-20 Thread Paulo Matos
On 20/09/17 17:07, Jeff Law wrote: > I'd strongly recommend using one of the existing infrastructures. I > know several folks (myself included) are using Jenkins/Hudson. There's > little to be gained building a completely new infrastructure to manage a > buildbot. > As David pointed out in an

Re: GCC Buildbot

2017-09-21 Thread Mark Wielaard
Hi, First let me say I am also a fan of buildbot. I use it for a couple of projects and it is really flexible, low on resources, easy to add new builders/workers and easily extensible if you like python. On Thu, 2017-09-21 at 07:18 +0200, Markus Trippelsdorf wrote: > And it has the basic problem

Re: GCC Buildbot

2017-09-21 Thread Christophe Lyon
ot > version nine (which is numerically 0.9.x). > That's something I'd have liked to discuss at the Cauldron, but I couldn't attend. > To make a long story short: https://gcc-buildbot.linki.tools > With brief documentation in: https://linkitools.github.io/gcc-buildbot >

Re: GCC Buildbot

2017-09-21 Thread Martin Sebor
On 09/20/2017 11:18 PM, Markus Trippelsdorf wrote: On 2017.09.20 at 18:01 -0500, Segher Boessenkool wrote: Hi! On Wed, Sep 20, 2017 at 05:01:55PM +0200, Paulo Matos wrote: This mail's intention is to gauge the interest of having a buildbot for GCC. +1. Or no, +100. - which machines we can

Re: GCC Buildbot

2017-09-21 Thread Joseph Myers
On Thu, 21 Sep 2017, Markus Trippelsdorf wrote: > And it has the basic problem of all automatic testing: that in the long > run everyone simply ignores it. Hence, see my comments about the value of having someone who monitors the results and files bugs / notifies patch authors / fixes issues. I

Re: GCC Buildbot

2017-09-21 Thread Paulo Matos
On 20/09/17 19:14, Joseph Myers wrote: > On Wed, 20 Sep 2017, Paulo Matos wrote: > >> - buildbot can notify people if the build fails or if there's a test >> regression. Notification can be sent to IRC and email for example. What >> would people prefer to have as the settings for notifications?

Re: GCC Buildbot

2017-09-21 Thread Paulo Matos
On 21/09/17 01:01, Segher Boessenkool wrote: > Hi! > > On Wed, Sep 20, 2017 at 05:01:55PM +0200, Paulo Matos wrote: >> This mail's intention is to gauge the interest of having a buildbot for >> GCC. > > +1. Or no, +100. > >> - which machines we can use as workers: we certainly need more worke

Re: GCC Buildbot

2017-09-21 Thread Paulo Matos
On 21/09/17 02:27, Joseph Myers wrote: > On Wed, 20 Sep 2017, Segher Boessenkool wrote: > >>> - buildbot can notify people if the build fails or if there's a test >>> regression. Notification can be sent to IRC and email for example. What >>> would people prefer to have as the settings for notif

Re: GCC Buildbot

2017-09-21 Thread Paulo Matos
On 21/09/17 14:11, Mark Wielaard wrote: > Hi, > > First let me say I am also a fan of buildbot. I use it for a couple of > projects and it is really flexible, low on resources, easy to add new > builders/workers and easily extensible if you like python. > > On Thu, 2017-09-21 at 07:18 +0200, Ma

Re: GCC Buildbot

2017-09-21 Thread Paulo Matos
On 21/09/17 16:41, Martin Sebor wrote: > > The regression and the testresults lists are useful but not nearly > as much as they could be.  For one, the presentation isn't user > friendly (a matrix view would be much more informative).  But even > beyond it, rather than using the pull model (peop

Re: GCC Buildbot

2017-09-21 Thread Paulo Matos
On 21/09/17 14:18, Christophe Lyon wrote: >> >> If this is something of interest, then we will need to understand what >> is required, among those: >> >> - which machines we can use as workers: we certainly need more worker >> (previously known as slave) machines to test GCC in different >> archs

Re: GCC Buildbot

2017-09-21 Thread Joseph Myers
On Thu, 21 Sep 2017, Paulo Matos wrote: > I totally agree that only if people get involved in checking if there > were regressions and keeping an eye on what's going on are things going > to improve. The framework can help a lot here by notifying the right > people and the mailing list if somethin

Re: GCC Buildbot

2017-09-21 Thread Joseph Myers
On Thu, 21 Sep 2017, Paulo Matos wrote: > Interesting suggestion. I haven't had the opportunity to look at the > compile farm. However, it could be interesting to have a mix of workers: > native compile farm ones and some x86_64 doing cross compilation and > testing. Note that even without a simu

Re: GCC Buildbot

2017-09-22 Thread Paulo Matos
On 22/09/17 01:23, Joseph Myers wrote: > On Thu, 21 Sep 2017, Paulo Matos wrote: > >> Interesting suggestion. I haven't had the opportunity to look at the >> compile farm. However, it could be interesting to have a mix of workers: >> native compile farm ones and some x86_64 doing cross compilati

Re: GCC Buildbot

2017-09-22 Thread Joseph Myers
On Fri, 22 Sep 2017, Paulo Matos wrote: > > Note that even without a simulator (but with target libc), you can test > > just the compilation parts of the testsuite using a board file with a > > dummy _load implementation. > > > > I was not aware of that. I will keep that in mind once I try to

Re: GCC Buildbot

2017-09-25 Thread Martin Liška
Hi Paulo. Thank you for working on that! To be honest, I've been running local buildbot on my desktop machine which does builds your buildbot instance can do (please see: https://pasteboard.co/GLZ0vLMu.png): - doing time to time (once a week) sanitizer builds: ASAN, UBSAN and run test-suite - do

Re: GCC Buildbot

2017-09-25 Thread Paulo Matos
rchitectures > > That's list of what I have and can be inspiration for you. I can help if you > want and we can find a reasonable resources > where this can be run. > Thanks. That's great. As you can see from #9 in https://github.com/LinkiTools/gcc-buildbot/issues/9, m

Re: GCC Buildbot

2017-09-25 Thread Jonathan Wakely
On 25 September 2017 at 11:13, Paulo Matos wrote: >> Apart from that, I fully agree with octoploid that >> http://toolchain.lug-owl.de/buildbot/ is duplicated effort which is running >> on GCC compile farm machines and uses a shell scripts to utilize. I would >> prefer to integrate it to Buildbot

Re: GCC Buildbot

2017-09-25 Thread Martin Liška
us handling of dependencies of external libraries can be irritating. >> - cross build for primary architectures >> >> That's list of what I have and can be inspiration for you. I can help if you >> want and we can find a reasonable resources >> where this can be

Re: GCC Buildbot

2017-09-25 Thread Paulo Matos
On 25/09/17 13:14, Jonathan Wakely wrote: > On 25 September 2017 at 11:13, Paulo Matos wrote: >>> Apart from that, I fully agree with octoploid that >>> http://toolchain.lug-owl.de/buildbot/ is duplicated effort which is running >>> on GCC compile farm machines and uses a shell scripts to utiliz

Re: GCC Buildbot

2017-09-25 Thread Paulo Matos
On 25/09/17 13:36, Martin Liška wrote: > > Would be great, what exactly do you want to visualize? For me, even having > green/red spots > works fine in order to quickly identify what builds are wrong. > There are several options and I think mostly it depends on what everyone would like to see

Re: GCC Buildbot

2017-09-26 Thread Martin Liška
On 09/25/2017 02:49 PM, Paulo Matos wrote: > For benchmarks like Qt, blitz (as mentioned in the gcc testing page), we > can plot the build time of the benchmark and resulting size when > compiling for size. > Please consider using LNT: http://llvm.org/docs/lnt/ Usage: http://lnt.llvm.org/ I've

Re: GCC Buildbot

2017-09-26 Thread Paulo Matos
On 26/09/17 10:43, Martin Liška wrote: > On 09/25/2017 02:49 PM, Paulo Matos wrote: >> For benchmarks like Qt, blitz (as mentioned in the gcc testing page), we >> can plot the build time of the benchmark and resulting size when >> compiling for size. >> > > Please consider using LNT: > http://ll

GCC Buildbot Update

2017-12-14 Thread Paulo Matos
is a irc bot currently silent. *Regression analysis:* This is one of the most important issues to tackle and I have a solution in a branch regression-testing : https://github.com/LinkiTools/gcc-buildbot/tree/regression-testing using jamais-vu from David Malcolm to analyze the regressions. It needs

Re: gcc buildbot?

2014-01-10 Thread Richard Sandiford
Hi, Philippe Baril Lecavalier writes: > I have been experimenting with buildbot lately, and I would be glad to > help in providing it. If there is interest, I could have a prototype and > a detailed proposal ready in a few days. It could serve GCC, binutils > and some important libraries as well.

Re: gcc buildbot?

2014-01-10 Thread Christophe Lyon
On 01/10/14 10:11, Richard Sandiford wrote: Hi, Philippe Baril Lecavalier writes: I have been experimenting with buildbot lately, and I would be glad to help in providing it. If there is interest, I could have a prototype and a detailed proposal ready in a few days. It could serve GCC, binutil

Re: GCC Buildbot Update

2017-12-14 Thread David Malcolm
27;svn help cleanup' for details) https://gcc-lnt.linki.tools/#/builders/3/builds/388/steps/0/logs/stdio Is there a bug-tracking location for the buildbot? Presumably: https://github.com/LinkiTools/gcc-buildbot/issues ? *Reporters:* > > There is a single reporter which is a irc bot c

Re: GCC Buildbot Update

2017-12-14 Thread Christophe Lyon
ugh ssh. > I suspect you are hitting a bug introduced recently, and fixed by: https://gcc.gnu.org/ml/gcc-patches/2017-12/msg00434.html > *Reporters:* > > There is a single reporter which is a irc bot currently silent. > > *Regression analysis:* > > This is one of the most im

Re: GCC Buildbot Update

2017-12-14 Thread Markus Trippelsdorf
On 2017.12.14 at 21:32 +0100, Christophe Lyon wrote: > On 14 December 2017 at 09:56, Paulo Matos wrote: > > I got an email suggesting I add some aarch64 workers so I did: > > 4 workers from CF (gcc113, gcc114, gcc115 and gcc116); > > > Great, I thought the CF machines were reserved for developpers

Re: GCC Buildbot Update

2017-12-15 Thread Paulo Matos
this please let me know. > https://gcc-lnt.linki.tools/#/builders/3/builds/388/steps/0/logs/stdio > Apologies, https://gcc-lnt.linki.tools is currently incorrectly forwarding you to https://gcc-buildbot.linki.tools. I meant to have it return an error until I open that up. > Is there a bug

Re: GCC Buildbot Update

2017-12-15 Thread Paulo Matos
On 14/12/17 21:32, Christophe Lyon wrote: > Great, I thought the CF machines were reserved for developpers. > Good news you could add builders on them. > Oh. I have seen similar things happening on CF machines so I thought it was not a problem. I have never specifically asked for permission. >

Re: GCC Buildbot Update

2017-12-15 Thread Christophe Lyon
On 15 December 2017 at 10:19, Paulo Matos wrote: > > > On 14/12/17 21:32, Christophe Lyon wrote: >> Great, I thought the CF machines were reserved for developpers. >> Good news you could add builders on them. >> > > Oh. I have seen similar things happening on CF machines so I thought it > was not

Re: GCC Buildbot Update

2017-12-15 Thread Paulo Matos
On 15/12/17 08:42, Markus Trippelsdorf wrote: > > I don't think this is good news at all. > As I pointed out in a reply to Chris, I haven't seeked permission but I am pretty sure something similar runs in the CF machines from other projects. The downside is that if we can't use the CF, I hav

Re: GCC Buildbot Update

2017-12-15 Thread Paulo Matos
On 15/12/17 10:21, Christophe Lyon wrote: > And the patch was committed last night (r255659), so maybe your builds now > work? > Forgot to mention that. Yes, it built! https://gcc-buildbot.linki.tools/#/builders/5 -- Paulo Matos

Re: GCC Buildbot Update

2017-12-15 Thread Markus Trippelsdorf
On 2017.12.15 at 10:21 +0100, Paulo Matos wrote: > > > On 15/12/17 08:42, Markus Trippelsdorf wrote: > > > > I don't think this is good news at all. > > > > As I pointed out in a reply to Chris, I haven't seeked permission but I > am pretty sure something similar runs in the CF machines from

Re: GCC Buildbot Update

2017-12-15 Thread David Malcolm
On Fri, 2017-12-15 at 10:16 +0100, Paulo Matos wrote: > > On 14/12/17 12:39, David Malcolm wrote: [...] > > It looks like you're capturing the textual output from "jv compare" > > and > > using the exit code. Would you prefer to import "jv" as a python > > module and use some kind of API? Or a

Re: GCC Buildbot Update

2017-12-15 Thread Segher Boessenkool
On Fri, Dec 15, 2017 at 08:42:18AM +0100, Markus Trippelsdorf wrote: > On 2017.12.14 at 21:32 +0100, Christophe Lyon wrote: > > On 14 December 2017 at 09:56, Paulo Matos wrote: > > > I got an email suggesting I add some aarch64 workers so I did: > > > 4 workers from CF (gcc113, gcc114, gcc115 and

Re: GCC Buildbot Update

2017-12-16 Thread Paulo Matos
On 15/12/17 15:29, David Malcolm wrote: > On Fri, 2017-12-15 at 10:16 +0100, Paulo Matos wrote: >> >> On 14/12/17 12:39, David Malcolm wrote: > > [...] > >>> It looks like you're capturing the textual output from "jv compare" >>> and >>> using the exit code. Would you prefer to import "jv" as

Re: GCC Buildbot Update

2017-12-16 Thread Paulo Matos
On 15/12/17 18:05, Segher Boessenkool wrote: > All the cfarm machines are shared resources. Benchmarking on them will > not work no matter what. And being a shared resource means all users > have to share and be mindful of others. > Yes, we'll definitely need better machines for benchmarking.

Re: GCC Buildbot Update

2017-12-20 Thread Paulo Matos
On 15/12/17 10:21, Christophe Lyon wrote: > On 15 December 2017 at 10:19, Paulo Matos wrote: >> >> >> On 14/12/17 21:32, Christophe Lyon wrote: >>> Great, I thought the CF machines were reserved for developpers. >>> Good news you could add builders on them. >>> >> >> Oh. I have seen similar thin

Re: GCC Buildbot Update

2017-12-20 Thread Christophe Lyon
On 20 December 2017 at 09:31, Paulo Matos wrote: > > > On 15/12/17 10:21, Christophe Lyon wrote: >> On 15 December 2017 at 10:19, Paulo Matos wrote: >>> >>> >>> On 14/12/17 21:32, Christophe Lyon wrote: Great, I thought the CF machines were reserved for developpers. Good news you could

Re: GCC Buildbot Update

2017-12-20 Thread Paulo Matos
On 20/12/17 10:51, Christophe Lyon wrote: > > The recent fix changed the Makefile and configure script in libatomic. > I guess that if your incremental builds does not run configure, it's > still using old Makefiles, and old options. > > You're right. I guess incremental builds should always c

Re: GCC Buildbot Update

2017-12-20 Thread Christophe Lyon
On 20 December 2017 at 11:02, Paulo Matos wrote: > > > On 20/12/17 10:51, Christophe Lyon wrote: >> >> The recent fix changed the Makefile and configure script in libatomic. >> I guess that if your incremental builds does not run configure, it's >> still using old Makefiles, and old options. >> >>

Re: GCC Buildbot Update

2017-12-20 Thread James Greenhalgh
On Wed, Dec 20, 2017 at 10:02:45AM +, Paulo Matos wrote: > > > On 20/12/17 10:51, Christophe Lyon wrote: > > > > The recent fix changed the Makefile and configure script in libatomic. > > I guess that if your incremental builds does not run configure, it's > > still using old Makefiles, and

Re: GCC Buildbot Update

2017-12-20 Thread Paulo Matos
On 20/12/17 12:48, James Greenhalgh wrote: > On Wed, Dec 20, 2017 at 10:02:45AM +, Paulo Matos wrote: >> >> >> On 20/12/17 10:51, Christophe Lyon wrote: >>> >>> The recent fix changed the Makefile and configure script in libatomic. >>> I guess that if your incremental builds does not run conf

GCC Buildbot Update - Definition of regression

2017-10-10 Thread Paulo Matos
Hi all, It's almost 3 weeks since I last posted on GCC Buildbot. Here's an update: * 3 x86_64 workers from CF are now installed; * There's one scheduler for trunk doing fresh builds for every Daily bump; * One scheduler doing incremental builds for each active branch; * An I

Re: GCC Buildbot Update - Definition of regression

2017-10-10 Thread Joseph Myers
On Tue, 10 Oct 2017, Paulo Matos wrote: > ANY -> no test ; Test disappears No, that's not a regression. Simply adding a line to a testcase will change the line number that appears in the PASS / FAIL line for an individual assertion therein. Or the names will change when e.g. -std=c++2a

Re: GCC Buildbot Update - Definition of regression

2017-10-10 Thread Markus Trippelsdorf
On 2017.10.10 at 21:45 +0200, Paulo Matos wrote: > Hi all, > > It's almost 3 weeks since I last posted on GCC Buildbot. Here's an update: > > * 3 x86_64 workers from CF are now installed; > * There's one scheduler for trunk doing fresh builds for every Da

Re: GCC Buildbot Update - Definition of regression

2017-10-10 Thread Paulo Matos
On 11/10/17 06:17, Markus Trippelsdorf wrote: > On 2017.10.10 at 21:45 +0200, Paulo Matos wrote: >> Hi all, >> >> It's almost 3 weeks since I last posted on GCC Buildbot. Here's an update: >> >> * 3 x86_64 workers from CF are now installed; >> * T

Re: GCC Buildbot Update - Definition of regression

2017-10-10 Thread Paulo Matos
On 10/10/17 23:25, Joseph Myers wrote: > On Tue, 10 Oct 2017, Paulo Matos wrote: > >> new test -> FAIL; New test starts as fail > > No, that's not a regression, but you might want to treat it as one (in the > sense that it's a regression at the higher level of "testsuite run should

Re: GCC Buildbot Update - Definition of regression

2017-10-10 Thread Markus Trippelsdorf
On 2017.10.11 at 08:22 +0200, Paulo Matos wrote: > > > On 11/10/17 06:17, Markus Trippelsdorf wrote: > > On 2017.10.10 at 21:45 +0200, Paulo Matos wrote: > >> Hi all, > >> > >> It's almost 3 weeks since I last posted on GCC Buildbot. Here's an

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Christophe Lyon
On 11 October 2017 at 08:34, Paulo Matos wrote: > > > On 10/10/17 23:25, Joseph Myers wrote: >> On Tue, 10 Oct 2017, Paulo Matos wrote: >> >>> new test -> FAIL; New test starts as fail >> >> No, that's not a regression, but you might want to treat it as one (in the >> sense that it's a

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Paulo Matos
On 11/10/17 10:35, Christophe Lyon wrote: > > FWIW, we consider regressions: > * any->FAIL because we don't want such a regression at the whole testsuite > level > * any->UNRESOLVED for the same reason > * {PASS,UNSUPPORTED,UNTESTED,UNRESOLVED}-> XPASS > * new XPASS > * XFAIL disappears (may me

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Christophe Lyon
On 11 October 2017 at 11:03, Paulo Matos wrote: > > > On 11/10/17 10:35, Christophe Lyon wrote: >> >> FWIW, we consider regressions: >> * any->FAIL because we don't want such a regression at the whole testsuite >> level >> * any->UNRESOLVED for the same reason >> * {PASS,UNSUPPORTED,UNTESTED,UNRE

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Paulo Matos
On 11/10/17 11:15, Christophe Lyon wrote: > > You can have a look at > https://git.linaro.org/toolchain/gcc-compare-results.git/ > where compare_tests is a patched version of the contrib/ script, > it calls the main perl script (which is not the prettiest thing :-) > Thanks, that's useful. I w

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Jonathan Wakely
On 11 October 2017 at 07:34, Paulo Matos wrote: > When someone adds a new test to the testsuite, isn't it supposed to not > FAIL? Yes, but sometimes it FAILs because the test is using a new feature that only works on some targets, and the new test was missing the right directives to make it UNSUPP

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread David Malcolm
On Wed, 2017-10-11 at 11:18 +0200, Paulo Matos wrote: > > On 11/10/17 11:15, Christophe Lyon wrote: > > > > You can have a look at > > https://git.linaro.org/toolchain/gcc-compare-results.git/ > > where compare_tests is a patched version of the contrib/ script, > > it calls the main perl script (

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Joseph Myers
On Wed, 11 Oct 2017, Paulo Matos wrote: > On 10/10/17 23:25, Joseph Myers wrote: > > On Tue, 10 Oct 2017, Paulo Matos wrote: > > > >> new test -> FAIL; New test starts as fail > > > > No, that's not a regression, but you might want to treat it as one (in the > > sense that it's a re

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Joseph Myers
On Wed, 11 Oct 2017, Christophe Lyon wrote: > * {PASS,UNSUPPORTED,UNTESTED,UNRESOLVED}-> XPASS I don't think any of these should be considered regressions. It's good if someone manually checks anything that's *consistently* XPASSing, to see if the XFAIL should be removed or restricted to narro

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Marc Glisse
On Wed, 11 Oct 2017, David Malcolm wrote: On Wed, 2017-10-11 at 11:18 +0200, Paulo Matos wrote: On 11/10/17 11:15, Christophe Lyon wrote: You can have a look at https://git.linaro.org/toolchain/gcc-compare-results.git/ where compare_tests is a patched version of the contrib/ script, it calls

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Martin Sebor
PASS -> ANY ; Test moves away from PASS No, only a regression if the destination result is FAIL (if it's UNRESOLVED then there might be a separate regression - execution test becoming UNRESOLVED should be accompanied by compilation becoming FAIL). If it's XFAIL, it might formally

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Andreas Schwab
On Okt 10 2017, Joseph Myers wrote: > Anything else -> FAIL and new FAILing tests aren't regressions at the > individual test level, but may be treated as such at the whole testsuite > level. An ICE FAIL is a regression, but this is always a new test. Andreas. -- Andreas Schwab, SUSE Labs,

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Joseph Myers
On Wed, 11 Oct 2017, Martin Sebor wrote: > I don't have a strong opinion on the definition of a Regression > in this context but I would very much like to see status changes > highlighted in the test results to indicate that something that There are lots of things that are useful *if* you have so

Re: GCC Buildbot Update - Definition of regression

2017-10-11 Thread Hans-Peter Nilsson
On Tue, 10 Oct 2017, Paulo Matos wrote: > This is a suggestion. I am keen to have corrections from people who use > this on a daily basis and/or have a better understanding of each status. Not mentioning them (oddly I don't see anyone mentioning them) makes me think you've not looked there so all

Re: GCC Buildbot Update - Definition of regression

2017-10-13 Thread David Malcolm
On Wed, 2017-10-11 at 16:17 +0200, Marc Glisse wrote: > On Wed, 11 Oct 2017, David Malcolm wrote: > > > On Wed, 2017-10-11 at 11:18 +0200, Paulo Matos wrote: > > > > > > On 11/10/17 11:15, Christophe Lyon wrote: > > > > > > > > You can have a look at > > > > https://git.linaro.org/toolchain/gcc-

jamais-vu can now ignore renumbering of source lines in dg output (Re: GCC Buildbot Update)

2018-01-24 Thread David Malcolm
On Sat, 2017-12-16 at 12:06 +0100, Paulo Matos wrote: > > On 15/12/17 15:29, David Malcolm wrote: > > On Fri, 2017-12-15 at 10:16 +0100, Paulo Matos wrote: > > > > > > On 14/12/17 12:39, David Malcolm wrote: > > > > [...] > > > > > > It looks like you're capturing the textual output from "jv >

Re: jamais-vu can now ignore renumbering of source lines in dg output (Re: GCC Buildbot Update)

2018-01-24 Thread Paulo Matos
On 24/01/18 20:20, David Malcolm wrote: > > I've added a new feature to jamais-vu (as of > 77849e2809ca9a049d5683571e27ebe190977fa8): it can now ignore test > results that merely changed line number. > > For example, if the old .sum file has a: > > PASS: g++.dg/diagnostic/param-type-mismat

Re: jamais-vu can now ignore renumbering of source lines in dg output (Re: GCC Buildbot Update)

2018-01-29 Thread Paulo Matos
On 24/01/18 20:20, David Malcolm wrote: > > I've added a new feature to jamais-vu (as of > 77849e2809ca9a049d5683571e27ebe190977fa8): it can now ignore test > results that merely changed line number. > > For example, if the old .sum file has a: > > PASS: g++.dg/diagnostic/param-type-mismat

Re: jamais-vu can now ignore renumbering of source lines in dg output (Re: GCC Buildbot Update)

2018-01-29 Thread David Malcolm
On Mon, 2018-01-29 at 14:55 +0100, Paulo Matos wrote: > > On 24/01/18 20:20, David Malcolm wrote: > > > > I've added a new feature to jamais-vu (as of > > 77849e2809ca9a049d5683571e27ebe190977fa8): it can now ignore test > > results that merely changed line number. > > > > For example, if the

Re: jamais-vu can now ignore renumbering of source lines in dg output (Re: GCC Buildbot Update)

2018-01-29 Thread Paulo Matos
On 29/01/18 15:19, David Malcolm wrote: >> >> Hi, >> >> I am looking at this today and I noticed that having the source file >> for >> all recent GCC revisions is costly in terms of time (if we wish to >> compress them) and space (for storage). I was instead thinking that >> jv >> could calculate