Re: Gtk+ unit tests (brainstorming)

2006-11-16 Thread Tim Janik
On Thu, 26 Oct 2006, Tristan Van Berkom wrote: Tim Janik wrote: (sometime one property has no meaning if another one hasnt been setup yet - in which case a g_return_if_fail() guard would be appropriate). wrong, some proeprty values are intentionally set up to support free-order settings.

Re: Gtk+ unit tests (brainstorming)

2006-11-16 Thread Iago Toral Quiroga
El mié, 15-11-2006 a las 10:51 +0100, Iago Toral Quiroga escribió: I'll add here some points supporting Check ;): ok, adressing them one by one, since i see multiple reasons for not using Check ;) [...] it's not clear that Check (besides than being an additional dependency in

Re: Gtk+ unit tests (brainstorming)

2006-11-15 Thread Iago Toral Quiroga
El mar, 14-11-2006 a las 15:33 +0100, Tim Janik escribió: I understand your point, however I still think that being able to get a wider report with all the tests failing at a given moment is also interesting (for example in a buildbot continuous integration loop, like the one being

Re: Gtk+ unit tests (brainstorming)

2006-11-14 Thread Tim Janik
On Thu, 26 Oct 2006, Iago Toral Quiroga wrote: - in the common case, test results should be reduced to a single boolean: all tests passed vs. at least one test failed many test frameworks provide means to count and report failing tests (even automake's standard check:-rule),

Re: Gtk+ unit tests (brainstorming)

2006-11-10 Thread Carl Worth
On Tue, 31 Oct 2006 10:26:41 -0800, Carl Worth wrote: On Tue, 31 Oct 2006 15:26:35 +0100 (CET), Tim Janik wrote: i.e. using averaging, your numbers include uninteresting outliers that can result from scheduling artefacts (like measuring a whole second for copying a single pixel), and they

Re: Gtk+ unit tests (brainstorming)

2006-10-31 Thread Iago Toral Quiroga
Havoc Pennington wrote: Tim Janik wrote: ah, interesting. could you please explain why you consider it such a big win? Without it I think I usually write about 10% coverage, and imagine in my mind that it is 50% or so ;-) I'm guessing this is pretty common. With it, it was easy

Re: Gtk+ unit tests (brainstorming)

2006-10-31 Thread Tim Janik
On Wed, 25 Oct 2006, Federico Mena Quintero wrote: On Wed, 2006-10-25 at 17:52 +0200, Tim Janik wrote: - GLib based test programs should never produce a CRITICAL **: or WARNING **: message and succeed. Definitely. I have some simple code in autotestfilechooser.c to catch this and fail

Re: Gtk+ unit tests (brainstorming)

2006-10-31 Thread Tim Janik
On Wed, 25 Oct 2006, Carl Worth wrote: On Wed, 25 Oct 2006 12:40:27 -0500, Federico Mena Quintero wrote: There are some things I really don't like in cairo's make check suite right now: 2. The tests take forever to link. Each test right now is a separate program. I chose this originally

Re: Gtk+ unit tests (brainstorming)

2006-10-31 Thread Stefan Kost
Hi Tim, Tim Janik wrote: Hi all. as mentioned in another email already, i've recently worked on improving unit test integration in Beast and summarized this in my last blog entry: http://blogs.gnome.org/view/timj/2006/10/23/0 # Beast and unit testing I did a presentation on check

Re: Gtk+ unit tests (brainstorming)

2006-10-30 Thread Tim Janik
On Wed, 25 Oct 2006, Havoc Pennington wrote: Hi, When coding dbus I thought I'd try a project with a focus on unit tests. It has (or at least had for a while) exceptionally high test coverage, around 75% of basic blocks executed in make check. The coverage-analyzer has been busted for a

Re: Gtk+ unit tests (brainstorming)

2006-10-30 Thread Havoc Pennington
Tim Janik wrote: ah, interesting. could you please explain why you consider it such a big win? Without it I think I usually write about 10% coverage, and imagine in my mind that it is 50% or so ;-) I'm guessing this is pretty common. With it, it was easy to just browse and say OK, this

Re: Gtk+ unit tests (brainstorming)

2006-10-30 Thread mathieu lacage
On Mon, 2006-10-30 at 15:34 +0100, Tim Janik wrote: On Wed, 25 Oct 2006, Havoc Pennington wrote: Hi, When coding dbus I thought I'd try a project with a focus on unit tests. It has (or at least had for a while) exceptionally high test coverage, around 75% of basic blocks executed in

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Andy Wingo
Hi, On Wed, 2006-10-25 at 17:52 +0200, Tim Janik wrote: - GLib based test programs should never produce a CRITICAL **: or WARNING **: message and succeed. Sometimes it is useful to check that a critical message was indeed shown, and then move on. GStreamer installs a log handler that aborts

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Michael Urman
On 10/25/06, Havoc Pennington [EMAIL PROTECTED] wrote: Testing those is like testing segfault handling, i.e. just nuts. The behavior is undefined once they print. (Well, for critical anyway. g_warning seems to be less consistently used) Certainly setting out to test all critical cases would

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Tim Janik
On Thu, 26 Oct 2006, Michael Urman wrote: On 10/25/06, Havoc Pennington [EMAIL PROTECTED] wrote: Testing those is like testing segfault handling, i.e. just nuts. The behavior is undefined once they print. (Well, for critical anyway. g_warning seems to be less consistently used) Certainly

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Tristan Van Berkom
Havoc Pennington wrote: Michael Urman wrote: On 10/25/06, Tim Janik [EMAIL PROTECTED] wrote: - GLib based test programs should never produce a CRITICAL **: or WARNING **: message and succeed. It would be good not to make it impossible to test WARNINGs and CRITICALs. After

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Tim Janik
On Thu, 26 Oct 2006, Tristan Van Berkom wrote: Havoc Pennington wrote: Michael Urman wrote: On 10/25/06, Tim Janik [EMAIL PROTECTED] wrote: - GLib based test programs should never produce a CRITICAL **: or WARNING **: message and succeed. It would be good not to make it impossible

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Iago Toral Quiroga
El mié, 25-10-2006 a las 17:52 +0200, Tim Janik escribió: - Unit tests should run fast - a test taking 1/10th of a second is a slow unit test, i've mentioned this in my blog entry already. Sure, very important, or otherwise developers will tend to neither use nor maintain the tests. - in

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Tristan Van Berkom
Tim Janik wrote: On Thu, 26 Oct 2006, Tristan Van Berkom wrote: Havoc Pennington wrote: Michael Urman wrote: On 10/25/06, Tim Janik [EMAIL PROTECTED] wrote: - GLib based test programs should never produce a CRITICAL **: or WARNING **: message and succeed. It would be good not to

Re: Gtk+ unit tests (brainstorming)

2006-10-26 Thread Bill Haneman
Hi: - homogeneous or consistent test output might be desirable in some contexts. Yes, it is an important point when thinking about a continuous integration tool for Gnome. If tests for all modules in Gnome agree on a common output format, then that data can be collected, processed

Re: Gtk+ unit tests (brainstorming)

2006-10-25 Thread Federico Mena Quintero
On Wed, 2006-10-25 at 17:52 +0200, Tim Janik wrote: while analysing the need for a testing framework and whether it makes sense for GLib and Gtk+ to depend on yet another package for the sole purpose of testing, i made/had the following observations/thoughts: Wooo! Thanks for working on

Re: Gtk+ unit tests (brainstorming)

2006-10-25 Thread Michael Urman
On 10/25/06, Tim Janik [EMAIL PROTECTED] wrote: - GLib based test programs should never produce a CRITICAL **: or WARNING **: message and succeed. It would be good not to make it impossible to test WARNINGs and CRITICALs. After all, error cases are often the least tested part of an