On Thu, 1 Nov 2007, Stefan Kost wrote: > Hi Tim,
> Now some comments about the API > g_test_create_case -> g_test_case_create > g_test_create_suite -> g_test_suite_create i think this is a bit of a philosophy issue. i'd like to think about the new testing framework as one integrated thing, and that allows creation of suites and cases (that leads to the names i suggested). alternatively, GTestCase and GTestSuite can be viewed as self standing objects with API, which would lead to the names you suggest. due to the very limited scope of the object APIs and the tight coupling with the rest of the suite for actually running suites/cases, i'm more inclined to the former naming/view though ;) > I would not export g_test_trap_fork(), it should be a parameter to the test > runner wheter it forks all tests or not. Does -k (keep-going) implies that > tests > are forked? no, the forking API is used only by test developers and only for tests that are supposed to fail or run asyncronously. i have parts of that already implemented, so these are working examples that use the forking API: /* prototype, updated from the initial proposal */ gboolean g_test_trap_fork (guint64 usec_timeout, GTestTrapFlags test_trap_flags); /* fork around a failing test and assert the intended cause of failure */ if (g_test_trap_fork (0, G_TEST_TRAP_SILENCE_STDERR)) { g_assert_not_reached(); } g_test_trap_assert_failed(); g_test_trap_assert_stderr ("*ERROR*should not be reached*"); /* fork out to interrupt the test after a specific timeout */ if (g_test_trap_fork (1.5 * 1000000, 0)) // give it 1.5 secs at maximum { /* loop and sleep forever */ while (TRUE) g_usleep (1000 * 1000); } g_test_trap_assert_failed(); // testing timeout logic here g_assert (g_test_trap_reached_timeout()); // ensure it failed due to timeout /* fork out to assert stdout pattern */ if (g_test_trap_fork (0, G_TEST_TRAP_SILENCE_STDOUT)) { g_print ("some stdout text: somagic17\n"); exit (0); } g_test_trap_assert_passed(); g_test_trap_assert_stdout ("*somagic17*"); the idea is to let the actual test binaries never do forks unless the above API is used (but then, it's the test writer who forks out) and to let the runner always fork+exec the test binaries. that way, test binaries stay very simple and straight forward according to the test implementation, so can be easily debugged in gdb. then, if launched via the runner, if a test binary actually fails a test (and thus needs to abort because we can't catch exceptions in C), the runner can fork+exec a new instance of the test binary to execute the next test, in case --keep-going was specified. > Would it make sense to allow overridable test_result loggers. This way a ide > can > drive the test and gather result without the need to parse stdout. we intend to either allow multiple runner implementations (e.g. gtester for the command line, and another for ide integration), or to simply interface with the xml logging output that gtester is supposed to produce. > One problem with unit-tests is that one can only tests the public api. The can > sometimes be worked around a bit with providing mock obejects. I am wondering > if > it would be useful to have internal tests inside the real code, call this from > the tests (maybe a separate aspect) and wrap those internal function with some > #ifdef so that is can be ommited by default from releases. i think at the very least, we'll need some hooks into Gtk+ for things like installing mock object vtable. my current idea is to wrap that up like: #ifdef GTK_INTERNAL_ABI void gtk_test_install_widget_vtable (GtkTestVTableWithManyHooks *vtable); #endif so it's not part of the public API, but can be used by test binaries built inside the gtk+ package tree which link against libgtk-2.0.so. but this isn't part of the glib testing framework, and as i said, i'll send out an email on gtk testing routines at another point. > Finally, how would the logger output look like. check support a normal and > verbose mode. Providing similar formated logs would give us instant reports on > http://build.gnome.org i'd ideally like gtester to supply all valuable information in an xml file, so we can implement arbitrary report generators on top of that. there are so many different report targets and needs by various people, that i think we can only cover all if people can plug their report generators. > Okay, before my mail exceeds yours, I'll stop here, heh ;) thanks for the input. > Stefan --- ciaoTJ _______________________________________________ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list