Re: Proposal for integration tests infrastructure

2014-11-04 Thread Honza Horak
Tim, thanks for your comments, I think we're on the same page in basically all aspects you mention. It seems like the issue was more wording and I still fail to find some better term for the tests that I was referring to -- so, how to ideally call the tests that are not unit tests any more,

Re: Proposal for integration tests infrastructure

2014-11-04 Thread Colin Walters
This looks related to: https://wiki.gnome.org/GnomeGoals/InstalledTests (Note that the Issues with make check is equivalent to issues with rpm %check) It's implemented by gnome-continuous, and there's been a bit of effort to make -tests subpackages for some pieces in Fedora, but AFAIK no

Re: Proposal for integration tests infrastructure

2014-11-04 Thread Matthias Clasen
On Tue, 2014-11-04 at 04:56 -0500, Colin Walters wrote: This looks related to: https://wiki.gnome.org/GnomeGoals/InstalledTests (Note that the Issues with make check is equivalent to issues with rpm %check) It's implemented by gnome-continuous, and there's been a bit of effort to make

Re: Proposal for integration tests infrastructure

2014-11-03 Thread Honza Horak
On 10/28/2014 08:08 AM, Nick Coghlan wrote: On 10/22/2014 09:43 PM, Honza Horak wrote: Fedora lacks integration testing (unit testing done during build is not enough). Taskotron will be able to fill some gaps in the future, so maintainers will be able to set-up various tasks after their

Re: Proposal for integration tests infrastructure

2014-11-03 Thread Honza Horak
On 10/24/2014 07:18 PM, John Dulaney wrote: How to deliver tests? a/ just use them directly from git (we need to keep some metadata for dependencies anyway) b/ package them as RPMs (we can keep metadata there; e.g. Taskotron will run only tests that have Provides: ci-tests(mariadb) after mariadb

Re: Proposal for integration tests infrastructure

2014-11-03 Thread Honza Horak
On 10/28/2014 08:08 AM, Nick Coghlan wrote: Note that any or all of the above may be appropriate, depending on the exact nature of the specific tests. For example, there are already some public Beaker installer tests at https://bitbucket.org/fedoraqa/fedora-beaker-tests for execution on

Re: Proposal for integration tests infrastructure

2014-11-03 Thread Tim Flink
On Mon, 03 Nov 2014 17:22:27 +0100 Honza Horak hho...@redhat.com wrote: On 10/28/2014 08:08 AM, Nick Coghlan wrote: Note that any or all of the above may be appropriate, depending on the exact nature of the specific tests. For example, there are already some public Beaker installer tests

Re: Proposal for integration tests infrastructure

2014-11-03 Thread Tim Flink
On Mon, 03 Nov 2014 17:08:40 +0100 Honza Horak hho...@redhat.com wrote: On 10/28/2014 08:08 AM, Nick Coghlan wrote: On 10/22/2014 09:43 PM, Honza Horak wrote: Fedora lacks integration testing (unit testing done during build is not enough). Taskotron will be able to fill some gaps in the

RE: Proposal for integration tests infrastructure

2014-10-24 Thread John Dulaney
Some thoughts: Where to keep tests? a/ in current dist-git for related components (problem with sharing parts of code, problem where to keep tests related for more components) b/ in separate git with similar functionality as dist-git (needs new infrastructure, components are not directly

Re: Proposal for integration tests infrastructure

2014-10-24 Thread Tim Flink
On Wednesday, October 22, 2014 01:43:57 PM you wrote: Fedora lacks integration testing (unit testing done during build is not enough). Taskotron will be able to fill some gaps in the future, so maintainers will be able to set-up various tasks after their component is built. But even before

Re: Proposal for integration tests infrastructure

2014-10-24 Thread Tim Flink
On Fri, 24 Oct 2014 14:10:23 -0600 Tim Flink tfl...@redhat.com wrote: On Wednesday, October 22, 2014 01:43:57 PM you wrote: Fedora lacks integration testing (unit testing done during build is not enough). Taskotron will be able to fill some gaps in the future, so maintainers will be able

Re: Proposal for integration tests infrastructure

2014-10-23 Thread Stef Walter
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22.10.2014 13:43, Honza Horak wrote: Fedora lacks integration testing (unit testing done during build is not enough). Taskotron will be able to fill some gaps in the future, so maintainers will be able to set-up various tasks after their

Proposal for integration tests infrastructure

2014-10-22 Thread Honza Horak
Fedora lacks integration testing (unit testing done during build is not enough). Taskotron will be able to fill some gaps in the future, so maintainers will be able to set-up various tasks after their component is built. But even before this works we can benefit from having the tests already