----- Original Message -----
> From: "Aleksandar Markovic" <aleksandar.m.m...@gmail.com>
> To: "Cleber Rosa" <cr...@redhat.com>
> Cc: "Wainer dos Santos Moschetta" <waine...@redhat.com>, "Aleksandar
> Markovic" <amarko...@wavecomp.com>,
> qemu-devel@nongnu.org, "Aleksandar Rikalo" <arik...@wavecomp.com>, "Eduardo
> Habkost" <ehabk...@redhat.com>,
> "Aurelien Jarno" <aurel...@aurel32.net>, "Philippe Mathieu-Daudé"
> <f4...@amsat.org>
> Sent: Wednesday, May 22, 2019 6:43:54 PM
> Subject: Re: [Qemu-devel] [PATCH 0/4] mips: Add more Avocado tests
>
> On May 22, 2019 11:46 PM, "Cleber Rosa" <cr...@redhat.com> wrote:
> >
> >
> >
> > ----- Original Message -----
> > > From: "Eduardo Habkost" <ehabk...@redhat.com>
> > > To: "Philippe Mathieu-Daudé" <f4...@amsat.org>
> > > Cc: qemu-devel@nongnu.org, "Aleksandar Rikalo" <arik...@wavecomp.com>,
> "Aleksandar Markovic"
> > > <aleksandar.m.m...@gmail.com>, "Aleksandar Markovic" <
> amarko...@wavecomp.com>, "Cleber Rosa" <cr...@redhat.com>,
> > > "Aurelien Jarno" <aurel...@aurel32.net>, "Wainer dos Santos Moschetta" <
> waine...@redhat.com>
> > > Sent: Wednesday, May 22, 2019 5:12:30 PM
> > > Subject: Re: [Qemu-devel] [PATCH 0/4] mips: Add more Avocado tests
> > >
> > > On Tue, May 21, 2019 at 01:19:06AM +0200, Philippe Mathieu-Daudé wrote:
> > > > Hi,
> > > >
> > > > It was a rainy week-end here, so I invested it to automatize some
> > > > of my MIPS tests.
> > > >
> > > > The BootLinuxSshTest is not Global warming friendly, it is not
> > > > meant to run on a CI system but rather on a workstation previous
> > > > to post a pull request.
> > > > It can surely be improved, but it is a good starting point.
> > >
> > > Until we actually have a mechanism to exclude the test case on
> > > travis-ci, I will remove patch 4/4 from the queue. Aleksandar,
> > > please don't merge patch 4/4 yet or it will break travis-ci.
> > >
> > > Cleber, Wainer, is it already possible to make "avocado run" skip
> > > tests tagged with "slow"?
> > >
> >
> > The mechanism exists, but we haven't tagged any test so far as slow.
> >
>
> Cleber,
>
> For the test from patch 4/4, there is no dilemma - it should be in the
> “slow” group, as Philippe envisioned and said, so that it is not humpered
> with stricter requirements for “fast” (default) group. Could you explain us
> how to do it, so that we can hopefully finally proceed?
>
Hi Aleksandar,
The point is that there's no "group" definition at this point. This is the
core of the discussion.
I think we're close to converging to something simple and effective. Please
let us know what you think of the proposals given.
Thanks!
- Cleber.
> Gratefully,
> Aleksandar
>
> > Should we define/document a criteria for a test to be slow? Given
> > that this is highly subjective, we have to think of:
> >
> > * Will we consider the average or maximum run time (the timeout
> > definition)?
> >
> > * For a single test, what is "slow"? Some rough numbers from Travis
> > CI[1] to help us with guidelines:
> > - boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: PASS (6.04 s)
> > - boot_linux_console.py:BootLinuxConsole.test_arm_virt: PASS (2.91 s)
> > -
> linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16:
> PASS (18.14 s)
> > - boot_linux.py:BootLinuxAarch64.test_virt: PASS (396.88 s)
> >
> > * Do we want to set a maximum job timeout? This way we can skip
> > tests after a given amount of time has passed. Currently we interrupt
> > the test running when the job timeout is reached, but it's possible
> > to add a option so that no new tests will be started, but currently
> > running ones will be waited on.
> >
> > Regards,
> > - Cleber.
> >
> > [1] - https://travis-ci.org/clebergnu/qemu/jobs/535967210#L3518
> >
> > > --
> > > Eduardo
> > >
>
