Thanks Robert for your answer.Grouping tests is a good idea, thanks for the
reminder. I'll use that if new flakiness
show up and if I have no countermeasures left :)
Etienne
Le lundi 21 janvier 2019 à 12:39 +0100, Robert Bradshaw a écrit :
> I am of the same opinion, this is the approach we're taking for Flinkas well.
> Various mitigations (e.g. capping the
> parallelism at 2 ratherthan the default of num cores) have helped.
> Several times the idea has been proposed to group unit tests togetherfor
> "expensive" backends. E.g. for self-contained
> tests one can createa single pipeline that contains all the tests with their
> asserts, andthen run that once to
> amortize the overhead (which is quitesignificant when you're only
> manipulating literally bytes of data).Only on
> failure would it exercise them individually (eithersequentially, or via a
> binary search).
> On Mon, Jan 21, 2019 at 10:07 AM Etienne Chauchot <echauc...@apache.org>
> wrote:
>
> Hi guys,
> Lately I have been fixing various Elasticsearch flakiness issues in the
> UTests by: introducing timeouts, countdown
> latches, force refresh, embedded cluster size decrease ...
> These flakiness issues are due to the embedded Elasticsearch not coping well
> with the jenkins overload. Still, IMHO I
> believe that having embedded backend for UTests are a lot better than mocks.
> Even if they are less tolerant to load, I
> prefer having UTests 100% representative of real backend and add
> countermeasures to protect against jenkins overload.
> WDYT ?
> Etienne
>
