On Fri, Feb 05, 2021 at 08:41:47AM +0100, Willy Tarreau wrote:
> Hi William,
>
> On Fri, Jan 29, 2021 at 02:44:27PM +0100, William Lallemand wrote:
> > Hello List,
> >
> > According to `make reg-tests-help` the REGTESTS_TYPES parameter must be
> > configured as an environment variable and not a make argument.
> >
> > However since patch 3bad3d5 ("BUILD: Makefile: exclude broken tests by
> > default"), it does not work anymore with an environment variable.
> >
> > Looking at the several CI files we have, it is used as a make
> > argument everywhere.
>
> I must confess I didn't even know it was expected to be used as an
> environment variable and have always used it as a make argument, given
> that these *are* exported as argument variables anyway.
I was surprised it was documented this way too, because we never used
the Makefile with environment variable before.
> Here's what I'm using for the make-reg-test-all script for example:
>
> make reg-tests VTEST_PROGRAM=/usr/local/bin/vtest
> REGTESTS_TYPES=default,bug,devel,slow -- --j 8 "$@" || exit 1
>
> > I'm going to update the `make reg-tests-help` command with the correct
> > syntax if there isn't any complain.
>
> OK.
>
> > We could fix the issue by using "?=" when setting the default value, but
> > it would make it the only variable that use "?=" in the Makefile and I'm
> > not sure we want to proceed this way.
>
> No please avoid this, I find the mixing of arguments and environment
> really confusing to use. The only valid use of "?=" is when you want
> to support pre-setting a variable that will be completed later,
> but this usually remains as it doesn't allow you to remove options,
> and in general it's really bad as you get different behaviors for
>
> $ VAR=xxx make
>
> and:
>
> $ make VAR=xxx
>
> I think the reg-test users are a small enough group that any required
> change can be quickly adopted if needed anyway, without having to take
> too much care about backwards compatibility, as long as it remains
> convenient to use.
>
Ok, I'm going to do the change in the help command then.
--
William Lallemand
