Re: [vpp-dev] Augment 'make test' with C build tests?

[Dave Wallace]

Burt,

On 08/23/2017 08:31 PM, Burt Silverman wrote:

Dave Wallace wrote:

IMO, "in-tree" .vs. "out-of-tree" really boils down to decoupling
the app's "Makefile.am" the rest of the vpp autotools
structure/configuration.  For example, I ran into the same issue
with .../vppsb/vcl-ldpreload/src/Makefile.am (which is literally
'out-of-tree') in the case where "VPP_DIR" is specified.  However,
the vcl-ldpreload build would work the same if it was moved
somewhere under vpp without integrating it into
.../vpp/src/Makefile.am -- thus it would equally be "out-of-tree"
even though it was stored in the vpp repo.



​I would modify this statement just a bit: It is really configure.ac 
 rather than Makefile.am that defines the package 
boundary. Looking at the AC_CONFIG_FILES statement of the configure.ac 
, we see that the VPP package has, underneath 
directory src,  Makefile.am, plugins/Makefile.am, 
vpp-api/python/Makefile.am, and vpp-api/java/Makefile.am. Well, what 
really matters is that there are no AC_CONFIG_SUBDIRS in 
/src/configure.ac , so that everything under 
/src is part of the VPP package. (Recall that Makefile.am 
includes a lot of .am fragments to address many of the subdirectories.)
Agreed, thanks for keeping me honest.  That's what I meant, while trying 
to not lose the audience in the details.  I'm just an egg when it comes 
to autotools, and I was assuming that most of the community are more 
familiar with make constructs.
The conclusion that extras/apps/test is out of tree remains valid. The 
only point I am making is that it is configure.ac 
 rather than Makefile.am that demarcates the VPP 
package. And that you first build and install (to a first order 
approximation) the stuff under /src, and then your out of tree 
test program should build against the installed VPP stuff (to a rough 
approximation.)

Exactly!

Thanks,
-daw-



Burt


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Burt Silverman]

Dave Wallace wrote:

IMO, "in-tree" .vs. "out-of-tree" really boils down to decoupling the app's
"Makefile.am" the rest of the vpp autotools structure/configuration.  For
example, I ran into the same issue with .../vppsb/vcl-ldpreload/src/Makefile.am
(which is literally 'out-of-tree') in the case where "VPP_DIR" is
specified.  However, the vcl-ldpreload build would work the same if it was
moved somewhere under vpp without integrating it into
.../vpp/src/Makefile.am -- thus it would equally be "out-of-tree" even
though it was stored in the vpp repo.



​I would modify this statement just a bit: It is really configure.ac rather
than Makefile.am that defines the package boundary. Looking at the
AC_CONFIG_FILES statement of the configure.ac, we see that the VPP package
has, underneath directory src,  Makefile.am, plugins/Makefile.am,
vpp-api/python/Makefile.am, and vpp-api/java/Makefile.am. Well, what really
matters is that there are no AC_CONFIG_SUBDIRS in /src/configure.ac,
so that everything under /src is part of the VPP package. (Recall that
Makefile.am includes a lot of .am fragments to address many of the
subdirectories.)

The conclusion that extras/apps/test is out of tree remains valid. The only
point I am making is that it is configure.ac rather than Makefile.am that
demarcates the VPP package. And that you first build and install (to a
first order approximation) the stuff under /src, and then your out of
tree test program should build against the installed VPP stuff (to a rough
approximation.)

Burt
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Jon Loeliger]

>> Feel free to nix this patch, of course, but I'm not really sure where
>> or how to get an equivalent test into the ..extras/apps approach yet.
>
> Cool.  Give me a few days, and I'll revise this patch to consolidate the
> test apps and integrate the 'make test-c-build' validation into 'make
> verify'.
>
> -daw-

Awesome!

Thank you!

jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Dave Wallace]

On 08/23/2017 05:45 PM, Jon Loeliger wrote:

On Wed, Aug 23, 2017 at 2:25 PM, Dave Wallace  wrote:

IMO, "in-tree" .vs. "out-of-tree" really boils down to decoupling the app's
"Makefile.am" the rest of the vpp autotools structure/configuration.  For
example, I ran into the same issue with
.../vppsb/vcl-ldpreload/src/Makefile.am (which is literally 'out-of-tree')
in the case where "VPP_DIR" is specified.  However, the vcl-ldpreload build
would work the same if it was moved somewhere under vpp without integrating
it into .../vpp/src/Makefile.am -- thus it would equally be "out-of-tree"
even though it was stored in the vpp repo.

For lack of exposure to much of that, I'm sure some of the subtlety
of that is lost on me still.


There are some other test apps in .../vpp/src/uri, which could be migrated
there as well if we want to consolidate all apps under one directory.
Personally I think this makes sense.

All the 'test apps' under one directory makes loads of sense.


Currently these are "noinst_PROGRAMS" in .../vpp/src/uri.am.  To build them
for testing, "s/noinst_PROGRAMS/bin_PROGRAMS/g" in uri.am, then rebuild --
which does build them 'in-tree' (i.e. using the vpp autotools
structure/configuration) --> the resulting binary executable files land in
.../vpp/build-root/install-vpp*-native/vpp/bin.

Hrm.  We don't want to pollute the install with non-real binaries and
random test apps.  (Is that what you said would happen here?)
Exactly.  This is why uri.am currently has these test apps defined as 
"noinst_PROGRAMS".





In any case, I think that it is entirely possible to implement your proposal
in such a way as to ensure that we can close the test escape that was the
impetus for your proposal.

That would be fabulous.  This is the third or fourth time I've been
caught by it.

You bet.



My recommendation is to consolidate test
applications in the same location in the process, in which case I think
.../vpp/extras/apps is a better location than either .../vpp/make/test/c or
.../vpp/src/apps

I'm happy to have it anywhere that makes sense.
Extra happy if it ends up as part of a 'make test', 'make verify' or some
other pre-patch-acceptance criteria.

I sent in a patch, https://gerrit.fd.io/r/#/c/8189/ , that demos what would
be my intent with a 'make test-c-build'.  It isn't currently wired into the
'make test' effort, but it could be readily.

Feel free to nix this patch, of course, but I'm not really sure where
or how to get an equivalent test into the ..extras/apps approach yet.
Cool.  Give me a few days, and I'll revise this patch to consolidate the 
test apps and integrate the 'make test-c-build' validation into 'make 
verify'.


-daw-



Thanks,
-daw-

jdl


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Dave Wallace]

Florin,

That makes sense to me.   Unless someone has a better suggestion, I'd 
just shorten that to "extras/apps/test".


Any other comments/suggestions?

Thanks,
-daw-

On 08/23/2017 05:41 PM, Florin Coras wrote:

Dave,

Agreed, modulo one comment: I’d like the folder to be named something 
more specific than “extras/apps”. We can keep that for genuine 
application, but for any apps meant for testing, I’d go with 
“extras/apps/testing” or something along those lines.


Cheers,
Florin

On Aug 23, 2017, at 2:21 PM, Dave Wallace > wrote:


Jon,

On 08/23/2017 03:41 PM, Jon Loeliger wrote:

On Wed, Aug 23, 2017 at 2:25 PM, Dave Wallace  wrote:

Jon,

I think this is an excellent idea as your example is clearly a test escape
that we should be detecting in our CI infra.

However, I'm not sure if "make test" is the appropriate place to add this
check.  IMHO, this would be better suited to be invoked under "make verify"
(like the clang test coverage).  I recommend that code itself live in
.../vpp/src/apps

This is the problem.  All the existing code is "in tree".  That always works.

I want something that is building based on out-of-tree (ie, RPM install
directory), like the 'make test' provides.
IMO, "in-tree" .vs. "out-of-tree" really boils down to decoupling the 
app's "Makefile.am" the rest of the vpp autotools 
structure/configuration.  For example, I ran into the same issue with 
.../vppsb/vcl-ldpreload/src/Makefile.am (which is literally 
'out-of-tree') in the case where "VPP_DIR" is specified.  However, 
the vcl-ldpreload build would work the same if it was moved somewhere 
under vpp without integrating it into .../vpp/src/Makefile.am -- thus 
it would equally be "out-of-tree" even though it was stored in the 
vpp repo.

There are some other test apps in .../vpp/src/uri, which could be migrated
there as well if we want to consolidate all apps under one directory.
Personally I think this makes sense.

Are these already being built from 'install' staging directories?
Currently these are "noinst_PROGRAMS" in .../vpp/src/uri.am.  To 
build them for testing, "s/noinst_PROGRAMS/bin_PROGRAMS/g" in uri.am, 
then rebuild -- which does build them 'in-tree' (i.e. using the vpp 
autotools structure/configuration) --> the resulting binary 
executable files land in .../vpp/build-root/install-vpp*-native/vpp/bin.


In any case, I think that it is entirely possible to implement your 
proposal in such a way as to ensure that we can close the test escape 
that was the impetus for your proposal.  My recommendation is to 
consolidate test applications in the same location in the process, in 
which case I think .../vpp/extras/apps is a better location than 
either .../vpp/make/test/c or .../vpp/src/apps


Thanks,
-daw-

jdl


___
vpp-dev mailing list
vpp-dev@lists.fd.io 
https://lists.fd.io/mailman/listinfo/vpp-dev




___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Jon Loeliger]

> On Wed, Aug 23, 2017 at 2:25 PM, Dave Wallace  wrote:
>

> IMO, "in-tree" .vs. "out-of-tree" really boils down to decoupling the app's
> "Makefile.am" the rest of the vpp autotools structure/configuration.  For
> example, I ran into the same issue with
> .../vppsb/vcl-ldpreload/src/Makefile.am (which is literally 'out-of-tree')
> in the case where "VPP_DIR" is specified.  However, the vcl-ldpreload build
> would work the same if it was moved somewhere under vpp without integrating
> it into .../vpp/src/Makefile.am -- thus it would equally be "out-of-tree"
> even though it was stored in the vpp repo.

For lack of exposure to much of that, I'm sure some of the subtlety
of that is lost on me still.

> There are some other test apps in .../vpp/src/uri, which could be migrated
> there as well if we want to consolidate all apps under one directory.
> Personally I think this makes sense.

All the 'test apps' under one directory makes loads of sense.

> Currently these are "noinst_PROGRAMS" in .../vpp/src/uri.am.  To build them
> for testing, "s/noinst_PROGRAMS/bin_PROGRAMS/g" in uri.am, then rebuild --
> which does build them 'in-tree' (i.e. using the vpp autotools
> structure/configuration) --> the resulting binary executable files land in
> .../vpp/build-root/install-vpp*-native/vpp/bin.

Hrm.  We don't want to pollute the install with non-real binaries and
random test apps.  (Is that what you said would happen here?)

> In any case, I think that it is entirely possible to implement your proposal
> in such a way as to ensure that we can close the test escape that was the
> impetus for your proposal.

That would be fabulous.  This is the third or fourth time I've been
caught by it.

> My recommendation is to consolidate test
> applications in the same location in the process, in which case I think
> .../vpp/extras/apps is a better location than either .../vpp/make/test/c or
> .../vpp/src/apps

I'm happy to have it anywhere that makes sense.
Extra happy if it ends up as part of a 'make test', 'make verify' or some
other pre-patch-acceptance criteria.

I sent in a patch, https://gerrit.fd.io/r/#/c/8189/ , that demos what would
be my intent with a 'make test-c-build'.  It isn't currently wired into the
'make test' effort, but it could be readily.

Feel free to nix this patch, of course, but I'm not really sure where
or how to get an equivalent test into the ..extras/apps approach yet.

> Thanks,
> -daw-

jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Florin Coras]

Dave, 

Agreed, modulo one comment: I’d like the folder to be named something more 
specific than “extras/apps”. We can keep that for genuine application, but for 
any apps meant for testing, I’d go with “extras/apps/testing” or something 
along those lines. 

Cheers, 
Florin

> On Aug 23, 2017, at 2:21 PM, Dave Wallace  wrote:
> 
> Jon,
> 
> On 08/23/2017 03:41 PM, Jon Loeliger wrote:
>> On Wed, Aug 23, 2017 at 2:25 PM, Dave Wallace  
>>  wrote:
>>> Jon,
>>> 
>>> I think this is an excellent idea as your example is clearly a test escape
>>> that we should be detecting in our CI infra.
>>> 
>>> However, I'm not sure if "make test" is the appropriate place to add this
>>> check.  IMHO, this would be better suited to be invoked under "make verify"
>>> (like the clang test coverage).  I recommend that code itself live in
>>> .../vpp/src/apps
>> This is the problem.  All the existing code is "in tree".  That always works.
>> 
>> I want something that is building based on out-of-tree (ie, RPM install
>> directory), like the 'make test' provides.
> IMO, "in-tree" .vs. "out-of-tree" really boils down to decoupling the app's 
> "Makefile.am" the rest of the vpp autotools structure/configuration.  For 
> example, I ran into the same issue with 
> .../vppsb/vcl-ldpreload/src/Makefile.am (which is literally 'out-of-tree') in 
> the case where "VPP_DIR" is specified.  However, the vcl-ldpreload build 
> would work the same if it was moved somewhere under vpp without integrating 
> it into .../vpp/src/Makefile.am -- thus it would equally be "out-of-tree" 
> even though it was stored in the vpp repo.
>>> There are some other test apps in .../vpp/src/uri, which could be migrated
>>> there as well if we want to consolidate all apps under one directory.
>>> Personally I think this makes sense.
>> Are these already being built from 'install' staging directories?
> Currently these are "noinst_PROGRAMS" in .../vpp/src/uri.am.  To build them 
> for testing, "s/noinst_PROGRAMS/bin_PROGRAMS/g" in uri.am, then rebuild -- 
> which does build them 'in-tree' (i.e. using the vpp autotools 
> structure/configuration) --> the resulting binary executable files land in 
> .../vpp/build-root/install-vpp*-native/vpp/bin.
> 
> In any case, I think that it is entirely possible to implement your proposal 
> in such a way as to ensure that we can close the test escape that was the 
> impetus for your proposal.  My recommendation is to consolidate test 
> applications in the same location in the process, in which case I think 
> .../vpp/extras/apps is a better location than either .../vpp/make/test/c or 
> .../vpp/src/apps
> 
> Thanks,
> -daw-
>> jdl
> 
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Dave Wallace]

Jon,

On 08/23/2017 03:41 PM, Jon Loeliger wrote:

On Wed, Aug 23, 2017 at 2:25 PM, Dave Wallace  wrote:

Jon,

I think this is an excellent idea as your example is clearly a test escape
that we should be detecting in our CI infra.

However, I'm not sure if "make test" is the appropriate place to add this
check.  IMHO, this would be better suited to be invoked under "make verify"
(like the clang test coverage).  I recommend that code itself live in
.../vpp/src/apps

This is the problem.  All the existing code is "in tree".  That always works.

I want something that is building based on out-of-tree (ie, RPM install
directory), like the 'make test' provides.
IMO, "in-tree" .vs. "out-of-tree" really boils down to decoupling the 
app's "Makefile.am" the rest of the vpp autotools 
structure/configuration.  For example, I ran into the same issue with 
.../vppsb/vcl-ldpreload/src/Makefile.am (which is literally 
'out-of-tree') in the case where "VPP_DIR" is specified.  However, the 
vcl-ldpreload build would work the same if it was moved somewhere under 
vpp without integrating it into .../vpp/src/Makefile.am -- thus it would 
equally be "out-of-tree" even though it was stored in the vpp repo.

There are some other test apps in .../vpp/src/uri, which could be migrated
there as well if we want to consolidate all apps under one directory.
Personally I think this makes sense.

Are these already being built from 'install' staging directories?
Currently these are "noinst_PROGRAMS" in .../vpp/src/uri.am.  To build 
them for testing, "s/noinst_PROGRAMS/bin_PROGRAMS/g" in uri.am, then 
rebuild -- which does build them 'in-tree' (i.e. using the vpp autotools 
structure/configuration) --> the resulting binary executable files land 
in .../vpp/build-root/install-vpp*-native/vpp/bin.


In any case, I think that it is entirely possible to implement your 
proposal in such a way as to ensure that we can close the test escape 
that was the impetus for your proposal.  My recommendation is to 
consolidate test applications in the same location in the process, in 
which case I think .../vpp/extras/apps is a better location than either 
.../vpp/make/test/c or .../vpp/src/apps


Thanks,
-daw-

jdl


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Jon Loeliger]

On Wed, Aug 23, 2017 at 2:25 PM, Dave Wallace  wrote:
> Jon,
>
> I think this is an excellent idea as your example is clearly a test escape
> that we should be detecting in our CI infra.
>
> However, I'm not sure if "make test" is the appropriate place to add this
> check.  IMHO, this would be better suited to be invoked under "make verify"
> (like the clang test coverage).  I recommend that code itself live in
> .../vpp/src/apps

This is the problem.  All the existing code is "in tree".  That always works.

I want something that is building based on out-of-tree (ie, RPM install
directory), like the 'make test' provides.

> There are some other test apps in .../vpp/src/uri, which could be migrated
> there as well if we want to consolidate all apps under one directory.
> Personally I think this makes sense.

Are these already being built from 'install' staging directories?

jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Augment 'make test' with C build tests?

[Dave Wallace]

Jon,

I think this is an excellent idea as your example is clearly a test 
escape that we should be detecting in our CI infra.


However, I'm not sure if "make test" is the appropriate place to add 
this check.  IMHO, this would be better suited to be invoked under "make 
verify" (like the clang test coverage).  I recommend that code itself 
live in .../vpp/src/apps


There are some other test apps in .../vpp/src/uri, which could be 
migrated there as well if we want to consolidate all apps under one 
directory.  Personally I think this makes sense.


Thanks,
-daw-


On 08/23/2017 02:59 PM, Jon Loeliger wrote:

Damjan and others,

Over the past 6 or 8 months, we have had several build failures
due to missing include files in the installation of built RPMs.

It is a really simple C test to identify the failure.  Here is an
almost minimal example:

 #include 
 #include 
 #include 
 #include 

 #define vl_typedefs
 #define vl_endianfun
 #include 
 #undef vl_typedefs
 #undef vl_endianfun

 int main(int argc, char *argv[])
 {
 printf("Hello, VPP World!\n");
 return 0;
 }

If this compiles, all of the VNET include files are present.
If one of the VNET include files (like )
is missing, it will not compile.

Other includes would be needed for the other plugins too.

There is currently no structure in place under vpp/test
using "make test" that is able to do this sort of compile
test of C code yet.

What about adding a vpp/test/c directory and a Makefile
to drive the build and execution of a couple simple test
cases like the above sample?

The trick here would be ensuring the use of the just-built
components, and not the pieces (possibly) already installed
on the host system.  That is, most of this stuff from vpp/Makefile
would have to be propagated for use as well:

 make -C test \
   TEST_DIR=$(WS_ROOT)/test \
   VPP_TEST_BUILD_DIR=$(BR)/build-$(2)-native \
   VPP_TEST_BIN=$(BR)/install-$(2)-native/vpp/bin/vpp \
   VPP_TEST_PLUGIN_PATH=$(call libexpand,$(libs),$(2),vpp_plugins) \
   VPP_TEST_INSTALL_PATH=$(BR)/install-$(2)-native/ \
   LD_LIBRARY_PATH=$(call libexpand,$(libs),$(2),) \
   EXTENDED_TESTS=$(EXTENDED_TESTS) \
   PYTHON=$(PYTHON) \
   $(3)

Thoughts?

jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] Augment 'make test' with C build tests?

[Jon Loeliger]

Damjan and others,

Over the past 6 or 8 months, we have had several build failures
due to missing include files in the installation of built RPMs.

It is a really simple C test to identify the failure.  Here is an
almost minimal example:

#include 
#include 
#include 
#include 

#define vl_typedefs
#define vl_endianfun
#include 
#undef vl_typedefs
#undef vl_endianfun

int main(int argc, char *argv[])
{
printf("Hello, VPP World!\n");
return 0;
}

If this compiles, all of the VNET include files are present.
If one of the VNET include files (like )
is missing, it will not compile.

Other includes would be needed for the other plugins too.

There is currently no structure in place under vpp/test
using "make test" that is able to do this sort of compile
test of C code yet.

What about adding a vpp/test/c directory and a Makefile
to drive the build and execution of a couple simple test
cases like the above sample?

The trick here would be ensuring the use of the just-built
components, and not the pieces (possibly) already installed
on the host system.  That is, most of this stuff from vpp/Makefile
would have to be propagated for use as well:

make -C test \
  TEST_DIR=$(WS_ROOT)/test \
  VPP_TEST_BUILD_DIR=$(BR)/build-$(2)-native \
  VPP_TEST_BIN=$(BR)/install-$(2)-native/vpp/bin/vpp \
  VPP_TEST_PLUGIN_PATH=$(call libexpand,$(libs),$(2),vpp_plugins) \
  VPP_TEST_INSTALL_PATH=$(BR)/install-$(2)-native/ \
  LD_LIBRARY_PATH=$(call libexpand,$(libs),$(2),) \
  EXTENDED_TESTS=$(EXTENDED_TESTS) \
  PYTHON=$(PYTHON) \
  $(3)

Thoughts?

jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev