[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
Hi Keith, On 03/04/2015 05:47 PM, Wiles, Keith wrote: > Hi Olivier > > On 3/4/15, 10:40 AM, "Olivier MATZ" wrote: > >> Hi Keith, >> >> On 03/04/2015 05:11 PM, Wiles, Keith wrote: >>> >>> >>> On 3/4/15, 3:08 AM, "Olivier MATZ" wrote: >>> Hi Keith, On 02/28/2015 05:56 PM, Keith Wiles wrote: > When building an external application like Pktgen and using the proper > makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would > use with example applications in the same RTE_SDK directory the > rte.extXYZ.mk > files are missing some defines/includes. > > 1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. > 2 - The build of external applications are forced to be verbose > ouput > as the Q=@ define is not present. > 3 - Missing include of target/generic/rte.vars.mk file which > includes > the information to locate the rte_config.h and other DPDK > include > files. > > A patch like this one was submitted before and was rejected because it > seemed it was not required, because target/generic/rte.vars.mk already > included by rte.vars.mk. > > This is not the cause for external applications like Pktgen which are > built outside of the RTE_SDK directory and only include the > rte.extXYZ.mk > makefile fragments. I still not understand what is your problem. If you take an example from dpdk, let's say examples/l2fwd. cd test # compile dpdk git clone http://dpdk.org/git/dpdk cd dpdk DPDK=${PWD} make -j8 install T=x86_64-native-linuxapp-gcc cd .. # copy l2fwd in an external directory cp -r dpdk/examples/l2fwd . cd l2fwd # build it make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK} >>> >>> Yes, this very trivial example works, but only because the makefiles are >>> combining the two different make fragments IMO. >>> >>> Then why do we have rte.extvars.mk fragment at all if it was not to be >>> used for building outside the DPDK build directory? >>> Why were the rte.extXYZ.mk make fragments created at all, but to >>> provide a >>> clean building system outside of DPDK build? >>> >>> It seem like to me we are combining two different build systems when >>> building the examples. If rte.extvars.mk is not used then lets delete it >>> or replace it with a single line to include rte.vars.mk. >>> >>> IMO combining the two different make fragment styles is confusing and we >>> need to remove rte.extvars.mk or replace it with my changes or replace >>> it >>> with a single line just to include rte.vars.mk, pick one. >> >> The examples and the documentation say to use "rte.vars.mk" for external >> applications. It's like this since the beginning, so changing the >> behavior now should be done with care to avoid breaking the working >> applications. I don't think it's a good idea. >> >> I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid >> people doing this mistake again, what do you think? > > Instead of moving the file and someone using it anyway (as it is broke > IMO) lets just replace the content of the file with a single line 'include > rte.vars.mk' and we solve the problem. Plus this solves my symmetry > problem :-) I don't understand why would someone include this file directly? What is the reason you did that at the first place? Usually, people start from an example or the documentation, which are both correct. We cannot replace the content of rte.extvars.mk by an include to rte.vars.mk because currently rte.vars.mk includes rte.extvars.mk. To me, the only problem is that rte.extvars.mk should be marked as internal. Allowing to include rte.extvars.mk in dpdk to fix the behavior of pktgen makefiles does not seem to be a good argument. Why not fixing pktgen instead? What is broken in dpdk? Regards, Olivier
[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
Hi Keith, On 03/04/2015 05:11 PM, Wiles, Keith wrote: > > > On 3/4/15, 3:08 AM, "Olivier MATZ" wrote: > >> Hi Keith, >> >> On 02/28/2015 05:56 PM, Keith Wiles wrote: >>> When building an external application like Pktgen and using the proper >>> makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would >>> use with example applications in the same RTE_SDK directory the >>> rte.extXYZ.mk >>> files are missing some defines/includes. >>> >>> 1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. >>> 2 - The build of external applications are forced to be verbose ouput >>> as the Q=@ define is not present. >>> 3 - Missing include of target/generic/rte.vars.mk file which includes >>> the information to locate the rte_config.h and other DPDK include >>> files. >>> >>> A patch like this one was submitted before and was rejected because it >>> seemed it was not required, because target/generic/rte.vars.mk already >>> included by rte.vars.mk. >>> >>> This is not the cause for external applications like Pktgen which are >>> built outside of the RTE_SDK directory and only include the >>> rte.extXYZ.mk >>> makefile fragments. >> >> I still not understand what is your problem. If you take an example from >> dpdk, let's say examples/l2fwd. >> >>cd test >># compile dpdk >>git clone http://dpdk.org/git/dpdk >>cd dpdk >>DPDK=${PWD} >>make -j8 install T=x86_64-native-linuxapp-gcc >>cd .. >># copy l2fwd in an external directory >>cp -r dpdk/examples/l2fwd . >>cd l2fwd >># build it >>make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK} > > Yes, this very trivial example works, but only because the makefiles are > combining the two different make fragments IMO. > > Then why do we have rte.extvars.mk fragment at all if it was not to be > used for building outside the DPDK build directory? > Why were the rte.extXYZ.mk make fragments created at all, but to provide a > clean building system outside of DPDK build? > > It seem like to me we are combining two different build systems when > building the examples. If rte.extvars.mk is not used then lets delete it > or replace it with a single line to include rte.vars.mk. > > IMO combining the two different make fragment styles is confusing and we > need to remove rte.extvars.mk or replace it with my changes or replace it > with a single line just to include rte.vars.mk, pick one. The examples and the documentation say to use "rte.vars.mk" for external applications. It's like this since the beginning, so changing the behavior now should be done with care to avoid breaking the working applications. I don't think it's a good idea. I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid people doing this mistake again, what do you think? Regards, Olivier
[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
Hi Olivier, On 3/4/15, 11:04 AM, "Olivier MATZ" wrote: >Hi Keith, > >On 03/04/2015 05:47 PM, Wiles, Keith wrote: >> Hi Olivier >> >> On 3/4/15, 10:40 AM, "Olivier MATZ" wrote: >> >>> Hi Keith, >>> >>> On 03/04/2015 05:11 PM, Wiles, Keith wrote: On 3/4/15, 3:08 AM, "Olivier MATZ" wrote: > Hi Keith, > > On 02/28/2015 05:56 PM, Keith Wiles wrote: >> When building an external application like Pktgen and using the >>proper >> makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would >> use with example applications in the same RTE_SDK directory the >> rte.extXYZ.mk >> files are missing some defines/includes. >> >> 1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. >> 2 - The build of external applications are forced to be verbose >> ouput >> as the Q=@ define is not present. >> 3 - Missing include of target/generic/rte.vars.mk file which >> includes >> the information to locate the rte_config.h and other DPDK >> include >> files. >> >> A patch like this one was submitted before and was rejected because >>it >> seemed it was not required, because target/generic/rte.vars.mk >>already >> included by rte.vars.mk. >> >> This is not the cause for external applications like Pktgen which >>are >> built outside of the RTE_SDK directory and only include the >> rte.extXYZ.mk >> makefile fragments. > > I still not understand what is your problem. If you take an example > from > dpdk, let's say examples/l2fwd. > > cd test > # compile dpdk > git clone http://dpdk.org/git/dpdk > cd dpdk > DPDK=${PWD} > make -j8 install T=x86_64-native-linuxapp-gcc > cd .. > # copy l2fwd in an external directory > cp -r dpdk/examples/l2fwd . > cd l2fwd > # build it > make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK} Yes, this very trivial example works, but only because the makefiles are combining the two different make fragments IMO. Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory? Why were the rte.extXYZ.mk make fragments created at all, but to provide a clean building system outside of DPDK build? It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it or replace it with a single line to include rte.vars.mk. IMO combining the two different make fragment styles is confusing and we need to remove rte.extvars.mk or replace it with my changes or replace it with a single line just to include rte.vars.mk, pick one. >>> >>> The examples and the documentation say to use "rte.vars.mk" for >>>external >>> applications. It's like this since the beginning, so changing the >>> behavior now should be done with care to avoid breaking the working >>> applications. I don't think it's a good idea. >>> >>> I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid >>> people doing this mistake again, what do you think? >> >> Instead of moving the file and someone using it anyway (as it is broke >> IMO) lets just replace the content of the file with a single line >>'include >> rte.vars.mk' and we solve the problem. Plus this solves my symmetry >> problem :-) > >I don't understand why would someone include this file directly? >What is the reason you did that at the first place? >Usually, people start from an example or the documentation, which are >both correct. > >We cannot replace the content of rte.extvars.mk by an include to >rte.vars.mk because currently rte.vars.mk includes rte.extvars.mk. >To me, the only problem is that rte.extvars.mk should be marked as >internal. > >Allowing to include rte.extvars.mk in dpdk to fix the behavior of >pktgen makefiles does not seem to be a good argument. Why not fixing >pktgen instead? What is broken in dpdk? I just posted your point before you finished you email and will submit a patch to move rte.extvars.mk to mk/rte.extvars.mk would that work? > >Regards, >Olivier >
[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
On 3/4/15, 10:47 AM, "Wiles, Keith" wrote: >Hi Olivier > >On 3/4/15, 10:40 AM, "Olivier MATZ" wrote: > >>Hi Keith, >> >>On 03/04/2015 05:11 PM, Wiles, Keith wrote: >>> >>> >>> On 3/4/15, 3:08 AM, "Olivier MATZ" wrote: >>> Hi Keith, On 02/28/2015 05:56 PM, Keith Wiles wrote: > When building an external application like Pktgen and using the >proper > makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would > use with example applications in the same RTE_SDK directory the > rte.extXYZ.mk > files are missing some defines/includes. > > 1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. > 2 - The build of external applications are forced to be verbose >ouput > as the Q=@ define is not present. > 3 - Missing include of target/generic/rte.vars.mk file which >includes > the information to locate the rte_config.h and other DPDK >include > files. > > A patch like this one was submitted before and was rejected because >it > seemed it was not required, because target/generic/rte.vars.mk >already > included by rte.vars.mk. > > This is not the cause for external applications like Pktgen which are > built outside of the RTE_SDK directory and only include the > rte.extXYZ.mk > makefile fragments. I still not understand what is your problem. If you take an example from dpdk, let's say examples/l2fwd. cd test # compile dpdk git clone http://dpdk.org/git/dpdk cd dpdk DPDK=${PWD} make -j8 install T=x86_64-native-linuxapp-gcc cd .. # copy l2fwd in an external directory cp -r dpdk/examples/l2fwd . cd l2fwd # build it make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK} >>> >>> Yes, this very trivial example works, but only because the makefiles >>>are >>> combining the two different make fragments IMO. >>> >>> Then why do we have rte.extvars.mk fragment at all if it was not to be >>> used for building outside the DPDK build directory? >>> Why were the rte.extXYZ.mk make fragments created at all, but to >>>provide a >>> clean building system outside of DPDK build? >>> >>> It seem like to me we are combining two different build systems when >>> building the examples. If rte.extvars.mk is not used then lets delete >>>it >>> or replace it with a single line to include rte.vars.mk. >>> >>> IMO combining the two different make fragment styles is confusing and >>>we >>> need to remove rte.extvars.mk or replace it with my changes or replace >>>it >>> with a single line just to include rte.vars.mk, pick one. >> >>The examples and the documentation say to use "rte.vars.mk" for external >>applications. It's like this since the beginning, so changing the >>behavior now should be done with care to avoid breaking the working >>applications. I don't think it's a good idea. >> >>I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid >>people doing this mistake again, what do you think? > >Instead of moving the file and someone using it anyway (as it is broke >IMO) lets just replace the content of the file with a single line 'include >rte.vars.mk' and we solve the problem. Plus this solves my symmetry >problem :-) The file can not be replaced with a single include line as it already includes rte.extvars.mk :-( OK lets just move it to mk/internal and I will submit a patch for that change. > >Regards >++Keith >> >>Regards, >>Olivier >> >
[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
Hi Olivier On 3/4/15, 10:40 AM, "Olivier MATZ" wrote: >Hi Keith, > >On 03/04/2015 05:11 PM, Wiles, Keith wrote: >> >> >> On 3/4/15, 3:08 AM, "Olivier MATZ" wrote: >> >>> Hi Keith, >>> >>> On 02/28/2015 05:56 PM, Keith Wiles wrote: When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk files are missing some defines/includes. 1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose ouput as the Q=@ define is not present. 3 - Missing include of target/generic/rte.vars.mk file which includes the information to locate the rte_config.h and other DPDK include files. A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk. This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk makefile fragments. >>> >>> I still not understand what is your problem. If you take an example >>>from >>> dpdk, let's say examples/l2fwd. >>> >>>cd test >>># compile dpdk >>>git clone http://dpdk.org/git/dpdk >>>cd dpdk >>>DPDK=${PWD} >>>make -j8 install T=x86_64-native-linuxapp-gcc >>>cd .. >>># copy l2fwd in an external directory >>>cp -r dpdk/examples/l2fwd . >>>cd l2fwd >>># build it >>>make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK} >> >> Yes, this very trivial example works, but only because the makefiles are >> combining the two different make fragments IMO. >> >> Then why do we have rte.extvars.mk fragment at all if it was not to be >> used for building outside the DPDK build directory? >> Why were the rte.extXYZ.mk make fragments created at all, but to >>provide a >> clean building system outside of DPDK build? >> >> It seem like to me we are combining two different build systems when >> building the examples. If rte.extvars.mk is not used then lets delete it >> or replace it with a single line to include rte.vars.mk. >> >> IMO combining the two different make fragment styles is confusing and we >> need to remove rte.extvars.mk or replace it with my changes or replace >>it >> with a single line just to include rte.vars.mk, pick one. > >The examples and the documentation say to use "rte.vars.mk" for external >applications. It's like this since the beginning, so changing the >behavior now should be done with care to avoid breaking the working >applications. I don't think it's a good idea. > >I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid >people doing this mistake again, what do you think? Instead of moving the file and someone using it anyway (as it is broke IMO) lets just replace the content of the file with a single line 'include rte.vars.mk' and we solve the problem. Plus this solves my symmetry problem :-) Regards ++Keith > >Regards, >Olivier >
[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
On 3/4/15, 3:08 AM, "Olivier MATZ" wrote: >Hi Keith, > >On 02/28/2015 05:56 PM, Keith Wiles wrote: >> When building an external application like Pktgen and using the proper >> makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would >> use with example applications in the same RTE_SDK directory the >>rte.extXYZ.mk >> files are missing some defines/includes. >> >>1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. >>2 - The build of external applications are forced to be verbose ouput >>as the Q=@ define is not present. >>3 - Missing include of target/generic/rte.vars.mk file which includes >>the information to locate the rte_config.h and other DPDK include >>files. >> >> A patch like this one was submitted before and was rejected because it >> seemed it was not required, because target/generic/rte.vars.mk already >> included by rte.vars.mk. >> >> This is not the cause for external applications like Pktgen which are >> built outside of the RTE_SDK directory and only include the >>rte.extXYZ.mk >> makefile fragments. > >I still not understand what is your problem. If you take an example from >dpdk, let's say examples/l2fwd. > > cd test > # compile dpdk > git clone http://dpdk.org/git/dpdk > cd dpdk > DPDK=${PWD} > make -j8 install T=x86_64-native-linuxapp-gcc > cd .. > # copy l2fwd in an external directory > cp -r dpdk/examples/l2fwd . > cd l2fwd > # build it > make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK} Yes, this very trivial example works, but only because the makefiles are combining the two different make fragments IMO. Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory? Why were the rte.extXYZ.mk make fragments created at all, but to provide a clean building system outside of DPDK build? It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it or replace it with a single line to include rte.vars.mk. IMO combining the two different make fragment styles is confusing and we need to remove rte.extvars.mk or replace it with my changes or replace it with a single line just to include rte.vars.mk, pick one. > >So if you use a Makefile similar to l2fwd, I think it should work. >As I explained in [1], you need to include "rte.vars.mk" at the >beginning of the Makefile, not "rte.extvars.mk". But you will use >"rte.extapp.mk" at the end, like in l2fwd Makefile. > >Regards, >Olivier > > >[1] http://dpdk.org/ml/archives/dev/2015-February/013301.html
[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
Hi Keith, On 02/28/2015 05:56 PM, Keith Wiles wrote: > When building an external application like Pktgen and using the proper > makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would > use with example applications in the same RTE_SDK directory the rte.extXYZ.mk > files are missing some defines/includes. > >1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. >2 - The build of external applications are forced to be verbose ouput >as the Q=@ define is not present. >3 - Missing include of target/generic/rte.vars.mk file which includes >the information to locate the rte_config.h and other DPDK include >files. > > A patch like this one was submitted before and was rejected because it > seemed it was not required, because target/generic/rte.vars.mk already > included by rte.vars.mk. > > This is not the cause for external applications like Pktgen which are > built outside of the RTE_SDK directory and only include the rte.extXYZ.mk > makefile fragments. I still not understand what is your problem. If you take an example from dpdk, let's say examples/l2fwd. cd test # compile dpdk git clone http://dpdk.org/git/dpdk cd dpdk DPDK=${PWD} make -j8 install T=x86_64-native-linuxapp-gcc cd .. # copy l2fwd in an external directory cp -r dpdk/examples/l2fwd . cd l2fwd # build it make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK} So if you use a Makefile similar to l2fwd, I think it should work. As I explained in [1], you need to include "rte.vars.mk" at the beginning of the Makefile, not "rte.extvars.mk". But you will use "rte.extapp.mk" at the end, like in l2fwd Makefile. Regards, Olivier [1] http://dpdk.org/ml/archives/dev/2015-February/013301.html
[dpdk-dev] [PATCH] External app builds need to locate common make fragments and includes.
When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk files are missing some defines/includes. 1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose ouput as the Q=@ define is not present. 3 - Missing include of target/generic/rte.vars.mk file which includes the information to locate the rte_config.h and other DPDK include files. A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk. This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk makefile fragments. Signed-off-by: Keith Wiles --- mk/rte.extvars.mk | 36 ++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/mk/rte.extvars.mk b/mk/rte.extvars.mk index 3e5a990..e6c6401 100644 --- a/mk/rte.extvars.mk +++ b/mk/rte.extvars.mk @@ -30,8 +30,19 @@ # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # -# directory where sources are located +# To be included at the beginning of all RTE external user Makefiles. This +# .mk will define the RTE environment variables by including the +# config file of SDK. It also includes the config file from external +# application if any. # + +ifeq ($(RTE_SDK),) +$(error RTE_SDK is not defined) +endif +ifeq ($(wildcard $(RTE_SDK)),) +$(error RTE_SDK variable points to an invalid location) +endif + ifdef S ifeq ("$(origin S)", "command line") RTE_SRCDIR := $(abspath $(S)) @@ -40,6 +51,16 @@ endif RTE_SRCDIR ?= $(CURDIR) export RTE_SRCDIR +# define Q to '@' or not. $(Q) is used to prefix all shell commands to +# be executed silently. +Q=@ +ifdef V +ifeq ("$(origin V)", "command line") +Q= +endif +endif +export Q + # # Makefile to call once $(RTE_OUTPUT) is created # @@ -51,6 +72,13 @@ endif RTE_EXTMK ?= $(RTE_SRCDIR)/Makefile export RTE_EXTMK +# RTE_TARGET is deducted from config when we are building the SDK. +# Else, when building an external app, RTE_TARGET must be specified +# by the user. +ifeq ($(RTE_TARGET),) +$(error RTE_TARGET is not defined) +endif + RTE_SDK_BIN := $(RTE_SDK)/$(RTE_TARGET) # @@ -78,4 +106,8 @@ RTE_MACHINE := $(CONFIG_RTE_MACHINE:"%"=%) RTE_EXEC_ENV := $(CONFIG_RTE_EXEC_ENV:"%"=%) RTE_TOOLCHAIN := $(CONFIG_RTE_TOOLCHAIN:"%"=%) - +ifneq ($(wildcard $(RTE_SDK)/mk/target/$(RTE_TARGET)/rte.vars.mk),) +include $(RTE_SDK)/mk/target/$(RTE_TARGET)/rte.vars.mk +else +include $(RTE_SDK)/mk/target/generic/rte.vars.mk +endif -- 2.3.0