Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-09-04 Thread Willem de Bruijn
Jakub Kicinski wrote: > On Fri, 30 Aug 2024 14:47:43 -0400 Willem de Bruijn wrote: > > > We have directories in net/lib, and it's a target, and it works, no? > > > > net/lib is not a TARGET in tools/testing/selftests/Makefile. Its > > Makefile only generates dependencies for other targets: TEST_

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-09-02 Thread Willem de Bruijn
Jakub Kicinski wrote: > On Mon, 2 Sep 2024 09:46:12 -0700 Jakub Kicinski wrote: > > PRESERVE_TEST_DIRS > > This was meant to be: > > PRESERVE_TEST_DIRS := 1 Oh I like this. > Testing this more, looks like rsync -aR breaks networking tests, too. > The net/lib target, specifically, is no longer a

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-09-02 Thread Jakub Kicinski
On Mon, 2 Sep 2024 09:46:12 -0700 Jakub Kicinski wrote: > PRESERVE_TEST_DIRS This was meant to be: PRESERVE_TEST_DIRS := 1 Testing this more, looks like rsync -aR breaks networking tests, too. The net/lib target, specifically, is no longer able to copy out the files outside of tools/testing/sel

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-09-02 Thread Jakub Kicinski
On Sun, 01 Sep 2024 17:15:38 -0400 Willem de Bruijn wrote: > Changing kselftests to preserve directories turns out to be trivial. > Patch inline below. > > But, existing TARGETS of course then start failing. Because they > depend on existing rsync without -R. In (at least) two ways: > > amd-pstat

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-09-01 Thread Willem de Bruijn
Willem de Bruijn wrote: > Jakub Kicinski wrote: > > On Fri, 30 Aug 2024 14:47:43 -0400 Willem de Bruijn wrote: > > > > We have directories in net/lib, and it's a target, and it works, no? > > > > > > net/lib is not a TARGET in tools/testing/selftests/Makefile. Its > > > Makefile only generates d

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-30 Thread Willem de Bruijn
Jakub Kicinski wrote: > On Fri, 30 Aug 2024 14:47:43 -0400 Willem de Bruijn wrote: > > > We have directories in net/lib, and it's a target, and it works, no? > > > > net/lib is not a TARGET in tools/testing/selftests/Makefile. Its > > Makefile only generates dependencies for other targets: TEST_

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-30 Thread Willem de Bruijn
Willem de Bruijn wrote: > Jakub Kicinski wrote: > > On Fri, 30 Aug 2024 11:20:05 -0400 Willem de Bruijn wrote: > > > Kselftest install does not preserve directories. > > > > > > So all .pkt files are copied into net/packetdrill root. This is messy. > > > More fundamentally it breaks the includes i

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-30 Thread Jakub Kicinski
On Fri, 30 Aug 2024 14:47:43 -0400 Willem de Bruijn wrote: > > We have directories in net/lib, and it's a target, and it works, no? > > net/lib is not a TARGET in tools/testing/selftests/Makefile. Its > Makefile only generates dependencies for other targets: TEST_FILES, > TEST_GEN_FILES and TEST

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-30 Thread Willem de Bruijn
Jakub Kicinski wrote: > On Fri, 30 Aug 2024 11:20:05 -0400 Willem de Bruijn wrote: > > Kselftest install does not preserve directories. > > > > So all .pkt files are copied into net/packetdrill root. This is messy. > > More fundamentally it breaks the includes in the files (e..g, `source > > ../co

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-30 Thread Jakub Kicinski
On Fri, 30 Aug 2024 11:20:05 -0400 Willem de Bruijn wrote: > Kselftest install does not preserve directories. > > So all .pkt files are copied into net/packetdrill root. This is messy. > More fundamentally it breaks the includes in the files (e..g, `source > ../common/defaults.sh`). Can you show

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-30 Thread Willem de Bruijn
Jakub Kicinski wrote: > On Wed, 28 Aug 2024 15:33:33 -0400 Willem de Bruijn wrote: > > That could work. > > > > Is reporting one KTAP and exitcode per directory vs per packetdrill > > invocation good/bad/neither? > > To me "neither", hopefully and unhelpfully I won't tell you anything > that's no

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Jakub Kicinski
On Wed, 28 Aug 2024 15:33:33 -0400 Willem de Bruijn wrote: > That could work. > > Is reporting one KTAP and exitcode per directory vs per packetdrill > invocation good/bad/neither? To me "neither", hopefully and unhelpfully I won't tell you anything that's not common sense :) The balance is betwe

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Willem de Bruijn
Jakub Kicinski wrote: > On Wed, 28 Aug 2024 10:03:55 -0400 Willem de Bruijn wrote: > > > > A single script is much > > > > simpler, optionally with nested KTAP (not supported yet by ksft). But, > > > > I'm afraid that running time without intermediate output will be very > > > > long when we integr

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Jakub Kicinski
On Wed, 28 Aug 2024 11:26:28 -0700 Mina Almasry wrote: > I did not know about run_kselftest.sh. From a quick look, it seems to > require selftests to be installed to run them. I think nipa relies on > something slightly different, it does something like this, AFAIU this > runs an individual kselfte

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Stanislav Fomichev
On 08/28, Jakub Kicinski wrote: > On Wed, 28 Aug 2024 10:20:47 -0700 Stanislav Fomichev wrote: > > > As Jakub responded. > > > > > > Indeed importing the whole application into the kernel sources is what > > > stopped me from attempting this before. Florian's nf_conntrack charted > > > the path he

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Mina Almasry
On Tue, Aug 27, 2024 at 12:34 PM Willem de Bruijn wrote: > > From: Willem de Bruijn > > Lay the groundwork to import into kselftests the over 150 packetdrill > TCP/IP conformance tests on github.com/google/packetdrill. > > Florian recently added support for packetdrill tests in nf_conntrack, > in

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Jakub Kicinski
On Wed, 28 Aug 2024 10:20:47 -0700 Stanislav Fomichev wrote: > > As Jakub responded. > > > > Indeed importing the whole application into the kernel sources is what > > stopped me from attempting this before. Florian's nf_conntrack charted > > the path here. > > Thanks both. I was expecting to f

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Stanislav Fomichev
On 08/28, Willem de Bruijn wrote: > Stanislav Fomichev wrote: > > On 08/27, Willem de Bruijn wrote: > > > From: Willem de Bruijn > > > > > > Lay the groundwork to import into kselftests the over 150 packetdrill > > > TCP/IP conformance tests on github.com/google/packetdrill. > > > > > > Florian

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Matthieu Baerts
Hi Willem, Paolo, Thank you for this series, that would be great to have the CI validating these packetdrill tests! (Having the Netdev CI validating these packetdrill tests was part of my suggestions for the discussion we will have at NetConf :) ) On 28/08/2024 16:03, Willem de Bruijn wrote: > P

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Jakub Kicinski
On Wed, 28 Aug 2024 10:03:55 -0400 Willem de Bruijn wrote: > > > A single script is much > > > simpler, optionally with nested KTAP (not supported yet by ksft). But, > > > I'm afraid that running time without intermediate output will be very > > > long when we integrate all packetdrill scripts. >

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Willem de Bruijn
Stanislav Fomichev wrote: > On 08/27, Willem de Bruijn wrote: > > From: Willem de Bruijn > > > > Lay the groundwork to import into kselftests the over 150 packetdrill > > TCP/IP conformance tests on github.com/google/packetdrill. > > > > Florian recently added support for packetdrill tests in nf

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Jakub Kicinski
On Wed, 28 Aug 2024 08:01:02 -0700 Stanislav Fomichev wrote: > I don't see anything about building the binary itself. Am I missing > something or should we also have some makefile magic to do it? > I'm not sure packetdrill is packaged by the distros... Presumably > we want the existing NIPA runners

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Stanislav Fomichev
On 08/27, Willem de Bruijn wrote: > From: Willem de Bruijn > > Lay the groundwork to import into kselftests the over 150 packetdrill > TCP/IP conformance tests on github.com/google/packetdrill. > > Florian recently added support for packetdrill tests in nf_conntrack, > in commit a8a388c2aae49 ("

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Willem de Bruijn
Paolo Abeni wrote: > Adding Mat(s) for awareness, it would be great (but difficult) to have > mptcp too in the long run ;) > > On 8/27/24 21:32, Willem de Bruijn wrote: > > From: Willem de Bruijn > > > > Lay the groundwork to import into kselftests the over 150 packetdrill > > TCP/IP conformanc

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Willem de Bruijn
Jakub Kicinski wrote: > Very exciting to see packetdrill tests making their way upstream! > > On Tue, 27 Aug 2024 15:32:29 -0400 Willem de Bruijn wrote: > > RFC points for discussion > > > > ksft: the choice for this python framework introduces a dependency on > > the YNL scripts, and some non-ob

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-28 Thread Paolo Abeni
Adding Mat(s) for awareness, it would be great (but difficult) to have mptcp too in the long run ;) On 8/27/24 21:32, Willem de Bruijn wrote: From: Willem de Bruijn Lay the groundwork to import into kselftests the over 150 packetdrill TCP/IP conformance tests on github.com/google/packetdrill.

Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-27 Thread Jakub Kicinski
Very exciting to see packetdrill tests making their way upstream! On Tue, 27 Aug 2024 15:32:29 -0400 Willem de Bruijn wrote: > RFC points for discussion > > ksft: the choice for this python framework introduces a dependency on > the YNL scripts, and some non-obvious code: > - to include the net/l

[PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

2024-08-27 Thread Willem de Bruijn
From: Willem de Bruijn Lay the groundwork to import into kselftests the over 150 packetdrill TCP/IP conformance tests on github.com/google/packetdrill. Florian recently added support for packetdrill tests in nf_conntrack, in commit a8a388c2aae49 ("selftests: netfilter: add packetdrill based conn