Re: [PATCH] dts: add per-test-suite configuration

2024-09-27 Thread Jeremy Spewock
Hi Luca, I apologize for not giving this patch more time/attention. I think this patch is worth having a more in-depth discussion about since it does add quite a bit more complexity, but the benefit of that is some valuable simplicity. I know we discussed in the recent team meeting about whether o

Re: [PATCH v1 3/3] dts: rework test suite and dts runner to include test_run configs

2024-09-27 Thread Jeremy Spewock
On Tue, Sep 10, 2024 at 7:05 AM Juraj Linkeš wrote: > > >> -tg_port.peer, > >> -tg_port.identifier, > >> -): > >> -self._port_links.append(PortLink(sut_port=sut_port, > >> tg_port=tg_port)) > >> +sut_ports = [] >

Re: [PATCH v1] dts: add send_packets to test_suite

2024-09-26 Thread Jeremy Spewock
On Thu, Sep 26, 2024 at 5:56 AM Juraj Linkeš wrote: > > > > On 25. 9. 2024 20:36, jspew...@iol.unh.edu wrote: > > From: Jeremy Spewock > > > > Currently the only methods provided in the test suite class for sending > > packets capture the resulting rece

Re: [PATCH v3 1/1] dts: rework packet addressing

2024-09-26 Thread Jeremy Spewock
On Thu, Sep 26, 2024 at 8:31 AM Juraj Linkeš wrote: > > > > diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py > > > +def _adjust_addresses(self, packets: list[Packet], expected: bool = > > False) -> list[Packet]: > > +# The packet is routed from TG egress to T

Re: [PATCH v4 1/1] dts: Remove XML-RPC server for Scapy TG and instead use PythonShell

2024-09-26 Thread Jeremy Spewock
On Thu, Sep 26, 2024 at 5:12 AM Juraj Linkeš wrote: > > The subject line should start with a lowercase character and it's also > over the character limit. This fits and basically says the same: > use PythonShell for Scapy instead of XML-RPC > > Scapy TG didn't fit but I think the above still works

Re: [PATCH v4 1/1] dts: Remove XML-RPC server for Scapy TG and instead use PythonShell

2024-09-26 Thread Jeremy Spewock
On Thu, Sep 26, 2024 at 5:12 AM Juraj Linkeš wrote: > > The subject line should start with a lowercase character and it's also > over the character limit. This fits and basically says the same: > use PythonShell for Scapy instead of XML-RPC > > Scapy TG didn't fit but I think the above still works

Re: [PATCH v6 1/1] dts: add text parser for testpmd verbose output

2024-09-26 Thread Jeremy Spewock
On Thu, Sep 26, 2024 at 4:25 AM Juraj Linkeš wrote: > > > > diff --git a/dts/framework/remote_session/testpmd_shell.py > > b/dts/framework/remote_session/testpmd_shell.py > > > @@ -581,6 +581,506 @@ class TestPmdPortStats(TextParser): > > tx_bps: int = field(metadata=TextParser.find_int(r"T

Re: [PATCH v2 1/1] dts: add send_packets to test suites and rework packet addressing

2024-09-25 Thread Jeremy Spewock
On Tue, Sep 24, 2024 at 10:30 AM Juraj Linkeš wrote: > > > > On 20. 9. 2024 20:08, jspew...@iol.unh.edu wrote: > > From: Jeremy Spewock > > > > Currently the only method provided in the test suite class for sending > > packets sends a single packet and then ca

Re: [PATCH v3 1/1] dts: Remove XML-RPC server for Scapy TG and instead use PythonShell

2024-09-24 Thread Jeremy Spewock
On Tue, Sep 24, 2024 at 6:55 AM Juraj Linkeš wrote: > > I like how this looks. I have a number of minor comments (mainly wording > and naming), but overall it looks very good. > > On 19. 9. 2024 21:02, jspew...@iol.unh.edu wrote: > > From: Jeremy Spewock > > > &

Re: [PATCH v2 1/1] dts: add binding to different drivers to TG node

2024-09-24 Thread Jeremy Spewock
ethod > > for binding ports into the node class rather than the SUT node class and > > adds an abstract method for getting the path to the devbind script into > > the node class. Then, binding ports to the correct drivers is moved into > > the build target setup and run on bo

Re: [PATCH v4 10/11] dts: add Rx offload capabilities

2024-09-23 Thread Jeremy Spewock
> Flag. Dynamic addition of Enum members runs into problems with typing > (mypy doesn't know about the members) and documentation generation > (Sphinx doesn't know about the members). > > Signed-off-by: Juraj Linkeš > Reviewed-by: Dean Marx Reviewed-by: Jeremy Spewock

Re: [PATCH v4 08/11] dts: add topology capability

2024-09-23 Thread Jeremy Spewock
nd > TopologyType.default are distinct. This is needed to distinguish between > a user passed value and the default value being used (which is used when > a test suite is or isn't decorated). > > Signed-off-by: Juraj Linkeš > Reviewed-by: Dean Marx Reviewed-by: Jeremy Spewock

Re: [PATCH v4 07/11] dts: add NIC capabilities from show rxq info

2024-09-23 Thread Jeremy Spewock
On Mon, Sep 23, 2024 at 11:02 AM Juraj Linkeš wrote: > > Add parsing for the show rxq info tespmd command > and add support for the Scattered Rx capability. > > Signed-off-by: Juraj Linkeš > Reviewed-by: Dean Marx Reviewed-by: Jeremy Spewock

Re: [PATCH v4 06/11] dts: add NIC capability support

2024-09-23 Thread Jeremy Spewock
The decorator API is designed to be simple to use. The arguments passed > to it are all from the testpmd shell. Everything else (even the actual > capability object creation) is done internally. > > Signed-off-by: Juraj Linkeš > Reviewed-by: Dean Marx Thank you for addressing all my comments! Reviewed-by: Jeremy Spewock

Re: [PATCH v4 03/11] dts: add mechanism to skip test cases or suites

2024-09-23 Thread Jeremy Spewock
esult. > > Signed-off-by: Juraj Linkeš > Reviewed-by: Dean Marx Reviewed-by: Jeremy Spewock

Re: [PATCH v4 02/11] dts: add test case decorators

2024-09-23 Thread Jeremy Spewock
One super nit-pick comment below, even without that though I still think this looks good. Reviewed-by: Jeremy Spewock On Mon, Sep 23, 2024 at 11:02 AM Juraj Linkeš wrote: > +def is_test_case(function: Callable) -> bool: > +if inspect.isfunction

Re: [PATCH v2 1/2] dts: add clearing port stats and verbose mode to testpmd

2024-09-23 Thread Jeremy Spewock
Apologies, please disregard this patch, I sent out a v3 with a fixed subject line.

Re: [PATCH v2] dts: add VLAN methods to testpmd shell

2024-09-20 Thread Jeremy Spewock
On Wed, Sep 18, 2024 at 3:41 PM Dean Marx wrote: > > + > +def tx_vlan_set(self, port: int, vlan: int, verify: bool = True) -> None: One thing to note is that I think this method (unlike rx_vlan_set for some reason) requires the ports to be stopped for it to work, so we should probably decora

Re: [PATCH v5 1/1] dts: add text parser for testpmd verbose output

2024-09-20 Thread Jeremy Spewock
On Thu, Sep 19, 2024 at 8:35 AM Juraj Linkeš wrote: > > > diff --git a/dts/framework/remote_session/testpmd_shell.py > > b/dts/framework/remote_session/testpmd_shell.py > > > @@ -577,6 +577,497 @@ class TestPmdPortStats(TextParser): > > tx_bps: int = field(metadata=TextParser.find_int(r"Tx-

Re: [PATCH v5 1/1] dts: add text parser for testpmd verbose output

2024-09-20 Thread Jeremy Spewock
On Thu, Sep 19, 2024 at 5:02 AM Juraj Linkeš wrote: > > > diff --git a/dts/framework/remote_session/testpmd_shell.py > > b/dts/framework/remote_session/testpmd_shell.py > > > @@ -577,6 +577,497 @@ class TestPmdPortStats(TextParser): > > tx_bps: int = field(metadata=TextParser.find_int(r"Tx-

Re: [PATCH v2] dts: fix runner target in the Dockerfile

2024-09-19 Thread Jeremy Spewock
On Thu, Sep 19, 2024 at 3:09 AM Juraj Linkeš wrote: > > > > On 18. 9. 2024 16:16, Jeremy Spewock wrote: > > On Wed, Sep 18, 2024 at 3:57 AM Juraj Linkeš > > wrote: > >> > >> > >>> diff --git a/dts/Dockerfile b/dts/Dockerfile > >>

Re: [PATCH 1/1] dts: add binding to different drivers to TG node

2024-09-18 Thread Jeremy Spewock
On Mon, Sep 16, 2024 at 6:04 AM Juraj Linkeš wrote: > > > > On 9. 9. 2024 17:55, Jeremy Spewock wrote: > > On Mon, Sep 9, 2024 at 8:16 AM Juraj Linkeš > > wrote: > >> > >> > >> > >> On 12. 8. 2024 19:22, jspew...@iol.unh.edu wrote:

Re: [PATCH v3 11/12] dts: add Rx offload capabilities

2024-09-18 Thread Jeremy Spewock
On Wed, Sep 18, 2024 at 10:27 AM Juraj Linkeš wrote: > > > > On 29. 8. 2024 17:40, Jeremy Spewock wrote: > > On Wed, Aug 28, 2024 at 1:44 PM Jeremy Spewock wrote: > >> > >> On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš > >> wrote: > >

Re: [PATCH v3 11/12] dts: add Rx offload capabilities

2024-09-18 Thread Jeremy Spewock
On Wed, Sep 18, 2024 at 10:18 AM Juraj Linkeš wrote: > > > > On 26. 8. 2024 19:24, Jeremy Spewock wrote: > > On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš > > wrote: > > > >> diff --git a/dts/framework/remote_session/testpmd_shell.py > >>

Re: [PATCH v3 08/12] dts: add NIC capability support

2024-09-18 Thread Jeremy Spewock
On Wed, Sep 18, 2024 at 8:58 AM Juraj Linkeš wrote: > > > > On 27. 8. 2024 18:36, Jeremy Spewock wrote: > > On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš > > wrote: > > > >> diff --git a/dts/framework/testbed_model/capability.py > >> b/dts/

Re: [PATCH v2] dts: fix runner target in the Dockerfile

2024-09-18 Thread Jeremy Spewock
On Wed, Sep 18, 2024 at 3:57 AM Juraj Linkeš wrote: > > > > diff --git a/dts/Dockerfile b/dts/Dockerfile > > > @@ -24,9 +27,12 @@ FROM base AS runner > > > +# Adds ~/.local/bin to PATH so that packages installed with pipx are > > callable. `pipx ensurepath` > > +# fixes this issue, but requires t

Re: [PATCH v3 1/1] dts: add text parser for testpmd verbose output

2024-09-17 Thread Jeremy Spewock
On Mon, Sep 9, 2024 at 7:44 AM Juraj Linkeš wrote: > > > > diff --git a/dts/framework/parser.py b/dts/framework/parser.py > > index 741dfff821..0b39025a48 100644 > > --- a/dts/framework/parser.py > > +++ b/dts/framework/parser.py > > @@ -160,6 +160,36 @@ def _find(text: str) -> Any: > > > >

Re: [PATCH v1] dts: fix runner target in the Dockerfile

2024-09-16 Thread Jeremy Spewock
On Mon, Sep 16, 2024 at 6:16 AM Juraj Linkeš wrote: > > > > On 11. 9. 2024 17:50, jspew...@iol.unh.edu wrote: > > From: Jeremy Spewock > > > > Currently the runner target in the Dockerfile attempts to run the > > `poetry install` command when building the ima

Re: [PATCH 1/1] dts: add binding to different drivers to TG node

2024-09-09 Thread Jeremy Spewock
On Mon, Sep 9, 2024 at 8:16 AM Juraj Linkeš wrote: > > > > On 12. 8. 2024 19:22, jspew...@iol.unh.edu wrote: > > From: Jeremy Spewock > > > > The DTS framework in its current state supports binding ports to > > different drivers on the SUT node but not the

Re: [PATCH v3 5/5] dts: add testpmd set ports queues

2024-09-09 Thread Jeremy Spewock
On Mon, Sep 9, 2024 at 7:03 AM Luca Vizzarro wrote: > > Add a facility to update the number of TX/RX queues during the runtime > of testpmd. > > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek Reviewed-by: Jeremy Spewock

Re: [PATCH v3 4/5] dts: add ability to start/stop testpmd ports

2024-09-09 Thread Jeremy Spewock
uire a specific state, removing this logic from the test > writer's duty. > > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek Reviewed-by: Jeremy Spewock

Re: [PATCH v2 4/5] dts: add ability to start/stop testpmd ports

2024-09-09 Thread Jeremy Spewock
On Fri, Sep 6, 2024 at 12:46 PM Luca Vizzarro wrote: > > On 23/08/2024 13:16, Juraj Linkeš wrote: > > As Jeremy mentioned, adding the verify argument may be worthwhile, but > > maybe only if we actually identify a usecase where we wouldn't want to > > do the verification. > > Yeah, as I pointed ou

Re: [PATCH v2 1/5] dts: add ability to send/receive multiple packets

2024-09-09 Thread Jeremy Spewock
On Fri, Sep 6, 2024 at 12:23 PM Luca Vizzarro wrote: > > On 09/08/2024 16:10, Jeremy Spewock wrote: > > > >> +def match_all_packets( > >> +self, expected_packets: list[Packet], received_packets: > >> list[Packet] > >> +) -> N

Re: [PATCH v3 1/1] dts: add methods for modifying MTU to testpmd shell

2024-09-06 Thread Jeremy Spewock
On Fri, Sep 6, 2024 at 9:58 AM Juraj Linkeš wrote: > > > diff --git a/dts/framework/remote_session/testpmd_shell.py > > b/dts/framework/remote_session/testpmd_shell.py > > index ca24b28070..c1462ba2d3 100644 > > --- a/dts/framework/remote_session/testpmd_shell.py > > +++ b/dts/framework/remote_se

Re: [PATCH v1] dts: fix testpmd port device error handling mode

2024-09-05 Thread Jeremy Spewock
Reviewed-by: Jeremy Spewock

Re: [PATCH v1] dts: add testpmd port information caching

2024-09-05 Thread Jeremy Spewock
Seems like a good change to me! Reviewed-by: Jeremy Spewock

Re: [PATCH v3 08/12] dts: add NIC capability support

2024-09-05 Thread Jeremy Spewock
On Thu, Sep 5, 2024 at 7:56 AM Juraj Linkeš wrote: > > > > On 26. 8. 2024 19:11, Jeremy Spewock wrote: > > On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš > > wrote: > > > >> @dataclass > >> class TestPmdPort(TextParser): > >>

Re: [PATCH v3 04/12] dts: add mechanism to skip test cases or suites

2024-09-05 Thread Jeremy Spewock
On Thu, Sep 5, 2024 at 5:23 AM Juraj Linkeš wrote: > >> +def _mark_results(self, result) -> None: > > > > Is it worth adding the type annotation for `result` here and to the > > other places where this is implemented? I guess it doesn't matter that > > much since it is a private method. > > >

Re: [PATCH v3 06/12] dst: add basic capability support

2024-09-05 Thread Jeremy Spewock
On Thu, Sep 5, 2024 at 5:50 AM Juraj Linkeš wrote: > >> @@ -63,6 +65,12 @@ class is to hold a subset of test cases (which could be > >> all test cases) because > >> > >> test_suite_class: type[TestSuite] > >> test_cases: list[type[TestCase]] > >> +required_capabilities: set[Capab

Re: [PATCH v3 03/12] dts: add test case decorators

2024-09-05 Thread Jeremy Spewock
On Thu, Sep 5, 2024 at 4:07 AM Juraj Linkeš wrote: > > > > On 26. 8. 2024 18:50, Jeremy Spewock wrote: > > On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš > > wrote: > > > >> class DTSRunner: > >> @@ -232,9 +231,9 @@ def _get_test_suites_with_cas

Re: [PATCH v3 2/2] dts: checksum offload test suite

2024-09-04 Thread Jeremy Spewock
Just one comment here but it is a super small doc-string thing that still passes the formatting script and everything, but whenever we add Ruff for checking format in the future, it would get caught so it might be better to just add it now. Otherwise: Reviewed-by: Jeremy Spewock On Mon, Aug 26

Re: [PATCH v3 1/2] dts: add csum HW offload to testpmd shell

2024-09-04 Thread Jeremy Spewock
Hey Dean, This patch is looking good, I like the changes from the previous version, there were just a few comments that I left but they should be pretty easy fixes. On Mon, Aug 26, 2024 at 5:22 PM Dean Marx wrote: > +def csum_set_hw(self, layer: ChecksumOffloadOptions, port_id: int, > veri

Re: [PATCH v14 2/2] dts: VLAN test suite implementation

2024-09-04 Thread Jeremy Spewock
All looks great to me, thank you for the patch and addressing comments, there is just one comment that I left here about something that I missed pointing out on the previous version, but once that is addressed I think this will be ready. On Fri, Aug 23, 2024 at 5:16 PM Dean Marx wrote: > +de

Re: [PATCH v14 1/2] dts: add VLAN methods to testpmd shell

2024-09-04 Thread Jeremy Spewock
for > show port info, removed $ at end of regex > > Signed-off-by: Dean Marx Reviewed-by: Jeremy Spewock

Re: [PATCH v2 2/2] dts: port over unified packet suite

2024-09-04 Thread Jeremy Spewock
I just left some general formatting comments and a few idea that I thought might be helpful, but it looks good to me in general: Reviewed-by: Jeremy Spewock On Fri, Aug 23, 2024 at 4:22 PM Dean Marx wrote: > > Port over unified packet testing suite from old DTS. This suite > tests th

Re: [PATCH v2 1/2] dts: add VXLAN port method to testpmd shell

2024-09-04 Thread Jeremy Spewock
Just a few comments on doc-strings, otherwise: Reviewed-by: Jeremy Spewock On Fri, Aug 23, 2024 at 4:22 PM Dean Marx wrote: > > Add rx_vxlan_port add/rm method to testpmd shell for adding > or removing a vxlan id to the specified port filter list. > > Signed-off-by: Dean Mar

Re: [PATCH v1 3/3] dts: rework test suite and dts runner to include test_run configs

2024-09-04 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 2:43 PM Nicholas Pratte wrote: > diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py > index 694b2eba65..fd51796a06 100644 > --- a/dts/framework/test_suite.py > +++ b/dts/framework/test_suite.py > @@ -20,6 +20,7 @@ > from scapy.layers.l2 import Ether #

Re: [PATCH v1 2/3] dts: rework testbed_model Port objects to contain unique identifiers

2024-09-04 Thread Jeremy Spewock
cholas Pratte Reviewed-by: Jeremy Spewock

Re: [PATCH v1 1/3] dts: rework port attributes in config module

2024-09-04 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 2:43 PM Nicholas Pratte wrote: > diff --git a/dts/framework/config/__init__.py > b/dts/framework/config/__init__.py > index df60a5030e..534821ed22 100644 > --- a/dts/framework/config/__init__.py > +++ b/dts/framework/config/__init__.py > @@ -151,11 +151,10 @@ class PortCo

Re: [PATCH v3 11/12] dts: add Rx offload capabilities

2024-08-29 Thread Jeremy Spewock
On Wed, Aug 28, 2024 at 1:44 PM Jeremy Spewock wrote: > > On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš > wrote: > > > diff --git a/dts/framework/remote_session/testpmd_shell.py > > b/dts/framework/remote_session/testpmd_shell.py > > index 48c31124d1..f83569669e

Re: [PATCH v3 11/12] dts: add Rx offload capabilities

2024-08-28 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > diff --git a/dts/framework/remote_session/testpmd_shell.py > b/dts/framework/remote_session/testpmd_shell.py > index 48c31124d1..f83569669e 100644 > --- a/dts/framework/remote_session/testpmd_shell.py > +++ b/dts/framework/remote_session/tes

Re: [PATCH v3 08/12] dts: add NIC capability support

2024-08-27 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > diff --git a/dts/framework/testbed_model/capability.py > b/dts/framework/testbed_model/capability.py > index 8899f07f76..9a79e6ebb3 100644 > --- a/dts/framework/testbed_model/capability.py > +++ b/dts/framework/testbed_model/capability.py >

Re: [PATCH v3 00/12] dts: add test skipping based on capabilities

2024-08-26 Thread Jeremy Spewock
Hey Juraj, Thanks for the series! This is definitely a large shift in how the framework operates, but I think a lot of these changes are hugely helpful and the code is very well written in general. I left some comments mostly about places where I think some things could be a little more clear, and

Re: [PATCH v3 12/12] dts: add NIC capabilities from show port info

2024-08-26 Thread Jeremy Spewock
;t know about the members). > > Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock

Re: [PATCH v3 11/12] dts: add Rx offload capabilities

2024-08-26 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > diff --git a/dts/framework/remote_session/testpmd_shell.py > b/dts/framework/remote_session/testpmd_shell.py > index 48c31124d1..f83569669e 100644 > --- a/dts/framework/remote_session/testpmd_shell.py > +++ b/dts/framework/remote_session/tes

Re: [PATCH v3 10/12] doc: add DTS capability doc sources

2024-08-26 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > > Add new files to generate DTS API documentation from. > > Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock

Re: [PATCH v3 09/12] dts: add topology capability

2024-08-26 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > > Add support for marking test cases as requiring a certain topology. The > default topology is a two link topology and the other supported > topologies are one link and no link topologies. > > The TestProtocol of test suites and cases is exte

Re: [PATCH v3 08/12] dts: add NIC capability support

2024-08-26 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > @dataclass > class TestPmdPort(TextParser): > """Dataclass representing the result of testpmd's ``show port info`` > command.""" > @@ -962,3 +1043,96 @@ def _close(self) -> None: > self.stop() > self.send_command("qu

Re: [PATCH v3 07/12] dts: add testpmd port information caching

2024-08-26 Thread Jeremy Spewock
ce the update. > > Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock

Re: [PATCH v3 06/12] dst: add basic capability support

2024-08-26 Thread Jeremy Spewock
Just one comment about adding something to a doc-string, otherwise looks good to me: Reviewed-by: Jeremy Spewock On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > diff --git a/dts/framework/test_result.py b/dts/framework/test_result.py > index 306b100bc6..b4b58ef348 100644 >

Re: [PATCH v3 05/12] dts: add support for simpler topologies

2024-08-26 Thread Jeremy Spewock
I just had one question below, otherwise: Reviewed-by: Jeremy Spewock On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > diff --git a/dts/framework/testbed_model/topology.py > b/dts/framework/testbed_model/topology.py > new file mode 100644 > index 00..19632ee890 >

Re: [PATCH v3 04/12] dts: add mechanism to skip test cases or suites

2024-08-26 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > --- a/dts/framework/test_result.py > +++ b/dts/framework/test_result.py > @@ -75,6 +75,20 @@ def create_config(self) -> TestSuiteConfig: > test_cases=[test_case.__name__ for test_case in self.test_cases], > ) > > +@p

Re: [PATCH v3 03/12] dts: add test case decorators

2024-08-26 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > class DTSRunner: > @@ -232,9 +231,9 @@ def _get_test_suites_with_cases( > > for test_suite_config in test_suite_configs: > test_suite_class = > self._get_test_suite_class(test_suite_config.test_suite) > -te

Re: [PATCH v3 02/12] dts: add the aenum dependency

2024-08-26 Thread Jeremy Spewock
tion that > retrieves the capabilities. > > Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock

Re: [PATCH v3 01/12] dts: fix default device error handling mode

2024-08-26 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 10:53 AM Juraj Linkeš wrote: > > The device_error_handling_mode of testpmd port may not be present, e.g. > in VM ports. > > Fixes: 61d5bc9bf974 ("dts: add port info command to testpmd shell") > > Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock

Re: [PATCH v2 2/2] dts: checksum offload test suite

2024-08-23 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 12:25 PM Dean Marx wrote: > > test suite for verifying layer 3/4 checksum offload > features on poll mode driver. > > Depends-on: patch-143033 > ("dts: add text parser for testpmd verbose output") > Depends-on: patch-142691 > ("dts: add send_packets to test suites and rewor

Re: [PATCH v2 1/2] dts: add csum HW offload to testpmd shell

2024-08-23 Thread Jeremy Spewock
On Wed, Aug 21, 2024 at 12:25 PM Dean Marx wrote: > > add csum_set_hw method to testpmd shell class. Port over > set_verbose and port start/stop from queue start/stop suite. Since we had that discussion in a DTS meeting about there not really being a rule against multiple dependencies or anything

Re: [PATCH v2 0/2] dts: port over checksum offload suite

2024-08-23 Thread Jeremy Spewock
Hey Dean, Thanks for the series! Looks good to me in general, I just left some comments with a few thoughts I had about some ways to potentially move some stuff around or swap implementations, but the functionality all looks good to me. Let me know what you think! Thanks, Jeremy On Wed, Aug 21,

Re: [RFC PATCH v2 0/5] dts: add VFs to the framework

2024-08-21 Thread Jeremy Spewock
Apologies for sending out the v2 twice for this series, when I attempted the first time the reply chain broke somehow and patchwork didn't consider all the patches as one series. This should be fixed now.

Re: [PATCH v2 5/5] dts: add testpmd set ports queues

2024-08-20 Thread Jeremy Spewock
On Fri, Aug 9, 2024 at 11:14 AM Jeremy Spewock wrote: > > > + > > +Raises: > > +InternalError: If `number_of` is invalid. > > I might be more in favor of this being an > InteractiveCommandExecutionError or some other DTSError just to have a > l

Re: [PATCH v7 1/2] dts: add methods for modifying MTU to testpmd shell

2024-08-20 Thread Jeremy Spewock
On Tue, Aug 20, 2024 at 9:05 AM Juraj Linkeš wrote: > > I'm trying to use this patch for the capabilities series. It works as I > need it to, so we just need to coordinate a bit to use this one patch > for both series. > > > diff --git a/dts/framework/remote_session/testpmd_shell.py > > b/dts/fra

Re: [PATCH v1] dts: correct typos in conf.yaml

2024-08-19 Thread Jeremy Spewock
Took me a few reads to even find the differences, haha. On Tue, Aug 13, 2024 at 1:17 PM Dean Marx wrote: > > correct docstring error in conf.yaml showing incorrect > example pci address for TG nodes > > Signed-off-by: Dean Marx Reviewed-by: Jeremy Spewock

Re: [PATCH v17 5/5] dts: add API doc generation

2024-08-14 Thread Jeremy Spewock
allows us to mock the imports. The missing > packages are taken from the DTS pyproject.toml file. > > And finally, the DTS API docs can be accessed from the DPDK API doxygen > page. > > [0] > https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings > > Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock

Re: [PATCH v17 3/5] dts: add doc generation dependencies

2024-08-14 Thread Jeremy Spewock
picks up the correct Python > installation, as pyelftools is required by the build system. > > Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock

Re: [PATCH v17 2/5] dts: replace the or operator in third party types

2024-08-14 Thread Jeremy Spewock
from module > 'framework'; the following exception was raised: > Traceback (most recent call last): > ... > TypeError: unsupported operand type(s) for |: 'Transport' and 'NoneType' > > The third part type here is Transport from the paramiko library. > > Signed-off-by: Juraj Linkeš > --- Reviewed-by: Jeremy Spewock

Re: [PATCH v17 1/5] dts: update params and parser docstrings

2024-08-14 Thread Jeremy Spewock
manipulation module") > Fixes: 967fc62b0a43 ("dts: refactor EAL parameters class") > Fixes: 818fe14e3422 ("dts: add parsing utility module") > Cc: luca.vizza...@arm.com > > Signed-off-by: Juraj Linkeš > Reviewed-by: Luca Vizzarro > --- Reviewed-by: Jeremy Spewock

Re: [RFC v1 2/2] dts: checksum offload test suite

2024-08-12 Thread Jeremy Spewock
On Mon, Aug 12, 2024 at 9:41 AM Dean Marx wrote: > > test suite for verifying layer 3/4 checksum offload > features on poll mode driver. > > Depends-on: patch-142762 > ("dts: add text parser for testpmd verbose output") > Depends-on: patch-142691 > ("dts: add send_packets to test suites and rework

Re: [RFC v1 1/2] dts: add csum HW offload to testpmd shell

2024-08-12 Thread Jeremy Spewock
On Mon, Aug 12, 2024 at 9:41 AM Dean Marx wrote: > > add csum_set_hw method to testpmd shell class. Port over > set_verbose and port start/stop from queue start/stop suite. > > Signed-off-by: Dean Marx > --- > +def csum_set_hw(self, layer: str, port_id: int, verify: bool = True) -> > None:

Re: [RFC v1 0/2] dts: initial checksum offload suite

2024-08-12 Thread Jeremy Spewock
Hey Dean, Thanks for the patch, I left some thoughts on the point you raise below and some of the larger things I noticed throughout the series. The logic seems to all be right though and I can understand what you were going for, so the broader ideas seem good to me. On Mon, Aug 12, 2024 at 9:41 

Re: [PATCH v1] dts: add verify argument to set forward mode

2024-08-12 Thread Jeremy Spewock
On Mon, Aug 12, 2024 at 10:23 AM Dean Marx wrote: > > Add optional verify argument to the set_forward_mode > method in testpmd shell. > > Signed-off-by: Dean Marx The patch all looks good to me, but it might be beneficial to add a "fixes" tag as well as a "bugzilla ID" tag to the commit body. Th

Re: [PATCH v16 5/5] dts: add API doc generation

2024-08-09 Thread Jeremy Spewock
Looks good to me, I just had a few comments about copyright lines but the actual functionality seems like it's there to me: Reviewed-by: Jeremy Spewock On Thu, Aug 8, 2024 at 4:55 AM Juraj Linkeš wrote: > > The tool used to generate DTS API docs is Sphinx, which is already in &g

Re: [PATCH v16 3/5] dts: add doc generation dependencies

2024-08-09 Thread Jeremy Spewock
picks up the correct Python > installation, as pyelftools is required by the build system. > > Signed-off-by: Juraj Linkeš > --- Reviewed-by: Jeremy Spewock

Re: [PATCH v16 2/5] dts: replace the or operator in third party types

2024-08-09 Thread Jeremy Spewock
ght it was just shipped with later Python versions. Regardless, Reviewed-by: Jeremy Spewock

Re: [PATCH v16 1/5] dts: update params and parser docstrings

2024-08-09 Thread Jeremy Spewock
manipulation module") > Fixes: 967fc62b0a43 ("dts: refactor EAL parameters class") > Fixes: 818fe14e3422 ("dts: add parsing utility module") > Cc: luca.vizza...@arm.com > > Signed-off-by: Juraj Linkeš > Reviewed-by: Luca Vizzarro > --- Reviewed-by: Jeremy Spewock

Re: [PATCH v13 2/2] dts: VLAN test suite implementation

2024-08-09 Thread Jeremy Spewock
On Wed, Aug 7, 2024 at 3:43 PM Dean Marx wrote: > > Test suite for verifying VLAN filtering, stripping, and insertion > functionality on Poll Mode Driver. > > Signed-off-by: Dean Marx > --- > +++ b/dts/tests/TestSuite_vlan.py > @@ -0,0 +1,168 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Co

Re: [PATCH v13 1/2] dts: add VLAN methods to testpmd shell

2024-08-09 Thread Jeremy Spewock
On Wed, Aug 7, 2024 at 3:43 PM Dean Marx wrote: > > added the following methods to testpmd shell class: > vlan set filter on/off, rx vlan add/rm, > vlan set strip on/off, port stop/start all/port, > tx vlan set/reset, set promisc/verbose > > fixed bug in vlan_offload for > show port info, removed

Re: [PATCH v9 2/2] dts: initial queue start/stop suite implementation

2024-08-09 Thread Jeremy Spewock
On Wed, Aug 7, 2024 at 3:36 PM Dean Marx wrote: > > This suite tests the ability of the Poll Mode Driver to enable > and disable Rx/Tx queues on a port. > > Signed-off-by: Dean Marx > --- Reviewed-by: Jeremy Spewock

Re: [PATCH v9 1/2] dts: add functions to testpmd shell

2024-08-09 Thread Jeremy Spewock
On Wed, Aug 7, 2024 at 3:36 PM Dean Marx wrote: > > added set promisc, set verbose, and port stop > commands to testpmd shell. > > Signed-off-by: Dean Marx > --- Reviewed-by: Jeremy Spewock

Re: [RFC v1 1/1] dts: split enums from primary json schema

2024-08-09 Thread Jeremy Spewock
Hey Nick, Thanks for the patch. The way you did things all makes sense to me and I like the idea of pulling out the definitions to make things more compact in the main schema, the one thought I did have though was that it isn't immediately obvious in the main schema where these values are coming f

Re: [PATCH] dts: add l2fwd test suite

2024-08-09 Thread Jeremy Spewock
Hey Luca, Thanks for the patch! Just a few pretty minor comments below. On Tue, Aug 6, 2024 at 8:53 AM Luca Vizzarro wrote: > > Add a basic L2 forwarding test suite which tests the correct > functionality of the forwarding facility built-in in the DPDK. > > The tests are performed with different

Re: [PATCH v2 5/5] dts: add testpmd set ports queues

2024-08-09 Thread Jeremy Spewock
On Tue, Aug 6, 2024 at 8:49 AM Luca Vizzarro wrote: > > Add a facility to update the number of TX/RX queues during the runtime > of testpmd. > > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek > --- > dts/framework/remote_session/testpmd_shell.py | 16 > 1 file chan

Re: [PATCH v2 4/5] dts: add ability to start/stop testpmd ports

2024-08-09 Thread Jeremy Spewock
I tried to address this very same problem in the scatter expansion patch series but, admittedly, I like your approach better. There is one thing that Juraj and I discussed on that thread however that is probably worth noting here regarding verification of the stopping and starting ports. The main i

Re: [PATCH v2 3/5] dts: add random packet generator

2024-08-09 Thread Jeremy Spewock
On Tue, Aug 6, 2024 at 8:49 AM Luca Vizzarro wrote: > > Add a basic utility that can create random L3 and L4 packets with random > payloads and port numbers (if L4). > > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek > Reviewed-by: Alex Chapman > --- Reviewed-by: Jeremy Spewock

Re: [PATCH v2 2/5] dts: add random generation seed setting

2024-08-09 Thread Jeremy Spewock
or one to be > generated and reported back to the user. > > Signed-off-by: Luca Vizzarro > Reviewed-by: Paul Szczepanek > Reviewed-by: Alex Chapman > --- Reviewed-by: Jeremy Spewock

Re: [PATCH v2 1/5] dts: add ability to send/receive multiple packets

2024-08-09 Thread Jeremy Spewock
Hey Luca, Thanks for the patch! I think this change makes a lot of sense in general, and I like the idea of removing the duplication between capturing traffic generators and the test suite. I just had one thought that I left below, but otherwise: Reviewed-by: Jeremy Spewock On Tue, Aug 6, 2024

Re: [PATCH v3 1/1] dts: add text parser for testpmd verbose output

2024-08-08 Thread Jeremy Spewock
On Thu, Aug 8, 2024 at 4:36 PM wrote: > > From: Jeremy Spewock > > Multiple test suites from the old DTS framework rely on being able to > consume and interpret the verbose output of testpmd. The new framework > doesn't have an elegant way for handling the verbose output

Re: [RFC v1 2/2] dts: port ethertype ethdev api test suite to new dts framework

2024-08-08 Thread Jeremy Spewock
On Mon, Aug 5, 2024 at 1:13 PM Nicholas Pratte wrote: > > Based off the test cases from the old DTS test suite, the following test > suite assess the behavior of ethertype configuration options found > within the ethdev api. > > Bugzilla-ID: 1505 > > depends-on: 142696 ("dts: add VLAN methods to t

Re: [RFC v1 1/2] dts: add additional vlan configuration to testpmd shell class

2024-08-08 Thread Jeremy Spewock
Hey Nick, Patch looks pretty good to me, just a few minor comments below. On Mon, Aug 5, 2024 at 1:13 PM Nicholas Pratte wrote: > > The ethertype test suite requires many different internal runtime > vlan offload options to test ethertype configuration. The following patch > adds a new RxVlanOf

Re: [PATCH v2] dts: add flow rule dataclass to testpmd shell

2024-08-02 Thread Jeremy Spewock
On Thu, Aug 1, 2024 at 5:24 AM Luca Vizzarro wrote: > > Hi Dean, thank you for your work! Some minor comments. > > On 26/07/2024 15:15, Dean Marx wrote: > > add dataclass for passing in flow rule creation arguments, as well as a > > __str__ method for converting to a sendable testpmd command. Add

Re: [PATCH v1] dts: add flow rule dataclass to testpmd shell

2024-08-02 Thread Jeremy Spewock
I think Luca made some great points and I agree with what he said, I just had one other though as well. Great work! On Fri, Jul 26, 2024 at 10:22 AM Dean Marx wrote: > + > class TestPmdShell(DPDKShell): > """Testpmd interactive shell. > > @@ -804,6 +841,25 @@ def show_port_stats(self, port

  1   2   3   4   >