On Thu, Oct 3, 2019 at 2:29 AM <pbhagavat...@marvell.com> wrote: > > From: Sunil Kumar Kori <sk...@marvell.com> > > Add documentation for l2fwd-event example. > Update release notes. > > Signed-off-by: Sunil Kumar Kori <sk...@marvell.com>
# Please fix the typos through with the following command aspell --lang=en_US --check doc/guides/sample_app_ug/l2_forward_event.rst # Remove the following warning. $ make doc-guides-html doc/guides/sample_app_ug/l2_forward_real_virtual.rst:39: WARNING: duplicate label figure_l2_fwd_benchmark_setup, > --- > MAINTAINERS | 1 + > doc/guides/rel_notes/release_19_11.rst | 6 + > doc/guides/sample_app_ug/index.rst | 1 + > doc/guides/sample_app_ug/intro.rst | 5 + > doc/guides/sample_app_ug/l2_forward_event.rst | 755 ++++++++++++++++++ > 5 files changed, 768 insertions(+) > create mode 100644 doc/guides/sample_app_ug/l2_forward_event.rst > > diff --git a/MAINTAINERS b/MAINTAINERS > index 292ac10c3..94a49b812 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -1461,6 +1461,7 @@ F: examples/l2fwd-cat/ > M: Sunil Kumar Kori <sk...@marvell.com> > M: Pavan Nikhilesh <pbhagavat...@marvell.com> > F: examples/l2fwd-event/ > +F: doc/guides/sample_app_ug/l2_forward_event.rst > T: git://dpdk.org/next/dpdk-next-eventdev > > F: examples/l3fwd/ > diff --git a/doc/guides/rel_notes/release_19_11.rst > b/doc/guides/rel_notes/release_19_11.rst > index 27cfbd9e3..071593e4d 100644 > --- a/doc/guides/rel_notes/release_19_11.rst > +++ b/doc/guides/rel_notes/release_19_11.rst > @@ -56,6 +56,12 @@ New Features > Also, make sure to start the actual text at the margin. > ========================================================= > > +* **Added new example l2fwd-event** Added new example l2fwd-event application > + > + Added an example application `l2fwd-event` that adds event device support > to > + traditional l2fwd example. The default poll mode is also preserved for > + readability. Please change the last sentence to the following or something similar "It demonstrates usage of poll and event mode IO mechanism under a single application" > + > > Removed Items > ------------- > diff --git a/doc/guides/sample_app_ug/index.rst > b/doc/guides/sample_app_ug/index.rst > index f23f8f59e..41388231a 100644 > --- a/doc/guides/sample_app_ug/index.rst > +++ b/doc/guides/sample_app_ug/index.rst > @@ -26,6 +26,7 @@ Sample Applications User Guides > l2_forward_crypto > l2_forward_job_stats > l2_forward_real_virtual > + l2_forward_event > l2_forward_cat > l3_forward > l3_forward_power_man > diff --git a/doc/guides/sample_app_ug/intro.rst > b/doc/guides/sample_app_ug/intro.rst > index 90704194a..84591c0a1 100644 > --- a/doc/guides/sample_app_ug/intro.rst > +++ b/doc/guides/sample_app_ug/intro.rst > @@ -87,6 +87,11 @@ examples are highlighted below. > forwarding, or ``l2fwd`` application does forwarding based on Ethernet MAC > addresses like a simple switch. > > +* :doc:`Network Layer 2 forwarding<l2_forward_event>`: The Network Layer 2 > + forwarding, or ``l2fwd-event`` application does forwarding based on > Ethernet MAC > + addresses like a simple switch. It demonstrate usage of poll and event > mode Rx/Tx > + mechanism. Please change the last sentence to the following or something similar "It demonstrates usage of poll and event mode IO mechanism under a single application" > + > * :doc:`Network Layer 3 forwarding<l3_forward>`: The Network Layer3 > forwarding, or ``l3fwd`` application does forwarding based on Internet > Protocol, IPv4 or IPv6 like a simple router. > diff --git a/doc/guides/sample_app_ug/l2_forward_event.rst > b/doc/guides/sample_app_ug/l2_forward_event.rst > new file mode 100644 > index 000000000..250d16887 > --- /dev/null > +++ b/doc/guides/sample_app_ug/l2_forward_event.rst > @@ -0,0 +1,755 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2010-2014 Intel Corporation. > + > +.. _l2_fwd_event_app: > + > +L2 Forwarding Eventdev Sample Application > +========================================= > + > +The L2 Forwarding eventdev sample application is a simple example of packet > +processing using the Data Plane Development Kit (DPDK) to demonstrate usage > of > +poll and event mode packet I/O mechanism. > + > +Overview > +-------- > + > +The L2 Forwarding eventdev sample application, performs L2 forwarding for > each > +packet that is received on an RX_PORT. The destination port is the adjacent > port > +from the enabled portmask, that is, if the first four ports are enabled > (portmask=0x0f), > +ports 1 and 2 forward into each other, and ports 3 and 4 forward into each > other. > +Also, if MAC addresses updating is enabled, the MAC addresses are affected > as follows: > + > +* The source MAC address is replaced by the TX_PORT MAC address > + > +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID > + > +Appliation receives packets from RX_PORT using below mentioned methods: > + > +* Poll mode > + > +* Eventdev mode (default) > + > +This application can be used to benchmark performance using a > traffic-generator, > +as shown in the :numref:`figure_l2_fwd_benchmark_setup`. > + > +.. _figure_l2_fwd_benchmark_setup: > + > +.. figure:: img/l2_fwd_benchmark_setup.* Looks like there is a typo in the original l2fwd application image. The diagram shows NUT instead of DUT. Please send a separate patch. > + > + Performance Benchmark Setup (Basic Environment) > + > +Compiling the Application > +------------------------- > + > +To compile the sample application see :doc:`compiling`. > + > +The application is located in the ``l2fwd-event`` sub-directory. > + > +Running the Application > +----------------------- > + > +The application requires a number of command line options: > + > +.. code-block:: console > + > + ./build/l2fwd-event [EAL options] -- -p PORTMASK [-q NQ] > --[no-]mac-updating --mode=MODE --eventq-sync=SYNC_MODE > + > +where, > + > +* p PORTMASK: A hexadecimal bitmask of the ports to configure > + > +* q NQ: A number of queues (=ports) per lcore (default is 1) > + > +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled > by default). > + > +* --mode=MODE: Packet transfer mode for I/O, poll or eventdev. Eventdev by > default. > + > +* --eventq-sync=SYNC_MODE: Event queue synchronization method, Ordered or > Atomic. Atomic by default. > + > +Sample usage commands are given below to run the application into different > mode: > + > +Poll mode on linux environment with 4 lcores, 16 ports and 8 RX queues per > lcore Nothing specific to Linux here, so please remove Linux environment > +Driver Initialization > +~~~~~~~~~~~~~~~~~~~~~ > + > +The main part of the code in the main() function relates to the > initialization > +of the driver. To fully understand this code, it is recommended to study the > +chapters that related to the Poll Mode and Event mode Driver in the > +*DPDK Programmer's Guide* - Rel 1.4 EAR and the *DPDK API Reference*. > + > +.. code-block:: c > + > + if (rte_pci_probe() < 0) > + rte_exit(EXIT_FAILURE, "Cannot probe PCI\n"); > + > + /* reset l2fwd_dst_ports */ > + > + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) > + l2fwd_dst_ports[portid] = 0; > + > + last_port = 0; > + > + /* > + * Each logical core is assigned a dedicated TX queue on each port. > + */ > + > + RTE_ETH_FOREACH_DEV(portid) { > + /* skip ports that are not enabled */ > + > + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0) > + continue; > + > + if (nb_ports_in_mask % 2) { > + l2fwd_dst_ports[portid] = last_port; > + l2fwd_dst_ports[last_port] = portid; > + } > + else > + last_port = portid; > + > + nb_ports_in_mask++; > + > + rte_eth_dev_info_get((uint8_t) portid, &dev_info); > + } > + > +Observe that: > + > +* rte_igb_pmd_init_all() simultaneously registers the driver as a PCI > driver Looks like rte_igb_pmd_init_all() from legacy l2fwd doc, Please fix in both docs. The original l2fwd doc fix send it as a separate patch. > +.. code-block:: c > + > + /* Start event device service */ > + ret = rte_event_dev_service_id_get(eventdev_rsrc.event_d_id, > + &service_id); > + if (ret != -ESRCH && ret != 0) > + rte_exit(EXIT_FAILURE, "Error in starting eventdev"); > + > + rte_service_runstate_set(service_id, 1); > + rte_service_set_runstate_mapped_check(service_id, 0); > + eventdev_rsrc.service_id = service_id; eventdev_rsrc removed recently in code review, Please sync code with sync in all code sections. > +.. note:: > + > + In the following code, one line for getting the output port requires some > + explanation. Do we need this note: ? > + > +During the initialization process, a static array of destination ports > +(l2fwd_dst_ports[]) is filled such that for each source port, a destination > port > +is assigned that is either the next or previous enabled port from the > portmask. > +If number of ports are odd in portmask then packet from last port will be > +forwarded to first port i.e. if portmask=0x07, then forwarding will take > place > +like p0--->p1, p1--->p2, p2--->p0. > + > +Also to optimize enqueue opeartion, l2fwd_simple_forward() stores incoming > mbus > +upto MAX_PKT_BURST. Once it reaches upto limit, all packets are transmitted > to > +destination ports. > + > +.. code-block:: c > + > + static void > + l2fwd_simple_forward(struct rte_mbuf *m, uint32_t portid) > + { > + uint32_t dst_port; > + int32_t sent; > + struct rte_eth_dev_tx_buffer *buffer; > + > + dst_port = l2fwd_dst_ports[portid]; > + > + if (mac_updating) > + l2fwd_mac_updating(m, dst_port); > + > + buffer = tx_buffer[dst_port]; > + sent = rte_eth_tx_buffer(dst_port, 0, buffer, m); > + if (sent) > + port_statistics[dst_port].tx += sent; > + } > + > +For this test application, the processing is exactly the same for all packets > +arriving on the same RX port. Therefore, it would have been possible to call > +the rte_eth_tx_buffer() function directly from the main loop to send all the > +received packets on the same TX port, using the burst-oriented send function, > +which is more efficient. > + > +However, in real-life applications (such as, L3 routing), > +packet N is not necessarily forwarded on the same port as packet N-1. > +The application is implemented to illustrate that, so the same approach can > be > +reused in a more complex application. > + > +To ensure that no packets remain in the tables, each lcore does a draining > of TX > +queue in its main loop. This technique introduces some latency when there are > +not many packets to send, however it improves performance: > + > +.. code-block:: c > + > + cur_tsc = rte_rdtsc(); > + > + /* > + * TX burst queue drain > + */ > + diff_tsc = cur_tsc - prev_tsc; > + if (unlikely(diff_tsc > drain_tsc)) { > + for (i = 0; i < qconf->n_rx_port; i++) { > + portid = l2fwd_dst_ports[qconf->rx_port_list[i]]; > + buffer = tx_buffer[portid]; > + sent = rte_eth_tx_buffer_flush(portid, 0, > + buffer); > + if (sent) > + port_statistics[portid].tx += sent; > + } > + > + /* if timer is enabled */ > + if (timer_period > 0) { > + /* advance the timer */ > + timer_tsc += diff_tsc; > + > + /* if timer has reached its timeout */ > + if (unlikely(timer_tsc >= timer_period)) { > + /* do this only on master core */ > + if (lcore_id == rte_get_master_lcore()) { > + print_stats(); > + /* reset the timer */ > + timer_tsc = 0; > + } > + } > + } > + > + prev_tsc = cur_tsc; > + } > + > +In the **l2fwd_main_loop_eventdev()** function, the main task is to read > ingress > +packets from the event ports. This is done using the following code: > + > +.. code-block:: c > + > + /* Read packet from eventdev */ > + nb_rx = rte_event_dequeue_burst(event_d_id, event_p_id, > + events, deq_len, 0); > + if (nb_rx == 0) { > + rte_pause(); > + continue; > + } > + > + for (i = 0; i < nb_rx; i++) { > + mbuf[i] = events[i].mbuf; > + rte_prefetch0(rte_pktmbuf_mtod(mbuf[i], void *)); > + } > + > + > +Before reading packets, deq_len is fetched to ensure correct allowed deq > length > +by the eventdev. > +The rte_event_dequeue_burst() function writes the mbuf pointers in a local > table > +and returns the number of available mbufs in the table. > + > +Then, each mbuf in the table is processed by the l2fwd_eventdev_forward() > +function. The processing is very simple: process the TX port from the RX > port, > +then replace the source and destination MAC addresses if MAC addresses > updating > +is enabled. > + > +.. note:: > + > + In the following code, one line for getting the output port requires some > + explanation. Do we need this note: ? > + > +During the initialization process, a static array of destination ports > +(l2fwd_dst_ports[]) is filled such that for each source port, a destination > port > +is assigned that is either the next or previous enabled port from the > portmask. > +If number of ports are odd in portmask then packet from last port will be > +forwarded to first port i.e. if portmask=0x07, then forwarding will take > place > +like p0--->p1, p1--->p2, p2--->p0. > + > +l2fwd_eventdev_forward() does not stores incoming mbufs. Packet will > forwarded > +be to destination ports via Tx adapter or generic event dev enqueue API > +depending H/W or S/W scheduler is used. > + > +.. code-block:: c > + > + static inline void > + l2fwd_eventdev_forward(struct rte_mbuf *m[], uint32_t portid, > + uint16_t nb_rx, uint16_t event_p_id) > + { > + uint32_t dst_port, i; > + > + dst_port = l2fwd_dst_ports[portid]; > + > + for (i = 0; i < nb_rx; i++) { > + if (mac_updating) > + l2fwd_mac_updating(m[i], dst_port); > + > + m[i]->port = dst_port; > + } > + > + if (timer_period > 0) { > + rte_spinlock_lock(&port_stats_lock); > + port_statistics[dst_port].tx += nb_rx; > + rte_spinlock_unlock(&port_stats_lock); > + } > + /* Registered callback is invoked for Tx */ > + eventdev_rsrc.send_burst_eventdev(m, nb_rx, event_p_id); > + } > -- > 2.17.1 >