Hi Steve,
> -----Original Message-----
> From: Liang, Cunming
> Sent: Sunday, January 4, 2015 10:11 AM
> To: Ouyang, Changchun; dev at dpdk.org
> Cc: Cao, Waterman
> Subject: RE: [PATCH v3 5/6] ixgbe: Config VF RSS
>
>
>
> > -----Original Message-----
> > From: Ouyang, Changchun
> > Sent: Wednesday, December 24, 2014 1:23 PM
> > To: dev at dpdk.org
> > Cc: Liang, Cunming; Cao, Waterman; Ouyang, Changchun
> > Subject: [PATCH v3 5/6] ixgbe: Config VF RSS
> >
> > It needs config RSS and IXGBE_MRQC and IXGBE_VFPSRTYPE to enable VF
> RSS.
> >
> > The psrtype will determine how many queues the received packets will
> > distribute to, and the value of psrtype should depends on both facet:
> > max VF rxq number which has been negotiated with PF, and the number of
> > rxq specified in config on guest.
> >
> > Signed-off-by: Changchun Ouyang <changchun.ouyang at intel.com>
> > ---
> > lib/librte_pmd_ixgbe/ixgbe_pf.c | 15 +++++++
> > lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 92
> > ++++++++++++++++++++++++++++++++++-----
> > 2 files changed, 97 insertions(+), 10 deletions(-)
> >
> > diff --git a/lib/librte_pmd_ixgbe/ixgbe_pf.c
> > b/lib/librte_pmd_ixgbe/ixgbe_pf.c index cbb0145..9c9dad8 100644
> > --- a/lib/librte_pmd_ixgbe/ixgbe_pf.c
> > +++ b/lib/librte_pmd_ixgbe/ixgbe_pf.c
> > @@ -187,6 +187,21 @@ int ixgbe_pf_host_configure(struct rte_eth_dev
> > *eth_dev)
> > IXGBE_WRITE_REG(hw, IXGBE_MPSAR_LO(hw-
> >mac.num_rar_entries),
> > 0);
> > IXGBE_WRITE_REG(hw, IXGBE_MPSAR_HI(hw-
> >mac.num_rar_entries),
> > 0);
> >
> > + /*
> > + * VF RSS can support at most 4 queues for each VF, even if
> > + * 8 queues are available for each VF, it need refine to 4
> > + * queues here due to this limitation, otherwise no queue
> > + * will receive any packet even RSS is enabled.
> > + */
> > + if (eth_dev->data->dev_conf.rxmode.mq_mode ==
> > ETH_MQ_RX_VMDQ_RSS) {
> > + if (RTE_ETH_DEV_SRIOV(eth_dev).nb_q_per_pool == 8) {
> > + RTE_ETH_DEV_SRIOV(eth_dev).active =
> ETH_32_POOLS;
> > + RTE_ETH_DEV_SRIOV(eth_dev).nb_q_per_pool = 4;
> > + RTE_ETH_DEV_SRIOV(eth_dev).def_pool_q_idx =
> > + dev_num_vf(eth_dev) * 4;
> > + }
> > + }
> > +
> > /* set VMDq map to default PF pool */
> > hw->mac.ops.set_vmdq(hw, 0,
> > RTE_ETH_DEV_SRIOV(eth_dev).def_vmdq_idx);
> >
> > diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> > b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> > index f69abda..a7c17a4 100644
> > --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> > +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> > @@ -3327,6 +3327,39 @@ ixgbe_alloc_rx_queue_mbufs(struct
> igb_rx_queue
> > *rxq)
> > }
> >
> > static int
> > +ixgbe_config_vf_rss(struct rte_eth_dev *dev) {
> > + struct ixgbe_hw *hw;
> > + uint32_t mrqc;
> > +
> > + ixgbe_rss_configure(dev);
> > +
> > + hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> > +
> > + /* MRQC: enable VF RSS */
> > + mrqc = IXGBE_READ_REG(hw, IXGBE_MRQC);
> > + mrqc &= ~IXGBE_MRQC_MRQE_MASK;
> > + switch (RTE_ETH_DEV_SRIOV(dev).active) {
> > + case ETH_64_POOLS:
> > + mrqc |= IXGBE_MRQC_VMDQRSS64EN;
> > + break;
> > +
> > + case ETH_32_POOLS:
> > + case ETH_16_POOLS:
> > + mrqc |= IXGBE_MRQC_VMDQRSS32EN;
> > + break;
> > +
> > + default:
> > + PMD_INIT_LOG(ERR, "Invalid pool number in IOV mode");
> > + return -EINVAL;
> > + }
> > +
> > + IXGBE_WRITE_REG(hw, IXGBE_MRQC, mrqc);
> > +
> > + return 0;
> > +}
> > +
> > +static int
> > ixgbe_dev_mq_rx_configure(struct rte_eth_dev *dev) {
> > struct ixgbe_hw *hw =
> > @@ -3358,24 +3391,38 @@ ixgbe_dev_mq_rx_configure(struct
> rte_eth_dev
> > *dev)
> > default: ixgbe_rss_disable(dev);
> > }
> > } else {
> > - switch (RTE_ETH_DEV_SRIOV(dev).active) {
> > /*
> > * SRIOV active scheme
> > * FIXME if support DCB/RSS together with VMDq & SRIOV
> > */
> > - case ETH_64_POOLS:
> > - IXGBE_WRITE_REG(hw, IXGBE_MRQC,
> > IXGBE_MRQC_VMDQEN);
> > + switch (dev->data->dev_conf.rxmode.mq_mode) {
> > + case ETH_MQ_RX_RSS:
> > + case ETH_MQ_RX_VMDQ_RSS:
> > + ixgbe_config_vf_rss(dev);
> > break;
> >
> > - case ETH_32_POOLS:
> > - IXGBE_WRITE_REG(hw, IXGBE_MRQC,
> > IXGBE_MRQC_VMDQRT4TCEN);
> > - break;
> > + default:
> > + switch (RTE_ETH_DEV_SRIOV(dev).active) {
> [Liang, Cunming] Just a minor comments. To avoid a switch branch inside
> another switch, we can have a ixgbe_config_vf_default(), which process all
> the things if no RSS/DCB required in multi-queue setting.
> Then we can put all the 'switch(SRIOV(dev).active){...}' in it.
Yes, will resolve it in v4 patch.
> > + case ETH_64_POOLS:
> > + IXGBE_WRITE_REG(hw, IXGBE_MRQC,
> > + IXGBE_MRQC_VMDQEN);
> > + break;
> >
> > - case ETH_16_POOLS:
> > - IXGBE_WRITE_REG(hw, IXGBE_MRQC,
> > IXGBE_MRQC_VMDQRT8TCEN);
> > + case ETH_32_POOLS:
> > + IXGBE_WRITE_REG(hw, IXGBE_MRQC,
> > + IXGBE_MRQC_VMDQRT4TCEN);
> > + break;
> > +
> > + case ETH_16_POOLS:
> > + IXGBE_WRITE_REG(hw, IXGBE_MRQC,
> > + IXGBE_MRQC_VMDQRT8TCEN);
> > + break;
> > + default:
> > + PMD_INIT_LOG(ERR,
> > + "invalid pool number in IOV mode");
> > + break;
> > + }
> > break;
> > - default:
> > - PMD_INIT_LOG(ERR, "invalid pool number in IOV
> mode");
> > }
> > }
> >
> > @@ -3989,10 +4036,32 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev)
> > uint16_t buf_size;
> > uint16_t i;
> > int ret;
> > + uint16_t valid_rxq_num;
> >
> > PMD_INIT_FUNC_TRACE();
> > hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> >
> > + valid_rxq_num = RTE_MIN(dev->data->nb_rx_queues, hw-
> > >mac.max_rx_queues);
> > +
> > + /*
> > + * VMDq RSS can't support 3 queues, so config it into 4 queues,
> > + * and give user a hint that some packets may loss if it doesn't
> > + * poll the queue where those packets are distributed to.
> > + */
> > + if (valid_rxq_num == 3)
> [Liang, Cunming] According to the inline comment, it makes more sense to
> use 'if (valid_rxq_num >= 3)'.
> In case, the value returned by max_rx_queues is not less equal than 4.
This will be resolved in v4 patch.
Thanks
Changchun
