On Thu, Jun 16, 2016 at 10:46:23PM +0200, Francois Romieu wrote:
> Netanel Belgazal <neta...@annapurnalabs.com> :
> [...]
>
> Very limited review below.
I'll comment on the documentation (since I edited it heavily) but will
leave some the other parts for Netanel to answer.
> > diff --git a/Documentation/networking/ena.txt
> > b/Documentation/networking/ena.txt
> > new file mode 100644
> > index 0000000..528544f
> > --- /dev/null
> > +++ b/Documentation/networking/ena.txt
> > @@ -0,0 +1,330 @@
> > +Linux kernel driver for Elastic Network Adapter (ENA) family:
> > +=============================================================
> > +
> > +Overview:
> > +=========
> > +The ENA driver provides a modern Ethernet device interface optimized
> > +for high performance and low CPU overhead.
>
> Marketing boilerplate. How much of it will still be true in 6 months ?
> 6 years ?
>
> Hint: stay technical. If in doubt, make it boring. :o)
Hopefully it will still be true for quite a while. But you make a good
point, the overview here should stick with the technical details.
> [...]
> > +ENA devices allow high speed and low overhead Ethernet traffic
> > +processing by providing a dedicated Tx/Rx queue pair per host CPU, a
> > +dedicated MSI-X interrupt vector per Tx/Rx queue pair, adaptive
> > +interrupt moderation, and CPU cacheline optimized data placement.
>
> How many queues exactly ?
It's negotiated with the adapter via the admin queue, and it will vary
based on the capabilities of a particular adapter. See:
rc = ena_com_get_feature(ena_dev, &get_resp,
ENA_ADMIN_MAX_QUEUES_NUM);
in ena_com.c
> [...]
> > +Memory Allocations:
> > +===================
> > +DMA Coherent buffers are allocated for the f`ollowing DMA rings:
> > +- Tx submission ring (For regular mode; for LLQ mode it is allocated
> > + using kzalloc)
> > +- Tx completion ring
> > +- Rx submission ring
> > +- Rx completion ring
> > +- Admin submission ring
> > +- Admin completion ring
> > +- AENQ ring
>
> Useless documentation (implementation detail).
>
> Nobody will maintain it when the driver changes. Please remove.
Ack.
> [...]
> > +MTU:
> > +====
> > +The driver supports an arbitrarily large MTU with a maximum that is
> > +negotiated with the device. The driver configures MTU using the
> > +SetFeature command (ENA_ADMIN_MTU property). The user can change MTU
> > +via ifconfig(8) and ip(8).
>
> Please avoid advertising legacy ifconfig.
Ack.
> [...]
> > diff --git a/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
> > b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
> > new file mode 100644
> > index 0000000..8516e5e
> > --- /dev/null
> > +++ b/drivers/net/ethernet/amazon/ena/ena_admin_defs.h
> [...]
> > +/* admin commands opcodes */
> > +enum ena_admin_aq_opcode {
> > + /* create submission queue */
> > + ENA_ADMIN_CREATE_SQ = 1,
> > +
> > + /* destroy submission queue */
> > + ENA_ADMIN_DESTROY_SQ = 2,
> > +
> > + /* create completion queue */
> > + ENA_ADMIN_CREATE_CQ = 3,
> > +
> > + /* destroy completion queue */
> > + ENA_ADMIN_DESTROY_CQ = 4,
> > +
> > + /* get capabilities of particular feature */
> > + ENA_ADMIN_GET_FEATURE = 8,
> > +
> > + /* get capabilities of particular feature */
> > + ENA_ADMIN_SET_FEATURE = 9,
> > +
> > + /* get statistics */
> > + ENA_ADMIN_GET_STATS = 11,
> > +};
>
> The documentation above simply duplicates the member names -> useless
> visual payload.
>
> Please replace space with tab before "=" to line things up.
I'll leave this to Netanel. A good amount of this comes from a unified
documentation / struct / register access generation script, and it
might need some massaging.
> [...]
> > +/* Basic Statistics Command. */
> > +struct ena_admin_basic_stats {
> > + /* word 0 : */
> > + u32 tx_bytes_low;
> > +
> > + /* word 1 : */
> > + u32 tx_bytes_high;
> > +
> > + /* word 2 : */
> > + u32 tx_pkts_low;
> > +
> > + /* word 3 : */
> > + u32 tx_pkts_high;
> > +
> > + /* word 4 : */
> > + u32 rx_bytes_low;
> > +
> > + /* word 5 : */
> > + u32 rx_bytes_high;
> > +
> > + /* word 6 : */
> > + u32 rx_pkts_low;
> > +
> > + /* word 7 : */
> > + u32 rx_pkts_high;
> > +
> > + /* word 8 : */
> > + u32 rx_drops_low;
> > +
> > + /* word 9 : */
> > + u32 rx_drops_high;
> > +};
>
> struct zorg {
> u32 ...;
> u32 ...;
> u32 ...;
> u32 ...;
>
> u32 ...;
> u32 ...;
> u32 ...;
> u32 ...;
>
> u32 ...;
> u32 ...;
> u32 ...;
> };
>
> -> No comment needed to figure the offset.
> Comment removed => no need to check if offset starts at word 0 or word 1.
>
> [...]
> > diff --git a/drivers/net/ethernet/amazon/ena/ena_com.c
> > b/drivers/net/ethernet/amazon/ena/ena_com.c
> > new file mode 100644
> > index 0000000..357e10f
> > --- /dev/null
> > +++ b/drivers/net/ethernet/amazon/ena/ena_com.c
> [...]
> > +static inline int ena_com_mem_addr_set(struct ena_com_dev *ena_dev,
> > + struct ena_common_mem_addr *ena_addr,
> > + dma_addr_t addr)
> > +{
> > + if ((addr & GENMASK_ULL(ena_dev->dma_addr_bits - 1, 0)) != addr) {
> > + ena_trc_err("dma address has more bits that the device
> > supports\n");
> > + return -EINVAL;
> > + }
> > +
> > + ena_addr->mem_addr_low = (u32)addr;
> > + ena_addr->mem_addr_high =
> > + ((addr & GENMASK_ULL(ena_dev->dma_addr_bits - 1, 32)) >> 32);
>
> No need to "... & GENMASK" given the test above.
Ack.
> > +
> > + return 0;
> > +}
> > +
> > +static int ena_com_admin_init_sq(struct ena_com_admin_queue *queue)
> > +{
> > + queue->sq.entries =
> > + dma_alloc_coherent(queue->q_dmadev,
> > + ADMIN_SQ_SIZE(queue->q_depth),
> > + &queue->sq.dma_addr,
> > + GFP_KERNEL | __GFP_ZERO);
>
> 1. Use dma_zalloc_coherent().
>
> 2. Style
> xyzab = dma_alloc_coherent(..., ..., ... ...,
> ...);
>
> 3. Add local variable for &queue->sq.
Ack.
> In a different part of the code: s/uint32_t/u32/
Argh! I thought I caught all of those.
Thanks for the review!
--msw
