Author: np
Date: Mon Mar 14 09:50:14 2011
New Revision: 219633
URL: http://svn.freebsd.org/changeset/base/219633
Log:
MFC cxgbe(4) and fixes.
r218792:
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters.
r219285:
Fix incorrect assertion.
r219286:
Resume tx immediately in response to an SGE egress update from the hardware.
r219287:
Upgrade the firmware on the card automatically if a better version is
available. Downgrade only for a major version mismatch.
r219288:
A txpkts work request should have a valid FID.
r219289:
Store the ifnet rather than the port_info in each txq and rxq struct.
r219290:
Tweaks for rx:
- everything related to LRO should be in #ifdef INET blocks
- reorder sge_iq's fields so that the most frequently used are all together
- pull all rx code into t4_intr_data directly
- let go of the ingress queue lock when passing up data
- refill the freelist only if it is short of at least 32 buffers
r219292:
Calculate how many descriptors can be reclaimed before calling
reclaim_tx_descs
r219293:
There is no need to hold an ingress queue's lock while processing its
descriptors
r219299:
Be sure to stay within the bounds of the mod_str array when displaying
the transceiver type.
r219392:
cxgbe shouldn't directly know of the UMA zones where network buffers
come from.
r219436:
Display holdoff timers and packet counts as a list of numbers.
Added:
stable/8/share/man/man4/cxgbe.4
- copied unchanged from r218792, head/share/man/man4/cxgbe.4
stable/8/sys/dev/cxgbe/
- copied from r218792, head/sys/dev/cxgbe/
stable/8/sys/modules/cxgbe/
- copied from r218792, head/sys/modules/cxgbe/
Modified:
stable/8/share/man/man4/Makefile
stable/8/share/man/man4/altq.4
stable/8/share/man/man4/vlan.4
stable/8/sys/conf/NOTES
stable/8/sys/conf/files
stable/8/sys/conf/kern.pre.mk
stable/8/sys/dev/cxgbe/adapter.h
stable/8/sys/dev/cxgbe/common/common.h
stable/8/sys/dev/cxgbe/t4_main.c
stable/8/sys/dev/cxgbe/t4_sge.c
stable/8/sys/modules/Makefile
stable/8/usr.sbin/sysinstall/devices.c
Directory Properties:
stable/8/share/man/man4/ (props changed)
stable/8/sys/ (props changed)
stable/8/sys/amd64/include/xen/ (props changed)
stable/8/sys/cddl/contrib/opensolaris/ (props changed)
stable/8/sys/contrib/dev/acpica/ (props changed)
stable/8/sys/contrib/pf/ (props changed)
stable/8/usr.sbin/sysinstall/ (props changed)
Modified: stable/8/share/man/man4/Makefile
==============================================================================
--- stable/8/share/man/man4/Makefile Mon Mar 14 05:29:45 2011
(r219632)
+++ stable/8/share/man/man4/Makefile Mon Mar 14 09:50:14 2011
(r219633)
@@ -81,6 +81,7 @@ MAN= aac.4 \
crypto.4 \
cue.4 \
cxgb.4 \
+ cxgbe.4 \
cy.4 \
da.4 \
dc.4 \
Modified: stable/8/share/man/man4/altq.4
==============================================================================
--- stable/8/share/man/man4/altq.4 Mon Mar 14 05:29:45 2011
(r219632)
+++ stable/8/share/man/man4/altq.4 Mon Mar 14 09:50:14 2011
(r219633)
@@ -127,6 +127,7 @@ They have been applied to the following
.Xr bfe 4 ,
.Xr bge 4 ,
.Xr cas 4 ,
+.Xr cxgbe 4 ,
.Xr dc 4 ,
.Xr de 4 ,
.Xr ed 4 ,
Copied: stable/8/share/man/man4/cxgbe.4 (from r218792,
head/share/man/man4/cxgbe.4)
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ stable/8/share/man/man4/cxgbe.4 Mon Mar 14 09:50:14 2011
(r219633, copy of r218792, head/share/man/man4/cxgbe.4)
@@ -0,0 +1,167 @@
+.\" Copyright (c) 2011, Chelsio Inc
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions are met:
+.\"
+.\" 1. Redistributions of source code must retain the above copyright notice,
+.\" this list of conditions and the following disclaimer.
+.\"
+.\" 2. Redistributions in binary form must reproduce the above copyright
+.\" notice, this list of conditions and the following disclaimer in the
+.\" documentation and/or other materials provided with the distribution.
+.\"
+.\" 3. Neither the name of the Chelsio Inc nor the names of its
+.\" contributors may be used to endorse or promote products derived from
+.\" this software without specific prior written permission.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+.\" POSSIBILITY OF SUCH DAMAGE.
+.\"
+.\" * Other names and brands may be claimed as the property of others.
+.\"
+.\" $FreeBSD$
+.\"
+.Dd February 14, 2011
+.Dt CXGBE 4
+.Os
+.Sh NAME
+.Nm cxgbe
+.Nd "Chelsio T4 10Gb and 1Gb Ethernet adapter driver"
+.Sh SYNOPSIS
+To compile this driver into the kernel,
+place the following lines in your
+kernel configuration file:
+.Bd -ragged -offset indent
+.Cd "device cxgbe"
+.Ed
+.Pp
+To load the driver as a
+module at boot time, place the following line in
+.Xr loader.conf 5 :
+.Bd -literal -offset indent
+if_cxgbe_load="YES"
+.Ed
+.Sh DESCRIPTION
+The
+.Nm
+driver provides support for PCI Express Ethernet adapters based on
+the Chelsio Terminator 4 (T4) ASIC.
+The driver supprts Jumbo Frames, Transmit/Receive checksum offload,
+TCP segmentation offload (TSO), Large Receive Offload (LRO), VLAN
+tag insertion/extraction, VLAN checksum offload, VLAN TSO, and
+Receive Side Steering (RSS).
+
+For further hardware information and questions related to hardware
+requirements, see
+.Pa http://www.chelsio.com/ .
+.Pp
+For more information on configuring this device, see
+.Xr ifconfig 8 .
+.Sh HARDWARE
+The
+.Nm
+driver supports 10Gb and 1Gb Ethernet adapters based on the T4 ASIC:
+.Pp
+.Bl -bullet -compact
+.It
+Chelsio T420-CR
+.It
+Chelsio T422-CR
+.It
+Chelsio T440-CR
+.It
+Chelsio T420-BCH
+.It
+Chelsio T440-BCH
+.It
+Chelsio T440-CH
+.It
+Chelsio T420-SO
+.It
+Chelsio T420-CX
+.It
+Chelsio T420-BT
+.It
+Chelsio T404-BT
+.El
+.Sh LOADER TUNABLES
+Tunables can be set at the
+.Xr loader 8
+prompt before booting the kernel or stored in
+.Xr loader.conf 5 .
+.Bl -tag -width indent
+.It Va hw.cxgbe.max_ntxq_10G_port
+The maximum number of tx queues to use for a 10Gb port.
+The default value is 8.
+.It Va hw.cxgbe.max_nrxq_10G_port
+The maximum number of rx queues to use for a 10Gb port.
+The default value is 8.
+.It Va hw.cxgbe.max_ntxq_1G_port
+The maximum number of tx queues to use for a 1Gb port.
+The default value is 2.
+.It Va hw.cxgbe.max_nrxq_1G_port
+The maximum number of rx queues to use for a 1Gb port.
+The default value is 2.
+.It Va hw.cxgbe.holdoff_timer_idx_10G
+.It Va hw.cxgbe.holdoff_timer_idx_1G
+The timer index value to use to delay interrupts.
+The holdoff timer list has the values 1, 5, 10, 50, 100, and 200
+by default (all values are in microseconds) and the index selects a
+value from this list.
+The default value is 1 for both 10Gb and 1Gb ports, which means the
+timer value is 5us.
+.It Va hw.cxgbe.holdoff_pktc_idx_10G
+.It Va hw.cxgbe.holdoff_pktc_idx_1G
+The packet-count index value to use to delay interrupts.
+The packet-count list has the values 1, 8, 16, and 32 by default
+and the index selects a value from this list.
+The default value is 2 for both 10Gb and 1Gb ports, which means 16
+packets (or the holdoff timer going off) before an interrupt is
+generated.
+.It Va hw.cxgbe.qsize_txq
+The size, in number of entries, of the descriptor ring used for a tx
+queue.
+A buf_ring of the same size is also allocated for additional
+software queuing. See
+.Xr ifnet 9 .
+The default value is 1024.
+.It Va hw.cxgbe.qsize_rxq
+The size, in number of entries, of the descriptor ring used for an
+rx queue.
+The default value is 1024.
+.Sh SUPPORT
+For general information and support,
+go to the Chelsio support website at:
+.Pa http://www.chelsio.com/ .
+.Pp
+If an issue is identified with this driver with a supported adapter,
+email all the specific information related to the issue to
+.Aq supp...@chelsio.com .
+.Sh SEE ALSO
+.Xr altq 4 ,
+.Xr arp 4 ,
+.Xr cxgb 4 ,
+.Xr netintro 4 ,
+.Xr ng_ether 4 ,
+.Xr ifconfig 8
+.Sh HISTORY
+The
+.Nm
+device driver first appeared in
+.Fx 9.0
+.Sh AUTHORS
+.An -nosplit
+The
+.Nm
+driver was written by
+.An Navdeep Parhar Aq n...@freebsd.org .
Modified: stable/8/share/man/man4/vlan.4
==============================================================================
--- stable/8/share/man/man4/vlan.4 Mon Mar 14 05:29:45 2011
(r219632)
+++ stable/8/share/man/man4/vlan.4 Mon Mar 14 09:50:14 2011
(r219633)
@@ -128,6 +128,7 @@ in the hardware is limited to the follow
.Xr bce 4 ,
.Xr bge 4 ,
.Xr cxgb 4 ,
+.Xr cxgbe 4 ,
.Xr em 4 ,
.Xr igb 4 ,
.Xr ixgb 4 ,
Modified: stable/8/sys/conf/NOTES
==============================================================================
--- stable/8/sys/conf/NOTES Mon Mar 14 05:29:45 2011 (r219632)
+++ stable/8/sys/conf/NOTES Mon Mar 14 09:50:14 2011 (r219633)
@@ -1877,6 +1877,8 @@ device xmphy # XaQti XMAC II
# cas: Sun Cassini/Cassini+ and National Semiconductor DP83065 Saturn
# cm: Arcnet SMC COM90c26 / SMC COM90c56
# (and SMC COM90c66 in '56 compatibility mode) adapters.
+# cxgbe: Support for PCI express 10Gb/1Gb adapters based on the Chelsio T4
+# (Terminator 4) ASIC.
# dc: Support for PCI fast ethernet adapters based on the DEC/Intel 21143
# and various workalikes including:
# the ADMtek AL981 Comet and AN985 Centaur, the ASIX Electronics
@@ -2048,6 +2050,7 @@ device xl # 3Com 3c90x
(``Boomerang'',
# PCI Ethernet NICs.
device bwi # Broadcom BCM430* BCM431*
+device cxgbe # Chelsio T4 10GbE PCIe adapter
device de # DEC/Intel DC21x4x (``Tulip'')
device em # Intel Pro/1000 Gigabit Ethernet
device igb # Intel Pro/1000 PCIE Gigabit Ethernet
Modified: stable/8/sys/conf/files
==============================================================================
--- stable/8/sys/conf/files Mon Mar 14 05:29:45 2011 (r219632)
+++ stable/8/sys/conf/files Mon Mar 14 09:50:14 2011 (r219633)
@@ -822,6 +822,12 @@ dev/cxgb/sys/uipc_mvec.c optional cxgb p
compile-with "${NORMAL_C} -I$S/dev/cxgb"
dev/cxgb/cxgb_t3fw.c optional cxgb cxgb_t3fw \
compile-with "${NORMAL_C} -I$S/dev/cxgb"
+dev/cxgbe/t4_main.c optional cxgbe pci \
+ compile-with "${NORMAL_C} -I$S/dev/cxgbe"
+dev/cxgbe/t4_sge.c optional cxgbe pci \
+ compile-with "${NORMAL_C} -I$S/dev/cxgbe"
+dev/cxgbe/common/t4_hw.c optional cxgbe pci \
+ compile-with "${NORMAL_C} -I$S/dev/cxgbe"
dev/cy/cy.c optional cy
dev/cy/cy_isa.c optional cy isa
dev/cy/cy_pci.c optional cy pci
Modified: stable/8/sys/conf/kern.pre.mk
==============================================================================
--- stable/8/sys/conf/kern.pre.mk Mon Mar 14 05:29:45 2011
(r219632)
+++ stable/8/sys/conf/kern.pre.mk Mon Mar 14 09:50:14 2011
(r219633)
@@ -79,8 +79,8 @@ INCLUDES+= -I$S/dev/twa
# ... and XFS
INCLUDES+= -I$S/gnu/fs/xfs/FreeBSD -I$S/gnu/fs/xfs/FreeBSD/support
-I$S/gnu/fs/xfs
-# ... and the same for cxgb
-INCLUDES+= -I$S/dev/cxgb
+# ... and the same for cxgb and cxgbe
+INCLUDES+= -I$S/dev/cxgb -I$S/dev/cxgbe
.endif
Modified: stable/8/sys/dev/cxgbe/adapter.h
==============================================================================
--- head/sys/dev/cxgbe/adapter.h Fri Feb 18 08:00:26 2011
(r218792)
+++ stable/8/sys/dev/cxgbe/adapter.h Mon Mar 14 09:50:14 2011
(r219633)
@@ -70,8 +70,8 @@ static __inline uint64_t
t4_bus_space_read_8(bus_space_tag_t tag, bus_space_handle_t handle,
bus_size_t offset)
{
- KASSERT(tag == X86_BUS_SPACE_IO,
- ("64-bit reads from I/O space not possible."));
+ KASSERT(tag == AMD64_BUS_SPACE_MEM,
+ ("%s: can only handle mem space", __func__));
return (*(volatile uint64_t *)(handle + offset));
}
@@ -80,8 +80,9 @@ static __inline void
t4_bus_space_write_8(bus_space_tag_t tag, bus_space_handle_t bsh,
bus_size_t offset, uint64_t value)
{
- KASSERT(tag == X86_BUS_SPACE_IO,
- ("64-bit writes to I/O space not possible."));
+ KASSERT(tag == AMD64_BUS_SPACE_MEM,
+ ("%s: can only handle mem space", __func__));
+
*(volatile uint64_t *)(bsh + offset) = value;
}
#else
@@ -114,7 +115,11 @@ enum {
RX_FL_ESIZE = 64, /* 8 64bit addresses */
- FL_BUF_SIZES = 4,
+#if MJUMPAGESIZE != MCLBYTES
+ FL_BUF_SIZES = 4, /* cluster, jumbop, jumbo9k, jumbo16k */
+#else
+ FL_BUF_SIZES = 3, /* cluster, jumbo9k, jumbo16k */
+#endif
TX_EQ_QSIZE = 1024,
TX_EQ_ESIZE = 64,
@@ -176,6 +181,7 @@ struct port_info {
struct link_config link_cfg;
struct port_stats stats;
+ struct taskqueue *tq;
struct callout tick;
struct sysctl_ctx_list ctx; /* lives from ifconfig up to down */
struct sysctl_oid *oid_rxq;
@@ -222,24 +228,25 @@ enum {
struct sge_iq {
bus_dma_tag_t desc_tag;
bus_dmamap_t desc_map;
- struct mtx iq_lock;
+ bus_addr_t ba; /* bus address of descriptor ring */
char lockname[16];
- unsigned int flags;
- struct adapter *adapter;
+ uint32_t flags;
+ uint16_t abs_id; /* absolute SGE id for the iq */
+ int8_t intr_pktc_idx; /* packet count threshold index */
+ int8_t pad0;
+ iq_intr_handler_t *handler;
+ __be64 *desc; /* KVA of descriptor ring */
- __be64 *desc; /* KVA of descriptor ring */
- bus_addr_t ba; /* bus address of descriptor ring */
+ struct mtx iq_lock;
+ struct adapter *adapter;
const __be64 *cdesc; /* current descriptor */
uint8_t gen; /* generation bit */
uint8_t intr_params; /* interrupt holdoff parameters */
- int8_t intr_pktc_idx; /* packet count threshold index */
uint8_t intr_next; /* holdoff for next interrupt */
uint8_t esize; /* size (bytes) of each entry in the queue */
uint16_t qsize; /* size (# of entries) of the queue */
uint16_t cidx; /* consumer index */
uint16_t cntxt_id; /* SGE context id for the iq */
- uint16_t abs_id; /* absolute SGE id for the iq */
- iq_intr_handler_t *handler;
};
enum {
@@ -274,6 +281,7 @@ struct sge_eq {
uint16_t cidx; /* consumer idx (desc idx) */
uint16_t pidx; /* producer idx (desc idx) */
uint16_t pending; /* # of descriptors used since last doorbell */
+ uint16_t iqid; /* iq that gets egr_update for the eq */
uint32_t cntxt_id; /* SGE context id for the eq */
/* DMA maps used for tx */
@@ -309,6 +317,9 @@ struct sge_fl {
struct sge_txq {
struct sge_eq eq; /* MUST be first */
struct mbuf *m; /* held up due to temporary resource shortage */
+ struct task resume_tx;
+
+ struct ifnet *ifp; /* the interface this txq belongs to */
/* stats for common events first */
@@ -336,9 +347,11 @@ struct sge_rxq {
struct sge_iq iq; /* MUST be first */
struct sge_fl fl;
+ struct ifnet *ifp; /* the interface this rxq belongs to */
unsigned int flags;
- struct port_info *port; /* the port this rxq belongs to */
+#ifdef INET
struct lro_ctrl lro; /* LRO state */
+#endif
/* stats for common events first */
@@ -544,13 +557,16 @@ static inline bool is_10G_port(const str
return ((pi->link_cfg.supported & FW_PORT_CAP_SPEED_10G) != 0);
}
+/* t4_main.c */
+void cxgbe_txq_start(void *, int);
int t4_os_find_pci_capability(struct adapter *, int);
int t4_os_pci_save_state(struct adapter *);
int t4_os_pci_restore_state(struct adapter *);
-
void t4_os_portmod_changed(const struct adapter *, int);
void t4_os_link_changed(struct adapter *, int, int);
+/* t4_sge.c */
+void t4_sge_modload(void);
void t4_sge_init(struct adapter *);
int t4_create_dma_tag(struct adapter *);
int t4_destroy_dma_tag(struct adapter *);
Modified: stable/8/sys/dev/cxgbe/common/common.h
==============================================================================
--- head/sys/dev/cxgbe/common/common.h Fri Feb 18 08:00:26 2011
(r218792)
+++ stable/8/sys/dev/cxgbe/common/common.h Mon Mar 14 09:50:14 2011
(r219633)
@@ -53,8 +53,8 @@ enum {
};
#define FW_VERSION_MAJOR 1
-#define FW_VERSION_MINOR 2
-#define FW_VERSION_MICRO 65
+#define FW_VERSION_MINOR 3
+#define FW_VERSION_MICRO 0
struct port_stats {
u64 tx_octets; /* total # of octets in good frames */
Modified: stable/8/sys/dev/cxgbe/t4_main.c
==============================================================================
--- head/sys/dev/cxgbe/t4_main.c Fri Feb 18 08:00:26 2011
(r218792)
+++ stable/8/sys/dev/cxgbe/t4_main.c Mon Mar 14 09:50:14 2011
(r219633)
@@ -36,11 +36,15 @@ __FBSDID("$FreeBSD$");
#include <sys/kernel.h>
#include <sys/bus.h>
#include <sys/module.h>
+#include <sys/malloc.h>
+#include <sys/queue.h>
+#include <sys/taskqueue.h>
#include <sys/pciio.h>
#include <dev/pci/pcireg.h>
#include <dev/pci/pcivar.h>
#include <dev/pci/pci_private.h>
#include <sys/firmware.h>
+#include <sys/sbuf.h>
#include <sys/smp.h>
#include <sys/socket.h>
#include <sys/sockio.h>
@@ -269,12 +273,14 @@ static void t4_get_regs(struct adapter *
static void cxgbe_tick(void *);
static int t4_sysctls(struct adapter *);
static int cxgbe_sysctls(struct port_info *);
+static int sysctl_int_array(SYSCTL_HANDLER_ARGS);
static int sysctl_holdoff_tmr_idx(SYSCTL_HANDLER_ARGS);
static int sysctl_holdoff_pktc_idx(SYSCTL_HANDLER_ARGS);
static int sysctl_qsize_rxq(SYSCTL_HANDLER_ARGS);
static int sysctl_qsize_txq(SYSCTL_HANDLER_ARGS);
static int sysctl_handle_t4_reg64(SYSCTL_HANDLER_ARGS);
-
+static inline void txq_start(struct ifnet *, struct sge_txq *);
+static int t4_mod_event(module_t, int, void *);
struct t4_pciids {
uint16_t device;
@@ -692,6 +698,15 @@ cxgbe_attach(device_t dev)
ifp->if_softc = pi;
callout_init(&pi->tick, CALLOUT_MPSAFE);
+ pi->tq = taskqueue_create("cxgbe_taskq", M_NOWAIT,
+ taskqueue_thread_enqueue, &pi->tq);
+ if (pi->tq == NULL) {
+ device_printf(dev, "failed to allocate port task queue\n");
+ if_free(pi->ifp);
+ return (ENOMEM);
+ }
+ taskqueue_start_threads(&pi->tq, 1, PI_NET, "%s taskq",
+ device_get_nameunit(dev));
if_initname(ifp, device_get_name(dev), device_get_unit(dev));
ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST;
@@ -746,6 +761,8 @@ cxgbe_detach(device_t dev)
if (rc != 0)
device_printf(dev, "port uninit failed: %d.\n", rc);
+ taskqueue_free(pi->tq);
+
ifmedia_removeall(&pi->media);
ether_ifdetach(pi->ifp);
if_free(pi->ifp);
@@ -951,13 +968,7 @@ cxgbe_start(struct ifnet *ifp)
for_each_txq(pi, i, txq) {
if (TXQ_TRYLOCK(txq)) {
- struct buf_ring *br = txq->eq.br;
- struct mbuf *m;
-
- m = txq->m ? txq->m : drbr_dequeue(ifp, br);
- if (m)
- t4_eth_tx(ifp, txq, m);
-
+ txq_start(ifp, txq);
TXQ_UNLOCK(txq);
}
}
@@ -1247,28 +1258,69 @@ prep_firmware(struct adapter *sc)
/* Check firmware version and install a different one if necessary */
rc = t4_check_fw_version(sc);
if (rc != 0 || force_firmware_install) {
+ uint32_t v = 0;
fw = firmware_get(T4_FWNAME);
- if (fw == NULL) {
- device_printf(sc->dev,
- "Could not find firmware image %s\n", T4_FWNAME);
- return (ENOENT);
+ if (fw != NULL) {
+ const struct fw_hdr *hdr = (const void *)fw->data;
+
+ v = ntohl(hdr->fw_ver);
+
+ /*
+ * The firmware module will not be used if it isn't the
+ * same major version as what the driver was compiled
+ * with. This check trumps force_firmware_install.
+ */
+ if (G_FW_HDR_FW_VER_MAJOR(v) != FW_VERSION_MAJOR) {
+ device_printf(sc->dev,
+ "Found firmware image but version %d "
+ "can not be used with this driver (%d)\n",
+ G_FW_HDR_FW_VER_MAJOR(v), FW_VERSION_MAJOR);
+
+ firmware_put(fw, FIRMWARE_UNLOAD);
+ fw = NULL;
+ }
}
- device_printf(sc->dev,
- "installing firmware %d.%d.%d on card.\n",
- FW_VERSION_MAJOR, FW_VERSION_MINOR, FW_VERSION_MICRO);
- rc = -t4_load_fw(sc, fw->data, fw->datasize);
- if (rc != 0) {
+ if (fw == NULL && (rc < 0 || force_firmware_install)) {
+ device_printf(sc->dev, "No usable firmware. "
+ "card has %d.%d.%d, driver compiled with %d.%d.%d, "
+ "force_firmware_install%s set",
+ G_FW_HDR_FW_VER_MAJOR(sc->params.fw_vers),
+ G_FW_HDR_FW_VER_MINOR(sc->params.fw_vers),
+ G_FW_HDR_FW_VER_MICRO(sc->params.fw_vers),
+ FW_VERSION_MAJOR, FW_VERSION_MINOR,
+ FW_VERSION_MICRO,
+ force_firmware_install ? "" : " not");
+ return (EAGAIN);
+ }
+
+ /*
+ * Always upgrade, even for minor/micro/build mismatches.
+ * Downgrade only for a major version mismatch or if
+ * force_firmware_install was specified.
+ */
+ if (fw != NULL && (rc < 0 || force_firmware_install ||
+ v > sc->params.fw_vers)) {
device_printf(sc->dev,
- "failed to install firmware: %d\n", rc);
- return (rc);
- } else {
- t4_get_fw_version(sc, &sc->params.fw_vers);
- t4_get_tp_version(sc, &sc->params.tp_vers);
+ "installing firmware %d.%d.%d.%d on card.\n",
+ G_FW_HDR_FW_VER_MAJOR(v), G_FW_HDR_FW_VER_MINOR(v),
+ G_FW_HDR_FW_VER_MICRO(v), G_FW_HDR_FW_VER_BUILD(v));
+
+ rc = -t4_load_fw(sc, fw->data, fw->datasize);
+ if (rc != 0) {
+ device_printf(sc->dev,
+ "failed to install firmware: %d\n", rc);
+ firmware_put(fw, FIRMWARE_UNLOAD);
+ return (rc);
+ } else {
+ /* refresh */
+ (void) t4_check_fw_version(sc);
+ }
}
- firmware_put(fw, FIRMWARE_UNLOAD);
+ if (fw != NULL)
+ firmware_put(fw, FIRMWARE_UNLOAD);
}
/* Contact firmware, request master */
@@ -2244,15 +2296,13 @@ t4_sysctls(struct adapter *sc)
SYSCTL_ADD_INT(ctx, children, OID_AUTO, "core_clock", CTLFLAG_RD,
&sc->params.vpd.cclk, 0, "core clock frequency (in KHz)");
- /* XXX: this doesn't seem to show up */
- SYSCTL_ADD_OPAQUE(ctx, children, OID_AUTO, "holdoff_tmr",
- CTLFLAG_RD, &intr_timer, sizeof(intr_timer), "IU",
- "interrupt holdoff timer values (us)");
-
- /* XXX: this doesn't seem to show up */
- SYSCTL_ADD_OPAQUE(ctx, children, OID_AUTO, "holdoff_pktc",
- CTLFLAG_RD, &intr_pktcount, sizeof(intr_pktcount), "IU",
- "interrupt holdoff packet counter values");
+ SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "holdoff_timers",
+ CTLTYPE_STRING | CTLFLAG_RD, &intr_timer, sizeof(intr_timer),
+ sysctl_int_array, "A", "interrupt holdoff timer values (us)");
+
+ SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "holdoff_pkt_counts",
+ CTLTYPE_STRING | CTLFLAG_RD, &intr_pktcount, sizeof(intr_pktcount),
+ sysctl_int_array, "A", "interrupt holdoff packet counter values");
return (0);
}
@@ -2304,7 +2354,7 @@ cxgbe_sysctls(struct port_info *pi)
#define SYSCTL_ADD_T4_REG64(pi, name, desc, reg) \
SYSCTL_ADD_OID(ctx, children, OID_AUTO, name, \
- CTLTYPE_U64 | CTLFLAG_RD, pi->adapter, reg, \
+ CTLTYPE_QUAD | CTLFLAG_RD, pi->adapter, reg, \
sysctl_handle_t4_reg64, "QU", desc)
SYSCTL_ADD_T4_REG64(pi, "tx_octets", "# of octets in good frames",
@@ -2428,7 +2478,7 @@ cxgbe_sysctls(struct port_info *pi)
#undef SYSCTL_ADD_T4_REG64
#define SYSCTL_ADD_T4_PORTSTAT(name, desc) \
- SYSCTL_ADD_UQUAD(ctx, children, OID_AUTO, #name, CTLFLAG_RD, \
+ SYSCTL_ADD_QUAD(ctx, children, OID_AUTO, #name, CTLFLAG_RD, \
&pi->stats.name, desc)
/* We get these from port_stats and they may be stale by upto 1s */
@@ -2455,6 +2505,22 @@ cxgbe_sysctls(struct port_info *pi)
}
static int
+sysctl_int_array(SYSCTL_HANDLER_ARGS)
+{
+ int rc, *i;
+ struct sbuf sb;
+
+ sbuf_new(&sb, NULL, 32, SBUF_AUTOEXTEND);
+ for (i = arg1; arg2; arg2 -= sizeof(int), i++)
+ sbuf_printf(&sb, "%d ", *i);
+ sbuf_trim(&sb);
+ sbuf_finish(&sb);
+ rc = sysctl_handle_string(oidp, sbuf_data(&sb), sbuf_len(&sb), req);
+ sbuf_delete(&sb);
+ return (rc);
+}
+
+static int
sysctl_holdoff_tmr_idx(SYSCTL_HANDLER_ARGS)
{
struct port_info *pi = arg1;
@@ -2578,7 +2644,31 @@ sysctl_handle_t4_reg64(SYSCTL_HANDLER_AR
val = t4_read_reg64(sc, reg);
- return (sysctl_handle_64(oidp, &val, 0, req));
+ return (sysctl_handle_quad(oidp, &val, 0, req));
+}
+
+static inline void
+txq_start(struct ifnet *ifp, struct sge_txq *txq)
+{
+ struct buf_ring *br;
+ struct mbuf *m;
+
+ TXQ_LOCK_ASSERT_OWNED(txq);
+
+ br = txq->eq.br;
+ m = txq->m ? txq->m : drbr_dequeue(ifp, br);
+ if (m)
+ t4_eth_tx(ifp, txq, m);
+}
+
+void
+cxgbe_txq_start(void *arg, int count)
+{
+ struct sge_txq *txq = arg;
+
+ TXQ_LOCK(txq);
+ txq_start(txq->ifp, txq);
+ TXQ_UNLOCK(txq);
}
int
@@ -2646,6 +2736,7 @@ t4_os_pci_restore_state(struct adapter *
pci_cfg_restore(dev, dinfo);
return (0);
}
+
void
t4_os_portmod_changed(const struct adapter *sc, int idx)
{
@@ -2656,10 +2747,13 @@ t4_os_portmod_changed(const struct adapt
if (pi->mod_type == FW_PORT_MOD_TYPE_NONE)
if_printf(pi->ifp, "transceiver unplugged.\n");
- else
+ else if (pi->mod_type > 0 && pi->mod_type < ARRAY_SIZE(mod_str)) {
if_printf(pi->ifp, "%s transceiver inserted.\n",
mod_str[pi->mod_type]);
-
+ } else {
+ if_printf(pi->ifp, "transceiver (type %d) inserted.\n",
+ pi->mod_type);
+ }
}
void
@@ -2737,10 +2831,20 @@ t4_ioctl(struct cdev *dev, unsigned long
return (rc);
}
+static int
+t4_mod_event(module_t mod, int cmd, void *arg)
+{
+
+ if (cmd == MOD_LOAD)
+ t4_sge_modload();
+
+ return (0);
+}
+
static devclass_t t4_devclass;
static devclass_t cxgbe_devclass;
-DRIVER_MODULE(t4nex, pci, t4_driver, t4_devclass, 0, 0);
+DRIVER_MODULE(t4nex, pci, t4_driver, t4_devclass, t4_mod_event, 0);
MODULE_VERSION(t4nex, 1);
DRIVER_MODULE(cxgbe, t4nex, cxgbe_driver, cxgbe_devclass, 0, 0);
Modified: stable/8/sys/dev/cxgbe/t4_sge.c
==============================================================================
--- head/sys/dev/cxgbe/t4_sge.c Fri Feb 18 08:00:26 2011 (r218792)
+++ stable/8/sys/dev/cxgbe/t4_sge.c Mon Mar 14 09:50:14 2011
(r219633)
@@ -34,6 +34,9 @@ __FBSDID("$FreeBSD$");
#include <sys/mbuf.h>
#include <sys/socket.h>
#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/queue.h>
+#include <sys/taskqueue.h>
#include <sys/sysctl.h>
#include <net/bpf.h>
#include <net/ethernet.h>
@@ -55,13 +58,9 @@ struct fl_buf_info {
uma_zone_t zone;
};
-/* t4_sge_init will fill up the zone */
-static struct fl_buf_info fl_buf_info[FL_BUF_SIZES] = {
- { MCLBYTES, EXT_CLUSTER, NULL},
- { MJUMPAGESIZE, EXT_JUMBOP, NULL},
- { MJUM9BYTES, EXT_JUMBO9, NULL},
- { MJUM16BYTES, EXT_JUMBO16, NULL}
-};
+/* Filled up by t4_sge_modload */
+static struct fl_buf_info fl_buf_info[FL_BUF_SIZES];
+
#define FL_BUF_SIZE(x) (fl_buf_info[x].size)
#define FL_BUF_TYPE(x) (fl_buf_info[x].type)
#define FL_BUF_ZONE(x) (fl_buf_info[x].zone)
@@ -118,7 +117,6 @@ static int alloc_fl_sdesc(struct sge_fl
static void free_fl_sdesc(struct sge_fl *);
static int alloc_eq_maps(struct sge_eq *);
static void free_eq_maps(struct sge_eq *);
-static struct mbuf *get_fl_sdesc_data(struct sge_fl *, int, int);
static void set_fl_tag_idx(struct sge_fl *, int);
static int get_pkt_sgl(struct sge_txq *, struct mbuf **, struct sgl *, int);
@@ -133,9 +131,35 @@ static inline void write_ulp_cpl_sgl(str
static int write_sgl_to_txd(struct sge_eq *, struct sgl *, caddr_t *);
static inline void copy_to_txd(struct sge_eq *, caddr_t, caddr_t *, int);
static inline void ring_tx_db(struct adapter *, struct sge_eq *);
+static inline int reclaimable(struct sge_eq *);
static int reclaim_tx_descs(struct sge_eq *, int, int);
static void write_eqflush_wr(struct sge_eq *);
static __be64 get_flit(bus_dma_segment_t *, int, int);
+static int handle_sge_egr_update(struct adapter *,
+ const struct cpl_sge_egr_update *);
+
+/*
+ * Called on MOD_LOAD and fills up fl_buf_info[].
+ */
+void
+t4_sge_modload(void)
+{
+ int i;
+ int bufsize[FL_BUF_SIZES] = {
+ MCLBYTES,
+#if MJUMPAGESIZE != MCLBYTES
+ MJUMPAGESIZE,
+#endif
+ MJUM9BYTES,
+ MJUM16BYTES
+ };
+
+ for (i = 0; i < FL_BUF_SIZES; i++) {
+ FL_BUF_SIZE(i) = bufsize[i];
+ FL_BUF_TYPE(i) = m_gettype(bufsize[i]);
+ FL_BUF_ZONE(i) = m_getzone(bufsize[i]);
+ }
+}
/**
* t4_sge_init - initialize SGE
@@ -151,11 +175,6 @@ t4_sge_init(struct adapter *sc)
struct sge *s = &sc->sge;
int i;
- FL_BUF_ZONE(0) = zone_clust;
- FL_BUF_ZONE(1) = zone_jumbop;
- FL_BUF_ZONE(2) = zone_jumbo9;
- FL_BUF_ZONE(3) = zone_jumbo16;
-
t4_set_reg_field(sc, A_SGE_CONTROL, V_PKTSHIFT(M_PKTSHIFT) |
V_INGPADBOUNDARY(M_INGPADBOUNDARY) |
F_EGRSTATUSPAGESIZE,
@@ -409,7 +428,6 @@ t4_intr_fwd(void *arg)
int ndesc_pending = 0, ndesc_total = 0;
int qid;
- IQ_LOCK(iq);
while (is_new_response(iq, &ctrl)) {
rmb();
@@ -436,7 +454,6 @@ t4_intr_fwd(void *arg)
iq_next(iq);
}
- IQ_UNLOCK(iq);
if (ndesc_total > 0) {
t4_write_reg(sc, MYPF_REG(A_SGE_PF_GTS),
@@ -469,7 +486,6 @@ t4_intr_evt(void *arg)
KASSERT(iq == &sc->sge.fwq, ("%s: unexpected ingress queue", __func__));
- IQ_LOCK(iq);
while (is_new_response(iq, &ctrl)) {
rmb();
@@ -492,21 +508,9 @@ t4_intr_evt(void *arg)
break;
}
- case CPL_SGE_EGR_UPDATE: {
- const struct cpl_sge_egr_update *cpl;
- unsigned int qid;
- struct sge *s = &sc->sge;
- struct sge_txq *txq;
-
- cpl = (const void *)(rss + 1);
- qid = G_EGR_QID(ntohl(cpl->opcode_qid));
- txq = (void *)s->eqmap[qid - s->eq_start];
- txq->egr_update++;
-
- /* XXX: wake up stalled tx */
-
+ case CPL_SGE_EGR_UPDATE:
+ handle_sge_egr_update(sc, (const void *)(rss + 1));
break;
- }
default:
device_printf(sc->dev,
@@ -524,7 +528,6 @@ t4_intr_evt(void *arg)
}
iq_next(iq);
}
- IQ_UNLOCK(iq);
if (ndesc_total > 0) {
t4_write_reg(sc, MYPF_REG(A_SGE_PF_GTS),
@@ -538,63 +541,73 @@ t4_intr_data(void *arg)
{
struct sge_rxq *rxq = arg;
struct sge_iq *iq = arg;
+ struct adapter *sc = iq->adapter;
struct rsp_ctrl *ctrl;
+ struct ifnet *ifp = rxq->ifp;
struct sge_fl *fl = &rxq->fl;
- struct port_info *pi = rxq->port;
- struct ifnet *ifp = pi->ifp;
- struct adapter *sc = pi->adapter;
+ struct fl_sdesc *sd = &fl->sdesc[fl->cidx], *sd_next;
const struct rss_header *rss;
const struct cpl_rx_pkt *cpl;
- int ndescs = 0, rsp_type;
uint32_t len;
+ int ndescs = 0, i;
struct mbuf *m0, *m;
#ifdef INET
struct lro_ctrl *lro = &rxq->lro;
struct lro_entry *l;
#endif
- IQ_LOCK(iq);
+ prefetch(sd->m);
+ prefetch(sd->cl);
+
iq->intr_next = iq->intr_params;
while (is_new_response(iq, &ctrl)) {
rmb();
rss = (const void *)iq->cdesc;
- cpl = (const void *)(rss + 1);
+ i = G_RSPD_TYPE(ctrl->u.type_gen);
- rsp_type = G_RSPD_TYPE(ctrl->u.type_gen);
+ if (__predict_false(i == X_RSPD_TYPE_CPL)) {
- if (__predict_false(rsp_type == X_RSPD_TYPE_CPL)) {
- const struct cpl_sge_egr_update *p = (const void *)cpl;
- unsigned int qid = G_EGR_QID(ntohl(p->opcode_qid));
+ /* Can't be anything except an egress update */
+ KASSERT(rss->opcode == CPL_SGE_EGR_UPDATE,
+ ("%s: unexpected CPL %x", __func__, rss->opcode));
- KASSERT(cpl->opcode == CPL_SGE_EGR_UPDATE,
- ("unexpected opcode on data ingress queue: %x",
- cpl->opcode));
-
- /* XXX: noone's waiting to be woken up... */
- wakeup(sc->sge.eqmap[qid - sc->sge.eq_start]);
+ handle_sge_egr_update(sc, (const void *)(rss + 1));
+ goto nextdesc;
+ }
+ KASSERT(i == X_RSPD_TYPE_FLBUF && rss->opcode == CPL_RX_PKT,
+ ("%s: unexpected CPL %x rsp %d", __func__, rss->opcode, i));
- ndescs++;
- iq_next(iq);
+ sd_next = sd + 1;
+ if (__predict_false(fl->cidx + 1 == fl->cap))
+ sd_next = fl->sdesc;
+ prefetch(sd_next->m);
+ prefetch(sd_next->cl);
- continue;
- }
+ cpl = (const void *)(rss + 1);
- KASSERT(G_RSPD_TYPE(ctrl->u.type_gen) == X_RSPD_TYPE_FLBUF,
- ("unexpected event on data ingress queue: %x",
- G_RSPD_TYPE(ctrl->u.type_gen)));
+ m0 = sd->m;
+ sd->m = NULL; /* consumed */
len = be32toh(ctrl->pldbuflen_qid);
+ if (__predict_false((len & F_RSPD_NEWBUF) == 0))
+ panic("%s: cannot handle packed frames", __func__);
+ len = G_RSPD_LEN(len);
- KASSERT(len & F_RSPD_NEWBUF,
- ("%s: T4 misconfigured to pack buffers.", __func__));
+ bus_dmamap_sync(fl->tag[sd->tag_idx], sd->map,
+ BUS_DMASYNC_POSTREAD);
- len = G_RSPD_LEN(len);
- m0 = get_fl_sdesc_data(fl, len, M_PKTHDR);
- if (m0 == NULL) {
- iq->intr_next = V_QINTR_TIMER_IDX(SGE_NTIMERS - 1);
- break;
+ m_init(m0, NULL, 0, M_NOWAIT, MT_DATA, M_PKTHDR);
+ if (len < MINCLSIZE) {
+ /* copy data to mbuf, buffer will be recycled */
+ bcopy(sd->cl, mtod(m0, caddr_t), len);
+ m0->m_len = len;
+ } else {
+ bus_dmamap_unload(fl->tag[sd->tag_idx], sd->map);
+ m_cljset(m0, sd->cl, FL_BUF_TYPE(sd->tag_idx));
+ sd->cl = NULL; /* consumed */
+ m0->m_len = min(len, FL_BUF_SIZE(sd->tag_idx));
}
len -= FL_PKTSHIFT;
@@ -623,16 +636,49 @@ t4_intr_data(void *arg)
rxq->vlan_extraction++;
}
+ i = 1; /* # of fl sdesc used */
+ sd = sd_next;
+ if (__predict_false(++fl->cidx == fl->cap))
+ fl->cidx = 0;
+
len -= m0->m_len;
m = m0;
while (len) {
- m->m_next = get_fl_sdesc_data(fl, len, 0);
- if (m->m_next == NULL)
- CXGBE_UNIMPLEMENTED("mbuf recovery");
+ i++;
+ sd_next = sd + 1;
+ if (__predict_false(fl->cidx + 1 == fl->cap))
+ sd_next = fl->sdesc;
+ prefetch(sd_next->m);
+ prefetch(sd_next->cl);
+
+ m->m_next = sd->m;
+ sd->m = NULL; /* consumed */
m = m->m_next;
+
+ bus_dmamap_sync(fl->tag[sd->tag_idx], sd->map,
+ BUS_DMASYNC_POSTREAD);
+
+ m_init(m, NULL, 0, M_NOWAIT, MT_DATA, 0);
+ if (len <= MLEN) {
+ bcopy(sd->cl, mtod(m, caddr_t), len);
+ m->m_len = len;
+ } else {
+ bus_dmamap_unload(fl->tag[sd->tag_idx],
+ sd->map);
+ m_cljset(m, sd->cl, FL_BUF_TYPE(sd->tag_idx));
+ sd->cl = NULL; /* consumed */
+ m->m_len = min(len, FL_BUF_SIZE(sd->tag_idx));
+ }
+
+ i++;
+ sd = sd_next;
+ if (__predict_false(++fl->cidx == fl->cap))
+ fl->cidx = 0;
+
len -= m->m_len;
}
+
#ifdef INET
if (cpl->l2info & htobe32(F_RXF_LRO) &&
rxq->flags & RXQ_LRO_ENABLED &&
@@ -640,17 +686,17 @@ t4_intr_data(void *arg)
/* queued for LRO */
} else
#endif
- (*ifp->if_input)(ifp, m0);
+ ifp->if_input(ifp, m0);
FL_LOCK(fl);
- if (fl->needed >= 32) {
+ fl->needed += i;
+ if (fl->needed >= 32)
*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
