RE: [PATCH] NTB: Add IDT 89HPESxNTx PCIe-switches support

[Allen Hubbe]

From: Serge Semin
> +static void idt_nt_write(struct idt_ntb_dev *ndev,
> +  const unsigned int reg, const u32 data)
> +{
> + /*
> +  * It's obvious bug to request a register exceeding the maximum possible
> +  * value as well as to have it unaligned.
> +  */
> + BUG_ON(reg > IDT_REG_PCI_MAX || !IS_ALIGNED(reg, IDT_REG_ALIGN));

Avoid BUG_ON.  Just warn and do nothing (at least, do nothing destructive) 
instead of crashing the system.  Here, and throughout the driver.

> +#define to_dev_ndev(ndev) (&((ndev)->ntb.dev))
> +#define to_pci_ndev(ndev) ((ndev)->ntb.pdev)

See Logan's recent patches in "Style fixes: open code obfuscating macros."

[PATCH] NTB: Add IDT 89HPESxNTx PCIe-switches support

[Serge Semin]

IDT 89HPESxNTx device series is PCIe-switches, which support Non-Transparent
bridging between domains connected to the device ports. Since new NTB API
exposes multi-port interface and messaging API, the IDT NT-functions can
be now supported in the kernel. This driver adds the following functionality:
1) Multi-port NTB API to have information of possible NT-functions activated
in compliance with available device ports.
2) Memory windows of direct and look up table based address translation with
all possible combinations of BARs setup.
3) Traditional doorbell NTB API.
4) One-on-one messaging NTB API.

There are some IDT PCIe-switch setups, which must be done before any of
the NTB peers started. It can be performed either by system BIOS via IDT
SMBus-slave interface or by pre-initialized IDT PCIe-switch EEPROM:
1) NT-functions of corresponding ports must be activated using SWPARTxCTL
and SWPORTxCTL registers.
2) BAR0 must be configured to expose NT-function configuration registers map.
3) The rest of the BARs must have at least one memory window configured,
otherwise the driver will just return an error.
Temperature sensor of IDT PCIe-switches can be also optionally activated by
BIOS or EEPROM.
(See IDT documentations for details of how the pre-initialization can be done)

Signed-off-by: Serge Semin 
---
 drivers/ntb/hw/Kconfig  |1 +
 drivers/ntb/hw/Makefile |1 +
 drivers/ntb/hw/idt/Kconfig  |   31 +
 drivers/ntb/hw/idt/Makefile |1 +
 drivers/ntb/hw/idt/ntb_hw_idt.c | 2628 +++
 drivers/ntb/hw/idt/ntb_hw_idt.h | 1160 +
 6 files changed, 3822 insertions(+)
 create mode 100644 drivers/ntb/hw/idt/Kconfig
 create mode 100644 drivers/ntb/hw/idt/Makefile
 create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.c
 create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.h

diff --git a/drivers/ntb/hw/Kconfig b/drivers/ntb/hw/Kconfig
index 7116472..a89243c 100644
--- a/drivers/ntb/hw/Kconfig
+++ b/drivers/ntb/hw/Kconfig
@@ -1,2 +1,3 @@
 source "drivers/ntb/hw/amd/Kconfig"
+source "drivers/ntb/hw/idt/Kconfig"
 source "drivers/ntb/hw/intel/Kconfig"
diff --git a/drivers/ntb/hw/Makefile b/drivers/ntb/hw/Makefile
index 532e085..87332c3 100644
--- a/drivers/ntb/hw/Makefile
+++ b/drivers/ntb/hw/Makefile
@@ -1,2 +1,3 @@
 obj-$(CONFIG_NTB_AMD)  += amd/
+obj-$(CONFIG_NTB_IDT)  += idt/
 obj-$(CONFIG_NTB_INTEL)+= intel/
diff --git a/drivers/ntb/hw/idt/Kconfig b/drivers/ntb/hw/idt/Kconfig
new file mode 100644
index 000..b360e56
--- /dev/null
+++ b/drivers/ntb/hw/idt/Kconfig
@@ -0,0 +1,31 @@
+config NTB_IDT
+   tristate "IDT PCIe-switch Non-Transparent Bridge support"
+   depends on PCI
+   help
+This driver supports NTB of cappable IDT PCIe-switches.
+
+Some of the pre-initializations must be made before IDT PCIe-switch
+exposes it NT-functions correctly. It should be done by either proper
+initialisation of EEPROM connected to master smbus of the switch or
+by BIOS using slave-SMBus interface changing corresponding registers
+value. Evidently it must be done before PCI bus enumeration is
+finished in Linux kernel.
+
+First of all partitions must be activated and properly assigned to all
+the ports with NT-functions intended to be activated (see SWPARTxCTL
+and SWPORTxCTL registers). Then all NT-function BARs must be enabled
+with chosen valid aperture. For memory windows related BARs the
+aperture settings shall determine the maximum size of memory windows
+accepted by a BAR. Note that BAR0 must map PCI configuration space
+registers.
+
+It's worth to note, that since a part of this driver relies on the
+BAR settings of peer NT-functions, the BAR setups can't be done over
+kernel PCI fixups. That's why the alternative pre-initialization
+techniques like BIOS using SMBus interface or EEPROM should be
+utilized. Additionally if one needs to have temperature sensor
+information printed to system log, the corresponding registers must
+be initialized within BIOS/EEPROM as well.
+
+If unsure, say N.
+
diff --git a/drivers/ntb/hw/idt/Makefile b/drivers/ntb/hw/idt/Makefile
new file mode 100644
index 000..a102cf1
--- /dev/null
+++ b/drivers/ntb/hw/idt/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_NTB_IDT) += ntb_hw_idt.o
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt.c b/drivers/ntb/hw/idt/ntb_hw_idt.c
new file mode 100644
index 000..f9d5737
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt.c
@@ -0,0 +1,2628 @@
+/*
+ *   This file is provided under a GPLv2 license.  When using or
+ *   redistributing this file, you may do so under that license.
+ *
+ *   GPL LICENSE SUMMARY
+ *
+ *   Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ *   This program is free software; you can redistribute it and/or modify it
+ *   under the terms and conditions of the GNU General Publi