Re: PCIe multipliers, how do they work?
I'm wondering how these things work. The wikipedia PCIe page [1] says: "PCI Express switches can create multiple endpoints out of one endpoint to allow sharing one endpoint with multiple devices." So maybe they use a PCIe switch? Poking around wikipedia and google has thus far uncovered very little info about PCIe switches. Wikipedia is less helpful than usual, and they keep making google less and less useful for no apparent reason. I don't see any other obvious keywords to google for. It isn't obvious how slot id/address is handled. How do commands and data get routed to/from the correct card? Is any firmware or OS support required? It's worth reading the following two pages closely, as they talk about PCI Express endpoints, and how PCI-Express switches have to work to enable multiple cards. http://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tutorial-guide-1 http://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tutorial-guide-2 One such family of PCI-express switch parts is from PLX Technologies, now owned by Broadcom. One of typical functions of such as switch is to provide "non transparent bridging" - basically mapping the address space from one machine's PCI address into the address space of another PCI system. This can be used a communication path between devices, for example, using a block and slot type software architecture. There is some support for NTB in FreeBSD's master branch, under sys/dev/ntb - with drivers for the PLX/Avago/Broadcom parts, as well as a driver for the Intel parts found on certain Xeon and Atom CPUs. -Kurt ___ freebsd-hardware@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-hardware To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"
Re: PCIe multipliers, how do they work?
On Fri, May 11, 2018 at 1:03 PM, Wojciech Pucharwrote: > There are also PCIe cards which provide multiple slots, typically >> connected with a usb cable. These tend to convert 1 PCIe_x1 slot >> into multiple PCIe_x1 slots. I get the impression that these do >> not require bifurcation support. They seem to be aimed at "miners" >> > > these cards consist of PCIe switch which is supported out of the box. > former ones require BIOS to reconfigure CPUs PCI lanes so instead of eg > one 16x lane there will be 4 4x lanes. > Usually they require a driver for management functions, like if you want to turn off one of the ports (because you know there's a bad card in it, for example). But for normal probe / attach they are usually drop in. Some mobos can require some BIOS adjustment, though to properly setup lane bifurcation and such... The ones that are true switches, and not just signal retimers, usually don't: they take the full 8 or 16 lanes and multiplex it amongst the 16-32 downstream lanes they typically provide. We're looking at one that does x4 lanes and expands to 4 cards at x4 lanes. The devices on the other side fill just over a lane each, but we'd run out of lanes if we did x2 fan-out. This the cards we're looking at, we can get x4 combined rates form the 4 cards that on their own are kinda crappy. Warner ___ freebsd-hardware@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-hardware To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"
Re: PCIe multipliers, how do they work?
There are also PCIe cards which provide multiple slots, typically connected with a usb cable. These tend to convert 1 PCIe_x1 slot into multiple PCIe_x1 slots. I get the impression that these do not require bifurcation support. They seem to be aimed at "miners" these cards consist of PCIe switch which is supported out of the box. former ones require BIOS to reconfigure CPUs PCI lanes so instead of eg one 16x lane there will be 4 4x lanes. ___ freebsd-hardware@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-hardware To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"
PCIe multipliers, how do they work?
Looking into ways to get additional expansion slots, since board designers cannot count past 7 and often not even that high. :-( There are PCIe riser cards that split a wide slot into 2 or more narrower slots, for example 1 x8 slot becomes 2 x4 slots. These would be very useful, except it appears that they require "bifurcation" support in the mainboard's firmware. Which most boards do not provide. And most boards are not supported by FLOSS firmware, so adding bifurcation support would be rather difficult. There are also PCIe cards which provide multiple slots, typically connected with a usb cable. These tend to convert 1 PCIe_x1 slot into multiple PCIe_x1 slots. I get the impression that these do not require bifurcation support. They seem to be aimed at "miners" for attaching multiple gpu cards. I'm not interested in mining or in attaching multiple gpu cards. I'm interesting in adding additional sata cards, Ethernet cards, and such. Unlike the bifurcation type riser splitter cards, which seem to conserve PCIe lanes, these are more like a sata port multiplier, with the same type of bandwidth limitation. I'm wondering how these things work. The wikipedia PCIe page [1] says: "PCI Express switches can create multiple endpoints out of one endpoint to allow sharing one endpoint with multiple devices." So maybe they use a PCIe switch? Poking around wikipedia and google has thus far uncovered very little info about PCIe switches. Wikipedia is less helpful than usual, and they keep making google less and less useful for no apparent reason. I don't see any other obvious keywords to google for. It isn't obvious how slot id/address is handled. How do commands and data get routed to/from the correct card? Is any firmware or OS support required? Is there some other solution that I haven't stumbled across? I'd really like to split an x8 slot into 4 x2 slots, which doesn't seem to be an off-the-shelf option either way. [1] en.wikipedia.org/wiki/PCI_Express ___ freebsd-hardware@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-hardware To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"