Re: Making tmpfs reserved memory configurable
On Thu, Jun 05, 2014 at 11:04:06PM +, Eduardo Horvath wrote: Maybe. It's set in uvmpd_tune(), which may be called by the page daemon in some circumstances. Take a look at sys/uvm/uvm_pdaemon.c. Hmm, should this be made more explicit in the mount_tmpfs documentation? Under such low memory conditions, an unbound tmpfs might not behave completely deterministially any more. Martin
Re: RFC: add MSI/MSI-X support to NetBSD
On Fri, May 30, 2014 at 05:55:25PM +0900, Kengo NAKAHARA wrote: Hello, I'm going to add MSI/MSI-X support to NetBSD. I list tasks about this. Would you comment following task list? I think that MSI/MSI-X logically separates into a few pieces, what do you think about these pieces? 1 An MI API for establishing mailboxes (or doorbells or whatever we may call them). A mailbox is a special physical address (PA) or PA/data-pair in correspondence with a callback (function, argument). An MI API for mapping the mailbox into various address spaces, but especially the message-signalling devices. In this way, the mailbox API is a use or an extension of bus_dma(9). Somewhere I have a draft proposal for this MI API, I will try to dig it up. 2 For each platform, an MD implementation of the MI mailbox API. 3 Extensions to pci(9) for establishing message-signalled interrupts using either a (function, argument) pair, a PA, or a (PA, data) pair. I am pretty sure that the implementation of these extensions can be MI. + [amd64 MD] refactor INTRSTUB - currently, it walks the interrupt handler list in assembly code - I want to use NetBSD's list library, so I want to convert this assembly code to C code. I support converting much of the interrupt dispatch code to C from assembly. Dave -- David Young dyo...@pobox.comUrbana, IL(217) 721-9981
Re: RFC: add MSI/MSI-X support to NetBSD
On Jun 6, 2014, at 10:40 AM, David Young dyo...@pobox.com wrote: 1 An MI API for establishing mailboxes (or doorbells or whatever we may call them). A mailbox is a special physical address (PA) or PA/data-pair in correspondence with a callback (function, argument). An MI API for mapping the mailbox into various address spaces, but especially the message-signalling devices. In this way, the mailbox API is a use or an extension of bus_dma(9). Somewhere I have a draft proposal for this MI API, I will try to dig it up. Note that some system need to point the MSI at a specific address (PPC PQIII for instance).
Re: RFC: add MSI/MSI-X support to NetBSD
On Jun 6, 2014, at 10:40 AM, David Young dyo...@pobox.com wrote: On Fri, May 30, 2014 at 05:55:25PM +0900, Kengo NAKAHARA wrote: Hello, I'm going to add MSI/MSI-X support to NetBSD. I list tasks about this. Would you comment following task list? I think that MSI/MSI-X logically separates into a few pieces, what do you think about these pieces? Don't forget being able to supply a kcpuset_t * In fact, now that we have SMP we need to rethink the intr_establish stuff to better deal with instead of hacking around it.
Re: RFC: add MSI/MSI-X support to NetBSD
On Fri, Jun 06, 2014 at 12:40:54PM -0500, David Young wrote:
On Fri, May 30, 2014 at 05:55:25PM +0900, Kengo NAKAHARA wrote:
Hello,
I'm going to add MSI/MSI-X support to NetBSD. I list tasks about this.
Would you comment following task list?
I think that MSI/MSI-X logically separates into a few pieces, what do
you think about these pieces?
1 An MI API for establishing mailboxes (or doorbells or whatever
we may call them). A mailbox is a special physical address (PA) or
PA/data-pair in correspondence with a callback (function, argument).
An MI API for mapping the mailbox into various address spaces,
but especially the message-signalling devices. In this way, the
mailbox API is a use or an extension of bus_dma(9).
Somewhere I have a draft proposal for this MI API, I will try to
dig it up.
Here is the proposal that I came up with many months (a few years?) ago
with input from Matt Thomas. I have tried to account for Matt's
requirements, but I'm not sure that I have done so.
Dave
--
David Young
dyo...@pobox.comUrbana, IL(217) 721-9981
BUS_MSI(9) NetBSD Kernel Developer's Manual BUS_MSI(9)
bus_msi(9) is a machine-independent interface for establishing in the
machine's physical address space a doorbell that when written with
a particular word, sends an interrupt vector to a set of CPUs. Using
bus_msi(9), the interrupt vector can be tied to interrupt handlers.
bus_msi(9) is the basis for a machine-independent implementation
of PCI Message-Signaled Interrupts (MSI) and MSI-X, however, the
bus_msi(9) implementation itself is highly machine-dependent. Any
NetBSD architecture that wants to support PCI MSI should provide a
bus_msi(9) implementation.
bus_msi(9) uses facilities provided by bus_dma(9).
typedef struct _bus_msi_t {
bus_addr_t mi_addr;
uint32_tmi_data;
uint32_tmi_count;
};
int
bus_msi_alloc(bus_dma_tag_t tag, bus_msi_reservation_t *msirp, size_t n,
uint32_t data_min, uint32_t data_max,
uint32_t data_alignment, uint32_t data_boundary, int flags);
Reserve `number' interrupt vectors on up to `ncpumax' CPUs
in the set `cpusetin' and reserve corresponding message
address/message data pairs. Record the message address/data-pair
reservations in up to `nintervals' consecutive bus_msi_interval_ts
beginning with `interval[0]'; overwrite `rintervals' with
the number of intervals used. Overwrite `cpusetout' with
the set of CPUs where interrupt vectors were established.
Each bus_msg_interval_t tells a message address, mi_addr,
and the mi_count different 32-bit message data words,
[mi_data,�mi_data�+�mi_count�-�1], to write to trigger
mi_count different interrupt vectors.
Each message data interval, [mi_data, mi_data + mi_count�-�1]
will satisfy the constraints passed to bus_msg_alloc():
[data_min,�data_max] must enclose each interval, each
interval must start at a multiple of data_alignment, and
no interval may cross a data_boundary boundary. A legal
value of data_alignment (or data_boundary) is either zero
or a power of 2. When zero, data_alignment (or data_boundary)
has no effect.
`tag' is the bus_dma_tag_t passed by the parent driver via
the bus _attach_args.
`flags' may be one of BUS_DMA_WAITOK or BUS_DMA_NOWAIT.
bus_msi_handle_t
bus_msi_establish(bus_dma_tag_t tag, bus_msi_reservation_t msir, int idx,
const kcpuset_t *cpusetin, int ncpumax, kcpuset_t *cpusetout,
int ipl, int (*func)(void *), void *arg);
Establish a callback (func, arg) to run at interrupt priority
level `ipl' whenever the `idx'th message in `intervals' is
delivered. Return an opaque handle for use with
bus_msi_disestablish().
You can establish more than one handler at each `idx'.
The correspondence between `idx's and message-address/data
pairs is like this:
idx 0 - (intervals[0].mi_addr, intervals[0].mi_data)
idx 1 - (intervals[0].mi_addr, intervals[0].mi_data + 1)
. . .
idx N - 1 - (intervals[0].mi_addr, intervals[0].mi_data +
intervals[0].mi_count - 1)
idx N - (intervals[1].mi_addr, intervals[1].mi_data)
idx N + 1 - (intervals[1].mi_addr, intervals[1].mi_data + 1)
. . .
idx N + K - 1 - (intervals[1].mi_addr, intervals[1].mi_data +
intervals[1].mi_count - 1)
void
bus_msi_disestablish(bus_dma_tag_t tag, bus_msi_handle_t);
Disestablish the callback established previously with
bus_msi_handle_t.
void
bus_msi_free(bus_dma_tag_t tag, bus_msi_reservation_t msir, int idx, size_t n);
Release intervals allocated with bus_msi_alloc().
Re: RFC: add MSI/MSI-X support to NetBSD
Date: Fri, 6 Jun 2014 12:56:53 -0500 From: David Young dyo...@pobox.com Here is the proposal that I came up with many months (a few years?) ago with input from Matt Thomas. I have tried to account for Matt's requirements, but I'm not sure that I have done so. For those ignoramuses among us who remain perplexed by the apparent difficulty of using a new interrupt delivery mechanism, could you add some notes to your proposal about what driver authors would need to know about it and when how one would use it in a driver? Would all architectures with PCI support bus_msi(9), or would PCI device drivers need to conditionally use it? Why isn't it just a matter of modifying pci_intr_map, or calling pci_intr_map_msi like in OpenBSD? Would there be other non-PCI buses with message-signalled interrupts too? (Still not having done my homework to study what this MSI business is all about, I'll note parenthetically that it seems FreeBSD and OpenBSD have supported MSI for a while, and I understand neither why it was so easy for them nor what advantage they lack by not having bus_msi(9).)
Re: RFC: add MSI/MSI-X support to NetBSD
On Jun 6, 2014, at 12:06 PM, Taylor R Campbell campbell+netbsd-tech-k...@mumble.net wrote: Date: Fri, 6 Jun 2014 12:56:53 -0500 From: David Young dyo...@pobox.com Here is the proposal that I came up with many months (a few years?) ago with input from Matt Thomas. I have tried to account for Matt's requirements, but I'm not sure that I have done so. For those ignoramuses among us who remain perplexed by the apparent difficulty of using a new interrupt delivery mechanism, could you add some notes to your proposal about what driver authors would need to know about it and when how one would use it in a driver? Would all architectures with PCI support bus_msi(9), or would PCI device drivers need to conditionally use it? Why isn't it just a matter of modifying pci_intr_map, or calling pci_intr_map_msi like in OpenBSD? Would there be other non-PCI buses with message-signalled interrupts too? Those that support PCIe or PCIX and support MSIs should change. But we can continue the legacy INT[A-D]. For performance, MSIs should be more efficient. (Still not having done my homework to study what this MSI business is all about, I'll note parenthetically that it seems FreeBSD and OpenBSD have supported MSI for a while, and I understand neither why it was so easy for them nor what advantage they lack by not having bus_msi(9).)
Re: RFC: add MSI/MSI-X support to NetBSD
On Fri, Jun 06, 2014 at 07:06:00PM +, Taylor R Campbell wrote: Date: Fri, 6 Jun 2014 12:56:53 -0500 From: David Young dyo...@pobox.com Here is the proposal that I came up with many months (a few years?) ago with input from Matt Thomas. I have tried to account for Matt's requirements, but I'm not sure that I have done so. For those ignoramuses among us who remain perplexed by the apparent difficulty of using a new interrupt delivery mechanism, could you add some notes to your proposal about what driver authors would need to know about it and when how one would use it in a driver? Driver authors do not need to know anything about bus_msi(9) unless they're doing something fancy. bus_msi(9) will be invisible to the author of a PCI driver because pci_intr(9) will establish the mailbox and all of that. bus_msi(9) hides differences between hardware platforms. Would all architectures with PCI support bus_msi(9), or would PCI device drivers need to conditionally use it? Why isn't it just a matter of modifying pci_intr_map, or calling pci_intr_map_msi like in OpenBSD? For a PCI driver, it *is* a matter of calling pci_intr_map(9), or whatever NetBSD comes up with for MSI/MSI-X. Would there be other non-PCI buses with message-signalled interrupts too? It's conceivable that there are existing non-PCI buses that use message-signalled interrupts. Any future bus probably will. (Still not having done my homework to study what this MSI business is all about, I'll note parenthetically that it seems FreeBSD and OpenBSD have supported MSI for a while, and I understand neither why it was so easy for them nor what advantage they lack by not having bus_msi(9).) NetBSD has supported MSI (but not MSI-X) for a while, too, at least on i386 or x86. Here are the nice things about MSI/MSI-X in a nutshell: you can have many interrupt sources per device (IIRC, there are just 4 interrupt lines on a legacy PCI bus), each interrupt can signal a different condition (so your interrupt service routine doesn't have to read an interrupt condition register), you can route each condition on a device to a different CPU, and the interrupt is a bus-master write that flushes all of the previous bus-master writes by the same device (according to the PCI ordering rules), so a driver doesn't have to poll a device register to land buffered bus-master writes before examining descriptor rings and other DMA-able regions. Dave -- David Young dyo...@pobox.comUrbana, IL(217) 721-9981
re: RFC: add MSI/MSI-X support to NetBSD
let's not forget my favourite mis-feature of MSI/MSI-X: if you misconfigure the address, interrupts might cause main memory to be corrupted. i've seen this happen, and it was rather difficult to diagnose the real culprit.. i'm a little confused about bus_msi(9) -- pci_intr(9) is already an MD interface, so if it was extended or if we copied the pci_intr_map_msi() functions from elsewhere, it's still MD code we have to write. what does bus_msi(9) add? who would use it? .mrg.
