Re: RFC: device tree ports, connectors and panels
On Sun, Apr 01, 2018 at 08:19:36AM -0300, Jared McNeill wrote: > LGTM. > > Any way I could convince you to ditch the _t typedefs? Only for consistency, > because the rest of the FDT code (for better or worse) doesn't use this > style. OK I'll remove them. typedefs makes lines a bit smaller, but that's not a big deal. > > I'd like to see this evolve at some point to work closer with the DRM2 APIs, > as it seems that there may be quite a bit of overlap. Sure. A working implementation of the fdt endpoints should help with this. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: Fixing excessive shootdowns during FFS read (kern/53124) on x86 - emap, direct map
2018-03-31 13:42 GMT+02:00 Jaromír Doleček : > 2018-03-25 17:27 GMT+02:00 Joerg Sonnenberger : >> Yeah, that's what ephemeral mappings where supposed to be for. The other >> question is whether we can't just use the direct map for this on amd64 >> and similar platforms? > > Right, we could/should use emap. I haven't realized emap is actually already > implemented. It's currently used for pipe for the loan/"direct" write. > > I don't know anything about emap thought. Are there any known issues, > do you reckon it's ready to be used for general I/O handling? Okay, so I've hacked to gether a patch to switch uvm_bio.c to ephemeral mapping: http://www.netbsd.org/~jdolecek/uvm_bio_emap.diff Seems to boot, no idea what else it will break. Looking at the state of usage though, the emap is only used for disabled code path for sys_pipe and nowhere else. That code had several on-and-off passes for being enabled in 2009, and no further use since then. Doesn't give too much confidence. The only port actually having optimization for emap is x86. Since amd64 is also the only one supporting direct map, we are really at liberty to pick either one. I'd lean towards direct map, since that doesn't require adding/removing any mapping in pmap_kernel() at all. From looking on the code, I gather direct map is quite easy to implement for other archs like sparc64. I'd say significantly easier than adding the necessary emap hooks into MD pmaps. Jaromir
Potential new syscall
I'm writing a userland emulator - basically, a hardware emulator, except that it just runs userland; instructions that would trap to privileged mode are instead handled by the emulator. While doing this, I ran into a problem: vfork. Most syscalls are fairly straightforward, but vfork is a problem. Most of the problems I've dealt with easily enough, but there is one that I feel a need for a new syscall for. I had a look at the modern vfork(2) manpage via the web interface, and the SEE ALSO section gives me no reason to think such a thing exists even in modern NetBSD. (I could just implement vfork as fork, but I'd prefer to be a faithful emulation if I can.) Basically, what I want is a syscall that a vfork()ed child can call to have the unsharing effects of execve(2) or _exit(2) (return the vmspace to the parent and let the it continue), while the child carries on with a clone of the vmspace without actually doing an exec or exit. This is because the emulator does not exec in the hosting-OS sense when the emulated process execs; I have found no other way to get the vfork semantics right without forking-and-exiting, and that gets process parenting wrong. (This would be fixable by adding a manager process, with everything for which parenting is relevant going via it, or by having a single emulator process timeshare among all simulated processes. A new syscall seems cleaner to me, especially as it fills a gap in the OS semantics.) It looks to me as though there is uvm support for the concept, in the form of vmspace_unshare(), in the version I'm working with; a little searching makes it appear it's been diked out of uvm_map.c more recently. The syscall wrapper around it is only a few lines, basically just uvmspace_unshare() plus, if PPWAIT is set, kicking the parent. I offer for consideration the thought that something of the sort might be worth adding. I have code written, but I don't yet know whether what I have works right - a test build is running as I type this - and it's for a version years behind -current. Thoughts? (Not restricted to just the above details; thoughts on the general idea would also be interesting to me.) /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: Potential new syscall
On 03.04.2018 01:29, Mouse wrote: > > Thoughts? (Not restricted to just the above details; thoughts on the > general idea would also be interesting to me.) > It might not be a satisfying answer... but go for fork(). I recommend this option - no kernel changes involved... unless someone wants good performance when executing memcached. fork() tends to replace vfork() in similar software. And while there, calling other functions than exec() after vfork() is less portable than vfork(). TSan notes that this is/was Undefined Behavior and workaround issues in openjdk (which closes file descriptors) with wrapping fork(). signature.asc Description: OpenPGP digital signature
Re: Potential new syscall
On 2018-04-02 07:29 PM, Mouse wrote: Basically, what I want is a syscall that a vfork()ed child can call to have the unsharing effects of execve(2) or _exit(2) (return the vmspace to the parent and let the it continue), while the child carries on with a clone of the vmspace without actually doing an exec or exit. This is That sounds suspiciously like Linux's unshare(2) call, with the CLONE_VM option. Eric
Re: Potential new syscall
Date:Mon, 2 Apr 2018 19:29:25 -0400 (EDT) From:Mouse Message-ID: <201804022329.taa19...@stone.rodents-montreal.org> | Basically, what I want is a syscall that a vfork()ed child can call to | have the unsharing effects of execve(2) or _exit(2) I have considered (and think I mentioned on some list some time ago) the same capability to improve sh performance. But as I don't want to think about pure performance changes until I am happy that the functionality is as correct as I can make it, pondering its usefulness is as far as I went so far. Having the mechanism available for testing (even if it was not committed to the standard NetBSD sources (yet?)) would be a real help, when it comes time to evaluate whether that method, or converting sh to use posix_spawn (an alternative that was suggested, but which I kind of dread - considering the amount of work that would be involved) would produce the best results. Kamil - "just use fork" is a very common response, but no matter how fork gets implemented, vfork() when used correctly always performs better by huge margins. You are of course correct that there is a very limited set of functions possible in a vfork()'d child - which is why the function proposed by Mouse would be useful - currently sh limits its use of vfork() to the times it is very unlikely any of those will be be needed, and uses fork() other times (costing performance if it was too conservative) - and when the vfork() is attempted, and it turns out something needs doing which won't work, the vfork()'d child (and whatever work it had already done) is simply abandoned and it is all done again after a fork() - also costing performance. The ability to just convert a vfork() into a fork() would avoid the waste in the latter case, and also allow far more agressive use of vfork() to help avoid the former losses. kre
Re: Fixing excessive shootdowns during FFS read (kern/53124) on x86 - emap, direct map
On Mon, Apr 02, 2018 at 09:28:43PM +0200, Jaromír Dole?ek wrote: > The only port actually having optimization for emap is x86. Since amd64 > is also the only one supporting direct map, we are really at liberty to pick > either one. I am not sure what you mean here - AFIK alpha and mips64 also have a direct map for all available ram set up (and some arm too, plus I guess the new aarch64 code as well). I have not seen arguments why to do it on sparc64 yet, as we can easily access all memory w/o any mapping, and a direct map has security implications. > I'd lean towards direct map, since that doesn't require > adding/removing any mapping in pmap_kernel() at all. From looking on the code, > I gather direct map is quite easy to implement for other archs like > sparc64. I'd say > significantly easier than adding the necessary emap hooks into MD pmaps. What would that hooks be? Are they documented? Martin
Re: Fixing excessive shootdowns during FFS read (kern/53124) on x86 - emap, direct map
Le 02/04/2018 à 21:28, Jaromír Doleček a écrit : 2018-03-31 13:42 GMT+02:00 Jaromír Doleček : 2018-03-25 17:27 GMT+02:00 Joerg Sonnenberger : Yeah, that's what ephemeral mappings where supposed to be for. The other question is whether we can't just use the direct map for this on amd64 and similar platforms? Right, we could/should use emap. I haven't realized emap is actually already implemented. It's currently used for pipe for the loan/"direct" write. I don't know anything about emap thought. Are there any known issues, do you reckon it's ready to be used for general I/O handling? Okay, so I've hacked to gether a patch to switch uvm_bio.c to ephemeral mapping: http://www.netbsd.org/~jdolecek/uvm_bio_emap.diff - pmap_emap_enter(va, pa, VM_PROT_READ); + pmap_emap_enter(va, pa, VM_PROT_READ | VM_PROT_WRITE); Mmh no, sys_pipe wanted it read-only, we shouldn't make it writable by default. Adding a prot argument to uvm_emap_enter would be better. Looking at the state of usage though, the emap is only used for disabled code path for sys_pipe and nowhere else. That code had several on-and-off passes for being enabled in 2009, and no further use since then. Doesn't give too much confidence. The only port actually having optimization for emap is x86. Since amd64 is also the only one supporting direct map, we are really at liberty to pick either one. I'd lean towards direct map, since that doesn't require adding/removing any mapping in pmap_kernel() at all. From looking on the code, I gather direct map is quite easy to implement for other archs like sparc64. I'd say significantly easier than adding the necessary emap hooks into MD pmaps. There is a good number of architectures where implementing a direct map is not possible, because of KVA consumption. With a direct map we consume more than once the physical space. If you have 4GB of ram, the direct map will consume 4GB of KVA; and in addition pmap_kernel will consume some KVA too. i386 for example has only 4GB of ram and 4GB of KVA, so we'll just never add a direct map there. Direct maps are good when the architecture has much, much more KVA than it has physical space. I saw some low-KVA architectures have a "partial direct map", where only a (tiny) area of the physical space is direct-mapped. There, we would have to adapt uvm_bio to use pmap_kernel instead, which seems ugly. As opposed to that, emaps can be implemented everywhere with no constraint on the arch. I think they are better than the direct map for uvm_bio.
