Funded project(s) to improve Linux emulation
The NetBSD core team is looking at finding volunteers or funded workers willing to help improve our linux emulation. We have an offer from Precedence Technologies to co-fund some of this developement, especially the Citrix sub-projects. At this point it is not clear how many of these projects will be funded, and how many takers would be involved. If you have knowledge and interest to help in some of this as a volunteer, please let us know. The current state of Linux emulation is greatly affected by incompatibilities in the futex emulation, so completition/merging of the thorpej-futex branch may be a prerequisite for some of the projects explained below. If you would like to work on some of this under a funding contract, please mail us a quote stating a timing estimate and if possible a fixed price, or an hourly rate and and maximum of hours you expect to need for this. The improvements we are looking for consist of multiple sub-projects, you can offer to work on single ones or a collection of items. If your work offered depends on some of the others being completed upfront, please state so (mostly it should be obvious, but just to make sure we topologicaly sort everything correctly). The tasks we are looking for takers are: - Implement missing timer sysctls and timerfd_* syscalls. For timerfd_* consider adding them natively and do a small compat shim for linux emul. Init public discussion about native timerfd_*. - Implement epoll(2) and friends for linux emulation. Consider a native implementation and small compat shim, and start a public discussion about it. - Improve stability (current code can easily panic the kernel) and add a test framework (might be pkgsrc based and create test binaries by cross compilation, does not need to use ATF). - Improve compat linux on arm - Improve linux audio support - Make current versions of the Citrix Workspace App runnable under linux emulation on NetBSD/i386. Precedence can provide test accounts on backend servers. - Extend support for the Citrix Workspace App to amd64. - Extend support for the Citrix Workspace App to some arm platform(s). - Add support for linux drm (DRI kernel support) for the version of DRI we support natively by the time this project runs. - Update our pkgsrc compat framework to newer libs if possible after the other improvements are done. Publish a list of other missing things if there are any blockers. Please contact us at c...@netbsd.org. Martin Husemann on behalf of the NetBSD core team.
Re: Funded project(s) to improve Linux emulation
Martin Husemann wrote: >The NetBSD core team is looking at finding volunteers or funded workers >willing to help improve our linux emulation. We have an offer from >Precedence Technologies to co-fund some of this developement, especially >the Citrix sub-projects. > >The tasks we are looking for takers are: > > - Improve compat linux on arm Does it work at all ? Linux on arm makes use of a page mapped at the top of the 32-bit address space containing some "helper" functions to do things like get thread local storage. I had a go at adding a pmap_helper() for Linux processes but didn't get it to work. > - Extend support for the Citrix Workspace App to some arm platform(s). Is this still arm or is there a requirement to be able to run an aarch64 application ? > - Add support for linux drm (DRI kernel support) for the version of DRI > we support natively by the time this project runs. Think we would want newer DRI kernel code to be able to use the Linux drivers for ARM GPUs, the Mali code makes use of the dri scheduler which isn't in our tree.
Re: Funded project(s) to improve Linux emulation
On Tue, Nov 10, 2020 at 06:36:06PM +0100, Martin Husemann wrote: > - Improve linux audio support Does anyone know what's needed yet?
Re: Funded project(s) to improve Linux emulation
> On Nov 10, 2020, at 9:36 AM, Martin Husemann wrote:
>
> The current state of Linux emulation is greatly affected by
> incompatibilities in the futex emulation, so completition/merging
> of the thorpej-futex branch may be a prerequisite for some of the
> projects explained below.
Let me provide some context here.
The thorpej-futex branch contains changes for two separate things, only one of
which is required for Linux emulation:
1- Changing from using a condvar internally to becoming its own synchronization
object primitive and using sleepqs directly. This is required in order to fix
a class of bugs related to thread selection on wakeup.
2- Implement a new set of operations that are NetBSD extensions to the futex
API, for use natively by the pthread library and possible other things as well.
(1) does indeed fix the bugs it was intended to fix (and there are test cases
for them), and it has been reported to improve the situation somewhat with
regard to Linux compatibility. However, there are some issues with the Linux
openjdk that lead to applications getting stuck. The bug isn't 100%
reproducible.
I have not yet written support for the PI ("priority-inheritance") futex
operations. Those should be fairly easy to implement, but will require changes
outside of the futex code ... in particular, we want to share the actual guts
of the priority-lending mechanism with the turnstile code used to back kernel's
mutex and rwlock implementations. Anyway, I think it's best to do as a
two-step approach, doing the PI bits later (and maybe as a lower priority
project; as far as I'm aware, priority-inheriting mutexes are not the default
in glibc, and applications have to opt-in).
(2) is currently disabled in the branch because of a weird crasher, but the
only extensions implemented are RW_WAIT and RW_HANDOFF designed for
implementing direct-handoff reader/writer locks in libpthread. I actually have
a bunch of changes to libpthread that sit on top of native futexes (for
pthread_mutex and pthread_cond), including support for process-shared
synchronization objects.
Anyway, if people are interested in helping carry this stuff across the finish
line, let me know, and I will be happy to provide guidance, etc.
-- thorpej
amd64 9.1, pre-wscons text colours?
NetBSD/amd64, 9.1. What controls the colours used for console text output before wscons takes over? When I build a kernel with WS_KERNEL_FG=WSCOL_GREEN and no WS_KERNEL_BG, I see green on black, both before and after the switch from text mode to text-in-graphics mode. WS_KERNEL_FG=WSCOL_BLACK and WS_KERNEL_BG=WSCOL_BLACK, and both are black-on-black. But WS_KERNEL_FG=WSCOL_LIGHT_BROWN (I wanted yellow, that looks like the closeset available approximation) and WS_KERNEL_BG=WSCOL_BLUE? Once wscons switches, everything is as I would expect - but, before that, it's white on black! So there's clearly something I don't understand going on. What? /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: amd64 9.1, pre-wscons text colours?
On Tue, Nov 10, 2020 at 06:39:53PM -0500, Mouse wrote: > NetBSD/amd64, 9.1. > > What controls the colours used for console text output before wscons > takes over? When I build a kernel with WS_KERNEL_FG=WSCOL_GREEN and no > WS_KERNEL_BG, I see green on black, both before and after the switch > from text mode to text-in-graphics mode. WS_KERNEL_FG=WSCOL_BLACK and > WS_KERNEL_BG=WSCOL_BLACK, and both are black-on-black. > > But WS_KERNEL_FG=WSCOL_LIGHT_BROWN (I wanted yellow, that looks like > the closeset available approximation) and WS_KERNEL_BG=WSCOL_BLUE? > Once wscons switches, everything is as I would expect - but, before > that, it's white on black! > > So there's clearly something I don't understand going on. What? That's all wscons; drmkms taking over isn't the same step as wscons taking over. But anyway: my guess is that you can't use the bright colors. I vaguely recall some issue with that long ago... -- David A. Holland dholl...@netbsd.org
Re: amd64 9.1, pre-wscons text colours?
>> What controls the colours used for console text output before wscons
>> takes over? [...]
>> So there's clearly something I don't understand going on. What?
> That's all wscons; drmkms taking over isn't the same step as wscons
> taking over.
Okay, that's _one_ thing I didn't understand :-)
> But anyway: my guess is that you can't use the bright colors. I
> vaguely recall some issue with that long ago...
I just trawled for WSCOL_ in the kernel source tree. The only thing I
see that could be responsible is this, in dev/rcons/rcons_subr.c:
/* Set ANSI colors */
void
rcons_setcolor(struct rconsole *rc, int fg, int bg)
{
int flg;
if (fg > WSCOL_WHITE || fg < 0)
return;
if (bg > WSCOL_WHITE || bg < 0)
return;
and the WSCOL_LIGHT_ colours are 8 higher than the non-LIGHT versions.
So if rcons is in use, this explains it. (I'm not sure it is, but the
symptoms match, and I don't, offhand, see anywhere else that could be
responsible. But I'm not at all certain I haven't missed something.)
Of course, the question now becomes _why_ that code is there.
According to cvsweb, it appeared in rev 1.3, on 1999-04-13, without any
indication what the purpose of it is; the commit message reads
| Many enchancements to rcons to support ANSI color and all attributes
| properly. All output now performed using a 'struct wsdisplay_emulops'.
(There are numerous other messages talking about "ANSI color", but I
_think_ it's actually ISO color; the SGR sequence is ANSI, but as I
understand it the colour-specifying arguments for it come from ISO
6429, not ANSI. I've never found an authoritative source to say
whether this is a misunderstanding or not)
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTMLmo...@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: amd64 9.1, pre-wscons text colours?
mo...@rodents-montreal.org (Mouse) writes: >But WS_KERNEL_FG=WSCOL_LIGHT_BROWN (I wanted yellow, that looks like >the closeset available approximation) and WS_KERNEL_BG=WSCOL_BLUE? >Once wscons switches, everything is as I would expect - but, before >that, it's white on black! >So there's clearly something I don't understand going on. What? VGA has only 8 colors and an intensity attribute. You cannot select the "light" ANSI colors. The VGA driver will fail attempts to allocate such color attributes, so you get white on black instead. if (__predict_false((unsigned int)fg >= sizeof(fgansitopc) || (unsigned int)bg >= sizeof(bgansitopc))) return (EINVAL); When DRM takes over, such hardware limitations no longer apply. The framebuffer code can handle more colors. -- -- Michael van Elst Internet: mlel...@serpens.de "A potential Snark may lurk in every tree."
Re: amd64 9.1, pre-wscons text colours?
On Wed, Nov 11, 2020 at 05:20:53AM -, Michael van Elst wrote: > >But WS_KERNEL_FG=WSCOL_LIGHT_BROWN (I wanted yellow, that looks like > >the closeset available approximation) and WS_KERNEL_BG=WSCOL_BLUE? > >Once wscons switches, everything is as I would expect - but, before > >that, it's white on black! > > >So there's clearly something I don't understand going on. What? > > > VGA has only 8 colors and an intensity attribute. You cannot select > the "light" ANSI colors. The VGA driver will fail attempts to > allocate such color attributes, so you get white on black instead. > > if (__predict_false((unsigned int)fg >= sizeof(fgansitopc) || > (unsigned int)bg >= sizeof(bgansitopc))) > return (EINVAL); > > > When DRM takes over, such hardware limitations no longer apply. > The framebuffer code can handle more colors. VGA hardware has 16 colors in text mode, eight each bright and dim. The "intensity attribute" is the upper of the four color selection bits and picks the bright colors. You can't use the bright colors as _backgrounds_ without poking a register somewhere: by default the top background color bit causes blinking instead. It's reasonable we might not have code to do that, just in case someone tries to boot on a machine with an EGA card or something that doesn't have the register. But if we can't use all 16 foreground colors, it's because of a driver problem; it's not a hardware limitation. -- David A. Holland dholl...@netbsd.org
