Hello, I have been working through reimplementing the mechanisms involved with launching programs in L4Re, managing to initiate programs apparently successfully under QEMU emulating amd64 and x86, but I have been less successful getting them to work under QEMU emulating a MIPS system (MIPS Malta).
My arrangement of threads within tasks should be fairly similar to what L4Re normally employs to host and run programs, and I wrote it up here: "Gradual Explorations of Filesystems, Paging and L4Re" https://blogs.fsfe.org/pboddie/?p=2540 The pictures at the end summarise what has been done. I have a region mapper running in a thread in the same task as the program to be launched. The region mapper handles page faults and interacts with dataspaces on the program's behalf. Faults caused by the region mapper are handled by the initiating task. What I seem to have avoided previously but now experience is that after the region mapper has been started, it is able to service page faults, but after a certain point in the execution of the program of interest, the region mapper itself experiences an exception in the __pthread_cleanup_push_defer function within uClib's pthreads implementation, specifically at this line: buffer->__canceltype = THREAD_GETMEM(self, p_canceltype); Digging into this, I found that self is apparently zero. Since self is obtained from a call to thread_self which in turn obtains the user[0] member of the thread control register structure (providing the thread descriptor address), I can only conclude that either this member has not been initialised or that it has been overwritten by the other thread as it started up. Investigating uClibc startup, I notice that pthread initialisation does indeed involve a traversal of UTCB areas, zeroing user[0] in the supposedly free UTCB blocks. And with the region mapper having failed, the main program will also fail fairly quickly, with an exception typically occurring in the __pthread_initialize_minimal function, coincidentally just after this zeroing has occurred, just before a call to __libc_setup_tls. I have been looking for hints as to how this work is done in L4Re normally. Given that the region mapper and the program being run are normal programs that are executed via their conventional entry points, this resulting in the initialisation of the C library and threading for each payload, it seems that some measures must be in place to prevent this initialisation from causing conflicts in the UTCB area. For instance, I see that in the file... pkg/l4re-core/l4re_kernel/server/src/main.cc ... the __libc_alloc_initial_tls function is redefined to presumably override the weakly referenced version in uClibc, but if this were invoked it would probably cause uClibc to fail during initialisation, at least unless the zero page is going to be used (but it would need allocation somewhere else), so maybe it doesn't override anything after all. In any case, this function would only change some aspects of how __libc_setup_tls would work, so maybe it is a distraction. I did wonder if there was something specific about MIPS that would cause this failure, investigating the user-local register (ULR) used to hold UTCB-related information, but getting the UTCB address via the ULR would appear to work: otherwise, the region mapper would probably not start up. I also wondered about program initialisation issues, since I did experience problems on actual MIPS hardware related to various registers not being set up correctly, but the generated code seems fairly comprehensive in this regard these days (and maybe it was all along but I encountered a confounding phenomenon). Does anyone have any familiarity with how this actually works and might be able to indicate what I am doing wrong? As always, thanks for any help that can be offered! Paul _______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de https://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
