Hello Everyone, I am Roy, and this is my progress report on what I have been doing.
First, based on the remarks of my mentors, I had to add an eBPF cleanup program to clean dead OpenMP thread TIDs from the omp_threads_map when the corresponding threads exit. Then, with the guest_ivshmem driver provided by my mentor, I created a systemd service that builds and loads this kernel module in the guest on each boot of a VM. Since this driver also exposes the bpf_guest_ivshmem_g2h_write kfunc used to write whether a vCPU is running an OpenMP thread or not, I adjusted the omp_thread_reg BPF program to utilize it as such. I also modified the host_ivshmem driver provided by the mentors by adding a kfunc, bpf_host_ivshmem_g2h_read, to read the information provided by the guest eBPF programs via the IVSHMEM guest-to-host page. Given now that the guest is providing this information (i.e., whether a vCPU is running an OpenMP thread or not), I now utilize it on the host in the phantom_switch_handler BPF program to mark a vCPU as phantom only if it is running an OpenMP thread and the vCPU is not running on the host. I then worked on making the phantom average computed on the host available to the guest userspace OpenMP programs. That involved adding a character device for the guest_ivshmem driver that exposes the phantom average on the host-to-guest page to userspace programs on the guest via file operations such as read on the char device's device node. Modifying the GCC code base by adding a new environment variable GOMP_WAIT_POLICY that retains the behaviour of OMP_WAIT_POLICY but introduces a new value pvsched for paravirtualised barrier synchronisation. I added the parsing logic for this environment variable and modified the do_spin function to read this phantom average while an OpenMP thread is spinning at a barrier and, when phantoms are detected, immediately block an OpenMP thread. I added tests to verify the interaction between the modified libgomp and Phantom Tracker, which involved the use of several eBPF programs: 1. A uprobe on the libgomp waiting path to identify when an OpenMP thread enters the waiting path. 2. A kprobe/kretprobe on the read operation of the guest_ivshmem character device to observe phantom-average reads. 3. A sys_enter_futex tracepoint to observe futex syscalls made by OpenMP threads. 4. A sched_switch tracepoint to observe when OpenMP threads are scheduled out. Following the latest discussions with my mentors, I am currently working on: - Replacing the read operation of the guest_ivshmem character device with an mmap operation so that the IVSHMEM pages can be directly mapped into guest userspace. This should avoid performing a read/pread operation every time the phantom average is accessed. - Adapting the libgomp implementation and test to the new mmap-based interface. - Exposing the relevant libgomp symbols in a way that provides a reliable interface for attaching uprobe eBPF programs. - Adding GCC tests for the new GOMP_WAIT_POLICY functionality. Best Regards Nchang Roy
