Hi Greg,
On Sun, Nov 25, 2018 at 10:13:52AM -0800, Greg Steuck wrote:
> Hi Anton,
>
> I tried to boot a kernel with kcov based on GENERIC.MP and the machine
> reboots without a peep immediately after
>
> vmm0 at mainbus0: VMX (using slow L1TF mitigation)
>
> Switching off either of kcov or MP results in normally working kernels. I'm
> attaching two concatenated dmesgs. The effect is reproducible on real HW
> and on GCE VM. Broken config is just:
> $ cat /sys/arch/amd64/conf/SYZKALLER
> include "arch/amd64/conf/GENERIC.MP"
> pseudo-device kcov 1
>
> Disabling either vmm or kcov in broken kernel UKC doesn't prevent crashes.
Known limitation, I haven't spent much time on making kcov MP-safe.
Especially since it's primarily used inside a VM through vmm which
currently is limited to a single CPU.
However, I did some investigation before and concluded that the problem
resides in the trace routine which is called from
cpu_boot_secondary_processors() before the secondary CPU is accessible
through curcpu(). I came up with a hackish solution to this problem (see
diff below) that got rejected; kettenis@ mentioned that we instead
should set MSR_GSBASE earlier in cpu_hatch() but I never managed to get
the right people involved with knowledge in this area. I might take a
look myself.
In the meantime, you could give the diff a try. It might be the case
that more functions are not eligible for tracing. OpenBSD as no method
of turning of tracing for a given source file like Linux does. This
might become necessary since I fear many more functions will not cope
with tracing.
Index: dev/kcov.c
===================================================================
RCS file: /cvs/src/sys/dev/kcov.c,v
retrieving revision 1.4
diff -u -p -r1.4 kcov.c
--- dev/kcov.c 27 Aug 2018 15:57:39 -0000 1.4
+++ dev/kcov.c 8 Sep 2018 21:51:20 -0000
@@ -49,6 +49,7 @@ struct kcov_dev {
};
void kcovattach(int);
+void kcov_attachhook(struct device *);
int kd_alloc(struct kcov_dev *, unsigned long);
void kd_free(struct kcov_dev *);
@@ -57,6 +58,7 @@ struct kcov_dev *kd_lookup(int);
static inline int inintr(void);
TAILQ_HEAD(, kcov_dev) kd_list = TAILQ_HEAD_INITIALIZER(kd_list);
+int kcov_attached = 0;
#ifdef KCOV_DEBUG
int kcov_debug = 1;
@@ -76,12 +78,11 @@ int kcov_debug = 1;
void
__sanitizer_cov_trace_pc(void)
{
- extern int cold;
struct kcov_dev *kd;
uint64_t idx;
- /* Do not trace during boot. */
- if (cold)
+ /* Do not trace before the root file system is mounted. */
+ if (!kcov_attached)
return;
/* Do not trace in interrupts to prevent noisy coverage. */
@@ -102,6 +103,13 @@ __sanitizer_cov_trace_pc(void)
void
kcovattach(int count)
{
+ config_mountroot(NULL, kcov_attachhook);
+}
+
+void
+kcov_attachhook(struct device *dev)
+{
+ kcov_attached = 1;
}
int
