On 5/26/2025 2:06 AM, Manuel Bouyer wrote: > On Sun, May 25, 2025 at 11:40:12PM -0400, Chuck Zmudzinski wrote: >> On 5/25/2025 11:25 AM, Manuel Bouyer wrote: >> > On Sun, May 25, 2025 at 08:54:37AM -0400, Chuck Zmudzinski wrote: >> >> No difference with the larger conring_size, unfortunately. >> > >> > Can you try a -current kernel, or a very recent kernel from the netbsd-10 >> > branch ? There have been fixes in the last few days that could help with >> > very recent CPUs >> > >> >> No change with the latest -current or -10 from daily builds. Still need to >> disable >> smt in Xen to enumerate all the vcpus. > > Well, I'm out of idea then, sorry. The next step would be to use the Xen > debug tools to see where the hang occurs >
I finally successfully booted NetBSD 10.1 PV dom0 on the Intel i5-14500 CPU! And it works using the latest pkgsrc version of Xen (4.18.5_20250521nb0). It also worked with the Fedora version of Xen hypervisor 4.18.4, and with that version of Xen, the cet=no-ibt setting was not needed. Thanks, Manuel, for your help! It is definitely not production ready, but I got it to work with the following tweaks and hacks. boot command used: menu=Boot normally with Xen:dev hd2d:;load /netbsd-XEN3_DOM0.gz -c console=xencons bootdev=wd1;multiboot /xen.gz dom0_mem=2G dom0_max_vcpus=4 com2=9600,8n1,0x40c0,16,1:0.0 console=com2 cet=no-ibt pv-l1tf=false With the setting of dom0_max_vcpus=4 for Xen, I got past a failed assertion gsi < NR_EVENT_CHANNELS from NetBSD dom0 that caused a panic in NetBSD dom0 without this setting. With the setting cet=no-ibt for Xen I got past a control-flow protection fault crash in Xen. This setting was not needed if I used version 4.18.4 of the Xen hypervisor from Fedora 40. I also needed to pass -c to the NetBSD dom0 kernel so I could disable com* interactively using userconf at boot time. Without doing this, the NetBSD dom0 panics when using the serial console for Xen. I could not get the kernel to invoke userconf to disable com* by any setting in boot.cfg; it was necessary to pass -c and disable com* interactively at boot time. I also needed to interactively set the root device because no bootdev setting in boot.cfg allowed the NetBSD dom0 kernel to correctly detect the root device. Here are two snippets from output of dmesg from dom0 illustrating how I got it to work: 1. As noted above, I passed -c to the NetBSD dom0 kernel to disable the com ports interactively at boot time using userconf (otherwise it crashes when NetBSD dom0 tries to attach the com port Xen is using): [ 1.000000] userconf: configure system autoconfiguration: [ 1.000000] uc> disable com* [ 1.000000] [ 132.000000] com* disabled [ 1.000000] [ 133.000000] com* disabled [ 1.000000] [ 134.000000] com* disabled [ 1.000000] [ 135.000000] com* disabled [ 1.000000] [ 136.000000] com* disabled [ 1.000000] [ 137.000000] com* disabled [ 1.000000] uc> exit [ 1.000000] Continuing... 2. I tried passing the bootdev to the NetBSD kernel as wd1, dk12, and NAME=<UUID> but it never worked. However, I was able to interactively set it at boot time: [ 5.159642] boot device: wd1 [ 5.159642] root on wd1a dumps on wd1b [ 5.159642] vfs_mountroot: can't open root device [ 5.159642] cannot mount root, error = 16 [ 5.159642] root device (default wd1a): dk12 [ 87.979641] dump device: dk11 [ 91.009641] file system (default generic): [ 92.249641] root on dk12 dumps on dk11 [ 92.259641] root file system type: ffs [ 92.259641] kern.module.path=/stand/amd64/10.1/modules [ 92.264648] init path (default /sbin/init): [ 94.929641] init: trying /sbin/init [ 95.449641] Your machine does not initialize mem_clusters; sparse_dumps disabled [ 105.499640] wsdisplay0: screen 1 added (default, vt100 emulation) [ 105.499640] wsdisplay0: screen 2 added (default, vt100 emulation) [ 105.509641] wsdisplay0: screen 3 added (default, vt100 emulation) [ 105.509641] wsdisplay0: screen 4 added (default, vt100 emulation) [ 105.759641] balloon0 at xenbus0 id 0: Xen Balloon driver [ 105.759641] balloon0: current reservation: 2097152 KiB Output of 'xl info' and 'xl list': netbsd# xl info host : netbsd release : 10.1 version : NetBSD 10.1 (XEN3_DOM0) #0: Mon Dec 16 13:08:11 UTC 2024 mkre...@mkrepro.netbsd.org:/usr/src/sys/arch/xen/compile/XEN3_DOM0 machine : amd64 nr_cpus : 20 max_cpu_id : 19 nr_nodes : 1 cores_per_socket : 10 threads_per_core : 2 cpu_mhz : 2611.200 hw_caps : bfebfbff:77faf3ff:2c100800:00000121:0000000f:239c27eb:9840078c:00000100 virt_caps : pv hvm hvm_directio pv_directio hap shadow iommu_hap_pt_share vmtrace gnttab-v1 gnttab-v2 total_memory : 32519 free_memory : 30139 sharing_freed_memory : 0 sharing_used_memory : 0 outstanding_claims : 0 free_cpus : 0 xen_major : 4 xen_minor : 18 xen_extra : .5_20250521nb0 xen_version : 4.18.5_20250521nb0 xen_caps : xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit2 xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : xen_commandline : dom0_mem=2G dom0_max_vcpus=4 com2=9600,8n1,0x40c0,16,1:0.0 console=com2 cet=no-ibt pv-l1tf=false cc_compiler : gcc (nb3 20231008) 10.5.0 cc_compile_by : chuckz cc_compile_domain : cc_compile_date : Fri May 23 18:59:50 EDT 2025 build_id : 86afac8b52e2d124001208bb6261d17088a2f26f xend_config_format : 4 netbsd# xl list Name ID Mem VCPUs State Time(s) Domain-0 0 2048 4 r----- 142.2 netbsd# It is not a user friendly setup, but it does work! Thanks again, Chuck Zmudzinski
