Hello all, I hope someone might be able to help me.
I'm developing a hypervisor myself, the hv_dir and the kernel_file from the
grub.cfg below are inside an ntfs partition. It's an UEFI platform, with
grub2.efi file inside a fat partition.
The grub version is 2.12 downloaded from the ftp
(https://ftp.gnu.org/gnu/grub/grub-2.12.tar.xz). Applied the patch with
extra_deps.lst. I built it with the commands in the build section.
The bios of this device I'm executing is handcrafted by myself also, (coreboot
+ edk2 - mrchromebox's uefipayload 25_02) - it's an x86 intel pentium platform.
The issue only occurs if I have the network stack + NIC driver on edk2, Also it
only occurs if no display port or a usb mass storage is connected.
This means if I have display port connected, or a USB stick the device boots
just fine (either one of them, it doesn't have to be both at the same time).
A bit more weird is that, if I delete grub.cfg file from the ntfs partition and
I recopy it ther, the exact same one it starts working again and issue does not
occur anymore.
Editing it on a non related line, like the "echo partition not found" makes it
work again too.
Also, if I build with disabled optimitaztion (CFLAGS=-O0) problem seems to go
away too.
Also, when executing the commands manually from grub command line or if I press
to edit the option then ctrl+X
I narrowed it down a bit using grub_printf to get some debug in parts like the
multiboot2 module to check the differences between good and bad boot, this is
in the Logs section.
It seems like, there are four calls for grub_efi_allocate_pages_real for a good
boot. However, when it throws the X64 Exception, it only has two calls of this
function. Then with
a system debugger, I discovered that this calls are being tried to be done
after the exit boot services was called by the multiboot2 module, which of
course will lead to an X64 Exception,
since allocate grub_efi_allocate_pages_real is trying to use boot services to
do the allocation and this functions were freed by the UEFI bios at this point.
All the 4 allocations are being done
by the line "module2 ($root)$kernel_dir/$kernel_file :p pagedir_mem_reg_id=0
vmdid=1 smp_cores=1" and the boot command issues the
"grub_multiboot2_make_mbi->Entry". So by some reason, when booting
the allocations are kinda being cut in the middle.
Also with system debugger. I got the stack trace from the allocate_pages_real
calls and it's located in the stack trace section. Unfortunately, I couldn't
use system debugger with non optimized build,
as the issue just goes away.
I'm not well versed in bootloader, nor the intrinsics of how grub works.
Therefore, any pointers on that would be very much appreciated.
Thanks in advance.
-------------- Build commands section --------------
chmod a+x autogen.sh
./autogen.sh
./configure --with-bootdir=/Boot --with-grubdir=GRUB2
--prefix=/home/user/grub-issue/Grub2.12-build/GRUB2/result_uefi --disable-nls
--disable-werror --with-platform=efi --target=x86_64
make install
./grub-mkimage -o /home/user/grub-issue/Grub2.12-build/GRUB2/grub2.efi -O
x86_64-efi -p /boot/grub2 -c /home/user/grub-issue/Grub2.12-build/grub_init.cfg
-d /home/user/grub-issue/Grub2.12-build/grub-2.12/grub-core normal video search
test configfile read echo fat ntfs part_gpt ext2
-------------- Logs section --------------
====== bad boot ======
[multiboot_elfxx.c - 255]-> calling calloc
grub_multiboot2_add_module->Entry
grub_multiboot2_add_module->Entry
grub_multiboot2_add_module->Entry
grub_multiboot2_add_module->Entry
grub_efi_allocate_pages_real_1-> Entry
grub_efi_allocate_pages_real_1-> Entry
grub_multiboot2_make_mbi->Entry
error: no suitable video mode found.
multiboot2 -> Calling grub_efi_finish_boot_services
grub_efi_finish_boot_services-> Calling ExitBootServices
!!!! X64 Exception Type - 0D(#GP - General Protection) CPU Apic ID - 00000000
!!!!
ExceptionData - 0000000000000000
RIP - 0000000071EFD431, CS - 0000000000000038, RFLAGS - 0000000000210006
RAX - 0000000075950018, RCX - 0000000000000001, RDX - 0000000000000002
RBX - 0000000000000006, RSP - 0000000004FD9EA0, RBP - 0000000004FD9F40
RSI - 0000000000000000, RDI - 000000000000022A
R8 - 0000000000000006, R9 - 0000000004FD9ED8, R10 - 000000006FDFB57C
R11 - 000000006FDFB57C, R12 - 0000000071F011AB, R13 - 0000000071F13EE8
R14 - 0000000000000001, R15 - 00000000FFFFFFFF
DS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030
GS - 0000000000000030, SS - 0000000000000030
CR0 - 0000000080010013, CR2 - 0000000000000000, CR3 - 0000000004A01000
CR4 - 0000000000000668, CR8 - 0000000000000000
DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000075A3A680
DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 000000000D0F0400
GDTR - 000000007594A000 0000000000000047, LDTR - 0000000000000000
IDTR - 00000000753DF018 0000000000000FFF, TR - 0000000000000000
FXSAVE_STATE - 0000000004FD9B00
!!!! Find image based on IP(0x71EFD431) (No PDB) (ImageBase=0000000071EF7000,
EntryPoint=0000000071EF8000) !!!!
====== good boot ======
grub_multiboot2_add_module->Entry
grub_multiboot2_add_module->Entry
grub_multiboot2_add_module->Entry
grub_multiboot2_add_module->Entry
grub_efi_allocate_pages_real_1-> Entry
grub_efi_allocate_pages_real_1-> Entry
grub_multiboot2_make_mbi->Entry
grub_efi_allocate_pages_real_1-> Entry
grub_efi_allocate_pages_real_1-> Entry
error: no suitable video mode found.
multiboot2 -> Calling grub_efi_finish_boot_services
grub_efi_finish_boot_services-> Calling ExitBootServices
-------------- Configuration file section --------------
grub_init.cfg:
echo GRUB_INIT.CFG
search -f /Boot/GRUB2/grub.cfg --set=root
set prefix=($root)/Boot/GRUB2
grub.cfg:
echo GRUB.CFG
if [ $grub_platform= pc ]; then
search -f /Boot/GRUB2/grub.cfg --set=root
fi
set prefix=($root)/Boot/GRUB2/modules
set enviro=($root)/Boot/GRUB2/env
insmod chain
insmod sleep
insmod multiboot
insmod multiboot2
insmod test
insmod all_video
set default=0
export default
set lang=en
export lang
set pager=1
export pager
set windiagdelay=10
export windiagdelay
search -f /Boot/GRUB2/grub.cfg --set=root
set prefix=($root)/Boot/GRUB2
function getpartition {
set pager=0
set searchtype=$1
set partsearch=$2
if [ $searchtype= label ] ; then
search.fs_label $partsearch rootaddress
else
search.file $partsearch rootaddress
fi
if [ $? = 0 ] ; then
set root=($rootaddress)
return 0
else
echo
echo
echo *** Grub did not find a boot partition with $searchtype $partsearch
***
echo
set pager=1
sleep -v -i 10
return 1
fi
}
timeout=5
default=1
if [ $grub_platform = efi ] ; then
set winfile=/efi/Microsoft/Boot/bootmgfw.efi
else
set winfile=+16
fi
menuentry 'Windows' --class windows --class icon-windows {
getpartition file $winfile
if [ $? = 0 ] ; then
chainloader ($root)$winfile
else
echo Partition not found
fi
}
menuentry 'Windows and My Kernel' --class matches --class icon-my_kernel
{
multiboot2 ($root)$hv_dir/HV_1st_stage.elf
if [ $? = 0 ] ; then
set hv_dir=/Boot/HV_DIR
set boot_partition_file=hv_file.000
set kernel_dir=/Boot/K_DIR
set kernel_file=My_kernel.elf
module2 ($root)/$kernel_dir/HV.cfg
module2 ($root)$hv_dir/HV_2nd_stage.elf
module2 ($root)$hv_dir/OVMF_modified.fd
getpartition file $hv_dir/$boot_partition_file
module2 ($root)$kernel_dir/$kernel_file :p pagedir_mem_reg_id=0 vmdid=1
smp_cores=1
boot
else
echo Hypervisor not found!
fi
}
-------------- Grub Kernel Map file (grub2.efi) --------------
0000000000000000 b cd_devices
0000000000000000 d grub_efidisk_dev
0000000000000000 d grub_module_license
0000000000000000 r apple.3930
0000000000000000 T _start
0000000000000000 T start
0000000000000008 b hd_devices
0000000000000010 b fd_devices
0000000000000010 d grub_module_license
0000000000000017 T grub_memalign_dma32
0000000000000018 B grub_command_list
0000000000000020 B grub_net_open
0000000000000040 B grub_disk_dev_list
0000000000000040 d block_io_guid
0000000000000048 B grub_disk_write_weak
0000000000000050 B grub_disk_firmware_is_tainted
0000000000000050 d acpi20_guid.3605
0000000000000058 B grub_disk_firmware_fini
0000000000000060 B grub_disk_cache_table
0000000000000060 d acpi_guid.3601
0000000000000070 d device_path_guid
0000000000000070 T grub_dma_free
0000000000000080 d loaded_image_guid
0000000000000080 T grub_dma_get_virt
0000000000000088 T grub_dma_get_phys
0000000000000090 d console_control_guid
0000000000000090 T grub_pci_make_address
00000000000000a0 d efi_variable_guid.3779
00000000000000bd T grub_pci_iterate
00000000000000c0 D shim_lock_verifier
0000000000000100 d shim_lock_guid
0000000000000110 D grub_current_context
0000000000000120 D grub_fs_blocklist
0000000000000180 D lockdown_verifier
000000000000018b T grub_pci_find_capability
00000000000001c0 D grub_gettext
00000000000001e0 d state_transitions
000000000000022f t get_device
0000000000000253 t grub_efidisk_close
000000000000028c t free_devices
00000000000002b6 T grub_efidisk_fini
0000000000000320 D grub_xputs
0000000000000328 D grub_term_highlight_color
0000000000000329 D grub_term_normal_color
0000000000000333 t grub_efidisk_open
0000000000000340 D verified_fs
00000000000003a0 d text_input_ex_guid.3892
00000000000003c0 d grub_console_term_output
0000000000000460 d grub_console_term_input
0000000000000480 R efi_codes
00000000000004a0 d text_colorstate
00000000000004a4 d cursor_mode
00000000000004a8 d text_mode
00000000000004c0 d tab.5008
000000000000057e t make_devices
00000000000006da t find_parent_device
000000000000076b t get_diskname_from_path_real
00000000000007e6 t get_diskname_from_path
000000000000086f t grub_efidisk_iterate
0000000000000a4f t grub_efidisk_get_device_name_iter
0000000000000ab8 t add_device
0000000000000b70 t grub_efidisk_readwrite
0000000000000c77 t grub_efidisk_read
0000000000000d69 t grub_efidisk_write
0000000000000e5e T grub_efidisk_init
0000000000001063 T grub_efidisk_get_device_handle
00000000000011b5 T grub_efidisk_get_device_name
0000000000001396 t grub_acpi_rsdt_find_table
0000000000001428 T grub_byte_checksum
000000000000143e T grub_acpi_find_table
0000000000001543 T grub_acpi_find_fadt
000000000000155d T grub_buffer_new
00000000000015cd T grub_buffer_free
00000000000015fc T grub_buffer_ensure_space
000000000000167f T grub_buffer_take_data
00000000000016a6 T grub_buffer_reset
00000000000016bb T grub_buffer_advance_read_pos
00000000000016fb t grub_cmd_lockdown
0000000000001720 T grub_register_command_prio
0000000000001820 T grub_register_command_lockdown
0000000000001878 T grub_unregister_command
00000000000018ba T memcpy
00000000000018ca T memmove
00000000000018da T memcmp
00000000000018ea T memset
00000000000018fa t grub_mini_print_files
000000000000193c t grub_mini_print_devices
0000000000001963 t grub_core_cmd_ls
0000000000001b03 t grub_core_cmd_insmod
0000000000001b6f t grub_core_cmd_set
0000000000001c39 t grub_core_cmd_unset
0000000000001c75 T grub_register_core_commands
0000000000001d51 t iterate_partition
0000000000001e03 T grub_device_open
0000000000001ef0 T grub_device_close
0000000000001f4d t iterate_disk
0000000000002049 T grub_device_iterate
00000000000020c1 t grub_disk_cache_store
0000000000002193 t grub_disk_cache_unlock
00000000000021ee t grub_disk_cache_fetch
000000000000224d t grub_disk_read_small
0000000000002507 T grub_disk_cache_invalidate_all
000000000000255b T grub_disk_dev_register
0000000000002574 T grub_disk_dev_unregister
00000000000025a1 T grub_disk_close
0000000000002655 T grub_disk_open
000000000000292d T grub_disk_read
0000000000002c9e T grub_disk_native_sectors
0000000000002cc4 t grub_dl_get
0000000000002d11 t grub_dl_find_section
0000000000002d95 T grub_dl_add
0000000000002de1 T grub_dl_register_symbol
0000000000002eb0 T grub_dl_ref
0000000000002efa T grub_dl_unref
0000000000002f44 T grub_dl_ref_count
0000000000002f53 T grub_dl_unload
0000000000003062 T grub_dl_load_core_noinit
0000000000003917 T grub_dl_load_core
0000000000003961 T grub_dl_load_file
0000000000003a2d T grub_dl_load
0000000000003b35 T grub_machine_acpi_get_rsdpv1
0000000000003b4f T grub_machine_acpi_get_rsdpv2
0000000000003b69 t grub_cmd_gdbinfo
0000000000003bab T grub_efi_register_debug_commands
0000000000003bde t grub_utf16_to_utf8
0000000000003ced t dump_vendor_path
0000000000003d76 T grub_efi_locate_protocol
0000000000003db4 T grub_efi_locate_handle
0000000000003eb6 T grub_efi_open_protocol
0000000000003f0b T grub_efi_close_protocol
0000000000003f43 T grub_efi_set_text_mode
0000000000003fca T grub_efi_stall
0000000000003ff3 T grub_efi_get_loaded_image
0000000000004012 T grub_reboot
0000000000004050 T grub_exit
000000000000409c T grub_efi_set_virtual_address_map
00000000000040f0 T grub_efi_set_variable_with_attributes
000000000000419f T grub_efi_set_variable
00000000000041b5 T grub_efi_get_variable_with_attributes
00000000000042ee T grub_efi_set_variable_to_string
000000000000436d T grub_efi_get_variable
0000000000004380 T grub_efi_section_addr
00000000000044d1 T grub_efi_get_filename
00000000000046ef T grub_efi_get_device_path
000000000000470e T grub_efi_find_last_device_path
000000000000475f T grub_efi_duplicate_device_path
00000000000047f3 T grub_efi_print_device_path
0000000000004f2b T grub_efi_compare_device_paths
000000000000503f T grub_efi_find_configuration_table
00000000000050ab T grub_efi_init
000000000000514f T grub_machine_get_bootlocation
00000000000051fd T grub_efi_fini
000000000000521e t find_card
00000000000052cd T grub_efi_free_pages
000000000000535a T grub_efi_allocate_pages_real
00000000000054f9 T grub_efi_allocate_any_pages
000000000000551b T grub_efi_allocate_fixed
0000000000005535 T grub_efi_get_memory_map
000000000000567c T grub_efi_finish_boot_services
0000000000005965 T grub_efi_find_mmap_size
00000000000059cb t grub_efi_mm_add_regions
0000000000005cf2 T grub_efi_memory_fini
0000000000005d27 T grub_efi_mm_init
0000000000005d7b t shim_lock_verifier_write
0000000000005dec t shim_lock_verifier_init
0000000000005e53 T grub_efi_get_secureboot
0000000000005fdf T grub_shim_lock_verifier_setup
00000000000060c5 T grub_is_shim_lock_enabled
00000000000060d3 t grub_env_find
0000000000006146 T grub_env_set
000000000000628c T grub_env_get
00000000000062bd T grub_env_get_bool
0000000000006356 T grub_env_unset
00000000000063f3 T grub_env_update_get_sorted
0000000000006488 T grub_register_variable_hook
00000000000064f3 T grub_env_export
0000000000006542 T grub_error
00000000000065c2 T grub_error_push
0000000000006646 T grub_error_pop
00000000000066bc T grub_print_error
000000000000675d T grub_file_get_device_name
00000000000067f8 T grub_file_read
00000000000068a4 T grub_file_close
00000000000068fa T grub_file_open
0000000000006afc T grub_file_seek
0000000000006b36 t probe_dummy_iter
0000000000006b40 t grub_fs_blocklist_read
0000000000006c3f t grub_fs_blocklist_open
0000000000006e2f T grub_fs_probe
0000000000007058 T grub_millisleep
000000000000708b T grub_machine_init
00000000000070ac T grub_machine_fini
00000000000070e3 T grub_tsc_calibrate_from_efi
000000000000716d t grub_tsc_get_time_ms
00000000000071b6 T grub_tsc_init
000000000000726d T grub_tsc_calibrate_from_pit
000000000000731b T grub_pmtimer_wait_count_tsc
000000000000751c T grub_tsc_calibrate_from_pmtimer
00000000000075db T grub_named_list_find
000000000000761e T grub_list_push
0000000000007639 T grub_list_remove
0000000000007668 t lockdown_verifier_init
00000000000076b6 T grub_lockdown
0000000000007725 T grub_is_lockdown
0000000000007733 t grub_env_write_root
000000000000778b T grub_modules_get_end
00000000000077ab T grub_main
0000000000007d6a t grub_gettext_dummy
0000000000007d72 t free_printf_args
0000000000007d92 T grub_memmove
0000000000007dd2 T grub_strcpy
0000000000007de9 T grub_puts_
0000000000007e29 T grub_memcmp
0000000000007e52 T grub_strcmp
0000000000007e72 T grub_strncmp
0000000000007ead T grub_strchr
0000000000007ec5 T grub_strrchr
0000000000007edc T grub_isspace
0000000000007f02 t grub_iswordseparator
0000000000007f50 T grub_strword
0000000000007fee T grub_memset
0000000000008067 T grub_strlen
000000000000807c T grub_debug_enabled
00000000000081be T grub_strdup
0000000000008213 T grub_strndup
0000000000008271 T grub_divmod64
00000000000082d6 t write_number
000000000000846a T grub_strtoull
0000000000008597 t parse_printf_arg_fmt
0000000000008944 t parse_printf_args
0000000000008a0c T grub_strtoul
0000000000008a1c t grub_vsnprintf_real
0000000000009044 T grub_vprintf
000000000000914c T grub_err_printf
000000000000914c T grub_printf
0000000000009195 T grub_printf_
00000000000091ed T grub_real_dprintf
000000000000928e t grub_vsnprintf.part.0
0000000000009308 T grub_vsnprintf
0000000000009320 T grub_snprintf
0000000000009361 T grub_xvasprintf
0000000000009410 T grub_xasprintf
0000000000009459 T grub_printf_fmt_check
000000000000956f T grub_abort
00000000000095c7 T grub_fatal
0000000000009635 T grub_utf8_to_utf16_alloc
0000000000009804 t get_header_from_pointer
00000000000098b9 T grub_memalign
0000000000009b4f T grub_calloc
0000000000009bbe T grub_malloc
0000000000009bbe T malloc
0000000000009bd3 T grub_zalloc
0000000000009c18 T free
0000000000009c18 T grub_free
0000000000009d36 T grub_mm_init_region
0000000000009fe8 b grub_last_time
000000000000a000 B grub_dl_head
000000000000a020 b grub_symtab
000000000000a05b T grub_realloc
000000000000a124 t check_varstate
000000000000a140 t grub_parser_execute_getline
000000000000a1bc t grub_buffer_peek_data_at
000000000000a205 t grub_buffer_append_data
000000000000a2a5 t terminate_arg
000000000000a310 t add_var
000000000000a3d5 T grub_parser_cmdline_state
000000000000a481 T grub_parser_split_cmdline
000000000000a7ae T grub_parser_execute
000000000000a828 T grub_partition_probe
000000000000a9d8 T grub_partition_iterate
000000000000aa4a T grub_partition_get_name
000000000000ab92 t grub_partition_check_containment
000000000000ac35 t probe_iter
000000000000aca3 t part_iterate
000000000000ad8b T grub_rescue_parse_line
000000000000aedd t grub_rescue_read_line
000000000000b008 B grub_efi_system_table
000000000000b010 B grub_efi_image_handle
000000000000b018 B grub_efi_net_config
000000000000b020 B grub_modbase
000000000000b020 T grub_rescue_run
000000000000b028 B grub_efi_is_finished
000000000000b030 b efi_allocated_memory
000000000000b038 b finish_desc_version
000000000000b040 b finish_desc_size
000000000000b048 b finish_key
000000000000b050 b finish_mmap_size
000000000000b058 b finish_mmap_buf
000000000000b060 b shim_lock_enabled
000000000000b080 b initial_context
000000000000b0a9 t grub_putcode_dumb
000000000000b100 B grub_err_printed_errors
000000000000b120 B grub_errmsg
000000000000b145 t grub_xputs_dumb
000000000000b1a8 T grub_getkey_noblock
000000000000b202 T grub_getkeystatus
000000000000b220 B grub_errno
000000000000b224 b grub_error_stack_assert
000000000000b228 b grub_error_stack_pos
000000000000b240 b grub_error_stack_items
000000000000b257 T grub_key_is_interrupt
000000000000b288 T grub_refresh
000000000000b2bf T grub_getkey
000000000000b2e9 T grub_get_time_ms
000000000000b2f9 T grub_install_get_time_ms
000000000000b30b t verified_read
000000000000b33f t verified_close
000000000000b3a4 t grub_verifiers_open
000000000000b67c T grub_verify_string
000000000000b700 T grub_verifiers_init
000000000000b719 T grub_arch_dl_check_header
000000000000b74c T grub_arch_dl_relocate_symbols
000000000000b8c2 t grub_efi_translate_key
000000000000b94c t grub_console_read_key_stroke
000000000000b9f1 t grub_console_setcolorstate
000000000000ba74 t grub_console_setcursor
000000000000bac8 t grub_console_getkeystatus
000000000000bb56 t grub_console_getkey
000000000000bc2e t grub_console_getxy
000000000000bc75 t grub_console_cls
000000000000bc80 B grub_file_progress_hook
000000000000bca0 B grub_file_filters
000000000000bcc0 B grub_grubnet_fini
000000000000bcc8 B grub_fs_autoload_hook
000000000000bcd0 B grub_fs_list
000000000000bcd8 b count.3524
000000000000bcdd t grub_efi_console_input_init
000000000000bce0 B grub_tsc_rate
000000000000bce8 b tsc_boot_time
000000000000bcf0 b lockdown
000000000000bcf8 b load_config
000000000000bd00 b buf.2132
000000000000bd36 t grub_efi_console_output_fini
000000000000bd80 t grub_prepare_for_text_output
000000000000be00 B grub_mm_add_region_fn
000000000000be08 B grub_mm_base
000000000000be10 B grub_partition_map_list
000000000000be20 b linebuf
000000000000be5c t grub_console_gotoxy
000000000000bea8 t grub_console_getwh
000000000000bf1d t grub_console_putchar
000000000000bf20 B grub_net_poll_cards_idle
000000000000bf28 B grub_term_poll_usb
000000000000bf30 B grub_term_inputs
000000000000bf38 B grub_term_outputs
000000000000bf40 B grub_term_inputs_disabled
000000000000bf48 B grub_term_outputs_disabled
000000000000bf50 b get_time_ms_func
000000000000bf58 B grub_file_verifiers
000000000000bf60 b key_data.3872
000000000000c0a3 T grub_console_init
000000000000c176 T grub_console_fini
000000000000c1c0 T grub_register_exported_symbols
-------------- stack trace section for grub_efi_allocate_pages_real
--------------
======== for the first two allocate_pages_real ========
grub_efi_allocate_pages_real
grub_efi_mm_add_regions
grub_memalign
grub_efidisk_readwrite
grub_efidisk_read
grub_disk_read_small_real
grub_disk_read_small
grub_disk_read
some call probably from multiboot2 (couldn't make symbols work for it)
======== for the last two allocate_pages_real ========
grub_efi_allocate_pages_real
grub_efi_mm_add_regions
grub_memalign
some call probably from multiboot2 (couldn't make symbols work for
it)[email protected]
