[PATCH 11/12] grub-shell: Use malta qemu-mips machine type instead off non-existant indy

2021-03-31 Thread Glenn Washburn 
Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index fbc809a8c..c69fbd080 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -119,7 +119,7 @@ case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" 
in
 mips-arc)
boot=cd
qemu=qemu-system-mips64
-   qemuopts="-M indy $qemuopts"
+   qemuopts="-M malta $qemuopts"
serial_port=arc/serial0/line0
console=
trim=1
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 04/12] grub-shell: Allow setting default timeout via GRUB_SHELL_DEFAULT_TIMEOUT envvar

2021-03-31 Thread Glenn Washburn 
Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index 9d8c417da..e80471126 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -211,7 +211,7 @@ case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" 
in
console=console;;
 esac
 
-timeout=60
+timeout=${GRUB_SHELL_DEFAULT_TIMEOUT:-60}
 mkimage_extra_arg=
 
 # Check the arguments.
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 09/12] grub-shell: Add --verbose to mkrescue when $debug is greater than 2

2021-03-31 Thread Glenn Washburn 
Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index 198b03057..854508750 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -404,7 +404,9 @@ fi
 
 if [ x$boot != xnet ] && [ x$boot != xemu ]; then
 pkgdatadir="@builddir@" \
-exec_show_error "@builddir@/grub-mkrescue" "--output=${isofile}" \
+exec_show_error "@builddir@/grub-mkrescue" \
+   ${debug:+$([ "$debug" -gt 2 ] && echo -n "--verbose")} \
+   "--output=${isofile}" \
"--override-directory=${builddir}/grub-core" \
--rom-directory="${rom_directory}" \
--locale-directory="@srcdir@/po" \
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 06/12] grub-shell: Add grub output logfile with grub-shell --debug

2021-03-31 Thread Glenn Washburn 
This allows seeing full qemu output of grub-shell, which can be invaluable
when debugging failing tests.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index 8c6ed76d7..c6d7860e9 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -379,6 +379,9 @@ echo "${halt_cmd}" >>${cfgfile}
 
 test -z "$debug" || echo "GRUB script: ${cfgfile}" >&2
 test -z "$debug" || echo "GRUB testcase script: ${tmpfile}" >&2
+
+goutfile="$work_directory/grub-qemu.log"
+test -z "$debug" || echo "GRUB output log: ${goutfile}" >&2
 test -z "$debug" || echo "Boot device: ${boot}" >&2
 
 isofile="$work_directory/grub.iso"
@@ -502,7 +505,7 @@ if [ x$boot = xnet ]; then
 cp "${cfgfile}" "$netdir/boot/grub/grub.cfg"
 cp "${source}" "$netdir/boot/grub/testcase.cfg"
 [ -z "$files" ] || copy_extra_files "$netdir" $files
-timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -boot n -net 
"user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext"
  -net nic  | cat | tr -d "\r" | do_trim
+timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -boot n -net 
"user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext"
  -net nic  | cat | tr -d "\r" | tee "${goutfile}" | do_trim
 elif [ x$boot = xemu ]; then
 rootdir="$work_directory/rootdir"
 grubdir="$rootdir/boot/grub"
@@ -521,11 +524,11 @@ elif [ x$boot = xemu ]; then
 [ -z "$files" ] || copy_extra_files "$rootdir" $files
 roottar="$work_directory/root.tar"
 (cd "$rootdir"; tar cf "$roottar" .)
-@builddir@/grub-core/grub-emu -m "$device_map" --memdisk "$roottar" -r 
memdisk -d "/boot/grub" | tr -d "\r" | do_trim
+@builddir@/grub-core/grub-emu -m "$device_map" --memdisk "$roottar" -r 
memdisk -d "/boot/grub" | tr -d "\r" | tee "${goutfile}" | do_trim
 test -n "$debug" || rm -rf "$rootdir"
 test -n "$debug" || rm -f "$roottar"
 else
-timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -${device}"${isofile}" ${bootdev} | cat | tr -d "\r" | do_trim
+timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -${device}"${isofile}" ${bootdev} | cat | tr -d "\r" | tee 
"${goutfile}" | do_trim
 fi
 if [ x$boot = xcoreboot ]; then
 test -n "$debug" || rm -f "${imgfile}"
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 12/12] grub-shell: Update qemu UEFI firmware names to be more generic

2021-03-31 Thread Glenn Washburn 
The current qemu firmware paths for arm-efi and arm64-efi are hardcoded
and not available on Ubuntu/Debian. Make these paths relative and basenames
generic. The idea is that regardless where or what names firmware images
are installed under they can be copied and renamed into a temporary
directory. Then use the -L argument to qemu to point to that directory.

For convenience, the GRUB source directory which build grub-shell will be
added to the list of firmware paths that qemu will search. So all that is
needed is to copy the firmwares to the source directory with potential
renames.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index c69fbd080..1bdfbbeba 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -74,7 +74,7 @@ exec_show_error () {
 work_directory=${WORKDIR:-`mktemp -d "${TMPDIR:-/tmp}/grub-shell.XX"`} 
|| exit 1
 
 . "${builddir}/grub-core/modinfo.sh"
-qemuopts="${GRUB_QEMU_OPTS}"
+qemuopts="${GRUB_QEMU_OPTS} -L ${srcdir}"
 serial_port=com0
 serial_null=
 halt_cmd=halt
@@ -194,7 +194,7 @@ case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" 
in
boot=hd
console=console
trim=1
-   qemuopts="-machine virt -cpu cortex-a57 -bios 
/usr/share/qemu-efi/QEMU_EFI.fd $qemuopts"
+   qemuopts="-machine virt -cpu cortex-a57 -bios OVMF-aarch64.fd $qemuopts"
disk="device virtio-blk-device,drive=hd1 -drive if=none,id=hd1,file="
serial_port=
;;
@@ -203,7 +203,7 @@ case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" 
in
boot=hd
console=console
trim=1
-   qemuopts="-machine virt -bios /usr/share/ovmf-arm/QEMU_EFI.fd $qemuopts"
+   qemuopts="-machine virt -bios OVMF-arm.fd $qemuopts"
disk="device virtio-blk-device,drive=hd1 -drive if=none,id=hd1,file="
serial_port=efi0
;;
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 07/12] grub-shell: Set exit status to qemu exit status

2021-03-31 Thread Glenn Washburn 
This allows us to test if unexpected output in test scripts is because of a
bug in grub, because there was an error in qemu, or qemu was killed due to a
timeout.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index c6d7860e9..40c62b9fd 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -498,6 +498,9 @@ copy_extra_files() {
 done
 }
 
+ret=0
+mkfifo "$work_directory/qemu-pipe"
+cat < "$work_directory/qemu-pipe" | tr -d "\r" | tee "${goutfile}" | do_trim &
 if [ x$boot = xnet ]; then
 netdir="$work_directory/netdir"
 mkdir -p "$netdir"
@@ -505,7 +508,7 @@ if [ x$boot = xnet ]; then
 cp "${cfgfile}" "$netdir/boot/grub/grub.cfg"
 cp "${source}" "$netdir/boot/grub/testcase.cfg"
 [ -z "$files" ] || copy_extra_files "$netdir" $files
-timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -boot n -net 
"user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext"
  -net nic  | cat | tr -d "\r" | tee "${goutfile}" | do_trim
+timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -boot n -net 
"user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext"
  -net nic > "$work_directory/qemu-pipe" || ret=$?
 elif [ x$boot = xemu ]; then
 rootdir="$work_directory/rootdir"
 grubdir="$rootdir/boot/grub"
@@ -524,11 +527,11 @@ elif [ x$boot = xemu ]; then
 [ -z "$files" ] || copy_extra_files "$rootdir" $files
 roottar="$work_directory/root.tar"
 (cd "$rootdir"; tar cf "$roottar" .)
-@builddir@/grub-core/grub-emu -m "$device_map" --memdisk "$roottar" -r 
memdisk -d "/boot/grub" | tr -d "\r" | tee "${goutfile}" | do_trim
+@builddir@/grub-core/grub-emu -m "$device_map" --memdisk "$roottar" -r 
memdisk -d "/boot/grub" > "$work_directory/qemu-pipe" || ret=$?
 test -n "$debug" || rm -rf "$rootdir"
 test -n "$debug" || rm -f "$roottar"
 else
-timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -${device}"${isofile}" ${bootdev} | cat | tr -d "\r" | tee 
"${goutfile}" | do_trim
+timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -${device}"${isofile}" ${bootdev} > 
"$work_directory/qemu-pipe" || ret=$?
 fi
 if [ x$boot = xcoreboot ]; then
 test -n "$debug" || rm -f "${imgfile}"
@@ -536,6 +539,7 @@ fi
 test -n "$debug" || rm -f "${isofile}"
 test -n "$debug" || rm -rf "${rom_directory}"
 test -n "$debug" || rm -f "${tmpfile}" "${cfgfile}"
-exit 0
+rm -f "$work_directory/qemu-pipe"
+exit $ret
 
 
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 10/12] grub-shell: Only turn on qemu head when large debug value is specified

2021-03-31 Thread Glenn Washburn 
There are several levels of debug and more may be added. But the qemu head
should be one of the highest debug levels. Set debug to a high value of 10
to turn on the head. We do not want to accidentally turn it on when adding
a new debug level to the testing system.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index 854508750..fbc809a8c 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -391,7 +391,7 @@ isofile="$work_directory/grub.iso"
 test -z "$debug" || echo "GRUB ISO file: ${isofile}" >&2
 test -z "$debug" || echo "GRUB ROM directory: ${rom_directory}" >&2
 
-if test -z "$debug"; then
+if test "${debug:-0}" -lt 10; then
   qemuopts="${qemuopts} -nographic -monitor file:/dev/null"
   # SeaBIOS 1.11.0 added support for VGA emulation over a serial port.  If
   # this is configured in SeaBIOS, then -nographic causes some extra junk to
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 08/12] tests: Allow turning on shell tracing from environment variables

2021-03-31 Thread Glenn Washburn 
This allows turning on shell tracing when its not practical or not possible
to use commandline arguments. Turn on tracing when the envvar is an integer
greater than 1, since these can log a lot of messages.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-fs-tester.in | 2 ++
 tests/util/grub-shell.in | 5 -
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index bfc425e1f..adeffbfba 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -2,6 +2,8 @@
 
 set -e
 
+[ "${GRUB_TEST_DEFAULT_DEBUG:-0}" -gt 1 ] && set -x
+
 fs="$1"
 
 GRUBFSTEST="@builddir@/grub-fstest"
diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index 40c62b9fd..198b03057 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -215,6 +215,7 @@ esac
 
 timeout=${GRUB_SHELL_DEFAULT_TIMEOUT:-60}
 mkimage_extra_arg=
+debug=${GRUB_SHELL_DEFAULT_DEBUG:-$GRUB_TEST_DEFAULT_DEBUG}
 
 # Check the arguments.
 for option in "$@"; do
@@ -234,7 +235,7 @@ for option in "$@"; do
 --no-trim)
trim=0 ;;
 --debug)
-debug=1 ;;
+debug=$((debug+1)) ;;
 --modules=*)
ms=`echo "$option" | sed -e 's/--modules=//' -e 's/,/ /g'`
modules="$modules $ms" ;;
@@ -319,6 +320,8 @@ for option in "$@"; do
 esac
 done
 
+[ "${debug:-0}" -gt 1 ] && set -x
+
 if [ "x${source}" = x ] ; then
 tmpfile="$work_directory/testcase.cfg"
 while read REPLY; do
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 00/12] Grub-shell improvements

2021-03-31 Thread Glenn Washburn 
I've extracted this patch series from the GitLab CI patch series, which some
minor modifications. Mostly these are improvements to grub-shell to facilitate
debugging.

The only patch that might be considered a fix, as opposed to an improvement,
would be patch #11, which fixes the issue where qemu-mips is given a
non-existant machine type. And while this was discussed here on the list as a
possible solution, I couldn't get the mips tests working, so I'm not sure it's
the right fix.

Glenn

Glenn Washburn (12):
  grub-shell: Allow specifying non-default trim line contents
  grub-shell: Trim line should always be matched from the beginning of
the line
  grub-shell: Only show grub-mkrescue output if it returns an error
  grub-shell: Allow setting default timeout via
GRUB_SHELL_DEFAULT_TIMEOUT envvar
  grub-shell: Put all generated files into working dir and use better
file names
  grub-shell: Add grub output logfile with grub-shell --debug
  grub-shell: Set exit status to qemu exit status
  tests: Allow turning on shell tracing from environment variables
  grub-shell: Add --verbose to mkrescue when $debug is greater than 2
  grub-shell: Only turn on qemu head when large debug value is specified
  grub-shell: Use malta qemu-mips machine type instead off non-existant
indy
  grub-shell: Update qemu UEFI firmware names to be more generic

 tests/util/grub-fs-tester.in |  2 +
 tests/util/grub-shell.in | 89 
 2 files changed, 63 insertions(+), 28 deletions(-)

-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 03/12] grub-shell: Only show grub-mkrescue output if it returns an error

2021-03-31 Thread Glenn Washburn 
The previous behavior ignored an error and the output from grub-mkrescue.
This made it a pain to discover that grub-mkrescue was the reason that tests
which rely on grub-shell were failing.  Even after discovering grub-mkrescue
was the culprit, there was no output to indicate why it was failing.  It
turns out that grub-mkrescue is a thin wrapper around xorriso. So if you do
not have xorriso installed it will fail with an error message about not
being able to find xorriso.

This change will allow grub-mkrescue output to be written to stderr, only if
grub-mkrescue fails.  If grub-mkrescue succeeds, there will be no output
from grub-mkrescue so as not to interfere with the functioning of tests.
This change should have no effect on the running of tests or other uses of
grub-shell as it only modifies the error path.

Also, if grub-mkrescue fails, the script exits early.  Since grub-shell
needs the iso image created by grub-mkresue to boot the qemu instance, a
failure here should be considered fatal.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 17 +++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index 602b16f3e..9d8c417da 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -60,6 +60,17 @@ Report bugs to .
 EOF
 }
 
+# Exec given argv and only show its output on STDERR if it returns an
+# error status.
+exec_show_error () {
+v=`$@ 2>&1`
+ret=$?
+if [ "$ret" != 0 ]; then
+echo "$v" >&2
+exit $ret
+fi
+}
+
 . "${builddir}/grub-core/modinfo.sh"
 qemuopts="${GRUB_QEMU_OPTS}"
 serial_port=com0
@@ -383,13 +394,15 @@ if test -z "$debug"; then
 fi
 
 if [ x$boot != xnet ] && [ x$boot != xemu ]; then
-pkgdatadir="@builddir@" "@builddir@/grub-mkrescue" "--output=${isofile}" 
"--override-directory=${builddir}/grub-core" \
+pkgdatadir="@builddir@" \
+exec_show_error "@builddir@/grub-mkrescue" "--output=${isofile}" \
+   "--override-directory=${builddir}/grub-core" \
--rom-directory="${rom_directory}" \
--locale-directory="@srcdir@/po" \
--themes-directory="@srcdir@/themes" \
$mkimage_extra_arg ${mkrescue_args} \
"/boot/grub/grub.cfg=${cfgfile}" "/boot/grub/testcase.cfg=${source}" \
-   ${files} >/dev/null 2>&1
+   ${files} || exit $?
 fi
 if [ x$boot = xhd ]; then
 if [ "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" = arm64-efi ] 
|| [ "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" = arm-efi ]; then
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 05/12] grub-shell: Put all generated files into working dir and use better file names

2021-03-31 Thread Glenn Washburn 
When running tests there are many invocations of grub-shell, and because the
output files are all random names in the same tmp directory, it becomes more
work to figure out which files went with which grub-shell invocations. So
all generated files from one invocation of grub-shell are put into a
randomly named directory, so as not to collide with other grub-shell
invocations. And now that the generated files can be put in a location where
they will not get stepped on, and they can be named sensible names.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 22 +-
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index e80471126..8c6ed76d7 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -71,6 +71,8 @@ exec_show_error () {
 fi
 }
 
+work_directory=${WORKDIR:-`mktemp -d "${TMPDIR:-/tmp}/grub-shell.XX"`} 
|| exit 1
+
 . "${builddir}/grub-core/modinfo.sh"
 qemuopts="${GRUB_QEMU_OPTS}"
 serial_port=com0
@@ -80,7 +82,7 @@ pseries=n
 disk="hda "
 case "${grub_modinfo_target_cpu}-${grub_modinfo_platform}" in
 *-emu)
-   device_map=`mktemp "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
+   device_map="$work_directory/device.map"
boot=emu
console=console
disk=0
@@ -318,14 +320,14 @@ for option in "$@"; do
 done
 
 if [ "x${source}" = x ] ; then
-tmpfile=`mktemp "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
+tmpfile="$work_directory/testcase.cfg"
 while read REPLY; do
echo "$REPLY" >> ${tmpfile}
 done
 source=${tmpfile}
 fi
 
-cfgfile=`mktemp "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
+cfgfile="$work_directory/grub.cfg"
 cat <${cfgfile}
 grubshell=yes
 enable_progress_indicator=0
@@ -357,7 +359,8 @@ if [ $trim = 1 ]; then
 echo "echo; echo $trim_head" >>${cfgfile}
 fi
 
-rom_directory=`mktemp -d "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
+rom_directory="$work_directory/rom_directory"
+mkdir -p "$rom_directory"
 
 for mod in ${modules}
 do
@@ -378,7 +381,7 @@ test -z "$debug" || echo "GRUB script: ${cfgfile}" >&2
 test -z "$debug" || echo "GRUB testcase script: ${tmpfile}" >&2
 test -z "$debug" || echo "Boot device: ${boot}" >&2
 
-isofile=`mktemp "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
+isofile="$work_directory/grub.iso"
 test -z "$debug" || echo "GRUB ISO file: ${isofile}" >&2
 test -z "$debug" || echo "GRUB ROM directory: ${rom_directory}" >&2
 
@@ -450,7 +453,7 @@ if [ x$boot = xmips_qemu ]; then
 fi
 
 if [ x$boot = xcoreboot ]; then
-imgfile=`mktemp "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
+imgfile="$work_directory/coreboot.img"
 cp "${GRUB_COREBOOT_ROM}" "${imgfile}"
 "${GRUB_CBFSTOOL}" "${imgfile}" add-payload -f 
"${rom_directory}/coreboot.elf" -n fallback/payload
 bootdev="-bios ${imgfile}"
@@ -493,14 +496,15 @@ copy_extra_files() {
 }
 
 if [ x$boot = xnet ]; then
-netdir=`mktemp -d "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
+netdir="$work_directory/netdir"
+mkdir -p "$netdir"
 pkgdatadir="@builddir@" "@builddir@/grub-mknetdir" 
"--grub-mkimage=${builddir}/grub-mkimage" "--directory=${builddir}/grub-core" 
"--net-directory=$netdir" ${mkrescue_args} > /dev/null
 cp "${cfgfile}" "$netdir/boot/grub/grub.cfg"
 cp "${source}" "$netdir/boot/grub/testcase.cfg"
 [ -z "$files" ] || copy_extra_files "$netdir" $files
 timeout -s KILL $timeout "${qemu}" ${qemuopts} ${serial_null} -serial 
file:/dev/stdout -boot n -net 
"user,tftp=$netdir,bootfile=/boot/grub/${grub_modinfo_target_cpu}-${grub_modinfo_platform}/core.$netbootext"
  -net nic  | cat | tr -d "\r" | do_trim
 elif [ x$boot = xemu ]; then
-rootdir="$(mktemp -d "${TMPDIR:-/tmp}/tmp.XX")"
+rootdir="$work_directory/rootdir"
 grubdir="$rootdir/boot/grub"
 mkdir -p "$grubdir/fonts"
 mkdir -p "$grubdir/themes"
@@ -515,7 +519,7 @@ elif [ x$boot = xemu ]; then
 cp "${cfgfile}" "$grubdir/grub.cfg"
 cp "${source}" "$grubdir/testcase.cfg"
 [ -z "$files" ] || copy_extra_files "$rootdir" $files
-roottar="$(mktemp "${TMPDIR:-/tmp}/tmp.XX")"
+roottar="$work_directory/root.tar"
 (cd "$rootdir"; tar cf "$roottar" .)
 @builddir@/grub-core/grub-emu -m "$device_map" --memdisk "$roottar" -r 
memdisk -d "/boot/grub" | tr -d "\r" | do_trim
 test -n "$debug" || rm -rf "$rootdir"
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 02/12] grub-shell: Trim line should always be matched from the beginning of the line

2021-03-31 Thread Glenn Washburn 
When turning on shell tracing the trim line will be output before we
actually want to start the trim. However, in this case the trim line never
starts from the beginning of the line. So start trimming from the correct
line by matching from the beginning of the line.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index b409962f1..602b16f3e 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -343,7 +343,7 @@ terminal_output ${term}
 EOF
 
 if [ $trim = 1 ]; then
-echo "echo $trim_head" >>${cfgfile}
+echo "echo; echo $trim_head" >>${cfgfile}
 fi
 
 rom_directory=`mktemp -d "${TMPDIR:-/tmp}/tmp.XX"` || exit 1
@@ -457,7 +457,7 @@ fi
 do_trim ()
 {
 if [ $trim = 1 ] || [ $trim = 2 ]; then
-   awk '{ if (have_head == 1) print $0; } /'"$trim_head"'/ { have_head=1; 
}'
+   awk '{ if (have_head == 1) print $0; } /^'"$trim_head"'/ { have_head=1; 
}'
 else
cat
 fi
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 01/12] grub-shell: Allow specifying non-default trim line contents

2021-03-31 Thread Glenn Washburn 
This will be useful for tests that have unwanted output from setup. This is
not documented because its only intended to be internal at the moment. Also,
--no-trim is allowed to explicitly turn off trim.

Signed-off-by: Glenn Washburn 
---
 tests/util/grub-shell.in | 14 +-
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/tests/util/grub-shell.in b/tests/util/grub-shell.in
index 93e9f5148..b409962f1 100644
--- a/tests/util/grub-shell.in
+++ b/tests/util/grub-shell.in
@@ -32,6 +32,7 @@ PATH="${builddir}:$PATH"
 export PATH
 
 trim=0
+trim_head=664cbea8-132f-4770-8aa4-1696d59ac35c
 
 # Usage: usage
 # Print the usage.
@@ -212,8 +213,13 @@ for option in "$@"; do
echo "$0 (GNU GRUB ${PACKAGE_VERSION})"
exit 0 ;;
 --trim)
-   trim=1
+   trim=1 ;;
+--trim=*)
+   trim=2
+   trim_head=`echo "$option" | sed -e 's/--trim=//' -e 's/,/ /g'`
;;
+--no-trim)
+   trim=0 ;;
 --debug)
 debug=1 ;;
 --modules=*)
@@ -336,8 +342,6 @@ terminal_input ${term}
 terminal_output ${term}
 EOF
 
-trim_head=664cbea8-132f-4770-8aa4-1696d59ac35c
-
 if [ $trim = 1 ]; then
 echo "echo $trim_head" >>${cfgfile}
 fi
@@ -452,8 +456,8 @@ fi
 
 do_trim ()
 {
-if [ $trim = 1 ]; then
-   awk '{ if (have_head == 1) print $0; } 
/664cbea8-132f-4770-8aa4-1696d59ac35c/ { have_head=1; }'
+if [ $trim = 1 ] || [ $trim = 2 ]; then
+   awk '{ if (have_head == 1) print $0; } /'"$trim_head"'/ { have_head=1; 
}'
 else
cat
 fi
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: multiboot2 and module2 boot issues via GRUB2

2021-03-31 Thread Roman Shaposhnik 
Hi Andrew!

first of all -- thanks for pointing me in the right direction. So after
reading relevant sources: comments inline.

On Tue, Mar 30, 2021 at 12:08 PM Andrew Cooper 
wrote:

> On 30/03/2021 19:28, Roman Shaposhnik wrote:
> > Hi!
> >
> > seems like I've run into an issue with multiboot2 and module2
> > commands that I can't quite explain. Since it may be something
> > super simply and silly -- I wanted to reach out here before I do
> > a GRUB/Xen/LK source deepdive.
> >
> > So here's the deal: whenever I boot straight up Linux kernel
> > I can do the following sequence of commands:
> >linux /kernel
> >initrd foo.cpio.gz bar.cpio.gz
> > and have linux kernel effectively stack content of bar.cpio.gz
> > on top of foo.cpio.gz and present a unified initramfs that way.
> >
> > I'm trying to replicate it with Xen, but:
> >  multiboot2 /boot/xen.gz
> >  module2 /kernel
> >  module2 foo.cpio.gz
> >  module2 bar.cpio.gz
> > only seems to be picking up foo.cpio.gz
> >
> > Has anyone run into this issue before?
>
> I can explain why that happens.  Not sure if it counts as a feature, bug
> or mis-expectation, but CC'ing grub-devel for their input.
>
> The initrd command is presumably concatenating those two files together
> in memory, and presenting Linux a single initrd pointer.
>

It is indeed what seems to be happening.


> For the module2 example, you're putting 3 distinct files in memory, and
> giving Xen a list 3 modules.
>

And that is also correct -- nothing like that is possible with modules.
Kernel actually needs to be aware of them. So the question then
becomes...


> Xen is capable of taking various things via modules, such as an
> XSM/Flask policy, or microcode, so has logic to identify these if
> present and separate them from "other stuff".  However, there is a
> hardcoded expectation that the first module is the dom0 kernel, and the
> next unrecognised module, if present, is *the* initrd.

I expect that Xen isn't handing bar.cpio.gz on to dom0, but I'm not sure
> whether passing two distinct initrd-like-things to Linux is even possible.
>
> What you presumably want is some `initrd` side effect in Grub so you can
> write `module2 foo.cpio.gz bar.cpio.gz` and have it concatenate things
> together in memory and present one MB2 module, but I suspect that exact
> syntax might be ambiguous with command line handling.  I have no idea
> whether such a command currently exists.
>

...I guess there's no mechanism out-of-the box to achieve what I want?

And the obvious next question: is my EVE usecase esoteric enough that
I should just go ahead and do a custom GRUB patch or is there a more
general interest in this?

Thanks,
Roman.
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 2/2] docs: Conform badmem and cutmem description indentations with other commands

2021-03-31 Thread Glenn Washburn 
Signed-off-by: Glenn Washburn 
---
 docs/grub.texi | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/grub.texi b/docs/grub.texi
index 93ae12a88..e2c58415a 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -4125,7 +4125,6 @@ can be changed only when using @samp{gfxterm} for 
terminal output.
 
 @deffn Command badram addr,mask[,addr,mask...]
 Filter out bad RAM.
-@end deffn
 
 This command notifies the memory manager that specified regions of
 RAM ought to be filtered out (usually, because they're damaged).  This
@@ -4147,6 +4146,7 @@ The command is similar to @command{cutmem} command.
 Note: The command is not allowed when lockdown is enforced (@pxref{Lockdown}).
   This prevents removing EFI memory regions to potentially subvert the
   security mechanisms provided by the UEFI secure boot.
+@end deffn
 
 @node blocklist
 @subsection blocklist
@@ -4317,7 +4317,6 @@ specified without dash separators.
 
 @deffn Command cutmem from[K|M|G] to[K|M|G]
 Remove any memory regions in specified range.
-@end deffn
 
 This command notifies the memory manager that specified regions of RAM ought to
 be filtered out. This remains in effect after a payload kernel has been loaded
@@ -4330,6 +4329,7 @@ The command is similar to @command{badram} command.
 Note: The command is not allowed when lockdown is enforced (@pxref{Lockdown}).
   This prevents removing EFI memory regions to potentially subvert the
   security mechanisms provided by the UEFI secure boot.
+@end deffn
 
 @node date
 @subsection date
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 1/2] docs: Add note to cryptomount that UUIDs should be specified without dashes

2021-03-31 Thread Glenn Washburn 
Signed-off-by: Glenn Washburn 
---
 docs/grub.texi | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/docs/grub.texi b/docs/grub.texi
index eeb3118eb..93ae12a88 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -4307,6 +4307,9 @@ GRUB suports devices encrypted using LUKS, LUKS2 and 
geli. Note that necessary
 modules (@var{luks}, @var{luks2} and @var{geli}) have to be loaded manually
 before this command can be used. For LUKS2 only the PBKDF2 key derivation
 function is supported, as Argon2 is not yet supported.
+
+Also, note that, unlike filesystem UUIDs, UUIDs for encrypted devices must be
+specified without dash separators.
 @end deffn
 
 @node cutmem
-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 0/2] Misc doc changes

2021-03-31 Thread Glenn Washburn 
The first patch adds a long overdue note to the cryptomount command that UUIDs
should be specified without dash, differently than how fs uuids are specified.

The second patch makes the indentation of command description text for cutmem
and badmem conform with the indentation of the rest of the commands.

Glenn

Glenn Washburn (2):
  docs: Add note to cryptomount that UUIDs should be specified without
dashes
  docs: Conform badmem and cutmem description indentations with other
commands

 docs/grub.texi | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

-- 
2.27.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH] password: Fix backspace in username prompt

2021-03-31 Thread egori 
From: Egor Ignatov 

Allow control characters in the bidi visual.
Resolves: #60114

Signed-off-by: Egor Ignatov 
---
 grub/grub-core/normal/charset.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/grub/grub-core/normal/charset.c b/grub/grub-core/normal/charset.c
index b0ab47d73..30e819bdf 100644
--- a/grub/grub-core/normal/charset.c
+++ b/grub/grub-core/normal/charset.c
@@ -925,6 +925,7 @@ grub_bidi_line_logical_to_visual (const grub_uint32_t 
*logical,
pop_stack ();
break;
  case GRUB_BIDI_TYPE_BN:
+   visual_len++;
break;
  case GRUB_BIDI_TYPE_R:
  case GRUB_BIDI_TYPE_AL:
-- 
2.25.4


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Dell BIOS issue reading Disk Extended data

2021-03-31 Thread Jordan Uggla 
This should *not* be made default because grub's native drivers don't
get very much testing, and they do cause grub to hang on some
hardware, but you can likely get around this limitation with:

grub-install --target=i386-pc --disk-module native /dev/sdX

If you want to test it without installing grub in this way to your
primary drive(s), you can boot from a grub USB and run "nativedisk" at
the grub shell. This will switch from firmware based disk access to
entirely grub native driver access (while accounting for the device
name changes and updating $prefix accordingly). For safety reasons,
this will disable firmware based access to all drives.

This works by including grub's native drivers in the embedded
core.img. For GPT disks (or arrays), which I assume you're using with
8TB capacity, grub's core.img is embedded in the BIOS Boot Partition.
As long as the BIOS Boot Partition is within 2TB of the beginning of
the disk (usually, it's just the first partition, and is 1MiB large)
you'll be able to load it just fine through INT13 and from that point
on (if your hardware is supported) you'll no longer be depending on
the boot firmware or be affected by its disk access bugs.


On Wed, Mar 31, 2021 at 7:45 AM Guilherme Piccoli
 wrote:
>
> On Tue, Mar 30, 2021 at 2:43 PM K, Narendra  wrote:
> > > On Fri, Jan 22, 2021 at 3:41 PM Limonciello, Mario
> > >  wrote:
> > > >
> > > > >
> > > > > Hello Dell folks, I'm Guilherme Piccoli from Canonical - first of
> > > > > all, apologies for the out-of-nowhere communication. We've been
> > > > > investigating an issue that seems to date long time ago, and
> > > > > eventually we could narrow it to what appears to be a Dell BIOS bug.
> > > > > Notice I'm also looping a kernel x86 ML and grub-devel, just for the
> > > > > purpose of archiving such discussion in public lists, to help others
> > > > > that may find such an issue in the future.
> > > > >
> > > > > Since I don't have contacts of Dell representatives, I've just
> > > > > raised a list of people from Dell contributing to kernel in the last
> > > > > 2 years - maybe one of you could point me towards the path of a
> > > > > proper contact/channel to discuss such an issue. If not, I'm sorry 
> > > > > for the
> > > noise.
> > > > > Let me detail the problem we're observing - notice all of this is
> > > > > about legacy BIOS mode, not UEFI.
> > > >
> > > > Most of the guys you CC'ed from Dell work on client related things,
> > > > not servers.  So I'll move some of the client guys off of this thread 
> > > > into BCC.
> > > > @K, Narendra can you please raise this with the appropriate team?
> > > >
> > >
> > > Thanks Mario and K, Narendra - do we have any news about that?
> > > Cheers,
> >
> > Hi Guilherme,
> >
> > Sorry for the delay in responding.
> >
>
> No problem Narendra K, thanks for your response!
>
> > I checked with the team about the findings shared in this thread. Please 
> > find the details -
> >
> > The behavior seems to be working as designed. Legacy BIOS Option ROM 
> > implements logical block addressing support only up to 2TB.
> > The option ROM will respond with 2TB as the size when queried by INT13h 
> > function 48h. For booting from volumes > 2TB, UEFI is needed.
>
> So, just confirming, Dell doesn't support > 2TB offsets even using the
> extended 64-bit offset from service 48h? I just want to confirm that
> is a support limitation from Dell, I've tested another vendor and that
> works in their machine, I could address > 2TB devices.
>
> In case this is unsupported by Dell, do you have an official
> documentation I can refer, to our customer?
> Thanks again,
>
> Guilherme
>
> ___
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: Dell BIOS issue reading Disk Extended data

2021-03-31 Thread Guilherme Piccoli 
On Tue, Mar 30, 2021 at 2:43 PM K, Narendra  wrote:
> > On Fri, Jan 22, 2021 at 3:41 PM Limonciello, Mario
> >  wrote:
> > >
> > > >
> > > > Hello Dell folks, I'm Guilherme Piccoli from Canonical - first of
> > > > all, apologies for the out-of-nowhere communication. We've been
> > > > investigating an issue that seems to date long time ago, and
> > > > eventually we could narrow it to what appears to be a Dell BIOS bug.
> > > > Notice I'm also looping a kernel x86 ML and grub-devel, just for the
> > > > purpose of archiving such discussion in public lists, to help others
> > > > that may find such an issue in the future.
> > > >
> > > > Since I don't have contacts of Dell representatives, I've just
> > > > raised a list of people from Dell contributing to kernel in the last
> > > > 2 years - maybe one of you could point me towards the path of a
> > > > proper contact/channel to discuss such an issue. If not, I'm sorry for 
> > > > the
> > noise.
> > > > Let me detail the problem we're observing - notice all of this is
> > > > about legacy BIOS mode, not UEFI.
> > >
> > > Most of the guys you CC'ed from Dell work on client related things,
> > > not servers.  So I'll move some of the client guys off of this thread 
> > > into BCC.
> > > @K, Narendra can you please raise this with the appropriate team?
> > >
> >
> > Thanks Mario and K, Narendra - do we have any news about that?
> > Cheers,
>
> Hi Guilherme,
>
> Sorry for the delay in responding.
>

No problem Narendra K, thanks for your response!

> I checked with the team about the findings shared in this thread. Please find 
> the details -
>
> The behavior seems to be working as designed. Legacy BIOS Option ROM 
> implements logical block addressing support only up to 2TB.
> The option ROM will respond with 2TB as the size when queried by INT13h 
> function 48h. For booting from volumes > 2TB, UEFI is needed.

So, just confirming, Dell doesn't support > 2TB offsets even using the
extended 64-bit offset from service 48h? I just want to confirm that
is a support limitation from Dell, I've tested another vendor and that
works in their machine, I could address > 2TB devices.

In case this is unsupported by Dell, do you have an official
documentation I can refer, to our customer?
Thanks again,

Guilherme

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel