Re: [PATCH 4/5] configure: Add dtc option
Am 12. Juni 2025 05:46:03 UTC schrieb Paolo Bonzini :
>On 6/12/25 07:12, Paolo Bonzini wrote:
>> I'll send a patch to the upstream dtc project and you can add it to QEMU via
>> diff_files (see commit 64644bc4eab2f for an example).
>This is the patch, backported to QEMU's version of the dtc subproject:
>
>diff --git a/libfdt/meson.build b/libfdt/meson.build
>index 0307ffb..6581965 100644
>--- a/libfdt/meson.build
>+++ b/libfdt/meson.build
>@@ -30,6 +30,7 @@ libfdt_dep = declare_dependency(
> include_directories: libfdt_inc,
> link_with: libfdt,
> )
>+meson.override_dependency('libfdt', libfdt_dep)
> install_headers(
> files(
>diff --git a/meson.build b/meson.build
>index b23ea1b..7def0a6 100644
>--- a/meson.build
>+++ b/meson.build
>@@ -54,6 +54,7 @@ version_gen_h = vcs_tag(
> subdir('libfdt')
> +dtc_tools = []
> if get_option('tools')
> flex = find_program('flex', required: true)
> bison = find_program('bison', required: true)
>@@ -77,7 +78,7 @@ if get_option('tools')
> )
>if cc.check_header('fnmatch.h')
>-executable(
>+dtc_tools += executable(
> 'convert-dtsv0',
> [
> lgen.process('convert-dtsv0-lexer.l'),
>@@ -88,7 +89,7 @@ if get_option('tools')
> )
> endif
> - executable(
>+ dtc_tools += executable(
> 'dtc',
> [
> lgen.process('dtc-lexer.l'),
>@@ -108,7 +109,7 @@ if get_option('tools')
> )
>foreach e: ['fdtdump', 'fdtget', 'fdtput', 'fdtoverlay']
>-executable(e, files(e + '.c'), dependencies: util_dep, install: true)
>+dtc_tools += executable(e, files(e + '.c'), dependencies: util_dep,
>install: true)
> endforeach
>install_data(
>@@ -118,6 +119,10 @@ if get_option('tools')
> )
> endif
> +foreach e: dtc_tools
>+ meson.override_find_program(e.name(), e)
>+endforeach
>+
> if not meson.is_cross_build()
> if py.found() and swig.found()
> subdir('pylibfdt')
>
>
>and this is how you can then find dtc in QEMU's meson.build:
>
>diff --git a/meson.build b/meson.build
>index 61595015802..831678b4580 100644
>--- a/meson.build
>+++ b/meson.build
>@@ -2121,13 +2121,15 @@ if numa.found() and not cc.links('''
> endif
> fdt = not_found
>+dtc = not_found
> fdt_opt = get_option('fdt')
> if fdt_opt == 'enabled' and get_option('wrap_mode') == 'nodownload'
> fdt_opt = 'system'
> endif
> if fdt_opt in ['enabled', 'system'] or (fdt_opt == 'auto' and have_system)
> fdt = cc.find_library('fdt', required: fdt_opt == 'system')
>- if fdt.found() and cc.links('''
>+ dtc = find_program('dtc', required: fdt_opt == 'system')
>+ if dtc.found() and fdt.found() and cc.links('''
> #include
> #include
> int main(void) { fdt_find_max_phandle(NULL, NULL); return 0; }''',
>@@ -2136,8 +2138,12 @@ if fdt_opt in ['enabled', 'system'] or (fdt_opt ==
>'auto' and have_system)
> elif fdt_opt != 'system'
> fdt_opt = get_option('wrap_mode') == 'nodownload' ? 'disabled' :
> 'internal'
> fdt = not_found
>+dtc = not_found
> else
>-error('system libfdt is too old (1.5.1 or newer required)')
>+if dtc.found()
>+ error('system libfdt is too old (1.5.1 or newer required)')
>+else
>+ error('device tree compiler not found')
> endif
> endif
> if fdt_opt == 'internal'
>@@ -2145,7 +2148,8 @@ if fdt_opt == 'internal'
> libfdt_proj = subproject('dtc', required: true,
>default_options: ['tools=false', 'yaml=disabled',
> 'python=disabled',
> 'default_library=static'])
>- fdt = libfdt_proj.get_variable('libfdt_dep')
>+ fdt = dependency('libfdt', required: true)
>+ dtc = find_program('dtc', required: true)
> endif
> rdma = not_found
>
Thanks Paolo, I'll send a v2.
Best regards,
Bernhard
Re: [PATCH 4/5] configure: Add dtc option
On Thu, 12 Jun 2025, Paolo Bonzini wrote: On 6/11/25 13:45, BALATON Zoltan wrote: On Wed, 11 Jun 2025, Daniel P. Berrangé wrote: On Wed, Jun 11, 2025 at 02:37:08AM +0200, BALATON Zoltan wrote: On Tue, 10 Jun 2025, Bernhard Beschow wrote: The next patch will make dtc mandatory for boards with a bundled DTB, causing these boards to be omitted when dtc is missing. Allow packagers to force the build of those boards by providing an option. I don't like this. This might result in some machines missing from some distros if they don't notice this and forget to update their packages and these are machines I care about. "Someone might not notice & screw up" is pretty much an argument for not doing anything at all. We publish release notes to alert distros More like "if it ain't broke don't fix it". to these kind of changes which is a reasonable effort on our part. Distros need to play their part by reading them and adapting to the changes. They can also compare the available options with new and old QEMU to identify any new build features they need to handle when updating. Also what guarantees that we don't get bugs by compiling the device tree with different dtc versions which might generate different binaries? The binaries we have now are known to work and don't take much space so keeping them for convenience might be simpler. If it worked so far why change it now? Ultimately the DTC project has responsibility to not cause regressions in new releases they issue, as is the case for any 3rd party dep that we rely on. We have CI jobs that can run on different distros. If we have coverage of the boards in functional test suite that provides a way to validate the operation with differing DTC versions as a sanity check. Distributing pre-built binaries in source release tarballs is a long standing issue with QEMU's releases. If we can eliminate the pre-bbuilt DTB and only ship the source that is a good thing IMHO. If this was using dtc from the submodule when not using system libfdt then at least it would not result in dropped machines and we had a known working dtc to compile with. Why isn't that used when we already have it for libfdt? Are there any platforms that we target which still lack dtc ? The submodule was there to plug gaps in distro support long ago, but if every distro has dtc available the submodule is redundant. OK, can we then make libfdt and dtc mandatory and not build at all instead of disabling boards without it? Disabling libfdt was done because of the x86 microvm board, which was adding a libfdt dependency to all of qemu-system-x86_64. However, several targets still have a hard dependency on libfdt, plus libfdt is supported as a subproject and will be built together with the emulators. So the impact of the change was minimal, and limited only to people using --disable-libfdt or --disable-download on the configure command line. This is not true of Bernard's proposal to use dtc. For now, one possibility is just to make dtc mandatory whenever one of this boards is used; the best would be to be able to use the dtc that comes from the libfdt subproject. This however requires patching dtc's meson.build so that it calls meson.override_find_program (and it could also use meson.override_dependency while you're at it). I'll send a patch to the upstream dtc project and you can add it to QEMU via diff_files (see commit 64644bc4eab2f for an example). This seems getting too complicated just to get rid of some small binaries so I'd just leave it as it is now but if Bernhard wants to make this change I'm OK with it as long as my boards won't get disabled in configs where now they are built I.e. if no system dtc is found now the dtbs are installed; if this used dtc from submodule instead that would be OK but dropping the board is not. I'm also OK with replacing the submodule with always requiring system libfdt and dtc but that may be more work for distros to update their package dependencies. Regards, BALATON Zoltan
Re: [PATCH 4/5] configure: Add dtc option
On 6/12/25 07:12, Paolo Bonzini wrote:
I'll send a patch to the upstream dtc project and you can add it to QEMU
via diff_files (see commit 64644bc4eab2f for an example).
This is the patch, backported to QEMU's version of the dtc subproject:
diff --git a/libfdt/meson.build b/libfdt/meson.build
index 0307ffb..6581965 100644
--- a/libfdt/meson.build
+++ b/libfdt/meson.build
@@ -30,6 +30,7 @@ libfdt_dep = declare_dependency(
include_directories: libfdt_inc,
link_with: libfdt,
)
+meson.override_dependency('libfdt', libfdt_dep)
install_headers(
files(
diff --git a/meson.build b/meson.build
index b23ea1b..7def0a6 100644
--- a/meson.build
+++ b/meson.build
@@ -54,6 +54,7 @@ version_gen_h = vcs_tag(
subdir('libfdt')
+dtc_tools = []
if get_option('tools')
flex = find_program('flex', required: true)
bison = find_program('bison', required: true)
@@ -77,7 +78,7 @@ if get_option('tools')
)
if cc.check_header('fnmatch.h')
-executable(
+dtc_tools += executable(
'convert-dtsv0',
[
lgen.process('convert-dtsv0-lexer.l'),
@@ -88,7 +89,7 @@ if get_option('tools')
)
endif
- executable(
+ dtc_tools += executable(
'dtc',
[
lgen.process('dtc-lexer.l'),
@@ -108,7 +109,7 @@ if get_option('tools')
)
foreach e: ['fdtdump', 'fdtget', 'fdtput', 'fdtoverlay']
-executable(e, files(e + '.c'), dependencies: util_dep, install: true)
+dtc_tools += executable(e, files(e + '.c'), dependencies: util_dep,
install: true)
endforeach
install_data(
@@ -118,6 +119,10 @@ if get_option('tools')
)
endif
+foreach e: dtc_tools
+ meson.override_find_program(e.name(), e)
+endforeach
+
if not meson.is_cross_build()
if py.found() and swig.found()
subdir('pylibfdt')
and this is how you can then find dtc in QEMU's meson.build:
diff --git a/meson.build b/meson.build
index 61595015802..831678b4580 100644
--- a/meson.build
+++ b/meson.build
@@ -2121,13 +2121,15 @@ if numa.found() and not cc.links('''
endif
fdt = not_found
+dtc = not_found
fdt_opt = get_option('fdt')
if fdt_opt == 'enabled' and get_option('wrap_mode') == 'nodownload'
fdt_opt = 'system'
endif
if fdt_opt in ['enabled', 'system'] or (fdt_opt == 'auto' and have_system)
fdt = cc.find_library('fdt', required: fdt_opt == 'system')
- if fdt.found() and cc.links('''
+ dtc = find_program('dtc', required: fdt_opt == 'system')
+ if dtc.found() and fdt.found() and cc.links('''
#include
#include
int main(void) { fdt_find_max_phandle(NULL, NULL); return 0; }''',
@@ -2136,8 +2138,12 @@ if fdt_opt in ['enabled', 'system'] or (fdt_opt ==
'auto' and have_system)
elif fdt_opt != 'system'
fdt_opt = get_option('wrap_mode') == 'nodownload' ? 'disabled' : 'internal'
fdt = not_found
+dtc = not_found
else
-error('system libfdt is too old (1.5.1 or newer required)')
+if dtc.found()
+ error('system libfdt is too old (1.5.1 or newer required)')
+else
+ error('device tree compiler not found')
endif
endif
if fdt_opt == 'internal'
@@ -2145,7 +2148,8 @@ if fdt_opt == 'internal'
libfdt_proj = subproject('dtc', required: true,
default_options: ['tools=false', 'yaml=disabled',
'python=disabled',
'default_library=static'])
- fdt = libfdt_proj.get_variable('libfdt_dep')
+ fdt = dependency('libfdt', required: true)
+ dtc = find_program('dtc', required: true)
endif
rdma = not_found
Re: [PATCH 4/5] configure: Add dtc option
On 6/11/25 13:45, BALATON Zoltan wrote: On Wed, 11 Jun 2025, Daniel P. Berrangé wrote: On Wed, Jun 11, 2025 at 02:37:08AM +0200, BALATON Zoltan wrote: On Tue, 10 Jun 2025, Bernhard Beschow wrote: The next patch will make dtc mandatory for boards with a bundled DTB, causing these boards to be omitted when dtc is missing. Allow packagers to force the build of those boards by providing an option. I don't like this. This might result in some machines missing from some distros if they don't notice this and forget to update their packages and these are machines I care about. "Someone might not notice & screw up" is pretty much an argument for not doing anything at all. We publish release notes to alert distros More like "if it ain't broke don't fix it". to these kind of changes which is a reasonable effort on our part. Distros need to play their part by reading them and adapting to the changes. They can also compare the available options with new and old QEMU to identify any new build features they need to handle when updating. Also what guarantees that we don't get bugs by compiling the device tree with different dtc versions which might generate different binaries? The binaries we have now are known to work and don't take much space so keeping them for convenience might be simpler. If it worked so far why change it now? Ultimately the DTC project has responsibility to not cause regressions in new releases they issue, as is the case for any 3rd party dep that we rely on. We have CI jobs that can run on different distros. If we have coverage of the boards in functional test suite that provides a way to validate the operation with differing DTC versions as a sanity check. Distributing pre-built binaries in source release tarballs is a long standing issue with QEMU's releases. If we can eliminate the pre-bbuilt DTB and only ship the source that is a good thing IMHO. If this was using dtc from the submodule when not using system libfdt then at least it would not result in dropped machines and we had a known working dtc to compile with. Why isn't that used when we already have it for libfdt? Are there any platforms that we target which still lack dtc ? The submodule was there to plug gaps in distro support long ago, but if every distro has dtc available the submodule is redundant. OK, can we then make libfdt and dtc mandatory and not build at all instead of disabling boards without it? Disabling libfdt was done because of the x86 microvm board, which was adding a libfdt dependency to all of qemu-system-x86_64. However, several targets still have a hard dependency on libfdt, plus libfdt is supported as a subproject and will be built together with the emulators. So the impact of the change was minimal, and limited only to people using --disable-libfdt or --disable-download on the configure command line. This is not true of Bernard's proposal to use dtc. For now, one possibility is just to make dtc mandatory whenever one of this boards is used; the best would be to be able to use the dtc that comes from the libfdt subproject. This however requires patching dtc's meson.build so that it calls meson.override_find_program (and it could also use meson.override_dependency while you're at it). I'll send a patch to the upstream dtc project and you can add it to QEMU via diff_files (see commit 64644bc4eab2f for an example). Thanks, Paolo
Re: [PATCH 4/5] configure: Add dtc option
On Wed, 11 Jun 2025, Daniel P. Berrangé wrote: On Wed, Jun 11, 2025 at 02:37:08AM +0200, BALATON Zoltan wrote: On Tue, 10 Jun 2025, Bernhard Beschow wrote: The next patch will make dtc mandatory for boards with a bundled DTB, causing these boards to be omitted when dtc is missing. Allow packagers to force the build of those boards by providing an option. I don't like this. This might result in some machines missing from some distros if they don't notice this and forget to update their packages and these are machines I care about. "Someone might not notice & screw up" is pretty much an argument for not doing anything at all. We publish release notes to alert distros More like "if it ain't broke don't fix it". to these kind of changes which is a reasonable effort on our part. Distros need to play their part by reading them and adapting to the changes. They can also compare the available options with new and old QEMU to identify any new build features they need to handle when updating. Also what guarantees that we don't get bugs by compiling the device tree with different dtc versions which might generate different binaries? The binaries we have now are known to work and don't take much space so keeping them for convenience might be simpler. If it worked so far why change it now? Ultimately the DTC project has responsibility to not cause regressions in new releases they issue, as is the case for any 3rd party dep that we rely on. We have CI jobs that can run on different distros. If we have coverage of the boards in functional test suite that provides a way to validate the operation with differing DTC versions as a sanity check. Distributing pre-built binaries in source release tarballs is a long standing issue with QEMU's releases. If we can eliminate the pre-bbuilt DTB and only ship the source that is a good thing IMHO. If this was using dtc from the submodule when not using system libfdt then at least it would not result in dropped machines and we had a known working dtc to compile with. Why isn't that used when we already have it for libfdt? Are there any platforms that we target which still lack dtc ? The submodule was there to plug gaps in distro support long ago, but if every distro has dtc available the submodule is redundant. OK, can we then make libfdt and dtc mandatory and not build at all instead of disabling boards without it? I only care that my boards will be in distro packages so users don't have to compile and I don't have to provide binaries which would also be ensured if we require dtc, then the dtbs can be dropped. There's still a potential issue that some of the dts files are old and may not compile with newer dtc and changing them may upset the guest but hopefully we won't get that. Regards, BALATON Zoltan
Re: [PATCH 4/5] configure: Add dtc option
On Wed, Jun 11, 2025 at 02:37:08AM +0200, BALATON Zoltan wrote: > On Tue, 10 Jun 2025, Bernhard Beschow wrote: > > The next patch will make dtc mandatory for boards with a bundled DTB, > > causing > > these boards to be omitted when dtc is missing. Allow packagers to force the > > build of those boards by providing an option. > > I don't like this. This might result in some machines missing from some > distros if they don't notice this and forget to update their packages and > these are machines I care about. "Someone might not notice & screw up" is pretty much an argument for not doing anything at all. We publish release notes to alert distros to these kind of changes which is a reasonable effort on our part. Distros need to play their part by reading them and adapting to the changes. They can also compare the available options with new and old QEMU to identify any new build features they need to handle when updating. > Also what guarantees that we don't get bugs > by compiling the device tree with different dtc versions which might > generate different binaries? The binaries we have now are known to work and > don't take much space so keeping them for convenience might be simpler. If > it worked so far why change it now? Ultimately the DTC project has responsibility to not cause regressions in new releases they issue, as is the case for any 3rd party dep that we rely on. We have CI jobs that can run on different distros. If we have coverage of the boards in functional test suite that provides a way to validate the operation with differing DTC versions as a sanity check. Distributing pre-built binaries in source release tarballs is a long standing issue with QEMU's releases. If we can eliminate the pre-bbuilt DTB and only ship the source that is a good thing IMHO. > If this was using dtc from the submodule when not using system libfdt then > at least it would not result in dropped machines and we had a known working > dtc to compile with. Why isn't that used when we already have it for libfdt? Are there any platforms that we target which still lack dtc ? The submodule was there to plug gaps in distro support long ago, but if every distro has dtc available the submodule is redundant. With regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PATCH 4/5] configure: Add dtc option
On Tue, 10 Jun 2025, Bernhard Beschow wrote:
The next patch will make dtc mandatory for boards with a bundled DTB, causing
these boards to be omitted when dtc is missing. Allow packagers to force the
build of those boards by providing an option.
I don't like this. This might result in some machines missing from some
distros if they don't notice this and forget to update their packages and
these are machines I care about. Also what guarantees that we don't get
bugs by compiling the device tree with different dtc versions which might
generate different binaries? The binaries we have now are known to work
and don't take much space so keeping them for convenience might be
simpler. If it worked so far why change it now?
If this was using dtc from the submodule when not using system libfdt then
at least it would not result in dropped machines and we had a known
working dtc to compile with. Why isn't that used when we already have it
for libfdt?
Regards,
BALATON Zoltan
Signed-off-by: Bernhard Beschow
---
meson.build | 12
Kconfig.host | 3 +++
meson_options.txt | 3 +++
pc-bios/dtb/meson.build | 1 -
scripts/meson-buildoptions.sh | 3 +++
5 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/meson.build b/meson.build
index 34729c2a3d..8a03fbaed8 100644
--- a/meson.build
+++ b/meson.build
@@ -2119,6 +2119,15 @@ if fdt_opt == 'internal'
fdt = libfdt_proj.get_variable('libfdt_dep')
endif
+dtc = not_found
+dtc_opt = get_option('dtc')
+if dtc_opt == 'enabled' or (dtc_opt == 'auto' and have_system)
+ dtc = find_program('dtc', required: dtc_opt == 'enabled')
+ if not dtc.found() and dtc_opt == 'enabled'
+error('dtc not found')
+ endif
+endif
+
rdma = not_found
if not get_option('rdma').auto() or have_system
rdma_libs = [cc.find_library('rdmacm', has_headers: ['rdma/rdma_cma.h'],
@@ -2477,6 +2486,7 @@ config_host_data.set('CONFIG_FREEBSD', host_os ==
'freebsd')
config_host_data.set('CONFIG_CAPSTONE', capstone.found())
config_host_data.set('CONFIG_COCOA', cocoa.found())
config_host_data.set('CONFIG_DARWIN', host_os == 'darwin')
+config_host_data.set('CONFIG_DTC', dtc.found())
config_host_data.set('CONFIG_FDT', fdt.found())
config_host_data.set('CONFIG_FUZZ', get_option('fuzzing'))
config_host_data.set('CONFIG_GCOV', get_option('b_coverage'))
@@ -3314,6 +3324,7 @@ host_kconfig = \
(libcbor.found() ? ['CONFIG_LIBCBOR=y'] : []) + \
(gnutls.found() ? ['CONFIG_GNUTLS=y'] : []) + \
(x11.found() ? ['CONFIG_X11=y'] : []) + \
+ (dtc.found() ? ['CONFIG_DTC=y'] : []) + \
(fdt.found() ? ['CONFIG_FDT=y'] : []) + \
(have_vhost_user ? ['CONFIG_VHOST_USER=y'] : []) + \
(have_vhost_vdpa ? ['CONFIG_VHOST_VDPA=y'] : []) + \
@@ -4703,6 +4714,7 @@ summary_info += {'sphinx-build': sphinx_build}
if config_host.has_key('GDB')
summary_info += {'gdb': config_host['GDB']}
endif
+summary_info += {'dtc': dtc}
summary_info += {'iasl': iasl}
summary_info += {'genisoimage': config_host['GENISOIMAGE']}
if host_os == 'windows' and have_ga
diff --git a/Kconfig.host b/Kconfig.host
index 933425c74b..82fc8848dd 100644
--- a/Kconfig.host
+++ b/Kconfig.host
@@ -29,6 +29,9 @@ config IVSHMEM
config TPM
bool
+config DTC
+bool
+
config FDT
bool
diff --git a/meson_options.txt b/meson_options.txt
index a442be2995..d0e56564fb 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -316,6 +316,9 @@ option('vduse_blk_export', type: 'feature', value: 'auto',
option('capstone', type: 'feature', value: 'auto',
description: 'Whether and how to find the capstone library')
+option('dtc', type: 'combo', value: 'auto',
+ choices: ['disabled', 'enabled', 'auto'],
+ description: 'Whether to find the device tree compiler')
option('fdt', type: 'combo', value: 'auto',
choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
description: 'Whether and how to find the libfdt library')
diff --git a/pc-bios/dtb/meson.build b/pc-bios/dtb/meson.build
index 993032949f..9fcdbeff14 100644
--- a/pc-bios/dtb/meson.build
+++ b/pc-bios/dtb/meson.build
@@ -5,7 +5,6 @@ dtbs = [
'petalogix-s3adsp1800.dtb',
]
-dtc = find_program('dtc', required: false)
if dtc.found()
foreach out : dtbs
f = fs.replace_suffix(out, '.dts')
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 73e0770f42..e8c452ffb0 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -116,6 +116,7 @@ meson_options_help() {
printf "%s\n" ' dmg dmg image format support'
printf "%s\n" ' docsDocumentations build support'
printf "%s\n" ' dsound DirectSound sound support'
+ printf "%s\n" ' dtc Whether to find the device tree compiler'
printf "%s\n" ' fuseFUSE block device export'
printf "%s\n" ' fuse-lseek SEEK_HOLE/SEEK_DATA support for FUSE exports'
printf "%s\n" ' gcrypt
