On Thu, Aug 8, 2024 at 2:56 PM John Snow <js...@redhat.com> wrote:
>
>
>
> On Mon, Aug 5, 2024 at 4:17 PM Octavian Purdila <ta...@google.com> wrote:
>>
>> From: Stefan Stanacar <stefa...@google.com>
>>
>> From: Stefan Stanacar <stefa...@google.com>
>>
>> The CMSIS System View Description format(CMSIS-SVD) is an XML based
>> description of Arm Cortex-M microcontrollers provided and maintained
>> by sillicon vendors. It includes details such as peripherals registers
>> (down to bitfields), peripheral register block addresses, reset
>> values, etc.
>>
>> This script uses this information to create header files that makes it
>> easier to emulate peripherals.
>>
>> The script can be used to create either peripheral specific headers or
>> board / system specific information. The script generated headers are
>> similar to the SVDConv utility.
>>
>> Peripheral specific headers contains information such as register
>> layout, register names and reset values for registers:
>>
>> typedef struct {
>> ...
>> union {
>> uint32_t PSELID; /* 0x00000FF8 Peripheral Select and
>> * Flexcomm module ID */
>> struct {
>> uint32_t PERSEL : 3; /* [2..0] Peripheral Select */
>> uint32_t LOCK : 1; /* [3..3] Lock the peripheral select */
>> uint32_t USARTPRESENT : 1; /* [4..4] USART present indicator */
>> uint32_t SPIPRESENT : 1; /* [5..5] SPI present indicator */
>> uint32_t I2CPRESENT : 1; /* [6..6] I2C present indicator */
>> uint32_t I2SPRESENT : 1; /* [7..7] I2S Present */
>> uint32_t : 4;
>> uint32_t ID : 20; /* [31..12] Flexcomm ID */
>> } PSELID_b;
>> };
>> ...
>> } FLEXCOMM_Type; /* Size = 4096 (0x1000) */
>>
>> #define FLEXCOMM_PSELID_PERSEL_Pos (0UL)
>> #define FLEXCOMM_PSELID_PERSEL_Msk (0x7UL)
>> #define FLEXCOMM_PSELID_LOCK_Pos (3UL)
>> #define FLEXCOMM_PSELID_LOCK_Msk (0x8UL)
>> ...
>>
>> typedef enum { /* FLEXCOMM_PSELID_LOCK */
>> /* Peripheral select can be changed by software. */
>> FLEXCOMM_PSELID_LOCK_UNLOCKED = 0,
>> /* Peripheral select is locked and cannot be changed until this
>> * Flexcomm module or the entire device is reset. */
>> FLEXCOMM_PSELID_LOCK_LOCKED = 1,
>> } FLEXCOMM_PSELID_LOCK_Enum;
>> ...
>>
>> #define FLEXCOMM_REGISTER_NAMES_ARRAY(_name) \
>> const char *_name[sizeof(FLEXCOMM_Type)] = { \
>> [4088 ... 4091] = "PSELID", \
>> [4092 ... 4095] = "PID", \
>> }
>>
>> Board specific headers contains information about peripheral base
>> register addresses.
>>
>> Signed-off-by: Stefan Stanacar <stefa...@google.com>
>> Signed-off-by: Octavian Purdila <ta...@google.com>
>> ---
>> configure | 2 +-
>> meson.build | 4 +
>> python/setup.cfg | 1 +
>> python/tests/minreqs.txt | 3 +
>> pythondeps.toml | 3 +
>> scripts/svd-gen-header.py | 342 ++++++++++++++++++++++++++++++++++++++
>> 6 files changed, 354 insertions(+), 1 deletion(-)
>> create mode 100755 scripts/svd-gen-header.py
>>
>> diff --git a/configure b/configure
>> index 5ad1674ca5..811bfa5d54 100755
>> --- a/configure
>> +++ b/configure
>> @@ -956,7 +956,7 @@ mkvenv="$python ${source_path}/python/scripts/mkvenv.py"
>> # Finish preparing the virtual environment using vendored .whl files
>>
>> $mkvenv ensuregroup --dir "${source_path}/python/wheels" \
>> - ${source_path}/pythondeps.toml meson || exit 1
>> + ${source_path}/pythondeps.toml meson svd-gen-header || exit 1
>
Hi John,
Thanks for reviewing!
>
> I haven't read the rest of this series; I'm chiming in solely from the
> build/python maintainer angle. Do we *always* need pysvd, no matter how QEMU
> was configured? Adding it to the meson line here is a very big hammer.
>
I think the minimum we can do is to install it only if CONFIG_ARM is
enabled. That might change in the future if the models we create with
pysvd are enabled for other architectures.
> If not, consider looking at how sphinx (the "docs" group) is only
> conditionally installed into the configure venv and mimic that using the
> appropriate configure flags that necessitate the availability of pyvsd for
> the QEMU build.
Thanks for the pointer, I'll take a look.
>
> We also need to provide a way for pysvd to be available offline; some
> packages are available via distro libs and if this package is available for
> every distro we officially support, that's sufficient (but requires updates
> to our various docker and VM test configuration files to add the new
> dependency). Otherwise, like we do for meson, we need to vendor the wheel in
> the tree so offline tarball builds will continue to work.
>
> It looks like pysvd is a pure python package with no dependencies, so it
> should be OK to vendor it in qemu.git/python/wheels/ - look at
> qemu.git/python/scripts/vendor.py and consider updating and running this
> script.
Thanks, I'll look at it and add it in v2.
>
> (The real blocker here is that RPM builds are performed offline and
> dependencies that cannot be satisfied via rpm can't be added through pip. We
> need any one of these to be true: (A) pyvsd is available (of a sufficient
> version) in all distro repositories we target; (B) This build feature is
> conditional and nobody minds if it never gets enabled for RPM builds; (C) The
> package can be vendored.)
>
> ~~js
>
> That said, you might be the first person I've seen outside of Paolo and I to
> brave mucking around with the python build venv. You deserve a bravery
> sticker :)
>
>>
>> # At this point, we expect Meson to be installed and available.
>> # We expect mkvenv or pip to have created pyvenv/bin/meson for us.
>> diff --git a/meson.build b/meson.build
>> index ec59effca2..dee587483b 100644
>> --- a/meson.build
>> +++ b/meson.build
>> @@ -3235,6 +3235,10 @@ tracetool_depends = files(
>> 'scripts/tracetool/vcpu.py'
>> )
>>
>> +svd_gen_header = [
>> + python, files('scripts/svd-gen-header.py')
>> +]
>> +
>> qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
>> meson.current_source_dir(),
>> get_option('pkgversion'), meson.project_version()]
>> diff --git a/python/setup.cfg b/python/setup.cfg
>> index 48668609d3..bc830c541a 100644
>> --- a/python/setup.cfg
>> +++ b/python/setup.cfg
>> @@ -45,6 +45,7 @@ devel =
>> urwid >= 2.1.2
>> urwid-readline >= 0.13
>> Pygments >= 2.9.0
>> + pysvd >= 0.2.3
>>
>> # Provides qom-fuse functionality
>> fuse =
>> diff --git a/python/tests/minreqs.txt b/python/tests/minreqs.txt
>> index a3f423efd8..7993fcd23c 100644
>> --- a/python/tests/minreqs.txt
>> +++ b/python/tests/minreqs.txt
>> @@ -22,6 +22,9 @@ distlib==0.3.6
>> # Dependencies for FUSE support for qom-fuse
>> fusepy==2.0.4
>>
>> +# Dependencies for svd-gen-regs
>> +pysvd==0.2.3
>> +
>> # Test-runners, utilities, etc.
>> avocado-framework==90.0
>>
>> diff --git a/pythondeps.toml b/pythondeps.toml
>> index 9c16602d30..8416b17650 100644
>> --- a/pythondeps.toml
>> +++ b/pythondeps.toml
>> @@ -32,3 +32,6 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed =
>> "1.1.1" }
>> # avocado-framework, for example right now the limit is 92.x.
>> avocado-framework = { accepted = "(>=88.1, <93.0)", installed = "88.1",
>> canary = "avocado" }
>> pycdlib = { accepted = ">=1.11.0" }
>> +
>> +[svd-gen-header]
>> +pysvd = { accepted = ">=0.2.3.", installed = "0.2.3" }
>> diff --git a/scripts/svd-gen-header.py b/scripts/svd-gen-header.py
>> new file mode 100755
>> index 0000000000..ab8cb4b665
>> --- /dev/null
>> +++ b/scripts/svd-gen-header.py
>> @@ -0,0 +1,342 @@
>> +#!/usr/bin/env python3
>> +
>> +# Copyright 2024 Google LLC
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +# See the COPYING file in the top-level directory.
>> +#
>> +# Use this script to generate a C header file from an SVD xml
>> +#
>> +# Two mode of operations are supported: peripheral and system.
>> +#
>> +# When running in peripheral mode a header for a specific peripheral
>> +# is going to be generated. It will define a type and structure with
>> +# all of the available registers at the bitfield level. An array that
>> +# contains the reigster names indexed by address is also going to be
>> +# generated as well as a function to initialize registers to their
>> +# reset values.
>> +#
>> +# Invocation example:
>> +#
>> +# svd_gen_header -i MIMXRT595S_cm33.xml -o flexcomm.h -p FLEXCOMM0 -t
>> FLEXCOMM
>> +#
>> +# When running in system mode a header for a specific system /
>> +# platform will be generated. It will define register base addresses
>> +# and interrupt numbers for selected peripherals.
>> +#
>> +# Invocation example:
>> +#
>> +# svd_gen_header -i MIMXRT595S_cm33.xml -o rt500.h -s RT500 -p FLEXCOMM0 \
>> +# -p CLKCTL0 -p CLKCTL1
>> +#
>> +
>> +import argparse
>> +import datetime
>> +import re
>> +import os
>> +import sys
>> +import xml.etree.ElementTree
>> +import pysvd
>> +
>> +data_type_by_bits = {
>> + 8: "uint8_t",
>> + 16: "uint16_t",
>> + 32: "uint32_t",
>> +}
>> +
>> +
>> +def get_register_array_name_and_size(register):
>> + """Return register name and register array size.
>> +
>> + The SVD can define register arrays and pysvd encodes the whole set
>> + as as regular register with their name prepended by [<array size>].
>> +
>> + Returns a tuple with the register name and the size of the array or
>> + zero if this is not a register set.
>> +
>> + """
>> +
>> + split = re.split(r"[\[\]]", register.name)
>> + return (split[0], int(split[1]) if len(split) > 1 else 0)
>> +
>> +
>> +def generate_register(register):
>> + """Generate register data.
>> +
>> + This include a field for accessing the full 32bits as we as
>> + bitfield based register fields.
>> +
>> + """
>> +
>> + data_type = data_type_by_bits[register.size]
>> +
>> + out = f" /* 0x{register.addressOffset:08X} {register.description}
>> */\n"
>> + out += " union {\n"
>> + out += f" {data_type} {register.name};\n"
>> + out += " struct {\n"
>> +
>> + fields = sorted(register.fields, key=lambda field: field.bitOffset)
>> + last_msb = -1
>> + for field in fields:
>> + reserve_bits = 0
>> + lsb = field.bitOffset
>> + msb = field.bitWidth + lsb - 1
>> +
>> + if last_msb == -1 and lsb > 0:
>> + reserve_bits = lsb
>> +
>> + if last_msb != -1 and (lsb - last_msb) > 1:
>> + reserve_bits = lsb - last_msb - 1
>> +
>> + if reserve_bits > 0:
>> + out += f" {data_type}:{reserve_bits};\n"
>> +
>> + out += f" /* [{msb}..{lsb}] {field.description} */\n"
>> + out += f" {data_type} {field.name} : {field.bitWidth};\n"
>> +
>> + last_msb = msb
>> +
>> + if register.size - last_msb > 1:
>> + out += f" {data_type}:{register.size - last_msb - 1};\n"
>> +
>> + reg_name, reg_array_size = get_register_array_name_and_size(register)
>> + if reg_array_size > 0:
>> + out += f" }} {reg_name}_b[{reg_array_size}];\n"
>> + else:
>> + out += f" }} {reg_name}_b;\n"
>> + out += " };\n\n"
>> +
>> + return out
>> +
>> +
>> +def generate_pos_and_msk_defines(name, periph):
>> + """Generate Pos and Msk defines"""
>> +
>> + out = "\n"
>> + for reg in periph.registers:
>> + for field in reg.fields:
>> + reg_name, _ = get_register_array_name_and_size(reg)
>> + field_name = f"{name}_{reg_name}_{field.name}"
>> + out += f"#define {field_name}_Pos ({field.bitOffset}UL)\n"
>> + mask = ((1 << field.bitWidth) - 1) << field.bitOffset
>> + out += f"#define {field_name}_Msk (0x{mask:x}UL)\n"
>> +
>> + return out
>> +
>> +
>> +def generate_enum_values(name, periph):
>> + """Generate enum values"""
>> +
>> + out = "\n"
>> + for reg in periph.registers:
>> + reg_name, _ = get_register_array_name_and_size(reg)
>> + for field in reg.fields:
>> + if hasattr(field, "enumeratedValues"):
>> + out += "\n"
>> + out += "typedef enum {\n"
>> + for enum in field.enumeratedValues.enumeratedValues:
>> + enum_name =
>> f"{name}_{reg_name}_{field.name}_{enum.name}"
>> + out += f" /* {enum.description} */\n"
>> + out += f" {enum_name} = {enum.value},\n"
>> + out += f"}} {name}_{reg_name}_{field.name}_Enum;\n"
>> +
>> + return out
>> +
>> +
>> +def generate_register_names_array_macro(name, periph):
>> + """Generate register names array macro"""
>> +
>> + out = f"#define {name}_REGISTER_NAMES_ARRAY(_name) \\\n"
>> + out += f" const char *_name[sizeof({name}_Type)] = {{ \\\n"
>> + for reg in periph.registers:
>> + reg_name, reg_array_size = get_register_array_name_and_size(reg)
>> + if reg_array_size > 0:
>> + for i in range(0, reg_array_size):
>> + start = reg.addressOffset + i * reg.size // 8
>> + stop = reg.addressOffset + (i + 1) * reg.size // 8 - 1
>> + out += f' [{start} ... {stop}] = "{reg_name}{i}",
>> \\\n'
>> + else:
>> + start = reg.addressOffset
>> + stop = reg.addressOffset + reg.size // 8 - 1
>> + out += f' [{start} ... {stop}] = "{reg.name}", \\\n'
>> + out += " }\n"
>> +
>> + return out
>> +
>> +
>> +def generate_reset_registers_function(name, periph):
>> + """Generate reset registers function"""
>> +
>> + out = "\n"
>> + fname = f"{name.lower()}_reset_registers"
>> + out += f"static inline void {fname}({name}_Type *regs)\n"
>> + out += "{\n"
>> + for reg in periph.registers:
>> + reg_name, reg_array_size = get_register_array_name_and_size(reg)
>> + if reg_array_size > 0:
>> + for i in range(0, int(reg_array_size)):
>> + out += f" regs->{reg_name}[{i}] =
>> {hex(reg.resetValue)};\n"
>> + else:
>> + out += f" regs->{reg_name} = {hex(reg.resetValue)};\n"
>> + out += "}\n"
>> +
>> + return out
>> +
>> +
>> +def generate_peripheral_header(periph, name):
>> + """Generate peripheral header
>> +
>> + The following information is generated:
>> +
>> + * typedef with all of the available registers and register fields,
>> + position and mask defines for register fields.
>> +
>> + * enum values that encode register fields options.
>> +
>> + * a macro that defines the register names indexed by the relative
>> + address of the register.
>> +
>> + * a function that sets the registers to their reset values
>> +
>> + """
>> +
>> + out = f"/* {name} - {periph.description} */\n\n"
>> + out += "typedef struct {\n"
>> +
>> + last_reg_offset = 0
>> + cnt = 0
>> + for reg in periph.registers:
>> + reserved_words = 0
>> + if (reg.addressOffset - last_reg_offset) > 0:
>> + reserved_words = int((reg.addressOffset - last_reg_offset) // 4)
>> + cnt += 1
>> +
>> + if cnt:
>> + show_count = cnt
>> + else:
>> + show_count = ""
>> +
>> + if reserved_words == 1:
>> + out += f" uint32_t RESERVED{show_count};\n\n"
>> + elif reserved_words > 1:
>> + out += f" uint32_t
>> RESERVED{show_count}[{reserved_words}];\n\n"
>> +
>> + out += generate_register(reg)
>> + last_reg_offset = reg.addressOffset + reg.size // 8
>> +
>> + size = periph.addressBlocks[0].size
>> + out += f"}} {name}_Type; /* Size = {size} (0x{size:X}) */\n"
>> +
>> + out += "\n\n"
>> +
>> + out += generate_pos_and_msk_defines(name, periph)
>> +
>> + out += generate_enum_values(name, periph)
>> +
>> + out += generate_register_names_array_macro(name, periph)
>> +
>> + out += generate_reset_registers_function(name, periph)
>> +
>> + return out
>> +
>> +
>> +def get_same_class_peripherals(svd, periph):
>> + """Get a list of peripherals that are instances of the same class."""
>> +
>> + return [periph] + [
>> + p
>> + for p in svd.peripherals
>> + if p.derivedFrom and p.derivedFrom.name == periph.name
>> + ]
>> +
>> +
>> +def generate_system_header(system, svd, periph):
>> + """Generate base and irq defines for given list of peripherals"""
>> +
>> + out = ""
>> +
>> + for p in get_same_class_peripherals(svd, periph):
>> + out += f"#define {system}_{p.name}_BASE 0x{p.baseAddress:X}UL\n"
>> + out += "\n"
>> +
>> + for p in get_same_class_peripherals(svd, periph):
>> + for irq in p.interrupts:
>> + out += f"#define {system}_{irq.name}_IRQn 0x{irq.value}UL\n"
>> + out += "\n"
>> +
>> + return out
>> +
>> +
>> +def main():
>> + """Script to generate C header file from an SVD file"""
>> +
>> + parser = argparse.ArgumentParser()
>> + parser.add_argument(
>> + "-i", "--input", type=str, help="Input SVD file", required=True
>> + )
>> + parser.add_argument(
>> + "-o", "--output", type=str, help="Output .h file", required=True
>> + )
>> + parser.add_argument(
>> + "-p",
>> + "--peripheral",
>> + action="append",
>> + help="peripheral name from the SVD file",
>> + required=True,
>> + )
>> + parser.add_argument(
>> + "-t",
>> + "--type-name",
>> + type=str,
>> + help="name to be used for peripheral definitions",
>> + required=False,
>> + )
>> + parser.add_argument(
>> + "-s",
>> + "--system",
>> + type=str,
>> + help="name to be used for the system definitions",
>> + required=False,
>> + )
>> +
>> + args = parser.parse_args()
>> +
>> + node = xml.etree.ElementTree.parse(args.input).getroot()
>> + svd = pysvd.element.Device(node)
>> +
>> + # Write license header
>> + out = "/*\n"
>> + license_text = svd.licenseText.split("\\n")
>> + for line in license_text:
>> + sline = f" * {line.strip()}"
>> + out += f"{sline.rstrip()}\n"
>> +
>> + now = datetime.datetime.utcnow().strftime("%Y-%m-%d %H:%M:%S")
>> + out += f" * @note Automatically generated by
>> {os.path.basename(__file__)}"
>> + out += f" on {now} UTC from {os.path.basename(args.input)}.\n"
>> + out += " *\n */\n\n"
>> +
>> + # Write some generic defines
>> + out += "#pragma once\n\n"
>> +
>> + for name in args.peripheral:
>> + periph = svd.find(name)
>> + if periph:
>> + if args.system:
>> + out += generate_system_header(args.system, svd, periph)
>> + else:
>> + out += generate_peripheral_header(
>> + periph, args.type_name if args.type_name else
>> periph.name
>> + )
>> + else:
>> + print(f"No such peripheral: {name}")
>> + return 1
>> +
>> + with open(args.output, "w", encoding="ascii") as output:
>> + output.write(out)
>> +
>> + return 0
>> +
>> +
>> +if __name__ == "__main__":
>> + sys.exit(main())
>> --
>> 2.46.0.rc2.264.g509ed76dc8-goog
>>
