Enable the generation of symbol & fixup information for usage with dynamic DT loading.
Passing the -@ option generates a __symbols__ node at the root node of the resulting blob for any node labels used. When using the /plugin/ tag all unresolved label references be tracked in the __fixups__ node, while all local phandle references will the tracked in the __local_fixups__ node. This is sufficient to implement a dynamic DT object loader. Signed-off-by: Pantelis Antoniou <pa...@antoniou-consulting.com> --- Documentation/dt-object-internal.txt | 295 +++++++++++++++++++++++++++++++++++ Documentation/dts-format.txt | 7 + Documentation/manual.txt | 8 + checks.c | 120 +++++++++++++- dtc-lexer.l | 5 + dtc-parser.y | 23 ++- dtc.c | 9 +- dtc.h | 38 +++++ flattree.c | 139 +++++++++++++++++ 9 files changed, 635 insertions(+), 9 deletions(-) create mode 100644 Documentation/dt-object-internal.txt diff --git a/Documentation/dt-object-internal.txt b/Documentation/dt-object-internal.txt new file mode 100644 index 0000000..a851721 --- /dev/null +++ b/Documentation/dt-object-internal.txt @@ -0,0 +1,295 @@ +Device Tree Dynamic Object format internals +------------------------------------------- + +The Device Tree for most platforms is a static representation of +the hardware capabilities. This is insufficient for many platforms +that need to dynamically insert device tree fragments to the +running kernel's live tree. + +This document explains the the device tree object format and the +modifications made to the device tree compiler, which make it possible. + +1. Simplified Problem Definition +-------------------------------- + +Assume we have a platform which boots using this simplified device tree: + +---- foo.dts ----------------------------------------------------------------- + /* FOO platform */ + / { + compatible = "corp,foo"; + + /* shared resources */ + res: res { + }; + + /* On chip peripherals */ + ocp: ocp { + /* peripherals that are always instantiated */ + peripheral1 { ... }; + } + }; +---- foo.dts ----------------------------------------------------------------- + +We have a number of peripherals that after probing (using some undefined method) +should result in different device tree configuration. + +We cannot boot with this static tree because due to the configuration of the +foo platform there exist multiple conficting peripherals DT fragments. + +So for the bar peripheral we would have this: + +---- foo+bar.dts ------------------------------------------------------------- + /* FOO platform + bar peripheral */ + / { + compatible = "corp,foo"; + + /* shared resources */ + res: res { + }; + + /* On chip peripherals */ + ocp: ocp { + /* peripherals that are always instantiated */ + peripheral1 { ... }; + + /* bar peripheral */ + bar { + compatible = "corp,bar"; + ... /* various properties and child nodes */ + } + } + }; +---- foo+bar.dts ------------------------------------------------------------- + +While for the baz peripheral we would have this: + +---- foo+baz.dts ------------------------------------------------------------- + /* FOO platform + baz peripheral */ + / { + compatible = "corp,foo"; + + /* shared resources */ + res: res { + /* baz resources */ + baz_res: res_baz { ... }; + }; + + /* On chip peripherals */ + ocp: ocp { + /* peripherals that are always instantiated */ + peripheral1 { ... }; + + /* baz peripheral */ + baz { + compatible = "corp,baz"; + /* reference to another point in the tree */ + ref-to-res = <&baz_res>; + ... /* various properties and child nodes */ + } + } + }; +---- foo+baz.dts ------------------------------------------------------------- + +We note that the baz case is more complicated, since the baz peripheral needs to +reference another node in the DT tree. + +2. Device Tree Object Format Requirements +----------------------------------------- + +Since the device tree is used for booting a number of very different hardware +platforms it is imperative that we tread very carefully. + +2.a) No changes to the Device Tree binary format. We cannot modify the tree +format at all and all the information we require should be encoded using device +tree itself. We can add nodes that can be safely ignored by both bootloaders and +the kernel. + +2.b) Changes to the DTS source format should be absolutely minimal, and should +only be needed for the DT fragment definitions, and not the base boot DT. + +2.c) An explicit option should be used to instruct DTC to generate the required +information needed for object resolution. Platforms that don't use the +dynamic object format can safely ignore it. + +2.d) Finally, DT syntax changes should be kept to a minimum. It should be +possible to express everything using the existing DT syntax. + +3. Implementation +----------------- + +The basic unit of addressing in Device Tree is the phandle. Turns out it's +relatively simple to extend the way phandles are generated and referenced +so that it's possible to dynamically convert symbolic references (labels) +to phandle values. + +We can roughly divide the operation into two steps. + +3.a) Compilation of the base board DTS file using the '-@' option +generates a valid DT blob with an added __symbols__ node at the root node, +containing a list of all nodes that are marked with a label. + +Using the foo.dts file above the following node will be generated; + +$ dtc -@ -O dtb -o foo.dtb -b 0 foo.dts +$ fdtdump foo.dtb +... +/ { + ... + res { + ... + linux,phandle = <0x00000001>; + phandle = <0x00000001>; + ... + }; + ocp { + ... + linux,phandle = <0x00000002>; + phandle = <0x00000002>; + ... + }; + __symbols__ { + res="/res"; + ocp="/ocp"; + }; +}; + +Notice that all the nodes that had a label have been recorded, and that +phandles have been generated for them. + +This blob can be used to boot the board normally, the __symbols__ node will +be safely ignored both by the bootloader and the kernel (the only loss will +be a few bytes of memory and disk space). + +3.b) The Device Tree fragments must be compiled with the same option but they +must also have a tag (/plugin/) that allows undefined references to labels +that are not present at compilation time to be recorded so that the runtime +loader can fix them. + +So the bar peripheral's DTS format would be of the form: + +/plugin/; /* allow undefined label references and record them */ +/ { + .... /* various properties for loader use; i.e. part id etc. */ + fragment@0 { + target = <&ocp>; + __overlay__ { + /* bar peripheral */ + bar { + compatible = "corp,bar"; + ... /* various properties and child nodes */ + } + }; + }; +}; + +Note that there's a target property that specifies the location where the +contents of the overlay node will be placed, and it references the label +in the foo.dts file. + +$ dtc -@ -O dtb -o bar.dtbo -b 0 bar.dts +$ fdtdump bar.dtbo +... +/ { + ... /* properties */ + fragment@0 { + target = <0xdeadbeef>; + __overlay__ { + bar { + compatible = "corp,bar"; + ... /* various properties and child nodes */ + } + }; + }; + __fixups__ { + ocp = "/fragment@0:target:0"; + }; +}; + +No __symbols__ has been generated (no label in bar.dts). +Note that the target's ocp label is undefined, so the phandle handle +value is filled with the illegal value '0xdeadbeef', while a __fixups__ +node has been generated, which marks the location in the tree where +the label lookup should store the runtime phandle value of the ocp node. + +The format of the __fixups__ node entry is + + <label> = "<local-full-path>:<property-name>:<offset>"; + +<label> Is the label we're referring +<local-full-path> Is the full path of the node the reference is +<property-name> Is the name of the property containing the + reference +<offset> The offset (in bytes) of where the property's + phandle value is located. + +Doing the same with the baz peripheral's DTS format is a little bit more +involved, since baz contains references to local labels which require +local fixups. + +/plugin/; /* allow undefined label references and record them */ +/ { + .... /* various properties for loader use; i.e. part id etc. */ + fragment@0 { + target = <&res>; + __overlay__ { + /* baz resources */ + baz_res: res_baz { ... }; + }; + }; + fragment@1 { + target = <&ocp>; + __overlay__ { + /* baz peripheral */ + baz { + compatible = "corp,baz"; + /* reference to another point in the tree */ + ref-to-res = <&baz_res>; + ... /* various properties and child nodes */ + } + }; + }; +}; + +Note that &baz_res reference. + +$ dtc -@ -O dtb -o baz.dtbo -b 0 baz.dts +$ fdtdump baz.dtbo +... +/ { + ... /* properties */ + fragment@0 { + target = <0xdeadbeef>; + __overlay__ { + res_baz { + .... + linux,phandle = <0x00000001>; + phandle = <0x00000001>; + }; + }; + }; + fragment@1 { + target = <0xdeadbeef>; + __overlay__ { + baz { + compatible = "corp,baz"; + ... /* various properties and child nodes */ + res=<0x00000001>; + } + }; + }; + __fixups__ { + res = "/fragment@0:target:0"; + ocp = "/fragment@1:target:0"; + }; + __local_fixups__ { + fixup = </fragment@1/__overlay__/baz:res:0>; + }; +}; + +This is similar to the bar case, but the reference of a local label by the +baz node generates a __local_fixups__ entry that records the place that the +local reference is being made. Since phandles are allocated starting at 1 +the run time loader must apply an offset to each phandle in every dynamic +DT object loaded. The __local_fixups__ node records the place of every +local reference so that the loader can apply the offset. diff --git a/Documentation/dts-format.txt b/Documentation/dts-format.txt index 41741df..4da515c 100644 --- a/Documentation/dts-format.txt +++ b/Documentation/dts-format.txt @@ -115,7 +115,14 @@ Version 1 DTS files have the overall layout: * C style (/* ... */) and C++ style (// ...) comments are supported. +Device Tree Objects +------------------- +Using the plugin tag enables dynamic tree objects. + + /plugin/; + +For the full details please see Documentation/dt-object-internal.txt -- David Gibson <da...@gibson.dropbear.id.au> -- Yoder Stuart <stuart.yo...@freescale.com> diff --git a/Documentation/manual.txt b/Documentation/manual.txt index 65c8540..d313715 100644 --- a/Documentation/manual.txt +++ b/Documentation/manual.txt @@ -133,6 +133,14 @@ Options: By default the most recent version is generated. Relevant for dtb and asm output only. + -@ + Dynamic resolution mode. For non /plugin/ compilations generate + a __symbols__ node containing a list of all nodes with a label. + When /plugin/ is used, unresolved references are recorded in + a __fixups__ node, while local phandle references are recorded + in a __local_fixups__ node. + See Documentation/dt-object-internal.txt + The <output_version> defines what version of the "blob" format will be generated. Supported versions are 1, 2, 3, 16 and 17. The default is diff --git a/checks.c b/checks.c index ee96a25..970c0a3 100644 --- a/checks.c +++ b/checks.c @@ -457,22 +457,93 @@ static void fixup_phandle_references(struct check *c, struct node *dt, struct node *node, struct property *prop) { struct marker *m = prop->val.markers; + struct fixup *f, **fp; + struct fixup_entry *fe, **fep; struct node *refnode; cell_t phandle; + int has_phandle_refs; + + has_phandle_refs = 0; + for_each_marker_of_type(m, REF_PHANDLE) { + has_phandle_refs = 1; + break; + } + + if (!has_phandle_refs) + return; for_each_marker_of_type(m, REF_PHANDLE) { assert(m->offset + sizeof(cell_t) <= prop->val.len); refnode = get_node_by_ref(dt, m->ref); - if (! refnode) { + if (!refnode && !symbol_fixup_support) { FAIL(c, "Reference to non-existent node or label \"%s\"\n", - m->ref); + m->ref); continue; } - phandle = get_node_phandle(dt, refnode); - *((cell_t *)(prop->val.val + m->offset)) = cpu_to_fdt32(phandle); + if (!refnode) { + /* allocate fixup entry */ + fe = xmalloc(sizeof(*fe)); + + fe->node = node; + fe->prop = prop; + fe->offset = m->offset; + fe->next = NULL; + + /* search for an already existing fixup */ + for_each_fixup(dt, f) + if (strcmp(f->ref, m->ref) == 0) + break; + + /* no fixup found, add new */ + if (f == NULL) { + f = xmalloc(sizeof(*f)); + f->ref = m->ref; + f->entries = NULL; + f->next = NULL; + + /* add it to the tree */ + fp = &dt->fixups; + while (*fp) + fp = &(*fp)->next; + *fp = f; + } + + /* and now append fixup entry */ + fep = &f->entries; + while (*fep) + fep = &(*fep)->next; + *fep = fe; + + /* mark the entry as unresolved */ + phandle = 0xdeadbeef; + } else { + phandle = get_node_phandle(dt, refnode); + + /* if it's a plugin, we need to record it */ + if (symbol_fixup_support && dt->is_plugin) { + + /* allocate a new local fixup entry */ + fe = xmalloc(sizeof(*fe)); + + fe->node = node; + fe->prop = prop; + fe->offset = m->offset; + fe->next = NULL; + + /* append it to the local fixups */ + fep = &dt->local_fixups; + while (*fep) + fep = &(*fep)->next; + *fep = fe; + } + } + + *((cell_t *)(prop->val.val + m->offset)) = + cpu_to_fdt32(phandle); } + } ERROR(phandle_references, NULL, NULL, fixup_phandle_references, NULL, &duplicate_node_names, &explicit_phandles); @@ -651,6 +722,45 @@ static void check_obsolete_chosen_interrupt_controller(struct check *c, } TREE_WARNING(obsolete_chosen_interrupt_controller, NULL); +static void check_auto_label_phandles(struct check *c, struct node *dt, + struct node *node) +{ + struct label *l; + struct symbol *s, **sp; + int has_label; + + if (!symbol_fixup_support) + return; + + has_label = 0; + for_each_label(node->labels, l) { + has_label = 1; + break; + } + + if (!has_label) + return; + + /* force allocation of a phandle for this node */ + (void)get_node_phandle(dt, node); + + /* add the symbol */ + for_each_label(node->labels, l) { + + s = xmalloc(sizeof(*s)); + s->label = l; + s->node = node; + s->next = NULL; + + /* add it to the symbols list */ + sp = &dt->symbols; + while (*sp) + sp = &((*sp)->next); + *sp = s; + } +} +NODE_WARNING(auto_label_phandles, NULL); + static struct check *check_table[] = { &duplicate_node_names, &duplicate_property_names, &node_name_chars, &node_name_format, &property_name_chars, @@ -669,6 +779,8 @@ static struct check *check_table[] = { &avoid_default_addr_size, &obsolete_chosen_interrupt_controller, + &auto_label_phandles, + &always_fail, }; diff --git a/dtc-lexer.l b/dtc-lexer.l index 254d5af..2cef406 100644 --- a/dtc-lexer.l +++ b/dtc-lexer.l @@ -112,6 +112,11 @@ static int pop_input_file(void); return DT_V1; } +<*>"/plugin/" { + DPRINT("Keyword: /plugin/\n"); + return DT_PLUGIN; + } + <*>"/memreserve/" { DPRINT("Keyword: /memreserve/\n"); BEGIN_DEFAULT(); diff --git a/dtc-parser.y b/dtc-parser.y index f412460..e444acf 100644 --- a/dtc-parser.y +++ b/dtc-parser.y @@ -20,6 +20,7 @@ %{ #include <stdio.h> +#include <inttypes.h> #include "dtc.h" #include "srcpos.h" @@ -56,9 +57,11 @@ static unsigned char eval_char_literal(const char *s); struct node *nodelist; struct reserve_info *re; uint64_t integer; + int is_plugin; } %token DT_V1 +%token DT_PLUGIN %token DT_MEMRESERVE %token DT_LSHIFT DT_RSHIFT DT_LE DT_GE DT_EQ DT_NE DT_AND DT_OR %token DT_BITS @@ -76,6 +79,7 @@ static unsigned char eval_char_literal(const char *s); %type <data> propdata %type <data> propdataprefix +%type <is_plugin> plugindecl %type <re> memreserve %type <re> memreserves %type <array> arrayprefix @@ -106,10 +110,23 @@ static unsigned char eval_char_literal(const char *s); %% sourcefile: - DT_V1 ';' memreserves devicetree + DT_V1 ';' plugindecl memreserves devicetree { - the_boot_info = build_boot_info($3, $4, - guess_boot_cpuid($4)); + $5->is_plugin = $3; + $5->is_root = 1; + the_boot_info = build_boot_info($4, $5, + guess_boot_cpuid($5)); + } + ; + +plugindecl: + /* empty */ + { + $$ = 0; + } + | DT_PLUGIN ';' + { + $$ = 1; } ; diff --git a/dtc.c b/dtc.c index a375683..30d9607 100644 --- a/dtc.c +++ b/dtc.c @@ -31,6 +31,7 @@ int reservenum; /* Number of memory reservation slots */ int minsize; /* Minimum blob size */ int padsize; /* Additional padding to blob */ int phandle_format = PHANDLE_BOTH; /* Use linux,phandle or phandle properties */ +int symbol_fixup_support = 0; static void fill_fullpaths(struct node *tree, const char *prefix) { @@ -96,6 +97,8 @@ static void __attribute__ ((noreturn)) usage(void) fprintf(stderr, "\t-W [no-]<checkname>\n"); fprintf(stderr, "\t-E [no-]<checkname>\n"); fprintf(stderr, "\t\t\tenable or disable warnings and errors\n"); + fprintf(stderr, "\t-@\n"); + fprintf(stderr, "\t\tSymbols and Fixups support\n"); exit(3); } @@ -118,7 +121,7 @@ int main(int argc, char *argv[]) minsize = 0; padsize = 0; - while ((opt = getopt(argc, argv, "hI:O:o:V:d:R:S:p:fqb:i:vH:sW:E:")) + while ((opt = getopt(argc, argv, "hI:O:o:V:d:R:S:p:fqb:i:vH:sW:E:@")) != EOF) { switch (opt) { case 'I': @@ -183,7 +186,9 @@ int main(int argc, char *argv[]) case 'E': parse_checks_option(false, true, optarg); break; - + case '@': + symbol_fixup_support = 1; + break; case 'h': default: usage(); diff --git a/dtc.h b/dtc.h index 3e42a07..0c8ecf6 100644 --- a/dtc.h +++ b/dtc.h @@ -54,6 +54,7 @@ extern int reservenum; /* Number of memory reservation slots */ extern int minsize; /* Minimum blob size */ extern int padsize; /* Additional padding to blob */ extern int phandle_format; /* Use linux,phandle or phandle properties */ +extern int symbol_fixup_support;/* enable symbols & fixup support */ #define PHANDLE_LEGACY 0x1 #define PHANDLE_EPAPR 0x2 @@ -133,6 +134,25 @@ struct label { struct label *next; }; +struct fixup_entry { + int offset; + struct node *node; + struct property *prop; + struct fixup_entry *next; +}; + +struct fixup { + char *ref; + struct fixup_entry *entries; + struct fixup *next; +}; + +struct symbol { + struct label *label; + struct node *node; + struct symbol *next; +}; + struct property { int deleted; char *name; @@ -159,6 +179,12 @@ struct node { int addr_cells, size_cells; struct label *labels; + + int is_root; + int is_plugin; + struct fixup *fixups; + struct symbol *symbols; + struct fixup_entry *local_fixups; }; #define for_each_label_withdel(l0, l) \ @@ -182,6 +208,18 @@ struct node { for_each_child_withdel(n, c) \ if (!(c)->deleted) +#define for_each_fixup(n, f) \ + for ((f) = (n)->fixups; (f); (f) = (f)->next) + +#define for_each_fixup_entry(f, fe) \ + for ((fe) = (f)->entries; (fe); (fe) = (fe)->next) + +#define for_each_symbol(n, s) \ + for ((s) = (n)->symbols; (s); (s) = (s)->next) + +#define for_each_local_fixup_entry(n, fe) \ + for ((fe) = (n)->local_fixups; (fe); (fe) = (fe)->next) + void add_label(struct label **labels, char *label); void delete_labels(struct label **labels); diff --git a/flattree.c b/flattree.c index 665dad7..6237715 100644 --- a/flattree.c +++ b/flattree.c @@ -262,6 +262,12 @@ static void flatten_tree(struct node *tree, struct emitter *emit, struct property *prop; struct node *child; int seen_name_prop = 0; + struct symbol *sym; + struct fixup *f; + struct fixup_entry *fe; + char *name, *s; + const char *fullpath; + int namesz, nameoff, vallen; if (tree->deleted) return; @@ -310,6 +316,139 @@ static void flatten_tree(struct node *tree, struct emitter *emit, flatten_tree(child, emit, etarget, strbuf, vi); } + if (!symbol_fixup_support) + goto no_symbols; + + /* add the symbol nodes (if any) */ + if (tree->symbols) { + + emit->beginnode(etarget, NULL); + emit->string(etarget, "__symbols__", 0); + emit->align(etarget, sizeof(cell_t)); + + for_each_symbol(tree, sym) { + + vallen = strlen(sym->node->fullpath); + + nameoff = stringtable_insert(strbuf, sym->label->label); + + emit->property(etarget, NULL); + emit->cell(etarget, vallen + 1); + emit->cell(etarget, nameoff); + + if ((vi->flags & FTF_VARALIGN) && vallen >= 8) + emit->align(etarget, 8); + + emit->string(etarget, sym->node->fullpath, + strlen(sym->node->fullpath)); + emit->align(etarget, sizeof(cell_t)); + } + + emit->endnode(etarget, NULL); + } + + /* add the fixup nodes */ + if (tree->fixups) { + + /* emit the external fixups */ + emit->beginnode(etarget, NULL); + emit->string(etarget, "__fixups__", 0); + emit->align(etarget, sizeof(cell_t)); + + for_each_fixup(tree, f) { + + namesz = 0; + for_each_fixup_entry(f, fe) { + fullpath = fe->node->fullpath; + if (fullpath[0] == '\0') + fullpath = "/"; + namesz += strlen(fullpath) + 1; + namesz += strlen(fe->prop->name) + 1; + namesz += 32; /* space for :<number> + '\0' */ + } + + name = xmalloc(namesz); + + s = name; + for_each_fixup_entry(f, fe) { + fullpath = fe->node->fullpath; + if (fullpath[0] == '\0') + fullpath = "/"; + snprintf(s, name + namesz - s, "%s:%s:%d", + fullpath, + fe->prop->name, fe->offset); + s += strlen(s) + 1; + } + + nameoff = stringtable_insert(strbuf, f->ref); + vallen = s - name - 1; + + emit->property(etarget, NULL); + emit->cell(etarget, vallen + 1); + emit->cell(etarget, nameoff); + + if ((vi->flags & FTF_VARALIGN) && vallen >= 8) + emit->align(etarget, 8); + + emit->string(etarget, name, vallen); + emit->align(etarget, sizeof(cell_t)); + + free(name); + } + + emit->endnode(etarget, tree->labels); + } + + /* add the local fixup property */ + if (tree->local_fixups) { + + /* emit the external fixups */ + emit->beginnode(etarget, NULL); + emit->string(etarget, "__local_fixups__", 0); + emit->align(etarget, sizeof(cell_t)); + + namesz = 0; + for_each_local_fixup_entry(tree, fe) { + fullpath = fe->node->fullpath; + if (fullpath[0] == '\0') + fullpath = "/"; + namesz += strlen(fullpath) + 1; + namesz += strlen(fe->prop->name) + 1; + namesz += 32; /* space for :<number> + '\0' */ + } + + name = xmalloc(namesz); + + s = name; + for_each_local_fixup_entry(tree, fe) { + fullpath = fe->node->fullpath; + if (fullpath[0] == '\0') + fullpath = "/"; + snprintf(s, name + namesz - s, "%s:%s:%d", + fullpath, fe->prop->name, + fe->offset); + s += strlen(s) + 1; + } + + nameoff = stringtable_insert(strbuf, "fixup"); + vallen = s - name - 1; + + emit->property(etarget, NULL); + emit->cell(etarget, vallen + 1); + emit->cell(etarget, nameoff); + + if ((vi->flags & FTF_VARALIGN) && vallen >= 8) + emit->align(etarget, 8); + + emit->string(etarget, name, vallen); + emit->align(etarget, sizeof(cell_t)); + + free(name); + + emit->endnode(etarget, tree->labels); + } + +no_symbols: emit->endnode(etarget, tree->labels); } -- 1.7.12 _______________________________________________ devicetree-discuss mailing list devicetree-discuss@lists.ozlabs.org https://lists.ozlabs.org/listinfo/devicetree-discuss