This resend fixes a couple things:
* Remove double import of os package
* Incorrect error checking in the "include-before-err" test.
Thanks,
Lluis
Lluís Vilanova writes:
> The primitive uses JSON syntax, and include paths are relative to the file
> using the directive:
> { 'include': 'path/to/file.json' }
> Signed-off-by: Lluís Vilanova <vilan...@ac.upc.edu>
> Reviewed-by: Eric Blake <ebl...@redhat.com>
> Reviewed-by: Markus Armbruster <arm...@redhat.com>
> ---
> docs/qapi-code-gen.txt | 11 +++++
> scripts/qapi.py | 64
> +++++++++++++++++++++++-----
> tests/Makefile | 5 ++
> tests/qapi-schema/include-before-err.err | 1
> tests/qapi-schema/include-before-err.exit | 1
> tests/qapi-schema/include-before-err.json | 2 +
> tests/qapi-schema/include-before-err.out | 0
> tests/qapi-schema/include-cycle-b.json | 1
> tests/qapi-schema/include-cycle-c.json | 1
> tests/qapi-schema/include-cycle.err | 3 +
> tests/qapi-schema/include-cycle.exit | 1
> tests/qapi-schema/include-cycle.json | 1
> tests/qapi-schema/include-cycle.out | 0
> tests/qapi-schema/include-format-err.err | 1
> tests/qapi-schema/include-format-err.exit | 1
> tests/qapi-schema/include-format-err.json | 2 +
> tests/qapi-schema/include-format-err.out | 0
> tests/qapi-schema/include-nested-err.err | 2 +
> tests/qapi-schema/include-nested-err.exit | 1
> tests/qapi-schema/include-nested-err.json | 1
> tests/qapi-schema/include-nested-err.out | 0
> tests/qapi-schema/include-no-file.err | 1
> tests/qapi-schema/include-no-file.exit | 1
> tests/qapi-schema/include-no-file.json | 1
> tests/qapi-schema/include-no-file.out | 0
> tests/qapi-schema/include-non-file.err | 1
> tests/qapi-schema/include-non-file.exit | 1
> tests/qapi-schema/include-non-file.json | 1
> tests/qapi-schema/include-non-file.out | 0
> tests/qapi-schema/include-relpath-sub.json | 2 +
> tests/qapi-schema/include-relpath.err | 0
> tests/qapi-schema/include-relpath.exit | 1
> tests/qapi-schema/include-relpath.json | 1
> tests/qapi-schema/include-relpath.out | 3 +
> tests/qapi-schema/include-self-cycle.err | 1
> tests/qapi-schema/include-self-cycle.exit | 1
> tests/qapi-schema/include-self-cycle.json | 1
> tests/qapi-schema/include-self-cycle.out | 0
> tests/qapi-schema/include-simple-sub.json | 2 +
> tests/qapi-schema/include-simple.err | 0
> tests/qapi-schema/include-simple.exit | 1
> tests/qapi-schema/include-simple.json | 1
> tests/qapi-schema/include-simple.out | 3 +
> tests/qapi-schema/include/relpath.json | 1
> 44 files changed, 110 insertions(+), 13 deletions(-)
> create mode 100644 tests/qapi-schema/include-before-err.err
> create mode 100644 tests/qapi-schema/include-before-err.exit
> create mode 100644 tests/qapi-schema/include-before-err.json
> create mode 100644 tests/qapi-schema/include-before-err.out
> create mode 100644 tests/qapi-schema/include-cycle-b.json
> create mode 100644 tests/qapi-schema/include-cycle-c.json
> create mode 100644 tests/qapi-schema/include-cycle.err
> create mode 100644 tests/qapi-schema/include-cycle.exit
> create mode 100644 tests/qapi-schema/include-cycle.json
> create mode 100644 tests/qapi-schema/include-cycle.out
> create mode 100644 tests/qapi-schema/include-format-err.err
> create mode 100644 tests/qapi-schema/include-format-err.exit
> create mode 100644 tests/qapi-schema/include-format-err.json
> create mode 100644 tests/qapi-schema/include-format-err.out
> create mode 100644 tests/qapi-schema/include-nested-err.err
> create mode 100644 tests/qapi-schema/include-nested-err.exit
> create mode 100644 tests/qapi-schema/include-nested-err.json
> create mode 100644 tests/qapi-schema/include-nested-err.out
> create mode 100644 tests/qapi-schema/include-no-file.err
> create mode 100644 tests/qapi-schema/include-no-file.exit
> create mode 100644 tests/qapi-schema/include-no-file.json
> create mode 100644 tests/qapi-schema/include-no-file.out
> create mode 100644 tests/qapi-schema/include-non-file.err
> create mode 100644 tests/qapi-schema/include-non-file.exit
> create mode 100644 tests/qapi-schema/include-non-file.json
> create mode 100644 tests/qapi-schema/include-non-file.out
> create mode 100644 tests/qapi-schema/include-relpath-sub.json
> create mode 100644 tests/qapi-schema/include-relpath.err
> create mode 100644 tests/qapi-schema/include-relpath.exit
> create mode 100644 tests/qapi-schema/include-relpath.json
> create mode 100644 tests/qapi-schema/include-relpath.out
> create mode 100644 tests/qapi-schema/include-self-cycle.err
> create mode 100644 tests/qapi-schema/include-self-cycle.exit
> create mode 100644 tests/qapi-schema/include-self-cycle.json
> create mode 100644 tests/qapi-schema/include-self-cycle.out
> create mode 100644 tests/qapi-schema/include-simple-sub.json
> create mode 100644 tests/qapi-schema/include-simple.err
> create mode 100644 tests/qapi-schema/include-simple.exit
> create mode 100644 tests/qapi-schema/include-simple.json
> create mode 100644 tests/qapi-schema/include-simple.out
> create mode 100644 tests/qapi-schema/include/relpath.json
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 63b03cf..051d109 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -40,6 +40,17 @@ enumeration types and union types.
> Generally speaking, types definitions should always use CamelCase for the
> type
> names. Command names should be all lower case with words separated by a
> hyphen.
> +
> +=== Includes ===
> +
> +The QAPI schema definitions can be modularized using the 'include' directive:
> +
> + { 'include': 'path/to/file.json'}
> +
> +The directive is evaluated recursively, and include paths are relative to the
> +file using the directive.
> +
> +
> === Complex types ===
> A complex type is a dictionary containing a single key whose value is a
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 07a7a82..ec806aa 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -11,6 +11,7 @@
> # This work is licensed under the terms of the GNU GPL, version 2.
> # See the COPYING file in the top-level directory.
> +import re
> from ordereddict import OrderedDict
> import os
> import sys
> @@ -36,9 +37,17 @@ builtin_type_qtypes = {
> 'uint64': 'QTYPE_QINT',
> }
> +def error_path(parent):
> + res = ""
> + while parent:
> + res = ("In file included from %s:%d:\n" % (parent['file'],
> + parent['line'])) + res
> + parent = parent['parent']
> + return res
> +
> class QAPISchemaError(Exception):
> def __init__(self, schema, msg):
> - self.fp = schema.fp
> + self.input_file = schema.input_file
> self.msg = msg
> self.col = 1
> self.line = schema.line
> @@ -47,23 +56,31 @@ class QAPISchemaError(Exception):
> self.col = (self.col + 7) % 8 + 1
> else:
> self.col += 1
> + self.info = schema.parent_info
> def __str__(self):
> - return "%s:%s:%s: %s" % (self.fp.name, self.line, self.col, self.msg)
> + return error_path(self.info) + \
> + "%s:%d:%d: %s" % (self.input_file, self.line, self.col, self.msg)
> class QAPIExprError(Exception):
> def __init__(self, expr_info, msg):
> - self.fp = expr_info['fp']
> - self.line = expr_info['line']
> + self.info = expr_info
> self.msg = msg
> def __str__(self):
> - return "%s:%s: %s" % (self.fp.name, self.line, self.msg)
> + return error_path(self.info['parent']) + \
> + "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
> class QAPISchema:
> - def __init__(self, fp):
> - self.fp = fp
> + def __init__(self, fp, input_relname=None, include_hist=[],
> parent_info=None):
> + input_fname = os.path.abspath(fp.name)
> + if input_relname is None:
> + input_relname = fp.name
> + self.input_dir = os.path.dirname(input_fname)
> + self.input_file = input_relname
> + self.include_hist = include_hist + [(input_relname, input_fname)]
> + self.parent_info = parent_info
> self.src = fp.read()
> if self.src == '' or self.src[-1] != '\n':
> self.src += '\n'
> @@ -74,10 +91,33 @@ class QAPISchema:
> self.accept()
> while self.tok != None:
> - expr_info = {'fp': fp, 'line': self.line}
> - expr_elem = {'expr': self.get_expr(False),
> - 'info': expr_info}
> - self.exprs.append(expr_elem)
> + expr_info = {'file': input_relname, 'line': self.line, 'parent':
> self.parent_info}
> + expr = self.get_expr(False)
> + if isinstance(expr, dict) and "include" in expr:
> + if len(expr) != 1:
> + raise QAPIExprError(expr_info, "Invalid 'include'
> directive")
> + include = expr["include"]
> + if not isinstance(include, str):
> + raise QAPIExprError(expr_info,
> + 'Expected a file name (string), got:
> %s'
> + % include)
> + include_path = os.path.join(self.input_dir, include)
> + if any(include_path == elem[1]
> + for elem in self.include_hist):
> + raise QAPIExprError(expr_info, "Inclusion loop for %s"
> + % include)
> + try:
> + fobj = open(include_path, 'r')
> + except IOError as e:
> + raise QAPIExprError(expr_info,
> + '%s: %s' % (e.strerror, include))
> + exprs_include = QAPISchema(fobj, include,
> + self.include_hist, expr_info)
> + self.exprs.extend(exprs_include.exprs)
> + else:
> + expr_elem = {'expr': expr,
> + 'info': expr_info}
> + self.exprs.append(expr_elem)
> def accept(self):
> while True:
> @@ -267,7 +307,7 @@ def check_exprs(schema):
> def parse_schema(input_file):
> try:
> schema = QAPISchema(open(input_file, "r"))
> - except QAPISchemaError, e:
> + except (QAPISchemaError, QAPIExprError), e:
> print >>sys.stderr, e
> exit(1)
> diff --git a/tests/Makefile b/tests/Makefile
> index 3aa9b0b..409bd24 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -164,7 +164,10 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
> duplicate-key.json union-invalid-base.json flat-union-no-base.json \
> flat-union-invalid-discriminator.json \
> flat-union-invalid-branch-key.json flat-union-reverse-define.json \
> - flat-union-string-discriminator.json)
> + flat-union-string-discriminator.json \
> + include-simple.json include-relpath.json include-format-err.json \
> + include-non-file.json include-no-file.json include-before-err.json \
> + include-nested-err.json include-self-cycle.json include-cycle.json)
> GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h
> tests/test-qmp-commands.h
> diff --git a/tests/qapi-schema/include-before-err.err
> b/tests/qapi-schema/include-before-err.err
> new file mode 100644
> index 0000000..5565275
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-before-err.json:2:13: Expected ":"
> diff --git a/tests/qapi-schema/include-before-err.exit
> b/tests/qapi-schema/include-before-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-before-err.json
> b/tests/qapi-schema/include-before-err.json
> new file mode 100644
> index 0000000..afb6cb6
> --- /dev/null
> +++ b/tests/qapi-schema/include-before-err.json
> @@ -0,0 +1,2 @@
> +{ 'include': 'include-simple-sub.json' }
> +{ 'command' 'missing-colon' }
> diff --git a/tests/qapi-schema/include-before-err.out
> b/tests/qapi-schema/include-before-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-cycle-b.json
> b/tests/qapi-schema/include-cycle-b.json
> new file mode 100644
> index 0000000..4fa985d
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle-b.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle-c.json' }
> diff --git a/tests/qapi-schema/include-cycle-c.json
> b/tests/qapi-schema/include-cycle-c.json
> new file mode 100644
> index 0000000..d12b592
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle-c.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle.json' }
> diff --git a/tests/qapi-schema/include-cycle.err
> b/tests/qapi-schema/include-cycle.err
> new file mode 100644
> index 0000000..602cf62
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.err
> @@ -0,0 +1,3 @@
> +In file included from tests/qapi-schema/include-cycle.json:1:
> +In file included from include-cycle-b.json:1:
> +include-cycle-c.json:1: Inclusion loop for include-cycle.json
> diff --git a/tests/qapi-schema/include-cycle.exit
> b/tests/qapi-schema/include-cycle.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-cycle.json
> b/tests/qapi-schema/include-cycle.json
> new file mode 100644
> index 0000000..6fcf1eb
> --- /dev/null
> +++ b/tests/qapi-schema/include-cycle.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-cycle-b.json' }
> diff --git a/tests/qapi-schema/include-cycle.out
> b/tests/qapi-schema/include-cycle.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-format-err.err
> b/tests/qapi-schema/include-format-err.err
> new file mode 100644
> index 0000000..721ff4e
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-format-err.json:1: Invalid 'include' directive
> diff --git a/tests/qapi-schema/include-format-err.exit
> b/tests/qapi-schema/include-format-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-format-err.json
> b/tests/qapi-schema/include-format-err.json
> new file mode 100644
> index 0000000..44980f0
> --- /dev/null
> +++ b/tests/qapi-schema/include-format-err.json
> @@ -0,0 +1,2 @@
> +{ 'include': 'include-simple-sub.json',
> + 'foo': 'bar' }
> diff --git a/tests/qapi-schema/include-format-err.out
> b/tests/qapi-schema/include-format-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-nested-err.err
> b/tests/qapi-schema/include-nested-err.err
> new file mode 100644
> index 0000000..1dacbda
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.err
> @@ -0,0 +1,2 @@
> +In file included from tests/qapi-schema/include-nested-err.json:1:
> +missing-colon.json:1:10: Expected ":"
> diff --git a/tests/qapi-schema/include-nested-err.exit
> b/tests/qapi-schema/include-nested-err.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-nested-err.json
> b/tests/qapi-schema/include-nested-err.json
> new file mode 100644
> index 0000000..5631e56
> --- /dev/null
> +++ b/tests/qapi-schema/include-nested-err.json
> @@ -0,0 +1 @@
> +{ 'include': 'missing-colon.json' }
> diff --git a/tests/qapi-schema/include-nested-err.out
> b/tests/qapi-schema/include-nested-err.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-no-file.err
> b/tests/qapi-schema/include-no-file.err
> new file mode 100644
> index 0000000..d5b9b22
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-no-file.json:1: No such file or directory:
> include-no-file-sub.json
> diff --git a/tests/qapi-schema/include-no-file.exit
> b/tests/qapi-schema/include-no-file.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-no-file.json
> b/tests/qapi-schema/include-no-file.json
> new file mode 100644
> index 0000000..9249ebd
> --- /dev/null
> +++ b/tests/qapi-schema/include-no-file.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-no-file-sub.json' }
> diff --git a/tests/qapi-schema/include-no-file.out
> b/tests/qapi-schema/include-no-file.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-non-file.err
> b/tests/qapi-schema/include-non-file.err
> new file mode 100644
> index 0000000..9658c78
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-non-file.json:1: Expected a file name (string),
> got: ['foo', 'bar']
> diff --git a/tests/qapi-schema/include-non-file.exit
> b/tests/qapi-schema/include-non-file.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-non-file.json
> b/tests/qapi-schema/include-non-file.json
> new file mode 100644
> index 0000000..cd43c3f
> --- /dev/null
> +++ b/tests/qapi-schema/include-non-file.json
> @@ -0,0 +1 @@
> +{ 'include': [ 'foo', 'bar' ] }
> diff --git a/tests/qapi-schema/include-non-file.out
> b/tests/qapi-schema/include-non-file.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-relpath-sub.json
> b/tests/qapi-schema/include-relpath-sub.json
> new file mode 100644
> index 0000000..4bd4af4
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath-sub.json
> @@ -0,0 +1,2 @@
> +{ 'enum': 'Status',
> + 'data': [ 'good', 'bad', 'ugly' ] }
> diff --git a/tests/qapi-schema/include-relpath.err
> b/tests/qapi-schema/include-relpath.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-relpath.exit
> b/tests/qapi-schema/include-relpath.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/include-relpath.json
> b/tests/qapi-schema/include-relpath.json
> new file mode 100644
> index 0000000..05018f3
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.json
> @@ -0,0 +1 @@
> +{ 'include': 'include/relpath.json' }
> diff --git a/tests/qapi-schema/include-relpath.out
> b/tests/qapi-schema/include-relpath.out
> new file mode 100644
> index 0000000..4ce3dcf
> --- /dev/null
> +++ b/tests/qapi-schema/include-relpath.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
> +[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
> +[]
> diff --git a/tests/qapi-schema/include-self-cycle.err
> b/tests/qapi-schema/include-self-cycle.err
> new file mode 100644
> index 0000000..981742a
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/include-self-cycle.json:1: Inclusion loop for
> include-self-cycle.json
> diff --git a/tests/qapi-schema/include-self-cycle.exit
> b/tests/qapi-schema/include-self-cycle.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/include-self-cycle.json
> b/tests/qapi-schema/include-self-cycle.json
> new file mode 100644
> index 0000000..55fb1b5
> --- /dev/null
> +++ b/tests/qapi-schema/include-self-cycle.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-self-cycle.json' }
> diff --git a/tests/qapi-schema/include-self-cycle.out
> b/tests/qapi-schema/include-self-cycle.out
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-simple-sub.json
> b/tests/qapi-schema/include-simple-sub.json
> new file mode 100644
> index 0000000..4bd4af4
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple-sub.json
> @@ -0,0 +1,2 @@
> +{ 'enum': 'Status',
> + 'data': [ 'good', 'bad', 'ugly' ] }
> diff --git a/tests/qapi-schema/include-simple.err
> b/tests/qapi-schema/include-simple.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/include-simple.exit
> b/tests/qapi-schema/include-simple.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/include-simple.json
> b/tests/qapi-schema/include-simple.json
> new file mode 100644
> index 0000000..1dd391a
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.json
> @@ -0,0 +1 @@
> +{ 'include': 'include-simple-sub.json' }
> diff --git a/tests/qapi-schema/include-simple.out
> b/tests/qapi-schema/include-simple.out
> new file mode 100644
> index 0000000..4ce3dcf
> --- /dev/null
> +++ b/tests/qapi-schema/include-simple.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
> +[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
> +[]
> diff --git a/tests/qapi-schema/include/relpath.json
> b/tests/qapi-schema/include/relpath.json
> new file mode 100644
> index 0000000..45dee24
> --- /dev/null
> +++ b/tests/qapi-schema/include/relpath.json
> @@ -0,0 +1 @@
> +{ 'include': '../include-relpath-sub.json' }
--
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth
