If 'sbt test' will run TDML tests on either or both backends, then I see no
need for the new C-backend to directly/natively support command-line invocation
of TDML tests at all.
Moreover, if the existing daffodil scala CLI can run TDML tests (which it can),
then if it can run TDML tests that select and use the C-generating back-end
when doing so, then that's more than sufficient.
So what additional purpose does having a specialized CLI provide for the C-code
backend?
1) A convenient way for a user to do ad-hoc testing outside of TDML context.
2) A production-worthy tool to script conversions of data from native to XML
(for now, other representations later perhaps) and back. I would certainly
expect it would startup and run much faster than scala-daffodil does for this
kind of scripted converter.
Neither of the above require that this C-code CLI have anything to do with the
scala daffodil CLI. In fact startup-time argues that it NOT have anything to do
with the scala code base.
But even If you are doing that, then might was well share code i.e., this
C-code CLI is a convenient way to implement TDML test rig adapter - conversion
to/from XML can be packaged so that this same mechanism is how the TDML runner
feeds XML Infosets to C code and gets infosets back from C code.
I would like to see this CLI and the XML conversion stuff (which depends on the
mxml library) to be kept separate from the "regular" runtime code, which
contains the primitive parsers/unparsers called from the generated code. This
latter runtime library is required for all uses of the generated C code from
its API. Selective linking against this could produce very small footprint DFDL
processors.
The CLI/XML library would be needed only for applications that want to convert
data to/from XML.
________________________________
From: Interrante, John A (GE Research, US) <inter...@research.ge.com>
Sent: Monday, October 5, 2020 12:09 PM
To: dev@daffodil.apache.org <dev@daffodil.apache.org>
Subject: How to use and trigger different Daffodil backends
Steve Lawrence and I would like to bring another topic from my runtime2 pull
request to the dev list for wider discussion. The question is how would you
expect to use and trigger different Daffodil backends? We believe these
requirements hold:
- Daffodil's Scala runtime/backend/whatchacallit will support the entire
DFDL 1.0 specification
- A new Daffodil backend will generate C code for a minimally viable subset
of the DFDL 1.0 specification for small footprint devices
- Daffodil will support running TDML tests against both backends (C code
generation, C code compilation, and invoking of compiled code all transparent
to the user)
- Daffodil may add more backends in the future (maybe runtime3 might
generate VHDL files for chips)
In addition, the following requirements may hold too:
- We may want to use the same TDML "snap compilation/execution" mechanism
to have "daffodil parse" and "daffodil unparse" CLI commands that work alike
across the backends, but a counter argument is that there's little benefit to
be gained when the C code could be executed directly using its own similar
"daffodil parse" and "daffodil unparse" CLI syntax, and a VHDL simulator is
needed to "snap compile/execute" VHDL files anyway.
- The C generator should generate an ".a" or ".so" C-callable library and
matching ".h" include files that implement C APIs for parsing and unparsing
binary data corresponding to the given DFDL schema. This generated code
library would be distinct from the C runtime2 library which contains common
low-level parsing/unparsing functions implemented in C. Actually, all
generated and runtime2 ".c" source files should be written to an output
directory in case both libraries have to be compiled on another platform.
Creating the generated library feels like what the CLI should do when using the
runtime2 backend. There is an analogy here to what invoking the "daffodil
save-parser" subcommand does (it creates a saved parser file which speeds up a
future parse). Likewise, it makes sense to generate & compile the C code
first, then run the C-code executable directly using its own "daffodil parse"
and "daffodil unparse" CLI syntax. Or generate the VHDL files first and then
load them into a VHDL simulator.
Steve, Mike, and I have discussed different CLI commands for creating the
generated library:
daffodil runtime2 -s schema.dfdl.xsd -o schema.bin
daffodil save-parser -s schema.dfdl.xsd -Truntime=runtime2 outfile
daffodil generate C -s schema.dfdl.xsd outdir/
Each has pros and cons. Using "daffodil <runtime>" means each runtime/backend
can use its own CLI options more easily. The outfile passed to "daffodil
save-parser" is a filename but the C generator needs a directory name since it
has to write multiple files into it. We can run different backends with
"daffodil generate C" or "daffodil generate VHDL" and if different backends
need different CLI options maybe the Daffodil Scala CLI can ask the different
backends for their CLI options.
What do devs think? Keep the Daffodil Scala CLI focused on Daffodil's Scala
runtime and make new runtimes use a separate CLI tool with a unique name and
its unique CLI syntax? Or shoehorn some additional functionality into the
Daffodil Scala CLI syntax?
