I've also just opened a PR to daffodil-sbt to fix the bug that causes "show
fullClasspath" and "export fullClasspath" to include daffodil dependencies:
https://github.com/apache/daffodil-sbt/pull/64
On 2024-10-24 08:24 AM, Steve Lawrence wrote:
Instead of `sbt dependencyTree`, you can run `sbt "show fullClasspath"` to
output all the dependencies that `packageDaffodilBin` uses. You can also run
`sbt "export fullClasspath"` to get an actual classpath string that you can drop
into DAFFODIL_CLASSPATH. In one line, I think you could do:
export DAFFODIL_CLASSPATH=$(sbt -batch -error "export fullClasspath")
Note that the -batch and -error are needed to disable [info] and other output
messages.
Also note that this includes the scala dependency and I think we might have a
bug in daffodil-sbt that causes it to also include Daffodil dependencies if any
schemas are layers/charsets/etc. I *think* the way the Daffodil CLI builds up
the classpath those extra dependencies will all be ignored, but if not you might
have to manually build up the classpath with just the paths you want.
If we want we could add a special sbt task that essentially mimics this
behavior, but I'd rather we just document this magic export command somewhere so
we don't have to maintain it.
The Daffodil synonm is an interesting idea. I guess it would just set fork the
daffodil script with DAFFODIL_CLASSPATH already set, and just pass any task
arguments to the script? I think that's possible in SBT, but I think the export
magic above is a bit more flexible and efficient since you don't need to keep
sbt running to run a Daffodil command.
All that said, I think what we really need is a way to move about from requiring
DAFFODIL_CLASSPATH at all. For example, if we could embed dependencies in the
actual saved parser file, then using that saved parser wouldn't need any
classpath modifications, it's just all already there and Daffodil internals
would uses those embedded dependencies for class lookups. I'm unsure of exactly
how to do that, but it's definitely possible--NiFi does something very similar
with it's "nar" format.
On 2024-10-23 06:00 PM, Mike Beckerle wrote:
I am trying to go from 'sbt test' to a schema I can play with from the
daffodil CLI.
The schema of interest is the DFDLSchemas envelope-payload example.
This schema depends on
* tcpMessage
* mil-std-2045
* pcap
The pcap schema in turn depends on
* ethernetIP
The ethernetIP schema defines a Daffodil layer plugin that exists in its
jar.
So far if I clone these all, and 'sbt publishLocal' all of the components,
then I can 'sbt test' in envelope-payload and it passes all tests.
So now I'd like to do 'daffodil save-parser
-s src/main/resources/io/github/dfdlschemas/envelopepayload/xsd/
envelopePayload.dfdl.xsd
-o /tmp/envPay.bin'
By adding these to the build.sbt
daffodilPackageBinVersions := Seq(daffodilVersion.value)
daffodilPackageBinInfos := Seq(
("/io/github/dfdlschemas/tcpMessage/xsd/tcpMessage.dfdl.xsd",
Some("message"), None)
)
Then 'sbt packageDaffodilBin' will create a compiled schema under the
target/ directory named:
dfdl-envelope-payload-1.1.0-daffodil390.bin
So far so good.
Now the challenge.
But this can't be used with the daffodil CLI without also setting up
DAFFODIL_CLASSPATH to have at least the ethernetIP jar file.
Which is where? (yes I know in the ~/.ivy2/local cache, but there is tons
of stuff in there.)
If I want to use xerces (aka full) validation then I also have to have all
the other component schema jar files on the DAFFODIL_CLASSPATH as well.
I tried issuing 'sbt dependencyTree', but the dependency tree is not just
the schemas, but all the dependencies on daffodil and everything
transitively it uses.
This is much too hard.
Why not have the daffodil-sbt plugin output a shell script that appends to
DAFFODIL_CLASSPATH with all the necessary component schema jars.
Then users could just run that script to establish the DAFFODIL_CLASSPATH
once, and then they could use the daffodil CLI normally.
In principle, a file could also be written intended to inform the daffodil
VSCode extension of the classpath to construct for the schema.
Thoughts?
One possible alternative is to add a daffodil command to sbt via the plugin
so that one can run daffodil command lines directly from the sbt prompt:
sbt
daffodil parse -p target/...bin foo.dat
This is really just a synonym for issuing sbt run for the Main class of the
daffodil-cli module, (after doing packageDaffodilBin) so might be very easy
to do.
Mike Beckerle
Apache Daffodil PMC | daffodil.apache.org
OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl
Owl Cyber Defense | www.owlcyberdefense.com
