Ultimately all jars need to end up on the classpath somehow.
Maybe one option is to add maven/ivy support to the Daffodil CLI and add
way to specify a dependency, e.g
daffodil parse --dependency com.example.transforms/[email protected] ...
Then Daffodil could download any dependencies (excluding Daffodil deps
that it already has) and update the classpath, maybe with a custom
classloader or maybe reexecing itself with a new claspath or something.
The feels a bit extreme though--I'm not sure I want the CLI to have
maven/ivy support.
I think maybe a more reasonable option is to document how to create a
fat jar for these kinds of things (with a way to exclude the Daffodil
dependencies), and then users only need to add a single jar to their
classpath. It still requires a little user intervention but it's not
terrible.
And it looks like the sbt-assembly plugin can already create a fat jar
and exclude dependencies using the "provided" keyword which just a
couple of sbt lines:
https://www.baeldung.com/scala/sbt
This is nice too because this will include schemas, transformers, and
dependencies (both Java and schema dependencies) all in a single jar, so
it's a all in one bundle that can easily be put on the class path that
contains everything you could need.
We've also discussed a Daffodil/DFDL plugin in the past to simplify
schema development. Having that plugin also configure these fat jars for
releases might make a lot of sense. This way, all you would do is add
the DFDL plugin to your project, define your dependencies the normal
way, put your schemas and transforms in the right place, and run "sbt
assembly", and you'll get a fat jar to put on the classpath.
On 2023-01-10 05:52 PM, Mike Beckerle wrote:
So when I use daffodil by way of sbt, all the managed dependencies of the
zip-layer-transformer (really just on apache commons compress) are taken
care of for me.
My main schema just declares a managed dependency on a specific version of
the zip-layer-transformer, and voila, the transitive dependency on the
apache commons compress takes care of itself.
When I use the CLI, I have to do this stuff manually. It's amazingly hard
to get DAFFODIL_CLASSPATH complete and correct.
'sbt show dependencyClasspath' shows me the dependency on apache commons
compress library, of course it also has all the daffodil jars and all their
dependencies.
How do we make the CLI or any environment for running Daffodil, as easy as
sbt?
On Tue, Jan 10, 2023 at 3:36 PM Steve Lawrence <[email protected]> wrote:
I think so. Daffodil can't know what libraries a layer transformer might
need, so we can't depend on them or include them in releases.
So if a transformer plugin requires one or more libraries that don't
already ship with Daffodil, then they should be marked as a dependency
of the transformer and their jars downloaded and included in the
classpath along with the transformer.
You could maybe get around his by making your transformer a fat jar, but
it might include all the Daffodil jars as well, which might cause some
problems.
On 2023-01-10 03:24 PM, Mike Beckerle wrote:
So I wrote a layer transformer that unzips zip files in data.
It depends on apache commons compress library.
Daffodil recognizes that a schema requires this, and dynamically loads
it.
But then things fail because apache commons compress library is not
found.
We have to manually add apache commons compress jar to the daffodil
classpath.
Is this expected behavior?
Mike Beckerle
Apache Daffodil PMC | daffodil.apache.org
OGF DFDL Workgroup Co-Chair |
www.ogf.org/ogf/doku.php/standards/dfdl/dfdl
Owl Cyber Defense | www.owlcyberdefense.com
