This trick works:
export DAFFODIL_CLASSPATH=$(sbt -batch -error "export fullClasspath")
I don't like it, but it works.
Here is what export fullClasspath outputs for this project:
export fullClasspath
/home/mbeckerle/dataiti/opensource/DFDLSchemas/envelope-payload/target/classes:/home/mbeckerle/.cache/coursier/v1/https/
repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.19/scala-library-2.12.19.jar:/home/mbeckerle/.ivy2/local/io.github.dfdlschemas/dfdl-tcpmessage/1.0.0/jars/dfdl-tcpmessage.jar:/home/mbeckerle/.ivy2/local/com.owlcyberdefense/dfdl-mil-std-2045/1.3.2/jars/dfdl-mil-std-2045.jar:/home/mbeckerle/.ivy2/local/com.tresys/dfdl-pcap/1.4.0/jars/dfdl-pcap.jar:/home/mbeckerle/.ivy2/local/com.owlcyberdefense/dfdl-ethernetip_2.12/1.4.0/jars/dfdl-ethernetip_2.12.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime1-layers_2.12/3.8.0/daffodil-runtime1-layers_2.12-3.8.0.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime1_2.12/3.8.0/daffodil-runtime1_2.12-3.8.0.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/os-lib_2.12/0.10.2/os-lib_2.12-0.10.2.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.12/2.3.0/scala-xml_2.12-2.3.0.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-parser-combinators_2.12/2.4.0/scala-parser-combinators_2.12-2.4.0.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/com/ibm/icu/icu4j/75.1/icu4j-75.1.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/xerces/xercesImpl/2.12.2/xercesImpl-2.12.2.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/xml-resolver/xml-resolver/1.2/xml-resolver-1.2.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/commons-io/commons-io/2.16.1/commons-io-2.16.1.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/com/typesafe/config/1.4.3/config-1.4.3.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/com/typesafe/scala-logging/scala-logging_2.12/3.9.5/scala-logging_2.12-3.9.5.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jdom/jdom2/2.0.6.1/jdom2-2.0.6.1.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/woodstox/woodstox-core/6.6.2/woodstox-core-6.6.2.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.16.1/jackson-core-2.16.1.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/daffodil/daffodil-io_2.12/3.8.0/daffodil-io_2.12-3.8.0.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/daffodil/daffodil-udf_2.12/3.8.0/daffodil-udf_2.12-3.8.0.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/geny_2.12/1.1.0/geny_2.12-1.1.0.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.19/scala-reflect-2.12.19.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/codehaus/woodstox/stax2-api/4.2.2/stax2-api-4.2.2.jar:/home/mbeckerle/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/daffodil/daffodil-lib_2.12/3.8.0/daffodil-lib_2.12-3.8.0.jar
>From this what is needed is only these:
/home/mbeckerle/.ivy2/local/io.github.dfdlschemas/dfdl-tcpmessage/1.0.0/jars/dfdl-tcpmessage.jar:\
/home/mbeckerle/.ivy2/local/com.owlcyberdefense/dfdl-mil-std-2045/1.3.2/jars/dfdl-mil-std-2045.jar:\
/home/mbeckerle/.ivy2/local/com.tresys/dfdl-pcap/1.4.0/jars/dfdl-pcap.jar:\
/home/mbeckerle/.ivy2/local/com.owlcyberdefense/dfdl-ethernetip_2.12/1.4.0/jars/dfdl-ethernetip_2.12.jar
All the rest are provided by daffodil, so don't need to be on the
DAFFODIL_CLASSPATH.
However, I am unsure of the downside of having all those extra jars on the
DAFFODIL_CLASSPATH.
On Thu, Oct 24, 2024 at 8:26 AM Steve Lawrence <slawre...@apache.org> wrote:
> Instead of `sbt dependencyTree`, you can run `sbt "show fullClasspath"` to
> output all the dependencies that `packageDaffodilBin` uses. You can also
> run
> `sbt "export fullClasspath"` to get an actual classpath string that you
> can drop
> into DAFFODIL_CLASSPATH. In one line, I think you could do:
>
> export DAFFODIL_CLASSPATH=$(sbt -batch -error "export fullClasspath")
>
> Note that the -batch and -error are needed to disable [info] and other
> output
> messages.
>
> Also note that this includes the scala dependency and I think we might
> have a
> bug in daffodil-sbt that causes it to also include Daffodil dependencies
> if any
> schemas are layers/charsets/etc. I *think* the way the Daffodil CLI builds
> up
> the classpath those extra dependencies will all be ignored, but if not you
> might
> have to manually build up the classpath with just the paths you want.
>
> If we want we could add a special sbt task that essentially mimics this
> behavior, but I'd rather we just document this magic export command
> somewhere so
> we don't have to maintain it.
>
>
> The Daffodil synonm is an interesting idea. I guess it would just set fork
> the
> daffodil script with DAFFODIL_CLASSPATH already set, and just pass any
> task
> arguments to the script? I think that's possible in SBT, but I think the
> export
> magic above is a bit more flexible and efficient since you don't need to
> keep
> sbt running to run a Daffodil command.
>
>
> All that said, I think what we really need is a way to move about from
> requiring
> DAFFODIL_CLASSPATH at all. For example, if we could embed dependencies in
> the
> actual saved parser file, then using that saved parser wouldn't need any
> classpath modifications, it's just all already there and Daffodil
> internals
> would uses those embedded dependencies for class lookups. I'm unsure of
> exactly
> how to do that, but it's definitely possible--NiFi does something very
> similar
> with it's "nar" format.
>
>
> On 2024-10-23 06:00 PM, Mike Beckerle wrote:
> > I am trying to go from 'sbt test' to a schema I can play with from the
> > daffodil CLI.
> >
> > The schema of interest is the DFDLSchemas envelope-payload example.
> >
> > This schema depends on
> > * tcpMessage
> > * mil-std-2045
> > * pcap
> >
> > The pcap schema in turn depends on
> > * ethernetIP
> >
> > The ethernetIP schema defines a Daffodil layer plugin that exists in its
> > jar.
> >
> > So far if I clone these all, and 'sbt publishLocal' all of the
> components,
> > then I can 'sbt test' in envelope-payload and it passes all tests.
> >
> > So now I'd like to do 'daffodil save-parser
> > -s
> src/main/resources/io/github/dfdlschemas/envelopepayload/xsd/envelopePayload.dfdl.xsd
> > -o /tmp/envPay.bin'
> >
> > By adding these to the build.sbt
> >
> > daffodilPackageBinVersions := Seq(daffodilVersion.value)
> > daffodilPackageBinInfos := Seq(
> > ("/io/github/dfdlschemas/tcpMessage/xsd/tcpMessage.dfdl.xsd",
> > Some("message"), None)
> > )
> >
> > Then 'sbt packageDaffodilBin' will create a compiled schema under the
> > target/ directory named:
> >
> > dfdl-envelope-payload-1.1.0-daffodil390.bin
> >
> > So far so good.
> >
> > Now the challenge.
> >
> > But this can't be used with the daffodil CLI without also setting up
> > DAFFODIL_CLASSPATH to have at least the ethernetIP jar file.
> >
> > Which is where? (yes I know in the ~/.ivy2/local cache, but there is tons
> > of stuff in there.)
> >
> > If I want to use xerces (aka full) validation then I also have to have
> all
> > the other component schema jar files on the DAFFODIL_CLASSPATH as well.
> >
> > I tried issuing 'sbt dependencyTree', but the dependency tree is not just
> > the schemas, but all the dependencies on daffodil and everything
> > transitively it uses.
> >
> > This is much too hard.
> >
> > Why not have the daffodil-sbt plugin output a shell script that appends
> to
> > DAFFODIL_CLASSPATH with all the necessary component schema jars.
> >
> > Then users could just run that script to establish the DAFFODIL_CLASSPATH
> > once, and then they could use the daffodil CLI normally.
> >
> > In principle, a file could also be written intended to inform the
> daffodil
> > VSCode extension of the classpath to construct for the schema.
> >
> > Thoughts?
> >
> > One possible alternative is to add a daffodil command to sbt via the
> plugin
> > so that one can run daffodil command lines directly from the sbt prompt:
> >
> > sbt
> >> daffodil parse -p target/...bin foo.dat
> >
> > This is really just a synonym for issuing sbt run for the Main class of
> the
> > daffodil-cli module, (after doing packageDaffodilBin) so might be very
> easy
> > to do.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Mike Beckerle
> > Apache Daffodil PMC | daffodil.apache.org
> > OGF DFDL Workgroup Co-Chair |
> www.ogf.org/ogf/doku.php/standards/dfdl/dfdl
> > Owl Cyber Defense | www.owlcyberdefense.com
> >
>
>
