I have a love/hate relationship with SBT, but today I love it, because of this
feature I discovered today
which is very cool, and super useful because it allows DFDL schemas to be split
into reusable modules easily.
SBT can automatically grab a github/git source release (or tag), download it,
build it in a temp location (recursively using sbt), and use it in the build
and test of dependent DFDL schema project. All transparently.
I have tried this with the PCAP DFDL Schema, which, due to recent factoring out
of the ethernetIP DFDL schema, now depends on that separate ethernetIP DFDL
schema.
The build.sbt for PCAP now looks like this:
name := "dfdl-pcap"
organization := "com.tresys"
version := "1.0.0"
scalaVersion := "2.12.14"
libraryDependencies ++= Seq(
//
// We want to get rid of this, because it requires a publishLocal
// or binary server to have this dependency schema present.
// instead we want to just pull the dependent schema from github
//
// "com.owlcyberdefense" % "dfdl-ethernetip" % "1.0.0",
//
"org.apache.daffodil" %% "daffodil-tdml-processor" % "3.1.0" % "test",
"com.novocode" % "junit-interface" % "0.11" % "test",
"junit" % "junit" % "4.12" % "test",
)
testOptions += Tests.Argument(TestFrameworks.JUnit, "-v")
crossPaths := false
//
// To express the dependency explicitly we have to create
// a project and use dependsOn
//
lazy val main = (project in file(".")).dependsOn(ethernetIP)
//
// This is a dependency on ethernetIP at github
// with a specific tag 1.0.0
//
lazy val ethernetIP =
RootProject(uri("git://github.com/DFDLSchemas/ethernetIP.git#1.0.0"))
The magic is the last line. SBT knows how to deal with a git repository and tag
as a dependency.
Now when I push any change to PCAP, the TravisCI run now works, because when
TravisCI runs sbt it will fetch the dependent schema and put it on classpath.
I have verified that this will work with git locations other than github. Ex:
this location works fine for a DI2E.net git schema:
lazy val main = (project in file(".")).dependsOn(geoUtil)
lazy val geoUtil =
RootProject(uri("ssh://[email protected]:7999/dfdl/geoutil.git#1.1"))
This should make the continuous integration for DFDL Schemas work nicely.
In general, this avoids the need for a binary server for DFDL schema jars to
provide sharing, or for users to fetch all the DFDL schemas on the dependency
chain (transitively), do an sbt publishLocal on them, etc.
This also makes it possible to download a DFDL schema as a zip, unzip to a
directory, and immediately 'sbt test' without having to worry about what other
schemas it depends on, as they will be pulled automatically.
You can try this for PCAP. Try these steps:
1. Web Browse https://github.com/DFDLSchemas/PCAP
2. Click the green "Code" button and choose download zip.
3. Unzip into a directory.
4. cd to that directory
5. sbt test
This runs the tests, transparently pulling the ethernetIP DFDL schema from
github for you.
