stevedlawrence commented on code in PR #183:
URL: https://github.com/apache/daffodil-sbt/pull/183#discussion_r3189319326
##########
src/main/scala/org/apache/daffodil/DaffodilPlugin.scala:
##########
@@ -840,6 +859,288 @@ object DaffodilPlugin extends AutoPlugin {
}
}
+ def flattenerSettings: Seq[Setting[_]] = Seq(
+ flattenTarget := target.value / s"${name.value}-${version.value}-flat.zip",
+ /* Paths in flattenExcludes/Includes that are not globbed at the start are
+ * generally only going to match paths within JAR files on the classpath.
+ * In order to deal with paths on the filesystem we need to glob the start
+ * of the path to account for different directory structure before the
+ * schema project path.
+ */
+ flattenExcludes := Seq(
+ // "**/src/test/resources/**",
+ // "Log4j*.xsd",
+ // "xsd/**", // This is coming from XSAT2
+ // "IBMdefined/**",
+ // "org/apache/xml/**",
+ // "edu/illinois/ncsa/daffodil/**",
+ "org/apache/daffodil/**"
+ // "**/*-tests.jar",
+ // "META-INF/**",
+ // "com/ibm/icu/**",
+ // "eclipse-xml-catalog.xml",
+ // "daffodil-built-in-catalog.xml"
+ ),
+ flattenIncludes := Seq(
+ "org/apache/daffodil/xsd/DFDLGeneralFormat*.dfdl.xsd"
+ ),
+
+ /**
+ * Whether or not to publish the flattened schemas zip. Defaults to false.
+ *
+ * If projects want to publish flattened schemas then they must explicitly
enable it by
+ * setting 'flattenSchemas / publishArtifact := true'.
+ *
+ * If false, flattened schemas will not be created unless you explicitly
run the
+ * flattenSchemas.
+ */
+ flattenSchemas / publishArtifact := false,
+
+ flattenSchemas / artifact := Artifact(
+ name.value,
+ "flat",
+ "zip",
+ Some("flat"),
+ Vector(),
+ None
+ ),
+ flattenSchemas := {
+
+ val logger = streams.value.log
+
+ val extractDir = Paths.get(target.value.getPath(), "flatExtractDir")
+ if (Files.exists(extractDir))
+ IO.delete(extractDir.toFile)
+ Files.createDirectory(extractDir)
+
+ val flatDir = Paths.get(target.value.getPath(), "flatDir")
+ if (Files.exists(flatDir))
+ IO.delete(flatDir.toFile)
+ Files.createDirectory(flatDir)
+
+ val projectXsdFiles = (Compile / resourceDirectories).value
+ .flatMap { dir => (dir ** "*.xsd").get }
+ .map(path => Paths.get(path.toString))
+ val projectXslFiles = (Compile / resourceDirectories).value
+ .flatMap { dir => (dir ** ("*.xsl" || "*.xslt")).get }
+ .map(path => Paths.get(path.toString))
+ val projectXmlFiles = (Compile / resourceDirectories).value
+ .flatMap { dir => (dir ** "*.xml").get }
+ .map(path => Paths.get(path.toString))
+
+ /* Copy schema files from the current project's src/main/resources
+ * directory
+ */
+ val filesFromProject =
+ (projectXsdFiles ++ projectXslFiles ++ projectXmlFiles).filterNot {
path =>
+ val matchesExcludes = flattenExcludes.value.exists(glob =>
glob.matches(path))
+ val matchesIncludes = flattenIncludes.value.exists(glob =>
glob.matches(path))
+ matchesExcludes && !matchesIncludes
+ }
+
+ val extractedProjectFiles = filesFromProject.map { origPath =>
+ val resourcePath = Paths
+ .get((Compile / resourceDirectories).value(0).toString)
+ .relativize(origPath)
+ .toString
+ val newPath = Paths.get(extractDir.toString, resourcePath)
+ Files.createDirectories(newPath.getParent())
+ Files.copy(origPath, newPath, StandardCopyOption.REPLACE_EXISTING)
+ newPath
+ }
+
+ /* Get all dependency jars and resources specific to this project. Note
+ * that the "Test" configuration is used for JAR files in order to ensure
+ * we pull in XSD files from daffodil-lib, as in many schema projects the
+ * daffodil dependencies are only used for testing, not compiling. Also
+ * note that we want to use Test/externalDependencyClasspath to get
+ * dependency jars, and not something like Test/fullClasspath or
+ * Test/dependencyClasspath, since those could trigger expensive resource
+ * generators or compilation of internal test jars that we don't need--we
+ * only need Compile/resources from this project and Test/dependency jars
+ * from external projects
+ */
+ val projectJarFiles = (Test /
externalDependencyClasspath).value.files.flatMap { file =>
+ (file ** "*.jar").get
+ }
+
+ projectJarFiles.reverse.map { jar =>
Review Comment:
Can you add a comment that explains that?
Alternatively, maybe what we should do is each time we find a reference we
should look for that file in the jar files by scanning the classpath in the
order the classpath is defined and extract the first file found? That's
essentially what Daffodil will do when it compiles things, so having the same
behavior ensures the same files are found. Using reverse to me isn't
immediately clear that we have the same behavior.
And we could probably do this by creating a `URLClassloader` based off the
classpath and then use getResource to find files, which takes into account the
order of the classpath and should return the right one. For example, the plugin
could do something like this:
```scala
val allClasspathURLs = (Compile / fullClasspath).value.map(_.toURI.toURL)
val classLoader = new URLClassLoader(allClasspathURLs, null)
// when we need to resolve a reference:
val url = classLoader.getResource(includeImportReference)
// extract url from the jar add it to the list of files to be flattened
```
This has the benefit that we don't have to extract all the jars using a
JarFileSystem into a temp directory, instead we just query the custom class
loader for individual files as we determine they are needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
