We also need to consider how to provide releases for different Ni-FI versions
Sincerely, Harinder Sood Senior Program Manager [email protected] 240 805 4219 owlcyberdefense.com The information contained in this transmission is for the personal and confidential use of the individual or entity to which it is addressed. If the reader is not the intended recipient, you are hereby notified that any review, dissemination, or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify the sender immediately. -----Original Message----- From: Steve Lawrence <[email protected]> Sent: Wednesday, September 3, 2025 8:59 AM To: [email protected] Subject: Re: [DISCUSS] Release Apache Daffodil v4.0.0 and Apache Daffodil SBT Plugin v1.5.0 Agreed about the need for testing multiple versions. It might be convenient if we could come up with a solution that doesn't require multiple branches. Multiple branches requires extra work to keep the branches in sync, and also makes it difficult to share common code or tests. I'm imagining a project structure where there could be multiple sub-projects, each one designed for a specific Daffodil version. The projects would share the src and test directories, so would only differ in things like dependencies to Daffodil or plugins. And for situations where code differences are needed between versions (e.g. plugins) we could have daffodil version specific directories, e.g. src/main/scala # code shared between all daffodil versions src/main/scala-daffodil3110 # code only used for daffodil 3.11.0 sub project src/main/scala-daffodil400 # code only used for daffodil 4.0.0 sub project This would also work for src/main/resources and src/test. This is kindof similar to how sbt supports projects building with Scala 2.x vs 3.x, and I think is not too difficult to do in SBT plugins. To support Daffodil plugins, I think we could have a new sbt-daffodil setting that defines daffodil plugin dependencies and each subproject would mutate that to depend on the right version. So an alternative SBT configuration might look something like this: name := "myFormat" version := "1.0.0" libraryDependencies := Seq(...) // normal non-plugin dependencies enablePlugin(DaffodilPlugin) daffodlPluginDependencies := Seq( "com.example.layers" %% "checksum" % "1.0.0" ) daffodilProjectVersion("3.11.0") daffodilProjectVersion("4.0.0") That would create subprojects for 3.11.0 and 4.0.0, so you could do something like: sbt daffodil3110/test sbt daffodil400/test And something like "sbt publish" would publish the main schema jar and saved parsers if configured. Each of those subprojects would depend on the "checksum" plugin but they would depend on a slightly modified names (e.g. "checksum_daffodil3110" vs "checksum_daffodil400"). Additional features would be needed to publish plugins with multiple versions and mutate the name to match. I imagine that would work similar to the multiple subprojects. I think there's still alot of details to work out, and it needs some testing to figure out if it will actually work, but I think in theory it's possible to not require branches so common code and tests can be easily shared and all tests run without needing to change branches. And I think it could be done without too much boilerplate. - Steve On 2025-09-03 08:17 AM, Mike Beckerle wrote: > +1 for creating a 4.0.0 release > > To me the biggest thing needed is not anything to hold up the release, > it's documentation on how to evolve a DFDL schema project on github > (or similar) and package server (e.g., like Artifactory) in such a way > that it can be maintained to work with Daffodil 3.7.0, 3.10.0, 3.11.0, > and 4.0.0 (and > subsequent) releases simultaneously, easily rebuilt and tested in > regression, etc. I pick those because the API (specifically packages) > changed after 3.7.0, 3.10.0 is the last of the Scala 2.12 releases, > 3.11.0 is Scala 2.13, and 4.0.0 is Scala 3. > > There are two scenarios - First is a pure schema - single component. > The DFDLSchemas FakeTDL is one such. > > The second is a complex multi-component schema. The PCAP schema on > DFDLSchemas is such. It uses EthernetIP as a component and that has a > Daffodil layer extension to compute IPv4 checksums. This should > illustrate all the challenges. > > I think this methodology is going to require use of multiple git > branches, since there are clearly code changes required for the > layers. I think these use only standard TDML tests however, so ad-hoc > test rigs shouldn't muddy the waters. But despite these git branches, > much of the schema will be independent of them, and an objective > should be to share what can be shared so as not to do too much cross-branch > duplication of changes. > > > > > > > > On Tue, Sep 2, 2025 at 3:03 PM Steve Lawrence <[email protected]> wrote: > >> Hi all, >> >> Although Daffodil 3.11.0 was release fairly recently, the current >> main branch of Daffodil has a number of major improvements, including >> switching to Scala 3, dropping official support for Java 8 and 11, a >> much improved API, and a number of important bug fixes. With these >> major changes complete, I think now would be a good time to release >> Daffodil 4.0.0. >> >> For reference, here is a page that describes all the breaking changes >> and how to migrate to the new API: >> >> https://daffodil.apache.org/migration-guides/4.0.0/ >> >> We should also plan to release the Daffodil SBT plugin 1.5.0 >> concurrently, since it has modifications needed to work with Daffodil >> 4.0.0. >> >> There are a dozen or so open pull requests to update dependencies >> that I think we can merge in the coming days in time for 4.0.0. >> Please let us know if there are any other issues you think should be >> fixed. >> >> If there are no additional changes or objections, I will volunteer as >> the release manager and plan to start the vote on Monday, Sep 8. >> >> Thanks, >> - Steve >> >
