Due on Sept 8 for board meeting on Sept 10. For comments/additions. I'll fix all the long lines and such later so ignore the formatting. VSCode folks please make sure this reflects your current status/objectives.
I do want people to consider the tone here. I don't want to over state things here, but something like 1/2 our developer bandwidth on the Daffodil library was used up by this for the last 6 to 8 months on the Scala version ports, and to me this has added very little value for our users and I fear until we add some substantial direct value (like a big performance boost over 3.11.0) it will be hard to get anyone to upgrade to 4.0.0. --------- ## Description: The mission of Apache Daffodil is the creation and maintenance of software related to an implementation of the Data Format Description Language (DFDL) used to convert between fixed format data and more readily processed forms such as XML or JSON ## Project Status: Current project status: Ongoing. Moderate activity. Issues for the board: None. ## Membership Data: Apache Daffodil was founded 2021-02-16 (4 years ago) There are currently 19 committers and 18 PMC members in this project. The Committer-to-PMC ratio is roughly 1:1. Community changes, past quarter: - No new PMC members. Last addition was Peter Katlic on 2024-03-17. - No new committers. Last addition was Peter Katlic on 2024-03-17. ## Project Activity: The past 6 to 8 months, the project has mostly been dealing with the evolution of our underlying Scala platform. Porting from Scala 2.12 to Scala 2.13 was quite challenging, as much of the Scala built-in XML support that we were strongly dependent on, was removed in Scala 2.13. The subsequent port to Scala 3 required significant changes that will impact all API-level Daffodil users. Daffodil 3.11.0 was released on 2025-06-17 along with the matching Daffodil SBT plugin 1.4.0 and Daffodil NiFi 1.21. This is the last release using Scala 2 technology, specifically the LTS Scala 2.13 version. If necessary we can support this longer term as needed for Daffodil library users who have fielded products using our Scala 2 code base who are unable to upgrade (more on this below) to the forthcoming Daffodil 4.0.0 any time soon. The Daffodil VSCode Extension 1.4.1 was released on 2025-06-30 and includes better overall error handling, improved reliability of Data Editor, correcting several schema completion usability issues and creating new-user onboarding and new developer onboarding documentation. Daffodil 4.0.0 is in preparation but imminent. This major release uses Scala 3 technology, and includes an entirely new and improved API necessitated by Scala-Java interoperability changes in Scala 3. The Daffodil API no longer has a Java/Scala dual API. It is defined entirely in Java. In addition to this major API discontinuity, we have chosen to fix issues that would also require a major version change - some non-conformities to the DFDL specification are fixed, for example. Daffodil 4.0.0 is clearly better than the prior versions and the API is more supportable, but the vast bulk of the required changes added no value for our users, who may wait for more value-add in subsequent releases before going through the upgrade and testing pain. ## Community Health: Good activity level in developer email and commit activity. User list activity has dropped, and this is problematic, as is activity of the related DFDL Workgroup at the Open Grid Forum/ISO. Our current user community is defense cyber-security and that world is happy using XML technology as a basis. But broader dev communities no longer use XML and there seem to be long term risks to use of XML technologies extending beyond just Daffodil. The DFDL schema language is based on XML Schema, and this is a sufficient barrier for adoption by developers. This is not new news as we have known about this since the time we were an incubator project. But this remains a challenge for us in that current XML-oriented users want performance improvements, bug fixes, and long-term supportability (e.g., Scala 3) and this easily uses up the development capabilities of our whole dev community, yet a reinvention of the technology independent of XML Schema is necessary to attract a more diverse community of developers in the long term. Our project roadmap already reflects these needs and we need to make progress on it.
