Mike,
After re-reading your email I may have misinterpreted some things. Here is
essentially the process I used to update all the schemas in our regression
suite to work against Daffodil 3.11.0.
For relatively simple schema projects (no Scala plugins):
*
Update all test rig code to be compatible with both Scala 2.12 and Scala 2.13
*
With updated version of sbt-daffodil, doing "sbt set daffodilPackageBinVersions
:= Seq("3.10.0", "3.11.0"); packageDaffodilBin" will build saved parsers for
both Daffodil 3.10.0 and 3.11.0.
For a more complicated project containing Scala plugins:
*
Update all test rig code to be compatible with both Scala 2.12 and Scala 2.13
*
Update all plugin code to be compatible with both Scala 2.12 and Scala 2.13
*
Publish a 2.12 version of the plugins
*
Publish a 2.13 version of the plugins
*
With updated version of sbt-daffodil set daffodilVersion := "3.10.0" and
daffodilPackageBinVersions := Seq("3.10.0") to build saved parsers against
Daffodil 3.10.0.
*
With updated version of sbt-daffodil set daffodilVersion := "3.11.0" and
daffodilPackageBinVersions := Seq("3.11.0") to build saved parsers against
Daffodil 3.11.0.
I believe that this is all I needed to do to get the schema projects working
with Daffodil 3.11.0 while also being relatively compatible with previous
versions. Note that for some Scala plugins like the zip layer transformer
doing ""sbt set daffodilPackageBinVersions := Seq("3.10.0", "3.11.0");
packageDaffodilBin" was able to generate saved parsers for both 3.10.0 and
3.11.0. This did not work with projects that required the ExtendedCharsets
plugin.
I still think that right now it is inadvisable to attempt to target Daffodil
3.11.0 until the updated sbt-daffodil plugin is ready as currently it will not
work at all if you attempt to target Daffodil 3.11.0. It might also be
possible to improve it's functionality regarding Scala pl;ugins, but we need to
investigate why some plugins can work for both Scala 2.12 and 2.13, but not
others.
Should I add a wiki page detailing this process?
Josh Adams
________________________________
From: Adams, Joshua <jad...@owlcyberdefense.com.INVALID>
Sent: Monday, June 2, 2025 11:53 AM
To: dev@daffodil.apache.org <dev@daffodil.apache.org>
Subject: Re: [VOTE] Release Apache Daffodil 3.11.0-rc2
Mike,
I think this might be a bit of a chicken and egg problem. Correct me if I'm
wrong, but I believe we were waiting for the release of Daffodil 3.11.0 before
performing the next release of sbt-daffodil. Without the updates to
sbt-daffodil, there a lot more manual steps that need to be taken. With the
changes in the upcoming sbt-daffodil release the only change I've had to make
for targeting older versions of Daffodil is to just set the corresponding
daffodilVersion and daffodilPackageBinVersions in whatever schema project I'm
in as well as it's dependencies.
Maybe in the future it makes more sense to perform the release of sbt-daffodil
(if changes are required) before releasing Daffodil so that we can provide
better transition instrucitons?
Josh
________________________________
From: Mike Beckerle <mbecke...@apache.org>
Sent: Monday, June 2, 2025 11:42 AM
To: dev@daffodil.apache.org <dev@daffodil.apache.org>
Subject: Re: [VOTE] Release Apache Daffodil 3.11.0-rc2
As of right now I'm considering voting -1.
I will change my vote if we provide sufficient documentation which may not
require a new RC3. Rationale follows.
I did not have time to thoroughly evaluate this release yet. But that's not
for lack of trying. It is because it is entirely unclear to me how to
evaluate it.
What is the methodology going forward to maintain a DFDL schema that must
work with say, Daffodil 3.5.0, Daffodil 3.9.0, and Daffodil 3.11.0, given
plug-ins, and test-rig code. Test rig code is scala and must be ported to
Scala 2.13.0. But, if the schema is to continue to work with Daffodil
3.10.0 or earlier then the code must remain Scala 2.12 linkable. Plugins
must be compiled for scala 2.12 and 2.13 and maintained side by side like
that.
I'd like to see specific instructions on how to cope with this transition.
What the sbt-daffodil plug in does for you, vs. what you must do yourself
by branching your DFDL schema project is entirely unclear.
Letting our users have to figure this out by trial-and-error is not
reasonable, particularly if it is not even clear to PMC insiders like me
how to best do this, or if it is even possible without maintaining a
branch/fork of all one's DFDL schemas corresponding to each Daffodil
release.
On Mon, Jun 2, 2025 at 11:00 AM Kilo, Olabusayo
<ok...@owlcyberdefense.com.invalid> wrote:
> +1 (binding)
>
> I checked:
>
> [NOT OK] links in email are correct except
> https://s.apache.org/daffodil-issues-3.11.0 which is giving a "can't view
> this page" error. I'm not logged in to jira, but I assume users shouldn't
> need a jira account to view the issues
> [OK] verified Summary of Changes pages (maven, sbt, download and dnf)
> [OK] verified download of binary and source
> [OK] Successfully used apache-daffodil-3.11.0-bin.tgz
> [OK] JavaDoc and ScalaDoc are correct
>
>
> --
> Lola Kilo
> ________________________________
> From: Adams, Joshua <jad...@owlcyberdefense.com.INVALID>
> Sent: Friday, May 30, 2025 2:18 PM
> To: dev@daffodil.apache.org <dev@daffodil.apache.org>
> Subject: Re: [VOTE] Release Apache Daffodil 3.11.0-rc2
>
> [You don't often get email from jad...@owlcyberdefense.com.invalid. Learn
> why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> +1
>
> I checked:
>
> [OK] hashes and signatures of source and helper binaries are correct
> [OK] source and helper binaries are 100% reproducible
> [OK] ~80 public and private DFDL schema projects pass tests
>
> Did notice one small bug, but not worth holding up a release - see
> https://issues.apache.org/jira/browse/DAFFODIL-3001
>
> Also ran some preliminary performance measurements across some schema
> projects. Seeing a consistent 5-10% improvement in throughput/latency
> across both parsing and unparsing when compared to Daffodil 3.10.0.
>
> Josh Adams
> ________________________________
> From: Steve Lawrence <slawre...@apache.org>
> Sent: Friday, May 30, 2025 7:36 AM
> To: dev@daffodil.apache.org <dev@daffodil.apache.org>
> Subject: Re: [VOTE] Release Apache Daffodil 3.11.0-rc2
>
> +1 (binding)
>
> I checked:
>
> [OK] hashes and signatures of source and helper binaries are correct
> [OK] source and helper binaries are 100% reproducible
> [OK] signature of git tag verifies
> [OK] source release matches git tag (minus KEYS file)
> [OK] source compiles and all tests pass (both en_US and de_DE)
> [OK] helper tgz/zip/msi/rpm/maven have the same content except where
> expected
> [OK] src, binaries, and jars include correct LICENSE/NOTICE
> [OK] RAT check passes
> [OK] no unexpected binaries in source
> [OK] rpm and msi install and run with basic usage
> [OK] ~80 public and private DFDL schema projects pass tests
> [OK] no issues found in JavaDoc and ScalaDoc
> [OK] no open CVEs found using sbt dependencyCheck
> [OK] Daffodil NiFi processor builds and tests pass
> [OK] Daffodil SBT plugin builds and tests pass
>
>
> On 2025-05-23 02:02 PM, Adams, Joshua wrote:
> > Hi all,
> >
> > I'd like to call a vote to release Apache Daffodil 3.11.0-rc2.
> >
> > All distribution packages, including signatures, digests, etc. can be
> found at:
> >
> > https://dist.apache.org/repos/dist/dev/daffodil/3.11.0-rc2/
> >
> > Staging artifacts can be found at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachedaffodil-1052/
> >
> > This release has been signed with PGP key 989D39300ACC1866,
> corresponding to jad...@apache.org, which is included in the KEYS file
> here:
> >
> > https://downloads.apache.org/daffodil/KEYS
> >
> > The release candidate has been tagged in git with v3.11.0-rc1.
> >
> > For reference, here is a list of all resolved JIRA issues tagged with
> 3.11.0:
> >
> > https://s.apache.org/daffodil-issues-3.11.0
> >
> > For a summary of the changes in this release, see:
> >
> > https://daffodil.apache.org/releases/3.11.0/
> >
> > Please review and vote. The vote will be open for at least 72 hours
> (Friday, 23 May 2025, 2:00 PM EST).
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > Thanks,
> > - Josh
>
>
