Once the revert/fix for regressions is merged, I think we're down to just these 3 tickets as really critical for Daffodil 3.3.0 release:
DAFFODIL-2652 <https://issues.apache.org/jira/browse/DAFFODIL-2652> - Ability to disable all alignment Given the number of outstanding bugs associated with unparser deadlock and alignment I think this feature is an important hedge allowing progress to be made by schema authors even if they run into these unparser/alignment related issues (like DAFFODIL-2662 or DAFFODIL-2666 which are hard to fix, and I think we don't want to hold back the release for those fixes because they will take a while.) DAFFODIL-2650 <https://issues.apache.org/jira/browse/DAFFODIL-2650> - using config file with cli parse or save parser causes backtrace Major usability issue when dealing with DFDL schemas that issue many warnings. DAFFODIL-2267 <https://issues.apache.org/jira/browse/DAFFODIL-2267> - Warnings emitted on pre-compiled parsers Major usability issue when dealing with DFDL schemas that issue many warnings in deployed Daffodil-based applications. Clutters the log with too many things users have to know can be ignored. I will say these latter 2 bugs have been a huge pain in the neck to me of late in debugging efforts associated with some DFDL schema work. They just so clutter the output that you really can't see what is going on sometimes. Thoughts? Is fixing these enough for us to do a release of 3.3.0 ? On Wed, Mar 9, 2022 at 2:06 PM Mike Beckerle <mbecke...@apache.org> wrote: > I just opened a PR which reverts a change which fixed a bug > (DAFFODIL-2626), but caused a number of regressions detected only by other > DFDL schemas such as NITF. (DAFFODIL-2666 and DAFFODIL-2662 are regressions > it caused.) > > The original bug is preferable to these regressions. > > This will get us closer to a releasable 3.3.0. > > > > > > On Wed, Mar 2, 2022 at 2:12 PM Mike Beckerle <mbecke...@apache.org> wrote: > >> I've marked all the alignment/cyclic-deadlock regressions as blockers for >> 3.3.0 along with the "hammer" to >> just turn off alignment. >> >> The fixing suggested in the thread here may be the fix, or the "hammer" >> fix, but the regressions on unparsing have to be addressed in 3.3.0, i.e., >> asap, before we can release it. >> >> I think other things we "almost" got working, like prefixed length fixes >> (of various bugs) could wait for a later release. >> >> There are numerous user projects I know about that are depending on 3.3.0 >> coming out quite soon now, without regressions. >> >> >> On Wed, Feb 23, 2022 at 11:19 AM Steve Lawrence <slawre...@apache.org> >> wrote: >> >>> I assume this is caused by alignment regions not getting optimized out >>> with the recent changes to the alignment algorithm. It's now more >>> correct, but it's more pessimistic. >>> >>> A hammer to just disable alignment might be a reasonable solution, but >>> I'd be concerned there are alignment regions that are needed, it's not >>> usually obvious, especially in complex schemas. >>> >>> I think the main change that causes regions to fail to optimize out is >>> that we can't optimize out alignment related to global declarations >>> because we don't know the alignment of the references. >>> >>> I added comments in https://issues.apache.org/jira/browse/DAFFODIL-2626 >>> that discuss this issue, and a potential fixe. I believe we just need to >>> require that alignment of global decl's to be the same as their >>> references. I hope that this would allow more optimization of alignment >>> regions. One issue was raised about global complexType's, who's >>> alignment only comes from the references, with no information on the >>> declaration. So that also causes issues with this approach. >>> >>> I think implementing one or both of these options as tunables might help >>> improve the alignment issue and would be reasonable to get in 3.3.0. >>> >>> >>> On 2/23/22 11:08 AM, Mike Beckerle wrote: >>> > So, we seem to be seeing a lot of regressions in various DFDL schemas >>> like >>> > most recently NITF, previously PNG. >>> > >>> > What if users run into this in their own DFDL schemas? >>> > >>> > These are showing unparser deadlocks due to cyclic relationships. At >>> one >>> > time we discussed adding a "big hammer" property or tunable that simply >>> > turns off alignment, as a workaround for all these sorts of alignment >>> > issues. I am wondering if we will need that so that users can work >>> around >>> > these alignment issues in their schemas. >>> > >>> > Changing these schemas for 3.3.0 compatibility is highly undesirable >>> (as >>> > was done for PNG), even if the changes are backward compatible. >>> > >>> > (Though if the schemas are actually incorrect in some way that we're >>> now >>> > detecting more effectively, that is the right fix.) >>> > >>> > >>> > >>> > On Tue, Feb 22, 2022 at 11:38 AM Interrante, John A (GE Research, US) < >>> > john.interra...@ge.com> wrote: >>> > >>> >> +1 >>> >> >>> >> I personally have no blocker or urgent issues that must be fixed >>> before >>> >> the next release (only some things I will need to start working on in >>> the C >>> >> backend, "Runtime 2," to handle some more complicated schemas). >>> >> >>> >> How will the roadmap for upcoming releases ( >>> >> >>> https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases >>> ) >>> >> change as a result of 3.3.0 being released asap? >>> >> >>> >> John >>> >> >>> >> -----Original Message----- >>> >> From: Mike Beckerle <mbecke...@apache.org> >>> >> Sent: Tuesday, February 22, 2022 10:44 AM >>> >> To: dev@daffodil.apache.org >>> >> Subject: EXT: [DISCUSS] need to release Daffodil 3.3.0 >>> >> >>> >> WARNING: This email originated from outside of GE. Please validate the >>> >> sender's email address before clicking on links or attachments as >>> they may >>> >> not be safe. >>> >> >>> >> A number of people are asking for 3.3.0, with its many bug fixes, to >>> be >>> >> released asap. >>> >> >>> >> Are there any remaining issues that must be fixed before this release? >>> >> >>> >> Otherwise I'd like to suggest we release 3.3.0. >>> >> >>> > >>> >>>
