I just opened a PR which reverts a change which fixed a bug (DAFFODIL-2626), but caused a number of regressions detected only by other DFDL schemas such as NITF. (DAFFODIL-2666 and DAFFODIL-2662 are regressions it caused.)
The original bug is preferable to these regressions. This will get us closer to a releasable 3.3.0. On Wed, Mar 2, 2022 at 2:12 PM Mike Beckerle <mbecke...@apache.org> wrote: > I've marked all the alignment/cyclic-deadlock regressions as blockers for > 3.3.0 along with the "hammer" to > just turn off alignment. > > The fixing suggested in the thread here may be the fix, or the "hammer" > fix, but the regressions on unparsing have to be addressed in 3.3.0, i.e., > asap, before we can release it. > > I think other things we "almost" got working, like prefixed length fixes > (of various bugs) could wait for a later release. > > There are numerous user projects I know about that are depending on 3.3.0 > coming out quite soon now, without regressions. > > > On Wed, Feb 23, 2022 at 11:19 AM Steve Lawrence <slawre...@apache.org> > wrote: > >> I assume this is caused by alignment regions not getting optimized out >> with the recent changes to the alignment algorithm. It's now more >> correct, but it's more pessimistic. >> >> A hammer to just disable alignment might be a reasonable solution, but >> I'd be concerned there are alignment regions that are needed, it's not >> usually obvious, especially in complex schemas. >> >> I think the main change that causes regions to fail to optimize out is >> that we can't optimize out alignment related to global declarations >> because we don't know the alignment of the references. >> >> I added comments in https://issues.apache.org/jira/browse/DAFFODIL-2626 >> that discuss this issue, and a potential fixe. I believe we just need to >> require that alignment of global decl's to be the same as their >> references. I hope that this would allow more optimization of alignment >> regions. One issue was raised about global complexType's, who's >> alignment only comes from the references, with no information on the >> declaration. So that also causes issues with this approach. >> >> I think implementing one or both of these options as tunables might help >> improve the alignment issue and would be reasonable to get in 3.3.0. >> >> >> On 2/23/22 11:08 AM, Mike Beckerle wrote: >> > So, we seem to be seeing a lot of regressions in various DFDL schemas >> like >> > most recently NITF, previously PNG. >> > >> > What if users run into this in their own DFDL schemas? >> > >> > These are showing unparser deadlocks due to cyclic relationships. At one >> > time we discussed adding a "big hammer" property or tunable that simply >> > turns off alignment, as a workaround for all these sorts of alignment >> > issues. I am wondering if we will need that so that users can work >> around >> > these alignment issues in their schemas. >> > >> > Changing these schemas for 3.3.0 compatibility is highly undesirable (as >> > was done for PNG), even if the changes are backward compatible. >> > >> > (Though if the schemas are actually incorrect in some way that we're now >> > detecting more effectively, that is the right fix.) >> > >> > >> > >> > On Tue, Feb 22, 2022 at 11:38 AM Interrante, John A (GE Research, US) < >> > john.interra...@ge.com> wrote: >> > >> >> +1 >> >> >> >> I personally have no blocker or urgent issues that must be fixed before >> >> the next release (only some things I will need to start working on in >> the C >> >> backend, "Runtime 2," to handle some more complicated schemas). >> >> >> >> How will the roadmap for upcoming releases ( >> >> >> https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases >> ) >> >> change as a result of 3.3.0 being released asap? >> >> >> >> John >> >> >> >> -----Original Message----- >> >> From: Mike Beckerle <mbecke...@apache.org> >> >> Sent: Tuesday, February 22, 2022 10:44 AM >> >> To: dev@daffodil.apache.org >> >> Subject: EXT: [DISCUSS] need to release Daffodil 3.3.0 >> >> >> >> WARNING: This email originated from outside of GE. Please validate the >> >> sender's email address before clicking on links or attachments as they >> may >> >> not be safe. >> >> >> >> A number of people are asking for 3.3.0, with its many bug fixes, to be >> >> released asap. >> >> >> >> Are there any remaining issues that must be fixed before this release? >> >> >> >> Otherwise I'd like to suggest we release 3.3.0. >> >> >> > >> >>
