vscode repo tag at start of our work

[Mike Beckerle]

Anyone mind if I push a long, highly-visible, tag
"Start-Apache-Daffodil-VSCode" to the daffodil-vscode repo at this
commit: 793e7e7e2cb4a303d6be865922dfb80afbca46bc
(last commit before our team, from a microsoft contributor)

This is just to make it easier when visualizing the history in tools like
gitk or qgit, etc.

Re: [DISCUSS] need to release Daffodil 3.3.0

[Steve Lawrence]

Agreed. +1 to start the release processes once these are resolved.

On 3/10/22 6:05 PM, Interrante, John A (GE Research, US) wrote:

Fixing these 3 tickets seems sufficient to me.

-Original Message-
From: Mike Beckerle 
Sent: Wednesday, March 9, 2022 6:00 PM
To: dev@daffodil.apache.org
Subject: EXT: Re: [DISCUSS] need to release Daffodil 3.3.0

Once the revert/fix for regressions is merged, I think we're down to just these 
3 tickets as really critical for Daffodil 3.3.0 release:

DAFFODIL-2652  - Ability 
to disable all alignment

Given the number of outstanding bugs associated with unparser deadlock and 
alignment I think this feature is an important hedge allowing progress to be 
made by schema authors even if they run into these unparser/alignment related 
issues (like DAFFODIL-2662 or DAFFODIL-2666 which are hard to fix, and I think 
we don't want to hold back the release for those fixes because they will take a 
while.)

DAFFODIL-2650  - using 
config file with cli parse or save parser causes backtrace

Major usability issue when dealing with DFDL schemas that issue many warnings.

DAFFODIL-2267  - Warnings 
emitted on pre-compiled parsers

Major usability issue when dealing with DFDL schemas that issue many warnings 
in deployed Daffodil-based applications. Clutters the log with too many things 
users have to know can be ignored.

I will say these latter 2 bugs have been a huge pain in the neck to me of late 
in debugging efforts associated with some DFDL schema work. They just so 
clutter the output that you really can't see what is going on sometimes.

Thoughts? Is fixing these enough for us to do a release of 3.3.0 ?

On Wed, Mar 9, 2022 at 2:06 PM Mike Beckerle  wrote:


I just opened a PR which reverts a change which fixed a bug
(DAFFODIL-2626), but caused a number of regressions detected only by
other DFDL schemas such as NITF. (DAFFODIL-2666 and DAFFODIL-2662 are
regressions it caused.)

The original bug is preferable to these regressions.

This will get us closer to a releasable 3.3.0.





On Wed, Mar 2, 2022 at 2:12 PM Mike Beckerle  wrote:


I've marked all the alignment/cyclic-deadlock regressions as blockers
for
3.3.0 along with the "hammer" to
just turn off alignment.

The fixing suggested in the thread here may be the fix, or the "hammer"
fix, but the regressions on unparsing have to be addressed in 3.3.0,
i.e., asap, before we can release it.

I think other things we "almost" got working, like prefixed length
fixes (of various bugs) could wait for a later release.

There are numerous user projects I know about that are depending on
3.3.0 coming out quite soon now, without regressions.


On Wed, Feb 23, 2022 at 11:19 AM Steve Lawrence

wrote:


I assume this is caused by alignment regions not getting optimized
out with the recent changes to the alignment algorithm. It's now
more correct, but it's more pessimistic.

A hammer to just disable alignment might be a reasonable solution,
but I'd be concerned there are alignment regions that are needed,
it's not usually obvious, especially in complex schemas.

I think the main change that causes regions to fail to optimize out
is that we can't optimize out alignment related to global
declarations because we don't know the alignment of the references.

I added comments in
https://issues.apache.org/jira/browse/DAFFODIL-2626
that discuss this issue, and a potential fixe. I believe we just
need to require that alignment of global decl's to be the same as
their references. I hope that this would allow more optimization of
alignment regions. One issue was raised about global complexType's,
who's alignment only comes from the references, with no information
on the declaration. So that also causes issues with this approach.

I think implementing one or both of these options as tunables might
help improve the alignment issue and would be reasonable to get in 3.3.0.


On 2/23/22 11:08 AM, Mike Beckerle wrote:

So, we seem to be seeing a lot of regressions in various DFDL
schemas

like

most recently NITF, previously PNG.

What if users run into this in their own DFDL schemas?

These are showing unparser deadlocks due to cyclic relationships.
At

one

time we discussed adding a "big hammer" property or tunable that
simply turns off alignment, as a workaround for all these sorts of
alignment issues. I am wondering if we will need that so that
users can work

around

these alignment issues in their schemas.

Changing these schemas for 3.3.0 compatibility is highly
undesirable

(as

was done for PNG), even if the changes are backward compatible.

(Though if the schemas are actually incorrect in some way that
we're

now

detecting more effectively, that is the right fix.)



On Tue, Feb 22, 2022 at 11:38 AM Interrante, John A (GE Research,
US) < john.interra...@ge.com> wrote:


+1

I