I'd be interested to hear others thoughts, but this sounds like the right behavior to me.
The use of isDefinedForLength for complex types is flawed--it is very reasonable for complex types to be too large to cache so we cannot call it to immediately determine if enough data exists for a complex type. Simple types (that aren't blobs) should never be as big as our cache, so it is reasonable to maintain the check in that case. Removing this will break some assumptions, such as isBitLimitOk always being true. It's possible there are others that some of these failing test will tease out. And it will also change the parse behavior which could cause some tests to run differently. Rather than immediately failing when there is not enough data for a complex type, it instead will only fail after parsing the children of the complex type and trying to skip any remaining bits. This means that the children of specified complex types will always be parsed, whereas in the past they would not be parsed if there was not enough data. If a test is specifically testing not enough data for complex types, the test should be fixed so that those child elements succeed. There may be other tweaks needed to tests depending on what tests are trying to accomplish. On 9/22/20 7:12 PM, Carlson, Ian wrote: > While pursuing DAFFODIL-2395 > <https://issues.apache.org/jira/browse/DAFFODIL-2395>, I experimented with > SpecifiedLengthParsers.scala, specifically the check isDefinedForLength. I > added > logic to only do this check for non-complex types so we wouldn’t look ahead > across types that might be so large that it breaks our backtracking. > > This immediately caused problems with the Assert.invariant(isLimitOk). It > appears that isLimitOk can now be false when the parser backtracks on points > of > uncertainty. As an attempted fix, I replaced the assert with a processing > error > for complexTypes. > > This causes some 50 tests to fail with new parse error messages, presumably > because we aren’t backtracking in the same way as before. > > This seems like it might require some additional thought, so I wanted to get > more eyes on the problem. Does the above solution make sense, or do these > results spark any new opinions? > > A picture containing object, clock Description automatically generatedIan > Carlson | Software Engineer > > Owl Cyber Defense > > W_icarlson@_owlcyberdefense.com <https://owlcyberdefense.com/> > > Connect with us! > > Facebook <https://www.facebook.com/owlcyberdefense/> | LinkedIn > <https://www.linkedin.com/company/owlcyberdefense/>| Twitter > <https://twitter.com/owlcyberdefense> > > Find us at our next event. Click Here. > <https://owlcyberdefense.com/resources/events/?utm_source=owl-cyber-defense&utm_medium=email&utm_content=banner&utm_campaign=2020-events> > > The information contained in this transmission is for the personal and > confidential use of the individual or entity to which it is addressed. > > If the reader is not the intended recipient, you are hereby notified that any > review, dissemination, or copying of this communication is strictly > prohibited. > > If you have received this transmission in error, please notify the sender > immediately*__* >
