(daffodil) branch main updated: Allow DataProcessors to be garbage collected in TDML tests

[slawrence]

This is an automated email from the ASF dual-hosted git repository.

slawrence pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil.git


The following commit(s) were added to refs/heads/main by this push:
     new dcf9079fd Allow DataProcessors to be garbage collected in TDML tests
dcf9079fd is described below

commit dcf9079fd9263848a4291c6827f9e19486af5693
Author: Steve Lawrence <slawre...@apache.org>
AuthorDate: Mon Jun 3 10:51:23 2024 -0400

    Allow DataProcessors to be garbage collected in TDML tests
    
    DataProcessors are currently stored in a "processor" variable in each
    TestCase for easy access while a TestCase is running. However, once a
    TestCase finishes running, it really shouldn't hold on to this
    DataProcessor anymore so that it can be garbage collected. We now set
    that variable to null when finished to allow this. Note that this does
    not remove DataProcessors from the compile cache--it only allows garbage
    collection of DataProcessor created with one of the withXYZ functions to
    tweak the cached processor for a specific test.
    
    Allowing garbage collection of DataProcessors is especially important
    when Xerces validation is enabled, since each DataProcessor hold a
    unique instance of a possibly very large Xerces validator. We likely
    need to change how we store Xerces validators so that they can be
    shared, but in the meantime this change at least allows a large number
    of TDML tests to run with validation enabled without running into
    OutOfMemory errors.
    
    DAFFODIL-2901
---
 .../org/apache/daffodil/tdml/TDMLRunner.scala      | 35 ++++++++++++++--------
 1 file changed, 23 insertions(+), 12 deletions(-)

diff --git 
a/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala 
b/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala
index 26397a599..ee4830291 100644
--- a/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala
+++ b/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala
@@ -934,18 +934,29 @@ abstract class TestCase(testCaseXML: NodeSeq, val parent: 
DFDLTestSuite) {
             }
           (diags, newNewProc)
       }
-      runProcessor(
-        newCompileResult,
-        optInputOrExpectedData,
-        nBits,
-        optExpectedErrors,
-        optExpectedWarnings,
-        optExpectedValidationErrors,
-        validationMode,
-        roundTrip,
-        implString
-      )
-
+      try {
+        runProcessor(
+          newCompileResult,
+          optInputOrExpectedData,
+          nBits,
+          optExpectedErrors,
+          optExpectedWarnings,
+          optExpectedValidationErrors,
+          validationMode,
+          roundTrip,
+          implString
+        )
+      } finally {
+        // runProcessor may have set the "processor" variable to the 
DataProcessor used for the
+        // test. This variable only exists as an easy way to avoid having to 
pass the
+        // DataProcessor around to a bunch of functions. At this point this 
TestCase is done
+        // with the DataProcessor, so we set it to null so it can be garbage 
collected if
+        // nothing else uses it. This is common if a test case calls withXYZ 
to create a
+        // temporary copy of a cached DataProcessor but with different 
variables, tunables,
+        // validation mode, etc. used only for this specific test. This is 
especially important
+        // if a DataProcessor stores any large information like a Xerces 
validator.
+        processor = null
+      }
     }
   }