Hello Daffodil Community, I'm interested in using Daffodil to access similar scientific data that's stored in files with several different binary encodings.
We've been successful in describing the data format with the DFDL specification, but so far the way we've been using Daffodil is to parse the binary data into an xml or json file. The issue here is that the binary data files are ~2 GB, and both the xml and json versions are more than an order of magnitude larger. And analysis can involve anywhere from 1 to 100 files. Has anyone used Daffodil to unparse data in a way that doesn't create an intermediate file? I'm imagining something that allows me to use Daffodil to look up and stream relevant data; I thought I'd reach out and see if this is a common use case, not possible, or somewhere in between. Thank you! Amy
