Dear List-Members, (Sorry, this has become a rather long posting, to give relevant context) We, that is Project Team 40 (http://www.cs.tut.fi/~varri/tc251/pt40/index.html) of CEN TC 251 "Health Informatics" (http://www.centc251.org/) Working Group IV (http://www.standards.nhsia.nhs.uk/centc251/wgiv/), try to define a file exchange format for biomedical data using ASN.1. Thanks to the Larmouth book and other resources, we got a good start, despite zero previous ASN.1 experience of the team members. When presenting our most recent draft at the last WGIV meeting, there were some voices, doubting the implementabilty of the file exchange format (FEF). We did choose BER encoding and first sample programs are working right now, but they are all hand-coded, not using available ASN.1 tools or libraries. The skeptical voices now claimed, that there are three (rather essential) features of the FEF, which would limit/exclude the implementabilty using existing tools and libraries. As the availabilty of (free and commercial) tools and libraries was one factor to choose ASN.1 in the first place, this would be rather bad news. 1. Large files, general The FEF files can grow rather large, not primarily because of millions of data elements (about 500 to 50000 primitive data elements are expected in a file), but by rather few (1 .. 50) very large primitive data elements holding the waveforms in a backward compatible binary format. In ASN.1 these are just declared as OCTET STRING. Issue raised: Existing tools/libraries will need to assemble the entire data structure in memory, including these BLOBs before writing and after reading the BER file. So this will easily overload virtual memory as file sizes approch the GB area. In addition primitive data elements larger than 4GB will not be supported in existing tools and libraries. 2. Large files, selective reading Most application would be happy to select only part of the BER file to read into memory at a any time, using some selection analogous to XPATH: "[APPLICATION 5003]:2/[APPLICATION 5004]:4" == "read only the fourth [APPLICATION 5004] data element within the second [APPLICATION 5003] data element" This is easily possibly in my hand-crafted application (as it build on top of a XML library). Issue raised: This functionality is not available in existing ASN.1 tools and libraries 3. Incomplete files or byte streams. For some applications it would be very useful to be able to read the FEF file while it is still written to, decoding relevant parts when they become available. Of coure the indefinite length encoding must be used for this scenario. Issue raised: This functionality is not available in existing ASN.1 tools and libraries, they will just give error messages that the file is not complete and won't return useful data, let alone be able to resume parsing when more data becomes available. If anyone knowledgeable about the existing base ASN.1 tools and libraries, can comment about the validity of these issues, it would be most helpful to us. Regards, Peter Jacobi
