Oh, I'm sorry, it involves re-thinking the building of the FO tree, using stream parsing.
Peter, are you saying that a pull parser is more computationally powerful than a SAX Parser--or is it just much more convenient? I don't think pull parsers can do more than SAX Parsers for the simple reason that you can implement a pull parser using a SAX Parser, no?
Firstly, my apologies to all for the tone of my previous message. Too little sleep, too much gall.
Defoe uses SAX for its stream parsing. Consequently, it's less computationally efficient. To use an extreme example, for many years compilers were less computationally efficient than coding with assembler. There are "inefficiencies" at every level of a computer system, from the microcode up. At any level, does the interface ease the development of solutions built on top?
For most of my initial stint at FOP, I was obsessively concerned with micro-efficiencies, and I displayed my ignorance concerning this on many occasions. (Ask Jeremias or Joerg.) I have watched improvements in the performance of JVMs overtake my obsessions while I have been working on FOP. So much for not teaching an old dog new tricks. In spite of those concerns, I went with an "inefficient" parsing mechanism, because it mightily clarified the parsing process.
As a completely unintentional side-effect, it gave me the tools to solve the really critical FOP performance problem on large files. This isn't going to be solved by micro-efficiencies on FO tree storage.
Unfortunately, software developers are an intensely conservative lot. R J Neuhaus has a lovely term, "the herd of independent minds". And he's not even describing software developers. It will be a long time before the SAX franchise falters. The curious thing is that Microsoft never went down the SAX road. They get by.
Defoe is a big job, and I need all the help I can get, but I'll get there.
Peter
