Tue, 26 Oct 2010 19:32:44 -0700, Walter Bright wrote: > Nick Sabalausky wrote: >> "Walter Bright" <[email protected]> wrote in message >> news:[email protected]... >>> 4. the tokens should be a value type, not a reference type >> >> I'm curious, is your reason for this purely to avoid allocations during >> lexing, or are there other reasons too? > > It's one big giant reason. Storage allocation gets unbelievably costly > in a lexer. Another is it makes tokens easy to copy. Another one is that > classes are for polymorphic behavior. What kind of polymorphic behavior > would one want with tokens?
This is why the basic data structure in functional languages, algebraic data types, suits better for this purpose.
