On Thursday, 2 August 2012 at 00:11:15 UTC, Walter Bright wrote:
3. tokens should be values, not classes
I agree with everything but this point. Tokens become quite large when they have all kinds of string and position information on them, much larger than the typical recommended sizes for pass-by-value. It's still possible to provide an interface for them that doesn't require as much copying (ref returns and getters and whatnot), but it's fiddly and way too easy to copy these huge Token structs around, especially if the output is a range of Token structs.
