On Wed, Sep 30, 2026 at 11:10 AM Paul Koning via cctalk <[email protected]> wrote: > > On Sep 30, 2026, at 9:30 AM, ben via cctalk <[email protected]> wrote: > > > > On 2026-09-29 5:57 p.m., Jim Davis via cctalk wrote: > >> And all our daily driver languages look like Algol or FORTRAN, > >> excepting Forth and APL, both written by mad scientists, > > > > And token parsing still is from the 1950's for many languages > > where blank spaces are ignored and a end of line/card or semicolon > > is needed to separate id's. > > Huh? The only language I can think of that ignores whitespace is TECO. Just > about everything else treats whitespace as a token separator.
Microsoft 8-bit BASIC tokenizes when you enter a new line of BASIC but whitespace is not required between tokens. It can get confused in very specific edge cases but mostly, it works. When executing tokenized code, outside of quotes, the actual next-char fetch routine in low mem (CHRGET) explicitly skips whitespace, so the token evaluator never sees it. The stream of tokens and untokenized args presented to the BASIC statement evaluator sees exactly the same stream from the following two lines... 10FORX=1TO100:PRINTX:NEXTX and 10 FOR X = 1 TO 100 : PRINT X : NEXT X Those spaces are stored in RAM but CHRGET skips them on each fetch. -ethan
