It might be interesting to know if the framework could support brainf*ck <https://esolangs.org/wiki/Brainfuck#Memory> (but I won't be the one to try to actually write the importer...).
On Monday, August 15, 2022 at 7:43:06 AM UTC-4 Edward K. Ream wrote: > Vitalije's importer code has primed the psychological pump. The effect on > the importers will be a spectacular collapse in complexity. > > *Aha 1*: A new method, *i.scan_all_lines*, can calculate all scan states > in a single pass. > > There is no need to store "helper" values in scan states! Instead, > i.scan_all_lines will maintain a few simple state-related vars. > > As a result, a single *NewScanState *class will suffice. Eventually, it > will become a named tuple with just two fields: context and level. Btw, > only i.gen_lines uses scan state classes, so the NewScanState class will > "disappear" from the view of all importers. > > *Aha 2*: The present scanning dictionaries contain two kinds of data: > level-related and token-related. Conceptually, those dictionaries should > *specify > only* tokenizing-related data. This Aha is moot because... > > *Aha 3*: None of the scanning dictionaries are needed!!! > > This last Aha arose while writing the first draft of i.scan_all_lines. At > present, i.scan_dict returns data telling how to: > > - compute the new state, > - compute the new level, > - increment the index into the line. > > But i.scan_all_lines can calculate all this without help! For any > language, i.scan_all_lines only needs to know the following *customizing > values*: > > - The (list of) characters that begin and end strings. > - The characters that begin and end all forms of comments. > - The (optional) characters (typically curly brackets) that increment and > decrement logical level. > > Subclasses of the Importer class will define these customizing values as > needed. > > *Summary* > > A new method, i.scan_all_lines, will compute all (new) scan states in a > prepass. > > New scan states will eventually be named tuples containing only context > and level fields. All custom scan state classes will disappear. > > As needed, importers will define constant customizing values for > i.scan_all_lines. > > Importers will no longer need ctors. All overriding data will be constants. > > The new code will be *slightly *faster and *much* simpler than the old. > > Edward > > P.S. I'm not going to apologize for the old code. It got us to our present > happy state. > > I could not have braved the coming changes without the new unit tests! > > EKR > -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/5daace0c-7a24-4de5-86b3-82f5e9e05aafn%40googlegroups.com.
