Re: ENB: About leoTokens.py

[Edward K. Ream]

On Saturday, January 6, 2024 at 7:17:41 AM UTC-6 Edward K. Ream wrote:

The work I did was a complex merger of three PR's.


This was, in essence, a Leonine version of git's cherry picking.

Cherry picking is generally frowned upon, but in this case I saw no 
alternative.

The ability to focus using clones allowed me to see patterns that are 
larger than refactoring patterns.

Dozens of cffs helped.

In short, refactoring with cffs and clones is the bees knees.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/499c0e19-2314-44f9-aea4-eb451ff3959dn%40googlegroups.com.

Re: ENB: About leoTokens.py

[Edward K. Ream]

On Sat, Jan 6, 2024 at 6:44 AM Thomas Passin  wrote:

It would be interesting to read more about this. Other IDEs claim to
> support refactoring.  How did Leo make it possible in a way that others
> would not have?


Libraries such as Rope have a fixed set of refactorings. The work I did was
a complex merger of three PR's.

I was continually looking at cloned nodes containing lists of subsidiary
nodes that could either be revised or removed. The processes are quite
different.

The Leonine way of refactoring does not depend on rigid tools or ideas.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS2U%3DwLqD%2BTkrhbZZMDyTdN9pPVXfOb1PfJpooaHeXppew%40mail.gmail.com.

Re: ENB: About leoTokens.py

[Thomas Passin]

On Saturday, January 6, 2024 at 7:31:52 AM UTC-5 Edward K. Ream wrote:

I could not have refactored the code without Leo.


It would be interesting to read more about this. Other IDEs claim to 
support refactoring.  How did Leo make it possible in a way that others 
would not have?

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/a3ecaf79-1567-4bec-a323-9c85abe76eb9n%40googlegroups.com.

ENB: About leoTokens.py

[Edward K. Ream]

 

This Engineering Notebook post introduces a new core module: *leoTokens.py*. 
This file contains Leo's new token-oriented beautifier, discussed in this 
ENB . 
I plan to transliterate leoTokens.py into Nim. 


*Background*


PR #3757  contains the 
present code. It is still a work in progress. PR #3757 is complex! Three 
earlier PRs contributed to the work. I could not have refactored the code 
without Leo.


*leoAst.py* contains Leo's legacy beautifier. This beautifier uses the 
*TokenOrderGenerator* (*TOG*) class, also defined in leoAst.py.


*Goals for leoTokens.py*


- Fixed type annotations of all data, functions, and methods.

- Minimal code size.

- Minimal dependence on other Leo modules.


*Aha: keep leoAst.py*


My initial thought was to *replace* the legacy beautifier with the 
tokens-only beautifier. But this was a bad idea for at least three reasons:


- The Orange class (the legacy beautifier) is an excellent demo of how to 
use the TOG.

- We want to be able to compare the old and new beautifiers.

- Adding another class to leoAst.py would complicate an already too-complex 
module.


*leoTokens.py*


leoTokens.py started life as a complex merger of the PRs mentioned above. 
Details omitted!


I then removed unnecessary classes, methods, and functions. En route, I 
simplified annotations. This pruning process ended today: leoTokens.py 
contains the minimum code required!


I saw that some helpers in *leoAst.py* were unnecessary. Again, details 
omitted.


*Summary*


Separating *leoTokens.py* from *leoAst.py* has simplified both files. 
Similar remarks apply to the corresponding unit test files.


leoTokens.py is now as small as possible. It is also fully annotated. This 
preliminary work will make transliterating leoTokens.py into Nim much 
easier.


leoTokens.py is still a work in progress. I must demonstrate that a 
tokens-only beautifier can infer context *without* the help of a parse tree.


I am confident that token scanning can do the job. The existing unit tests 
will be my guide!


 Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/d3a7bc63-bb4d-4fcd-bb6d-d7bf39283fa2n%40googlegroups.com.