Re: Rev 789350d in devel: a big step for Leo's blacken and beautify commands

[Edward K. Ream]

On Wednesday, September 11, 2019 at 5:35:42 PM UTC-5, Edward K. Ream wrote:

> About 5 lines of code in this the SyntaxSanitizer class suffice to 
support the two new settings...
> Black does not need to change in any way...
> A new setting prevent blacks from messing with hundreds of existing 
overindented comments.

I want to brag a bit. The SyntaxSanitizer class is short, elegant and 
useful. It:

- allows Leo's devs to carry on, regardless of black's limitations, 
philosophy, whatever.
- allows the beautify-tree command to verify that it doesn't change the 
code's meaning (parse tree).

Leo's adaptation of black is *way* more elegant than the equivalent mods 
would be in black itself!

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/d1f3a600-9d0c-425d-ae1b-4c1b52b233bf%40googlegroups.com.

Rev 789350d in devel: a big step for Leo's blacken and beautify commands

[Edward K. Ream]

This work completes #1325 
: Beautifiers should 
optionally preserve indentation of overindented comments.

Leo now sports two new settings:

@bool black-keep-comment-indentation
@bool orange-keep-comment-indentation

The former applies to the blacken-tree and blacken-node commands. The 
latter applies to the beautify-tree and beautify-node commands. Maybe it 
should be renamed beautify-keep-comment-indentation.

About 5 lines of code in this the SyntaxSanitizer class suffice to support 
the two new settings.

*Significance*

The way is now clear to finish #1322 
: beautify/blacken 
all of Leo's sources.

I plan to use *black* (blacken-tree) not orange (beautify-tree). This will 
provide a solid baseline for working on orange.

*Black does not need to change in any way.* Leo's blacken-commands use the 
SyntaxSanitizer class to shield black from Leo's syntax. There should be 
little or no need to apply the external version of black on external files!


*Next steps*

1. I'll use Leo's internal blacken-tree command  to beautify all of Leo's 
sources This will be done in a new *test-black* branch. As noted earlier, 
blacken-tree fails for one or two of Leo's sources.  I'll attempt to alter 
Leo's sources so that blacken-tree can be used everywhere.

2. After thorough testing, the test-black branch will be merged into the 
devel and beauty branches.

3. Finally, I'll complete #1266 
: (orange is the new 
black) in the beauty branch. A new *test-orange* branch will contain the 
results of using beautify-tree everywhere. This branch can then be compared 
against devel.

*Summary*

Leo now contains settings that allow *black* to beautify all of Leo's 
sources, using Leo's blacken-tree command.  A new setting prevent blacks 
from messing with hundreds of existing overindented comments.

Black does not need to change in any way! The blacken-tree command reflects 
*Leo's* philosophy: we never argue about preferences. Leonistas can use 
black as *they* choose, not as black's devs demand.

Work on orange (beautify-tree, enhanced with line breaking/joining) will 
continue.  At present, beautify-tree is four times faster than 
blacken-tree. The token-based code for beautify-tree is interesting in 
itself. Imo, it is significantly simpler than black's parse-tree based 
code.  For example, regularizing strings is very easy using tokens.  It's a 
nightmare using parse trees.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/cb912ed4-64d1-4f28-929f-4a4d3741c8a8%40googlegroups.com.

Re: DANGER: don't run black on files containing Leo sentinels

[Edward K. Ream]

On Wednesday, September 11, 2019 at 6:16:55 AM UTC-5, Edward K. Ream wrote:

I'll ask the black devs to create an option that would allow Leonistas to 
> use black safely.
>

Done here .

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/cb251bd7-8773-4587-b023-bbc868354684%40googlegroups.com.

Re: Global variables in user scripts

[Brian Theado]

Thanks, Vitalije. Only after seeing your response did it occur to me to
take a closer look at the scripting docs and of course the user_dict ivar
is documented there.

On Wed, Sep 11, 2019 at 7:42 AM vitalije  wrote:

> You have c.user_dict for these kind of things. It is an ordinary python
> dictionary and you can put in it anything you want. Values you set in this
> dictionary will be there until Leo exits. If you need to make them more
> permanent then you can put values in c.db (looks like and behaves like a
> dictionary, but values must be pickleable).
>
> Vitalije
>
> On Wednesday, September 11, 2019 at 1:28:39 PM UTC+2, btheado wrote:
>>
>> I'm writing a script button in which I want to transform some outline
>> data into html and then display that html in a leo doc. The viewrendered
>> functionality doesn't seem to really fit my needs as the render pane output
>> is tied to whatever node is currently selected.
>>
>> I just want to be able to display the html "on-demand" (i.e. through
>> script button click) and refresh it on-demand.
>>
>> I have working code for it, but it requires storing the dock object
>> somewhere so it can be used again the next time the script is called. I was
>> wondering if there is some good, already-established convention for storing
>> "global" state for scripts.
>>
>> In my case, the variable makes sense to be on a per-commander basis, so I
>> just stored my information on the commander 'c'. That doesn't seem very
>> clean to me and I was wondering if there is a better approach?
>>
>> Here is my function which stores and uses global state on c:
>>
>> def display_widget_in_leo_pane(c, w, name):
>> """
>> w is the widget to display
>> name is the name the widget should appear in pane menu
>> """
>> dw = c.frame.top
>> if not hasattr(c, '*my_docks*'): c.my_docks = {}
>> dock = c.*my_docks*.get(name)
>> if not dock:
>> dock = g.app.gui.create_dock_widget(
>>  closeable=True, moveable=True, height=50, name=name)
>> c.*my_docks*[name] = dock
>> dw.addDockWidget(QtCore.Qt.RightDockWidgetArea, dock)
>> dock.setWidget(w)
>> dock.show()
>>
>>
>> And a function to use the above:
>>
>> def display_html(html, name = 'test html'):
>> w = QtWidgets.QTextBrowser()
>> w.setHtml(html)
>> display_widget_in_leo_pane(c, w, name)
>>
>> Also, could someone comment on whether the above code is "leaking"
>> widgets? Should I be calling dock.widget() to retrieve the old widget each
>> time to perform some sort of delete/cleanup?
>>
>> Brian
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "leo-editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to leo-editor+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/leo-editor/39d49062-9e58-402d-9a97-bd98ed090441%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAO5X8CwqLZybR5dyZEKU94%2BhJBQ9GjUDRGxc8Z-do3eDXdhAVw%40mail.gmail.com.

Re: Global variables in user scripts

[vitalije]

You have c.user_dict for these kind of things. It is an ordinary python 
dictionary and you can put in it anything you want. Values you set in this 
dictionary will be there until Leo exits. If you need to make them more 
permanent then you can put values in c.db (looks like and behaves like a 
dictionary, but values must be pickleable).

Vitalije

On Wednesday, September 11, 2019 at 1:28:39 PM UTC+2, btheado wrote:
>
> I'm writing a script button in which I want to transform some outline data 
> into html and then display that html in a leo doc. The viewrendered 
> functionality doesn't seem to really fit my needs as the render pane output 
> is tied to whatever node is currently selected.
>
> I just want to be able to display the html "on-demand" (i.e. through 
> script button click) and refresh it on-demand.
>
> I have working code for it, but it requires storing the dock object 
> somewhere so it can be used again the next time the script is called. I was 
> wondering if there is some good, already-established convention for storing 
> "global" state for scripts.
>
> In my case, the variable makes sense to be on a per-commander basis, so I 
> just stored my information on the commander 'c'. That doesn't seem very 
> clean to me and I was wondering if there is a better approach?
>
> Here is my function which stores and uses global state on c:
>
> def display_widget_in_leo_pane(c, w, name):
> """
> w is the widget to display
> name is the name the widget should appear in pane menu
> """
> dw = c.frame.top
> if not hasattr(c, '*my_docks*'): c.my_docks = {}
> dock = c.*my_docks*.get(name)
> if not dock:
> dock = g.app.gui.create_dock_widget(
>  closeable=True, moveable=True, height=50, name=name)
> c.*my_docks*[name] = dock
> dw.addDockWidget(QtCore.Qt.RightDockWidgetArea, dock)
> dock.setWidget(w)
> dock.show()
>
>
> And a function to use the above:
>
> def display_html(html, name = 'test html'):
> w = QtWidgets.QTextBrowser()
> w.setHtml(html)
> display_widget_in_leo_pane(c, w, name)
>
> Also, could someone comment on whether the above code is "leaking" 
> widgets? Should I be calling dock.widget() to retrieve the old widget each 
> time to perform some sort of delete/cleanup?
>
> Brian
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/39d49062-9e58-402d-9a97-bd98ed090441%40googlegroups.com.

Global variables in user scripts

[Brian Theado]

I'm writing a script button in which I want to transform some outline data
into html and then display that html in a leo doc. The viewrendered
functionality doesn't seem to really fit my needs as the render pane output
is tied to whatever node is currently selected.

I just want to be able to display the html "on-demand" (i.e. through script
button click) and refresh it on-demand.

I have working code for it, but it requires storing the dock object
somewhere so it can be used again the next time the script is called. I was
wondering if there is some good, already-established convention for storing
"global" state for scripts.

In my case, the variable makes sense to be on a per-commander basis, so I
just stored my information on the commander 'c'. That doesn't seem very
clean to me and I was wondering if there is a better approach?

Here is my function which stores and uses global state on c:

def display_widget_in_leo_pane(c, w, name):
"""
w is the widget to display
name is the name the widget should appear in pane menu
"""
dw = c.frame.top
if not hasattr(c, '*my_docks*'): c.my_docks = {}
dock = c.*my_docks*.get(name)
if not dock:
dock = g.app.gui.create_dock_widget(
 closeable=True, moveable=True, height=50, name=name)
c.*my_docks*[name] = dock
dw.addDockWidget(QtCore.Qt.RightDockWidgetArea, dock)
dock.setWidget(w)
dock.show()


And a function to use the above:

def display_html(html, name = 'test html'):
w = QtWidgets.QTextBrowser()
w.setHtml(html)
display_widget_in_leo_pane(c, w, name)

Also, could someone comment on whether the above code is "leaking" widgets?
Should I be calling dock.widget() to retrieve the old widget each time to
perform some sort of delete/cleanup?

Brian

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAO5X8CyrW7Uy7aaN_eMPiHu3FR1ay3o8PVAgLRb7OmGzscPRZw%40mail.gmail.com.

DANGER: don't run black on files containing Leo sentinels

[Edward K. Ream]

Don't panic:

- The warning applies only when running black from the command line.

- The warning apples only when running black on files created by @file.

- Leo's blacken-* command are completely safe.  As noted previously, they 
can fail in rare cases, but failures are clearly reported and leave 
outlines unchanged.

There is, *in principle*, no way for *Leo *to avoid the danger...

*Blackening external files with Leo sentinels is inherently dangerous*

The reason is shocking.  After many hours of investigation, I finally 
realized that Leo's read code *already* recognizes *blackened sentinels*, 
sentinels with a blank between the opening comment delimiter and the '@' 
sign that starts all sentinels.  No change is necessary (or possible!) to 
Leo's read code.

The problem lies entirely outside of Leo's read code.  After blackening, 
*formerly 
innocent comments* can become sentinel lines.  There is no way, *in 
principle*, for the read code to know that these *apparent* sentinels 
should be ignored. Honoring these *newly active sentinels* can destroy an 
@file tree.

The problem is really a case of *missing @verbatim sentinels*. For example, 
leoTangle.py contains the following comment:

# if you're not going to use { } for pascal comments, use
# @comment (* *)
# to specify the alternative

Yikes!  After blackening outside Leo, the second comment becomes a real 
@comment sentinel.  It passes the corresponding regex used by the read code!

So we're stuck:

- Leo creates @verbatim sentinels only for python comments that start with 
"#@".

- To avoid missing @verbatim sentinels, Leo would have to create @verbatim 
sentinels for *newly dangerous comments*. These are comments that:

a) are preceded only by whitespace,
b) start with "# @", (notice the blank)
c) actually look like a valid Leo sentinel comment.

Alas, generating @verbatim sentinels for newly-dangerous comments would 
change the format of Leo's file format.

I know from experience that this kind of change can have subtle, rare (and 
therefore all the more dangerous) unintended consequences. Old versions of 
Leo might not handle these new @verbatim sentinels properly. The potential 
for misery is real. There is *no way* I shall allow Leo to create more 
@verbatim sentinels!

*Workarounds*

There are several workarounds, none of them perfect.

1. Poor: A script might warn about newly dangerous comments. You would have 
to remember to run the script before blackening external files, and then 
you would have to change the comments.

2. Fair: Convert @file trees to @clean before blackening, and convert them 
back to @file afterwards.

3. Good: Use Leo's tangle-tree and tangle-node commands. They usually work 
just fine.

4. Possible: Modify black so it doesn't insert a blank between # and @.

Last night I discovered how! In def make_comment, change:

if content and content[0] not in " !:#'%":
content = " " + content

to:

if content and content[0] not in " @!:#'%": # EKR: This string should be an 
option.
content = " " + content

Imo, this is an urgently needed option.  We'll see what the black devs say.

*Summary*

Leo's read code in fast_at.scan_lines *already* recognizes blackened 
sentinels.  That's very bad news.

Blacking an external file can turn *formerly innocent comments* into *newly 
dangerous sentinels*. scan_lines can not, in principle, tell which sentinel 
lines should be ignored. The needed @verbatim sentinels that do not exist.

I shall reject any proposal to write new @verbatim sentinels. The worst 
quality disasters in Leo's history have arisen from similar "small" changes.

There are several workarounds. Using Leo's blacken-tree command will 
usually be the best.

I'll ask the black devs to create an option that would allow Leonistas to 
use black safely.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/7a2d6f0b-077d-4e4c-aea1-62a62819164a%40googlegroups.com.