Re: UUID gnx support

[spike]

On 2022-09-13 17:51, Thomas Passin wrote:
> It's far from clear to me that any code that uses gnx's needs to know 
> anything except their string value.  You want to avoid inadvertent 
> duplications of gnx's (so copies can be distinguished from clones, for one 
> thing), that's clear.  Not that I've studied the code or anything useful 
> like that!  it ought to be possible to combine a UUID with the user name, 
> so people to whom it matters (like @edward) could still see that.
> 
> Don't forget, there are several kinds of UUIDs.  Some are more 
> collision-proof than others, or harder to generate, or the scope of 
> uniqueness varies (e.g., local computer vs the world).  ISTR that 
> TheOldNewThing had a post or series of posts on the differences between 
> different kinds of UUIDs.
> 
> File format changes should be backwards compatible - Please Please Please!
> On Tuesday, September 13, 2022 at 6:17:54 PM UTC-4 Edward K. Ream wrote:
> 
>> On Tuesday, September 13, 2022 at 4:36:05 PM UTC-5 spike wrote:
>>
>> Having read the discussion on #1348 (thank you for the link), I must 
>>> strongly disagree with this assessment. 
>>>
>>> There were two problems involved in that bug: 
>>> 1) Leo created a spurious vnode when reading an outline. 
>>> 2) This exposed a timing race due to using timestamp based IDs. 
>>>
>>> Introducing UUIDs was a proposed solution. Sequence numbers were the 
>>> chosen alternative. Both were reasonable. 
>>>
>>
>> Interesting. I don't remember the details, so you may well be correct. 
>>
>> Using username and timestamp information in your workflow is both fair 
>>> and reasonable. Blaming UUIDs for a bug caused by improper use of 
>>> timestamps is neither. 
>>>
>>
>> Thanks for these remarks. All I remember was the pain of the bugs, so 
>> perhaps you are correct that UUIDs were not the real culprit. This is not a 
>> hot button issue for me. I'm often wrong about details.
>>
>> It seems we both agree that the present scheme is reasonable. Let's leave 
>> it at that.
>>
>> Anyway, it's way too late to consider significant changes to Leo for 
>> 6.7.0, and any change to Leo's file format qualifies as a *highly* 
>> significant change :-)
>>
>> Edward
>>
> 
IDs should always be treated as blind tokens. Attempting to decode them
is asking for trouble precisely because it causes compatibility problems
and hard to trace bugs. The token generator needs to check against
existing tokens to verify uniqueness (Leo does this) and may perform
heuristics if it needs to ensure sequencing, but care must be taken to
safely handle unexpected input.

Leo decodes gnx in exactly two places:

The unit test for the token generator, which should never see tokens
created by a different version and thus is safe.

The ToDo plugin, in todo.py on line 1303. It takes proper precautions to
fail gracefully when it hits an ID it can't read because failures were a
known and expected issue when it was first created. Even something as
simple as a '.' in a user name is enough to trip it up.

There is a more general problem in Leo with not escaping output, which
can create unreadable files if unexpected characters are present. Raw
gnxes are written to both .leo files and in @file annotations,
preventing the use of the characters '<', '>', '"', '&', ':', and '\n'.
None of those characters occur in UUIDs, which are limited to [0-9A-F]
and '-'.

My proposed patch used UUID4, which are randomly generated and provide
strong guarantees of uniqueness. It was also gated by a user setting
that defaulted to 'off'.

UUID gnx is verified compatible with any recent Leo and should be
compatible all the way back to 4.1. That's why I felt comfortable
submitting it for 6.7.0.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/5fb0ec1d-0f03-39ea-a64e-7f1d3f8c5ded%40runbox.com.

Re: UUID gnx support

[spike]

On 2022-09-14 06:28, Thomas Passin wrote:
> I wondered about that message.  I saw it after pulling the latest 
> changeset.  I haven't set any uAs of any variety, myself.  Maybe some 
> plugin?
> 
> On Wednesday, September 14, 2022 at 4:53:20 AM UTC-4 Edward K. Ream wrote:
> 
>> On Wednesday, September 14, 2022 at 3:38:40 AM UTC-5 Edward K. Ream wrote:
>>
>> P.S. I did not sufficiently consider compatibility issues when approving 
>>> PR #2818 , which 
>>> says:
>>>
>>> "Older versions of Leo will not be able to read files that use this 
>>> feature. Files not using this feature are not affected."
>>>
>>
>>  I've just taken a look at the relevant read code: FastRead.resolveUa*.* 
>> In fact, older version of Leo *will* be able to read files with _json uAs.
>>
>> As I read the code, Leo will issue the warning: f"can not unhexlify 
>> {attr}={val}" for _json uAs, but almost-never-issued warnings shouldn't 
>> disqualify the PR.
>>
>> Edward
>>
> 
User attributes are used by several plugins in Leo. I asked for JSON
support because the only options were 'plain string' and 'pickle'. The
former is too limiting and the latter isn't text, which can be a problem
when you need to examine or edit a .leo file by hand. Debugging user
icons was needlessly troublesome because of it.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/e5a2a556-fbd4-3ea1-f66b-593bb8802db7%40runbox.com.

Re: UUID gnx support

[spike]

On 2022-09-14 02:38, Edward K. Ream wrote:
> On Tue, Sep 13, 2022 at 6:23 PM spike  wrote:
> 
> We both have reasonable objectives. That's why I asked for an option
>> setting, so both use cases are covered, but I don't mind keeping a
>> private fork.
>>
> 
> Leo is not going to change its file format. Not now. Not ever.  There are
> two problems:
> 
> 1. [Minor] Supporting two formats is messy within Leo.
> 2. [Fatal] Creating a new format now means that old versions of Leo won't
> be able to read .leo files created in the new format.
> 
This does not apply to UUID gnx. Older versions of Leo all the way back
to 4.1 will accept them without problems. The only affected code is
'sort by creation date', which already fails gracefully because user
names aren't guaranteed to match Leo's expected format, and that breaks
decoding the timestamp.

> Use a fork if you must. Leo isn't going to use UUIDs.
> 
> Adding a command-line option is too horrible to contemplate. Adding a user
> setting won't work because user options are read too late.
> 
The user settings works as expected since new ID's aren't created until
after the settings are read. This was tested both ways: Disabled in
global settings and enabled in the file, and enabled in global and
disabled in the file. Both worked as expected.

> On that note I have some ideas for cleaning up and extending Leo's file
>> format and undo helpers -- user attribute handling in particular is
>> pretty cludgy -- but that would be for 7.0 at minimum. Such changes
>> would run far and deep and I've been trying to avoid that with my
>> submissions.
>>
> 
> I will not consider such changes. Leo is a mature program, constrained by
> its past. Stability is much more important than behind-the-scenes
> improvements, or supposed improvements.
> 
> 
Edward
So be it.

> 
> P.S. I did not sufficiently consider compatibility issues when approving PR
> #2818 <https://github.com/leo-editor/leo-editor/pull/2818>, which says:
> 
> "Older versions of Leo will not be able to read files that use this
> feature. Files not using this feature are not affected."
> 
> I'm considering whether to back out of this PR.
> 
> Edward
> 

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/11300d9b-0e6e-78f8-dfd9-c879c7b287f9%40runbox.com.

Re: UUID gnx support

[spike]

On 2022-09-13 16:17, Edward K. Ream wrote:
> On Tuesday, September 13, 2022 at 4:36:05 PM UTC-5 spike wrote:
> 
> Having read the discussion on #1348 (thank you for the link), I must 
>> strongly disagree with this assessment. 
>>
>> There were two problems involved in that bug: 
>> 1) Leo created a spurious vnode when reading an outline. 
>> 2) This exposed a timing race due to using timestamp based IDs. 
>>
>> Introducing UUIDs was a proposed solution. Sequence numbers were the 
>> chosen alternative. Both were reasonable. 
>>
> 
> Interesting. I don't remember the details, so you may well be correct. 
> 
> Using username and timestamp information in your workflow is both fair 
>> and reasonable. Blaming UUIDs for a bug caused by improper use of 
>> timestamps is neither. 
>>
> 
> Thanks for these remarks. All I remember was the pain of the bugs, so 
> perhaps you are correct that UUIDs were not the real culprit. This is not a 
> hot button issue for me. I'm often wrong about details.
> 
> It seems we both agree that the present scheme is reasonable. Let's leave 
> it at that.
> 
> Anyway, it's way too late to consider significant changes to Leo for 6.7.0, 
> and any change to Leo's file format qualifies as a *highly* significant 
> change :-)
> 
> Edward
> 
I understand the pain and frustration of hard-to-trace bugs. We've all
been there. I also apologize for being harsh. I get attached to things,
too. ;)

We both have reasonable objectives. That's why I asked for an option
setting, so both use cases are covered, but I don't mind keeping a
private fork.

I agree that file format changes are a big deal. That's why I was hoping
to get this in for 6.7.0, but I guess I rushed it. Sorry about that.

On that note I have some ideas for cleaning up and extending Leo's file
format and undo helpers -- user attribute handling in particular is
pretty cludgy -- but that would be for 7.0 at minimum. Such changes
would run far and deep and I've been trying to avoid that with my
submissions.

Cheers.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/f5d72add-4814-0aca-88ac-1e872df833a2%40runbox.com.

Re: UUID gnx support

[spike]

On 2022-09-13 04:37, Edward K. Ream wrote:
> On Tuesday, September 13, 2022 at 5:14:19 AM UTC-5 Edward K. Ream wrote:
> 
>> Being wary of UUIDs should be part of Leo's institutional memory.
> 
> PR #2827  adds a 
> warning to NodeIndices.computeNewIndex:
> 
> def computeNewIndex(self) -> str:
> """Return a new gnx."""
> 
> # Warning! Warning! Warning!
> 
> # Don't even *think* about changing the format of gnxs!
> # Doing so could introduce the worst kind of bugs: subtle and rare.
> 
> # See this post: 
> https://groups.google.com/g/leo-editor/c/Lldywoievn4/m/RUMMzB7fBgAJ
> 
> t_s = self.update()  # Updates self.lastTime and self.lastIndex.
> gnx = g.toUnicode(f"{self.userId}.{t_s}.{self.lastIndex:d}")
> return gnx
> 
> Edward
> 
Having read the discussion on #1348 (thank you for the link), I must
strongly disagree with this assessment.

There were two problems involved in that bug:
1) Leo created a spurious vnode when reading an outline.
2) This exposed a timing race due to using timestamp based IDs.

Introducing UUIDs was a proposed solution. Sequence numbers were the
chosen alternative. Both were reasonable.

Using username and timestamp information in your workflow is both fair
and reasonable. Blaming UUIDs for a bug caused by improper use of
timestamps is neither.

This seems to be a hot button for you, so I'll drop the issue.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/4effea75-c9da-896c-a5f1-11efe60bee7a%40runbox.com.

Re: UUID gnx support

[spike]

On 2022-09-12 15:39, Edward K. Ream wrote:
> On Mon, Sep 12, 2022 at 3:39 PM spike  wrote:
> 
>> A longstanding pet peeve of mine is that Leo includes the user name in
>> the gnx string, so I patched it to use UUIDs instead. I don't know if
>> anyone else cares enough to warrant including this upstream, but I've
>> created a PR (#2843) in case anyone is interested.
>>
> 
> I care, and I want to retain the user ids.
> 
> Edward
> 
Fair enough.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/7711c703-146b-038f-696b-a6e0c60ff2f9%40runbox.com.

UUID gnx support

[spike]

A longstanding pet peeve of mine is that Leo includes the user name in
the gnx string, so I patched it to use UUIDs instead. I don't know if
anyone else cares enough to warrant including this upstream, but I've
created a PR (#2843) in case anyone is interested.

I've been using this for a couple of months without difficulty, but YMMV.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/ebb74412-2c7e-40a9-2cb1-2a66fa201a97%40runbox.com.

Re: PR #2826 merged into devel

[spike]

On 2022-09-10 16:39, Edward K. Ream wrote:
> PR #2826  greatly 
> simplifies how Leo handles text indices.
> 
> This PR is a mass update. I have carefully compared diffs, all tests pass, 
> mypy and pylint are happy, and I have been eating my own dog food without 
> upset. Nevertheless, the PR may have introduced new bugs. Please report any 
> problems immediately.
> 
> The PR contains a high-level summary in the first comment. However, the 
> changes do not capture my excitement about the changes:
> 
> 1. This PR took about 24 hours. This incredible productivity is the direct 
> result of mypy checks. At last we are reaping the benefit of strong type 
> checking.
> 
> 2. Usually I have some notion of what the final code will look like. Not 
> this time! Here is the list of the happy surprises:
> 
> - All indices *everywhere* are ints, not a Union.
> - Leo's core knows *nothing* about g.toPythonIndex.
> - g.toPythonIndex takes a *string* argument, not a Union!
> - All calls to g.toPythonIndex occur within toInt local helper functions.
> - It was *easy* to delete all the faux-helper x.toPythonIndex *methods.*
> 
> *Testing*
> 
> The PR removes the recently-introduced int_i, int_j, etc. vars. I have 
> reviewed various kinds of diffs, including git-diff-pr between the 6.6.4 
> and ekr-index branches. What stands out are the many missing calls to 
> g.toPythonIndex.
> 
> However, Leo's unit tests don't cover qt_text.py. It would be possible to 
> do so, but not straightforward.  See #2827 
> . In their absence I 
> have relied on mypy, pylint, and code reviews. It's not enough :-)
> 
> *Summary*
> 
> The new code base is a spectacular simplification. We'll deal with bugs as 
> they arise.
> 
> The difference between Union[int, str] and just int is hard to overstate. 
> The new code should help the leoJS project considerably.
> 
> Edward
> 
Looks good. Will try it out.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/9cbd1fd8-cf29-5228-3299-924c30a5211f%40runbox.com.

Re: PEP 561 (mypy annotations)

[spike]

On 2022-09-10 04:43, Edward K. Ream wrote:
> On Fri, Sep 9, 2022 at 8:04 PM spike  wrote:
> 
>> In mypy's documentation they suggest including an empty "py.typed" file
>> in the package directory to identify it as a typed package.
>>
>> Leo supports out-of-tree plugins this would allow them to use mypy for
>> type checking.
>>
> 
> Good idea. Please open an issue.
> 
> Edward
> 
PR #2833 is ready for review.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/825f829d-c44e-6c9c-5017-5c0b0e0a47d1%40runbox.com.

PEP 561 (mypy annotations)

[spike]

In mypy's documentation they suggest including an empty "py.typed" file
in the package directory to identify it as a typed package.

Leo supports out-of-tree plugins this would allow them to use mypy for
type checking.

https://mypy.readthedocs.io/en/stable/installed_packages.html

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/7d152d45-71fe-be3c-e54a-66a4ae5f77ac%40runbox.com.

Re: User attribute bug fixes and a new JSON attribute feature [3 patches]

[spike]

On 2022-08-21 03:33, Edward K. Ream wrote:
> On Fri, Aug 19, 2022 at 1:26 PM spike  wrote:
> 
>> Attached are two bug fixes, one of them serious, and a feature request.
>> Patches apply cleanly in order to v6.6.4-devel but are otherwise
>> independent.
>>
> 
> Thanks for these contributions!
> 
> Please submit a Pull Request.
> 
> Edward
> 
I'm not sure if this was sent before because there was an error.


I posted the patches to gitlab at:

https://gitlab.com/spikethehobbitmage/leo-editor.git


There are three branches:

fix-user-attributes (fixes two bugs in user attributes)

fix-REPLACE-REST (fixes a bug in Declutter)

support-JSON-attributes (adds a new feature to user attributes)


All of them are from the current 'devel' branch.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/6954fdfe-81a2-7476-7fe9-d032fd84ba73%40runbox.com.

Bug in Declutter REPLACE-REST

[spike]

file "leo/plugins/qt_tree.py"

in function "declutter_replace" on line 375

elif cmd == 'REPLACE-REST':
s = (text[:m.start] + text[m.end() :]).strip()

missing "()" after "m.start"

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/45b0edd3-a085-ffc9-17e3-e82bc73ef7b9%40runbox.com.

User attribute bug fixes and a new JSON attribute feature [3 patches]

[spike]

Attached are two bug fixes, one of them serious, and a feature request.
Patches apply cleanly in order to v6.6.4-devel but are otherwise
independent.


"fix attribute quoting.diff"

This fixes a serious file corruption bug that occurs when a user
attribute has a name starting with 'str_' and includes a literal '"'
character in the value.

Serious due to affected files becoming unreadable.

Compatibility note: Unpatched versions of Leo will be able to read
affected files created by patched versions but will remain unable to
save them correctly. Affected users are strongly encouraged to upgrade.


"sort user attributes.diff"

This fixes a cosmetic bug that causes user attributes to be stored in
random order. This causes spurious deltas when affected .leo files are
stored under version control but is otherwise harmless.

Compatibility note: This change is fully backward compatible.


"support JSON attributes.diff"

This new feature adds support for JSON formatted attributes, derived
from the current 'str_' attribute special handling. Only attributes with
names starting with 'json_' are affected. The new code automatically
falls back to standard pickling if any JSON errors occur.

Compatibility note: Leo files using this new feature won't be readable
by older versions of the program. No other files are affected.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/1b88d4a8-2094-7f5d-5c1b-8a2219ebefe6%40runbox.com.
392bbc4828b6134465c63367119519a30c7c68f4
diff --git a/leo/core/leoFileCommands.py b/leo/core/leoFileCommands.py
index c53899665..359da0507 100644
--- a/leo/core/leoFileCommands.py
+++ b/leo/core/leoFileCommands.py
@@ -2167,7 +2167,7 @@ class FileCommands:
 if key.startswith('str_'):
 if isinstance(val, (str, bytes)):
 val = g.toUnicode(val)
-attr = f' {key}="{xml.sax.saxutils.escape(val)}"'
+attr = f' {key}={xml.sax.saxutils.quoteattr(val)}'
 return attr
 g.trace(type(val), repr(val))
 g.warning("ignoring non-string attribute", key, "in", torv)
614a19ffcef8cfecdeabdc8d329a89e6f22ad016
diff --git a/leo/core/leoFileCommands.py b/leo/core/leoFileCommands.py
index 359da0507..79eee8923 100644
--- a/leo/core/leoFileCommands.py
+++ b/leo/core/leoFileCommands.py
@@ -2182,7 +2182,7 @@ class FileCommands:
 if isinstance(attrDict, dict):
 val = ''.join(
 [self.putUaHelper(v, key, val)
-for key, val in attrDict.items()])
+for key, val in sorted(attrDict.items())])
 return val
 g.warning("ignoring non-dictionary unknownAttributes for", v)
 return ''
2a1d0f809f24ab8d455b24585d53172c989b0a90
diff --git a/leo/core/leoFileCommands.py b/leo/core/leoFileCommands.py
index 79eee8923..456a2cea5 100644
--- a/leo/core/leoFileCommands.py
+++ b/leo/core/leoFileCommands.py
@@ -190,6 +190,14 @@ class FastRead:
 if attr.startswith('str_'):
 if isinstance(val, (str, bytes)):
 return g.toUnicode(val)
+# Support JSON encoded attributes
+if attr.startswith('json_'):
+if isinstance(val, (str, bytes)):
+try:
+return json.loads(g.toUnicode(val))
+except json.JSONDecodeError:
+# fall back to standard handling
+g.trace(f"attribute not JSON encoded {attr}={val}")
 try:
 # Throws a TypeError if val is not a hex string.
 binString = binascii.unhexlify(val)
@@ -2172,6 +2180,17 @@ class FileCommands:
 g.trace(type(val), repr(val))
 g.warning("ignoring non-string attribute", key, "in", torv)
 return ''
+# Support JSON encoded attributes
+if key.startswith('json_'):
+try:
+val = json.dumps(val, ensure_ascii=False, separators=(',', ':'), sort_keys=True)
+except TypeError:
+# fall back to pickle
+g.trace(type(val), repr(val))
+g.warning("pickling JSON incompatible attribute", key, "in", torv)
+else:
+attr = f' {key}={xml.sax.saxutils.quoteattr(val)}'
+return attr
 return self.pickle(torv=torv, val=val, tag=key)
 #@+node:EKR.20040526202501: *5* fc.putUnknownAttributes
 def putUnknownAttributes(self, v):

Re: Icons with relative paths - status report

[spike]

Within Leo, header icons are only used by the default status icon, a few
UI commands, the 'todo' plugin, the 'declutter' feature, and a handful
of unused functions in editCommands. All live references are limited to
only four files:
commands/editCommands.py
core/leoCommands.py
plugins/qt_tree.py
plugins/todo.py


There are a few dead references as well, limited to two files:
plugins/auto_colorize2_0.py contains a commented out block and tkGui was
obsoleted in 2013.


There doesn't appear to be any published third party code that uses
header icons. Searching online for various related keywords only brings
up references to Leo's source code and documentation, and bug report
1559. This doesn't rule out unpublished scripts, of course.


The commands and documented API functions are quite limited. I still
need to read through the documentation on these, but having read the
code they are all very limited. User scripts must therefore be equally
limited or use raw data access.

insert-icon insertIcon()
insertIconFromFile()
cloneFindByPredicate()

delete-first-icon deleteFirstIcon()
delete-last-icon deleteLastIcon()
delete-node-icons deleteNodeIcons()


deleteIconByName() is an undocumented function left over from tkGui and
is the only surviving code that reads the 'relPath' attribute. It is a
logical complement to insertIconFromFile() so it might be used by user
scripts.


Internally Leo uses getIconList(), appendImageDictToList(), and
setIconList(), none of which are documented. These all involve raw
access to icon data so there is no telling what user scripts might do
with them.

appendImageDictToList() is the best place to catch legacy code modifying
icon data.


I still haven't traced all of the icon data attributes to see what they
mean and which would be good candidates for API access, but the 'type'
attribute should probably be deprecated. It was only ever used by tkGui,
and only ever had one defined value: 'file'.


The 'relPath' attribute is troublesome because it may hold an absolute
path or a theme relative path, and because of deleteIconByName() might
actually be used by someone. It otherwise hasn't been used since tkGui.


Current code depends on the 'path' attribute always being an absolute
path, even when it isn't valid. This behaviour needs to be removed, but
that causes compatibility issues. On the other hand sharing files with
these paths is already broken unless the sender and receiver have
matching filesystems, so there are already compatibility issues.

Only 'declutter' isn't affected, because it generates 'path' at runtime.


There are very good reasons for backwards compatibility, but there needs
to be a way forward as well. There is some complexity since there are
five independent sources of icons getting mixed together and the order
may be significant, especially if some symbols are used by more than one
source. Adding in document-relative paths just adds another layer to that.

I'm going to need to sleep on this and think about it some more.

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/d5923c71-ec0a-5983-1009-d185f52cd6cf%40runbox.com.

Re: minor selection inconsistency with move-lines-up/down

[spike]

The KWrite editor works almost exactly the same way, just
Ctrl-Shift-Up/Down, and it doesn't insert blank lines if you try to go
past the top or bottom of the file.

It can be a bit surprising that it works this way, but it's consistent
because you highlighted the newline character at the end of the upper
line. If you cut, it gets cut, too.

Try highlighting between the 'i' and 'n' characters instead of at the
end to see why that makes sense.

Cheers.

On 2022-07-19 03:24, jkn wrote:
> Hi all
> I have been using the 'move-lines-up' and 'move-lines-down' commands
> (Ctrl-up and Ctrl-down) quite a bit recently. They are very handy and I miss
> them in other editors I use ;-)
> 
> I have seen a small inconsistency though; I flag it here for consideration
> as an enhancement. It's to do with how the lines to move are selected.
> 
> Lets say I have three lines:
> 
> line1
> line2
> line3
> 
> - if I have my cursor on line 1, then Ctrl-up/down moves just line 1
> (fine)
> 
> - if I have my cursor at the beginning of line 1, then press shift-down,
>   the whole of line 1 is selected, and my cursor is at the beginning of 
> line2
>  (fine so far)
> 
> - but if I then press Ctrl-up, both Line 1 *and* line 2 move: not what I 
> expect!
> I have to move my cursor 'left' (to be on the end of line 1), for only
> line 1 to move.
> 
> I can see how this occurs, but it is different if, say, having selected
> line 1 in the same way, I were to Ctrl-x (cut) it. Only the selected text
> in line 1 would be cut, not line 2 as well.
> 
> If possible I would request that this behaviour is changed. However
> it's pretty low priority, I would agree.
> 
> Thanks for Leo!
> Jon N
> 

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/e5314dcb-1f1d-c2f1-c992-33fcc984edc0%40runbox.com.

Re: Icons with relative paths

[spike]

Since we need an argument to distinguish between document and theme
relative paths, I'm using the absence of that argument to set legacy
mode. For absolute paths it doesn't make any difference. I just need to
test for absolute or relative.

The devil is, of course, in the details. I keep overthinking things and
wandering off on tangents, or I think I'd be done already. It really
shouldn't be complicated.

On 2022-07-18 19:41, tbp1...@gmail.com wrote:
> You can add a new parameter to the method call, say use_legacy = True.  
> Then all existing code will work as always with the default value of 
> use_legacy.  You add a new code branch for when use_legacy is False so you 
> can get what you want in that case.  Tests for file existence will work 
> with relative paths or absolute paths, and you can easily determine which 
> you have.  I may be missing something (I often do), but it doesn't sound 
> very tricky.
> 
> On Monday, July 18, 2022 at 7:16:47 PM UTC-4 spike wrote:
> 

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/5b1147a7-dae3-061d-f845-2fc756ee3e12%40runbox.com.

Re: Icons with relative paths

[spike]

I can kludge it, but it introduces some subtle but potentially nasty
bugs. It really needs to be fixed properly in the next major update.


I suppose adding a 'pathType' parameter to insertIconFromFile would 
work. If the parameter is missing then it uses legacy mode and issues a 
deprecation warning to the log.


Splitting the "insert-icon" command and menu option isn't a problem
since it isn't meant to be scripted.


On the data side, a new attribute is needed for document relative paths,
'docRelPath', with absolute paths in the other two attributes to mimic
the current behaviour when using legacy code.

The problems with this are that the legacy paths are possibly invalid 
and legacy code directly modifying icon data won't know to update the 
new path, putting them out of sync. New code can't necessarily fix it 
because the updated absolute paths may still be invalid. This is 
especially troublesome when sharing outlines between users with 
different Leo versions. On the other hand the 'relPath' attribute 
suffers from the same problems.



On 2022-07-18 14:36, tbp1...@gmail.com wrote:

I'm all in favor of improving features, but please find a way to make
them continue to work with existing code.  It seems to me that the
easiest way to go about this would be to make the code fall back to a
document or outline-relative path if it can't find the icon files in
the other standard places it already looks for.

On Monday, July 18, 2022 at 4:23:06 PM UTC-4 spike wrote:

g.os_path_exists works with relative paths but 
c.editCommands.insertIconFromFile forces relative paths to be

relative to the Leo theme directory. Document relative paths are
rejected.

Running the following command from Leo's Python Console tab fails: 
c.editCommands.insertIconFromFile('./icons/xfsm-lock.png', p=p)


This works, but only for theme icons: 
c.editCommands.insertIconFromFile('light/checkbox_checked.svg',

p=p)

This also works, but forces an absolute path: import os

c.editCommands.insertIconFromFile(os.path.abspath('./icons/xfsm-lock.png'),




p=p)


c.editCommands.insertIconFromFile calls appendImageDictToList,
which then calls g.app.gui.getTreeImage, which requires relative
paths to be relative to the current Leo theme. Document-relative
paths are rejected with an unhelpful "can not load image: None"
message in the log.

There are three possible use cases for icons, but the current logic
only supports two of them. 1) Theme relative path. This is used by
the "to-do" plugin, which seems to be why the icon feature exists
in the first place. 2) Document relative path. Useful but
unsupported. 3) System relative path. Supported, limited use, but
not portable.


As it stands the current implementation is incomplete. If there are
no objections I will work on a fix.

TO DO: * The 'relPath' attribute needs to be replaced by an
'isThemeIcon' flag. Old icon data needs to be automatically updated
to the new format on file load. Theme paths should also respect the
current theme.

* I'm debating adding an option to
c.editCommands.insertIconFromFile or making a new
insertThemeIconFromFile function for theme icons.

* The insert-icon command should get a companion insert-theme-icon 
command so that the Open dialog starts in the right folder.


* The to-do plugin needs to be updated to the new API.

* There should also be a visible indicator when an icon file is
missing or unreadable. I'm thinking a red 'x' in a white box, like
web browsers do.

* Test cases need to be written.

* Then documentation.


I'm still new to Leo coding, so I'll probably have questions.

Thank you for your patience.


On 2022-07-18 08:04, tbp1...@gmail.com wrote:

Typically, either you write code and try different paths
depending on whether your file is in one place or another, or a
Leo method does the

same

thing for you, in which case you need to know where Leo is going
to look. For example, in the part of the Cheatsheet you mention:

fn = g.os_path_finalize_join(g.app.loadDir, '..', 'Icons',
'Tango', '16x16', 'status', icon) if g.os_path_exists(fn): 
c.editCommands.insertIconFromFile(path=fn)


g.os_path_exists() works with relative paths as wel as absolute
paths.

It would be helpful if you explain just how you are trying to use
these icons. If it is via a Leo setting, which settings? If via
code you have written, please show the code. On Monday, July 18,
2022 at 5:55:21 AM UTC-4 spike wrote:


On 2022-07-17 11:29, Félix wrote:

Thanks for the examples spike, but you replied to me only
instead of

reply

all, so your examples are not showing in the thread.

Here is the example spike provided: (see attachments)

On the other hand, i've looked at the 'unknownattributes' of
the nodes,

and

indeed the paths provided for the

Re: Icons with relative paths

[spike]

g.os_path_exists works with relative paths but 
c.editCommands.insertIconFromFile forces relative paths to be relative 
to the Leo theme directory. Document relative paths are rejected.


Running the following command from Leo's Python Console tab fails:
 c.editCommands.insertIconFromFile('./icons/xfsm-lock.png', p=p)

This works, but only for theme icons:
 c.editCommands.insertIconFromFile('light/checkbox_checked.svg', p=p)

This also works, but forces an absolute path:
 import os

c.editCommands.insertIconFromFile(os.path.abspath('./icons/xfsm-lock.png'), 
p=p)


c.editCommands.insertIconFromFile calls appendImageDictToList, which 
then calls g.app.gui.getTreeImage, which requires relative paths to be 
relative to the current Leo theme. Document-relative paths are rejected 
with an unhelpful "can not load image: None" message in the log.


There are three possible use cases for icons, but the current logic only 
supports two of them.
1) Theme relative path. This is used by the "to-do" plugin, which seems 
to be why the icon feature exists in the first place.

2) Document relative path. Useful but unsupported.
3) System relative path. Supported, limited use, but not portable.


As it stands the current implementation is incomplete. If there are no 
objections I will work on a fix.


TO DO:
* The 'relPath' attribute needs to be replaced by an 'isThemeIcon' flag. 
Old icon data needs to be automatically updated to the new format on 
file load. Theme paths should also respect the current theme.


* I'm debating adding an option to c.editCommands.insertIconFromFile or 
making a new insertThemeIconFromFile function for theme icons.


* The insert-icon command should get a companion insert-theme-icon 
command so that the Open dialog starts in the right folder.


* The to-do plugin needs to be updated to the new API.

* There should also be a visible indicator when an icon file is missing 
or unreadable. I'm thinking a red 'x' in a white box, like web browsers do.


* Test cases need to be written.

* Then documentation.


I'm still new to Leo coding, so I'll probably have questions.

Thank you for your patience.


On 2022-07-18 08:04, tbp1...@gmail.com wrote:

Typically, either you write code and try different paths depending on
whether your file is in one place or another, or a Leo method does the same
thing for you, in which case you need to know where Leo is going to look.
For example, in the part of the Cheatsheet you mention:

 fn = g.os_path_finalize_join(g.app.loadDir,
 '..', 'Icons', 'Tango', '16x16', 'status', icon)
 if g.os_path_exists(fn):
 c.editCommands.insertIconFromFile(path=fn)

g.os_path_exists() works with relative paths as wel as absolute paths.

It would be helpful if you explain just how you are trying to use these
icons.  If it is via a Leo setting, which settings?  If via code you have
written, please show the code.
On Monday, July 18, 2022 at 5:55:21 AM UTC-4 spike wrote:


On 2022-07-17 11:29, Félix wrote:

Thanks for the examples spike, but you replied to me only instead of

reply

all, so your examples are not showing in the thread.

Here is the example spike provided: (see attachments)

On the other hand, i've looked at the 'unknownattributes' of the nodes,

and

indeed the paths provided for the icons is not relative to the Leo file,
but are instead complete paths that will not work anywhere else: such as
*/home/keith/tmp/test/icons/blablabla.png*

instead of *icons/blablabla.png*

Lastly, i tried using / setting / finding out about icons on nodes as I
was totally unaware that this was a thing... i could not find anything

...

Where / how did you get those icons and icon attributes? searching for
'icons' in the documentation yields some other unrelated stuff... Like

can

you take screenshots on how you set those?

(sorry i'm totally ignorant about those 'node icons' that you speak of,
although i can see they exists in your screenshots and that the

attributes

are there in the .leo file.)

Félix



On Sunday, July 17, 2022 at 7:46:03 AM UTC-4 Edward K. Ream wrote:


On Sat, Jul 16, 2022 at 9:50 PM Robert-Felix 

wrote:


Leo uses relative paths for most stuff so moving the leo file and your

'project' or 'resources' folder to accompany it along should keep
everything working fine. whatever you had going on...



Leo's path expressions
<http://leoeditor.com/directives.html#path-expressions> should suffice
for any conceivable purpose :-)

Edward




Sorry about the misdirected reply. I'll try to do better in future.

Thanks for reposting test file attachment. I'm posting a shot of the
menu option as well as the 'good.png' and 'bad.png' example screenshots.


The documentation is sparse a

Icons with relative paths

[spike]

I have a .leo file that has a companion folder containing custom icons. 
Leo doesn't seem to support relative paths, so moving the .leo file and 
its companion folder breaks all of the icons in the file. On top of 
that, if Leo can't find an icon then there is no visual indication that 
something is missing.


Any suggestions?

--
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/f080eec2-b3b4-a3ab-28a5-2fcfc0890065%40runbox.com.

v6.4b2 "newline-and-indent" unit test failure [patch]

[spike]

patch 73eaf41fff7c150012448f8113ca45623bb5aa23
   "Use textwrap.dedent instead of g.adjustTripleString everywhere"
   in v6.4b2 breaks "newline-and-indent" unit test.

The problem is that textwrap.dedent clears empty lines and the test 
needs to insert an indented blank line in order to work. Attached patch 
fixes the problem by disabling textwrap.dedent for the affected test.



Error messages:
Test case for newline-and-indent ... mismatch in body
expected:
[
 'first line\n',
 'line 1\n',
 '\n',
 'line a\n',
 'line b\n',
 'line c\n',
 'last line\n'
]
got:
[
 'first line\n',
 'line 1\n',
 '\n',
 'line a\n',
 'line b\n',
 'line c\n',
 'last line\n'
]
parent_p.b ''
FAIL

Traceback (most recent call last):
   File "leo/unittests/commands/test_editCommands.py", line 2498, in 
test_newline_and_indent

 self.run_test(
   File "leo/unittests/commands/test_editCommands.py", line 53, in run_test
 assert False
AssertionError

--
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/28b940cc-4bb7-965e-29ad-1edd5a0c4e85%40runbox.com.
diff --git a/leo/unittests/commands/test_editCommands.py b/leo/unittests/commands/test_editCommands.py
index d9ccff847..2ce0b75bd 100644
--- a/leo/unittests/commands/test_editCommands.py
+++ b/leo/unittests/commands/test_editCommands.py
@@ -18,7 +18,8 @@ class EditCommandsTest(unittest.TestCase):
 before_b, after_b,  # before/after body text.
 before_sel, after_sel,  # before and after selection ranges.
 command_name,
-directives=''
+directives='',
+dedent=True
 ):
 c = self.c
 # For shortDescription().
@@ -27,9 +28,12 @@ class EditCommandsTest(unittest.TestCase):
 command = c.commandsDict.get(command_name)
 assert command, f"no command: {command_name}"
 # Set the text.
-parent_b = textwrap.dedent(directives)
-before_b = textwrap.dedent(before_b)
-after_b = textwrap.dedent(after_b)
+if dedent:
+parent_b = textwrap.dedent(directives)
+before_b = textwrap.dedent(before_b)
+after_b = textwrap.dedent(after_b)
+else:
+parent_b = directives
 self.parent_p.b = parent_b
 self.tempNode.b = before_b
 self.before_p.b = before_b
@@ -2498,9 +2502,10 @@ class EditCommandsTest(unittest.TestCase):
 self.run_test(
 before_b=before_b,
 after_b=after_b,
-before_sel=("2.6", "2.6"),
-after_sel=("3.4", "3.4"),
+before_sel=("2.10", "2.10"),
+after_sel=("3.8", "3.8"),
 command_name="newline-and-indent",
+dedent=False
 )
 #@+node:ekr.20201130090918.92: *3* next-line
 def test_next_line(self):