Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 2:48 PM, vitalije  wrote:

@Edward, I have another question. Where is the point where the actual
reading of Leo document ends and data is transferred in the tree.

 To be more concrete I have found in method getLeoFileHelper of
FileCommands class the following lines:

c.setRootVnode(v)

fc.rootVnode = v

I thought that is what I was looking for, but on my surprise c.setRootVnode
is empty method. It does nothing.

​You have found the spot. At this point, the sax code has set c.hiddenRootNode.


It looks like *both* the statements you show can be eliminated. Not only
does c.setRootVnode do nothing, but fc.rootVnode is never used! I'll clean
this up soon. My apologies for the confusion.

So, how c commander gets the result of loading Leo document?

​Using c.hiddenRootNode, set by the sax read code.​
​ ​
Here is c.rootPosition
​:

def rootPosition(self):

c = self
if c.hiddenRootNode.children:
v = c.hiddenRootNode.children[0]
return leoNodes.Position(v, childIndex=0, stack=None)
else:
return None

If I knew that my prototype will be finished very soon.

​Oh good. We can have a race :-)

I think I know how to make Leo reloadable, without changing Leo's core much.

The timing of this discussion is extremely lucky, as I have just spent lots
of time delving into caching. The plan is as follows:

1. A new *reload-leo* command will write every open Leo outline to a *cache
list*, using a variation of cacher.makeCacheList:

def makeCacheList(self, p):
return [
p.h, p.b, p.gnx,
[self.makeCacheList(p2) for p2 in p.children()]]

The command will also write leoSettings.leo and myLeoSettings.leo
to their own cache lists, if they are not already open.

2. The reload-leo will then do the following:

- set a g.app.reloading flag
- reload all Leo modules
- re-execute all of Leo's startup code, for each previously open outline,
including leoSettings.leo and myLeoSettings.leo.

If g.app.reloading is True, fc.getLeoFile will get *everything* from the
proper complete cache list via a single (recursive) call to
cacher.createOutlineFromCacheList.

Boom. No calls to sax. No need to reload external files. This should be
almost instantaneous.
Because all modules have already been reloaded, all newly-created data
reflect changes to, say, the VNode, Position or Commander classes.

There will likely be other tests against the g.app.reloading flag. The easy
way would be to have Leo re-parse the cached settings file. Later, the
settings themselves (the settings dicts) could be saved and restored.

It's conceivable that this could be done in just a few days...

*Summary*

Caching all open outlines, plus settings files, will allow Leo to
re-execute its *complete* startup logic after reloading all affected
modules. This will allow Leo to update *all* its data structures
*quickly *without
special cases. This is a fascinating possibility.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 12:48:06 -0700 (PDT)
vitalije  wrote:

> > ​I would be happy with that, provided that a) settings can be
> > organized 
> >> and b) per-document (.leo file) settings are possible.​ ​ The best
> >> way of organizing anything is in a .leo file.​
> >
> As I wrote in the earlier post today, it is trivial in sqlite to
> open connection to several different databases (files). So it is easy
> to have one database file for settings machine wide, another one for
> settings per user, and yet another one for settings per folder (if I
> am not mistaken there used to be possible for user to define settings
> per project/folder, although I have never used that possibility). It
> is easy to make script to dump all relevant (leo,
> myLeo) settings-files in separate databases. 

The docs. on settings location are here:
http://leoeditor.com/customizing.html#id3

not 100% sure but I assume `homeDir` is ~/.leo or $HOME\.leo and
`configDir` is .../leo/config/, and `localDir` is the folder containing
the .leo file.

Hmm, I'd forgotten about the machine-name variation, although it can be
very useful when the same filesystem is accessed from different
hardware setups.

Essential: 1a, 2b, 3
Consider dropping: 1b, 1c, 2a - really when would you use these?
Useful: 2c, 2d - 2d when you want .leo files in a folder to share
 settings

So there's a question of when to use separate DB files vs. info. in the
DB to separate contexts.

1a and 2b would obviously be their own DB files, with a search that
fails in 2b being repeated in 1a, trusting for now that that won't be a
performance issue.

3 is trickier.  We're going to (ok, proposing to ;-) separate the
settings data from the rest of the .leo file(*).  Managing two files
side by side seems obnoxious, from the users point of view.  I think it
would be better to put the outline specific info. in the same DB as
used for 2b (e.g. ~/.leo/myLeoSettings.sqlite3).  So the setting table
might look like:

setting: long - primary key for settings table
name: text - setting name
body: text - text data for setting for docs. and @data settings
type: long - link to types table (@bool, @int, @data, etc.)
parent: long - link to settings table for hierarchy
outline: long - link to outline table

We'd start by looking at c.db to see how the outline table might manage
outline identities.

(*) this separation is a high cost thing.  It means the user can't just
move the .leo file to another system and have that file's local
settings follow along.  This is consistent with myLeoSettings.leo also
not following along, but still seems like a regression.

Which suggests that at least for file local settings we might want to
continue to support .leo file @settings trees.  Hmm.

That leads to wanting to make .leo file @settings trees more writable
than they are now, which leads to the possibility of using Leo outlines
rather than sqlite as a storage format.  Before this is interpreted as
scraping the idea completely, I'll point out the leosax.py module which
reads .leo files into a light weight dict representation of an outline
many many times faster than Leo's regular read code.  It ignores
external files, and is read only.

Alternatives are

(a) keeping the .leo and settings DBs side by side,
something.leo and something.leo.settings.sqlite?
(b) embedding the sqlite data in a the .leo XML file - 
that seems like a horrible idea.

So I think the question of the cost of separating file-local settings
from the .leo file is an issue here.  It sort of runs contrary to the
idea of XML, which should allow everything to be in one file.

But anything that deviates from simple vanilla use of sqlite DB files
starts to lose the simplicity gain.  Hmm.

Cheers -Terry


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[vitalije]

>
> ​I would be happy with that, provided that a) settings can be organized 
>> and b) per-document (.leo file) settings are possible.​ ​ The best way of 
>> organizing anything is in a .leo file.​
>
> As I wrote in the earlier post today, it is trivial in sqlite to open 
connection to several different databases (files). So it is easy to have 
one database file for settings machine wide, another one for settings per 
user, and yet another one for settings per folder (if I am not mistaken 
there used to be possible for user to define settings per project/folder, 
although I have never used that possibility). It is easy to make script to 
dump all relevant (leo, myLeo)settings-files in separate databases. 

@Edward, I have another question. Where is the point where the actual 
reading of Leo document ends and data is transferred in the tree. To be 
more concrete I have found in method getLeoFileHelper of FileCommands class 
the following lines:

c.setRootVnode(v)

fc.rootVnode = v

I thought that is what I was looking for, but on my surprise c.setRootVnode 
is empty method. It does nothing. 
So, how c commander gets the result of loading Leo document? If I knew that 
my prototype will be finished very soon. I have already made possible for 
launchLeo.py to accept '.db' as extension for Leo documents in sqlite 
format and it passes sqlite connection as theFile argument to 
getLeoFileHelper. It could be easily intercepted even before in 
LoadManager.openFileByName 
method and control transferred to the caller with complete new outline read 
from database. I just don't know where to transfer the tree restored from 
database and built in memory. 
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 13:25:39 -0500
"Edward K. Ream"  wrote:

> On Wed, Jun 28, 2017 at 12:05 PM, Terry Brown 
> wrote:
> 
> > On Wed, 28 Jun 2017 06:39:41 -0700 (PDT)
> 
> The success of the current
> > ​ ​
> > code in the context of 1. is questionable, seeing new and even
> > experienced users struggle with settings management.
> 
> ​Imo few (none?) of those problems arise because Leo represents
> settings in .leo files.​

Well, no, I guess using .leo files for storage isn't a problem from the
user point of view.  I think having to edit an outline and use text
markup and decide which file to edit and how to express values
are challenges for users, depending on their backgrounds.  Using a DB
for setting storage would, to me anyway, make it much easier to write a
more friendly interface for updating settings.

> ​I object to changing the UI on the grounds that it would simplify
> code. I don't want people to have to change their settings files!
> Maybe a prototype will convince me that settings files are dumb
> idea.  Until then, I remain skeptical.

Of course.  I think simplifying code has value in its own right,
although it may not always be a priority.  And in this case simplifying
code may make it easier to simplify part of the user experience.
But degrading or even perturbing the user experience simply to simplify
code would not make sense.

> Incremental proposals
> would likely be easier to understand and evaluate.

Hmm, I suppose a settings DB could be made to shadow the existing
settings and then used as a basis to demo alternative settings
editing.

Cheers -Terry


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 12:05 PM, Terry Brown  wrote:

> On Wed, 28 Jun 2017 06:39:41 -0700 (PDT)
>

The success of the current
> ​ ​
> code in the context of 1. is questionable, seeing new and even
> experienced users struggle with settings management.
>

​Imo few (none?) of those problems arise because Leo represents settings in
.leo files.​



> 2. GUI wise Leo has discarded GUI's in the past, and now has two.  I
> think what you're saying is that setting management *must* remain
> something that can be done within Leo using the tree and body panes.
> I guess that's an understandable desire, although to date it doesn't
> help resolve the settings management challenges.
>

​Yes, I think that's what I meant ;-)​


I'm not sure if you meant to apply the scripting api clause to this
> discussion, but it doesn't seem relevant.  There would be no change to
> the signature of g|c.config().


​Good.  I wasn't sure about this point.
​


> > For example, yesterday I discovered a settings table that inited
> > several commander ivars.  My first thought was, maybe this table
> > could go away. But no.  These ivars appear all over Leo.  It would be
> > horrible to use c.config.getInt or c.config.getBool in their stead.
>
> If you're referring to ivarsData, I don't think that's relevant - the
> issue of objects "caching" config. vars. as ivars is orthogonal to the
> proposed alternative g|c.config() implementation.
>

​I was referring to ivarsData.  ​I didn't mean to imply that this was
directly relevant to the discussion, although Vitalije's suggestion to
replace these ivars with python properties is interesting.

>
> I think a confusing part of this discussion is that it's unclear
> whether it's about simplifying code or improving the settings ui.  I
> think it's both, i.e. a proposal to use a DB as a key value store that
> would hugely simplify loading of settings, and also reduce the
> technical barriers to implementing a less challenging settings UI.
>

​I object to changing the UI on the grounds that it would simplify code. I
don't want people to have to change their settings files!  Maybe a
prototype will convince me that settings files are dumb idea.  Until then,
I remain skeptical.
​


> But as long as there's a requirement for Leo settings to be editable as
> Leo outlines, the DB implementation is probably not useful.
>

​That's the rub.  But maybe I'll get really excited by the prototype.​


With respect to the settings struggles
> ​ [big snip] ​
> I think it's clear that the current settings
> management options will always be a barrier for a significant number of
> users.


​I don't have that sense, but if it's true I would welcome proposals to
simplify matters. A proposal that says, in effect, I have a great idea for
throwing away all of Leo's settings machinery is going to be a hard sell,
but possibly not impossible sell.  Incremental proposals would likely be
easier to understand and evaluate.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 11:01 AM, vitalije  wrote:

By transferring to ClojureScript I have discovered some very cool projects
>> that I could not imagine were possible at all. One of the coolest things is
>> writing reloadable code. I am amazed how cool it is to make small change in
>> your code and in a few milliseconds browser shows the effect of changed
>> code. It is impossible to express in words the feeling. Once you try it and
>> feel it for yourself you can never tolerate again any tool that requires
>> you to reload or to wait more than a second for the effect of your action
>> to be visible.
>>
> ​
I keep coming back to this in my thoughts. Making Leo reloadable would
instantly make Leo a killer app.

How to do a prototype in the least amount of time? The way forward is
murky. Yes, Python has imp.reload, but it's not enough to reload code. All
of Leo's objects must be reloaded.

Prototypes can sweep all kinds of inconvenient details under the rug.  All
hacks are allowed, including hacks to avoid reloading settings files,
plugins, whatever. But hacks are not enough.  A strategy is needed.

The situation is akin to the time before Leo's first breakthrough, namely
the Aha that I could use MORE as a prototype for Leo.  What kind of
prototype could steer us in the direction of fully reloadable python code?
What kind of clever scaffolding is needed?

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Offray Vladimir Luna Cárdenas]

Yes, Smalltalk has liveness build in the core experience since early
releases. Is something that you can not find almost anywhere the common
experience of programming and computing in general. Recently I was
reading about how this liveness impressed Steve Jobs and his team and
inspired much of the computing after that (despite GUI, paradoxically
hid the live coding idea). Here are the details:

https://www.quora.com/What-was-it-like-to-be-at-Xerox-PARC-when-Steve-Jobs-visited/answer/Alan-Kay-11

With Grafoscopio, I want to leave outlining for writing and exploratory
computing in data storytelling, but also use Pharo browsers and code
inspectors when fits the problem better. I want to stretch the outline
metaphor as much as I can, while recognizing other ways of dealing with
coding and user experience.

Cheers,

Offray

On 28/06/17 12:22, Terry Brown wrote:
> On Wed, 28 Jun 2017 12:10:27 -0500
> Offray Vladimir Luna Cárdenas  wrote:
>
>> Using Pharo Smalltalk I have experienced a similar feeling about a
>> direct live system where changes are reflected immediately without
>> any reload.
> Years ago someone showed me the Squeak Smalltalk environment.  It
> seemed that it had "liveness" built in from the lowest levels.  As in
> open an inspector / mutator dialog for the desktop background color
> object and watch it change dynamically as you change the value.  If
> Pharo's similar, I think it has more liveness intrinsically.
>
> But there is certainly potential for making some things more live in
> Leo.  We just have to work within PyQt etc. and how they work.
>
> Cheers -Terry
>


-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Wed, Jun 28, 2017 at 11:01 AM, vitalije  wrote:

Every ivar that caches settings can be turned into property that gets its
>> value from c.config.get<
>> ​something>.
>>
>
​Excellent idea. There is a performance penalty, but it is not likely
significant.

It wouldn't be so hard to even make a script that would search and find
> every occurrence in Leos code of such ivars and turned them into properties.
>
​
Yes.
​


> By transferring to ClojureScript I have discovered some very cool projects
> that I could not imagine were possible at all. One of the coolest things is
> writing reloadable code.
>


> I am amazed how cool it is to make small change in your code and in a few
> milliseconds browser shows the effect of changed code. It is impossible to
> express in words the feeling. Once you try it and feel it for yourself you
> can never tolerate again any tool that requires you to reload or to wait
> more than a second for the effect of your action to be visible. Maybe I
> become a bit spoiled after experiencing all that and now I am expecting
> from each tool I use to have such feature.
>

​Very cool.  This is purpose-level thinking.
​


> @Edward, you have mention the Aha moment when @clean file become possible.
> I had proposed that few years earlier but you didn't understand my idea. I
> don't know is it possible to find those messages on this list but I
> remember that I proposed for shadow files that were used to be kept in
> hidden folder as ordinary files that their content could be stored inside
> Leo document itself. That is precisely what @clean nodes do today. Isn't it?
>

​Close enough. The "diff" is between the external file (without sentinels)
and and the results of simulating writes from the @clean tree to two strings*,
with and without* sentinels.

Well if I were able to explain that idea better @clean feature could have
> been implemented few years earlier.
>

I see.  My biggest regret is not realizing @shadow was sound (ambiguities
don't permanently matter) before Bernhard Mulder died.  He may have thought
the theorem was trivial, so didn't bother to state and prove it.
​

> In Leo's site there is paragraph where Edward explains why he likes Python
> (here )
>
> Before using Python I never fully realized how difficult and dangerous
> memory allocation is in C++. Try doing:
>
> aList[i:j] = list(aString)
>
> in C. You will write about 20 lines of C code. Any error in this code will
> create a memory allocation crash or leak.
>
> Now I am trying to say the same thing about settings code. All settings
> could be present in just about two lines of code: sqlite3.connect(...) and
> conn.execute('select ...').fetchone().
>

​That would be great internally. Can you init​

​sqlite from leoSettings.leo, myLeoSettings.leo and the local .leo files?
​

> Surely, Leo has proven that it is possible to keep settings in the same
> file format in which Leo documents are kept. But it was achieved by writing
> a lot lines of code and that means lot of possible places for bugs to hide.
> It would not make a memory leak or hard crash, but nevertheless it can
> cause a lot of frustration to the user (especially to spoiled one) . What
> is more important it can prevent (and if I may say so it does prevent)
> achieving such a cool feature as reloadability. Who knows what other
> features could be discovered once the reloadability is achieved.
>

​Reloadability is a separate issue.​

​ Leo could cache settings so that reload time is minimized.​

>
> Present scheme for keeping settings allows hierarchical management. Well
> it does. But honestly speaking how big is that benefit? I for example don't
> remember when I touched myLeoSettings.leo file last time. But I do remember
> that it was painful every time I did. Being able to see my settings
> in hierarchical view isn't that useful if you must read a ton of
> documentation to figure out what is the proper way to set something.
>

​Leo has hundreds of settings.  I change them in myLeoSettings.leo or a
local .leo file all the time​

​for testing.  There have been few complaints about @bool, @int, etc. The
confusion comes from knowing which settings are in effect.

I suppose we could say that *all* settings come only from
myLeoSettings.leo.  This would mean a) copying settings from
leoSettings.leo and b) foregoing the possibility of per-file settings.
This does not seem attractive to me.
​

> If you are lucky and everything work as advertised you have to open
> settings file make change and then reopen all your documents.
>

​The full reload-settings command is supposed to do this without reloading
files.
​


> But if it doesn't work you have to go through the documentation to figure
> where you went wrong and why your new setting is not accepted. If you are
> very unlucky, there can be a bug somewhere that prevents your settings to
> take place and there would be a lot of combinations to try (as 

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 12:10:27 -0500
Offray Vladimir Luna Cárdenas  wrote:

> Using Pharo Smalltalk I have experienced a similar feeling about a
> direct live system where changes are reflected immediately without
> any reload.

Years ago someone showed me the Squeak Smalltalk environment.  It
seemed that it had "liveness" built in from the lowest levels.  As in
open an inspector / mutator dialog for the desktop background color
object and watch it change dynamically as you change the value.  If
Pharo's similar, I think it has more liveness intrinsically.

But there is certainly potential for making some things more live in
Leo.  We just have to work within PyQt etc. and how they work.

Cheers -Terry

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Offray Vladimir Luna Cárdenas]

Hi,

Using Pharo Smalltalk I have experienced a similar feeling about a
direct live system where changes are reflected immediately without any
reload. Also I have shared that dealing with traversing the settings
three could be cumbersome and you need to read a lot before having the
change you want, if everything goes well. I would like something like
using the minibuffer or some interactive console to make such changes
possible and direct and despite using Leo less (I'm focusing on using
and improving Grafoscopio [1]) I advocate the exploration of improving
liveness in the Leo environment, so Sqlite seems a good candidate for that.

[1] http://mutabit.com/grafoscopio/index.en.html

As I've told, Leo (tool and community) is an important source of
inspiration and all this criticisms are directed towards making it
better and not on authors or their pretty good engineering decisions in
general.

Cheers,

Offray

On 28/06/17 11:01, vitalije wrote:
>
>
> For example, yesterday I discovered a settings table that inited
> several commander ivars.  My first thought was, maybe this table
> could go away.  But no.  These ivars appear all over Leo.  It
> would be horrible to use c.config.getInt or c.config.getBool in
> their stead.  So this table creates a major convenience in Leo's
> code, even while it complicates the startup code and (eventually)
> will complicate the reload-settings command.  It a choice, and I
> think the tradeoff is a good one.
>
> Edward
>
>
> Every ivar that caches settings can be turned into property that gets
> its value from c.config.get. It wouldn't be so hard to even
> make a script that would search and find every occurrence in Leos code
> of such ivars and turned them into properties. That way Leo would have
> single source of settings data and every change in the source would be
> visible every where instantaneously. 
>
> Last few months I have been studying ClojureScript and experimenting
> with it. Maybe last 5 years or so, my programming was mostly oriented
> towards client scripting in web browsers. Until recently I was using
> almost exclusively  CoffeeScript for all programming tasks. By
> transferring to ClojureScript I have discovered some very cool
> projects that I could not imagine were possible at all. One of the
> coolest things is writing reloadable code. I am amazed how cool it is
> to make small change in your code and in a few milliseconds browser
> shows the effect of changed code. It is impossible to express in words
> the feeling. Once you try it and feel it for yourself you can never
> tolerate again any tool that requires you to reload or to wait more
> than a second for the effect of your action to be visible. Maybe I
> become a bit spoiled after experiencing all that and now I am
> expecting from each tool I use to have such feature. 
>
> For all my computer related work I use Leo as editor, as a database as
> scripting environment, for processing all sorts of files, for all
> types of scripting. And I am very grateful to Edward and whole
> community for all the work they all put in this project. I am no
> newbie in Leo world nor I am newbie in python or for that matter
> almost to any programming language from Z80 and 6502 assembly language
> to C, C++, Java, Scala, Groovy, Javascript ActionScript, ... up to
> Clojure and ClojureScript.
>
> @Edward, you have mention the Aha moment when @clean file become
> possible. I had proposed that few years earlier but you didn't
> understand my idea. I don't know is it possible to find those messages
> on this list but I remember that I proposed for shadow files that were
> used to be kept in hidden folder as ordinary files that their content
> could be stored inside Leo document itself. That is precisely what
> @clean nodes do today. Isn't it?
> Well if I were able to explain that idea better @clean feature could
> have been implemented few years earlier.
> But at that time my suggestion was refused and I didn't care too much
> because I work almost exclusively alone and I have never had problems
> with synchronizing files nor with the sentinel lines. 
>
> In Leo's site there is paragraph where Edward explains why he likes
> Python (here ) 
>
> Before using Python I never fully realized how difficult and dangerous
> memory allocation is in C++. Try doing:
>
> aList[i:j] = list(aString)
>
> in C. You will write about 20 lines of C code. Any error in this code
> will create a memory allocation crash or leak.
>
>
> Now I am trying to say the same thing about settings code. All
> settings could be present in just about two lines of code:
> sqlite3.connect(...) and conn.execute('select ...').fetchone().
> Surely, Leo has proven that it is possible to keep settings in the
> same file format in which Leo documents are kept. But it was achieved
> by writing a lot lines of code and that means lot of possible places
> for bugs to hide. It would not 

Re: How good Leo handles settings?

[Terry Brown]

On Wed, 28 Jun 2017 06:39:41 -0700 (PDT)
"Edward K. Ream"  wrote:

> 1. Clarity and simplicity of *purpose*.
> 2. Stability and simplicity of *user interface*, including gui and 
> scripting api.
> 3. Simplicity of *code*.

[snip]

> In this discussion, the first two priorities demand that Leo continue
> to use outlines to represent the hierarchy of settings. 

I don't really think that follows in any particularly clear manner.

1. is a general goal no one can argue with.  The success of the current
code in the context of 1. is questionable, seeing new and even
experienced users struggle with settings management.

2. GUI wise Leo has discarded GUI's in the past, and now has two.  I
think what you're saying is that setting management *must* remain
something that can be done within Leo using the tree and body panes.
I guess that's an understandable desire, although to date it doesn't
help resolve the settings management challenges.

I'm not sure if you meant to apply the scripting api clause to this
discussion, but it doesn't seem relevant.  There would be no change to
the signature of g|c.config().  If you mean that it would no longer be
possible to edit settings by editing settings .leo files with Leo's
API, that would be true, but I would argue that no one does that
anyway.

[snip]

> For example, yesterday I discovered a settings table that inited
> several commander ivars.  My first thought was, maybe this table
> could go away. But no.  These ivars appear all over Leo.  It would be
> horrible to use c.config.getInt or c.config.getBool in their stead.

If you're referring to ivarsData, I don't think that's relevant - the
issue of objects "caching" config. vars. as ivars is orthogonal to the
proposed alternative g|c.config() implementation.

I think a confusing part of this discussion is that it's unclear
whether it's about simplifying code or improving the settings ui.  I
think it's both, i.e. a proposal to use a DB as a key value store that
would hugely simplify loading of settings, and also reduce the
technical barriers to implementing a less challenging settings UI.

But as long as there's a requirement for Leo settings to be editable as
Leo outlines, the DB implementation is probably not useful.

With respect to the settings struggles I think it's important to
remember these numbers, which I'm about to make up:

0% - percent of Leo users whose primary use of Leo is developing Leo,
rounded to the nearest percent ;-)

10%?, 20%?, 50%? - percent of Leo users whose primary use of Leo is
developing in Python (I have no idea what this number is)

10%?, 20%?, 50%? - percent of Leo users who do not use Leo for coding
at all (I have no idea what this number is either)

Given those numbers, I think it's clear that the current settings
management options will always be a barrier for a significant number of
users.  Even, hard though it is to believe, with cff :-)

Cheers -Terry



-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: I've stoped *saying* all tests pass in commit messages

[Terry Brown]

On Wed, 28 Jun 2017 07:55:41 -0700 (PDT)
"Edward K. Ream"  wrote:

> But in fact, I run all unit tests before virtually every commit.  All
> other devs should too.

Here are the fails I get from d1608d30.

I always forget to run unit tests before making changes, so I have to
make a local commit, checkout the version before my changes, count the
fails, checkout my changes again, and check the the fail count remains
unchanged.  So it would really help if the fail count could be assumed
to be zero on pulls from master.  But I appreciate how hard that is
with two version of Python, 3+ OSes, two major and many minor versions
of PyQt, and different window managers.

Trying to create a QVariant instance of QMetaType::Void type, an
invalid QVariant will be constructed instead

[this is repeated many many times]

.
==
ERROR: runTest (leo.core.leoTest.GeneralTestCase)
@test rST import test

--
Traceback (most recent call last):
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoTest.py", 
line 211, in runTest
builtins.execfile(scriptFile, d)
  File "/home/tbrown/.leo/scriptFile.py", line 74, in 
assert root.h.startswith('@@'), root.h
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 1635, in __get_h
return p.headString()
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 672, in headString
return self.v.headString()
AttributeError: 'NoneType' object has no attribute 'headString'

==
ERROR: runTest (leo.core.leoTest.GeneralTestCase)
@test rST import test: simple

--
Traceback (most recent call last):
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoTest.py", 
line 211, in runTest
builtins.execfile(scriptFile, d)
  File "/home/tbrown/.leo/scriptFile.py", line 22, in 
assert root.h.startswith('@@'), root.h
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 1635, in __get_h
return p.headString()
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 672, in headString
return self.v.headString()
AttributeError: 'NoneType' object has no attribute 'headString'

==
ERROR: runTest (leo.core.leoTest.GeneralTestCase)
@test rST import test: no double-underlines

--
Traceback (most recent call last):
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoTest.py", 
line 211, in runTest
builtins.execfile(scriptFile, d)
  File "/home/tbrown/.leo/scriptFile.py", line 73, in 
assert root.h.startswith('@@'), root.h
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 1635, in __get_h
return p.headString()
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 672, in headString
return self.v.headString()
AttributeError: 'NoneType' object has no attribute 'headString'

==
ERROR: runTest (leo.core.leoTest.GeneralTestCase)
@test rST import test: long underlines

--
Traceback (most recent call last):
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoTest.py", 
line 211, in runTest
builtins.execfile(scriptFile, d)
  File "/home/tbrown/.leo/scriptFile.py", line 34, in 
assert root.h.startswith('@@'), root.h
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 1635, in __get_h
return p.headString()
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 672, in headString
return self.v.headString()
AttributeError: 'NoneType' object has no attribute 'headString'

==
ERROR: runTest (leo.core.leoTest.GeneralTestCase)
@test rST import test: long overlines

--
Traceback (most recent call last):
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoTest.py", 
line 211, in runTest
builtins.execfile(scriptFile, d)
  File "/home/tbrown/.leo/scriptFile.py", line 35, in 
assert root.h.startswith('@@'), root.h
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 1635, in __get_h
return p.headString()
  File 
"/mnt/usr1/usr1/home/tbrown/t/Package/leo/git/leo-editor/leo/core/leoNodes.py", 
line 672, in 

Re: Recovered nodes and empty lines difference

[vitalije]

Thanks.

On Wednesday, June 28, 2017 at 4:51:56 PM UTC+2, Edward K. Ream wrote:
>
> On Wednesday, June 28, 2017 at 8:42:46 AM UTC-5, vitalije wrote:
>>
>> I would prefer if Leo wouldn't report nodes that differ in any number of 
>> empty lines.
>>
>
> Rev d1608d of master should fix this.  Let me know if it doesn't.
>
> Edward
>

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[vitalije]

Someone asked how this proposition can help users. I wasn't clear about 
that. It won't help that much unless reloading is also addressed. But there 
are some settings that aren't so obvious when represented as a text. 
Colors, shortcuts for example. Now it is required that a user writes them 
as a text and there is possibility of spelling mistakes. Add to it need to 
open another Leo file, edit settings, save it, close it and then reopen 
your document again to accept new changes. Then you realize that the color 
is not so pleasing and you have to go again through the whole process. Very 
soon I gave up adjusting colors. I think that settings like colors, fonts, 
shortcuts and similar should not be edited as a text but as something that 
user can see and feel without understanding any particular syntax. They 
certainly can be kept as a text, but user should be allowed to edit them in 
more natural way.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[vitalije]

>
>
> For example, yesterday I discovered a settings table that inited several 
> commander ivars.  My first thought was, maybe this table could go away.  
> But no.  These ivars appear all over Leo.  It would be horrible to use 
> c.config.getInt or c.config.getBool in their stead.  So this table creates 
> a major convenience in Leo's code, even while it complicates the startup 
> code and (eventually) will complicate the reload-settings command.  It a 
> choice, and I think the tradeoff is a good one.
>
> Edward
>

Every ivar that caches settings can be turned into property that gets its 
value from c.config.get. It wouldn't be so hard to even make a 
script that would search and find every occurrence in Leos code of such 
ivars and turned them into properties. That way Leo would have single 
source of settings data and every change in the source would be visible 
every where instantaneously. 

Last few months I have been studying ClojureScript and experimenting with 
it. Maybe last 5 years or so, my programming was mostly oriented towards 
client scripting in web browsers. Until recently I was using almost 
exclusively  CoffeeScript for all programming tasks. By transferring to 
ClojureScript I have discovered some very cool projects that I could not 
imagine were possible at all. One of the coolest things is writing 
reloadable code. I am amazed how cool it is to make small change in your 
code and in a few milliseconds browser shows the effect of changed code. It 
is impossible to express in words the feeling. Once you try it and feel it 
for yourself you can never tolerate again any tool that requires you to 
reload or to wait more than a second for the effect of your action to be 
visible. Maybe I become a bit spoiled after experiencing all that and now I 
am expecting from each tool I use to have such feature. 

For all my computer related work I use Leo as editor, as a database as 
scripting environment, for processing all sorts of files, for all types of 
scripting. And I am very grateful to Edward and whole community for all the 
work they all put in this project. I am no newbie in Leo world nor I am 
newbie in python or for that matter almost to any programming language from 
Z80 and 6502 assembly language to C, C++, Java, Scala, Groovy, Javascript 
ActionScript, ... up to Clojure and ClojureScript.

@Edward, you have mention the Aha moment when @clean file become possible. 
I had proposed that few years earlier but you didn't understand my idea. I 
don't know is it possible to find those messages on this list but I 
remember that I proposed for shadow files that were used to be kept in 
hidden folder as ordinary files that their content could be stored inside 
Leo document itself. That is precisely what @clean nodes do today. Isn't it?
Well if I were able to explain that idea better @clean feature could have 
been implemented few years earlier.
But at that time my suggestion was refused and I didn't care too much 
because I work almost exclusively alone and I have never had problems with 
synchronizing files nor with the sentinel lines. 

In Leo's site there is paragraph where Edward explains why he likes Python (
here ) 

Before using Python I never fully realized how difficult and dangerous 
memory allocation is in C++. Try doing:

aList[i:j] = list(aString)

in C. You will write about 20 lines of C code. Any error in this code will 
create a memory allocation crash or leak.

Now I am trying to say the same thing about settings code. All settings 
could be present in just about two lines of code: sqlite3.connect(...) and 
conn.execute('select ...').fetchone(). Surely, Leo has proven that it is 
possible to keep settings in the same file format in which Leo documents 
are kept. But it was achieved by writing a lot lines of code and that means 
lot of possible places for bugs to hide. It would not make a memory leak or 
hard crash, but nevertheless it can cause a lot of frustration to the user 
(especially to spoiled one) . What is more important it can prevent (and if 
I may say so it does prevent) achieving such a cool feature as 
reloadability. Who knows what other features could be discovered once the 
reloadability is achieved. 

Present scheme for keeping settings allows hierarchical management. Well it 
does. But honestly speaking how big is that benefit? I for example don't 
remember when I touched myLeoSettings.leo file last time. But I do remember 
that it was painful every time I did. Being able to see my settings 
in hierarchical view isn't that useful if you must read a ton of 
documentation to figure out what is the proper way to set something. If you 
are lucky and everything work as advertised you have to open settings file 
make change and then reopen all your documents. But if it doesn't work you 
have to go through the documentation to figure where you went wrong and why 
your new setting is not accepted. If you are very 

I've stoped *saying* all tests pass in commit messages

[Edward K. Ream]

But in fact, I run all unit tests before virtually every commit.  All other 
devs should too.

And periodically I also run pylint -a

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: Recovered nodes and empty lines difference

[Edward K. Ream]

On Wednesday, June 28, 2017 at 8:42:46 AM UTC-5, vitalije wrote:
>
> I would prefer if Leo wouldn't report nodes that differ in any number of 
> empty lines.
>

Rev d1608d of master should fix this.  Let me know if it doesn't.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: Recovered nodes and empty lines difference

[Edward K. Ream]

On Wed, Jun 28, 2017 at 8:42 AM, vitalije  wrote:

> The other day I think I have read somewhere in Leo's code that it is
> supposed not to report in recovered nodes those nodes that differ in just
> one empty line. In my tool collection something is regularly causing Leo to
> report a lot of recovered nodes that differ not in one empty line but in
> two successive empty lines at the end of node. I am not sure what is
> causing those changes but I would prefer if Leo wouldn't report nodes that
> differ in any number of empty lines.
>

​This should be easily fixed.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Recovered nodes and empty lines difference

[vitalije]

The other day I think I have read somewhere in Leo's code that it is 
supposed not to report in recovered nodes those nodes that differ in just 
one empty line. In my tool collection something is regularly causing Leo to 
report a lot of recovered nodes that differ not in one empty line but in 
two successive empty lines at the end of node. I am not sure what is 
causing those changes but I would prefer if Leo wouldn't report nodes that 
differ in any number of empty lines.
Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

Re: How good Leo handles settings?

[Edward K. Ream]

On Tuesday, June 27, 2017 at 12:24:24 PM UTC-5, Terry Brown wrote:

> On Tue, 27 Jun 2017 11:27:27 -0500 
> "Edward K. Ream"  wrote: 
>
 

> > ​How does that help new users?​ 
>
> That's the problem with this proposal, it's so overloaded with benefits 
> it's hard to keep them categorized ;-)  


> This doesn't help new users directly, it just simplifies Leo's code 
> base and init.  It does make it easier to write a @settings GUI 
> that's simpler for new users, but that's a separate benefit. 
>

I was up in the middle of the night thinking about this, and afterwords had 
a vivid dream about leadership.

In this post, I want to make sure that you, Terry, understand how and why I 
make the choices I do.

As project leader, I have several goals that are in tension:

- to encourage newbies to dive in and try new things.
- to encourage experimentation, improvement and simplification where 
possible.
- to keep Leo's user interface and scripting api as stable as possible.

But I do not want my tolerance for ideas to imply that I am willing to 
sacrifice my fundamental priorities for Leo:

1. Clarity and simplicity of *purpose*.
2. Stability and simplicity of *user interface*, including gui and 
scripting api.
3. Simplicity of *code*.

Sometimes, rarely, simplicity of purpose and code go together.  The 
Mulder-Ream update algorithm is the most important example.  Simplified 
code was an essential part of doing @clean.

Much more commonly, simplicity of purpose results in difficult, complex 
code behind the scenes. In that case, the *only *proper course is to *prefer 
clarity and simplicity of purpose over code simplicity*. It is naive to 
think that code complexity is a symptom that something is wrong.  I am 
willing to endure this misunderstanding among newbies, but I want you, 
Terry, to understand that code complexity is *not *a reliable guide to 
evaluating proposals.

In this discussion, the first two priorities demand that Leo continue to 
use outlines to represent the hierarchy of settings. Yes, the code is 
complex. However, it mirrors inherent, *unexpected-but-unavoidable*, 
behind-the-scenes complexity. This is *not* going to change, even if, 
heaven forbid, someone decides to rewrite the startup code.

This a "teachable moment" relating to other topics:

1. From very early days, simplicity of purpose demanded that Leo's core be 
independent of gui. This creates a small amount of complexity in the core 
itself, and more code complexity in wrapper classes of various kinds.  

*Without this code complexity it would have not been possible to create the 
npyscreen gui.*Early on, I often muttered to myself about the npyscreen 
code. The phrase that kept running through my head was 'a twisty little 
maze of rooms, all alike'.  But I didn't waste *any *time complaining to 
the npyscreen people about their code. The npyscreen folks are *never *going 
to change their code. *They can't*, because it would break their users' 
existing applications. Creators of npyscreen apps must suck up the 
complexity and carry on.

2. Many years ago now, Leo moved to the "one node world" in which all data 
was held in vnodes, rather than a combination of (extinct) tnodes and 
vnodes.  An essential part of the project was a transition plan so that 
existing Leonine scripts would continue to work.  That transition went so 
well that only a few remnants of the old world exist.

A major Aha allowed the *same *vnode to be expanded/contracted *independently 
*in different positions of the outline. New, complex, code created the 
valuable *illusion* that cloned (v)nodes have independent expansion bits. 
Similarly, root @ nodes contain complex data that create the 
*illusion* that (v)nodes may be marked independently.

3. Leo's settings-handling code is remarkably complex, and sometimes buggy. 
But it would be a huge mistake to think that this complexity is for 
nothing, and that somehow newbies' first ideas about settings can be taken 
at face value.  I can not allow you, Terry, to make this mistake.

*Summary*

There is no Elysian realm in which all code complexities disappear. Simple 
user interface is an *illusion*, often created by very complex code. Using 
cff, git, pylint and pyflakes, devs can *discover* the complex, surprising, 
and unavoidable reasons why code is as it is.

For example, yesterday I discovered a settings table that inited several 
commander ivars.  My first thought was, maybe this table could go away.  
But no.  These ivars appear all over Leo.  It would be horrible to use 
c.config.getInt or c.config.getBool in their stead.  So this table creates 
a major convenience in Leo's code, even while it complicates the startup 
code and (eventually) will complicate the reload-settings command.  It a 
choice, and I think the tradeoff is a good one.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop 

Rev 609f3aa in master binds find-quick-selected to Ctrl-Shift-F

[Edward K. Ream]

This was the binding (created in quicksearch.py) in effect before a recent 
change. My apologies for the trouble this has caused.  I did not do proper 
testing after disabling the binding.

Eventually, leoSettings.leo will contain this default setting, but #510: 
bindings to commands... 
must be fixed first. 
That work is happening in the "settings" branch.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To post to this group, send email to leo-editor@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.