Re: #1467: Use traditional unit test (almost?) everywhere

[Terry Brown]

On Sat, 28 Dec 2019 02:17:06 -0800 (PST)
"Edward K. Ream"  wrote:

> Bob's remarks yesterday about unit testing had me look at Leo's unit 
> testing framework with new eyes. 
> 
> #1467  suggest
> using traditional unit tests everywhere, or maybe almost everywhere.

Sounds like a good idea.  I suspect pytest https://docs.pytest.org/
is used by major projects more routinely than Python's "built-in" library, so I
would evaluate that before making the switch.

Cheers -Terry

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/20191228123737.5a80d28b%40lakeview.

Leo requires tkinter module to be configured in Python environment

[Viktor Ransmayr]

Hello Edward, hello Matt,

I just noticed something interesting/ strange/ weird.

Actually two issues:

First: Leo requires that the tkinter module is configured in the Python 
installation.

(Backgound: I just upgraded to Python 3.7.6 - and - decided to de-select 
everything but pip ...)

Second: I noticed this only, when I started w/o an existing '~/.leo' 
directory.

The traceback I receive is:

(Leo-devel) PS C:\Users\Viktor\pyve\github\Leo-devel> leo-m

--global-docks: False

Traceback (most recent call last):
  File "C:\Users\Viktor\PyVE\GitHub\Leo-devel\Scripts\leo-m-script.py", 
line 11, in 
load_entry_point('leo', 'console_scripts', 'leo-m')()
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\runLeo.py",
 
line 73, in run
g.app.loadManager.load(fileName, pymacs)
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoApp.py",
 
line 2512, in load
lm.doPrePluginsInit(fileName, pymacs)
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoApp.py",
 
line 2691, in doPrePluginsInit
lm.initApp(verbose)
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoApp.py",
 
line 2914, in initApp
g.app.setLeoID(verbose=verbose)
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoApp.py",
 
line 1157, in setLeoID
self.setIdFromDialog()
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoApp.py",
 
line 1244, in setIdFromDialog
dialog = g.TkIDDialog()
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoGlobals.py",
 
line 526, in __init__
super().__init__(self.title, self.message)
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoGlobals.py",
 
line 446, in __init__
self.createTopFrame()
  File 
"c:\users\viktor\pyve\github\leo-devel\devel\leo-editor-devel\leo\core\leoGlobals.py",
 
line 538, in createTopFrame
import tkinter as Tk
ModuleNotFoundError: No module named 'tkinter'
(Leo-devel) PS C:\Users\Viktor\pyve\github\Leo-devel>

As soon as I re-use my previously saved '~/.leo' directory, the 
ModuleFoundError disappears ...

Can you please explain 

* why the module 'tkinter' is required, although PyQt is used as the GUI 
framework - and - 

* secondly, why the problem goes away, if I'm using "old/ saved" '~/.leo' 
content ...

With kind regards,

Viktor

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/8a779c7d-b3df-442a-addf-428adb773a79%40googlegroups.com.

Re: Progress on ViewRendered2

[Edward K. Ream]

On Fri, Dec 27, 2019 at 6:35 PM Thomas Passin  wrote:

> I have made a lot of progress on re-implementing *Viewrendered2 *in the
> QT5 era, and I wanted to share where I am with it.
>

Many thanks for this work!  Others have expressed interest.


> So it's been painful, but progress is good.
>

Excellent.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAMF8tS0dFQwjHSDOxM7fOkfENOzfNBO%2BWMQK3qo2i%2B2Cq3B1NQ%40mail.gmail.com.

Re: #1467: Use traditional unit test (almost?) everywhere

[Brian Theado]

I've been experimenting lately writing pytest tests for leo. I just
published my work at
https://github.com/btheado/leo-editor/tree/pytest-experiment.

You should be able try it out with these commands (untested):

git origin add btheado https://github.com/btheado/leo-editor.git
git checkout btheado pytest-experiment
pip install pytest
pytest leo/test/pytest


The tests I wrote are in leo/test/pytest.leo

The first set of tests I wrote are for testing the leoNode.py file. I was
interested in seeing what it was like to try to get full code coverage by
the tests. There is a project called coverage.py (
https://coverage.readthedocs.io) which can produce nice reports about which
lines of code have been executed and which have not. I tried using this
coverage.py against leo's current unit tests. Somehow coverage.py was not
able to properly mark all the executed code. I suspect something in Leo's
unit tests caused coverage.py to lose track but I'm not sure. I planned on
doing binary search through the unit tests (enable/disable) until I found
which ones caused coverage to lose its way. I never got around to doing
that and instead used the pytest coverage.py plugin on some unit tests I
wrote.

My leo/test/pytest/leoNodes_test.py file contains 24 tests. I picked some
methods of the Position class in leoNodes.py and strove to get full
coverage on them. With 24 tests, I did fully cover them, but it is only a
small percentage of the whole file. You can see the coverage achieved by
running these commands:

pip install pytest-cov
pytest --cov-report html --cov-report term-missing --cov=leo.core.leoNodes
leo/test/pytest/leoNodes_test.py
firefox htmlcov/leo_core_leoNodes_py.html  ;# Or whatever your web browser
is


The resulting web page highlights all the lines of code which haven't been
executed in red. From the tests I wrote, you should see 100% coverage for
several of the Position methods including the comparison
operators, convertTreeToString, moreHead,
moreBody, children, following_siblings, nearest_roots,
and nearest_unique_roots.

In the process of writing those tests, I found what I think is a bug and I
discovered the very nice xfail (
https://docs.pytest.org/en/latest/skipping.html#xfail) feature of pytest.
With it you can mark test that you expect to fail with the xfail decorator.
This will suppress all the verbose information pytest gives when there is a
failure (stack trace, etc), Normally, this verbose information is helpful
in tracking down the reason for failures. But if the bug is one which can't
or won't be fixed right away, all the extra information can get in the way.
So the xfail marker will suppress all the failure details unless you run
using the --runxfail command line option. That way you can keep your pytest
runs clean, but then easily access the failure details when you are ready
to fix the bug.

To see the details of the expected failures I've identified as bugs (2 of
them so far), run it like this:

pytest leo/test/pytest --runxfail


I'm a big fan of writing tests which fail before a bug is fixed and pass
after a bug is fixed. The xfail feature is very helpful in this regard.

Pytest also has the feature of fixtures (
https://docs.pytest.org/en/latest/fixture.html) which I've made use of. It
allows all tests to be written as simple functions. No classes needed.
Anything which normally would be done in a setup method can be implemented
as a fixture instead. It seems there are some tradeoffs here, but overall I
like it a lot.

Fixtures can be defined in any test file, but if they are common to
multiple test files, they can be defined in a file named conftest.py. This
makes the fixtures available to all the files in the directory. I've
defined my fixtures there.

My fixtures include a bridge fixture which can be used to access the leo
bridge. I also have fixtures for several example outlines. I didn't like
that the fixtures ended up "distant" from the tests themselves, so I came
up with a naming convention for the example outlines which allows you to
know the exact structure and contents of the outline just by looking at the
name of the outline. I tried to explain this naming convention in
conftest.py, but I'm not sure if it will be clear to anyone other than
myself.

Using pytest provides a lot of benefits:

   - Information provided about failures is excellent
   - marking expected failures with xfail is very useful for documenting
   bugs before they are fixed
   - fixtures allow all tests to be written as simple functions
   - coverage plugin allows code coverage to be measured
   - there are many, many other plugins

I'm not suggesting leo switch to using pytest. With this work I've shared,
I hope it is easy for those familiar with leo unit tests to be able to
evaluate the nice features of pytest and decide whether it is worth further
consideration.

On Sat, Dec 28, 2019 at 8:40 AM Edward K. Ream  wrote:

>
> On Saturday, December 28, 2019 at 6:07:03 AM UTC-5, vitalije wrote:
>
> For a lo

Re: #1467: Use traditional unit test (almost?) everywhere

[Edward K. Ream]

On Saturday, December 28, 2019 at 6:07:03 AM UTC-5, vitalije wrote:

For a long time I've been feeling that Leo unit tests don't prove anything. 
> They usually don't exercise real Leo code at all or if they do, they 
> exercise just a small portion of it. So, the fact that unit tests are 
> passing doesn't mean Leo would work properly for real users.
>

This is a separate issue. As I understand it, unit tests are meant to test 
small portions of code. They can also ensure that specific bugs don't 
happen again.

It might take a huge effort to fully eliminate all `if g.unitTesting` 
> conditionals from Leo core, but it might be worth doing.
>

A second separate issue. It's not likely to happen, because tests in 
unitTest.leo test outline operations. It's natural to run those tests in a 
real outline.  Indeed, I don't see how else to run those tests.  I've just 
updated the title and first comment of #1467 to indicate that leoTest.py 
and unitTest.leo will likely remain.

*Summary*

unitTest.leo seems necessary to test outline operations. @test nodes are 
natural in that environment.

For all other work, using stand-alone test classes should be easier and 
more natural. Even in unitTest.leo, there will likely be ways of leveraging 
stand-alone test classes. I'll be investigating the possibilities...

Using traditional unit tests where possible will remove another objection 
to using Leo.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/dd029cac-6860-4f8f-a565-7a3083f16120%40googlegroups.com.

Re: Why is Script 'leo-m.exe' not available in Leo 6.1-final on PyPI?

[Viktor Ransmayr]

Hello Matt,

Am Do., 26. Dez. 2019 um 20:49 Uhr schrieb Matt Wilkie :

> Why is the script 'leo-m.exe' available in the development-/ beta version
>> of Leo (e.g. 6.1-b1-devel) but not in the master-/ final version of Leo
>> (e.g. 6.1-final) at PyPI?
>>
>
> It's because master lags behind devel. It was inadvertently left out of
> the last master (final) release, and will be restored in next one.
>

Do you plan to deliver that as part of Leo 6.1.1?

With kind regards,

Viktor

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/CAAeSrGJGU%2B6CGrJfLAZbwnQ73KfPeX0YzBmujDpZGoY%3DEUMK1Q%40mail.gmail.com.

Re: #1467: Use traditional unit test (almost?) everywhere

[vitalije]

This is a good idea. I suppose you don't plan to make a lot of changes in 
the test cases. But even without changing test cases this would be a big 
step forward.

For a long time I've been feeling that Leo unit tests don't prove anything. 
They usually don't exercise real Leo code at all or if they do, they 
exercise just a small portion of it. So, the fact that unit tests are 
passing doesn't mean Leo would work properly for real users.

It might take a huge effort to fully eliminate all `if g.unitTesting` 
conditionals from Leo core, but it might be worth doing. Anyway it would be 
much clearer to see if it is possible to eliminate those conditionals after 
transferring unit tests into external files.

Vitalije

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/56e1d31a-870d-4139-a317-7de5e061c463%40googlegroups.com.

#1467: Use traditional unit test (almost?) everywhere

[Edward K. Ream]

Bob's remarks yesterday about unit testing had me look at Leo's unit 
testing framework with new eyes. 

#1467  suggest using 
traditional unit tests everywhere, or maybe almost everywhere. Imo, this 
will be a significant improvement.  I'll use a script to do the conversion, 
so it shouldn't be a major project.

*Benefits of traditional unit tests*

Yesterday I converted (by hand) the @test nodes for leoAst.py in 
unitTest.leo to several Test classes within leoAst.py itself. There were no 
significant problems. Instead, I saw more and more advantages to 
traditional test classes and functions:

- Using classes and functions allows the test code to reside within the 
file being tested. Such code can be used by people who have not installed 
Leo.

- Common code can reside in, say, a BaseTest class. With @test nodes, such 
common code requires the ugly @common-code hack.

- Tests can be grouped by putting them in distinct Test classes, which will 
typically be subclasses of BaseTest.  The unittest module can run all tests 
within a module, or within a specific test class, or even a single test 
function:

python -m unittest test_module1 test_module2python -m unittest 
test_module.TestClasspython -m unittest test_module.TestClass.test_method

  
Such commands commands can be run within Leo.  For example, I used the 
following @button node yesterday:

import os
g.cls()
leo_editor_dir = g.os_path_finalize_join(g.app.loadDir, '..', '..')
os.chdir(leo_editor_dir)
commands = r'python -m unittest leo.core.leoAst.TestTOG'
g.execute_shell_commands(commands, trace=False)

*Grouping tests*

The big advantage (or so I thought) to @test nodes was that it is easy to 
run individual tests or group of tests.  But this advantage is an 
illusion!  Using clones, it is easy to run any desired group of test 
functions:

- Create a RecentTests test class.

- Use clones to add test functions to that class.

- Run the script in an @command run-recent-tests node, similar to that 
shown above.

*Summary*

@test nodes aren't nearly as useful as I thought they were. Traditional 
unit tests have many advantages over @test nodes.

Traditional unit tests play well with Leo. @command/button node can run 
groups of tests. Clones can be used to add test functions to test classes.

The new test classes will simplify Test Driven Development of Leo itself.

I'll use scripts to convert tests in unitTest.leo to test classes in Leo's 
core modules. unitTest.leo and (maybe) leoTest.py will likely be retired.

Edward

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/9c1089c5-53bd-4ce8-b81b-a299813c2946%40googlegroups.com.

Re: Why is Script 'leo-m.exe' not available in Leo 6.1-final on PyPI?

[Viktor Ransmayr]

Hello Thomas,

Am Samstag, 28. Dezember 2019 00:54:15 UTC+1 schrieb Thomas Passin:
>
>
>
> On Thursday, December 26, 2019 at 2:49:52 PM UTC-5, Matt Wilkie wrote:
>>
>> Why is the script 'leo-m.exe' available in the development-/ beta version 
>>> of Leo (e.g. 6.1-b1-devel) but not in the master-/ final version of Leo 
>>> (e.g. 6.1-final) at PyPI?
>>>
>>
>> It's because master lags behind devel. It was inadvertently left out of 
>> the last master (final) release, and will be restored in next one.
>>  
>>
>>> I think this script should be there at all times, since it is really 
>>> helpful when something goes wrong unexpectedly.
>>>
>>  
>> `python path\to\launchLeo.py` is the same as leo-messages.exe is always 
>> available, regardless of what does or doesn't happens with pip's install.
>>
>> -matt
>>
>
> BTW, it's easier to run these modules using python -m, since you don't 
> have to keep coming up with the full path.  That is:
>
> python -m leo.core.runLeo # or launchLeo or whatever
>
> Of course, the leo directory location has to be known to Python if it's 
> not in the standard location (e.g., Lib/site-packages).
>

Thanks for this reminder. - Yes, it does work for leo.core.runLeo - but - 
it does not work for launchLeo.py, since this file is not part of the Leo 
package when installed from PyPI.

AFAICT this is related to what Matt called "Leo's issue with (newer) Python 
Packaging Preferences" here [1].

With kind regards,

Viktor
---
[1] https://groups.google.com/d/msg/leo-editor/JxE7zwStAhA/03064iAhDgAJ

-- 
You received this message because you are subscribed to the Google Groups 
"leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to leo-editor+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/leo-editor/6c73bb24-03b3-476a-919d-27f5b342e127%40googlegroups.com.