[issue15629] Add to regrtest the ability to run Lib and Doc doctests

STINNER Victor added the comment: The issue has been fixed by bpo-35240. commit a22df4896f6b83c8741203118790ae281716bca5 Author: Victor Stinner Date: Wed Nov 28 10:24:08 2018 +0100 bpo-35240: Add "doctest" job to Travis CI (GH-10753) Create a new "doctest" job in Travis CI to

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Change by Mark Lawrence : -- nosy: -BreamoreBoy ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Mark Lawrence added the comment: It appears, especially from the later messages, that this is not wanted. However noting msg170616 should we close this with #10967 to supersede it? -- nosy: +BreamoreBoy versions: +Python 3.5 -Python 3.4 ___ Python

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Ezio Melotti added the comment: We discussed about this on IRC, and I'll try to summarize the outcomes. If we want devs to run doctests, the syntax should be as simple as possible, ideally there shouldn't be any extra syntax -- at least for the common case. Note that there are two kind of

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Antoine Pitrou added the comment: I think I'm -1 to run doctests as part of regrtest. Documentation examples are meant to be read, not executed: it is fine for them to omit unimportant information or trivial boilerplate (such as imports). Any complication in the way doc examples must be

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Ezio Melotti added the comment: Documentation examples are meant to be read, not executed I agree, but several of them already run, and others run with the addition of doctests directives (that are hidden in the html output). If the code needs to be changed in a way that is less readable,

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

R. David Murray added the comment: I've given this some more thought, and I'm leaning toward Antoine's POV here. The point of running the doctests in the docs is not to test python, but to test the docs. Sphinx has a facility to do that: make doctest. So I think it is better to use that

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Antoine Pitrou added the comment: I think we should be moving *away* from having special infrastructure in regrtest. As much stuff as makes sense should be moved to unittest, and we've been slowly doing that. Correspondingly, we should use Sphinx's native test facilities, not add special

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Éric Araujo added the comment: RDM’s arguments make a lot of sense. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15629 ___ ___

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

R. David Murray added the comment: We should improve the unittest help, then :) I will also note that the couple of times I tried it I couldn't figure out how to use the regrtest test selection :) What we probably need for regrtest is a way to pass through a selection string from the

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Brett Cannon added the comment: I should mention that http://bugs.python.org/issue10967 exists as a meta issue about trying to gut regrtest in favour of using other things in unittest (and doctest in this case) when possible. -- ___ Python tracker

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Changes by Aaron Iles aaron.i...@gmail.com: -- nosy: +aliles ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15629 ___ ___ Python-bugs-list mailing

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Changes by Ezio Melotti ezio.melo...@gmail.com: Added file: http://bugs.python.org/file27206/issue15629-3.2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15629 ___

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Chris Jerdonek added the comment: Attached is a file of doctest statistics (counts of example failures, exceptions, and successes) for almost every file in the Doc directory when running the doctests with vanilla doctest. I did this to get a sense of which files it would be easiest to get

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

R. David Murray added the comment: turtle uses it because that was the file that I made work when I was playing with 'make doctest'. I think being able to use the testsetup directive would be good. It could also them be used (I think!) to put resource directives in the docs that would

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Chris Jerdonek added the comment: I am attaching an updated version of the patch for feedback. The latest version of the patch I developed and used in the real-world example of fixing both the module and documentation file doctests for the ipaddress module (and its HOWTO) in issue 15888. The

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Chris Jerdonek added the comment: You should also see if there are more doctest-safe or more doctest-unsafe tests, and mark the ones in the smallest group. Running the doctests for all .rst files in the Doc/ directory using the latest patch gave me the following: 299 tests [files] OK. 126

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Chris Jerdonek added the comment: Integrating this with regrtest requires some work, for example you would have to count successes/failures, add skips for certain tests and resources to control what tests should be run (there's currently a turtle going around on my screen when I run `make

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Chris Jerdonek added the comment: Sphinx does have a way to mark doctest snippets as run this, don't run this. I believe that requires using 'make doctest' as the runner, The doctest module supports this natively (i.e. without the need for Sphinx) via its skip directive (#doctest: +SKIP):

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Chris Jerdonek added the comment: The reason is that Sphinx has extra facilities that allow docttests to work without having to have boilerplate code in places where doing so would disrupt the narrative flow. Yes, this seems to be true. Sphinx has a testsetup directive which lets you

[issue15629] Add to regrtest the ability to run Lib and Doc doctests

Chris Jerdonek added the comment: Attached is a patch that adds to regrtest the ability to run doctests in both library modules and documentation files (specifically, in the non-test modules in Lib/ and in the *.rst files in Doc/). The syntax to run all doctests is-- python -m test