On Mon, Jul 15, 2024, at 9:53 AM, Florian Schulze wrote: > On 15 Jul 2024, at 14:03, Bruno Oliveira wrote: > > > Hi Florian, > > > > I find this plugin very useful, in fact I think it would make sense for > > this to be in pytest-core given that `--lf` already exists. > > I wouldn't mind having it in core. I just wanted people to easily try it > first and maybe improve it before inclusion. Like pytest-warnings was done > initially. Using ``config._do_configure`` would also be a non-issue if it is > in core. > > > What if the --lf option can optionally accept a file name, which when > > given, will read the set of failed tests from that file instead of reading > > it from the cache. This way you can just save the > > `.pytest_cache/lastfailed` file from CI, and reuse it locally: > > > > pytest --lf=lastfailed-file-from-ci > > > > Seems like the changes would be minimal for that to happen. > > Adding a json parser for that file would be trivial and it would also remove > my gripe with the ``--restore-lf`` name. I don't think it is possible or wise > to overwrite ``--lf`` from the plugin though.
Oh I would document that file to be "opaque" for users (meaning we can change it at will, including its format), so users are expected to just copy it from one location to another. > > Also changing ``--lf`` to have an optional filename might cause problems with > invocation if file paths are involved. Could be a backward compatibility > issue. Good point. > > How would one access ``.pytest_cache/lastfailed`` from GitHub Actions for > example? Just copy'n'paste from the log output seems more intuitive for most > people, so the parsing should be kept. > > I'm not sure how user-friendly it is to explain how to copy > ``.pytest_cache/lastfailed``, so I think ``--dump-lf`` would still be useful. Seems just a matter of documenting to use the `upload-artifact` action to upload the file unchanged, without users needing to worry about the contents of the file. > > > I know this is not exactly the feedback you wanted, but thought I would > > throw the idea in there for your consideration. > > Totally valid feedback. Thanks! > > Regards, > Florian > > > > > Cheers, > > Bruno > > > > On Mon, Jul 15, 2024, at 2:29 AM, Florian Schulze wrote: > >> Hi! > >> > >> I wrote https://github.com/fschulze/pytest-failed and would like to get > >> some feedback on a few things before I publish it. > >> > >> What is it? > >> > >> The plugin adds ``--dump-lf`` and ``--restore-lf`` options. > >> > >> My use case is two fold for test suites with runtimes of several minutes: > >> > >> 1. when tests fail in CI, I can copy the summary from the report and paste > >> it into a file like ``failed.txt`` then re-run just those tests with > >> ``--restore-lf=failed.txt`` > >> > >> 2. when fixing a bug I often fix only some of tests at the first attempt, > >> then refactor the fix and want to re-run all initially failing tests. With > >> this plugin I only run the full test suite once, then run with > >> ``--dump-lf=failed.txt`` and can restore the initial bunch of tests with > >> ``--restore-lf=failed.txt`` whenever I like, without ever re-running the > >> full test suite. > >> > >> This is also super useful with ``git bisect`` and likely many other > >> use-cases. > >> > >> The ``=`` in the options seems to be required, as otherwise the filename > >> might be interpreted as ``[file_or_dir]`` by pytest in some forms of > >> invokation. > >> > >> My questions: > >> > >> In > >> [``pytest_cmdline_main``](https://github.com/fschulze/pytest-failed/blob/main/src/pytest_failed/plugin.py#L44) > >> I use ``config._do_configure``, which is internal API. Is there a better > >> way? > >> > >> Anyone have suggestions for better names of the plugin itself and its > >> options? I think ``--dump-lf`` is ok, but I don't really like > >> ``--restore-lf``. > >> > >> When reading the list of test names it knows about the pytest report > >> summary syntax, are there any other formats that would be useful to > >> support for cut'n'paste from CI? > >> > >> If you try the plugin, please give feedback on whether you found it useful > >> and understand how it works. > >> > >> Regards, > >> Florian Schulze > >> _______________________________________________ > >> pytest-dev mailing list > >> pytest-dev@python.org > >> https://mail.python.org/mailman/listinfo/pytest-dev > >> >
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
