Hi Florian,
I find this plugin very useful, in fact I think it would make sense for this to
be in pytest-core given that `--lf` already exists.
What if the --lf option can optionally accept a file name, which when given,
will read the set of failed tests from that file instead of reading it from the
cache. This way you can just save the `.pytest_cache/lastfailed` file from CI,
and reuse it locally:
pytest --lf=lastfailed-file-from-ci
Seems like the changes would be minimal for that to happen.
I know this is not exactly the feedback you wanted, but thought I would throw
the idea in there for your consideration.
Cheers,
Bruno
On Mon, Jul 15, 2024, at 2:29 AM, Florian Schulze wrote:
> Hi!
>
> I wrote https://github.com/fschulze/pytest-failed and would like to get some
> feedback on a few things before I publish it.
>
> What is it?
>
> The plugin adds ``--dump-lf`` and ``--restore-lf`` options.
>
> My use case is two fold for test suites with runtimes of several minutes:
>
> 1. when tests fail in CI, I can copy the summary from the report and paste it
> into a file like ``failed.txt`` then re-run just those tests with
> ``--restore-lf=failed.txt``
>
> 2. when fixing a bug I often fix only some of tests at the first attempt,
> then refactor the fix and want to re-run all initially failing tests. With
> this plugin I only run the full test suite once, then run with
> ``--dump-lf=failed.txt`` and can restore the initial bunch of tests with
> ``--restore-lf=failed.txt`` whenever I like, without ever re-running the full
> test suite.
>
> This is also super useful with ``git bisect`` and likely many other use-cases.
>
> The ``=`` in the options seems to be required, as otherwise the filename
> might be interpreted as ``[file_or_dir]`` by pytest in some forms of
> invokation.
>
> My questions:
>
> In
> [``pytest_cmdline_main``](https://github.com/fschulze/pytest-failed/blob/main/src/pytest_failed/plugin.py#L44)
> I use ``config._do_configure``, which is internal API. Is there a better way?
>
> Anyone have suggestions for better names of the plugin itself and its
> options? I think ``--dump-lf`` is ok, but I don't really like
> ``--restore-lf``.
>
> When reading the list of test names it knows about the pytest report summary
> syntax, are there any other formats that would be useful to support for
> cut'n'paste from CI?
>
> If you try the plugin, please give feedback on whether you found it useful
> and understand how it works.
>
> Regards,
> Florian Schulze
> _______________________________________________
> pytest-dev mailing list
> pytest-dev@python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
>
_______________________________________________
pytest-dev mailing list
pytest-dev@python.org
https://mail.python.org/mailman/listinfo/pytest-dev
