potiuk commented on PR #33355:
URL: https://github.com/apache/airflow/pull/33355#issuecomment-1679615177
> I propose I could add a note to the docs.
Yes. Note in the docs should be sufficient (avoid modifying the venv after
it is created or similar ) . checking venv context is not a good idea (not only
because of expensiveness of it but also things like .pycaching and potential
unintended small but harmless modifications
I also think we should have a way to clean such broken venvs. You know
"cache invalidation". Currently we have no easy/documented way of doing it.
It does not have to be sophisticated, but one thing that works well is -
rather easy - adding a separate unique ID/prefix to such venv so that you can
effectively "clean-it" by changing the prefix. This is how it is done in GitHub
Actions for example. There you can specify "key" alongside cache definition -
and it used to determine if this "the same" or different cache to be used.
In our case you could simply change "path" where the venv is created, but
maybe also adding "cache-key" (as sub-folder in the path) might make sense?
This way you could store (and sync?) all the cache files by using the same
PATH, but then have a key that you could change in case you would like to
invalidate this particular venv you use?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]