potiuk commented on PR #33355:
URL: https://github.com/apache/airflow/pull/33355#issuecomment-1679615177

   > I propose I could add a note to the docs.
   
   Yes. Note in the docs should be sufficient (avoid modifying the venv after 
it is created or similar ) . checking venv context is not a good idea (not only 
because of expensiveness of it but also things like .pycaching and potential 
unintended small but harmless modifications
   
   I also think we should have a way to clean such broken venvs. You know 
"cache invalidation".  Currently we have no easy/documented way of doing it.
   
    It does not have to be sophisticated, but one thing that works well is - 
rather easy - adding a separate unique ID/prefix to such venv so that you can 
effectively "clean-it" by changing the prefix. This is how it is done in GitHub 
Actions for example. There you can specify "key" alongside cache definition - 
and it used to determine if this "the same" or different cache to be used.
   
   In our case you could simply change "path" where the venv is created, but 
maybe also adding "cache-key" (as sub-folder in the path) might make sense? 
This way you could store (and sync?) all the cache files by using the same 
PATH, but then have a key that you could change in case you would like to 
invalidate this particular venv you use? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to