On 19/04/2021 01.46, Jason Friedman wrote: > I should state at the start that I have a solution to my problem. I am > writing to see if there is a better solution. > > I have a program that runs via crontab every five minutes. It polls a > Box.com folder for files and, if any are found, it copies them locally and > performs a computation on them that can exceed five minutes. It pushes the > results back up to Box. (Box.com ensures that only complete files are > visible when I poll.) Files are dropped into this Box.com folder rarely, > but to ensure a good customer experience I do not want to set my crontab to > run less frequently. My hardware cannot support multiple simultaneous > computations. > > I have written a piece of code to detect if more than 1 instance of my > program is running, and I put this code into a separate module (support.py) > so that other programs can use it.
In a similar situation, one of my teams used an (OpSys) environment variable (available in both *nux and MS-Win). - when the application starts, it checks for the variable - if exists, stops running, else may proceed During code review (when I noticed this tactic) I was slightly surprised, because back when I was young (men were men, and knights were bold, ...), we used file-stubs. However, such systems face two complementary, potential-problems: 'single-instance' (which is being addressed), and 'blocking-instance'. If there is a risk that the long-running computations may fail into a never-ending loop, the system effectively dies (but silently!) and source files (ie at Box.com) may accumulate without receiving attention. Accordingly, the above-mentioned environment-variable was filled with a time-stamp. Then a second step in the check-routine reviewed the time since the 'blocking' instance started, in order to log or raise suitable alerts if things went awry. YMMV! An alternative, if the system already uses a database, is to keep a local record in the DB of all the files lodged at box.com. This can include a note that each file has/not been processed (plus any other stats or logging you may deem appropriate). A third state would be 'in process'. Now, at start-up, the application can quickly check to see if there is any file in that state... -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list