Marko Rauhamaa <ma...@pacujo.net> writes: > dieter <die...@handshake.de>: >> Marko Rauhamaa <ma...@pacujo.net> writes: >>> Keeping the number of long-term objects low is key. >> >> Right, if I need near realtime behaviour and must live >> with [C]Python's garbage collector. > > Or any other GC ever invented.
There are "realtime garbage collection" algorithms. For them, the creation of new objects must cooperate with the garbage collection - likely with the need to acquire a lock for a short period. But there is no (principle) need to block all "normal" activity during the complete garbage collection (as [C]Python's garbage collector has done this at the time of my problem). >> But, a web application does usually not need near realtime behaviour. >> An occasional (maybe once in a few days) garbage collection and >> associated reduced response time is acceptable. >> A problem only arises if a badly designed component produces >> quite frequently hundreds of thousands of temporary objects >> likely triggering (frequent) garbage collections. > > But I think you are barking up the wrong tree. You could rightly blame > GC itself as an unworkable paradigm and switch to, say, C++ or Rust. I am happy that [C]Python uses mainly reference counting for its memory management and that GC is used quite sparingly. > Or you could blame the parts of the software that create too many > long-term objects. I do not do that because I understand why in my application there are many long living objects. > You shouldn't blame the parts of the software that churn out zillions of > short-term objects. I do precisely that: the blamed component produced a very large number of short living objects -- without need and while it should have been aware that it operates on mass data - among others from the fact that its complete environment took special care to work with this mass data efficiently. I solved my problem by replacing this single component by one knowing what it does. No need to rewrite the complete application, get rid of Python object caches or even switch to a different language. -- https://mail.python.org/mailman/listinfo/python-list