If you are really just logging, use separate loggers/files, sync the clocks, and timestamp records, then merge when you are ready to process if you need timeline related results.
On Wed, Mar 14, 2012 at 1:29 PM, Preston Landers <pland...@gmail.com> wrote: > On Wed, Mar 14, 2012 at 2:55 PM, Tony Cappellini <cappy2...@gmail.com> wrote: >> >>> >>in which objects can truly move from one process to another is >>> >>recreating them in the other process. Even fork() makes copies of >>> >> everything. >> >> Recreating an object in another process means it's a different object, not a >> shared one. > > Yeah, I know. I was trying to make that point. There's no real way > for the same object to exist in multiple processes other than SYSV > shared mem. > > Truly shared memory (i.e, SYSV style) is tricky, not very portable, > and usually the wrong answer in my experience. As fas as I know stock > Python doesn't support that, and definitely never will on Windows. > > The point is that you need to figure out what problem you're really > trying to solve (logging to one file from multiple processes, it > sounds like) and then find the best / simplest approach, which I can > tell you definitely doesn't involve SYSV shared memory. It's > probably just creating separate logging objects in each process, > pointing to the same file, and protected by file locking if necessary. > > >>> >>Have you tried pickle or other techniques of serialization? Not sure >>> >>offhand if the logger module supports pickle but it might. >> >> Yes. I've just tried this, even though I expected it not to work. >> >> If process A pickles a logger object, and process B unpickles it, >> referencing of an object in a different process is meaningless. >> >> In my case, when the process attempted to write to the logger, no entries >> were seen in the logfile. Surprisingly, no exceptions occurred- but this >> could just be a coincidence. > > Probably because the logger object, when serialized, saves a reference > to an open filehandle, which won't be automatically transfered to the > other process. > > (There might ultimately be a way to make that work by inheriting > filehandles, but again, if you can find something simpler...) > > >> >> That may work, and with less effort than my original idea. >> >> But if two processes write to the logfile at the same time (especially on a >> multicore machine), >> hard-to-read logfiles may result. >> it's worth a try. > > Yes, that type of thing can occur, but you can also get around that > with simple file locking. By the way, that same problem certainly > exists even if you somehow shared the object between two processes - > what if the two processes made a log call at the same time? > > File locking may introduce some performance issues if the logging is > very frequent, but usually you can find ways to mitigate that. > > The main app I work on uses log files extensively, and the same file > is appended to by unrelated processes without a locking mechanism. > Occasionally you do see some interleaving of log entries but in my > experience it's fairly rare and in my particular case we don't care > much about that anyway. > > regards, > Preston > _______________________________________________ > python-win32 mailing list > python-win32@python.org > http://mail.python.org/mailman/listinfo/python-win32 _______________________________________________ python-win32 mailing list python-win32@python.org http://mail.python.org/mailman/listinfo/python-win32