On 03/24/2016 09:17 AM, Chris Nulk wrote: > > I did a little searching around our lists. It seems the attachments > directory is always created and populated whether or not scrub_nondigest > was ever set to Yes. So I can't really use the existence of the > attachments directory or the lack of it being populated as a check for > if scrub_nondigest was ever set to Yes.
No, but here's what you can look for. If scrub_nondigest is True, the message will be scrubbed during processing before any delivery and before the message is added to the cumulative mbox. Thus there will be one copy of each scrubbed attachment in the attachments directory and a link in the message body and the scrubbed attachment will not be in the cumulative mbox. If scrub_nondigest is False, archiving will scrub the message and any scrubbed attachments will be in the attachments directory and replaced by links in the archived message, and the attachment will be in the cumulative mbox. Additionally, if the list is digestable, digest processing will also scrub the message for the plain text digest even if there are no plain digest subscribers or even no MIME digest subscribers either. This will create second copies of each attachment in the attachments directory, but not until the digest is actually created. > Is the anything that can be checked for that would indicate > scrub_nondigest was Yes at any point in the past? Obviously, if it is > currently Yes it doesn't matter. Not 100% reliably, but if there are periods of time when the attachments (sub-)directories contain two copies of each attachment, you know scrub_nondigest was No at that point, and the absence of any duplicates may say scrub_nondigest was Yes at that point or it may say that this archive was imported or previously processed with bin/arch --wipe which will not replace the duplicate attachments. > Lacking the ability to check, I suppose a warning should be generated > prior to running the bin/arch. In addition, the '-b/--rebuild_archives' > option should default to false unless there is a positive indication the > user wants to run it ('-b y / --rebuild_archives=Yes'). The default for > the rebuild_archives option would be No. > > If there is any way to tell if scrub_nondigest has been set to Yes in > the list's past, please let me know. I am going to add the warning and > try making the changes to the rebuild_archives options. I will post the > diff from the Mark's original clone_list. It just occurred to me that there is a way. Check the lists cumulative mbox for messages which are single part text/plain and have content somewhere in the body matching the regexp '\nAn? .* was scrubbed\.\.\.\n' This would indicate the message was scrubbed before being archived (or possibly is a message quoting something from a plain text digest, there's always a gotcha) and would probably mean that at the time of that message, scrub_nondigest was Yes. It's actually OK to run bin/arch --wipe if scrub_nondigest is Yes. The two 'bad' things that happen here are if scrub_nondigest was No in the past, there will be extra, unreferenced copies of attachments, and for the periods when scrub_nondigest was Yes, the links to scrubbed attachments will still have the wrong listname. The loss of data problem is if scrub_nondigest is No and was Yes in the past, the attachments that were scrubbed when it was Yes will be lost. -- Mark Sapiro <m...@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org