Re: [Mailman-Developers] Patch for HyperArch

2016-03-12 Thread Stephen J. Turnbull
Mark Sapiro writes: > On 03/12/2016 08:23 AM, Stephen J. Turnbull wrote: > > Mark Sapiro writes: > > > > > The Received: header check is important. For an "imported" mbox, the > > > From_ separators may reflect when the mbox was exported from it's source > > > rather than the message date

Re: [Mailman-Developers] Patch for HyperArch

2016-03-12 Thread Mark Sapiro
On 03/12/2016 08:23 AM, Stephen J. Turnbull wrote: > Mark Sapiro writes: > > > The Received: header check is important. For an "imported" mbox, the > > From_ separators may reflect when the mbox was exported from it's source > > rather than the message date. If the messages have Received: heade

Re: [Mailman-Developers] Patch for HyperArch

2016-03-12 Thread Stephen J. Turnbull
Mark Sapiro writes: > The Received: header check is important. For an "imported" mbox, the > From_ separators may reflect when the mbox was exported from it's source > rather than the message date. If the messages have Received: headers, > the later ones at least tend to have good dates. Over

Re: [Mailman-Developers] Patch for HyperArch

2016-03-11 Thread Mark Sapiro
On 03/11/2016 10:14 AM, Sebastian Hagedorn wrote: > > As far as I'm concerned, the problem is fixed. Thank you very much for > all your help! And thank you for reporting and testing. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense

Re: [Mailman-Developers] Patch for HyperArch

2016-03-11 Thread Sebastian Hagedorn
I have fixed this and pushed rev 1634. The entire patch combining revs 1633 and 1644 is attached. Thanks again. This time it ran through, and all mails with broken date headers are archived as expected: the ones with only the "From foo@bar" line have the date of the migration from Majordomo to

Re: [Mailman-Developers] Patch for HyperArch

2016-03-11 Thread Mark Sapiro
On 03/11/2016 06:39 AM, Sebastian Hagedorn wrote: > --On 10. März 2016 um 17:00:18 -0800 Mark Sapiro wrote: > >> I have reported this bug and fixed it. The bug is >> and the fix is >>

Re: [Mailman-Developers] Patch for HyperArch

2016-03-11 Thread Sebastian Hagedorn
--On 10. März 2016 um 17:00:18 -0800 Mark Sapiro wrote: I have reported this bug and fixed it. The bug is and the fix is The fix looks at message timestamps in the followin

Re: [Mailman-Developers] Patch for HyperArch

2016-03-10 Thread Mark Sapiro
On 03/10/2016 10:10 AM, Mark Sapiro wrote: > > For the actual "fix", my inclination is to modify the _set_date method > in pipermail.py (this is called from Hyperarch.py as > self.__super_set_date(message) just before it does self.fromdate = > time.ctime(int(self.date)). > > I would have this che

Re: [Mailman-Developers] Patch for HyperArch

2016-03-10 Thread Mark Sapiro
On 03/10/2016 03:19 AM, Sebastian Hagedorn wrote: > > Unless you're really interested in the other differences you referred to > in your other message, I won't bother to analyze them further. It seems > clear to me that you have identified the main issue. I understand the issue, and I know how t

Re: [Mailman-Developers] Patch for HyperArch

2016-03-10 Thread Sebastian Hagedorn
--On 9. März 2016 um 15:15:20 -0800 Mark Sapiro wrote: We played around and found that the error is related to our version of Python. Here's a minimal test script that shows the issue: from email.Utils import parseaddr, parsedate_tz, mktime_tz, formatdate print mktime_tz(parsedate_tz("Fri, 4 F

Re: [Mailman-Developers] Patch for HyperArch

2016-03-09 Thread Mark Sapiro
> > We played around and found that the error is related to our version of > Python. Here's a minimal test script that shows the issue: > > from email.Utils import parseaddr, parsedate_tz, mktime_tz, formatdate > print mktime_tz(parsedate_tz("Fri, 4 Feb 100 00:51:42 +0100 (MET)")); > > That's th

Re: [Mailman-Developers] Patch for HyperArch

2016-03-09 Thread Mark Sapiro
On 03/09/2016 05:26 AM, Sebastian Hagedorn wrote: > > --On 8. März 2016 um 15:43:11 -0800 Mark Sapiro wrote: > >> I'm still having difficulty duplicating what you saw. >> >> For the above three messages, the first one just gets detected as an >> invalid date and archived under the current date b

Re: [Mailman-Developers] Patch for HyperArch

2016-03-09 Thread Sebastian Hagedorn
Hi Mark, thanks for looking into this. --On 8. März 2016 um 15:43:11 -0800 Mark Sapiro wrote: I'm still having difficulty duplicating what you saw. For the above three messages, the first one just gets detected as an invalid date and archived under the current date by the current bin/arch.

Re: [Mailman-Developers] Patch for HyperArch

2016-03-08 Thread Mark Sapiro
On 03/08/2016 04:42 AM, Sebastian Hagedorn wrote: > > Here are the culprits. They are now easy to find, because they are all > new in the archive :-) (which is public, btw: > ) > ... > > Then there are a few broken spam mes

Re: [Mailman-Developers] Patch for HyperArch

2016-03-08 Thread Sebastian Hagedorn
--On 7. März 2016 um 14:35:47 -0800 Mark Sapiro wrote: If an exception is caught, the date is simply set to the current time. I understand the patch, but I'm not sure if setting the current time is appropriate. In particular, the self.__super_set_date(message) method, if it doesn't find a val

Re: [Mailman-Developers] Patch for HyperArch

2016-03-07 Thread Mark Sapiro
On 03/07/2016 04:00 AM, Sebastian Hagedorn wrote: > Hi, > > we recently needed to rebuild a rather old list archive. The oldest > mails are from 2001, and as far as I could tell the last complete > rebuild happened in 2005. When we ran "arch --wipe" now, it failed: > ... > ValueError: timestamp o