[Mailman-Developers] Dates again

[Steven Kuck]

I see this topic has been discussed years ago, but I recently started 
running some MailMan lists and this problem came up almost as soon as I 
started running it.  (Version 2.1.5)

Users WILL NOT set their clocks to some nicely synchronized time.  This 
is especially true since many of my subscribers are computer neophytes.  
When sorted by date, some replies come before the original posting.  The 
worst offender is actually a highly competent computer user who claims 
to have a defective clock in his laptop that produces messages from the 
year 2080.  (The truth of his claim is immaterial.)  In an ideal world, 
these things would never happen.  This is not an ideal world.

There was a discussion about "not munging" the headers as a validation 
for objecting to altering the "Date" header.  Considering the actions of 
"CookHeaders.py" and other files that variously alter the "Reply To" 
header, delete unwanted headers, and obviously alter the "Subject" line, 
these objections seem weak, if not ironic.  The date set by the senders 
computer is not sacrosanct.  While it may have some "meaning," it is of 
no "use."  Also, if the date isn't set properly, it probably isn't even 
important to the sender.

Since all of these messages are, in fact, being sent by my server I 
think it quite reasonable to change the "Date" to reflect the time that 
it was processed and changed by the server.  I'm the one who has to 
answer questions about why the archive dates seem wrong and who is 
ultimately responsible for the smooth operation of the service.  I would 
LIKE this to be an option that I could set.  I do not have the time or 
inclination to write the code to make this a user option.  However, 
since it something I wanted applied to all of my lists, I added the code 
to copy the "Date" field to "X-Original-Date" and put in the server's 
time as the "Date."

Suddenly, my new archives are coherent.  I don't have to worry about 
aberrant or obnoxious users.  I only added six lines of code (and two 
imports) to "CookHeaders.py" to make this work.  PLEASE tell me I wasted 
my time and there is a simple option I can set!  Tell me that I don't 
have to port this code when I want to upgrade!

I don't speak Python, and I know I imported more than necessary, but it 
seems to work.  I'm including the code changes both to share what I 
found to be Very Useful and to have others sanity-check my code.  As I 
said, I never wrote a line of Python before this.

Steven Kuck
PS  I considered sending this message from the year 2080 as I've had to 
deal with, but I thought I'd give you a chance to implement this patch 
first.
PPS  No, I don't WANT there to be a "fuzzfactor" allowing "close enough" 
dates.  Server Time Only.

# SKuck Header additions to CookHeaders.py of Mailman 2.1.5 - Nov 18, 2004
import time
from email.Utils import parsedate_tz, mktime_tz, formatdate
# End SKuck Header additions
   ...
   msg['List-Id'] = listid_h
   # The above included as an insertion location reference.  I think 
it'll work from anywhere in "def process"
   # SKuck added to pounce on bad email time stamps.
   originaldate = msg.get('date')
   receivedtime = formatdate(time.time())
   del msg['date']
   del msg['x-original-date']
   msg['Date'] = receivedtime
   msg['X-Original-Date'] = originaldate
   # End SKuck added
   ...

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Re: [Mailman-Developers] Dates again

[Jim Tittsler]

On Sat, Nov 20, 2004 at 03:31:38AM -0500, Steven Kuck wrote:
> Suddenly, my new archives are coherent.  I don't have to worry about 

To keep your archives coherent, you can just set:
ARCHIVER_CLOBBER_DATE_POLICY = 1
in your mm_cfg.py.

> my time and there is a simple option I can set!  Tell me that I don't 
> have to port this code when I want to upgrade!

Rather than patching CookHeaders, you might consider making
your own handler that only mungs the date, and then add that
handler to the global pipeline in your mm_cfg.  Then a new
version of CookHeaders.py is not a problem.

-- 
Jim Tittsler http://www.OnJapan.net/  GPG: 0x01159DB6
Python Starship  http://Starship.Python.net/
Ringo MUG Tokyo  http://www.ringo.net/rss.html

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Re: [Mailman-Developers] Dates again

[Brad Knowles]

At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
 Since all of these messages are, in fact, being sent by my server I think
 it quite reasonable to change the "Date" to reflect the time that it was
 processed and changed by the server.
	The problem is that you can't tell which "Date:" headers are good 
and which ones aren't.  The message may have been held up in a queue 
for several days, or the date it was sent could be off by several 
days.  By the time the message gets to you, it is impossible to 
distinguish between these two events.

	IMO, you should adhere to the principle of minimal munging, and 
not replace a "Date:" header unless you can pretty conclusively prove 
that it was set wrong to begin with.

   I'm the one who has to answer
 questions about why the archive dates seem wrong and who is ultimately
 responsible for the smooth operation of the service.  I would LIKE this
 to be an option that I could set.  I do not have the time or inclination
 to write the code to make this a user option.  However, since it something
 I wanted applied to all of my lists, I added the code to copy the "Date"
 field to "X-Original-Date" and put in the server's time as the "Date."
	I would be violently opposed to any system-wide modification that 
would arbitrarily replace all "Date:" headers with ones based on the 
time of reception.

	If there were a way to effectively detect when a "Date:" header 
was wrong and when it was okay, then I might be willing to allow the 
system to correct the "Date:" header in those particular cases.  If 
you've got a patch or additional code that can do that, I'd like to 
see it, although I can't promise it would be accepted by the Mailman 
developers for inclusion in an upcoming version.

 PS  I considered sending this message from the year 2080 as I've had to
 deal with, but I thought I'd give you a chance to implement this patch first.
We've all had the problem.
 PPS  No, I don't WANT there to be a "fuzzfactor" allowing "close enough"
 dates.  Server Time Only.
	I can't speak for Barry, Tokio, or the other Mailman developers, 
but I would be violently opposed to a change of this sort.

--
Brad Knowles, <[EMAIL PROTECTED]>
"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755
  SAGE member since 1995.  See  for more info.
___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Re: [Mailman-Developers] Dates again

[Steven Kuck]

Brad Knowles wrote:
At 3:31 AM -0500 2004-11-20, Steven Kuck wrote:
 Since all of these messages are, in fact, being sent by my server I 
think
 it quite reasonable to change the "Date" to reflect the time that it 
was
 processed and changed by the server.
The problem is that you can't tell which "Date:" headers are good 
and which ones aren't.  The message may have been held up in a queue 
for several days, or the date it was sent could be off by several 
days.  By the time the message gets to you, it is impossible to 
distinguish between these two events.
I feel as though I've suggested making burgers from a sacred cow...  I 
would like to figure out why it is sacred, so pardon my sacrilegious 
question: why does it matter?  Please, no flames...  Can you point me to 
the definitive text for mail header definitions?  RFC 733 only defines 
the format - it doesn't say they are any more inviolate than the 
"Subject" line.

True, you can't tell if a message was sent from a misbehaving computer 
or if it got stuck in the queue of a misbehaving server.  Surely, any 
dates in the future would be fair game for alteration - they can't have 
been "stuck" there since NEXT week.  I'd also be suspect of any dates 
before 1970.  Frankly, I'd be suspect of any date older than a three 
days (over a weekend).

I still think that the time on the originating computer (assuming the 
sender isn't inserting false dates) is a matter for trivia, and archival 
in a "X-Sender-Universal-Disjointed-Time" field.  Why do I care if the 
message was stuck (unless it's my server) or if the user's clock is wrong?

IMO, you should adhere to the principle of minimal munging, and 
not replace a "Date:" header unless you can pretty conclusively prove 
that it was set wrong to begin with.
Come now, it's not like I'm adding content to the body of the message!  
Oh wait, mailman does that too.

I would be violently opposed to any system-wide modification that 
would arbitrarily replace all "Date:" headers with ones based on the 
time of reception.
As I said, I can guarantee messages from the future are wrong.  Disagree?
Perhaps messages from more than a day (or N days) in the past could be 
bounced saying:
"Either your system clock is wrong or your message was unreasonably 
delayed.  Either fix your clock, or make sure your message is still 
current and send it again."
Alternately, the message could be held for approval or date fixing, and 
you could set that user as "Date Impaired" so that all messages from 
that individual get fixed - if they're off.


 PS  I considered sending this message from the year 2080 as I've had to
 deal with, but I thought I'd give you a chance to implement this 
patch first.
We've all had the problem.
At least you recognize it as a "problem."  I would be more than happy to 
work on a reasonable, if not totally infallible "solution."  I know as a 
current solution we tolerate a message archive on this list from 2006.  
If date > now, replace.  Can we at least get that in there?  I'll worry 
about messages from 1989 or earlier on my own.

Steven Kuck - no need for "violent opposition" - I'll behave.
___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Re: [Mailman-Developers] Dates again

[Barry Warsaw]

On Sat, 2004-11-20 at 04:04, Jim Tittsler wrote:
> On Sat, Nov 20, 2004 at 03:31:38AM -0500, Steven Kuck wrote:
> > Suddenly, my new archives are coherent.  I don't have to worry about 
> 
> To keep your archives coherent, you can just set:
> ARCHIVER_CLOBBER_DATE_POLICY = 1
> in your mm_cfg.py.

Right, and if Steven really wants server time to always override, you
can set ARCHIVE_CLOBBER_DATE_POLICY = 2 and
ARCHIVER_ALLOWABLE_SANE_DATE_SKEW to some really small value (minutes(5)
or even 0).

-Barry



signature.asc
Description: This is a digitally signed message part
___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Re: [Mailman-Developers] Dates again

[Kenneth Porter]

--On Saturday, November 20, 2004 6:22 AM -0500 Steven Kuck <[EMAIL PROTECTED]> 
wrote:

As I said, I can guarantee messages from the future are wrong.  Disagree?
Perhaps messages from more than a day (or N days) in the past could be
bounced saying:
"Either your system clock is wrong or your message was unreasonably
delayed.  Either fix your clock, or make sure your message is still
current and send it again."
Alternately, the message could be held for approval or date fixing, and
you could set that user as "Date Impaired" so that all messages from that
individual get fixed - if they're off.
This seems a reasonable approach, given a configurable delivery delay 
tolerance. One could also cross-check References headers against messages 
already received, to set a lower limit on the time stamp. If a message 
claims to have been sent prior to one it references (modulo some 
tolerance), then it can be bounced/modified/moderated.
___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Re: [Mailman-Developers] Dates again

[John W. Baxter]

On 11/20/2004 13:26, "Kenneth Porter" <[EMAIL PROTECTED]> wrote:

> --On Saturday, November 20, 2004 6:22 AM -0500 Steven Kuck <[EMAIL PROTECTED]>
> wrote:
> 
>> As I said, I can guarantee messages from the future are wrong.  Disagree?
>> 
>> Perhaps messages from more than a day (or N days) in the past could be
>> bounced saying:
>> "Either your system clock is wrong or your message was unreasonably
>> delayed.  Either fix your clock, or make sure your message is still
>> current and send it again."
>> Alternately, the message could be held for approval or date fixing, and
>> you could set that user as "Date Impaired" so that all messages from that
>> individual get fixed - if they're off.
> 
> This seems a reasonable approach, given a configurable delivery delay
> tolerance. One could also cross-check References headers against messages
> already received, to set a lower limit on the time stamp. If a message
> claims to have been sent prior to one it references (modulo some
> tolerance), then it can be bounced/modified/moderated.

Keep in mind that there are MUAs which set the Date: as the message is
begun, not when it is sent.  If the sender leaves it as a draft for a couple
of days before sending, the Date: will be "two days old".  (What one wants
to do with that date is another matter.)

  --John
___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Re: [Mailman-Developers] Dates again

[Brad Knowles]

At 6:22 AM -0500 2004-11-20, Steven Kuck wrote:
 I feel as though I've suggested making burgers from a sacred cow...  I
 would like to figure out why it is sacred, so pardon my sacrilegious
 question: why does it matter?  Please, no flames...
Rght.
	You feel free to include all your own inflamatory comments before 
the question, and then insist that this can't lead to a flame war. 
This is not a good start.

  Can you point me to
 the definitive text for mail header definitions?  RFC 733 only defines
 the format - it doesn't say they are any more inviolate than the
 "Subject" line.
	I believe that the current RFC on this matter is 2822, see 
.  The last paragraph of 
Section 3.6 says:

|   The only required header fields are the origination date field and
|   the originator address field(s).  All other header fields are
|   syntactically optional.  More information is contained in the table
|   following this definition.
	This implies to me that the "Date:" header is one of the two most 
important headers on the entire message.  Keep that in mind.  Section 
3.6.1 says:

|3.6.1. The origination date field
|
|   The origination date field consists of the field name "Date" followed
|   by a date-time specification.
|
|orig-date   =   "Date:" date-time CRLF
|
|   The origination date specifies the date and time at which the creator
|   of the message indicated that the message was complete and ready to
|   enter the mail delivery system.  For instance, this might be the time
|   that a user pushes the "send" or "submit" button in an application
|   program.  In any case, it is specifically not intended to convey the
|   time that the message is actually transported, but rather the time at
|   which the human or other creator of the message has put the message
|   into its final form, ready for transport.  (For example, a portable
|   computer user who is not connected to a network might queue a message
|   for delivery.  The origination date is intended to contain the date
|   and time that the user queued the message, not the time when the user
|   connected to the network to send the message.)
	There's nothing here that says that an MTA should, or can, 
correct a "Date:" header that it believes to be incorrect.  There's 
nothing here that says an application can or should correct a "Date:" 
header that it believes to be incorrect.

	You should also read section 3.6.6, and note that it is not 
applicable since "Resent-" fields are for when a user has 
re-introduced a message into the transport system, and this is not 
what we are doing.  Moreover, the "Resent-" fields are purely 
informational, and do not over-ride the original fields in the 
message in any way.

	For my part, all this means that the "Date:" header is even more 
inviolable than the "Reply-to:" header, which is a very old flamewar. 
See 
.


	As for the rest of your message, I consider it to be sufficiently 
inflamatory that I will not attempt to respond.  You seem to have a 
clear agenda here, and there is nothing that will deter you from this 
path.  If you wish to modify your system to function the way you 
want, there's not really much that any of us can do to try to change 
your mind.

	However, if you do actually want to try to benefit the rest of 
the Mailman community, or at least to avoid having to keep re-porting 
your change to every new version of Mailman, you're going to have to 
make some concessions to the patch you are proposing.

--
Brad Knowles, <[EMAIL PROTECTED]>
"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755
  SAGE member since 1995.  See  for more info.
___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org