Re: INTERNALDATE one hour in future for sent message
Unix systems should be run in GMT/UTC (almost the same thing; GMT is _not_ British time"). You then use $TZ in the environment, or some OS-dependent way of setting 'localtime' (eg, a symlink /etc/localtime, or some other method) to let programs show the time in the local zone. That's normally handled by libc. Machine's internal clock is (and has always been) in UTC. TZ environment variable is set (in /etc/TIMEZONE) to US/Eastern which supposedly does automatically handle timezone change. From what I can tell by comparing files found in /usr/share/lib/zoneinfo, US/Eastern is same as setting to EST5EDT (to test this I changed setting TZ in TIMEZONE file to EST5EDT and rebooted. Same behavior as if TZ set to US/Eastern). It's really just asking the operating system for "the current time", so the OS is not using GMT. IS there any way of verifying that INTERNALDATE has access/is using TZ environment variable i.e. are we setting/using it correctly? Some here are thinking that it sounds like it might be a cyrus bug if changing the server time changes the behavior. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: INTERNALDATE one hour in future for sent message
On 2006-06-28 at 16:43 -0400, Jim Brett wrote: > Thanks, your response is greatly appreciated. Here's OS info: > > # uname -a > SunOS machine.company.com 5.8 Generic_117350-13 sun4u sparc > SUNW,Sun-Fire-V240 Edit /etc/TIMEZONE, zone information available in /usr/share/lib/zoneinfo/ $ man -s 4 timezone You may also want to look at /etc/defaults/cron, if you want to make cronjobs stick to GMT or somesuch. You'll probably need a reboot somewhere in there, and to make sure that you adjust the system clock to compensate for adjusting the zone information. A lot of time issues just go away if you keep system clocks automatically synchronised. ntpd helps here, which on Solaris 9 (don't know about 8, sorry) is in: SUNWntpr SUNWntpu (NTP = Network Time Protocol) That's the terse version, but should provide enough pointers for going on with. I'm only assuming that you're not already using NTP, but I suspect that it would've been difficult to keep a system clock outside GMT whilst using ntpd. Social benefit to using GMT with mail-servers, even outside the UK (full disclosure: I'm expat British, so perhaps biased) is that when it comes to reporting abuse and providing logs to correlate events, it's *really* useful to have a common timezone standard which everyone needs to be able to map to their local time easily, without needing to learn acronyms. Even better is when you can say, "Log timestamps are in GMT with time synchronised via NTP to high stratum, so we believe them to be accurate." -- "Everything has three factors: politics, money, and the right way to do it. In that order." -- Gary Donahue Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: INTERNALDATE one hour in future for sent message
Thanks, your response is greatly appreciated. Here's OS info: # uname -a SunOS machine.company.com 5.8 Generic_117350-13 sun4u sparc SUNW,Sun-Fire-V240 Phil wrote: On 2006-06-28 at 10:21 -0400, Jim wrote: INTERNALDATE (hence received date?) one hour in future for sent message. Unix systems should be run in GMT/UTC (almost the same thing; GMT is _not_ "British time"). You then use $TZ in the environment, or some OS-dependent way of setting 'localtime' (eg, a symlink /etc/localtime, or some other method) to let programs show the time in the local zone. That's normally handled by libc. 've searched cyrus wiki plus web but haven't found an answer. From what I can tell, setting of the received date for sent message is in the sphere of influence of the IMAP server. It's really just asking the operating system for "the current time", so the OS is not using GMT. orrect for sent mail i.e. would work 6 months per year. <- Update: I did try that and it works correctly so there is a timezone issue somewhere. Don't run the OS with the kernel's clock in local time, or you'll have all sorts of ongoing problems, biting you in subtle ways. You don't mention the OS you're using (or I missed it) so I can't say what the actual method of setting the system "localtime" is whilst leaving the kernel in GMT. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: INTERNALDATE one hour in future for sent message
On 2006-06-28 at 10:21 -0400, Jim Brett wrote: > INTERNALDATE (hence received date?) one hour in future for sent > message. Unix systems should be run in GMT/UTC (almost the same thing; GMT is _not_ "British time"). You then use $TZ in the environment, or some OS-dependent way of setting 'localtime' (eg, a symlink /etc/localtime, or some other method) to let programs show the time in the local zone. That's normally handled by libc. > I've searched cyrus wiki plus web but haven't found an answer. From > what I can tell, setting of the received date for sent message is in the > sphere of influence of the IMAP server. It's really just asking the operating system for "the current time", so the OS is not using GMT. > correct for sent mail i.e. would work 6 months per year. <- Update: I > did try that and it works correctly so there is a timezone issue somewhere. Don't run the OS with the kernel's clock in local time, or you'll have all sorts of ongoing problems, biting you in subtle ways. You don't mention the OS you're using (or I missed it) so I can't say what the actual method of setting the system "localtime" is whilst leaving the kernel in GMT. -- "Everything has three factors: politics, money, and the right way to do it. In that order." -- Gary Donahue Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
INTERNALDATE one hour in future for sent message
INTERNALDATE (hence received date?) one hour in future for sent
message. I realize that a received date on a message in sent folder
doesn't really have meaning but, if a user moves from sent to inbox (or
trash), then clients (including outlook and outlook express) sort by
received date which is consistently one hour in the future which bothers
our QA department.
Version: Cyrus IMAP4 v2.2.12 [ESMTP Postfix (2.2.1) "Brand X"]
I've searched cyrus wiki plus web but haven't found an answer. From
what I can tell, setting of the received date for sent message is in the
sphere of influence of the IMAP server.
Steps to recreate: user sends mail via our telephone user-interface to
themself (resulting in one message each in sent and inbox folder).
telnet to solaris box and select/fetch info. Sent at 11:27 but
INTERNALDATE for sent message is 12:27 (one hour in future). Values for
message in inbox both 11:27.
// Is there some timezone setting/issue that we're missing. This seems
most likely.
Note that INTERNALDATE for sent message ends with "-400" while date in
envelope info ends with "0400 (EDT)". Not sure if doesn't matter, if
there's a cyrus internal timezone setting we need to set/change or
whether, if machine changed from EDT to EST, INTERNALDATE would be
correct for sent mail i.e. would work 6 months per year. <- Update: I
did try that and it works correctly so there is a timezone issue somewhere.
// We're not sure if it's us or if something else isn't involved ...
Though isn't Javamail still responsible for the APPEND to the mail
folder? If so, it could still be to blame.
// Data. Message in question sent at 11:27:45
bash-2.03# telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK computername.com Cyrus IMAP4 v2.2.12 server ready
// login as user
1 login 555 555
1 OK User logged in
// select sent folder
2 select inbox.sent-mail
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)]
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1134760645]
* OK [UIDNEXT 155]
2 OK [READ-WRITE] Completed
// fetch info on message sent at roughly 11:27
3 fetch 1:1 (ENVELOPE INTERNALDATE RFC822.size)
* 1 FETCH (INTERNALDATE "22-Jun-2006 12:27:45 -0400" RFC822.SIZE 31441
ENVELOPE ("Thu, 22 Jun 2006 11:27:45 -0400 (EDT)" "Voicemail"
(("Jay Bee" NIL "3391111" "computername.com")) (("Jay Bee"
NIL "555" "computername.com")) (("Jay Bee" NIL "555"
"computername.com")) (("Jay Bee" NIL "555" "
computername.com")) NIL NIL NIL
"<[EMAIL PROTECTED]-dt>"))
3 OK Completed (0.000 sec)
// select inbox
4 select inbox
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)]
* 1 EXISTS
* 0 RECENT
* OK [UNSEEN 1]
* OK [UIDVALIDITY 1134760644]
* OK [UIDNEXT 492]
4 OK [READ-WRITE] Completed
// fetch info on received message sent at roughly 11:27
5 fetch 1:1 (ENVELOPE INTERNALDATE RFC822.size)
* 1 FETCH (INTERNALDATE "22-Jun-2006 11:27:45 -0400" RFC822.SIZE 31825
ENVELOPE ("Thu, 22 Jun 2006 11:27:45 -0400 (EDT)" "Voicemail"
(("Jay Bee" NIL "3391111" "computername.com")) (("Jay Bee"
NIL "555" "computername.com")) (("Jay Bee" NIL "555"
"computername.com")) (("Jay Bee" NIL "555" "
computername.com")) NIL NIL NIL
"<[EMAIL PROTECTED]-dt>"))
5 OK Completed (0.000 sec)
// That's it ...
6 logout
* BYE LOGOUT received
6 OK Completed
// - - -
// contents of cyrus.conf
bash-2.03# cat ./usr/local/imapldap/util/cyrus.conf
START {
# do not delete these entries!
mboxlist cmd="ctl_mboxlist -r"
deliver cmd="ctl_deliver -r"
# this is only necessary if using idled for IMAP IDLE
#idledcmd="idled"
}
# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
imap cmd="imapd" listen="imap" prefork=0
# imaps cmd="imapd -s" listen="imaps" prefork=0
# pop3 cmd="pop3d" listen="pop3" prefork=0
# pop3s cmd="pop3d -s" listen="pop3s" prefork=0
# sieve cmd="timsieved" listen="sieve" prefork=0
# at least one LMTP is required for delivery
lmtpunix cmd="lmtpd" listen="/var/spool/postfix/public/lmtp"
prefork=1
}
EVENT
Re: [PATCH] Use INTERNALDATE for ipurge (was: ipurge to use -d as receive date and not sent date)
On Tue, Oct 14, Henrique de Moraes Holschuh wrote:
> Please try the attached patch.
>
> It switches default behaviour to least-surprise (INTERNALDATE), adds
> -X to get the old behaviour, and enhances ipurge by adding -i to invert
> match logic.
I am a little bit late on this topic, but I also made a patch and solved it a
little bit different. Btw.: I don't like changing defaults, even if they do
not make a lot of sense.
--
With best regards,
Carsten Hoeger
--- imap/ipurge.c 2003/11/04 09:27:08 1.1
+++ imap/ipurge.c 2003/11/05 10:00:05
@@ -77,6 +77,7 @@
extern int optopt;
/* globals for callback functions */
+int sent_day = 1;
int days = -1;
int size = -1;
int exact = -1;
@@ -114,7 +115,7 @@
usage(argv[0]);
}
- while ((option = getopt(argc, argv, "C:hxd:b:k:m:f")) != EOF) {
+ while ((option = getopt(argc, argv, "C:hxd:b:k:m:fR")) != EOF) {
switch (option) {
case 'C': /* alt config file */
alt_config = optarg;
@@ -125,6 +126,9 @@
}
days = atoi(optarg) * 86400 /* nominal # of seconds in a 'day' */;
} break;
+case 'R': {
+ sent_day = 0;
+} break;
case 'b': {
if (optarg == 0) {
usage(argv[0]);
@@ -192,11 +196,13 @@
int
usage(char *name) {
- printf("usage: %s [-f] [-C ] [-x] {-d days &| -b bytes|-k Kbytes|-m
Mbytes}\n\t[mboxpattern1 ... [mboxpatternN]]\n", name);
+ printf("usage: %s [-f] [-C ] [-x] {-d days [-R|-S] &| -b bytes|-k
Kbytes|-m Mbytes}\n\t[mboxpattern1 ... [mboxpatternN]]\n", name);
printf("\tthere are no defaults and at least one of -d, -b, -k, -m\n\tmust be
specified\n");
printf("\tif no mboxpattern is given %s works on all mailboxes\n", name);
printf("\t -x specifies an exact match for days or size\n");
printf("\t -f force also to delete mail below user.* and INBOX.*\n");
+ printf("\t -R use received date to purge mails older then -d days\n");
+ printf("\t(default is sent date)\n");
exit(0);
}
@@ -265,7 +271,11 @@
bit32 senttime;
bit32 msgsize;
- senttime = ntohl(*((bit32 *)(buf + OFFSET_SENTDATE)));
+ if( sent_day ) {
+ senttime = ntohl(*((bit32 *)(buf + OFFSET_SENTDATE)));
+ } else {
+ senttime = ntohl(*((bit32 *)(buf + OFFSET_INTERNALDATE)));
+ }
msgsize = ntohl(*((bit32 *)(buf + OFFSET_SIZE)));
stats->total++;
--- man/ipurge.82003/11/04 09:37:16 1.1
+++ man/ipurge.82003/11/04 10:09:27
@@ -57,6 +57,9 @@
.br
[
.BI \-d " days"
+[
+.B \-R
+]
|
.BI \-b " bytes"
|
@@ -99,6 +102,10 @@
.BI \-d " days"
Age of message in \fIdays\fR.
.TP
+.BI \-R
+Use the received date to purge mails in conjunction with \fB-d\fR instead of
+the sent date, which is the default.
+.TP
.BI \-b " bytes"
Size of message in \fIbytes\fR.
.TP
pgp0.pgp
Description: PGP signature
[PATCH] Use INTERNALDATE for ipurge (was: ipurge to use -d as receive date and not sent date)
Please try the attached patch.
It switches default behaviour to least-surprise (INTERNALDATE), adds
-X to get the old behaviour, and enhances ipurge by adding -i to invert
match logic.
On Tue, 14 Oct 2003, Carsten Hoeger wrote:
> On Tue, Oct 14, Rob Siemborski wrote:
> > If a patch was supplied to make this be handled by a command-line option,
> > we'd probably take it (it shouldn't be that much work).
>
> Yes.
> It's quite simple, but I don't want to do it without asking.
> I'll submit it if it is ready.
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
Index: ipurge.c
===
RCS file: /home/cvs/debian/cyrus21-imapd/imap/ipurge.c,v
retrieving revision 1.8
diff -u -r1.8 ipurge.c
--- ipurge.c22 Apr 2003 21:15:03 - 1.8
+++ ipurge.c14 Oct 2003 16:06:24 -
@@ -82,6 +82,8 @@
int exact = -1;
int pattern = -1;
int skipflagged = 0;
+int datemode = OFFSET_INTERNALDATE;
+int invertmatch = 0;
/* for statistical purposes */
typedef struct mbox_stats_s {
@@ -114,7 +116,7 @@
usage(argv[0]);
}
- while ((option = getopt(argc, argv, "C:hxd:b:k:m:fs")) != EOF) {
+ while ((option = getopt(argc, argv, "C:hxd:b:k:m:fsXi")) != EOF) {
switch (option) {
case 'C': /* alt config file */
alt_config = optarg;
@@ -152,6 +154,12 @@
case 's' : {
skipflagged = 1;
} break;
+case 'X' : {
+ datemode = OFFSET_SENTDATE;
+} break;
+case 'i' : {
+ invertmatch = 1;
+} break;
case 'h':
default: usage(argv[0]);
}
@@ -195,12 +203,14 @@
int
usage(char *name) {
- printf("usage: %s [-f] [-s] [-C ] [-x] {-d days &| -b bytes|-k
Kbytes|-m Mbytes}\n\t[mboxpattern1 ... [mboxpatternN]]\n", name);
+ printf("usage: %s [-f] [-s] [-C ] [-x] [-X] [-i] {-d days | -b bytes|-k
Kbytes|-m Mbytes}\n\t[mboxpattern1 ... [mboxpatternN]]\n", name);
printf("\tthere are no defaults and at least one of -d, -b, -k, -m\n\tmust be
specified\n");
printf("\tif no mboxpattern is given %s works on all mailboxes\n", name);
printf("\t -x specifies an exact match for days or size\n");
printf("\t -f force also to delete mail below user.* and INBOX.*\n");
printf("\t -s skip over messages that are flagged.\n");
+ printf("\t -X use Date: header instead of delivery time for date matches.\n");
+ printf("\t -i invert match logic: -x means not equal, date is for newer, size is
for smaller.\n");
exit(0);
}
@@ -270,7 +280,7 @@
bit32 msgsize;
bit32 flagged;
- senttime = ntohl(*((bit32 *)(buf + OFFSET_SENTDATE)));
+ senttime = ntohl(*((bit32 *)(buf + datemode)));
msgsize = ntohl(*((bit32 *)(buf + OFFSET_SIZE)));
flagged = ntohl(*((bit32 *)(buf + OFFSET_SYSTEM_FLAGS))) & FLAG_FLAGGED;
@@ -285,6 +295,11 @@
my_time = time(0);
/*printf("comparing %ld :: %ld\n", my_time, the_record->sentdate); */
if (((my_time - (time_t) senttime)/86400) == (days/86400)) {
+ if (invertmatch) return 0;
+ deleteit(msgsize, stats);
+ return 1;
+ } else {
+ if (!invertmatch) return 0;
deleteit(msgsize, stats);
return 1;
}
@@ -292,6 +307,11 @@
if (size >= 0) {
/* check size */
if (msgsize == size) {
+ if (invertmatch) return 0;
+ deleteit(msgsize, stats);
+ return 1;
+ } else {
+ if (!invertmatch) return 0;
deleteit(msgsize, stats);
return 1;
}
@@ -301,14 +321,22 @@
if (days >= 0) {
my_time = time(0);
/*printf("comparing %ld :: %ld\n", my_time, the_record->sentdate); */
- if ((my_time - (time_t) senttime) > days) {
+ if (!invertmatch && ((my_time - (time_t) senttime) > days)) {
+ deleteit(msgsize, stats);
+ return 1;
+ }
+ if (invertmatch && ((my_time - (time_t) senttime) < days)) {
deleteit(msgsize, stats);
return 1;
}
}
if (size >= 0) {
/* check size */
- if (msgsize > size) {
+ if (!invertmatch && (msgsize > size)) {
+ deleteit(msgsize, stats);
+ return 1;
+ }
+ if (invertmatch && (msgsize < size)) {
deleteit(msgsize, stats);
return 1;
}
Re: internaldate
On Wed, 2003-04-02 at 16:55, Ken Murchison wrote:
> Alexander Brill wrote:
> >
> > I am trying to set the internaldate for a message, but it doesn't store
> > properly.
> >
> > This is what I try (using python)
> > imap.store(1,"+FLAGS","INTERNALDATE 22-Mar-2003 02:10:31 +0100")
> > > ('OK', ['1 (FLAGS (INTERNALDATE 22-Mar-2003 02:10:31 +0100))'])
> > It replies with an OK (prefixed with an >), and that the INTERNALDATE
> > has been altered.
> >
> > But when I try to fetch the same message it tells me that the
> > INTERNALDATE is:
> > ('OK', ['1 (INTERNALDATE "25-Mar-2003 02:10:31 +0100")'])
> >
> > So it doesn't store the values, even though it returns an OK. Any hints?
>
> You can't change INTERNALDATE with STORE, only flags can be changed (see
> sec 6.4.6 of RFC 3501). Typically, INTERNALDATE is a read-only
> attribute which is set when a message gets stored in a mailbox, either
> by SMTP/LMTP, COPY or APPEND (see sec 2.3.3 of RFC 3501).
Is there some other way of setting it then? I have a bunch of mails
dated the 25 of March :)
--
Alexander Brill <[EMAIL PROTECTED]>
http://www.project23.no
PGP-key: http://www.nettstudio.no/pgp/alexander.brill.asc
signature.asc
Description: This is a digitally signed message part
Re: internaldate
Alexander Brill wrote:
>
> I am trying to set the internaldate for a message, but it doesn't store
> properly.
>
> This is what I try (using python)
> imap.store(1,"+FLAGS","INTERNALDATE 22-Mar-2003 02:10:31 +0100")
> > ('OK', ['1 (FLAGS (INTERNALDATE 22-Mar-2003 02:10:31 +0100))'])
> It replies with an OK (prefixed with an >), and that the INTERNALDATE
> has been altered.
>
> But when I try to fetch the same message it tells me that the
> INTERNALDATE is:
> ('OK', ['1 (INTERNALDATE "25-Mar-2003 02:10:31 +0100")'])
>
> So it doesn't store the values, even though it returns an OK. Any hints?
You can't change INTERNALDATE with STORE, only flags can be changed (see
sec 6.4.6 of RFC 3501). Typically, INTERNALDATE is a read-only
attribute which is set when a message gets stored in a mailbox, either
by SMTP/LMTP, COPY or APPEND (see sec 2.3.3 of RFC 3501).
--
Kenneth Murchison Oceana Matrix Ltd.
Software Engineer 21 Princeton Place
716-662-8973 x26 Orchard Park, NY 14127
--PGP Public Key--http://www.oceana.com/~ken/ksm.pgp
internaldate
I am trying to set the internaldate for a message, but it doesn't store
properly.
This is what I try (using python)
imap.store(1,"+FLAGS","INTERNALDATE 22-Mar-2003 02:10:31 +0100")
> ('OK', ['1 (FLAGS (INTERNALDATE 22-Mar-2003 02:10:31 +0100))'])
It replies with an OK (prefixed with an >), and that the INTERNALDATE
has been altered.
But when I try to fetch the same message it tells me that the
INTERNALDATE is:
('OK', ['1 (INTERNALDATE "25-Mar-2003 02:10:31 +0100")'])
So it doesn't store the values, even though it returns an OK. Any hints?
--
Alexander Brill <[EMAIL PROTECTED]>
http://www.project23.no
PGP-key: http://alexb.egil.org/key.pub
signature.asc
Description: This is a digitally signed message part
