Re: Queue Time
On Sat, 19 Aug 2000 [EMAIL PROTECTED] wrote: > Well spotted Richard. > > I haven't looked at this particular paper, but one of the benefits of all > the ATM development work that the Telcos have done over the last 5 or so > years is the intense focus on scheduling algorithms with an emphasis > on fairness and optimal resource usage (oh, and charging for every > packet at every QOS level). Admittedly it tends to be for very short > lived queues (such as cell queues in an ATM switch), but if you're into > reasonably heavy mathematics then this area is rich in related reading > material. Personally I only recommend it for insomniacs... an interesting point, below are my comments on where the differences betwen the models might lead to problems in appying the theories acorss the technologies (In a previous job I built large campus networks based upon X.25, ethernet, fddi and ATM; as well as mail systems based upon greybook mail, X.400, cc:mail and SMTP) Queue numbers - In an ATM network devices only have a finite number of known destination (next hop) devices whose ststus they have to be concerened about when determing the queue behavior.r Open email routers (like postfix and qmail) (based on SMTP and the DNS) have a practically infinite targets (destination addresses) and the number of them is unknown. However in comparison to the others below they have the higest performance as they avoid modifying the contents of the message. The number of queues will be most unlike that of an ATM device. If one considers a closed mail system with similar characteristics to the TAM model such as cc:Mail where the post-offices are connected by cc:Mail routers with a known number of other post-offices as targets (routing to the Internet is handled like another post-office with a gateway from the closed to the open system) then each router might be considered to behave like an ATM switch, however in this system the routers can only implement round-robin serving of the queues based upon alphabetical sorting of the target post-office names and so the QoS is frankly appalling. (if you have a workgroup where people most often communicate within a post-office this is okay, but for cross-postoffice workgroups it just doesn't work) Multi-protocol mail routers like PP provides are rich in gateway functionality, but the performance of these systems is really horrible as they have to cope with all of the transformations necessary to move data between protocols. In PP's case the generic configuration converted generally (as one might expect) but judiciously tweaking the configuration it was possible to stop it reformatting headers and body parts where not necessary. there were internal queues where a message was queued for reforatting, first of the header and then the body, for delivery, and finally for deletion. The overhead of the ISO ROSE layer for IPC really did kill its performance, and the number of sycronous directory writes did not help either Discard of messages --- For data applications (ie not voice ot video) the AAL's expect there to be some overlying network prorocol such as IP or ethernet which will retry the sending of data if they run out of queueing resources. (if the ATM device decides to discard a cell within a higher layer data frame then it can determine which are the following cells, and so discard the rest of the frame alleviating congestion without affecting more frames than ar necessary. In the case of mail messages we rarely see large binary attachments split across multiple mail messages which have to be manually reassembled. It is generally considered VERY BAD for a mail system to loose or purposefully discard a message to alleviate congestion. Blocking architecture - Finally, almost all ATM switches operate with non-blocking routing architectures. conversely mail systems work by receiving a message into a queue, and then effectively blocking until the message is pulled up out of the queue (think q-smtp -> q-queue -> q-send -> q-rspawn -> q-remote) rather than doing something like q-smtp -> q-remote \ \ failure -> q-queue q-send -> q-rspawn so if the destination MTA is available the message is forwarded immediately (and written into the queue) so the routing of messages through a gateway to internal systems is very fast, but if the remote MTA is down (or goes down) then the message is written into the queue for qmail-send to retry later. I think this is where some hard performwance gains will be made in the next generation of mail routing systems after qmail and postfix. to prove the block exists where I say it does, remove the trigger file from the qmail queue directory and see the performance of the system plummet as the trigger mechanism for the queue notifying qmail-send there is work to do in the queue fails. RjL
Re: Queue Time
On Sat, Aug 19, 2000 at 07:42:04PM +0100, [EMAIL PROTECTED] wrote: > On Fri, 18 Aug 2000, Rogerio Brito wrote: > > I was looking for more detains about the mathematical side of > > the things (e.g., what is the measure of "hurt", in your words > > or the cost to which Dan refers?) and like why the optimal > > retry schedule is essentially independent of the actual > > distribution of message delay times. And why did Dan choose a > > quadratic retry schedule and not, say, a cubic one? For some > > convenience? > > The abstract of the paper: > > Chao-Ju Hou and Kang G. Shin, "Determination of an optimal retry time in > multiple-module computing systems," IEEE Trans. on Computers, Vol. 45, No. > 3, pages 374--379, March, 1996 Well spotted Richard. I haven't looked at this particular paper, but one of the benefits of all the ATM development work that the Telcos have done over the last 5 or so years is the intense focus on scheduling algorithms with an emphasis on fairness and optimal resource usage (oh, and charging for every packet at every QOS level). Admittedly it tends to be for very short lived queues (such as cell queues in an ATM switch), but if you're into reasonably heavy mathematics then this area is rich in related reading material. Personally I only recommend it for insomniacs... Mark.
Re: Queue Time
On Fri, 18 Aug 2000, Rogerio Brito wrote: > I was looking for more detains about the mathematical side of > the things (e.g., what is the measure of "hurt", in your words > or the cost to which Dan refers?) and like why the optimal > retry schedule is essentially independent of the actual > distribution of message delay times. And why did Dan choose a > quadratic retry schedule and not, say, a cubic one? For some > convenience? The abstract of the paper: Chao-Ju Hou and Kang G. Shin, "Determination of an optimal retry time in multiple-module computing systems," IEEE Trans. on Computers, Vol. 45, No. 3, pages 374--379, March, 1996 looks relevant, as do the titles of the papers listed at http://ftp.ust.hk/dblp/db/indices/a-tree/s/Shin:Kang_G=.html Hope this helps. RjL
Re: Queue Time
On Aug 18 2000, Michael T. Babcock wrote: > I'm not Dan, but this is slightly less mathematical than it sounds. The > main point (if I understand DJB here) is: > > Its only an hour late? Another 10 minutes will hurt about "this much". > Its a day late? Another hour will probably also hurt about "this much". > Its a week late? Another (day?) won't hurt more than, oh, "this much". > > "this much" being more or less equal ... djb: '...is worth the same as...' > > ... where the amount of delay is respective to the amount of accumulated > delay. Oh, thanks. Yes, I think that I understand the intuition behind those claims. I was looking for more detains about the mathematical side of the things (e.g., what is the measure of "hurt", in your words or the cost to which Dan refers?) and like why the optimal retry schedule is essentially independent of the actual distribution of message delay times. And why did Dan choose a quadratic retry schedule and not, say, a cubic one? For some convenience? If Dan (or any other poster) could help, I'd be very gateful. :-) Thanks for your insight, Roger... -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Rogerio Brito - [EMAIL PROTECTED] - http://www.ime.usp.br/~rbrito/ Nectar homepage: http://www.linux.ime.usp.br/~rbrito/nectar/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Re: Queue Time
[EMAIL PROTECTED] wrote: > > On Thu, 17 Aug 2000, Eric Cox wrote: > > > [EMAIL PROTECTED] wrote: > > > > > > If you only go to an hour granularity and assume a queuelifetime of no > > > more than seven days, then you only need 168 instances. I was kinda thinking > > > of something a little more elegant than that... > > > > How about using Netscape's X-Priority header to set the queue lifetime > > according to the admin's wishes. Set 5 different queue lifetimes > > according to based on the 5 settings of the X-Priority header. This > > could be accomplished with Ian's patch, and some preprocessing before > > qmail-inject. > > qmail-send doesn't read the contents of the mesdsage, Yep, hence the "preprocessing". > Considering the case of 'this mail message is no longer of use unless > delivered to the recipient BY time X'. On a qmail system we only have > control of the retry schedule on suystems which we control: Agreed - no system is perfect. I was just pointing out that some of the MUAs had already come halfway, and with some scripting and Ian's patch, one might be able to meet them in the middle. Personally I don't have any use for this functionality, but thought I'd throw it into the mix... Eric
Re: Queue Time
On Thu, 17 Aug 2000, Eric Cox wrote: > [EMAIL PROTECTED] wrote: > > > > If you only go to an hour granularity and assume a queuelifetime of no > > more than seven days, then you only need 168 instances. I was kinda thinking > > of something a little more elegant than that... > > How about using Netscape's X-Priority header to set the queue lifetime > according to the admin's wishes. Set 5 different queue lifetimes > according to based on the 5 settings of the X-Priority header. This > could be accomplished with Ian's patch, and some preprocessing before > qmail-inject. qmail-send doesn't read the contents of the mesdsage, so qmail-queue would have to do this to set the appropriate value into the queue control files that qmail-send does read. Considering the case of 'this mail message is no longer of use unless delivered to the recipient BY time X'. On a qmail system we only have control of the retry schedule on suystems which we control: if the MX records for my mother's ISP's mail servers points to something other than her machine (and it does) then I loose control of the delivery timing. In fact if I'm emailing her to tell her what time I'm going to be home for lunch (improbably given we're several thousand miles apart) I need to set the time to much less than the real value by which I need to let her know when I'll be home, so any bounce message has time to make it back from her ISP to me so I know to telephone her. Consider the case of 'this message is URGENT, deliver it first' this is okay, however when email systems become congested I have observer that people start marking all messages as urgent just so thier mail gets delivered quickly (this is analogous to ordinary users being able to use negative values for nice(1) and so boosting their prioity) eventually everyone figures out how to do this and the benefit diminishes. I'm not sure that retrying delivery frequently is going to get an urgent delivery delivered /that/ much more frequently than it would have anyway. At the start of the delivery schedule the times are every few seconds, growing to every few minutes and then to every few hours. if the message were truly URGENT then if immediate delivery is not available it should be returned quicker than for normal delivery. (This principle was used in the Glasgow mail package I was using on prime9955 minicomputers in the 1980's.) Urgent messages got returned in half the time of normal messages and non-urgent messages had twice the queuelifetime of normal messages. so, the last case is one which is worthwhile, but only applies to This MTA, once this mta has disposed of a message then all bets are off as to when it will get delivered (if at all). oh, for MUA integrated delivery status tracking like you can get out of X.400 (I've received X.400 DSNs vie internet mail and the first time you get one you think what on earth is this huge chunk of binary data) RjL
Re: Queue Time
I'm not Dan, but this is slightly less mathematical than it sounds. The main point (if I understand DJB here) is: Its only an hour late? Another 10 minutes will hurt about "this much". Its a day late? Another hour will probably also hurt about "this much". Its a week late? Another (day?) won't hurt more than, oh, "this much". "this much" being more or less equal ... djb: '...is worth the same as...' ... where the amount of delay is respective to the amount of accumulated delay. - Original Message - From: "Rogerio Brito" <[EMAIL PROTECTED]> > BTW, Dan, where could I read more about optimal schedules? I'm > particularly interested in learning more about the following > paragraph of yours (I'm not very experienced on Statistics, > but I'm willing to learn more about it): > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > Mathematical amusement: The optimal retry schedule is essentially, > though not exactly, independent of the actual distribution of message > delay times. What really matters is how much cost you assign to retries > and to particular increases in latency. qmail's current quadratic retry > schedule says that an hour-long delay in a day-old message is worth the > same as a ten-minute delay in an hour-old message; this doesn't seem so > unreasonable. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Re: Queue Time
On Aug 17 2000, David Dyer-Bennet wrote: > I'd agree for simply changing the schedule (assuming the new > algorithm isn't more complicated). My response was to a proposal > for making the schedule *variable* by message. Oh, sorry for responding to a different matter (I confess that I read that pretty quickly without paying as much attention as I should). I'd imagine that yes, each message having its own retry schedule would really blow the complexity without limits (each message having a data structure to describe its schedule?) So, we do agree on that. And I just love the KISS principle. []s, Roger... -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Rogerio Brito - [EMAIL PROTECTED] - http://www.ime.usp.br/~rbrito/ Nectar homepage: http://www.linux.ime.usp.br/~rbrito/nectar/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Re: Queue Time
Rogerio Brito <[EMAIL PROTECTED]> writes on 17 August 2000 at 22:20:26 -0300 > On Aug 17 2000, David Dyer-Bennet wrote: > > The harm is in the increased complexity of the queue itself, and in > > the programs that manage and access it. Increased complexity costs > > in reliability, security, and resources consumed. > > As far as I can see (but I may be wrong here), there's no > increased complexity. Just change a function in qmail-send.c > and your new version of qmail, with a different retry schedule > is there, brand spanking new. I'd agree for simply changing the schedule (assuming the new algorithm isn't more complicated). My response was to a proposal for making the schedule *variable* by message. -- Photos: http://dd-b.lighthunters.net/ Minicon: http://www.mnstf.org/minicon Bookworms: http://ouroboros.demesne.com/ SF: http://www.dd-b.net/dd-b David Dyer-Bennet / Welcome to the future! / [EMAIL PROTECTED]
Re: Queue Time
On Aug 17 2000, David Dyer-Bennet wrote: > The current algorithm is essentially exponential backoff by host. I don't think so. qmail uses a quadratic schedule for deliveries (both local and remote, with the difference being the coefficient of the quadratic function -- a function of the number of retires; the coefficients are 10^2 for local deliveries and 20^2 for remote deliveries). I've posted this to the list some time ago and Dave Sill's excellent document also contains a discussion about this. I guess that he could post the formula of next retry in a next version of his document. []s, Roger... -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Rogerio Brito - [EMAIL PROTECTED] - http://www.ime.usp.br/~rbrito/ Nectar homepage: http://www.linux.ime.usp.br/~rbrito/nectar/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Re: Queue Time
On Aug 17 2000, David Dyer-Bennet wrote: > The harm is in the increased complexity of the queue itself, and in > the programs that manage and access it. Increased complexity costs > in reliability, security, and resources consumed. As far as I can see (but I may be wrong here), there's no increased complexity. Just change a function in qmail-send.c and your new version of qmail, with a different retry schedule is there, brand spanking new. The function's name is nextretry (and it is quite short). BUT, before changing qmail's retry schedule, please read the comments that Dan has put on THOUGHTS (which I reproduce below). BTW, Dan, where could I read more about optimal schedules? I'm particularly interested in learning more about the following paragraph of yours (I'm not very experienced on Statistics, but I'm willing to learn more about it): - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Mathematical amusement: The optimal retry schedule is essentially, though not exactly, independent of the actual distribution of message delay times. What really matters is how much cost you assign to retries and to particular increases in latency. qmail's current quadratic retry schedule says that an hour-long delay in a day-old message is worth the same as a ten-minute delay in an hour-old message; this doesn't seem so unreasonable. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Any references would be greatly appreciated. Thanks for any information, Roger... -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Rogerio Brito - [EMAIL PROTECTED] - http://www.ime.usp.br/~rbrito/ Nectar homepage: http://www.linux.ime.usp.br/~rbrito/nectar/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Re: Queue Time
[EMAIL PROTECTED] wrote: > > If you only go to an hour granularity and assume a queuelifetime of no > more than seven days, then you only need 168 instances. I was kinda thinking > of something a little more elegant than that... How about using Netscape's X-Priority header to set the queue lifetime according to the admin's wishes. Set 5 different queue lifetimes according to based on the 5 settings of the X-Priority header. This could be accomplished with Ian's patch, and some preprocessing before qmail-inject. Eric
Re: Queue Time
On Thu, Aug 17, 2000 at 02:39:11PM -0700, M.B. wrote: > why not just run 2 instances of qmail. one w/ a queuelifetime of a few > days or a week and one with a lifetime of a few hours. if it has to go > out and can't it'll end up bouncing out of the queue quickly. it'll be > queued often in that short amount of time as desired. So if lunch is four hours away I need a four-hour queue, if lunch is six hours away, I also need a six-hour queue, if lunch is two days away, I also need a two-day queue, and if I happen to send it an hour earlier than usual, I need a five-hour queue, a seven hour queue and a two-days+one-hour queue. If you only go to an hour granularity and assume a queuelifetime of no more than seven days, then you only need 168 instances. I was kinda thinking of something a little more elegant than that... Regards.
RE: Queue Time
why not just run 2 instances of qmail. one w/ a queuelifetime of a few days or a week and one with a lifetime of a few hours. if it has to go out and can't it'll end up bouncing out of the queue quickly. it'll be queued often in that short amount of time as desired. -- Michael Boyiazis [EMAIL PROTECTED] Mail Architect, NetZero, Inc. ___ Why pay for something you could get for free? NetZero provides FREE Internet Access and Email http://www.netzero.net/download/index.html
Re: Queue Time
On Thu, Aug 17, 2000 at 03:27:35PM -0400, Dave Sill wrote: > [EMAIL PROTECTED] wrote: > > >Hmm. A more devious hack might be to adjust the mtime of the info file to be > >time() + QMAILQUEUELIFETIME - control/queuelifetime. The cost would be > >much closer to zero then - albeit at the cost of a misleading info file... > > And qmail-send would be using the tail end of the usual retry > schedule, not the front end. Yeah, but apart from these really good counterpoints, what else is wrong with my suggestion? (with apologies to Mr Cleese). Regards.
Re: Queue Time
[EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > One frequently-proposed (and possibly implemented) solution for such > > time-critical email is to avoid queuing the message in the first place. > > Instead, you call qmail-remote directly with your message. If it succeeds, > > you know immediately that the message reached the MX you pointed it at. If > > it fails -- then you can queue it, if you think it might still get there > > before the information is obsolete. > That might help the >0.1% of the population that composes and submits email > on a Unix system that has qmail-remote installed, but what of the other poor > sods? They're out of luck; they're trying to do reliable, realtime communication with a protocol which was never designed for either. They need to either change their tools, change their requirements, or pick up the phone. Charles -- --- Charles Cazabon<[EMAIL PROTECTED]> GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ Any opinions expressed are just that -- my opinions. ---
Re: Queue Time
Michael T. Babcock <[EMAIL PROTECTED]> wrote: > > One frequently-proposed (and possibly implemented) solution for such > > time-critical email is to avoid queuing the message in the first place. > > Instead, you call qmail-remote directly with your message. If it succeeds, > > you know immediately that the message reached the MX you pointed it at. If > > it fails -- then you can queue it, if you think it might still get there > > before the information is obsolete. > ... on which note ... how much of a hack would it be to qmail-inject to add > this as an option? Not a hack at all. Don't touch the qmail-inject code; instead, write a wrapper around it (in Python, Perl, or even C). Try qmail-remote; check it's exit code, and if necessary, call /var/qmail/bin/qmail-inject-orig or whatever you move the 'real' qmail-inject to. Make sure you return the same exit codes that the real one does, or you'll confuse qmail when it's injecting bounce messages and such. Charles -- --- Charles Cazabon<[EMAIL PROTECTED]> GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ Any opinions expressed are just that -- my opinions. ---
Re: Queue Time
Date: Thu, 17 Aug 2000 12:21:33 -0700 From: [EMAIL PROTECTED] Off tangent, how does an SMTP submission get at this feature? Interestingly, it should probably be something that each MTA knows about to be truly useful, consider secondary MXes gateway systems, etc. Ahh, a change in SMTP - fat chance! Yes, doing this properly would require an SMTP extension. I think it could be fairly easily added to the DSN extension in RFC 1891. (Of course, qmail doesn't support DSN.) Ian
Re: Queue Time
- Original Message - From: "Charles Cazabon" <[EMAIL PROTECTED]> > > If I send an email to mother saying "I'll be home for lunch" I'd like to tell my > > MTA to drop/bounce the mail after that event has occurred. > > One frequently-proposed (and possibly implemented) solution for such > time-critical email is to avoid queuing the message in the first place. > Instead, you call qmail-remote directly with your message. If it succeeds, > you know immediately that the message reached the MX you pointed it at. > If it fails -- then you can queue it, if you think it might still get there > before the information is obsolete. ... on which note ... how much of a hack would it be to qmail-inject to add this as an option?
Re: Queue Time
[EMAIL PROTECTED] wrote: >Hmm. A more devious hack might be to adjust the mtime of the info file to be >time() + QMAILQUEUELIFETIME - control/queuelifetime. The cost would be >much closer to zero then - albeit at the cost of a misleading info file... And qmail-send would be using the tail end of the usual retry schedule, not the front end. -Dave
Re: Queue Time
> > If I send an email to mother saying "I'll be home for lunch" I'd like to tell my > > MTA to drop/bounce the mail after that event has occurred. > > One frequently-proposed (and possibly implemented) solution for such > time-critical email is to avoid queuing the message in the first place. > Instead, you call qmail-remote directly with your message. If it succeeds, > you know immediately that the message reached the MX you pointed it at. > If it fails -- then you can queue it, if you think it might still get there > before the information is obsolete. That might help the >0.1% of the population that composes and submits email on a Unix system that has qmail-remote installed, but what of the other poor sods? Regards.
Re: Queue Time
On Thu, Aug 17, 2000 at 12:13:11PM -0700, Ian Lance Taylor wrote: >Date: Thu, 17 Aug 2000 12:07:24 -0700 >From: [EMAIL PROTECTED] > >> The information is passed via the environment variable >> QMAILQUEUELIFETIME to qmail-inject, which uses a new code (L) in the >> todo file. qmail-send moves this new code over to the info file, and >> honors it when bouncing messages. > >Hmm. A more devious hack might be to adjust the mtime of the info file to be >time() + QMAILQUEUELIFETIME - control/queuelifetime. The cost would be >much closer to zero then - albeit at the cost of a misleading info file... > > The problem is that the information has to get from qmail-inject to > the info file. qmail-inject creates a todo file. It is qmail-send > which reads the todo file, and creates the info file. So the > information must be recorded in the todo file in some fashion. It > could be done by fiddling the mtime of the todo file, and having > qmail-send copy the mtime of the todo file over to the info file. But > I don't think that is significantly cheaper than what I implemented. > > Also it would make the qmail-qread output misleading. Agreed. Off tangent, how does an SMTP submission get at this feature? Interestingly, it should probably be something that each MTA knows about to be truly useful, consider secondary MXes gateway systems, etc. Ahh, a change in SMTP - fat chance! Regards.
Re: Queue Time
[EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > On Thu, Aug 17, 2000 at 02:17:37PM -0700, Tim Hunter wrote: > > This doesn't even make sense, are you looking to configure a different time > > period for different users? > > > > This isn't even possible, or neccessary. > > Hmm. I think this'd be a pretty useful feature actually. > > If I send an email to mother saying "I'll be home for lunch" I'd like to tell my > MTA to drop/bounce the mail after that event has occurred. One frequently-proposed (and possibly implemented) solution for such time-critical email is to avoid queuing the message in the first place. Instead, you call qmail-remote directly with your message. If it succeeds, you know immediately that the message reached the MX you pointed it at. If it fails -- then you can queue it, if you think it might still get there before the information is obsolete. Charles -- --- Charles Cazabon<[EMAIL PROTECTED]> GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ Any opinions expressed are just that -- my opinions. ---
Re: Queue Time
Date: Thu, 17 Aug 2000 12:07:24 -0700 From: [EMAIL PROTECTED] > The information is passed via the environment variable > QMAILQUEUELIFETIME to qmail-inject, which uses a new code (L) in the > todo file. qmail-send moves this new code over to the info file, and > honors it when bouncing messages. Hmm. A more devious hack might be to adjust the mtime of the info file to be time() + QMAILQUEUELIFETIME - control/queuelifetime. The cost would be much closer to zero then - albeit at the cost of a misleading info file... The problem is that the information has to get from qmail-inject to the info file. qmail-inject creates a todo file. It is qmail-send which reads the todo file, and creates the info file. So the information must be recorded in the todo file in some fashion. It could be done by fiddling the mtime of the todo file, and having qmail-send copy the mtime of the todo file over to the info file. But I don't think that is significantly cheaper than what I implemented. Also it would make the qmail-qread output misleading. Ian
Re: Queue Time
Michael T. Babcock <[EMAIL PROTECTED]> writes on 17 August 2000 at 14:52:40 -0400 > > - Original Message - > From: "David Dyer-Bennet" <[EMAIL PROTECTED]> > > > > [EMAIL PROTECTED] <[EMAIL PROTECTED]> writes on 17 August 2000 at > 11:32:00 -0700 > > > > > I can't see much harm in being able to define the queuelifetime on > > > an individual submission - perhaps limited to between 0 and some > > > multiple of control/queuelifetime. > > > > The harm is in the increased complexity of the queue itself, and in > > the programs that manage and access it. Increased complexity costs in > > reliability, security, and resources consumed. > > > > I do agree that that feature has some benefit. I'm doubtful it's > > worth the cost, but I have little idea what the cost would *actually* > > be. Remember that there'd have to be some way to get the information > > passed to the queueing engine, too. > > Oh come now. It may cost in speed. But it adds functionality. Any > functionality like this can be coded in a secure and reliable manner, so > those shouldn't be cited as reasons to avoid the issue. I'm not an expert on software complexity and error probabilities. But what you've just said is someting I understand to be a very common misconception. Features add complexity. Complexity adds chances for things to go wrong in unexpected ways. Even apparently simple, isolated, features. But to get deeper analysis and more detail, you'd need to find somebody more knowledgable than me; I know the results more than the evidence. -- Photos: http://dd-b.lighthunters.net/ Minicon: http://www.mnstf.org/minicon Bookworms: http://ouroboros.demesne.com/ SF: http://www.dd-b.net/dd-b David Dyer-Bennet / Welcome to the future! / [EMAIL PROTECTED]
Re: Queue Time
On Thu, Aug 17, 2000 at 11:59:08AM -0700, Ian Lance Taylor wrote: >From: "David Dyer-Bennet" <[EMAIL PROTECTED]> >Date: Thu, 17 Aug 2000 13:49:00 -0500 (CDT) > >[EMAIL PROTECTED] <[EMAIL PROTECTED]> writes on 17 August 2000 at 11:32:00 -0700 > > > I can't see much harm in being able to define the queuelifetime on > > an individual submission - perhaps limited to between 0 and some > > multiple of control/queuelifetime. > >The harm is in the increased complexity of the queue itself, and in >the programs that manage and access it. Increased complexity costs in >reliability, security, and resources consumed. > >I do agree that that feature has some benefit. I'm doubtful it's >worth the cost, but I have little idea what the cost would *actually* >be. Remember that there'd have to be some way to get the information >passed to the queueing engine, too. > > I think my patch shows the actual cost of this feature. It is > non-zero, but is not high. > > The information is passed via the environment variable > QMAILQUEUELIFETIME to qmail-inject, which uses a new code (L) in the > todo file. qmail-send moves this new code over to the info file, and > honors it when bouncing messages. Hmm. A more devious hack might be to adjust the mtime of the info file to be time() + QMAILQUEUELIFETIME - control/queuelifetime. The cost would be much closer to zero then - albeit at the cost of a misleading info file... Regards.
Re: Queue Time
From: "David Dyer-Bennet" <[EMAIL PROTECTED]> Date: Thu, 17 Aug 2000 13:49:00 -0500 (CDT) [EMAIL PROTECTED] <[EMAIL PROTECTED]> writes on 17 August 2000 at 11:32:00 -0700 > I can't see much harm in being able to define the queuelifetime on > an individual submission - perhaps limited to between 0 and some > multiple of control/queuelifetime. The harm is in the increased complexity of the queue itself, and in the programs that manage and access it. Increased complexity costs in reliability, security, and resources consumed. I do agree that that feature has some benefit. I'm doubtful it's worth the cost, but I have little idea what the cost would *actually* be. Remember that there'd have to be some way to get the information passed to the queueing engine, too. I think my patch shows the actual cost of this feature. It is non-zero, but is not high. The information is passed via the environment variable QMAILQUEUELIFETIME to qmail-inject, which uses a new code (L) in the todo file. qmail-send moves this new code over to the info file, and honors it when bouncing messages. Ian
Re: Queue Time
- Original Message - From: "David Dyer-Bennet" <[EMAIL PROTECTED]> > [EMAIL PROTECTED] <[EMAIL PROTECTED]> writes on 17 August 2000 at 11:32:00 -0700 > > > I can't see much harm in being able to define the queuelifetime on > > an individual submission - perhaps limited to between 0 and some > > multiple of control/queuelifetime. > > The harm is in the increased complexity of the queue itself, and in > the programs that manage and access it. Increased complexity costs in > reliability, security, and resources consumed. > > I do agree that that feature has some benefit. I'm doubtful it's > worth the cost, but I have little idea what the cost would *actually* > be. Remember that there'd have to be some way to get the information > passed to the queueing engine, too. Oh come now. It may cost in speed. But it adds functionality. Any functionality like this can be coded in a secure and reliable manner, so those shouldn't be cited as reasons to avoid the issue.
Re: Queue Time
[EMAIL PROTECTED] <[EMAIL PROTECTED]> writes on 17 August 2000 at 11:32:00 -0700 > I can't see much harm in being able to define the queuelifetime on > an individual submission - perhaps limited to between 0 and some > multiple of control/queuelifetime. The harm is in the increased complexity of the queue itself, and in the programs that manage and access it. Increased complexity costs in reliability, security, and resources consumed. I do agree that that feature has some benefit. I'm doubtful it's worth the cost, but I have little idea what the cost would *actually* be. Remember that there'd have to be some way to get the information passed to the queueing engine, too. -- Photos: http://dd-b.lighthunters.net/ Minicon: http://www.mnstf.org/minicon Bookworms: http://ouroboros.demesne.com/ SF: http://www.dd-b.net/dd-b David Dyer-Bennet / Welcome to the future! / [EMAIL PROTECTED]
Re: Queue Time
Date: Thu, 17 Aug 2000 11:32:00 -0700 From: [EMAIL PROTECTED] On Thu, Aug 17, 2000 at 02:17:37PM -0700, Tim Hunter wrote: > This doesn't even make sense, are you looking to configure a different time > period for different users? > > This isn't even possible, or neccessary. Hmm. I think this'd be a pretty useful feature actually. If I send an email to mother saying "I'll be home for lunch" I'd like to tell my MTA to drop/bounce the mail after that event has occurred. Likewise I'm sure that e-letter establishments have different cutoff times for when a mailout should be dropped - especially for very time-sensitive info. I can't see much harm in being able to define the queuelifetime on an individual submission - perhaps limited to between 0 and some multiple of control/queuelifetime. Setting queuelifetime isn't the same thing as controlling the backoff algorithm. I personally agree that setting queuelifetime is useful. I wrote up a patch to do that a while back: http://www.ornl.gov/its/archives/mailing-lists/qmail/2000/05/msg00140.html Ian
Re: Queue Time
On Thu, Aug 17, 2000 at 02:17:37PM -0700, Tim Hunter wrote: > This doesn't even make sense, are you looking to configure a different time > period for different users? > > This isn't even possible, or neccessary. Hmm. I think this'd be a pretty useful feature actually. If I send an email to mother saying "I'll be home for lunch" I'd like to tell my MTA to drop/bounce the mail after that event has occurred. Likewise I'm sure that e-letter establishments have different cutoff times for when a mailout should be dropped - especially for very time-sensitive info. I can't see much harm in being able to define the queuelifetime on an individual submission - perhaps limited to between 0 and some multiple of control/queuelifetime. Regards.
Re: Queue Time
This doesn't even make sense, are you looking to configure a different time period for different users? This isn't even possible, or neccessary. This is not a feature that really even affects a user. qmail will continue to deliver throughout the week in the essentially exponential backoff by host algorithm. - Original Message - From: "Michael T. Babcock" <[EMAIL PROTECTED]> To: "David Dyer-Bennet" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Sent: Thursday, August 17, 2000 10:23 AM Subject: Re: Queue Time > The current algorithm is probably fine for most users, but what about > configuring the initial frequency? I can see some people being interested > in trying again in 5 or 30 seconds, and others wanting to wait a few hours. > > - Original Message - > From: "David Dyer-Bennet" <[EMAIL PROTECTED]> > > > > Shane Wise <[EMAIL PROTECTED]> writes on 17 August 2000 at > 11:12:30 -0500 > > > Is there any way to change the queue time to where it will retry more > often? > > > > No. Why do you think you need this? > > > > The current algorithm is essentially exponential backoff by host. It > > tries more often at first, and gradually less often as the message > > gets older, until eventually it gives up and bounces after one last > > try. > >
Re: Queue Time
The current algorithm is probably fine for most users, but what about configuring the initial frequency? I can see some people being interested in trying again in 5 or 30 seconds, and others wanting to wait a few hours. - Original Message - From: "David Dyer-Bennet" <[EMAIL PROTECTED]> > Shane Wise <[EMAIL PROTECTED]> writes on 17 August 2000 at 11:12:30 -0500 > > Is there any way to change the queue time to where it will retry more often? > > No. Why do you think you need this? > > The current algorithm is essentially exponential backoff by host. It > tries more often at first, and gradually less often as the message > gets older, until eventually it gives up and bounces after one last > try.
Re: Queue Time
Shane Wise <[EMAIL PROTECTED]> writes on 17 August 2000 at 11:12:30 -0500 > Is there any way to change the queue time to where it will retry more often? No. Why do you think you need this? The current algorithm is essentially exponential backoff by host. It tries more often at first, and gradually less often as the message gets older, until eventually it gives up and bounces after one last try. Do you think you have something that's a general improvement on this? Or special requirements? -- Photos: http://dd-b.lighthunters.net/ Minicon: http://www.mnstf.org/minicon Bookworms: http://ouroboros.demesne.com/ SF: http://www.dd-b.net/dd-b David Dyer-Bennet / Welcome to the future! / [EMAIL PROTECTED]
Re: Queue Time
On Thu, Aug 17, 2000 at 11:12:30AM -0500, Shane Wise wrote: > Is there any way to change the queue time to where it will retry more often? No, I'm afraid not. However, giving qmail-send and ALRM signal will cause all deferred deliveries to be retried. james -- James Raftery (JBR54) - Programmer Hostmaster - IE TLD Hostmaster IE Domain Registry - www.domainregistry.ie - (+353 1) 706 2375 "Managing 4000 customer domains with BIND has been a lot like herding cats." - Mike Batchelor, on [EMAIL PROTECTED]
Queue Time
Is there any way to change the queue time to where it will retry more often?
Re: queue time
> How long qmail can queue a messages? Is there any control to > change it? RTFM qmail-send Chris
queue time
Hi all, How long qmail can queue a messages? Is there any control to change it? Sifat.
