Stripping binaries?
I'd like to set up my qmail-based mailhosts to strip attachments automatically, save them to a dir, and put a "this attachment saved to /some/path" message at the bottom... instead of distributing binaries to all and sundry. Anyone doing this with qmail yet, and/or got any hints? -- ~jv -- [EMAIL PROTECTED] : [EMAIL PROTECTED] -- ... threads rolling ... -- >http://homepage.mac.com/seclorum/FileSharing.html<
Re: Single UID mailboxes - mail getting lost
I agree... because I just got it working finally! I dont know what was wrong, but I reinstalled and started over. That seemed to work. About the only disadvantage to not using a vpop type manager is the automation. But even a simple perl script can solve that problem. Jay On Monday 30 July 2001 8:56 pm, you wrote: > On Mon, Jul 30, 2001 at 10:55:10PM +0200, Henning Brauer wrote: > > Singe UID setups, usually called Virtual User Setups, are requiring a > > virtual user manager. > > That's not true. I've been using a roll-my-own single UID setup for ages, > with no vpopmail or vmailmgr or anything but a custom checkpassword. It > works beautifully and I don't have to worry about the security of non-DJB > code (except for the cdb-enabled checkpassword, which I wrote myself). > > For someone with a small set of domains and a user base that doesn't change > that often, I'd recommend a single-UID setup. It'll be more secure, and > it'll give him a better understanding of how all the pieces of qmail work. > > Chris Content-Type: application/pgp-signature; charset="us-ascii"; name="Attachment: 1" Content-Transfer-Encoding: 7bit Content-Description: -- Jay Kline [EMAIL PROTECTED] http://www.slushpupie.com A people living under the perpetual menace of war and invasion is very easy to govern. It demands no social reforms. It does not haggle over expenditures on armaments and military equipment. It pays without discussion, it ruins itself, and that is an excellent thing for the syndicates of financiers and manufacturers for whom patriotic terrors are an abundant source of gain. -- Anatole France
Single UID mailboxes - mail getting lost
I need to have a mail server that accepts mail from several domains. I read through your HOWTO and thought that would be the best way to implement it, however I have not had any success with it. When I try to send a message, it seems to go ok, there are no errors in the log files or anything, but the message seems to just disappear. To save time and space, I will give you the basic setup I have, listing only one user. The rest use the same setup, just change the names, etc. I created a user/group popuser (both have ID 101) The email address I will use is [EMAIL PROTECTED] The host machine is dogbert.slushpupie.com /var/qmail/control/users/assign: =jay-tarsk-com:popuser:101:101:/var/qmail/popboxes/tarsk-com/tarsk-jay::: /var/qmail/control/users/poppasswd: (the password in this case is tarsk) tarsk-jay:W.I8jJCHbKvBQ:popuser:/var/qmail/popboxes/tarsk-com/tarsk-jay /var/qmail/control/virtualdomains: tarsk.com:tarsk-com /var/qmail/control/rcpthosts: localhost dogbert.slushpupie.com slushpupie.com tarsk.com /var/qmail/popboxes/tarsk-com/tarsk-jay/.qmail: ./Maildir/ And here is what the syslog looks like when I try to send a message: Jul 30 13:22:21 dogbert qmail: 996517341.588676 new msg 109593 Jul 30 13:22:21 dogbert qmail: 996517341.589130 info msg 109593: bytes 233 from <> qp 4889 uid 64011 Jul 30 13:22:21 dogbert qmail: 996517341.596701 starting delivery 39: msg 109593 to local @dogbert.slushpupie.com Jul 30 13:22:21 dogbert qmail: 996517341.597090 status: local 1/10 remote 0/20 Jul 30 13:22:21 dogbert qmail: 996517341.600277 delivery 39: success: Jul 30 13:22:21 dogbert qmail: 996517341.600649 status: local 0/10 remote 0/20 Jul 30 13:22:21 dogbert qmail: 996517341.600799 end msg 109593 After sending the message there is nothing. No error, nothing bounces back, I check the queue dirs, and nothing in there. I am afraid this is a bit over my head, and could really use some help. Any help you can provide would be greatly appreciated. Thanks, Jay -- Jay Kline [EMAIL PROTECTED] http://www.slushpupie.com/
Re: Coding scripts for QMAILQUEUE patch
"jdomingo" == jdomingo writes: jdomingo> I'm sure this script is quite simple (maybe just opening jdomingo> a couple of fd from which to read the messages from), jdomingo> but I'd like to get a working script to build on it and jdomingo> learn (a bash script would be perfect :). Here's one I wrote in Perl that you should be able to get started with: http://www.soffian.org/downloads/qmail/qqrbl j.
Remote server connects then QUIT's, why?
I've been seeing a lot of the following ever since I started accepting mail for a particular domain. > 988341350.667080 tcpserver: pid 10036 from 138.220.29.7 > 988341350.668046 tcpserver: ok 10036 matrixit.net:216.58.86.3:25 > :138.220.29.7::2037 > 988341350.672839 10036 > 220 matrixit.net ESMTP ready^M > 988341350.739657 10036 < EHLO worldbank.org^M > 988341350.739847 10036 > 250-matrixit.net^M > 988341350.739958 10036 > 250-SIZE 10485760^M > 988341350.740083 10036 > 250-PIPELINING^M > 988341350.740182 10036 > 250 8BITMIME^M > 988341350.808618 10036 < QUIT^M > 988341350.808958 tcpserver: end 10036 status 0 The remote mail server at 138.220.29.7 just connects, then QUIT's immediately, NOT delivering the mail it has for me (and yes, it does have some). So it's not like qmail is refusing the mail, the remote just decides to QUIT before sending it. It has been doing this every 5 minutes over the past few days. Everything else is honky-dory. I'm totally at a loss for why this is happening. Any possible explanations? >From the occasional messages I have received, it appears to be running Lotus Domino Release 5.0.5 and Mailsweeper SMTPRS 4.2.1 > Received: from worldbank.org ([138.220.29.7]) > by WBLN0014.worldbank.org (Lotus Domino Release 5.0.5) > with ESMTP id 2001042623080385:20693 ; > Thu, 26 Apr 2001 23:08:03 -0400 > Received: from matrixit.net (matrixit.net) by worldbank.org > (Content Technologies SMTPRS 4.2.1) I'm going to contact the postmaster, but I doubt that'll give me any clue.
argh
Anyone else have trouble unsubscribing to the list? Not that I don't love you guys or anything, I just need to get off of some these lists that I'm on. I've tried unsubscribing 5 separate times. I have confirmation messages that say I have been removed from the list, and I stop receiving messages for anywhere from 2 hours to a day, and then I'm magically back on the list again. How do I get off the list and stay off? -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> http://cbs.marketwatch.com http://www.bigcharts.com
RE: Using a RAMDISK for /var/qmail/queue thoughts ?
> -Original Message- > From: Sean Reifschneider [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, December 13, 2000 12:12 PM > > 10ns is much faster than 5.3ms... It works, I've done it, > it's reasonably I was thinking about doing this awhile back. I made a ramdisk and used both bonnie and bonnie2 to do some benchmarking on it, and I actually got worse performance on the Ramdisk. You still have the overhead of the file system, and you have no hardware DMA controller with a Ramdisk. It was slower in my experience. The box I tested it on had 1GB of Ram, and a 512MB ramdisk. No swap was being used during the tests. Jay > -Original Message- > From: Sean Reifschneider [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, December 13, 2000 12:12 PM > To: Greg Cope > Cc: qmail mailing list > Subject: Re: Using a RAMDISK for /var/qmail/queue thoughts ? > > > On Wed, Dec 13, 2000 at 04:20:19PM +, Greg Cope wrote: > >Has anyone any empirical evidence for the speed increases I > may expect > >(as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI > system (eg 10k, > >5.3 ms seek 25mb/s) ? > > 10ns is much faster than 5.3ms... It works, I've done it, > it's reasonably > fast, but you still have to worry about things like swamping the todo > and on top of that you may have to worry about filling up your queue > disc. You can get QMail into a situation where it's completely wedged > until you manually remove some files from the ram disc to > give it enough > space to continue delivering mail. > > Sean > -- > I never thought I'd live in a country where physical > violence would be used > to disenfranchise voters. Have you heard about Bush > supporters rioting? > Sean Reifschneider, Inimitably Superfluous <[EMAIL PROTECTED]> > tummy.com - Linux Consulting since 1995. Qmail, KRUD, > Firewalls, Python >
RE: Newsletter
Not if you set the security options. I have mine set up so that only moderators can post, others get a message that tells them they don't have permission. It's in the FAQ at http://www.ezmlm.org. Jay > -Original Message- > From: Fernando Costa de Almeida [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, November 01, 2000 9:23 AM > To: Charles Cazabon > Cc: qmaillist > Subject: Re: Newsletter > > > Charles Cazabon wrote: > > > Fernando Costa de Almeida <[EMAIL PROTECTED]> wrote: > > > > > > > > > 1) Is there a specific newsletter software that works together > > > qmail? > > > > Yes. ezmlm, ezmlm-idx both work natively with qmail. > Majordomo can be made > > to work with qmail. See www.qmail.org; mailing lists are > mentioned a few > > times there > >Ok, but if the user that receives the email see the > header: Delivered-To or > Mailing-List and sends a email to this address, he will send > a email to MY > newsletter. > > > > > > > > > 2) Has anyone applied the big-concurrency patch in a > Red-Hat 6.2 > > > System? > > > > Yes. Or did you want some specific piece of information? > > Yes, I cant do this because when I try to recompile > qmail, it shows a msg > like this "Your system has a hidden FD_SET() limit"... I dont > know what to > change in the kernel to made this work This is the piece > Im looking > for. > > Thanks, > > > > > > > > Charles > > -- > > > -- > - > > Charles Cazabon > <[EMAIL PROTECTED]> > > GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ > Any opinions expressed are just that -- my opinions. > --- -- To boldly go where I surely don't belong.
RE: people are definately starting to harvest emailadresses on this list...
Why are you such an asshole? Who's the owner of this list? I'm getting sick of hearing Felix's shit. -Original Message- From: Felix von Leitner [mailto:[EMAIL PROTECTED]] Sent: Saturday, October 28, 2000 6:56 PM To: Qmail mailing list Subject: Re: people are definately starting to harvest emailadresses on this list... Thus spake Martin Jespersen ([EMAIL PROTECTED]): > Nice to see that people are able to be constructive around here *pats Felix on his little head* While we are talking about "constructive", please construct yourself a gut and shoot yourself, idiot. Felix
fixcrio
I'm calling tcpserver with this line: tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u 503 -g 503 0 smtp /var/qmail/bin/qmail-smtpd 2>&1 | /var/qmail/bin/splogger smtpd 3 & I need to use fixcrio to fix stupid emailers that put stray 's in their messages. How do I integrate fixcrio into this? Do I just do: tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u 503 -g 503 0 smtp /usr/local/bin/fixcrio | /var/qmail/bin/qmail-smtpd 2>&1 | /var/qmail/bin/splogger smtpd 3 & Jay
RE: How many outbound messages can you send per hour?
The reason it's so slow is a large number of the connections are most likely being tied up for long periods by mailservers than don't respond. About 80 of my 120 connections when I sent out mailings would do the same thing. Apply the big concurrency patch. My concurrency is at 508 now, and I can do millions of messages per day. During sending, about 100-120 of the connections are tied up by slow or nonresponding mailservers. I'd raise my concurrency higher, but I have to make some kernel mods first (FD_SET problem). Jay > -Original Message- > From: Greg Jorgensen [mailto:[EMAIL PROTECTED]] > Sent: Friday, October 27, 2000 3:44 PM > To: qmail > Subject: How many outbound messages can you send per hour? > > > We send lots of newsletters and other subscription-type > emails. Our qmail > server seems to max out at 10,000 outbound messages per hour. > I already have > concurrencyremote set to 120, and reverse DNS lookups and > other slowdowns > turned off. The CPU (a dual UltraSparc) is not maxing out; it > looks like we've > reached a limit of the SMTP protocol and the number of > connections allowed. > > I can see that separating outbound and incoming mail on > different servers would > help; most of the bounces come in while the bulk of the > subscriptions are going > out. We are going to add some servers, but I'm wondering if > 10,000/hour is a > typical limit, or is there some way I can send more messages > in an hour? > > Thanks! > > > = > Greg Jorgensen > Deschooling Society > Portland, Oregon, USA > [EMAIL PROTECTED] > > __ > Do You Yahoo!? > Yahoo! Messenger - Talk while you surf! It's FREE. > http://im.yahoo.com/ >
RE: How to ignore .qmail-* files for given users?
If you're using wu-ftpd or some derivative of, you should be able to do this in the /etc/ftpaccess file. Do a 'man ftpaccess' for the correct syntax. Jay -Original Message- From: John Chronakis [mailto:[EMAIL PROTECTED]] Sent: Monday, October 23, 2000 9:55 PM To: qmail-list Subject: How to ignore .qmail-* files for given users? Hello, I would like to prevent users with ftp access to change the delivery instructions for qmail. To be more specific, I want some users not to be able to receive mail at all (they all bellong to the same group) and others not be able to change delivery instruction by creating their own dot-qmail files via ftp (they all belong to another group). I suppose that this is controled by QMAILHOME/users/assign, but the man page is not very helpful. Can anyone help? John
RE: MTA and \r\n problems
Just had this exact same problem 2 days ago with ASPmail. The script was pulling some HTML from a web page and emailing it off. There was a lone linefeed character at the end of one of the tables on the page. One of our developer guys fixed the page somehow, or made the aspmail thing look for lone 's. Someone mentioned that you could start smtpd with something like fixcrio | /var/qmail/bin/qmail-smtpd, but I haven't tried it yet. Jay -Original Message- From: Rich Feather [mailto:[EMAIL PROTECTED]] Sent: Friday, October 13, 2000 2:36 PM To: [EMAIL PROTECTED] Subject: MTA and \r\n problems Okay, I have inherited a legacy Active Server Pages/VB environment that uses ASPEmail to send HTML email messages to a qmail relay server. I keep getting the dreaded LF bounce. After talking to Persits Software (the makers of the component), they're convinced that qmail is stripping the CRLF and replacing it with simply a LF. My questions are, is this a reasonable behavior from qmail and, if so, how would I fix it? Thanks.
RE: A bug or am I being daft?
Doesn't the case change violate RFC821 or 822? I seem to remember reading that case in the user portion of the email address should never be changed because the accounts "Bob" and "bob" are two completely different accounts on a unix machine. Jay -Original Message- From: Kai MacTane [mailto:[EMAIL PROTECTED]] Sent: Friday, October 13, 2000 1:07 PM To: [EMAIL PROTECTED] Subject: Re: A bug or am I being daft? At 10/13/00 10:47 AM , Ben Cody Houston wrote: >Basically, I'm trying to deliver mail to: > >[EMAIL PROTECTED] > >so, I have an ~alias/.qmail-Bob:Hanson file... but it won't work. It will >work if it's ~alias/.qmail-bob:hanson - should either one work? No, only the latter one should work. See http://cr.yp.to/qmail/faq/incominguser.html#alias-dots. As it says, "Dots are converted to colons, and uppercase is converted to lowercase." --Kai MacTane -- "Uh-oh... Gravity works." --Batty Koda, "Ferngully" (Hey, bats are gothic.)
RE: statistics
Hrm, I grabbed qmail analog, I piped one of my logfiles through tai64nfrac and into zoverall, and I get 0 for completed requests and total delivery attempts. If I put it through any other program that came with qmail analog, I get no output. Any idea why this would happen? tai64nfrac works just fine too. Jay -Original Message- From: Peter Green [mailto:[EMAIL PROTECTED]] Sent: Friday, October 13, 2000 10:15 AM To: Andy Abshagen Cc: Qmail Mailing List Subject: Re: statistics also sprach andy: > Where are the MRTG pieces for qmail? What Does The Archive Say(TM)? (See <http://www-archive.ornl.gov:8000/>. Search on qmail-mrtg.) (Hint: http://www.prodigysolutions.com/qmail-mrtg.1.0.tar.gz :) /pg -- Peter Green : Gospel Communications Network, SysAdmin : [EMAIL PROTECTED] --- /* * [...] Note that 120 sec is defined in the protocol as the maximum * possible RTT. I guess we'll have to use something other than TCP * to talk to the University of Mars. * PAWS allows us longer timeouts and large windows, so once implemented * ftp to mars will work nicely. */ (from /usr/src/linux/net/inet/tcp.c, concerning RTT [round trip time])
statistics
Does anyone know of any log analyzers for qmail? I need stats on how many messages are going out, how many of those are bouncing, speed, and whatever else I can find. What about something that would offer real-time stats, like number of concurrent connections, messages waiting in queue, current sending rate, etc.? -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> http://cbs.marketwatch.com http://www.bigcharts.com
RE: smtp speed limit??
Woohoo! Figured it out. The stupid windows mailserver didn't log anything useful, so I took one of the messages and used 'nc' to pipe it into qmail. The message was autogenerate by some .asp file somewhere and had a stray in it. Jay -Original Message- From: Aaron L. Meehan [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 12, 2000 11:54 AM To: '[EMAIL PROTECTED]' Subject: Re: smtp speed limit?? Quoting Austad, Jay ([EMAIL PROTECTED]): > I'm looking here: > http://cr.yp.to/docs/smtplf.html > > Maybe one of the messages contains an not preceded by a . Where > would this be logged on the qmail box? Unfortunately, in the stock qmail, nowhere. However, you may start up your favorite packet capturing software, sniffit or its equivilent, and see what is going on between the two servers easily enough. I've never used it, but there is a djb piece of software known as recordio that you can use to log every smtp command sent or received by qmail-smtpd. Again, though, if this is the problem the exchange server should be logging qmail's smtp response! If there's a stray newline, qmail-smtpd will spit out a 4xx error code and the URL quoted above. Aaron
RE: smtp speed limit??
I'm looking here: http://cr.yp.to/docs/smtplf.html Maybe one of the messages contains an not preceded by a . Where would this be logged on the qmail box? -Original Message- From: Andy Bradford [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 6:31 PM To: Chris Johnson Cc: '[EMAIL PROTECTED]' Subject: Re: smtp speed limit?? On Wed, 11 Oct 2000 19:12:42 EDT, Chris Johnson wrote: > If I were troubleshooting a communication problem between two hosts, and on one > end of the connection was some piece of Microsoft software, and on the other > end of the connection was some software written by Dan Bernstein, can you guess > where I'd start looking for the problem? I know where I would look, but just because it is DJB's software doesn't mean that it was configured properly. :-) Andy
RE: smtp speed limit??
It's the builtin MS smtp server that come with win2000 internet services. I couldn't find logs for it, but I forgot to look in the stupid Event viewer. Here's what I just found: "Message delivery to the remote domain 'marketwatchmail.com' failed. The error message is 'The connection was dropped by the remote host.'." Any ideas why it would drop the connection after only receiving one message? I couldn't find any errors on the qmail box. Jay -Original Message- From: Aaron L. Meehan [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 5:37 PM To: '[EMAIL PROTECTED]' Subject: Re: smtp speed limit?? Quoting Austad, Jay ([EMAIL PROTECTED]): > We have a little script that sends 5 different messages in rapid succession > to one email address. When it gets pointed at our mailserver running MS > Exchange, it works fine, however, when pointed at the qmail box, only the > first message makes it through, the other 4 get delayed or deffered and put > back into the queue (MS Exchange or some MS mailserver thing). Fascinating. I wonder if your exchange server bothers to log the response from the remote mail server upon delivery, failure, or deferral? That would help (and even more to show them to us -- I think that shall be my mantra from now on). > Does qmail limit how fast one host can send messages via smtp? > > Here's my tcpserver line: > tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u > 503 -g 503 0 smtp /var/qmail/bin/qmail-smtpd 2>&1 | /var/qmail/bin/splogger > smtpd 3 & Since tcpserver is handling the smtp connection, the proper question would be whether tcpserver itself can limit how fast it will receive mail. The answer to the question as you put it is "no." However, tcpserver can limit the number of smtp connections it will accept. The default is 40 simultaneous connections, and you have actually specified 500. You would not seem to be crossing that threshold, but I'm just guessing without seeing any logs--tcpserver's logging will show if you're going over 500 connections. Personally, my bet is that the exchange server is puking. Five messages in rapid succession.. that must put real stress on the bloatware. Aaron
smtp speed limit??
We have a little script that sends 5 different messages in rapid succession to one email address. When it gets pointed at our mailserver running MS Exchange, it works fine, however, when pointed at the qmail box, only the first message makes it through, the other 4 get delayed or deffered and put back into the queue (MS Exchange or some MS mailserver thing). Does qmail limit how fast one host can send messages via smtp? Here's my tcpserver line: tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u 503 -g 503 0 smtp /var/qmail/bin/qmail-smtpd 2>&1 | /var/qmail/bin/splogger smtpd 3 & -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> http://cbs.marketwatch.com http://www.bigcharts.com
RE: concurrencyremote
I had the same problem for awhile at around 300 something concurrency. In /proc/sys/fs, you need to increase the values of some stuff in there. I just doubled the value of everything until it worked. :) Jay -Original Message- From: Ricardo Albano [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 3:17 PM To: Doug Schmidt; 'Kris Kelley'; QMail Mailing List Subject: Re: concurrencyremote This is probably an OS limit, I think has relation with the max. file descriptor that the system can handle, try setting a line line 'ulimit -n 65000' in the script that lunch the qmail-smtpd process. What OS/Version are you running ? RDA.- -Original Message- From: Doug Schmidt <[EMAIL PROTECTED]> To: 'Kris Kelley' <[EMAIL PROTECTED]>; QMail Mailing List <[EMAIL PROTECTED]> Date: Wednesday, October 11, 2000 4:44 PM Subject: RE: concurrencyremote >Yep, I found the problem right after sending the message to the list. >control/* are owned by root:root w/ 644 permissions. When I originally >created the file as root, because of umask, permissions were 640 >all is working fine now w/ concurrencyremote at 30. When I make it 40, >I get messages in my maillog that say: > >Oct 11 11:24:18 server qmail: 971277858.753401 delivery 1078: deferral: >qmail-spawn_unable_to_create_pipe._(#4.3.0) > >Is this becuase the server is running out of memory? > >~Doug > >-Original Message- >From: Kris Kelley [mailto:[EMAIL PROTECTED]] >Sent: Wednesday, October 11, 2000 11:29 AM >To: QMail Mailing List >Subject: Re: concurrencyremote > > >Doug Schmidt wrote: >> I would like to increase qmail's concurrencyremote from the default 20 to >> 40. When I create: >> /var/qmail/control/concurrencyremote >> and put a value of 40 in the file, >> I restart qmail and get the error: >> Oct 10 16:53:14 server qmail: 971211194.211356 alert: cannot start: unable >> to read controls > >Could be a matter of permissions. What are the permissions of your control >directory and the control files now? On my machine, the control directory >is owned by root:qmail with 755 permissions, and all the files within are >owned by root:root with 644 permissions. > >---Kris Kelley
RE: Running Multiple Copies of Qmail on the same server...
The queues aren't bad now, we're pretty good about prompty removing any addresses that are bad, and total garbage emails don't even get subscribed to the list. Doesn't really matter if we lose a queue. I did play with memory filesystems a couple of months ago, and I got worse performance on that than I did on the 30GB IDE drive on the machine! I think I may have figured out a way to distribute the queue across multiple servers. I just have to figure out if ezmlm would still be able to handle bounces OK. Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 8:43 PM To: Qmail Subject: Re: Running Multiple Copies of Qmail on the same server... On Tue, Oct 10, 2000 at 08:08:03PM -0500, Austad, Jay wrote: > I'm going to have to put some QMQP servers in our other data centers to get > more speed out of it. We've got 90Mbit/sec worth of bandwidth at each one > of those, but it'll burst higher. > > I noticed that when the messages are broken up into 20,000 rcpt chunks, they > go out way faster than a bunch of 150,000 rcpt chunks. At our current > growth rate, we expect over 5,000,000 users per list during the first part > of 2001. I just have to figure out how to increase the disk IO performance > or reduce the need for it on the list server box. How big are your queues? (in disk space terms)? Can you afford to lose a queue occasionally? Can you afford a ram disk? Just as an exercise you might want to run a queue as a memory file system for a little while, just to see the sort of benefit you could gain. Regards.
RE: Running Multiple Copies of Qmail on the same server...
I'm going to have to put some QMQP servers in our other data centers to get more speed out of it. We've got 90Mbit/sec worth of bandwidth at each one of those, but it'll burst higher. I noticed that when the messages are broken up into 20,000 rcpt chunks, they go out way faster than a bunch of 150,000 rcpt chunks. At our current growth rate, we expect over 5,000,000 users per list during the first part of 2001. I just have to figure out how to increase the disk IO performance or reduce the need for it on the list server box. Jay -Original Message- From: James Stevens [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 8:03 PM To: Austad, Jay; Qmail Subject: Re: Running Multiple Copies of Qmail on the same server... ROFL, ok.. I feel small now ... ... ... ... --JT - Original Message - From: "Austad, Jay" <[EMAIL PROTECTED]> To: "'James Stevens'" <[EMAIL PROTECTED]> Sent: Tuesday, October 10, 2000 5:56 PM Subject: RE: Running Multiple Copies of Qmail on the same server... > Here's what I did: > > I have a main server running qmail and ezmlm. I set up ezmlm to use qmqp to > send it's messages. I modified qmail-qmqmc.c to randomly pick a QMQP server > instead of just choosing the first one. Each list is broken up into 52 > sublists. Right now, I only have 3 QMQP servers. When a message goes out, > it randomly distributes 52 separate messages between 3 QMQP servers, which > ends up being pretty even. All of the sending is offloaded to other > machines, and split up, so the messages go out super fast. 1,000,000 > addresses in under 15 minutes. :) I saturate one of our DS3's everytime it > runs. My modifications can support up to 255 QMQP servers. > > The bounces are starting to load down the main box though, I'm going to have > to lower the ezmlm-warn timeout to about 4 days, it's still at 11.6. > > Jay > > -Original Message- > From: James Stevens [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, October 10, 2000 7:38 PM > To: Goran Blazic; Qmail > Subject: Re: Running Multiple Copies of Qmail on the same server... > > > Well when a list of 200k+ is mailing and another list gets fired up behind > it I would like the server which sits almost twiddling it's thumbs and has > all kinds of resources left over to be able to send out to more than one > list at a time without a queue delay Right now 'Majordomo' funnels all > messages into qmail via qmail-send and anything that gets queued after that > has to wait for the current queue to get done. To get around this I plan on > setting up multiple qmails under the same server each having it's own queue > ofcourse and then each client would have his or her own smtp server for > sending to there lists on. Meaning that with multiple qmails running one > client could send to her or his list and a secound client could send to her > or his list and a third client could send to her or his list and I could > send to my staff list and nowone would experience any delays and maybe just > maybe my server might get above a 5% load avaerage for the first time in > it's life. > > Anyways if anyone has actually done this please point me in the right > direction I'd really appreciate it. ;) > > --JT > - Original Message - > From: "Goran Blazic" <[EMAIL PROTECTED]> > To: "'James Stevens'" <[EMAIL PROTECTED]>; "Qmail" > <[EMAIL PROTECTED]> > Sent: Tuesday, October 10, 2000 5:22 PM > Subject: RE: Running Multiple Copies of Qmail on the same server... > > > > I dont really see no good point on why you would want to run multiple > copies > > of qmail... > > Or what you would understand by that ??!!?? > > > > Goran > > > > -Original Message- > > From: James Stevens [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, October 11, 2000 2:19 AM > > To: Qmail > > Subject: Running Multiple Copies of Qmail on the same server... > > > > > > Can someone point me to a web page that has some explanation of setting up > > concurrent running qmails on the same machine and what edits I need to > make > > to avoid conflicks.. > > > > Thanks in advance.. > > > > --JT > > > > > > >
RE: Running Multiple Copies of Qmail on the same server...
You're better off with RAID 0+1. I run RAID 5 on mine and when it runs through the queue, disk IO is the bottleneck. -Original Message- From: James Stevens [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 7:45 PM To: Michael Boyiazis; Qmail Subject: Re: Running Multiple Copies of Qmail on the same server... Thanks! .. ;) The machine I'm running qmail on has like a full c block assigned to it and a RAID5 array so I don't think the disk i/o will be a problem ?? But if needed I also have a RAID1 Array setup on it with twin 20gigs so I could place the secound qmail over there but I think the RAID5 should be fast enough for it Thoughts? --JT - Original Message - From: "Michael Boyiazis" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, October 10, 2000 5:38 PM Subject: RE: Running Multiple Copies of Qmail on the same server... > We have found inbound mail to be very disk i/o bound > w/o doing much to the cpu. so we added another disk > and have two instances running. it lets us handle twice > the load. > > you need the box to handle 2 IPs; > > for the second instance recompile w/ the value in: > /export/home/qmail-1.03/conf-qmail to hold the > home of the second queue, say /var/qmail2 instead > of the default /var/qmail > > the spot in your tcpserver line that says 0 smtp > should be changed to be: > > mail_instance_1.domain.com smtp > > and repeat the tcpserver startup for another instance. > > make sure to both qmail instances are started in your > init script. > > -- > Michael Boyiazis > Mail Architect, NetZero, Inc. > > > -Original Message- > > From: Goran Blazic [mailto:[EMAIL PROTECTED]] > > Sent: Tuesday, October 10, 2000 5:23 PM > > To: 'James Stevens'; Qmail > > Subject: RE: Running Multiple Copies of Qmail on the same server... > > > > > > I dont really see no good point on why you would want to run > > multiple copies > > of qmail... > > Or what you would understand by that ??!!?? > > > > Goran > > > > -Original Message- > > From: James Stevens [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, October 11, 2000 2:19 AM > > To: Qmail > > Subject: Running Multiple Copies of Qmail on the same server... > > > > > > Can someone point me to a web page that has some explanation > > of setting up > > concurrent running qmails on the same machine and what edits > > I need to make > > to avoid conflicks.. > > > > Thanks in advance.. > > > > --JT > > > > >
RE: qmail list reply-to
>Considering that the majority of Internet users these days are so young that >the have never seen carbon paper, Regardless of age, if you were a trouble maker in school, you got quickly acquainted with it. In 4th grade, I remember having to write 5000 times: "When singing Old Dan Tucker, I will not call him Old D*mn F*cker". Carbon paper saved me about 4000 sentences. My parents were not pleased. -Original Message- From: Chris Garrigues [mailto:[EMAIL PROTECTED]] Sent: Monday, October 09, 2000 1:53 PM To: Brett Randall Cc: Robin S. Socha; qmail Subject: Re: qmail list reply-to > From: "Brett Randall" <[EMAIL PROTECTED]> > Date: Mon, 9 Oct 2000 07:00:06 +1000 > > > Then the sender should ask for a Cc: - remember kids, it isn't called > > Courtesy Copy for nothing. > > I thought it was Carbon Copy? Considering that the majority of Internet users these days are so young that the have never seen carbon paper, that term seems to be as obsolete as "dialing" a telephone. At Stan Freburg said, "That went out with button shoes!" Chris -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 4314 Avenue C Austin, TX 78751-3709 +1 512 374 0500 My email address is an experiment in SPAM elimination. For an explanation of what we're doing, see http://www.DeepEddy.Com/tms.html Nobody ever got fired for buying Microsoft, but they could get fired for relying on Microsoft.
RE: spam alarm as result from "help with girlfriend"
I got one of those too. Lily is not cute. :) -Original Message- From: Don Wright [mailto:[EMAIL PROTECTED]] Sent: Sunday, October 08, 2000 1:27 PM To: Qmail Subject: RE: spam alarm as result from "help with girlfriend" > From: Alexander Jernejcic > > i think "Wheres Mybrudda [mailto:[EMAIL PROTECTED]]" did > gather adresses for spam-mail! > just received spam from "From Lily [[EMAIL PROTECTED]]" It doesn't have to be from that hotmail account, it could be any lurker on the list, or it could be from the maillist to newsgroup mirror at newsgate.muc.de. Check news:muc.lists.qmail. (Bet you didn't know you were a USENET star.) Here are the headers from your message, as posted to the newsgroup. More than enough opportunity for anonymous harvesting. Note: Giganews is my newsfeed provider. Path: news3.aus1.giganews.com!nntp3.aus1.giganews.com!nntp.giganews.com!feed1.news .rcn.net!rcn!news.maxwell.syr.edu!tank.news.pipex.net!pipex!do.de.uu.net!inf ormatik.tu-muenchen.de!news.muc.de!newsgate.muc.de!not-for-mail From: [EMAIL PROTECTED] (Alexander Jernejcic) Newsgroups: muc.lists.qmail Subject: spam alarm as result from "help with girlfriend" Date: 8 Oct 2000 12:08:55 +0200 Organization: Newsgate at muc.de e.V. Lines: 22 Approved: [EMAIL PROTECTED] Distribution: world Message-ID: <[EMAIL PROTECTED]> Reply-To: [EMAIL PROTECTED] NNTP-Posting-Host: marvin.muc.de Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Trace: marvin.muc.de 970999735 12097 193.149.48.2 (8 Oct 2000 10:08:55 GMT) X-Complaints-To: [EMAIL PROTECTED] NNTP-Posting-Date: 8 Oct 2000 10:08:55 GMT Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm Precedence: bulk X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Importance: Normal X-Newsgate-CVS-Id: $Id: newsgate.pl,v 1.2 1998/10/22 13:23:53 mort Exp $ X-No-Archive: Yes Xref: nntp3.aus1.giganews.com muc.lists.qmail:39178
RE: check password
I don't know. But, if it's not currently possible, you could always hack RADIUS support into vpopmail. Then just set up a RADIUS server on your Domain controller for authentication. With some of the RADIUS libraries, it doesn't look that hard. Just a thought, I'm sure someone has a better way. Jay -Original Message-From: Stano Pa¹ka [mailto:[EMAIL PROTECTED]]Sent: Friday, October 06, 2000 12:03 AMTo: qmail konferenciaSubject: Re: check password No one knows? No one answers... Stano. - Original Message - From: Stano Pa¹ka To: qmail konferencia Sent: Monday, October 02, 2000 7:03 AM Subject: check password Can I use users/passwords from WIN NT server for qmail/vpopmail? And how? Stano.
RE: To send 500,000 messages
I also had to use the big concurrency patch. I do a single message with 150,000 recipients. Our lists range from 80,000 people to around 200,000 people depending on the list. I have multiple QMQP servers that do the sending and the main box load balances between the QMQP servers. So if I break the messages into smaller groups of recipients, it will most likely go out faster since I have more than one server doing the sending. I haven't done this yet, but I'll eventually be setting up sublists in my ezmlm setup to do just this. All of our newsletters are stock market/financial related, so the ones that go out during the day are time sensitive. Jay -Original Message- From: Henrique Pantarotto [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 8:00 PM To: Austad, Jay; qmail Subject: RE: To send 500,000 messages Jay, I really loved this VERH patch. That's exactly what I need, to create a custom "To" field and perhaps a "click here to unsubscribe" field. To send 150,000 messages, do you send a single message with all the recipients or do you send multiple messages (multiple queues)? And another dumb question: did you apply any other patch to get concurrency level of 509 or you did that with Qmail as-is? Thanks a lot! ___ Henrique Pantarotto SysOp Site São Paulo Terra Networks Brasil S/A A Internet mais sua do que nunca Tel: (11) 5505-5728 r.316/238 ICQ: 6934285 IT: henpa [EMAIL PROTECTED] -Original Message- From: Austad, Jay [mailto:[EMAIL PROTECTED]] Sent: quinta-feira, 5 de outubro de 2000 21:33 To: 'Brett Randall'; Henrique Pantarotto; qmail Subject: RE: To send 500,000 messages How personalized are they? Are you just putting the persons email address in the body of the message? the VERH patch for qmail will let you do this. We use this patch and a concurrency level of 509 and it puts out about 150,000 messages every 15 minutes. I run DJB dns with 100MB memory reserved for the cache so the mailservers won't have to wait long for queries. Overall, I'm really impressed with how fast qmail and djbdns work. I can saturate one of our DS3's by just sending out mail. :) Jay -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 7:48 PM To: Henrique Pantarotto; qmail Subject: RE: To send 500,000 messages > 1) I have searched this mailing-list and I understood that Qmail can > handle this demand very nicely, right? Are those patches for "high > load servers" really necessary or is the default qmail distribution > (with concurrencyremote set to 100 or something) enough? They're there for a reason...of course, these presume that your link can handle a greater concurrency. > 2) Whats the best way to send the 500,000 mails? Send a single > message with 500,000 destination addresses or to send 500,000 > messages, one for each email address? One message with 500,000 destination addresses. Could you imagine opening up 500,000 smtp connections to a single server? Uh uh. > 3) If I really need to send 500,000 different messages (for custom > text inside the message) to the users, will this stress Qmail or will > it handle it okay? qmail won't have a problem. But how big is your server? Your server will likely cack it if you send 500,000 messages from another machine. My best advice? Generate a template with metatags in it (like , ), have a database ready, and run it through a perl script that calls qmail-inject. If you don't need to customise messages, consider using ezmlm-idx. It will remove addresses that bounce after 14 days if they can't be resolved, it is easy to administer, users can subscribe and unsubscribe at whim, it can me moderated so messages can't be sent to the mailing list unless you authorise them to be... /BR Manager InterPlanetary Solutions http://ipsware.com/
RE: To send 500,000 messages
How personalized are they? Are you just putting the persons email address in the body of the message? the VERH patch for qmail will let you do this. We use this patch and a concurrency level of 509 and it puts out about 150,000 messages every 15 minutes. I run DJB dns with 100MB memory reserved for the cache so the mailservers won't have to wait long for queries. Overall, I'm really impressed with how fast qmail and djbdns work. I can saturate one of our DS3's by just sending out mail. :) Jay -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 7:48 PM To: Henrique Pantarotto; qmail Subject: RE: To send 500,000 messages > 1) I have searched this mailing-list and I understood that Qmail can > handle this demand very nicely, right? Are those patches for "high > load servers" really necessary or is the default qmail distribution > (with concurrencyremote set to 100 or something) enough? They're there for a reason...of course, these presume that your link can handle a greater concurrency. > 2) Whats the best way to send the 500,000 mails? Send a single > message with 500,000 destination addresses or to send 500,000 > messages, one for each email address? One message with 500,000 destination addresses. Could you imagine opening up 500,000 smtp connections to a single server? Uh uh. > 3) If I really need to send 500,000 different messages (for custom > text inside the message) to the users, will this stress Qmail or will > it handle it okay? qmail won't have a problem. But how big is your server? Your server will likely cack it if you send 500,000 messages from another machine. My best advice? Generate a template with metatags in it (like , ), have a database ready, and run it through a perl script that calls qmail-inject. If you don't need to customise messages, consider using ezmlm-idx. It will remove addresses that bounce after 14 days if they can't be resolved, it is easy to administer, users can subscribe and unsubscribe at whim, it can me moderated so messages can't be sent to the mailing list unless you authorise them to be... /BR Manager InterPlanetary Solutions http://ipsware.com/
RE: Help with my girlfriend?
If she's just not paying attention to you, you should probably remove /dev/vbrtr0 and /dev/othrbf0. -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 5:46 PM To: qmail Subject: Re: Help with my girlfriend? On Fri, Oct 06, 2000 at 09:35:06AM +1000, Brett Randall wrote: > > strace /dev/gf0 > > No, I think you've got it wrong. I think its strace /dev/gf6 at the > moment... That depends on your interface renumbering standards. Rebooting in between relations is a good thing. Greetz, Peter -- dataloss networks '/ignore-ance is bliss' - me
RE: Remotely subscribing multiple addresses
A solution someone gave me awhile back was to save the emails to a file, one per line, and do: cat emails.txt | xargs --max-args 20 ezmlm-sub ~/ Worked great. -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Wednesday, September 20, 2000 8:28 PM To: Ben Beuchler; qmail Subject: RE: Remotely subscribing multiple addresses > Is there any way for a remote admin to subscribe multiple addresses in > one request? I'm guessing not, but it sure would be nice... A work-around way, but set up your own admin user (like multadmin), that has a .qmail file which pipes to a script you made up, which checks that the address it is coming from is an administrator's address, and then feeds the contents of the message line-by-line to ezmlm-sub. You might want to use some method of authentication in the message just so people don't learn about your multiple-subscription method and abuse it via a simple e-mail (could be as simple as your favourite word, or could be as complex and safe as public key encryption). /BR Manager InterPlanetary Solutions http://ipsware.com/
RE: concurrency remote patch
RAID 5 sucks for writes in the first place, but is excellent for reads. I'm running raid 5 on my mailing list box for availability reasons, but that distributes to my qmqp servers which are all RAID 0. I don't care that much if I lose a drive on the QMQP servers since I can have a new one built in about 15 minutes. I'll just lose my queue, which is only newsletter subscriptions anyway. It would suck to lose the queue, but it's not mission critical and the chances of it happening are low. Although, I'm sure it will happen sometime. Jay -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 2:30 PM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch On Wed, Sep 20, 2000 at 04:19:30AM +0900, James T. Perry wrote: [snip] > > I had both of my QMQP servers bouncing off of the 120 limit > > yesterday, and they were pretty much idle (Dell 2450's with > > 2 striped 9GB 10k rpm drives). > > Which RAID level? > I remember somebody mentioning in this list that 0+1 will perform > faster than 3 (or 5 obviously ;). > I can't confirm this since I don't have that kind of artillery > here at home. Anybody? I know that RAID5 sucks on Mylex DAC1100 controllers, and that RAID0+1 is blindingly fast :) Greetz, Peter -- dataloss networks '/ignore-ance is bliss' - me
RE: qmail error
Make sure your files in /var/qmail/control have the correct settings in them. I ran into this yesterday and it turned out I had bad info in defaultdomain, locals, me, and plusdomain. Jay -Original Message- From: Jens Georg [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 2:56 PM To: qmail mailinglist Subject: qmail error hi, while sending emails to AOL qmail reports the following error-messages: Remote host said: 501 syntactically invalid HELO argument(s) Remote host said: 501 HELO requires domain address i did not find anything about this in the faqs. somebody here who can help me ? regards, jens
log analyzers
So what do most people consider the best log analyzer for qmail logs (I'm using multilog)? I'd like to see real-time stats if possible, or at least near realtime... :) Jay ------ Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: concurrency remote patch
>I haven't reached a concurrency greater than 100 (*blush*) yet so >I can't say what would exactly happen when the concurrency really >hits a high number - above the real 1024 limit (or 509 in qmail). I had both of my QMQP servers bouncing off of the 120 limit yesterday, and they were pretty much idle (Dell 2450's with 2 striped 9GB 10k rpm drives). I think even if I could get the concurrency up to 1024 or above, it still wouldn't be enough to make a difference on the box. I'll find out soon if I can make it bounce off of the 509 limit. Our Midday Market Report is due to go out within the hour. Hopefully when the next version of qmail comes out, it will have the big-concurrency and big-todo patch already installed. What happens if I start a second copy of qmail using /var/qmail2, different uids, and bind to another IP on the same box? Will I be able to do 509 concurrency out of each copy since they are running as different users? Jay -Original Message- From: James T. Perry [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 11:41 AM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch Hi Jay, "Austad, Jay" wrote: > > Here's what I did to rebuild the rpm: [snip] Thanks for the information! I gotta get used to building RPMs... (after all, I am using an RPM distro ;) > As for the FD_SET problem, Dell sucks and ships a RAID card > that requires a proprietary driver on their so-called "Linux > approved" servers. It's a pain to recompile the kernel with > any modifications because that damn module they have might > not work. Everyone keeps pushing them to just release the > source so it can be incorporated into the kernel, but they're > being stupid about it. I know _exactly_ what you mean. (see RANT below) I guess I wasn't clear on the info I had previously posted. I didn't recompile the kernel... I just modified the sources to "goose" the qmail compile process and it somehow worked (call me kraziej :). I haven't reached a concurrency greater than 100 (*blush*) yet so I can't say what would exactly happen when the concurrency really hits a high number - above the real 1024 limit (or 509 in qmail). As for performance, my IDE ATA disk is slower than what qmail can really handle so setting the concurrency below 500 may not be a problem after all, now that I think of it... And procmail + /var/spool/mail is another "wide-load" I have which affects performance compared to Maildir. If I were to have a RAID 0+1 spinning above 1rpms, maybe a different story ( read smokin' gun :) cheers, jamie Recently, I am getting more annoyed with big corporations leeching off on all of the efforts the open-source spirit has built up in the past years, giving the community not much good publicity nor credit in return either... Steal everything and yet spitting all over us. Sorry to mention it here folks - no flame please :) #-#-#-#-#-#-#-# -- If somebody can help create a search engine for my room, I will call them a Saint... GUI == Graphical User Interference
RE: Problems receiving mail
If you're on a machine on the inside and you're trying to hit an ip on the outside of a firewall or router that does NAT, it won't work. I think Checkpoint makes a firewall that works around this problem, but that's the only one I can think of. You need to do your testing from a remote machine if you're hitting the 200.201.1.1 ip. Is 200.201.1.1 the real ip? I tried connecting to both port 80 and 25 and neither worked. In any case though, you won't be able to hit your external ip's from the internal network if the firewall is NATing them. Jay -Original Message- From: Wagner R. Landgraf [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 7:01 AM To: [EMAIL PROTECTED] Subject: Re: Problems receiving mail I cannot receive mail. Let's say that my internal IP is 192.168.1.20 (local net inside the firewall) and my external IP is 200.201.1.1 . If I do (from another machine in the local network) telnet 192.168.1.20 25 , it works fine, and I can send mail to the mail server using telnet commands. However, if I use telnet 200.201.1.1 25 , it can't connect. I though it was a firewall problem, but look at this: I have an http server running in another machine (192.168.1.2) in the local network. If I do telnet 192.168.1.2 80 it works ok, connecting to the port 80. If I do 200.201.1.1 80, it also can't connect. However, my http server is running and it's ok. So, it maybe not be a problem with firewall. Can someone help me and explain this? Thank you Wagner R. Landgraf Automa Consultoria & Informática Ltda. [EMAIL PROTECTED] - Original Message - From: "Daniel Augusto Fernandes" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Monday, September 18, 2000 5:15 PM Subject: Re: Problems receiving mail > > "Wagner R. Landgraf" wrote: > > > > Ok, now TELNET test is ok (I can receive mail in qmail sending it by > > telnet local connect). However, I cannot send remote-local mail. > > > > My qmail server is under a firewall. I've set the firewall to redirect > > mail packets (port 25) to qmail server. > > > > How are you going to be able to send mail packets to the internet? > > > However, when I send mail to [EMAIL PROTECTED] (my IP address), I > > receive a Undeliverable message, saying that "The recipient name is > > not recognized". > > > > As in the man pages: qmail doesn't send msg to root@*. > > > Can someone help me? > > > > I've also tried to set a POP account in my e-mail client, using my POP > > server as 200.201.34.197 (or even 192.168.1.20, my local IP address), > > but the client cannot connect to server. Any ideas? > > > > Thank you all > > > > Wagner R. Landgraf > > Automa Consultoria & Informática Ltda. > > [EMAIL PROTECTED] > > > -- > > Daniel Augusto Fernandes (DAF tm) [EMAIL PROTECTED] > GCSNethttp://www.gcsnet.com.br/ > > Se você não encontra > o sentido das coisas > é porque este não > se encontra, se cria. >Antoine Saint-Exupéry
RE: concurrency remote patch
Here's what I did to rebuild the rpm: rpm -ivh qmail-1.03-16.src.rpm cd /usr/src/redhat/SOURCES tar zxvf qmail-1.03.tar.gz patch -p0 mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 4:01 AM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch Hi Jay, "Austad, Jay" wrote: > > I grabbed the source rpm and just applied the patch to it and > rebuilt it. Works great. Congrats! Please teach me how you did it! > Except, FD_SET is limited to 1024 descriptors. Don't you hate it when that happens? ;) > How do I change this? I assume I can't just echo something into > /proc... I want to be able to do more than 509 concurrency. I did succeed to raise the concurrency level to 1000, but its an unofficial "dirty hack" and I can't be liable for any damages but here's what I did: (BTW, I am thinking of writing up a mini-howto) This was originally posted to the qmail list at about Mon, 18 Sep 2000 20:36:09 +0900 with the title : Re: conf-spawn and FD_SET SUCCESS! but updated a little since I forgot to include some more info. Warning/Disclaimer: This worked for me. I am not responsible if something breaks, or causes any damage to everything or anything related to the person following the steps below to modify their system and/or qmail setup (or whatever...). This was originally posted to the qmail list at about Mon, 18 Sep 2000 20:36:09 +0900 with the title : Re: conf-spawn and FD_SET SUCCESS! but updated since I forgot to include some more info. linux kernel 2.2.17 qmail-1.03 procmail rc file (install was accomplished by closely following lwq :) plus localtime, DNS, big-todo, big-concurrency patches in that order changes in qmail configuration: $qmailhome/control/concurrencylocal 500 $qmailhome/control/concurrencyremote 500 $qmailsrc/conf-spawn set to 1000 $qmailsrc/conf-split set to 100 $qmailsrc/conf-cc set to cc -O2 -D__FD_SETSIZE=2048 ucspi-tcp-0.88 daemontools-0.70 1. Inside the big-concurrency patch: it said to modify "/usr/src/linux/include/linux/tasks.h NR_TASKS from 512 to e.g. 2048" - directly quoted - which I did. 2. As P.Y. Adi Prasaja mentioned: raised the __FD_SETSIZE value in /usr/include/bits/types.h from 1024 to 2048 3. Just to be sure: raised the __FD_SETSIZE value in /usr/src/linux/include/linux/posix_types.h from 1024 to 2048 4. As Peter van Dijk mentioned: added -D__FD_SETSIZE=2048 to $qmailsrc/conf-cc 5. Edit the following: $qmailhome/control/concurrencylocal 500 $qmailhome/control/concurrencyremote 500 $qmailsrc/conf-spawn set to 1000 $qmailsrc/conf-split set to 100 $qmailsrc/conf-cc set to cc -O2 -D__FD_SETSIZE=2048 6. make setup: bingo! no compile errors. qmail is idling ok. 7. Change sources back to default value: (incase I break something ;) /usr/include/bits/types.h /usr/src/linux/include/linux/posix_types.h both back to 1024 #-#-#-#-#-#-#-# -- If somebody can help create a search engine for my room, I will call them a Saint... GUI == Graphical User Interference
RE: concurrency remote patch
I grabbed the source rpm and just applied the patch to it and rebuilt it. Works great. Except, FD_SET is limited to 1024 descriptors. How do I change this? I assume I can't just echo something into /proc... I want to be able to do more than 509 concurrency. Jay -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Monday, September 18, 2000 7:11 PM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch On Mon, Sep 18, 2000 at 07:06:26PM -0500, Austad, Jay wrote: > Unfortunately, I installed the boxes that need the modification with an rpm. > I suppose this will possibly screw things up if I use some binaries from a > tarball and some from the possibly modified rpm version. In that case, don't take any chances but rebuild the rpm's from the SRPMS, including the patch along the way. My rpm knowledge ends with that this is possible. Sorry :) Greetz, Peter -- dataloss networks '/ignore-ance is bliss' - me
RE: concurrency remote patch
Unfortunately, I installed the boxes that need the modification with an rpm. I suppose this will possibly screw things up if I use some binaries from a tarball and some from the possibly modified rpm version. -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Monday, September 18, 2000 6:58 PM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch On Mon, Sep 18, 2000 at 06:54:41PM -0500, Austad, Jay wrote: > If I use this patch to increase the concurrencyremote limit to 65535, can I > just compile qmail-remote and drop it in place? Or do I have to replace > everything? If you mean the big-concurrency patch, no. It changes the interface between qmail-send on one side, and qmail-lspawn/rspawn on the other side. You need to update at least these 3. I'd go for a - patch, make - stop qmail - make setup check - start qmail Greetz, Peter. -- [ircoper][EMAIL PROTECTED] - Peter van Dijk / Hardbeat [student]Undernet:#groningen/wallops | IRCnet:/#alliance [developer]_ [disbeliever - the world is backwards](__VuurWerk__(--*-
concurrency remote patch
If I use this patch to increase the concurrencyremote limit to 65535, can I just compile qmail-remote and drop it in place? Or do I have to replace everything? Thanks. -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: C API for queueing messages
Title: RE: C API for queueing messages Hi, Replies to your questions below. >I don't know how zmailer does it, but qmail, postfix and sendmail >all use a set*id program to protect their queue. (Does anyone know >whether zmailer does a fork/exec under the covers of the API?) No, it does not do a fork/exec under the covers. From my understanding of the zmailer code, mail_open() it opens a file under an intermediate queue directory called router and returns a FILE pointer for us to write the message. Then a seperate router process picks up the message from there. The advantage here is that there is very little over head in writing a file as opposed a popen("sendmail -i -t") call. >Have you measured that execing qmail-queue is "not very efficient" in >the full context of your system or are you surmising this? Yes, we have done a lot of timing analysis and it was found that in excess of 50% of the processing time is being spent in the code segment of the application that invokes the sendmail exe thro popen(). We also explored the possibilities of using fork(), which is faster. But we are afraid that the system might run out of ulimit with all these small processes spawned. So now I understand why qmail does not have an API because otherwise the security is compromised. Thanks for your email. Regards, - Jay On Wed, Sep 13, 2000 at 11:58:04AM -0500, Jay Balakrishna wrote: > Hi All, > We are trying to find out what is the most efficient way of queueing the > message in qmail from our C Programs. So, would like to know if there are > any C API's that are available for queuing messages. Basically I am looking > for library routines that act as the native submission interface(API) for > qmail. I would like to use the API's from my C Program that we currently > have where we compose and build the messages and to queue the email message > directly. Something similar to mail_open() and mail_close() provided by > zmailer where you can submit messages composed in the MSG_RFC822 format. > I can always call sendmail wrapper for queue-Inject or queue-inject itself > from my C program thro a popen() or system() call, but I want to avoid the > overhead of starting a new UNIX process for each and every email that we > send and also popen() is not very efficient. > Any help will be appreciated. Any other ideas are also most welcome > Thanks and Regards, > - Jay >
C API for queueing messages
Title: C API for queueing messages Hi All, We are trying to find out what is the most efficient way of queueing the message in qmail from our C Programs. So, would like to know if there are any C API's that are available for queuing messages. Basically I am looking for library routines that act as the native submission interface(API) for qmail. I would like to use the API's from my C Program that we currently have where we compose and build the messages and to queue the email message directly. Something similar to mail_open() and mail_close() provided by zmailer where you can submit messages composed in the MSG_RFC822 format. I can always call sendmail wrapper for queue-Inject or queue-inject itself from my C program thro a popen() or system() call, but I want to avoid the overhead of starting a new UNIX process for each and every email that we send and also popen() is not very efficient. Any help will be appreciated. Any other ideas are also most welcome Thanks and Regards, - Jay
RE: Spamming .....
There is no way to send to 100k recipients at a time. Most spammers hire minimum wage workers to cut and paste messages into Outlook Express and send them to each recipient. You can always point your Outlook Express client at your qmail machine, but it's more efficient to just point it at the mailserver of your ISP. Always put your real email address in the messages. Most spammers make the mistake of putting fake ones and all of the potential customers can't respond. Usually a phone number is good also since many people prefer not to use email for business purposes. -Original Message- From: Rick Harris [mailto:[EMAIL PROTECTED]] Sent: Monday, September 11, 2000 8:55 PM To: Qmail Subject: RE: Spamming . Perhaps you should watch the list before you wish to declare yourself an evil spammer :) , but perhaps some other list members might have a viewpoint ? Rick -Original Message- From: Jerry Hsieh [mailto:[EMAIL PROTECTED]] Sent: Monday, September 11, 2000 8:41 PM To: Qmail Subject: Spamming . Hi, I would like to get into the spamming business and I have a basic qmail server setup already. I have no idea how to send mail to 100k receiptents at one time. Can someone give me some hints? Thanks for your time. Regards, Jerry
RE: QMTP
I understand it fine. I just have to get as much info as possible for our developers so they can code it up. Just want to make sure they don't have to do a ton of debugging due some subtle mistake somewhere. Jay -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 05, 2000 12:51 PM To: '[EMAIL PROTECTED]' Subject: Re: QMTP On Tue, Sep 05, 2000 at 10:35:08AM -0500, Austad, Jay wrote: > Where would I find detailed specs on the QMTP protocol? I've found some > stuff at http://cr.yp.to/proto/qmtp.txt, but I need more. What part of that document is unclear to you, or what don't you understand? You might want to read serialqmtp.c (in serialmail) and qmail-qmtpd.c (in qmail) I am glancing over qmtp.txt and it's mostly quite clear to me, except the stuff about 'safe messages'. If none of the bytes in a safe message can be <0a>, when the hell *do* we see a safe message? Looking at serialqmtp, it seems to just *disregard* this problem and pump the message into the packet. Cool, no more CRLF/LF conversions :) If it's netstrings that confuse you, read http://cr.yp.to/proto/netstrings.txt Greetz, Peter -- dataloss networks
QMTP
Where would I find detailed specs on the QMTP protocol? I've found some stuff at http://cr.yp.to/proto/qmtp.txt, but I need more. We're writing a little piece of code that's going to sit on a couple of hundred Win2000 webservers that can talk QMTP to our qmail box for faster delivery. Users come to our site and sign up for automatically generated charts and other such things and the webservers generate the content for the email and send it off using SMTP right now. We need to speed this up as much as possible, and QMTP looks like the best way to do it since we already have a qmail box sending most of our other subscriber mail. We also have several QMQP servers. Would it maybe be better to make it talk QMQP to the boxes running qmail-qmqpd? = Here's my setup below in case anyone is wondering (warning: may be confusing without a picture): I have one main machine that runs ezmlm and a full version of qmail, it also runs a second copy of qmail (actually mini-qmail, we'll get to that in a minute). qmail-qmqpc is modified on the machine to load balance between all of my QMQP servers. I can add QMQP servers as necessary if the load gets to high on the rest of them. The full version of qmail is running to handle bounces, and to accept mail from webservers. Ezmlm-idx is set up to use QMQP for delivery, so messages being sent out get distributed to multiple QMQP servers. If the email comes in from a webserver, box running ezmlm would try to send it off itself... I didn't want that to happen because I want to keep the disk activity and other resources down as much as possible, so the logical choice would be to offload all mail not destined for the local box to the QMQP servers. Since I couldn't figure out a better way to do it, I installed a copy of mini-qmail with qmail-smtpd listening on port 26, and my modified qmail-qmqpc. smtproutes is set up so all outgoing mail goes to mini-qmail (on port 26) which distributes it to the QMQP servers. I wish smtproutes would let you specify a protocol to use... :) So the way this is set up, all outgoing mail gets sent to QMQP servers to let them handle the queueing and sending. Main destined for the local machine is still accepted and processed normally. Right now, I have 2 bottlenecks... The SMTP connection that the webservers use to send mail through it, and the SMTP connection to port 26 that the main copy of qmail redirects outgoing mail to. -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: queueing and using qmqp at the same time
I don't want it to go to my smarthost, I want it to use qmail-qmqpc to send it to my QMQP servers. If only I could specify a protocol in smtproutes... Otherwise I could run "mini-qmail" on the same box with an smtp server listening on port 26, and use smtproutes to redirect outgoing mail to localhost:26. That way it would use my qmail-qmqpc to send it off to the QMQP servers. I want to use qmail-qmqpc because I modified to load balance between multiple servers, and I don't want to bog down my list server box with queueing outgoing mail. It's going to be ALOT of outgoing mail... Jay -Original Message- From: Charles Cazabon [mailto:[EMAIL PROTECTED]] Sent: Friday, August 25, 2000 9:43 AM To: '[EMAIL PROTECTED]' Subject: Re: queueing and using qmqp at the same time Chris Johnson <[EMAIL PROTECTED]> wrote: > > On Thu, Aug 24, 2000 at 04:37:41PM -0500, Austad, Jay wrote: > > > I have a mail server that needs to receive mail for local users and handle > > > bounces. However, I would like all outgoing mail to use my QMQP servers > > > instead of being queued locally. Is this possible? > > > No. > > You might look at Bruce Guenter's nullmailer, http://em.ca/~bruceg/nullmailer/ > I've never used it, but it looks like it might do what you need. nullmailer doesn't provide an SMTP daemon for incoming mail; it is only for sending locally-injected mail from a dumb host to a smart relay. However, it's excellent for that. However, smtproutes could be used with qmail to provide this functionality, I think. Set up /var/qmail/control/locals to handle local deliveries, and put the single line :smarthost.domain into /var/qmail/control/smtproutes. This should make everything that can't be delivered locally be forwarded to your smarthost. Anyone care to correct me on this? Charles -- -- Charles Cazabon <[EMAIL PROTECTED]> QCC Communications Corporation Saskatoon, SK My opinions do not necessarily represent those of my employer. --
queueing and using qmqp at the same time
I have a mail server that needs to receive mail for local users and handle bounces. However, I would like all outgoing mail to use my QMQP servers instead of being queued locally. Is this possible? -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: linux filesystem
Wasn't there a problem with qmail and reiserfs awhile back? Seems to me qmail had to be slightly modified due to some file locking issues or something... I must've deleted the thread on it though. Jay -Original Message- From: Fucht, Rob van (ELS) [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 24, 2000 9:38 AM To: 'Van Liedekerke Franky'; 'qmail list' Subject: RE: linux filesystem You mean perhaps Reiserfs? (a journalling file system). Rob -Original Message- From: Van Liedekerke Franky [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 24, 2000 3:27 PM To: 'qmail list' Subject: linux filesystem A while back there was talk about a good filesystem for linux (next to ext2fs). Anybody remebers how this is called, and does anybody use it with satisfaction? I have looked through the archives, but I'm just not able to fill in the right search terms to find the answer... Franky
RE: Cool powered by qmail logo.
If anyone is interested in printing up a bunch of these, my friend works for a place here in Minneapolis called Signature Concepts and he gets a hefty discount (they do all of the University of MN stuff). We printed up some shirts for the DSM Racing club (http://www.dsm.org) and it cost us around $9 for a short sleeve and $12 for a long sleeve shirt. Good quality too, heavy cotton. Drop me an email if you're interested and I'll get you in touch with him. If you can send a JPG of what you want printed on the shirts, that would be even better. The more colors, the higher the cost. We had 2 colors on our shirts. The turn around time is usually pretty quick (a week or two depending on the season), so we had people send their money first so we knew how many to print up. Jay -Original Message- From: Russell Nelson [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 10, 2000 2:38 PM To: qmail Subject: Re: Cool powered by qmail logo. Keith Warno \(@HaggleWare.com\) writes: > Whatever happened to the qmail shirt idea? Last I knew most people agreed > "don't queue mail with sendmail; send mail with qmail" was the slogan of > choice, but where'd it go from there? Remember this posting? Russ Nelson writes: >Vern Hart writes: > > > http://www.nerdgear.com/search.php?@category=100 > > > > Those prices at nerdgear are pretty good. Especially for > > embroidery. Even with shipping. > >The extra-large is $18.18 with shipping. I'll let the list know if >the shirts don't suck. They don't suck. I've already worn it to a customer's site. :) -- -russ nelson <[EMAIL PROTECTED]> http://russnelson.com | If you think Crynwr sells support for free software | PGPok | health care is expensive now 521 Pleasant Valley Rd. | +1 315 268 1925 voice | now, wait until you see Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | what it costs when it's free.
RE: Protection
Put a .forward file in with this evil users new email address. Then all mail sent to them will really get to them. If nothing else, it will get them to unsubscribe from all of the lists. -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Wednesday, August 09, 2000 7:26 AM To: qmail Subject: RE: Protection Set up an automatic revenge flood? Maybe not... :> It depends if it is mailing lists or spam. First start by unsubscribing from REAL mailing lists. If it is spam, change your domain name...I would personally sue the ex user for breaching your 'reasonable use policy' (what? you don't have one? doh!) or at least for ongoing damages since you are now virtually permanently committed to wasting bandwidth on unsolicited e-mails. Only other option is to refuse the e-mails (ie using common spam killing techniques) at the last relay before it is transferred over your link. Brett Manager InterPlanetary Solutions http://ipsware.com/ > -Original Message- > From: Slider [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, August 09, 2000 7:06 PM > To: [EMAIL PROTECTED] > Subject: Protection > > > > > Hi all, > > Please can you help with advise about protecting my mail servers > from one of > my on ex users!! He/She has subscribed to about 30 mailing lists with the > address that falls under my mail service! I am now recieving > about 10 mails > a minute for that user! Removing the maildir and letting them > bounce is not > helping as I thought it would... any other suggestions?? > > Slider >
RE: updated load balancing qmail-qmqpc.c mods
I agree with you, I forgot to mention that, sorry. I didn't have enough Mountain Dew yet. :) -Original Message- From: Michael T. Babcock [mailto:[EMAIL PROTECTED]] Sent: Wednesday, August 02, 2000 10:44 AM To: Austad, Jay; [EMAIL PROTECTED] Subject: Re: updated load balancing qmail-qmqpc.c mods Re-read my point: its unnecessary. I didn't say it wouldn't work. I said the CPU use of doing it this way was unnecessary over a simpler round-robin approach (After picking an initial random server). Note: I think using an array of pointers to server addresses would allow you to do your rotations a lot faster. Think "point quicksort". - Original Message - From: "Austad, Jay" <[EMAIL PROTECTED]> > Even if one server is getting hit multiple times in a row, I don't think it > will matter. I put in 10 servers in my qmqpservers file and some well > placed printf's in qmqpc and ran it 100,000 times several times through. > Each server was getting picked about 10% of the time +/- 1% or so. If we're > sending millions of messages, this +/- 1% isn't going to make a significant > difference.
RE: updated load balancing qmail-qmqpc.c mods
Even if one server is getting hit multiple times in a row, I don't think it will matter. I put in 10 servers in my qmqpservers file and some well placed printf's in qmqpc and ran it 100,000 times several times through. Each server was getting picked about 10% of the time +/- 1% or so. If we're sending millions of messages, this +/- 1% isn't going to make a significant difference. I could just use the following code to shuffle the array instead of just rotating it: void Shuffle(void *array, int n, size_t size) { int f,g; char *temp,*p; temp=malloc(size); p=array; for(f=0;fmailto:[EMAIL PROTECTED]] Sent: Wednesday, August 02, 2000 8:58 AM To: JuanE; [EMAIL PROTECTED] Subject: Re: updated load balancing qmail-qmqpc.c mods Again, its unnecessary and does not add anything to the 'system' of distributing load across servers. Distributing in a true round-robin fashion is sufficient if the servers are of equal quality (and/or drop connections sooner if they are not). True round-robin distribution cannot easily be accomplished (without shared memory or a temporary file) and so picking a random starting point is the closest that can be achieved. Using random servers each time though, especially from a near-true random number source, could cause one server to be hit 3 or 4 (or, technically, an infinite number of) times in a row. There is no need to go to the expense of randomly computing each successive server (for example:) #1 #2 #3 #4 #5 1 3 4 1 2 2 4 1 2 3 3 1 3 4 4 4 1 1 2 2 "1,3,4,1,2" were picked as my random numbers and the number of messages sent out in a round varying ... each server was hit almost evenly: 1 xx 2 x 3 4 x (Disclaimer: My old stats teacher would kill me for this insufficiently long example and lack of mathematical proof) - Original Message - From: "JuanE" <[EMAIL PROTECTED]> > A small modification to what you have done can get rid of the problem of > banging one server. Instead of just sampling a random starting point for > the search, just get a random number between 1 and N (N being the number of > servers) and if that server is down just sample again until you find a > server that's up. This way, you will redistribute the load randomly to all > servers and not just the next one on the list.
RE: updated load balancing qmail-qmqpc.c mods
You are correct. However, since qmail-qmqpc does not continually run, it is not possible to do a non-random round robin (as far as I know). If qmqpc could be turned into a process that ran continuously, it would be possible to do regular round-robin and there would be no need for the random stuff at all, this may also increase performance because it wouldn't have to read /var/qmail/control/qmqpservers every time it started. I haven't looked at it, but does qmail-inject work by just piping it's data into qmail-queue (qmail-queue is a link to qmail-qmqpc in a mini-qmail setup)? If so, couldn't qmail-queue be turned into a named pipe, and qmqpc could be modified slightly to continously run and wait for input on that named pipe? Just a random thought. Of course, I'm not a programmer and I may not have any idea what the hell I'm talking about. :) This approach that I'm using obviously isn't going to spread the load between servers exactly evenly, but it should be close enough when mailing large amounts of mail, unless one qmqp server dies. However, our qmqp servers have port 628 for QMQP monitored closely and I'll get paged within a couple of minutes if it goes down, so I'm not too worried about it. I guess I could redo it so instead of just rotating the array by a random number, it would shuffle it it randomly. That would keep it from banging on the server immediately after the dead one, and give everything an even chance again. Jay -Original Message- From: JuanE [mailto:[EMAIL PROTECTED]] Sent: Tuesday, August 01, 2000 6:57 PM To: [EMAIL PROTECTED] Subject: Re: updated load balancing qmail-qmqpc.c mods Jay, If I understand this correctly, then I think there is a flaw in this type of random round robin. I think I can best describe it with an example. Assume you have 4 servers A, B, C, D, and that server C is down. When your random seed hits server A, B or D (with probability 1/4, respectively) the message will go through fine. When your random seeds hits server C (again with p=1/4), you will realize that it is down and proceed to D. So in effect you will hit servers A and B with probability 1/4 each, and server D with probability 1/2. This is not very good load balancing, because basically the load of the server that's down is inherited by the next server, which is not what you set out to accomplish. With simple, non-random round-robin, when ever you hit server C, you will skip it and go to server D. The next message will go to server A, the next to B, etc... Thus, the probability of hitting server D is 1/3 (as well as A and B). All this rests on my interpretation of what you have done, since I am not familiar with the code. JES Austad, Jay writes: > Ok, I fixed it. It now just rotates the serverindex.pos[] by a random > amount, and then loops through until it finds a good server. rand() is still > seeded with milliseconds from the system clock. I used memcpy() to move the > array around instead of loops to make it more efficient. > > Would it be possible to run qmqpc as some sort of daemon? Then it could > have memory of which server it contacted last and make sure it went to the > next one, instead of having the chance of picking the same one again. > > Jay > > > > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include "substdio.h" > #include "getln.h" > #include "readwrite.h" > #include "exit.h" > #include "stralloc.h" > #include "slurpclose.h" > #include "error.h" > #include "sig.h" > #include "ip.h" > #include "timeoutconn.h" > #include "timeoutread.h" > #include "timeoutwrite.h" > #include "auto_qmail.h" > #include "control.h" > #include "fmt.h" > > #define PORT_QMQP 628 > > void die_success() { _exit(0); } > void die_perm() { _exit(31); } > void nomem() { _exit(51); } > void die_read() { if (errno == error_nomem) nomem(); _exit(54); } > void die_control() { _exit(55); } > void die_socket() { _exit(56); } > void die_home() { _exit(61); } > void die_temp() { _exit(71); } > void die_conn() { _exit(74); } > void die_format() { _exit(91); } > > int lasterror = 55; > int qmqpfd; > > int saferead(fd,buf,len) int fd; char *buf; int len; > { > int r; > r = timeoutread(60,qmqpfd,buf,len); > if (r <= 0) die_conn(); > return r; > } > int safewrite(fd,buf,len) int fd; char *buf; int len; > { > int r; > r = timeoutwrite(60,qmqpfd,buf,len); > if (r <= 0) die_conn(); > return r; > } > > char buf[1024]; > substdio to = SUBSTDIO_FDBUF(safewrite,-1,buf,sizeof
updated load balancing qmail-qmqpc.c mods
Ok, I fixed it. It now just rotates the serverindex.pos[] by a random amount, and then loops through until it finds a good server. rand() is still seeded with milliseconds from the system clock. I used memcpy() to move the array around instead of loops to make it more efficient. Would it be possible to run qmqpc as some sort of daemon? Then it could have memory of which server it contacted last and make sure it went to the next one, instead of having the chance of picking the same one again. Jay #include #include #include #include #include #include #include #include #include #include #include "substdio.h" #include "getln.h" #include "readwrite.h" #include "exit.h" #include "stralloc.h" #include "slurpclose.h" #include "error.h" #include "sig.h" #include "ip.h" #include "timeoutconn.h" #include "timeoutread.h" #include "timeoutwrite.h" #include "auto_qmail.h" #include "control.h" #include "fmt.h" #define PORT_QMQP 628 void die_success() { _exit(0); } void die_perm() { _exit(31); } void nomem() { _exit(51); } void die_read() { if (errno == error_nomem) nomem(); _exit(54); } void die_control() { _exit(55); } void die_socket() { _exit(56); } void die_home() { _exit(61); } void die_temp() { _exit(71); } void die_conn() { _exit(74); } void die_format() { _exit(91); } int lasterror = 55; int qmqpfd; int saferead(fd,buf,len) int fd; char *buf; int len; { int r; r = timeoutread(60,qmqpfd,buf,len); if (r <= 0) die_conn(); return r; } int safewrite(fd,buf,len) int fd; char *buf; int len; { int r; r = timeoutwrite(60,qmqpfd,buf,len); if (r <= 0) die_conn(); return r; } char buf[1024]; substdio to = SUBSTDIO_FDBUF(safewrite,-1,buf,sizeof buf); substdio from = SUBSTDIO_FDBUF(saferead,-1,buf,sizeof buf); substdio envelope = SUBSTDIO_FDBUF(read,1,buf,sizeof buf); /* WARNING: can use only one of these at a time! */ stralloc beforemessage = {0}; stralloc message = {0}; stralloc aftermessage = {0}; char strnum[FMT_ULONG]; stralloc line = {0}; struct sindex { int pos[256]; int len; }; void getmess() { int match; if (slurpclose(0,&message,1024) == -1) die_read(); strnum[fmt_ulong(strnum,(unsigned long) message.len)] = 0; if (!stralloc_copys(&beforemessage,strnum)) nomem(); if (!stralloc_cats(&beforemessage,":")) nomem(); if (!stralloc_copys(&aftermessage,",")) nomem(); if (getln(&envelope,&line,&match,'\0') == -1) die_read(); if (!match) die_format(); if (line.len < 2) die_format(); if (line.s[0] != 'F') die_format(); strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0; if (!stralloc_cats(&aftermessage,strnum)) nomem(); if (!stralloc_cats(&aftermessage,":")) nomem(); if (!stralloc_catb(&aftermessage,line.s + 1,line.len - 2)) nomem(); if (!stralloc_cats(&aftermessage,",")) nomem(); for (;;) { if (getln(&envelope,&line,&match,'\0') == -1) die_read(); if (!match) die_format(); if (line.len < 2) break; if (line.s[0] != 'T') die_format(); strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0; if (!stralloc_cats(&aftermessage,strnum)) nomem(); if (!stralloc_cats(&aftermessage,":")) nomem(); if (!stralloc_catb(&aftermessage,line.s + 1,line.len - 2)) nomem(); if (!stralloc_cats(&aftermessage,",")) nomem(); } } void doit(server) char *server; { struct ip_address ip; char ch; if (!ip_scan(server,&ip)) return; qmqpfd = socket(AF_INET,SOCK_STREAM,0); if (qmqpfd == -1) die_socket(); if (timeoutconn(qmqpfd,&ip,PORT_QMQP,10) != 0) { lasterror = 73; if (errno == error_timeout) lasterror = 72; close(qmqpfd); return; } strnum[fmt_ulong(strnum,(unsigned long) (beforemessage.len + message.len + aftermessage.len))] = 0; substdio_puts(&to,strnum); substdio_puts(&to,":"); substdio_put(&to,beforemessage.s,beforemessage.len); substdio_put(&to,message.s,message.len); substdio_put(&to,aftermessage.s,aftermessage.len); substdio_puts(&to,","); substdio_flush(&to); for (;;) { substdio_get(&from,&ch,1); if (ch == 'K') die_success(); if (ch == 'Z') die_temp(); if (ch == 'D') die_perm(); } } stralloc servers = {0}; main() { int i; int j; int randj; int randarr[256]; struct timeb tp; struct sindex serverindex; sig_pipeignore(); if (chdir(auto_qmail) == -1) die_home(); if (control_init() == -1) die_control(); if (control_readfile(&servers,"control/qmqpservers",0) != 1) die_control(); getmess(); serverindex.len = 1; serverindex.pos[0]=0; for (j = 0; j < se
RE: mail server location question
You can't connect to the external side of your firewall from a machine on the inside. Make sure you're testing it from a machine outside of your firewall. Jay -Original Message- From: Greg Owen [mailto:[EMAIL PROTECTED]] Sent: Thursday, July 27, 2000 8:00 AM To: qmail Subject: RE: mail server location question > OK, I think I have my firewall masquerading the firewall > external IP port 25 to the qmail box internal IP port 25 > > I'm getting connection rejects, when I try to telnet to > port 25 on the firewall. This should redirect me to port > 25 on the qmail box, right? If your firewall is set up right, it should. Does your qmail box accept connections on port 25 at all? While logged into your qmail box, type 'telnet localhost 25'. If you get connection refused, then you aren't running qmail-smtpd properly. If your connection is accepted and you get the SMTP banner, then test the firewall's port 25 again. If the first suceeds and the second fails, then the firewall is probably not configured correctly. > I'm not sure that it's the qmail box that's causing the > problem, but is there anything I need to do to allow smtp > connections from the internet? Not on the connection level. Once you get port 25 responding to the outside world, you may need to tweak your configuration as far as rcpthosts and relaying goes, but first let's get plain old connectivity going. -- gowen -- Greg Owen -- [EMAIL PROTECTED]
qmail-qmqpc.c load balancing mods
Here's the modified version of qmqpc that will pick a random server instead of the first available. I haven't tested it yet, but it compiles, and printf statements in strategic places give me the output I'm looking for. The rand() function is seeded with the current milliseconds from the system clock. The for loop for doit() will loop twice the number of the servers we have to make sure it looked at all of them, this was easier than keeping track of which servers were already checked. Don't make too much fun of my code, I haven't coded anything for about 3 years. :) -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com === #include #include #include #include #include #include #include #include "substdio.h" #include "getln.h" #include "readwrite.h" #include "exit.h" #include "stralloc.h" #include "slurpclose.h" #include "error.h" #include "sig.h" #include "ip.h" #include "timeoutconn.h" #include "timeoutread.h" #include "timeoutwrite.h" #include "auto_qmail.h" #include "control.h" #include "fmt.h" #define PORT_QMQP 628 void die_success() { _exit(0); } void die_perm() { _exit(31); } void nomem() { _exit(51); } void die_read() { if (errno == error_nomem) nomem(); _exit(54); } void die_control() { _exit(55); } void die_socket() { _exit(56); } void die_home() { _exit(61); } void die_temp() { _exit(71); } void die_conn() { _exit(74); } void die_format() { _exit(91); } int lasterror = 55; int qmqpfd; int saferead(fd,buf,len) int fd; char *buf; int len; { int r; r = timeoutread(60,qmqpfd,buf,len); if (r <= 0) die_conn(); return r; } int safewrite(fd,buf,len) int fd; char *buf; int len; { int r; r = timeoutwrite(60,qmqpfd,buf,len); if (r <= 0) die_conn(); return r; } char buf[1024]; substdio to = SUBSTDIO_FDBUF(safewrite,-1,buf,sizeof buf); substdio from = SUBSTDIO_FDBUF(saferead,-1,buf,sizeof buf); substdio envelope = SUBSTDIO_FDBUF(read,1,buf,sizeof buf); /* WARNING: can use only one of these at a time! */ stralloc beforemessage = {0}; stralloc message = {0}; stralloc aftermessage = {0}; char strnum[FMT_ULONG]; stralloc line = {0}; struct sindex { int pos[256]; int len; }; void getmess() { int match; if (slurpclose(0,&message,1024) == -1) die_read(); strnum[fmt_ulong(strnum,(unsigned long) message.len)] = 0; if (!stralloc_copys(&beforemessage,strnum)) nomem(); if (!stralloc_cats(&beforemessage,":")) nomem(); if (!stralloc_copys(&aftermessage,",")) nomem(); if (getln(&envelope,&line,&match,'\0') == -1) die_read(); if (!match) die_format(); if (line.len < 2) die_format(); if (line.s[0] != 'F') die_format(); strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0; if (!stralloc_cats(&aftermessage,strnum)) nomem(); if (!stralloc_cats(&aftermessage,":")) nomem(); if (!stralloc_catb(&aftermessage,line.s + 1,line.len - 2)) nomem(); if (!stralloc_cats(&aftermessage,",")) nomem(); for (;;) { if (getln(&envelope,&line,&match,'\0') == -1) die_read(); if (!match) die_format(); if (line.len < 2) break; if (line.s[0] != 'T') die_format(); strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0; if (!stralloc_cats(&aftermessage,strnum)) nomem(); if (!stralloc_cats(&aftermessage,":")) nomem(); if (!stralloc_catb(&aftermessage,line.s + 1,line.len - 2)) nomem(); if (!stralloc_cats(&aftermessage,",")) nomem(); } } void doit(server) char *server; { struct ip_address ip; char ch; if (!ip_scan(server,&ip)) return; qmqpfd = socket(AF_INET,SOCK_STREAM,0); if (qmqpfd == -1) die_socket(); if (timeoutconn(qmqpfd,&ip,PORT_QMQP,10) != 0) { lasterror = 73; if (errno == error_timeout) lasterror = 72; close(qmqpfd); return; } strnum[fmt_ulong(strnum,(unsigned long) (beforemessage.len + message.len + aftermessage.len))] = 0; substdio_puts(&to,strnum); substdio_puts(&to,":"); substdio_put(&to,beforemessage.s,beforemessage.len); substdio_put(&to,message.s,message.len); substdio_put(&to,aftermessage.s,aftermessage.len); substdio_puts(&to,","); substdio_flush(&to); for (;;) { substdio_get(&from,&ch,1); if (ch == 'K') die_success(); if (ch == 'Z') die_temp(); if (ch == 'D') die_perm(); } } stralloc servers = {0}; main() { int i; int j; int randj; struct timeb tp; struct sindex serverindex; //used to keep an i
RE: qmqpc load balancing
It would probably be much more efficient to round-robin them. Otherwise you end up banging on one until it's buried (or at it's set limit), then banging on the next, and so on. What happens when they all decide their load is too high and shut down qmqpd? Isn't it much easier to code round-robin into it anyway? Would it be easier to just make it so I could put a hostname into the qmqpservers file and do round robin dns for it? Wouldn't that just be a simple addition of gethostbyname()? I have to say though, I really like the way qmail is laid out. Lots of small programs with specific functions, it makes it very easy to do modifications. Jay -Original Message- From: Michael T. Babcock [mailto:[EMAIL PROTECTED]] Sent: Saturday, July 22, 2000 3:55 PM To: Russell Nelson Cc: '[EMAIL PROTECTED]' Subject: Re: qmqpc load balancing True, but its quite valid to round-robin several servers to keep any one from ever getting a high load in the first place. eg. the way load-balancing HTTP usually works. Russell Nelson wrote: > Austad, Jay writes: > > Instead of having qmqpc picking the first available server, I would like it > > to load balance between all servers I have listed as QMQP servers. > > Do it the other way around. If a server thinks its load is too high, > it should shut down its qmqpc service.
RE: qmqpc load balancing
Even if it does hit a down server twice, it'll still just randomize and try again hopefully hitting one that's not down. This shouldn't lose any mail and shouldn't take much, if any, extra system resources. So the actual string that 'j' scans looks something like this: 192.168.20.1\0192.168.20.2\0192.168.20.3 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, July 21, 2000 3:44 PM To: [EMAIL PROTECTED] Subject: Re: qmqpc load balancing On Fri, Jul 21, 2000 at 04:35:57PM -0400, Paul Jarc wrote: > [EMAIL PROTECTED] writes: > > On Fri, Jul 21, 2000 at 03:10:58PM -0500, Austad, Jay wrote: > > > Instead of having qmqpc picking the first available server, I would like it > > > to load balance between all servers I have listed as QMQP servers. > ... > > > This way, "i" will be a random number from 0 to (servers.len-1). > > > > Almost. Note that 'i' has to point to the start of the \0 terminated > > string that 'j' is currently scanning. servers.s is a series of \0 > > terminated strings. You need to do some work to make sure that i, is > > pointing that the start of the string that j randomizes into. > > Also note that if you choose one at random every time, you just might > choose the same one, which happens to be down, every time. It'd be > better to rotate the list by a random number of entries, and then step > through it normally. I think he as trying to randomize the start point and cycle thru the list as qmqpc does now. So even if he hits the same starting point, it'll still find a good server. Leastwise that's what I was originally trying to show with the posted code-fragment. Regards.
qmqpc load balancing
Instead of having qmqpc picking the first available server, I would like it to load balance between all servers I have listed as QMQP servers. In qmail-qmqpc.c on line 153, it says: i = 0; for (j = 0;j < servers.len;++j) if (!servers.s[j]) { doit(servers.s + i); i = j + 1; } Would it work if I change it to: i = 0; for (j = 0;j < servers.len;++j) i = (servers.len*1.0)*rand()/(RAND_MAX+1.0); if (!servers.s[j]) { doit(servers.s + i); } This way, "i" will be a random number from 0 to (servers.len-1). -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
dot-qmail with ./named.pipe == invalid argument #4.3.0
Greetings from a happy new qmail user (first post), ** qmail 1.03 ** linux-2.2.16-:i386-:-:pentium-:- As part of a ritualistic adventure to the edges of qmail sanity, I brazenly created a fifo to be read by a perl "daemon". Using a .qmail file in a real-user's home dir that read: ./inmail.pipe (BTW - I'm also using fastforward, but this is an actual user account without aliasing) Long story short -- qmail wrote to the pipe just fine but justifiably barfs at the end. (Uhh, stat call or something?) Jul 20 10:43:03 mn qmail: 964089783.401047 delivery 3225: deferral: Unable_to_write_./inmail.pipe:_invalid_argument._(#4.3.0)/ So .. I proceeded to create a writer.sh: #!/bin/sh cat > ./inmail.pipe and modified .qmail: | ./writer.sh This appears to work fine, as I had hoped. Why would I do such a crazy thing in the first place? I've compiled with QUEUE_EXTRA and plan to use it to mangle incoming/outgoing messages for [EMAIL PROTECTED] & [EMAIL PROTECTED] More precisely, I plan to attach ticket # headers, update the "bug reports" database, and route incoming replies to the sales/support person who was first to respond. Blah blah blah .. using Perl. I switched to qmail 1) specifically for QUEUE_EXTRA, 2) security, security, security, and 3) out of desperation and displeasure with sendmail (Sorry sendmail hackers). So why the FIFO? I'm concerned about spawning many-a-Perl at 2.5MB each on my lowly P75 DNS/Firewall/qmail box. So the idea of a single reader is attractive .. Is that crazy? Suggestions? (Other than "Pentium II's are cheap") :-) -Jay J p.s. Many thanks for qmail & friends
RE: questions about performance and setup
>As for performance though, I'd be interested in seeing the actual numbers from >the ramdisk test to check against my 10k RPM disk stats. I used bonnie++ to test it. I'll post the results sometime today, when I get some time. Jay -Original Message- From: Michael T. Babcock [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 18, 2000 10:41 AM To: Austad, Jay; Qmail Mailing List Subject: Re: questions about performance and setup Nothing wrong with 100% CPU usage. It just means that the kernel was able to soak the CPU with work ... which is good. Maxing out your performance on a RAM disk at 75% CPU usage means your system has a problem somewhere. As for performance though, I'd be interested in seeing the actual numbers from the ramdisk test to check against my 10k RPM disk stats. "Austad, Jay" wrote: > I did some benchmarking using a standard 7200 RPM disk and a 128MB ramdisk. > The machine was not using any swap, so there was no chance of the ramdisk > accidentally making it to disk. > > In short, performance on it sucked. The throughput was about 10% less than > IDE, but seeks/sec were 5-10 times more. However, the CPU was maxed at 100% > during tests to the ramdisk.
RE: IPCHAINS and slow POP/SMTP access
Since ipchains is a not really a firewall but a packet filter, you need to make sure you have the line: ipchains -A input -s 0.0.0.0/0.0.0.0 -d /255.255.255.255 -p 6 -j ACCEPT ! -y Make sure your firewall also accepts all packets from 192.168.1.0/24 also. Ipchains configs are sorta off topic, so if you have any other questions just email me instead of the list. Check these 2 things though, you probably already have them, but make sure. Chances are you don't even need the ipchains rules with the masquerade though, depends on what's running on the firewall box. You machines behind it are inherently protected by the fact that you're using port address translation for net access to them. Jay -Original Message- From: Doug Oucharek [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 18, 2000 3:09 AM To: [EMAIL PROTECTED] Subject: IPCHAINS and slow POP/SMTP access Hello, I've searched the archives on this topic and though there are a lot of people who have reported this issue, I have not really seen a solution yet. I have a Linux box (Redhad 5.?) which I use as a firewall/server/NAT machine. One ethernet card is connected to an ADSL modem and has a fixed IP address. The other ethernet card is connected to a mini-hub and has the address of 192.168.1.1. I have a set of Mac's connected to the hub with addresses 192.168.1.100 and up. I've got Qmail running just great for both SMTP and POP!! However, as soon as I activate my firewall (using ipchains), sending or receiving email from a local machine takes over 3 minutes!! In the archives, some people have speculated that this is a DNS issue or a problem with auth. I have TCP port 113 (auth) opened to the world (local and internet) and am still having a problem. I suspect that there must be some other port I need to open up. Does anyone have a suggestion of where I can go from here? I am a bit new to Qmail and not too familiar with the debugging tools. By the way, I do not have DNS active on my Linux box and am relying on my ISP's DNS server (they have my domain name set up in their server). Doug
RE: questions about performance and setup
I did some benchmarking using a standard 7200 RPM disk and a 128MB ramdisk. The machine was not using any swap, so there was no chance of the ramdisk accidentally making it to disk. In short, performance on it sucked. The throughput was about 10% less than IDE, but seeks/sec were 5-10 times more. However, the CPU was maxed at 100% during tests to the ramdisk. Jay -Original Message- From: Oliver White [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 18, 2000 12:27 AM To: [EMAIL PROTECTED] Subject: Re: questions about performance and setup Steve Wolfe wrote: > > With all of the emails I recieved, I get the impression that I'm going to > > I/O bound instead of processor or memory bound. How much disk will be > > sufficient for the queue? 1GB? More? > > It's not so much a matter of disk size (I don't think you'll have a 1 gig > queue!), You could quite easily get a 1 Gig queue, even if you don't run into the obvious problem of temporary loss of network connectivity. Say you've got 200,000 subscribers and you generate your messages twice as fast as qmail can send them, then when you've finished generating the messages you've still got 100,000 in the queue. If the messages are 10Kb each, that's 1 Gb. > > (I can put 2GB of ram in the box)? Linux has support for making a disk in > > memory, putting a filesystem on it and mounting it. Wouldn't this take > > care of I/O problems? > >That's about as good of I/O as you can get, I would imagine. ; ) As > another author stated, the largest gain would be in writes, but that's where > the largest expenditure is anyway. Just make dang, dang sure that your > machine is NOT going to have any hiccups or lose power while the queue is > full, or you'll instantly lose it all. What if you put the 2 Gb RAM in the box, but let Linux use it as a disk cache? I'm not sure how the disk caching under Linux works, but if you create a file and then delete it before it actually gets written to disk, is there any disk activity required? Sure, the disks will be thrashing away, trying to keep up, but would the I/O actually block if there was still room in the disk cache? - Oliver.
RE: questions about performance and setup
It's all newsletter subscription email so an occasional loss of the queue would be fine. It won't be pumping out mission critical email or anything like that. I'm benchmarking a ramdisk filesystem right now, unfortunately the only thing I have to compare it to is an IDE disk because I can't take down our other machines to increase the ramdisk size during the day. Even then, I'd only be able to test on a 10,000 rpm scsi disk, no RAID. Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Monday, July 17, 2000 10:37 AM To: [EMAIL PROTECTED] Subject: Re: questions about performance and setup On Mon, Jul 17, 2000 at 10:29:03AM -0500, Austad, Jay wrote: > With all of the emails I recieved, I get the impression that I'm going to > I/O bound instead of processor or memory bound. How much disk will be > sufficient for the queue? 1GB? More? > > I'm just grasping here to figure out the best solution, so bear with me... > What if I only needed a 1GB queue, and what if that queue was a 1GB ramdisk > (I can put 2GB of ram in the box)? Linux has support for making a disk in > memory, putting a filesystem on it and mounting it. Wouldn't this take care > of I/O problems? The I/O cost is simply there to protect again machine failure, reboots, power-loss, OS bugs, that sort of thing. Your memory file system will be ok if it's battery-backed up and running on a system as reliable as a hard-disk. Otherwise you increase the chances of losing part of your queue at some point. Having said that, you may find the trade-off acceptable. That is putting the queue in a memory file system and accepting a total queue loss once every now and and again. If eg, it's advertising email and occassional losses are tolerable, then this may be a perfectly acceptable cost/reliability trade-off for you. Regards.
RE: questions about performance and setup
With all of the emails I recieved, I get the impression that I'm going to I/O bound instead of processor or memory bound. How much disk will be sufficient for the queue? 1GB? More? I'm just grasping here to figure out the best solution, so bear with me... What if I only needed a 1GB queue, and what if that queue was a 1GB ramdisk (I can put 2GB of ram in the box)? Linux has support for making a disk in memory, putting a filesystem on it and mounting it. Wouldn't this take care of I/O problems? Jay -Original Message- From: Oliver White [mailto:[EMAIL PROTECTED]] Sent: Monday, July 17, 2000 12:34 AM To: [EMAIL PROTECTED] Subject: Re: questions about performance and setup We're in a similar situation at the moment. However, we want to send out 100,000 UNIQUE emails per day, expanding to 500,000 or more in the near future. Also, our send window is only actually a couple of hours. I'm trying to work out the best settings for the concurrencyremote and conf-split parameters. Our system is a HP Netserver 2000r PIII-667 RAID5 running Linux. Are there any problems in setting conf-split to a very large value? Is it necessary on a Linux system, assuming a queue size of, say 100,000? Any information appreciated. - Oliver. "Austad, Jay" wrote: > Non-unique emails will most likely be generated by other machines and send > the box running mini-qmail via smtp. Non-unique emails will be a small > percentage of what gets sent out, for now. > > Jay > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > Sent: Saturday, July 15, 2000 12:10 AM > To: '[EMAIL PROTECTED]' > Subject: Re: questions about performance and setup > > On Fri, Jul 14, 2000 at 07:01:46PM -0500, Austad, Jay wrote: > > >Then have the script that does the mailing call randomly > > >on of the /var/qmail#/bin/qmail-inject. This will emulate round robin > > >without any patching. > > > > Won't this way be a performance hit though? I admit, it is an easy > solution > > No. My experience is that the cost of running a script to inject the mail > in a way similar to that mentioned above, is pretty small compared to the > queue injection cost and the delivery cost. sh or perl will be fine. > > > and would work excellent, but I have to think about efficiency also. C > code > > is much faster than shell or perl, and I'd like to set it up once and not > > have to ever worry about again, or at least for a long, long time. > > > > As I said, we're doing 50 million emails a month right now, but this is > > increasing substantially each month, and as we rollout new subscription > > services, we'll have even more load. Sending 10 times this amount by the > > same time next year is a good possibility, possibly sooner as we seem to > > underestimate the rate at which we're growing much of the time... > > You may also need to look at the scalability of the generation of the > emails. One system I recently looked at claimed to be able to generate > nicely unique emails at a targetted database, but it burned CPU like > it was free - just in generating the content. > > Mark. > > > > > Jay > > > > -Original Message- > > From: JuanE [mailto:[EMAIL PROTECTED]] > > Sent: Friday, July 14, 2000 5:55 PM > > To: '[EMAIL PROTECTED]' > > Subject: Re: questions about performance and setup > > > > > > > > Jay, > > > > That's the beauty of having multiple instances, not having to patch qmail. > > All you need to do is install qmail once per machine (ie, /var/qmail1, > > /var/qmail2,...). Then have the script that does the mailing call randomly > > on of the /var/qmail#/bin/qmail-inject. This will emulate round robin > > without any patching. > > > > JES > > > > Austad, Jay writes: > > > > > Where would I start in the code to modify the QMQP servers list so that > it > > > would load balance between all of the servers in the list instead of > just > > > using the first one it can contact? This would be very useful to me. I > > > assume qmail-qmqpc.c is one of them, are there others I would need to > play > > > around with? > > > > > > Jay > > > > > > -Original Message- > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > > > Sent: Friday, July 14, 2000 3:55 PM > > > To: '[EMAIL PROTECTED]' > > > Subject: Re: questions about performance and setup > > > > > > > > > On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote: > > > > I alread
RE: questions about performance and setup
Non-unique emails will most likely be generated by other machines and send the box running mini-qmail via smtp. Non-unique emails will be a small percentage of what gets sent out, for now. Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Saturday, July 15, 2000 12:10 AM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup On Fri, Jul 14, 2000 at 07:01:46PM -0500, Austad, Jay wrote: > >Then have the script that does the mailing call randomly > >on of the /var/qmail#/bin/qmail-inject. This will emulate round robin > >without any patching. > > Won't this way be a performance hit though? I admit, it is an easy solution No. My experience is that the cost of running a script to inject the mail in a way similar to that mentioned above, is pretty small compared to the queue injection cost and the delivery cost. sh or perl will be fine. > and would work excellent, but I have to think about efficiency also. C code > is much faster than shell or perl, and I'd like to set it up once and not > have to ever worry about again, or at least for a long, long time. > > As I said, we're doing 50 million emails a month right now, but this is > increasing substantially each month, and as we rollout new subscription > services, we'll have even more load. Sending 10 times this amount by the > same time next year is a good possibility, possibly sooner as we seem to > underestimate the rate at which we're growing much of the time... You may also need to look at the scalability of the generation of the emails. One system I recently looked at claimed to be able to generate nicely unique emails at a targetted database, but it burned CPU like it was free - just in generating the content. Mark. > > Jay > > -Original Message- > From: JuanE [mailto:[EMAIL PROTECTED]] > Sent: Friday, July 14, 2000 5:55 PM > To: '[EMAIL PROTECTED]' > Subject: Re: questions about performance and setup > > > > Jay, > > That's the beauty of having multiple instances, not having to patch qmail. > All you need to do is install qmail once per machine (ie, /var/qmail1, > /var/qmail2,...). Then have the script that does the mailing call randomly > on of the /var/qmail#/bin/qmail-inject. This will emulate round robin > without any patching. > > JES > > Austad, Jay writes: > > > Where would I start in the code to modify the QMQP servers list so that it > > would load balance between all of the servers in the list instead of just > > using the first one it can contact? This would be very useful to me. I > > assume qmail-qmqpc.c is one of them, are there others I would need to play > > around with? > > > > Jay > > > > -Original Message- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > > Sent: Friday, July 14, 2000 3:55 PM > > To: '[EMAIL PROTECTED]' > > Subject: Re: questions about performance and setup > > > > > > On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote: > > > I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes > > > with no trouble, some of them took work to get going, but it runs well. > I > > > have a few Crystal PC's here also that I may use instead, dual PIII > 550's > > > with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these > for > > > testing. > > > > I agree with the earlier poster that more spindles for your queue > > (c/- raid) is a good thing in general. > > > > > The bulk of the messages will be the same content to many rcpt's. > > However, > > > once in awhile we'll have 100,000 different messages go out to 100,000 > > > different people. > > > > > > Since the QMQP support under mini-qmail doesn't load balance, can I feed > > it > > > a hostname with multiple dns entries (round-robin dns)? Or better yet, > > how > > > easy would it be to modify the qmail code to just load balance between > > them? > > > > The manpage for qmail-qmqpc tells us that they have to be IP addresses > > in qmqpservers so a RR DNS won't help. If all of the messages are > generated > > on one machine, then I'd be inclined to go for a much simpler solution > > than modifying qmail. I'd have an instance of qmail for each outbound > > server with the appropriate qmqpservers entry, then have your queue > > insertion script do a round-robin itself by simply cycling thru > > the qmail-inject command associated with each instance. > > > > for instance in 1 2 3 4 5
RE: questions about performance and setup
>Then have the script that does the mailing call randomly >on of the /var/qmail#/bin/qmail-inject. This will emulate round robin >without any patching. Won't this way be a performance hit though? I admit, it is an easy solution and would work excellent, but I have to think about efficiency also. C code is much faster than shell or perl, and I'd like to set it up once and not have to ever worry about again, or at least for a long, long time. As I said, we're doing 50 million emails a month right now, but this is increasing substantially each month, and as we rollout new subscription services, we'll have even more load. Sending 10 times this amount by the same time next year is a good possibility, possibly sooner as we seem to underestimate the rate at which we're growing much of the time... Jay -Original Message- From: JuanE [mailto:[EMAIL PROTECTED]] Sent: Friday, July 14, 2000 5:55 PM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup Jay, That's the beauty of having multiple instances, not having to patch qmail. All you need to do is install qmail once per machine (ie, /var/qmail1, /var/qmail2,...). Then have the script that does the mailing call randomly on of the /var/qmail#/bin/qmail-inject. This will emulate round robin without any patching. JES Austad, Jay writes: > Where would I start in the code to modify the QMQP servers list so that it > would load balance between all of the servers in the list instead of just > using the first one it can contact? This would be very useful to me. I > assume qmail-qmqpc.c is one of them, are there others I would need to play > around with? > > Jay > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > Sent: Friday, July 14, 2000 3:55 PM > To: '[EMAIL PROTECTED]' > Subject: Re: questions about performance and setup > > > On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote: > > I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes > > with no trouble, some of them took work to get going, but it runs well. I > > have a few Crystal PC's here also that I may use instead, dual PIII 550's > > with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these for > > testing. > > I agree with the earlier poster that more spindles for your queue > (c/- raid) is a good thing in general. > > > The bulk of the messages will be the same content to many rcpt's. > However, > > once in awhile we'll have 100,000 different messages go out to 100,000 > > different people. > > > > Since the QMQP support under mini-qmail doesn't load balance, can I feed > it > > a hostname with multiple dns entries (round-robin dns)? Or better yet, > how > > easy would it be to modify the qmail code to just load balance between > them? > > The manpage for qmail-qmqpc tells us that they have to be IP addresses > in qmqpservers so a RR DNS won't help. If all of the messages are generated > on one machine, then I'd be inclined to go for a much simpler solution > than modifying qmail. I'd have an instance of qmail for each outbound > server with the appropriate qmqpservers entry, then have your queue > insertion script do a round-robin itself by simply cycling thru > the qmail-inject command associated with each instance. > > for instance in 1 2 3 4 5 > do > getnext_message_details() > /var/qmail{$instance}/bin/qmail-inject currentmessage details > done > > Or some such. > > > Alternatively, if you have money to burn, maybe a layer four switch > with load-balancing skills. > > > Mark. > > > > > > Jay > > > > > > > > -Original Message- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > > Sent: Friday, July 14, 2000 2:09 PM > > To: '[EMAIL PROTECTED]' > > Subject: Re: questions about performance and setup > > > > > > > Here's what I need to know: > > > > > > 1. How well does qmail take advantage of multiple processors? How much > > > > Indreectly, quite well as it forks many processes, thus if the OS takes > > good advantage of your CPUs, then qmail inherits that advantage. > > > > > memory and disk will I need? (we're at 50 million messages per month > now, > > > > Are these message unique per target address or the same. If unique, your > > requirements are vastly different and very queue/disk intensive. If they > > are the same and you take advantage or VERP support on qmail, then > > your load will mainly be sending related which will
RE: questions about performance and setup
Where would I start in the code to modify the QMQP servers list so that it would load balance between all of the servers in the list instead of just using the first one it can contact? This would be very useful to me. I assume qmail-qmqpc.c is one of them, are there others I would need to play around with? Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, July 14, 2000 3:55 PM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote: > I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes > with no trouble, some of them took work to get going, but it runs well. I > have a few Crystal PC's here also that I may use instead, dual PIII 550's > with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these for > testing. I agree with the earlier poster that more spindles for your queue (c/- raid) is a good thing in general. > The bulk of the messages will be the same content to many rcpt's. However, > once in awhile we'll have 100,000 different messages go out to 100,000 > different people. > > Since the QMQP support under mini-qmail doesn't load balance, can I feed it > a hostname with multiple dns entries (round-robin dns)? Or better yet, how > easy would it be to modify the qmail code to just load balance between them? The manpage for qmail-qmqpc tells us that they have to be IP addresses in qmqpservers so a RR DNS won't help. If all of the messages are generated on one machine, then I'd be inclined to go for a much simpler solution than modifying qmail. I'd have an instance of qmail for each outbound server with the appropriate qmqpservers entry, then have your queue insertion script do a round-robin itself by simply cycling thru the qmail-inject command associated with each instance. for instance in 1 2 3 4 5 do getnext_message_details() /var/qmail{$instance}/bin/qmail-inject currentmessage details done Or some such. Alternatively, if you have money to burn, maybe a layer four switch with load-balancing skills. Mark. > > Jay > > > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] > Sent: Friday, July 14, 2000 2:09 PM > To: '[EMAIL PROTECTED]' > Subject: Re: questions about performance and setup > > > > Here's what I need to know: > > > > 1. How well does qmail take advantage of multiple processors? How much > > Indreectly, quite well as it forks many processes, thus if the OS takes > good advantage of your CPUs, then qmail inherits that advantage. > > > memory and disk will I need? (we're at 50 million messages per month now, > > Are these message unique per target address or the same. If unique, your > requirements are vastly different and very queue/disk intensive. If they > are the same and you take advantage or VERP support on qmail, then > your load will mainly be sending related which will benefit from > more memory, multiple instances, etc. > > > and we only send out monday-friday, so that's over 2 million messages per > > day, and it's only going up) > > > > 2. How many messages per day would one estimate that each of these > servers > > could do? > > > > 3. I read about mini-qmail and how it's about 100 times faster blasting > out > > email to QMQP servers. Since you can specify multiple QMQP servers, if I > > have a fourth machine running mini-qmail and managing the actual mailing > > list, can I add the other 3 as QMQP servers and have it load balance > between > > all 3 for sending out mail? (this way I could add more servers easily if > I > > needed to) > > The qmqp support doesn't load balance. It simply takes the first one > it can connect to. > > > 4. Can I easily make qmail run an external script for each bounced mail? > > Absolutely. > > > 5. Anything else I should know? > > That all hinges on whether your emails are unique for each recipient or > not. Or more importantly, the average number of recipients per unique > email. > > > Regards.
RE: questions about performance and setup
I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes with no trouble, some of them took work to get going, but it runs well. I have a few Crystal PC's here also that I may use instead, dual PIII 550's with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these for testing. The bulk of the messages will be the same content to many rcpt's. However, once in awhile we'll have 100,000 different messages go out to 100,000 different people. Since the QMQP support under mini-qmail doesn't load balance, can I feed it a hostname with multiple dns entries (round-robin dns)? Or better yet, how easy would it be to modify the qmail code to just load balance between them? Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, July 14, 2000 2:09 PM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup > Here's what I need to know: > > 1. How well does qmail take advantage of multiple processors? How much Indreectly, quite well as it forks many processes, thus if the OS takes good advantage of your CPUs, then qmail inherits that advantage. > memory and disk will I need? (we're at 50 million messages per month now, Are these message unique per target address or the same. If unique, your requirements are vastly different and very queue/disk intensive. If they are the same and you take advantage or VERP support on qmail, then your load will mainly be sending related which will benefit from more memory, multiple instances, etc. > and we only send out monday-friday, so that's over 2 million messages per > day, and it's only going up) > > 2. How many messages per day would one estimate that each of these servers > could do? > > 3. I read about mini-qmail and how it's about 100 times faster blasting out > email to QMQP servers. Since you can specify multiple QMQP servers, if I > have a fourth machine running mini-qmail and managing the actual mailing > list, can I add the other 3 as QMQP servers and have it load balance between > all 3 for sending out mail? (this way I could add more servers easily if I > needed to) The qmqp support doesn't load balance. It simply takes the first one it can connect to. > 4. Can I easily make qmail run an external script for each bounced mail? Absolutely. > 5. Anything else I should know? That all hinges on whether your emails are unique for each recipient or not. Or more importantly, the average number of recipients per unique email. Regards.
questions about performance and setup
I've been given the task of setting up our own "blaster" for sending out emails of our financial news and charts to our subscribers. We outsource this right now, and it's abysmally expensive. Basically, we want 3 boxes (or so) that run in parallel and blast out the emails, about 50 million per month, but the subscription rate is growing rapidly each month. It needs to handle bounced mail by dumping the addresses into a file for later retrieval so they can be removed from the database, or by running an external script for each bounced address. I'm looking at getting 3 dell dual PIII 750's, with a 18 or 36GB 1rpm disk, and 512M or 1G of mem each. Each will run Linux or BSD. Here's what I need to know: 1. How well does qmail take advantage of multiple processors? How much memory and disk will I need? (we're at 50 million messages per month now, and we only send out monday-friday, so that's over 2 million messages per day, and it's only going up) 2. How many messages per day would one estimate that each of these servers could do? 3. I read about mini-qmail and how it's about 100 times faster blasting out email to QMQP servers. Since you can specify multiple QMQP servers, if I have a fourth machine running mini-qmail and managing the actual mailing list, can I add the other 3 as QMQP servers and have it load balance between all 3 for sending out mail? (this way I could add more servers easily if I needed to) 4. Can I easily make qmail run an external script for each bounced mail? 5. Anything else I should know? Thanks. -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
EZMLM problems
I'm using ezmlm 0.53 with ezmlm-idx 0.40. Have never set up a moderated mailing list before, but decided I wanted to try it out tonight. Set it up with: ezmlm-make -q -m /path/to/list /path/to/. list domain.com Set up a couple of test subscribers, and set up a moderator with ezmlm-sub /path/to/list/mod [EMAIL PROTECTED] Tried to send a subscribe request, got the confirmation back, sent the "cookie" back to be accepted to the list, all while tail -f'ing my logfiles. Got this in the logfile... May 6 01:55:59 domain qmail: 957603359.650961 status: local 1/10 remote 2/20 May 6 01:55:59 elementdesign qmail: 957603359.760669 delivery 4325: failure: ezmlm-manage:_fatal:_Command_not_available_(#5.1.1)/ May 6 01:55:59 domain qmail: 957603359.761031 status: local 0/10 remote 2/20 What could be causing this? ezmlm-manage is there, its in the path, and it is taking the correct command line options (ezmlm-manage 'path/to/list', correct?) When I try to run it from the command line with these command line options, I get "SENDER not set". I know this is an environment variable, but where is it set, and what is it to be set to? It's pretty vague in the manpages and the FAQ. Thanks for any help... appreciate it. j
Local mail for virtual domains
I have a user [EMAIL PROTECTED] that has an account on our qmail box and would like the mail delivered to /home/testme/Maildir. The qmail box is also a virtual domain for sailnet.com. The default domain for the qmail box is sailnet.net. How do I get her mail delievered /home/test/Maildir instead of /home/vpopmail/domains/sailnet.com/testme/Maildir? Thanx, Jay
queue backup
I have qmail/ezmlm-idx. We are serving over 100 mailing lists with a total of about 50,000 subscribers. The output from qmail-qstat is: messages in queue: 2293 messages in queue but not yet preprocessed: 0 My first question is how to I purge the queue. The second question is what causes the queue to hold these messages. The queue size seems to continue grow it never comes down. We are not having any complaints from our subscribers at all. Thanx, Jay
Re: Qmail System Users
You can find some of this info on www.inter7.com >> Original Message << On 3/29/00, 8:45:21 AM, [EMAIL PROTECTED] wrote regarding Qmail System Users: > I am trying to justify continuing the use of qmail in our organization. Is > there a comprehensive list out there of large organizations that use qmail > and the volume of mail they have pass through their mail exchangers? > Appreciate any insight into this. Thanks.
start/stop
I'm running starting qmail with the following script, how do I start and stop qmail with tcpserver? env - PATH="/var/qmail/bin:/usr/local/bin" \ qmail-start ./Maildir/ /usr/local/bin/accustamp \ | /usr/local/bin/setuser qmaill /usr/local/bin/cyclog -s 104 \ -n 200 /var/log/qmail & echo "Starting qmail ..." env - PATH="/var/qmail/bin:/usr/local/bin" \ tcpserver -H -R -c100 0 pop3 /var/qmail/bin/qmail-popup \ sailnet.com \ /home/vpopmail/bin/vchkpw /var/qmail/bin/qmail-pop3d Maildir & echo "Starting pop ..." env - PATH="/var/qmail/bin:/usr/local/bin" \ tcpserver -H -R -x /etc/tcp.smtp.cdb -c100 -u1001 -g102 0 smtp \ /var/qmail/bin/qmail-smtpd 2>&1 > /dev/null & echo "Starting smtp ..."
Re: TCPRULES help
I upgraded to v0.86, however I still see nothing in the docs on the webpage or in the man pages about using hostnames! Where did you find this info? Thanx, Jay >>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<< On 3/14/00, 11:09:59 AM, Anand Buddhdev <[EMAIL PROTECTED]> wrote regarding Re: TCPRULES help: > On Tue, Mar 14, 2000 at 04:43:30PM +0100, Petr Novotny wrote: > > > I would like to add the following to my tcp.smtp file for tcprules: > > > > > > .domain.com:allow,RELAYCLIENT="" > > > > > > My question is can you use domain names with tcprules, or does it have > > > to be IP addresses. In the man page all expamples use Ips and not > > > domains. > > > > It has to be IP address. > With the new tcpserver v 0.86, you can use hostnames. See the > documentation. > -- > See complete headers for more info
TCPRULES help
I would like to add the following to my tcp.smtp file for tcprules: .domain.com:allow,RELAYCLIENT="" My question is can you use domain names with tcprules, or does it have to be IP addresses. In the man page all expamples use Ips and not domains. Thanx, Jay
Re: Example Please
Along the same lines, I would like qmail to forward an email to [EMAIL PROTECTED] to all the email lists I have setup with ezmlm. I have a .qmail-all-lists in my ~alias dir. It contains [EMAIL PROTECTED] [EMAIL PROTECTED] When I send a message to [EMAIL PROTECTED] I get: <[EMAIL PROTECTED]>: ezmlm-reject: fatal: List address must be in To: or Cc: (#5.7.0) for every list in the alias. Also, how do I protect this alias so only a select few can send a message to it. Thanx, Jay >>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<< On 3/10/00, 10:26:10 AM, Jonathan McDowell <[EMAIL PROTECTED]> wrote regarding Re: Example Please: > On Fri, Mar 10, 2000 at 10:16:27AM +, Christopher Tarricone wrote: > > How would I setup and alias to someones pager > > ([EMAIL PROTECTED]) without creating a local account > > to forward the mail from. Or can I not do that. I am new to qmail.. In > > sendmail I just added the line to /etc/alliases and was done with > > it... > You want to create a .qmail-foo file in ~alias that contains > &[EMAIL PROTECTED] > as the only line. This will forward all mail to foo@yourdomain to > [EMAIL PROTECTED] > J. > -- > Programmer, | Conscience is the fear of getting | Tel/SMS (Orange): > student and | caught. | +44-7974-678646 > BHMF. | | Made by HuggieTag
Trailer for every message
I want to add a trailer to all outgoing list messages for every list created. I used ezmlm-make - t to create a list, however I never see the trailer from DIR/text/trailer appended to any of the lists messages. Any Ideas? Thanx, Jay
POP and pine/elm
Here's the deal. I set up vpopmail (or vchkpass, whatever you want to call it) for pop mail. It keeps everything in /home/vpopmail. But some of my users want to be able to check their mail with pine if they need to, or be able to download it if they need to. (like if they are on the road, pine is more convenient) I'm very new to qmail and am still a bit confused about how it works, so can anyone give me any suggestions on how to make this work? For instance... I set up a pop account [EMAIL PROTECTED] Someone sends mail to [EMAIL PROTECTED] it goes into /home/vpopmail/domains/domain.com/jay/Maildir but I also want it to go into /home/jay/Maildir, so I can check it in pine as well... Thanks for any help
qmail vs. sendmail weirdness
Alright, here's the situation. I set up vpopmail for pop accounts, and qmailadmin to manage those accounts. I set up an account [EMAIL PROTECTED] I added that pop user to my mail client, and sent out a few messages to test. When I send a message to a domain that uses sendmail, I get a bounce saying: <[EMAIL PROTECTED]> ... Domain must exist Simple question. WTF? All of the other domains I sent it to worked fine (hotmail, a few boxes I set qmail up on, a few domains that use alternative mail clients) Any reason why this is happening? Thanks.
.qmail questions
Here's the deal... I'm new to qmail, and I think it's just the coolest thing I've ever seen. But I am having problems with it. I have a domain in the rcpthosts and virtualdomains files and I want to set up a pop account for somebody on my domain. So I create a file called .qmail-mydomain-user in my home directory and put "/home/user/Maildir/" in it. When I send mail to that user, qmail cannot chdir to /home/user/Maildir/, so the mail isn't recieved. I've tried changing permissions on /home/user/Maildir/ but the mail still isn't delivered... temporary error. Don't know the reason for this. I've tried using the users/assign method of delivering mail to this user, but no matter what I put in the users/assign file (even if it's empty!), and I run qmail-newu, I get the following error: qmail-newu: fatal: bad format in users/assign Are there any workaround for my problem, or am I doing something wrong? Also, what is wrong with users/assign? Just for reference, here is the line I put in users/assign: =myuser-mydomain-nathan:nate:1035:109:/home/nate::: ^^^this is set to what I set in control/virtualdomains like: mydomain.com:myuser-mydomain Any help appreciated.
Re: Help on qmailanalog
"Jay" == Jay Soffian <[EMAIL PROTECTED]> writes: Jay> I've attached a patch for daemontools-0.61 which creates tai64nepoch, Jay> something I hacked from tai64nlocal. I happen to rotate my logs Oops, forgot to attach the patch. Attached. j. -- Jay Soffian <[EMAIL PROTECTED]>UNIX Systems Engineer 404.572.1941 Cox Interactive Media diff -c -N daemontools-0.61.orig/Makefile daemontools-0.61/Makefile *** daemontools-0.61.orig/Makefile Thu Aug 26 02:56:44 1999 --- daemontools-0.61/Makefile Mon Nov 1 21:44:37 1999 *** *** 318,324 ./compile open_write.c prog: \ ! svscan supervise svok svstat svc fghack multilog tai64n tai64nlocal \ softlimit setuidgid envuidgid rts matchtest prot.o: \ --- 318,324 ./compile open_write.c prog: \ ! svscan supervise svok svstat svc fghack multilog tai64n tai64nlocal tai64nepoch \ softlimit setuidgid envuidgid rts matchtest prot.o: \ *** *** 557,562 --- 557,570 tai64n.o: \ compile tai64n.c timestamp.h substdio.h readwrite.h exit.h ./compile tai64n.c + + tai64nepoch: \ + load tai64nepoch.o substdio.a error.a str.a fs.a + ./load tai64nepoch substdio.a error.a str.a fs.a + + tai64nepoch.o: \ + compile tai64nepoch.c substdio.h subfd.h substdio.h exit.h fmt.h + ./compile tai64nepoch.c tai64nlocal: \ load tai64nlocal.o substdio.a error.a str.a fs.a diff -c -N daemontools-0.61.orig/tai64nepoch.c daemontools-0.61/tai64nepoch.c *** daemontools-0.61.orig/tai64nepoch.c Wed Dec 31 19:00:00 1969 --- daemontools-0.61/tai64nepoch.c Tue Jan 4 13:09:42 2000 *** *** 0 --- 1,68 + #include + #include + #include "substdio.h" + #include "subfd.h" + #include "exit.h" + #include "fmt.h" + + char num[FMT_ULONG]; + + void get(ch) + char *ch; + { + int r; + + r = substdio_get(subfdin,ch,1); + if (r == 1) return; + if (r == 0) _exit(0); + _exit(111); + } + + void out(buf,len) + char *buf; + int len; + { + if (substdio_put(subfdout,buf,len) == -1) + _exit(111); + } + + time_t secs; + unsigned long nanosecs; + unsigned long u; + struct tm *t; + + main() + { + char ch; + + for (;;) { + get(&ch); + if (ch == '@') { + secs = 0; + nanosecs = 0; + for (;;) { + get(&ch); + u = ch - '0'; + if (u >= 10) { + u = ch - 'a'; + if (u >= 6) break; + u += 10; + } + secs <<= 4; + secs += nanosecs >> 28; + nanosecs &= 0xfff; + nanosecs <<= 4; + nanosecs += u; + } + secs -= 4611686018427387914ULL; + out(num,fmt_ulong(num,(unsigned long) (secs))); + out(".",1); + out(num,fmt_uint0(num,(unsigned int) nanosecs,9)); + } + for (;;) { + out(&ch,1); + if (ch == '\n') break; + get(&ch); + } + } + }
Re: Help on qmailanalog
"Ronny" == Ronny Haryanto <[EMAIL PROTECTED]> writes: Ronny> I use this script to send me a log analysis nightly (via crontab). Ronny> Add the z* commands as you like before ")| qmail-inject". Ronny> #!/bin/sh Ronny> PATH=/usr/local/qmailanalog/bin:/var/qmail/bin:/bin Ronny> QMAILLOG="/tmp/q.$$" Ronny> umask 077 Ronny> cat /var/log/qmail/* | matchup > $QMAILLOG 5>/dev/null Ronny> DATE=`date +'%a %d %b'` Ronny> (echo "To: [EMAIL PROTECTED]" Ronny> echo "From: [EMAIL PROTECTED]" Ronny> echo "Subject: qmail report $DATE" Ronny> echo "" Ronny> zoverall < $QMAILLOG Ronny> zfailures < $QMAILLOG Ronny> zdeferrals < $QMAILLOG)| qmail-inject Ronny> rm -f $QMAILLOG You really shouldn't be throwing away the output from fd5. It's there for a reason. I use this script: #!/bin/sh PATH="/usr/local/bin/qmailanalog:/usr/local/bin:$PATH" export PATH umask 077 TMP_FILE="/var/log/qmail/qmailanalog.tmp" EXT_FILE="/var/log/qmail/qmailanalog.ext" OUT_FILE="/var/log/qmail/qmailanalog.out" LOG_FILE="/var/log/qmail/log.1.gz" rm -f $TMP_FILE $OUT_FILE cat << MAIL_HEADER > $OUT_FILE From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: `hostname` qmail statistics MAIL_HEADER touch $EXT_FILE (cat $EXT_FILE; gunzip -c $LOG_FILE | tai64nepoch) | matchup > $TMP_FILE 5>$EXT_FILE.new mv $EXT_FILE.new $EXT_FILE zoverall < $TMP_FILE >> $OUT_FILE echo "--" >> $OUT_FILE zddist < $TMP_FILE >> $OUT_FILE /var/qmail/bin/qmail-inject < $OUT_FILE rm -f $TMP_FILE $OUT_FILE I've attached a patch for daemontools-0.61 which creates tai64nepoch, something I hacked from tai64nlocal. I happen to rotate my logs nightly, using qfilelog instead of using multilog (rotating logs based on size is fine, but multilog really should support logs rotated based on time as well - something as simply as rotating whenever it receives a HUP or USR1 would make me happy). j. -- Jay Soffian <[EMAIL PROTECTED]>UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: help pls! RELAYCLIENT won't bypass rcpthosts
"Reece" == Reece Markowsky <[EMAIL PROTECTED]> writes: Reece> You can see that I am running tcpserver correctly too: /usr/local/bin/tcpserver -v -p -x /etc/tcp.smtp.cdb -u 137 -g 223 0 smtp /var/qmail/bin/qmail-smtpd Try making sure that the path to the CDB file is part of the '-x' argument, not an additional argument (remove that extra space, that is '-xFOO', not '-x FOO'). Also, use cdbdump to make sure that the cdb file is up to date (or just rebuild it). Reece> Any ideas?? If that doesn't work, instead of exec'ing qmail-smtpd, exec an sh script which dumps the env and them execs qmail-smtpd, as in: #!/bin/sh env > /var/tmp/debug/qmail-smtpd.$$ exec /var/qmail/bin/smtpd Then, examine the environment and make sure RELAYCLIENT really is set. BTW, if might want to use ofmipd for "internal" hosts to give you rewriting flexibility in case you need it. I do so like this: /usr/local/bin/tcpserver -R -H -learthquake -x/var/qmail/etc/tcp.smtp.cdb -u61 -g60 -v 0 smtp /var/qmail/libexec/qmail-smtpd+ofmipd /var/qmail/etc/tcp.smtp: 192.168.249.:allow,OFMIPCLIENT="yes" 192.168.250.:allow,OFMIPCLIENT="yes" 206.251.18.:allow,OFMIPCLIENT="yes" 204.71.180.:allow,OFMIPCLIENT="yes" :allow,DATABYTES="1048576" /var/qmail/libexec/qmail-smtpd+ofmipd #!/bin/sh if [ -n "$OFMIPCLIENT" ] ; then exec /var/qmail/bin/ofmipd else exec /var/qmail/bin/qmail-smtpd fi Good luck. j. -- Jay Soffian <[EMAIL PROTECTED]>UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Sendmail Virtusertable equivalent?
"Richard" == Richard Roderick <[EMAIL PROTECTED]> writes: Richard> I saw fastforward, I just didn't have a clean Richard> understanding of the capabilities and how it could be Richard> used to solve this problem. :) Richard> I thought you did a great job explaining the Richard> virtusertable conversion process, that's what I think Richard> needs to be added. Perhaps to the HOWTO or David Sill's Richard> "Life with Qmail". Qmail is a pretty flexible tool. It's hard to envision what everyone might want to do with it, but I agree that converting from sendmail's virtusertable is likely a common scenario. I'd be surprised if a search of the qmail list archive didn't turn this up being discussed before. It was clear to me how to piece together qmail's various components to arrive at the solution I needed. Hmm, maybe I'll expand a little on my post and ask to have it added to the qmail FAQ. It would certainly complement the "How do I use sendmail's /etc/aliases with qmail?" question: "How do I use sendmail's /etc/virtusertable with qmail?" Looks like DJB maintaines the FAQ? Dan - interested in contributions? j. -- Jay Soffian <[EMAIL PROTECTED]>UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Sendmail Virtusertable equivalent?
"Richard" == Richard Roderick <[EMAIL PROTECTED]> writes: Richard> AAAHHH. That's what I was looking for! The part about Richard> using the fastforward to do it was what I could not find, Richard> and I'm no expert. Richard> Someone needs to add this to the Qmail web site. It is. See 'Author's Enhancement Software for qmail': The fastforward package supports forwarding tables under qmail. j. -- Jay Soffian <[EMAIL PROTECTED]>UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Sendmail Virtusertable equivalent?
"Jay" == Jay Soffian <[EMAIL PROTECTED]> writes: Jay> [...] Jay> Don't forget to turn the virtualaliases file into a cdb: Jay> [...] I presume you've read the qmail instructions and are familiar with the basics of setup. I did leave out a pretty important step though... you need to add the virtual domains (the LHS in the virtusertable) to either /var/qmail/control/rcpthosts on /var/qmail/control/morercpthosts (I used morercpthosts, don't forget to run qmail-newmrh). j. -- Jay Soffian <[EMAIL PROTECTED]>UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Sendmail Virtusertable equivalent?
"Richard" == Richard Roderick <[EMAIL PROTECTED]> writes: Richard> In answer to the original question: It's not easy. I just Richard> finished developing the tools to deal with all of it. I handled this conversion in < 1 hour for over 30 domains. 300 or 3000 domains would have taken no longer since I built myself tools. I found it pretty damn trivial. Perl is your friend. But you could do this with sed, awk and grep. What I did: Grep out all the domains on the LHS from the sendmail virtualuser table and put them in /var/qmail/control/virtualdomains all with the same RHS like so: domain-a.com:alias-virtualdomains domain-b.com:alias-virtualdomains domain-c.com:alias-virtualdomains ... awk makes that pretty simple: awk '/^#/{next};/^ *$/{next};{split($1,a,/@/);print a[2]":alias-virtualdomains"}' \ < /etc/mail/virtusertable >> /var/qmail/control/virtualdomains Install fastforward (see http://www.qmail.org). Setup ~alias/.qmail-virtualdomains-default: echo "|/var/qmail/bin/fastforward -d /var/qmail/etc/virtualaliases.cdb" > ~alias/.qmail-virtualdomains-default Build /var/qmail/etc/virtualaliases.cdb. I'd use a perl script for that depending upon complexity. I'd send you the one I used if I could still find it... virtualaliases is similar to sendmail's virtusertable, but more flexible. For example, virtualaliases allows multiple addresses on the RHS. With sendmail, you can only use a single address on the RHS which you then need to expand in the aliases file to multiple addresses. If you only have a single address on the RHS in virtusertable, the conversion is simple, just add a ':' after every address in the LHS and add a ';' after every address on the RHS. You can do that with awk: awk '/^#/{next};/^ *$/{next};{print $1":\t"$2";"}' \ < virtusertable >> /var/qmail/etc/virtualaliases Clean up virtualaliases as needed, you'll want to fully qualify addresses on the RHS. If any of the RHS addresses actually appear in your /etc/aliases, then you'll want to expand those out. If you have a bunch, I suggest you write a script to iterate through all the LHS addresses and call sendmail -bv to expand out the RHS. Don't forget to turn the virtualaliases file into a cdb: setforward virtualaliases.cdb virtualaliases.tmp < virtualaliases Good luck. You'll be happier with qmail in the long run. Trust me. I've administered both for quite some time. Also, people on this list are more receptive if you lose the attitude. j. -- Jay Soffian <[EMAIL PROTECTED]>UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Maildrop samples
From: Keith Burdis <[EMAIL PROTECTED]> > There are some examples in the maildropex(5) man page. There are dozens more examples in the qmail-uce anti-spam package on Mr.Sam's page. http://www.geocities.com/SiliconValley/Peaks/5799/qmail-uce.html -- Jay Swackhamer <[EMAIL PROTECTED]> WCi system/network administrator
Re: ezmlm-manage acceping multiple domains in inhost
"Frederik" == Frederik Lindberg <[EMAIL PROTECTED]> writes: Frederik> You could install ezmlm-idx-0.322 (www.ezmlm.org) and Frederik> not worry about it any more. By default (as spam Frederik> counter-measure), ezmlm+idx requires the list name in Frederik> To/Cc, so for moving lists you need to disable this Frederik> (ezmlm-reject -T in DIR/editor). Cool, I'll look into that. Disabling the spam counter-measure is a non-issue since this is a moderated list. Frederik> You could also forward messages: ~/.qmail -> Frederik> newlist@newhost, ~/.qmail-default -> Frederik> newlist-$DEFAULT@newhost. I thought about that, but wouldn't that then change the envelope sender? I ended up installing mess822. The original list was [EMAIL PROTECTED], new list is [EMAIL PROTECTED] I run this list under the alias user. I set things up like so: /var/qmail/control/virtualdomains lists.storm98.com:storm98 /var/qmail/control/rewrite =lists.storm98.com:lists.storm99.com /var/qmail/alias/.qmail-/storm98-default |/var/qmail/bin/new-inject "-f$SENDER" Seems to work properly. It would have been easier to just use ofmipd, but ofmipd is an open relay. j. -- Jay Soffian <[EMAIL PROTECTED]> UNIX Systems Administrator 404.572.1941 Cox Interactive Media
ezmlm-manage acceping multiple domains in inhost
I have a list that was recently moved to a new hostname and I'd like ezmlm-manage to be able to accept messages at either address. That is, I'd like to be able to put multiple domains into inhost, but ezmlm-manage doesn't support this. So I can either patch ezmlm-manage or rewrite the incoming messages using new-inject from the mess822 package. Does this sound correct, or am I missing another option? Does anyone have any suggestions / pitfalls about doing this? My idea is to use virtualdomains to deliver to an alternate address that runs new-inject to rewrite the To address and then deliver to the canonical list address. Thoughts? j. -- Jay Soffian <[EMAIL PROTECTED]> UNIX Systems Administrator 404.572.1941 Cox Interactive Media
REPLY TO THIS ADDRESS -ERR user has no $HOME/Maildir
Sorry, I sent that last one using the wrong addy Hi,I'm trying to get the qmail pop3d working, and the error message I keepgetting when I am checking mail is -ERR user has no $HOME/Maildir/home/whoever/Mailbox exists, and I specify /bin/checkpassword/var/qmail/bin/qmail-pop3d Mailboxin inetd.confany ideas?
-ERR user has no $HOME/Maildir
Hi, I'm trying to get the qmail pop3d working, and the error message I keep getting when I am checking mail is -ERR user has no $HOME/Maildir /home/whoever/Mailbox exists, and I specify /bin/checkpassword /var/qmail/bin/qmail-pop3d Mailbox in inetd.conf any ideas?
ERR User has no $HOME/maildir (STILL GETTING ERROR)
OK, thanks for your help so far, but I am still getting the error /home/whoever/Maildir exists now, and it has mail in it but outlook express responds first with rejecting the password, but gives me that same error again. ERR User has no $HOME/maildir any other ideas?
Re: ERR User has no $HOME/maildir (STILL GETTING ERROR)
- Original Message - From: Chris Johnson <[EMAIL PROTECTED]> To: Jay <[EMAIL PROTECTED]> Sent: Monday, April 19, 1999 2:16 PM Subject: Re: ERR User has no $HOME/maildir (STILL GETTING ERROR) > On Mon, Apr 19, 1999 at 02:01:13PM -0400, Jay wrote: > > OK, thanks for your help so far, but I am still getting the error > > > > /home/whoever/Maildir exists now, and it has mail in it but outlook express > > responds first with rejecting the password, but gives me that same error > > again. ERR User has no $HOME/maildir > > Is Maildir a file or a directory? And if it's a directory, is it a maildir, as > created by /var/qmail/bin/maildirmake? > > Chris >
problem with Maildir
I am getting the error: unable to chdir to Maildir in /log/messages when I am trying to inject. I made a switch from mbox to Maildir, and my incoming mail doesn't seem to work anymore.. Anybody who can help?
using pine or sendmail
Question: I don't undertand why everyone is talking about pine and sendmail is it necessary to use another email program with qmail or just optional? Jay
Installing Qmail - HELP!
Hi, I have a question about installing qmailI am somewhat of a newbie to linux, however I have somehow managed toinstall it, and network it with two other windows machines...anyway, here is my question..I read the install instructions, but when I do the tests - this is whathappens for each testNOTE: - I have already made rc* in the var/qmail directoryIn my /var/log/messages I get that status: local 0/10 remote 0/20 line..whatever it is supposed to beWhen I type ps:Nothing QMAIL related comes up, even though I have initialized it withcsh -cf '/var/qmail/rc &'When I run the qmail-inject commandI get no mail (to get my mail I type #mail?, it says I have no mail forroot) however, in the /var/log/messages file it says "delivery (1-17 or so): sorry, although I'm listed as the best performance MX or A for that host,it isn't in my control/locals file, so I don't treat it as a local 5.4.6/"(along with a whole lot of other stuff... my qmail /control/locals file isas such:localdungeon.dyndns.orgmail.dungeon.dyndns.orglocalhost.dungeon.dyndns.org(but I just kind of guessed at this configuration..)Any Idea on how to get this working?Sencerely in suspense, Jay [EMAIL PROTECTED]P.S. here is a sample from my /var/log/messages fileApr 16 14:41:18 dungeon qmail: 924273678.890680 new msg 92382Apr 16 14:41:18 dungeon qmail: 924273678.896461 info msg 92382: bytes 231from $Apr 16 14:41:19 dungeon qmail: 924273679.010769 end msg 92382Apr 16 14:44:41 dungeon in.telnetd[2675]: connect from 209.239.8.50Apr 16 14:44:49 dungeon login[2676]: ROOT LOGIN on `ttyp1' from`roam.wk.execul$Apr 16 14:53:39 dungeon qmail: 924274419.048157 new msg 92382Apr 16 14:53:39 dungeon qmail: 924274419.049211 info msg 92382: bytes 218from $Apr 16 14:53:39 dungeon qmail: 924274419.206967 starting delivery 1: msg92382 $Apr 16 14:53:39 dungeon qmail: 924274419.207822 status: local 0/10 remote1/20Apr 16 14:53:39 dungeon qmail: 924274419.602018 delivery 1: failure:Sorry._Alt$Apr 16 14:53:39 dungeon qmail: 924274419.602941 status: local 0/10 remote0/20Apr 16 14:53:39 dungeon qmail: 924274419.863543 bounce msg 92382 qp 2694Apr 16 14:53:39 dungeon qmail: 924274419.867961 end msg 92382Apr 16 14:53:39 dungeon qmail: 924274419.874358 new msg 92385Apr 16 14:53:39 dungeon qmail: 924274419.877159 info msg 92385: bytes 877from $Apr 16 14:53:40 dungeon qmail: 924274420.136926 starting delivery 2: msg92385 $Apr 16 14:53:40 dungeon qmail: 924274420.138680 status: local 0/10 remote1/20Apr 16 14:53:40 dungeon qmail: 924274420.194083 delivery 2: failure:Sorry._Alt$Apr 16 14:53:40 dungeon qmail: 924274420.198476 status: local 0/10 remote0/20
sendmail like lusers in qmail
I'm trying to get qmail to support something like the LUSER support in sendmail. Here's the situation: This is a client machine. That is, it relays everything off of a mail hub (which happens to be running sendmail). I've set defaulthost and defaultdomain to "cimedia.com" so any mail sent to an unqualfied address on the machine is qualified with "@cimedia.com" and then forwarded to the mailhub (since I have also setup smtproutes to forward everything to the mail hub). fetchmail is used to grab mail for the machine and it delivers it through qmail-smtpd via tcpserver. That all works fine. The machine has a limited set of users, let's say it has root and bob. I'd like mail which originates on the machine but is addressed to either root or bob (that is, anyone in /var/qmail/users/cdb on this machine) to be qualified with /var/qmail/control/me instead of with defaulthost. Is there anyway to do this w/o wrapping qmail-inject or replacing it with new-inject? Can I even do this with new-inject? j. -- Jay Soffian <[EMAIL PROTECTED]> UNIX Systems Administrator 404.572.1941 Cox Interactive Media