Stripping binaries?
I'd like to set up my qmail-based mailhosts to strip attachments automatically, save them to a dir, and put a this attachment saved to /some/path message at the bottom... instead of distributing binaries to all and sundry. Anyone doing this with qmail yet, and/or got any hints? -- ~jv -- [EMAIL PROTECTED] : [EMAIL PROTECTED] -- ... threads rolling ... -- http://homepage.mac.com/seclorum/FileSharing.html
Single UID mailboxes - mail getting lost
I need to have a mail server that accepts mail from several domains. I read through your HOWTO and thought that would be the best way to implement it, however I have not had any success with it. When I try to send a message, it seems to go ok, there are no errors in the log files or anything, but the message seems to just disappear. To save time and space, I will give you the basic setup I have, listing only one user. The rest use the same setup, just change the names, etc. I created a user/group popuser (both have ID 101) The email address I will use is [EMAIL PROTECTED] The host machine is dogbert.slushpupie.com /var/qmail/control/users/assign: =jay-tarsk-com:popuser:101:101:/var/qmail/popboxes/tarsk-com/tarsk-jay::: /var/qmail/control/users/poppasswd: (the password in this case is tarsk) tarsk-jay:W.I8jJCHbKvBQ:popuser:/var/qmail/popboxes/tarsk-com/tarsk-jay /var/qmail/control/virtualdomains: tarsk.com:tarsk-com /var/qmail/control/rcpthosts: localhost dogbert.slushpupie.com slushpupie.com tarsk.com /var/qmail/popboxes/tarsk-com/tarsk-jay/.qmail: ./Maildir/ And here is what the syslog looks like when I try to send a message: Jul 30 13:22:21 dogbert qmail: 996517341.588676 new msg 109593 Jul 30 13:22:21 dogbert qmail: 996517341.589130 info msg 109593: bytes 233 from qp 4889 uid 64011 Jul 30 13:22:21 dogbert qmail: 996517341.596701 starting delivery 39: msg 109593 to local @dogbert.slushpupie.com Jul 30 13:22:21 dogbert qmail: 996517341.597090 status: local 1/10 remote 0/20 Jul 30 13:22:21 dogbert qmail: 996517341.600277 delivery 39: success: Jul 30 13:22:21 dogbert qmail: 996517341.600649 status: local 0/10 remote 0/20 Jul 30 13:22:21 dogbert qmail: 996517341.600799 end msg 109593 After sending the message there is nothing. No error, nothing bounces back, I check the queue dirs, and nothing in there. I am afraid this is a bit over my head, and could really use some help. Any help you can provide would be greatly appreciated. Thanks, Jay -- Jay Kline [EMAIL PROTECTED] http://www.slushpupie.com/
Re: Single UID mailboxes - mail getting lost
I agree... because I just got it working finally! I dont know what was wrong, but I reinstalled and started over. That seemed to work. About the only disadvantage to not using a vpop type manager is the automation. But even a simple perl script can solve that problem. Jay On Monday 30 July 2001 8:56 pm, you wrote: On Mon, Jul 30, 2001 at 10:55:10PM +0200, Henning Brauer wrote: Singe UID setups, usually called Virtual User Setups, are requiring a virtual user manager. That's not true. I've been using a roll-my-own single UID setup for ages, with no vpopmail or vmailmgr or anything but a custom checkpassword. It works beautifully and I don't have to worry about the security of non-DJB code (except for the cdb-enabled checkpassword, which I wrote myself). For someone with a small set of domains and a user base that doesn't change that often, I'd recommend a single-UID setup. It'll be more secure, and it'll give him a better understanding of how all the pieces of qmail work. Chris Content-Type: application/pgp-signature; charset=us-ascii; name=Attachment: 1 Content-Transfer-Encoding: 7bit Content-Description: -- Jay Kline [EMAIL PROTECTED] http://www.slushpupie.com A people living under the perpetual menace of war and invasion is very easy to govern. It demands no social reforms. It does not haggle over expenditures on armaments and military equipment. It pays without discussion, it ruins itself, and that is an excellent thing for the syndicates of financiers and manufacturers for whom patriotic terrors are an abundant source of gain. -- Anatole France
Re: Coding scripts for QMAILQUEUE patch
jdomingo == jdomingo Jos writes: jdomingo I'm sure this script is quite simple (maybe just opening jdomingo a couple of fd from which to read the messages from), jdomingo but I'd like to get a working script to build on it and jdomingo learn (a bash script would be perfect :). Here's one I wrote in Perl that you should be able to get started with: http://www.soffian.org/downloads/qmail/qqrbl j.
Remote server connects then QUIT's, why?
I've been seeing a lot of the following ever since I started accepting mail for a particular domain. 988341350.667080 tcpserver: pid 10036 from 138.220.29.7 988341350.668046 tcpserver: ok 10036 matrixit.net:216.58.86.3:25 :138.220.29.7::2037 988341350.672839 10036 220 matrixit.net ESMTP ready^M 988341350.739657 10036 EHLO worldbank.org^M 988341350.739847 10036 250-matrixit.net^M 988341350.739958 10036 250-SIZE 10485760^M 988341350.740083 10036 250-PIPELINING^M 988341350.740182 10036 250 8BITMIME^M 988341350.808618 10036 QUIT^M 988341350.808958 tcpserver: end 10036 status 0 The remote mail server at 138.220.29.7 just connects, then QUIT's immediately, NOT delivering the mail it has for me (and yes, it does have some). So it's not like qmail is refusing the mail, the remote just decides to QUIT before sending it. It has been doing this every 5 minutes over the past few days. Everything else is honky-dory. I'm totally at a loss for why this is happening. Any possible explanations? From the occasional messages I have received, it appears to be running Lotus Domino Release 5.0.5 and Mailsweeper SMTPRS 4.2.1 Received: from worldbank.org ([138.220.29.7]) by WBLN0014.worldbank.org (Lotus Domino Release 5.0.5) with ESMTP id 2001042623080385:20693 ; Thu, 26 Apr 2001 23:08:03 -0400 Received: from matrixit.net (matrixit.net) by worldbank.org (Content Technologies SMTPRS 4.2.1) I'm going to contact the postmaster, but I doubt that'll give me any clue.
argh
Anyone else have trouble unsubscribing to the list? Not that I don't love you guys or anything, I just need to get off of some these lists that I'm on. I've tried unsubscribing 5 separate times. I have confirmation messages that say I have been removed from the list, and I stop receiving messages for anywhere from 2 hours to a day, and then I'm magically back on the list again. How do I get off the list and stay off? -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: Using a RAMDISK for /var/qmail/queue thoughts ?
-Original Message- From: Sean Reifschneider [mailto:[EMAIL PROTECTED]] Sent: Wednesday, December 13, 2000 12:12 PM 10ns is much faster than 5.3ms... It works, I've done it, it's reasonably I was thinking about doing this awhile back. I made a ramdisk and used both bonnie and bonnie2 to do some benchmarking on it, and I actually got worse performance on the Ramdisk. You still have the overhead of the file system, and you have no hardware DMA controller with a Ramdisk. It was slower in my experience. The box I tested it on had 1GB of Ram, and a 512MB ramdisk. No swap was being used during the tests. Jay -Original Message- From: Sean Reifschneider [mailto:[EMAIL PROTECTED]] Sent: Wednesday, December 13, 2000 12:12 PM To: Greg Cope Cc: qmail mailing list Subject: Re: Using a RAMDISK for /var/qmail/queue thoughts ? On Wed, Dec 13, 2000 at 04:20:19PM +, Greg Cope wrote: Has anyone any empirical evidence for the speed increases I may expect (as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI system (eg 10k, 5.3 ms seek 25mb/s) ? 10ns is much faster than 5.3ms... It works, I've done it, it's reasonably fast, but you still have to worry about things like swamping the todo and on top of that you may have to worry about filling up your queue disc. You can get QMail into a situation where it's completely wedged until you manually remove some files from the ram disc to give it enough space to continue delivering mail. Sean -- I never thought I'd live in a country where physical violence would be used to disenfranchise voters. Have you heard about Bush supporters rioting? Sean Reifschneider, Inimitably Superfluous [EMAIL PROTECTED] tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python
RE: Newsletter
Not if you set the security options. I have mine set up so that only moderators can post, others get a message that tells them they don't have permission. It's in the FAQ at http://www.ezmlm.org. Jay -Original Message- From: Fernando Costa de Almeida [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 01, 2000 9:23 AM To: Charles Cazabon Cc: qmaillist Subject: Re: Newsletter Charles Cazabon wrote: Fernando Costa de Almeida [EMAIL PROTECTED] wrote: 1) Is there a specific newsletter software that works together qmail? Yes. ezmlm, ezmlm-idx both work natively with qmail. Majordomo can be made to work with qmail. See www.qmail.org; mailing lists are mentioned a few times there Ok, but if the user that receives the email see the header: Delivered-To or Mailing-List and sends a email to this address, he will send a email to MY newsletter. 2) Has anyone applied the big-concurrency patch in a Red-Hat 6.2 System? Yes. Or did you want some specific piece of information? Yes, I cant do this because when I try to recompile qmail, it shows a msg like this "Your system has a hidden FD_SET() limit"... I dont know what to change in the kernel to made this work This is the piece Im looking for. Thanks, Charles -- -- - Charles Cazabon [EMAIL PROTECTED] GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ Any opinions expressed are just that -- my opinions. --- -- To boldly go where I surely don't belong.
RE: people are definately starting to harvest emailadresses on this list...
Why are you such an asshole? Who's the owner of this list? I'm getting sick of hearing Felix's shit. -Original Message- From: Felix von Leitner [mailto:[EMAIL PROTECTED]] Sent: Saturday, October 28, 2000 6:56 PM To: Qmail mailing list Subject: Re: people are definately starting to harvest emailadresses on this list... Thus spake Martin Jespersen ([EMAIL PROTECTED]): Nice to see that people are able to be constructive around here *pats Felix on his little head* While we are talking about "constructive", please construct yourself a gut and shoot yourself, idiot. Felix
RE: How many outbound messages can you send per hour?
The reason it's so slow is a large number of the connections are most likely being tied up for long periods by mailservers than don't respond. About 80 of my 120 connections when I sent out mailings would do the same thing. Apply the big concurrency patch. My concurrency is at 508 now, and I can do millions of messages per day. During sending, about 100-120 of the connections are tied up by slow or nonresponding mailservers. I'd raise my concurrency higher, but I have to make some kernel mods first (FD_SET problem). Jay -Original Message- From: Greg Jorgensen [mailto:[EMAIL PROTECTED]] Sent: Friday, October 27, 2000 3:44 PM To: qmail Subject: How many outbound messages can you send per hour? We send lots of newsletters and other subscription-type emails. Our qmail server seems to max out at 10,000 outbound messages per hour. I already have concurrencyremote set to 120, and reverse DNS lookups and other slowdowns turned off. The CPU (a dual UltraSparc) is not maxing out; it looks like we've reached a limit of the SMTP protocol and the number of connections allowed. I can see that separating outbound and incoming mail on different servers would help; most of the bounces come in while the bulk of the subscriptions are going out. We are going to add some servers, but I'm wondering if 10,000/hour is a typical limit, or is there some way I can send more messages in an hour? Thanks! = Greg Jorgensen Deschooling Society Portland, Oregon, USA [EMAIL PROTECTED] __ Do You Yahoo!? Yahoo! Messenger - Talk while you surf! It's FREE. http://im.yahoo.com/
fixcrio
I'm calling tcpserver with this line: tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u 503 -g 503 0 smtp /var/qmail/bin/qmail-smtpd 21 | /var/qmail/bin/splogger smtpd 3 I need to use fixcrio to fix stupid emailers that put stray lf's in their messages. How do I integrate fixcrio into this? Do I just do: tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u 503 -g 503 0 smtp /usr/local/bin/fixcrio | /var/qmail/bin/qmail-smtpd 21 | /var/qmail/bin/splogger smtpd 3 Jay
RE: How to ignore .qmail-* files for given users?
If you're using wu-ftpd or some derivative of, you should be able to do this in the /etc/ftpaccess file. Do a 'man ftpaccess' for the correct syntax. Jay -Original Message- From: John Chronakis [mailto:[EMAIL PROTECTED]] Sent: Monday, October 23, 2000 9:55 PM To: qmail-list Subject: How to ignore .qmail-* files for given users? Hello, I would like to prevent users with ftp access to change the delivery instructions for qmail. To be more specific, I want some users not to be able to receive mail at all (they all bellong to the same group) and others not be able to change delivery instruction by creating their own dot-qmail files via ftp (they all belong to another group). I suppose that this is controled by QMAILHOME/users/assign, but the man page is not very helpful. Can anyone help? John
statistics
Does anyone know of any log analyzers for qmail? I need stats on how many messages are going out, how many of those are bouncing, speed, and whatever else I can find. What about something that would offer real-time stats, like number of concurrent connections, messages waiting in queue, current sending rate, etc.? -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: statistics
Hrm, I grabbed qmail analog, I piped one of my logfiles through tai64nfrac and into zoverall, and I get 0 for completed requests and total delivery attempts. If I put it through any other program that came with qmail analog, I get no output. Any idea why this would happen? tai64nfrac works just fine too. Jay -Original Message- From: Peter Green [mailto:[EMAIL PROTECTED]] Sent: Friday, October 13, 2000 10:15 AM To: Andy Abshagen Cc: Qmail Mailing List Subject: Re: statistics also sprach andy: Where are the MRTG pieces for qmail? What Does The Archive Say(TM)? (See http://www-archive.ornl.gov:8000/. Search on qmail-mrtg.) (Hint: http://www.prodigysolutions.com/qmail-mrtg.1.0.tar.gz :) /pg -- Peter Green : Gospel Communications Network, SysAdmin : [EMAIL PROTECTED] --- /* * [...] Note that 120 sec is defined in the protocol as the maximum * possible RTT. I guess we'll have to use something other than TCP * to talk to the University of Mars. * PAWS allows us longer timeouts and large windows, so once implemented * ftp to mars will work nicely. */ (from /usr/src/linux/net/inet/tcp.c, concerning RTT [round trip time])
RE: A bug or am I being daft?
Doesn't the case change violate RFC821 or 822? I seem to remember reading that case in the user portion of the email address should never be changed because the accounts "Bob" and "bob" are two completely different accounts on a unix machine. Jay -Original Message- From: Kai MacTane [mailto:[EMAIL PROTECTED]] Sent: Friday, October 13, 2000 1:07 PM To: [EMAIL PROTECTED] Subject: Re: A bug or am I being daft? At 10/13/00 10:47 AM , Ben Cody Houston wrote: Basically, I'm trying to deliver mail to: [EMAIL PROTECTED] so, I have an ~alias/.qmail-Bob:Hanson file... but it won't work. It will work if it's ~alias/.qmail-bob:hanson - should either one work? No, only the latter one should work. See http://cr.yp.to/qmail/faq/incominguser.html#alias-dots. As it says, "Dots are converted to colons, and uppercase is converted to lowercase." --Kai MacTane -- "Uh-oh... Gravity works." --Batty Koda, "Ferngully" (Hey, bats are gothic.)
RE: MTA and \r\n problems
Just had this exact same problem 2 days ago with ASPmail. The script was pulling some HTML from a web page and emailing it off. There was a lone linefeed character at the end of one of the tables on the page. One of our developer guys fixed the page somehow, or made the aspmail thing look for lone lf's. Someone mentioned that you could start smtpd with something like fixcrio | /var/qmail/bin/qmail-smtpd, but I haven't tried it yet. Jay -Original Message- From: Rich Feather [mailto:[EMAIL PROTECTED]] Sent: Friday, October 13, 2000 2:36 PM To: [EMAIL PROTECTED] Subject: MTA and \r\n problems Okay, I have inherited a legacy Active Server Pages/VB environment that uses ASPEmail to send HTML email messages to a qmail relay server. I keep getting the dreaded LF bounce. After talking to Persits Software (the makers of the component), they're convinced that qmail is stripping the CRLF and replacing it with simply a LF. My questions are, is this a reasonable behavior from qmail and, if so, how would I fix it? Thanks.
RE: smtp speed limit??
Woohoo! Figured it out. The stupid windows mailserver didn't log anything useful, so I took one of the messages and used 'nc' to pipe it into qmail. The message was autogenerate by some .asp file somewhere and had a stray lf in it. Jay -Original Message- From: Aaron L. Meehan [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 12, 2000 11:54 AM To: '[EMAIL PROTECTED]' Subject: Re: smtp speed limit?? Quoting Austad, Jay ([EMAIL PROTECTED]): I'm looking here: http://cr.yp.to/docs/smtplf.html Maybe one of the messages contains an lf not preceded by a cr. Where would this be logged on the qmail box? Unfortunately, in the stock qmail, nowhere. However, you may start up your favorite packet capturing software, sniffit or its equivilent, and see what is going on between the two servers easily enough. I've never used it, but there is a djb piece of software known as recordio that you can use to log every smtp command sent or received by qmail-smtpd. Again, though, if this is the problem the exchange server should be logging qmail's smtp response! If there's a stray newline, qmail-smtpd will spit out a 4xx error code and the URL quoted above. Aaron
RE: concurrencyremote
I had the same problem for awhile at around 300 something concurrency. In /proc/sys/fs, you need to increase the values of some stuff in there. I just doubled the value of everything until it worked. :) Jay -Original Message- From: Ricardo Albano [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 3:17 PM To: Doug Schmidt; 'Kris Kelley'; QMail Mailing List Subject: Re: concurrencyremote This is probably an OS limit, I think has relation with the max. file descriptor that the system can handle, try setting a line line 'ulimit -n 65000' in the script that lunch the qmail-smtpd process. What OS/Version are you running ? RDA.- -Original Message- From: Doug Schmidt [EMAIL PROTECTED] To: 'Kris Kelley' [EMAIL PROTECTED]; QMail Mailing List [EMAIL PROTECTED] Date: Wednesday, October 11, 2000 4:44 PM Subject: RE: concurrencyremote Yep, I found the problem right after sending the message to the list. control/* are owned by root:root w/ 644 permissions. When I originally created the file as root, because of umask, permissions were 640 all is working fine now w/ concurrencyremote at 30. When I make it 40, I get messages in my maillog that say: Oct 11 11:24:18 server qmail: 971277858.753401 delivery 1078: deferral: qmail-spawn_unable_to_create_pipe._(#4.3.0) Is this becuase the server is running out of memory? ~Doug -Original Message- From: Kris Kelley [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 11:29 AM To: QMail Mailing List Subject: Re: concurrencyremote Doug Schmidt wrote: I would like to increase qmail's concurrencyremote from the default 20 to 40. When I create: /var/qmail/control/concurrencyremote and put a value of 40 in the file, I restart qmail and get the error: Oct 10 16:53:14 server qmail: 971211194.211356 alert: cannot start: unable to read controls Could be a matter of permissions. What are the permissions of your control directory and the control files now? On my machine, the control directory is owned by root:qmail with 755 permissions, and all the files within are owned by root:root with 644 permissions. ---Kris Kelley
smtp speed limit??
We have a little script that sends 5 different messages in rapid succession to one email address. When it gets pointed at our mailserver running MS Exchange, it works fine, however, when pointed at the qmail box, only the first message makes it through, the other 4 get delayed or deffered and put back into the queue (MS Exchange or some MS mailserver thing). Does qmail limit how fast one host can send messages via smtp? Here's my tcpserver line: tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u 503 -g 503 0 smtp /var/qmail/bin/qmail-smtpd 21 | /var/qmail/bin/splogger smtpd 3 -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: smtp speed limit??
It's the builtin MS smtp server that come with win2000 internet services. I couldn't find logs for it, but I forgot to look in the stupid Event viewer. Here's what I just found: "Message delivery to the remote domain 'marketwatchmail.com' failed. The error message is 'The connection was dropped by the remote host.'." Any ideas why it would drop the connection after only receiving one message? I couldn't find any errors on the qmail box. Jay -Original Message- From: Aaron L. Meehan [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 5:37 PM To: '[EMAIL PROTECTED]' Subject: Re: smtp speed limit?? Quoting Austad, Jay ([EMAIL PROTECTED]): We have a little script that sends 5 different messages in rapid succession to one email address. When it gets pointed at our mailserver running MS Exchange, it works fine, however, when pointed at the qmail box, only the first message makes it through, the other 4 get delayed or deffered and put back into the queue (MS Exchange or some MS mailserver thing). Fascinating. I wonder if your exchange server bothers to log the response from the remote mail server upon delivery, failure, or deferral? That would help (and even more to show them to us -- I think that shall be my mantra from now on). Does qmail limit how fast one host can send messages via smtp? Here's my tcpserver line: tcpserver -q -c 500 -x /etc/smtp.cdb -H -l mail.marketwatchmail.com -R -u 503 -g 503 0 smtp /var/qmail/bin/qmail-smtpd 21 | /var/qmail/bin/splogger smtpd 3 Since tcpserver is handling the smtp connection, the proper question would be whether tcpserver itself can limit how fast it will receive mail. The answer to the question as you put it is "no." However, tcpserver can limit the number of smtp connections it will accept. The default is 40 simultaneous connections, and you have actually specified 500. You would not seem to be crossing that threshold, but I'm just guessing without seeing any logs--tcpserver's logging will show if you're going over 500 connections. Personally, my bet is that the exchange server is puking. Five messages in rapid succession.. that must put real stress on the bloatware. Aaron
RE: smtp speed limit??
I'm looking here: http://cr.yp.to/docs/smtplf.html Maybe one of the messages contains an lf not preceded by a cr. Where would this be logged on the qmail box? -Original Message- From: Andy Bradford [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 6:31 PM To: Chris Johnson Cc: '[EMAIL PROTECTED]' Subject: Re: smtp speed limit?? On Wed, 11 Oct 2000 19:12:42 EDT, Chris Johnson wrote: If I were troubleshooting a communication problem between two hosts, and on one end of the connection was some piece of Microsoft software, and on the other end of the connection was some software written by Dan Bernstein, can you guess where I'd start looking for the problem? I know where I would look, but just because it is DJB's software doesn't mean that it was configured properly. :-) Andy
RE: Running Multiple Copies of Qmail on the same server...
You're better off with RAID 0+1. I run RAID 5 on mine and when it runs through the queue, disk IO is the bottleneck. -Original Message- From: James Stevens [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 7:45 PM To: Michael Boyiazis; Qmail Subject: Re: Running Multiple Copies of Qmail on the same server... Thanks! .. ;) The machine I'm running qmail on has like a full c block assigned to it and a RAID5 array so I don't think the disk i/o will be a problem ?? But if needed I also have a RAID1 Array setup on it with twin 20gigs so I could place the secound qmail over there but I think the RAID5 should be fast enough for it Thoughts? --JT - Original Message - From: "Michael Boyiazis" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 10, 2000 5:38 PM Subject: RE: Running Multiple Copies of Qmail on the same server... We have found inbound mail to be very disk i/o bound w/o doing much to the cpu. so we added another disk and have two instances running. it lets us handle twice the load. you need the box to handle 2 IPs; for the second instance recompile w/ the value in: /export/home/qmail-1.03/conf-qmail to hold the home of the second queue, say /var/qmail2 instead of the default /var/qmail the spot in your tcpserver line that says 0 smtp should be changed to be: mail_instance_1.domain.com smtp and repeat the tcpserver startup for another instance. make sure to both qmail instances are started in your init script. -- Michael Boyiazis Mail Architect, NetZero, Inc. -Original Message- From: Goran Blazic [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 5:23 PM To: 'James Stevens'; Qmail Subject: RE: Running Multiple Copies of Qmail on the same server... I dont really see no good point on why you would want to run multiple copies of qmail... Or what you would understand by that ??!!?? Goran -Original Message- From: James Stevens [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 2:19 AM To: Qmail Subject: Running Multiple Copies of Qmail on the same server... Can someone point me to a web page that has some explanation of setting up concurrent running qmails on the same machine and what edits I need to make to avoid conflicks.. Thanks in advance.. --JT
RE: Running Multiple Copies of Qmail on the same server...
I'm going to have to put some QMQP servers in our other data centers to get more speed out of it. We've got 90Mbit/sec worth of bandwidth at each one of those, but it'll burst higher. I noticed that when the messages are broken up into 20,000 rcpt chunks, they go out way faster than a bunch of 150,000 rcpt chunks. At our current growth rate, we expect over 5,000,000 users per list during the first part of 2001. I just have to figure out how to increase the disk IO performance or reduce the need for it on the list server box. Jay -Original Message- From: James Stevens [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 8:03 PM To: Austad, Jay; Qmail Subject: Re: Running Multiple Copies of Qmail on the same server... ROFL, ok.. I feel small now ... ... ... ... --JT - Original Message - From: "Austad, Jay" [EMAIL PROTECTED] To: "'James Stevens'" [EMAIL PROTECTED] Sent: Tuesday, October 10, 2000 5:56 PM Subject: RE: Running Multiple Copies of Qmail on the same server... Here's what I did: I have a main server running qmail and ezmlm. I set up ezmlm to use qmqp to send it's messages. I modified qmail-qmqmc.c to randomly pick a QMQP server instead of just choosing the first one. Each list is broken up into 52 sublists. Right now, I only have 3 QMQP servers. When a message goes out, it randomly distributes 52 separate messages between 3 QMQP servers, which ends up being pretty even. All of the sending is offloaded to other machines, and split up, so the messages go out super fast. 1,000,000 addresses in under 15 minutes. :) I saturate one of our DS3's everytime it runs. My modifications can support up to 255 QMQP servers. The bounces are starting to load down the main box though, I'm going to have to lower the ezmlm-warn timeout to about 4 days, it's still at 11.6. Jay -Original Message- From: James Stevens [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 7:38 PM To: Goran Blazic; Qmail Subject: Re: Running Multiple Copies of Qmail on the same server... Well when a list of 200k+ is mailing and another list gets fired up behind it I would like the server which sits almost twiddling it's thumbs and has all kinds of resources left over to be able to send out to more than one list at a time without a queue delay Right now 'Majordomo' funnels all messages into qmail via qmail-send and anything that gets queued after that has to wait for the current queue to get done. To get around this I plan on setting up multiple qmails under the same server each having it's own queue ofcourse and then each client would have his or her own smtp server for sending to there lists on. Meaning that with multiple qmails running one client could send to her or his list and a secound client could send to her or his list and a third client could send to her or his list and I could send to my staff list and nowone would experience any delays and maybe just maybe my server might get above a 5% load avaerage for the first time in it's life. Anyways if anyone has actually done this please point me in the right direction I'd really appreciate it. ;) --JT - Original Message - From: "Goran Blazic" [EMAIL PROTECTED] To: "'James Stevens'" [EMAIL PROTECTED]; "Qmail" [EMAIL PROTECTED] Sent: Tuesday, October 10, 2000 5:22 PM Subject: RE: Running Multiple Copies of Qmail on the same server... I dont really see no good point on why you would want to run multiple copies of qmail... Or what you would understand by that ??!!?? Goran -Original Message- From: James Stevens [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 11, 2000 2:19 AM To: Qmail Subject: Running Multiple Copies of Qmail on the same server... Can someone point me to a web page that has some explanation of setting up concurrent running qmails on the same machine and what edits I need to make to avoid conflicks.. Thanks in advance.. --JT
RE: Running Multiple Copies of Qmail on the same server...
The queues aren't bad now, we're pretty good about prompty removing any addresses that are bad, and total garbage emails don't even get subscribed to the list. Doesn't really matter if we lose a queue. I did play with memory filesystems a couple of months ago, and I got worse performance on that than I did on the 30GB IDE drive on the machine! I think I may have figured out a way to distribute the queue across multiple servers. I just have to figure out if ezmlm would still be able to handle bounces OK. Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 10, 2000 8:43 PM To: Qmail Subject: Re: Running Multiple Copies of Qmail on the same server... On Tue, Oct 10, 2000 at 08:08:03PM -0500, Austad, Jay wrote: I'm going to have to put some QMQP servers in our other data centers to get more speed out of it. We've got 90Mbit/sec worth of bandwidth at each one of those, but it'll burst higher. I noticed that when the messages are broken up into 20,000 rcpt chunks, they go out way faster than a bunch of 150,000 rcpt chunks. At our current growth rate, we expect over 5,000,000 users per list during the first part of 2001. I just have to figure out how to increase the disk IO performance or reduce the need for it on the list server box. How big are your queues? (in disk space terms)? Can you afford to lose a queue occasionally? Can you afford a ram disk? Just as an exercise you might want to run a queue as a memory file system for a little while, just to see the sort of benefit you could gain. Regards.
RE: spam alarm as result from help with girlfriend
I got one of those too. Lily is not cute. :) -Original Message- From: Don Wright [mailto:[EMAIL PROTECTED]] Sent: Sunday, October 08, 2000 1:27 PM To: Qmail Subject: RE: spam alarm as result from "help with girlfriend" From: Alexander Jernejcic i think "Wheres Mybrudda [mailto:[EMAIL PROTECTED]]" did gather adresses for spam-mail! just received spam from "From Lily [[EMAIL PROTECTED]]" It doesn't have to be from that hotmail account, it could be any lurker on the list, or it could be from the maillist to newsgroup mirror at newsgate.muc.de. Check news:muc.lists.qmail. (Bet you didn't know you were a USENET star.) Here are the headers from your message, as posted to the newsgroup. More than enough opportunity for anonymous harvesting. Note: Giganews is my newsfeed provider. Path: news3.aus1.giganews.com!nntp3.aus1.giganews.com!nntp.giganews.com!feed1.news .rcn.net!rcn!news.maxwell.syr.edu!tank.news.pipex.net!pipex!do.de.uu.net!inf ormatik.tu-muenchen.de!news.muc.de!newsgate.muc.de!not-for-mail From: [EMAIL PROTECTED] (Alexander Jernejcic) Newsgroups: muc.lists.qmail Subject: spam alarm as result from "help with girlfriend" Date: 8 Oct 2000 12:08:55 +0200 Organization: Newsgate at muc.de e.V. Lines: 22 Approved: [EMAIL PROTECTED] Distribution: world Message-ID: [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] NNTP-Posting-Host: marvin.muc.de Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Trace: marvin.muc.de 970999735 12097 193.149.48.2 (8 Oct 2000 10:08:55 GMT) X-Complaints-To: [EMAIL PROTECTED] NNTP-Posting-Date: 8 Oct 2000 10:08:55 GMT Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm Precedence: bulk X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Importance: Normal X-Newsgate-CVS-Id: $Id: newsgate.pl,v 1.2 1998/10/22 13:23:53 mort Exp $ X-No-Archive: Yes Xref: nntp3.aus1.giganews.com muc.lists.qmail:39178
RE: qmail list reply-to
Considering that the majority of Internet users these days are so young that the have never seen carbon paper, Regardless of age, if you were a trouble maker in school, you got quickly acquainted with it. In 4th grade, I remember having to write 5000 times: "When singing Old Dan Tucker, I will not call him Old D*mn F*cker". Carbon paper saved me about 4000 sentences. My parents were not pleased. -Original Message- From: Chris Garrigues [mailto:[EMAIL PROTECTED]] Sent: Monday, October 09, 2000 1:53 PM To: Brett Randall Cc: Robin S. Socha; qmail Subject: Re: qmail list reply-to From: "Brett Randall" [EMAIL PROTECTED] Date: Mon, 9 Oct 2000 07:00:06 +1000 Then the sender should ask for a Cc: - remember kids, it isn't called Courtesy Copy for nothing. I thought it was Carbon Copy? Considering that the majority of Internet users these days are so young that the have never seen carbon paper, that term seems to be as obsolete as "dialing" a telephone. At Stan Freburg said, "That went out with button shoes!" Chris -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 4314 Avenue C Austin, TX 78751-3709 +1 512 374 0500 My email address is an experiment in SPAM elimination. For an explanation of what we're doing, see http://www.DeepEddy.Com/tms.html Nobody ever got fired for buying Microsoft, but they could get fired for relying on Microsoft.
RE: check password
I don't know. But, if it's not currently possible, you could always hack RADIUS support into vpopmail. Then just set up a RADIUS server on your Domain controller for authentication. With some of the RADIUS libraries, it doesn't look that hard. Just a thought, I'm sure someone has a better way. Jay -Original Message-From: Stano Pa¹ka [mailto:[EMAIL PROTECTED]]Sent: Friday, October 06, 2000 12:03 AMTo: qmail konferenciaSubject: Re: check password No one knows? No one answers... Stano. - Original Message - From: Stano Pa¹ka To: qmail konferencia Sent: Monday, October 02, 2000 7:03 AM Subject: check password Can I use users/passwords from WIN NT server for qmail/vpopmail? And how? Stano.
RE: Help with my girlfriend?
If she's just not paying attention to you, you should probably remove /dev/vbrtr0 and /dev/othrbf0. -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 5:46 PM To: qmail Subject: Re: Help with my girlfriend? On Fri, Oct 06, 2000 at 09:35:06AM +1000, Brett Randall wrote: strace /dev/gf0 No, I think you've got it wrong. I think its strace /dev/gf6 at the moment... That depends on your interface renumbering standards. Rebooting in between relations is a good thing. Greetz, Peter -- dataloss networks '/ignore-ance is bliss' - me
RE: To send 500,000 messages
How personalized are they? Are you just putting the persons email address in the body of the message? the VERH patch for qmail will let you do this. We use this patch and a concurrency level of 509 and it puts out about 150,000 messages every 15 minutes. I run DJB dns with 100MB memory reserved for the cache so the mailservers won't have to wait long for queries. Overall, I'm really impressed with how fast qmail and djbdns work. I can saturate one of our DS3's by just sending out mail. :) Jay -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 7:48 PM To: Henrique Pantarotto; qmail Subject: RE: To send 500,000 messages 1) I have searched this mailing-list and I understood that Qmail can handle this demand very nicely, right? Are those patches for "high load servers" really necessary or is the default qmail distribution (with concurrencyremote set to 100 or something) enough? They're there for a reason...of course, these presume that your link can handle a greater concurrency. 2) Whats the best way to send the 500,000 mails? Send a single message with 500,000 destination addresses or to send 500,000 messages, one for each email address? One message with 500,000 destination addresses. Could you imagine opening up 500,000 smtp connections to a single server? Uh uh. 3) If I really need to send 500,000 different messages (for custom text inside the message) to the users, will this stress Qmail or will it handle it okay? qmail won't have a problem. But how big is your server? Your server will likely cack it if you send 500,000 messages from another machine. My best advice? Generate a template with metatags in it (like name, company), have a database ready, and run it through a perl script that calls qmail-inject. If you don't need to customise messages, consider using ezmlm-idx. It will remove addresses that bounce after 14 days if they can't be resolved, it is easy to administer, users can subscribe and unsubscribe at whim, it can me moderated so messages can't be sent to the mailing list unless you authorise them to be... /BR Manager InterPlanetary Solutions http://ipsware.com/
RE: To send 500,000 messages
I also had to use the big concurrency patch. I do a single message with 150,000 recipients. Our lists range from 80,000 people to around 200,000 people depending on the list. I have multiple QMQP servers that do the sending and the main box load balances between the QMQP servers. So if I break the messages into smaller groups of recipients, it will most likely go out faster since I have more than one server doing the sending. I haven't done this yet, but I'll eventually be setting up sublists in my ezmlm setup to do just this. All of our newsletters are stock market/financial related, so the ones that go out during the day are time sensitive. Jay -Original Message- From: Henrique Pantarotto [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 8:00 PM To: Austad, Jay; qmail Subject: RE: To send 500,000 messages Jay, I really loved this VERH patch. That's exactly what I need, to create a custom "To" field and perhaps a "click here to unsubscribe" field. To send 150,000 messages, do you send a single message with all the recipients or do you send multiple messages (multiple queues)? And another dumb question: did you apply any other patch to get concurrency level of 509 or you did that with Qmail as-is? Thanks a lot! ___ Henrique Pantarotto SysOp Site São Paulo Terra Networks Brasil S/A A Internet mais sua do que nunca Tel: (11) 5505-5728 r.316/238 ICQ: 6934285 IT: henpa [EMAIL PROTECTED] -Original Message----- From: Austad, Jay [mailto:[EMAIL PROTECTED]] Sent: quinta-feira, 5 de outubro de 2000 21:33 To: 'Brett Randall'; Henrique Pantarotto; qmail Subject: RE: To send 500,000 messages How personalized are they? Are you just putting the persons email address in the body of the message? the VERH patch for qmail will let you do this. We use this patch and a concurrency level of 509 and it puts out about 150,000 messages every 15 minutes. I run DJB dns with 100MB memory reserved for the cache so the mailservers won't have to wait long for queries. Overall, I'm really impressed with how fast qmail and djbdns work. I can saturate one of our DS3's by just sending out mail. :) Jay -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 05, 2000 7:48 PM To: Henrique Pantarotto; qmail Subject: RE: To send 500,000 messages 1) I have searched this mailing-list and I understood that Qmail can handle this demand very nicely, right? Are those patches for "high load servers" really necessary or is the default qmail distribution (with concurrencyremote set to 100 or something) enough? They're there for a reason...of course, these presume that your link can handle a greater concurrency. 2) Whats the best way to send the 500,000 mails? Send a single message with 500,000 destination addresses or to send 500,000 messages, one for each email address? One message with 500,000 destination addresses. Could you imagine opening up 500,000 smtp connections to a single server? Uh uh. 3) If I really need to send 500,000 different messages (for custom text inside the message) to the users, will this stress Qmail or will it handle it okay? qmail won't have a problem. But how big is your server? Your server will likely cack it if you send 500,000 messages from another machine. My best advice? Generate a template with metatags in it (like name, company), have a database ready, and run it through a perl script that calls qmail-inject. If you don't need to customise messages, consider using ezmlm-idx. It will remove addresses that bounce after 14 days if they can't be resolved, it is easy to administer, users can subscribe and unsubscribe at whim, it can me moderated so messages can't be sent to the mailing list unless you authorise them to be... /BR Manager InterPlanetary Solutions http://ipsware.com/
RE: Remotely subscribing multiple addresses
A solution someone gave me awhile back was to save the emails to a file, one per line, and do: cat emails.txt | xargs --max-args 20 ezmlm-sub ~/listname Worked great. -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Wednesday, September 20, 2000 8:28 PM To: Ben Beuchler; qmail Subject: RE: Remotely subscribing multiple addresses Is there any way for a remote admin to subscribe multiple addresses in one request? I'm guessing not, but it sure would be nice... A work-around way, but set up your own admin user (like multadmin), that has a .qmail file which pipes to a script you made up, which checks that the address it is coming from is an administrator's address, and then feeds the contents of the message line-by-line to ezmlm-sub. You might want to use some method of authentication in the message just so people don't learn about your multiple-subscription method and abuse it via a simple e-mail (could be as simple as your favourite word, or could be as complex and safe as public key encryption). /BR Manager InterPlanetary Solutions http://ipsware.com/
RE: concurrency remote patch
I grabbed the source rpm and just applied the patch to it and rebuilt it. Works great. Except, FD_SET is limited to 1024 descriptors. How do I change this? I assume I can't just echo something into /proc... I want to be able to do more than 509 concurrency. Jay -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Monday, September 18, 2000 7:11 PM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch On Mon, Sep 18, 2000 at 07:06:26PM -0500, Austad, Jay wrote: Unfortunately, I installed the boxes that need the modification with an rpm. I suppose this will possibly screw things up if I use some binaries from a tarball and some from the possibly modified rpm version. In that case, don't take any chances but rebuild the rpm's from the SRPMS, including the patch along the way. My rpm knowledge ends with that this is possible. Sorry :) Greetz, Peter -- dataloss networks '/ignore-ance is bliss' - me
RE: concurrency remote patch
Here's what I did to rebuild the rpm: rpm -ivh qmail-1.03-16.src.rpm cd /usr/src/redhat/SOURCES tar zxvf qmail-1.03.tar.gz patch -p0 big-concurrency.patch Edit qmail-1.03/conf-spawn down to 509 or less so it doesn't blow up because of the FD_SET descriptor limit rm qmail-1.03.tar.gz tar zcvf qmail-1.03.tar.gz qmail-1.03/ rm -rf qmail-1.03 Edit the init script to use multilog instead of splogger if you want cd ../SPEC rpm -ba qmail.spec Then your rpms should magically appear in /usr/src/redhat/RPMS/i386/ I guess I should've changed the name of the rpm and all that, but it was late and I was tired. :) As for the FD_SET problem, Dell sucks and ships a RAID card that requires a proprietary driver on their so-called "Linux approved" servers. It's a pain to recompile the kernel with any modifications because that damn module they have might not work. Everyone keeps pushing them to just release the source so it can be incorporated into the kernel, but they're being stupid about it. Jay -Original Message- From: James T. Perry [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 4:01 AM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch Hi Jay, "Austad, Jay" wrote: I grabbed the source rpm and just applied the patch to it and rebuilt it. Works great. Congrats! Please teach me how you did it! Except, FD_SET is limited to 1024 descriptors. Don't you hate it when that happens? ;) How do I change this? I assume I can't just echo something into /proc... I want to be able to do more than 509 concurrency. I did succeed to raise the concurrency level to 1000, but its an unofficial "dirty hack" and I can't be liable for any damages but here's what I did: (BTW, I am thinking of writing up a mini-howto) This was originally posted to the qmail list at about Mon, 18 Sep 2000 20:36:09 +0900 with the title : Re: conf-spawn and FD_SET SUCCESS! but updated a little since I forgot to include some more info. Warning/Disclaimer: This worked for me. I am not responsible if something breaks, or causes any damage to everything or anything related to the person following the steps below to modify their system and/or qmail setup (or whatever...). This was originally posted to the qmail list at about Mon, 18 Sep 2000 20:36:09 +0900 with the title : Re: conf-spawn and FD_SET SUCCESS! but updated since I forgot to include some more info. linux kernel 2.2.17 qmail-1.03 procmail rc file (install was accomplished by closely following lwq :) plus localtime, DNS, big-todo, big-concurrency patches in that order changes in qmail configuration: $qmailhome/control/concurrencylocal 500 $qmailhome/control/concurrencyremote 500 $qmailsrc/conf-spawn set to 1000 $qmailsrc/conf-split set to 100 $qmailsrc/conf-cc set to cc -O2 -D__FD_SETSIZE=2048 ucspi-tcp-0.88 daemontools-0.70 1. Inside the big-concurrency patch: it said to modify "/usr/src/linux/include/linux/tasks.h NR_TASKS from 512 to e.g. 2048" - directly quoted - which I did. 2. As P.Y. Adi Prasaja mentioned: raised the __FD_SETSIZE value in /usr/include/bits/types.h from 1024 to 2048 3. Just to be sure: raised the __FD_SETSIZE value in /usr/src/linux/include/linux/posix_types.h from 1024 to 2048 4. As Peter van Dijk mentioned: added -D__FD_SETSIZE=2048 to $qmailsrc/conf-cc 5. Edit the following: $qmailhome/control/concurrencylocal 500 $qmailhome/control/concurrencyremote 500 $qmailsrc/conf-spawn set to 1000 $qmailsrc/conf-split set to 100 $qmailsrc/conf-cc set to cc -O2 -D__FD_SETSIZE=2048 6. make setup: bingo! no compile errors. qmail is idling ok. 7. Change sources back to default value: (incase I break something ;) /usr/include/bits/types.h /usr/src/linux/include/linux/posix_types.h both back to 1024 #-#-#-#-#-#-#-# -- If somebody can help create a search engine for my room, I will call them a Saint... GUI == Graphical User Interference
RE: Problems receiving mail
If you're on a machine on the inside and you're trying to hit an ip on the outside of a firewall or router that does NAT, it won't work. I think Checkpoint makes a firewall that works around this problem, but that's the only one I can think of. You need to do your testing from a remote machine if you're hitting the 200.201.1.1 ip. Is 200.201.1.1 the real ip? I tried connecting to both port 80 and 25 and neither worked. In any case though, you won't be able to hit your external ip's from the internal network if the firewall is NATing them. Jay -Original Message- From: Wagner R. Landgraf [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 7:01 AM To: [EMAIL PROTECTED] Subject: Re: Problems receiving mail I cannot receive mail. Let's say that my internal IP is 192.168.1.20 (local net inside the firewall) and my external IP is 200.201.1.1 . If I do (from another machine in the local network) telnet 192.168.1.20 25 , it works fine, and I can send mail to the mail server using telnet commands. However, if I use telnet 200.201.1.1 25 , it can't connect. I though it was a firewall problem, but look at this: I have an http server running in another machine (192.168.1.2) in the local network. If I do telnet 192.168.1.2 80 it works ok, connecting to the port 80. If I do 200.201.1.1 80, it also can't connect. However, my http server is running and it's ok. So, it maybe not be a problem with firewall. Can someone help me and explain this? Thank you Wagner R. Landgraf Automa Consultoria Informática Ltda. [EMAIL PROTECTED] - Original Message - From: "Daniel Augusto Fernandes" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, September 18, 2000 5:15 PM Subject: Re: Problems receiving mail "Wagner R. Landgraf" wrote: Ok, now TELNET test is ok (I can receive mail in qmail sending it by telnet local connect). However, I cannot send remote-local mail. My qmail server is under a firewall. I've set the firewall to redirect mail packets (port 25) to qmail server. How are you going to be able to send mail packets to the internet? However, when I send mail to [EMAIL PROTECTED] (my IP address), I receive a Undeliverable message, saying that "The recipient name is not recognized". As in the man pages: qmail doesn't send msg to root@*. Can someone help me? I've also tried to set a POP account in my e-mail client, using my POP server as 200.201.34.197 (or even 192.168.1.20, my local IP address), but the client cannot connect to server. Any ideas? Thank you all Wagner R. Landgraf Automa Consultoria Informática Ltda. [EMAIL PROTECTED] -- Daniel Augusto Fernandes (DAF tm) [EMAIL PROTECTED] GCSNethttp://www.gcsnet.com.br/ Se você não encontra o sentido das coisas é porque este não se encontra, se cria. Antoine Saint-Exupéry
RE: concurrency remote patch
I haven't reached a concurrency greater than 100 (*blush*) yet so I can't say what would exactly happen when the concurrency really hits a high number - above the real 1024 limit (or 509 in qmail). I had both of my QMQP servers bouncing off of the 120 limit yesterday, and they were pretty much idle (Dell 2450's with 2 striped 9GB 10k rpm drives). I think even if I could get the concurrency up to 1024 or above, it still wouldn't be enough to make a difference on the box. I'll find out soon if I can make it bounce off of the 509 limit. Our Midday Market Report is due to go out within the hour. Hopefully when the next version of qmail comes out, it will have the big-concurrency and big-todo patch already installed. What happens if I start a second copy of qmail using /var/qmail2, different uids, and bind to another IP on the same box? Will I be able to do 509 concurrency out of each copy since they are running as different users? Jay -Original Message- From: James T. Perry [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 11:41 AM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch Hi Jay, "Austad, Jay" wrote: Here's what I did to rebuild the rpm: [snip] Thanks for the information! I gotta get used to building RPMs... (after all, I am using an RPM distro ;) As for the FD_SET problem, Dell sucks and ships a RAID card that requires a proprietary driver on their so-called "Linux approved" servers. It's a pain to recompile the kernel with any modifications because that damn module they have might not work. Everyone keeps pushing them to just release the source so it can be incorporated into the kernel, but they're being stupid about it. I know _exactly_ what you mean. (see RANT below) I guess I wasn't clear on the info I had previously posted. I didn't recompile the kernel... I just modified the sources to "goose" the qmail compile process and it somehow worked (call me kraziej :). I haven't reached a concurrency greater than 100 (*blush*) yet so I can't say what would exactly happen when the concurrency really hits a high number - above the real 1024 limit (or 509 in qmail). As for performance, my IDE ATA disk is slower than what qmail can really handle so setting the concurrency below 500 may not be a problem after all, now that I think of it... And procmail + /var/spool/mail is another "wide-load" I have which affects performance compared to Maildir. If I were to have a RAID 0+1 spinning above 1rpms, maybe a different story ( read smokin' gun :) cheers, jamie RANT Recently, I am getting more annoyed with big corporations leeching off on all of the efforts the open-source spirit has built up in the past years, giving the community not much good publicity nor credit in return either... Steal everything and yet spitting all over us. Sorry to mention it here folks - no flame please :) /RANT #-#-#-#-#-#-#-# -- If somebody can help create a search engine for my room, I will call them a Saint... GUI == Graphical User Interference
log analyzers
So what do most people consider the best log analyzer for qmail logs (I'm using multilog)? I'd like to see real-time stats if possible, or at least near realtime... :) Jay -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: qmail error
Make sure your files in /var/qmail/control have the correct settings in them. I ran into this yesterday and it turned out I had bad info in defaultdomain, locals, me, and plusdomain. Jay -Original Message- From: Jens Georg [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 2:56 PM To: qmail mailinglist Subject: qmail error hi, while sending emails to AOL qmail reports the following error-messages: Remote host said: 501 syntactically invalid HELO argument(s) Remote host said: 501 HELO requires domain address i did not find anything about this in the faqs. somebody here who can help me ? regards, jens
RE: concurrency remote patch
RAID 5 sucks for writes in the first place, but is excellent for reads. I'm running raid 5 on my mailing list box for availability reasons, but that distributes to my qmqp servers which are all RAID 0. I don't care that much if I lose a drive on the QMQP servers since I can have a new one built in about 15 minutes. I'll just lose my queue, which is only newsletter subscriptions anyway. It would suck to lose the queue, but it's not mission critical and the chances of it happening are low. Although, I'm sure it will happen sometime. Jay -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 19, 2000 2:30 PM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch On Wed, Sep 20, 2000 at 04:19:30AM +0900, James T. Perry wrote: [snip] I had both of my QMQP servers bouncing off of the 120 limit yesterday, and they were pretty much idle (Dell 2450's with 2 striped 9GB 10k rpm drives). Which RAID level? I remember somebody mentioning in this list that 0+1 will perform faster than 3 (or 5 obviously ;). I can't confirm this since I don't have that kind of artillery here at home. Anybody? I know that RAID5 sucks on Mylex DAC1100 controllers, and that RAID0+1 is blindingly fast :) Greetz, Peter -- dataloss networks '/ignore-ance is bliss' - me
concurrency remote patch
If I use this patch to increase the concurrencyremote limit to 65535, can I just compile qmail-remote and drop it in place? Or do I have to replace everything? Thanks. -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: concurrency remote patch
Unfortunately, I installed the boxes that need the modification with an rpm. I suppose this will possibly screw things up if I use some binaries from a tarball and some from the possibly modified rpm version. -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Monday, September 18, 2000 6:58 PM To: '[EMAIL PROTECTED]' Subject: Re: concurrency remote patch On Mon, Sep 18, 2000 at 06:54:41PM -0500, Austad, Jay wrote: If I use this patch to increase the concurrencyremote limit to 65535, can I just compile qmail-remote and drop it in place? Or do I have to replace everything? If you mean the big-concurrency patch, no. It changes the interface between qmail-send on one side, and qmail-lspawn/rspawn on the other side. You need to update at least these 3. I'd go for a - patch, make - stop qmail - make setup check - start qmail Greetz, Peter. -- [ircoper][EMAIL PROTECTED] - Peter van Dijk / Hardbeat [student]Undernet:#groningen/wallops | IRCnet:/#alliance [developer]_ [disbeliever - the world is backwards](__VuurWerk__(--*-
C API for queueing messages
Title: C API for queueing messages Hi All, We are trying to find out what is the most efficient way of queueing the message in qmail from our C Programs. So, would like to know if there are any C API's that are available for queuing messages. Basically I am looking for library routines that act as the native submission interface(API) for qmail. I would like to use the API's from my C Program that we currently have where we compose and build the messages and to queue the email message directly. Something similar to mail_open() and mail_close() provided by zmailer where you can submit messages composed in the MSG_RFC822 format. I can always call sendmail wrapper for queue-Inject or queue-inject itself from my C program thro a popen() or system() call, but I want to avoid the overhead of starting a new UNIX process for each and every email that we send and also popen() is not very efficient. Any help will be appreciated. Any other ideas are also most welcome Thanks and Regards, - Jay
RE: C API for queueing messages
Title: RE: C API for queueing messages Hi, Replies to your questions below. I don't know how zmailer does it, but qmail, postfix and sendmail all use a set*id program to protect their queue. (Does anyone know whether zmailer does a fork/exec under the covers of the API?) No, it does not do a fork/exec under the covers. From my understanding of the zmailer code, mail_open() it opens a file under an intermediate queue directory called router and returns a FILE pointer for us to write the message. Then a seperate router process picks up the message from there. The advantage here is that there is very little over head in writing a file as opposed a popen(sendmail -i -t) call. Have you measured that execing qmail-queue is not very efficient in the full context of your system or are you surmising this? Yes, we have done a lot of timing analysis and it was found that in excess of 50% of the processing time is being spent in the code segment of the application that invokes the sendmail exe thro popen(). We also explored the possibilities of using fork(), which is faster. But we are afraid that the system might run out of ulimit with all these small processes spawned. So now I understand why qmail does not have an API because otherwise the security is compromised. Thanks for your email. Regards, - Jay On Wed, Sep 13, 2000 at 11:58:04AM -0500, Jay Balakrishna wrote: Hi All, We are trying to find out what is the most efficient way of queueing the message in qmail from our C Programs. So, would like to know if there are any C API's that are available for queuing messages. Basically I am looking for library routines that act as the native submission interface(API) for qmail. I would like to use the API's from my C Program that we currently have where we compose and build the messages and to queue the email message directly. Something similar to mail_open() and mail_close() provided by zmailer where you can submit messages composed in the MSG_RFC822 format. I can always call sendmail wrapper for queue-Inject or queue-inject itself from my C program thro a popen() or system() call, but I want to avoid the overhead of starting a new UNIX process for each and every email that we send and also popen() is not very efficient. Any help will be appreciated. Any other ideas are also most welcome Thanks and Regards, - Jay
RE: Spamming .....
There is no way to send to 100k recipients at a time. Most spammers hire minimum wage workers to cut and paste messages into Outlook Express and send them to each recipient. You can always point your Outlook Express client at your qmail machine, but it's more efficient to just point it at the mailserver of your ISP. Always put your real email address in the messages. Most spammers make the mistake of putting fake ones and all of the potential customers can't respond. Usually a phone number is good also since many people prefer not to use email for business purposes. -Original Message- From: Rick Harris [mailto:[EMAIL PROTECTED]] Sent: Monday, September 11, 2000 8:55 PM To: Qmail Subject: RE: Spamming . Perhaps you should watch the list before you wish to declare yourself an evil spammer :) , but perhaps some other list members might have a viewpoint ? Rick -Original Message- From: Jerry Hsieh [mailto:[EMAIL PROTECTED]] Sent: Monday, September 11, 2000 8:41 PM To: Qmail Subject: Spamming . Hi, I would like to get into the spamming business and I have a basic qmail server setup already. I have no idea how to send mail to 100k receiptents at one time. Can someone give me some hints? Thanks for your time. Regards, Jerry
QMTP
Where would I find detailed specs on the QMTP protocol? I've found some stuff at http://cr.yp.to/proto/qmtp.txt, but I need more. We're writing a little piece of code that's going to sit on a couple of hundred Win2000 webservers that can talk QMTP to our qmail box for faster delivery. Users come to our site and sign up for automatically generated charts and other such things and the webservers generate the content for the email and send it off using SMTP right now. We need to speed this up as much as possible, and QMTP looks like the best way to do it since we already have a qmail box sending most of our other subscriber mail. We also have several QMQP servers. Would it maybe be better to make it talk QMQP to the boxes running qmail-qmqpd? = Here's my setup below in case anyone is wondering (warning: may be confusing without a picture): I have one main machine that runs ezmlm and a full version of qmail, it also runs a second copy of qmail (actually mini-qmail, we'll get to that in a minute). qmail-qmqpc is modified on the machine to load balance between all of my QMQP servers. I can add QMQP servers as necessary if the load gets to high on the rest of them. The full version of qmail is running to handle bounces, and to accept mail from webservers. Ezmlm-idx is set up to use QMQP for delivery, so messages being sent out get distributed to multiple QMQP servers. If the email comes in from a webserver, box running ezmlm would try to send it off itself... I didn't want that to happen because I want to keep the disk activity and other resources down as much as possible, so the logical choice would be to offload all mail not destined for the local box to the QMQP servers. Since I couldn't figure out a better way to do it, I installed a copy of mini-qmail with qmail-smtpd listening on port 26, and my modified qmail-qmqpc. smtproutes is set up so all outgoing mail goes to mini-qmail (on port 26) which distributes it to the QMQP servers. I wish smtproutes would let you specify a protocol to use... :) So the way this is set up, all outgoing mail gets sent to QMQP servers to let them handle the queueing and sending. Main destined for the local machine is still accepted and processed normally. Right now, I have 2 bottlenecks... The SMTP connection that the webservers use to send mail through it, and the SMTP connection to port 26 that the main copy of qmail redirects outgoing mail to. -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: QMTP
I understand it fine. I just have to get as much info as possible for our developers so they can code it up. Just want to make sure they don't have to do a ton of debugging due some subtle mistake somewhere. Jay -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Tuesday, September 05, 2000 12:51 PM To: '[EMAIL PROTECTED]' Subject: Re: QMTP On Tue, Sep 05, 2000 at 10:35:08AM -0500, Austad, Jay wrote: Where would I find detailed specs on the QMTP protocol? I've found some stuff at http://cr.yp.to/proto/qmtp.txt, but I need more. What part of that document is unclear to you, or what don't you understand? You might want to read serialqmtp.c (in serialmail) and qmail-qmtpd.c (in qmail) I am glancing over qmtp.txt and it's mostly quite clear to me, except the stuff about 'safe messages'. If none of the bytes in a safe message can be 0a, when the hell *do* we see a safe message? Looking at serialqmtp, it seems to just *disregard* this problem and pump the message into the packet. Cool, no more CRLF/LF conversions :) If it's netstrings that confuse you, read http://cr.yp.to/proto/netstrings.txt Greetz, Peter -- dataloss networks
RE: queueing and using qmqp at the same time
I don't want it to go to my smarthost, I want it to use qmail-qmqpc to send it to my QMQP servers. If only I could specify a protocol in smtproutes... Otherwise I could run "mini-qmail" on the same box with an smtp server listening on port 26, and use smtproutes to redirect outgoing mail to localhost:26. That way it would use my qmail-qmqpc to send it off to the QMQP servers. I want to use qmail-qmqpc because I modified to load balance between multiple servers, and I don't want to bog down my list server box with queueing outgoing mail. It's going to be ALOT of outgoing mail... Jay -Original Message- From: Charles Cazabon [mailto:[EMAIL PROTECTED]] Sent: Friday, August 25, 2000 9:43 AM To: '[EMAIL PROTECTED]' Subject: Re: queueing and using qmqp at the same time Chris Johnson [EMAIL PROTECTED] wrote: On Thu, Aug 24, 2000 at 04:37:41PM -0500, Austad, Jay wrote: I have a mail server that needs to receive mail for local users and handle bounces. However, I would like all outgoing mail to use my QMQP servers instead of being queued locally. Is this possible? No. You might look at Bruce Guenter's nullmailer, http://em.ca/~bruceg/nullmailer/ I've never used it, but it looks like it might do what you need. nullmailer doesn't provide an SMTP daemon for incoming mail; it is only for sending locally-injected mail from a dumb host to a smart relay. However, it's excellent for that. However, smtproutes could be used with qmail to provide this functionality, I think. Set up /var/qmail/control/locals to handle local deliveries, and put the single line :smarthost.domain into /var/qmail/control/smtproutes. This should make everything that can't be delivered locally be forwarded to your smarthost. Anyone care to correct me on this? Charles -- -- Charles Cazabon [EMAIL PROTECTED] QCC Communications Corporation Saskatoon, SK My opinions do not necessarily represent those of my employer. --
RE: linux filesystem
Wasn't there a problem with qmail and reiserfs awhile back? Seems to me qmail had to be slightly modified due to some file locking issues or something... I must've deleted the thread on it though. Jay -Original Message- From: Fucht, Rob van (ELS) [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 24, 2000 9:38 AM To: 'Van Liedekerke Franky'; 'qmail list' Subject: RE: linux filesystem You mean perhaps Reiserfs? (a journalling file system). Rob -Original Message- From: Van Liedekerke Franky [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 24, 2000 3:27 PM To: 'qmail list' Subject: linux filesystem A while back there was talk about a good filesystem for linux (next to ext2fs). Anybody remebers how this is called, and does anybody use it with satisfaction? I have looked through the archives, but I'm just not able to fill in the right search terms to find the answer... Franky
queueing and using qmqp at the same time
I have a mail server that needs to receive mail for local users and handle bounces. However, I would like all outgoing mail to use my QMQP servers instead of being queued locally. Is this possible? -- Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: Cool powered by qmail logo.
If anyone is interested in printing up a bunch of these, my friend works for a place here in Minneapolis called Signature Concepts and he gets a hefty discount (they do all of the University of MN stuff). We printed up some shirts for the DSM Racing club (http://www.dsm.org) and it cost us around $9 for a short sleeve and $12 for a long sleeve shirt. Good quality too, heavy cotton. Drop me an email if you're interested and I'll get you in touch with him. If you can send a JPG of what you want printed on the shirts, that would be even better. The more colors, the higher the cost. We had 2 colors on our shirts. The turn around time is usually pretty quick (a week or two depending on the season), so we had people send their money first so we knew how many to print up. Jay -Original Message- From: Russell Nelson [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 10, 2000 2:38 PM To: qmail Subject: Re: Cool powered by qmail logo. Keith Warno \(@HaggleWare.com\) writes: Whatever happened to the qmail shirt idea? Last I knew most people agreed "don't queue mail with sendmail; send mail with qmail" was the slogan of choice, but where'd it go from there? Remember this posting? Russ Nelson writes: Vern Hart writes: http://www.nerdgear.com/search.php?@category=100 Those prices at nerdgear are pretty good. Especially for embroidery. Even with shipping. The extra-large is $18.18 with shipping. I'll let the list know if the shirts don't suck. They don't suck. I've already worn it to a customer's site. :) -- -russ nelson [EMAIL PROTECTED] http://russnelson.com | If you think Crynwr sells support for free software | PGPok | health care is expensive now 521 Pleasant Valley Rd. | +1 315 268 1925 voice | now, wait until you see Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | what it costs when it's free.
RE: Protection
Put a .forward file in with this evil users new email address. Then all mail sent to them will really get to them. If nothing else, it will get them to unsubscribe from all of the lists. -Original Message- From: Brett Randall [mailto:[EMAIL PROTECTED]] Sent: Wednesday, August 09, 2000 7:26 AM To: qmail Subject: RE: Protection Set up an automatic revenge flood? Maybe not... : It depends if it is mailing lists or spam. First start by unsubscribing from REAL mailing lists. If it is spam, change your domain name...I would personally sue the ex user for breaching your 'reasonable use policy' (what? you don't have one? doh!) or at least for ongoing damages since you are now virtually permanently committed to wasting bandwidth on unsolicited e-mails. Only other option is to refuse the e-mails (ie using common spam killing techniques) at the last relay before it is transferred over your link. Brett Manager InterPlanetary Solutions http://ipsware.com/ -Original Message- From: Slider [mailto:[EMAIL PROTECTED]] Sent: Wednesday, August 09, 2000 7:06 PM To: [EMAIL PROTECTED] Subject: Protection Hi all, Please can you help with advise about protecting my mail servers from one of my on ex users!! He/She has subscribed to about 30 mailing lists with the address that falls under my mail service! I am now recieving about 10 mails a minute for that user! Removing the maildir and letting them bounce is not helping as I thought it would... any other suggestions?? Slider
RE: updated load balancing qmail-qmqpc.c mods
I agree with you, I forgot to mention that, sorry. I didn't have enough Mountain Dew yet. :) -Original Message- From: Michael T. Babcock [mailto:[EMAIL PROTECTED]] Sent: Wednesday, August 02, 2000 10:44 AM To: Austad, Jay; [EMAIL PROTECTED] Subject: Re: updated load balancing qmail-qmqpc.c mods Re-read my point: its unnecessary. I didn't say it wouldn't work. I said the CPU use of doing it this way was unnecessary over a simpler round-robin approach (After picking an initial random server). Note: I think using an array of pointers to server addresses would allow you to do your rotations a lot faster. Think "point quicksort". - Original Message - From: "Austad, Jay" [EMAIL PROTECTED] Even if one server is getting hit multiple times in a row, I don't think it will matter. I put in 10 servers in my qmqpservers file and some well placed printf's in qmqpc and ran it 100,000 times several times through. Each server was getting picked about 10% of the time +/- 1% or so. If we're sending millions of messages, this +/- 1% isn't going to make a significant difference.
updated load balancing qmail-qmqpc.c mods
Ok, I fixed it. It now just rotates the serverindex.pos[] by a random
amount, and then loops through until it finds a good server. rand() is still
seeded with milliseconds from the system clock. I used memcpy() to move the
array around instead of loops to make it more efficient.
Would it be possible to run qmqpc as some sort of daemon? Then it could
have memory of which server it contacted last and make sure it went to the
next one, instead of having the chance of picking the same one again.
Jay
#include sys/types.h
#include string.h
#include memory.h
#include sys/socket.h
#include stddef.h
#include netinet/in.h
#include arpa/inet.h
#include stdlib.h
#include sys/time.h
#include sys/timeb.h
#include "substdio.h"
#include "getln.h"
#include "readwrite.h"
#include "exit.h"
#include "stralloc.h"
#include "slurpclose.h"
#include "error.h"
#include "sig.h"
#include "ip.h"
#include "timeoutconn.h"
#include "timeoutread.h"
#include "timeoutwrite.h"
#include "auto_qmail.h"
#include "control.h"
#include "fmt.h"
#define PORT_QMQP 628
void die_success() { _exit(0); }
void die_perm() { _exit(31); }
void nomem() { _exit(51); }
void die_read() { if (errno == error_nomem) nomem(); _exit(54); }
void die_control() { _exit(55); }
void die_socket() { _exit(56); }
void die_home() { _exit(61); }
void die_temp() { _exit(71); }
void die_conn() { _exit(74); }
void die_format() { _exit(91); }
int lasterror = 55;
int qmqpfd;
int saferead(fd,buf,len) int fd; char *buf; int len;
{
int r;
r = timeoutread(60,qmqpfd,buf,len);
if (r = 0) die_conn();
return r;
}
int safewrite(fd,buf,len) int fd; char *buf; int len;
{
int r;
r = timeoutwrite(60,qmqpfd,buf,len);
if (r = 0) die_conn();
return r;
}
char buf[1024];
substdio to = SUBSTDIO_FDBUF(safewrite,-1,buf,sizeof buf);
substdio from = SUBSTDIO_FDBUF(saferead,-1,buf,sizeof buf);
substdio envelope = SUBSTDIO_FDBUF(read,1,buf,sizeof buf);
/* WARNING: can use only one of these at a time! */
stralloc beforemessage = {0};
stralloc message = {0};
stralloc aftermessage = {0};
char strnum[FMT_ULONG];
stralloc line = {0};
struct sindex
{
int pos[256];
int len;
};
void getmess()
{
int match;
if (slurpclose(0,message,1024) == -1) die_read();
strnum[fmt_ulong(strnum,(unsigned long) message.len)] = 0;
if (!stralloc_copys(beforemessage,strnum)) nomem();
if (!stralloc_cats(beforemessage,":")) nomem();
if (!stralloc_copys(aftermessage,",")) nomem();
if (getln(envelope,line,match,'\0') == -1) die_read();
if (!match) die_format();
if (line.len 2) die_format();
if (line.s[0] != 'F') die_format();
strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0;
if (!stralloc_cats(aftermessage,strnum)) nomem();
if (!stralloc_cats(aftermessage,":")) nomem();
if (!stralloc_catb(aftermessage,line.s + 1,line.len - 2)) nomem();
if (!stralloc_cats(aftermessage,",")) nomem();
for (;;) {
if (getln(envelope,line,match,'\0') == -1) die_read();
if (!match) die_format();
if (line.len 2) break;
if (line.s[0] != 'T') die_format();
strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0;
if (!stralloc_cats(aftermessage,strnum)) nomem();
if (!stralloc_cats(aftermessage,":")) nomem();
if (!stralloc_catb(aftermessage,line.s + 1,line.len - 2)) nomem();
if (!stralloc_cats(aftermessage,",")) nomem();
}
}
void doit(server)
char *server;
{
struct ip_address ip;
char ch;
if (!ip_scan(server,ip)) return;
qmqpfd = socket(AF_INET,SOCK_STREAM,0);
if (qmqpfd == -1) die_socket();
if (timeoutconn(qmqpfd,ip,PORT_QMQP,10) != 0) {
lasterror = 73;
if (errno == error_timeout) lasterror = 72;
close(qmqpfd);
return;
}
strnum[fmt_ulong(strnum,(unsigned long) (beforemessage.len + message.len +
aftermessage.len))] = 0;
substdio_puts(to,strnum);
substdio_puts(to,":");
substdio_put(to,beforemessage.s,beforemessage.len);
substdio_put(to,message.s,message.len);
substdio_put(to,aftermessage.s,aftermessage.len);
substdio_puts(to,",");
substdio_flush(to);
for (;;) {
substdio_get(from,ch,1);
if (ch == 'K') die_success();
if (ch == 'Z') die_temp();
if (ch == 'D') die_perm();
}
}
stralloc servers = {0};
main()
{
int i;
int j;
int randj;
int randarr[256];
struct timeb tp;
struct sindex serverindex;
sig_pipeignore();
if (chdir(auto_qmail) == -1) die_home();
if (control_init() == -1) die_control();
if (control_readfile(servers,"control/qmqpservers",0) != 1)
die_control();
getmess();
serverindex.len = 1;
serverindex.pos[0]=0;
for (j = 0; j servers.len; j++)
{
if (servers.s[j] == NULL) {
serverindex.pos[serverindex.len]
RE: mail server location question
You can't connect to the external side of your firewall from a machine on the inside. Make sure you're testing it from a machine outside of your firewall. Jay -Original Message- From: Greg Owen [mailto:[EMAIL PROTECTED]] Sent: Thursday, July 27, 2000 8:00 AM To: qmail Subject: RE: mail server location question OK, I think I have my firewall masquerading the firewall external IP port 25 to the qmail box internal IP port 25 I'm getting connection rejects, when I try to telnet to port 25 on the firewall. This should redirect me to port 25 on the qmail box, right? If your firewall is set up right, it should. Does your qmail box accept connections on port 25 at all? While logged into your qmail box, type 'telnet localhost 25'. If you get connection refused, then you aren't running qmail-smtpd properly. If your connection is accepted and you get the SMTP banner, then test the firewall's port 25 again. If the first suceeds and the second fails, then the firewall is probably not configured correctly. I'm not sure that it's the qmail box that's causing the problem, but is there anything I need to do to allow smtp connections from the internet? Not on the connection level. Once you get port 25 responding to the outside world, you may need to tweak your configuration as far as rcpthosts and relaying goes, but first let's get plain old connectivity going. -- gowen -- Greg Owen -- [EMAIL PROTECTED]
qmail-qmqpc.c load balancing mods
Here's the modified version of qmqpc that will pick a random server instead
of the first available. I haven't tested it yet, but it compiles, and
printf statements in strategic places give me the output I'm looking for.
The rand() function is seeded with the current milliseconds from the system
clock. The for loop for doit() will loop twice the number of the servers we
have to make sure it looked at all of them, this was easier than keeping
track of which servers were already checked. Don't make too much fun of my
code, I haven't coded anything for about 3 years. :)
--
Jay Austad
Network Administrator
CBS Marketwatch
612.817.1271
[EMAIL PROTECTED]
http://cbs.marketwatch.com
http://www.bigcharts.com
===
#include sys/types.h
#include sys/socket.h
#include netinet/in.h
#include arpa/inet.h
#include stdlib.h
#include sys/time.h
#include sys/timeb.h
#include "substdio.h"
#include "getln.h"
#include "readwrite.h"
#include "exit.h"
#include "stralloc.h"
#include "slurpclose.h"
#include "error.h"
#include "sig.h"
#include "ip.h"
#include "timeoutconn.h"
#include "timeoutread.h"
#include "timeoutwrite.h"
#include "auto_qmail.h"
#include "control.h"
#include "fmt.h"
#define PORT_QMQP 628
void die_success() { _exit(0); }
void die_perm() { _exit(31); }
void nomem() { _exit(51); }
void die_read() { if (errno == error_nomem) nomem(); _exit(54); }
void die_control() { _exit(55); }
void die_socket() { _exit(56); }
void die_home() { _exit(61); }
void die_temp() { _exit(71); }
void die_conn() { _exit(74); }
void die_format() { _exit(91); }
int lasterror = 55;
int qmqpfd;
int saferead(fd,buf,len) int fd; char *buf; int len;
{
int r;
r = timeoutread(60,qmqpfd,buf,len);
if (r = 0) die_conn();
return r;
}
int safewrite(fd,buf,len) int fd; char *buf; int len;
{
int r;
r = timeoutwrite(60,qmqpfd,buf,len);
if (r = 0) die_conn();
return r;
}
char buf[1024];
substdio to = SUBSTDIO_FDBUF(safewrite,-1,buf,sizeof buf);
substdio from = SUBSTDIO_FDBUF(saferead,-1,buf,sizeof buf);
substdio envelope = SUBSTDIO_FDBUF(read,1,buf,sizeof buf);
/* WARNING: can use only one of these at a time! */
stralloc beforemessage = {0};
stralloc message = {0};
stralloc aftermessage = {0};
char strnum[FMT_ULONG];
stralloc line = {0};
struct sindex
{
int pos[256];
int len;
};
void getmess()
{
int match;
if (slurpclose(0,message,1024) == -1) die_read();
strnum[fmt_ulong(strnum,(unsigned long) message.len)] = 0;
if (!stralloc_copys(beforemessage,strnum)) nomem();
if (!stralloc_cats(beforemessage,":")) nomem();
if (!stralloc_copys(aftermessage,",")) nomem();
if (getln(envelope,line,match,'\0') == -1) die_read();
if (!match) die_format();
if (line.len 2) die_format();
if (line.s[0] != 'F') die_format();
strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0;
if (!stralloc_cats(aftermessage,strnum)) nomem();
if (!stralloc_cats(aftermessage,":")) nomem();
if (!stralloc_catb(aftermessage,line.s + 1,line.len - 2)) nomem();
if (!stralloc_cats(aftermessage,",")) nomem();
for (;;) {
if (getln(envelope,line,match,'\0') == -1) die_read();
if (!match) die_format();
if (line.len 2) break;
if (line.s[0] != 'T') die_format();
strnum[fmt_ulong(strnum,(unsigned long) line.len - 2)] = 0;
if (!stralloc_cats(aftermessage,strnum)) nomem();
if (!stralloc_cats(aftermessage,":")) nomem();
if (!stralloc_catb(aftermessage,line.s + 1,line.len - 2)) nomem();
if (!stralloc_cats(aftermessage,",")) nomem();
}
}
void doit(server)
char *server;
{
struct ip_address ip;
char ch;
if (!ip_scan(server,ip)) return;
qmqpfd = socket(AF_INET,SOCK_STREAM,0);
if (qmqpfd == -1) die_socket();
if (timeoutconn(qmqpfd,ip,PORT_QMQP,10) != 0) {
lasterror = 73;
if (errno == error_timeout) lasterror = 72;
close(qmqpfd);
return;
}
strnum[fmt_ulong(strnum,(unsigned long) (beforemessage.len + message.len +
aftermessage.len))] = 0;
substdio_puts(to,strnum);
substdio_puts(to,":");
substdio_put(to,beforemessage.s,beforemessage.len);
substdio_put(to,message.s,message.len);
substdio_put(to,aftermessage.s,aftermessage.len);
substdio_puts(to,",");
substdio_flush(to);
for (;;) {
substdio_get(from,ch,1);
if (ch == 'K') die_success();
if (ch == 'Z') die_temp();
if (ch == 'D') die_perm();
}
}
stralloc servers = {0};
main()
{
int i;
int j;
int randj;
struct timeb tp;
struct sindex serverindex; //used to keep an index of where each
server starts in servers.s
sig_pipeignore();
if (chdir(auto_qmail) == -1) die_home();
if (control_init() == -1) die_control();
if (control_readfile(servers,"control/qmqpservers",0)
RE: qmqpc load balancing
It would probably be much more efficient to round-robin them. Otherwise you end up banging on one until it's buried (or at it's set limit), then banging on the next, and so on. What happens when they all decide their load is too high and shut down qmqpd? Isn't it much easier to code round-robin into it anyway? Would it be easier to just make it so I could put a hostname into the qmqpservers file and do round robin dns for it? Wouldn't that just be a simple addition of gethostbyname()? I have to say though, I really like the way qmail is laid out. Lots of small programs with specific functions, it makes it very easy to do modifications. Jay -Original Message- From: Michael T. Babcock [mailto:[EMAIL PROTECTED]] Sent: Saturday, July 22, 2000 3:55 PM To: Russell Nelson Cc: '[EMAIL PROTECTED]' Subject: Re: qmqpc load balancing True, but its quite valid to round-robin several servers to keep any one from ever getting a high load in the first place. eg. the way load-balancing HTTP usually works. Russell Nelson wrote: Austad, Jay writes: Instead of having qmqpc picking the first available server, I would like it to load balance between all servers I have listed as QMQP servers. Do it the other way around. If a server thinks its load is too high, it should shut down its qmqpc service.
qmqpc load balancing
Instead of having qmqpc picking the first available server, I would like it
to load balance between all servers I have listed as QMQP servers. In
qmail-qmqpc.c on line 153, it says:
i = 0;
for (j = 0;j servers.len;++j)
if (!servers.s[j]) {
doit(servers.s + i);
i = j + 1;
}
Would it work if I change it to:
i = 0;
for (j = 0;j servers.len;++j)
i = (servers.len*1.0)*rand()/(RAND_MAX+1.0);
if (!servers.s[j]) {
doit(servers.s + i);
}
This way, "i" will be a random number from 0 to (servers.len-1).
------
Jay Austad
Network Administrator
CBS Marketwatch
612.817.1271
[EMAIL PROTECTED]
http://cbs.marketwatch.com
http://www.bigcharts.com
RE: qmqpc load balancing
Even if it does hit a down server twice, it'll still just randomize and try again hopefully hitting one that's not down. This shouldn't lose any mail and shouldn't take much, if any, extra system resources. So the actual string that 'j' scans looks something like this: 192.168.20.1\0192.168.20.2\0192.168.20.3 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, July 21, 2000 3:44 PM To: [EMAIL PROTECTED] Subject: Re: qmqpc load balancing On Fri, Jul 21, 2000 at 04:35:57PM -0400, Paul Jarc wrote: [EMAIL PROTECTED] writes: On Fri, Jul 21, 2000 at 03:10:58PM -0500, Austad, Jay wrote: Instead of having qmqpc picking the first available server, I would like it to load balance between all servers I have listed as QMQP servers. ... This way, "i" will be a random number from 0 to (servers.len-1). Almost. Note that 'i' has to point to the start of the \0 terminated string that 'j' is currently scanning. servers.s is a series of \0 terminated strings. You need to do some work to make sure that i, is pointing that the start of the string that j randomizes into. Also note that if you choose one at random every time, you just might choose the same one, which happens to be down, every time. It'd be better to rotate the list by a random number of entries, and then step through it normally. I think he as trying to randomize the start point and cycle thru the list as qmqpc does now. So even if he hits the same starting point, it'll still find a good server. Leastwise that's what I was originally trying to show with the posted code-fragment. Regards.
dot-qmail with ./named.pipe == invalid argument #4.3.0
Greetings from a happy new qmail user (first post), ** qmail 1.03 ** linux-2.2.16-:i386-:-:pentium-:- As part of a ritualistic adventure to the edges of qmail sanity, I brazenly created a fifo to be read by a perl "daemon". Using a .qmail file in a real-user's home dir that read: ./inmail.pipe (BTW - I'm also using fastforward, but this is an actual user account without aliasing) Long story short -- qmail wrote to the pipe just fine but justifiably barfs at the end. (Uhh, stat call or something?) error Jul 20 10:43:03 mn qmail: 964089783.401047 delivery 3225: deferral: Unable_to_write_./inmail.pipe:_invalid_argument._(#4.3.0)/ /error So .. I proceeded to create a writer.sh: #!/bin/sh cat ./inmail.pipe and modified .qmail: | ./writer.sh This appears to work fine, as I had hoped. Why would I do such a crazy thing in the first place? I've compiled with QUEUE_EXTRA and plan to use it to mangle incoming/outgoing messages for [EMAIL PROTECTED] [EMAIL PROTECTED] More precisely, I plan to attach ticket # headers, update the "bug reports" database, and route incoming replies to the sales/support person who was first to respond. Blah blah blah .. using Perl. I switched to qmail 1) specifically for QUEUE_EXTRA, 2) security, security, security, and 3) out of desperation and displeasure with sendmail (Sorry sendmail hackers). So why the FIFO? I'm concerned about spawning many-a-Perl at 2.5MB each on my lowly P75 DNS/Firewall/qmail box. So the idea of a single reader is attractive .. Is that crazy? Suggestions? (Other than "Pentium II's are cheap") :-) -Jay J p.s. Many thanks for qmail friends
RE: questions about performance and setup
I did some benchmarking using a standard 7200 RPM disk and a 128MB ramdisk. The machine was not using any swap, so there was no chance of the ramdisk accidentally making it to disk. In short, performance on it sucked. The throughput was about 10% less than IDE, but seeks/sec were 5-10 times more. However, the CPU was maxed at 100% during tests to the ramdisk. Jay -Original Message- From: Oliver White [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 18, 2000 12:27 AM To: [EMAIL PROTECTED] Subject: Re: questions about performance and setup Steve Wolfe wrote: With all of the emails I recieved, I get the impression that I'm going to I/O bound instead of processor or memory bound. How much disk will be sufficient for the queue? 1GB? More? It's not so much a matter of disk size (I don't think you'll have a 1 gig queue!), You could quite easily get a 1 Gig queue, even if you don't run into the obvious problem of temporary loss of network connectivity. Say you've got 200,000 subscribers and you generate your messages twice as fast as qmail can send them, then when you've finished generating the messages you've still got 100,000 in the queue. If the messages are 10Kb each, that's 1 Gb. (I can put 2GB of ram in the box)? Linux has support for making a disk in memory, putting a filesystem on it and mounting it. Wouldn't this take care of I/O problems? That's about as good of I/O as you can get, I would imagine. ; ) As another author stated, the largest gain would be in writes, but that's where the largest expenditure is anyway. Just make dang, dang sure that your machine is NOT going to have any hiccups or lose power while the queue is full, or you'll instantly lose it all. What if you put the 2 Gb RAM in the box, but let Linux use it as a disk cache? I'm not sure how the disk caching under Linux works, but if you create a file and then delete it before it actually gets written to disk, is there any disk activity required? Sure, the disks will be thrashing away, trying to keep up, but would the I/O actually block if there was still room in the disk cache? - Oliver.
RE: IPCHAINS and slow POP/SMTP access
Since ipchains is a not really a firewall but a packet filter, you need to make sure you have the line: ipchains -A input -s 0.0.0.0/0.0.0.0 -d your public IP/255.255.255.255 -p 6 -j ACCEPT ! -y Make sure your firewall also accepts all packets from 192.168.1.0/24 also. Ipchains configs are sorta off topic, so if you have any other questions just email me instead of the list. Check these 2 things though, you probably already have them, but make sure. Chances are you don't even need the ipchains rules with the masquerade though, depends on what's running on the firewall box. You machines behind it are inherently protected by the fact that you're using port address translation for net access to them. Jay -Original Message- From: Doug Oucharek [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 18, 2000 3:09 AM To: [EMAIL PROTECTED] Subject: IPCHAINS and slow POP/SMTP access Hello, I've searched the archives on this topic and though there are a lot of people who have reported this issue, I have not really seen a solution yet. I have a Linux box (Redhad 5.?) which I use as a firewall/server/NAT machine. One ethernet card is connected to an ADSL modem and has a fixed IP address. The other ethernet card is connected to a mini-hub and has the address of 192.168.1.1. I have a set of Mac's connected to the hub with addresses 192.168.1.100 and up. I've got Qmail running just great for both SMTP and POP!! However, as soon as I activate my firewall (using ipchains), sending or receiving email from a local machine takes over 3 minutes!! In the archives, some people have speculated that this is a DNS issue or a problem with auth. I have TCP port 113 (auth) opened to the world (local and internet) and am still having a problem. I suspect that there must be some other port I need to open up. Does anyone have a suggestion of where I can go from here? I am a bit new to Qmail and not too familiar with the debugging tools. By the way, I do not have DNS active on my Linux box and am relying on my ISP's DNS server (they have my domain name set up in their server). Doug
RE: questions about performance and setup
As for performance though, I'd be interested in seeing the actual numbers from the ramdisk test to check against my 10k RPM disk stats. I used bonnie++ to test it. I'll post the results sometime today, when I get some time. Jay -Original Message- From: Michael T. Babcock [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 18, 2000 10:41 AM To: Austad, Jay; Qmail Mailing List Subject: Re: questions about performance and setup Nothing wrong with 100% CPU usage. It just means that the kernel was able to soak the CPU with work ... which is good. Maxing out your performance on a RAM disk at 75% CPU usage means your system has a problem somewhere. As for performance though, I'd be interested in seeing the actual numbers from the ramdisk test to check against my 10k RPM disk stats. "Austad, Jay" wrote: I did some benchmarking using a standard 7200 RPM disk and a 128MB ramdisk. The machine was not using any swap, so there was no chance of the ramdisk accidentally making it to disk. In short, performance on it sucked. The throughput was about 10% less than IDE, but seeks/sec were 5-10 times more. However, the CPU was maxed at 100% during tests to the ramdisk.
RE: questions about performance and setup
With all of the emails I recieved, I get the impression that I'm going to I/O bound instead of processor or memory bound. How much disk will be sufficient for the queue? 1GB? More? I'm just grasping here to figure out the best solution, so bear with me... What if I only needed a 1GB queue, and what if that queue was a 1GB ramdisk (I can put 2GB of ram in the box)? Linux has support for making a disk in memory, putting a filesystem on it and mounting it. Wouldn't this take care of I/O problems? Jay -Original Message- From: Oliver White [mailto:[EMAIL PROTECTED]] Sent: Monday, July 17, 2000 12:34 AM To: [EMAIL PROTECTED] Subject: Re: questions about performance and setup We're in a similar situation at the moment. However, we want to send out 100,000 UNIQUE emails per day, expanding to 500,000 or more in the near future. Also, our send window is only actually a couple of hours. I'm trying to work out the best settings for the concurrencyremote and conf-split parameters. Our system is a HP Netserver 2000r PIII-667 RAID5 running Linux. Are there any problems in setting conf-split to a very large value? Is it necessary on a Linux system, assuming a queue size of, say 100,000? Any information appreciated. - Oliver. "Austad, Jay" wrote: Non-unique emails will most likely be generated by other machines and send the box running mini-qmail via smtp. Non-unique emails will be a small percentage of what gets sent out, for now. Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Saturday, July 15, 2000 12:10 AM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup On Fri, Jul 14, 2000 at 07:01:46PM -0500, Austad, Jay wrote: Then have the script that does the mailing call randomly on of the /var/qmail#/bin/qmail-inject. This will emulate round robin without any patching. Won't this way be a performance hit though? I admit, it is an easy solution No. My experience is that the cost of running a script to inject the mail in a way similar to that mentioned above, is pretty small compared to the queue injection cost and the delivery cost. sh or perl will be fine. and would work excellent, but I have to think about efficiency also. C code is much faster than shell or perl, and I'd like to set it up once and not have to ever worry about again, or at least for a long, long time. As I said, we're doing 50 million emails a month right now, but this is increasing substantially each month, and as we rollout new subscription services, we'll have even more load. Sending 10 times this amount by the same time next year is a good possibility, possibly sooner as we seem to underestimate the rate at which we're growing much of the time... You may also need to look at the scalability of the generation of the emails. One system I recently looked at claimed to be able to generate nicely unique emails at a targetted database, but it burned CPU like it was free - just in generating the content. Mark. Jay -Original Message- From: JuanE [mailto:[EMAIL PROTECTED]] Sent: Friday, July 14, 2000 5:55 PM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup Jay, That's the beauty of having multiple instances, not having to patch qmail. All you need to do is install qmail once per machine (ie, /var/qmail1, /var/qmail2,...). Then have the script that does the mailing call randomly on of the /var/qmail#/bin/qmail-inject. This will emulate round robin without any patching. JES Austad, Jay writes: Where would I start in the code to modify the QMQP servers list so that it would load balance between all of the servers in the list instead of just using the first one it can contact? This would be very useful to me. I assume qmail-qmqpc.c is one of them, are there others I would need to play around with? Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, July 14, 2000 3:55 PM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote: I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes with no trouble, some of them took work to get going, but it runs well. I have a few Crystal PC's here also that I may use instead, dual PIII 550's with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these for testing. I agree with the earlier poster that more spindles for your queue (c/- raid) is a good thing in general. The bulk of the messages will be the same content to many rcpt's. However, once in awhile we'll have 100,000 different messages go out to 100,000 different people. Since the QMQP support under mini-qmail doesn't load balance, can I feed it a hostname with mu
questions about performance and setup
I've been given the task of setting up our own "blaster" for sending out emails of our financial news and charts to our subscribers. We outsource this right now, and it's abysmally expensive. Basically, we want 3 boxes (or so) that run in parallel and blast out the emails, about 50 million per month, but the subscription rate is growing rapidly each month. It needs to handle bounced mail by dumping the addresses into a file for later retrieval so they can be removed from the database, or by running an external script for each bounced address. I'm looking at getting 3 dell dual PIII 750's, with a 18 or 36GB 1rpm disk, and 512M or 1G of mem each. Each will run Linux or BSD. Here's what I need to know: 1. How well does qmail take advantage of multiple processors? How much memory and disk will I need? (we're at 50 million messages per month now, and we only send out monday-friday, so that's over 2 million messages per day, and it's only going up) 2. How many messages per day would one estimate that each of these servers could do? 3. I read about mini-qmail and how it's about 100 times faster blasting out email to QMQP servers. Since you can specify multiple QMQP servers, if I have a fourth machine running mini-qmail and managing the actual mailing list, can I add the other 3 as QMQP servers and have it load balance between all 3 for sending out mail? (this way I could add more servers easily if I needed to) 4. Can I easily make qmail run an external script for each bounced mail? 5. Anything else I should know? Thanks. ------ Jay Austad Network Administrator CBS Marketwatch 612.817.1271 [EMAIL PROTECTED] http://cbs.marketwatch.com http://www.bigcharts.com
RE: questions about performance and setup
I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes with no trouble, some of them took work to get going, but it runs well. I have a few Crystal PC's here also that I may use instead, dual PIII 550's with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these for testing. The bulk of the messages will be the same content to many rcpt's. However, once in awhile we'll have 100,000 different messages go out to 100,000 different people. Since the QMQP support under mini-qmail doesn't load balance, can I feed it a hostname with multiple dns entries (round-robin dns)? Or better yet, how easy would it be to modify the qmail code to just load balance between them? Jay -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, July 14, 2000 2:09 PM To: '[EMAIL PROTECTED]' Subject: Re: questions about performance and setup Here's what I need to know: 1. How well does qmail take advantage of multiple processors? How much Indreectly, quite well as it forks many processes, thus if the OS takes good advantage of your CPUs, then qmail inherits that advantage. memory and disk will I need? (we're at 50 million messages per month now, Are these message unique per target address or the same. If unique, your requirements are vastly different and very queue/disk intensive. If they are the same and you take advantage or VERP support on qmail, then your load will mainly be sending related which will benefit from more memory, multiple instances, etc. and we only send out monday-friday, so that's over 2 million messages per day, and it's only going up) 2. How many messages per day would one estimate that each of these servers could do? 3. I read about mini-qmail and how it's about 100 times faster blasting out email to QMQP servers. Since you can specify multiple QMQP servers, if I have a fourth machine running mini-qmail and managing the actual mailing list, can I add the other 3 as QMQP servers and have it load balance between all 3 for sending out mail? (this way I could add more servers easily if I needed to) The qmqp support doesn't load balance. It simply takes the first one it can connect to. 4. Can I easily make qmail run an external script for each bounced mail? Absolutely. 5. Anything else I should know? That all hinges on whether your emails are unique for each recipient or not. Or more importantly, the average number of recipients per unique email. Regards.
RE: questions about performance and setup
Where would I start in the code to modify the QMQP servers list so that it
would load balance between all of the servers in the list instead of just
using the first one it can contact? This would be very useful to me. I
assume qmail-qmqpc.c is one of them, are there others I would need to play
around with?
Jay
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 3:55 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote:
I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes
with no trouble, some of them took work to get going, but it runs well. I
have a few Crystal PC's here also that I may use instead, dual PIII 550's
with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these for
testing.
I agree with the earlier poster that more spindles for your queue
(c/- raid) is a good thing in general.
The bulk of the messages will be the same content to many rcpt's.
However,
once in awhile we'll have 100,000 different messages go out to 100,000
different people.
Since the QMQP support under mini-qmail doesn't load balance, can I feed
it
a hostname with multiple dns entries (round-robin dns)? Or better yet,
how
easy would it be to modify the qmail code to just load balance between
them?
The manpage for qmail-qmqpc tells us that they have to be IP addresses
in qmqpservers so a RR DNS won't help. If all of the messages are generated
on one machine, then I'd be inclined to go for a much simpler solution
than modifying qmail. I'd have an instance of qmail for each outbound
server with the appropriate qmqpservers entry, then have your queue
insertion script do a round-robin itself by simply cycling thru
the qmail-inject command associated with each instance.
for instance in 1 2 3 4 5
do
getnext_message_details()
/var/qmail{$instance}/bin/qmail-inject currentmessage details
done
Or some such.
Alternatively, if you have money to burn, maybe a layer four switch
with load-balancing skills.
Mark.
Jay
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 2:09 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
Here's what I need to know:
1. How well does qmail take advantage of multiple processors? How much
Indreectly, quite well as it forks many processes, thus if the OS takes
good advantage of your CPUs, then qmail inherits that advantage.
memory and disk will I need? (we're at 50 million messages per month
now,
Are these message unique per target address or the same. If unique, your
requirements are vastly different and very queue/disk intensive. If they
are the same and you take advantage or VERP support on qmail, then
your load will mainly be sending related which will benefit from
more memory, multiple instances, etc.
and we only send out monday-friday, so that's over 2 million messages
per
day, and it's only going up)
2. How many messages per day would one estimate that each of these
servers
could do?
3. I read about mini-qmail and how it's about 100 times faster blasting
out
email to QMQP servers. Since you can specify multiple QMQP servers, if
I
have a fourth machine running mini-qmail and managing the actual mailing
list, can I add the other 3 as QMQP servers and have it load balance
between
all 3 for sending out mail? (this way I could add more servers easily
if
I
needed to)
The qmqp support doesn't load balance. It simply takes the first one
it can connect to.
4. Can I easily make qmail run an external script for each bounced mail?
Absolutely.
5. Anything else I should know?
That all hinges on whether your emails are unique for each recipient or
not. Or more importantly, the average number of recipients per unique
email.
Regards.
RE: questions about performance and setup
Then have the script that does the mailing call randomly
on of the /var/qmail#/bin/qmail-inject. This will emulate round robin
without any patching.
Won't this way be a performance hit though? I admit, it is an easy solution
and would work excellent, but I have to think about efficiency also. C code
is much faster than shell or perl, and I'd like to set it up once and not
have to ever worry about again, or at least for a long, long time.
As I said, we're doing 50 million emails a month right now, but this is
increasing substantially each month, and as we rollout new subscription
services, we'll have even more load. Sending 10 times this amount by the
same time next year is a good possibility, possibly sooner as we seem to
underestimate the rate at which we're growing much of the time...
Jay
-Original Message-
From: JuanE [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 5:55 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
Jay,
That's the beauty of having multiple instances, not having to patch qmail.
All you need to do is install qmail once per machine (ie, /var/qmail1,
/var/qmail2,...). Then have the script that does the mailing call randomly
on of the /var/qmail#/bin/qmail-inject. This will emulate round robin
without any patching.
JES
Austad, Jay writes:
Where would I start in the code to modify the QMQP servers list so that it
would load balance between all of the servers in the list instead of just
using the first one it can contact? This would be very useful to me. I
assume qmail-qmqpc.c is one of them, are there others I would need to play
around with?
Jay
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 3:55 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote:
I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell boxes
with no trouble, some of them took work to get going, but it runs well.
I
have a few Crystal PC's here also that I may use instead, dual PIII
550's
with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these
for
testing.
I agree with the earlier poster that more spindles for your queue
(c/- raid) is a good thing in general.
The bulk of the messages will be the same content to many rcpt's.
However,
once in awhile we'll have 100,000 different messages go out to 100,000
different people.
Since the QMQP support under mini-qmail doesn't load balance, can I feed
it
a hostname with multiple dns entries (round-robin dns)? Or better yet,
how
easy would it be to modify the qmail code to just load balance between
them?
The manpage for qmail-qmqpc tells us that they have to be IP addresses
in qmqpservers so a RR DNS won't help. If all of the messages are
generated
on one machine, then I'd be inclined to go for a much simpler solution
than modifying qmail. I'd have an instance of qmail for each outbound
server with the appropriate qmqpservers entry, then have your queue
insertion script do a round-robin itself by simply cycling thru
the qmail-inject command associated with each instance.
for instance in 1 2 3 4 5
do
getnext_message_details()
/var/qmail{$instance}/bin/qmail-inject currentmessage details
done
Or some such.
Alternatively, if you have money to burn, maybe a layer four switch
with load-balancing skills.
Mark.
Jay
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 2:09 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
Here's what I need to know:
1. How well does qmail take advantage of multiple processors? How
much
Indreectly, quite well as it forks many processes, thus if the OS takes
good advantage of your CPUs, then qmail inherits that advantage.
memory and disk will I need? (we're at 50 million messages per month
now,
Are these message unique per target address or the same. If unique, your
requirements are vastly different and very queue/disk intensive. If they
are the same and you take advantage or VERP support on qmail, then
your load will mainly be sending related which will benefit from
more memory, multiple instances, etc.
and we only send out monday-friday, so that's over 2 million messages
per
day, and it's only going up)
2. How many messages per day would one estimate that each of these
servers
could do?
3. I read about mini-qmail and how it's about 100 times faster
blasting
out
email to QMQP servers. Since you can specify multiple QMQP servers,
if
I
have a fourth machine running mini-qmail and managing the actual
mailing
list, can I add the other 3 as QMQP servers and have it load balance
between
all 3 for sending out mail? (this way I could add
RE: questions about performance and setup
Non-unique emails will most likely be generated by other machines and send
the box running mini-qmail via smtp. Non-unique emails will be a small
percentage of what gets sent out, for now.
Jay
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Saturday, July 15, 2000 12:10 AM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
On Fri, Jul 14, 2000 at 07:01:46PM -0500, Austad, Jay wrote:
Then have the script that does the mailing call randomly
on of the /var/qmail#/bin/qmail-inject. This will emulate round robin
without any patching.
Won't this way be a performance hit though? I admit, it is an easy
solution
No. My experience is that the cost of running a script to inject the mail
in a way similar to that mentioned above, is pretty small compared to the
queue injection cost and the delivery cost. sh or perl will be fine.
and would work excellent, but I have to think about efficiency also. C
code
is much faster than shell or perl, and I'd like to set it up once and not
have to ever worry about again, or at least for a long, long time.
As I said, we're doing 50 million emails a month right now, but this is
increasing substantially each month, and as we rollout new subscription
services, we'll have even more load. Sending 10 times this amount by the
same time next year is a good possibility, possibly sooner as we seem to
underestimate the rate at which we're growing much of the time...
You may also need to look at the scalability of the generation of the
emails. One system I recently looked at claimed to be able to generate
nicely unique emails at a targetted database, but it burned CPU like
it was free - just in generating the content.
Mark.
Jay
-Original Message-
From: JuanE [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 5:55 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
Jay,
That's the beauty of having multiple instances, not having to patch qmail.
All you need to do is install qmail once per machine (ie, /var/qmail1,
/var/qmail2,...). Then have the script that does the mailing call randomly
on of the /var/qmail#/bin/qmail-inject. This will emulate round robin
without any patching.
JES
Austad, Jay writes:
Where would I start in the code to modify the QMQP servers list so that
it
would load balance between all of the servers in the list instead of
just
using the first one it can contact? This would be very useful to me. I
assume qmail-qmqpc.c is one of them, are there others I would need to
play
around with?
Jay
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 3:55 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
On Fri, Jul 14, 2000 at 02:29:06PM -0500, Austad, Jay wrote:
I already have Mandrake Linux 7.0 and 7.1 running on multiple Dell
boxes
with no trouble, some of them took work to get going, but it runs
well.
I
have a few Crystal PC's here also that I may use instead, dual PIII
550's
with 512MB ram and 9 or 18GB 1rpm drives. I'll probably use these
for
testing.
I agree with the earlier poster that more spindles for your queue
(c/- raid) is a good thing in general.
The bulk of the messages will be the same content to many rcpt's.
However,
once in awhile we'll have 100,000 different messages go out to 100,000
different people.
Since the QMQP support under mini-qmail doesn't load balance, can I
feed
it
a hostname with multiple dns entries (round-robin dns)? Or better
yet,
how
easy would it be to modify the qmail code to just load balance between
them?
The manpage for qmail-qmqpc tells us that they have to be IP addresses
in qmqpservers so a RR DNS won't help. If all of the messages are
generated
on one machine, then I'd be inclined to go for a much simpler solution
than modifying qmail. I'd have an instance of qmail for each outbound
server with the appropriate qmqpservers entry, then have your queue
insertion script do a round-robin itself by simply cycling thru
the qmail-inject command associated with each instance.
for instance in 1 2 3 4 5
do
getnext_message_details()
/var/qmail{$instance}/bin/qmail-inject currentmessage details
done
Or some such.
Alternatively, if you have money to burn, maybe a layer four switch
with load-balancing skills.
Mark.
Jay
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Friday, July 14, 2000 2:09 PM
To: '[EMAIL PROTECTED]'
Subject: Re: questions about performance and setup
Here's what I need to know:
1. How well does qmail take advantage of multiple processors? How
much
Indreectly, quite well as it forks many processes, thus if the OS
takes
good
EZMLM problems
I'm using ezmlm 0.53 with ezmlm-idx 0.40. Have never set up a moderated mailing list before, but decided I wanted to try it out tonight. Set it up with: ezmlm-make -q -m /path/to/list /path/to/. list domain.com Set up a couple of test subscribers, and set up a moderator with ezmlm-sub /path/to/list/mod [EMAIL PROTECTED] Tried to send a subscribe request, got the confirmation back, sent the "cookie" back to be accepted to the list, all while tail -f'ing my logfiles. Got this in the logfile... May 6 01:55:59 domain qmail: 957603359.650961 status: local 1/10 remote 2/20 May 6 01:55:59 elementdesign qmail: 957603359.760669 delivery 4325: failure: ezmlm-manage:_fatal:_Command_not_available_(#5.1.1)/ May 6 01:55:59 domain qmail: 957603359.761031 status: local 0/10 remote 2/20 What could be causing this? ezmlm-manage is there, its in the path, and it is taking the correct command line options (ezmlm-manage 'path/to/list', correct?) When I try to run it from the command line with these command line options, I get "SENDER not set". I know this is an environment variable, but where is it set, and what is it to be set to? It's pretty vague in the manpages and the FAQ. Thanks for any help... appreciate it. j
Local mail for virtual domains
I have a user [EMAIL PROTECTED] that has an account on our qmail box and would like the mail delivered to /home/testme/Maildir. The qmail box is also a virtual domain for sailnet.com. The default domain for the qmail box is sailnet.net. How do I get her mail delievered /home/test/Maildir instead of /home/vpopmail/domains/sailnet.com/testme/Maildir? Thanx, Jay
queue backup
I have qmail/ezmlm-idx. We are serving over 100 mailing lists with a total of about 50,000 subscribers. The output from qmail-qstat is: messages in queue: 2293 messages in queue but not yet preprocessed: 0 My first question is how to I purge the queue. The second question is what causes the queue to hold these messages. The queue size seems to continue grow it never comes down. We are not having any complaints from our subscribers at all. Thanx, Jay
Re: Qmail System Users
You can find some of this info on www.inter7.com Original Message On 3/29/00, 8:45:21 AM, [EMAIL PROTECTED] wrote regarding Qmail System Users: I am trying to justify continuing the use of qmail in our organization. Is there a comprehensive list out there of large organizations that use qmail and the volume of mail they have pass through their mail exchangers? Appreciate any insight into this. Thanks.
start/stop
I'm running starting qmail with the following script, how do I start and stop qmail with tcpserver? env - PATH="/var/qmail/bin:/usr/local/bin" \ qmail-start ./Maildir/ /usr/local/bin/accustamp \ | /usr/local/bin/setuser qmaill /usr/local/bin/cyclog -s 104 \ -n 200 /var/log/qmail echo "Starting qmail ..." env - PATH="/var/qmail/bin:/usr/local/bin" \ tcpserver -H -R -c100 0 pop3 /var/qmail/bin/qmail-popup \ sailnet.com \ /home/vpopmail/bin/vchkpw /var/qmail/bin/qmail-pop3d Maildir echo "Starting pop ..." env - PATH="/var/qmail/bin:/usr/local/bin" \ tcpserver -H -R -x /etc/tcp.smtp.cdb -c100 -u1001 -g102 0 smtp \ /var/qmail/bin/qmail-smtpd 21 /dev/null echo "Starting smtp ..."
TCPRULES help
I would like to add the following to my tcp.smtp file for tcprules: .domain.com:allow,RELAYCLIENT="" My question is can you use domain names with tcprules, or does it have to be IP addresses. In the man page all expamples use Ips and not domains. Thanx, Jay
Re: TCPRULES help
I upgraded to v0.86, however I still see nothing in the docs on the webpage or in the man pages about using hostnames! Where did you find this info? Thanx, Jay Original Message On 3/14/00, 11:09:59 AM, Anand Buddhdev [EMAIL PROTECTED] wrote regarding Re: TCPRULES help: On Tue, Mar 14, 2000 at 04:43:30PM +0100, Petr Novotny wrote: I would like to add the following to my tcp.smtp file for tcprules: .domain.com:allow,RELAYCLIENT="" My question is can you use domain names with tcprules, or does it have to be IP addresses. In the man page all expamples use Ips and not domains. It has to be IP address. With the new tcpserver v 0.86, you can use hostnames. See the documentation. -- See complete headers for more info
Trailer for every message
I want to add a trailer to all outgoing list messages for every list created. I used ezmlm-make - t to create a list, however I never see the trailer from DIR/text/trailer appended to any of the lists messages. Any Ideas? Thanx, Jay
POP and pine/elm
Here's the deal. I set up vpopmail (or vchkpass, whatever you want to call it) for pop mail. It keeps everything in /home/vpopmail. But some of my users want to be able to check their mail with pine if they need to, or be able to download it if they need to. (like if they are on the road, pine is more convenient) I'm very new to qmail and am still a bit confused about how it works, so can anyone give me any suggestions on how to make this work? For instance... I set up a pop account [EMAIL PROTECTED] Someone sends mail to [EMAIL PROTECTED] it goes into /home/vpopmail/domains/domain.com/jay/Maildir but I also want it to go into /home/jay/Maildir, so I can check it in pine as well... Thanks for any help
qmail vs. sendmail weirdness
Alright, here's the situation. I set up vpopmail for pop accounts, and qmailadmin to manage those accounts. I set up an account [EMAIL PROTECTED] I added that pop user to my mail client, and sent out a few messages to test. When I send a message to a domain that uses sendmail, I get a bounce saying: [EMAIL PROTECTED] ... Domain must exist Simple question. WTF? All of the other domains I sent it to worked fine (hotmail, a few boxes I set qmail up on, a few domains that use alternative mail clients) Any reason why this is happening? Thanks.
.qmail questions
Here's the deal... I'm new to qmail, and I think it's just the coolest thing I've ever seen. But I am having problems with it. I have a domain in the rcpthosts and virtualdomains files and I want to set up a pop account for somebody on my domain. So I create a file called .qmail-mydomain-user in my home directory and put "/home/user/Maildir/" in it. When I send mail to that user, qmail cannot chdir to /home/user/Maildir/, so the mail isn't recieved. I've tried changing permissions on /home/user/Maildir/ but the mail still isn't delivered... temporary error. Don't know the reason for this. I've tried using the users/assign method of delivering mail to this user, but no matter what I put in the users/assign file (even if it's empty!), and I run qmail-newu, I get the following error: qmail-newu: fatal: bad format in users/assign Are there any workaround for my problem, or am I doing something wrong? Also, what is wrong with users/assign? Just for reference, here is the line I put in users/assign: =myuser-mydomain-nathan:nate:1035:109:/home/nate::: ^^^this is set to what I set in control/virtualdomains like: mydomain.com:myuser-mydomain Any help appreciated.
Re: help pls! RELAYCLIENT won't bypass rcpthosts
"Reece" == Reece Markowsky [EMAIL PROTECTED] writes: Reece You can see that I am running tcpserver correctly too: /usr/local/bin/tcpserver -v -p -x /etc/tcp.smtp.cdb -u 137 -g 223 0 smtp /var/qmail/bin/qmail-smtpd Try making sure that the path to the CDB file is part of the '-x' argument, not an additional argument (remove that extra space, that is '-xFOO', not '-x FOO'). Also, use cdbdump to make sure that the cdb file is up to date (or just rebuild it). Reece Any ideas?? If that doesn't work, instead of exec'ing qmail-smtpd, exec an sh script which dumps the env and them execs qmail-smtpd, as in: #!/bin/sh env /var/tmp/debug/qmail-smtpd.$$ exec /var/qmail/bin/smtpd Then, examine the environment and make sure RELAYCLIENT really is set. BTW, if might want to use ofmipd for "internal" hosts to give you rewriting flexibility in case you need it. I do so like this: /usr/local/bin/tcpserver -R -H -learthquake -x/var/qmail/etc/tcp.smtp.cdb -u61 -g60 -v 0 smtp /var/qmail/libexec/qmail-smtpd+ofmipd /var/qmail/etc/tcp.smtp: 192.168.249.:allow,OFMIPCLIENT="yes" 192.168.250.:allow,OFMIPCLIENT="yes" 206.251.18.:allow,OFMIPCLIENT="yes" 204.71.180.:allow,OFMIPCLIENT="yes" :allow,DATABYTES="1048576" /var/qmail/libexec/qmail-smtpd+ofmipd #!/bin/sh if [ -n "$OFMIPCLIENT" ] ; then exec /var/qmail/bin/ofmipd else exec /var/qmail/bin/qmail-smtpd fi Good luck. j. -- Jay Soffian [EMAIL PROTECTED]UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Help on qmailanalog
"Ronny" == Ronny Haryanto [EMAIL PROTECTED] writes: Ronny I use this script to send me a log analysis nightly (via crontab). Ronny Add the z* commands as you like before ")| qmail-inject". Ronny #!/bin/sh Ronny PATH=/usr/local/qmailanalog/bin:/var/qmail/bin:/bin Ronny QMAILLOG="/tmp/q.$$" Ronny umask 077 Ronny cat /var/log/qmail/* | matchup $QMAILLOG 5/dev/null Ronny DATE=`date +'%a %d %b'` Ronny (echo "To: [EMAIL PROTECTED]" Ronny echo "From: [EMAIL PROTECTED]" Ronny echo "Subject: qmail report $DATE" Ronny echo "" Ronny zoverall $QMAILLOG Ronny zfailures $QMAILLOG Ronny zdeferrals $QMAILLOG)| qmail-inject Ronny rm -f $QMAILLOG You really shouldn't be throwing away the output from fd5. It's there for a reason. I use this script: #!/bin/sh PATH="/usr/local/bin/qmailanalog:/usr/local/bin:$PATH" export PATH umask 077 TMP_FILE="/var/log/qmail/qmailanalog.tmp" EXT_FILE="/var/log/qmail/qmailanalog.ext" OUT_FILE="/var/log/qmail/qmailanalog.out" LOG_FILE="/var/log/qmail/log.1.gz" rm -f $TMP_FILE $OUT_FILE cat MAIL_HEADER $OUT_FILE From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: `hostname` qmail statistics MAIL_HEADER touch $EXT_FILE (cat $EXT_FILE; gunzip -c $LOG_FILE | tai64nepoch) | matchup $TMP_FILE 5$EXT_FILE.new mv $EXT_FILE.new $EXT_FILE zoverall $TMP_FILE $OUT_FILE echo "--" $OUT_FILE zddist$TMP_FILE $OUT_FILE /var/qmail/bin/qmail-inject $OUT_FILE rm -f $TMP_FILE $OUT_FILE I've attached a patch for daemontools-0.61 which creates tai64nepoch, something I hacked from tai64nlocal. I happen to rotate my logs nightly, using qfilelog instead of using multilog (rotating logs based on size is fine, but multilog really should support logs rotated based on time as well - something as simply as rotating whenever it receives a HUP or USR1 would make me happy). j. -- Jay Soffian [EMAIL PROTECTED]UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Help on qmailanalog
"Jay" == Jay Soffian [EMAIL PROTECTED] writes:
Jay I've attached a patch for daemontools-0.61 which creates tai64nepoch,
Jay something I hacked from tai64nlocal. I happen to rotate my logs
Oops, forgot to attach the patch. Attached.
j.
--
Jay Soffian [EMAIL PROTECTED]UNIX Systems Engineer
404.572.1941 Cox Interactive Media
diff -c -N daemontools-0.61.orig/Makefile daemontools-0.61/Makefile
*** daemontools-0.61.orig/Makefile Thu Aug 26 02:56:44 1999
--- daemontools-0.61/Makefile Mon Nov 1 21:44:37 1999
***
*** 318,324
./compile open_write.c
prog: \
! svscan supervise svok svstat svc fghack multilog tai64n tai64nlocal \
softlimit setuidgid envuidgid rts matchtest
prot.o: \
--- 318,324
./compile open_write.c
prog: \
! svscan supervise svok svstat svc fghack multilog tai64n tai64nlocal tai64nepoch \
softlimit setuidgid envuidgid rts matchtest
prot.o: \
***
*** 557,562
--- 557,570
tai64n.o: \
compile tai64n.c timestamp.h substdio.h readwrite.h exit.h
./compile tai64n.c
+
+ tai64nepoch: \
+ load tai64nepoch.o substdio.a error.a str.a fs.a
+ ./load tai64nepoch substdio.a error.a str.a fs.a
+
+ tai64nepoch.o: \
+ compile tai64nepoch.c substdio.h subfd.h substdio.h exit.h fmt.h
+ ./compile tai64nepoch.c
tai64nlocal: \
load tai64nlocal.o substdio.a error.a str.a fs.a
diff -c -N daemontools-0.61.orig/tai64nepoch.c daemontools-0.61/tai64nepoch.c
*** daemontools-0.61.orig/tai64nepoch.c Wed Dec 31 19:00:00 1969
--- daemontools-0.61/tai64nepoch.c Tue Jan 4 13:09:42 2000
***
*** 0
--- 1,68
+ #include sys/types.h
+ #include sys/time.h
+ #include "substdio.h"
+ #include "subfd.h"
+ #include "exit.h"
+ #include "fmt.h"
+
+ char num[FMT_ULONG];
+
+ void get(ch)
+ char *ch;
+ {
+ int r;
+
+ r = substdio_get(subfdin,ch,1);
+ if (r == 1) return;
+ if (r == 0) _exit(0);
+ _exit(111);
+ }
+
+ void out(buf,len)
+ char *buf;
+ int len;
+ {
+ if (substdio_put(subfdout,buf,len) == -1)
+ _exit(111);
+ }
+
+ time_t secs;
+ unsigned long nanosecs;
+ unsigned long u;
+ struct tm *t;
+
+ main()
+ {
+ char ch;
+
+ for (;;) {
+ get(ch);
+ if (ch == '@') {
+ secs = 0;
+ nanosecs = 0;
+ for (;;) {
+ get(ch);
+ u = ch - '0';
+ if (u = 10) {
+ u = ch - 'a';
+ if (u = 6) break;
+ u += 10;
+ }
+ secs = 4;
+ secs += nanosecs 28;
+ nanosecs = 0xfff;
+ nanosecs = 4;
+ nanosecs += u;
+ }
+ secs -= 4611686018427387914ULL;
+ out(num,fmt_ulong(num,(unsigned long) (secs)));
+ out(".",1);
+ out(num,fmt_uint0(num,(unsigned int) nanosecs,9));
+ }
+ for (;;) {
+ out(ch,1);
+ if (ch == '\n') break;
+ get(ch);
+ }
+ }
+ }
Re: Sendmail Virtusertable equivalent?
"Richard" == Richard Roderick [EMAIL PROTECTED] writes:
Richard In answer to the original question: It's not easy. I just
Richard finished developing the tools to deal with all of it.
I handled this conversion in 1 hour for over 30 domains. 300 or 3000
domains would have taken no longer since I built myself tools. I found
it pretty damn trivial. Perl is your friend. But you could do this
with sed, awk and grep.
What I did:
Grep out all the domains on the LHS from the sendmail virtualuser
table and put them in /var/qmail/control/virtualdomains all with the
same RHS like so:
domain-a.com:alias-virtualdomains
domain-b.com:alias-virtualdomains
domain-c.com:alias-virtualdomains
...
awk makes that pretty simple:
awk '/^#/{next};/^ *$/{next};{split($1,a,/@/);print a[2]":alias-virtualdomains"}' \
/etc/mail/virtusertable /var/qmail/control/virtualdomains
Install fastforward (see http://www.qmail.org).
Setup ~alias/.qmail-virtualdomains-default:
echo "|/var/qmail/bin/fastforward -d /var/qmail/etc/virtualaliases.cdb"
~alias/.qmail-virtualdomains-default
Build /var/qmail/etc/virtualaliases.cdb. I'd use a perl script for
that depending upon complexity. I'd send you the one I used if I could
still find it... virtualaliases is similar to sendmail's
virtusertable, but more flexible.
For example, virtualaliases allows multiple addresses on the RHS. With
sendmail, you can only use a single address on the RHS which you then
need to expand in the aliases file to multiple addresses.
If you only have a single address on the RHS in virtusertable, the
conversion is simple, just add a ':' after every address in the LHS
and add a ';' after every address on the RHS. You can do that with
awk:
awk '/^#/{next};/^ *$/{next};{print $1":\t"$2";"}' \
virtusertable /var/qmail/etc/virtualaliases
Clean up virtualaliases as needed, you'll want to fully qualify
addresses on the RHS. If any of the RHS addresses actually appear in
your /etc/aliases, then you'll want to expand those out. If you have a
bunch, I suggest you write a script to iterate through all the LHS
addresses and call sendmail -bv to expand out the RHS.
Don't forget to turn the virtualaliases file into a cdb:
setforward virtualaliases.cdb virtualaliases.tmp virtualaliases
Good luck. You'll be happier with qmail in the long run. Trust
me. I've administered both for quite some time.
Also, people on this list are more receptive if you lose the attitude.
j.
--
Jay Soffian [EMAIL PROTECTED]UNIX Systems Engineer
404.572.1941 Cox Interactive Media
Re: Sendmail Virtusertable equivalent?
"Jay" == Jay Soffian [EMAIL PROTECTED] writes: Jay [...] Jay Don't forget to turn the virtualaliases file into a cdb: Jay [...] I presume you've read the qmail instructions and are familiar with the basics of setup. I did leave out a pretty important step though... you need to add the virtual domains (the LHS in the virtusertable) to either /var/qmail/control/rcpthosts on /var/qmail/control/morercpthosts (I used morercpthosts, don't forget to run qmail-newmrh). j. -- Jay Soffian [EMAIL PROTECTED]UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Sendmail Virtusertable equivalent?
"Richard" == Richard Roderick [EMAIL PROTECTED] writes: Richard AAAHHH. That's what I was looking for! The part about Richard using the fastforward to do it was what I could not find, Richard and I'm no expert. Richard Someone needs to add this to the Qmail web site. It is. See 'Author's Enhancement Software for qmail': The fastforward package supports forwarding tables under qmail. j. -- Jay Soffian [EMAIL PROTECTED]UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Sendmail Virtusertable equivalent?
"Richard" == Richard Roderick [EMAIL PROTECTED] writes: Richard I saw fastforward, I just didn't have a clean Richard understanding of the capabilities and how it could be Richard used to solve this problem. :) Richard I thought you did a great job explaining the Richard virtusertable conversion process, that's what I think Richard needs to be added. Perhaps to the HOWTO or David Sill's Richard "Life with Qmail". Qmail is a pretty flexible tool. It's hard to envision what everyone might want to do with it, but I agree that converting from sendmail's virtusertable is likely a common scenario. I'd be surprised if a search of the qmail list archive didn't turn this up being discussed before. It was clear to me how to piece together qmail's various components to arrive at the solution I needed. Hmm, maybe I'll expand a little on my post and ask to have it added to the qmail FAQ. It would certainly complement the "How do I use sendmail's /etc/aliases with qmail?" question: "How do I use sendmail's /etc/virtusertable with qmail?" Looks like DJB maintaines the FAQ? Dan - interested in contributions? j. -- Jay Soffian [EMAIL PROTECTED]UNIX Systems Engineer 404.572.1941 Cox Interactive Media
Re: Maildrop samples
From: Keith Burdis [EMAIL PROTECTED] There are some examples in the maildropex(5) man page. There are dozens more examples in the qmail-uce anti-spam package on Mr.Sam's page. http://www.geocities.com/SiliconValley/Peaks/5799/qmail-uce.html -- Jay Swackhamer [EMAIL PROTECTED] WCi system/network administrator
ezmlm-manage acceping multiple domains in inhost
I have a list that was recently moved to a new hostname and I'd like ezmlm-manage to be able to accept messages at either address. That is, I'd like to be able to put multiple domains into inhost, but ezmlm-manage doesn't support this. So I can either patch ezmlm-manage or rewrite the incoming messages using new-inject from the mess822 package. Does this sound correct, or am I missing another option? Does anyone have any suggestions / pitfalls about doing this? My idea is to use virtualdomains to deliver to an alternate address that runs new-inject to rewrite the To address and then deliver to the canonical list address. Thoughts? j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: ezmlm-manage acceping multiple domains in inhost
"Frederik" == Frederik Lindberg [EMAIL PROTECTED] writes: Frederik You could install ezmlm-idx-0.322 (www.ezmlm.org) and Frederik not worry about it any more. By default (as spam Frederik counter-measure), ezmlm+idx requires the list name in Frederik To/Cc, so for moving lists you need to disable this Frederik (ezmlm-reject -T in DIR/editor). Cool, I'll look into that. Disabling the spam counter-measure is a non-issue since this is a moderated list. Frederik You could also forward messages: ~/.qmail - Frederik newlist@newhost, ~/.qmail-default - Frederik newlist-$DEFAULT@newhost. I thought about that, but wouldn't that then change the envelope sender? I ended up installing mess822. The original list was [EMAIL PROTECTED], new list is [EMAIL PROTECTED] I run this list under the alias user. I set things up like so: /var/qmail/control/virtualdomains lists.storm98.com:storm98 /var/qmail/control/rewrite =lists.storm98.com:lists.storm99.com /var/qmail/alias/.qmail-/storm98-default |/var/qmail/bin/new-inject "-f$SENDER" Seems to work properly. It would have been easier to just use ofmipd, but ofmipd is an open relay. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
problem with Maildir
I am getting the error: unable to chdir to Maildir in /log/messages when I am trying to inject. I made a switch from mbox to Maildir, and my incoming mail doesn't seem to work anymore.. Anybody who can help?
Re: ERR User has no $HOME/maildir (STILL GETTING ERROR)
- Original Message - From: Chris Johnson [EMAIL PROTECTED] To: Jay [EMAIL PROTECTED] Sent: Monday, April 19, 1999 2:16 PM Subject: Re: ERR User has no $HOME/maildir (STILL GETTING ERROR) On Mon, Apr 19, 1999 at 02:01:13PM -0400, Jay wrote: OK, thanks for your help so far, but I am still getting the error /home/whoever/Maildir exists now, and it has mail in it but outlook express responds first with rejecting the password, but gives me that same error again. ERR User has no $HOME/maildir Is Maildir a file or a directory? And if it's a directory, is it a maildir, as created by /var/qmail/bin/maildirmake? Chris
ERR User has no $HOME/maildir (STILL GETTING ERROR)
OK, thanks for your help so far, but I am still getting the error /home/whoever/Maildir exists now, and it has mail in it but outlook express responds first with rejecting the password, but gives me that same error again. ERR User has no $HOME/maildir any other ideas?
REPLY TO THIS ADDRESS -ERR user has no $HOME/Maildir
Sorry, I sent that last one using the wrong addy Hi,I'm trying to get the qmail pop3d working, and the error message I keepgetting when I am checking mail is -ERR user has no $HOME/Maildir/home/whoever/Mailbox exists, and I specify /bin/checkpassword/var/qmail/bin/qmail-pop3d Mailboxin inetd.confany ideas?
sendmail like lusers in qmail
I'm trying to get qmail to support something like the LUSER support in sendmail. Here's the situation: This is a client machine. That is, it relays everything off of a mail hub (which happens to be running sendmail). I've set defaulthost and defaultdomain to "cimedia.com" so any mail sent to an unqualfied address on the machine is qualified with "@cimedia.com" and then forwarded to the mailhub (since I have also setup smtproutes to forward everything to the mail hub). fetchmail is used to grab mail for the machine and it delivers it through qmail-smtpd via tcpserver. That all works fine. The machine has a limited set of users, let's say it has root and bob. I'd like mail which originates on the machine but is addressed to either root or bob (that is, anyone in /var/qmail/users/cdb on this machine) to be qualified with /var/qmail/control/me instead of with defaulthost. Is there anyway to do this w/o wrapping qmail-inject or replacing it with new-inject? Can I even do this with new-inject? j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: maildir and You have new mail
"Miquel" == Miquel van Smoorenburg [EMAIL PROTECTED] writes: Miquel It turns out that while in mbox format (see also the Miquel mbox(5) manpage that comes with qmail) it only takes one Miquel stat() to find out if new mail arrived, you need to scan Miquel the entire new/ and cur/ directories for maildir Miquel format. With a lot of users who potentially have hundreds Miquel of messages in their spool, this can be quite time- and Miquel disk intensive. Miquel Now, with a bit of thinking I found out that this isn't Miquel nessecary at all. The modification time on the tmp/ Miquel directory indicates when a new message was last delivered, Miquel since that always goes through tmp/. The access time on Miquel the new/ directory (set by readdir()) indicates when a Miquel mail program last checked the maildir mbox. So, a complete Miquel scan of new/ and cur/ stat()ing all files isn't nessecary Miquel at all. Doesn't the presence of any messages in the new dir indicate "You have new mail." and the presence of any messages in the cur dir indicate "You have mail." So you need to do an opendir and a readdir, but you can stop at the first directory entry that looks like a valid message. Hopefully I'm not being to naive. I'm hoping that the OS does _not_ in fact read in the entire directory and then libc just returns one entry at a time. If this is the case, then while you are saving some CPU by not iterating though all of the directory entries, the disk access is still the same, which is likely the expensive part. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: Melissa Virus
"Chris" == Chris Garrigues [EMAIL PROTECTED] writes: Chris Software could be so much better than it is, but the fact Chris that it isn't better doesn't excuse the Chris companies/programers who made it the way it is. I find this especially true of MS software. With the resources that Microsoft has, their software should be the best in the world. It should be so good that no one would even think of using anyone else's software. Yet MS software is at best, mediocre, and at worst, total dung. I fault Bill Gates more than anyone else in the company for this. It's truely sad. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: poor documentation example
"Russ" == Russ Allbery [EMAIL PROTECTED] writes: Russ make Russ make install Russ ./install /depot/col/package/bin BIN Russ ./install /depot/col/package/man MAN For what is that supposed to work? It doesn't work for ucspi-tcp-0.84. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: poor documentation example
"Mark" == Mark Delany [EMAIL PROTECTED] writes: But I'm sure djb knows his way is better, so this is all a waste of breath now, isn't it. Mark So lemme get this right. Dan B. has written and made freely Mark available an MTA that many people like. Russell N. has set Mark up a web site to help distribute information about that Mark MTA. Numerous others are running mirrors for the web site Mark and ftp archives all over the planet. Plenty of people have Mark made patches and alternative distributions freely Mark available. Even more people have provided thousands of hours Mark of free support on this list and Dave S. is running a free Mark web site that archives this list for future prosperity. I don't think that's fair. I didn't criticize the contributions of any of those people. For what it's worth: Thank you all who help to make the Internet a better place. That includes everyone Mark mentioned. Mark And the best you have to offer is what? A sarcastic tirade? A tirade? Please. It was one sarcastic remark. I'm glad that Dan has written the software he has. We rely on it daily. And Dan can do whatever the heck he wants with his software. But everyone on this list knows how difficult it can be to convince Dan that there might be a better way. It's a little frustrating sometimes. Thus my remark. In any case, it was inappropriate, so I apologize. Dan often laments about how fractured the Unix world is. Yet his installation method is yet another non-standard installation method I have to deal with. His software is easy to build and install as long as it conforms to his ideas about how software should be built and installed. I'm willing to live with /var/qmail. I am not willing to live with stuffing everything under /usr/local. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: poor documentation example
"Scott" == Scott D Yelich [EMAIL PROTECTED] writes: I've had tcpserver compile just fine even with HP's broken compiler. It seems as if you're trying to find fault just to try and prove your point. If you know/knew in advance of your non-standard compiler setup you'd be prepared for it. Scott *sigh* Scott You just don't get it... do you. Scott I have a standard compiler set up. I have gcc. I do not Scott have cc. Scott I get 99% of my programs in source and they tell me to edit Scott the Make file and change the "cc" line to "gcc" or to type Scott ./Configure. Both of these get me to compile (maybe I have Scott to define solaris, etc.) just fine. Scott Then comes qmail, et al., does it use Makefile with CC=gcc? Scott no. Does it use ./Configure? no. It says "type make; make Scott config check; # that's all!" Scott BUT IT IS NOT ALL. Scott That's all (I'm trying to say). Well, at least I agree with this. tcpserver is the ONLY package (well, not including other djb packages which have this same breakage) I have ever compiled where not even this works: make CC=gcc Also, none of djb's packages like the idea of compiling a program to look in one place and installing it in another. We use depot for all our package installes, so we compile packages to look in /usr/local, but install them in /depot/col/package_name. All of djb's packages require extra work to get them to install this way. For qmail, it isn't that big a deal because we put it in /var/qmail, but for tcpserver and other djb packages, we want them depotized. For pacakges that use GNU autoconf, this is trivial. 'make install prefix=/depot/col/package_name' and you're done. For all of djb's stuff, you either install by hand or futz around with the various conf-FOO files and make sure you preserve their timestamps when editing them after you've built the package. blech. But I'm sure djb knows his way is better, so this is all a waste of breath now, isn't it. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: mail forward if unread
"Harald" == Harald Hanche-Olsen [EMAIL PROTECTED] writes: Harald Careful, or a crash at the wrong moment might cause a mail Harald to be moved but not forwarded. Of course. I intentionally left out the part about running an occasional cron job (or some such) to check for orphans. We need to leave some exercises for the reader. Harald I think the race condition Harald you are refering to is more benign: It might cause a Harald message to be forwarded more than once (extremely Harald unlikely), or a qmail-inject or mv to fail because the Harald message was yanked under the script's feet (more likely, Harald since the user may be reading his mail at the same time Harald the cron job is running). True. Harald In fact, even moving the Harald message first does not get rid of this latter sort of race Harald condition, since there is a time interval between find Harald discovering the file and the mv command running (or Harald between the user's MUA seeing the file and trying to move Harald it). Ah, true again. Harald If the MUA is not able to cope with the possibility Harald of a message disappearing like that, then the user must Harald precede the reading of his mail by first moving the Harald messages using a more fault tolerant program. My simple Harald script could also need some work in that area. I never Harald intended it to be a complete solution, just a reasonable Harald starting point. Agreed. The user of the script should be aware of the race conditions in any case. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
Re: mail forward if unread
"Harald" == Harald Hanche-Olsen [EMAIL PROTECTED] writes: Harald A simple shell script should do the job: #!/bin/sh find Maildir/new -type f -mtime +1 -print | while read m; do /var/qmail/bin/qmail-inject -a bill mv $m Forwarded/new done There is a race condition here. Move the file before you run qmail inject on it. If the qmail inject fails, move it back. Harald The forwarded message should inherit the original's Harald envelope sender (picked up from the Return-Path field). Harald Override with -f if you prefer. You should also be aware Harald that qmail-inject does some error checking and header Harald processing that is not totally appropriate for mail Harald forwarding. In particular, if the incoming mail has Harald syntax errors in the headers (not an unheard of situation) Harald the above script will fail. A better solution might be to Harald use forward, but then you have to set up the environment Harald variables that forward expects, so this is harder to do. j. -- Jay Soffian [EMAIL PROTECTED] UNIX Systems Administrator 404.572.1941 Cox Interactive Media
