Re: SMTP AUTH in trunk
On 25 Jan 2006, at 22:47, John Peacock wrote: As you might remember, I have been pounding my head against the Danga::* pollserver implementation with reference to the use of AUTH in trunk. Matt gave me some ideas, but I was not able to figure out how to use the CONTINUATION to get the next line of data (since login may require up to 2 additional round-trips and both plain and cram-md5 require 1). So I punted. I added code to Pollserver so that if a transaction is in the AUTH_PENDING state, it will continue to loop through auth_process() until it is complete. I'm too tired to commit this now, but I'll do so in the morning unless I hear lots of complaints (or word from Matt on how to simplify the code using the real continuation logic). The more I think about it this might be the only way to do it. Sorry I haven't been much help on this :-/ Commit away. Matt.
Re: Problem forwarding with plugins/queue/smtp-forward
On 20 Jan 2006, at 19:44, Ovid wrote:
What appears to be the relevant part of the error message from qpsmtpd
is this:
10797 trying to get config for me
10797 FATAL PLUGIN ERROR: Invalid argument at
./plugins/queue/smtp-forward line 49, STDIN line 13.
10797 451 Queuing declined or disabled; try again later
10797 dispatching QUIT
And what's actually throwing the error (comments are the values):
my $smtp = Net::SMTP-new(
$self-{_smtp_server},# localhost
Port = $self-{_smtp_port}, # 2525
Timeout = 60,
Hello = $self-qp-config(me), # Curtis-Poes-Computer.local
) || die $!;
From what I can tell from the Net::SMTP docs, Port is not a valid
parameter. LocalPort is a valid parameter, but that fails with
Address already in use (since postfix is using it).
Does anyone know what I'm doing wrong?
Probably postfix is rejecting the connection for some reason. Can you
tcpdump the connection and see what happens? Is there anything in the
postfix logs?
Matt.
Re: queue plugins and DECLINED question
On 16 Jan 2006, at 16:56, Jeff Macdonald wrote: I have my own queue directory structure that I call minutedir. I took the maildir plugin to get an idea of what to do and I've created my own working plugin. However, I'm curious why folks return a DECLINED vs a DENYSOFT when an error happens. I would think one would like the sending MTA to try again later. The basic premise is to allow stackable queue plugins, so that if one fails (returns DECLINED) then the next one can run and just work. I guess in a live environment you probably don't want that to happen though. Matt.
Re: Support for RFC 1893 - Enhanced Mail System Status Codes
On 12 Jan 2006, at 08:01, John Peacock wrote: Hanno Hecker wrote: That was on Matt's wishlist before it can be included That's interesting, since I was thinking last night it would make the most sense to add this whole block of code to Qpsmtpd::Plugin and you could just do $self-relaying_denied(DENY); from within *any* plugin and cut to the chase. I don't see any performance benefit from making Perl dispatch a class method rather than an object method (and having in directly in the plugin object means that an inheritance walk isn't even required). If we start adding all this stuff to Plugin.pm it has the possibility of getting very bloated - there's so much we could just add in there for convenience. I kind of saw the DSN stuff as a helper class, something not required but useful, hence the externalisation. Matt.
Re: smtp-forward plugin
On 12 Jan 2006, at 07:49, Allan Joergensen wrote: Hi, shouldn't the smtp-forward plugin issue the same errorcode to the connection server as it gets from the backend smtp server? ie. if the backend server says 550 address: Recipient address rejected: User unknown qpsmtpd issues softdeny which means that the remote server will keep on trying to deliver mails to this non-existing user. Yes, probably. A patch for this would be welcome - the smtp-forward plugin was rather hurriedly put together (by yours truly). Matt.
Re: virus alert to recipient
On 12 Jan 2006, at 05:53, rik wrote: Thanks to everybody for your precious help. Matt is it possible to use your code/plugin? I don't have one unfortunately. I think the group would be interested in whatever you come up with though.
Re: SpeedyCGI doesn't work with qpsmtpd on a 64bit OS
On 11 Jan 2006, at 10:57, Mark Powell wrote: Hi, Been quiet for sometime, because I've not had to touch my FreeBSD5S+qpsmtpd+SpeedyCGI setup since my flurry of activity 18months ago. However, now I'm moving the relays onto Dell Poweredge 2850's with the 64bit amd64 version of FreeBSD 6. I'm using the FreeBSD port of SpeedyCGI, but it doesn't seem to work properly on an amd64 compile. Anyone else seen this? I can find no mention of it anywhere which seems strange. Failing getting SpeedyCGI working again, what's the preferred way to speed up qpsmtpd startup? Cheers. The speedycgi code is kind of complex (it embeds the perl interpreter in its own server code) - have you tried pperl? I wrote pperl because of the complexity of speedycgi. Matt.
Re: SpeedyCGI doesn't work with qpsmtpd on a 64bit OS
On 11 Jan 2006, at 18:24, Mark Powell wrote: On Wed, 11 Jan 2006, Matt Sergeant wrote: On 11 Jan 2006, at 10:57, Mark Powell wrote: The speedycgi code is kind of complex (it embeds the perl interpreter in its own server code) - have you tried pperl? I wrote pperl because of the complexity of speedycgi. Well I tried PPerl all those months ago and one relay hung at one point, tho that was on FBSD 4.x. SpeedyCGI also hung on that version of FBSD, I think. On FBSD 5.x SpeedyCGI has been perfect. I'll give PPerl a run on FBSD 6, but I remember you saying once that there were still some issues in PPerl that you'd like to iron out, but never got the time. Did you ever get the time? You comments in that one message on this list made me test only SpeedyCGI on FBSD 5.x. PPerl still seems to be at the same version I would've tried in summer 2004? Yeah, though I suspect more likely changes to qpsmtpd have fixed the issues in pperl, but this stuff is a bit magic and really hard to debug. Sorry not been keeping up with this list. Wasn't there some version of qpsmtpd that did all the pre-forking stuff itself? Yeah I should have just suggested that. Use qpsmtpd-forkserver and you'll be happy. Matt.
Re: virus alert to recipient
On 11 Jan 2006, at 18:18, rik wrote: Hi all, I'm testing Qpsmtpd as a front-end mx for my mail hosting system and I think i'ts great. I've a question for you.. I'd to know if this is possibile without coding my own plugin. If one of my user receive a virus, is it possibile to send an alert to his recipient and save the email in a quarantine directory insted of blocking the message? Yes it's possible, but it requires you write your own plugin (as nobody has done that yet, or they may have but haven't open sourced it). Basically the way I'd do it is via a deny hook which looks at why it was denied and creates a mail to the recipient if it was a virus deny. I'd create the mail using something simple like Email::Send rather than trying to use the qpsmtpd API. Matt.
Re: check_validrcptto_cdb plugin
On 10 Jan 2006, at 05:44, Robin Bowes wrote:
# cdb file containing valid local email addresses
my $validrcptto_file = 'config/validrcptto.cdb';
# don't check if file not readable
return (DECLINED) if ( ! -f $validrcptto_file );
# Open the cdb file and tie to hash
tie my %rcpt, 'CDB_File', $validrcptto_file or
return (DECLINED);
As Ask suggested, use:
my $rcpt = $self-config('validrcptto', 'map') || return DECLINED;
Saves you a bit of code :-)
Re: check_validrcptto_cdb plugin
On 10 Jan 2006, at 10:42, Robin Bowes wrote: What am I doing wrong? Got it. Several things really... 1. Use $validrcptto not %validrcptto 2. Don't specify the .cdb extension (can someone confirm how this works? Does the code take the filename and append .cdb ? Yes. It's a bit strange, and really screws with caching unfortunately. I wish we could just re-write it all - there have been a number of suggestions to do so. 3. (and this is the main problem) The config object is under $self-qp, e.g.: # map to cdb file containing valid local email addresses my $validrcptto = $self-qp-config( 'validrcptto', 'map' ) || return DECLINED; Ah, I adding a proxy to allow $self-config() in trunk, hence why I wrote what I wrote. Matt.
Re: Help with qpsmtpd trunk code
On 5 Jan 2006, at 15:33, John Peacock wrote:
I've attached a segment of the server log, as well as the complete
swaks
session. As you can see, it looks like cram-md5 code is not able to
read the client input at line lib/Qpsmtpd/Auth.pm:280, so it gets
passed
as an unrecognized command. Is there something different I should be
doing while running under pollserver to read STDIN?
Yeah you can't read from STDIN because the socket is non-blocking, and
so you have to just wait to be told there's a line available. This will
require a continuation - I'll look into it now.
I'd also appreciate any hints you could give me on how to understand
the
Danga::* code as well. I expect it would be esential to know how to do
the virus scanning under an asynchronous scheme. It seems like the
Danga::Socket code came from Danga.com's wcmtools, but it isn't
documented (well, not in such a way that I can make heads or tails of
it).
ok, from the top then...
Yes, Danga::Socket is code from Brad Fitzpatrick's livejournal stuff.
It's basically a module that does essentially the same thing as POE or
Event or LibEvent, but is lighter and faster than POE and Event.pm, and
doesn't require XS like LibEvent.
The basic idea is you have a socket (file descriptor) that you want to
receive notification events on - usually just event_read - i.e. data is
available. In order to process these you subclass Danga::Socket to
create a class that handles all the relevant events, and construct an
instance of your new class passing in the socket:
MyDangaSubclass-new($socket). You don't need to store that object
anywhere, it is automatically stored for you in the sea of objects
processed by Danga::Socket.
Danga::Socket uses use fields for field safety and a bit of a
performance gain. So subclasses need to follow suit.
I've also created subclasses of Danga::Socket called Danga::Client and
Danga::TimeoutSocket, which basically add in extra capabilities (mostly
stacked read suspension, line by line processing and timeouts). Usually
you should be subclassing Danga::Client instead of Danga::Socket.
Now in order to get Clamd, smtp-forward and spamd working in a
non-blocking manner each will require their own subclass of
Danga::Client to handle the communication, and a continuation so that
other clients can be processed while those things occur. That's not
really as hard as it sounds, but it does take a bit of getting your
head around.
The easiest thing to do might be to see a simple example client class.
So here's the start of an SMTP client. Note this is subclassing
Danga::Socket instead of Danga::Client, so I've re-implemented some of
the line by line and timeout stuff that Danga::Client does for you:
package SMTPSock;
use base qw(Danga::Socket);
use fields qw(connected ip line);
use IO::Handle;
use Socket;
# total session time
use constant SESSION_TIMEOUT = 3600;
# connection timeout
use constant CONNECT_TIMEOUT = 60;
BEGIN {
our $PROTO = getprotobyname('tcp');
our $PORT = getservbyname('smtp', 'tcp');
}
sub new {
my SMTPSock $self = shift;
$self = fields::new($self) unless ref $self;
$self-{ip} = shift;
my $sin = sockaddr_in($PORT,inet_aton($self-{ip}));
socket(my $sock, PF_INET, SOCK_STREAM, $PROTO);
IO::Handle::blocking($sock, 0);
$self-{connected} = connect($sock, $sin);
$self-SUPER::new( $sock );
$self-watch_read(1);
# connect timeout
$self-AddTimer(CONNECT_TIMEOUT, sub { connect_timeout($self) });
return $self;
}
sub log {
my SMTPSock $self = shift;
warn($self-{ip} : , @_);
}
sub connect_timeout {
my SMTPSock $self = shift;
if (!$self-{connected} !$self-{closed}) {
$self-log(Connection timed out\n);
$self-close(connect timeout);
}
}
sub session_timeout {
my SMTPSock $self = shift;
$self-close(session timeout);
}
sub event_read {
my SMTPSock $self = shift;
if (!$self-{connected}) {
if (eof($self-sock)) {
$self-close(connection failed);
}
else {
$self-log(connected\n);
$self-{connected} = 1;
$self-AddTimer(SESSION_TIMEOUT, sub {
session_timeout($self) });
}
}
else {
my $bref = $self-read(8192);
return $self-close($!) unless defined $bref;
$self-process_read_buf($bref);
}
}
sub close {
my SMTPSock $self = shift;
my $code = shift;
$self-log(Closing: $code\n);
$self-SUPER::close($code);
}
sub event_err {
my SMTPSock $self = shift;
$self-log(error on socket\n);
$self-close($self-{ip} : error on socket);
}
sub event_hup {
my SMTPSock $self = shift;
$self-log(HUP\n);
$self-close($self-{ip} : remote end hangup);
}
sub process_read_buf {
my SMTPSock $self = shift;
my $bref = shift;
$self-{line} .= $$bref;
while ($self-{line} =~ s/^(.*?\n)//) {
my $line = $1;
my $resp
Re: transmitting before SMTP greeting error?
On 6 Jan 2006, at 13:58, Les Mikesell wrote: I'm running an ancient network monitoring tool called 'spong' and it's test for a working smtp service is to connect to the port, send 'quit' and parse what it receives for a '220'. With qpsmtpd as set up in the SMEserver beta, it is reporting a failure: 450 Connecting host started transmitting before SMTP greeting Is there an RFC requirement that the connecting host MUST wait before sending anything? Yes, see RFC 2821 section 4.3.1. Matt.
Re: Fix for tls problems
On 4 Jan 2006, at 12:19, John Peacock wrote: John Peacock wrote: I'm going to work on some other changes and apply them to the 0.3x branch (and then merge back to trunk), since it is odd that Qpsmtpd::Connection-new() doesn't do anything with any initializing parameters passed in (there is the start() sub to do that). Actually, how do the following patches look. I abstracted out the hard parameters of a Qpsmtpd::Connection object and added a clone() method. I don't like that plugins/tls needs to know what parameters of the connection object are important enough to copy. I'm not so sure. The TLS spec actually specifies what details need to be kept...
Re: Thank You for qpsmtpd ... and a deployment note....
On 3 Jan 2006, at 07:35, Robin Bowes wrote: Robin Bowes said the following on 03/01/2006 00:20: 1. a softlimit value of 3500 works for me on x86_64 (qpsmtpd process is ~30M) I just enabled the tls plugin and memory usage jumped to ~45M! 7086 qpsmtpd 15 0 44112 12m 2788 S 0.0 0.6 0:00.31 qpsmtpd That's kinda big, isn't it? Sadly that's Perl for you. I'm going to be installing a mail server on an x86_64 box soon with an incoming concurrency of around 50. That's 2250MB RAM just for the qpsmtpd processes. Eek. No, most of the RAM is in modules which are loaded in the parent process, so the forked children use Copy On Write, meaning the memory is shared between them unless it's written to. You should be using a lot less than that. Matt.
Re: [OT] swaks patch
On 3 Jan 2006, at 12:00, John Peacock wrote: I needed a simple way to send lots of messages (for testing the trunk code) so I patched swaks: http://jetmore.org/john/code/#swaks to send multiple identical messages at once (via fork). Please see the attached patch. Remember if you are doing this, you will need to up the --limit-connections and --max-from-ip options when running qpsmtpd. I was able to get the pollserver from trunk to accept 100 simultaneous messages without breaking a sweat (although it took 30 seconds so the last swaks session actually timed out and got hit by check_earlytalker). The nice thing was the load never went above 0.55 (700MHz Celeron)! Can you explain why it got hit by check_earlytalker? Doesn't seem to make sense...
Re: go crazy with me
On 20 Dec 2005, at 12:00, Ken Simpson wrote: Not sure what you mean by it not scaling - can you elaborate? Sure it uses more ram than multiplexing, but even for a high traffic mail server like apache.org's the mail-in-apache2 model works well (apache.org uses Apache::Qpsmtpd for email). I'm curious as to how you've mixed things up though - if the details aren't private IP I'd love to know more. I'll cut in so that Stas can save his keyboarding wrist. [snip] Thanks for that - that's what we found too, but not many places needed the scalability. Anyhow if anyone's interested in an open source version of what you've done they can check out the current svn trunk of Qpsmtpd - http://smtpd.develooper.com/ - it uses Danga::Socket instead of Event::Lib (similar concepts though) to multiplex all connections. Matt.
Re: go crazy with me
On 19 Dec 2005, at 14:33, Stas Bekman wrote: JT Smith wrote: Yup, I've actually already done it that way with both Parallel::ForkManager in one instance and Proc::Queue as an alternative. I added in event handling with both Event and Event::Lib as seperate trials. All those implementations were relatively easy to do. But the question becomes, why? If everything else is running in Apache, why start a seperate service to run these tasks? And again, I said I want to go crazy. Let's not figure out how else we could do that (I already know that), but how could we do it using Apache? Here at mailchannels.com we have first used mp2 to handle the email traffic shaping entirely inside mod_perl2, but the nature of our product is so different from serving HTTP, it just won't scale (mostly memory-wise, but also too many processes). Not sure what you mean by it not scaling - can you elaborate? Sure it uses more ram than multiplexing, but even for a high traffic mail server like apache.org's the mail-in-apache2 model works well (apache.org uses Apache::Qpsmtpd for email). I'm curious as to how you've mixed things up though - if the details aren't private IP I'd love to know more. Matt.
Re: Catalyst w/ AxKit
I guess there's probably not much of AxKit you'd be using, so might be easier to just use an XSLT transformer inside Catalyst... Alternatively if there's bits of AxKit you want to use (stylechoosers, etc) then a Catalyst Provider would be a good idea. On 16 Dec 2005, at 07:37, [EMAIL PROTECTED] wrote: Interested in this too. However, this is with an application build in axkit but switching to catalyst instead of taglibs (more or less same pipeline as yours example). So I was wondering if anyone here had any experience using AxKit for presentation side (front end) of an application build using Catalyst? I've done some preliminary googling and haven't been able to find much. I'm thinking of doing something with a pipeline like this: Request - Apache (AxKit) - Catalyst - XML - AxKit - XSLT - Response If someone here could point me in the right direction or shout out a couple of pointers/pitfalls it be greatly appreciated. Thanks, - Jason Mills - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: clamav plugin: Return DENYSOFT when clamd not available
On 16 Dec 2005, at 06:31, Filippo Carletti wrote: If clamdscan cannot connect to clamd, clamdscan plugin returns DECLINED and the virus is delivered to the user. A safer option would be to return DENYSOFT when clamd is not reachable. I agree with the patch. I'm sure some idiots will turn on the clamd plugin without running clamd and wonder why they get no mail, but oh well.
Re: XMLBeans help requested
On 15 Dec 2005, at 10:33, javan (sent by Nabble.com) wrote: I am glad this worked for you. Please have a practice of rating any response you get or read so that it's helpful for other people searching for same issue resolution. Err, I'm not sure what's going on here, but this is a mailing list for AxKit Users. Please remove this bridge to the mailing list and please don't encourage Java Beans questions here - this has nothing to do with AxKit. Thanks. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Body rewrite
On 14 Dec 2005, at 11:25, Oscar Retana wrote: Let's say a plugin finds something really bad inside an email (an exe attachment). I want to DENY that email, but I also want to notify the recipient[*]. I was thinking of creating a new e-mail from scratch and send it to the recipients... but then, I thought it will be simpler to strip completely the contents (body, any attachments, everything! - keep only the headers) and let the poor email pass through with a new body including some kind of message for the end user. So, my question is if the current qpsmtpd modules will allow me to rewrite easily the body of an email. Here's a way you can do that: Set config/memory_threshold to 0 In your AV plugin call $transaction-body_resetpos(); And then write your new body with $transaction-body_write($data); You'll also need to change the headers too probably. I'll leave that part as an exercise ;-) Matt.
Re: SMTP store and forward requires DSN for integrity
On 10 Dec 2005, at 16:54, Douglas Otis wrote: The BATV is a few lines of code that adds a private tag with a time limit set in days. BATV helps dramatically by eliminating the DATA phase and all that is involved in handling messages. In addition, once BATV becomes more widely deployed, the DSN refusal offers an alert about accepting more such messages from that IP address. BATV will make forged DSNs a thing of the past, irrespective of where a recipient list is checked, an AV or SPAM filter is added, etc. And BATV will never be widely deployed because it breaks every single system out there that keys off the return path. And there are a lot of these systems. Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: SMTP store and forward requires DSN for integrity
On 12 Dec 2005, at 15:50, John Levine wrote: And BATV will never be widely deployed because it breaks every single system out there that keys off the return path. And there are a lot of these systems. I keep hearing that, but other than a few ezmlm lists and the occasional tired fax gateway, I never run into them. Where are they? I can't enumerate them all for you - I don't know them all. That's the point though - you can't know all the systems you may be breaking by changing the way your MAIL FROM works. Here's a few that I've told you about before: Whitelist/Blacklist entries Quarantine systems Anything based on .qmail files Some auto-forwarders Auto-responders Now you can potentially whitelist around these issues (to some extent), but while that works for geek mail systems it doesn't scale up very well. I know you (personally) whitelist around some problem systems in your implementation but can you expect Grandma to do that? Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 5 Dec 2005, at 13:23, [EMAIL PROTECTED] wrote: I added a test case (check-in [2798]) that checks to make sure that sqlite3_result_text is able to deal with embedded '\000' characters in a string. I appears to work fine. I cannot reproduce the problem Can you suggest other ways of producing the problem? Not yet. Let me write a test case and see if there's really a bug or not. Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 2 Dec 2005, at 13:07, [EMAIL PROTECTED] wrote:
Right. So it's retreival that's the issue when this occurs, because I
do:
int col_type = sqlite3_column_type(stmt, i);
and it returns SQLITE_TEXT, so I then do:
val = (char*)sqlite3_column_text(stmt, i);
which doesn't return a length for me.
Would sqlite3_column_bytes() return the right length there rather than
me doing strlen() on the resulting data?
yes it will.
OK, so 1.11 is on CPAN which fixes this. However I have another bug
report about this not working for user defined functions, where I do
this:
s = SvPV(result, len);
sqlite3_result_text( context, s, len, SQLITE_TRANSIENT );
(SvPV is a macro that retrieves a char* from result, and as a side
effect sets len to the length of the string in bytes, even if it
contains nuls).
Is this maybe a bug in sqlite3_result_text()? I could patch it to do:
if (memchr(s, 0, len)) {
/* if the result contains NUL(s) treat it as a blob */
sqlite3_result_blob(context, s, len, SQLITE_TRANSIENT );
}
else {
sqlite3_result_text( context, s, len, SQLITE_TRANSIENT );
}
But that seems a waste of resources if it's a bug in
sqlite3_result_text().
Matt.
__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
__
Re: [sqlite] problem with blobs (perl code)
On 2 Dec 2005, at 13:07, [EMAIL PROTECTED] wrote:
Right. So it's retreival that's the issue when this occurs, because I
do:
int col_type = sqlite3_column_type(stmt, i);
and it returns SQLITE_TEXT, so I then do:
val = (char*)sqlite3_column_text(stmt, i);
which doesn't return a length for me.
Would sqlite3_column_bytes() return the right length there rather than
me doing strlen() on the resulting data?
yes it will.
OK, so 1.11 is on CPAN which fixes this. However I have another bug
report about this not working for user defined functions, where I do
this:
s = SvPV(result, len);
sqlite3_result_text( context, s, len, SQLITE_TRANSIENT );
(SvPV is a macro that retrieves a char* from result, and as a side
effect sets len to the length of the string in bytes, even if it
contains nuls).
Is this maybe a bug in sqlite3_result_text()? I could patch it to do:
if (memchr(s, 0, len)) {
/* if the result contains NUL(s) treat it as a blob */
sqlite3_result_blob(context, s, len, SQLITE_TRANSIENT );
}
else {
sqlite3_result_text( context, s, len, SQLITE_TRANSIENT );
}
But that seems a waste of resources if it's a bug in
sqlite3_result_text().
Matt.
__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
__
Re: [sqlite] problem with blobs (perl code)
On 5 Dec 2005, at 13:23, [EMAIL PROTECTED] wrote: I added a test case (check-in [2798]) that checks to make sure that sqlite3_result_text is able to deal with embedded '\000' characters in a string. I appears to work fine. I cannot reproduce the problem Can you suggest other ways of producing the problem? Not yet. Let me write a test case and see if there's really a bug or not. Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 2 Dec 2005, at 08:07, [EMAIL PROTECTED] wrote: Would sqlite3_column_bytes() return the right length there rather than me doing strlen() on the resulting data? yes it will. Doh! In that case then 1.11 will head to CPAN with blobs working transparently. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 1 Dec 2005, at 21:52, [EMAIL PROTECTED] wrote: SQLite does has a separate BLOB type. But for TEXT types, SQLite still works like Perl and carries around a length so that the string can have embedded '\000' characters. I just added a test to the test suite to verify that this works. Suppose you do this: sqlite3_bind_text(pVm, 1, "abc\000xyz\000pq", 10, SQLITE_STATIC); If this is part of an INSERT, say, then you will insert a 10-character string that happens to contain a couple of extra \000 characters. Right. So it's retreival that's the issue when this occurs, because I do: int col_type = sqlite3_column_type(stmt, i); and it returns SQLITE_TEXT, so I then do: val = (char*)sqlite3_column_text(stmt, i); which doesn't return a length for me. Would sqlite3_column_bytes() return the right length there rather than me doing strlen() on the resulting data? Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 1 Dec 2005, at 21:52, [EMAIL PROTECTED] wrote: SQLite does has a separate BLOB type. But for TEXT types, SQLite still works like Perl and carries around a length so that the string can have embedded '\000' characters. I just added a test to the test suite to verify that this works. Suppose you do this: sqlite3_bind_text(pVm, 1, abc\000xyz\000pq, 10, SQLITE_STATIC); If this is part of an INSERT, say, then you will insert a 10-character string that happens to contain a couple of extra \000 characters. Right. So it's retreival that's the issue when this occurs, because I do: int col_type = sqlite3_column_type(stmt, i); and it returns SQLITE_TEXT, so I then do: val = (char*)sqlite3_column_text(stmt, i); which doesn't return a length for me. Would sqlite3_column_bytes() return the right length there rather than me doing strlen() on the resulting data? Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 2 Dec 2005, at 08:07, [EMAIL PROTECTED] wrote: Would sqlite3_column_bytes() return the right length there rather than me doing strlen() on the resulting data? yes it will. Doh! In that case then 1.11 will head to CPAN with blobs working transparently. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 1 Dec 2005, at 15:10, [EMAIL PROTECTED] wrote: So in the example of $sth->execute($blob), if $blob contains an integer, use sqlite3_bind_int64(), or if $blob contains a string use sqlite3_bind_text(), or if $blob contains a blob, then use sqlite3_bind_blob(), and so forth. Is there something about perl internals that prevents the above from working? Yes. Perl has no concept of blobs. A scalar variable can be one of: IV (integer) UV (unsigned integer) NV (double) PV (string) so a blob is just a string - but perl carries a length around with it so you can have binary data in there. You'd have to check something like strlen(data) != len to determine if it contained NULs (or just scan for the NUL - same difference). Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On Thu, 1 Dec 2005, Matt Sergeant wrote: > > Looking now at the DBI documentation, I see that values bound using > > execute are 'usually treated as "SQL_VARCHAR" types unless the driver > > can determine the correct type (which is rare)'. Because it is simple > > to scan the string for NUL's, I guess I consider this one of those > > rare cases where the driver can just 'do the right thing'. > > It's pointless though to do that, because then when they try and get data > out the same way they won't understand why it's truncated. At least this > way they have to bind properly on both in and out. Oh, I forgot to answer the issue - if I did a scan for NUL bytes on every string going into the system it would really hurt those doing performance sensitive apps. Unfortunately for you that includes me :-) Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On Thu, 1 Dec 2005, Matt Sergeant wrote: Looking now at the DBI documentation, I see that values bound using execute are 'usually treated as SQL_VARCHAR types unless the driver can determine the correct type (which is rare)'. Because it is simple to scan the string for NUL's, I guess I consider this one of those rare cases where the driver can just 'do the right thing'. It's pointless though to do that, because then when they try and get data out the same way they won't understand why it's truncated. At least this way they have to bind properly on both in and out. Oh, I forgot to answer the issue - if I did a scan for NUL bytes on every string going into the system it would really hurt those doing performance sensitive apps. Unfortunately for you that includes me :-) Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [sqlite] problem with blobs (perl code)
On 1 Dec 2005, at 15:10, [EMAIL PROTECTED] wrote: So in the example of $sth-execute($blob), if $blob contains an integer, use sqlite3_bind_int64(), or if $blob contains a string use sqlite3_bind_text(), or if $blob contains a blob, then use sqlite3_bind_blob(), and so forth. Is there something about perl internals that prevents the above from working? Yes. Perl has no concept of blobs. A scalar variable can be one of: IV (integer) UV (unsigned integer) NV (double) PV (string) so a blob is just a string - but perl carries a length around with it so you can have binary data in there. You'd have to check something like strlen(data) != len to determine if it contained NULs (or just scan for the NUL - same difference). Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Time::localtime under mod_perl2 showing output in client-side time zones
On 6 Nov 2005, at 10:12, Richard N. Fogle wrote: We've tried the script with CGI(), I currently have it posted without. Same results. We've tried placing the time functions in a subroutine with private variables with the same results. Tried using Thread::Semaphore with the same results. If anyone would like the URL to see this for themselves write me off list. If there is need for additional information than I apologize in advance for not submitting it and will do so upon request. Try calling POSIX::tzset() before the call. It should fix it.
Re: Problem sending
On 30 Nov 2005, at 14:17, John Peacock wrote: [EMAIL PROTECTED] wrote: I think the point was he's sending to me (and I'm running the trunk). Ah, I missed that part of the equation... Unfortunately I'm now up to about 200k spams a day on that machine, so finding his disconnect in the log of all those is not going to be easy. fgrep??? :( I still need to do some work on the logging - it currently only logs the fd# with the lines to distinguish them. So you have to find the connecting IP, and find the fd# and grep out lines for that fd#. Kind of a pain to say the least. Matt.
Re: Problem sending
On 30 Nov 2005, at 15:17, John Peacock wrote: Matt Sergeant wrote: I still need to do some work on the logging - it currently only logs the fd# with the lines to distinguish them. So you have to find the connecting IP, and find the fd# and grep out lines for that fd#. Kind of a pain to say the least. But if you had the hook_reject log lines like Robin is suggesting, it would be a simple matter to grep out the summary line with all of the information in one place. I don't think it's that simple. I suspect the disconnect is happening because qpsmtpd for some reason isn't seeing the line from the client, so eventually disconnects the end user with a timeout. And that doesn't get logged as a reject. That plus logging hooks were are broken in trunk for reasons I'm unsure of. Matt.
Re: pperl? high_perf?
On 28 Nov 2005, at 21:41, John Peacock wrote: Is it worth me trying the high_perf branch? The high_perf branch was merged to trunk and currently doesn't function as a complete system (like tags/0.31 does) since not all of the plugins have been rewritten to use the new code. I'd like those who couldn't get it working to try again now. I have it running on a real (non-spamtrap) system now, and would be happy to help debug things. Matt.
Re: Amavisd-new to qpsmtpd
On 28 Nov 2005, at 18:03, David Beveridge wrote: No, the external postfix process opens up a real time connection to qpsmtpd. So when you are talking to port 25 (postfix) it is proxying it through to qpsmtpd, Kind of like how http requests are proxied through squid. That is why 550 errors from qpsmtpd end up back at the original sender. I'm confused by what this gains you though. Why not just qpsmtpd-postfix?
Re: forkserver: when to drop privileges
On 21 Nov 2005, at 20:07, Robert Spier wrote: Since I want to include this logging plugin in my qpsmtpd-0.31.1 RPMs to get rid of the dependency on an external timestamping tool, I'm asking if anybody relies on qpsmtpd-forkserver loading the plugins as root. I hope not. (Although the not-running-as-root bit me once before... but i worked around it -- and wouldn't really want to support it anyway.) I agree. Lets drop root ASAP.
Re: Segmentation Fault
On 18 Nov 2005, at 12:27, Lars Skjærlund wrote: Hi list, I'm running AxKit 1.62 - and all of a sudden, my kit has stopped working. Instead, I get loads of 'Segmentation fault (11)' in the logfile. Googling a little around, I see this is a wellknown phenomena, however, most hits are rather old. What's the present state - and what can I do about it? Generally caused by upgrading libxml2/libxslt without recompiling the relevant perl modules. Hard to say though with so little info. Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Rate limiting invalid address senders?
On 16 Nov 2005, at 19:35, [EMAIL PROTECTED] wrote: So, I finally got qpsmtpd working, it was actually very easy. Now I'm going to make my sysadmin actually install it for the company spam filtering. One of the ideas he had for filtering was to throttle people who sent more than two or three emails to addresses that didn't exist. I thought this would be a fairly easy plugin to write, but I had a couple of concerns. First I wanted to ask if it already existed someplace? Secondly, we would be using qpsmtpd merely as a front end to an already running mail daemon (I think it's postfix), so I would need some way for qpsmtpd to consult the mail daemon to decide if the address was valid or not. Can anyone suggest me some documentation to read for those problems? You could adapt the plugin I hacked together at the end of my article: http://www.oreillynet.com/pub/a/sysadmin/2005/09/15/qpsmtpd.html It's designed for rate limiting hosts on dnsbl's, but you could easily modify for invalid accounts. Matt.
Re: AxKit SVN repos
On 13 Nov 2005, at 17:39, Matt Sergeant wrote: OK guys, AxKit is ready to be migrated to SVN, and the process has started. Could you please all take a quick look at http://svn.apache.org/repos/test/xml/axkit/ - this is the standard migration style, and looks right to me, but maybe I've missed something - if so let me know ASAP. If all goes well we'll be migrated this week. And then we'll migrate the mailing list too. OK, AxKit is all migrated to SVN. CVS is read only. http://svn.apache.org/repos/asf/xml/axkit/ svn co https://svn.apache.org/repos/asf/xml/axkit/trunk xml-axkit
AxKit SVN repos
OK guys, AxKit is ready to be migrated to SVN, and the process has started. Could you please all take a quick look at http://svn.apache.org/repos/test/xml/axkit/ - this is the standard migration style, and looks right to me, but maybe I've missed something - if so let me know ASAP. If all goes well we'll be migrated this week. And then we'll migrate the mailing list too. Matt.
Fwd: Thanks
Does anyone know of an answer to this question? Begin forwarded message: From: Dillip Kandy [EMAIL PROTECTED] Date: 12 November 2005 05:46:17 GMT-05:00 To: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: Thanks Hi Matt Thanks for writing such an wonderful article on qpsmtpd on oreillynet website. I use qmail for our mail server and I could infuse qpsmtpd alongwith qmail on our server efficiently and the result is just amazing, one can see the spammers running away and legitimate traffic coming in :-) . Kudos to Ask Bjørn Hansen for the creativity and it is all achieved with Perl that's more amazing. I have a small question...if I want some particular domain as whitelist for qpsmtpd then what I should do? Sorry, I am not good at perl :-( if you have some plugin for it then I can readily use it or just provide me any link. Again, thanks for the nice article regards Dillip Kandy
Re: multi-valued dns lookups in lib/Danga/DNS?
On 11 Nov 2005, at 09:24, Matt Sergeant wrote: On 10 Nov 2005, at 18:55, Brian Grossman wrote: I think so. Net::DNS::Resolver::send() returns either a Packet object or undef, so it seems reasonable to assume there will be only one packet even when using the asyncronous interface (bgsend). Also, the bgread() docs say that you should close the socket after reading it; I take that as confirmation that there will only be one packet. A plugin can emit several unrelated queries before calling for a CONTINUATION (mine does), so I think changing the Danga::DNS callback API will be necessary. Actually I don't think it's that hard. Patch coming. Does that work for you?
Re: multi-valued dns lookups in lib/Danga/DNS?
On 11 Nov 2005, at 16:09, Brian Grossman wrote: A plugin can emit several unrelated queries before calling for a CONTINUATION (mine does), so I think changing the Danga::DNS callback API will be necessary. Actually I don't think it's that hard. Patch coming. Does that work for you? Doesn't this (from trunk/plugins/dnsbl): -$qp-finish_continuation unless $pending; +$qp-finish_continuation if $qp-input_sock-readable; make the continuation finish after the first callback, regardless or what else may be in the response packet? No, it should wait until the DNS.pm instance disappears (because its DESTROY method sets $client-readable to on again). I suspect this because trunk/lib/Danga/DNS.pm:new() sends the query then immediately calls $client-enable_read(). Hmm - are you sure it doesn't set readable to off? (Wow, lib/Danga/DNS has changed a lot since I last synced it to my production branch.) :-) I'm using it for other production stuff (outside of Qpsmtpd - such as processing log files and doing hundreds of thousands of DNS queries - and it really rocks for that), so needed it to be a bit more complete. Matt.
Re: multi-valued dns lookups in lib/Danga/DNS?
On 9 Nov 2005, at 23:06, Brian Grossman wrote:
I was thinking about implementing dnsbl return codes and a brief
perusal of lib/Danga/DNS/Resolver.pm makes me think that resolver is
a
bit hostile to the idea of multiple returns to a lookup.
Why? You should just get the callback called twice.
Confirmed.
But that makes notes('pending_dns_queries') (in trunk/plugins/dnsbl) go
negative, revealing a race between receiving the rest of the answers
and
going on to the next plugin.
Got any suggestions on a more resilient way to lock here?
Hmm. I don't know too much about DNS, but can we assume that all the
responses will come back in one packet? If so I guess we can accumulate
the results and do the callback once. Or if we don't want to change the
API I guess we could have a finished() callback. Thoughts?
Matt.
__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
__
Re: To, CC Bcc
On 5 Nov 2005, at 09:25, Anilkumar wrote:
Dear Matt,
I found the command for getting all the e-mail id
from mail.
my @recipients = $transaction-recipients;
foreach my $recip (@recipients)
{
$user = $user . , . $recip-address;
}
$user = substr($user,2);
This is easier as:
$user = join(', ', map { $_-address } @recipients);
my $size= @recipients;
$user will give me all the E-mail seperated by comma
and $size will give me total no of receipents.
Now i am working for finding the Subject.
$transaction-mail-header('Subject');
Note that this will be the possibly encoded subject (i.e. it might be
in base64).
Matt.
Re: To, CC Bcc
On 4 Nov 2005, at 19:01, Bob Dodds wrote: If I hand off a list of recipients to qmail using the same method as qmail-queue, the recipient's source will show To: undisclosed recipients if there is a bcc. qmail must be doing something with the bcc header, because I passed it a simple string of [EMAIL PROTECTED], not any [EMAIL PROTECTED] It's a confusion about how mail works with BCCs. BCC is purely a client facility - there is no BCC header as such - it should be immediately stripped by the client before it sends to the SMTP server, and turned into RCPT TO commands.
Re: To, CC Bcc
On 5 Nov 2005, at 05:46, Anilkumar wrote: what will i get my @recipients = $transaction-recipients ; suppose i am sending mail with 2 id in To: field 3 id in Cc: field and 4 id in Bcc: field i want some logic to get the answer 9 and all the e-mail id. Have you tried it yet? This will work, and will give you 9 addresses. Matt.
Re: multi-valued dns lookups in lib/Danga/DNS?
On 4 Nov 2005, at 20:02, Brian Grossman wrote: I was thinking about implementing dnsbl return codes and a brief perusal of lib/Danga/DNS/Resolver.pm makes me think that resolver is a bit hostile to the idea of multiple returns to a lookup. Why? You should just get the callback called twice. Matt.
ANNOUNCE: AxKit 1.7
It really has been a long time, hasn't it... Well AxKit 1.7 is finally out. The core developers have been very busy getting on with real life, so it took a long time to make a release - sorry about that. The focus now is on porting to Apache2, for what will be called AxKit 2. While we had early hopes of a new design/architecture for AxKit 2, I think right now we need to concentrate on just a straight port. I've started coding that already, but don't expect anything soon :-) Features of this release: - Upgrade license to Apache 2.0 - Extended SimpleTaglib with several new features - Added pseudo-protocol registry - Added POST data provider (Apache::AxKit::Provider::POST) - Fix: Apache::Test params were ignored - Added AxDirHandler config option - Many UTF-8 related fixes - Log errors on separate lines for newer Apache's - Improve gzip cache handling and delivery Available now at the usual download location: http://www.apache.org/dyn/closer.cgi/xml/axkit/ Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: To, CC Bcc
On 4 Nov 2005, at 13:11, Anilkumar wrote: Deal All John, Now i am again struck. No i need list of E-mail id in To, CC, and BCC. if i get a single list then its good, individual list is also okay. i also need subject of mail from plugin. please give me variable name for finding out these and where to write the plugin to get proper values. John, Thanks in Advance. Pretty much these values will be in $plugin-qpsmtpd-transaction-recipients (as Qpsmtpd::Address objects), though you could go looking in the headers of the email - but Bcc should never be seen, and To/CC may contain entries that aren't in the list of local recipients (because they got delivered elsewhere). Hard to say what you should look at without knowing what it is you want to do.
Re: [sqlite] Request for comment: Proposed SQLite API changes
On 3 Nov 2005, at 08:18, [EMAIL PROTECTED] wrote: As currently implemented, when an error occurs during sqlite3_step(), the function returns SQLITE_ERROR. Then you have to call either sqlite3_reset() or sqlite3_finalize() to find the actual error code. Suppose this where to change in version 3.3.0 so that the actual error code was returned by sqlite3_step(). That would mean that moving from version 3.2.7 to 3.3.0 might involve some minor code changes. The API would not be 100% backwards compatible. But the API would be cleaner. What does the community think about such a change? Changing backwards compatibility considered very bad by me. What a mess that would be for DBD::SQLite if/when someone updates sqlite.so and everything stops working in very subtle ways. Yuck! Wrap it in a new function please. Another proposal: Suppose that when creating an sqlite3_stmt using sqlite3_prepare, the original SQL text was stored in the sqlite3_stmt. Then when a schema change occurred, the statement was automatically recompiled and rebound. There would no more SQLITE_SCHEMA errors. But sqlite3_stmts would use a little more memory. And sqlite3_step might take a little longer to initialize sometimes if it found it needed to rerun the parser. What about this change? Is it a worth-while tradeoff? Judging by the followup you posted this can't be done easily, but I'd be happy if you can find a way. Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Worm Blowback Storm
On 3 Nov 2005, at 11:51, John Peacock wrote:
Guillaume Filion wrote:
Remember that once you start VERPing your outgoing mail, you should
wait for some time (I'd say one week) before rejecting bounces that
aren't VERPed. That way, you can receive bounces to the mails that
were sent when you didn't do VERP.
Yeah, I know. I would probably opt for 10 days, since there may be
some weird MTA's out there with a longer decay rate.
I get very few legit bounces in any case that it might be more
worthwhile to create a tarbaby defense (accept but not deliver), at
least for the moment.
I went through the blowback storm thing around the time I started using
qpsmtpd. Here's the plugin I use. It's probably not what you want to
use though.
#!/usr/bin/perl -w
sub register {
my $self = shift;
$self-register_hook('mail' = 'mail_handler');
}
sub mail_handler {
my ($self, $tran, $sender) = @_;
if ($sender-format eq ) {
return DENY, Returned mail not accepted here;
}
elsif ($sender-format =~ /DAEMON/i) {
return DENY, MAILER DAEMON mail not accepted here;
}
return DECLINED;
}
Re: [sqlite] Request for comment: Proposed SQLite API changes
On 3 Nov 2005, at 08:18, [EMAIL PROTECTED] wrote: As currently implemented, when an error occurs during sqlite3_step(), the function returns SQLITE_ERROR. Then you have to call either sqlite3_reset() or sqlite3_finalize() to find the actual error code. Suppose this where to change in version 3.3.0 so that the actual error code was returned by sqlite3_step(). That would mean that moving from version 3.2.7 to 3.3.0 might involve some minor code changes. The API would not be 100% backwards compatible. But the API would be cleaner. What does the community think about such a change? Changing backwards compatibility considered very bad by me. What a mess that would be for DBD::SQLite if/when someone updates sqlite.so and everything stops working in very subtle ways. Yuck! Wrap it in a new function please. Another proposal: Suppose that when creating an sqlite3_stmt using sqlite3_prepare, the original SQL text was stored in the sqlite3_stmt. Then when a schema change occurred, the statement was automatically recompiled and rebound. There would no more SQLITE_SCHEMA errors. But sqlite3_stmts would use a little more memory. And sqlite3_step might take a little longer to initialize sometimes if it found it needed to rerun the parser. What about this change? Is it a worth-while tradeoff? Judging by the followup you posted this can't be done easily, but I'd be happy if you can find a way. Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Qpsmtpd::Address POD and tests
On 2 Nov 2005, at 13:52, John Peacock wrote: I just committed some preliminary POD describing the available methods in Qpsmtpd::Address. I also threw in a simple test of the overloaded comparison method. Thanks. I assigned the copyright to Peter, since he started it, but I wonder if we want/need to get better about being consistent. Should we assign our own copyrights as we add stuff or should we sign over the copyright to Ask for everything? Does anyone care? If we assign copyright to anyone it should be a foundation (TPF?). For a project the size of qpsmtpd though I don't think we need to be re-assigning copyright. *shrug* Matt.
Re: [svn:qpsmtpd] r557 - branches/0.31/lib/Qpsmtpd
On 31 Oct 2005, at 12:51, [EMAIL PROTECTED] wrote: +'=' = \spaceship, Is this wise? cmp yes, but spaceship?
Re: [svn:qpsmtpd] r557 - branches/0.31/lib/Qpsmtpd
On 1 Nov 2005, at 10:15, John Peacock wrote: I guess I'll just remove the '=' binding and rename the sub to addr_cmp instead. See what you think of the next commit... Yeah I like it. I hate to be a pain in the ass but I'd love to see you add something to t/qpsmtpd-address.t too :-)
Re: turning off the cache iin a plugin
On 26 Oct 2005, at 13:33, Ken Neighbors wrote: By the way, I'm curious, what method are you using to generate the content on the fly? I'm using XSP and the problem I'm working around is that it always turns *off* AxKit's caching for the final output (PDF in my case) even when I say $r-no_cache( 0 ). I've worked around this problem by subclassing Language/XSP.pm to keep the cache enabled. (If anyone needs this feature, let me know.) For the next version of AxKit (not 1.7, which I'm ready to release now once I remember how to do apache's release procedures) I'll be implementing XSP caching properly end to end. Currently you can cache the output of the XSP (i.e. the XML that the XSP produces) but it will still run the rest of the phases after that, which is bad. In order to fix that I need to have the cache dependency checking code call into the XSP page (assuming it has been compiled) to check if we have a cached page. That adds a bit of complexity to the dependency checking, but it's the right thing to do. Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
A competitor
http://svn.apache.org/repos/asf/httpd/mod_smtpd/trunk/ It's actually turned out fairly interesting looking.
AxKit 1.7 Final RC
OK, there's a final release candidate of 1.7 up at http://axkit.org/download/AxKit-1.7.tar.gz If there are no test failures within 24 hours we'll make this 1.7 final. Only test failure fixes will make it into CVS.
Re: Qsmtpd::Address design question
On 10 Oct 2005, at 17:29, Brian Grossman wrote: Forkserver's going to be miserable whether it's array or hash. I was thinking in terms of PollServer and Matt's (I think it was Matt's) claim of concurrency 10,000 in his spamtrap. Of course, there's so many hashes running around already, surely another one won't make all that much difference. Indeed. We run about 160M handling a peak of 3000 concurrent connections. So just multiply by 4 to get 10k.
Re: Qsmtpd::Address design question
On 10 Oct 2005, at 18:48, Peter Eisch wrote: On 10/10/05 4:58 PM, Matt Sergeant [EMAIL PROTECTED] wrote: Indeed. We run about 160M handling a peak of 3000 concurrent connections. So just multiply by 4 to get 10k. Do you run spamassassin on these systems? If so, how do you get around the load that it imposes? No, it's a spamtrap, so we want all the spam we can get :-) I run only about 4 plugins, and have to work my ass off to get them to have as little affect on performance as possible. Matt.
Re: Qsmtpd::Address design question
On 7 Oct 2005, at 15:13, John Peacock wrote: What was it that bothered you? Should we convert that fat block of BNF into much more readable POD? We could also use extended regex's and document the various bits inline rather than having to go back up to figure out the BNF... new() vs parse() for a start (new if the address is , and parse() if it isn't !!!) parse() returning a list or undef. format() in general. We should use overload. user()/host() don't cache their values. I think that was it.
Re: deja vu ... axkit: URIs don't support query parameters
On 19 Sep 2005, at 02:31, S. Woodside wrote: So, is axkit: supposed to support query parameters / it would be nice! It's kind of on the todo list. The thing is axkit: URIs run in the current request's process space, and so we need to find a way of temporarily replacing the request's idea of query params and bring them back after we're done. It just needs a bit of JFDI I think :-)
Re: TaglibHelper -unwanted tags
On 3 Oct 2005, at 17:06, Chuck Phillips wrote: Am I oversimplying the modifications needed to add this functionallity? Is there something that I'm missing? No, that should work just fine. Not sure if we want to do it though - the whole plural thing scares me as it puts us in the realms of parsing english :-) I always figured if people want that level of control over the XML generated that they'd just use as_xml. Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Use of Auth.pm not working in 0.31-dev
On 4 Oct 2005, at 03:56, Rusty Conover wrote: I'm working with a SVN checkout of 0.31-dev from 2 days ago. Make sure you're not running trunk, as there are still unresolved issues there - use the branch instead.
Qpsmtpd Article
Tell all your friends: http://www.oreillynet.com/pub/a/sysadmin/2005/09/15/qpsmtpd.html
Re: a faster html::template?
On 8 Sep 2005, at 14:18, Perrin Harkins wrote: Not sure why you say certainly not. XSLT is very fast and easy to optimise (because it's assignment free) and the C implementations are very quick. Remember that an XSLT transform is executing all its tight loops in C-space, not in perl-space. Yes, but HTML::Template doesn't do any parsing when executing a template. XSLT always has to parse XML, and then take some pains to generate well-formed XML. I admit that it's possible that an XML parser could be written that is so fast that it actually beats the speed of accessing variables in perl, but it seems very unlikely. It depends what architecture you use. You have to compare like-with-like. XSP in AxKit is perl code that executes to build a DOM tree, not a XML string, so there's no parsing involved - it's just iterating over a data structure. And it's very fast. To give a counter example using XML strings though, I produce the AxKit mailing list archives from a static XML file containing around 10k records. If I time this: xsltproc --stringparam month 1 --stringparam year 2005 ../stylesheets/archive-pickmonth.xsl index.xml | xsltproc ../stylesheets/archive-thread.xsl - | xsltproc ../stylesheets/archive-gather-orphans.xsl - | xsltproc ../stylesheets/archive-sort.xsl - | xsltproc ../stylesheets/archive-display.xsl - That's 5 XSLT stylesheets, which means parsing XML five times, and spawning 5 processes to do this. It executes in 0.75s on my server. Now to achieve this you have to know how to use XSLT, including using things like keys() to get efficient indexing, but I think you can see that it's very quick indeed. Matt.
Re: a faster html::template?
On 9 Sep 2005, at 12:12, Perrin Harkins wrote: HTML::Template (or HTML::Template::JIT, to compare C code to C code) has a *very* limited set of logical operations (not Turing complete by any stretch) that operates directly on Perl variables. By comparison, XSP will be making method calls to use the DOM tree and supports a very rich programming language. It does much more, and that's why I expect it to be slower. Not slow -- just slower. Yes I know that's what you're saying. I'm just saying I'd be willing to bet it's the other way around (even when compared with HT::JIT).
Re: strange
On 7 Sep 2005, at 04:38, Pascal Dreissen wrote: John Peacock wrote: Pascal Dreissen wrote: Depending on your distro, there may be several different ways to change which character set is used. Choose one that doesn't include Unicode characters (for example, I use en_US). You can see what language setting is in force by typing: $ echo $LANG from any command line. [EMAIL PROTECTED] root]# echo $LANG en_US.UTF-8 Now you just need to change that in /etc/sysconfig/i18n (i.e. remove all the .UTF-8 parts) and reboot. Matt.
Re: Does TaglibHelper support element attributes?
On 25 Aug 2005, at 14:41, Chuck Phillips wrote: Does TaglibHelper have the ability to return XML elements that have attributes? No it doesn't. What I would like to do is create a taglib that returns XML much like this: someElement someAttribute=value blah blah /someElement Is there any way of doing this using TaglibHelper without resorting to enabling as_xml and forming the XML myself? Without using as_xml, you'll have to use an extra XSLT phase to convert a child tag into an attribute. Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: milter interface?
On 25 Aug 2005, at 16:10, Les Mikesell wrote: Oops - no... I spent a couple of minutes looking through the available addon plugins but hadn't downloaded the base package to see that it is built in. Moving on to the next question: has anyone tried it with MimeDefang? If you haven't seen MimeDefang, it basically splits mail attachments out and runs your choice of scanners over them with the actions controlled by a custom snippet of perl. It already has hooks for most of the popular virus scanners and spamassassin and knows how to remove attachments you might consider dangerous and replace them with a warning message. The milter plugin is relatively untested. Will you be our guinea pig?
Re: Preliminary design proposal for charset normalization support in SpamAssassin
On 23 Aug 2005, at 14:51, John Gardiner Myers wrote: Matt Sergeant wrote: Wasn't there unicode normalisation in the original email parser that I submitted to the project (that Theo turned into the current parser) ? Certainly it would make sense to use that if you could. It works very well on a very large set of test data. That code only deals with MIME-labeled charsets. It has no provision for charset detection. Really? I must have written that later in my local version of the code. I can probably provide some code for charset detection - it's fairly simple once you have the heuristics figured out. Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Style chooser working for firefox not for IE6
On 23 Aug 2005, at 06:22, David Holden wrote: It works fine for both if I remove the HTML version but I'm not sure why one browser/server combination would behave differently for this though, any idea? I'm confused as to how it works at all. What is mapping the non-suffix version to the .xml or .html files? Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Preliminary design proposal for charset normalization support in SpamAssassin
On 22 Aug 2005, at 21:38, Justin Mason wrote: looking at http://cvs.sourceforge.net/viewcvs.py/spamassassin/spamassassin3/lib/ Mail/SpamAssassin/Message/Parser.pm?rev=1.2view=markup it appears to be Text::Iconv. Yeah, later iterations of that use Encode. Does SA now require 5.8.0 so that you don't need the abstraction? __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Integrating seperate taglibs
On 16 Aug 2005, at 19:52, Christopher H. Laco wrote: Is there a way in the api to add namespaces on the fly? So loading: xmlns:checkout=http://today.icantfocus.com/CPAN/AxKit/XSP/Handel/ Checkout would automatically add xmlns:order=http://today.icantfocus.com/CPAN/AxKit/XSP/Handel/Order; behind the scenes? No. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: baffled by intermittant failure to correctly apply xslt
On 19 Aug 2005, at 07:08, Gilbert Laycock wrote: I'm hoping that somebody can suggest where I should start looking. Previously when I've had odd issues like this it is due to XSLT trying to load the w3c HTML DTD. Try using an XML Catalog containing that DTD. Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: PassiveTeX Questions
On 19 Aug 2005, at 13:22, Aaron Steager wrote: Is PassiveTeX the tool I should use or is the something else that is better suited? Most people prefer HTMLDoc because the input is HTML 3.2, which they can generate very easily. However you definitely have more control of the output via XSL-FO (and PassiveTeX). If it is does anyone have some simple xsp fo xsl files I could look at? I'm not asking for anyone to do my work but a few simple files I could look through would be greatly appreciated. Try google for xsl-fo examples. There are hundreds on the 'net. w3schools.org might be a good place to start. Matt. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 0.31 release candidate 2
On 22 Aug 2005, at 03:57, Peter J. Holzer wrote: Note this is why I implemented $Include. Maybe my cold is addling my brain, but I don't see how $Include can be used to load plugins from different locations. In my RPM, qpsmtpd and qpsmtpd-forkserver are in /usr/bin (they probably should be in /usr/sbin instead). By default, qpsmtpd would now look in /usr/bin/plugins for the plugins, but they are in /usr/share/qpsmtpd/plugins instead. So a config option is needed to tell qpsmtpd where they are. In addition, a sysadmin might want to install his own plugins into /usr/local/qpsmtpd/plugins - this is what Devin's patch allows. Ah I see. Yes ignore me. $Include is there so that a plugin RPM/DEB/Whatever can just drop a plugin in the plugins dir and a config file in a plugin.d directory and qpsmtpd will automatically pick it up. In short: $Include is there so that an RPM doesn't have to modify config/plugins. Matt.
Re: Preliminary design proposal for charset normalization support in SpamAssassin
Wasn't there unicode normalisation in the original email parser that I submitted to the project (that Theo turned into the current parser) ? Certainly it would make sense to use that if you could. It works very well on a very large set of test data. Matt. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: 0.31 release candidate 2
On 21 Aug 2005, at 05:22, Peter J. Holzer wrote: On 2005-08-20 20:59:10 -0700, Devin Carraway wrote: On Sat, Aug 20, 2005 at 07:38:24PM +0200, Peter J. Holzer wrote: * My patch for a configurable plugin_dir (needed for the RPMs directory structure). I ship a similar patch with Debian's qpsmtpd, for FHS compliance. It varies from Peter's original in allowing multiple values in plugin_dirs and checking for plugins in each -- this helps provide sysadmins with a safe spot to drop their own plugins or override the distributed ones without worrying about interfering with the package management. Ah, that's even better, of course. Note this is why I implemented $Include.
Re: Reverse SRS plugin?
On 18 Aug 2005, at 03:30, Gavin Carr wrote: On Thu, Aug 18, 2005 at 11:44:06AM +1000, Gavin Carr wrote: Just wondering if anyone has written a Reverse SRS plugin? I think this would just be a recipient plugin that rewrites SRS recipient addresses early during RCPT? (before rcpt checks, specifically) Replying to self - nope, that won't work. SRS rewriting has to be done _after_ rcpt_ok, of course, because our local domain is probably only on the un-rewritten recipient. But it really should still be at RCPT time because we need to be able to deny individual recipients where the rewrite fails. Maybe I'm not thinking straight, but I didn't see the need to do that when I wrote a plugin that did something like this. It's in the old CVS archive in the Attic if you're interested in looking (can't remember what I called it, but it's the only plugin we ever attic'd).
Re: Out of memory! ?
On 13 Aug 2005, at 09:40, Pascal Dreissen wrote: Robert Spier wrote: What does your run script look like? #!/bin/sh QMAILDUID=`id -u smtpd` NOFILESGID=`id -g smtpd` exec /usr/bin/softlimit -m 2500 \ /usr/bin/tcpserver -c 10 -v -R -p \ -u $QMAILDUID -g $NOFILESGID `head -1 config/IP` smtp \ ./qpsmtpd 21 And what version is this?
Towards 1.7
OK, so we've been dragging our feet for some time now, but I've finally gotten around to doing *something*, so I'm hoping I can encourage others to do the same... I've released XML::LibXSLT 1.58 to CPAN. This *should* fix the axkit: URI stuff. AxKit CVS now passes make test. So what do I need *you* to do? I need you all to check out the current CVS, and with XML::LibXSLT 1.58, make sure make test passes. If we can all do that, 1.7 can go out soon, we can migrate to SVN, and we can start looking more seriously at the TomKit code. Matt.
Re: bad name error in TagLibHelper
On 3 Aug 2005, at 16:03, ahg wrote: The site used to work fine. Any ideas? This rather cryptic error message bad name is caused when you try and create an XML element with invalid characters in it. I'm not sure why that might be happening here because you don't show anything else. What does descConjunto() do? Perhaps it returns a hashref and accidentally one of the keys is blank? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: UDP Protocol Handler?
On 20 Jul 2005, at 10:40, Dan Sully wrote: Has anyone written a protocol handler for UDP? If so - could you share any examples? I haven't, but IIRC there is/was a talk at YAPC::EU about using mod_perl as a DNS server... I think it is by mock, who did the mod_perl as a mail server in Belfast last year. I guess that's not much help, except to know it's possible. Matt.
Re: UDP Protocol Handler?
On 2 Aug 2005, at 13:27, Ken Simpson wrote: BTW Matt: Have you released the super-caching DNS query system that you were discussing a few months back? The one that works way better than Net::DNS for doing nasty things like lots of SPF checks? Sort of. There's code as part of Qpsmtpd that will show you how to do Net::DNS with a single socket. And SpamAssassin has integrated similar ideas. But I don't have anything generic. Matt.
Re: TLS+relay problem
On 20 Jul 2005, at 14:13, Marco Aurelio Monteiro wrote: I found a problem related to TLS and check_relay plugins. After the starttls command the server responds relaying denied. Check_relay plugin is configured properly. Check_relay plugin sets to 1 a variable of $connection called relay_client, however it is not saved in the tls plugin. There is a patch attached. Please take a very good look at this, because I am starting with qpsmtpd. The code was downloaded from svn and the revision is 519. Applied, thanks.
Re: Seeking *non* thread-friendly DBD
On 22 Jul 2005, at 12:30, Dean Arnold wrote: Hoping someone can point me at a DBD thats known to be non-thread-friendly (ie, hasn't implemented the clone() methods). However, its underlying client libs and XS subs (if any) do need to be thread-capable (ie, no writing to process-global variables wo/ locking). I'm trying to test out a theory that DBIx::Threaded will make non-thread-friendly DBD's not only friendly, but thread-safe. DBD::SQLite.
Re: qpsmtpd in Debian
On 28 Jul 2005, at 17:13, Devin Carraway wrote: For anyone who's into that sort of thing: http://packages.debian.org/unstable/mail/qpsmtpd This is 0.30, which mercifully held still for the packaging effort. The only plugins included are those in qpsmtpd svn, plus my own logging and exim-queue plugins specifically written for Debian (exim4 is Debian's default MTA). I may uploaded a qpsmtpd-plugins package with third-party plugins not in svn, but haven't started on that yet. Great news. Any chance you'll be donating your exim-queue plugin to the project?
Re: Seeking *non* thread-friendly DBD
On 22 Jul 2005, at 12:30, Dean Arnold wrote: Hoping someone can point me at a DBD thats known to be non-thread-friendly (ie, hasn't implemented the clone() methods). However, its underlying client libs and XS subs (if any) do need to be thread-capable (ie, no writing to process-global variables wo/ locking). I'm trying to test out a theory that DBIx::Threaded will make non-thread-friendly DBD's not only friendly, but thread-safe. DBD::SQLite.
Re: PerMsgStatus
[Lots of stuff snipped] You know, it'd be nice if Daniel, or anyone else, checked in my optimised PMS.pm [*] in as a branch. That way it can be worked on easily by multiple people. An optimisation branch would mean you can continue with the current release work, while others work on performance. Matt. [*] I changed a few parts about how rules get compiled while at CEAS that should have been faster, only benchmarks showed no visible sign of improvements. I'd be happy to send it to anyone interested. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Mailing List Detection
On 27 Jul 2005, at 02:52, Mark Fowler wrote:
So, as part of my mission to replace (most) of my hacked together
procmail
and pperl mail filtering solution with something that can reject mail
at
SMTP time, I realised I needed a list detection plugin. Which I
thought I
might share here:
use Mail::ListDetector;
sub hook_data_post {
my ($self, $transaction) = @_;
# can we work out what the list was called? if so, add a header
my $object = Mail::ListDetector-new($transaction);
if ($object)
{ $transaction-header-add(X-DetectedListName,
$object-listname) }
return DECLINED;
}
There's some code for doing list detection in SpamAssassin that I wrote
that is much improved over what's in Mail::ListDetector (or at least
was at the time - I haven't checked in years), and thus might be easier
to just borrow that code and re-write it as a plugin...
Doing all this interface stuff for Email::Abstract seems awfully
complex to me. I'm just a simple guy.
Matt.
Re: perl 5.005
On 19 Jul 2005, at 09:01, John Peacock wrote: [EMAIL PROTECTED] wrote: I'm all for dropping support of 5.5. Move up to 5.8. For a pure Perl module (I don't even think any of our dependencies require XS), if we can maintain compatibility without too much trouble, I think we should. There were things added after 5.5 such as easy lexical filehandles [ open(my $fh, $filename) ] that I just wouldn't want to give up now. What I don't want to do is have us all have to maintain an old 5.6 install just to test it there before every release. Matt.
Make capabilities more core?
Currently I think the only thing that uses the capabilities notes
field is the new tls plugin. My suggestion is to make this not a notes
field any more, but an integral part of the $transaction object.
The reason being that currently we have a horrible hoop jump with it
being in notes. Things like this:
my @capabilities = $self-transaction-notes('capabilities')
? @{ $self-transaction-notes('capabilities') }
: ();
And this:
my $cap = $transaction-notes('capabilities');
$cap ||= [];
push @$cap, 'STARTTLS';
$transaction-notes('capabilities', $cap);
Which would be much easier if we could just say:
$transaction-add_capability('STARTTLS');
And:
my @capabilities = $transaction-capabilities();
Any objections?
