sbrewin 2004/02/13 10:25:30 Modified: src/xdocs Tag: branch_2_1_fcs fetchmail_configuration_2_2.xml src/java/org/apache/james/fetchmail Tag: branch_2_1_fcs ParsedConfiguration.java ProcessorAbstract.java MessageProcessor.java src/conf Tag: branch_2_1_fcs james-fetchmail.xml Added: src/conf/samples/fetchmail Tag: branch_2_1_fcs remoteReceivedHeader.xml Log: Added filter for invalid Received header at the specified index.
See http://nagoya.apache.org/jira/secure/ViewIssue.jspa?key=JAMES-149. Revision Changes Path No revision No revision 1.1.2.5 +38 -7 james-server/src/xdocs/fetchmail_configuration_2_2.xml Index: fetchmail_configuration_2_2.xml =================================================================== RCS file: /home/cvs/james-server/src/xdocs/fetchmail_configuration_2_2.xml,v retrieving revision 1.1.2.4 retrieving revision 1.1.2.5 diff -u -r1.1.2.4 -r1.1.2.5 --- fetchmail_configuration_2_2.xml 9 Feb 2004 16:15:45 -0000 1.1.2.4 +++ fetchmail_configuration_2_2.xml 13 Feb 2004 18:25:30 -0000 1.1.2.5 @@ -131,7 +131,7 @@ <li><strong><a href="#maxmessagesize">maxmessagesize</a></strong> (0, 1)</li> <li><strong><a href="#recipientnotfound">recipientnotfound</a></strong> (1, 1)</li> <li><strong><a href="#recursesubfolders">recursesubfolders</a></strong> (1, 1)</li> -<li><strong><a href="#remoteReceivedHeaderIndex">remoteReceivedHeaderIndex</a></strong> (0, 1)</li> +<li><strong><a href="#remoteReceivedHeader">remoteReceivedHeader</a></strong> (0, 1)</li> <li><strong><a href="#remoterecipient">remoterecipient</a></strong> (1, 1)</li> <li><strong><a href="#undeliverable">undeliverable</a></strong> (1, 1)</li> <li><strong><a href="#userundefined">userundefined</a></strong> (1, 1)</li> @@ -469,10 +469,11 @@ </p> </subsection> -<subsection name="remoteReceivedHeaderIndex"> -<p>The <strong>remoteReceivedHeaderIndex</strong> tag declares the zero based +<subsection name="remoteReceivedHeader"> +<p>The <strong>remoteReceivedHeader</strong> tag declares the zero based index of the RFC2822 compliant RECEIVED header used to determine the address and -host name of the remote MTA that sent a fetched message.</p> +host name of the remote MTA that sent a fetched message and what happens to +messages when the specified header is invalid.</p> <p>Typically, the first (index = 0) RECEIVED header is for the local MTA that delivered the message to the message store and the second (index = 1) RECEIVED @@ -490,17 +491,47 @@ <p>Matchers such as InSpammerBlacklist use the remote address and/or remote host name to identify illegitimate remote MTAs. If you do not use such matchers, the <strong>remoteReceivedHeaderIndex</strong> tag may be omitted or the default -value of -1 can be specified. This causes the remote address to be set to +index value of -1 can be specified. This causes the remote address to be set to <code>127.0.0.1</code> and the remote host name to be set to <code>localhost</code>. Matchers almost always considered these values to be legitimate.</p> -<p>The tag value is an integer whose meaning is described above.</p> +<p>The tag has these attributes: +<dl> +<dt><strong>index</strong></dt> +<dd>An integer whose meaning is described above. +</dd> +<dt><strong>reject</strong></dt> +<dd>A boolean. If "true", mail whose specified recieved header is invalid +will not be injected into the James input spool. If "false", mail whose +specified recieved header is invalid will be injected into the James input spool with +the Mail Attribute <code>org.apache.james.fetchmail.isInvalidReceivedHeader</code> +added to the mail. +</dd> +<dt><strong>leaveonserver</strong></dt> +<dd>A boolean. If "true", mail whose specified recieved header is invalid +will be left on the server. If "false", mail whose specified recieved header +is invalid will be marked for deletion.</dd> +<dt><strong>markseen</strong></dt> +<dd>A boolean. If "true", mail whose specified recieved header is invalid +will be marked as seen on the server. If "false", mail whose specified +recieved header is invalid will not be marked as seen.</dd> +</dl> +</p> <p> <source> -<remoteReceivedHeaderIndex>1</remoteReceivedHeaderIndex> +<remoteReceivedHeader + index="1" + reject="true" + leaveonserver="true" + markseen="true"/> </source> +</p> + +<p>An example configuration using James mailet processing to notify the postmaster +of fetched messages that contain an invalid Received header can be found in the file +<code>$PHOENIX_HOME/apps/james/conf/samples/fetchmail/remoteReceivedHeader.xml</code>. </p> </subsection> No revision No revision 1.1.2.7 +93 -4 james-server/src/java/org/apache/james/fetchmail/ParsedConfiguration.java Index: ParsedConfiguration.java =================================================================== RCS file: /home/cvs/james-server/src/java/org/apache/james/fetchmail/ParsedConfiguration.java,v retrieving revision 1.1.2.6 retrieving revision 1.1.2.7 diff -u -r1.1.2.6 -r1.1.2.7 --- ParsedConfiguration.java 11 Feb 2004 17:17:07 -0000 1.1.2.6 +++ ParsedConfiguration.java 13 Feb 2004 18:25:30 -0000 1.1.2.7 @@ -218,7 +218,25 @@ * The index of the received header to use to compute the remote address * and remote host name for a message. This is 0 based and defaults to -1. */ - private int fieldRemoteReceivedHeaderIndex; + private int fieldRemoteReceivedHeaderIndex = -1; + + /** + * Keep messages with an invalid received header on the remote mailserver. + * Normally, messages are kept in the folder on the mailserver if they have been + * rejected + */ + private boolean fieldLeaveRemoteReceivedHeaderInvalid = true; + + /** + * Mark messages with an invalid received header on the remote mailserver as + * seen. Normally, messages are not marked as seen if they have been rejected. + */ + private boolean fieldMarkRemoteReceivedHeaderInvalidSeen = false; + + /** + * Reject messages with an invalid received header. + */ + private boolean fieldRejectRemoteReceivedHeaderInvalid; /** * Reject messages for which a recipient could not be determined. @@ -367,8 +385,18 @@ setMarkUndeliverableSeen( undeliverable.getAttributeAsBoolean("markseen")); - setRemoteReceivedHeaderIndex( - conf.getChild("remoteReceivedHeaderIndex").getValueAsInteger(-1)); + Configuration remotereceivedheader = conf.getChild("remotereceivedheader", false); + if (null != remotereceivedheader) + { + setRemoteReceivedHeaderIndex( + remotereceivedheader.getAttributeAsInteger("index")); + setRejectRemoteReceivedHeaderInvalid( + remotereceivedheader.getAttributeAsBoolean("reject")); + setLeaveRemoteReceivedHeaderInvalid( + remotereceivedheader.getAttributeAsBoolean("leaveonserver")); + setMarkRemoteReceivedHeaderInvalidSeen( + remotereceivedheader.getAttributeAsBoolean("markseen")); + } Configuration maxmessagesize = conf.getChild("maxmessagesize", false); if (null != maxmessagesize) @@ -456,7 +484,11 @@ && isLeaveUserUndefined() && !isMarkUserUndefinedSeen() && isLeaveUndeliverable() - && !isMarkUndeliverableSeen() + && !isMarkUndeliverableSeen() + && isLeaveMaxMessageSizeExceeded() + && !isMarkMaxMessageSizeExceededSeen() + && isLeaveRemoteReceivedHeaderInvalid() + && !isMarkRemoteReceivedHeaderInvalidSeen() ; } @@ -1066,6 +1098,63 @@ protected void setRejectMaxMessageSizeExceeded(boolean rejectMaxMessageSize) { fieldRejectMaxMessageSizeExceeded = rejectMaxMessageSize; + } + + /** + * Returns the leaveRemoteReceivedHeaderInvalid. + * @return boolean + */ + public boolean isLeaveRemoteReceivedHeaderInvalid() + { + return fieldLeaveRemoteReceivedHeaderInvalid; + } + + /** + * Returns the markRemoteReceivedHeaderInvalidSeen. + * @return boolean + */ + public boolean isMarkRemoteReceivedHeaderInvalidSeen() + { + return fieldMarkRemoteReceivedHeaderInvalidSeen; + } + + /** + * Returns the rejectRemoteReceivedHeaderInvalid. + * @return boolean + */ + public boolean isRejectRemoteReceivedHeaderInvalid() + { + return fieldRejectRemoteReceivedHeaderInvalid; + } + + /** + * Sets the leaveRemoteReceivedHeaderInvalid. + * @param leaveRemoteReceivedHeaderInvalid The leaveRemoteReceivedHeaderInvalid to set + */ + protected void setLeaveRemoteReceivedHeaderInvalid(boolean leaveRemoteReceivedHeaderInvalid) + { + fieldLeaveRemoteReceivedHeaderInvalid = + leaveRemoteReceivedHeaderInvalid; + } + + /** + * Sets the markRemoteReceivedHeaderInvalidSeen. + * @param markRemoteReceivedHeaderInvalidSeen The markRemoteReceivedHeaderInvalidSeen to set + */ + protected void setMarkRemoteReceivedHeaderInvalidSeen(boolean markRemoteReceivedHeaderInvalidSeen) + { + fieldMarkRemoteReceivedHeaderInvalidSeen = + markRemoteReceivedHeaderInvalidSeen; + } + + /** + * Sets the rejectRemoteReceivedHeaderInvalid. + * @param rejectRemoteReceivedHeaderInvalid The rejectRemoteReceivedHeaderInvalid to set + */ + protected void setRejectRemoteReceivedHeaderInvalid(boolean rejectRemoteReceivedHeaderInvalid) + { + fieldRejectRemoteReceivedHeaderInvalid = + rejectRemoteReceivedHeaderInvalid; } } 1.1.2.6 +27 -0 james-server/src/java/org/apache/james/fetchmail/ProcessorAbstract.java Index: ProcessorAbstract.java =================================================================== RCS file: /home/cvs/james-server/src/java/org/apache/james/fetchmail/ProcessorAbstract.java,v retrieving revision 1.1.2.5 retrieving revision 1.1.2.6 diff -u -r1.1.2.5 -r1.1.2.6 --- ProcessorAbstract.java 11 Feb 2004 17:17:07 -0000 1.1.2.5 +++ ProcessorAbstract.java 13 Feb 2004 18:25:30 -0000 1.1.2.6 @@ -288,6 +288,15 @@ } /** + * Returns the leaveRemoteReceivedHeaderInvalid. + * @return boolean + */ + protected boolean isLeaveRemoteReceivedHeaderInvalid() + { + return getConfiguration().isLeaveRemoteReceivedHeaderInvalid(); + } + + /** * Returns the LeaveMaxMessageSizeExceeded. * @return boolean */ @@ -315,6 +324,15 @@ } /** + * Returns the RejectRemoteReceivedHeaderInvalid. + * @return boolean + */ + protected boolean isRejectRemoteReceivedHeaderInvalid() + { + return getConfiguration().isRejectRemoteReceivedHeaderInvalid(); + } + + /** * Returns the RejectMaxMessageSizeExceeded. * @return boolean */ @@ -394,6 +412,15 @@ { return getConfiguration().isMarkUserUndefinedSeen(); } + + /** + * Returns the markRemoteReceivedHeaderInvalidSeen. + * @return boolean + */ + protected boolean isMarkRemoteReceivedHeaderInvalidSeen() + { + return getConfiguration().isMarkRemoteReceivedHeaderInvalidSeen(); + } /** * Returns the MarkMaxMessageSizeExceededSeen. 1.1.2.6 +120 -16 james-server/src/java/org/apache/james/fetchmail/MessageProcessor.java Index: MessageProcessor.java =================================================================== RCS file: /home/cvs/james-server/src/java/org/apache/james/fetchmail/MessageProcessor.java,v retrieving revision 1.1.2.5 retrieving revision 1.1.2.6 diff -u -r1.1.2.5 -r1.1.2.6 --- MessageProcessor.java 11 Feb 2004 17:17:07 -0000 1.1.2.5 +++ MessageProcessor.java 13 Feb 2004 18:25:30 -0000 1.1.2.6 @@ -57,19 +57,22 @@ * <p>Messages written to the input spool have the following Mail Attributes * set if the corresponding condition is satisfied: * <dl> - * <dt>org.apache.james.fetchmail.isRemoteRecipient</dt> - * <dd>The recipient is on a remote host</dd> - * <dt>org.apache.james.fetchmail.isUserUndefined</dt> - * <dd>The recipient is on a localhost but not defined to James</dd> * <dt>org.apache.james.fetchmail.isBlacklistedRecipient</dt> * <dd>The recipient is in the configured blacklist</dd> - * <dt>org.apache.james.fetchmail.isRecipientNotFound</dt> - * <dd>The recipient could not be found. Delivery is to the configured recipient. - * See the discussion of delivery to a sole intended recipient below.</dd> * <dt>org.apache.james.fetchmail.isMaxMessageSizeExceeded (java.lang.String)</dt> * <dd>The message size exceeds the configured limit. An empty message is * written to the input spool. The Mail Attribute value is a String * representing the size of the original message in bytes.</dd> + * <dt>org.apache.james.fetchmail.isRecipientNotFound</dt> + * <dd>The recipient could not be found. Delivery is to the configured recipient. + * See the discussion of delivery to a sole intended recipient below.</dd> + * <dt>org.apache.james.fetchmail.isRemoteRecievedHeaderInvalid</dt> + * <dd>The Receieved header at the index specified by parameter + * <code>remoteReceivedHeaderIndex</code> is invalid.</dd> + * <dt>org.apache.james.fetchmail.isRemoteRecipient</dt> + * <dd>The recipient is on a remote host</dd> + * <dt>org.apache.james.fetchmail.isUserUndefined</dt> + * <dd>The recipient is on a localhost but not defined to James</dd> * </dl> * * <p>Configuration settings - @@ -89,6 +92,8 @@ * <dd>See the discussion of delivery to a sole intended recipient below</dd> * <dt>RejectMaxMessageSizeExceeded</dt> * <dd>Rejects messages whose size exceeds the configured limit</dd> + * <dt>RejectRemoteReceievedHeaderInvalid</dt> + * <dd>Rejects messages whose Received header is invalid.</dd> * </dl> * * <p>Rejection processing is intentionally limited to managing the status of the @@ -168,6 +173,11 @@ private boolean fieldRemoteRecipient = true; /** + * The mail's Received header at index remoteReceivedHeaderIndex is invalid. + */ + private Boolean fieldRemoteReceivedHeaderInvalid; + + /** * Recipient is not a local user */ private boolean fieldUserUndefined = false; @@ -325,6 +335,13 @@ rejectMaxMessageSizeExceeded(getMessageIn().getSize()); return; } + + if (isRejectRemoteReceivedHeaderInvalid() + && isRemoteReceivedHeaderInvalid().booleanValue()) + { + rejectRemoteReceivedHeaderInvalid(); + return; + } // Create the mail // If any of the mail addresses are malformed, we will get a @@ -484,7 +501,29 @@ logStatusInfo(messageBuffer.toString()); return; - } + } + + /** + * Method rejectRemoteReceivedHeaderInvalid. + * @throws MessagingException + */ + protected void rejectRemoteReceivedHeaderInvalid() + throws MessagingException + { + // Update the flags of the received message + if (!isLeaveRemoteReceivedHeaderInvalid()) + setMessageDeleted(); + + if (isMarkRemoteReceivedHeaderInvalidSeen()) + setMessageSeen(); + + StringBuffer messageBuffer = + new StringBuffer("Rejected mail with an invalid Received: header at index "); + messageBuffer.append(getRemoteReceivedHeaderIndex()); + messageBuffer.append("."); + logStatusInfo(messageBuffer.toString()); + return; + } /** * <p>Method createMessage answers a new <code>MimeMessage</code> from the @@ -627,16 +666,16 @@ */ protected String computeRemoteDomain() throws MessagingException { - String[] headers = getMessageIn().getHeader(RFC2822Headers.RECEIVED); - StringBuffer domainBuffer = new StringBuffer(); - - // It isn't documented, but getHeader() will answer null if there - // are no matching headers, so we need to check! + StringBuffer domainBuffer = new StringBuffer(); + String[] headers = null; + if (getRemoteReceivedHeaderIndex() > -1) + getMessageIn().getHeader(RFC2822Headers.RECEIVED); + if (null != headers) { // If there are RECEIVED headers and the index to begin at is greater // than -1, try and extract the domain - if (headers.length > 0 && getRemoteReceivedHeaderIndex() > -1) + if (headers.length > 0) { final String headerTokens = " \n\r"; @@ -731,10 +770,10 @@ // Update the flags of the received message if (!isLeaveUndeliverable()) setMessageDeleted(); - + if (isMarkUndeliverableSeen()) setMessageSeen(); - + logStatusWarn("Message could not be delivered due to an error determining the remote domain."); if (getLogger().isDebugEnabled()) { @@ -1188,6 +1227,11 @@ aMail.setAttribute( getAttributePrefix() + "isMaxMessageSizeExceeded", new Integer(getMessageIn().getSize()).toString()); + + if (isRemoteReceivedHeaderInvalid().booleanValue()) + aMail.setAttribute( + getAttributePrefix() + "isRemoteReceivedHeaderInvalid", + null); } /** @@ -1472,5 +1516,65 @@ { fieldMaxMessageSizeExceeded = maxMessageSizeExceeded; } + + /** + * Returns the remoteReceivedHeaderInvalid, lazily initialised. + * @return Boolean + */ + protected Boolean isRemoteReceivedHeaderInvalid() throws MessagingException + { + Boolean isInvalid = null; + if (null == (isInvalid = isRemoteReceivedHeaderInvalidBasic())) + { + updateRemoteReceivedHeaderInvalid(); + return isRemoteReceivedHeaderInvalid(); + } + return isInvalid; + } + + /** + * Computes the remoteReceivedHeaderInvalid. + * @return Boolean + */ + protected Boolean computeRemoteReceivedHeaderInvalid() + throws MessagingException + { + Boolean isInvalid = Boolean.FALSE; + try + { + getRemoteAddress(); + } + catch (UnknownHostException e) + { + isInvalid = Boolean.TRUE; + } + return isInvalid; + } + + /** + * Returns the remoteReceivedHeaderInvalid. + * @return Boolean + */ + private Boolean isRemoteReceivedHeaderInvalidBasic() + { + return fieldRemoteReceivedHeaderInvalid; + } + + /** + * Sets the remoteReceivedHeaderInvalid. + * @param remoteReceivedHeaderInvalid The remoteReceivedHeaderInvalid to set + */ + protected void setRemoteReceivedHeaderInvalid(Boolean remoteReceivedHeaderInvalid) + { + fieldRemoteReceivedHeaderInvalid = remoteReceivedHeaderInvalid; + } + + /** + * Updates the remoteReceivedHeaderInvalid. + */ + protected void updateRemoteReceivedHeaderInvalid() throws MessagingException + { + setRemoteReceivedHeaderInvalid(computeRemoteReceivedHeaderInvalid()); + } } No revision No revision 1.1.2.7 +29 -14 james-server/src/conf/james-fetchmail.xml Index: james-fetchmail.xml =================================================================== RCS file: /home/cvs/james-server/src/conf/james-fetchmail.xml,v retrieving revision 1.1.2.6 retrieving revision 1.1.2.7 diff -u -r1.1.2.6 -r1.1.2.7 --- james-fetchmail.xml 9 Feb 2004 16:15:45 -0000 1.1.2.6 +++ james-fetchmail.xml 13 Feb 2004 18:25:30 -0000 1.1.2.7 @@ -68,19 +68,6 @@ <!-- How frequently this host is checked - in milliseconds. 600000 is every ten minutes --> <interval>600000</interval> - <!-- The zero based index of the RECEIVED Header used to compute the remote address and remote host name --> - <!-- of the MTA that delivered a fetched message. --> - <!-- Typically, the first (index = 0) RECEIVED Header is for the local MTA that delivers mail to the message store --> - <!-- and the second RECEIVED Header (index = 1) is the remote domain delivering mail to the MTA, so the second --> - <!-- RECEIVED Header is the one to use. --> - <!-- Some configurations will differ. Look at the RECEIVED Headers and set the index to point to the first --> - <!-- remote MTA. --> - <!-- Matchers such as InSpammerBlacklist use the remote address and/or remote host name to identify illegitimate --> - <!-- remote MTAs. --> - <!-- This tag is optional. If omitted, it will default to an index of -1, which is interpreted as use 127.0.0.1 for the --> - <!-- remote address and use 'localhost' for the remote host name. Both are almost always considered legitimate. --> - <remoteReceivedHeaderIndex>1</remoteReceivedHeaderIndex> - <!-- name of the javamail provider you wish to use --> <!-- (pop3, imap, etc. --> <javaMailProviderName>pop3</javaMailProviderName> @@ -131,6 +118,34 @@ <!-- if false, messages left on the server will not be marked as seen --> <fetched leaveonserver="false" markseen="true"/> + <!-- Specify the index of the RECEIVED Header used to compute the remote address and remote host name --> + <!-- and what happens to messages that contain an invalid header at that index. --> + <!-- Matchers such as InSpammerBlacklist use the remote address and/or remote host name to identify illegitimate --> + <!-- remote MTAs. --> + <!-- This tag is optional. If omitted, the remote address will be set to 127.0.0.1 and the remote domain --> + <!-- will be set to 'localhost'. Both are almost always considered legitimate. --> + <!-- index --> + <!-- The zero based index of the RECEIVED Header used to compute the remote address and remote host name --> + <!-- of the MTA that delivered a fetched message. --> + <!-- Typically, the first (index = 0) RECEIVED Header is for the local MTA that delivers mail to the message store --> + <!-- and the second RECEIVED Header (index = 1) is the remote domain delivering mail to the MTA, so the second --> + <!-- RECEIVED Header is the one to use. --> + <!-- Some configurations will differ. Look at the RECEIVED Headers and set the index to point to the first remote MTA. --> + <!-- An index of -1 is is interpreted as use 127.0.0.1 for the remote address and use 'localhost' for the remote host name. --> + <!-- reject --> + <!-- if true, messages whose received header at the specified index is invalid will be rejected --> + <!-- if false, messages whose received header at the specified index is invalid will be accepted --> + <!-- and the Mail Attribute "org.apache.james.fetchmail.isInvalidReceivedHeader" --> + <!-- will be added to the message. Use the HasMailAttribute matcher to detect them. --> + <!-- The following apply if reject="true" and a message is rejected... --> + <!-- leaveonserver --> + <!-- if true, messages will be left on the server --> + <!-- if false, messages will be deleted from the server --> + <!-- markseen --> + <!-- if true, messages left on the server will be marked as seen --> + <!-- if false, messages left on the server will not be marked as seen --> + <remotereceivedheader index="1" reject="true" leaveonserver="true" markseen="false"/> + <!-- Specify what happens to messages whose size exceeds the specified limit. --> <!-- This tag is optional. If omitted, there is no limit. --> <!-- limit --> @@ -149,7 +164,7 @@ <!-- markseen --> <!-- if true, messages left on the server will be marked as seen --> <!-- if false, messages left on the server will not be marked as seen --> - <maxmessagesize limit=0 reject="false" leaveonserver="true" markseen="false"/> + <maxmessagesize limit="0" reject="false" leaveonserver="true" markseen="false"/> <!-- Specify what happens to undeliverable messages --> <!-- leaveonserver --> No revision No revision 1.1.2.1 +101 -0 james-server/src/conf/samples/fetchmail/Attic/remoteReceivedHeader.xml --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]