RE: MxSorter and changes to DNSServer.java
> > I just realized that like an idiot, I just committed the > > changes before testing them. :-( I'll be doing that > > immediately. > Sounds great, (except for the non-testing part ;-)) Well, at least the dogfood tastes good ... Changing to use org.xbill.DNS.Address instead of InetAddress for the lookups seems to be running fine so far. I haven't tried Fetchmail, though. > > I find it bothersome that Sun's DNS service for JNDI apparently > > ignores the TTL provided by the DNS server. > Agreed, [the] very least they could have done was to provide > a hook to circumvent it. OK, this was a false alarm, and wrong. Sun's documentation is just poorly worded, but I found a clarifying e-mail. http://java.sun.com/j2se/1.4.2/docs/guide/net/properties.html says that "networkaddress.cache.ttl [indicates] the caching policy for successful name lookups from the name service." That is misleading, especially since they later refer to sun.net.spi.nameservice.provider as specifying the name service provider, and giving JNDI DNS as an example. However, Jayalaxmi Hangal from Sun provided details in http://archives.java.sun.com/cgi-bin/wa?A2=ind0402&L=jndi-interest&F=&S=&P=4 67, and clarified that the caching is not used by the JNDI DNS provider. I don't know what Sun's provider does do in terms of caching. If we want to replace it with dnsjava, we should be able to do so. I thought I had remembered finding a name service stub for dnsjava on the net somewhere, but I cannot locate it anymore. However, I did find some stub code: http://java-house.jp/ml/archive/j-h-b/051361.html http://java-house.jp/ml/archive/j-h-b/051940.html It looks as if it would be fairly straightforward to plug in dnsjava, at least with JREs that support the sun.net.spi.nameservice.provider property. Perhaps Brian knows where there might be a ready-made adapter. I have e-mailed him to ask. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: MxSorter and changes to DNSServer.java
Yes, I did, and dnsjava-1.6.2.jar is in lib. But dnsjava-1.4.1.jar was still sitting there (shouldn't it have been deleted by the CVS update?), causing the problem. After removing it, the build went ok. Vincenzo > -Original Message- > From: Noel J. Bergman [mailto:[EMAIL PROTECTED] > Sent: martedì 13 aprile 2004 16.48 > To: James Developers List > Subject: RE: MxSorter and changes to DNSServer.java > > > > after your commit I'm getting the following build error: > > Did you update the whole project? There are an updated dnsjava JAR and > build.xml. > > --- Noel > > See: > > http://cvs.apache.org/viewcvs.cgi/james-server/lib/?only_with_tag=branch_2_1 > _fcs > > http://cvs.apache.org/viewcvs.cgi/james-server/build.xml > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: MxSorter and changes to DNSServer.java
> Also, was a JIRA issue created for this change? For the dnsjava change? Kind of bundled with another, but I've created an explicit one for it. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: MxSorter and changes to DNSServer.java
> after your commit I'm getting the following build error: Did you update the whole project? There are an updated dnsjava JAR and build.xml. --- Noel See: http://cvs.apache.org/viewcvs.cgi/james-server/lib/?only_with_tag=branch_2_1 _fcs http://cvs.apache.org/viewcvs.cgi/james-server/build.xml - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: MxSorter and changes to DNSServer.java
On Tuesday 13 April 2004 03:49, Noel J. Bergman wrote: > > I haven't looked at the org.xbill.DNS.Address but it > > sounds like a good solution. > > Agreed. I'll look at making that change. > > > I have put it on my todo-list to go through the code and > > weed-out InetAddress, just haven't gotten around to it yet. > > I just did: > > $ grep --recursive "InetAddress\.get.*ByName" src/java/ -l > src/java/org/apache/james/James.java > src/java/org/apache/james/core/AbstractJamesService.java > src/java/org/apache/james/dnsserver/DNSServer.java > src/java/org/apache/james/fetchmail/MessageProcessor.java > > src/java/org/apache/james/transport/mailets/RemoteDeliverySocketFactory.jav >a src/java/org/apache/james/transport/mailets/RemoteDelivery.java > src/java/org/apache/james/transport/matchers/InSpammerBlacklist.java > src/java/org/apache/james/util/NetMatcher.java > > James, AbstractJamesService, and RemoteDeliverySocketFactory were all local > stuff. > > I've updated DNSServer, MessageProcessor, RemoteDelivery, > InSpammerBlacklist, and NetMatcher it use org.xbill.DNS.Address for the two > static methods. I fully qualified the name to (a) make it easier to find, > and (b) because it was necessary in some cases to distinguish from > javax.mail.Address, anyway. Of course, I just realized that like an idiot, > I just committed the changes before testing them. :-( I'll be doing that > immediately. Sounds great, (except for the non-testing part ;-)) > > Another side-effect of the changes since yours and Richard's is that by > passing an IP address to JavaMail, we eliminate its lookup, which would > also have used InetAddress' caching. Yes, I was aware of that "sideeffect". > > I find it bothersome that Sun's DNS service for JNDI apparently ignores the > TTL provided by the DNS server. Agreed, the Sun is absolutely NOT geared towards a mail-server solution, and their reason for the "cache-forever" srategy (that it is made to prevent spoofing attacks) is IMO a bit farfetched, the very least they could have done was to provide a hook to circumvent it. But as it is we are not too pleased with their JavaMail implementation either ;-) --Søren > > --- Noel > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] -- Søren Hilmer, M.Sc. R&D manager Phone: +45 70 27 64 00 TietoEnator IT+ A/S Fax:+45 70 27 64 40 Ved Lunden 12 Direct: +45 87 46 64 57 DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: MxSorter and changes to DNSServer.java
Vincenzo Gianferrari Pini wrote: Noel, after your commit I'm getting the following build error: compile: Compiling James Java sources Compiling 228 source files to C:\cvsssh\james-branch_2_1_fcs\build\classes C:\cvsssh\james-branch_2_1_fcs\src\java\org\apache\james\dnsserver\DNSServer.java:257: lookupRecords(org.xbill.DNS.Name,int,byte) in org.xbill.DNS.Cache cannot be applied to (org.xbill.DNS.Name,int,int) SetResponse cached = cache.lookupRecords(name, type, dnsCredibility); ^ Note: Some input files use or override a deprecated API. Note: Recompile with -deprecation for details. 1 error BUILD FAILED file:C:/cvsssh/james-branch_2_1_fcs/build.xml:274: Compile failed; see the compiler error output for details. Vincenzo Also, was a JIRA issue created for this change? -- Serge Knystautas President Lokitech >>> software . strategy . design >> http://www.lokitech.com p. 301.656.5501 e. [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: MxSorter and changes to DNSServer.java
Noel, after your commit I'm getting the following build error: compile: Compiling James Java sources Compiling 228 source files to C:\cvsssh\james-branch_2_1_fcs\build\classes C:\cvsssh\james-branch_2_1_fcs\src\java\org\apache\james\dnsserver\DNSServer.java:257: lookupRecords(org.xbill.DNS.Name,int,byte) in org.xbill.DNS.Cache cannot be applied to (org.xbill.DNS.Name,int,int) SetResponse cached = cache.lookupRecords(name, type, dnsCredibility); ^ Note: Some input files use or override a deprecated API. Note: Recompile with -deprecation for details. 1 error BUILD FAILED file:C:/cvsssh/james-branch_2_1_fcs/build.xml:274: Compile failed; see the compiler error output for details. Vincenzo > -Original Message- > From: Noel J. Bergman [mailto:[EMAIL PROTECTED] > Sent: martedì 13 aprile 2004 3.50 > To: James Developers List > Subject: RE: MxSorter and changes to DNSServer.java > > > > I haven't looked at the org.xbill.DNS.Address but it > > sounds like a good solution. > > Agreed. I'll look at making that change. > > > I have put it on my todo-list to go through the code and > > weed-out InetAddress, just haven't gotten around to it yet. > > I just did: > > $ grep --recursive "InetAddress\.get.*ByName" src/java/ -l > src/java/org/apache/james/James.java > src/java/org/apache/james/core/AbstractJamesService.java > src/java/org/apache/james/dnsserver/DNSServer.java > src/java/org/apache/james/fetchmail/MessageProcessor.java > > src/java/org/apache/james/transport/mailets/RemoteDeliverySocketFactory.java > src/java/org/apache/james/transport/mailets/RemoteDelivery.java > src/java/org/apache/james/transport/matchers/InSpammerBlacklist.java > src/java/org/apache/james/util/NetMatcher.java > > James, AbstractJamesService, and RemoteDeliverySocketFactory were all local > stuff. > > I've updated DNSServer, MessageProcessor, RemoteDelivery, > InSpammerBlacklist, and NetMatcher it use org.xbill.DNS.Address for the two > static methods. I fully qualified the name to (a) make it easier to find, > and (b) because it was necessary in some cases to distinguish from > javax.mail.Address, anyway. Of course, I just realized that like an idiot, > I just committed the changes before testing them. :-( I'll be doing that > immediately. > > Another side-effect of the changes since yours and Richard's is that by > passing an IP address to JavaMail, we eliminate its lookup, which would also > have used InetAddress' caching. > > I find it bothersome that Sun's DNS service for JNDI apparently ignores the > TTL provided by the DNS server. > > --- Noel > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: MxSorter and changes to DNSServer.java
> I haven't looked at the org.xbill.DNS.Address but it > sounds like a good solution. Agreed. I'll look at making that change. > I have put it on my todo-list to go through the code and > weed-out InetAddress, just haven't gotten around to it yet. I just did: $ grep --recursive "InetAddress\.get.*ByName" src/java/ -l src/java/org/apache/james/James.java src/java/org/apache/james/core/AbstractJamesService.java src/java/org/apache/james/dnsserver/DNSServer.java src/java/org/apache/james/fetchmail/MessageProcessor.java src/java/org/apache/james/transport/mailets/RemoteDeliverySocketFactory.java src/java/org/apache/james/transport/mailets/RemoteDelivery.java src/java/org/apache/james/transport/matchers/InSpammerBlacklist.java src/java/org/apache/james/util/NetMatcher.java James, AbstractJamesService, and RemoteDeliverySocketFactory were all local stuff. I've updated DNSServer, MessageProcessor, RemoteDelivery, InSpammerBlacklist, and NetMatcher it use org.xbill.DNS.Address for the two static methods. I fully qualified the name to (a) make it easier to find, and (b) because it was necessary in some cases to distinguish from javax.mail.Address, anyway. Of course, I just realized that like an idiot, I just committed the changes before testing them. :-( I'll be doing that immediately. Another side-effect of the changes since yours and Richard's is that by passing an IP address to JavaMail, we eliminate its lookup, which would also have used InetAddress' caching. I find it bothersome that Sun's DNS service for JNDI apparently ignores the TTL provided by the DNS server. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: MxSorter and changes to DNSServer.java
On Friday 09 April 2004 17:12, Noel J. Bergman wrote:
> > My concern on this is that InetAddress caches successfull DNS
> > lookups forever (at least on default) and this strategy is not
> > very sound for a mailserver
>
> Good point, Søren. And that happens in InetAddress, but through
> contamination with sun.* classes. It isn't pluggable. The is a comment in
> the code that suggests that the author realizes it is a problem. Even
> replacing the default DNS provider with dnsjava (using
> sun.net.spi.nameservice.provider), would not help.
>
> However, since the use of InetAddress within DNSServer is opaque, we could
> trivially switch to using org.xbill.DNS.Address, which is a InetAddress
> clone that uses dnsjava. How does that sound?
>
I haven't looked at the org.xbill.DNS.Address but it sounds like a good
solution.
> I haven't checked the rest of the code, but InSpammerBlacklist also has
> this problem. That should probably be changed to use dnsjava, and perhaps
> JNDI in the future (portable, but more overhead). That would also allow us
> to get the TXT record, which some DNS RBLs use to provide useful
> information, e.g.,
>
> attrs = dnsContext.getAttributes(rblString, new String[] {"A", "TXT"});
>
> in JNDI-speak.
>
Yes, Noel I was aware that other places of the code might malfunction because
of this, and I have put it on my todo-list to go through the code and
weed-out InetAddress, just haven't gotten around to it yet.
-- Søren
--
Søren Hilmer, M.Sc.
R&D manager Phone: +45 70 27 64 00
TietoEnator IT+ A/S Fax:+45 70 27 64 40
Ved Lunden 12 Direct: +45 87 46 64 57
DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: MxSorter and changes to DNSServer.java
> My concern on this is that InetAddress caches successfull DNS
> lookups forever (at least on default) and this strategy is not
> very sound for a mailserver
Good point, Søren. And that happens in InetAddress, but through
contamination with sun.* classes. It isn't pluggable. The is a comment in
the code that suggests that the author realizes it is a problem. Even
replacing the default DNS provider with dnsjava (using
sun.net.spi.nameservice.provider), would not help.
However, since the use of InetAddress within DNSServer is opaque, we could
trivially switch to using org.xbill.DNS.Address, which is a InetAddress
clone that uses dnsjava. How does that sound?
I haven't checked the rest of the code, but InSpammerBlacklist also has this
problem. That should probably be changed to use dnsjava, and perhaps JNDI
in the future (portable, but more overhead). That would also allow us to
get the TXT record, which some DNS RBLs use to provide useful information,
e.g.,
attrs = dnsContext.getAttributes(rblString, new String[] {"A", "TXT"});
in JNDI-speak.
--- Noel
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: MxSorter and changes to DNSServer.java
Hi Noel, My concern on this is that InetAddress caches successfull DNS lookups forever (at least on default) and this strategy is not very sound for a mailserver, as I happened to discover :-( A possible (not optimal) solution is to set the security property networkaddress.cache.ttl to something less than the maximum retry-time for an undelivered mail, somewhere in James (RemoteDelivery ?). --Soren On Wednesday 07 April 2004 21:18, Noel J. Bergman wrote: > I have modified getSMTPHostAddresses to call findMXRecords and then use the > Iterator from that collection. We could go back to using MxSorter (the > comment I just put in about being able to us MxSorter within findMXRecords > is wrong, since we don't ever see a Collection over which to iterate; we > get an iterator), if we make some changes to it. > > The more important change in DNSServer was reverting to the previous > technique of using InetAddress.getAllByName to get the IP addresses for the > SMTP hosts, rather than the Type.A lookup. The code was failing to resolve > hosts that use CNAME on the right hand side of an MX record. That may be > an incorrect DNS configuration, but it is all too common as I have noticed > over the past couple weeks of testing the new code. > > Comments? Any strong feelings either way about using MxSorter to sort on > the fly versus pre-sorting, which is what findMXRecords does? If people > want to use MxSorter, the fixes should be fairly straightforward. The > drawback is having two sections of code doing largely the same thing, but > if we end up deprecating findMXRecords, that issue goes away. > > I am not sure if MxSorter is faster unless there would be a lot of records. > Even if you are using the first record, we have already got the records, > and sorting a small Collection in place is quick. MxSorter makes a linear > pass through the entire Collection, engages in object creation (at the > least, it has to call toString()) for each host matching the then > leastPriorityFound, and may create and delete entries from its working > collection depending upon the order in which they are present from the > lookup call. At least that's my take on the code. > > --- Noel > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] -- Søren Hilmer, M.Sc. R&D manager Phone: +45 70 27 64 00 TietoEnator IT+ A/S Fax:+45 70 27 64 40 Ved Lunden 12 Direct: +45 87 46 64 57 DK-8230 Åbyhøj Email: soren.hilmer tietoenator.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: MxSorter and changes to DNSServer.java
> I haven't looked at this code recently, but I guess this change > might end the reliance upon the org.xbill jar. Do you know > offhand if it does? No, it does not. All that the change really does is revert to the same use we made before the change to handle multi-homed hosts. We use the DNS to resolve the MX records, and then InetAddress to resolve the host addresses. One of the reasons for updating to the latest dnsjava was to clear up once and for all any licensing questions, although Serge had previously received proper permission from Brian to include dnsjava with James. > It is good that you caught this, Noel. Yes, well I was watching it fail on my monitor (I keep a watch on my mail logs in a corner of the screen), so I had some motivaton to find and fix it. ;-) > I don't know of a good reference which tells how mailservers > should use DNS. RFCs 974, 2821, etc., all provide good information. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: MxSorter and changes to DNSServer.java
Noel J. Bergman wrote: The more important change in DNSServer was reverting to the previous technique of using InetAddress.getAllByName to get the IP addresses for the SMTP hosts, rather than the Type.A lookup. I haven't looked at this code recently, but I guess this change might end the reliance upon the org.xbill jar. Do you know offhand if it does? > ... The code was failing to resolve hosts that use CNAME on the right hand side of an MX record. That may be an incorrect DNS configuration, but it is all too common as I have noticed over the past couple weeks of testing the new code. It is good that you caught this, Noel. When I was working with this functionality back in December, I felt unsure of the proper ways to use DNS since I don't know of a good reference which tells how mailservers should use DNS. Do you know of such a reference? or do we just push ahead learning from our experience? Rich Hammer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
MxSorter and changes to DNSServer.java
I have modified getSMTPHostAddresses to call findMXRecords and then use the Iterator from that collection. We could go back to using MxSorter (the comment I just put in about being able to us MxSorter within findMXRecords is wrong, since we don't ever see a Collection over which to iterate; we get an iterator), if we make some changes to it. The more important change in DNSServer was reverting to the previous technique of using InetAddress.getAllByName to get the IP addresses for the SMTP hosts, rather than the Type.A lookup. The code was failing to resolve hosts that use CNAME on the right hand side of an MX record. That may be an incorrect DNS configuration, but it is all too common as I have noticed over the past couple weeks of testing the new code. Comments? Any strong feelings either way about using MxSorter to sort on the fly versus pre-sorting, which is what findMXRecords does? If people want to use MxSorter, the fixes should be fairly straightforward. The drawback is having two sections of code doing largely the same thing, but if we end up deprecating findMXRecords, that issue goes away. I am not sure if MxSorter is faster unless there would be a lot of records. Even if you are using the first record, we have already got the records, and sorting a small Collection in place is quick. MxSorter makes a linear pass through the entire Collection, engages in object creation (at the least, it has to call toString()) for each host matching the then leastPriorityFound, and may create and delete entries from its working collection depending upon the order in which they are present from the lookup call. At least that's my take on the code. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
