Tom,
> did you guys break the sslbot again? (doesn't look like it. :) I had to
Nope, it still works, however OpenSRS is having some problems with the RWI
timing out lately (probably because of all the people running sslbot!). I
have it set to automatically run every night and the connection times out
more than it is successful.
By the way, if anyone is interested, I can look into posting my hack of
sslbot which writes the retrieved data to a mysql table.
> write another one for a different supplier and came up with a much more
> robust page parsing algorithm... :-) [split out the tables, then split out
> the rows... then parse the rows] Almost looking forward to rewriting that
> sslbot script :-)
and the ETA for this is what, yesterday? ;)
> Considering that forcing clients to write robot scripts to pound your
> servers into the dirt is a just generally a bad idea, and a simple text
> output screen to dump everything would be _trivial_ to write... you guys
> don't make this easy... (I know we've discussed this before, and I don't
> hold you responsible Chuck.... :)
Exactly. As I mentioned above, I wonder if this is part of the cause for
the slowdown seen in the RWI. After all it shouldn't be that slow at 5am
EST when I run my script.
> That said, I have no real desire to grab all the customer fields, and I
> would probably do that via a whois lookup instead anyway... much easier to
> parse text output than HTML, and it should be faster than using an SSL
> connection.
This is what I would suggest as well. Just get the list of active domains
and then use WHOIS to get the current contact information.
-Bill
