Hi Greg,
When a Web Crawler retrieves the page, it is mostly relying on response
from web server - parses html returned, and extracts stuff out. It is very
rare that these tools will actually try to run javascript or any dynamic
content that may be present in the returned html to be able to see that
page as it would appear to someone using browser.
With developer tools, you can inspect the DOM as it is at this very moment,
when sitting at the browser. I know that some versions of IE developer
tools were displaying out-of-sync DOM until little refresh icon was
clicked, not sure how it is these days as I haven't been using IE's tooling
for a while now.
Cheers,
jano
On 20 September 2015 at 11:35, Greg Keogh wrote:
> Folks, back in the mid 1990s as a challenge I wrote a web page with some
> raw JavaScript in it to generate a large table of random but vaguely
> realistic looking email addresses (using letter distribution frequencies).
> It was folklore back then that pages like this would "poison" spam address
> harvesters.
>
> Almost 20 years later I'm still getting daily 404 requests for that
> ancient page, so I found a copy, modernised it a bit and it still works.
> BUT ... in IE's View Source you do not see any of the html emitted by the
> script. Is by design? Does it mean that the spam harvesters would never
> drink their poison?
>
> Technically, I'm surprised that the browser is ignorant of script changes
> to the DOM. Can anyone explain the behaviour?
>
> *Greg*
>
