Hi, I have a query regarding manifoldCF. Is this having some kind of functionality to check, if the URL it is crawling, does exist actually or page not found(404).
Like I have a requirement in which i am crawling data for university and job i continuously running.After some period it found that the certain URL's have been removed from University site but its is getting indexed still also. Some pages have been marked as status 404. How can manifold be automatise to check this , that if the URL is corresponding to 404(does not exist anymore), it should be indexed Thanks Priya.
