Hi, You aren't giving me enough information to know why your job isn't rechecking URLs. Please tell me how your job is configured, specifically whether it's continuous or not. Thanks.
Karl On Mon, Sep 2, 2019 at 4:47 AM Priya Arora <pr...@smartshore.nl> wrote: > Hi, > > I have a query regarding manifoldCF. Is this having some kind of > functionality to check, if the URL it is crawling, does exist actually or > page not found(404). > > Like I have a requirement in which i am crawling data for university and > job i continuously running.After some period it found that the certain > URL's have been removed from University site but its is getting indexed > still also. > > Some pages have been marked as status 404. > How can manifold be automatise to check this , that if the URL is > corresponding to 404(does not exist anymore), it should be indexed > > Thanks > Priya. >
