Re: WebSite Crawling with Customheader Info

2018-12-05 Thread Karl Wright
Hi Jasvinder, That sounds like a customization you would have to make. The Web Connector is designed for generic web crawling, not as the basis for a custom connector. Indeed, I would strongly suggest that you not try to use the web connector to retrieve your repository content but rather develo

WebSite Crawling with Customheader Info

2018-12-05 Thread jasvinder . singh
Can you please suggest is there a way to pass custom header info with Seed URL so that the target application can determine this request is coming for crawling i.e if my site is xxx.yy.com so when the request hits from ManifoldCF for crawling can i pass some header which I can parse in my xxx.y