Hi , I am trying to crawl this url http://www.amazon.com/Levis-Mens-550-Relaxed-Jean/dp/B0018OKX68 and getting the crawled page content in my ParseFIlter plugin like this String html = new String(webPage.getContent().array()); Then I am using this html to extract my required information....
But its not returning me complete html of page. I have logged the 'html' and I can see that log file contains incomplete html for the above link.... Is there any size limit of page' content ? Or I am doing something wrong here ? Thanks, Tony.
