check your HTTP request header, especially Cookie, Refer, User-Agent fields, can use wget or curl to test first.
在 2014年10月16日，下午6:27，chun ho chui <[email protected]> 写道： > I have been trying to crawl this website, built with ajax, but no luck. > > I knew it is ajax, so I tried to crawl it by sending an HTTP request ( > http://www.investing.com/economic-calendar/filter )and get the result back. > > According to the Firebug > Request URL:http://www.investing.com/economic-calendar/filter > Request Method:POST > Status Code:200 OK > > However, it doesn't work as expected. ( * the website is updating itself by > post request when filter changed, and also regularly update by the refresh > get request) > > Highly appreciate with your help. > > > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
