Hello, I'm trying to scrape an ASP.NET website. The steps are as follows:
1. Submit a form with stock code = '00001' at http://www.hkexnews.hk/listedco/listconews/advancedsearch/search_active_main.aspx. 2. Scrape data in the page that returns The callback function after_post does not get called so I'm not sure if I am setting up the FormRequest properly. Appreciate any help. --- class HKFilingSpider(BaseSpider): name = "hkex_filing" allowed_domains = ["http://www.hkexnews.hk/";] start_urls = ["http://www.hkexnews.hk/listedco/listconews/advancedsearch/search_active_main.aspx";] def parse(self, response): print "Submitting form..." sel = Selector(response) req = FormRequest.from_response(response, formname='aspnetForm', formdata={ '__VIEWSTATEENCRYPTED':'', 'ctl00$txt_today':'20140116', 'ctl00$hfStatus':'AEM', 'ctl00$hfAlert':'', 'ctl00$txt_stock_code':'00001', 'ctl00$txt_stock_name':'', 'ctl00$rdo_SelectDocType':'rbAll', 'ctl00$sel_tier_1':'-2', 'ctl00$sel_DocTypePrior2006':'-1', 'ctl00$sel_tier_2_group':'-2', 'ctl00$sel_tier_2':'-2', 'ctl00$ddlTierTwo':'23,1,3', 'ctl00$ddlTierTwoGroup':'10,2', 'ctl00$txtKeyWord':'', 'ctl00$rdo_SelectDateOfRelease':'rbManualRange', 'ctl00$sel_DateOfReleaseFrom_d':'01', 'ctl00$sel_DateOfReleaseFrom_m':'04', 'ctl00$sel_DateOfReleaseFrom_y':'1999', 'ctl00$sel_DateOfReleaseTo_d':'16', 'ctl00$sel_DateOfReleaseTo_m':'01', 'ctl00$sel_DateOfReleaseTo_y':'2014', 'ctl00$sel_defaultDateRange':'SevenDays', 'ctl00$rdo_SelectSortBy':'rbDateTime', }, callback=self.after_post) def after_post(self, response): # does not get called -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/groups/opt_out.
