Hello Ramzay Ak At load time, HttpProxyMiddleware checks whether it can find proxies using six.moves.urllib.request.getproxies(), (in Python 2, this is https://docs.python.org/2/library/urllib.html#urllib.getproxies)
If it cannot find proxies this way, it will not be configured for the crawl. See https://github.com/scrapy/scrapy/blob/129421c7e31b89b9b0f9c5f7d8ae59e47df36091/scrapy/downloadermiddlewares/httpproxy.py#L23 HttpProxyMiddleware does not currently understand HTTP_PROXY setting. You can call your spider with this to set an env var for the proxy $ http_proxy="http://127.0.0.1:8080"; scrapy crawl myspider Hope this helps. /Paul On Monday, December 5, 2016 at 12:23:16 PM UTC+1, Ramzay Ak wrote: > > Hello. I want that each request goes throw local proxy (127.0.0.1:8080). > But when i start spider, i did not see HttpProxyMiddleware in list of > enabled middleware: > > 2016-12-04 10:27:12 [scrapy] INFO: Enabled downloader middlewares: >> ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', >> 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', >> 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', >> 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', >> 'scrapy.downloadermiddlewares.retry.RetryMiddleware', >> 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', >> 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', >> 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', >> 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', >> 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware', >> 'scrapy.downloadermiddlewares.stats.DownloaderStats'] > > > in settings.py i add strings: > > >> HTTP_PROXY = "http://127.0.0.1:8080"; >> >> DOWNLOADER_MIDDLEWARES = { >> 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 100, >> } >> >> > Please tell me what should I do to make it work > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
