Hello Curl Community, i'm building a webcrawler, now i need to tune, my crawler, the crawler was build in php, but i have some problems.
Is this in Cpp possible: e.g. I make a request to http://domain.com/4/dir/somepdf.pdf When my crawler makes a request to that url, and the header returns, a for example application/pdf , than I want, that curl does not download the file (in one request). (I just want only text/html to download). In php i could do this in two requests like just receive the header, check, if ok than make a new request. I want todo this in* one request* check, stop or go. The same applies to 404, 403, 503 HTTP errors i dont want fetch the body if the header returns a these errors. Real Multi-threading, on PHP you dont have a real multi-threading, curl is so fast as your slowest request. Is this in Cpp possible to make a real multi threaded application? As you can see im coming from PHP, does the cpp client has some other feat, which the php not has? which are worth to mention? Thanks for help -- You received this message because you are subscribed to the Google Groups "curlpp" group. To view this discussion on the web visit https://groups.google.com/d/msg/curlpp/-/KNI7yTu0qbcJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/curlpp?hl=en.
