On Fri, Jul 18, 2014 at 1:05 PM, Santosh Kumar <rhce....@gmail.com> wrote: > its a curl command caliing a http function. > > for example: > > curl <username>:<password> <xml> GET http://<somelink>/app/somefunction
Hi Santosh, Ok, good. This is some of the information that we wanted to know. So you are performing HTTP requests to get content, and you also need to be able to provide username and password for authentication. Sounds good so far. > so i have to use this to create a wrapper which can scale better. This actually doesn't follow from the requirement. Let me explain. You're fixated on using the "curl" shell command to do this task. But that's not the only way you can get the content of a URL from Python. For example, Python's standard library includes a URL-downloading library in urllib (https://docs.python.org/2/library/urllib.html) which might work just as well as curl. You're saying that you need to scale better. In that case, you must identify what exactly is causing that slowness. To that end, please respond to Martin Brown's questions in this thread. You should try to address them, because I share those questions too. If you're just talking about a single HTTP request, for example, then I don't see an obvious solution to speed up a single download besides throwing more hardware and networking equipment at the problem. But if we're talking about many HTTP requests at once, where the current solution is getting each URL's content in a serial manner, then perhaps some kind of concurrent solution will be appropriate. (e.g. http://www.ibm.com/developerworks/aix/library/au-threadingpython/) But such a concurrent approach is only effective if you aren't already saturating your network, and you're connecting to slow servers, and... etc. Those are the details that matter, the ones you need to talk about and describe. "Slowness" isn't sufficiently descriptive to let us give you technical advice. Please say more about the problem by answering Martin's questions. Good luck! _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
