On Dec 17, 1:18 am, Kottiyath <n.kottiy...@gmail.com> wrote: > Hi all, > I have to connect to a secure website every second to get the data > and then post to it. I have been investigating on many web clients in > python, but nothing fits the bill properly. > The ones I tried implementing are: > 1. httplib based - I created myself. (I cannot use urllib2 since I > have to transfer files, and urllib2 doesnt have multipart content-type > support) > 2. Twisted web client. > I also looked at mechanize etc too. > > The problems I face are - > 1. I liked twisted a lot, but when I implemented it, I found that > client support is there only for twisted.web and not twisted.web2. > Since I connect to the same website every time, I would like to have > persistent connections and since twisted.web is HTTP/1.0, persistent > connection support is not yet there. Without persistent connections, I > would have to have TCP connection handshake everytime and it is taking > too much time. > 2. Since I connect to the website every second, I have to have > many connections running at the same time. I am worried that creating > threads for each connection is going to be a big problem (esp if the > server response is slow), since the processor will get swamped - > especially since there are many other activities going on in the > machine. > 3. I would also like to pipe line the requests - esp if the > response is slow. > > Other requirements: > 1. HTTPS Support > 2. Connection through proxy.
You could take a look at pycurl - python bindings for libcurl. http://pycurl.sourceforge.net/ srp > > Is there any good web client which I can use straight up? Or would > I have to implement the whole thing myself? It looks like a big beast > and I was wondering whether python provides it straight up. > > Regards > K -- http://mail.python.org/mailman/listinfo/python-list