Re: Direct interaction with subprocess - the curse of blocking I/O
> So well, I'd like to know, do you people know any solution to this > simple problem - making a user interact directly with a subprocess? you might want something like Expect. check out the pexpect module: http://pexpect.sourceforge.net/pexpect.html -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: handling https sites
> Is there any module in python to open https > sites through a proxy. yes, you can use "urllib2". from the urllib2 docs: "The default is to read the list of proxies from the environment variables" So if you have a proxy setup, it should detect it from your environment vars. If you want to specify something different, you want to build an opener with a ProxyHandler: http://docs.python.org/library/urllib2.html#urllib2.ProxyHandler -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: Graphical library - charts
> I suggest you look at matplotlib. +1 Another vote Matplotlib. It has impressive graphing/plotting capabilities and is used as a Python module/library. Description from site: "matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and ipython shell (ala matlab or mathematica), web application servers, and six graphical user interface toolkits." http://matplotlib.sourceforge.net/ -Corey Goldberg -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 slow for multiple requests
> It might be, if the local server doesn't scale well enough to handle > 100 concurrent requests. true.. I didn't think of that. I was assuming the client machine wasn't resource constrained. That would definitely lead to inaccurate timings if that was the case. -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 slow for multiple requests
> The problem is, that CentOS is running on the server and there is only > 2.4 available. On wich version did you ran these tests? I tested with Windows XP and Python 2.5.4. I don't have a 2.4 setup I can easily test with. you can try httplib rather than urllib2. httplib is slightly lower level and is actually used inside urllib2 for transport. -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 slow for multiple requests
> Bascally it just grabs a page xy > times and tells me how long it took. you aren't doing a read(), so technically you are just connecting to the web server and sending the request but never reading the content back from the socket. So your timing wouldn't be accurate. try this instead: response = urllib2.urlopen(req).read() But that is not the problem you are describing... > when I increase the number of repetitions, it is > slowing down considerably (1 is like 3 ms, 100 takes 6 seconds). > Maybe it is a known issue in urllib2 I ran your code and can not reproduce that behavior. No matter how many repetitions, I still get a similar response time per transaction. any more details or code samples you can provide? -Corey Goldberg -- http://mail.python.org/mailman/listinfo/python-list
Re: free chart lib for Python?
> You could try matplotlib:http://matplotlib.sourceforge.net. +1 Matplotlib. It's a very nice module with impressive graphing/charting/plotting capabilities. -- http://mail.python.org/mailman/listinfo/python-list
Re: Scraping a web page
> Is there anyway I > can get almost a screen capture of the page? I'm not sure exactly what you mean by "screen capture". But the webbrowser module in the standard lib might be of some help. You can use it to drive a web browser from Python. to load a page in your browser, you can do something like this: --- #! /usr/bin/env python import webbrowser url = 'http://www.google.com' webbrowser.open(url) --- -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: debuglevel for a HTTP request
> It doesn't show the debug output, any ideas? I think like this: opener = urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1)) -- http://mail.python.org/mailman/listinfo/python-list
Re: Guidance on writing a top-like console
> >> I am interested in writing an application that functions like a Unix > >> or Linux top in the way it displays data. > >> It should be command-line based but dynamically refreshing. also check out the source for "dstat". It is written in python and displays top-like information and more. It dynamically updates the way top does. -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: PyWin32 for Python 3.x
> Release 213 is out already: Tim, Mark, this is great news.. thanks for tracking 3.x so closely. I big barrier for me to eventually adopt 3.x is the ability to use pywin32. thanks! -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: parallel/concurrent process in python
> Why not use os.fork(), it is the same as C's fork? os.fork is not cross platform. It is *nix only. Subprocess runs on Windows also. The OP never specified his platform. -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: Themed TK (tk Tile) at last?!
> This is a big deal in that while tkinter came with (just about) every > Python the desire to use wxWidgets or Qt etc was high because tkinter > widgets just look so horrid. I liked Tk a lot, but also moved to wx because of Tk's L&F. Tk is great for simple tool interfaces. Great news. -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: getting all HTTP headers from urllib2 Request?
> > Looking at the httplib sources, the only headers it may add are Host, > > Accept-Encoding: identity, and Content-Length. now that I think of it, if it is only 3 headers, I can just override them explicitly from urllib2 and then log that. thanks a lot for looking into the httplib source! -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: getting all HTTP headers from urllib2 Request?
> Looking at the httplib sources, the only headers it may add are Host, > Accept-Encoding: identity, and Content-Length. those are exactly the headers I want to capture. do you know how to get a hold of them from a request using urllib2. -Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: getting all HTTP headers from urllib2 Request?
> I didn't try it, but the Request Class from urllib2 has a method > called, header_items(). That could be what your looking for. yes, that method only shows you all the headers added by urllib2. there are other headers that are produced by httplib under the covers that are added to the outgoing http request. That is what I am trying to get at. -Corey -- http://mail.python.org/mailman/listinfo/python-list
getting all HTTP headers from urllib2 Request?
I have a Python web client that uses urllib2. It is easy enough to add my own HTTP headers to the outgoing requests. I just create a dictionary of the headers I want to add, and pass it to the Request initializer. These custom headers are not all that gets sent. urllib2 attaches headers also. You can view the headers that urrlib2 adds by looking at unredirected_hdrs. However, these aren't the only HTTP headers that get sent on the wire. Other standard HTTP headers get added to outgoing requests as well as the custom ones I explicitly add and urllib2 adds. When I sniff the request using Wireshark, I see headers besides the ones I added myself. My question is how do a I get access to *all* of these headers? I want to log every request (including the full set of HTTP headers that get sent) that gets sent from my program, and can't figure out how. any pointers? In a nutshell: How do I get *all* the outgoing headers from an HTTP request created by urllib2? - Corey Goldberg -- http://mail.python.org/mailman/listinfo/python-list
