Re: difference between urllib2.urlopen and firefox view 'page source'?

2007-03-21 Thread cjl
Group: Thank you for all the informative replies, they have helped me figure things out. Next up is learning beautiful soup. Thank you for the code example, but I am trying to learn how to 'screen scrape', because Yahoo does make historical stock data available using the CSV format, but they do

Re: difference between urllib2.urlopen and firefox view 'page source'?

2007-03-20 Thread Tina I
cjl wrote: Hi. I am trying to screen scrape some stock data from yahoo, so I am trying to use urllib2 to retrieve the html and beautiful soup for the parsing. Maybe (most likely) I am doing something wrong, but when I use urllib2.urlopen to fetch a page, and when I view 'page source' of

Re: difference between urllib2.urlopen and firefox view 'page source'?

2007-03-20 Thread kyosohma
On Mar 20, 1:56 am, Tina I [EMAIL PROTECTED] wrote: cjl wrote: Hi. I am trying to screen scrape some stock data from yahoo, so I am trying to use urllib2 to retrieve the html and beautiful soup for the parsing. Maybe (most likely) I am doing something wrong, but when I use

Re: difference between urllib2.urlopen and firefox view 'page source'?

2007-03-20 Thread John Nagle
Here's a useful online tool that might help you see what's happening: http://www.sitetruth.com/experimental/viewer.html We use this to help webmasters see what our web crawler is seeing. This reads a page, using Python and FancyURLOpener, with a USER-AGENT string of

Re: difference between urllib2.urlopen and firefox view 'page source'?

2007-03-20 Thread wesley chun
On Mar 20, 8:33 am, [EMAIL PROTECTED] wrote: On Mar 20, 1:56 am, Tina I [EMAIL PROTECTED] wrote: I am trying to screen scrape some stock data from yahoo, so I am trying to use urllib2 to retrieve the html and beautiful soup for the parsing. You can do this fairly easily. I found a

difference between urllib2.urlopen and firefox view 'page source'?

2007-03-19 Thread cjl
Hi. I am trying to screen scrape some stock data from yahoo, so I am trying to use urllib2 to retrieve the html and beautiful soup for the parsing. Maybe (most likely) I am doing something wrong, but when I use urllib2.urlopen to fetch a page, and when I view 'page source' of the exact same URL

Re: difference between urllib2.urlopen and firefox view 'page source'?

2007-03-19 Thread zacherates
On Mar 19, 10:30 pm, cjl [EMAIL PROTECTED] wrote: Hi. I am trying to screen scrape some stock data from yahoo, so I am trying to use urllib2 to retrieve the html and beautiful soup for the parsing. Maybe (most likely) I am doing something wrong, but when I use urllib2.urlopen to fetch a

Re: difference between urllib2.urlopen and firefox view 'page source'?

2007-03-19 Thread Steve Holden
cjl wrote: Hi. I am trying to screen scrape some stock data from yahoo, so I am trying to use urllib2 to retrieve the html and beautiful soup for the parsing. Maybe (most likely) I am doing something wrong, but when I use urllib2.urlopen to fetch a page, and when I view 'page source' of