On May 27, 12:29 pm, powah <wong_po...@yahoo.ca> wrote: > I want to download all mib files from the web > page:http://www.juniper.net/techpubs/software/junos/junos94/swconfig-net-m... > > All mib filenames are of this format:www.juniper.net/techpubs... .txt > > I write this program but has the following error. > Please help. > Thanks. > > [code] > #!/usr/bin/env python > import urllib2,os,urlparse > url="http://www.juniper.net/techpubs/software/junos/junos94/swconfig- > net-mgmt/juniper-specific-mibs-junos-nm.html#jN18E19" > page=urllib2.urlopen(url) > f=0 > links=[] > data=page.read().split("\n") > for item in data: > if "www.juniper.net/techpubs" in item: > httpind=item.index("www.juniper.net/techpubs") > item=item[httpind:] > #print "item " + item > ind=item.index("<") > links.append(item[:ind]) #grab all links > # download all links > for link in links: > print "link " + link > filename=link.split("/")[-1] > print "downloading ... " + filename > u=urllib2.urlopen(link) > p=u.read() > open(filename,"w").write(p) > [/code] > > $ ~/python/downloadjuniper.py > linkwww.juniper.net/techpubs/software/junos/junos94/swconfig-net-mgmt/mib... > downloading ... mib-jnx-user-aaa.txt > Traceback (most recent call last): > File "/home/powah/python/downloadjuniper.py", line 20, in ? > u=urllib2.urlopen(link) > File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen > return _opener.open(url, data) > File "/usr/lib/python2.4/urllib2.py", line 350, in open > protocol = req.get_type() > File "/usr/lib/python2.4/urllib2.py", line 233, in get_type > raise ValueError, "unknown url type: %s" % self.__original > ValueError: unknown url > type:www.juniper.net/techpubs/software/junos/junos94/swconfig-net-mgmt/mib... > > $ python > Python 2.4.4 (#1, Oct 23 2006, 13:58:00) > [GCC 4.1.1 20061011 (Red Hat 4.1.1-30)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > > > > My computer is FC6 linux.
There's only a couple dozen of them, right-click->Save As. I'm sure Juniper would appreciate that much more than an automated crawler. As far as your ValueError is concerned, consider that 'www.juniper.com' doesn't start with a protocol specification when passed into urllib2.urlopen. -Jeff mcjeff.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list