On 05/05/2017 08:45 AM, Rafael Knuth wrote:
Hi there,
I just recently learned how to build a basic web scraper with Python
3.5 (I am learning Python for data analytics purposes). Being new to
coding, I have a question:
How do I know which libraries I need to perform a certain task?
For example, in case of this web scraper (which I built with help of a
tutorial on YouTube) I need to have urrlib and Beautiful Soup
import urllib
import urllib.request
from bs4 import BeautifulSoup
theurl = "https://twitter.com/rafaelknuth";
thepage = urllib.request.urlopen(theurl)
soup = BeautifulSoup(thepage, "html.parser")
print(soup.title.text)
i = 1
for tweets in soup.findAll("div",{"class":"content"}):
print(i)
print(tweets.find("p").text)
i = i + 1
Is there a way I can figure out which libraries I need when drafting my code?
Can you share your experiences? Right now, if I wanted for example to
populate a Google Sheet with my scraped web content - how would I know
which libraries I would need to actually make this happen? I am trying
wondering if there is a process to figure out what I exactly need
library-wise.
There is a Python API to google sheets but when I had a look, it seemed
fairly complex. I haven't tried it yet but depending on what you need to
do this library may be what you need:
https://pypi.python.org/pypi/gspread.
Regards, Jim
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
