ok i found something that works. instead of using the def i did this: for incident in row('div', {'class': 'food' or 'drink' }):
and it worked! only thing is that i think i am messing up the logic and here is why So when i run my script i get results, meaning it scrapes some stuff out, but then i get errors and where i am told: TypeError: unsupported operand type(s) for +: 'NullType' and 'str' Is this because of the logic in my code? i mean what i want the script to do is look for the <Tr> tag and then find the first div tag named food or drink, find its sibling named food, drink or dessert and then find the button tag which is the following sibling and THEN scrape out the href. is it possible that after it finds that first div and looks for the next sibling and then the next siblings href that it then tries to run the same process all over again starting with the 2nd div tag and being that it can't find the another div tag after the 2nd div tag that it trips up? know what i mean? here is my code: for row in bs('tr'): for incident in row('div', {'class': 'food' or 'drink'}): b = incident.findNextSibling('div', {'class': 'food' or 'drink' or 'dessert'}) n = b.findNextSibling('a', {'class': 'btn'}) link= n['href'] + "','" -- http://mail.python.org/mailman/listinfo/python-list