Hey Gaurang, What's the OS, version of python, version of scrapy you're using?
Does scrapy use urlib? or better, if you know, what lib does scrapy use for the url/xpath processing? On Mon, Dec 29, 2014 at 11:32 AM, Gaurang shah <[email protected]> wrote: > Sorry guys, Forgot to mentioned. All these xpath is able to identify the > elemenet using firepath add-on of firefox. > > *//video * > *//video[contains(@class,'html5-main-video')]/@src* > > *//div[@class='html5-video-container']/video/@src* > > *//div[@id='movie_player']/div[1]/video/@src* > > *//div[@id='player-api']/div[1]/div[1]/video/@src* > > *However none of them is working in scrapy ???* > > Gaurang Shah > Blog: qtp-help.blogspot.com > Mobile: +91 738756556 > > On Mon, Dec 29, 2014 at 9:41 PM, bruce <[email protected]> wrote: > >> Are you able to effectively create an xpath using your browser's >> xpath/dev tools? >> >> in firefox, you can use dom inspector, there are others as well, not sure >> of your browser.. >> >> In other words, is the issue with the "video" element, or something else >> in your xpath? >> >> If you can resolve the xpath with a separate tool, that should give you >> direction to solve the issue. >> >> >> >> On Mon, Dec 29, 2014 at 7:38 AM, Gaurang shah <[email protected]> >> wrote: >> >>> Hi Guys, >>> >>> I am trying to scrap the youtube site. And somehow the xpath which >>> fetches the video src is not working in scrapy. >>> >>> Url: https://www.youtube.com/watch?v=1EFnX1UkXVU >>> >>> >>> following xpaths is not working >>> *//video * >>> *//video[contains(@class,'html5-main-video')]/@src* >>> >>> >>> <https://lh3.googleusercontent.com/--_vqbGQxgWg/VKFLFyraflI/AAAAAAAACLY/2352f1VU0ds/s1600/Image%2B004.jpg> >>> I am able to retrive xpath till,* //div[@id='player-api']*, after that >>> it's dead end. scrapy is not able to find any more node in this. However >>> there are nodes inside that as well. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "scrapy-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/scrapy-users. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "scrapy-users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/scrapy-users/nGisMymqofU/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/scrapy-users. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
