Leleu thank you for your time and for advice, I'm going to check out selenium.
субота, 22. новембар 2014. 17.43.35 UTC+1, Travis Leleu је написао/ла: > > I usually look at the ajax calls made from the page to see if I can't just > mock that request and get the data back right from the server. (Note it's > possible the site is sending the product name via the original html page, > and just uses js to swap it out -- in that case, I would recommend selenium > w/ phantomjs [only because it was easy for me to setup, and just worked for > me]). > > That'll take up to 10x the resources of just a basic http request, but > it's still generally minimal unless you're scraping millions of pages. > > > On Sat, Nov 22, 2014 at 8:39 AM, JEBI93 <[email protected] > <javascript:>> wrote: > >> Yep thats what i got to, whats easiest way to scrape it, Selenium, >> Phantom.js? I don't have expirience with scraping JavaScript. >> >> субота, 22. новембар 2014. 17.35.17 UTC+1, Travis Leleu је написао/ла: >>> >>> Yes, it's a piece of javascript loading the h1 value. When I load the >>> page w/o js enabled, it just reads " >>> Proizvod >>> " >>> >>> >>> >>> On Sat, Nov 22, 2014 at 8:26 AM, JEBI93 <[email protected]> >>> wrote: >>> >>>> http://emmi.rs/procesori-%28cpu%29-cpu-fm2-amd-athlon% >>>> E2%84%A2-x4-quad-core-750k.11.html?productId=29644 is link. >>>> Here is screenshot of what i would like to scrape(content is same as >>>> you can see):http://i.imgur.com/QYW5Cgp.png. >>>> The lower one(orange h1) just prints Product, maybe there's some >>>> javascript code running in background or something I can't figure it out. >>>> Price, description and everything else works just fine. >>>> >>>> субота, 22. новембар 2014. 17.05.50 UTC+1, Travis Leleu је написао/ла: >>>>> >>>>> You didn't provide a link to the site in question, but how about >>>>> trying something like this? (I often mix CSS and XPath for convenience.) >>>>> >>>>> sel.css( 'div#blinks' )[0].xpath( './/text()' ) >>>>> >>>>> (I think the circled text is in the div's text node -- if it's in the >>>>> span, just make the xpath selector start with the span. >>>>> >>>>> >>>>> On Sat, Nov 22, 2014 at 6:35 AM, JEBI93 <[email protected]> >>>>> wrote: >>>>> >>>>>> So i was scraping ecommerce site and i can't figure which Xpath to >>>>>> use in this div tag:http://i.imgur.com/ltuVfTQ.png >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "scrapy-users" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at http://groups.google.com/group/scrapy-users. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "scrapy-users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at http://groups.google.com/group/scrapy-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "scrapy-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at http://groups.google.com/group/scrapy-users. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
