On 22/06/2011 00:40, Gary Yang wrote:
I use get and getstore() to retrieve links. However, the page I got with “get” and “getstore()” is different than the page I got from the browser. To debug it, I copied and pasted the link below to the browser’s address bar. Then, I view the HTML source code from browser. I searched the keyword, offer-listing. I found nine of them. Below is one of nine keyword offer-listing I found: <div class="mbcOlpLink"><a class="buyAction" href="/gp/offer-listing/1567651151/ref=dp_olp_all_mbc? Below is the URL: http://www.amazon.com/Vocabulary-School-Student-Norman-Levine/dp/1567651151 I use the script below to retrieve page and save it at found_it.html. However, I cannot find any “offer-listing” in it. The page generated by get / getstore is different than the browser with the same URL. What did I miss? Can someone help? #!/usr/bin/perl -w use strict; use LWP::Simple; my $link = "http://www.amazon.com/Vocabulary-School-Student-Norman-Levine/dp/1567651151";; my $found_it = get("$link"); print "$found_it\n"; my $status = getstore("$link", "found_it.html"); print "Success\n" unless is_error($status);
The amazon.com site makes extensive use of client-side JavaScript, which changes the displayed page comprehensively once it is loaded. It is also likely to be contrary to the Amazon site rules to access it robotically, as you are trying to do. Rob
