Hi, I am starting coding in r and one of the things that i want to do is to scrape some data from the web. The problem that I am having is that I cannot get passed the disclaimer page (which produces a session cookie). I have been able to collect some ideas and combine them in the code below but I dont get passed the disclaimer page. I am trying to agree the disclaimer with the postForm and write the cookie to a file, but I cannot do it succesfully.... The webpage cookies are written to the file but the value is FALSE... So any ideas of what I should do or what I am doing wrong with? Thank you for your help,
library(RCurl) library(XML) site <- " http://www.wateroffice.ec.gc.ca/graph/graph_e.html?mode=text&stn=05ND012&prm1=3&syr=2012&smo=09&sday=15&eyr=2012&emo=09&eday=18"; postForm(site, disclaimer_action="I Agree") cf <- "cookies.txt" no_cookie <- function() { curlHandle <- getCurlHandle(cookiefile=cf, cookiejar=cf) getURL(site, curl=curlHandle) rm(curlHandle) gc() } if ( file.exists(cf) == TRUE ) { file.create(cf) no_cookie() } allTables <- readHTMLTable(site) allTables [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
