On Tue, 2023-01-24 at 12:13 +0100, Miluji Sb wrote: > Greetings everyone, > > I have a question on extracting country-level data from CRU ( > https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.06/crucy.2205251923.v4.06/countries/tmp/ > ).
Something like: To get all links/filenames in one table: a <- rvest::read_html("https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.06/crucy.2205251923.v4.06/countries/tmp/ ") tbl <- a |> rvest::html_table() |> as.data.frame() tbl <- tbl[-c(1,2),] To download them all to specific directory my_download_function <- function(myurl ="", output_dir = "data") { if(!dir.exists({{output_dir}})) {dir.create({{output_dir}})} .destfile = paste0({{output_dir}}, "/", {{myurl}}) .myurl <- paste0("https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.06/crucy.2205251923.v4.06/countries/tmp/ ", {{myurl}}) download.file(url = .myurl, destfile = .destfile, method = "wget", extra = "-c --progress=bar:force") NULL } invisible(lapply(seq(nrow(tbl)), function(i) my_download_function(tbl[i,1], "data"))) Now, having it locally you can read them one by one with read.csv, like: f <- list.files(path = "data", pattern = "crucy*", full.names = TRUE) read.csv(f[i], skip = 3, header = TRUE) It doesn't make sense without adding additional information about country/territotry, but at least you have starting point. Regards, Grzegorz _______________________________________________ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo