Re: [R] [External] Re: read.csv fails in R console in Ubuntu terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?
On my Ubuntu system the download with read.csv succeeds in an R
console if I set the HTTPUserAgent and download.file.method options to
match the ones used by RStudio.
Given how picky the server is being I would worry about whether this
use is in line with the site's terms of service.
Best,
luke
On Thu, 16 Jul 2020, Ista Zahn wrote:
On Thu, Jul 16, 2020 at 5:15 PM Ista Zahn wrote:
On Thu, Jul 16, 2020 at 8:18 AM Rui Barradas wrote:
Hello,
Thanks, but no, download.file still gives 403 Forbidden with both method
= "libcurl" and method = "wget".
I think that makes it "not an R question". Ask on
https://unix.stackexchange.com/ maybe?
Oh, sorry I misread your message. Nevertheless:
$ curl
"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
Access Denied
Access Denied
You don't have permission to access
"httpoldnasdaqcomscreeningcompaniesbynameaspx"
on this server.
Reference185506d2171594934303938edcb
$ wget
"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
--2020-07-16 17:19:12--
https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving old.nasdaq.com (old.nasdaq.com)... 2600:1400:9000:28f::1b46,
2600:1400:9000:29b::1b46, 23.78.161.120
Connecting to old.nasdaq.com
(old.nasdaq.com)|2600:1400:9000:28f::1b46|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-07-16 17:19:12 ERROR 403: Forbidden.
I don't think this is an R problem.
Best,
Ista
Best,
Ista
Rui Barradas
Às 05:31 de 16/07/20, Jeff Newmiller escreveu:
Perhaps read FAQ 7.43? [1]
[1]
https://cran.r-project.org/doc/FAQ/R-FAQ.html#How-can-I-enable-secure-https-downloads-in-R_003f
On July 15, 2020 4:02:27 PM PDT, Rui Barradas wrote:
Hello,
R 4.0.2 on Ubuntu 20.04 LTS, sessionInfo below.
I'm also unable to read the file with Rscript from the Ubuntu terminal
but the error is not the same as the OP's.
The first try was a file test1.R with the following commands.
x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
read.csv(x, as.is=TRUE, na="n/a")
And run with Rscript
rui@rui:~$ Rscript --vanilla test1.R
Error in file(file, "rt") :
cannot open the connection to
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
Calls: read.csv -> read.table -> file
In addition: Warning message:
In file(file, "rt") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
HTTP status was '403 Forbidden'
Execution halted
The second try was download.file() and then read it.
File test2.R is:
x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
download.file(x, "companylist.csv")
read.csv("companylist.csv", as.is=TRUE, na="n/a")
But this too failed with error 403 Forbiden.
rui@rui:~$ Rscript --vanilla test2.R
trying URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
Error in download.file(x, "companylist.csv") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
In addition: Warning message:
In download.file(x, "companylist.csv") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
HTTP status was '403 Forbidden'
Execution halted
This is my session info.
rui@rui:~$ Rscript --vanilla -e 'sessionInfo()'
R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
locale:
[1] LC_CTYPE=pt_PT.UTF-8 LC_NUMERIC=C
[3] LC_TIME=pt_PT.UTF-8LC_COLLATE=pt_PT.UTF-8
[5] LC_MONETARY=pt_PT.UTF-8LC_MESSAGES=pt_PT.UTF-8
[7] LC_PAPER=pt_PT.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=pt_PT.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.0.2
Às 08:45 de 15/07/20, Sam H escreveu:
Hi,
I am trying to download some data using read.csv and it works
perfectly in
RStudio and fails in the R console in the terminal in Ubuntu 18.04
after
upgrading from R 3.6.3 to 4.0.2. Before upgrading this worked in the
R
console in the terminal also without any issues.
Why would that be? How to fix this?
Below please find R code output and sessionInfo().
*Works in RStudio*
read.csv("https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;,
header=TRUE, as.is=TRUE, na="n/a")
SymbolName
LastSale MarketCap IPOyear1 TXG
10x Genomics, Inc. 87.4400 $8.6B20192 YI
111, Inc. 6.4800 $533.69M20183
PIH 1347 Property Insurance
Re: [R] Probabilities in discrete event simulation model in R
Please read and follow the posting guide linked below when asking for help here. This is not a "we do the work for you" forum. Bert Gunter "The trouble with having an open mind is that people keep coming along and sticking things into it." -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) On Thu, Jul 16, 2020 at 3:39 PM Shah Alam wrote: > I am designing a programme in R to simulate two events that are 1. a person > gets cancer, or 2. die of other causes. I have data for the probability of > getting cancer by age and probability of death due to causes other than > cancer. > >1. > >My dataset has 3 columns 1) age 2) prob getting cancer 3) prob dying >other causes. I generate random number using runif(1, min = 0, max = 1). >2. > >This random number compares with all the values in the second column >(prob getting cancer) and give the age which has the same value of >probability as this random number. Then it compares with all the values > in >the 3rd column and gives the age at which women die because of causes > other >than cancer. > > The problem is: > >1. > >I want an R code in which the random number compares in both the columns >simultaneously (prob of getting cancer, and prob of death) step by step. >for example, if the random number is 0.2, first the code checks the >observation 1 in the column of getting cancer if it fails to get the > same >values. Then it checks the observation 1 in the column of death due to >other causes. If it fails then move to 2nd observation of prob getting >cancer The process should go on until it finds the age which >has the same value of probability as this random number. Kindly help me > on >how to do this in R. > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Probabilities in discrete event simulation model in R
I am designing a programme in R to simulate two events that are 1. a person gets cancer, or 2. die of other causes. I have data for the probability of getting cancer by age and probability of death due to causes other than cancer. 1. My dataset has 3 columns 1) age 2) prob getting cancer 3) prob dying other causes. I generate random number using runif(1, min = 0, max = 1). 2. This random number compares with all the values in the second column (prob getting cancer) and give the age which has the same value of probability as this random number. Then it compares with all the values in the 3rd column and gives the age at which women die because of causes other than cancer. The problem is: 1. I want an R code in which the random number compares in both the columns simultaneously (prob of getting cancer, and prob of death) step by step. for example, if the random number is 0.2, first the code checks the observation 1 in the column of getting cancer if it fails to get the same values. Then it checks the observation 1 in the column of death due to other causes. If it fails then move to 2nd observation of prob getting cancer The process should go on until it finds the age which has the same value of probability as this random number. Kindly help me on how to do this in R. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] read.csv fails in R console in Ubuntu terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?
On Thu, Jul 16, 2020 at 5:15 PM Ista Zahn wrote:
>
> On Thu, Jul 16, 2020 at 8:18 AM Rui Barradas wrote:
> >
> > Hello,
> >
> > Thanks, but no, download.file still gives 403 Forbidden with both method
> > = "libcurl" and method = "wget".
>
> I think that makes it "not an R question". Ask on
> https://unix.stackexchange.com/ maybe?
Oh, sorry I misread your message. Nevertheless:
$ curl
"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
Access Denied
Access Denied
You don't have permission to access
"httpoldnasdaqcomscreeningcompaniesbynameaspx"
on this server.
Reference185506d2171594934303938edcb
$ wget
"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
--2020-07-16 17:19:12--
https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving old.nasdaq.com (old.nasdaq.com)... 2600:1400:9000:28f::1b46,
2600:1400:9000:29b::1b46, 23.78.161.120
Connecting to old.nasdaq.com
(old.nasdaq.com)|2600:1400:9000:28f::1b46|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-07-16 17:19:12 ERROR 403: Forbidden.
I don't think this is an R problem.
Best,
Ista
>
> Best,
> Ista
>
> >
> > Rui Barradas
> >
> > Às 05:31 de 16/07/20, Jeff Newmiller escreveu:
> > > Perhaps read FAQ 7.43? [1]
> > >
> > > [1]
> > > https://cran.r-project.org/doc/FAQ/R-FAQ.html#How-can-I-enable-secure-https-downloads-in-R_003f
> > >
> > > On July 15, 2020 4:02:27 PM PDT, Rui Barradas
> > > wrote:
> > >> Hello,
> > >>
> > >> R 4.0.2 on Ubuntu 20.04 LTS, sessionInfo below.
> > >>
> > >> I'm also unable to read the file with Rscript from the Ubuntu terminal
> > >> but the error is not the same as the OP's.
> > >>
> > >>
> > >> The first try was a file test1.R with the following commands.
> > >>
> > >> x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
> > >> read.csv(x, as.is=TRUE, na="n/a")
> > >>
> > >>
> > >> And run with Rscript
> > >>
> > >> rui@rui:~$ Rscript --vanilla test1.R
> > >> Error in file(file, "rt") :
> > >>cannot open the connection to
> > >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
> > >> Calls: read.csv -> read.table -> file
> > >> In addition: Warning message:
> > >> In file(file, "rt") :
> > >>cannot open URL
> > >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
> > >>
> > >> HTTP status was '403 Forbidden'
> > >> Execution halted
> > >>
> > >>
> > >>
> > >> The second try was download.file() and then read it.
> > >> File test2.R is:
> > >>
> > >> x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
> > >> download.file(x, "companylist.csv")
> > >> read.csv("companylist.csv", as.is=TRUE, na="n/a")
> > >>
> > >>
> > >> But this too failed with error 403 Forbiden.
> > >>
> > >> rui@rui:~$ Rscript --vanilla test2.R
> > >> trying URL
> > >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
> > >> Error in download.file(x, "companylist.csv") :
> > >>cannot open URL
> > >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
> > >> In addition: Warning message:
> > >> In download.file(x, "companylist.csv") :
> > >>cannot open URL
> > >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
> > >>
> > >> HTTP status was '403 Forbidden'
> > >> Execution halted
> > >>
> > >>
> > >> This is my session info.
> > >>
> > >> rui@rui:~$ Rscript --vanilla -e 'sessionInfo()'
> > >> R version 4.0.2 (2020-06-22)
> > >> Platform: x86_64-pc-linux-gnu (64-bit)
> > >> Running under: Ubuntu 20.04 LTS
> > >>
> > >> Matrix products: default
> > >> BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
> > >> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
> > >>
> > >> locale:
> > >> [1] LC_CTYPE=pt_PT.UTF-8 LC_NUMERIC=C
> > >> [3] LC_TIME=pt_PT.UTF-8LC_COLLATE=pt_PT.UTF-8
> > >> [5] LC_MONETARY=pt_PT.UTF-8LC_MESSAGES=pt_PT.UTF-8
> > >> [7] LC_PAPER=pt_PT.UTF-8 LC_NAME=C
> > >> [9] LC_ADDRESS=C LC_TELEPHONE=C
> > >> [11] LC_MEASUREMENT=pt_PT.UTF-8 LC_IDENTIFICATION=C
> > >>
> > >> attached base packages:
> > >> [1] stats graphics grDevices utils datasets methods base
> > >>
> > >> loaded via a namespace (and not attached):
> > >> [1] compiler_4.0.2
> > >>
> > >>
> > >>
> > >> Às 08:45 de 15/07/20, Sam H escreveu:
> > >>> Hi,
> > >>>
> > >>> I am trying to download some data using read.csv and it works
> > >> perfectly in
> > >>> RStudio and fails in the R console in the terminal in Ubuntu 18.04
> > >> after
> > >>> upgrading from R 3.6.3 to 4.0.2. Before upgrading this worked in the
> > >> R
> > >>> console in the terminal also without any issues.
> > >>>
> > >>> Why would that be? How to fix this?
> > >>>
> > >>> Below please find R code output and sessionInfo().
> > >>>
> > >>> *Works in
Re: [R] read.csv fails in R console in Ubuntu terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?
On Thu, Jul 16, 2020 at 8:18 AM Rui Barradas wrote:
>
> Hello,
>
> Thanks, but no, download.file still gives 403 Forbidden with both method
> = "libcurl" and method = "wget".
I think that makes it "not an R question". Ask on
https://unix.stackexchange.com/ maybe?
Best,
Ista
>
> Rui Barradas
>
> Às 05:31 de 16/07/20, Jeff Newmiller escreveu:
> > Perhaps read FAQ 7.43? [1]
> >
> > [1]
> > https://cran.r-project.org/doc/FAQ/R-FAQ.html#How-can-I-enable-secure-https-downloads-in-R_003f
> >
> > On July 15, 2020 4:02:27 PM PDT, Rui Barradas wrote:
> >> Hello,
> >>
> >> R 4.0.2 on Ubuntu 20.04 LTS, sessionInfo below.
> >>
> >> I'm also unable to read the file with Rscript from the Ubuntu terminal
> >> but the error is not the same as the OP's.
> >>
> >>
> >> The first try was a file test1.R with the following commands.
> >>
> >> x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
> >> read.csv(x, as.is=TRUE, na="n/a")
> >>
> >>
> >> And run with Rscript
> >>
> >> rui@rui:~$ Rscript --vanilla test1.R
> >> Error in file(file, "rt") :
> >>cannot open the connection to
> >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
> >> Calls: read.csv -> read.table -> file
> >> In addition: Warning message:
> >> In file(file, "rt") :
> >>cannot open URL
> >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
> >>
> >> HTTP status was '403 Forbidden'
> >> Execution halted
> >>
> >>
> >>
> >> The second try was download.file() and then read it.
> >> File test2.R is:
> >>
> >> x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
> >> download.file(x, "companylist.csv")
> >> read.csv("companylist.csv", as.is=TRUE, na="n/a")
> >>
> >>
> >> But this too failed with error 403 Forbiden.
> >>
> >> rui@rui:~$ Rscript --vanilla test2.R
> >> trying URL
> >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
> >> Error in download.file(x, "companylist.csv") :
> >>cannot open URL
> >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
> >> In addition: Warning message:
> >> In download.file(x, "companylist.csv") :
> >>cannot open URL
> >> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
> >>
> >> HTTP status was '403 Forbidden'
> >> Execution halted
> >>
> >>
> >> This is my session info.
> >>
> >> rui@rui:~$ Rscript --vanilla -e 'sessionInfo()'
> >> R version 4.0.2 (2020-06-22)
> >> Platform: x86_64-pc-linux-gnu (64-bit)
> >> Running under: Ubuntu 20.04 LTS
> >>
> >> Matrix products: default
> >> BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
> >> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
> >>
> >> locale:
> >> [1] LC_CTYPE=pt_PT.UTF-8 LC_NUMERIC=C
> >> [3] LC_TIME=pt_PT.UTF-8LC_COLLATE=pt_PT.UTF-8
> >> [5] LC_MONETARY=pt_PT.UTF-8LC_MESSAGES=pt_PT.UTF-8
> >> [7] LC_PAPER=pt_PT.UTF-8 LC_NAME=C
> >> [9] LC_ADDRESS=C LC_TELEPHONE=C
> >> [11] LC_MEASUREMENT=pt_PT.UTF-8 LC_IDENTIFICATION=C
> >>
> >> attached base packages:
> >> [1] stats graphics grDevices utils datasets methods base
> >>
> >> loaded via a namespace (and not attached):
> >> [1] compiler_4.0.2
> >>
> >>
> >>
> >> Às 08:45 de 15/07/20, Sam H escreveu:
> >>> Hi,
> >>>
> >>> I am trying to download some data using read.csv and it works
> >> perfectly in
> >>> RStudio and fails in the R console in the terminal in Ubuntu 18.04
> >> after
> >>> upgrading from R 3.6.3 to 4.0.2. Before upgrading this worked in the
> >> R
> >>> console in the terminal also without any issues.
> >>>
> >>> Why would that be? How to fix this?
> >>>
> >>> Below please find R code output and sessionInfo().
> >>>
> >>> *Works in RStudio*
> >>>
>
> >> read.csv("https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;,
> >> header=TRUE, as.is=TRUE, na="n/a")
> >>> SymbolName
> >>> LastSale MarketCap IPOyear1 TXG
> >>> 10x Genomics, Inc. 87.4400 $8.6B20192 YI
> >>> 111, Inc. 6.4800 $533.69M20183
> >>> PIH 1347 Property Insurance Holdings, Inc. 4.5350
> >>> $27.52M2014
> >>>sessionInfo()
> >>> R version 4.0.2 (2020-06-22)
> >>> Platform: x86_64-pc-linux-gnu (64-bit)
> >>> Running under: Ubuntu 18.04.4 LTS
> >>>
> >>> Matrix products: default
> >>> BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
> >>> LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
> >>>
> >>> locale:
> >>>[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> >>> LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
> >>>[5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
> >>> LC_PAPER=en_US.UTF-8 LC_NAME=C
> >>>[9] LC_ADDRESS=C LC_TELEPHONE=C
> >>> LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> >>>
> >>> attached base packages:[1] stats
Re: [R] read.csv fails in R console in Ubuntu terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?
On 7/15/2020 4:02 PM, Rui Barradas wrote:
Hello,
R 4.0.2 on Ubuntu 20.04 LTS, sessionInfo below.
I'm also unable to read the file with Rscript from the Ubuntu terminal
but the error is not the same as the OP's.
The first try was a file test1.R with the following commands.
x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
read.csv(x, as.is=TRUE, na="n/a")
And run with Rscript
rui@rui:~$ Rscript --vanilla test1.R
Error in file(file, "rt") :
cannot open the connection to
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
Calls: read.csv -> read.table -> file
In addition: Warning message:
In file(file, "rt") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
HTTP status was '403 Forbidden'
Execution halted
The second try was download.file() and then read it.
File test2.R is:
x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
download.file(x, "companylist.csv")
read.csv("companylist.csv", as.is=TRUE, na="n/a")
But this too failed with error 403 Forbiden.
rui@rui:~$ Rscript --vanilla test2.R
trying URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
Error in download.file(x, "companylist.csv") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
In addition: Warning message:
In download.file(x, "companylist.csv") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
HTTP status was '403 Forbidden'
Execution halted
<<>>
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Works fine in Windows 10 64-bit with R-4.0.2, so I would echo Bert
Gunter's advise to try r-sig-debian list.
Dan
--
Daniel Nordlund
Port Townsend, WA USA
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Year and month from a sequence
Dear Jim, Eric, and Rui,
Great solutions! Thank you so much.
Best,
Milu
On Wed, Jul 15, 2020 at 9:20 AM Rui Barradas wrote:
> Hello,
>
> Yet another way, with package lubridate.
> When creating the "Date" objects, I set day = 1 to compare to TRUE below.
>
>
> library(lubridate)
>
> start <- as.Date(ISOdate(1861, 1, 1))
> end <- as.Date(ISOdate(2005, 12, 1))
>
> start + months(4139 - 2400)
> #[1] "2005-12-01"
>
> end == start + months(4139 - 2400)
> #[1] TRUE
>
>
> The argument to function months can be a vector of integers.
>
>
> start + months(1:(4139 - 2400))
>
>
>
> Then extract the months.
>
> month(start)
> month(end)
> month(start + months())
> month(start + months(4139 - 2400))
>
>
> Hope this helps,
>
> Rui Barradas
>
> Às 23:26 de 14/07/20, Miluji Sb escreveu:
> > Dear all,
> > I have a panel dataset with a large number of groups (200K+) with a
> > sequence representing the month of the observations (2400 - 4139). The
> data
> > is from January 1861 to December 2005.
> > My goal is to extract the corresponding month and year for each
> observation.
> >
> > I tried the following;
> > x$dt <- seq(from=as.Date("1861-01-01"),by='1 month',length=209000)
> >
> > However, the end date is clearly out of the range. I would like to
> convert
> > the month number (2400 - 4139) to January 1861 to December 2005 by ID.
> How
> > can I achieve this? Any help will be highly appreciated.
> >
> > Best,
> >
> > Milu
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error in chol.default
The error msg says it all if you know how to read it.
> When I run the optimization (given that I can't find parameters that
> fit the data by eyeball), I get the error:
> ```
> Error in chol.default(object$hessian) :
> the leading minor of order 1 is not positive definite
Your Jacobian (derivatives of the residual function w.r.t. the parameters)
is singular -- rather spectacularly so.
Try the short addition to your code to use analytic derivatives:
rich = function(p, x) {
a = p["curvature"]
k = p["finalPop"]
r = p["growthRate"]
y = r * x * (1-(x/k)^a)
return(y)
}
ricky = function(p, x, y) p$r * x * (1-(x/p$k)^p$a) -y
# values
Y <- c(41, 41, 41, 41, 41, 41, 45, 62, 121, 198, 275,
288, 859, 1118)
X <- 1:14
a = 1
k = 83347
r = 40
fit = rich(c(curvature=a, finalPop=k, growthRate=r), X)
plot(Y ~ X,
col = "red", lwd = 2,
main = "Richards model",
xlab = expression(bold("Days")),
ylab = expression(bold("Cases")))
points(X, fit, type = "l", lty = 2, lwd = 2)
library("minpack.lm")
o <- nls.lm(par = list(a=a, k=k, r=r), fn = ricky, x = X, y = Y)
summary(o)
print(o1)
library(nlsr)
xy=data.frame(x=X, y=Y)
rcky2 = y ~ r * x * (1-(x/k)^a) -y
o1 <- nlxb(rcky2, start = c(a=a, k=k, r=r), data=xy, trace=TRUE)
You should find
> o1
nlsr object: x
residual sumsquares = 3161769 on 14 observations
after 8Jacobian and 13 function evaluations
namecoeff SE tstat pval gradient
JSingval
a294.113NA NA NA 0
31.86
k 83347NA NA NA 0
0
r74.9576NA NA NA -6.064e-05
0
>
Note the singular values of the Jacobian. Actual zeros!
Even so, nlsr had managed to make some progress.
nls() and nls.lm() use approximate derivatives. Often that's fine (and it is
more flexible
in handling awkward functions), but a lot of the time it is NOT OK.
JN
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Error in chol.default
Hello,
I am trying to fit a Richards model to some cumulative incidence data
of infection. I got this example:
```
rich = function(p, x) {
a = p["curvature"]
k = p["finalPop"]
r = p["growthRate"]
y = r * x * (1-(x/k)^a)
return(y)
}
ricky = function(p, x, y) p$r * x * (1-(x/p$k)^p$a) -y
# values
Y <- c(41, 41, 41, 41, 41, 41, 45, 62, 121, 198, 275,
288, 859, 1118)
X <- 1:14
a = 1
k = 83347
r = 40
fit = rich(c(curvature=a, finalPop=k, growthRate=r), X)
plot(Y ~ X,
col = "red", lwd = 2,
main = "Richards model",
xlab = expression(bold("Days")),
ylab = expression(bold("Cases")))
points(X, fit, type = "l", lty = 2, lwd = 2)
library("minpack.lm")
o <- nls.lm(par = list(a=a, k=k, r=r), fn = ricky, x = X, y = Y)
summary(o)
```
When I run the optimization (given that I can't find parameters that
fit the data by eyeball), I get the error:
```
Error in chol.default(object$hessian) :
the leading minor of order 1 is not positive definite
```
What is this about?
Thank you
--
Best regards,
Luigi
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R-es] Rcomander (Anova de dos factores con interacción)
Buenas tardes, Tengo una tabla de doble entrada sobre la que deseo efectuar una análisis anova de dos factores con interacción a través de Rcomander. La tabla tiene esta forma: ritmo X Y Z Altura A 10,15 10,22 15,24 B 11,22 14,3 65,31 Mi pregunta es: ¿Debo dar otra forma a la tabla para que Rcomander la reconozca? Por ejemplo, así cardio ritmo altura 10 X A 15 X A 11 X B 22 X B (…) (…) (…) ¿O hay alguna forma para que Rcomander reconozca la tabla de doble entrada a partir de Excel? Gracias David Aguinaga Abad [[alternative HTML version deleted]] ___ R-help-es mailing list R-help-es@r-project.org https://stat.ethz.ch/mailman/listinfo/r-help-es
Re: [R] read.csv fails in R console in Ubuntu terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?
Hello,
Thanks, but no, download.file still gives 403 Forbidden with both method
= "libcurl" and method = "wget".
Rui Barradas
Às 05:31 de 16/07/20, Jeff Newmiller escreveu:
Perhaps read FAQ 7.43? [1]
[1]
https://cran.r-project.org/doc/FAQ/R-FAQ.html#How-can-I-enable-secure-https-downloads-in-R_003f
On July 15, 2020 4:02:27 PM PDT, Rui Barradas wrote:
Hello,
R 4.0.2 on Ubuntu 20.04 LTS, sessionInfo below.
I'm also unable to read the file with Rscript from the Ubuntu terminal
but the error is not the same as the OP's.
The first try was a file test1.R with the following commands.
x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
read.csv(x, as.is=TRUE, na="n/a")
And run with Rscript
rui@rui:~$ Rscript --vanilla test1.R
Error in file(file, "rt") :
cannot open the connection to
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
Calls: read.csv -> read.table -> file
In addition: Warning message:
In file(file, "rt") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
HTTP status was '403 Forbidden'
Execution halted
The second try was download.file() and then read it.
File test2.R is:
x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;
download.file(x, "companylist.csv")
read.csv("companylist.csv", as.is=TRUE, na="n/a")
But this too failed with error 403 Forbiden.
rui@rui:~$ Rscript --vanilla test2.R
trying URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
Error in download.file(x, "companylist.csv") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
In addition: Warning message:
In download.file(x, "companylist.csv") :
cannot open URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
HTTP status was '403 Forbidden'
Execution halted
This is my session info.
rui@rui:~$ Rscript --vanilla -e 'sessionInfo()'
R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
locale:
[1] LC_CTYPE=pt_PT.UTF-8 LC_NUMERIC=C
[3] LC_TIME=pt_PT.UTF-8LC_COLLATE=pt_PT.UTF-8
[5] LC_MONETARY=pt_PT.UTF-8LC_MESSAGES=pt_PT.UTF-8
[7] LC_PAPER=pt_PT.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=pt_PT.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.0.2
Às 08:45 de 15/07/20, Sam H escreveu:
Hi,
I am trying to download some data using read.csv and it works
perfectly in
RStudio and fails in the R console in the terminal in Ubuntu 18.04
after
upgrading from R 3.6.3 to 4.0.2. Before upgrading this worked in the
R
console in the terminal also without any issues.
Why would that be? How to fix this?
Below please find R code output and sessionInfo().
*Works in RStudio*
read.csv("https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;,
header=TRUE, as.is=TRUE, na="n/a")
SymbolName
LastSale MarketCap IPOyear1 TXG
10x Genomics, Inc. 87.4400 $8.6B20192 YI
111, Inc. 6.4800 $533.69M20183
PIH 1347 Property Insurance Holdings, Inc. 4.5350
$27.52M2014
sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.4 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:[1] stats graphics grDevices utils
datasets methods base
loaded via a namespace (and not attached):[1] compiler_4.0.2
tools_4.0.2
*Fails in R console in terminal*
>
read.csv("https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download;,
header=TRUE, as.is=TRUE, na="n/a")
Error in file(file, "rt") :
cannot open the connection to
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download'
In addition: Warning message:
In file(file, "rt") :
URL
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=1=download':
status was 'Failure when receiving data from the peer'> traceback()3:
file(file, "rt")2: read.table(file = file, header = header, sep =
sep,
quote = quote,
dec = dec, fill = fill, comment.char = comment.char, ...)1:
[R] aws.s3::s3sync Error in curl::curl_fetch_disk(url, x$path, handle = handle) : Failed to open file
I am trying to use aws.s3::s3sync to sync the contents of a bucket to an
aws ec2 instance ( via rstudio server)
my script is as follows I cannot make a repro since that would need to be
run from the instance with my account key and details ( replaced with
xx below).
all files I am trying to sync are under the s3 prefix FAME_FMR which is
the only top level "directory" in the bucket
Have I missed a setting?
If not any suggestions on this error?
thanks
Nevil Amos
library(aws.s3)
Sys.setenv("AWS_ACCESS_KEY_ID" = "x",
"AWS_SECRET_ACCESS_KEY" =
"x",
"AWS_DEFAULT_REGION" = "ap-southeast-2")
myBucket<-"ecological-risk-analysis"
myPath<-"~/ShinyApps"
> bucket_exists(myBucket)[1][1] TRUE> dir.exists(myPath)[1] TRUE
>s3sync(path = myPath ,bucket =myBucket)
[998]
"FAME_FMR/HDMS/75m/BinaryThresholded/Satin_Bowerbird_Spp10679_Thresholded_Binary.tif"
[999]
"FAME_FMR/HDMS/75m/BinaryThresholded/Satin_Flycatcher_Spp10366_Thresholded_Binary.tif"
[1000]
"FAME_FMR/HDMS/75m/BinaryThresholded/Scaly_breasted_Lorikeet_Spp10256_Thresholded_Binary.tif"
[ reached getOption("max.print") -- omitted 191 entries ]1191 bucket
objects not found in local directory<== Saving object 'FAME_FMR/' to
'~/ShinyApps/FAME_FMR/'Error in curl::curl_fetch_disk(url, x$path,
handle = handle) :
Failed to open file /home/rstudio/ShinyApps/FAME_FMR/.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
