Re: [Rd] Hard memory limit of 16GB under Windows?

[Tomas Kalibera]

Hi Samuel,

could you please provide more information? Where do you see the limit 
reported or how did you trigger it, what version of Windows do you have, 
are you using 64-bit build of R (sessionInfo())


Please check help("Memory-limits") and section 8 of R Admin manual

Best,
Tomas

On 4/7/20 1:35 PM, Samuel Granjeaud IR/Inserm wrote:

Hi,
I am not not sure whether this topic belongs to this mail list, but I
feel the subscribers here should be the right audience.
I noticed that the memory limit reported under Windows is 16 GB. I am
wondering how to increase it. I didn't found anything in Rprofile.site
nor .Rprofile. Is this limit hard coded at compilation?
Best,
Samuel
[[alternative HTML version deleted]]

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Hard memory limit of 16GB under Windows?

[Samuel Granjeaud IR/Inserm]

Hi Tomas,

Many thanks for your answer.

Here is a copy of a fresh session under RStudio, and after a copy under 
Rgui.
Strangely enough the result of memory.limit() is not the same. Without 
your question I would not have looked to RGui, being used to work with 
RStudio.


The value under RGui sounds to correspond to the total RAM of the 
computer. It makes me noticing that the value is in MB.


The value under Rstudio was so huge (1.759219e+13) that I just 
interpreted it as GB. But I was totally wrong. So in fact I don't know 
what it refers to. The documentation says "For a 64-bit versions of R 
under 64-bit Windows the limit is currently 8Tb.", but it looks like 
being 16TB, which my computer don't have of course.


I still have to understand why my colleague has a problem of memory 
allocation (cannot allocate vector of size 12.6 Gb).


Sorry for the wrong interpretation, but thanks for the help,
Samuel

--- RStudio

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> memory.limit()
[1] 1.759219e+13
> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252
LC_MONETARY=French_France.1252 LC_NUMERIC=C

[5] LC_TIME=French_France.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.6.3 tools_3.6.3
>

--- RGui

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> ls()
character(0)
> memory.limit()
[1] 32627
> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252
LC_MONETARY=French_France.1252

[4] LC_NUMERIC=C   LC_TIME=French_France.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.6.3
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Hard memory limit of 16GB under Windows?

[Tomas Kalibera]

Hi Samuel,

please also have a look at ?memory.limit. You can set this limit at R 
startup. It is in megabytes. Maybe R Studio sets it at runtime.


Best
Tomas

On 4/7/20 3:57 PM, Samuel Granjeaud IR/Inserm wrote:

Hi Tomas,

Many thanks for your answer.

Here is a copy of a fresh session under RStudio, and after a copy 
under Rgui.
Strangely enough the result of memory.limit() is not the same. Without 
your question I would not have looked to RGui, being used to work with 
RStudio.


The value under RGui sounds to correspond to the total RAM of the 
computer. It makes me noticing that the value is in MB.


The value under Rstudio was so huge (1.759219e+13) that I just 
interpreted it as GB. But I was totally wrong. So in fact I don't know 
what it refers to. The documentation says "For a 64-bit versions of R 
under 64-bit Windows the limit is currently 8Tb.", but it looks like 
being 16TB, which my computer don't have of course.


I still have to understand why my colleague has a problem of memory 
allocation (cannot allocate vector of size 12.6 Gb).


Sorry for the wrong interpretation, but thanks for the help,
Samuel

--- RStudio

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> memory.limit()
[1] 1.759219e+13
> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252 
LC_MONETARY=French_France.1252 LC_NUMERIC=C

[5] LC_TIME=French_France.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods base

loaded via a namespace (and not attached):
[1] compiler_3.6.3 tools_3.6.3
>

--- RGui

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> ls()
character(0)
> memory.limit()
[1] 32627
> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252 
LC_MONETARY=French_France.1252

[4] LC_NUMERIC=C   LC_TIME=French_France.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods base

loaded via a namespace (and not attached):
[1] compiler_3.6.3
>



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Hard memory limit of 16GB under Windows?

[Marc Schwartz via R-devel]

Hi Samuel,

You may already be aware, but if not, RStudio has their own support mechanisms 
here:

  https://support.rstudio.com/hc/en-us

If this does turn out to be RStudio specific, you may wish to check there for 
additional insights.

Regards,

Marc Schwartz


> On Apr 7, 2020, at 10:24 AM, Tomas Kalibera  wrote:
> 
> Hi Samuel,
> 
> please also have a look at ?memory.limit. You can set this limit at R 
> startup. It is in megabytes. Maybe R Studio sets it at runtime.
> 
> Best
> Tomas
> 
> On 4/7/20 3:57 PM, Samuel Granjeaud IR/Inserm wrote:
>> Hi Tomas,
>> 
>> Many thanks for your answer.
>> 
>> Here is a copy of a fresh session under RStudio, and after a copy under Rgui.
>> Strangely enough the result of memory.limit() is not the same. Without your 
>> question I would not have looked to RGui, being used to work with RStudio.
>> 
>> The value under RGui sounds to correspond to the total RAM of the computer. 
>> It makes me noticing that the value is in MB.
>> 
>> The value under Rstudio was so huge (1.759219e+13) that I just interpreted 
>> it as GB. But I was totally wrong. So in fact I don't know what it refers 
>> to. The documentation says "For a 64-bit versions of R under 64-bit Windows 
>> the limit is currently 8Tb.", but it looks like being 16TB, which my 
>> computer don't have of course.
>> 
>> I still have to understand why my colleague has a problem of memory 
>> allocation (cannot allocate vector of size 12.6 Gb).
>> 
>> Sorry for the wrong interpretation, but thanks for the help,
>> Samuel
>> 
>> --- RStudio
>> 
>> R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
>> Copyright (C) 2020 The R Foundation for Statistical Computing
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> 
>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>> You are welcome to redistribute it under certain conditions.
>> Type 'license()' or 'licence()' for distribution details.
>> 
>> R is a collaborative project with many contributors.
>> Type 'contributors()' for more information and
>> 'citation()' on how to cite R or R packages in publications.
>> 
>> Type 'demo()' for some demos, 'help()' for on-line help, or
>> 'help.start()' for an HTML browser interface to help.
>> Type 'q()' to quit R.
>> 
>> > memory.limit()
>> [1] 1.759219e+13
>> > sessionInfo()
>> R version 3.6.3 (2020-02-29)
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> Running under: Windows 10 x64 (build 18363)
>> 
>> Matrix products: default
>> 
>> locale:
>> [1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252 
>> LC_MONETARY=French_France.1252 LC_NUMERIC=C
>> [5] LC_TIME=French_France.1252
>> 
>> attached base packages:
>> [1] stats graphics  grDevices utils datasets  methods base
>> 
>> loaded via a namespace (and not attached):
>> [1] compiler_3.6.3 tools_3.6.3
>> >
>> 
>> --- RGui
>> 
>> R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
>> Copyright (C) 2020 The R Foundation for Statistical Computing
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> 
>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>> You are welcome to redistribute it under certain conditions.
>> Type 'license()' or 'licence()' for distribution details.
>> 
>> R is a collaborative project with many contributors.
>> Type 'contributors()' for more information and
>> 'citation()' on how to cite R or R packages in publications.
>> 
>> Type 'demo()' for some demos, 'help()' for on-line help, or
>> 'help.start()' for an HTML browser interface to help.
>> Type 'q()' to quit R.
>> 
>> > ls()
>> character(0)
>> > memory.limit()
>> [1] 32627
>> > sessionInfo()
>> R version 3.6.3 (2020-02-29)
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> Running under: Windows 10 x64 (build 18363)
>> 
>> Matrix products: default
>> 
>> locale:
>> [1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252 
>> LC_MONETARY=French_France.1252
>> [4] LC_NUMERIC=C   LC_TIME=French_France.1252
>> 
>> attached base packages:
>> [1] stats graphics  grDevices utils datasets  methods base
>> 
>> loaded via a namespace (and not attached):
>> [1] compiler_3.6.3
>> >
>> 
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Hard memory limit of 16GB under Windows?

[Thierry Onkelinx via R-devel]

Dear Samuel,

The most important information from your mail was the actual error message:
"cannot allocate vector of size 12.6 Gb".

You'll need to know what code generated this error message. And then figure
out if the code does something sensible. Often that is not the case when
you get a similar error message.

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkel...@inbo.be
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be

///
To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to say
what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey
///




Op di 7 apr. 2020 om 16:16 schreef Samuel Granjeaud IR/Inserm <
samuel.granje...@inserm.fr>:

> Hi Tomas,
>
> Many thanks for your answer.
>
> Here is a copy of a fresh session under RStudio, and after a copy under
> Rgui.
> Strangely enough the result of memory.limit() is not the same. Without
> your question I would not have looked to RGui, being used to work with
> RStudio.
>
> The value under RGui sounds to correspond to the total RAM of the
> computer. It makes me noticing that the value is in MB.
>
> The value under Rstudio was so huge (1.759219e+13) that I just
> interpreted it as GB. But I was totally wrong. So in fact I don't know
> what it refers to. The documentation says "For a 64-bit versions of R
> under 64-bit Windows the limit is currently 8Tb.", but it looks like
> being 16TB, which my computer don't have of course.
>
> I still have to understand why my colleague has a problem of memory
> allocation (cannot allocate vector of size 12.6 Gb).
>
> Sorry for the wrong interpretation, but thanks for the help,
> Samuel
>
> --- RStudio
>
> R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
> Copyright (C) 2020 The R Foundation for Statistical Computing
> Platform: x86_64-w64-mingw32/x64 (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under certain conditions.
> Type 'license()' or 'licence()' for distribution details.
>
> R is a collaborative project with many contributors.
> Type 'contributors()' for more information and
> 'citation()' on how to cite R or R packages in publications.
>
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for an HTML browser interface to help.
> Type 'q()' to quit R.
>
>  > memory.limit()
> [1] 1.759219e+13
>  > sessionInfo()
> R version 3.6.3 (2020-02-29)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows 10 x64 (build 18363)
>
> Matrix products: default
>
> locale:
> [1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252
> LC_MONETARY=French_France.1252 LC_NUMERIC=C
> [5] LC_TIME=French_France.1252
>
> attached base packages:
> [1] stats graphics  grDevices utils datasets  methods   base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.6.3 tools_3.6.3
>  >
>
> --- RGui
>
> R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
> Copyright (C) 2020 The R Foundation for Statistical Computing
> Platform: x86_64-w64-mingw32/x64 (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under certain conditions.
> Type 'license()' or 'licence()' for distribution details.
>
> R is a collaborative project with many contributors.
> Type 'contributors()' for more information and
> 'citation()' on how to cite R or R packages in publications.
>
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for an HTML browser interface to help.
> Type 'q()' to quit R.
>
>  > ls()
> character(0)
>  > memory.limit()
> [1] 32627
>  > sessionInfo()
> R version 3.6.3 (2020-02-29)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows 10 x64 (build 18363)
>
> Matrix products: default
>
> locale:
> [1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252
> LC_MONETARY=French_France.1252
> [4] LC_NUMERIC=C   LC_TIME=French_France.1252
>
> attached base packages:
> [1] stats graphics  grDevices utils datasets  methods   base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.6.3
>  >
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

[[alternative HTML version deleted]]

Re: [Rd] Hard memory limit of 16GB under Windows?

[Hervé Pagès]

Agreed. I don't see any evidence of a "hard limit" here. It just looks 
like the code you're running has exhausted the memory of your machine. 
It doesn't require creating a lot of 12.6 Gb vectors to do that, even on 
a machine with a lot of memory ;-)


H.

On 4/7/20 08:05, Thierry Onkelinx via R-devel wrote:

Dear Samuel,

The most important information from your mail was the actual error message:
"cannot allocate vector of size 12.6 Gb".

You'll need to know what code generated this error message. And then figure
out if the code does something sensible. Often that is not the case when
you get a similar error message.

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkel...@inbo.be
Havenlaan 88 bus 73, 1000 Brussel
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.inbo.be&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=r6WLJ5dXWo2qb7mQwONaCxYeeWgKwycd3y89JoqY-oY&s=VNmTfuiByvneqd4lcQS0o961TZa3ygo2Rdp7AtiGnk0&e=

///
To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to say
what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey
///




Op di 7 apr. 2020 om 16:16 schreef Samuel Granjeaud IR/Inserm <
samuel.granje...@inserm.fr>:


Hi Tomas,

Many thanks for your answer.

Here is a copy of a fresh session under RStudio, and after a copy under
Rgui.
Strangely enough the result of memory.limit() is not the same. Without
your question I would not have looked to RGui, being used to work with
RStudio.

The value under RGui sounds to correspond to the total RAM of the
computer. It makes me noticing that the value is in MB.

The value under Rstudio was so huge (1.759219e+13) that I just
interpreted it as GB. But I was totally wrong. So in fact I don't know
what it refers to. The documentation says "For a 64-bit versions of R
under 64-bit Windows the limit is currently 8Tb.", but it looks like
being 16TB, which my computer don't have of course.

I still have to understand why my colleague has a problem of memory
allocation (cannot allocate vector of size 12.6 Gb).

Sorry for the wrong interpretation, but thanks for the help,
Samuel

--- RStudio

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

  > memory.limit()
[1] 1.759219e+13
  > sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252
LC_MONETARY=French_France.1252 LC_NUMERIC=C
[5] LC_TIME=French_France.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.6.3 tools_3.6.3
  >

--- RGui

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

  > ls()
character(0)
  > memory.limit()
[1] 32627
  > sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Runni

Re: [Rd] Hard memory limit of 16GB under Windows?

[Samuel Granjeaud IR/Inserm]

Thanks to all contributors for your time and useful feedbacks.
They pointed me to the right direction and brought me new knowledges.
Samuel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel