Re: [sage-support] Re: Tracking memory usage and time
On Thu, 19 Sep 2019, J wrote: > In detail I found the syndrome_decoders crashes while initiating with > big codes as the system runs out of memory Hmm... An idea: $ fgrep VmPeak /proc/6649/status VmPeak:31204 kB (6649 is PID of my bash) If you add delay to the end of sage script, you could check how much was max memory usage. -- Jori Mäntysalo Tampereen yliopisto - Ihminen ratkaisee -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/alpine.DEB.2.21..1909191424090.110721%40shell.sis.uta.fi.
Re: [sage-support] Re: Tracking memory usage and time
More or less this yes; But it seem I will stick to putting it in a sage ipyhton session wrapped in a screen session In detail I found the syndrome_decoders crashes while initiating with big codes as the system runs out of memory and that prompted me to plot RAM vs max_error for the syndrome decoder On 19-09-19 08:22:29, Jori Mäntysalo (TAU) wrote: > On Wed, 18 Sep 2019, J wrote: > > > The most problematic part for me is: > > > > I would like to script it > > I'm not sure what you mean. You have some list L of objects, and want to > know how much memory it takes to run f(x) for each x in L? > > -- > Jori Mäntysalo > > Tampereen yliopisto - Ihminen ratkaisee > > -- > You received this message because you are subscribed to the Google Groups > "sage-support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-support+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-support/alpine.DEB.2.21..1909191120130.110721%40shell.sis.uta.fi. -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/20190919101531.bwbjb2ghhqdowu6d%40deathbolt.927589452.space.
Re: [sage-support] Re: Tracking memory usage and time
On Wed, 18 Sep 2019, J wrote: > The most problematic part for me is: > > I would like to script it I'm not sure what you mean. You have some list L of objects, and want to know how much memory it takes to run f(x) for each x in L? -- Jori Mäntysalo Tampereen yliopisto - Ihminen ratkaisee -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/alpine.DEB.2.21..1909191120130.110721%40shell.sis.uta.fi.
Re: [sage-support] Re: Tracking memory usage and time
The most problematic part for me is: I would like to script it On 19-09-18 10:20:12, Jori Mäntysalo (TAU) wrote: > On Tue, 17 Sep 2019, J wrote: > > > TBH I can't get it to work > > and > > "sagemath %mprun" > > gives to pages of search results including this thread ^^ > > Duh. Somebody should add a page on this to the doc. > > First, said > > ./sage -pip install memory_profiler > > and then normally > > ./sage --notebook=jupyter > > In a notebook I did > > def power2(x): > L = range(x) > s = 0 > for i in L: > s += 2*i+1 > return s > > and then loaded the extension > > %load_ext memory_profiler > > and last ran > > %memit > power2(5) > > It works. However, > > %mprun -f power2 > power2(5) > > does not. Is there an easy way to profile memory usage on line-by-line > basis? > > -- > Jori Mäntysalo > > Tampereen yliopisto - Ihminen ratkaisee > > -- > You received this message because you are subscribed to the Google Groups > "sage-support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-support+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-support/alpine.DEB.2.21..1909181316180.53292%40shell.sis.uta.fi. -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/20190918140345.5bpq6xehlrncq4hk%40deathbolt.927589452.space.
Re: [sage-support] Re: Tracking memory usage and time
On Tue, 17 Sep 2019, J wrote: > TBH I can't get it to work > and > "sagemath %mprun" > gives to pages of search results including this thread ^^ Duh. Somebody should add a page on this to the doc. First, said ./sage -pip install memory_profiler and then normally ./sage --notebook=jupyter In a notebook I did def power2(x): L = range(x) s = 0 for i in L: s += 2*i+1 return s and then loaded the extension %load_ext memory_profiler and last ran %memit power2(5) It works. However, %mprun -f power2 power2(5) does not. Is there an easy way to profile memory usage on line-by-line basis? -- Jori Mäntysalo Tampereen yliopisto - Ihminen ratkaisee -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/alpine.DEB.2.21..1909181316180.53292%40shell.sis.uta.fi.
Re: [sage-support] Re: Tracking memory usage and time
On 19-09-03 09:43:23, Jori Mäntysalo (TAU) wrote: > On Mon, 2 Sep 2019, J wrote: > > > Thanks get_memory_usage sounds good; I want to run several decoders from > > the coding theory module to see better show there ups and downs; > > There is also at least %mprun magic. Googling that will give you some > examples. > TBH I can't get it to work and "sagemath %mprun" gives to pages of search results including this thread ^^ > -- > Jori Mäntysalo > > Tampereen yliopisto - Ihminen ratkaisee > > -- > You received this message because you are subscribed to the Google Groups > "sage-support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-support+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-support/alpine.DEB.2.21..1909031237550.45316%40shell.sis.uta.fi. -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/20190917205815.p2z3d7lxbkbqtawx%40deathbolt.927589452.space.
Re: [sage-support] Re: Tracking memory usage and time
On 19-09-03 11:31:32, Nils Bruin wrote:
>
>
> On Tuesday, September 3, 2019 at 2:43:27 AM UTC-7, Jori Mäntysalo (TAU)
> wrote:
> >
> > There is also at least %mprun magic. Googling that will give you some
> > examples.
> >
> > Looking at the memory footprint of the entire process (as a function of
> time) gives some indication of memory use of a certain implementation of an
> algorithm, but there are many factors that influence it. CPython probably
> has a slight preference for reusing (freed/reclaimed) memory over
> requesting new memory from the OS, but there is not an actual guarantee.
> And CPython might be seriously lax in reclaiming memory, or it might be
> prevented by a memory leak in sage that is not due to the implementation of
> the algorithm. So you can take results like that only as an indication and
> not as authoritative. Determining memory usage of an algorithm in the
> mathematical sense probably needs code analysis.
>
> (that said, memory claimed from the OS definitely gives an UPPER BOUND on
> the memory usage of a certain algorithm; for obvious reasons)
>
Well, one of the algorithms stores
sum {l^e} {\binom{n}{e}}
with l being the number of elements of the field
which gets big quite fast
> --
> You received this message because you are subscribed to the Google Groups
> "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-support+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-support/66674d87-2f64-4da1-89f9-009b2898d3a4%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sage-support/20190903195428.zge4htnopcsadfko%40deathbolt.927589452.space.
Re: [sage-support] Re: Tracking memory usage and time
On Tuesday, September 3, 2019 at 2:43:27 AM UTC-7, Jori Mäntysalo (TAU) wrote: > > There is also at least %mprun magic. Googling that will give you some > examples. > > Looking at the memory footprint of the entire process (as a function of time) gives some indication of memory use of a certain implementation of an algorithm, but there are many factors that influence it. CPython probably has a slight preference for reusing (freed/reclaimed) memory over requesting new memory from the OS, but there is not an actual guarantee. And CPython might be seriously lax in reclaiming memory, or it might be prevented by a memory leak in sage that is not due to the implementation of the algorithm. So you can take results like that only as an indication and not as authoritative. Determining memory usage of an algorithm in the mathematical sense probably needs code analysis. (that said, memory claimed from the OS definitely gives an UPPER BOUND on the memory usage of a certain algorithm; for obvious reasons) -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/66674d87-2f64-4da1-89f9-009b2898d3a4%40googlegroups.com.
[sage-support] Re: Tracking memory usage and time
On 2019-09-03, Jori Mäntysalo wrote: > On Mon, 2 Sep 2019, J wrote: > >> Thanks get_memory_usage sounds good; I want to run several decoders from >> the coding theory module to see better show there ups and downs; > > There is also at least %mprun magic. Googling that will give you some > examples. Cool! I didn't know about %mprun before. Thanks! Simon -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/qkm6b8%241593%242%40blaine.gmane.org.
Re: [sage-support] Re: Tracking memory usage and time
On 19-09-03 09:43:23, Jori Mäntysalo (TAU) wrote: > On Mon, 2 Sep 2019, J wrote: > > > Thanks get_memory_usage sounds good; I want to run several decoders from > > the coding theory module to see better show there ups and downs; > > There is also at least %mprun magic. Googling that will give you some > examples. > Thank you, I will research this too -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/20190903111334.xwsd2v63vlyl5jwc%40deathbolt.927589452.space.
Re: [sage-support] Re: Tracking memory usage and time
On Mon, 2 Sep 2019, J wrote: > Thanks get_memory_usage sounds good; I want to run several decoders from > the coding theory module to see better show there ups and downs; There is also at least %mprun magic. Googling that will give you some examples. -- Jori Mäntysalo Tampereen yliopisto - Ihminen ratkaisee -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/alpine.DEB.2.21..1909031237550.45316%40shell.sis.uta.fi.
Re: [sage-support] Re: Tracking memory usage and time
On 19-09-02 10:35:18, Simon King wrote: > Hi J, > > On 2019-08-24, J wrote: > > to do a overview of a rather different set of `SAGE` methods, I would > > like to not only track the time used to run a command, but also the > > memory usage of the commands. > > > > Is there a recommended way to do this? > > I am a bit surprised that nobody answered this question yet. Sorry. > > There is the get_memory_usage command, that might provide what you asked > for. But I am not sure if I understand correctly what you want to > achieve: Do you have a lengthy program and you want to understand how > much resources each individual command in your program takes? In that > case, it might make sense to use a profiler (e.g., %prun or %crun). Or do > you only want to know how much time and memore the program takes in total? > In that case, %time and get_memory_usage would probably give you the > answer. > Thanks get_memory_usage sounds good; I want to run several decoders from the coding theory module to see better show there ups and downs; it is expected for the syndrome decoder to be quicker but more memory hungry while the nearest neighbor is more time expensive. -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/20190902112113.qzcbu4shd2eirs37%40deathbolt.927589452.space.
[sage-support] Re: Tracking memory usage and time
Hi J, On 2019-08-24, J wrote: > to do a overview of a rather different set of `SAGE` methods, I would > like to not only track the time used to run a command, but also the > memory usage of the commands. > > Is there a recommended way to do this? I am a bit surprised that nobody answered this question yet. Sorry. There is the get_memory_usage command, that might provide what you asked for. But I am not sure if I understand correctly what you want to achieve: Do you have a lengthy program and you want to understand how much resources each individual command in your program takes? In that case, it might make sense to use a profiler (e.g., %prun or %crun). Or do you only want to know how much time and memore the program takes in total? In that case, %time and get_memory_usage would probably give you the answer. Best regards, Simon -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/qkir96%245tgi%242%40blaine.gmane.org.
