Re: getting memory usage of varaibles

2017-05-03 Thread Ben Finney
Erik writes: > The thing about functions or classes is that you can't (at the literal > source level) define them *without* giving them a name: Even a function is commonly defined without giving it a name. >>> strategies = [ ... (lambda x: x + 2), ... (lambda x: x ** 3),

Re: getting memory usage of varaibles

2017-05-03 Thread Ben Finney
Larry Martell writes: > On Wed, May 3, 2017 at 12:57 AM, Chris Angelico wrote: > > Coming right back to the beginning here: What do you expect the name > > of an object to be? > > The name of the variable in the program, e.g. sql, db_conn, rows, etc. That assumes that the object has exactly one

Re: getting memory usage of varaibles

2017-05-03 Thread Terry Reedy
On 5/3/2017 6:21 PM, Larry Martell wrote: On Wed, May 3, 2017 at 6:15 PM, Terry Reedy wrote: Python already uses this trick for functions, classes, and modules by giving them .__name__ attribute. Code objects have a .co_name attribute. These are used for tracing and tracebacks. I left out

Re: getting memory usage of varaibles

2017-05-03 Thread Chris Angelico
On Thu, May 4, 2017 at 10:32 AM, Ned Batchelder wrote: > On Wednesday, May 3, 2017 at 8:09:59 PM UTC-4, Steve D'Aprano wrote: >> On Thu, 4 May 2017 09:30 am, Ned Batchelder wrote: >> >> > Functions, classes, and modules can also be referred to by a number of >> > variables: >> > >> > def foo()

Re: getting memory usage of varaibles

2017-05-03 Thread Ned Batchelder
On Wednesday, May 3, 2017 at 8:09:59 PM UTC-4, Steve D'Aprano wrote: > On Thu, 4 May 2017 09:30 am, Ned Batchelder wrote: > > > Functions, classes, and modules can also be referred to by a number of > > variables: > > > > def foo(): pass > > bar = baz = foo > > > > But functions (by virt

Re: getting memory usage of varaibles

2017-05-03 Thread Steve D'Aprano
On Thu, 4 May 2017 09:30 am, Ned Batchelder wrote: > Functions, classes, and modules can also be referred to by a number of > variables: > > def foo(): pass > bar = baz = foo > > But functions (by virtue of the name in the def statement) have an > inherent name, Indeed; but we also hav

Re: getting memory usage of varaibles

2017-05-03 Thread Ned Batchelder
On Wednesday, May 3, 2017 at 6:22:28 PM UTC-4, larry@gmail.com wrote: > On Wed, May 3, 2017 at 6:15 PM, Terry Reedy wrote: > > On 5/3/2017 8:40 AM, Larry Martell wrote: > >> > >> On Wed, May 3, 2017 at 8:29 AM, Chris Angelico wrote: > >>> > >>> On Wed, May 3, 2017 at 10:12 PM, Larry Martell

Re: getting memory usage of varaibles

2017-05-03 Thread Erik
On 03/05/17 23:21, Larry Martell wrote: But not for a variable like a list or dict? What name should "[1, 2, 3]", or "{1, 'a': 2: 'b'}" be given? The thing about functions or classes is that you can't (at the literal source level) define them *without* giving them a name: def func(): pass c

Re: getting memory usage of varaibles

2017-05-03 Thread Larry Martell
On Wed, May 3, 2017 at 6:15 PM, Terry Reedy wrote: > On 5/3/2017 8:40 AM, Larry Martell wrote: >> >> On Wed, May 3, 2017 at 8:29 AM, Chris Angelico wrote: >>> >>> On Wed, May 3, 2017 at 10:12 PM, Larry Martell >>> wrote: On Wed, May 3, 2017 at 12:57 AM, Chris Angelico wrote:

Re: getting memory usage of varaibles

2017-05-03 Thread Terry Reedy
On 5/3/2017 8:40 AM, Larry Martell wrote: On Wed, May 3, 2017 at 8:29 AM, Chris Angelico wrote: On Wed, May 3, 2017 at 10:12 PM, Larry Martell wrote: On Wed, May 3, 2017 at 12:57 AM, Chris Angelico wrote: On Wed, May 3, 2017 at 5:53 AM, Larry Martell wrote: And I can see it getting larger

Re: getting memory usage of varaibles

2017-05-03 Thread Ned Batchelder
On Wednesday, May 3, 2017 at 8:40:09 AM UTC-4, larry@gmail.com wrote: > On Wed, May 3, 2017 at 8:29 AM, Ned Batchelder wrote: > > On Tuesday, May 2, 2017 at 11:49:37 PM UTC-4, larry@gmail.com wrote: > >> On Tue, May 2, 2017 at 7:01 PM, Erik wrote: > >> > On 02/05/17 23:28, Larry Martell w

Re: getting memory usage of varaibles

2017-05-03 Thread Larry Martell
On Wed, May 3, 2017 at 8:29 AM, Chris Angelico wrote: > On Wed, May 3, 2017 at 10:12 PM, Larry Martell > wrote: >> On Wed, May 3, 2017 at 12:57 AM, Chris Angelico wrote: >>> On Wed, May 3, 2017 at 5:53 AM, Larry Martell >>> wrote: And I can see it getting larger and larger. But I want to

Re: getting memory usage of varaibles

2017-05-03 Thread Larry Martell
On Wed, May 3, 2017 at 8:29 AM, Ned Batchelder wrote: > On Tuesday, May 2, 2017 at 11:49:37 PM UTC-4, larry@gmail.com wrote: >> On Tue, May 2, 2017 at 7:01 PM, Erik wrote: >> > On 02/05/17 23:28, Larry Martell wrote: >> >> Anyone have any thoughts on how I can monitor the variables'

Re: getting memory usage of varaibles

2017-05-03 Thread Chris Angelico
On Wed, May 3, 2017 at 10:12 PM, Larry Martell wrote: > On Wed, May 3, 2017 at 12:57 AM, Chris Angelico wrote: >> On Wed, May 3, 2017 at 5:53 AM, Larry Martell >> wrote: >>> And I can see it getting larger and larger. But I want to see what it >>> is that is causing this. My thought was to put

Re: getting memory usage of varaibles

2017-05-03 Thread Ned Batchelder
On Tuesday, May 2, 2017 at 11:49:37 PM UTC-4, larry@gmail.com wrote: > On Tue, May 2, 2017 at 7:01 PM, Erik wrote: > > On 02/05/17 23:28, Larry Martell wrote: > > Anyone have any thoughts on how I can monitor the variables' memory > usage as the script runs? > >>> > >>> > >>> Th

Re: getting memory usage of varaibles

2017-05-03 Thread Larry Martell
On Wed, May 3, 2017 at 12:57 AM, Chris Angelico wrote: > On Wed, May 3, 2017 at 5:53 AM, Larry Martell wrote: >> And I can see it getting larger and larger. But I want to see what it >> is that is causing this. My thought was to put all the objects in a >> dict with their sizes and compare them a

Re: getting memory usage of varaibles

2017-05-02 Thread INADA Naoki
I recommend tracemalloc. Start with PYTHONTRACEMALLOC=3, and increase depth only when stack trace is too short. 2017/05/03 午後0:50 "Larry Martell" : > On Tue, May 2, 2017 at 7:01 PM, Erik wrote: > > On 02/05/17 23:28, Larry Martell wrote: > > Anyone have any thoughts on how I can monit

Re: getting memory usage of varaibles

2017-05-02 Thread Chris Angelico
On Wed, May 3, 2017 at 5:53 AM, Larry Martell wrote: > And I can see it getting larger and larger. But I want to see what it > is that is causing this. My thought was to put all the objects in a > dict with their sizes and compare them as the program runs and report > on the one that are growing.

Re: getting memory usage of varaibles

2017-05-02 Thread Larry Martell
On Tue, May 2, 2017 at 7:01 PM, Erik wrote: > On 02/05/17 23:28, Larry Martell wrote: Anyone have any thoughts on how I can monitor the variables' memory usage as the script runs? >>> >>> >>> This is application-specific, but sometimes it helps to look at the >>> objects' types, or

Re: getting memory usage of varaibles

2017-05-02 Thread Erik
On 02/05/17 23:28, Larry Martell wrote: Anyone have any thoughts on how I can monitor the variables' memory usage as the script runs? This is application-specific, but sometimes it helps to look at the objects' types, or even their values. The types are dict and list, so they are not very use

Re: getting memory usage of varaibles

2017-05-02 Thread Larry Martell
On Tue, May 2, 2017 at 5:57 PM, Dan Stromberg wrote: > On Tue, May 2, 2017 at 12:53 PM, Larry Martell > wrote: >> I have a script that consumes more and more memory as it runs. It has >> no globals and the large data structures go out of scope often so >> should be garbage collected. I've looked

Re: getting memory usage of varaibles

2017-05-02 Thread breamoreboy
On Tuesday, May 2, 2017 at 8:54:46 PM UTC+1, larry@gmail.com wrote: > I have a script that consumes more and more memory as it runs. It has > no globals and the large data structures go out of scope often so > should be garbage collected. I've looked at the most likely suspects > with sys.getsi

Re: getting memory usage of varaibles

2017-05-02 Thread Dan Stromberg
On Tue, May 2, 2017 at 12:53 PM, Larry Martell wrote: > I have a script that consumes more and more memory as it runs. It has > no globals and the large data structures go out of scope often so > should be garbage collected. I've looked at the most likely suspects > with sys.getsizeof and they are

getting memory usage of varaibles

2017-05-02 Thread Larry Martell
I have a script that consumes more and more memory as it runs. It has no globals and the large data structures go out of scope often so should be garbage collected. I've looked at the most likely suspects with sys.getsizeof and they are not growing in size. I did this: sum([sys.getsizeof(o) for o