Re: [Tutor] global variables/constants versus volatile variables/constants
On 14/06/2014 20:37, S Tareq wrote: You appear to be having problems with your keyboard. Could you please resend your message, stating precisely what you wanted to say. Thank you. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
Thank you very much Allan! 100% clear. I thank each and every one of you who contributed to all my question on the above subject. A lot of things became very clear. May you all be well. On Sun, Jun 15, 2014 at 3:15 AM, Alan Gauld wrote: > On 14/06/14 13:53, diliup gabadamudalige wrote: > > Say if I have a lot of Lists, strings and variables used to carry data >> to and from various functions why can't I have a class with all these in >> it? >> > > You can but it should also have the functions that operate on the data > too. Thats the point of classes they link function and data together > so you don't need to pass lots of parameters around. If you have a class > that jusat holds lots of disparate data items you might as well just use a > list/tuple or dictionary. The value of classes is when you add behaviour or > methods to the mix. > > > > I thought on the lines of blood carrying various things to different >> organs in the body. >> > > But even there the blood has a distinct set of related items, it doesn't > carry the food to your stomach or the video signals from > your eye to your brain. And blood has operations - it can flow, > coagulate, increase/decrease red-cell count etc. It has > behaviour as well as data. > > > class Containers(): >> def __init__(self): >> self.last_selected_scale= "" >> self.scale_notes = "" >> self.arpeggionotes = "" >> self.handplayed_notes = [] >> self.MIDIscale = [] # the MIDI scale to be played is here >> self.play_MIDIscale = False >> self.play_MIDIarpeggio = False >> self.play_MIDI_scale_note = False >> self.play_MIDI_arpeggio_note = False >> self.MIDInote_inscale = 0 >> > > Now that could be a Tune... And it might have record/play/pause operations > > > > now say if I play the MIDI keyboard, I collect the notes played which >> will be appended to the list v.hanplayednotes. This will be done from a >> function which will scan the keyboard and append played notes to that >> list. >> > > But that could be a method of the Tune sub-class, MidiTune, which knows > how to record from a MidiSequencer object. > > > This is how I have used these variables, lists and strings in this >> program. So far it has run error free. >> > > The issue is not how to make it run - you can do that in assembler. > The issue is how easy is it to read and how easy os it to modify and fix > later. 80% of the cost of commercial software is in "maintenance" > The goal of professional software engineers is to reduce maintenance > costs even if that increases the initial 20% development budget. > > > Is this not correct or bad design. I know it is not the normal way of >> using class but is it wrong? >> > > Almost certainly, because it adds a layer of complexity for little or no > gain. > > Whereas adding the methods to the class improves the maintainability and > readability (and sometimes the performance, although that's a secondary > benefit). > > HTH > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Diliup Gabadamudalige http://www.diliupg.com http://soft.diliupg.com/ ** This e-mail is confidential. It may also be legally privileged. If you are not the intended recipient or have received it in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Any unauthorized reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ** ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On 14/06/14 13:53, diliup gabadamudalige wrote: Say if I have a lot of Lists, strings and variables used to carry data to and from various functions why can't I have a class with all these in it? You can but it should also have the functions that operate on the data too. Thats the point of classes they link function and data together so you don't need to pass lots of parameters around. If you have a class that jusat holds lots of disparate data items you might as well just use a list/tuple or dictionary. The value of classes is when you add behaviour or methods to the mix. I thought on the lines of blood carrying various things to different organs in the body. But even there the blood has a distinct set of related items, it doesn't carry the food to your stomach or the video signals from your eye to your brain. And blood has operations - it can flow, coagulate, increase/decrease red-cell count etc. It has behaviour as well as data. class Containers(): def __init__(self): self.last_selected_scale= "" self.scale_notes = "" self.arpeggionotes = "" self.handplayed_notes = [] self.MIDIscale = [] # the MIDI scale to be played is here self.play_MIDIscale = False self.play_MIDIarpeggio = False self.play_MIDI_scale_note = False self.play_MIDI_arpeggio_note = False self.MIDInote_inscale = 0 Now that could be a Tune... And it might have record/play/pause operations now say if I play the MIDI keyboard, I collect the notes played which will be appended to the list v.hanplayednotes. This will be done from a function which will scan the keyboard and append played notes to that list. But that could be a method of the Tune sub-class, MidiTune, which knows how to record from a MidiSequencer object. This is how I have used these variables, lists and strings in this program. So far it has run error free. The issue is not how to make it run - you can do that in assembler. The issue is how easy is it to read and how easy os it to modify and fix later. 80% of the cost of commercial software is in "maintenance" The goal of professional software engineers is to reduce maintenance costs even if that increases the initial 20% development budget. Is this not correct or bad design. I know it is not the normal way of using class but is it wrong? Almost certainly, because it adds a layer of complexity for little or no gain. Whereas adding the methods to the class improves the maintainability and readability (and sometimes the performance, although that's a secondary benefit). HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On Friday, 13 June 2014, 12:45, Steven D'Aprano wrote:
On Fri, Jun 13, 2014 at 12:51:25PM +0530, diliup gabadamudalige wrote:
> Hi All!
> Hope everyone is
well.
>
> In my code there are many dictionaries and lists which are used in various
> functions. Is it better/pythonic/efficient to have these inside the
> function itself or declared at the beginning of the program in which case
> they will be global? They are all read only. I understand that global
> constants and variable have memory allocated to them but when declared
> inside a function are created on the fly, used and discarded. Please
> enlighten me further on this and correct me if i'm wrong.
A good question.
Depending on the size of these dictionaries and lists, worrying about
efficiency here may be premature optimization. As they say:
"We should forget about small efficiencies, say about 97% of
the time: premature optimization is the root of all evil."
-- Donald Knuth
"The First Rule of Program
Optimization: Don't do it. The
Second Rule of Program Optimization (for experts only!):
Don't do it yet." -- Michael A. Jackson
"More computing sins are committed in the name of efficiency
(without necessarily achieving it) than for any other single
reason — including blind stupidity." -- W.A. Wulf
If these lists and dicts are small, say, fewer than a dozen items, the
time to create and destroy them is probably trivial, especially if
you construct them from constant literals. In that case, it's a
matter of personal taste whether you prefer them as global constants or
local to a function.
But if it takes a long time to build the list, then you definitely
should move it outside the function and perform the initialisation step
only once:
# This may take a while...
PRIMES = [prime(i) for i in range(1,
101)]
If your lists really are read-only, then you should consider turning
them into tuples:
# Not this:
SOME_ITEMS = ["this", "that", "another"]
# Better:
SOME_ITEMS = ("this", "that", "another")
One advantage of the tuple is that in recent versions of Python, the
tuple may be compiled as a constant instead of being built at runtime.
This is from Python 2.7:
py> from dis import dis
py> code = compile("x = [2, 4, 8]", "", "exec")
py> dis(code)
1 0 LOAD_CONST 0 (2)
3 LOAD_CONST 1 (4)
6 LOAD_CONST 2 (8)
9 BUILD_LIST
3
12 STORE_NAME 0 (x)
15 LOAD_CONST 3 (None)
18 RETURN_VALUE
py> code = compile("x = (2, 4, 8)", "", "exec")
py> dis(code)
1 0 LOAD_CONST 4 ((2, 4, 8))
3 STORE_NAME 0 (x)
6 LOAD_CONST 3 (None)
9 RETURN_VALUE
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On Sat, Jun 14, 2014 at 06:23:08PM +0530, diliup gabadamudalige wrote: > Say if I have a lot of Lists, strings and variables used to carry data to > and from various functions why can't I have a class with all these in it? Of course you *can*, but you *should not*. Read on... [...] > so I declare > > class Containers(): > def __init__(self): > self.last_selected_scale= "" > self.scale_notes = "" > self.arpeggionotes = "" etc. > now i initiate the class with > v=Containers() So now you write code where everything you use starts with "v.". What does the v. do? How does it help your programming? Answer: it doesn't. It just makes an extra two characters to type. Compare the code you would write if you just defined the variables directly: # Option 1 last_selected_scale = "" scale_notes = "" arpeggionotes = "" do_something(last_selected_scale) do_another_thing(scale_notes, arpeggionotes) # Versus option 2: class Containers(): def __init__(self): self.last_selected_scale = "" self.scale_notes = "" self.arpeggionotes = "" v = Container() do_something(v.last_selected_scale) do_another_thing(v.scale_notes, v.arpeggionotes) The two options do *exactly* the same thing, but Option 2 takes much more typing and reading. For what benefit? The only benefit is if you might have multiple Containers at the same time: keyboard_a = Container() keyboard_b = Container() do_something(keyboard_a.last_selected_scale) do_another_thing(keyboard_b.scale_notes, keyboard_b.arpeggionotes) Now you can play and record with two keyboards at the same time. (Perhaps you have four arms? *smile*) In this case, the extra cost of using a class pays for itself. Otherwise, it is just wasted effort. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
Thank you all for those great clarifications. I learnt a lot from these. My roots in programming are from the early 80s and with a gap of nearly 30 years till I restarted again last year in March with Python. :) So some of the new concepts like classes are a bit alien to me but I am catching on. I made a mistake by using the word variables on variables and constants both. My design structure would most probably be quite unconventional to most like Allen Gauld says. I am still learning and I like to jump into the deep end and learn to swim there and so far have managed to stay alive! :) I have gained immensely from these few emails and hope I can gain more knowledge to write better code. Say if I have a lot of Lists, strings and variables used to carry data to and from various functions why can't I have a class with all these in it? I thought on the lines of blood carrying various things to different organs in the body. One common container class. Containers for data exchange between all classes and functions. so I declare class Containers(): def __init__(self): self.last_selected_scale= "" self.scale_notes = "" self.arpeggionotes = "" self.handplayed_notes = [] self.MIDIscale = [] # the MIDI scale to be played is here self.play_MIDIscale = False self.play_MIDIarpeggio = False self.play_MIDI_scale_note = False self.play_MIDI_arpeggio_note = False self.MIDInote_inscale = 0 now i initiate the class with v=Containers() now say if I play the MIDI keyboard, I collect the notes played which will be appended to the list v.hanplayednotes. This will be done from a function which will scan the keyboard and append played notes to that list. This is how I have used these variables, lists and strings in this program. So far it has run error free. Is this not correct or bad design. I know it is not the normal way of using class but is it wrong? Thank you all in advance. -- Diliup Gabadamudalige http://www.diliupg.com http://soft.diliupg.com/ ** This e-mail is confidential. It may also be legally privileged. If you are not the intended recipient or have received it in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Any unauthorized reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ** ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On 14Jun2014 11:21, Steven D'Aprano wrote: On Fri, Jun 13, 2014 at 09:42:34PM +0530, diliup gabadamudalige wrote: 2. Lists that are read only are better off being tuples? Possibly. It depends. As a general rule, tuples may be used for heterogeneous data, lists should be used for homogeneous data. To make this less clear, I would say this is a useful rule some of the time. It certainly isn't the only criterion one might use for choosing between a tuple and a list. What do I mean by this? This is a good use for a tuple: customer = (name, address, phone_number, status, balance) You have a fixed number of fields, and each field means something different (they are hetrogeneous). You should not use a list for something like this. Were I using the "heterogeneous data" rule I would be advocaing going to whole hog and using a "namedtuple" from the "collections" module. This has the advantage of giving each element of the tuple a nice name so that you can refer to "customer.address" instead of using magic indices like "customer[1]". That makes the code easier to read and easier to write. But if you're not using namedtuple-like attributes to access the tuple elements, then I would not be considering the "heterogeneous data" rule to be as relevant in my choice of tuple or list. There are some advantages to using a tuple as a frozen list, which can't have its values modified nor have its size changed. Specificly, if that is your intent and your code later tried to change the values (because it is buggy) then Python will raise an exception immediately, making it easier to find your mistake. If you stick to a list for this situation then the buggy changes will go ahead and the error will only be apparent much later when you get the wrong results. Fail early, fail often! Conversely, there are downsides to using tuples as frozen lists. The most obvious is that you can't add lists and tuples together, just lists and lists. If you need to add tuples and lists all the time then the surround fiddliness required may be enough to push you back to using lists throughout. Finally, another argument against mixing tuples and lists is simply that it may be early optimisation. Unless you're benefiting from the "you can't accidentally modify a tuple later" effect, just write it all with lists; it will be simpler. You can always come back later after the code is functioning correctly and think about tuples. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
Albert-Jan, I've been meaning to ask for a long time... I don't suppose you're hitting "Forward" rather than "Reply" in your posts are you? Because I've never seen replies from anyone else use the same style as your replies. Further comments below. On Fri, Jun 13, 2014 at 10:14:46AM -0700, Albert-Jan Roskam wrote: > Hmm, if I create the namedtuple with 'verbose=True' it *really* makes clear > why it took so much longer. > (what's '_property' btw? I know 'property' and the newer decorator wih the > same name, but not _property). _property is just property renamed. If you read the source code to namedtuple, you will see something like the line: _property = property or possibly: from builtins import property as _property That's likely done to make it a local variable, and so a little faster (although I don't know why the author bothered, it will make so little difference), or possibly to prevent clashes in case the caller defines a field name called "property". I'm not sure, but they are my guesses. [...] > v000 = _property(_itemgetter(0), doc='Alias for field number 0') > > v001 = _property(_itemgetter(1), doc='Alias for field number 1') > > v002 = _property(_itemgetter(2), doc='Alias for field number 2') [snip 97 more lines] Yes, thank you, we get the picture. No need to show 100 examples. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On Fri, Jun 13, 2014 at 09:42:34PM +0530, diliup gabadamudalige wrote: > Thank you all for these elaborate explanations. > so would it be wrong to assume that, > 1. the list or dict if 'large' and not mutated is better declared once in a > Class and used throughout the program? There is *no* advantage to writing this: class Foo: data = ['a', 'large', 'list'] result = function(Foo.data) instead of this: data = ['a', 'large', 'list'] result = function(data) Just dumping things into a class is not very helpful. You should ask yourself: * Does my code represent a "thing", like a string, or a web server, or a customer record? Then a class might be a good solution. * Does this thing have data and functions that go together? For example, a customer record might have data like Account Balance, and a function like update_account_balance. Then a class might be a good solution. * Could I have more than one of these "things" at the same time? For example, you obviously could have more than one string or customer record. Then a class is probably a good solution. If none of these things are true, then a class is probably a bad solution. > or > 2. Lists that are read only are better off being tuples? Possibly. It depends. As a general rule, tuples may be used for heterogeneous data, lists should be used for homogeneous data. What do I mean by this? This is a good use for a tuple: customer = (name, address, phone_number, status, balance) You have a fixed number of fields, and each field means something different (they are hetrogeneous). You should not use a list for something like this. This is a good use for a list: [value, another_value, a_third_value, ...] There is an arbitrary number of items, and each item represents the same kind of thing (they are homogeneous). Don't think of types, like "all the items are floats", but instead think "all the items are a person's height". As a micro-optimization, you might treat tuple as a frozen-list. That's not so unusual that you will confuse people by it, but it is a little unusual, and a matter of personal taste. > or > 3.the list or dict if 'large' and not mutated is better declared once as a > global variable and used throughout the program? If it's not mutated, it isn't really a *variable* is it? But yes, I would consider this the simplest thing that can work. This would be my preference. > The variables, lists carrying variables, and dict. carrying variables are > declared as a class. So why not include all above in the same class? > Varables and constans which are used throughout the program in one class > declared beofr the program enters the main executing loop? > wouldn't this be fast and the memory usage static up to a point (for all > declared items in the variables class)? I'm afraid I don't understand this paragraph. Can you try explaining more carefully please? (By the way: please trim unnecessary quoted text. Your email contained dozens of lines of double-quoted > > comments that had nothing to do with the questions you asked. If the comments aren't relevant, please delete them. Thank you.) -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On 13/06/14 19:20, diliup gabadamudalige wrote:
I declare the following dict at the beginning of the program.
KEY_SIGNATURES = {"C": [], "G": ["F"], "D": ["F", "C"], "A": ["F", "C",
"G"], "E": ["F", "C", "G", "D"], "B": ["F", "C", "G", "D", "A"], "F#":
now in the functions when ever I need to access i just use it.
eg:
sharps_or_flats = KEY_SIGNATURES[key]
or I can create it like this
class Variables():
def __init__(self, adv):
self.KEY_SIGNATURES = {"C": [], "G": ["F"],
now I call the class with
v=Variables()
now the same dict is
v.Key_SIGNATURES
my questions:
which is better to use and why?
As always there is no absolute answer. It depends on what you
are doing in a wider context.
However from your message you may be missing the point of the class
solution. The pointy of a class is to bring together the variables
*and the functions* that operate on those variables. So you should
not be doing v.Key_SIGNATURES from functions outside the class,
rather you should be defining your functions as methods of the
class and inside those methods using self.Key_SIGNATURES
Also a class called 'variables' is usually a bad design smell.
The class should be called something meaningful in the context
of your program. Something that represents the data and operations
that you have captured within it.
If you discover you have captured two unrelated concepts then
split the class into two. That will encourage reuse but also make
your code much more readable.
Do both remain in memory(separately of course) till I exit the program
once I run it? Will memory usage be the same?
For data this small what happens in memory should be the least of your
worries. Its much more important to get a design that is readable,
maintainable and reliable. Only when you have proved there is a
memory issue to solve do you need to start worrying about it.
will the creation of the dict take the same time in both above?
But if you are really worried about it use the timeit module or the
profiler to find out, don't guess or rely on others guesses.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
Thank you all for these elaborate explanations.
so would it be wrong to assume that,
1. the list or dict if 'large' and not mutated is better declared once in a
Class and used throughout the program?
or
2. Lists that are read only are better off being tuples?
or
3.the list or dict if 'large' and not mutated is better declared once as a
global variable and used throughout the program?
The variables, lists carrying variables, and dict. carrying variables are
declared as a class. So why not include all above in the same class?
Varables and constans which are used throughout the program in one class
declared beofr the program enters the main executing loop?
wouldn't this be fast and the memory usage static up to a point (for all
declared items in the variables class)?
On Fri, Jun 13, 2014 at 6:38 PM, Steven D'Aprano
wrote:
> On Fri, Jun 13, 2014 at 05:10:28AM -0700, Albert-Jan Roskam wrote:
>
> > The other day I used collections.namedtuple and I re-initialized
> > Record (see below) with every function*) call. Bad idea! It looks
> > nicer because I did not need a global (and globals are baaad, mkay?),
> > but it was *much* slower. I processed a log of a few million lines, I
> > think.
> >
> > # bad --> time-consuming
> > import collections
> >
> > def do_something_with(raw_record):
> >Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in
> range(100)))
> >return Record(*raw_record.split())
>
> Look at how much work you do here. First, you create a long string of
> the form:
>
> "v000 v001 v002 v003 ... v099"
>
> representing 1000 v-digits names. Then you create a brand new Record
> class that takes those 100 v-digits names as arguments. Creating that
> class requires building a string, parsing it as Python code, and then
> running it. (You're not expected to know that, but if you read the
> source code for namedtuple you will see that's how it works.) So
> creating that class is slow. Every time you call the function, it builds
> a new "v000 ... v099" string, from scratch, then builds a new class,
> also from scratch, and finally populates an instance of that class with
> 100 values from the raw_record.
>
> Only that last step needs to be done inside the function.
>
>
> > # better --> even though it uses a global variable
> > import collections
> >
> > Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in
> range(100)))
>
> [Aside: you may find it easier to debug problems with this if you give
> the namedtuple class a sensible name, like "Record", rather than "_".]
>
> How is that a global *variable*? It's a global name, "Record", but it is
> no more a "variable" than it would be if you did:
>
> class Record(tuple):
> def __new__(cls, v000, v001, v002, ... , v099):
> # code goes here
>
> @property
> def v000(self):
> return self[0]
>
> # likewise for v001, v002, ... v099
> # plus additional methods
>
>
> namedtuple is a factory function which creates a class. Buried deep
> within it is a class statement, just as if you had written the class
> yourself. Normally, when you create a class, you don't treat it as a
> variable, you treat it as a constant, like functions. That is no
> different from classes you create with the class keyword. So "global
> variables are bad" doesn't apply because it's not a variable.
>
> Even if it were a variable, what really matters is not that it gets
> stored in the global scope, but whether or not it gets explicitly passed
> to functions as arguments, or implicitly modified secretly behind the
> scenes. For example:
>
> # Not bad
> data = ["put", "stuff", "here"]
> process(data)
> do_things_with(data)
>
>
> # Bad, for various reasons
> data = ["put", "stuff", "here"]
> process() # process what?
> do_things_with() # What are we doing things with?
>
>
>
> In the first case, "data" may be stored in the global scope, but inside
> each function it is treated as a regular local variable. Let's contrast
> how one might operate on a second set of data in each case:
>
> # Too easy
> process(some_other_data)
>
>
> # Ouch, this is painful
> save_data = data
> data = some_other_data
> process()
> data = save_data # restore the previous value
>
>
> Global variables aren't bad because Moses came down from the mountains
> with a stone tablet that declares that they are bad. They're bad because
> they cause excessive coupling, they operate by side-effect, they spoil
> idepotent code, and they are implicit instead of explicit.
>
>
>
>
> > def do_something_with(raw_record):
> >return Record(*raw_record.split())
>
> Much more sensible!
>
>
>
> --
> Steven
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
--
Diliup Gabadamudalige
http://www.diliupg.com
http://soft.diliupg.com/
**
Th
Re: [Tutor] global variables/constants versus volatile variables/constants
Thank you all for these elaborate explanations.
so would it be wrong to assume that,
1. the list or dict if 'large' and not mutated is better declared once in a
Class and used throughout the program?
or
2. Lists that are read only are better off being tuples?
or
3.the list or dict if 'large' and not mutated is better declared once as a
global variable and used throughout the program?
The variables, lists carrying variables, and dict. carrying variables are
declared as a class. So why not include all above in the same class?
Variables and constants which are used throughout the program in one class
declared before the program enters the main executing loop?
wouldn't this be fast and the memory usage static up to a point (for all
declared items in the variables class)?
On Fri, Jun 13, 2014 at 9:42 PM, diliup gabadamudalige
wrote:
> Thank you all for these elaborate explanations.
> so would it be wrong to assume that,
> 1. the list or dict if 'large' and not mutated is better declared once in
> a Class and used throughout the program?
> or
> 2. Lists that are read only are better off being tuples?
> or
> 3.the list or dict if 'large' and not mutated is better declared once as a
> global variable and used throughout the program?
>
> The variables, lists carrying variables, and dict. carrying variables are
> declared as a class. So why not include all above in the same class?
> Varables and constans which are used throughout the program in one class
> declared beofr the program enters the main executing loop?
> wouldn't this be fast and the memory usage static up to a point (for all
> declared items in the variables class)?
>
>
> On Fri, Jun 13, 2014 at 6:38 PM, Steven D'Aprano
> wrote:
>
>> On Fri, Jun 13, 2014 at 05:10:28AM -0700, Albert-Jan Roskam wrote:
>>
>> > The other day I used collections.namedtuple and I re-initialized
>> > Record (see below) with every function*) call. Bad idea! It looks
>> > nicer because I did not need a global (and globals are baaad, mkay?),
>> > but it was *much* slower. I processed a log of a few million lines, I
>> > think.
>> >
>> > # bad --> time-consuming
>> > import collections
>> >
>> > def do_something_with(raw_record):
>> >Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in
>> range(100)))
>> >return Record(*raw_record.split())
>>
>> Look at how much work you do here. First, you create a long string of
>> the form:
>>
>> "v000 v001 v002 v003 ... v099"
>>
>> representing 1000 v-digits names. Then you create a brand new Record
>> class that takes those 100 v-digits names as arguments. Creating that
>> class requires building a string, parsing it as Python code, and then
>> running it. (You're not expected to know that, but if you read the
>> source code for namedtuple you will see that's how it works.) So
>> creating that class is slow. Every time you call the function, it builds
>> a new "v000 ... v099" string, from scratch, then builds a new class,
>> also from scratch, and finally populates an instance of that class with
>> 100 values from the raw_record.
>>
>> Only that last step needs to be done inside the function.
>>
>>
>> > # better --> even though it uses a global variable
>> > import collections
>> >
>> > Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in
>> range(100)))
>>
>> [Aside: you may find it easier to debug problems with this if you give
>> the namedtuple class a sensible name, like "Record", rather than "_".]
>>
>> How is that a global *variable*? It's a global name, "Record", but it is
>> no more a "variable" than it would be if you did:
>>
>> class Record(tuple):
>> def __new__(cls, v000, v001, v002, ... , v099):
>> # code goes here
>>
>> @property
>> def v000(self):
>> return self[0]
>>
>> # likewise for v001, v002, ... v099
>> # plus additional methods
>>
>>
>> namedtuple is a factory function which creates a class. Buried deep
>> within it is a class statement, just as if you had written the class
>> yourself. Normally, when you create a class, you don't treat it as a
>> variable, you treat it as a constant, like functions. That is no
>> different from classes you create with the class keyword. So "global
>> variables are bad" doesn't apply because it's not a variable.
>>
>> Even if it were a variable, what really matters is not that it gets
>> stored in the global scope, but whether or not it gets explicitly passed
>> to functions as arguments, or implicitly modified secretly behind the
>> scenes. For example:
>>
>> # Not bad
>> data = ["put", "stuff", "here"]
>> process(data)
>> do_things_with(data)
>>
>>
>> # Bad, for various reasons
>> data = ["put", "stuff", "here"]
>> process() # process what?
>> do_things_with() # What are we doing things with?
>>
>>
>>
>> In the first case, "data" may be stored in the global scope, but inside
>> each function it is treated as a regular local variable. Let's contrast
>> how one might operate on a second set of data in ea
Re: [Tutor] global variables/constants versus volatile variables/constants
- Original Message -
> From: Steven D'Aprano
> To: tutor@python.org
> Cc:
> Sent: Friday, June 13, 2014 3:08 PM
> Subject: Re: [Tutor] global variables/constants versus volatile
> variables/constants
>
> On Fri, Jun 13, 2014 at 05:10:28AM -0700, Albert-Jan Roskam wrote:
>
>> The other day I used collections.namedtuple and I re-initialized
>> Record (see below) with every function*) call. Bad idea! It looks
>> nicer because I did not need a global (and globals are baaad, mkay?),
>> but it was *much* slower. I processed a log of a few million lines, I
>> think.
>>
>> # bad --> time-consuming
>> import collections
>>
>> def do_something_with(raw_record):
>> Record = collections.namedtuple("_", "
> ".join("v%%03d" % i for i in range(100)))
>> return Record(*raw_record.split())
>
> Look at how much work you do here. First, you create a long string of
> the form:
>
> "v000 v001 v002 v003 ... v099"
>
> representing 1000 v-digits names. Then you create a brand new Record
> class that takes those 100 v-digits names as arguments. Creating that
> class requires building a string, parsing it as Python code, and then
> running it. (You're not expected to know that, but if you read the
> source code for namedtuple you will see that's how it works.) So
> creating that class is slow. Every time you call the function, it builds
> a new "v000 ... v099" string, from scratch, then builds a new class,
> also from scratch, and finally populates an instance of that class with
> 100 values from the raw_record.
>
> Only that last step needs to be done inside the function.
Hmm, if I create the namedtuple with 'verbose=True' it *really* makes clear why
it took so much longer.
(what's '_property' btw? I know 'property' and the newer decorator wih the same
name, but not _property).
In [1]: import collections
In [2]: Record = collections.namedtuple("_", " ".join("v%03d" % i for i in
range(100)),verbose=True)
class _(tuple):
'_(v000, v001, v002, v003, v004, v005, v006, v007, v008, v009, v010, v011,
v012, v013, v014, v015, v016, v017, v018, v019, v020, v021, v022, v023, v024,
v025, v026, v027, v028, v029, v030, v031, v032, v033, v034, v035, v036, v037,
v038, v039, v040, v041, v042, v043, v044, v045, v046, v047, v048, v049, v050,
v051, v052, v053, v054, v055, v056, v057, v058, v059, v060, v061, v062, v063,
v064, v065, v066, v067, v068, v069, v070, v071, v072, v073, v074, v075, v076,
v077, v078, v079, v080, v081, v082, v083, v084, v085, v086, v087, v088, v089,
v090, v091, v092, v093, v094, v095, v096, v097, v098, v099)'
__slots__ = ()
_fields = ('v000', 'v001', 'v002', 'v003', 'v004', 'v005', 'v006', 'v007',
'v008', 'v009', 'v010', 'v011', 'v012', 'v013', 'v014', 'v015', 'v016', 'v017',
'v018', 'v019', 'v020', 'v021', 'v022', 'v023', 'v024', 'v025', 'v026', 'v027',
'v028', 'v029', 'v030', 'v031', 'v032', 'v033', 'v034', 'v035', 'v036', 'v037',
'v038', 'v039', 'v040', 'v041', 'v042', 'v043', 'v044', 'v045', 'v046', 'v047',
'v048', 'v049', 'v050', 'v051', 'v052', 'v053', 'v054', 'v055', 'v056', 'v057',
'v058', 'v059', 'v060', 'v061', 'v062', 'v063', 'v064', 'v065', 'v066', 'v067',
'v068', 'v069', 'v070', 'v071', 'v072', 'v073', 'v074', 'v075', 'v076', 'v077',
'v078', 'v079', 'v080', 'v081', 'v082', 'v083', 'v084', 'v085', 'v086', 'v087',
'v088', 'v089', 'v090', 'v091', 'v092', 'v093', 'v094', 'v095', 'v096', 'v097',
'v098', 'v099')
def __new__(_cls, v000, v001, v002, v003, v004, v005, v006, v007, v008,
v009, v010, v011, v012, v013, v014, v015, v016, v017, v018, v019, v020, v021,
v022, v023, v024, v025, v026, v027, v028, v029, v030, v031, v032, v033, v03
Re: [Tutor] global variables/constants versus volatile variables/constants
On Fri, Jun 13, 2014 at 05:10:28AM -0700, Albert-Jan Roskam wrote:
> The other day I used collections.namedtuple and I re-initialized
> Record (see below) with every function*) call. Bad idea! It looks
> nicer because I did not need a global (and globals are baaad, mkay?),
> but it was *much* slower. I processed a log of a few million lines, I
> think.
>
> # bad --> time-consuming
> import collections
>
> def do_something_with(raw_record):
> Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in
> range(100)))
> return Record(*raw_record.split())
Look at how much work you do here. First, you create a long string of
the form:
"v000 v001 v002 v003 ... v099"
representing 1000 v-digits names. Then you create a brand new Record
class that takes those 100 v-digits names as arguments. Creating that
class requires building a string, parsing it as Python code, and then
running it. (You're not expected to know that, but if you read the
source code for namedtuple you will see that's how it works.) So
creating that class is slow. Every time you call the function, it builds
a new "v000 ... v099" string, from scratch, then builds a new class,
also from scratch, and finally populates an instance of that class with
100 values from the raw_record.
Only that last step needs to be done inside the function.
> # better --> even though it uses a global variable
> import collections
>
> Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in
> range(100)))
[Aside: you may find it easier to debug problems with this if you give
the namedtuple class a sensible name, like "Record", rather than "_".]
How is that a global *variable*? It's a global name, "Record", but it is
no more a "variable" than it would be if you did:
class Record(tuple):
def __new__(cls, v000, v001, v002, ... , v099):
# code goes here
@property
def v000(self):
return self[0]
# likewise for v001, v002, ... v099
# plus additional methods
namedtuple is a factory function which creates a class. Buried deep
within it is a class statement, just as if you had written the class
yourself. Normally, when you create a class, you don't treat it as a
variable, you treat it as a constant, like functions. That is no
different from classes you create with the class keyword. So "global
variables are bad" doesn't apply because it's not a variable.
Even if it were a variable, what really matters is not that it gets
stored in the global scope, but whether or not it gets explicitly passed
to functions as arguments, or implicitly modified secretly behind the
scenes. For example:
# Not bad
data = ["put", "stuff", "here"]
process(data)
do_things_with(data)
# Bad, for various reasons
data = ["put", "stuff", "here"]
process() # process what?
do_things_with() # What are we doing things with?
In the first case, "data" may be stored in the global scope, but inside
each function it is treated as a regular local variable. Let's contrast
how one might operate on a second set of data in each case:
# Too easy
process(some_other_data)
# Ouch, this is painful
save_data = data
data = some_other_data
process()
data = save_data # restore the previous value
Global variables aren't bad because Moses came down from the mountains
with a stone tablet that declares that they are bad. They're bad because
they cause excessive coupling, they operate by side-effect, they spoil
idepotent code, and they are implicit instead of explicit.
> def do_something_with(raw_record):
> return Record(*raw_record.split())
Much more sensible!
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
- Original Message -
> From: Steven D'Aprano
> To: tutor@python.org
> Cc:
> Sent: Friday, June 13, 2014 1:45 PM
> Subject: Re: [Tutor] global variables/constants versus volatile
> variables/constants
>
> On Fri, Jun 13, 2014 at 12:51:25PM +0530, diliup gabadamudalige wrote:
>> Hi All!
>> Hope everyone is well.
>>
>> In my code there are many dictionaries and lists which are used in various
>> functions. Is it better/pythonic/efficient to have these inside the
>> function itself or declared at the beginning of the program in which case
>> they will be global? They are all read only. I understand that global
>> constants and variable have memory allocated to them but when declared
>> inside a function are created on the fly, used and discarded. Please
>> enlighten me further on this and correct me if i'm wrong.
>
> A good question.
>
> Depending on the size of these dictionaries and lists, worrying about
> efficiency here may be premature optimization. As they say:
>
>
> "We should forget about small efficiencies, say about 97% of
> the time: premature optimization is the root of all evil."
> -- Donald Knuth
The other day I used collections.namedtuple and I re-initialized Record (see
below) with every function*) call. Bad idea!
It looks nicer because I did not need a global (and globals are baaad, mkay?),
but it was *much* slower. I processed a log of a few million lines, I think.
# bad --> time-consuming
import collections
def do_something_with(raw_record):
Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in
range(100)))
return Record(*raw_record.split())
# better --> even though it uses a global variable
import collections
Record = collections.namedtuple("_", " ".join("v%%03d" % i for i in range(100)))
def do_something_with(raw_record):
return Record(*raw_record.split())
*) the function contained more code, of course!
Albert-Jan
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On Fri, Jun 13, 2014 at 12:51:25PM +0530, diliup gabadamudalige wrote:
> Hi All!
> Hope everyone is well.
>
> In my code there are many dictionaries and lists which are used in various
> functions. Is it better/pythonic/efficient to have these inside the
> function itself or declared at the beginning of the program in which case
> they will be global? They are all read only. I understand that global
> constants and variable have memory allocated to them but when declared
> inside a function are created on the fly, used and discarded. Please
> enlighten me further on this and correct me if i'm wrong.
A good question.
Depending on the size of these dictionaries and lists, worrying about
efficiency here may be premature optimization. As they say:
"We should forget about small efficiencies, say about 97% of
the time: premature optimization is the root of all evil."
-- Donald Knuth
"The First Rule of Program Optimization: Don't do it. The
Second Rule of Program Optimization (for experts only!):
Don't do it yet." -- Michael A. Jackson
"More computing sins are committed in the name of efficiency
(without necessarily achieving it) than for any other single
reason — including blind stupidity." -- W.A. Wulf
If these lists and dicts are small, say, fewer than a dozen items, the
time to create and destroy them is probably trivial, especially if
you construct them from constant literals. In that case, it's a
matter of personal taste whether you prefer them as global constants or
local to a function.
But if it takes a long time to build the list, then you definitely
should move it outside the function and perform the initialisation step
only once:
# This may take a while...
PRIMES = [prime(i) for i in range(1, 101)]
If your lists really are read-only, then you should consider turning
them into tuples:
# Not this:
SOME_ITEMS = ["this", "that", "another"]
# Better:
SOME_ITEMS = ("this", "that", "another")
One advantage of the tuple is that in recent versions of Python, the
tuple may be compiled as a constant instead of being built at runtime.
This is from Python 2.7:
py> from dis import dis
py> code = compile("x = [2, 4, 8]", "", "exec")
py> dis(code)
1 0 LOAD_CONST 0 (2)
3 LOAD_CONST 1 (4)
6 LOAD_CONST 2 (8)
9 BUILD_LIST 3
12 STORE_NAME 0 (x)
15 LOAD_CONST 3 (None)
18 RETURN_VALUE
py> code = compile("x = (2, 4, 8)", "", "exec")
py> dis(code)
1 0 LOAD_CONST 4 ((2, 4, 8))
3 STORE_NAME 0 (x)
6 LOAD_CONST 3 (None)
9 RETURN_VALUE
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
diliup gabadamudalige Wrote in message: > there are many dictionaries and lists which are used in various functions. Is it better/pythonic/efficient to have these inside the function itself or declared at the beginning of the program in which case they will be global? They are all read only. I understand that global constants and variable have memory allocated to them but when declared inside a function are created on the fly, used and discarded. > (Please use text mail to post here, not html. Html messages can cause several kinds of problems in a text mailing list. ) If they are read-only, you don't want to waste time recreating them over and over. Neither do you want to be repeating the code that initializes them. So make them either global or a class member, and be sure to name them all caps to signify they shouldn't be altered. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
CCing the tutor list. Please use ReplyAll when responding to the list. Alan Gauld Author of the Learn To Program website http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos > > From: diliup gabadamudalige >To: Alan Gauld >Sent: Friday, 13 June 2014, 9:32 >Subject: Re: [Tutor] global variables/constants versus volatile >variables/constants > > > >thanks Alan! > >in the program that i am writing i used a class called variables and declare >all variables i use in the program in the def __init__ section. >so all variables which otherwise would have to be declared as global so that >they can be modified by other functions are in a class. >Is this ok? >The strings, lists and dictionaries which are not read only are used as locals >in functions and classes. >If there is a string, list or dict. which is used in several functions is it >better to use a global or several locals? > > > >On Fri, Jun 13, 2014 at 1:25 PM, Alan Gauld wrote: > >On 13/06/14 08:21, diliup gabadamudalige wrote: >> >> >>In my code there are many dictionaries and lists which are used in >>>various functions. Is it better/pythonic/efficient to have these inside >>>the function itself or declared at the beginning of the program in which >>>case they will be global? >>> >> If you are sharing a set of data structures between several functions thats often a sign they should be in a class. The data then becomes the class instance attributes and the functions become the methods. >> >> >> >>global constants and variable have memory allocated to them but when >>>declared inside a function are created on the fly, used and discarded. >>> >> That's not usually the main issue when deciding for global/local. >>Its more about the maintainability of the code and the over >>use of side effects which makes the code difficult to read. >>If the data is read only then things are not so bad and globals >>can be an OK solution. >> >>hth >>-- >>Alan G >>Author of the Learn to Program web site >>http://www.alan-g.me.uk/ >>http://www.flickr.com/photos/alangauldphotos >> >>___ >>Tutor maillist - Tutor@python.org >>To unsubscribe or change subscription options: >>https://mail.python.org/mailman/listinfo/tutor >> > > > >-- > >Diliup Gabadamudalige > >http://www.diliupg.com >http://soft.diliupg.com/ > >** >This e-mail is confidential. It may also be legally privileged. If you are not >the intended recipient or have received it in error, please delete it and all >copies from your system and notify the sender immediately by return e-mail. >Any unauthorized reading, reproducing, printing or further dissemination of >this e-mail or its contents is strictly prohibited and may be unlawful. >Internet communications cannot be guaranteed to be timely, secure, error or >virus-free. The sender does not accept liability for any errors or omissions. >** > > > >___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] global variables/constants versus volatile variables/constants
On 13/06/14 08:21, diliup gabadamudalige wrote: In my code there are many dictionaries and lists which are used in various functions. Is it better/pythonic/efficient to have these inside the function itself or declared at the beginning of the program in which case they will be global? If you are sharing a set of data structures between several functions thats often a sign they should be in a class. The data then becomes the class instance attributes and the functions become the methods. global constants and variable have memory allocated to them but when declared inside a function are created on the fly, used and discarded. That's not usually the main issue when deciding for global/local. Its more about the maintainability of the code and the over use of side effects which makes the code difficult to read. If the data is read only then things are not so bad and globals can be an OK solution. hth -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] global variables/constants versus volatile variables/constants
Hi All! Hope everyone is well. In my code there are many dictionaries and lists which are used in various functions. Is it better/pythonic/efficient to have these inside the function itself or declared at the beginning of the program in which case they will be global? They are all read only. I understand that global constants and variable have memory allocated to them but when declared inside a function are created on the fly, used and discarded. Please enlighten me further on this and correct me if i'm wrong. Thank you for your time and may you be well. -- Diliup Gabadamudalige http://www.diliupg.com http://soft.diliupg.com/ ** This e-mail is confidential. It may also be legally privileged. If you are not the intended recipient or have received it in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Any unauthorized reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ** ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
