Re: [Tutor] lists of lists: more Chutes Ladders!
On 31/12/2013 07:30, Keith Winston wrote: Never mind, I figured out that the slice assignment is emptying the previous lists, before the .reset() statements are creating new lists that I then populate and pass on. It makes sense. On Tue, Dec 31, 2013 at 12:59 AM, Keith Winston keithw...@gmail.com mailto:keithw...@gmail.com wrote: I resolved a problem I was having with lists, but I don't understand how! I caught my code inadvertently resetting/zeroing two lists TWICE at the invocation of the game method, and it was leading to all the (gamechutes gameladders) lists returned by that method being zeroed out except the final time the method is called. That is: the game method below is iterated iter times (this happens outside the method), and every time gamechutes and gameladders (which should be lists of all the chutes and ladders landed on during the game) were returned empty, except for the last time, in which case they were correct. I can see that doing the multiple zeroing is pointless, but I can't understand why it would have any effect on the returned values. Note that self.reset() is called in __init__, so the lists exist before this method is ever called, if I understand properly. def game(self, iter): Single game self.gamechutes[:] = [] #when I take out these two slice assignments, self.gameladders[:] = [] # then gamechutes gameladders work properly self.gamechutes = [] # these were actually in a call to self.reset() self.gameladders = [] # other stuff in reset() while self.position 100: gamecandl = self.move() if gamecandl[0] != 0: self.gamechutes.append(gamecandl[0]) if gamecandl[1] != 0: self.gameladders.append(gamecandl[1]) return [iter, self.movecount, self.numchutes, self.numladders, self.gamechutes,self.gameladders] I'm happy to share the rest of the code if you want it, though I'm pretty sure the problem lies here. If it's not obvious, I'm setting myself up to analyse chute ladder frequency: how often, in a sequence of games, one hits specific chutes ladders, and related stats. As always, any comments on style or substance are appreciated. Please intersperse or bottom post, top posting makes things very difficult to follow, especially in long threads. TIA. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists of lists: more Chutes Ladders!
On 12/31/2013 06:59 AM, Keith Winston wrote: I resolved a problem I was having with lists, but I don't understand how! I caught my code inadvertently resetting/zeroing two lists TWICE at the invocation of the game method, and it was leading to all the (gamechutes gameladders) lists returned by that method being zeroed out except the final time the method is called. That is: the game method below is iterated iter times (this happens outside the method), and every time gamechutes and gameladders (which should be lists of all the chutes and ladders landed on during the game) were returned empty, except for the last time, in which case they were correct. I can see that doing the multiple zeroing is pointless, but I can't understand why it would have any effect on the returned values. Note that self.reset() is called in __init__, so the lists exist before this method is ever called, if I understand properly. def game(self, iter): Single game self.gamechutes[:] = [] #when I take out these two slice assignments, self.gameladders[:] = [] # then gamechutes gameladders work properly self.gamechutes = [] # these were actually in a call to self.reset() self.gameladders = [] # other stuff in reset() while self.position 100: gamecandl = self.move() if gamecandl[0] != 0: self.gamechutes.append(gamecandl[0]) if gamecandl[1] != 0: self.gameladders.append(gamecandl[1]) return [iter, self.movecount, self.numchutes, self.numladders, self.gamechutes,self.gameladders] I'm happy to share the rest of the code if you want it, though I'm pretty sure the problem lies here. If it's not obvious, I'm setting myself up to analyse chute ladder frequency: how often, in a sequence of games, one hits specific chutes ladders, and related stats. As always, any comments on style or substance are appreciated. Well you have found the reason for weird behaviour, but there is one point you may have missed, I mean not fully understood, else you would probably never have coded that way. Imagine there is somewhere in your code, before the resetting of either, some reference to this list; then you empty this list using the first idiom, the emptying instruction with [:], like: original = [1, 2, 3] # ... ref_to_list = original # not a copy # ... original[:] = []# empty the same list # assert checks something is true; else we get an AssertionError: assert(ref_to_list is original) assert(ref_to_list == []) Now, replace the emptying instruction with: original = [] # assigns another list Now, assertions are false (both). `ref_to_list` actually still refers to the... original list; The folllowing is true in this version: assert(ref_to_list == [1,2,3]) But the symbol `original` itself is not bound to this orignal list anymore (it's now a liar! a misnomer). In this version, we do not *modify* the list element (to empty it, for the matter), but *replace* it by a new one (empty right from the start), bound to `original`. And as you ask for style comments, I would (this is subject to personal views): * use '_' for multi-word ids: game_chutes * say what the 'game' method computes and returns, in comment/doc and in its *name* (many think a true function, that produces a result, should be named after its product -- I share this view -- eg 'square' compute the square of its input) * 'iter' is a clear misnomer: I would have never guessed if you didn't say it in text: use eg n_iters or number_iters or such [*] ( Denis [*] In addition, iter is also the name of a builtin function, like print. It is rarely a good idea to reuse such names, but maybe you did not know it. Here is what it does (if you don't know yet about iterator objects, this will give you a fore-taste, but don't bother with that topic before you actually need them): spir@ospir:~$ python3 Python 3.3.1 (default, Sep 25 2013, 19:29:01) [GCC 4.7.3] on linux Type help, copyright, credits or license for more information. iter built-in function iter l = [1,2,3] it = iter(l) it list_iterator object at 0x7f5a63b41c50 for i in it: print(i) ... 1 2 3 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] can't install
On Sunday, December 29, 2013 9:11 PM, Tobias M. t...@tobix.eu wrote: prefix/bin/pyvenv ~/my_venv source ~/my_venv/bin/activate wget python-distribute.org/distribute_setup.py python distribute_setup.py Hi, can i ask why the name ~/my_venv/ .. is that just to indicate ~ as the home directory? so pyvenv already has access to virtualenv? i thought i would have needed pypi https://pypi.python.org/pypi/virtualenv or easy_install before i got install it, but here you create the virtualenv before getting distribute_setup.py. Everything seems fine now, at last! I really appreciate your help;)___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] same python script now running much slower
Thanks for all of your comments! I am working with human genome information which is in the form of many very short DNA sequence reads. I am using a script that sorts through all of these sequences and picks out ones that contain a particular sequence I'm interested in. Because my data set is so big, I have the data on an external hard drive (but that's where I had it before when it was faster too). As for how much slower it is running, I don't know because I keep having to move my computer before it is finished. The size of the data is the same, the script has not been modified, and the data is still in the same place. Essentially, I'm doing exactly what I did before (as a test) but it is now slower. How would I test your suggestion, Bill, that the script is paging itself to death? The data has not grown and I don't think the number of processes occupying memory has changed. By the way, I am using a Mac and I've tried two different computers. Thanks so much for all of your help! Meredith Sent from my iPhone On Dec 30, 2013, at 2:37 PM, William Ray Wing w...@mac.commailto:w...@mac.com wrote: On Dec 30, 2013, at 1:37 PM, Protas, Meredith prot...@vision.ucsf.edumailto:prot...@vision.ucsf.edu wrote: Hi, I'm very new to python so I'm sorry about such a basic question. I am using a python script generated by another person. I have used this script multiple times before and it takes around 24 hours to run. Recently, I have tried to run the script again (the same exact command lines) and it is much much slower. I have tried on two different computers with the same result. I used top to see if there were any suspicious functions that were happening but there seems to not be. I also ran another python script I used before and that went at the same speed as before so the problem seems unique to the first python script. Does anyone have any idea why it is so much slower now than it used to be (just around a month ago). Thanks for your help! Meredith Meredith, This is just a slight expansion on the note you received from Alan. Is there any chance that the script now is paging itself to death? That is, if you are reading a huge amount of data into a structure in memory, and if it no longer fits in available physical memory (either because the amount of data to be read has grown or the number of other processes that are occupying memory have grown), then that data structure may have gone virtual and the OS may be swapping it out to disk. That would dramatically increase the amount of elapsed wall time the program takes to run. If you can tell us more about what the program actually is doing or calculating, we might be able to offer more help. -Bill ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] same python script now running much slower
On Dec 31, 2013 12:43 PM, Protas, Meredith prot...@vision.ucsf.edu wrote: Thanks for all of your comments! I am working with human genome information which is in the form of many very short DNA sequence reads. I am using a script that sorts through all of these sequences and picks out ones that contain a particular sequence I'm interested in. Because my data set is so big, I have the data on an external hard drive (but that's where I had it before when it was faster too). Meredith, you really haven't given enough information for anyone to know what your program does and why it's slow. How long is your code? If you could just post the code here then you will likely get a much more helpful response. Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] subtyping builtin type
Hello, I don't remember exactly how to do that. As an example: class Source (str): __slots__ = ['i', 'n'] def __init__ (self, string): self.i = 0 # current matching index in source self.n = len(string)# number of ucodes (Unicode code points) #~ str.__init__(self, string) I thought I needed to call str's __init__, as in the line comented out, but (1) python refuses with a TypeError (2) all seems to work fine (meaning, the string is well stored, *implicitely*). Am I missing some point, or is this the way to do? How does it work? I particular, how does python know which param to take as source string? (There could be other params to __init__.) Thank you, Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Dec 31, 2013 2:37 PM, spir denis.s...@gmail.com wrote: Hello, I don't remember exactly how to do that. As an example: class Source (str): __slots__ = ['i', 'n'] def __init__ (self, string): self.i = 0 # current matching index in source self.n = len(string)# number of ucodes (Unicode code points) #~ str.__init__(self, string) I thought I needed to call str's __init__, as in the line comented out, but (1) python refuses with a TypeError (2) all seems to work fine (meaning, the string is well stored, *implicitely*). Am I missing some point, or is this the way to do? How does it work? I particular, how does python know which param to take as source string? (There could be other params to __init__.) The underlying built-in object is created by str.__new__ before __init__ is called. If you override __new__ you'll need to defer to str.__new__ and can choose to intercept the arguments and replace them. Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 8:35 AM, spir denis.s...@gmail.com wrote:
Hello,
I don't remember exactly how to do that. As an example:
class Source (str):
__slots__ = ['i', 'n']
def __init__ (self, string):
self.i = 0 # current matching index in source
self.n = len(string)# number of ucodes (Unicode code points)
#~ str.__init__(self, string)
I thought I needed to call str's __init__, as in the line comented out, but
(1) python refuses with a TypeError (2) all seems to work fine (meaning, the
string is well stored, *implicitely*). Am I missing some point, or is this
the way to do? How does it work? I particular, how does python know which
param to take as source string? (There could be other params to __init__.)
class Source(str):
... __slots__ = ['i', 'n']
... def __init__(self, string):
... self.i = 0
... self.n = len(string)
...
s = Source('testing')
s
'testing'
s.i
0
s.n
7
If you look at the repr of str.__init__, you'll see that it is
inherited from object:
str.__init__
slot wrapper '__init__' of 'object' objects
str.__init__ is object.__init__
True
Compare this to the __init__ of list, which is a mutable type:
list.__init__
slot wrapper '__init__' of 'list' objects
list.__init__ is not object.__init__
True
Being immutable, str uses __new__ to create a new str object; it
doesn't use __init__ at all. Since you're not overriding __new__ in
your subclass, you don't need to worry about calling str.__new__
because it's already done by the time Source.__init__ is called.
HTH,
--
Zach
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On 12/31/2013 04:03 PM, Zachary Ware wrote:
On Tue, Dec 31, 2013 at 8:35 AM, spir denis.s...@gmail.com wrote:
Hello,
I don't remember exactly how to do that. As an example:
class Source (str):
__slots__ = ['i', 'n']
def __init__ (self, string):
self.i = 0 # current matching index in source
self.n = len(string)# number of ucodes (Unicode code points)
#~ str.__init__(self, string)
I thought I needed to call str's __init__, as in the line comented out, but
(1) python refuses with a TypeError (2) all seems to work fine (meaning, the
string is well stored, *implicitely*). Am I missing some point, or is this
the way to do? How does it work? I particular, how does python know which
param to take as source string? (There could be other params to __init__.)
class Source(str):
... __slots__ = ['i', 'n']
... def __init__(self, string):
... self.i = 0
... self.n = len(string)
...
s = Source('testing')
s
'testing'
s.i
0
s.n
7
If you look at the repr of str.__init__, you'll see that it is
inherited from object:
str.__init__
slot wrapper '__init__' of 'object' objects
str.__init__ is object.__init__
True
Compare this to the __init__ of list, which is a mutable type:
list.__init__
slot wrapper '__init__' of 'list' objects
list.__init__ is not object.__init__
True
Being immutable, str uses __new__ to create a new str object; it
doesn't use __init__ at all. Since you're not overriding __new__ in
your subclass, you don't need to worry about calling str.__new__
because it's already done by the time Source.__init__ is called.
Thank you, Oscar Zachary. I guess thus the way it is done is correct (for my
case), is it? Seems your last remark shows the source of my confusion: probably,
in past times, I subtyped builtin types and overrided their __new__, thus had to
call the original one.
Would I have to do this if Source had other __init__ params? Or would it work
anyway provided the string remains 1st param? Or what else?
Denis
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 9:35 AM, spir denis.s...@gmail.com wrote: I[n] particular, how does python know which param to take as source string? (There could be other params to __init__.) You override __new__, and you might also have to override __init__, but not in this case. object.__init__ ignores the extra args because __new__ isn't object.__new__. Since you're using 3.x, str.__new__ can take optional arguments for bytes `encoding` and `errors`. You may want to preserve this feature as keyword arguments. For example: class Source(str): __slots__ = ['i', 'n'] def __new__(cls, s, i, n, **kwds): self = super(Source, cls).__new__(cls, s, **kwds) self.i = i self.n = n return self Source(b'abc', 0, 3) # probably wrong b'abc' Source(b'abc', 0, 3, encoding='ascii') 'abc' ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 9:22 AM, spir denis.s...@gmail.com wrote:
Thank you, Oscar Zachary. I guess thus the way it is done is correct (for
my case), is it? Seems your last remark shows the source of my confusion:
probably, in past times, I subtyped builtin types and overrided their
__new__, thus had to call the original one.
Would I have to do this if Source had other __init__ params? Or would it
work anyway provided the string remains 1st param? Or what else?
The interactive interpreter is great for this kind of thing, you know
;). I didn't know the answer to this question right off, so here's
what I tried:
class Source(str):
... __slots__ = ['i', 'n', 'a', 'k']
... def __init__(self, *args, **kwargs):
... self.i = 0
... self.n = len(self)
... self.a = args
... self.k = kwargs
...
s = Source('testing')
s
'testing'
s.i
0
s.n
7
s.a
('testing',)
s.k
{}
s = Source('testing', 'tester', b='kwarg test')
Traceback (most recent call last):
File interactive input, line 1, in module
TypeError: 'b' is an invalid keyword argument for this function
s = Source('testing', 'tester')
Traceback (most recent call last):
File interactive input, line 1, in module
TypeError: decoding str is not supported
So it seems the answer is: if you want Source to be able to take args
that str doesn't, you'll have to define __new__ and intercept the
arguments you want (or the arguments meant for str) and pass only
str's arguments to str.__new__ to get self. Depending on what you're
trying to do, it may turn out easier (and less confusing) to not
subclass str at all, and just keep a _string attribute for the string.
--
Zach
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On 31/12/2013 15:54, Zachary Ware wrote: On Tue, Dec 31, 2013 at 9:22 AM, spir denis.s...@gmail.com wrote: Thank you, Oscar Zachary. I guess thus the way it is done is correct (for my case), is it? Seems your last remark shows the source of my confusion: probably, in past times, I subtyped builtin types and overrided their __new__, thus had to call the original one. Would I have to do this if Source had other __init__ params? Or would it work anyway provided the string remains 1st param? Or what else? The interactive interpreter is great for this kind of thing, you know ;). Good point, i've been letting the newbies off with this one recently. class Source(str): ... __slots__ = ['i', 'n', 'a', 'k'] The glossary entry for __slots__ states A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminating instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved for rare cases where there are large numbers of instances in a memory-critical application. I'll admit that I really don't understand what's tricky about it, can someone explain please. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 10:21 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: The glossary entry for __slots__ states A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminating instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved for rare cases where there are large numbers of instances in a memory-critical application. I'll admit that I really don't understand what's tricky about it, can someone explain please. I'm not sure, exactly. My suspicion is that it can be rather annoying to make sure you don't add just one extra attribute that isn't in __slots__ (yet), or to make sure that you found every attribute you ever access or might need to access in every branch of every method of everything that uses the class or an instance thereof (since using __slots__ means you can't add anything to the class dynamically, unless you add __dict__ to __slots__ which defeats half the purpose). Also, it's probably meant to add just a faint touch of FUD since it really isn't necessary 95+% of the time, especially for newbies. -- Zach ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 11:21 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: The glossary entry for __slots__ states A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminating instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved for rare cases where there are large numbers of instances in a memory-critical application. I'll admit that I really don't understand what's tricky about it, can someone explain please. Refer to the language reference: http://docs.python.org/3/reference/datamodel.html#notes-on-using-slots Minor correction: It says str requires empty __slots__, but that's a bug in the docs. It's referring to 2.x str. Else this thread wouldn't exist. In 3.x, str is basically the unicode type from 2.x. Its __itemsize__ is 0 because the character array is allocated separately. Actually in CPython 3.3 there's a compact string object (i.e. PyCompactUnicodeObject), but that's not used for a subclass. Appending new slots to an instance poses no problem for a subclass of str. Regarding __class__ assignment, I'll add that it also breaks if the types include a __dict__ or __weakref__ slot in addition to other slots. For example, this works fine: class C: __slots__ = '__weakref__', class D: __slots__ = '__weakref__', C().__class__ = D But adding another slot pushes __weakref__ out of its expected position in the layout, so the test for a compatible layout fails: class C: __slots__ = '__weakref__', 'a' class D: __slots__ = '__weakref__', 'a' C().__class__ = D Traceback (most recent call last): File stdin, line 1, in module TypeError: __class__ assignment: 'C' object layout differs from 'D' The layout is identical, but the test it uses can't see that. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 11:53 AM, eryksun eryk...@gmail.com wrote: Minor correction: It says str requires empty __slots__, but that's a bug in the docs. It's referring to 2.x str. Else this thread wouldn't exist. In 3.x, str is basically the unicode type from 2.x. Its __itemsize__ is 0 because the character array is allocated separately. Actually in CPython 3.3 there's a compact string object (i.e. PyCompactUnicodeObject), but that's not used for a subclass. Appending new slots to an instance poses no problem for a subclass of str. It is still true for bytes objects, though. Thanks for pointing that out, it is now fixed. Regarding __class__ assignment, I'll add that it also breaks if the types include a __dict__ or __weakref__ slot in addition to other slots. For example, this works fine: class C: __slots__ = '__weakref__', class D: __slots__ = '__weakref__', C().__class__ = D But adding another slot pushes __weakref__ out of its expected position in the layout, so the test for a compatible layout fails: class C: __slots__ = '__weakref__', 'a' class D: __slots__ = '__weakref__', 'a' C().__class__ = D Traceback (most recent call last): File stdin, line 1, in module TypeError: __class__ assignment: 'C' object layout differs from 'D' The layout is identical, but the test it uses can't see that. Would you mind opening an issue for this? This looks like it may be fixable (or the doc should be updated). -- Zach ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On 12/31/2013 06:53 PM, eryksun wrote: On Tue, Dec 31, 2013 at 11:21 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: The glossary entry for __slots__ states A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminating instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved for rare cases where there are large numbers of instances in a memory-critical application. I'll admit that I really don't understand what's tricky about it, can someone explain please. Refer to the language reference: http://docs.python.org/3/reference/datamodel.html#notes-on-using-slots Minor correction: It says str requires empty __slots__, but that's a bug in the docs. It's referring to 2.x str. Else this thread wouldn't exist. In 3.x, str is basically the unicode type from 2.x. Its __itemsize__ is 0 because the character array is allocated separately. Actually in CPython 3.3 there's a compact string object (i.e. PyCompactUnicodeObject), but that's not used for a subclass. Appending new slots to an instance poses no problem for a subclass of str. Regarding __class__ assignment, I'll add that it also breaks if the types include a __dict__ or __weakref__ slot in addition to other slots. For example, this works fine: class C: __slots__ = '__weakref__', class D: __slots__ = '__weakref__', C().__class__ = D But adding another slot pushes __weakref__ out of its expected position in the layout, so the test for a compatible layout fails: class C: __slots__ = '__weakref__', 'a' class D: __slots__ = '__weakref__', 'a' C().__class__ = D Traceback (most recent call last): File stdin, line 1, in module TypeError: __class__ assignment: 'C' object layout differs from 'D' The layout is identical, but the test it uses can't see that. Oh, that's close to what i first thought when reading about the trickiness of using __slots__: that it may relate to subtyping supertypes with __slots__ (right, I should try myself, but here it's 21.28, on dec 31, time to move my ass... ;-) Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] ValueError: could not convert string to float: '13,2'
Hi, I'm trying to convert a list of strings to float. Unfortunately the numbers are written with a decimal comma instead of a decimal point. What is the best way to replace these commas with decimal points? Do I need to write a function that will iterate over every alphanumeric, replace the comma with a point and then rewrite the list, or is there a better way? Thank you, PierreD. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists of lists: more Chutes Ladders!
Thanks Denis, I found out about the iter builtin last night, a few hours after I'd coded/posted that. Oops. Thanks for your other comments, I am clearer now about the distinction of creating a new, empty list vs. clearing the same list out, and the subsequent implications on other symbols bound to the same list (is that the right language?). Not to beat a dead horse: you mention the name of the game method: in my code, game plays a game of Chutes Ladders (does a series of moves until the game is over), compiles the statistics from said game, and passes those, as a list of ints lists, to be gathered into a list of lists at the next level (games is the list of lists, composed of many game lists). I should absolutely document it better, but does that still not seem like a good name to you? Thanks for your feedback. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] ValueError: could not convert string to float: '13,2'
On 31/12/2013 13:53, Pierre Dagenais wrote: Hi, I'm trying to convert a list of strings to float. Unfortunately the numbers are written with a decimal comma instead of a decimal point. What is the best way to replace these commas with decimal points? Do I need to write a function that will iterate over every alphanumeric, replace the comma with a point and then rewrite the list, or is there a better way? Thank you, PierreD. Change your locale. import locale # Set to users preferred locale: locale.setlocale(locale.LC_ALL, '') # Or a specific locale: locale.setlocale(locale.LC_NUMERIC, en_DK.UTF-8) print(locale.atof(3,14)) See http://docs.python.org/3/library/locale.html#background-details-hints-tips-and-caveats for the background. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On 31/12/2013 17:53, eryksun wrote: On Tue, Dec 31, 2013 at 11:21 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: The glossary entry for __slots__ states A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminating instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved for rare cases where there are large numbers of instances in a memory-critical application. I'll admit that I really don't understand what's tricky about it, can someone explain please. Refer to the language reference: http://docs.python.org/3/reference/datamodel.html#notes-on-using-slots Not found on the windows CHM file. Looks like another bug report to keep our lazy, bone idle core developers like that Zach what's his name busy over the New Year :) Minor correction: It says str requires empty __slots__, but that's a bug in the docs. It's referring to 2.x str. Else this thread wouldn't exist. In 3.x, str is basically the unicode type from 2.x. Its __itemsize__ is 0 because the character array is allocated separately. Actually in CPython 3.3 there's a compact string object (i.e. PyCompactUnicodeObject), but that's not used for a subclass. Appending new slots to an instance poses no problem for a subclass of str. Regarding __class__ assignment, I'll add that it also breaks if the types include a __dict__ or __weakref__ slot in addition to other slots. For example, this works fine: class C: __slots__ = '__weakref__', class D: __slots__ = '__weakref__', C().__class__ = D But adding another slot pushes __weakref__ out of its expected position in the layout, so the test for a compatible layout fails: class C: __slots__ = '__weakref__', 'a' class D: __slots__ = '__weakref__', 'a' C().__class__ = D Traceback (most recent call last): File stdin, line 1, in module TypeError: __class__ assignment: 'C' object layout differs from 'D' The layout is identical, but the test it uses can't see that. Thank you for this detailed explanation. Happy New Year to your good self and everybody else. Let's hope 2014 is more code, less bugs. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] ValueError: could not convert string to float: '13,2'
Playing with this, a list comprehension is perfect: fnumlist = [float(num.replace(,, .)) for num in snumlist] ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] ValueError: could not convert string to float: '13,2'
Hi PierreD, I think if you iterate over your strings with something like this, it will do what you want, if I understand correctly (snum is your string number, like 123,321): fnum = float(snum.replace(,, .) keith: rank beginner, take everything with a grain of salt! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On 31/12/2013 21:11, Mark Lawrence wrote: On 31/12/2013 17:53, eryksun wrote: Refer to the language reference: http://docs.python.org/3/reference/datamodel.html#notes-on-using-slots Not found on the windows CHM file. Looks like another bug report to keep our lazy, bone idle core developers like that Zach what's his name busy over the New Year :) *facepalm* so used to using the index that I completely forgot about the search facility, first hit!!! -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists of lists: more Chutes Ladders!
On 12/31/2013 09:46 PM, Keith Winston wrote: Thanks Denis, I found out about the iter builtin last night, a few hours after I'd coded/posted that. Oops. Thanks for your other comments, I am clearer now about the distinction of creating a new, empty list vs. clearing the same list out, and the subsequent implications on other symbols bound to the same list (is that the right language?). Thanks, thus I did not spoil my time ;-) Not to beat a dead horse: you mention the name of the game method: in my code, game plays a game of Chutes Ladders (does a series of moves until the game is over), compiles the statistics from said game, and passes those, as a list of ints lists, to be gathered into a list of lists at the next level (games is the list of lists, composed of many game lists). I should absolutely document it better, but does that still not seem like a good name to you? Thanks for your feedback. In the part you showed us, unless my memory trumps myself, the method barely collected stat data _about_ the game. I did not look like representing the game's *playing* globally. But indeed, it was probably only a snippet. Well, actually the case looks ambiguous. If at all possible, i would separate in a sub-method the piece of code that compiles statistic data. Call it stats or game_stats or whatever looks clear and correct. The super-method that plays the game and, as an after-thought (even if it's not the case for you) calls this sub-method that collects data _about_ the game, may be called 'play'. This, because the whole of the app is, conceptually, the 'game', isn't it? or am I wrong on this point? Also, again conceptually, this method is an action (playing the game), thus a verb fits well as a name for it. A function that only computes new data (which is the case for stats) would requires a noun as name, the name of what it computes. But this is all rather personal and you may have a different perspective on all this. (However, this scheme of verbs for actions and nouns for functions works fine, I'm not inventing it; for this reason, I do recommend it as a consistent starting point; then you may later forge your own standard shortcuts, or adopt the ones of your preferred programming community). Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 03:35:55PM +0100, spir wrote: Hello, I don't remember exactly how to do that. As an example: class Source (str): __slots__ = ['i', 'n'] def __init__ (self, string): self.i = 0 # current matching index in source self.n = len(string)# number of ucodes (Unicode code points) #~ str.__init__(self, string) The easiest way to do that is: class Source(str): def __init__(self, *args, **kwargs): self.i = 0 self.n = len(self) As a (premature) memory optimization, you can use __slots__ to reduce the amount of memory per instance. But this (probably) is the wrong way to solve this problem. Your design makes Source a kind of string: issubclass(Source, str) = True I expect that it should not be. (Obviously I'm making some assumptions about the design here.) To decide whether you should use subclassing here, ask yourself a few questions: * Does it make sense to call string methods on Source objects? In Python 3.3, there are over 40 public string methods. If *just one* of them makes no sense for a Source object, then Source should not be a subclass of str. e.g. source.isnumeric(), source.isidentifier() * Do you expect to pass Source objects to arbitrary functions which expect strings, and have the result be meaningful? * Does it make sense for Source methods to return plain strings? source.upper() returns a str, not a Source object. * Is a Source thing a kind of string? If so, what's the difference between a Source and a str? Why not just use a str? If all you want is to decorate a string with a couple of extra pieces of information, then a limitation of Python is that you can only do so by subclassing. * Or does a Source thing *include* a string as a component part of it? If that is the case -- and I think it is -- then composition is the right approach. The difference between has-a and is-a relationships are critical. I expect that the right relationship should be: a Source object has a string rather than is a string. That makes composition a better design than inheritance. Here's a lightweight mutable solution, where all three attributes are public and free to be varied after initialisation: class Source: def __init__(self, string, i=0, n=None): if n is None: n = len(string) self.i = i self.n = n self.string = string An immutable solution is nearly as easy: from collections import namedtuple class Source(namedtuple(Source, string i n)): def __new__(cls, string, i=0, n=None): if n is None: n = len(string) return super(Source, cls).__new__(cls, string, i, n) Here's a version which makes the string attribute immutable, and the i and n attributes mutable: class Source: def __init__(self, string, i=0, n=None): if n is None: n = len(string) self.i = i self.n = n self._string = string @property def string(self): return self._string -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subtyping builtin type
On Tue, Dec 31, 2013 at 7:26 PM, Steven D'Aprano st...@pearwood.info wrote: from collections import namedtuple class Source(namedtuple(Source, string i n)): def __new__(cls, string, i=0, n=None): if n is None: n = len(string) return super(Source, cls).__new__(cls, string, i, n) namedtuple is a factory to create a tuple subclass that has properties that use operator.itemgetter, a __dict__ property that returns a collections.OrderedDict, plus the convenience functions _make and _replace. It also sets __slots__ = () to prevent instances from getting a dict. If you subclass a 2nd time, remember to also set __slots__ = (). But this only matters if you need better performance and smaller memory footprint when creating thousands of instances. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists of lists: more Chutes Ladders!
On Tue, Dec 31, 2013 at 4:27 AM, spir denis.s...@gmail.com wrote: In addition, iter is also the name of a builtin function, like print. While iter is a built-in function, it would be clearer if you referenced the __builtins__ namespace. Built-in objects are linked into the interpreter, either statically or from a shared library (.so, .pyd). But the __builtins__ namespace can and does include names for non-built-in objects (e.g. help and exit). The important point in this context is that iter is in the builtins module, not that it's a built-in function. There are lots of built-in objects that aren't in builtins. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists of lists: more Chutes Ladders!
On Tue, Dec 31, 2013 at 10:00:32PM -0500, eryksun wrote: On Tue, Dec 31, 2013 at 4:27 AM, spir denis.s...@gmail.com wrote: In addition, iter is also the name of a builtin function, like print. While iter is a built-in function, it would be clearer if you referenced the __builtins__ namespace. Don't use __builtins__! Firstly, it's an implementation detail only in CPython, not part of the language. So it doesn't exist in Jython or IronPython: Jython 2.5.1+ (Release_2_5_1, Aug 4 2010, 07:18:19) [OpenJDK Server VM (Sun Microsystems Inc.)] on java1.6.0_27 Type help, copyright, credits or license for more information. import __builtins__ Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named __builtins__ __builtins__ with-an-s is a crappy hack that has never worked correctly and has caused more confusion than help: https://mail.python.org/pipermail/python-3000/2007-March/006170.html Restricted mode in CPython has never worked correctly, __builtins__ has always been an implementation-specific detail, and you should never use it. The one you actually want is __builtin__ (no s) in Python 2, or builtins (no underscores) in Python 3. Built-in objects are linked into the interpreter, either statically or from a shared library (.so, .pyd). But the __builtins__ namespace can and does include names for non-built-in objects (e.g. help and exit). Only when running interactively, and only when site.py runs. If you remove or disable site.py, the help and exit functions don't get added. Actually, there's nothing special about site.py, anyone or anything could monkey-patch builtins. Don't do this: py spam # Does spam variable exist? Traceback (most recent call last): File stdin, line 1, in module NameError: name 'spam' is not defined py import builtins py builtins.spam = spam spam spam py del builtins py spam 'spam spam spam' The important point in this context is that iter is in the builtins module, not that it's a built-in function. There are lots of built-in objects that aren't in builtins. I'm afraid I've lost the context, and don't understand why this is important. It's true that not all built-in objects are in builtins, and not all objects in builtins are built-in, but other than for pedantic correctness, why does this matter? -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists of lists: more Chutes Ladders!
On Tue, Dec 31, 2013 at 10:53 PM, Steven D'Aprano st...@pearwood.info wrote:
__builtins__ with-an-s is a crappy hack that has never worked correctly
and has caused more confusion than help:
https://mail.python.org/pipermail/python-3000/2007-March/006170.html
Restricted mode in CPython has never worked correctly, __builtins__
has always been an implementation-specific detail, and you should never
use it. The one you actually want is __builtin__ (no s) in Python 2,
or builtins (no underscores) in Python 3.
But I didn't say to use the __builtins__ module. __builtin__ works
in Jython, but the module I referenced was builtins, per the 3.x
example given by Denis.
I'm definitely not talking about the weird hack that __builtins__ is a
module in __main__ in CPython. I've never understood the reason for
that, and I loathe it. In every other module it's a dict:
d = {}
exec(x = 1, d)
type(d['__builtins__'])
class 'dict'
I'm not familiar with the implementation of Jython's builtins
namespace. Using the name __builtins__ was based on CPython. Just to
be clear, I'm talking about the builtins namespace and scope --
however it's implemented in Jython or IronPython. I don't use them and
never plan to, so I'm grateful for the correction, Steven.
But the __builtins__ namespace can and does include names for
non-built-in objects (e.g. help and exit).
Only when running interactively, and only when site.py runs. If you
remove or disable site.py, the help and exit functions don't get added.
I'm aware of that, and have even mentioned it at least twice in the past.
The important point in this
context is that iter is in the builtins module, not that it's a
built-in function. There are lots of built-in objects that aren't in
builtins.
I'm afraid I've lost the context, and don't understand why this is
important. It's true that not all built-in objects are in builtins, and
not all objects in builtins are built-in, but other than for pedantic
correctness, why does this matter?
Denis said:
iter is also the name of a builtin function, like print
My point was that the issue with iter is a namespace issue. It doesn't
matter that it's a built-in function like print. Denis could have
meant that either way, so I thought it important to clarify. Why? A
while back there was a thread on the list confusing built-in
functions/methods in the io module and the builtins namespace, and
there was even an erroneous bug report filed on a doc string.
https://mail.python.org/pipermail/tutor/2013-June/096218.html
http://bugs.python.org/issue18249
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] Shelve immutable objects
I'm working my way slowly through Programming Python by Mark Lutz, and as an example of data persistence, he uses this example: -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shelve immutable objects
So sorry, I hit return: here's the example:
import shelve
db = shelve.open('class-shelve')
sue = db['sue']
sue.giveRaise(.25)
db['sue'] = sue
tom = db['tom']
tom.giveRaise(.20)
db['tom'] = tom
db.close()
Is it possible to dispense with the assignment/reassignment and just use
(open shelve)
db['sue'].giveRaise(.25)
db['sue'].giveRaise(.25)
(close shelve)
or is the assignment (or bounding?) necessary to unshelve/reshelve the
items...
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shelve immutable objects
According to:
http://docs.python.org/2/library/shelve.html
The shelve can be opened in 'writeback' mode, which I think might be
relevant to your question.
By default modified objects are written only when assigned to the
shelf (see Example). If the optional writebackparameter is set to
True, all entries accessed are also cached in memory, and written back
on sync() and close(); this can make it handier to mutate mutable
entries in the persistent dictionary, but, if many entries are
accessed, it can consume vast amounts of memory for the cache, and it
can make the close operation very slow since all accessed entries are
written back (there is no way to determine which accessed entries are
mutable, nor which ones were actually mutated).
Let's try it:
##
import shelve
db = shelve.open('class-shelve')
db['a-list'] = [1, 2, 3]
db.close()
db = shelve.open('class-shelve', writeback=True)
db['a-list'].append(four)
db.close()
db = shelve.open('class-shelve')
db['a-list']
[1, 2, 3, 'four']
##
So yes, you should be able to use a shelve in writeback mode to
automatically persist the mutable structures. That being said, the
docs do say to be aware of the implications: it means every accessed
entry's going to be re-persisted because the shelve does not really
watch for mutations: it just checks for access.
Happy New Year!
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shelve immutable objects
On Tue, Dec 31, 2013 at 11:01 PM, Keith Winston keithw...@gmail.com wrote: I'm working my way slowly through Programming Python by Mark Lutz, and as an example of data persistence, he uses this example: Ooops; the email got cut off a bit early. Can you try again? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists of lists: more Chutes Ladders!
On 01/01/2014 07:13 AM, eryksun wrote: I'm afraid I've lost the context, and don't understand why this is important. It's true that not all built-in objects are in builtins, and not all objects in builtins are built-in, but other than for pedantic correctness, why does this matter? Denis said: iter is also the name of a builtin function, like print My point was that the issue with iter is a namespace issue. It doesn't matter that it's a built-in function like print. Denis could have meant that either way, so I thought it important to clarify. Why? A while back there was a thread on the list confusing built-in functions/methods in the io module and the builtins namespace, and there was even an erroneous bug report filed on a doc string. https://mail.python.org/pipermail/tutor/2013-June/096218.html http://bugs.python.org/issue18249 You are very right, eryksyn, I was not clear at all, in fact it was not clear in my knowledge. My point was: `iter` the func exists in python (built-in with '-'), one may use it at times. Giving an application var this name hides, which accosionnally leads to bugs. I have been bitten by such a bug more than once in the past, and once hard to find, asp. with the built-in func `range` (a very tempting var name, isn't it?). Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
