Re: Tuples and immutability
On Tue, 11 Mar 2014 23:25:19 -0400, Terry Reedy wrote: On 3/11/2014 10:01 PM, Rick Johnson wrote: On Thursday, February 27, 2014 4:18:01 PM UTC-6, Ian wrote: x += y is meant to be equivalent, except possibly in-place and more efficient, than x = x + y. The manual actually says An augmented assignment expression like x += 1 can be rewritten as x = x + 1 to achieve a similar, but not exactly equal effect. In the augmented version, x is only evaluated once. Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object and assigning that to the target, the old object is modified instead. In an ideal world, the speed of these two codes should be the same, Nope, 'similar' is not 'equivalent'. Evaluating x twice instead of once and possibly allocating a new object versus not take extra time. In a statement like x.y.z[3*n+m] += 1, calculating the target dominates the time to increment, so this form should be nearly twice as fast. Excellent point Terry! I always forget that the target of an augmented assignment may not be a simple name like x but can be an arbitrary complex reference, anything that is a legal assignment target. Because += is documented as only evaluating the expression once it can behave quite differently to the `spam = spam + 1` case. Evaluating the right hand side may have side- effects that change what the left hand side evaluates to. This is not the case with the augmented assignment. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
On 03/11/2014 08:25 PM, Terry Reedy wrote: On 3/11/2014 10:01 PM, Rick Johnson wrote: On Thursday, February 27, 2014 4:18:01 PM UTC-6, Ian wrote: x += y is meant to be equivalent, except possibly in-place and more efficient, than x = x + y. The manual actually says An augmented assignment expression like x += 1 can be rewritten as x = x + 1 to achieve a similar, but not exactly equal effect. In the augmented version, x is only evaluated once. Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object and assigning that to the target, the old object is modified instead. ... and reassigned to the target. :) (If it doesn't say that last part, it should.) -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: beautiful soup get class info
Christopher Welborn wrote:
On 03/06/2014 02:22 PM, teddyb...@gmail.com wrote:
I am using beautifulsoup to get the title and date of the website.
title is working fine but I am not able to pull the date. Here is the
code in the url:
span class=dateOctober 22, 2011/span
In Python, I am using the following code:
date1 = soup.span.text
data=soup.find_all(date=value)
Results in:
[]
March 5, 2014
What is the proper way to get this info?
Thanks.
I believe it's the 'attrs' argument.
http://www.crummy.com/software/BeautifulSoup/bs4/doc/
# Workaround the 'class' problem:
data = soup.find_all(attrs={'class': 'date'})
I haven't tested it, but it's worth looking into.
Yes there are two ways to filtr by class:
soup = bs4.BeautifulSoup(
... span class=onealpha/span
... span class=twobeta/span)
Use attrs:
soup.find_all(attrs={class: one})
[span class=onealpha/span]
Append an underscore:
soup.find_all(class_=two)
[span class=twobeta/span]
--
https://mail.python.org/mailman/listinfo/python-list
Re: Balanced trees
Op 11-03-14 00:24, Roy Smith schreef: In article 8761nmrnfk@elektro.pacujo.net, Marko Rauhamaa ma...@pacujo.net wrote: Anyway, this whole debate is rather unnecessary since every developer is supposed to have both weapons in their arsenal. The problem with having a choice is that it opens up the possibility of making the wrong one :-) This is just a standard defense for the status quo. Introducing the decimal module also added a choice. As this discussion has shown, figuring out whether a hash table or a tree is better for a given problem is non-trivial. My guess is that if you gave 1000 typical developers both data structures and let them pick freely, the number of cases where it really mattered and the developer picked the right one would be approximately equal to the number of cases where they picked the wrong one. You are only illustrating one part. How about all those cases now where the wrong choice is more or less forced on the developer for lack of the alternative? -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
Op 12-03-14 07:28, Steven D'Aprano schreef:
On Tue, 11 Mar 2014 23:25:19 -0400, Terry Reedy wrote:
Nope, 'similar' is not 'equivalent'. Evaluating x twice instead of once
and possibly allocating a new object versus not take extra time. In a
statement like x.y.z[3*n+m] += 1, calculating the target dominates the
time to increment, so this form should be nearly twice as fast.
Excellent point Terry!
I always forget that the target of an augmented assignment may not be a
simple name like x but can be an arbitrary complex reference, anything
that is a legal assignment target. Because += is documented as only
evaluating the expression once it can behave quite differently to the
`spam = spam + 1` case. Evaluating the right hand side may have side-
effects that change what the left hand side evaluates to. This is not the
case with the augmented assignment.
The documentation is wrong at that point as the following code illustrates.
| import sys
| write = sys.stdout.write
|
| class logdict:
| def __init__(self):
| self.lst = {}
|
| def __setitem__(self, key, value):
| write('[%s] = %s\n' % (key, value))
| self.lst[key] = value
|
| def __getitem__(self, key):
| value = self.lst[key]
| write('[%s] = %s\n' % (key, value))
| return value
|
| tab = logdict()
| tab['key'] = 'value'
| tab['key'] += ' with extra tail'
| write('\n')
| tab = logdict()
| tab['key'] = 'value'
| tab['key'] = tab['key'] + ' with extra tail'
If you run this code, you get the following result:
| [key] = value
| [key] = value
| [key] = value with extra tail
|
| [key] = value
| [key] = value
| [key] = value with extra tail
As you can see there is no difference here in the evaluations done
between using
| tab['key'] += ' with extra tail'
or
| tab['key'] = tab['key'] + ' with extra tail'
--
https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
On Wed, Mar 12, 2014 at 12:28 AM, Steven D'Aprano st...@pearwood.info wrote: On Tue, 11 Mar 2014 23:25:19 -0400, Terry Reedy wrote: Nope, 'similar' is not 'equivalent'. Evaluating x twice instead of once and possibly allocating a new object versus not take extra time. In a statement like x.y.z[3*n+m] += 1, calculating the target dominates the time to increment, so this form should be nearly twice as fast. Excellent point Terry! I always forget that the target of an augmented assignment may not be a simple name like x but can be an arbitrary complex reference, anything that is a legal assignment target. Because += is documented as only evaluating the expression once it can behave quite differently to the `spam = spam + 1` case. Evaluating the right hand side may have side- effects that change what the left hand side evaluates to. This is not the case with the augmented assignment. Of course one could also do: z = x.y.z i = 3*n+m z[i] = z[i] + 1 which reduces the duplicated work down to storing and loading a couple of locals, and also prevents side-effects from affecting the LHS evaluation. -- https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
On Wed, Mar 12, 2014 at 3:39 AM, Antoon Pardon antoon.par...@rece.vub.ac.be wrote: The documentation is wrong at that point as the following code illustrates. Either way it still has to do a getitem and a setitem, but if you have a more nested structure then the extra getitems are not repeated. For example, using your logdict class: tab = logdict() tab[1] = logdict() [1] = __main__.logdict object at 0x02A2E430 tab[1][2] = logdict() [1] = __main__.logdict object at 0x02A2E430 [2] = __main__.logdict object at 0x02A2EB10 tab[1][2][3] = ['value'] [1] = __main__.logdict object at 0x02A2E430 [2] = __main__.logdict object at 0x02A2EB10 [3] = ['value'] tab[1][2][3] += [' with extra tail'] [1] = __main__.logdict object at 0x02A2E430 [2] = __main__.logdict object at 0x02A2EB10 [3] = ['value'] [3] = ['value', ' with extra tail'] versus: tab[1][2][3] = ['value'] [1] = __main__.logdict object at 0x02A2E430 [2] = __main__.logdict object at 0x02A2EB10 [3] = ['value'] tab[1][2][3] = tab[1][2][3] + [' with extra tail'] [1] = __main__.logdict object at 0x02A2E430 [2] = __main__.logdict object at 0x02A2EB10 [3] = ['value'] [1] = __main__.logdict object at 0x02A2E430 [2] = __main__.logdict object at 0x02A2EB10 [3] = ['value', ' with extra tail'] As you can see the += version does two fewer getitem calls in this case. -- https://mail.python.org/mailman/listinfo/python-list
Re: which async framework?
On Tue, Mar 11, 2014 at 5:38 PM, Chris Angelico ros...@gmail.com wrote: On Wed, Mar 12, 2014 at 10:18 AM, Marko Rauhamaa ma...@pacujo.net wrote: Chris Angelico ros...@gmail.com: What corner cases are there with threads that you don't have with anything else? There are numerous. Here's one example: deadlocks due to two threads taking locks in a different order. The problem crops up naturally with two intercommunicating classes. It can sometimes be very difficult to spot or avoid. Yep. Now how is that not a problem when you use some other model, like an event loop? The standard methods of avoiding deadlocks (like acquiring locks in strict order) work exactly the same for all models, and are just as necessary. If you don't have threads then the only locks you need to acquire are for external resources. You might still deadlock, but at least your process won't be deadlocking with itself. -- https://mail.python.org/mailman/listinfo/python-list
SMTPHandler and mail subject
Hi I use logging.handlers.SMTPHandler and i have tried to change subject of mail. Are there exists easy way to do that ? Subject of mail can change repeatedly depends on content of mail. Eras -- https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
On 12 March 2014 03:25, Terry Reedy tjre...@udel.edu wrote:
On 3/11/2014 10:01 PM, Rick Johnson wrote:
On Thursday, February 27, 2014 4:18:01 PM UTC-6, Ian wrote:
x += y is meant to be equivalent, except possibly in-place and
more efficient, than x = x + y.
The manual actually says An augmented assignment expression like x += 1 can
be rewritten as x = x + 1 to achieve a similar, but not exactly equal
effect. In the augmented version, x is only evaluated once. Also, when
possible, the actual operation is performed in-place, meaning that rather
than creating a new object and assigning that to the target, the old object
is modified instead.
In an ideal world, the speed of these two codes should be the same,
Nope, 'similar' is not 'equivalent'. Evaluating x twice instead of once and
possibly allocating a new object versus not take extra time. In a statement
like x.y.z[3*n+m] += 1, calculating the target dominates the time to
increment, so this form should be nearly twice as fast.
An example where the result is semantically different:
from numpy import array
a = array([1, 2, 3], dtype=int)
a
array([1, 2, 3])
a + 1.1
array([ 2.1, 3.1, 4.1])
a += 1.1
a
array([2, 3, 4])
The reason is that numpy arrays are both mutable and have statically
determined elementary data type:
(a + 1.1).dtype
dtype('float64')
a.dtype
dtype('int64')
Oscar
--
https://mail.python.org/mailman/listinfo/python-list
Re: SMTPHandler and mail subject
Eras, You have to override getSubject method of SMTPHandler. http://hg.python.org/cpython/file/677327810121/Lib/logging/handlers.py#l907 Thanks. Andriy Kornatskyy On Mar 12, 2014, at 12:08 PM, eras.rasmu...@gmail.com wrote: Hi I use logging.handlers.SMTPHandler and i have tried to change subject of mail. Are there exists easy way to do that ? Subject of mail can change repeatedly depends on content of mail. Eras -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On 03/11/2014 08:36 PM, Terry Reedy wrote:
On 3/11/2014 6:13 PM, John Gordon wrote:
In mailman.8062.1394573210.18130.python-l...@python.org Ethan Furman
et...@stoneleaf.us writes:
if missing:
raise ValueError('invoices %r missing from batch' % missing)
It's been a while since I wrote test cases, but I recall using the assert*
methods (assertEqual, assertTrue, etc.) instead of raising exceptions.
Perhaps that's the issue?
Yes. I believe the methods all raise AssertionError on failure, and the test
methods are wrapped with try:.. except
AssertionError as err:
if missing:
raise ValueError('invoices %r missing from batch' % missing)
should be assertEqual(missing, [], 'invoices missing from batch') and if that
fails, the non-empty list is printed
along with the message.
I've tried it both ways, and both ways my process is being killed, presumably
by the O/S.
I will say it's an extra motivating factor to have few failing tests -- if more than two of my tests fail, all I see are
'.'s, 'E's, and 'F's, with no clues as to which test failed nor why. Thank goodness for '-v' and being able to specify
which method of which class to run!
--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list
Re: which async framework?
Grant Edwards wrote: On 2014-03-11, Antoine Pitrou solip...@pitrou.net wrote: Sturla Molden sturla.molden at gmail.com writes: Chris Withers chris at simplistix.co.uk wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to learn a framework? You will find epoll and kqueue/kevent in the select module and iocp in pywin32. Yes, why use a library when you can rewrite it all yourself? Actually, you should probably issue system calls to the kernel directly, the libc is overrated (as is portability, I suppose). And don't bother with device drivers for the network adapters either. Just map their PCI regions in to user-space and twiddle the reigisters directly! ;) [I do that when testing PCI boards with C code, and one of these days I'm going to figure out how to do it with Python.] Use numpy with mmap -- https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
Op 12-03-14 10:51, Ian Kelly schreef: On Wed, Mar 12, 2014 at 3:39 AM, Antoon Pardon antoon.par...@rece.vub.ac.be wrote: The documentation is wrong at that point as the following code illustrates. Either way it still has to do a getitem and a setitem, but if you have a more nested structure then the extra getitems are not repeated. For example, using your logdict class: Sure, but the documentation doesn't say for sufficiently nested structures some evaluations are not repeated. Take the following example. | tab['key'] = [1] | tab['key'] += [2] Now let us rewrite that second statment in two different ways. | tab['key'] = tab['key'] + [2] or | tmp = tab['key'] | tmp += [2] Now which of these two rewritings is closer to the augmented concatenation? A natural reading of the documentation would conclude the second one, because in that case we only need to evaluate tab['key'] once as righthand sided. However it turns out the augmented concantenation is closer to the first rewriting here, evaluating tab['key'] twice once a lefthand sided and once as right hand sided, which IMO will surprise people that rely on the documentation. -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
locale getlocale returns None on OSX
Hi, locale.getlocale() sometimes returns (None, None) under OSX (Python 2, not sure about Python 3, but I think so). The problem is outlined here: http://stackoverflow.com/questions/1629699/locale-getlocale-problems-on-osx What is the cause of this? Is it limited to just Darwin systes? Does the 'horrible hack' solution on OS have any drawbacks? I like it better because it is not needed to set the LC_ALL environment variable prior to starting the Python program. Regards, Albert-Jan ~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~ -- https://mail.python.org/mailman/listinfo/python-list
Save to a file, but avoid overwriting an existing file
Hi! I would like to assure that when writing to a file I do not overwrite an existing file, but I'm unsure which is the best way to approach to this problem. As I can see, there are at least two possibilities: 1. I could use fd = os.open(x, os.O_WRONLY | os.O_CREAT | os.O_EXCL) which will fail - if the file exists. However, I would prefer if the program would try to save under different name in this case, instead of discarding all the calculation done until now - but I' not too well with catching exceptions. 2. Alternatively, a unique string could be generated to assure that no same file exists. I can see one approach to this is to include date and time in the file name. But this seems to me a bit clumsy, and is not unique, i.e. it could happen (at least in theory) that two processes finish in the same second. Any suggestions, please? -- https://mail.python.org/mailman/listinfo/python-list
Re: SMTPHandler and mail subject
It works. Thank you :) Eras -- https://mail.python.org/mailman/listinfo/python-list
Re: Save to a file, but avoid overwriting an existing file
This seems to be an application-level decision. If so, in your application, why not just check to see if the file exists, and implement whatever workaround you deem correct for your needs? For example (to choose a simple, but rather silly, file naming strategy): fname = x while os.path.exists(fname): fname = %s.%f % (fname, random.random()) fd = open(fname, w) It's clearly not going to be safe from race conditions, but I leave solving that problem as an exercise for the reader. Skip -- https://mail.python.org/mailman/listinfo/python-list
How do we pass default argument value to create thread object?
Hi, I am using Thread class to create threads. CODE thread = threading.Thread(target=Fun, args=[arg1, arg2, arg3=val]) thread.start() /CODE This code is throwing compilation error(Ipython). In [19]: import threading In [20]: def Fun(agr1, arg2, arg3=None): : pass : In [21]: thread = threading.Thread(target=Fun, args=[arg1, arg2, arg3=val ]) File ipython console, line 1 thread = threading.Thread(target=Fun, args=[arg1, arg2, arg3=val]) ^ SyntaxError: invalid syntax How do we pass the value to default arguments while creating thread object? Regards, ~Piyush Facebook https://www.facebook.com/piyushkv1 Twitterhttps://twitter.com/SocializePiyush -- https://mail.python.org/mailman/listinfo/python-list
Re: Save to a file, but avoid overwriting an existing file
On 2014-03-12 13:29, zoom wrote: 2. Alternatively, a unique string could be generated to assure that no same file exists. I can see one approach to this is to include date and time in the file name. But this seems to me a bit clumsy, and is not unique, i.e. it could happen (at least in theory) that two processes finish in the same second. Python offers a tempfile module that gives this (and a whole lot more) to you out of the box. -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
In article mailman.8082.1394620172.18130.python-l...@python.org,
Ethan Furman et...@stoneleaf.us wrote:
I've tried it both ways, and both ways my process is being killed, presumably
by the O/S.
What evidence do you have the OS is killing the process?
Some systems have an oom (Out Of Memory) process killer, which nukes
(semi-random) process when the system exhausts memory. Is it possible
this is happening? If so, you should see some log message in one of
your system logs.
You didn't mention (or maybe I misssed it) which OS you're using. I'm
assuming you've got some kind of system call tracer (strace, truss,
dtrace, etc). Try running your tests under that. If something is
sending your process a kill signal, you'll see it:
[gazillions of lines elided]
write(1, , 4 ) = 4
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
select(1, [0], NULL, NULL, NULL)= ? ERESTARTNOHAND (To be
restarted)
--- SIGTERM (Terminated) @ 0 (0) ---
+++ killed by SIGTERM +++
Alternatively, maybe something inside your process is just calling
sys.exit(), or even os._exit(). You'll see the exit() system call in
the strace output.
And, of course, the standard suggestion to reduce this down to the
minimum test case. You posted:
def test_xxx_1(self):
p = self.pp.lockbox_payments[0]
# affirm we have what we're expecting
self.assertEqual(
(p.payer, p.ck_num, p.credit),
('a customer', '010101', 1),
)
self.assertEqual(p.invoices.keys(), ['XXX'])
self.assertEqual(p.invoices.values()[0].amount, 1)
# now make sure we get back what we're expecting
np, b = self.pp._match_invoices(p)
missing = []
for inv_num in ('123456', '789012', '345678'):
if inv_num not in b:
missing.append(inv_num)
if missing:
raise ValueError('invoices %r missing from batch' % missing)
what happens if you reduce that to:
def test_xxx_1(self):
self.fail()
do you still get this strange behavior? What if you get rid of your
setUpClass()? Keep hacking away at the test suite until you get down to
a single line of code which, if run, exhibits the behavior, and if
commented out, does not. At that point, you'll have a clue what's
causing this. If you're lucky :-)
--
https://mail.python.org/mailman/listinfo/python-list
Re: How do we pass default argument value to create thread object?
- Original Message -
Hi,
I am using Thread class to create threads.
CODE
thread = threading.Thread(target=Fun, args=[arg1, arg2, arg3=val])
thread.start()
/CODE
This code is throwing compilation error(Ipython).
In [19]: import threading
In [20]: def Fun(agr1, arg2, arg3=None):
: pass
:
In [21]: thread = threading.Thread(target=Fun, args=[arg1, arg2,
arg3=val ])
File ipython console, line 1
thread = threading.Thread(target=Fun, args=[arg1, arg2, arg3=val])
^
SyntaxError: invalid syntax
How do we pass the value to default arguments while creating thread
object?
Regards,
~Piyush
Facebook Twitter
Hi,
try something like
thread = threading.Thread(target=Fun, args=(val1, val2),
kwargs={arg3:val3})
This will call
Fun(val1, val2, arg3=val3)
Browse the python tutorial for details on arguments and keyword arguments.
Cheers,
JM
-- IMPORTANT NOTICE:
The contents of this email and any attachments are confidential and may also be
privileged. If you are not the intended recipient, please notify the sender
immediately and do not disclose the contents to any other person, use it for
any purpose, or store or copy the information in any medium. Thank you.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How do we pass default argument value to create thread object?
On Wed, Mar 12, 2014 at 8:09 AM, Piyush Verma 114piy...@gmail.com wrote:
Hi,
I am using Thread class to create threads.
CODE
thread = threading.Thread(target=Fun, args=[arg1, arg2, arg3=val])
thread.start()
/CODE
This code is throwing compilation error(Ipython).
In [19]: import threading
In [20]: def Fun(agr1, arg2, arg3=None):
: pass
:
In [21]: thread = threading.Thread(target=Fun, args=[arg1, arg2,
arg3=val])
File ipython console, line 1
thread = threading.Thread(target=Fun, args=[arg1, arg2, arg3=val])
^
SyntaxError: invalid syntax
How do we pass the value to default arguments while creating thread object?
Use the 'kwargs' argument:
thread = threading.Thread(target=Fun, args=[1, 2], kwargs={'arg3': val})
See http://docs.python.org/3.4/library/threading#threading.Thread for more info.
Hope this helps,
--
Zach
--
https://mail.python.org/mailman/listinfo/python-list
Deep vs. shallow copy?
I think I understand the difference between deep vs. shallow copies but
I was bitten by this:
with open(os.path.join('path', 'foo.txt', 'rb') as txt:
reader = csv.reader(txt)
data = [row.append(year) for row in reader]
This does not work although the append does complete. The below works:
with open(os.path.join('path', 'foo.txt', 'rb') as txt:
reader = csv.reader(txt)
data = [row + [year] for row in reader]
However in this context I am baffled. If someone can explain what is
going on here, I would be most grateful.
Alex van der Spek
--
https://mail.python.org/mailman/listinfo/python-list
Re: Deep vs. shallow copy?
On Wed, Mar 12, 2014 at 9:25 AM, Alex van der Spek zd...@xs4all.nl wrote:
with open(os.path.join('path', 'foo.txt', 'rb') as txt:
reader = csv.reader(txt)
data = [row.append(year) for row in reader]
Forget deep v. shallow copies. What is the value of the variable year?
And why would you expect list.append to return anything?
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: Deep vs. shallow copy?
On Wed, Mar 12, 2014 at 9:25 AM, Alex van der Spek zd...@xs4all.nl wrote:
I think I understand the difference between deep vs. shallow copies but
I was bitten by this:
with open(os.path.join('path', 'foo.txt', 'rb') as txt:
reader = csv.reader(txt)
data = [row.append(year) for row in reader]
This does not work although the append does complete. The below works:
with open(os.path.join('path', 'foo.txt', 'rb') as txt:
reader = csv.reader(txt)
data = [row + [year] for row in reader]
However in this context I am baffled. If someone can explain what is
going on here, I would be most grateful.
Deep/shallow copying doesn't really come into this. row.append()
mutates the list (row), it doesn't return a new list. Like most
in-place/mutating methods in Python, it returns None instead of self
to show that mutation was done, so your listcomp fills `data` with
Nones; there is no copying done at all. The second example works as
you expected because `row + [year]` results in a new list, which the
listcomp is happy to append to `data`--which does mean that `row` is
copied.
To avoid the copy that the second listcomp is doing (which really
shouldn't be necessary anyway, unless your rows are astronomically
huge), you have a couple of options. First, you can expand your
listcomp and use append:
with open(os.path.join('path', 'foo.txt'), 'rb') as txt: # with
your typo fixed ;)
reader = csv.reader(txt)
data = []
for row in reader:
row.append(year)
data.append(row)
To me, that's pretty readable and pretty clear about what it's doing.
Then there's this option, which I don't recommend:
import operator
with open(os.path.join('path', 'foo.txt'), 'rb') as txt:
reader = csv.reader(txt)
data = [operator.iadd(row, [year]) for row in reader]
This works because operator.iadd is basically shorthand for
row.__iadd__([year]), which does return self (otherwise, the
assignment part of `row += [year]` couldn't work). But, it's not as
clear about what's happening, and only saves a whole two lines (maybe
3 if you already have operator imported).
Hope this helps,
--
Zach
--
https://mail.python.org/mailman/listinfo/python-list
Re: Deep vs. shallow copy?
On Wed, 12 Mar 2014 10:00:09 -0500, Zachary Ware wrote:
On Wed, Mar 12, 2014 at 9:25 AM, Alex van der Spek zd...@xs4all.nl
wrote:
I think I understand the difference between deep vs. shallow copies but
I was bitten by this:
with open(os.path.join('path', 'foo.txt', 'rb') as txt:
reader = csv.reader(txt)
data = [row.append(year) for row in reader]
This does not work although the append does complete. The below works:
with open(os.path.join('path', 'foo.txt', 'rb') as txt:
reader = csv.reader(txt)
data = [row + [year] for row in reader]
However in this context I am baffled. If someone can explain what is
going on here, I would be most grateful.
Deep/shallow copying doesn't really come into this. row.append()
mutates the list (row), it doesn't return a new list. Like most
in-place/mutating methods in Python, it returns None instead of self to
show that mutation was done, so your listcomp fills `data` with Nones;
there is no copying done at all. The second example works as you
expected because `row + [year]` results in a new list, which the
listcomp is happy to append to `data`--which does mean that `row` is
copied.
To avoid the copy that the second listcomp is doing (which really
shouldn't be necessary anyway, unless your rows are astronomically
huge), you have a couple of options. First, you can expand your
listcomp and use append:
with open(os.path.join('path', 'foo.txt'), 'rb') as txt: # with
your typo fixed ;)
reader = csv.reader(txt)
data = []
for row in reader:
row.append(year)
data.append(row)
To me, that's pretty readable and pretty clear about what it's doing.
Then there's this option, which I don't recommend:
import operator
with open(os.path.join('path', 'foo.txt'), 'rb') as txt:
reader = csv.reader(txt)
data = [operator.iadd(row, [year]) for row in reader]
This works because operator.iadd is basically shorthand for
row.__iadd__([year]), which does return self (otherwise, the assignment
part of `row += [year]` couldn't work). But, it's not as clear about
what's happening, and only saves a whole two lines (maybe 3 if you
already have operator imported).
Hope this helps,
Thank you, that helped immensely!
Having been taught programming in Algol60 Python still defeats me at times!
Particularly since Algol60 wasn't very long lived and what came
thereafter (FORTRAN) much worse.
I get it now, the below is equivalent!
I am perfectly happy with the one copy of the list row + [year].
Just wanted to learn something here and I have!
Python 2.6.5 (r265:79063, Feb 27 2014, 19:44:14)
[GCC 4.4.3] on linux2
Type help, copyright, credits or license for more information.
a = [1,2,3]
b = 'val'
a.append(b)
a
[1, 2, 3, 'val']
c = a.append(b)
print c
None
--
https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On 03/12/2014 06:44 AM, Roy Smith wrote:
In article mailman.8082.1394620172.18130.python-l...@python.org,
Ethan Furman et...@stoneleaf.us wrote:
I've tried it both ways, and both ways my process is being killed, presumably
by the O/S.
What evidence do you have the OS is killing the process?
I put a bare try/except around the call to unittest.main, with a print
statement in the except, and nothing ever prints.
Some systems have an oom (Out Of Memory) process killer, which nukes
(semi-random) process when the system exhausts memory. Is it possible
this is happening? If so, you should see some log message in one of
your system logs.
That would explain why my editor windows were being killed.
You didn't mention (or maybe I misssed it) which OS you're using.
Ubuntu 13 something or other.
I'm
assuming you've got some kind of system call tracer (strace, truss,
dtrace, etc).
Sadly, I have no experience with those programs yet, and until now didn't even
know they existed.
Try running your tests under that. If something is
sending your process a kill signal, you'll see it:
[gazillions of lines elided]
write(1, , 4 ) = 4
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
select(1, [0], NULL, NULL, NULL)= ? ERESTARTNOHAND (To be
restarted)
--- SIGTERM (Terminated) @ 0 (0) ---
+++ killed by SIGTERM +++
Alternatively, maybe something inside your process is just calling
sys.exit(), or even os._exit(). You'll see the exit() system call in
the strace output.
My bare try/except would have caught that.
And, of course, the standard suggestion to reduce this down to the
minimum test case. You posted:
def test_xxx_1(self):
p = self.pp.lockbox_payments[0]
# affirm we have what we're expecting
self.assertEqual(
(p.payer, p.ck_num, p.credit),
('a customer', '010101', 1),
)
self.assertEqual(p.invoices.keys(), ['XXX'])
self.assertEqual(p.invoices.values()[0].amount, 1)
# now make sure we get back what we're expecting
np, b = self.pp._match_invoices(p)
missing = []
for inv_num in ('123456', '789012', '345678'):
if inv_num not in b:
missing.append(inv_num)
if missing:
raise ValueError('invoices %r missing from batch' % missing)
what happens if you reduce that to:
def test_xxx_1(self):
self.fail()
I only get the strange behavior if more than two (or maybe three) of my test cases fail. Less than that magic number,
and everything works just fine. It doesn't matter which two or three, either.
do you still get this strange behavior? What if you get rid of your
setUpClass()? Keep hacking away at the test suite until you get down to
a single line of code which, if run, exhibits the behavior, and if
commented out, does not. At that point, you'll have a clue what's
causing this. If you're lucky :-)
I strongly suspect it's memory. When I originally wrote the code I tried to include six months worth of EoM data, but
had to back it down to three as my process kept mysteriously dying at four or more months. There must be waaay too
much stuff being kept alive by the stack traces of the failed tests.
Thanks for your help!
--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to create an instance of a python class from C++
Barry Scott, 11.03.2014 22:37: On 5 Mar 2014, at 00:14, Bill wrote: I can't figure out how to create an instance of a python class from 'C++': Why not use pycxx from http://sourceforge.net/projects/cxx/? This lib does all the heavy lifting for you for both python2 and python3. Has docs and examples. Yes, tool support definitely helps here. I was going to suggest Cython (also for obvious reasons), where the code that the OP posted would look like this: def RegisterClass(class_decl): an = type(class_decl)() print(an.description()) return 0 Clearly substantially simpler than the posted C code (and certainly safer, faster and more correct) - although that doesn't really help me much with understanding what the intention of this code is, looks rather weird... Stefan -- https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
In article mailman.8095.1394640002.18130.python-l...@python.org, Ethan Furman et...@stoneleaf.us wrote: Alternatively, maybe something inside your process is just calling sys.exit(), or even os._exit(). You'll see the exit() system call in the strace output. My bare try/except would have caught that. A bare except would catch sys.exit(), but not os._exit(). Well, no that's not actually true. Calling os._exit() will raise: TypeError: _exit() takes exactly 1 argument (0 given) but it won't catch os._exit(0) :-) what happens if you reduce that to: def test_xxx_1(self): self.fail() I only get the strange behavior if more than two (or maybe three) of my test cases fail. Less than that magic number, and everything works just fine. It doesn't matter which two or three, either. OK, well, assuming this is a memory problem, what if you do: def test_xxx_1(self): l = [] while True: l.append(0) That should eventually run out of memory. Does that get you the same behavior in a single test case? If so, that at least would be evidence supporting the memory exhaustion theory. I strongly suspect it's memory. When I originally wrote the code I tried to include six months worth of EoM data, but had to back it down to three as my process kept mysteriously dying at four or more months. There must be waaay too much stuff being kept alive by the stack traces of the failed tests. One thing you might try is running your tests under nose (http://nose.readthedocs.org/). Nose knows how to run unittest tests, and one of the gazillion options it has is to run each test case in an isolated process: --process-restartworker If set, will restart each worker process once their tests are done, this helps control memory leaks from killing the system. [NOSE_PROCESS_RESTARTWORKER] that might be what you need. -- https://mail.python.org/mailman/listinfo/python-list
Re:Save to a file, but avoid overwriting an existing file
zoom z...@yahoo.com Wrote in message: Hi! I would like to assure that when writing to a file I do not overwrite an existing file, but I'm unsure which is the best way to approach to this problem. As I can see, there are at least two possibilities: 1. I could use fd = os.open(x, os.O_WRONLY | os.O_CREAT | os.O_EXCL) which will fail - if the file exists. However, I would prefer if the program would try to save under different name in this case, instead of discarding all the calculation done until now - but I' not too well with catching exceptions. The tempfile module is your best answer, but if you really need to keep the file afterwards, you'll have the same problem when you rename it later. I suggest you learn about try/except. For simple cases it's not that tough, though if you want to ask about it, you'll need to specify your Python version. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re: Deep vs. shallow copy?
On 12 Mar 2014 15:29:59 GMT, Alex van der Spek zd...@xs4all.nl wrote: On Wed, 12 Mar 2014 10:00:09 -0500, Zachary Ware wrote: On Wed, Mar 12, 2014 at 9:25 AM, Alex van der Spek zd...@xs4all.nl wrote: === 8 === Having been taught programming in Algol60 Python still defeats me at times! Particularly since Algol60 wasn't very long lived and what came thereafter (FORTRAN) much worse. Actually, Algol 60 lived on (and lives on, though not so much used now outside of Scandinavia) in an improved and OOP-extended form in Simula 67 (now just Simula). Most implementations excpt that for DEC-System10 were, however, overpriced and poorly marketed, so we had to wait for C++ (improved and OOP-extended C) for OOP to catch on. === 8 === -- https://mail.python.org/mailman/listinfo/python-list
What does gc.get_objects() return?
Hi all. I was wondering if someone could explain gc.get_objects() in a bit more detail to me. Does it return a list of 'all objects known to Python'? Only some of them? Which does it return? Which it does not? For example (done using CPython 3.4 interactive interpreter) here it does not contain an object I know exists: a = object() a in gc.get_objects() False but here it does: class Foo: pass ... a = Foo() a in gc.get_objects() True It also does not seem to contain string objects. At first I thought that perhaps interned strings were not placed under GC control, so I tested this: a = asdkjfhk23498237$(*$*($ksjdhfkjsd b = asdkjfhk23498237$(*$*($ksjdhfkjsd a is b False a in gc.get_objects() False b in gc.get_objects() False Which shows that non-interned scripts such as these are not getting listed in gc.get_objects() either. :-s Many thanks. Best regards, Jurko Gospodnetić -- https://mail.python.org/mailman/listinfo/python-list
Re: Save to a file, but avoid overwriting an existing file
On 3/12/2014 5:29 AM, zoom wrote: 2. Alternatively, a unique string could be generated to assure that no same file exists. I can see one approach to this is to include date and time in the file name. But this seems to me a bit clumsy, and is not unique, i.e. it could happen (at least in theory) that two processes finish in the same second. I tend to use this method -- prepending the job name or targeting different directories per job precludes duplication. Unless you're running the same job at the same time, in which case tempfile is the way to go (which I use for archiving spooled print files which can occur simultaneously.) Emile -- https://mail.python.org/mailman/listinfo/python-list
Re: locale getlocale returns None on OSX
From: Albert-Jan Roskam fo...@yahoo.com
To: Python python-list@python.org
Sent: Wednesday, March 12, 2014 1:22 PM
Subject: locale getlocale returns None on OSX
Hi,
locale.getlocale() sometimes returns (None, None) under OSX (Python 2, not
sure about Python 3, but I think so). The problem is outlined here:
http://stackoverflow.com/questions/1629699/locale-getlocale-problems-on-osx
What is the cause of this? Is it limited to just Darwin systes? Does the
'horrible hack' solution on OS have any drawbacks? I like it better because it
is not needed to set the LC_ALL environment variable prior to starting the
Python program.
Regards,
Albert-Jan
Ok, here are some tests on my own system:
albertjan@debian:~$ uname -a
Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.54-2 x86_64 GNU/Linux
Even if locale.setlocale is used first, OSX returns (None, None) when
locale.getlocale() is called. Another thing that surprises me in the examples
below is the output of the python -c example using Python 2.7. Isn't this
supposed to be exactly equivalent to the code that follows?
# Python 2.7 albertjan@debian:~$ python -c import locale;
locale.setlocale(locale.LC_ALL, ); print(locale.getlocale())
(None, None) # why is this?
albertjan@debian:~$ python
Python 2.7.3 (default, Jan 2 2013, 13:56:14)
[GCC 4.7.2] on linux2
Type help, copyright, credits or license for more information.
import locale
locale.setlocale(locale.LC_ALL, )
'en_US.UTF-8'
locale.getlocale()
('en_US', 'UTF-8') # --- OSX (sometimes?) returns (None, None) here.
# Python 3.3
albertjan@debian:~$ python3 -c import locale; locale.setlocale(locale.LC_ALL,
); print(locale.getlocale())
('en_US', 'UTF-8')
albertjan@debian:~$ python3
Python 3.3.4 (default, Feb 17 2014, 19:23:00)
[GCC 4.7.2] on linux
Type help, copyright, credits or license for more information.
import locale
locale.setlocale(locale.LC_ALL, )
'en_US.UTF-8'
locale.getlocale()
('en_US', 'UTF-8')
# Pypy
albertjan@debian:~$ pypy -c import locale; locale.setlocale(locale.LC_ALL,
); print locale.getlocale()
(None, None)
albertjan@debian:~$ pypy
Python 2.7.3 (87aa9de10f9ca71da9ab4a3d53e0ba176b67d086, Mar 10 2014, 14:07:15)
[PyPy 2.2.1 with GCC 4.7.2] on linux2
Type help, copyright, credits or license for more information.
And now for something completely different: ``pypy is a better kind of
foolishness - lac''
import locale
locale.setlocale(locale.LC_ALL, )
'en_US.UTF-8'
locale.getlocale()
('en_US', 'UTF-8')
--
https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
On Wed, Mar 12, 2014 at 12:34 PM, Jurko Gospodnetić jurko.gospodne...@pke.hr wrote: Hi all. I was wondering if someone could explain gc.get_objects() in a bit more detail to me. Does it return a list of 'all objects known to Python'? Only some of them? Which does it return? Which it does not? For example (done using CPython 3.4 interactive interpreter) here it does not contain an object I know exists: a = object() a in gc.get_objects() False but here it does: class Foo: pass ... a = Foo() a in gc.get_objects() True I don't know why some objects aren't showing up in get_objects, but here's a quick note to say it's not just 3.4: $ ./pythons 'import gc; var=data; print(var in gc.get_objects())' /usr/local/cpython-2.4/bin/python [6600 refs] False /usr/local/cpython-2.5/bin/python False [7536 refs] /usr/local/cpython-2.6/bin/python False [15233 refs] /usr/local/cpython-2.7/bin/python False [18526 refs] /usr/local/cpython-3.0/bin/python Traceback (most recent call last): File string, line 1, in module File /usr/local/cpython-3.0/lib/python3.0/_weakrefset.py, line 121, in __eq__ return self.data == set(ref(item) for item in other) File /usr/local/cpython-3.0/lib/python3.0/_weakrefset.py, line 121, in genexpr return self.data == set(ref(item) for item in other) TypeError: cannot create weak reference to 'str' object [34510 refs] /usr/local/cpython-3.1/bin/python False [37270 refs] /usr/local/cpython-3.2/bin/python False [37700 refs] /usr/local/cpython-3.3/bin/python False [42662 refs] /usr/local/cpython-3.4/bin/python False /usr/local/pypy-2.2/bin/pypy True /usr/local/jython-2.7b1/bin/jython Traceback (most recent call last): File string, line 1, in module NotImplementedError: not applicable to Java GC Interesting that it is present in Pypy. -- https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
On 2014-03-12 19:34, Jurko Gospodnetić wrote: Hi all. I was wondering if someone could explain gc.get_objects() in a bit more detail to me. Does it return a list of 'all objects known to Python'? Only some of them? Which does it return? Which it does not? [snip] gc.is_tracked(...) might be relevant to your question. -- https://mail.python.org/mailman/listinfo/python-list
Re: locale getlocale returns None on OSX
In article 1394626979.46880.yahoomailba...@web163806.mail.gq1.yahoo.com, Albert-Jan Roskam fo...@yahoo.com wrote: locale.getlocale() sometimes returns (None, None) under OSX (Python 2, not sure about Python 3, but I think so). The problem is outlined here: http://stackoverflow.com/questions/1629699/locale-getlocale-problems-on-osx Python's locale uses the plaform's locale implementation and POSIX locale is an old and somewhat weird animal. Make sure you thoroughly read the description of the locale module, in particular, the caveats section: http://docs.python.org/2/library/locale.html#background-details-hints-tip s-and-caveats The first gotcha is that you need to explicitly call locale.setlocale(LC_ALL,) to get the preferred locale environment, as either specified by the user, by LANG or other LC_* environment variables, or the platform defaults. In general, OS X does not provide a default for your process. However, various environments do. The OS X Terminal.app has profile settings (Preferences - Settings - (Profile name) - Advanced) to specific a character encoding and a checkbox to Set locale environment variables on startup. With that set properly, programs run under Terminal.app will see LANG set. The user can also set an explicit LANG value in a shell profile, like .profile or .bashrc, but those only apply when a shell is being used. Depending on which profile it is set in, that might not work under all conditions, like under sudo or in an ssh session. Setting an env variable in a shell profile would also not apply to an double-clickable app bundle since no shell is involved in launching it; it is possible to set environment variables in the app's Info.plist, though. What is the cause of this? Is it limited to just Darwin systes? Does the 'horrible hack' solution on OS have any drawbacks? I like it better because it is not needed to set the LC_ALL environment variable prior to starting the Python program. As the caveats section points out, setting locale env vars may have unwanted side effects on other parts of the process it is running in or creates. So, if you are using it in a standalone program, it may be OK. If you are using it a module intended to be used by other programs, you probably shouldn't be changing something that could break other parts of the calling program. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
Hi. On 12.3.2014. 21:02, MRAB wrote: I was wondering if someone could explain gc.get_objects() in a bit more detail to me. Does it return a list of 'all objects known to Python'? Only some of them? Which does it return? Which it does not? gc.is_tracked(...) might be relevant to your question. Thanks! The documentation for gc.is_tracked() straightened me out. :-) So gc.collect() returns a list of all the objects GC is in charge of, and which instances are and are not tracked by the GC is, I guess, an interpreter implementation detail. For CPython 3.4 I guess strings and other atomic types such as ints are not, as well as raw object() instances. Custom class instances on the other hand seem to be under GC control. Thanks again. Best regards, Jurko Gospodnetić -- https://mail.python.org/mailman/listinfo/python-list
DB API question - where is a stored procedure's return value?
I've stumbled on a problem with the python-sybase module. If I have a
stored procedure like this:
create stored procedure test_proc
as
return 1
and call it from Python like this:
curs.callproc(test_proc, {})
it's not clear to me where the return status is stored. Currently, the
python-sybase module provides (I believe) the ability to set output
parameters (those specified as such in the argument dictionary), and
you can iterate over all the result sets the stored procedure
produces.
Looking at the Python database API (PEP 249), I saw no mention of the
return status:
.callproc( procname [, parameters ] )
(This method is optional since not all databases provide stored procedures. [3])
Call a stored database procedure with the given name. The sequence of
parameters must contain one entry for each argument that the procedure
expects. The result of the call is returned as modified copy of the
input sequence. Input parameters are left untouched, output and
input/output parameters replaced with possibly new values.
The procedure may also provide a result set as output. This must then
be made available through the standard.fetch*() methods.
I see no mention of how to handle the return value. The modified copy
of the input sequence is just that. In the case of the python-sybase
module, any parameter values in the input dictionary of type
DataBufType are rewritten with values from the first row of the first
result set. The Cursor class also defines a _status_result() method,
which sounds promising. I haven't had a chance to try it out yet. My
concern is more that the Python database API doesn't mention stored
procedure return values at all.
What do other database adaptors do about stored procedure return
values? Does PEP 249 need revision?
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
On Thu, Mar 13, 2014 at 8:28 AM, Jurko Gospodnetić jurko.gospodne...@pke.hr wrote: So gc.collect() returns a list of all the objects GC is in charge of, and which instances are and are not tracked by the GC is, I guess, an interpreter implementation detail. I assume you don't mean collect() there, as that returns the amount of garbage that it just collected :) It's not strictly an implementation detail, beyond that there are certain optimizations. For instance... For CPython 3.4 I guess strings and other atomic types such as ints are not, as well as raw object() instances. Custom class instances on the other hand seem to be under GC control. ... strings and ints should never be listed, and custom objects should always be listed, but I'd say the non-tracking of object() would be an implementation-specific optimization. Definitely the non-tracking of a dict of nothing but atomic keys and values would be that. The concept is that the GC tracks (in that sense; everything in CPython is refcounted, but that's not what these functions look at) anything that could be a part of a reference cycle. That's all it concerns itself with, so something that can't have references to arbitrary objects can't possibly be worth tracking. Interestingly, a tuple of integers is tracked: a=1,2,3 gc.is_tracked(a) True So not all optimizations are done that could be done. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
TALK.ORIGINS CENSORS THE TRUTH
EVIDENCE THAT HUMANS LIVED IN THE DEVONIAN: https://groups.google.com/group/sci.bio.paleontology/browse_thread/thread/6f501c469c7af24f# https://groups.google.com/group/sci.bio.paleontology/browse_thread/thread/3aad75c16afb0b82# http://thrinaxodon.wordpress.com/ === THRINAXODON ONLY HAD THIS TO SAY: I..I...I...Can't believe it. This completely disproved Darwinian orthodoxy. === THE BASTARDS AT THE SMITHSONIAN, AND THE LEAKEY FOUNDATION ARE ERODING WITH FEAR. === THESE ASSHOLES ARE GOING TO DIE: THOMAS AQUINAS; ALDOUS HUXLEY; BOB CASANVOVA; SkyEyes; DAVID IAIN GRIEG; MARK ISAAK; JOHN HARSHAM; RICHARD NORMAN; DR. DOOLITTLE; CHARLES DARWIN; MARK HORTON; ERIK SIMPSON; HYPATIAB7; PAUL J. GANS; JILLERY; WIKI TRIK; THRINAXODON; PETER NYIKOS; RON OKIMOTO; JOHN S. WILKINS === THRINAXODON WAS SCOURING ANOTHER DEVONIAN FOSSIL BED, AND FOUND A HUMAN SKULL, AND A HUMAN FEMUR. HE ANALYSED THE FINDS, AND SAW THAT THEY WERE NOT NORMAL ROCKS. THESE WERE FOSSILIZED BONES. THEY EVEN HAD TOOTH MARKS ON THEM. SO, THRINAXODON BROUGHT THEM TO THE LEAKEY FOUNDATION, THEY UTTERLY DISMISSED IT, AND SAID, We want to keep people thinking that humans evolved 2 Ma. THRINAXODON BROUGHT HIS SWORD, AND SAID, SCIENCE CORRECTS ITSELF. RICHARD LEAKEY SAID, That is a myth, for people to believe in science. THRINAXODON PLANS TO BRING DOOM TO SCIENCE, ITSELF. THRINAXODON IS NOW ON REDDIT -- ---Thrinaxodon -- https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
On Wed, Mar 12, 2014 at 3:44 PM, Chris Angelico ros...@gmail.com wrote: The concept is that the GC tracks (in that sense; everything in CPython is refcounted, but that's not what these functions look at) anything that could be a part of a reference cycle. That's all it concerns itself with, so something that can't have references to arbitrary objects can't possibly be worth tracking. Interestingly, a tuple of integers is tracked: a=1,2,3 gc.is_tracked(a) True So not all optimizations are done that could be done. Or is it? a = 1,2,3 gc.is_tracked(a) True gc.collect() 0 gc.is_tracked(a) False -- https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
On Wed, Mar 12, 2014 at 4:35 PM, Ian Kelly ian.g.ke...@gmail.com wrote: So not all optimizations are done that could be done. Or is it? a = 1,2,3 gc.is_tracked(a) True gc.collect() 0 gc.is_tracked(a) False I guess the reason for this is that when PyTuple_New is called, it knows how many elements it will contain but doesn't know yet what they are, so it defers the issue by just going ahead and adding itself to the GC -- https://mail.python.org/mailman/listinfo/python-list
Re: Save to a file, but avoid overwriting an existing file
On 12Mar2014 13:29, zoom z...@yahoo.com wrote: I would like to assure that when writing to a file I do not overwrite an existing file, but I'm unsure which is the best way to approach to this problem. As I can see, there are at least two possibilities: 1. I could use fd = os.open(x, os.O_WRONLY | os.O_CREAT | os.O_EXCL) which will fail - if the file exists. However, I would prefer if the program would try to save under different name in this case, instead of discarding all the calculation done until now - but I' not too well with catching exceptions. Others have menthions tempfile, though of course you have the same collision issue when you come to rename the temp file if you are keeping it. I would run with option 1 for your task. Just iterate until os.open succeeds. However, you need to distinuish _why_ an open fails. For example, if you were trying to make files in a directory to which you do not have write permission, or just a directory that did not exist, os.open would fail not matter what name you used, so your loop would run forever. Therefore you need to continue _only_ if you get EEXIST. Otherwise abort. So you'd have some code like this (totally untested): # at top of script import errno # where you make the file def open_new(primary_name): try: fd = os.open(primary_name, os.O_WRONLY | os.O_CREAT | os.O_EXCL) except OSError as e: if e.errno != errno.EEXIST: raise else: return primary_name, fd n = 1 while True: secondary_name = %s.%d % (primary_name, n) try: fd = os.open(secondary_name, os.O_WRONLY | os.O_CREAT | os.O_EXCL) except OSError as e: if e.errno != errno.EEXIST: raise else: return secondary_name, fd n += 1 # where you need the file path, fd = open_new(x) That gets you a function your can reuse which returns the file's name and the file descriptor. Cheers, -- Cameron Simpson c...@zip.com.au Reason #173 to fear technology: o o o o o o o o ^|\ ^|^ v|^ v|v |/v |X| \| | /\ \ / \ / \ / \ o o o o o o o o \ x / | / \ ) |\ / \ / \ / \L Mr. email does the Macarena. -- https://mail.python.org/mailman/listinfo/python-list
Re: DB API question - where is a stored procedure's return value?
On Wed, Mar 12, 2014 at 3:44 PM, Skip Montanaro s...@pobox.com wrote: What do other database adaptors do about stored procedure return values? Does PEP 249 need revision? I can't speak generally, but in cx_Oracle you either execute a query like this: result = cursor.var(cx_Oracle.NUMBER) cursor.execute(:1 := test_proc(), [result]) print(result.getvalue()) or you use the non-standard callfunc method: print(cursor.callfunc(test_proc, cx_Oracle.NUMBER)) As a general solution, one might wrap a stored procedure that returns a value into a stored procedure that has an output parameter and call it with callproc. Some implementations might include a return value in the parameter list anyway. -- https://mail.python.org/mailman/listinfo/python-list
Re: Save to a file, but avoid overwriting an existing file
On 12/03/2014 22:19, Cameron Simpson wrote: On 12Mar2014 13:29, zoom z...@yahoo.com wrote: I would like to assure that when writing to a file I do not overwrite an existing file, but I'm unsure which is the best way to approach to this problem. As I can see, there are at least two possibilities: 1. I could use fd = os.open(x, os.O_WRONLY | os.O_CREAT | os.O_EXCL) which will fail - if the file exists. However, I would prefer if the program would try to save under different name in this case, instead of discarding all the calculation done until now - but I' not too well with catching exceptions. Others have menthions tempfile, though of course you have the same collision issue when you come to rename the temp file if you are keeping it. I would run with option 1 for your task. Just iterate until os.open succeeds. However, you need to distinuish _why_ an open fails. For example, if you were trying to make files in a directory to which you do not have write permission, or just a directory that did not exist, os.open would fail not matter what name you used, so your loop would run forever. Therefore you need to continue _only_ if you get EEXIST. Otherwise abort. So you'd have some code like this (totally untested): # at top of script import errno # where you make the file def open_new(primary_name): try: fd = os.open(primary_name, os.O_WRONLY | os.O_CREAT | os.O_EXCL) except OSError as e: if e.errno != errno.EEXIST: raise else: return primary_name, fd n = 1 while True: secondary_name = %s.%d % (primary_name, n) try: fd = os.open(secondary_name, os.O_WRONLY | os.O_CREAT | os.O_EXCL) except OSError as e: if e.errno != errno.EEXIST: raise else: return secondary_name, fd n += 1 # where you need the file path, fd = open_new(x) That gets you a function your can reuse which returns the file's name and the file descriptor. Cheers, I haven't looked but would things be easier if the new exception hierarchy were used http://docs.python.org/3.3/whatsnew/3.3.html#pep-3151-reworking-the-os-and-io-exception-hierarchy ? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: DB API question - where is a stored procedure's return value?
On Wed, Mar 12, 2014 at 5:00 PM, Ian Kelly ian.g.ke...@gmail.com wrote: result = cursor.var(cx_Oracle.NUMBER) cursor.execute(:1 := test_proc(), [result]) print(result.getvalue()) Sorry, that should properly be: result = cursor.var(cx_Oracle.NUMBER) cursor.execute(BEGIN :1 := test_proc(); END;, [result]) print(result.getvalue()) -- https://mail.python.org/mailman/listinfo/python-list
Re: Deep vs. shallow copy?
On Wed, 12 Mar 2014 15:29:59 +, Alex van der Spek wrote: Having been taught programming in Algol60 Python still defeats me at times! Particularly since Algol60 wasn't very long lived and what came thereafter (FORTRAN) much worse. Fortran came first. Fortran was the first high-level language which allowed the programmer to write things that looked rather like the sorts of mathematical expressions they were used to. There were a few higher-level assembly languages that came before Fortran, such as SpeedCoding, Fortran's predecessor, but Fortran was the first truly high-level programming language, and even in 1957 it came with an optimizing compiler. I'm not really familiar with Algol, but I do know Pascal, and you should think of the append method to be like a Pascal procedure. Because Python doesn't have true procedures, it follows the convention that returning the special object None signals the intention to return nothing at all. Hence your example below: c = a.append(b) print c None -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
On Thu, Mar 13, 2014 at 9:35 AM, Ian Kelly ian.g.ke...@gmail.com wrote: Or is it? a = 1,2,3 gc.is_tracked(a) True gc.collect() 0 gc.is_tracked(a) False Huh, *that* is interesting! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: DB API question - where is a stored procedure's return value?
On Mar 13, 2014, at 12:00 AM, Ian Kelly ian.g.ke...@gmail.com wrote: As a general solution, one might wrap a stored procedure that returns a value into a stored procedure that has an output parameter and call it with callproc. Some implementations might include a return value in the parameter list anyway. Alternatively… if it’s really a function… wrap it in a select statement… such as: select foo() as value from dual -- https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On Wed, 12 Mar 2014 08:32:29 -0700, Ethan Furman wrote: There must be waaay too much stuff being kept alive by the stack traces of the failed tests. I believe that unittest does keep stack traces alive until the process ends. I thought that there was a recent bug report for it, but the only one I can find was apparently fixed more than a decade ago: http://bugs.python.org/issue451309 -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
On Tue, 11 Mar 2014 17:06:43 -0600, Ian Kelly wrote: On Tue, Mar 11, 2014 at 10:46 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: There are a number of possible solutions. One possibility would be to copy the Circle as an Ellipse and return the new object instead of mutating it. Then you have the situation where, given a mutable object x that satisfies isinstance(x, Ellipse), the stretch method *may* be able to update the object in-place, or it *may* not. That is a really lousy design. Of course we are free to create classes with ugly, inconsistent, stupid or unworkable APIs if we like. Python won't stop us: That's true but irrelevant to my point, which was to counter the assertion that mutable types can always be assumed to be able to perform operations in-place. Always? Not so fast. This is Python. We have freedom to abuse nearly everything, and if you want to shoot yourself in the foot, you can. With the exception of a handful of things which cannot be overridden (e.g. None, numeric literals, syntax) you cannot strictly assume anything about anything. Python does not enforce that iterators raise StopIteration when empty, or that indexing beyond the boundaries of a sequence raises IndexError, or that __setitem__ of a mapping sets the key and value, or that __len__ returns a length. Augmented assignment is no different. The docs describe the intention of the designers and the behaviour of the classes that they control, so with standard built-in classes like int, str, list, tuple etc. you can safely assume that mutable types will perform the operation in place and immutable types won't, but with arbitrary types from some arbitrarily eccentric or twisted programmer, who knows what it will do? -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On Tue, 11 Mar 2014 13:58:17 -0700, Ethan Furman wrote:
class Test_wfbrp_20140225(TestCase):
@classmethod
def setUpClass(cls):
cls.pp = wfbrp.PaymentProcessor(
'.../lockbox_file',
'.../aging_file',
[
Path('month_end_1'),
Path('month_end_2'),
Path('month_end_3'),
],
)
This has nothing to do with your actual problem, which appears to be the
Linux(?) OOM killer reaping your applications, just some general
observations on your test.
def test_xxx_1(self):
Having trouble thinking up descriptive names for the test? That's a sign
that the test might be doing too much. Each test should check one self-
contained thing. That may or may not be a single call to a unittest
assert* method, but it should be something you can describe in a few
words:
it's a regression test for bug 23
test that the database isn't on fire
invoices should have a valid debtor
the foo report ought to report all the foos
...and nothing but the foos.
This hints -- its just a hint, mind you, since I lack all specific
knowledge of your application -- that the following affirm tests should
be pulled out into separate tests.
p = self.pp.lockbox_payments[0]
# affirm we have what we're expecting
self.assertEqual(
(p.payer, p.ck_num, p.credit),
('a customer', '010101', 1),
)
self.assertEqual(p.invoices.keys(), ['XXX'])
self.assertEqual(p.invoices.values()[0].amount, 1)
which would then leave this to be the Actual Thing Being Tested for this
test, which then becomes test_no_missing_invoices rather than test_xxx_1.
# now make sure we get back what we're expecting
np, b = self.pp._match_invoices(p)
missing = []
for inv_num in ('123456', '789012', '345678'):
if inv_num not in b:
missing.append(inv_num)
if missing:
raise ValueError('invoices %r missing from batch' %
missing)
Raising an exception directly inside the test function should only occur
if the test function is buggy. As Terry has already suggested, this
probably communicates your intention much better:
self.assertEqual(missing, [])
--
Steven D'Aprano
http://import-that.dreamwidth.org/
--
https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On Wed, 12 Mar 2014 08:32:29 -0700, Ethan Furman wrote: Some systems have an oom (Out Of Memory) process killer, which nukes (semi-random) process when the system exhausts memory. Is it possible this is happening? If so, you should see some log message in one of your system logs. That would explain why my editor windows were being killed. Try opening a second console tab and running top in it. It will show the amount of memory being used. Then run the tests in the first, jump back to top, and watch to see if memory use goes through the roof: top -Mm -d 0.5 will sort by memory use, display memory in more sensible human-readable units instead of bytes, and update the display every 0.5 second. You can then hit the i key to toggle display of idle processes and only show those that are actually doing something (which presumably will include Python running the tests). This at least will allow you to see whether or not memory is the concern. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Save to a file, but avoid overwriting an existing file
Cameron Simpson c...@zip.com.au writes: Therefore you need to continue _only_ if you get EEXIST. Otherwise abort. If you target Python 3.3 or later, you can catch “FileExistsError” URL:http://docs.python.org/3/library/exceptions.html#FileExistsError which is far simpler than messing around with ‘errno’ values. -- \ “I know you believe you understood what you think I said, but I | `\ am not sure you realize that what you heard is not what I | _o__) meant.” —Robert J. McCloskey | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On 03/12/2014 04:47 PM, Steven D'Aprano wrote: top -Mm -d 0.5 Cool, thanks! -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On 03/12/2014 04:38 PM, Steven D'Aprano wrote: [snip lots of good advice for unit testing] I was just removing the Personally Identifiable Information. Each test is pulling a payment from a batch of payments, so the first couple asserts are simply making sure I have the payment I think I have, then I run the routine that is supposed to match that payment with a bunch of invoices, and then I test to make sure I got back the invoices that I have manually verified are the correct ones to be returned. There are many different tests because there are many different paths through the code, depending on exactly which combination of insanities the bank, the customer, and the company choose to inflict at that moment. ;) -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
On Wed, Mar 12, 2014 at 5:20 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
On Tue, 11 Mar 2014 17:06:43 -0600, Ian Kelly wrote:
That's true but irrelevant to my point, which was to counter the
assertion that mutable types can always be assumed to be able to perform
operations in-place.
Always? Not so fast.
This is Python. We have freedom to abuse nearly everything, and if you
want to shoot yourself in the foot, you can. With the exception of a
handful of things which cannot be overridden (e.g. None, numeric
literals, syntax) you cannot strictly assume anything about anything.
Python does not enforce that iterators raise StopIteration when empty, or
that indexing beyond the boundaries of a sequence raises IndexError, or
that __setitem__ of a mapping sets the key and value, or that __len__
returns a length.
Thank you; you've stated my point more succinctly than I did.
Augmented assignment is no different. The docs describe the intention of
the designers and the behaviour of the classes that they control, so with
standard built-in classes like int, str, list, tuple etc. you can safely
assume that mutable types will perform the operation in place and
immutable types won't, but with arbitrary types from some arbitrarily
eccentric or twisted programmer, who knows what it will do?
This got me curious about how consistent the standard library is about
this exactly, so I did some grepping. In the standard library there
are 5 mutable types that support concatenation that I was able to
find: list, deque, array, bytearray, and Counter. There are none that
support addition, which I find interesting in that the language
provides hooks for in-place addition but never uses them itself.
All of the classes above appear to follow the rule that if you can
concatenate an operand, you can in-place concatenate the same operand.
The converse however does not hold: list.__iadd__ and
Counter.__iadd__ are both more permissive in what types they will
accept than their __add__ counterparts, and especially interesting to
me is that deque implements __iadd__ but does not implement __add__ at
all. This last in particular seems to support the assertion that +=
should be viewed more as a shorthand for an in-place operation, less
as an equivalent for x = x + y.
l = [1,2,3]
l + (4,5,6)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: can only concatenate list (not tuple) to list
l += (4,5,6)
l
[1, 2, 3, 4, 5, 6]
c = collections.Counter('mississippi')
c + collections.Counter('alabama')
Counter({'s': 4, 'a': 4, 'i': 4, 'p': 2, 'm': 2, 'b': 1, 'l': 1})
c + dict({'a': 4, 'l': 1, 'b': 1, 'm': 1})
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for +: 'Counter' and 'dict'
c += dict({'a': 4, 'l': 1, 'b': 1, 'm': 1})
c
Counter({'s': 4, 'a': 4, 'i': 4, 'p': 2, 'm': 2, 'b': 1, 'l': 1})
d = collections.deque([1,2,3])
d += [4,5,6]
d
deque([1, 2, 3, 4, 5, 6])
d + [7,8,9]
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for +: 'collections.deque' and 'list'
d.__add__
Traceback (most recent call last):
File stdin, line 1, in module
AttributeError: 'collections.deque' object has no attribute '__add__'
--
https://mail.python.org/mailman/listinfo/python-list
Re: Tuples and immutability
On 3/12/2014 9:35 PM, Ian Kelly wrote:
On Wed, Mar 12, 2014 at 5:20 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
On Tue, 11 Mar 2014 17:06:43 -0600, Ian Kelly wrote:
That's true but irrelevant to my point, which was to counter the
assertion that mutable types can always be assumed to be able to perform
operations in-place.
Always? Not so fast.
This is Python. We have freedom to abuse nearly everything, and if you
want to shoot yourself in the foot, you can. With the exception of a
handful of things which cannot be overridden (e.g. None, numeric
literals, syntax) you cannot strictly assume anything about anything.
Python does not enforce that iterators raise StopIteration when empty, or
that indexing beyond the boundaries of a sequence raises IndexError, or
that __setitem__ of a mapping sets the key and value, or that __len__
returns a length.
Thank you; you've stated my point more succinctly than I did.
Augmented assignment is no different. The docs describe the intention of
the designers and the behaviour of the classes that they control, so with
standard built-in classes like int, str, list, tuple etc. you can safely
assume that mutable types will perform the operation in place and
immutable types won't, but with arbitrary types from some arbitrarily
eccentric or twisted programmer, who knows what it will do?
This got me curious about how consistent the standard library is about
this exactly, so I did some grepping. In the standard library there
are 5 mutable types that support concatenation that I was able to
find: list, deque, array, bytearray, and Counter. There are none that
support addition, which I find interesting in that the language
provides hooks for in-place addition but never uses them itself.
All of the classes above appear to follow the rule that if you can
concatenate an operand, you can in-place concatenate the same operand.
The converse however does not hold: list.__iadd__ and
Counter.__iadd__ are both more permissive in what types they will
accept than their __add__ counterparts, and especially interesting to
me is that deque implements __iadd__ but does not implement __add__ at
all. This last in particular seems to support the assertion that +=
should be viewed more as a shorthand for an in-place operation, less
as an equivalent for x = x + y.
l = [1,2,3]
l + (4,5,6)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: can only concatenate list (not tuple) to list
l += (4,5,6)
l
[1, 2, 3, 4, 5, 6]
Like it or not, one should actually think of 'somelist += iterable' as
equivalent to 'somelist.extend(iterable)'. Without looking at the C
code, I suspect that the latter is the internal implementation.
Collections.deque also has .extend. Collections.Counter has .update and
that is += seems to be doing.
c = collections.Counter('mississippi')
c + collections.Counter('alabama')
Counter({'s': 4, 'a': 4, 'i': 4, 'p': 2, 'm': 2, 'b': 1, 'l': 1})
c + dict({'a': 4, 'l': 1, 'b': 1, 'm': 1})
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for +: 'Counter' and 'dict'
c += dict({'a': 4, 'l': 1, 'b': 1, 'm': 1})
c
Counter({'s': 4, 'a': 4, 'i': 4, 'p': 2, 'm': 2, 'b': 1, 'l': 1})
d = collections.deque([1,2,3])
d += [4,5,6]
d
deque([1, 2, 3, 4, 5, 6])
d + [7,8,9]
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for +: 'collections.deque' and 'list'
d.__add__
Traceback (most recent call last):
File stdin, line 1, in module
AttributeError: 'collections.deque' object has no attribute '__add__'
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
Re: unittest weirdness
On 3/12/2014 11:32 AM, Ethan Furman wrote: I strongly suspect it's memory. When I originally wrote the code I tried to include six months worth of EoM data, but had to back it down to three as my process kept mysteriously dying at four or more months. There must be waaay too much stuff being kept alive by the stack traces of the failed tests. There is an issue or two about unittest not releasing memory. Also, modules are not cleared from sys.modules, so anything accessible from global scope is kept around. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Save to a file, but avoid overwriting an existing file
On 13Mar2014 11:22, Ben Finney ben+pyt...@benfinney.id.au wrote: Cameron Simpson c...@zip.com.au writes: Therefore you need to continue _only_ if you get EEXIST. Otherwise abort. If you target Python 3.3 or later, you can catch “FileExistsError” URL:http://docs.python.org/3/library/exceptions.html#FileExistsError which is far simpler than messing around with ‘errno’ values. Yes and no. Firstly, we Old Farts find these new ways confusing:-) Secondly, and perhaps more importantly, if I were using the O_EXCL et al constants in a call to os.open as the OP is doing, I would want to be equally picky about what I recognise in the exception; I _want_ to use error explicitly. In particular, the behaviour of os.open is pretty much a bare metal call to the POSIX open(2) OS interface, and so I want my response to be framed directly in terms of the documented error responses for that API. Thus, errno values. Finally, if I were grouping my responses to the exception by type and perhaps performing a few different actions depending on the error, then the new class hierachy is a good way to do it. But for this instance, IMO, we're catching exactly one failure mode as retry with another name and reraising anything else. So I don't find an errno test any less readable that a class comparison, and it still feels more precise to me. Cheers, -- Cameron Simpson c...@zip.com.au The road less traveled is FASTER !! - Luc Marcouiller, marco...@ireq.hydro.qc.ca -- https://mail.python.org/mailman/listinfo/python-list
Re: What does gc.get_objects() return?
On 3/12/2014 3:34 PM, Jurko Gospodnetić wrote: I was wondering if someone could explain gc.get_objects() in a bit more detail to me. Does it return a list of 'all objects known to Python'? Only some of them? Which does it return? Which it does not? This took about 10 seconds. import gc help(gc.get_objects) Help on built-in function get_objects in module gc: get_objects(...) get_objects() - [...] Return a list of objects tracked by the collector (excluding the list returned). --- Now, what object does gc track? CPython answer: objects that could be involved in and kept alive by reverence cycles and therefore not deleted when not needed by reference counting alone. Object instances, numbers, and strings cannot be involved in reference cycles, as there are no user settable references. Tuples can be: a = [] b = (a,) a[0] = b A tuple of, for instance, ints would not need to be tracked, but if it is, someone probably decided that the cost of checking is not worth the benefit. Details are subject to change with versions. In 3.4, gc can break cycles of objects with delete methods, whereas they previously waiting until program shutdown. Some year ago, I remember a suggestion to turn off gc while creating lots of tuples that did not need to be tracked, but I do not know if that is still useful. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Deep vs. shallow copy?
On Thursday, March 13, 2014 4:37:53 AM UTC+5:30, Steven D'Aprano wrote:
On Wed, 12 Mar 2014 15:29:59 +, Alex van der Spek wrote:
Having been taught programming in Algol60 Python still defeats me at
times! Particularly since Algol60 wasn't very long lived and what came
thereafter (FORTRAN) much worse.
I'm not really familiar with Algol, but I do know Pascal, and you should
think of the append method to be like a Pascal procedure. Because Python
doesn't have true procedures, it follows the convention that returning
the special object None signals the intention to return nothing at all.
Hence your example below:
Yes... Algol I dont know. But to the extent that it corresponds to
Pascal the following may help.
Pascal is remarkably clean and cleaner than what came before -- Lisp
-- and after -- C, all its derivatives, python... almost everything.
First off there are expressions and statements, and no messing with
the distinction. C started laissez-faire. No void, just default
functions intended to be used as procedures to int. This was
problematic to enough people that void was introduced soon enough.
But its still messy:
An assignment like x = 3 is both an expression and (like) a statement.
It is mostly used as a statement by appending a semicolon.
It can also be used as an expression as in y = x = 3.
Try explaining this to a first year class and you'd know what a mess it is.
For the most part if you think youve got it you probably havent.
Python is in some respects worse than C, in some better.
Its better in that assignment statements are not re-purposeable as
expressions. Its worse in that procedures are simulated by None
returning functions -- this means syntax errors which C would give
when using a void function as an expression, you wont get in python.
On the whole, whether your language supports it or not, its best to
think of the world as separated into actions and values.
Call the action-world the 'imperative' world.
Call the value-world the 'functional' world ('declarative' would be
better but 'functional' is too entrenched).
[Following table meant to be read with fixed (courier) font]
| | Imperative | Functional |
| Language entity | Statement | Expression |
| Denote (and think with) | Action | Value |
| Abstracted into | Procedure | Function|
| Atoms are | Assignment | Constant/Variable |
| Data Structures | Mutable| Immutable |
| Loop primitive | Recursion | Iteration |
| World is| In time| Timeless (Platonic) |
Generally mixing these two worlds is necessary for any non-trivial
programming but is commonly problematic.
Your error was to use a functional form -- list comprehension -- and
embed an imperative construct -- row.append -- into that. This is
always a gaffe
Legal mixing is done thus
Expression - Statement by putting the expression on the rhs of an assignment
Statement - Expression by putting the statement(s) into a function
While most programmers think this unproblematic, Backus' widely-cited
Turing Award lecture is devoted to showing why this is a problem
http://www.thocp.net/biographies/papers/backus_turingaward_lecture.pdf
For an old Algol/Fortran programmer it may be a worthwhile read considering
Backus invented Fortran :-)
--
https://mail.python.org/mailman/listinfo/python-list
[issue1617161] Instance methods compare equal when their self's are equal
Westley Martínez added the comment: I updated the tests to be in sync, but the implementation of the fix is not so trivial. The conversion from cmp() to rich comparison is the primary culprit, so it will take time for me to get familiar enough with the C source to update the fix. I couldn't seem to get the patch to apply even to the 2.x branch (I think it's because it's an SVN patch...) to see if the fix actually works. That said, this enhancement is so old that it might not warrant a fix at all. Maybe it should be brought up on python-dev? -- Added file: http://bugs.python.org/file34367/1617161_test_update.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1617161 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19627] python open built-in function - updating is not defined
Roundup Robot added the comment: New changeset 1f823c1559bd by Éric Araujo in branch '2.7': Define what “updating” means in docs for open (#19627). http://hg.python.org/cpython/rev/1f823c1559bd -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19627 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19627] python open built-in function - updating is not defined
Éric Araujo added the comment: Edited slightly and committed, thank you! -- assignee: docs@python - eric.araujo resolution: - fixed stage: needs patch - committed/rejected ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19627 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20135] FAQ need list mutation answers
Éric Araujo added the comment: I agree; if someone goes to the docs, “Programming FAQ” looks more enticing than “Design and History”. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20135 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20627] Add context manager support to xmlrpc.client.ServerProxy
Éric Araujo added the comment: Looks good to me; left some comments on Rietveld. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20627 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue809163] Can't add files with spaces
Éric Araujo added the comment: Previous comment: “If there is a way to make rpm read one line at a time, we should use it.” If there isn’t such an option, and if sed is guaranteed to be installed on rpm-based systems, we can try the sed script. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue809163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4931] distutils does not show any error msg when can't build C module extensions due to a missing C compiler
Roundup Robot added the comment: New changeset a865f6fb82b4 by Éric Araujo in branch '2.7': Avoid “error: None” messages from distutils (#4931). http://hg.python.org/cpython/rev/a865f6fb82b4 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4931 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4931] distutils does not show any error msg when can't build C module extensions due to a missing C compiler
Roundup Robot added the comment: New changeset 0f1237b61f58 by Éric Araujo in branch '2.7': Restore missing part of error message (#4931) http://hg.python.org/cpython/rev/0f1237b61f58 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4931 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16104] Use multiprocessing in compileall script
Claudiu.Popa added the comment: Thank you for the review, Éric! Here's the updated patch. -- Added file: http://bugs.python.org/file34368/issue16104_1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16104 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19627] python open built-in function - updating is not defined
Georg Brandl added the comment: Can this be closed? -- nosy: +georg.brandl ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19627 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19627] python open built-in function - updating is not defined
Changes by Éric Araujo mer...@netwok.org: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19627 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1222585] C++ compilation support for distutils
Éric Araujo added the comment: Now that distutils2 development is stopped and that distutils is no longer under a feature freeze, this is a reasonable thing to add to 3.5. Thanks for your efforts Arfrever, and sorry you had to port and maintain this to so many different versions. -- components: +Distutils -Distutils2 keywords: -easy versions: +Python 3.5 -3rd party, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1222585 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20853] pdb args crashes when an arg is not printable
Jurjen N.E. Bos added the comment: Thanks for your reaction. The object is not printable, since I was debugging an __init__ of an object, and some fields where being initialized: class foo: def __init__(self): foo.bar = hello def repr(self): return foo.bar I tried to make a useable patch file (with neater layout and using Exeption), see attach. -- keywords: +patch Added file: http://bugs.python.org/file34369/pdb.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20853 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4931] distutils does not show any error msg when can't build C module extensions due to a missing C compiler
Roundup Robot added the comment: New changeset 504eb00998f2 by Éric Araujo in branch '3.3': Avoid “error: None” messages from distutils (#4931). http://hg.python.org/cpython/rev/504eb00998f2 New changeset c7bd0f953687 by Éric Araujo in branch 'default': Merge 3.3 (#4931) http://hg.python.org/cpython/rev/c7bd0f953687 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4931 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20853] pdb args crashes when an arg is not printable
Jurjen N.E. Bos added the comment:
Oops. Here the correct example:
class foo:
... def __init__(self):
... foo.bar = hello
... def __repr__(self): return foo.bar
...
pdb.runcall(foo)
stdin(3)__init__()
(Pdb) a
Traceback (most recent call last):
File .\pdb.py, line 1132, in do_args
self.message('%s = %r' % (name, dict[name]))
File stdin, line 4, in __repr__
AttributeError: type object 'foo' has no attribute 'bar'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File stdin, line 1, in module
File .\pdb.py, line 1580, in runcall
return Pdb().runcall(*args, **kwds)
File C:\Python33\lib\bdb.py, line 439, in runcall
res = func(*args, **kwds)
File stdin, line 3, in __init__
File stdin, line 3, in __init__
File C:\Python33\lib\bdb.py, line 47, in trace_dispatch
return self.dispatch_line(frame)
File C:\Python33\lib\bdb.py, line 65, in dispatch_line
self.user_line(frame)
File .\pdb.py, line 266, in user_line
self.interaction(frame, None)
File .\pdb.py, line 345, in interaction
self._cmdloop()
File .\pdb.py, line 318, in _cmdloop
self.cmdloop()
File C:\Python33\lib\cmd.py, line 138, in cmdloop
stop = self.onecmd(line)
File .\pdb.py, line 411, in onecmd
return cmd.Cmd.onecmd(self, line)
File C:\Python33\lib\cmd.py, line 217, in onecmd
return func(arg)
File .\pdb.py, line 1134, in do_args
self.message('%s = *** repr failed: %s ***' % (name,))
TypeError: not enough arguments for format string
At least, I expect pdb to not crash, but a clearer error (as in the patch) is
nice to have.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20853
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19549] PKG-INFO is created with CRLF on Windows
Éric Araujo added the comment: The docs and PEPs only define the PKG-INFO file as “a set of RFC 822-style headers”. Character encoding was defined later to be UTF-8; there is no mention of explicit choice of platform-consistent or cross-platform linefeeds. Are there tools out there that suffer from the use of CRLF in PKG-INFO files? If not, I’ll close this as invalid: the behavior does not contradict the documentation. -- nosy: +dholth, ncoghlan versions: -Python 2.7, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19549 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19394] distutils.core.Extension: empty strings in library_dirs and include_dirs should not be allowed
Éric Araujo added the comment: Thanks for the report. What fix do you suggest? Printing an error if empty strings are passed for include/library_dirs? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19394 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20853] pdb args crashes when an arg is not printable
Jurjen N.E. Bos added the comment:
I am not good at this. Sorry for the mess.
Here is a good example, and a good patch:
class foo:
... def __init__(self):
... foo.bar = hello
... def __repr__(self): return foo.bar
...
pdb.runcall(foo)
Traceback (most recent call last):
File stdin, line 1, in module
NameError: name 'pdb' is not defined
import pdb
pdb.runcall(foo)
stdin(3)__init__()
(Pdb) a
Traceback (most recent call last):
File stdin, line 1, in module
File C:\Python33\lib\pdb.py, line 1577, in runcall
return Pdb().runcall(*args, **kwds)
File C:\Python33\lib\bdb.py, line 439, in runcall
res = func(*args, **kwds)
File stdin, line 3, in __init__
File stdin, line 3, in __init__
File C:\Python33\lib\bdb.py, line 47, in trace_dispatch
return self.dispatch_line(frame)
File C:\Python33\lib\bdb.py, line 65, in dispatch_line
self.user_line(frame)
File C:\Python33\lib\pdb.py, line 266, in user_line
self.interaction(frame, None)
File C:\Python33\lib\pdb.py, line 345, in interaction
self._cmdloop()
File C:\Python33\lib\pdb.py, line 318, in _cmdloop
self.cmdloop()
File C:\Python33\lib\cmd.py, line 138, in cmdloop
stop = self.onecmd(line)
File C:\Python33\lib\pdb.py, line 411, in onecmd
return cmd.Cmd.onecmd(self, line)
File C:\Python33\lib\cmd.py, line 217, in onecmd
return func(arg)
File C:\Python33\lib\pdb.py, line 1131, in do_args
self.message('%s = %r' % (name, dict[name]))
File stdin, line 4, in __repr__
AttributeError: type object 'foo' has no attribute 'bar'
--
Added file: http://bugs.python.org/file34370/pdb.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20853
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14270] Can't install a project in a specific directory
Éric Araujo added the comment: pysetup was replaced by pip as official installer. This issue may affect distlib. -- nosy: +vinay.sajip resolution: - out of date stage: test needed - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14270 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4931] distutils does not show any error msg when can't build C module extensions due to a missing C compiler
Éric Araujo added the comment: Happy to close this as fixed. -- resolution: accepted - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4931 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5342] packaging: add tests for old versions cleanup on update
Éric Araujo added the comment: packaging is not developped anymore. The issue described here may be relevant for projects like wheel, distlib or pip. -- resolution: - out of date stage: test needed - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5342 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12944] Accept arbitrary files for packaging's upload command
Changes by Éric Araujo mer...@netwok.org: -- Removed message: http://bugs.python.org/msg193103 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12944 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12944] Accept arbitrary files for distutils' upload command
Éric Araujo added the comment: With the lift of the distutils feature freeze decided at PyCon 2013, this feature request could be done for 3.5. -- title: Accept arbitrary files for packaging's upload command - Accept arbitrary files for distutils' upload command versions: +Python 3.5 -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12944 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17574] pysetup failing with OSError: [Errno 18] Invalid cross-device link.
Changes by Éric Araujo mer...@netwok.org: -- resolution: - out of date stage: - committed/rejected status: open - closed type: crash - behavior versions: -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17574 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17640] from distutils.util import byte_compile hangs
Éric Araujo added the comment: Can’t reproduce (linux 3.12). If someone on FreeBSD can reproduce the bug, please reopen this ticket. -- resolution: - works for me stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17640 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18045] get_python_version is not import in bdist_rpm.py
Roundup Robot added the comment: New changeset 677327810121 by Éric Araujo in branch '2.7': Fix missing import in bdist_rpm (#18045) http://hg.python.org/cpython/rev/677327810121 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18045 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18045] get_python_version is not import in bdist_rpm.py
Éric Araujo added the comment: Thanks for the report and patches. -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18045 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18970] run_setup() behavior differs from cli invocation of setup.py
Éric Araujo added the comment: The principle of least surprise suggests that run_setup() should behave equivalently to a command line invocation of setup.py. That’s debatable: distutils.core.setup corresponds to a command-line invocation of setup.py, but run_setup is documented as something else: http://docs.python.org/3.4/distutils/apiref#distutils.core.run_setup In general, distutils lends itself poorly to programmatic usage. Can you tell more about your use case for calling run_setup directly? Many setup.py scripts use the idiomatic 'conditional script' stanza if __name__ == '__main__'. That’s surprising to me: setup.py scripts are not modules-that-also-work-as-scripts, only scripts. -- versions: -Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18970 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16326] distutils build_ext fails to set library_dirs in 2.7.2 on Linux
Éric Araujo added the comment: FTR a patch in #18976 is said to also fix this one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16326 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14894] distutils.LooseVersion fails to compare number and a word
Éric Araujo added the comment: I’m still torn between improving this as best as we can and waiting for clean new spec and code. Last PyCon Nick said he had a PEP in mind to specify how distutils would be updated to support new standards; I think we’ll revisit this issue when that policy exists. -- nosy: +ncoghlan versions: +Python 3.4 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14894 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14915] pysetup3.3 install is case insensitive, remove is case sensitive
Changes by Éric Araujo mer...@netwok.org: -- resolution: - out of date stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14915 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10530] distutils2 should allow the installing of python files with invalid syntax
Éric Araujo added the comment: This is now irrelevant, unless it also applies to distutils (in which case, please reopen). -- assignee: tarek - eric.araujo resolution: - out of date stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10530 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11416] netrc module does not handle multiple entries for a single host
Lucas Hoffmann added the comment: What is the status of this issue? On version 3.3 I still can not get more than one entry per host. The stopping issue #12009 seems to be closed. (Sorry, if I violate some etiquette by bumping this, just tell me.) -- nosy: +luc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11416 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20884] importlib/__init__.py can not be loaded without __file__ - breaks cxFreeze
Jurko Gospodnetić added the comment: As mentioned by Anthony Tuininga at: https://bitbucket.org/anthony_tuininga/cx_freeze/issue/61/python-340-nameerror-name-__file__-is-not#comment-9077661 The value of __file__ in the problematic importlib/__init__.py module with Python 3.3. is 'frozen' while with Python 3.4 it is not defined. And this 'frozen' string seems to be something coming from Python and not cx_Freeze (I see it in 'Python/import.c' in both Python 3.3.5 3.4). Hope this helps. Best regards, Jurko Gospodnetić -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20884 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13198] Remove duplicate definition of write_record_file
Éric Araujo added the comment: wheel/distlib/pip don’t have the same code, and if they had the same issue it would probably be found by a user. Closing. -- resolution: - out of date stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13198 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13175] packaging uses wrong line endings in RECORD files on Windows
Éric Araujo added the comment: This issue is now obsolete for distutils2. Other projects that write RECORD files are hopefully more robust in their code and tests. -- resolution: - out of date stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13175 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
