Re: even faster heaps

[srinivas devaki]

Hi,
sorry i didn't get to your last mail as i'm having exams at that time.

Everything is good so far.

but if at all the purpose is to add features to the stdlib heap and
keeping the speed offered by the stdlib, why aren't you allowing the
duplicate elements which is possible with stdlib heap.

to be able to add duplicate elements you just have to make the set as
a dict with counter as the value. afterall it is highly common to have
priority task scheduling with equal priorities to the tasks.

as far as the benchmarks are concerned it didn't change much with the
current dataset. but it is not valid to compare them as the current
dataset is intended for unique elements. but with the dict the dataset
can contain elements which have equal keys.

on my machine with 10 repetitions in the test_xheap_time.py
** results for the code which supports duplicate elements
h...@github.com/eightnoteight/xheap **

(u'init', u'heapq  ', u' 0.45 ( 1.00x)  4.59 ( 1.00x) 52.24 (
1.00x) 553.80 ( 1.00x)')
(u'', u'Heap   ', u' 0.48 ( 1.07x)  5.41 ( 1.18x) 75.89 (
1.45x) 786.46 ( 1.42x)')
(u'', u'RemovalHeap', u' 0.86 ( 1.93x)  9.98 ( 2.18x) 131.28 (
2.51x) 1364.14 ( 2.46x)')

(u'pop ', u'heapq  ', u' 0.09 ( 1.00x)  1.27 ( 1.00x) 15.64 (
1.00x) 184.76 ( 1.00x)')
(u'', u'Heap   ', u' 0.10 ( 1.07x)  1.33 ( 1.05x) 16.30 (
1.04x) 191.43 ( 1.04x)')
(u'', u'RemovalHeap', u' 0.15 ( 1.64x)  1.87 ( 1.47x) 21.33 (
1.36x) 242.31 ( 1.31x)')

(u'push', u'heapq  ', u' 0.04 ( 1.00x)  0.33 ( 1.00x)  3.53 (
1.00x) 32.99 ( 1.00x)')
(u'', u'Heap   ', u' 0.05 ( 1.20x)  0.41 ( 1.25x)  4.37 (
1.24x) 44.14 ( 1.34x)')
(u'', u'RemovalHeap', u' 0.06 ( 1.58x)  0.56 ( 1.70x)  5.85 (
1.66x) 59.59 ( 1.81x)')


(u'init', u'heapq', u' 0.45 ( 1.00x)  8.00 ( 1.00x) 109.50 (
1.00x) 898.50 ( 1.00x)')
(u'', u'OrderHeap', u' 0.50 ( 1.10x)  8.75 ( 1.09x) 112.63 (
1.03x) 1108.26 ( 1.23x)')
(u'', u'XHeap', u' 0.93 ( 2.06x) 13.88 ( 1.74x) 170.97 (
1.56x) 1709.94 ( 1.90x)')

(u'pop ', u'heapq', u' 0.04 ( 1.00x)  0.54 ( 1.00x)  6.50 ( 1.00x)
76.64 ( 1.00x)')
(u'', u'OrderHeap', u' 0.06 ( 1.73x)  0.80 ( 1.49x)  9.16 ( 1.41x)
101.68 ( 1.33x)')
(u'', u'XHeap', u' 0.10 ( 2.65x)  1.14 ( 2.13x) 12.60 ( 1.94x)
137.73 ( 1.80x)')

(u'push', u'heapq', u' 0.01 ( 1.00x)  0.17 ( 1.00x)  1.86 ( 1.00x)
14.85 ( 1.00x)')
(u'', u'OrderHeap', u' 0.04 ( 3.88x)  0.47 ( 2.86x)  4.80 ( 2.58x)
42.98 ( 2.89x)')
(u'', u'XHeap', u' 0.04 ( 3.79x)  0.47 ( 2.85x)  4.85 ( 2.61x)
43.94 ( 2.96x)')


(u'remove', u'RemovalHeap', u' 0.07 ( 1.00x)  0.73 ( 1.00x)  7.38 (
1.00x) 75.18 ( 1.00x)')
(u'  ', u'XHeap  ', u' 0.05 ( 0.79x)  0.55 ( 0.75x)  5.52 (
0.75x) 55.08 ( 0.73x)')




** benchmark for current git HEAD d0707fba2401a3cef8aed54028fe6d7e9497faa5 **


(u'init', u'heapq  ', u' 0.49 ( 1.00x)  5.03 ( 1.00x) 58.71 (
1.00x) 600.91 ( 1.00x)')
(u'', u'Heap   ', u' 0.51 ( 1.05x)  5.83 ( 1.16x) 82.88 (
1.41x) 886.22 ( 1.47x)')
(u'', u'RemovalHeap', u' 0.58 ( 1.19x)  7.19 ( 1.43x) 106.80 (
1.82x) 1108.52 ( 1.84x)')

(u'pop ', u'heapq  ', u' 0.10 ( 1.00x)  1.41 ( 1.00x) 17.64 (
1.00x) 209.82 ( 1.00x)')
(u'', u'Heap   ', u' 0.11 ( 1.07x)  1.47 ( 1.04x) 18.27 (
1.04x) 215.14 ( 1.03x)')
(u'', u'RemovalHeap', u' 0.15 ( 1.52x)  1.91 ( 1.35x) 22.64 (
1.28x) 258.68 ( 1.23x)')

(u'push', u'heapq  ', u' 0.04 ( 1.00x)  0.32 ( 1.00x)  3.49 (
1.00x) 33.92 ( 1.00x)')
(u'', u'Heap   ', u' 0.05 ( 1.18x)  0.39 ( 1.22x)  4.21 (
1.20x) 42.03 ( 1.24x)')
(u'', u'RemovalHeap', u' 0.06 ( 1.52x)  0.52 ( 1.62x)  5.60 (
1.60x) 56.54 ( 1.67x)')


(u'init', u'heapq', u' 0.44 ( 1.00x)  7.92 ( 1.00x) 106.52 (
1.00x) 915.20 ( 1.00x)')
(u'', u'OrderHeap', u' 0.50 ( 1.14x)  8.67 ( 1.10x) 111.99 (
1.05x) 1129.89 ( 1.23x)')
(u'', u'XHeap', u' 0.61 ( 1.38x) 10.75 ( 1.36x) 140.86 (
1.32x) 1417.84 ( 1.55x)')

(u'pop ', u'heapq', u' 0.04 ( 1.00x)  0.55 ( 1.00x)  6.59 ( 1.00x)
76.81

Re: Pyhon 2.x or 3.x, which is faster?

[Steven D'Aprano]

On Tuesday 08 March 2016 12:41, BartC wrote:

> On 08/03/2016 01:19, Steven D'Aprano wrote:
>> On Tue, 8 Mar 2016 07:19 am, BartC wrote:
>>
>>> I don't have to hand a jpeg file that it can't
>>> decode.
>>
>> Run this under Python 2:
>>
>> from random import randint
>> blob = [randint(0, 256) for i in range(16*1024)]
>>
>> with open('broken.jpg', 'wb') as f:
>>  f.write(''.join(blob))
>>
>> If your software can decode that, it will be a miracle.
>>
> 
> That's not a jpeg file.

Nevertheless, in the real world, you have to deal with corrupt JPGs and 
files mislabelled as JPGs. And "crashing" doesn't count as "deal with" :-)


> (You code didn't run even on Python 2


Serves me right for not testing it before posting :-(

Change the second line to:

blob = [chr(randint(0, 255)) for i in range(16*1024)]

and it works for me.




-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Terry Reedy]

On 3/7/2016 7:22 PM, BartC wrote:


(Is a byte string the same as a byte array?


No, if by 'byte array' mean (mutable) bytearray.
Yes, in the sense that a byte string is an (immutable) array of ints in 
range(256).



Is a byte array the same as an array.array?

No, if by 'byte array' you mean the same as bytes.
Yes, if you mean bytearray and array.array('B'), in the sense that 
bytearray is based on array.array('B').




If I remove this line from my code, where 'data' has

> just been read from a file:


data=array.array('B',data)

then it still works - Python 3.


In 3.3, data = bytearray(data) would have the same effect, and the only 
effect is to make data mutable, but if you do not mutate the image 
bytes, that is useless.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 08/03/2016 03:39, Terry Reedy wrote:

On 3/7/2016 8:33 PM, BartC wrote:

On Tue, Mar 8, 2016 at 12:00 PM, BartC  wrote:



def whiletest():
|   i=0
|   while i<=1:
|   |   i+=1

whiletest()

Python 2.7:  8.4 seconds
Python 3.1: 12.5 seconds
Python 3.4: 18.0 seconds

Even if you don't care about speed, you must admit that there appears
to be
something peculiar going on here: why would 3.4 take more than twice
as long
as 2.7? What do they keep doing to 3.x to cripple it on each new
version?



Let me ask you a follow-on question first: how slow does a new Python
version have to be before even you would take notice?


We now run a suite of benchmarks daily on latest versions of 2.7 and
default (3.6) to compare not to each other but to previous days to
detect changes within either branch.

I verified your result with installed 64 bit 2.7.11 versus 3.5.1

import time
def test():
 i=0
 while i<=1:
 i+=1
start = time.time()
test()
print(time.time() - start)

4.4 (2.7) versus 10.7 (3.5)

Running loop at top level instead of inside the function doubled the
time.  Replacing globals dict lookup with function locals array lookup
really helps.

Next, I replaced the body of the function with
 for i in range(1): pass

This time, 3.5 wins: 3.8 versus 2.7.  Whoops, unfair. Change range to
xrange in 2.7 and the time is 1.5.

Neither version optimizes the do-nothing loop away.  A further version
of CPython might.  There are people working on improving the speed of
CPython.  Integer operations are not their focus, though, because that
can be done in numpy.  Text operations have been getting work, and at
least one person is actively working on an ast optimizer.



For reference all of the tips of the trade are given here 
https://wiki.python.org/moin/PythonSpeed/PerformanceTips


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Mark Lawrence]

On 07/03/2016 23:09, Fillmore wrote:


I must be missing something simple because I can't find a way to break
out of a nested loop in Python.

Is there a way to label loops?

For the record, here's a Perl script of mine I'm trying to port...there
may be 'malformed' lines in a TSV file I'm parsing that are better
discarded than fixed.



[snipped the code as I don't understand it]

Another interesting read that I've found 
http://nedbatchelder.com/blog/201203/breaking_out_of_two_loops_at_once.html


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Terry Reedy]

On 3/7/2016 8:33 PM, BartC wrote:

On Tue, Mar 8, 2016 at 12:00 PM, BartC  wrote:



def whiletest():
|   i=0
|   while i<=1:
|   |   i+=1

whiletest()

Python 2.7:  8.4 seconds
Python 3.1: 12.5 seconds
Python 3.4: 18.0 seconds

Even if you don't care about speed, you must admit that there appears
to be
something peculiar going on here: why would 3.4 take more than twice
as long
as 2.7? What do they keep doing to 3.x to cripple it on each new
version?



Let me ask you a follow-on question first: how slow does a new Python
version have to be before even you would take notice?


We now run a suite of benchmarks daily on latest versions of 2.7 and 
default (3.6) to compare not to each other but to previous days to 
detect changes within either branch.


I verified your result with installed 64 bit 2.7.11 versus 3.5.1

import time
def test():
i=0
while i<=1:
i+=1
start = time.time()
test()
print(time.time() - start)

4.4 (2.7) versus 10.7 (3.5)

Running loop at top level instead of inside the function doubled the 
time.  Replacing globals dict lookup with function locals array lookup 
really helps.


Next, I replaced the body of the function with
for i in range(1): pass

This time, 3.5 wins: 3.8 versus 2.7.  Whoops, unfair. Change range to 
xrange in 2.7 and the time is 1.5.


Neither version optimizes the do-nothing loop away.  A further version 
of CPython might.  There are people working on improving the speed of 
CPython.  Integer operations are not their focus, though, because that 
can be done in numpy.  Text operations have been getting work, and at 
least one person is actively working on an ast optimizer.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Terry Reedy]

On 3/7/2016 6:49 PM, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 10:42 AM, Chris Kaynor  wrote:

And the same rough example, using an exception without a function:

for file in files:
 try:
 for line in file:
 section = line.split()
 for section in line:
 if sectionCorrupt:
 raise Exception('Section corrupt') # You probably want
more details here, for possible manual repair. You could also have a custom
exception class for more control.
 except Exception as e:
 print('Failed to process the file {} with error {}.'.format(file,
str(e))) # Probably should also print the entire traceback to aid in
repairing errors, especially if they are due to a bug in the code, but this
is the rough idea.


Yes, although I would more strongly suggest the custom exception
class. In fact, the way I'd word it is: Never raise Exception, always
a subclass. Otherwise it's too easy to accidentally catch something
elsewhere in the code (a ValueError or TypeError or even
AttributeError).


I would consider raising and catching StopIteration.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 07/03/2016 11:38, BartC wrote:

On 07/03/2016 11:11, Marko Rauhamaa wrote:

BartC :


I've also found that 3 was consistently slower than 2 on various
benchmarks. Perhaps 10 to 20% slower (also 3.4 vs. 2.7).


Python doesn't exist for performance-critical parts of your solution.
Also, Python programs tend to take huge amounts of space.


I'm not complaining. For my purposes (implementing a similar language
that competes with Python for speed), I welcome the slower speed of
Python 3!

(Although competing with CPython is too easy. PyPy is more of a problem.
With the Jpeg benchmark I mentioned, I can beat PyPy up to 6Mpix, but
then PyPy starts to get faster. At 80Mpix, PyPy is 60% faster.)



I suspect that all that is happening is you are ignoring the Python 
startup time, so the more data you process, the better the figures will 
look for any version of Python.


As for the real world, as opposed to benchmarks, how about this 
https://www.willmcgugan.com/blog/tech/post/realtime-events-with-the-inthingio-python-api/ 
?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[MRAB]

On 2016-03-08 01:33, BartC wrote:

On 08/03/2016 01:23, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 12:00 PM, BartC  wrote:

Yes of course it does. As does 'being slow'. Take another microbenchmark:

def whiletest():
|   i=0
|   while i<=1:
|   |   i+=1

whiletest()

Python 2.7:  8.4 seconds
Python 3.1: 12.5 seconds
Python 3.4: 18.0 seconds

Even if you don't care about speed, you must admit that there appears to be
something peculiar going on here: why would 3.4 take more than twice as long
as 2.7? What do they keep doing to 3.x to cripple it on each new version?


How do your benchmarks compare on this code:

pass


Let me ask you a follow-on question first: how slow does a new Python
version have to be before even you would take notice?

Compared with 2.7, 3.4 above is spending nearly an extra ten seconds
doing  what? I can't understand why someone just wouldn't care.

Part of it will be that Python 2 has 2 integer types: 'int' (fixed 
length) and 'long' (variable length).


Originally, 'int' addition could overflow, but it was more friendly to 
promote the result to 'long' instead.


Python 3 dropped 'int' and renamed 'long' to 'int'.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 08/03/2016 01:47, BartC wrote:

On 08/03/2016 01:12, Mark Lawrence wrote:

On 08/03/2016 01:00, BartC wrote:


If your efforts manage to double the speed of reading file A, then
probably the reading file B is also going to be improved! In practice
you use a variety of files, but one at a time will do.



What is the difference in your timing when you first read the file, and
then read it a second time when it's been cached by the OS?  In other
words, you are probably measuring more of the response time of the disk
than the code that does the reading, hence making your figures useless.



It's not going to be significant. My hard drive is going to read at,
what, 100MB per second? Probably more.

One test file is 0.2MB. Load time is going to be negligible whether
cached or not.

The Python timing for that file is around 20 seconds, time enough to
read 1 copies from the disk.

And a C program reads /and decodes/ the same file from the same disk in
between 0.1 and 0.2 seconds.



So how much of that time is Python startup time, compared to C which is 
effectively zero?  Or are you suggesting that C code is always 100 times 
faster than Python?  Of course I'd like to see you write C code 100 
times faster than Python, but of course that's where Python shines, 
which is why it is so popular.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[MRAB]

On 2016-03-08 01:37, Jinghui Niu wrote:

On Monday, March 7, 2016 at 5:16:44 PM UTC-8, Ben Finney wrote:

Jinghui Niu  writes:

> May I take this opportunity to ask a bold question for a beginner

No problem, your questions are on topic here. Thank you for being civil.

> if I want to mimic Sublime Text's fuzzy search for a file search in
> given directories, which module of these two would be more suitable?

You'll need to describe the requirements, I don't know exactly what that
behaviour is.

--
 \  "He that loveth father or mother more than me is not worthy of |
  `\me: and he that loveth son or daughter more than me is not |
_o__)worthy of me." --Jesus, as quoted in Matthew 10:37 |
Ben Finney


Let me try to describe the behaviours as much as I can here: It is a real-time search, updating the search result as you type; It 
is very smart, not limiting itself into the verbatim words, but extracting feature strings automatically, e.g., typing 
"fontz" will match not just "*fontz*", but also "font-size", "fontzipper", etc; It can be 
switched for certain type of files according specific rules, such as not including ".pyc" files.

My gut feeling is that it is using Regex, but just wonder how I can fit Regex 
into either fnmatch() or glob() module.


Could it be as simple as looking for _those_ characters in _that_ order?

It might be giving a higher prominence to those whose matching letters 
are closer together.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Ben Finney]

BartC  writes:

> On 08/03/2016 01:40, Chris Angelico wrote:
> > You need to be VERY clear about exactly what you're measuring. Are
> > you using the 'timeit' module to measure execution of one line of
> > code? Are you putting your code into a file and running that with
> > /usr/bin/time? Are you putting the code into Idle and running it in
> > a loop with 'exec' and using time.time() around the outside? Your
> > numbers do not concern me *because they mean nothing*.
>
> So they're just random numbers?

That's not an implication from what Chris is saying. Rather, the
implication is the numbers do not have the meaning you're ascribing to
them.

> I write interpreters. If I suddenly saw a 115% increase in runtime
> from one version to another, /even on a trivial 2-line loop/ (or even
> on /any/ program doing the same task), then I'd want to know why!
> Because experience tells me that something is wrong.

Sure. Experience is not, though, going to tell you *what* is wrong until
you can be much more specific in your observations, and verifiably
attribute observations to behaviour.

Until then, with all the experience in the world, to claim you know the
explanation is merely unfounded assertion.

-- 
 \ “Skepticism is the highest duty and blind faith the one |
  `\   unpardonable sin.” —Thomas Henry Huxley, _Essays on |
_o__)   Controversial Questions_, 1889 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Steven D'Aprano]

On Tue, 8 Mar 2016 12:04 pm, Jinghui Niu wrote:

> May I take this opportunity to ask a bold question for a beginner: if I
> want to mimic Sublime Text's fuzzy search for a file search in given
> directories, which module of these two would be more suitable? Could you
> shed some insights? Thanks.

Read the documentation. See what functionality each module provides.

Here is the documentation for Python 3:

https://docs.python.org/3/library/glob.html
https://docs.python.org/3/library/fnmatch.html


and for Python 2:

https://docs.python.org/2/library/glob.html
https://docs.python.org/2/library/fnmatch.html


Reading the source is also a great way to learn about the module, and to
learn what is considered best practice in Python:


https://hg.python.org/cpython/file/3.5/Lib/glob.py
https://hg.python.org/cpython/file/3.5/Lib/fnmatch.py



This may also be helpful:


https://pymotw.com/2/glob/
https://pymotw.com/2/fnmatch/



But I shall give you a hint: the glob module imports the fnmatch module, and
uses it to perform the matching.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Steven D'Aprano]

On Tue, 8 Mar 2016 09:39 am, BartC wrote:

> On 07/03/2016 20:47, Chris Angelico wrote:
>> On Tue, Mar 8, 2016 at 7:19 AM, BartC  wrote:
> 
>>> What can be more perfect for comparing two implementations?
> 
>> rosuav@sikorsky:~$ python2 -m timeit -s 'from fp import Float'
>> 'Float("1234.567")'
>> 100 loops, best of 3: 1.84 usec per loop
>> rosuav@sikorsky:~$ python3 -m timeit -s 'from fp import Float'
>> 'Float("1234.567")'
>> 10 loops, best of 3: 2.76 usec per loop
>>
>> Look! Creating a floating-point value is faster under Python 2 than
>> Python 3. What could be more perfect?
> 
>> This is like a microbenchmark in that it doesn't tell you anything
>> about real-world usage.
> 
> Microbenchmarks have their uses, when you are concentrating on a
> particular aspect of a language. But I tried your code, calling Float a
> million times, and 2.7 took 8.3 seconds; 3.4 took 10.5 seconds. But that
> is meaningless according to you.

I think Chris is looking at this as a fair test of Python's speed. Nobody in
their right mind would use his pure-Python Float when built-in floats
exist.

But I think Chris is wrong. He may remember that Python gained a Decimal
class written in pure Python. Does he think that it is invalid to ask how
fast Decimal is? Surely not. Creating millions of Decimal instances might
not be the single biggest bottleneck slowing your code down, but I'm pretty
sure we would want that to be as fast as possible.

(In fact, in Python 3.4 [by memory], Decimal was re-written in C for speed.)

Anyway, just for comparisons sake, I ran the same micro-benchmark:

[steve@ando ~]$ python2.6 -m timeit -s 'from fp import
Float' 'Float("1234.567")'
10 loops, best of 3: 11.1 usec per loop

[steve@ando ~]$ python2.7 -m timeit -s 'from fp import
Float' 'Float("1234.567")'
10 loops, best of 3: 12.1 usec per loop

[steve@ando ~]$ python3.3 -m timeit -s 'from fp import
Float' 'Float("1234.567")'
10 loops, best of 3: 13.6 usec per loop

[steve@ando ~]$ python/python-dev/3.5/python -m timeit -s 'from fp import
Float' 'Float("1234.567")'
1 loops, best of 3: 54 usec per loop


So you can see on my computer, there is an apparent slowdown between 2.6 and
2.7, and 2.7 and 3.3. I say "apparent" because it may not be statistically
meaningful: time results are notoriously fickle. I ran 2.7 again three
times, and got three faster results:

10 loops, best of 3: 11.8 usec per loop
10 loops, best of 3: 11.7 usec per loop
10 loops, best of 3: 11.6 usec per loop


I'm pretty sure that the "11.8 .7 .6" pattern is just a coincidence, because
I ran it again:

10 loops, best of 3: 12.4 usec per loop

So there is considerable variation in timing results.

What about 3.5? That's four times slower than 3.3? What happened there?

Simple: I compiled 3.3 with all the debugging code turned on.



> (3.1 took 8.5 seconds. What happened between 3.1 and 3.4 to cause such a
> slow-down? Do you think it might be a good idea for /someone/ at least
> to take pay some attention to that, before it grinds to a halt
> completely by version 4.0?)

This is the whole point of the speed.python.org site, to monitor and
benchmark the speed of the language.


[...]
>> CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
>> going to differ in performance. And that's without even looking at
>> what core devs would refer to as "other Pythons", which would include
>> IronPython, Jython, PyPy (well, you got that, but you're treating it
>> as an afterthought), MicroPython, Brython, wpython, Nuitka,
>> Cython. these are *other Pythons*.
> 
> What are you suggesting here? That all these Pythons are going to be
> faster or slower than each other? I would guess that most of them are
> going to be roughly the same, other than PyPy. If there was a fast
> version, then I would have heard about it!

I think Chris exaggerates the performance differences between bug fix
releases, at least in general. There's unlikely to be a major change in the
interpreter applied to (say) 2.7.11 that would speed it up drastically
compared to 2.7.0. But there may be a series of small performance
enhancements which, *together*, add up to a moderate increase in overall
speed (while being all but invisible to sufficiently small
micro-benchmarks.

But typically, it is very, very hard to definitively say that one version or
implementation of Python is faster than another. That will often depend on
what you're trying to do! But, with lots of hand-waving and a certain
amount of trepidation, I'd like to offer this as the "conventional wisdom"
for the speed of pure-Python on a semi-arbitrary set of benchmarks which
may or may not reflect actual use by anyone:

# slowest
jython
cpython + stackless
ironpython
pypy
# fastest



>> the
>> performance of array.array() is far from stable. It's not a core
>> language feature; you're using it because it's the most obvious
>> translation of your C algorithm,
> 
> I'm using it because this kind of file r

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 08/03/2016 01:40, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 12:33 PM, BartC  wrote:



Let me ask you a follow-on question first: how slow does a new Python
version have to be before even you would take notice?

Compared with 2.7, 3.4 above is spending nearly an extra ten seconds doing
 what? I can't understand why someone just wouldn't care.


Performance matters, when you actually have something useful to
measure. Startup performance matters enormously if interpreter startup
is what you're doing a lot of (for example, the "feel" of Mercurial
depends heavily on Python startup performance). It matters not a whit
if your process keeps running for a long time, and handles many
requests (for example, a web server).

You need to be VERY clear about exactly what you're measuring. Are you
using the 'timeit' module to measure execution of one line of code?
Are you putting your code into a file and running that with
/usr/bin/time? Are you putting the code into Idle and running it in a
loop with 'exec' and using time.time() around the outside? Your
numbers do not concern me *because they mean nothing*.


So they're just random numbers?

I write interpreters. If I suddenly saw a 115% increase in runtime from 
one version to another, /even on a trivial 2-line loop/ (or even on 
/any/ program doing the same task), then I'd want to know why! Because 
experience tells me that something is wrong.


Of course I don't know here what they've been doing to Python. Maybe 
they know exactly why it's slower. But then they should explain why it 
is, otherwise it could be a bug. Maybe someone left a debug flag turned 
on or something. Or there's some other technical reason, but it would be 
nice to know what it is.


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[justin walters]

Well, from that error message, it seems like you may have a permissions
issue. Also, the git shell is not the sane as a real shell. There are
multiple windows terminal emulators. Why not use one of those?
On Mar 7, 2016 5:46 PM, "Jon Ribbens"  wrote:

> On 2016-03-07, Ian Kelly  wrote:
> > On Mon, Mar 7, 2016 at 11:51 AM, Jon Ribbens
> > wrote:
> >> I must say that Python on Windows was a very poor experience indeed,
> >> "virtualenv" does not work and "venv" refuses to create the 'activate'
> >> shell script so does not work either
> >
> > I've used both of these on Windows (although not recently) and had no
> > trouble with them. I never had a problem with venv not creating the
> > activate.bat file.
>
> It's not activate.bat, it's activate (no file extension) the posix
> shell script. I installed Git for Windows which provides bash (or
> something that looks like it). Python venv doesn't cope with this
> situation at all.
>
> 'virtualenv' works even less well, it just says:
>
> $ virtualenv test
> Using base prefix 'd:\\program files (x86)\\python35-32'
> New python executable in D:\Users\Jon
> Ribbens\Documents\Python\test\Scripts\python.exe
> ERROR: The executable "D:\Users\Jon
> Ribbens\Documents\Python\test\Scripts\python.exe" could not be run:
> [WinError 5] Access is denied
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Chris Angelico]

On Tue, Mar 8, 2016 at 12:37 PM, Jinghui Niu  wrote:
> Let me try to describe the behaviours as much as I can here: It is a 
> real-time search, updating the search result as you type; It is very smart, 
> not limiting itself into the verbatim words, but extracting feature strings 
> automatically, e.g., typing "fontz" will match not just "*fontz*", but also 
> "font-size", "fontzipper", etc; It can be switched for certain type of files 
> according specific rules, such as not including ".pyc" files.
>
> My gut feeling is that it is using Regex, but just wonder how I can fit Regex 
> into either fnmatch() or glob() module.

Sounds to me like you're looking at two different things.

1) File name matching, whether with fnmatch or glob, will tell you
which files to look at. You mention excluding .pyc files; you could
specify that only "*.py" be matched, or "*q*.py", or whatever.

2) Regular expression or other searching, within the files.

Either fnmatch or glob will serve you well for the first; I don't
think they're appropriate for the second. But it depends what you're
trying to do.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Ben Finney]

Jinghui Niu  writes:

> Let me try to describe the behaviours as much as I can here: It is a
> real-time search, updating the search result as you type; It is very
> smart, not limiting itself into the verbatim words, but extracting
> feature strings automatically, e.g., typing "fontz" will match not
> just "*fontz*", but also "font-size", "fontzipper", etc; It can be
> switched for certain type of files according specific rules, such as
> not including ".pyc" files.

None of that really depends on the difference between ‘glob’ versus
‘fnmatch’. Either of them could be at the bowels of an implementation of
the behaviour you describe.

> My gut feeling is that it is using Regex, but just wonder how I can
> fit Regex into either fnmatch() or glob() module.

You don't. Those two are *much* lower level than the behaviour you
describe, and don't have anything to say about that behaviour.

You will need all the extra framework implementing all the other stuff
you describe, before you make any decision between ‘fnmatch’ versus
‘glob’.

-- 
 \ “Skepticism is the highest duty and blind faith the one |
  `\   unpardonable sin.” —Thomas Henry Huxley, _Essays on |
_o__)   Controversial Questions_, 1889 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 12:34 PM, Steven D'Aprano  wrote:
> On Tue, 8 Mar 2016 07:47 am, Chris Angelico wrote:
>
>> You write *real world* code and then profile that. You get actual real
>> programs that you actually really use, and you run those through
>> timing harnesses.
>
>
> Chris, I think that's exactly what BartC has done: he has a program that he
> actually uses, one which processes JPG files. It's written in pure Python,
> so he's not comparing the quality of C libraries, he's comparing Python
> versus Python.

The trouble is that we can't be sure _what_ he's done. All we have is
that it takes longer under some circumstances than others. Without
knowing a lot more about how the measurements are done, I don't know
that we can get anything from it.

And I remember seeing a performance analysis that showed that
array.array() actually sucks for performance; hence my recommendation
to try other ways of doing things, before saying "Python 3 is slower
than Python 2".

> Could Bart's code be improved for production use? Almost certainly. I'm sure
> that by using a C image processing library, like pillow, it would be ten or
> a hundred times faster. If Bart were saying "Python is crap, it's too slow"
> then a perfectly acceptable response would be "no, you're just misusing it,
> here you want to use it as an interface to this library and let the library
> do the heavy lifting". That's what Python is designed for. But that's not
> what Bart is saying.
>
> I'm impressed that pure Python code running in CPython is even *usable* for
> whatever sort of image processing BartC is doing. He must be doing
> something right, given that its not unusably slow.

Fair point. Although these are only small files (as far as I know), so
even if there's an O(N**2) memory allocation (eg with a naive "take a
byte off the front and give me back the rest" algorithm), it'd quite
probably still be usable. I've had N Squareds in code that sat there
until the day I did some stupid benchmark on a gig of data, and only
then started seeing problems. Readably-written code isn't necessarily
that much slower than tightly-optimized code.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 08/03/2016 01:12, Mark Lawrence wrote:

On 08/03/2016 01:00, BartC wrote:


If your efforts manage to double the speed of reading file A, then
probably the reading file B is also going to be improved! In practice
you use a variety of files, but one at a time will do.



What is the difference in your timing when you first read the file, and
then read it a second time when it's been cached by the OS?  In other
words, you are probably measuring more of the response time of the disk
than the code that does the reading, hence making your figures useless.



It's not going to be significant. My hard drive is going to read at, 
what, 100MB per second? Probably more.


One test file is 0.2MB. Load time is going to be negligible whether 
cached or not.


The Python timing for that file is around 20 seconds, time enough to 
read 1 copies from the disk.


And a C program reads /and decodes/ the same file from the same disk in 
between 0.1 and 0.2 seconds.


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 08/03/2016 01:19, Steven D'Aprano wrote:

On Tue, 8 Mar 2016 07:19 am, BartC wrote:


I don't have to hand a jpeg file that it can't
decode.



Run this under Python 2:



from random import randint
blob = [randint(0, 256) for i in range(16*1024)]

with open('broken.jpg', 'wb') as f:
 f.write(''.join(blob))




If your software can decode that, it will be a miracle.



That's not a jpeg file.

(You code didn't run even on Python 2, but I created a 16KB file of 
random bytes by other means. It said 'Reading past end of data', 
presumably looking for some marker or other.)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, Ian Kelly  wrote:
> On Mon, Mar 7, 2016 at 11:51 AM, Jon Ribbens
> wrote:
>> I must say that Python on Windows was a very poor experience indeed,
>> "virtualenv" does not work and "venv" refuses to create the 'activate'
>> shell script so does not work either
>
> I've used both of these on Windows (although not recently) and had no
> trouble with them. I never had a problem with venv not creating the
> activate.bat file.

It's not activate.bat, it's activate (no file extension) the posix
shell script. I installed Git for Windows which provides bash (or
something that looks like it). Python venv doesn't cope with this
situation at all.

'virtualenv' works even less well, it just says:

$ virtualenv test
Using base prefix 'd:\\program files (x86)\\python35-32'
New python executable in D:\Users\Jon 
Ribbens\Documents\Python\test\Scripts\python.exe
ERROR: The executable "D:\Users\Jon 
Ribbens\Documents\Python\test\Scripts\python.exe" could not be run: [WinError 
5] Access is denied
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 12:33 PM, BartC  wrote:
> On 08/03/2016 01:23, Chris Angelico wrote:
>>
>> On Tue, Mar 8, 2016 at 12:00 PM, BartC  wrote:
>>>
>>> Yes of course it does. As does 'being slow'. Take another microbenchmark:
>>>
>>> def whiletest():
>>> |   i=0
>>> |   while i<=1:
>>> |   |   i+=1
>>>
>>> whiletest()
>>>
>>> Python 2.7:  8.4 seconds
>>> Python 3.1: 12.5 seconds
>>> Python 3.4: 18.0 seconds
>>>
>>> Even if you don't care about speed, you must admit that there appears to
>>> be
>>> something peculiar going on here: why would 3.4 take more than twice as
>>> long
>>> as 2.7? What do they keep doing to 3.x to cripple it on each new version?
>>
>>
>> How do your benchmarks compare on this code:
>>
>> pass
>
>
> Let me ask you a follow-on question first: how slow does a new Python
> version have to be before even you would take notice?
>
> Compared with 2.7, 3.4 above is spending nearly an extra ten seconds doing
>  what? I can't understand why someone just wouldn't care.

Performance matters, when you actually have something useful to
measure. Startup performance matters enormously if interpreter startup
is what you're doing a lot of (for example, the "feel" of Mercurial
depends heavily on Python startup performance). It matters not a whit
if your process keeps running for a long time, and handles many
requests (for example, a web server).

You need to be VERY clear about exactly what you're measuring. Are you
using the 'timeit' module to measure execution of one line of code?
Are you putting your code into a file and running that with
/usr/bin/time? Are you putting the code into Idle and running it in a
loop with 'exec' and using time.time() around the outside? Your
numbers do not concern me *because they mean nothing*.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Jinghui Niu]

On Monday, March 7, 2016 at 5:16:44 PM UTC-8, Ben Finney wrote:
> Jinghui Niu  writes:
> 
> > May I take this opportunity to ask a bold question for a beginner
> 
> No problem, your questions are on topic here. Thank you for being civil.
> 
> > if I want to mimic Sublime Text's fuzzy search for a file search in
> > given directories, which module of these two would be more suitable?
> 
> You'll need to describe the requirements, I don't know exactly what that
> behaviour is.
> 
> -- 
>  \  "He that loveth father or mother more than me is not worthy of |
>   `\me: and he that loveth son or daughter more than me is not |
> _o__)worthy of me." --Jesus, as quoted in Matthew 10:37 |
> Ben Finney

Let me try to describe the behaviours as much as I can here: It is a real-time 
search, updating the search result as you type; It is very smart, not limiting 
itself into the verbatim words, but extracting feature strings automatically, 
e.g., typing "fontz" will match not just "*fontz*", but also "font-size", 
"fontzipper", etc; It can be switched for certain type of files according 
specific rules, such as not including ".pyc" files.

My gut feeling is that it is using Regex, but just wonder how I can fit Regex 
into either fnmatch() or glob() module.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, Chris Angelico  wrote:
> 3) Use standard Python, and ditch Windows. This is what I do. :)

This is also what I do nearly all of the time, but for PyWeek I needed
a computer running Python that actually had a monitor attached ;-)

Although to be fair, Python under Ubuntu Server has for a long time
been a very poor experience too, what with the pip problem.

(I can't remember what the error message is but if you do
'sudo apt-get install python python-pip' and then use pip to upgrade
either pip or requests, it breaks pip completely and irreversibly,
you have to manually re-install it using get-pip.py. It's probably
this one:
https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1306991)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Ben Finney]

BartC  writes:

> Let me ask you a follow-on question first: how slow does a new Python
> version have to be before even you would take notice?

Slow *at doing what*, in particular? It matters which specific operation
we're discussing.

-- 
 \   “The generation of random numbers is too important to be left |
  `\to chance.” —Robert R. Coveyou |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Steven D'Aprano]

On Tue, 8 Mar 2016 07:47 am, Chris Angelico wrote:

> You write *real world* code and then profile that. You get actual real
> programs that you actually really use, and you run those through
> timing harnesses.


Chris, I think that's exactly what BartC has done: he has a program that he
actually uses, one which processes JPG files. It's written in pure Python,
so he's not comparing the quality of C libraries, he's comparing Python
versus Python.

I haven't looked at his code, so I don't know if its a fair comparison of
Python versus Bart's Private Language. But that's not the point. Idiomatic
Python or not, "best practice" production level code or not, it is a fair
comparison of Python 2 versus Python 3, or CPython versus PyPy, because
it's the same Python code being run each time.

Could Bart's code be improved for production use? Almost certainly. I'm sure
that by using a C image processing library, like pillow, it would be ten or
a hundred times faster. If Bart were saying "Python is crap, it's too slow"
then a perfectly acceptable response would be "no, you're just misusing it,
here you want to use it as an interface to this library and let the library
do the heavy lifting". That's what Python is designed for. But that's not
what Bart is saying.

I'm impressed that pure Python code running in CPython is even *usable* for
whatever sort of image processing BartC is doing. He must be doing
something right, given that its not unusably slow.




-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 08/03/2016 01:23, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 12:00 PM, BartC  wrote:

Yes of course it does. As does 'being slow'. Take another microbenchmark:

def whiletest():
|   i=0
|   while i<=1:
|   |   i+=1

whiletest()

Python 2.7:  8.4 seconds
Python 3.1: 12.5 seconds
Python 3.4: 18.0 seconds

Even if you don't care about speed, you must admit that there appears to be
something peculiar going on here: why would 3.4 take more than twice as long
as 2.7? What do they keep doing to 3.x to cripple it on each new version?


How do your benchmarks compare on this code:

pass


Let me ask you a follow-on question first: how slow does a new Python 
version have to be before even you would take notice?


Compared with 2.7, 3.4 above is spending nearly an extra ten seconds 
doing  what? I can't understand why someone just wouldn't care.


--
Bartc

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 12:00 PM, BartC  wrote:
> Yes of course it does. As does 'being slow'. Take another microbenchmark:
>
> def whiletest():
> |   i=0
> |   while i<=1:
> |   |   i+=1
>
> whiletest()
>
> Python 2.7:  8.4 seconds
> Python 3.1: 12.5 seconds
> Python 3.4: 18.0 seconds
>
> Even if you don't care about speed, you must admit that there appears to be
> something peculiar going on here: why would 3.4 take more than twice as long
> as 2.7? What do they keep doing to 3.x to cripple it on each new version?

How do your benchmarks compare on this code:

pass

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Steven D'Aprano]

On Tue, 8 Mar 2016 07:19 am, BartC wrote:

> I don't have to hand a jpeg file that it can't
> decode.


Run this under Python 2:



from random import randint
blob = [randint(0, 256) for i in range(16*1024)]

with open('broken.jpg', 'wb') as f:
f.write(''.join(blob))




If your software can decode that, it will be a miracle.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Ben Finney]

Jinghui Niu  writes:

> May I take this opportunity to ask a bold question for a beginner

No problem, your questions are on topic here. Thank you for being civil.

> if I want to mimic Sublime Text's fuzzy search for a file search in
> given directories, which module of these two would be more suitable?

You'll need to describe the requirements, I don't know exactly what that
behaviour is.

-- 
 \  “He that loveth father or mother more than me is not worthy of |
  `\me: and he that loveth son or daughter more than me is not |
_o__)worthy of me.” —Jesus, as quoted in Matthew 10:37 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 08/03/2016 01:00, BartC wrote:


If your efforts manage to double the speed of reading file A, then
probably the reading file B is also going to be improved! In practice
you use a variety of files, but one at a time will do.



What is the difference in your timing when you first read the file, and 
then read it a second time when it's been cached by the OS?  In other 
words, you are probably measuring more of the response time of the disk 
than the code that does the reading, hence making your figures useless.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 08/03/2016 00:05, Ben Finney wrote:

BartC  writes:


On 07/03/2016 20:47, Chris Angelico wrote:

Look! Creating a floating-point value is faster under Python 2 than
Python 3. What could be more perfect?



This is like a microbenchmark in that it doesn't tell you anything
about real-world usage.


Microbenchmarks have their uses, when you are concentrating on a
particular aspect of a language.


The “micro” is not referring to the focus of the operation; that's a
good thing to do and is a way to produce useful, salient results. Choose
a specific aspect and design a test case which focusses as narrowly as
possible on that aspect. Then you can gather data to make meaningful
statements about that aspect in isolation from others.

What is “micro” about the benchmark you showed is that it doesn't gather
enough data to be useful. The test should not just do one statement; it
should do some relevant complete unit of the algorithm. Focussed, but
interesting: a huge amount of data, or a complex data conversion, or
something else that qualifies as “real”.


What's the purpose of the measurement? In my case, I'm not just 
measuring, but also implementing: either developing a program (the 
decoder for example) and trying to get it fast, and/or using it to tweak 
some interpreter or other I'm working on. Then I might well want to 
focus on particular single byte-code or some narrow aspect of the program.



Anyway, is reading a jpeg not real world usage? It's a task normally
done with a compiled language, but can also be a good test for an
interpreted one.


Try testing on a large battery of highly varied JPEG images, and getting
the aggregate results from that. Or something that is more interesting
(i.e. relevant to the real usage) than one image.


If your efforts manage to double the speed of reading file A, then 
probably the reading file B is also going to be improved! In practice 
you use a variety of files, but one at a time will do.



If there was a fast version, then I would have heard about it!


The mistake, I think, is in assuming a “fast version” means anything.


Yes of course it does. As does 'being slow'. Take another microbenchmark:

def whiletest():
|   i=0
|   while i<=1:
|   |   i+=1

whiletest()

Python 2.7:  8.4 seconds
Python 3.1: 12.5 seconds
Python 3.4: 18.0 seconds

Even if you don't care about speed, you must admit that there appears to 
be something peculiar going on here: why would 3.4 take more than twice 
as long as 2.7? What do they keep doing to 3.x to cripple it on each new 
version?


(For comparison, my interpreter managed 0.4s, PyPy 0.3s (it's hot on 
loops) and part-optimised C might be 0.07s (fully-optimised would be 0s).)


So you might say that a 'fast' version wouldn't give silly timings like 
the above.


Trying something like the silly loop above on a new language can be 
quite revealing.


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Jinghui Niu]

On Monday, March 7, 2016 at 5:00:57 PM UTC-8, Ben Finney wrote:
> Jinghui Niu  writes:
> 
> > Thank you for your reply. So if for a beginner learner who doesn't
> > care so much about Unix-compliant, which one is a better choice for
> > learning purpose?
> 
> Learn to care about the difference :-) They are both useful and both do
> different things.
> 
> -- 
>  \ "I must say that I find television very educational. The minute |
>   `\   somebody turns it on, I go to the library and read a book." |
> _o__)--Groucho Marx |
> Ben Finney

May I take this opportunity to ask a bold question for a beginner: if I want to 
mimic Sublime Text's fuzzy search for a file search in given directories, which 
module of these two would be more suitable? Could you shed some insights? 
Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Ben Finney]

Jinghui Niu  writes:

> Thank you for your reply. So if for a beginner learner who doesn't
> care so much about Unix-compliant, which one is a better choice for
> learning purpose?

Learn to care about the difference :-) They are both useful and both do
different things.

-- 
 \ “I must say that I find television very educational. The minute |
  `\   somebody turns it on, I go to the library and read a book.” |
_o__)—Groucho Marx |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Jinghui Niu]

On Monday, March 7, 2016 at 1:37:45 PM UTC-8, Ben Finney wrote:
> Jinghui Niu  writes:
> 
> > Hi, I've been studying python 3 modules. I'm a bit confused about the
> > possibly overlap between fnmatch() and glob(), they seem to achieve
> > the same goals exactly. Why duplicate?
> 
> >From the module documentation:
> 
> Note that unlike fnmatch.fnmatch(), glob treats filenames beginning
> with a dot (.) as special cases.
> 
> https://docs.python.org/2/library/glob.html>
> 
> So the goals are different: 'glob.glob' has the goal of matching closer
> to the Unix meaning of glob patterns, where filenames starting with a
> "." character are conventionally treated as "hidden by default".
> 
> -- 
>  \  "We are stuck with technology when what we really want is just |
>   `\ stuff that works." --Douglas Adams |
> _o__)  |
> Ben Finney

Thank you for your reply. So if for a beginner learner who doesn't care so much 
about Unix-compliant, which one is a better choice for learning purpose?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[MRAB]

On 2016-03-08 00:22, BartC wrote:

On 07/03/2016 23:40, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 9:39 AM, BartC  wrote:



I'm using it because this kind of file reading in Python is a mess. If I do
a read, will I get a string, a byte sequence object, a byte-array, or
array-array, or what?


Uhh you'll get either a text string or a byte string, depending on
whether you opened the file as text or binary. Where's the mess?


(Is a byte string the same as a byte array? Is a byte array the same as
an array.array? If I remove this line from my code, where 'data' has
just been read from a file:

 data=array.array('B',data)

then it still works - Python 3. But not on Python 2. If I do .read on a
binary file I get a byte string in Python 3, but a string in Python 2.
That sort of mess.

In Python 2, an unmarked string literal _is_ a bytestring; in Python 3, 
an unmarked string literal is Unicode.





And how do I write that deceptively simple header on the output file
without array.array because I tried all sorts:

 f = open(file+".ppm", "wb")
 s="P6\n%d %d\n255\n" % (hdr.width, hdr.height)
 sbytes=array.array('B',list(map(ord,s)))
 f.write(sbytes)


You don't need to use array.array:

sbytes = bytes(map(ord, s))

In Python 3.5, you can use '%' with a bytestring:

s = b"P6\n%d %d\n255\n" % (hdr.width, hdr.height)

[snip]

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 11:22 AM, BartC  wrote:
>
> (Is a byte string the same as a byte array? Is a byte array the same as an
> array.array? If I remove this line from my code, where 'data' has just been
> read from a file:
>
>data=array.array('B',data)
>
> then it still works - Python 3. But not on Python 2. If I do .read on a
> binary file I get a byte string in Python 3, but a string in Python 2. That
> sort of mess.

The default string in Py2 *is* a byte string.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 08/03/2016 00:22, BartC wrote:



The slow speed of Python (and a number of other scripting languages) is
well known. But obviously that is not any sort of issue, and these
people working flat out on tracing JIT compilers are all wasting their
time too!



The slow speed of Python hasn't stopped it gaining a huge following in 
just about every possible area of computing.  Who cares about speed when 
you're waiting for the file, database or network to respond?  Sure, some 
people want things faster, but I suspect that 99% couldn't care less.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Mark Lawrence]

On 07/03/2016 23:09, Fillmore wrote:


I must be missing something simple because I can't find a way to break
out of a nested loop in Python.

Is there a way to label loops?

For the record, here's a Perl script of mine I'm trying to port...there
may be 'malformed' lines in a TSV file I'm parsing that are better
discarded than fixed.

my $ctr = 0;
OUTER:
while($line = ) {

 $ctr++;
 if ($ctr < 5) {next;}

 my @allVals  = split /\t/,$line;

 my $newline;
 foreach my $i (0..$#allVals) {

 if ($i == 0) {
 if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}

 $newline =  $allVals[0];
 }

 if (defined $headers{$i}) {

 #if column not a number, skip line
 if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}

 $newline .= "\t".$allVals[$i+1];
 }
 }
 print $newline."\n";

}


I suggest you read and digest the various responses here 
http://stackoverflow.com/questions/189645/how-to-break-out-of-multiple-loops-in-python, 
some of which are similar if not identical to answers you've all ready 
been given.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 23:40, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 9:39 AM, BartC  wrote:



I'm using it because this kind of file reading in Python is a mess. If I do
a read, will I get a string, a byte sequence object, a byte-array, or
array-array, or what?


Uhh you'll get either a text string or a byte string, depending on
whether you opened the file as text or binary. Where's the mess?


(Is a byte string the same as a byte array? Is a byte array the same as 
an array.array? If I remove this line from my code, where 'data' has 
just been read from a file:


   data=array.array('B',data)

then it still works - Python 3. But not on Python 2. If I do .read on a 
binary file I get a byte string in Python 3, but a string in Python 2. 
That sort of mess.


And how do I write that deceptively simple header on the output file 
without array.array because I tried all sorts:


   f = open(file+".ppm", "wb")
   s="P6\n%d %d\n255\n" % (hdr.width, hdr.height)
   sbytes=array.array('B',list(map(ord,s)))
   f.write(sbytes)

Note this file mixes text and binary mode - blame the ppm format - but 
is mostly binary.)



But I also said I am interested in using the languages to directly implement
programs and algorithms not just for scripting. Then you need to be able to
measure what the core language is capable of.


Yeah, because the value of human legs is proven by walking from one
city to the next and seeing how long it takes. You're still using the
language inefficiently, and then proving that one inefficient use is
less inefficient than another. Congratulations.


So whoever created the speed comparison site linked to at the start of 
the thread is wasting their time?


Obviously some people are interested in this stuff, even if you're not.

The slow speed of Python (and a number of other scripting languages) is 
well known. But obviously that is not any sort of issue, and these 
people working flat out on tracing JIT compilers are all wasting their 
time too!


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Joel Goldstick]

On Mon, Mar 7, 2016 at 7:13 PM, Fillmore 
wrote:

> On 3/7/2016 7:08 PM, Chris Angelico wrote:
>
>
>> Yep, which is why we're offering a variety of new paradigms. Because
>> it's ever so much easier to get your head around three than one!
>>
>> We are SO helpful, guys. So helpful. :)
>>
>
> not too dissimilarly from human languages, speaking a foreign language is
> more often than not a matter of learning to think differently...
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

I found this on break and continue.  I don't know perl, but if what you
want to do is quit continuing in a loop and go to the next iteration, then
continue will do that.
http://www.tutorialspoint.com/python/python_loop_control.htm

-- 
Joel Goldstick
http://joelgoldstick.com/ 
http://cc-baseballstats.info/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 7:08 PM, Chris Angelico wrote:



Yep, which is why we're offering a variety of new paradigms. Because
it's ever so much easier to get your head around three than one!

We are SO helpful, guys. So helpful. :)


not too dissimilarly from human languages, speaking a foreign language 
is more often than not a matter of learning to think differently...


--
https://mail.python.org/mailman/listinfo/python-list

What will I get when reading from a file? (was: Pyhon 2.x or 3.x, which is faster?)

[Ben Finney]

BartC  writes:

> I'm using [a third-party library function] because this kind of file
> reading in Python is a mess. If I do a read, will I get a string, a
> byte sequence object, a byte-array, or array-array, or what?

That's a strange statement; there is virtually no mystery in what you'll
get from a Python 3 file object, because you're eessentially forced to
declare explicitly what kind of results you want from it.

In fact, this no-mystery is fairly special to Python. Most other
languages will guess aspects of the file and won't force the programmer
to be explicit; if anything, newcomers are frustrated at how *little*
guesswork (i.e. how much explicit declaration) is involved in opening a
Python 3 file.

If you want bytes, you open the file in that mode. If you want text, you
must instead open it with that mode and tell it what text codec to use.

And so on. Where are you seeing mysterious results?

-- 
 \  “Any intelligent fool can make things bigger and more complex… |
  `\It takes a touch of genius – and a lot of courage – to move in |
_o__)the opposite direction.” —Albert Einstein |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Angelico]

On Tue, Mar 8, 2016 at 11:02 AM, Fillmore  wrote:
> On 3/7/2016 6:09 PM, Fillmore wrote:
>>
>>
>> I must be missing something simple because I can't find a way to break
>> out of a nested loop in Python.
>
>
> Thanks to everyone who has tried to help so far. I suspect this may be a
> case where I just need to get my head around a new paradigm

Yep, which is why we're offering a variety of new paradigms. Because
it's ever so much easier to get your head around three than one!

We are SO helpful, guys. So helpful. :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Ben Finney]

BartC  writes:

> On 07/03/2016 20:47, Chris Angelico wrote:
> > Look! Creating a floating-point value is faster under Python 2 than
> > Python 3. What could be more perfect?
>
> > This is like a microbenchmark in that it doesn't tell you anything
> > about real-world usage.
>
> Microbenchmarks have their uses, when you are concentrating on a
> particular aspect of a language.

The “micro” is not referring to the focus of the operation; that's a
good thing to do and is a way to produce useful, salient results. Choose
a specific aspect and design a test case which focusses as narrowly as
possible on that aspect. Then you can gather data to make meaningful
statements about that aspect in isolation from others.

What is “micro” about the benchmark you showed is that it doesn't gather
enough data to be useful. The test should not just do one statement; it
should do some relevant complete unit of the algorithm. Focussed, but
interesting: a huge amount of data, or a complex data conversion, or
something else that qualifies as “real”.

> Anyway, is reading a jpeg not real world usage? It's a task normally
> done with a compiled language, but can also be a good test for an
> interpreted one.

Try testing on a large battery of highly varied JPEG images, and getting
the aggregate results from that. Or something that is more interesting
(i.e. relevant to the real usage) than one image.

> If you want an interpreted language to take on wider range of tasks,
> then you need to make it faster, and that means measuring how
> efficient it is at executing algorithms itself, not measuring how long
> some library in a different language takes to execute.

That's quite true. The trick is to avoid being *so* focussed that you
are mostly measuring a corner case, instead of a relevant use case.

> > CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
> > going to differ in performance. And that's without even looking at
> > what core devs would refer to as "other Pythons", which would
> > include IronPython, Jython, PyPy (well, you got that, but you're
> > treating it as an afterthought), MicroPython, Brython, wpython,
> > Nuitka, Cython. these are *other Pythons*.
>
> What are you suggesting here? That all these Pythons are going to be
> faster or slower than each other?

At specific, highly-focussed operations? Of course. That is going to be
true almost by definition: the implementations are different, so if your
test cases are extremely focussed they will be much more likely to find
variations in the implementations — differences that are not relevant to
which version of the Python source you're coming from.

> I would guess that most of them are going to be roughly the same,
> other than PyPy.

That's the kind of guess which should be subjected to testing. We humans
are *very* bad at estimating those kinds of behaviour in implementation.
This is especially true because we humans also demand baroque,
highly-complex implementations in order to get our speed improvements.

And those complexities of implementation will be a major factor
thwarting our intuitions about how those implementations will perform.
If you think you know how an implementation will perform, the data
indicates you should not trust that instinct.

> If there was a fast version, then I would have heard about it!

The mistake, I think, is in assuming a “fast version” means anything.

Different implementations will be faster at some things, slower at other
things, where “things” can be a very lumpy and convoluted landscape of
complex use cases. Test those assumptions with realistic benchmarks run
under each implementation.

-- 
 \  “When [science] permits us to see the far side of some new |
  `\  horizon, we remember those who prepared the way – seeing for |
_o__)  them also.” —Carl Sagan, _Cosmos_, 1980 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 6:09 PM, Fillmore wrote:


I must be missing something simple because I can't find a way to break
out of a nested loop in Python.


Thanks to everyone who has tried to help so far. I suspect this may be a 
case where I just need to get my head around a new paradigm




--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Angelico]

On Tue, Mar 8, 2016 at 10:42 AM, Chris Kaynor  wrote:
> And the same rough example, using an exception without a function:
>
> for file in files:
> try:
> for line in file:
> section = line.split()
> for section in line:
> if sectionCorrupt:
> raise Exception('Section corrupt') # You probably want
> more details here, for possible manual repair. You could also have a custom
> exception class for more control.
> except Exception as e:
> print('Failed to process the file {} with error {}.'.format(file,
> str(e))) # Probably should also print the entire traceback to aid in
> repairing errors, especially if they are due to a bug in the code, but this
> is the rough idea.

Yes, although I would more strongly suggest the custom exception
class. In fact, the way I'd word it is: Never raise Exception, always
a subclass. Otherwise it's too easy to accidentally catch something
elsewhere in the code (a ValueError or TypeError or even
AttributeError).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Angelico]

On Tue, Mar 8, 2016 at 10:09 AM, Fillmore  wrote:
> For the record, here's a Perl script of mine I'm trying to port...there may
> be 'malformed' lines in a TSV file I'm parsing that are better discarded
> than fixed.
>
> my $ctr = 0;
> OUTER:
> while($line = ) {
>
> $ctr++;
> if ($ctr < 5) {next;}
>
> my @allVals  = split /\t/,$line;
>
> my $newline;
> foreach my $i (0..$#allVals) {
>
> if ($i == 0) {
> if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}
>
> $newline =  $allVals[0];
> }
>
> if (defined $headers{$i}) {
>
> #if column not a number, skip line
> if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}
>
> $newline .= "\t".$allVals[$i+1];
> }
> }
> print $newline."\n";
>
> }

I'm not too fluent in Perl, but my understanding here is that "next
OUTER" is equivalent to Python's 'continue' statement, right? So your
flow control is roughly this:

for _ in range(5): skip header row
for line in file:
for cell in split(line):
if bad_cell_value:
break # Skip out of the inner loop now
else:
print_stuff # We didn't break

Does that look right? The key here is that a 'for' loop has an 'else'
clause, which happens if there's no 'break' in the main loop. I think
that's what you're after, here; you get to break out of "the loop and
a little bit more", so to speak. The outer loop is actually immaterial
here.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 6:29 PM, Rob Gaddi wrote:


You're used to Perl, you're used to exceptions being A Thing.  This is
Python, and exceptions are just another means of flow control.

class MalformedLineError(Exception): pass

for line in file:
 try:
 for part in line.split('\t'):
 if thispartisbadforsomereason:
 raise MalformedLineError()
 otherwisewedothestuff
 except MalformedLineError:
 pass



I am sure you are right, but adapting this thing here into something 
that is a fix to my problem seems like abusing my 'system 2' (for those 
who read a certain book by a guy called Daniel Kanheman :)


--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Kaynor]

On Mon, Mar 7, 2016 at 3:21 PM, Fillmore 
wrote:

> On 3/7/2016 6:17 PM, Ian Kelly wrote:
>
>> On Mon, Mar 7, 2016 at 4:09 PM, Fillmore 
>> wrote:
>>
>>>
>>> I must be missing something simple because I can't find a way to break
>>> out
>>> of a nested loop in Python.
>>>
>>> Is there a way to label loops?
>>>
>>
>> No, you can't break out of nested loops,
>>
>
> wow...this is a bit of a WTF moment to me :(
>
> apart from structuring your
>> code such that return does what you want.
>>
>>
> Can you elaborate? apologies, but I'm new to python and trying to find my
> way out of perl


The normal way of breaking out of some set of loops would be to structure
your code so that you have a function, and when you want to break out of
some of multiple loops, you can just call return.


As a very rough example:

def processFile(file):
for line in file:
section = line.split()
for section in line:
if sectionCorrupt:
return # Stops processing the entire file, by exiting the
function, but keeps processing the list of files.

for file in files:
processFile(file)




Alternatively, as you mention corrupted lines, perhaps raising an exception
would be the best approach.

And the same rough example, using an exception without a function:

for file in files:
try:
for line in file:
section = line.split()
for section in line:
if sectionCorrupt:
raise Exception('Section corrupt') # You probably want
more details here, for possible manual repair. You could also have a custom
exception class for more control.
except Exception as e:
print('Failed to process the file {} with error {}.'.format(file,
str(e))) # Probably should also print the entire traceback to aid in
repairing errors, especially if they are due to a bug in the code, but this
is the rough idea.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 9:39 AM, BartC  wrote:
>
> What are you suggesting here? That all these Pythons are going to be faster
> or slower than each other? I would guess that most of them are going to be
> roughly the same, other than PyPy. If there was a fast version, then I would
> have heard about it!

Try them, then you'll know. They all have different strengths and weaknesses.

>> the
>> performance of array.array() is far from stable. It's not a core
>> language feature; you're using it because it's the most obvious
>> translation of your C algorithm,
>
> I'm using it because this kind of file reading in Python is a mess. If I do
> a read, will I get a string, a byte sequence object, a byte-array, or
> array-array, or what?

Uhh you'll get either a text string or a byte string, depending on
whether you opened the file as text or binary. Where's the mess?

> Are you saying there array.array will not work on some implementations? In
> that case it's more of a mess than I thought!

No, it'll always be there (at least, I'm pretty sure it will). I said
that its *performance* is all over the place. Try coding in the
simplest possible way first, and only turn to array.array afterward.

> Real world use of Python appears to be as a scripting language and for glue
> code, with most of the real work done in libraries implemented in native
> code. I acknowledge that. And I understand that the 10-20% difference
> between Py2 and Py3 will largely disappear for these kinds of uses.

Real world use of Python includes writing full-scale applications in
it... not sure whether the myriad web apps written with
Flask/Django/etc count as "most of the real work done in native
libraries". But what does "real work" mean? When you write a data
processing program in Python, you could spend most of your time doing
integer arithmetic using the native 'int' type, or you could spend
most of your time using numpy. One of those is using a third-party
library written in Fortran; the other is using the language facilities
directly. But in CPython, the latter means you're using C. Where's the
difference? All your most important code - all the "real work" in
terms of algorithms etc - is in Python. The underlying facilities are
completely generic.

> But I also said I am interested in using the languages to directly implement
> programs and algorithms not just for scripting. Then you need to be able to
> measure what the core language is capable of.

Yeah, because the value of human legs is proven by walking from one
city to the next and seeing how long it takes. You're still using the
language inefficiently, and then proving that one inefficient use is
less inefficient than another. Congratulations.

I'm done.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Rob Gaddi]

Fillmore wrote:

>
> I must be missing something simple because I can't find a way to break 
> out of a nested loop in Python.
>
> Is there a way to label loops?
>
> For the record, here's a Perl script of mine I'm trying to port...there 
> may be 'malformed' lines in a TSV file I'm parsing that are better 
> discarded than fixed.
>
> my $ctr = 0;
> OUTER:
> while($line = ) {
>
>  $ctr++;
>  if ($ctr < 5) {next;}
>
>  my @allVals  = split /\t/,$line;
>
>  my $newline;
>  foreach my $i (0..$#allVals) {
>
>   if ($i == 0) {
>   if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}
>
>   $newline =  $allVals[0];
>   }
>
>   if (defined $headers{$i}) {
>
>   #if column not a number, skip line
>   if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}
>
>   $newline .= "\t".$allVals[$i+1];
>   }
>  }
>  print $newline."\n";
>
> }

You're used to Perl, you're used to exceptions being A Thing.  This is
Python, and exceptions are just another means of flow control.

class MalformedLineError(Exception): pass

for line in file:
try:
for part in line.split('\t'):
if thispartisbadforsomereason:
raise MalformedLineError()
otherwisewedothestuff
except MalformedLineError:
pass

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 6:17 PM, Ian Kelly wrote:

On Mon, Mar 7, 2016 at 4:09 PM, Fillmore  wrote:


I must be missing something simple because I can't find a way to break out
of a nested loop in Python.

Is there a way to label loops?


No, you can't break out of nested loops,


wow...this is a bit of a WTF moment to me :(


apart from structuring your
code such that return does what you want.



Can you elaborate? apologies, but I'm new to python and trying to find 
my way out of perl


--
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Mark Lawrence]

On 07/03/2016 22:51, Fillmore wrote:


learning Python from Perl here. Want to do things as Pythonicly as
possible.

I am reading a TSV, but need to skip the first 5 lines. The following
works, but wonder if there's a more pythonc way to do things. Thanks

ctr = 0
with open(prfile,mode="rt",encoding='utf-8') as pfile:
 for line in pfile:
 ctr += 1

 if ctr < 5:
 continue

 allVals = line.strip().split("\t")
 print(allVals)


Something like this, completely untested.

with open(prfile,mode="rt",encoding='utf-8') as pfile:
lineIter = iter(pfile)
for _ in range(5):
next(lineIter)
for line in lineIter:
 allVals = line.strip().split("\t")
 print(allVals)

I'd also recommend that you use the csv module 
https://docs.python.org/3/library/csv.html which can also cope with tsv 
files, it's far more reliable than relying on a simple call to split().


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Ian Kelly]

On Mon, Mar 7, 2016 at 4:09 PM, Fillmore  wrote:
>
> I must be missing something simple because I can't find a way to break out
> of a nested loop in Python.
>
> Is there a way to label loops?

No, you can't break out of nested loops, apart from structuring your
code such that return does what you want.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Fillmore]

On 3/7/2016 6:03 PM, sohcahto...@gmail.com wrote:


On a side note, your "with open..." line uses inconsistent quoting.

> You have "" on one string, but '' on another.

Thanks. I'll make sure I flog myself three times later tonight...



--
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Ian Kelly]

On Mon, Mar 7, 2016 at 3:51 PM, Fillmore  wrote:
>
> learning Python from Perl here. Want to do things as Pythonicly as possible.
>
> I am reading a TSV, but need to skip the first 5 lines. The following works,
> but wonder if there's a more pythonc way to do things. Thanks

I'd probably use itertools.islice.

from itertools import islice
for line in isiice(pfile, 5, None):
allVals = line.strip().split("\t")
print(allVals)
-- 
https://mail.python.org/mailman/listinfo/python-list

breaking out of outer loops

[Fillmore]

I must be missing something simple because I can't find a way to break 
out of a nested loop in Python.


Is there a way to label loops?

For the record, here's a Perl script of mine I'm trying to port...there 
may be 'malformed' lines in a TSV file I'm parsing that are better 
discarded than fixed.


my $ctr = 0;
OUTER:
while($line = ) {

$ctr++;
if ($ctr < 5) {next;}

my @allVals  = split /\t/,$line;

my $newline;
foreach my $i (0..$#allVals) {

if ($i == 0) {
if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}

$newline =  $allVals[0];
}

if (defined $headers{$i}) {

#if column not a number, skip line
if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}

$newline .= "\t".$allVals[$i+1];
}
}
print $newline."\n";

}
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[sohcahtoa82]

On Monday, March 7, 2016 at 2:51:50 PM UTC-8, Fillmore wrote:
> learning Python from Perl here. Want to do things as Pythonicly as possible.
> 
> I am reading a TSV, but need to skip the first 5 lines. The following 
> works, but wonder if there's a more pythonc way to do things. Thanks
> 
> ctr = 0
> with open(prfile,mode="rt",encoding='utf-8') as pfile:
>  for line in pfile:
>  ctr += 1
> 
>  if ctr < 5:
>  continue
> 
>  allVals = line.strip().split("\t")
>  print(allVals)

I'd read all the lines at once and then just slice the list.

with open(prfile, mode="rt", encoding="utf-8") as pfile:
lines = pfile.readlines()[5:]
for line in lines:
allVals = line.strip().split("\t")
print(allVals)

Obviously, this will only work well if your file is a reasonable size, as it 
will store the entire thing in memory at once.

On a side note, your "with open..." line uses inconsistent quoting.  You have 
"" on one string, but '' on another.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Rob Gaddi]

Fillmore wrote:

>
> learning Python from Perl here. Want to do things as Pythonicly as possible.
>
> I am reading a TSV, but need to skip the first 5 lines. The following 
> works, but wonder if there's a more pythonc way to do things. Thanks
>
> ctr = 0
> with open(prfile,mode="rt",encoding='utf-8') as pfile:
>  for line in pfile:
>  ctr += 1
>
>  if ctr < 5:
>  continue
>
>  allVals = line.strip().split("\t")
>  print(allVals)

for lineno, line in enumerate(pfile, start=1):

will save you maintaining your own counter.  But other than that, nah,
that's pretty much the approach you're looking for.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list

Pythonic love

[Fillmore]

learning Python from Perl here. Want to do things as Pythonicly as possible.

I am reading a TSV, but need to skip the first 5 lines. The following 
works, but wonder if there's a more pythonc way to do things. Thanks


ctr = 0
with open(prfile,mode="rt",encoding='utf-8') as pfile:
for line in pfile:
ctr += 1

if ctr < 5:
continue

allVals = line.strip().split("\t")
print(allVals)
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 20:47, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 7:19 AM, BartC  wrote:



What can be more perfect for comparing two implementations?



rosuav@sikorsky:~$ python2 -m timeit -s 'from fp import Float'
'Float("1234.567")'
100 loops, best of 3: 1.84 usec per loop
rosuav@sikorsky:~$ python3 -m timeit -s 'from fp import Float'
'Float("1234.567")'
10 loops, best of 3: 2.76 usec per loop

Look! Creating a floating-point value is faster under Python 2 than
Python 3. What could be more perfect?



This is like a microbenchmark in that it doesn't tell you anything
about real-world usage.


Microbenchmarks have their uses, when you are concentrating on a 
particular aspect of a language. But I tried your code, calling Float a 
million times, and 2.7 took 8.3 seconds; 3.4 took 10.5 seconds. But that 
is meaningless according to you.


(3.1 took 8.5 seconds. What happened between 3.1 and 3.4 to cause such a 
slow-down? Do you think it might be a good idea for /someone/ at least 
to take pay some attention to that, before it grinds to a halt 
completely by version 4.0?)


(I tried near-equivalent code on my byte-coded language. It took 0.7 
seconds. Then I tried a fairer test, as I use a bit of ASM to help the 
interpreter along: I created a C source version of the interpreter, 
compiled it with Tiny C, which generates some of the worst code around, 
and ran it again on the byte-code. It took 3 seconds; still faster than 
either of the Pythons!)


Anyway, is reading a jpeg not real world usage? It's a task normally 
done with a compiled language, but can also be a good test for an 
interpreted one.



You write *real world* code and then profile that. You get actual real
programs that you actually really use, and you run those through
timing harnesses.


Sorry, but that is useless. It tells you that slow, interpreted 
languages can be viable with certain kinds of usage. Everyone knows that 
(I've been creating applications which balance compiled code and 
interpreted code since about the mid-80s).


If you want an interpreted language to take on wider range of tasks, 
then you need to make it faster, and that means measuring how efficient 
it is at executing algorithms itself, not measuring how long some 
library in a different language takes to execute.



CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
going to differ in performance. And that's without even looking at
what core devs would refer to as "other Pythons", which would include
IronPython, Jython, PyPy (well, you got that, but you're treating it
as an afterthought), MicroPython, Brython, wpython, Nuitka,
Cython. these are *other Pythons*.


What are you suggesting here? That all these Pythons are going to be 
faster or slower than each other? I would guess that most of them are 
going to be roughly the same, other than PyPy. If there was a fast 
version, then I would have heard about it!



the
performance of array.array() is far from stable. It's not a core
language feature; you're using it because it's the most obvious
translation of your C algorithm,


I'm using it because this kind of file reading in Python is a mess. If I 
do a read, will I get a string, a byte sequence object, a byte-array, or 
array-array, or what?


Are you saying there array.array will not work on some implementations? 
In that case it's more of a mess than I thought!



not because it's the best way to use
Python. So I say again, your measurement has little to no significance
to real-world code.


Real world use of Python appears to be as a scripting language and for 
glue code, with most of the real work done in libraries implemented in 
native code. I acknowledge that. And I understand that the 10-20% 
difference between Py2 and Py3 will largely disappear for these kinds of 
uses.


But I also said I am interested in using the languages to directly 
implement programs and algorithms not just for scripting. Then you need 
to be able to measure what the core language is capable of.



--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Terry Reedy]

On 3/7/2016 6:54 AM, Andrew Jaffe wrote:


Dumb question, and this probably isn't the place for it, but the three
Pythons being tested are:

  64-bit CPython on Li... latest in branch '2.7'
  64-bit CPython on Li... latest in branch '3.5'
  64-bit CPython on Li... latest


> I understand that the first two are the released 2.7 and 3.5
> versions; is the third the most recent 3.x build?

I am rather sure that none are released versions and that all three 
refer to the current tip ('latest') of each branch in the repository, 
with third being the default branch.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Terry Reedy]

On 3/7/2016 12:23 PM, Tony van der Hoff wrote:


However, more generally, how am I supposed to know that a module is part
of a package, and needs a "magic" stanza to get a module loaded?


The doc for a package usually mentions the submodules it contains, as in
https://docs.python.org/3/library/tkinter.html#tkinter-modules

The doc for a submodule has the package name in its title line.

25.4. tkinter.scrolledtext — Scrolled Text Widget
8.4. collections.abc — Abstract Base Classes for Containers

And yes, tkinter docs need considerable work.  That is why there are 
several external resources list in the intro.  I mostly use #3 Tkinter 
reference (not perfect) supplements by #2, #4, and #5.  Not mentioned 
there are the docstrings (also not perfect), accessible via interactive 
help.


>>> import tkinter as tk
>>> help(tk.Button.configure)
Help on function configure in module tkinter:

configure(self, cnf=None, **kw)
Configure resources of a widget.

The values for resources are specified as keyword
arguments. To get an overview about
the allowed keyword arguments call the method keys.

--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Roel Schroeven]

Faling Dutchman schreef op 2016-03-07 09:24:

class Item:
def __init__(self, id, productId, quantity, pageCount, files, option, 
metadata):
self.id = id
self.productId = productId
self.quantity = quantity
self.pageCount = pageCount
self.files = files
self.option = option
self.metadata = metadata

itm = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
print(itm)

it prints: <__main__.Item object at 0x02EBF3B0>


I'm not 100% sure exactly what you need, but namedtuple might be what 
you're looking for:


>>> import collections
>>> Item = collections.namedtuple('Item', 'id productId quantity 
pageCount files option metadata')

>>> itm = Item(1, None, 1, 1, 'asdf', {'asdf': 3, 'ads': 55}, None)
>>> print(itm)
Item(id=1, productId=None, quantity=1, pageCount=1, files='asdf', 
option={'ads': 55, 'asdf': 3}, metadata=None)


See 
https://docs.python.org/3/library/collections.html?highlight=namedtuple#collections.namedtuple



--
The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom.
  -- Isaac Asimov

Roel Schroeven

--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Mark Lawrence]

On 07/03/2016 20:07, MRAB wrote:

On 2016-03-07 19:08, Mark Lawrence wrote:

On 07/03/2016 17:38, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff
 wrote:

Thanks to all who replied to my cry for help; I understand it better
now.
But:

On 07/03/16 16:08, Chris Angelico wrote:



The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.



Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.


You're looking at an ancient version of the docs. Here's the newest
docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA



As I happen to be playing with tkinter I was searching literally minutes
ago for the way to put a grid onto a notebook tab.  Regrettably the
notebook grid is conspicious by its absence from the docs :(


It's in the tkinter.ttk docs:

https://docs.python.org/3/library/tkinter.ttk.html



https://docs.python.org/3/library/tkinter.ttk.html#tkinter.ttk.Notebook 
lists add, forget, hide, identify, index, insert, select, tab, tabs and

enable_traversal, but no grid, possibly others.  yet:-

>>> import tkinter.ttk
>>> help(tkinter.ttk.Notebook.grid)
Help on function grid_configure in module tkinter:

grid_configure(self, cnf={}, **kw)
Position a widget in the parent widget in a grid. Use as options:
column=number - use cell identified with given column (starting with 0)
columnspan=number - this widget will span several columns
in=master - use master to contain this widget
in_=master - see 'in' option description
ipadx=amount - add internal padding in x direction
ipady=amount - add internal padding in y direction
padx=amount - add padding in x direction
pady=amount - add padding in y direction
row=number - use cell identified with given row (starting with 0)
rowspan=number - this widget will span several rows
sticky=NSEW - if cell is larger on which sides will this
  widget stick to the cell boundary

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Ben Finney]

Jinghui Niu  writes:

> Hi, I've been studying python 3 modules. I'm a bit confused about the
> possibly overlap between fnmatch() and glob(), they seem to achieve
> the same goals exactly. Why duplicate?

From the module documentation:

Note that unlike fnmatch.fnmatch(), glob treats filenames beginning
with a dot (.) as special cases.

https://docs.python.org/2/library/glob.html>

So the goals are different: ‘glob.glob’ has the goal of matching closer
to the Unix meaning of glob patterns, where filenames starting with a
“.” character are conventionally treated as “hidden by default”.

-- 
 \  “We are stuck with technology when what we really want is just |
  `\ stuff that works.” —Douglas Adams |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 7:19 AM, BartC  wrote:
>>> I disagree. The program does its job perfectly (you will have to take it
>>> further to verify the results, such as writing out the .ppm file and
>>> viewing
>>> the contents).
>>>
>>> But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for
>>> a
>>> smaller file). /This is in line with other observations./
>>
>>
>> What's your meaning of "perfectly"? You're implementing things in a
>> very C way, and then showing that two different Python interpreters
>> have different performance.
>
>
> Two interpreters executing exactly the same code and exactly the same
> algorithm. And for a real task which deliberately doesn't just delegate to
> high-level features (otherwise you're just comparing libraries).
>
> What can be more perfect for comparing two implementations?

def valueOf(digit):
return ord(digit) - ord('0')

class Float:
def __init__(self, string):
self.value = 0
self.scale = 0
have_dot = 0
for i in range(len(string)):
if string[i] == '.': have_dot = 1
else:
self.value = self.value * 10 + valueOf(string[i])
if have_dot: self.scale += 1

rosuav@sikorsky:~$ python2 -m timeit -s 'from fp import Float'
'Float("1234.567")'
100 loops, best of 3: 1.84 usec per loop
rosuav@sikorsky:~$ python3 -m timeit -s 'from fp import Float'
'Float("1234.567")'
10 loops, best of 3: 2.76 usec per loop

Look! Creating a floating-point value is faster under Python 2 than
Python 3. What could be more perfect?

This is like a microbenchmark in that it doesn't tell you anything
about real-world usage. Your example also has the problem that it does
file I/O, which adds a ton of noise to your numbers. When you
reimplement a C-designed algorithm in Python, it's often NOT going to
be the best use of the language; and if you're comparing two poor uses
of a language, what do you prove by showing that one interpreter is
faster than another?

>> Did you try the 'pillow' library?
>>
>> https://pypi.python.org/pypi/Pillow
>>
>> It supports 2.6/2.7 and 3.2+, and would be most people's "one obvious
>> way" to do things. At very least, it should get a mention in your
>> performance comparisons.
>
>
> I don't understand. This is an external library that appears to be written
> in C. How does that help me compare the performance of two implementations
> of Python?

I'm not sure that it is written in C, but the point is that this is a
much better way to compare code. Use the language well, not badly.

> One could be ten times slower than the other (in executing its native
> bytecode), but if it spends 99% of its time an a C image library, how do you
> measure that?

You write *real world* code and then profile that. You get actual real
programs that you actually really use, and you run those through
timing harnesses. (Or throughput harnesses, which are pretty much the
same thing. With a web application, "performance" doesn't so much mean
"how long it takes to run", but "how many requests per second the
system can handle". Same thing, opposite way of measuring it.)

>>> (I'm quite pleased with my version: smaller, faster, works on all the
>>> Pythons, supports all 3 colour formats and no decoding bugs that I'm
>>> aware
>>> of, and it's the first Python program I've written that does something
>>> useful.)
>>
>>
>> "all the Pythons" meaning what, exactly?
>
>
> I means versions 2 and 3 of the language, tested on CPython versions 2.5,
> 2.7, 3.1 and 3.4, as well as PyPy. The other decoders I tried were for 2.x.

CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
going to differ in performance. And that's without even looking at
what core devs would refer to as "other Pythons", which would include
IronPython, Jython, PyPy (well, you got that, but you're treating it
as an afterthought), MicroPython, Brython, wpython, Nuitka,
Cython. these are *other Pythons*. What you're looking at is
closer to *all the versions of CPython*, but not even that, since
there are so many different ways that they can be built, and the
performance of array.array() is far from stable. It's not a core
language feature; you're using it because it's the most obvious
translation of your C algorithm, not because it's the best way to use
Python. So I say again, your measurement has little to no significance
to real-world code.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[justin walters]

Unfortunately, it's difficult for the core devs to know every hardware and
os combination there is. Maybe you could submit a bug report?
On Mar 7, 2016 10:56 AM, "Jon Ribbens"  wrote:

> On 2016-03-07, mm0fmf  wrote:
> > On 07/03/2016 18:09, Jon Ribbens wrote:
> >> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
> >> not 64-bit Intel processors.
> >
> > You didn't read the bit that says
> >
> > "The binaries for AMD64 will also work on processors that implement the
> > Intel 64 architecture. (Also known as the "x64" architecture, and
> > formerly known as both "EM64T" and "x86-64".) "
>
> You are quite correct that I did not read the whole of the
> ridiculously-long and badly-organised download page.
>
> I would strongly suggest that, at the very least, the 'Description'
> for the x86-64 files should say "for all 64-bit PCs (except Itanium*)"
> with the '*' linking to a footnote that explains how to tell if you
> have one of those.
>
> I must say that Python on Windows was a very poor experience indeed,
> "virtualenv" does not work and "venv" refuses to create the 'activate'
> shell script so does not work either (and pygame doesn't work, but
> that's presumably not Python's fault).
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 19:10, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 5:34 AM, BartC  wrote:



def abortjpeg(mess):
 print ("Error:",mess)
 raise

Here's my Python:



RuntimeError: No active exception to reraise


OK I'll have a look. (Or maybe just change 'raise' to 'exit(0)'. It just 
needs to crash out at this point, but should be a bit more graceful than 
what you had.)



I disagree. The program does its job perfectly (you will have to take it
further to verify the results, such as writing out the .ppm file and viewing
the contents).

But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for a
smaller file). /This is in line with other observations./


What's your meaning of "perfectly"? You're implementing things in a
very C way, and then showing that two different Python interpreters
have different performance.


Two interpreters executing exactly the same code and exactly the same 
algorithm. And for a real task which deliberately doesn't just delegate 
to high-level features (otherwise you're just comparing libraries).


What can be more perfect for comparing two implementations?


You haven't actually shown how a
properly-written program would perform.


If I could find one that worked, I would have used that! (But I was more 
interested in 3 vs. PyPy than 2 vs. 3.)



Did you try the 'pillow' library?

https://pypi.python.org/pypi/Pillow

It supports 2.6/2.7 and 3.2+, and would be most people's "one obvious
way" to do things. At very least, it should get a mention in your
performance comparisons.


I don't understand. This is an external library that appears to be 
written in C. How does that help me compare the performance of two 
implementations of Python?


One could be ten times slower than the other (in executing its native 
bytecode), but if it spends 99% of its time an a C image library, how do 
you measure that?



(I'm quite pleased with my version: smaller, faster, works on all the
Pythons, supports all 3 colour formats and no decoding bugs that I'm aware
of, and it's the first Python program I've written that does something
useful.)


"all the Pythons" meaning what, exactly?


I means versions 2 and 3 of the language, tested on CPython versions 
2.5, 2.7, 3.1 and 3.4, as well as PyPy. The other decoders I tried were 
for 2.x.



And, no decoding bugs? Maybe,
but I wouldn't bet my life on it. Unless your code is a pure port of
someone else's (and unless that someone else could guarantee that the
original C code was bug free), I wouldn't trust code that I can't
completely analyze in one sitting. Virtually all code has bugs in it;
the only code that doesn't is code that's so trivially simple that you
can completely understand it in one "headspace".


Let's put it this way: I don't have to hand a jpeg file that it can't 
decode. When one turns up then I will fix that. (I can't say for sure 
there aren't any subtle chroma or quality problems, but there's nothing 
obvious.) I can't say that for the decoders I looked at.


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

fnmatch() vs. glob.glob()

[Jinghui Niu]

Hi, I've been studying python 3 modules. I'm a bit confused about the possibly 
overlap between fnmatch() and glob(), they seem to achieve the same goals 
exactly. Why duplicate?
Or maybe something I've missed?

Thank you.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[MRAB]

On 2016-03-07 19:08, Mark Lawrence wrote:

On 07/03/2016 17:38, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff  wrote:

Thanks to all who replied to my cry for help; I understand it better now.
But:

On 07/03/16 16:08, Chris Angelico wrote:



The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.



Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.


You're looking at an ancient version of the docs. Here's the newest docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA



As I happen to be playing with tkinter I was searching literally minutes
ago for the way to put a grid onto a notebook tab.  Regrettably the
notebook grid is conspicious by its absence from the docs :(


It's in the tkinter.ttk docs:

https://docs.python.org/3/library/tkinter.ttk.html

--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Andrew Farrell]

I'm going to echo Chris Angelo's suggestion #2 to use a python
distribution. This page  has the links
to download Anaconda. It is free and if you need to download libraries
which require compiled external code like numpy you can just run `conda
install numpy`.

On Mon, Mar 7, 2016 at 1:47 PM, Chris Angelico  wrote:

> On Tue, Mar 8, 2016 at 5:51 AM, Jon Ribbens
>  wrote:
> > I must say that Python on Windows was a very poor experience indeed,
> > "virtualenv" does not work and "venv" refuses to create the 'activate'
> > shell script so does not work either (and pygame doesn't work, but
> > that's presumably not Python's fault).
>
> To be quite frank, Python on Windows has had a *lot* of problems, and
> most of them have been the fault of Windows. Starting with Python 3.5,
> there've been some big packaging changes that ought to make Windows
> Python a bit easier to use; however, there've been some teething
> troubles, and there are still some ways in which setting up a
> properly-working Python is a pain.
>
> So you have a few choices:
>
> 1) Stick with the vanilla Python on the vanilla Windows. It's not
> horrendous, but there will be rough edges. Report those rough edges,
> and hopefully they can be fixed in time for 3.6 (or even 3.5.2).
>
> 2) Use a different Python distribution, eg Anaconda or ActiveState.
> Some of them cost money; when you pay money for open source software,
> what you're getting is a level of support, which will mean you can
> complain to them when something doesn't work, and insist that they fix
> it. This may or may not give you better results than option 1.
>
> 3) Use standard Python, and ditch Windows. This is what I do. :)
>
> 4) Keep using Windows, but do your Python work in a virtual machine.
> Either full-on virtualization software like VirtualBox or VMWare, or
> something that gives you a web browser interface to a VM hosted in the
> cloud (eg Cloud 9 or Nitrous).
>
> Personally, I don't see any reason to run "real Windows" on any of my
> systems. Windows programs get run under Wine or VirtualBox, never
> directly on the hardware. But if your needs are different, there are
> plenty of ways to improve your Windows+Python setup; just be patient,
> and accept that you might have a bit of extra work to do.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Chris Angelico]

On Tue, Mar 8, 2016 at 5:51 AM, Jon Ribbens
 wrote:
> I must say that Python on Windows was a very poor experience indeed,
> "virtualenv" does not work and "venv" refuses to create the 'activate'
> shell script so does not work either (and pygame doesn't work, but
> that's presumably not Python's fault).

To be quite frank, Python on Windows has had a *lot* of problems, and
most of them have been the fault of Windows. Starting with Python 3.5,
there've been some big packaging changes that ought to make Windows
Python a bit easier to use; however, there've been some teething
troubles, and there are still some ways in which setting up a
properly-working Python is a pain.

So you have a few choices:

1) Stick with the vanilla Python on the vanilla Windows. It's not
horrendous, but there will be rough edges. Report those rough edges,
and hopefully they can be fixed in time for 3.6 (or even 3.5.2).

2) Use a different Python distribution, eg Anaconda or ActiveState.
Some of them cost money; when you pay money for open source software,
what you're getting is a level of support, which will mean you can
complain to them when something doesn't work, and insist that they fix
it. This may or may not give you better results than option 1.

3) Use standard Python, and ditch Windows. This is what I do. :)

4) Keep using Windows, but do your Python work in a virtual machine.
Either full-on virtualization software like VirtualBox or VMWare, or
something that gives you a web browser interface to a VM hosted in the
cloud (eg Cloud 9 or Nitrous).

Personally, I don't see any reason to run "real Windows" on any of my
systems. Windows programs get run under Wine or VirtualBox, never
directly on the hardware. But if your needs are different, there are
plenty of ways to improve your Windows+Python setup; just be patient,
and accept that you might have a bit of extra work to do.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Ian Kelly]

On Mon, Mar 7, 2016 at 11:51 AM, Jon Ribbens
 wrote:
> I must say that Python on Windows was a very poor experience indeed,
> "virtualenv" does not work and "venv" refuses to create the 'activate'
> shell script so does not work either

I've used both of these on Windows (although not recently) and had no
trouble with them. I never had a problem with venv not creating the
activate.bat file.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 5:34 AM, BartC  wrote:
> On 07/03/2016 15:31, Chris Angelico wrote:
>>
>> On Tue, Mar 8, 2016 at 12:25 AM, BartC  wrote:
>
>
>>> (The Python version of that program is here:
>>> http://pastebin.com/cHx3UhQb. It should work with any Python.)
>>
>>
>> Actually, it won't work with any Python - not if it gets a broken
>> file. Your abortjpeg() function doesn't work :)
>
>
> What does it do when it doesn't work? I just get 'can't load file' then it
> stops if there's no such file. With an extra message if there's a format
> error. (I'm not sure what 'raise' does, but it doesn't complain about it. I
> think I realised I didn't know what came next and left it.)

Here's your code:

def abortjpeg(mess):
print ("Error:",mess)
raise

Here's my Python:

$ python3
Python 3.6.0a0 (default:bbfbde6ee9d0, Feb 19 2016, 12:43:06)
[GCC 5.3.1 20160121] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> raise
Traceback (most recent call last):
  File "", line 1, in 
RuntimeError: No active exception to reraise

The only try/except in your code (an except block is the only way
you'll have an "active exception") is this abomination:

def getfile(file):
try:
data=open(file,"rb").read()
except:
return 0
...

>> But what you have there is a messy mixture of old-style classes used
>> as if they were C structs, array.array() as if it were a C array, and
>> utterly uncommented code that looks like it was ported from, again, C.
>> Benchmarking Py2 vs Py3 on that kind of code is like trying to figure
>> out whether it's easier to drag a 747 by your teeth or your toes.
>> Yeah, you might get a result, but it's a useless one.
>
>
> I disagree. The program does its job perfectly (you will have to take it
> further to verify the results, such as writing out the .ppm file and viewing
> the contents).
>
> But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for a
> smaller file). /This is in line with other observations./

What's your meaning of "perfectly"? You're implementing things in a
very C way, and then showing that two different Python interpreters
have different performance. You haven't actually shown how a
properly-written program would perform.

>> You're not going to prove anything useful about Python - any version
>> of Python - by using it badly. Try implementing JPEG decoding in a
>> more Pythonic way - or, better still, use pillow and don't write that
>> kind of code yourself at all.
>
> Before I wrote this, I looked at three other existing Python decoders, all
> more Pythonic (one had been ported from C++ I think and was crammed full of
> classes).
>
> One worked so poorly that it was dismissed. The other two were full of bugs
> and didn't fully support the 3 main colour formats, but when they did work,
> were 3-6 times slower than my version IIRC.
>
> They also only worked with Python 2 (so presumably would have been even
> slower on 3!) It also meant I couldn't test PyPy on them. I tried psyco but
> it made little difference. And actually the main purpose was to have
> something substantial to try PyPy on.
>
> So I don't know who's using Python more badly: me or those other authors.

Did you try the 'pillow' library?

https://pypi.python.org/pypi/Pillow

It supports 2.6/2.7 and 3.2+, and would be most people's "one obvious
way" to do things. At very least, it should get a mention in your
performance comparisons.

> (I'm quite pleased with my version: smaller, faster, works on all the
> Pythons, supports all 3 colour formats and no decoding bugs that I'm aware
> of, and it's the first Python program I've written that does something
> useful.)

"all the Pythons" meaning what, exactly? And, no decoding bugs? Maybe,
but I wouldn't bet my life on it. Unless your code is a pure port of
someone else's (and unless that someone else could guarantee that the
original C code was bug free), I wouldn't trust code that I can't
completely analyze in one sitting. Virtually all code has bugs in it;
the only code that doesn't is code that's so trivially simple that you
can completely understand it in one "headspace".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Mark Lawrence]

On 07/03/2016 17:38, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff  wrote:

Thanks to all who replied to my cry for help; I understand it better now.
But:

On 07/03/16 16:08, Chris Angelico wrote:



The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.



Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.


You're looking at an ancient version of the docs. Here's the newest docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA



As I happen to be playing with tkinter I was searching literally minutes 
ago for the way to put a grid onto a notebook tab.  Regrettably the 
notebook grid is conspicious by its absence from the docs :(


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 07/03/2016 11:02, BartC wrote:


PyPy I think is only compatible with Python 3 code (there are a few
other issues too).



Quoting from http://doc.pypy.org/en/latest/release-0.7.0.html "The 
interpreter and object model implementations shipped with the 0.7 
version can run on their own and implement the core language features of 
Python as of CPython 2.4.".


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, mm0fmf  wrote:
> On 07/03/2016 18:09, Jon Ribbens wrote:
>> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
>> not 64-bit Intel processors.
>
> You didn't read the bit that says
>
> "The binaries for AMD64 will also work on processors that implement the 
> Intel 64 architecture. (Also known as the "x64" architecture, and 
> formerly known as both "EM64T" and "x86-64".) "

You are quite correct that I did not read the whole of the
ridiculously-long and badly-organised download page.

I would strongly suggest that, at the very least, the 'Description'
for the x86-64 files should say "for all 64-bit PCs (except Itanium*)"
with the '*' linking to a footnote that explains how to tell if you
have one of those.

I must say that Python on Windows was a very poor experience indeed,
"virtualenv" does not work and "venv" refuses to create the 'activate'
shell script so does not work either (and pygame doesn't work, but
that's presumably not Python's fault).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 15:31, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 12:25 AM, BartC  wrote:



(The Python version of that program is here:
http://pastebin.com/cHx3UhQb. It should work with any Python.)


Actually, it won't work with any Python - not if it gets a broken
file. Your abortjpeg() function doesn't work :)


What does it do when it doesn't work? I just get 'can't load file' then 
it stops if there's no such file. With an extra message if there's a 
format error. (I'm not sure what 'raise' does, but it doesn't complain 
about it. I think I realised I didn't know what came next and left it.)



But what you have there is a messy mixture of old-style classes used
as if they were C structs, array.array() as if it were a C array, and
utterly uncommented code that looks like it was ported from, again, C.
Benchmarking Py2 vs Py3 on that kind of code is like trying to figure
out whether it's easier to drag a 747 by your teeth or your toes.
Yeah, you might get a result, but it's a useless one.


I disagree. The program does its job perfectly (you will have to take it 
further to verify the results, such as writing out the .ppm file and 
viewing the contents).


But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for 
a smaller file). /This is in line with other observations./


> You're not going to prove anything useful about Python - any version
> of Python - by using it badly. Try implementing JPEG decoding in a
> more Pythonic way - or, better still, use pillow and don't write that
> kind of code yourself at all.

Before I wrote this, I looked at three other existing Python decoders, 
all more Pythonic (one had been ported from C++ I think and was crammed 
full of classes).


One worked so poorly that it was dismissed. The other two were full of 
bugs and didn't fully support the 3 main colour formats, but when they 
did work, were 3-6 times slower than my version IIRC.


They also only worked with Python 2 (so presumably would have been even 
slower on 3!) It also meant I couldn't test PyPy on them. I tried psyco 
but it made little difference. And actually the main purpose was to have 
something substantial to try PyPy on.


So I don't know who's using Python more badly: me or those other authors.

(I'm quite pleased with my version: smaller, faster, works on all the 
Pythons, supports all 3 colour formats and no decoding bugs that I'm 
aware of, and it's the first Python program I've written that does 
something useful.)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Random832]

On Mon, Mar 7, 2016, at 13:09, Jon Ribbens wrote:
> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
> not 64-bit Intel processors.

Current 64-bit processors produced by Intel use the "AMD64"
architecture, not the Intel IA-64 (Itanium) architecture.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Chris Warrick]

On 7 March 2016 at 19:09, Jon Ribbens  wrote:
> On 2016-03-07, Ian Kelly  wrote:
>> On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:
>>> I am trying to download Python but I have windows 10 and I do not see a 64
>>> bit download for my operating system. Do you have a 64 bit for windows?
>>
>> What page are you looking at?
>> https://www.python.org/downloads/release/python-351/ has downloads for
>> both Windows x86 and Windows x86-64.
>
> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
> not 64-bit Intel processors.
> --
> https://mail.python.org/mailman/listinfo/python-list

Modern Intel processors use the amd64 (aka x86_64) architecture.
Intel’s Itanium architecture (IA-64) never really took off, and was
not supported by the consumer versions of Windows (other than XP x64).

(Not to mention 32-bit processors are sometimes called i386…i686,
where the i stands for Intel, and those processors were also
manufactured by AMD and others)

-- 
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[mm0fmf]

On 07/03/2016 18:09, Jon Ribbens wrote:

On 2016-03-07, Ian Kelly  wrote:

On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:

I am trying to download Python but I have windows 10 and I do not see a 64
bit download for my operating system. Do you have a 64 bit for windows?


What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.


It only appears to have downloads for 32-bit, or 64-bit AMD processors,
not 64-bit Intel processors.



You didn't read the bit that says

"The binaries for AMD64 will also work on processors that implement the 
Intel 64 architecture. (Also known as the "x64" architecture, and 
formerly known as both "EM64T" and "x86-64".) "




--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Mark Lawrence]

On 07/03/2016 17:42, Ian Kelly wrote:

On Mon, Mar 7, 2016 at 10:25 AM, Mark Lawrence  wrote:

On 07/03/2016 16:57, Ian Kelly wrote:


On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales 
wrote:


I am trying to download Python but I have windows 10 and I do not see a
64
bit download for my operating system. Do you have a 64 bit for windows?



What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.

The other question is are you sure that 64-bit Python is what you
want? If your Python is 64-bit then I believe that any extension
modules you use need to be compiled 64-bit as well. On a 64-bit
Windows system you can run either 32-bit or 64-bit Python, and AFAIK
it's more common to use 32-bit Python.



I've been running 64 bit Python on Windows for years with no problems. Why
use 32 bit?  I certainly don't understand why you'd need to.


It seems to be easier to find 32-bit binaries for libraries. For
example, the official Windows build of pygame is only 32-bit. If
64-bit only works for you though, then by all means use it.



As http://www.pygame.org/download.shtml hasn't been updated since August 
6th 2009 and only goes up to Python 3.2 I'd stick with 
http://www.lfd.uci.edu/~gohlke/pythonlibs/#pygame which gives a 64 bit 
3.5 download.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, Ian Kelly  wrote:
> On Mon, Mar 7, 2016 at 10:25 AM, Mark Lawrence  
> wrote:
>> I've been running 64 bit Python on Windows for years with no problems. Why
>> use 32 bit?  I certainly don't understand why you'd need to.
>
> It seems to be easier to find 32-bit binaries for libraries. For
> example, the official Windows build of pygame is only 32-bit.

There doesn't appear to be an official Windows build of pygame at all,
if you want anything more recent than Python 3.1.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, Ian Kelly  wrote:
> On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:
>> I am trying to download Python but I have windows 10 and I do not see a 64
>> bit download for my operating system. Do you have a 64 bit for windows?
>
> What page are you looking at?
> https://www.python.org/downloads/release/python-351/ has downloads for
> both Windows x86 and Windows x86-64.

It only appears to have downloads for 32-bit, or 64-bit AMD processors,
not 64-bit Intel processors.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Ian Kelly]

On Mon, Mar 7, 2016 at 10:25 AM, Mark Lawrence  wrote:
> On 07/03/2016 16:57, Ian Kelly wrote:
>>
>> On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales 
>> wrote:
>>>
>>> I am trying to download Python but I have windows 10 and I do not see a
>>> 64
>>> bit download for my operating system. Do you have a 64 bit for windows?
>>
>>
>> What page are you looking at?
>> https://www.python.org/downloads/release/python-351/ has downloads for
>> both Windows x86 and Windows x86-64.
>>
>> The other question is are you sure that 64-bit Python is what you
>> want? If your Python is 64-bit then I believe that any extension
>> modules you use need to be compiled 64-bit as well. On a 64-bit
>> Windows system you can run either 32-bit or 64-bit Python, and AFAIK
>> it's more common to use 32-bit Python.
>>
>
> I've been running 64 bit Python on Windows for years with no problems. Why
> use 32 bit?  I certainly don't understand why you'd need to.

It seems to be easier to find 32-bit binaries for libraries. For
example, the official Windows build of pygame is only 32-bit. If
64-bit only works for you though, then by all means use it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Ian Kelly]

On Mon, Mar 7, 2016 at 10:23 AM, Tony van der Hoff  wrote:
> However, more generally, how am I supposed to know that a module is part of
> a package, and needs a "magic" stanza to get a module loaded?

If the import path of the module has a dot in it, then it's part of a package.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Chris Angelico]

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff  wrote:
> Thanks to all who replied to my cry for help; I understand it better now.
> But:
>
> On 07/03/16 16:08, Chris Angelico wrote:
>>
>>
>> The documentation should tell you what you need to import to make
>> something work. In this case, I would guess that "import
>> tkinter.messagebox" or "from tkinter import messagebox" would be the
>> recommended way to use this module.
>
>
> Well, I found the tkinter documentation to be sparse in the extreme
> (https://docs.python.org/release/3.1.3/library/tkinter.html), and it
> certainly doesn't go into that sort of detail.

You're looking at an ancient version of the docs. Here's the newest docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Mark Lawrence]

On 07/03/2016 17:23, Tony van der Hoff wrote:

Thanks to all who replied to my cry for help; I understand it better
now. But:

On 07/03/16 16:08, Chris Angelico wrote:


The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.


Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.

However, more generally, how am I supposed to know that a module is part
of a package, and needs a "magic" stanza to get a module loaded?

Cheers,


It all depends on whether or not there is an __init__.py file in the 
directory, see e.g. http://pythoncentral.io/how-to-create-a-python-package/


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Mark Lawrence]

On 07/03/2016 16:38, Tim Golden wrote:

On 07/03/2016 16:25, Xiang Zhang wrote:

Hi,

I know I can get the attribute name in some way, but since I just
want the attribute name when an AttributeError caused by it raised, I
really don't want to inspect the string or introduce one more layer
over getattr. I hope I can get the attribute which causes the
exception from the AttributeError raised directly. But it seems I
can't.



As things stand, you can't. But if you were to chime in on this issue:

   https://bugs.python.org/issue18156

then you might bring discussion there back to life again and see some
forward movement. There seemed to be a consensus (and involving some
active developers) so it's possible that new interest might revive interest.

TJG



Thanks, you saved me searching for the link that I knew existed :)

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Mark Lawrence]

On 07/03/2016 16:57, Ian Kelly wrote:

On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:

I am trying to download Python but I have windows 10 and I do not see a 64
bit download for my operating system. Do you have a 64 bit for windows?


What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.

The other question is are you sure that 64-bit Python is what you
want? If your Python is 64-bit then I believe that any extension
modules you use need to be compiled 64-bit as well. On a 64-bit
Windows system you can run either 32-bit or 64-bit Python, and AFAIK
it's more common to use 32-bit Python.



I've been running 64 bit Python on Windows for years with no problems. 
Why use 32 bit?  I certainly don't understand why you'd need to.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Tony van der Hoff]

Thanks to all who replied to my cry for help; I understand it better 
now. But:


On 07/03/16 16:08, Chris Angelico wrote:


The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.


Well, I found the tkinter documentation to be sparse in the extreme 
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it 
certainly doesn't go into that sort of detail.


However, more generally, how am I supposed to know that a module is part 
of a package, and needs a "magic" stanza to get a module loaded?


Cheers,
--
Tony van der Hoff| mailto:t...@vanderhoff.org
Buckinghamshire, England |
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Xiang Zhang]

On Tuesday, March 8, 2016 at 12:38:31 AM UTC+8, Tim Golden wrote:
> On 07/03/2016 16:25, Xiang Zhang wrote:
> > Hi,
> > 
> > I know I can get the attribute name in some way, but since I just
> > want the attribute name when an AttributeError caused by it raised, I
> > really don't want to inspect the string or introduce one more layer
> > over getattr. I hope I can get the attribute which causes the
> > exception from the AttributeError raised directly. But it seems I
> > can't.
> > 
> 
> As things stand, you can't. But if you were to chime in on this issue:
> 
>   https://bugs.python.org/issue18156
> 
> then you might bring discussion there back to life again and see some
> forward movement. There seemed to be a consensus (and involving some
> active developers) so it's possible that new interest might revive interest.
> 
> TJG

Thanks for mentioning the issue. I'll ping it to see if there is still anyone 
interested in it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Ian Kelly]

On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:
> I am trying to download Python but I have windows 10 and I do not see a 64
> bit download for my operating system. Do you have a 64 bit for windows?

What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.

The other question is are you sure that 64-bit Python is what you
want? If your Python is 64-bit then I believe that any extension
modules you use need to be compiled 64-bit as well. On a 64-bit
Windows system you can run either 32-bit or 64-bit Python, and AFAIK
it's more common to use 32-bit Python.
-- 
https://mail.python.org/mailman/listinfo/python-list

Question

[Ben Morales]

I am trying to download Python but I have windows 10 and I do not see a 64
bit download for my operating system. Do you have a 64 bit for windows?
Sincerely,

*Ben Morales*
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A mistake which almost went me mad

[Chris Angelico]

On Tue, Mar 8, 2016 at 3:19 AM, Ian Kelly  wrote:
> On Thu, Mar 3, 2016 at 11:50 AM, Tim Chase
>  wrote:
>> I think that relative imports should ameliorate this, as I usually
>> hit it when I'm using smtplib which in turn imports "email" (and, in
>> 2.x when it found my local email.py would crash and burn). If it used
>> a relative import that forced it to find the one in the stdlib, it
>> should(?) prevent it from finding my local version first.
>
> Relative imports only work inside packages. You can't use a relative
> import to import one top-level module from another.
>
> Besides, the relative import doesn't help to disambiguate in this
> case. The absolute path of the stdlib email module is "email". The
> absolute path of the module in your CWD is also "email". Why should a
> relative import prefer one over the other? So I would think that even
> if it worked, it would still just end up importing the first one it
> finds on your sys.path.

So the solution is to treat the current directory as a pseudo-package.
It'd be a backward-incompatible change, so it would need to be
explicitly invoked. Something like:

python3 -p somefile.py

which would pretend to create an __init__.py in the current directory,
change to the parent, and "from dirname import somefile". I mentioned
this one time before, but it got kinda missed; I think I'll reraise it
on python-ideas. It's a protection on par with not having the current
directory in $PATH.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Tim Golden]

On 07/03/2016 16:25, Xiang Zhang wrote:
> Hi,
> 
> I know I can get the attribute name in some way, but since I just
> want the attribute name when an AttributeError caused by it raised, I
> really don't want to inspect the string or introduce one more layer
> over getattr. I hope I can get the attribute which causes the
> exception from the AttributeError raised directly. But it seems I
> can't.
> 

As things stand, you can't. But if you were to chime in on this issue:

  https://bugs.python.org/issue18156

then you might bring discussion there back to life again and see some
forward movement. There seemed to be a consensus (and involving some
active developers) so it's possible that new interest might revive interest.

TJG
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Xiang Zhang]

Hi,

I know I can get the attribute name in some way, but since I just want the 
attribute name when an AttributeError caused by it raised, I really don't want 
to inspect the string or introduce one more layer over getattr. I hope I can 
get the attribute which causes the exception from the AttributeError raised 
directly. But it seems I can't.
-- 
https://mail.python.org/mailman/listinfo/python-list