Re: How can I speed up a script that iterates over a large range (600 billion)?

[Paul Rubin]

Chris Torek  writes:
> def primes():
> """
> Yields sequence of prime numbers via Sieve of Eratosthenes.
> """

I think this has the same order-complexity but is enormously slower in
practice, and runs out of recursion stack after a while.  Exercise: spot
the recursion.

from itertools import islice, ifilter, count

def primes():
a = count(2)
while True:
p = a.next()
yield p
a = ifilter(lambda t,p=p: t%p, a)

# print first 100 primes
print list(islice(primes(), 100))
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Chris Torek]

Now that the exercise has been solved...

Instead of "really short code to solve the problem", how about 
some "really long code"? :-)

I was curious about implementing prime factorization as a generator,
using a prime-number generator to come up with the factors, and
doing memoization of the generated primes to produce a program that
does what "factor" does, e.g.:

$ factor 9
9: 3 3 2071723 5363222357

The python code is rather too slow for this particular number (I
gave up on it finding 5363222357) but works quite well on 600851475143,
or even, e.g., 12186606004219:

$ python factor.py 600851475143 12186606004219
600851475143: 71 839 1471 6857
12186606004219: 2071723 5882353

While searching for speed hacks I came across a few interesting
tricks, particularly TZTZIOY's mod-30 scheme (erat3) at
stackoverflow.com (see URLs in the comments), which only really
works well in Python 2.7 and later (using itertools.compress).
Here is the end result (run with 2.5 and 2.7, I have no 3.x installed
anywhere convenient, and of course the print calls would change):

import itertools
import sys

def count(start = 0, step = 1):
"""
Yield count starting from  and counting up by .
Same as itertools.count() except for the  argument, and
allowing non-"int" arguments.

Python 2.7 and later provides this directly, via itertools.

Note: it's usually faster to use itertools.islice(itertools.count(...))
than to run the "while True" loop below, so we do that if we can.
"""
if (sys.version_info[0] > 2 or
(sys.version_info[0] == 2 and sys.version_info[1] >= 7)):
return itertools.count(start, step)
if isinstance(start, int) and isinstance(step, int):
if step == 1:
return itertools.count(start)
if 1 < step < 5: # 5 upper bound is a guess
return itertools.islice(itertools.count(start), 0, None, step)
def f(start, step):
while True:
yield start
start += step
return f(start, step)

# Sieve of Eratosthenes-based prime-number generator.
#
# Based on David Eppstein's for python 2.3(?) and subsequent
# discussion -- see
# http://code.activestate.com/recipes/117119-sieve-of-eratosthenes/
#
# See also:
# http://oreilly.com/pub/a/python/excerpt/pythonckbk_chap1/index1.html?page=last
#
# 
http://stackoverflow.com/questions/2211990/how-to-implement-an-efficient-infinite-generator-of-prime-numbers-in-python/3796442#3796442
def primes():
"""
Yields sequence of prime numbers via Sieve of Eratosthenes.
"""
# Keep the state from the last time we abandoned the
# generator, in primes.primes and primes.D.  We can then
# very quickly re-yield previously-saved primes.  We're
# going to skip even numbers below, we so prime the
# "primes so far" list with 2.
#
# For the fancy (erat3) version, we need to pre-provide
# 2, 3, and 5, and pre-load D.  Having done that we can
# start either version at 7.
try:
primes.primes
except AttributeError:
primes.primes = [2, 3, 5]
for p in primes.primes:
yield p
# OK, now we need a mapping holding known-composite values
# (numbers "struck from the sieve").
try:
D = primes.D
except AttributeError:
D = primes.D = { 9: 3, 25: 5 }
# D[q] exists if q is composite; its value is the first prime
# number that proves that q is composite.  (However, only odd
# numbers appear in D.)
q = p + 2 # where we start sieve-ing, below
if sys.version_info[0] == 2 and sys.version_info[1] < 7:
for q in count(q, 2):
p = D.pop(q, None)
if p is None:
# q was not marked composite, so it's prime; moreover,
# q-squared is composite (and odd) and its first prime
# factor is q.
primes.primes.append(q)
D[q * q] = q
yield q
else:
# q is composite, p is its first prime factor -- e.g.,
# q=9 and p=3, or q=15 and p=3.  Extend the sieve:
# find first natural number k where q + 2kp is not already
# already known as composite.  (e.g., if q=9 and p=3, k=1
# so that we mark D[15] as composite, with 3 as its first
# prime factor.)  Note that we are skipping even numbers,
# so p and q are both odd, so q+p is even, q+2p is odd,
# q+3p is even, q+4p is odd, etc.  We don't need to mark
# even-numbered composites in D, which is why we only look
# at q + 2kp.
twop = p + p
x = q + twop # next odd multiple of p
while x in D: # skip already-known composites
x += twop
D[x] = p
else:
#   7  9 11 13 15 17 19 21 23 25 27 29 31 33 35
MASK = (1, 0, 1, 1

Re: Security test of embedded Python

[Dennis]

Hi,

The Google App Engine product seems to sandbox Python code, however it
comes with a lot of limitations and maybe those can be an inspiration
for how you design your infrastructure.

http://code.google.com/appengine/docs/python/overview.html

http://code.google.com/appengine/kb/commontasks.html

I hope this helps somewhat - I know lacking some specifics.

Dennis O.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Paul Rubin]

Terry Reedy  writes:
> If the best C program for a problem takes 10 seconds or more, then
> applying the same 1 minute limit to Python is insane, and contrary to
> the promotion of good algorithm thinking.

The Euler problems are all designed to be doable in 1 minute or less and
the Euler project started in 2001, when personal computers were probably
10x slower than they are today.  So they shouldn't take more than 6
seconds on a modern computer if you're thoughtful.  On a reasonable
modern computer (maybe even a 2001 computer), they should all be doable
in < 1 minute in python, probably well under.  They can probably all be
done in under 1 second in C.

The "largest prime factor of 600851475143" problem we're discussing took
0.017 user cpu seconds in completely unoptimized python on my laptop
using the second-most-naive algoritm possible, including loading the
interpreter from the command line.  Executing an empty file takes the
same amount of time.  The algorithm would probably be >10x faster in C
with a little bit of tweaking.  The problems are about math cleverness,
not CPU resources.  Some of them are pretty hard, but if your solution
is taking more than a minute in Python, it means you should be looking
for a better algorithm, not a faster language.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter/scrollbar/canvas question

[Chris Angelico]

On Wed, Jun 22, 2011 at 1:50 PM, Saul Spatz  wrote:
> This is the third time I've tried to post this reply.  If you see multiple 
> answers from me, that's why.
>

All three came through on the mailing list, but out of order - this
one came in second.

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter/scrollbar/canvas question

[Saul Spatz]

It works if you change it like so:

from tkinter import *
class ShowList(Frame):
def __init__(self, root):
Frame.__init__(self, root)
self.grid()
self.draw_widgets()
def draw_widgets(self):
cframe = Frame(self)
cframe.grid(row=1, sticky=N+S+E+W)
canv = Canvas(cframe)
canv.grid(row=0, column=0, sticky=N+S+E+W)
vscroll = Scrollbar(cframe, orient=VERTICAL, command=canv.yview)
hscroll = Scrollbar(cframe, orient=HORIZONTAL, 
command=canv.xview)
vscroll.grid(row=0, column=1, sticky=N+S)
hscroll.grid(row=1, column=0, sticky=E+W)
canv["xscrollcommand"] = hscroll.set
canv["yscrollcommand"] = vscroll.set
aframe = Frame(canv)
id = canv.create_window(0,0,window=aframe, anchor=N+W)
for i in range(0,100):
Label(aframe, text=str(i), anchor=N+W).grid(row=i, 
column=0)
aframe.update_idletasks()
canv["scrollregion"]=canv.bbox(ALL)
root  = Tk()
m=ShowList(root)
root.mainloop()

You have to call update_idletasks to force the canvas to be mapped to screen; 
until it is mapped, its bounding box is (0,0,1,1).  You can call 
update_idletasks through any widget.

That said, I wonder if it wouldn't be better to put canvas objects directly on 
the canvas, instead of putting widgets in a frame on the canvas.  The way 
you're doing it, you can't give the widgets tags, which are what give the 
canvas its real power. 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Handling import errors

[Chris Rebert]

On Tue, Jun 21, 2011 at 1:51 PM, Guillaume Martel-Genest
 wrote:
> What is the pythonic way to handle imports error? What is bugging me
> is that the imports can't be inside a function (because I use them in
> different places in the script and thus they have to be in the global
> scope). I would write something like:
>
> try:
>    import foo
> except ImportError:
>    logging.error('could not import foo')
>    sys.exit(1)
>
> But logging is not configured at this point as my main() have not been
> called yet.
>
> Should I define a global variable and assign it to my module later? Or
> should I let the exception happen and let the stack trace be the error
> message?

If your users are technical, the latter, since it's much more
informative to anyone with programming/sysadmin skills. It's also not
really the sort of error your program can usefully recover from.
(Although in some cases, the module being imported may be considered
truly optional, in which case `try...except ImportError` makes sense;
but this is fairly uncommon unless a program is intended to have
extensions/plug-ins.)

If your users aren't technical, then you should have a top-level
try...except around almost the entire program that displays a simple
error message in the event of an unhandled exception, preferably with
an option to display the gory details (i.e. exception & stack trace).

Cheers,
Chris
--
http://rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Emails backup in python 3.2

[Michael Hrivnak]

Why not use one of the many projects and products that are designed to
store email?  Do you have a special reason for wanting to implement
your own email storage?

I'm thinking that you can use fetchmail with your favorite mail store,
and you won't need to write any code at all.

http://fetchmail.berlios.de/

Michael

On Tue, Jun 21, 2011 at 9:03 AM, TheSaint  wrote:
> Hello,
> I'm looking for an idea how to backup emails retrieved by poplib and save
> them into mailbox.mbox file.
> The problem is the received message which is a list of bytes streams,
> mailbox.mbox don't expect a list.
> What conversion should I do?
> A file type io.StringIO ?
> decoding every bytes stream which might not have any declared codec?
>
> As far as python moved into unicode, why doesn't it handle these undecoded
> bytes as it was with strings before?
>
> --
> goto /dev/null
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Terry Reedy]

On 6/21/2011 8:00 PM, Paul Rubin wrote:

Terry Reedy  writes:

efficient implementation will allow a solution to be obtained on a
modestly powered computer in less than one minute."

If something really takes a minute in C,
allow yourself at least 10 minutes or even more with plain CPython.


No.


No what? My conditional statement is correct. It turns out not to apply 
here. Great.



The idea of the Euler problems is to think up sane algorithms to
solve them, not micro-optimize or use low level languages on crappy
algorithms.

  n=600851475143
  for d in xrange(2,n):
if d*d>  n: break
   while n%d == 0: n //= d
  print n

finishes on my laptop with no noticable pause.


If the best C program for a problem takes 10 seconds or more, then 
applying the same 1 minute limit to Python is insane, and contrary to 
the promotion of good algorithm thinking.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter/scrollbar/canvas question

[Saul Spatz]

This is the third time I've tried to post this reply.  If you see multiple 
answers from me, that's why.

Your script will work if you change it like so:

from tkinter import *
class ShowList(Frame):
def __init__(self, root):
Frame.__init__(self, root)
self.grid()
self.draw_widgets()
def draw_widgets(self):
cframe = Frame(self)
cframe.grid(row=1, sticky=N+S+E+W)
canv = Canvas(cframe)
canv.grid(row=0, column=0, sticky=N+S+E+W)
vscroll = Scrollbar(cframe, orient=VERTICAL, command=canv.yview)
hscroll = Scrollbar(cframe, orient=HORIZONTAL, 
command=canv.xview)
vscroll.grid(row=0, column=1, sticky=N+S)
hscroll.grid(row=1, column=0, sticky=E+W)
canv["xscrollcommand"] = hscroll.set
canv["yscrollcommand"] = vscroll.set
aframe = Frame(canv)
id = canv.create_window(0,0,window=aframe, anchor=N+W)
for i in range(0,100):
Label(aframe, text=str(i), anchor=N+W).grid(row=i, 
column=0)
aframe.update_idletasks()
canv["scrollregion"]=canv.bbox(ALL)
root  = Tk()
m=ShowList(root)
root.mainloop()

You need to call update_idletasks to force the canvas to be mapped to the 
screen before you compute the bounding box.  Until it's mapped, the bounding 
box is (0,0,1,1), and of course you need to put the widgets on before you 
compute the bounding box.  You can call update_idletasks through any widget.  

That said, I wonder if it wouldn't be better to put canvas objects directly on 
the canvas, instead of putting widgets in a frame.  The way you'r doing it, you 
can't give the widgets tags, and tags are what give the canvas its power.

Saul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode codepoints

[Chris Angelico]

On Wed, Jun 22, 2011 at 1:37 PM, Saul Spatz  wrote:
> Hi,
>
> I'm just starting to learn a bit about Unicode. I want to be able to read a 
> utf-8 encoded file, and print out the codepoints it encodes.  After many 
> false starts, here's a script that seems to work, but it strikes me as 
> awfully awkward and unpythonic.  Have you a better way?

Once you have your data as a Unicode string (and you seem to be using
Python 3, so 's' will be a Unicode string), wouldn't a list of its
codepoints simply be this?

for c in s:
  print('U+'+hex(ord(c))[2:])

But if you do need the codePoints() function, I'd do it as a generator.

> def codePoints(s):
>    ''' return a list of the Unicode codepoints in the string s '''
>    skip = False
>    for k, c in enumerate(s):
>        if skip:
>            skip = False
>            yield ord(s[k-1:k+1])
>            continue
>        if not 0xd800 <= ord(c) <= 0xdfff:
>            yield ord(c)
>        else:
>            skip = True

Your main function doesn't even have to change - it's iterating over
the list, so it may as well iterate over the generator instead.

But I don't really understand what codePoints() does. Is it expecting
the parameter to be a string of bytes or of Unicode characters?

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter/scrollbar/canvas question

[Saul Spatz]

It works if you change it like so:

from tkinter import *
class ShowList(Frame):
def __init__(self, root):
Frame.__init__(self, root)
self.grid()
self.draw_widgets()
def draw_widgets(self):
cframe = Frame(self)
cframe.grid(row=1, sticky=N+S+E+W)
canv = Canvas(cframe)
canv.grid(row=0, column=0, sticky=N+S+E+W)
vscroll = Scrollbar(cframe, orient=VERTICAL, command=canv.yview)
hscroll = Scrollbar(cframe, orient=HORIZONTAL, 
command=canv.xview)
vscroll.grid(row=0, column=1, sticky=N+S)
hscroll.grid(row=1, column=0, sticky=E+W)
canv["xscrollcommand"] = hscroll.set
canv["yscrollcommand"] = vscroll.set
aframe = Frame(canv)
id = canv.create_window(0,0,window=aframe, anchor=N+W)
for i in range(0,100):
Label(aframe, text=str(i), anchor=N+W).grid(row=i, 
column=0)
aframe.update_idletasks()
canv["scrollregion"]=canv.bbox(ALL)
root  = Tk()
m=ShowList(root)
root.mainloop()

You need to do the update_idletasks to force the canvas to be mapped before you 
figure out the bounding box.  Until the canvas is mapped to the screen, the 
bounding box is (0,0,1,1) so there no scrolling possible.  (You can call 
update_ideltasks through any widget.)

That said, I wonder why you're putting widgets in the frame instead of putting 
objects directly on the canvas.  The way you're doing it you can't use tags, 
which are what really give the canvas its power.

Saul

   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[John Salerno]

On Jun 21, 10:02 pm, Mel  wrote:
> John Salerno wrote:
> > ::sigh:: Well, I'm stuck again and it has to do with my get_factors
> > function again, I think. Even with the slight optimization, it's
> > taking forever on 20! (factorial, not excitement)  :) It's frustrating
> > because I have the Python right, but I'm getting stuck on the math.
>
> > The problem:
>
> > "What is the smallest positive number that is evenly divisible by all
> > of the numbers from 1 to 20?"
>
> > Here's the function (it's in the problem3.py file, hence the import
> > below):
>
> > import math
>
> > def get_factors(number):
> >     factors = []
>
> >     for n in range(2, int(math.sqrt(number))):
> >         if number % n == 0:
> >             factors.append(n)
> >             factors.append(number // n)
>
> >     return factors
>
> > And here's my new script for the new exercise:
>
> > import math
> > from problem3 import get_factors
>
> > max_num = 20
> > n = math.factorial(max_num)
> > factors = get_factors(n)
> > div_all = []
>
> > for x in factors:
> >     for y in range(2, max_num+1):
> >         if x % y != 0:
> >             break
> >         elif y == max_num:
> >             div_all.append(x)
>
> > print(min(div_all))
>
> > It could easily be that I'm simply approaching it all wrong. I just
> > thought that maybe using the factorial of the highest number in the
> > range (in this case, 20) would be an easy way of finding which numbers
> > to test.
>
> These are almost "trick questions" in a way, because of the math behind
> them.  If the question were "What is the tallest high-school student in
> Scranton, PA?" then searching a population for the property would be the
> only way to go.  BUT you can also build up the answer knowing the
> factorization of all the numbers up to 20.
>
>         Mel.

I think you're right. I just read the next problem and it is similar
in style, i.e. the example solution involves a small set of numbers
which I can write a script for and it would execute within a second,
but when I expand the script to include the larger set of numbers for
the problem, it then takes ages to execute, making me feel like the
trick isn't to figure out the right code, but the right math.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Security test of embedded Python

[Paul Rubin]

Chris Angelico  writes:
> Meanwhile, I'm looking into V8 and whether we can do everything we
> need to that way, and how much dev time it's going to take me to
> change languages...

If you want to run Python, one obvious approach is a
controlled-execution wrapper like Geordi uses.
-- 
http://mail.python.org/mailman/listinfo/python-list

Unicode codepoints

[Saul Spatz]

Hi,

I'm just starting to learn a bit about Unicode. I want to be able to read a 
utf-8 encoded file, and print out the codepoints it encodes.  After many false 
starts, here's a script that seems to work, but it strikes me as awfully 
awkward and unpythonic.  Have you a better way?

def codePoints(s):
''' return a list of the Unicode codepoints in the string s '''
answer = []
skip = False
for k, c in enumerate(s):
if skip:
skip = False
answer.append(ord(s[k-1:k+1]))
continue
if not 0xd800 <= ord(c) <= 0xdfff:
answer.append(ord(c))
else:
skip = True
return answer

if __name__ == '__main__':
s = open('test.txt', encoding = 'utf8', errors = 'replace').read()
code = codePoints(s)
for c in code:
print('U+'+hex(c)[2:])

Thanks for any help you can give me.

Saul


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Security test of embedded Python

[Chris Angelico]

On Wed, Jun 22, 2011 at 1:09 PM, Benjamin Kaplan
 wrote:
> Use Pyjamas with that and now you have your sandboxed Python :)
>

Not a day goes past without a reminder that I haven't yet explored Pyjamas! :)

Monty's back online now in a restricted environment. I'm going to a
meeting in a couple of hours where we will decide where to go from
here; between now and then, if anyone can gain filesystem or OS
access, that will probably put the final nail in the coffin of us
using Python.

Meanwhile, I'm looking into V8 and whether we can do everything we
need to that way, and how much dev time it's going to take me to
change languages...

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[John Salerno]

On Jun 21, 9:09 pm, Paul Rubin  wrote:
> John Salerno  writes:
> > It's frustrating because I have the Python right, but I'm getting
> > stuck on the math
> > "What is the smallest positive number that is evenly divisible by all
> > of the numbers from 1 to 20?"
>
> The answer is lcm [1,2,3, ... 20].  You can figure out how to implement
> lcm.
>
> The Euler problems are not really programming exercises.  They are
> exercises in math and algorithms.  Quite a lot of them involve thinking
> clever and fast ways to do stuff that would be trivial (but too slow) by
> brute force.  In general, once you figure out the right algorithm,
> writing the code is easy.  But you have to be fairly mathematically
> attuned, to have any chance of spotting the algorithm.
>
> If you want programming exercises that are less mathematical, there are
> some nice ones at rubyquiz.com.  They are intended for Ruby but of
> course you can solve them in Python.

Thanks. So far they are helping me with Python too, but definitely not
as much as more general exercises would, I'm sure. The part about
writing the code is fun, but once that's done, I seem to end up stuck
with an inefficient implementation because I don't know the math
tricks behind the problem.

I'll check out rubyquiz.com. I've been searching for some Python
exercises to do but haven't found too many sites with them, at least
not in such a nice and organized way as Project Euler.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Security test of embedded Python

[Benjamin Kaplan]

On Tue, Jun 21, 2011 at 7:40 PM, Paul Rubin  wrote:
> Chris Angelico  writes:
>> I'll also be looking into Pike. Unfortunately its community is far
>> smaller than Python's, so security holes may be less obvious.
>
> Actually the most obvious and widespread sandboxed language these days
> is Javascript.  There's several embeddable implementations.  Maybe you
> should just use one of those.

Use Pyjamas with that and now you have your sandboxed Python :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Benjamin Kaplan]

On Tue, Jun 21, 2011 at 4:30 PM, Terry Reedy  wrote:
> On 6/21/2011 3:48 PM, John Salerno wrote:
>
>> Absolutely not! Each problem has been designed according to a "one-
>> minute rule", which means that although it may take several hours to
>> design a successful algorithm with more difficult problems, an
>> efficient implementation will allow a solution to be obtained on a
>> modestly powered computer in less than one minute."
>
> That statement is for C, not Python. Python is efficient with human time,
> but not machine time. If something really takes a minute in C, allow
> yourself at least 10 minutes or even more with plain CPython.
>
> --
> Terry Jan Reedy

Python is the second most popular language on Project Euler, at 14358
users compared to 15897 who use C/C++. I'm pretty sure they don't
assume you use C. Although Python's longs do make some of the early
problems really really easy.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Mel]

John Salerno wrote:

> ::sigh:: Well, I'm stuck again and it has to do with my get_factors
> function again, I think. Even with the slight optimization, it's
> taking forever on 20! (factorial, not excitement)  :) It's frustrating
> because I have the Python right, but I'm getting stuck on the math.
> 
> The problem:
> 
> "What is the smallest positive number that is evenly divisible by all
> of the numbers from 1 to 20?"
> 
> 
> 
> Here's the function (it's in the problem3.py file, hence the import
> below):
> 
> import math
> 
> def get_factors(number):
> factors = []
> 
> for n in range(2, int(math.sqrt(number))):
> if number % n == 0:
> factors.append(n)
> factors.append(number // n)
> 
> return factors
> 
> And here's my new script for the new exercise:
> 
> import math
> from problem3 import get_factors
> 
> max_num = 20
> n = math.factorial(max_num)
> factors = get_factors(n)
> div_all = []
> 
> for x in factors:
> for y in range(2, max_num+1):
> if x % y != 0:
> break
> elif y == max_num:
> div_all.append(x)
> 
> print(min(div_all))
> 
> It could easily be that I'm simply approaching it all wrong. I just
> thought that maybe using the factorial of the highest number in the
> range (in this case, 20) would be an easy way of finding which numbers
> to test.

These are almost "trick questions" in a way, because of the math behind 
them.  If the question were "What is the tallest high-school student in 
Scranton, PA?" then searching a population for the property would be the 
only way to go.  BUT you can also build up the answer knowing the 
factorization of all the numbers up to 20.

Mel.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Security test of embedded Python

[Paul Rubin]

Chris Angelico  writes:
> I'll also be looking into Pike. Unfortunately its community is far
> smaller than Python's, so security holes may be less obvious.

Actually the most obvious and widespread sandboxed language these days
is Javascript.  There's several embeddable implementations.  Maybe you
should just use one of those.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Security test of embedded Python

[Chris Angelico]

Followup: The test box has been administratively taken offline after
about an hour of testing. Thank you to everyone who participated; it
seems we have a lot of changes to make!

Monty failed the test. But it was an incredibly successful test. And
hopefully, we'll be bringing things back online for another shot once
things are sorted out!

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Security test of embedded Python

[Chris Angelico]

On Wed, Jun 22, 2011 at 12:02 PM, Paul Rubin  wrote:
> Chris Angelico  writes:
>> users to supply scripts which will then run on our servers...
>> The environment is Python 3.3a0 embedded in C++, running on Linux.
>
> This doesn't sound like a bright idea, given the well-known difficulty
> of sandboxing Python.

So it seems! Less than half an hour after I made the announcement
post, the box had been compromised.

> Geordi  has some interesting
> examples (C++) you might want to try translating to Python and running
> on your server.  It uses ptrace to control the execution of potentially
> hostile code.  I don't know if any exploits have been found or whether
> it's still active.

Thanks, will look into it.

> Maybe you want to look at Lua.  IMHO it's not a very nice language, but
> I've heard that it's easy to embed and sandbox.

Yeah, I've used Lua before (in a game called Angband), and it's not
that great. But security's more important than ideal language syntax.

I'll also be looking into Pike. Unfortunately its community is far
smaller than Python's, so security holes may be less obvious.

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[MRAB]

On 22/06/2011 02:21, John Salerno wrote:

::sigh:: Well, I'm stuck again and it has to do with my get_factors
function again, I think. Even with the slight optimization, it's
taking forever on 20! (factorial, not excitement)  :) It's frustrating
because I have the Python right, but I'm getting stuck on the math.

The problem:

"What is the smallest positive number that is evenly divisible by all
of the numbers from 1 to 20?"


You don't need factorials, just remember that each of the numbers can
be expressed as the product of a multiset of prime factors.
--
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Paul Rubin]

John Salerno  writes:
> It's frustrating because I have the Python right, but I'm getting
> stuck on the math
> "What is the smallest positive number that is evenly divisible by all
> of the numbers from 1 to 20?"

The answer is lcm [1,2,3, ... 20].  You can figure out how to implement
lcm.

The Euler problems are not really programming exercises.  They are
exercises in math and algorithms.  Quite a lot of them involve thinking
clever and fast ways to do stuff that would be trivial (but too slow) by
brute force.  In general, once you figure out the right algorithm,
writing the code is easy.  But you have to be fairly mathematically
attuned, to have any chance of spotting the algorithm.

If you want programming exercises that are less mathematical, there are
some nice ones at rubyquiz.com.  They are intended for Ruby but of
course you can solve them in Python.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Security test of embedded Python

[Paul Rubin]

Chris Angelico  writes:
> users to supply scripts which will then run on our servers...
> The environment is Python 3.3a0 embedded in C++, running on Linux.

This doesn't sound like a bright idea, given the well-known difficulty
of sandboxing Python.

Geordi  has some interesting
examples (C++) you might want to try translating to Python and running
on your server.  It uses ptrace to control the execution of potentially
hostile code.  I don't know if any exploits have been found or whether
it's still active.

Maybe you want to look at Lua.  IMHO it's not a very nice language, but
I've heard that it's easy to embed and sandbox.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[John Salerno]

::sigh:: Well, I'm stuck again and it has to do with my get_factors
function again, I think. Even with the slight optimization, it's
taking forever on 20! (factorial, not excitement)  :) It's frustrating
because I have the Python right, but I'm getting stuck on the math.

The problem:

"What is the smallest positive number that is evenly divisible by all
of the numbers from 1 to 20?"



Here's the function (it's in the problem3.py file, hence the import
below):

import math

def get_factors(number):
factors = []

for n in range(2, int(math.sqrt(number))):
if number % n == 0:
factors.append(n)
factors.append(number // n)

return factors

And here's my new script for the new exercise:

import math
from problem3 import get_factors

max_num = 20
n = math.factorial(max_num)
factors = get_factors(n)
div_all = []

for x in factors:
for y in range(2, max_num+1):
if x % y != 0:
break
elif y == max_num:
div_all.append(x)

print(min(div_all))

It could easily be that I'm simply approaching it all wrong. I just
thought that maybe using the factorial of the highest number in the
range (in this case, 20) would be an easy way of finding which numbers
to test.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using django ORM from web browser and from command line apps

[News123]

On 06/22/2011 03:08 AM, Ian Kelly wrote:
> On Tue, Jun 21, 2011 at 6:21 PM, News123  wrote:
>> Out of curiousity: Do you know whether the imports would be executed for
>> each potential command as soon as I call manage.py or only
>> 'on demand'?
> 
> Off the top of my head, I don't know.

Never mind.
WIill jsut add a print statement when I make my first test.
This should clearly show whether the module is imported all the time or
only on demand.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using only the django ORM (DB access model) and nothing else.

[News123]

On 06/22/2011 03:02 AM, Roy Smith wrote:
> In article <4e012e8d$0$23682$426a3...@news.free.fr>,
>  News123  wrote:
> 
> 
> I don't see any reason you couldn't use the Model layer by itself, if 
> you want to.  It pretty much stands on its own.

Thanks a lot for confirming,

I have now my small example.

Just wanted to be sure I don't overlook some tiny, but really annoying
detail which would strongly advise against using the model outside of a
web framework.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using only the django ORM (DB access model) and nothing else.

[News123]

On 06/22/2011 03:04 AM, Ian Kelly wrote:
>>
>> So I need at least a little more to make my script work.
> 
> There's a bit of magic in the way Django finds things, and I think
> you'll still need to keep the basic structure of a Django project --
> models should be in a "models.py" file located in an "app" package,
> which should be included in the INSTALLED_APPS setting.  You just
> won't have any views or urlconfs or templates or admin sites or
> anything like that.

Hi Ian,

Thanks for your answer.
Ourt messages crossed. I had exactly the same idea and started playing..
and you are right.

The settings module needs only
 DATABASES
and INSTALLED_APPS with one app

and in the apps dir I need apart from the compulsory __init__.py only
models.py

Cool






-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using only the django ORM (DB access model) and nothing else.

[News123]

It seems I found a solution (refer to end of this tessage).

Not sure though if there are any drawbacks or if this method of working
could cause any other issues.


 On 06/22/2011 02:42 AM, News123 wrote:
> On 06/22/2011 01:51 AM, News123 wrote:
>> Hi,
>>
>> I have a small application running on a host without web server and
>> without any need for django except its ORM accessing data bases without
>> explicitely writing sql queries.)
>>
>> I assume there's many libraries (SQL Alchemy or others), which could do
>> this job. and which migh have less overhead than django.
>>
>> As I am already implementing a web server application with django on
>> another host I wanted to use the same syntax / API for my non web
>> application.
>>
>> Now my question:
>>
>> What would be the minimal setup required to use django orms and nothing
>> else.
>>
>>
>> What entries could I remove from settings.py
>> would I still have to add INSATLLED_APPS to the settings or could I just
>> write one script
>>
>> defining only defining settings.DATABASES, and the typical contents of
>> models.py of an application.
>>
>>
>> Thanks in advance  for some suggestions or ideas how you would approach
>> writing a tiny non web application with django.db.models.Models
>>
> I made a very first brute force test:
> 
> settings.py: (only DATABASES is set)
> ===
> import os
> MYDIR = os.path.abspath(os.path.dirname(__file__))
> 
> DATABASES = {
> 'default' : {
> 'ENGINE': 'django.db.backends.sqlite3',
> 'NAME': os.path.join(MYDIR, "tiny.db"),
> 'HOST': '',
> 'USER': '',
> 'PASSWORD': '',
> 'PORT': '',
> }
> }
> 
> 
> myapp.py
> ==
> #!/usr/bin/env python
> import os
> 
> # just set the env prior to importing a django module
> os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
> from django.db import models
> 
> print "yes this line is executed"
> 
> # this will fail :-(
> class Mini(models.Model):
> name   = models.CharField(max_length=80)
> 
> 
> 
> 
> ###
> If running myapp.py I get following output:
> 
> yes this line is executed
> Traceback (most recent call last):
>   File "./myapp.py", line 11, in 
> class Mini(models.Model):
>   File
> "/opt/my_python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/base.py",
> line 52, in __new__
> kwargs = {"app_label": model_module.__name__.split('.')[-2]}
> IndexError: list index out of range
> (my_python)n1234@mypc:~/minidjango$
> 
> 
> So I need at least a little more to make my script work.
> 


directory structure is now

settings.py
myapp/__init__.py
myapp/models.py
tst.py

settings.py

import os
MYDIR = os.path.abspath(os.path.dirname(__file__))

DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(MYDIR, "tiny.db"),
'HOST': '',
'USER': '',
'PASSWORD': '',
'PORT': '',
}
}

INSTALLED_APPS = (
'myapp',
)


myapp/models.py

from django.db import models

class Mini(models.Model):
name   = models.CharField(max_length=80)


tst.py
---
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

import myapp.models as mymodels

for name in ["one", "two", "three"]:
mymodels.Mini(name=name).save()

print mymodels.Mini.objects.all().values()




now I can call syncdb with:
django-admin syncdb --settings=settings --pythonpath=`pwd`


and run my test app with

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using django ORM from web browser and from command line apps

[Ian Kelly]

On Tue, Jun 21, 2011 at 6:21 PM, News123  wrote:
> Out of curiousity: Do you know whether the imports would be executed for
> each potential command as soon as I call manage.py or only
> 'on demand'?

Off the top of my head, I don't know.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using only the django ORM (DB access model) and nothing else.

[Roy Smith]

In article <4e012e8d$0$23682$426a3...@news.free.fr>,
 News123  wrote:

> Hi,
> 
> I have a small application running on a host without web server and
> without any need for django except its ORM accessing data bases without
> explicitely writing sql queries.)

You would do much better to ask this question on the django mailing list 
(http://groups.google.com/group/django-users).


> I assume there's many libraries (SQL Alchemy or others), which could do
> this job. and which migh have less overhead than django.

Ugh.  I've played with SQL Alchemy a few times and every time I've run 
away screaming in the other direction.  I can see how it's useful if you 
need to be totally cross-platform, but, man, if that's what it takes to 
be cross platform, I'm happy being a MySQL bigot all day long.

> As I am already implementing a web server application with django on
> another host I wanted to use the same syntax / API for my non web
> application.
> 
> Now my question:
> 
> What would be the minimal setup required to use django orms and nothing
> else.

I don't see any reason you couldn't use the Model layer by itself, if 
you want to.  It pretty much stands on its own.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using only the django ORM (DB access model) and nothing else.

[Ian Kelly]

On Tue, Jun 21, 2011 at 6:42 PM, News123  wrote:
> ###
> If running myapp.py I get following output:
>
> yes this line is executed
> Traceback (most recent call last):
>  File "./myapp.py", line 11, in 
>    class Mini(models.Model):
>  File
> "/opt/my_python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/base.py",
> line 52, in __new__
>    kwargs = {"app_label": model_module.__name__.split('.')[-2]}
> IndexError: list index out of range
> (my_python)n1234@mypc:~/minidjango$
>
>
> So I need at least a little more to make my script work.

There's a bit of magic in the way Django finds things, and I think
you'll still need to keep the basic structure of a Django project --
models should be in a "models.py" file located in an "app" package,
which should be included in the INSTALLED_APPS setting.  You just
won't have any views or urlconfs or templates or admin sites or
anything like that.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is the mailing list to usenet gateway borked?

[Ben Finney]

Steven D'Aprano  writes:

> The last couple of messages on this list  show
> up fine on the mailman archives, but are empty posts on
> comp.lang.python. Is there a problem with the mail -> usenet gateway?

I don't see empty messages through Usenet.

However, at around the time you reported this, I started seeing some
(not many) messages each day with strange mangling of the header: the
Date field showing Unix epoch, the Subject field truncated, and the From
field showing the remainder from the Subject.

Another distortion is that the summary of message size (which I think is
derived from the Lines field) shows zero for all those problematic
messages. That may indicate a link between the problems we're seeing.

For example:

=
O  [20110621T054352:  2.0k: deathweaselx86  ]─── Is there any advantage or 
disadvantage to using sets over list comps to ensure a list
O  [20110621T111058:  2.8k: Steven D'Aprano ]├─► [...]
O  [20110621T135956:  1.9k: rusi]└─► [...]
O  [19700101T10:  0.0k: comps to ensure a li]*── Re: Is there any advantage 
or disadvantage to using sets over list
O  <19700101T10:  0.0k: comps to ensure a li>  └─► [...]
O  [20110618T161430:  3.5k: Gregory Ewing   ]─── ANN: PyGUI 2.5.1
=

The problematic messages in the above example are Message-ID:
 and
. The messages, when
retrieved, have the right content and even the fields in the header
appear fine.

My news service for ‘comp.lang.python’ is currently provided from
‘astraweb.com’.

-- 
 \ “I'm a born-again atheist.” —Gore Vidal |
  `\   |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Security test of embedded Python

[Chris Angelico]

I'm involved in the construction of an environment that allows end
users to supply scripts which will then run on our servers. We need to
be able to offer the full flexibility of a scripting language, but
without the risk of compromise to our computers. To that end, we have
set up a system with pretty much the same facilities as our live
system will have, and are offering this to the world to hammer on -
and requesting the world's assistance in hunting down bugs.

The environment is Python 3.3a0 embedded in C++, running on Linux.
It's currently home-hosted to keep things simple, with only one port
forwarded to it from our NAT router (so don't bother port scanning,
you aren't looking at Monty).

And yes, that's right. I have no imagination when it comes to names.
Our test box really is called Monty. And to sign up for our forums,
you'll need to prove you're a human by knowing that the name "Python"
goes with "Monty".

Launch page: http://www.pythontest.com/
PHPBB forum: http://www.pythontest.com/forum/
  (feedback here please, no need to clutter the python-list)
Actual thing to whump into submission: http://www.pythontest.com:8000/

Find a bug, get noted as a contributor! :)

Thanks!

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using only the django ORM (DB access model) and nothing else.

[News123]

On 06/22/2011 01:51 AM, News123 wrote:
> Hi,
> 
> I have a small application running on a host without web server and
> without any need for django except its ORM accessing data bases without
> explicitely writing sql queries.)
> 
> I assume there's many libraries (SQL Alchemy or others), which could do
> this job. and which migh have less overhead than django.
> 
> As I am already implementing a web server application with django on
> another host I wanted to use the same syntax / API for my non web
> application.
> 
> Now my question:
> 
> What would be the minimal setup required to use django orms and nothing
> else.
> 
> 
> What entries could I remove from settings.py
> would I still have to add INSATLLED_APPS to the settings or could I just
> write one script
> 
> defining only defining settings.DATABASES, and the typical contents of
> models.py of an application.
> 
> 
> Thanks in advance  for some suggestions or ideas how you would approach
> writing a tiny non web application with django.db.models.Models
> 
I made a very first brute force test:

settings.py: (only DATABASES is set)
===
import os
MYDIR = os.path.abspath(os.path.dirname(__file__))

DATABASES = {
'default' : {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(MYDIR, "tiny.db"),
'HOST': '',
'USER': '',
'PASSWORD': '',
'PORT': '',
}
}


myapp.py
==
#!/usr/bin/env python
import os

# just set the env prior to importing a django module
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from django.db import models

print "yes this line is executed"

# this will fail :-(
class Mini(models.Model):
name   = models.CharField(max_length=80)




###
If running myapp.py I get following output:

yes this line is executed
Traceback (most recent call last):
  File "./myapp.py", line 11, in 
class Mini(models.Model):
  File
"/opt/my_python/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/db/models/base.py",
line 52, in __new__
kwargs = {"app_label": model_module.__name__.split('.')[-2]}
IndexError: list index out of range
(my_python)n1234@mypc:~/minidjango$


So I need at least a little more to make my script work.










-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using django ORM from web browser and from command line apps

[News123]

Hi Ian,

On 06/22/2011 02:09 AM, Ian Kelly wrote:
> On Tue, Jun 21, 2011 at 5:39 PM, News123  wrote:
>> I'm having a django browser application.
>>
>> There's certain administrative tasks, that I'd like to perform from the
>> command line (cronjob or manually).

> It sounds like you probably want a custom management command:
> 
> https://docs.djangoproject.com/en/1.3/howto/custom-management-commands/
Didn't know this existed.
Yes this is definitely a way to go. At least for part of the commands,
that I plan to implement.
Out of curiousity: Do you know whether the imports would be executed for
each potential command as soon as I call manage.py or only
'on demand'?

> 
>> I wanted to know whether there are any precautions to take if I do this.
>>
>> The one issue, that I could imagine is that Django (if I understood
>> correctly) performs normally database transaction based an a http request.
> 
> If you have the TransactionMiddleware enabled, yes.  Otherwise the
> default is to commit everything immediately.

Good to know.
> 
>> What would happen if I don't do anythong special in a script.
>> Would the entire runtime of the script be considered a transaction?
> 
> The default here is also to commit everything immediately.  Since
> there is no HTTP request, the TransactionMiddleware does not get
> invoked even if enabled.  For controlled transactions you will need to
> use the commit_on_success or commit_manually decorator /
> context-managers:
> 
> https://docs.djangoproject.com/en/1.3/topics/db/transactions/
> 
Thanks this url is very clear. :-)

I will use the commit model as required for my different tasks
-- 
http://mail.python.org/mailman/listinfo/python-list

Tkinter/scrollbar/canvas question

[agb]

Dear Pythonistas,

I've been trying to write a script that results in a set of widgets in a
scrollable widget. Since tkinter is "the" gui for Python (and about the
only one that I can be sure will not require additional software
installation), I went that route, and wrote the following:

from tkinter import *
class ShowList(Frame):
def __init__(self, root):
Frame.__init__(self, root)
self.grid()
self.draw_widgets()
def draw_widgets(self):
cframe = Frame()
cframe.grid(row=1, sticky=N+S+E+W)
canv = Canvas(cframe)
canv["scrollregion"]=canv.bbox(ALL)
canv.grid(row=0, column=0, sticky=N+S+E+W)
vscroll = Scrollbar(cframe, orient=VERTICAL, command=canv.yview)
hscroll = Scrollbar(cframe, orient=HORIZONTAL, 
command=canv.xview)
vscroll.grid(row=0, column=1, sticky=N+S)
hscroll.grid(row=1, column=0, sticky=E+W)
canv["xscrollcommand"] = hscroll.set
canv["yscrollcommand"] = vscroll.set
aframe = Frame(canv)
canv.create_window(0,0,window=aframe, anchor=N+W)
for i in range(0,100):
Label(aframe, text=str(i), anchor=N+W).grid(row=i, 
column=0)

root  = Tk()
m=ShowList(root)
root.mainloop()


...which is great in that it does display the list of items (in this case, a
bunch of Labels) but not so great in the fact that the vertical scrollbar's
scroll thumb is the list of the entire scroll trough, making skip down/up a
screenful impossible. Both the horizontal scrollbar and the vertical
scrollbar will scroll until the list is off the screen (not the desired
behavior). 

Initially, I thought the canv["scrollregion"] = canv.bbox(ALL) would take
care of the off-screen scrolling by making the scrollable region no larger
than it needs to be. Reading many examples of Python+tkinter code, intended
to demonstrate the way to code scrollbars and canvases, didn't enlighten me
enough to figure out the bug(s) in my code. Any suggestions as to what I
did wrong?

Many thanks in advance,

--agb

--
http://gall.mine.nu
free to get comics,free chat-1,dc++ (dcplusplus),
mute webcache,group update program,torrent,atomic time server,tool to find your 
ip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using django ORM from web browser and from command line apps

[Ian Kelly]

On Tue, Jun 21, 2011 at 5:39 PM, News123  wrote:
> Hi,
>
> I'm having a django browser application.
>
> There's certain administrative tasks, that I'd like to perform from the
> command line (cronjob or manually).
> As these scripts might be huge and might consume quite some memory I'd
> prefer, that they were not part of the normal application and would just
> consume memory during the administrative phase.
>
> I am not sure, whether this is a use case really being intended.

It sounds like you probably want a custom management command:

https://docs.djangoproject.com/en/1.3/howto/custom-management-commands/

> I wanted to know whether there are any precautions to take if I do this.
>
> The one issue, that I could imagine is that Django (if I understood
> correctly) performs normally database transaction based an a http request.

If you have the TransactionMiddleware enabled, yes.  Otherwise the
default is to commit everything immediately.

> What would happen if I don't do anythong special in a script.
> Would the entire runtime of the script be considered a transaction?

The default here is also to commit everything immediately.  Since
there is no HTTP request, the TransactionMiddleware does not get
invoked even if enabled.  For controlled transactions you will need to
use the commit_on_success or commit_manually decorator /
context-managers:

https://docs.djangoproject.com/en/1.3/topics/db/transactions/

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Paul Rubin]

John Salerno  writes:
> However, even after reading the Wikipedia page about prime numbers and
> trial division, I'm still a little unclear as to why the square root
> of the number is the upper bound of the range you need to check.

Suppose p is the smallest divisor of n, and p > sqrt(n).
("Divisor" here means p=1 doesn't count).

p being a divisor of n means there is some q such that n = pq.
That means q = n / p.

If p > sqrt(n) that means that q must be < sqrt(n).  But that
contradicts the claim that p is the smallest divisor.

So we know that if there is a divisor at all, it must be <= sqrt(n)
and if we don't find one by then, n must be prime.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Paul Rubin]

Terry Reedy  writes:
>> efficient implementation will allow a solution to be obtained on a
>> modestly powered computer in less than one minute."
> If something really takes a minute in C,
> allow yourself at least 10 minutes or even more with plain CPython.

No.  The idea of the Euler problems is to think up sane algorithms to
solve them, not micro-optimize or use low level languages on crappy
algorithms.

 n=600851475143
 for d in xrange(2,n):
   if d*d > n: break
  while n%d == 0: n //= d
 print n

finishes on my laptop with no noticable pause.  The trick is to stop
testing once you hit the square root of the number.  There is at least
one extremely obvious optimization I didn't bother with (above 2, only
test odd divisors), that would have doubled the speed on top of that.
-- 
http://mail.python.org/mailman/listinfo/python-list

using only the django ORM (DB access model) and nothing else.

[News123]

Hi,

I have a small application running on a host without web server and
without any need for django except its ORM accessing data bases without
explicitely writing sql queries.)

I assume there's many libraries (SQL Alchemy or others), which could do
this job. and which migh have less overhead than django.

As I am already implementing a web server application with django on
another host I wanted to use the same syntax / API for my non web
application.

Now my question:

What would be the minimal setup required to use django orms and nothing
else.


What entries could I remove from settings.py
would I still have to add INSATLLED_APPS to the settings or could I just
write one script

defining only defining settings.DATABASES, and the typical contents of
models.py of an application.


Thanks in advance  for some suggestions or ideas how you would approach
writing a tiny non web application with django.db.models.Models

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Vlastimil Brom]

2011/6/21 John Salerno :
> However, even after reading the Wikipedia page about prime numbers and
> trial division, I'm still a little unclear as to why the square root
> of the number is the upper bound of the range you need to check.
> --

There are likely be some more elaborated proofs, but it seems
sufficiently evident, that with the factors being the square root you
get some kind of "middle position"; with other factors (e.g. two for
simplicity), one of them could be greater, while the another has to be
smaller; this smaller one would have been discovered already, while
searching continuously until the square root of the given number.

vbr
-- 
http://mail.python.org/mailman/listinfo/python-list

Using django ORM from web browser and from command line apps

[News123]

Hi,

I'm having a django browser application.

There's certain administrative tasks, that I'd like to perform from the
command line (cronjob or manually).
As these scripts might be huge and might consume quite some memory I'd
prefer, that they were not part of the normal application and would just
consume memory during the administrative phase.

I am not sure, whether this is a use case really being intended.

I wanted to know whether there are any precautions to take if I do this.

The one issue, that I could imagine is that Django (if I understood
correctly) performs normally database transaction based an a http request.

What would happen if I don't do anythong special in a script.
Would the entire runtime of the script be considered a transaction?

Are there any special commands that I have to use in order to indicate
when a transaction starts / stops?


Thanks in advance for suggestions.

Please look also at  a related question.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Terry Reedy]

On 6/21/2011 3:48 PM, John Salerno wrote:


Absolutely not! Each problem has been designed according to a "one-
minute rule", which means that although it may take several hours to
design a successful algorithm with more difficult problems, an
efficient implementation will allow a solution to be obtained on a
modestly powered computer in less than one minute."


That statement is for C, not Python. Python is efficient with human 
time, but not machine time. If something really takes a minute in C, 
allow yourself at least 10 minutes or even more with plain CPython.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Finding greatest prime factor, was Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Chris Angelico]

Oops, realized after posting that there's a bug in my code - it
returns 1 for a perfect square. Need another check in the 'while'
loop, thus:

On Wed, Jun 22, 2011 at 8:59 AM, Chris Angelico  wrote:
> exec 600851475143; for (int i=2;ii) ret/=i
>
>  while not ret%i and ret>i:

Definitely room for improvement here!

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new string-formatting preferred? (was "What is this syntax ?")

[Tim Chase]

On 06/21/2011 05:19 PM, Terry Reedy wrote:

On 6/21/2011 7:33 AM, Tim Chase wrote:

http://docs.python.org/library/stdtypes.html#str.format>


Is there a good link to a thread-archive on when/why/how .format(...)
became "preferred to the % formatting"?


That is a controversial statement.


I'm not sure whether you're "controversial" refers to

- the documentation at that link,


I meant the preceding statement (derived from the linked source, but
that is not important) that .format is preferred to %.


I guess then with all the contention, having such a vocal 
preference in the docs (even if tempered by "in new code") seems 
to unnecessarily polarize when I find myself very "meh" in either 
direction.



multi_warn = '''\
Warning: testing multiple {0}s against an iterator will only test
the first {0} unless the iterator is reiterable; most are not.'''.format
...
print(multiwarn('function'))
...
print(multiwarn('iterator'))


Does the gotcha of a non-restarting iterator


Huh? What iterator?


Your string-body text warns about behavior regarding iterators. 
I was thrown by your warning.



The other new feature I saw was the use of __format__()


Suppose one had a Money class with currency and decimal amount fields.
.__str__ can add a currency symbol (before or after as appropriate) but
has to use a standard format for the amount field. .__float__ can be
post-processed according to a %...f spec, but cannot include a currency
symbol. Money.__format__(self,spec) can format the amount at it wishes,
including its rounding rules, *and* add a currency symbol.


A Money class was one of the first things I thought of, but 
figured that would be better relegated to an i18n wrapper if you 
wanted it.  Such a wrapper would handle currency-symbol choice & 
positioning positioning (before vs. after; relation to the +/-; 
optional characters for thousands-separators and decimal 
separators; and partitioning at thousands-or-other-multiples, etc).



Or suppose one has a multi-precision float. %80.40f will require an mpf
instance to appoximate itself as a float, possibly with error.
mpg.__format__ should be able to do better.


This case makes a better argument, showing me some new value 
added by __format__().


-tkc



--
http://mail.python.org/mailman/listinfo/python-list

Finding greatest prime factor, was Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Chris Angelico]

On Wed, Jun 22, 2011 at 7:48 AM, John Salerno  wrote:
> Thanks for the all the advice everyone. Now I'm on to problem #4, and
> I'm stumped again, but that's what's fun! :)

So now that you've solved it, I'd like to see some fast one-liners to
do the job. (Since Python cares about whitespace, it might not
_technically_ fit on one line, but in the spirit of one-liners, try to
keep it short.)

Here's what I did in Pike, very quickly:
exec 600851475143; for (int i=2;ihttp://mail.python.org/mailman/listinfo/python-list

Re: something about performence

[Terry Reedy]

On 6/20/2011 10:59 PM, king6c...@gmail.com wrote:
> Hi，
> I have two large files,each has more than 2 lines,and each line 
> consists of two fields,one is the id and the other a value,
> the ids are sorted.
> 
> for example:
> 
> file1
> (uin_a y)
> 1 1245
> 2 12333
> 3 324543
> 5 3464565
> 
> 
> 
> file2
> (uin_b gift)
> 1 34545
> 3 6436466
> 4 35345646
> 5 463626
> 
> 
> I want to merge them and get a file,the lines of which consists of an id 
> and the sum of the two values in file1 and file2。
> the codes are as below:

One minor thing you can do is use bound methods
> 
> uin_y=open('file1')
> uin_gift=open(file2')

ynext = open('file1').next
gnext = open(file1').next

> y_line=uin_y.next()
> gift_line=uin_gift.next()

y_line = ynext()
gift_list = gnext()

and similarly for all .next appearances in what follows.

> while 1:
> try:
> uin_a,y=[int(i) for i in y_line.split()]

This creates an unnecessary second temporary  list. Unroll the loop.

pair = y_line.split
uin_a = int(pair[0])
y = int(pair[1])

> uin_b,gift=[int(i) for i in gift_line.split()]

same for this line

> if uin_a==uin_b:
> score=y+gift
> print uin_a,score
> y_line=uin_y.next()
> gift_line=uin_gift.next()
> if uin_a print uin_a,y
> y_line=uin_y.next()
> if uin_a>uin_b:
> print uin_b,gift
> gift_line=uin_gift.next()
> except StopIteration:
> break



-- 
Terry Jan Reedy


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new string-formatting preferred? (was "What is this syntax ?")

[Terry Reedy]

On 6/21/2011 7:33 AM, Tim Chase wrote:

On 06/20/2011 09:17 PM, Terry Reedy wrote:

On 6/20/2011 8:46 PM, Tim Chase wrote:

On 06/20/2011 05:19 PM, Ben Finney wrote:

“This method of string formatting is the new standard in
Python 3.0, and should be preferred to the % formatting
described in String Formatting Operations in new code.”

http://docs.python.org/library/stdtypes.html#str.format>


Is there a good link to a thread-archive on when/why/how .format(...)
became "preferred to the % formatting"?


That is a controversial statement.


I'm not sure whether you're "controversial" refers to

- the documentation at that link,
- Ben's quote of the documentation at that link,
- my quotation of Ben's quote of the documentation,
- or my request for a "thread-archive on the when/why/how"

I _suspect_ you mean the first one :)


I meant the preceding statement (derived from the linked source, but 
that is not important) that .format is preferred to %. Guido prefers it. 
I prefer it. At least a couple of developers vocally do not prefer it 
and might prefer that the statement was not there. Guido recognizes that 
deprecation of % formatting would at least require a conversion function 
that does not now exist.


I see that the linked doc says 'in new code'. That makes the statement 
less (but only less) controversial.





I haven't seen any great wins of the new formatting over
the classic style.


It does not abuse the '%' operator,


Weighed against the inertia of existing code/documentation/tutorials, I
consider this a toss-up. If .format() had been the preferred way since
day#1, I'd grouse about adding/overloading '%', but going the other
direction, there's such a large corpus of stuff using '%', the addition
of .format() feels a bit schizophrenic.


it does not make a special case of tuples (a source of bugs),


Having been stung occasionaly by this, I can see the benefit here over
writing the less-blatant

"whatever %s" % (tupleish,)


and it is more flexible, especially
indicating objects to be printed. Here is a simple example from my code
that would be a bit more difficult with %.

multi_warn = '''\
Warning: testing multiple {0}s against an iterator will only test
the first {0} unless the iterator is reiterable; most are not.'''.format
...
print(multiwarn('function'))
...
print(multiwarn('iterator'))


Does the gotcha of a non-restarting iterator


Huh? What iterator?

> trump pulling each field  you want and passing it explicitly?

Huh? I explicitly pass the strings to be printed.


In pre-.format(), I'd just use dictionary formatting:

"we have %(food)s & eggs and %(food)s, bacon & eggs" % {
"food": "spam", # or my_iterator.next()?
}


A better parallel to my example would be

menu = "We have %(meat)s & eggs or %(meat)s and potatoes."
print(menu % {'meat':'spam'})
print(menu % {'meat':'ham'})

The exact duplicate of that with .format is

menu = "We have {meat} & eggs or {meat} and potatoes.".format
print(menu(meat = 'spam'))
print(menu(meat = 'ham'))

One knock against '.format' is that it is 6 chars more that '%'. But for 
repeat usage, it is only needed once. And look: '%(meat)s' is 2 more 
chars than '{meat}' and, to me, {} is easier to type than (). Then " % 
{'meat':"spam"}" is 3 more chars than "(meat = 'ham')" and definitely 
harder to type. While I prefer '}' to ')', I prefer '))' to the mixed 
'})'. The % way is at least 'a bit more difficult' even compared to the 
longer and harder .format with named fields.


menu = "We have {0} & eggs or {0} and potatoes.".format
print(menu('spam'))
print(menu('ham'))

it a little easier yet, though perhaps less clear, especially if there 
were multiple substitutions.



The other new feature I saw was the use of __format__() which may have
good use-cases, but I don't yet have a good example of when I'd want
per-stringification formatting compared to just doing my desired
formatting in __str__() instead.


__str__ always returns the same string for an instance in a given state.
Similarly, __float__ and __int__ will return the same float or int 
version of an unchanged instance. __format__(spec) can directly adjust 
the result according to spec without the restriction of going through an 
intermediary str, int, or float.


Suppose one had a Money class with currency and decimal amount fields. 
.__str__ can add a currency symbol (before or after as appropriate) but 
has to use a standard format for the amount field. .__float__ can be 
post-processed according to a %...f spec, but cannot include a currency 
symbol. Money.__format__(self,spec) can format the amount at it wishes, 
including its rounding rules, *and* add a currency symbol.


Or suppose one has a multi-precision float. %80.40f will require an mpf 
instance to appoximate itself as a float, possibly with error. 
mpg.__format__ should be able to do better.


(Sadly, this new ability to more accurately represent objects is not yet 
used for ints and is broken for fractions.Fraction. I will probably 

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[John Salerno]

On Jun 21, 4:41 pm, Ian Kelly  wrote:
> On Tue, Jun 21, 2011 at 3:09 PM, John Salerno  wrote:
> > Don't worry, I was still unclear about what to do after reading all
> > the responses, even yours! But one thing that made me feel better was
> > that I wasn't having a Python problem as much as a *math* problem. I
> > changed my get_factors function to only go as far as the square root
> > of the number in question, and that yielded an answer immediately. :)
>
> > However, even after reading the Wikipedia page about prime numbers and
> > trial division, I'm still a little unclear as to why the square root
> > of the number is the upper bound of the range you need to check.
>
> Careful, note that the greatest prime factor may actually be greater
> than the square root.  It's just that it's possible to find it without
> iterating past the square root.  This is because for each p that is a
> factor of n, q is also a factor of n, where p * q = n.  If p >
> sqrt(n), then q < sqrt(n).  Therefore you can find p by finding q and
> dividing n / q.

Oh! Now it makes sense! That first sentence helped to put it into
perspective, too. The Wikipedia page says more or less the same thing,
but this paragraph just made more sense to me. :)

Thanks for the all the advice everyone. Now I'm on to problem #4, and
I'm stumped again, but that's what's fun! :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: those darn exceptions

[Chris Torek]

>On Tue, 21 Jun 2011 01:43:39 +, Chris Torek wrote:
>> But how can I know a priori
>> that os.kill() could raise OverflowError in the first place?

In article <4e006912$0$29982$c3e8da3$54964...@news.astraweb.com>
Steven D'Aprano   wrote:
>You can't. Even if you studied the source code, you couldn't be sure that 
>it won't change in the future. Or that somebody will monkey-patch 
>os.kill, or a dependency, introducing a new exception.

Indeed.  However, if functions that "know" which exceptions they
themselves can raise "declare" this (through an __exceptions__
attribute for instance), then whoever changes the source or
monkey-patches os.kill can also make the appropriate change to
os.kill.__exceptions__.

>More importantly though, most functions are reliant on their argument. 
>You *cannot* tell what exceptions len(x) will raise, because that depends 
>on what type(x).__len__ does -- and that could be anything. So, in 
>principle, any function could raise any exception.

Yes; this is exactly why you need a type-inference engine to make
this work.  In this case, len() is more (though not quite exactly)
like the following user-defined function:

def len2(x):
try:
fn = x.__len__
except AttributeError:
raise TypeError("object of type %r has no len()" % type(x))
return fn()

eg:

>>> len(3)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: object of type 'int' has no len()
>>> len2(3)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 5, in len2
TypeError: object of type  has no len()

In this case, len would not have any __exceptions__ field (or if
it does, it would not be a one-element tuple, but I currently think
it makes more sense for many of the built-ins to resort to rules
in the inference engine).  This is also the case for most operators,
e.g., ordinary "+" (or operator.add) is syntactic sugar for:

first_operand.__add__(second_operand)

or:

second_operand.__radd__(first_operand)

depending on both operands' types and the first operand's __add__.

The general case is clearly unsolveable (being isomorphic to the
halting problem), but this is not in itself an excuse for attempting
to solve more-specific cases.  A better excuse -- which may well
be "better enough" :-) -- occurs when the specific cases that *can*
be solved are so relatively-rare that the approach degenerates into
uselessness.

It is worth noting that the approach I have in mind does not
survive pickling, which means a very large subset of Python code
is indigestible to a pylint-like exception-inference engine.

>Another question -- is the list of exceptions part of the function's 
>official API? *All* of the exceptions listed, or only some of them?

All the ones directly-raised.  What to do about "invisible"
dependencies (such as those in len2() if len2 is "invisible",
e.g., coded in C rather than Python) is ... less obvious. :-)

>In general, you can't do this at compile-time, only at runtime. There's 
>no point inspecting len.__exceptions__ at compile-time if len is a 
>different function at runtime.

Right.  Which is why pylint is fallible ... yet pylint is still
valuable.  At least, I find it so.  It misses a lot of important
things -- it loses types across list operations, for instance --
but it catches enough to help.  Here is a made-up example based on
actual errors I have found via pylint:

"doc"
class Frob(object):
"doc"
def __init__(self, arg1, arg2):
self.arg1 = arg1
self.arg2 = arg2

def frob(self, nicate):
"frobnicate the frob"
self.arg1 += nicate

def quux(self):
"return the frobnicated value"
example = self # demonstrate that pylint is not using the *name*
return example.argl # typo, meant arg1
...

$ pylint frob.py
* Module frob
E1101: 15:Frob.quux: Instance of 'Frob' has no 'argl' member

("Loses types across list operations" means that, e.g.:

def quux(self):
return [self][0].argl

hides the type, and hence the typo, from pylint.  At some point I
intend to go in and modify it to track the element-types of list
elements: in "enough" cases, a list's elements all have the same
type, which means we can predict the type of list[i].  If a list
contains mixed types, of course, we have to fall back to the
failure-to-infer case.)

(This also shows that much real code might raise IndexError: any
list subscript that is out of range does so.  So a lot of real
functions *might* raise IndexError, etc., which is another argument
that "in real code, an exception inference engine will wind up
concluding that every line might raise every exception".  Which
might be true, but I still believe, for the moment, that a tool
for inferring exceptions would have some value.)
-- 
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake 

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Ian Kelly]

On Tue, Jun 21, 2011 at 3:09 PM, John Salerno  wrote:
> Don't worry, I was still unclear about what to do after reading all
> the responses, even yours! But one thing that made me feel better was
> that I wasn't having a Python problem as much as a *math* problem. I
> changed my get_factors function to only go as far as the square root
> of the number in question, and that yielded an answer immediately. :)
>
> However, even after reading the Wikipedia page about prime numbers and
> trial division, I'm still a little unclear as to why the square root
> of the number is the upper bound of the range you need to check.

Careful, note that the greatest prime factor may actually be greater
than the square root.  It's just that it's possible to find it without
iterating past the square root.  This is because for each p that is a
factor of n, q is also a factor of n, where p * q = n.  If p >
sqrt(n), then q < sqrt(n).  Therefore you can find p by finding q and
dividing n / q.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Irmen de Jong]

On 21-6-2011 23:09, John Salerno wrote:
> On Jun 21, 3:22 pm, Irmen de Jong  wrote:
>> On 21-06-11 22:10, Irmen de Jong wrote:
>> [stuff]
>>
>> I didn't read the last paragraph of John's message until just now, and
>> now realize that what I wrote is likely way too much information for
>> what he asked.
>> I'm sorry. Next time I'll read everything until and including the last
>> full stop.
>>
>> Irmen
> 
> Don't worry, I was still unclear about what to do after reading all
> the responses, even yours! But one thing that made me feel better was
> that I wasn't having a Python problem as much as a *math* problem. I
> changed my get_factors function to only go as far as the square root
> of the number in question, and that yielded an answer immediately. :)

Ok, cool :)


> However, even after reading the Wikipedia page about prime numbers and
> trial division, I'm still a little unclear as to why the square root
> of the number is the upper bound of the range you need to check.

If there is an exact divisor d >= √n, then the quotient n/d is also a divisor 
of n, and
that quotient is <= √n. So if we don't find such a quotient before reaching √n, 
it's not
useful to search for d > √n (because no divisors would be found).

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Handling import errors

[Mel]

Guillaume Martel-Genest wrote:

> What is the pythonic way to handle imports error? What is bugging me
> is that the imports can't be inside a function (because I use them in
> different places in the script and thus they have to be in the global
> scope).

Actually, you can if you declare them global:

Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
... global os
... import os
... 
>>> dir (os)
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'os' is not defined
>>> f()
>>> dir (os)
['EX_CANTCREAT', 'EX_CONFIG', 'EX_DATAERR', 'EX_IOERR', 'EX_NOHOST', 
'EX_NOINPUT', 'EX_NOPERM', 'EX_NOUSER', 'EX_OK', 'EX_OSERR', 'EX_OSFILE', 
'EX_PROTOCOL', 'EX_SOFTWARE', 'EX_TEMPFAIL', 'EX_UNAVAILABLE', 'EX_USAGE', 
'F_OK', 'NGROUPS_MAX', 'O_APPEND', 'O_ASYNC', 'O_CREAT', 'O_DIRECT', 
'O_DIRECTORY', 'O_DSYNC', 'O_EXCL', 'O_LARGEFILE', 'O_NDELAY', 'O_NOATIME', 
'O_NOCTTY', 'O_NOFOLLOW', 'O_NONBLOCK

etc.

Mel.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Handling import errors

[Miki Tebeka]

> try:
> import foo
> except ImportError:
> logging.error('could not import foo')
> sys.exit(1)
Why not just let the exception terminate the program? It will have even more 
information about the problem that caused foo not to load.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Handling import errors

[Tim Johnson]

* Guillaume Martel-Genest  [110621 12:53]:
> What is the pythonic way to handle imports error? What is bugging me
> is that the imports can't be inside a function (because I use them in
> different places in the script and thus they have to be in the global
> scope). I would write something like:
  Suppose you do something like this:
  try :
mod = __import__('mymodulename')
  except ImportError:
pass ## replace with error handling here 

  so `mod' is global if you execute the above code as top-level,
  *but* you can also pass `mod' as an argument like any other
  variable, as far as I know and far as I have done.

-- 
Tim 
tim at johnsons-web dot com or akwebsoft dot com
http://www.akwebsoft.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[John Salerno]

On Jun 21, 3:22 pm, Irmen de Jong  wrote:
> On 21-06-11 22:10, Irmen de Jong wrote:
> [stuff]
>
> I didn't read the last paragraph of John's message until just now, and
> now realize that what I wrote is likely way too much information for
> what he asked.
> I'm sorry. Next time I'll read everything until and including the last
> full stop.
>
> Irmen

Don't worry, I was still unclear about what to do after reading all
the responses, even yours! But one thing that made me feel better was
that I wasn't having a Python problem as much as a *math* problem. I
changed my get_factors function to only go as far as the square root
of the number in question, and that yielded an answer immediately. :)

However, even after reading the Wikipedia page about prime numbers and
trial division, I'm still a little unclear as to why the square root
of the number is the upper bound of the range you need to check.
-- 
http://mail.python.org/mailman/listinfo/python-list

Handling import errors

[Guillaume Martel-Genest]

What is the pythonic way to handle imports error? What is bugging me
is that the imports can't be inside a function (because I use them in
different places in the script and thus they have to be in the global
scope). I would write something like:

try:
import foo
except ImportError:
logging.error('could not import foo')
sys.exit(1)

But logging is not configured at this point as my main() have not been
called yet.

Should I define a global variable and assign it to my module later? Or
should I let the exception happen and let the stack trace be the error
message?
-- 
http://mail.python.org/mailman/listinfo/python-list

Handling import errors

[Guillaume Martel-Genest]

What is the pythonic way to handle imports error? What is bugging me
is that the imports can't be inside a function (because I use them in
different places in the script and thus they have to be in the global
scope). I would write something like:

try:
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: running an existing script

[Adam Chapman]

On Jun 21, 8:00 pm, Ethan Furman  wrote:
> Adam Chapman wrote:
> > Thanks Ethan
>
> > No way could I have worked that out in my state of stress!
>
> > For your second idea, would I need to type that into the python command
> > line interface (the one that looks like a DOS window?
>
> If you are actually in a python CLI, at the top of that screen does it
> say something like
>
> Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit
> (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>
> ?
>
> If yes, then what I wrote earlier should actually work (I downloaded
> jBoost and looked at the nfold.py script).  Here it is again:
>
> --> import os
> --> os.chdir('path/to/nfold.py') # don't include nfold.py  ;)
> --> import nfold
> --> import sys
> --> sys.argv = ["nfold.py", "--folds=5", "--data=spambase.data",
> ... "--spec=spambase.spec", "--rounds=500", "--tree=ADD_ALL",
> ... "--generate" ]
> ...
> --> nfold.main()
>
> I fixed the sys.argv line from last time.
>
> Good luck!
>
> ~Ethan~

Thanks to both of you for your help.

It's getting late here, I'll give it another try tomorrow
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[MRAB]

On 21/06/2011 20:48, John Salerno wrote:

I'm working on the Project Euler exercises and I'm stumped on problem
3:

"What is the largest prime factor of the number 600851475143 ?"

Now, I've actually written functions to get a list of the factors of
any given number, and then another function to get the prime numbers
from that list. It works fine with small numbers, but when I try to
feed my get_factors function with the above number (600 billion),
naturally it takes forever! But according to the Project Euler
website:

"I've written my program but should it take days to get to the answer?

Absolutely not! Each problem has been designed according to a "one-
minute rule", which means that although it may take several hours to
design a successful algorithm with more difficult problems, an
efficient implementation will allow a solution to be obtained on a
modestly powered computer in less than one minute."

But it definitely takes more than a minute, and I still haven't gotten
it to end yet without just canceling it myself.


[snip]
A non-prime is the product of a prime and another number which may or
may not be a prime. Look for the smallest prime and repeat.

On a modern PC, if it takes more than, say, a second for the given
number, you're doing it wrong. :-)
--
http://mail.python.org/mailman/listinfo/python-list

sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

[Irmen de Jong]

On 21-06-11 22:10, Irmen de Jong wrote:
[stuff]

I didn't read the last paragraph of John's message until just now, and 
now realize that what I wrote is likely way too much information for 
what he asked.
I'm sorry. Next time I'll read everything until and including the last 
full stop.


Irmen
--
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Mel]

John Salerno wrote:
> I'm working on the Project Euler exercises and I'm stumped on problem
> 3:
> 
> "What is the largest prime factor of the number 600851475143 ?"
[ ... ]
> Here is what I have so far. Initially the get_factors function just
> iterated over the entire range(2, n + 1), but since a number can't
> have a factor greater than half of itself, I tried to shorten the
> range by doing range(2, n //2), but that still leaves 300 billion
> numbers to go through.
> 
> def get_factors(number):
> factors = [number]
> 
> for n in range(2, number // 2):
> if number % n == 0:
> factors.append(n)
> 
> return factors
> 
> 
> def get_primes(number_list):
> primes = number_list[:]
> 
> for n in number_list:
> for x in range(2, n):
> if n % x == 0:
> primes.remove(n)
> break
> 
> return primes
> 
> 
> print(max(get_primes(get_factors(600851475143
> 
> 
> Also, I want to make it clear that I DO NOT WANT THE ANSWER. I really
> want to solve this myself, but the reason I'm asking about it is to
> see if there really is some way to change this code so that it can get
> an answer in less than one minute, as the website says should be
> possible. A hint about what I need to do would be nice, but not an
> answer. I just don't see any way to get the factors without iterating
> over the entire range of values, though.

It certainly can be done faster.  I ran it against the factor finder that I 
wrote, and it popped up the answer

mwilson@tecumseth:~$ bin/factors.py 600851475143
71 839 1471 ...

before I could glance at my watch.  factors.py works, as does yours, by 
testing for small factors first, but it divides them out as it goes, so it 
tends to do its work on smallish numbers.  And since the smallest factors 
are taken out as soon as possible, they have to be the prime ones.

Good hunting,   Mel.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to get return values of a forked process

[Chris Torek]

>> On Tue, Jun 21, 2011 at 12:26 PM, Ian  wrote:
>>> myForkedScript has code like this:
>>> if fail:
>>> os._exit(1)
>>> else:
>>>os._exit(os.EX_OK)
>>>
>>> Is using os._exit() the correct way to get a return value back to the
>>> main process?

"The" correct way, no, but it is "a" correct way (and cheaper than
using a pipe to pickle and unpickle failure, the way the subprocess
module does it, for instance).  In any case, you *should* call
os._exit() either directly or indirectly after a successful fork
but a failed exec.

>On Jun 21, 1:54 pm, Ian Kelly  wrote:
>> sys.exit() is the preferred way.

Using sys.exit() after a fork() has other risks (principally,
duplication of pending output when flushing write-mode streams),
which is why os._exit() is provided.

>>> I thought the value 'n', passed in os._exit(n) would be the value I
>>> get returned.  In the case of a failure, I get 256 returned rather
>>> than 1.

>> According to the docs ...
   [snip documentation and description]
>> However, I would advise using the subprocess module for this instead
>> of the os module (which is just low-level wrappers around system
>> calls).

Indeed, subprocess gives you convenience, safety, and platform
independence (at least across POSIX-and-Windows) with a relatively
low cost.  As long as the cost is low enough (and it probably is)
I agree with this.

In article 
Ian   wrote:
>Where did you find the Unix docs you pasted in?  I didn't find it in
>the man pages.  Thank you.  Based on what you say, I will change my
>os._exit() to sys.exit().

Not sure where Ian Kelly's documentation came from, but note that on
Unix, the "os" module also provides os.WIFSIGNALED, os.WTERMSIG,
os.WIFEXITED, and os.WEXITSTATUS for dissecting the "status"
integer returned from the various os.wait* calls.

Again, if you use the subprocess module, you are insulated from
this sort of detail (which, as always, has both advantages and
disadvantages).
-- 
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W)  +1 801 277 2603
email: gmail (figure it out)  http://web.torek.net/torek/index.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Irmen de Jong]

On 21-06-11 21:48, John Salerno wrote:

I'm working on the Project Euler exercises and I'm stumped on problem
3:

"What is the largest prime factor of the number 600851475143 ?"

Now, I've actually written functions to get a list of the factors of
any given number, and then another function to get the prime numbers
from that list. It works fine with small numbers, but when I try to
feed my get_factors function with the above number (600 billion),
naturally it takes forever!


You need a better algorithm to calculate primes, and iterate over primes 
instead of over the full (half, or even better, sqrt(n)) range of 
possible values. You also should optimize the stop condition, once you 
find that the number can no longer be divided by larger primes you can 
stop the loop.


For instance to get the prime factors of the number 1000 you'd iterate
over the prime numbers 2,3,5 and conclude that

1000=2*2*2*5*5*5, so 5 would be the largest prime factor.

No need to try larger primes than 5, let alone go through 1..sqrt(1000).

The Sieve of Eratosthenes is a well known algorithm to calculate primes 
with reasonable efficiency.


Irmen
--
http://mail.python.org/mailman/listinfo/python-list

Re: basic bytecode to machine code compiler (part 3)

[Rouslan Korneychuk]

On 06/21/2011 06:55 AM, Ulrich Eckhardt wrote:

Rouslan Korneychuk wrote:

if i != pindex:
 (less if x<= pivot else greater).append(x)


Just curious, is there a reason why you wrote this last line that way
instead of using a "normal" if/else clause?


Cheers!

Uli




No special reason. I just like concise code.
--
http://mail.python.org/mailman/listinfo/python-list

Re: How can I speed up a script that iterates over a large range (600 billion)?

[Ian Kelly]

On Tue, Jun 21, 2011 at 1:48 PM, John Salerno  wrote:
> Here is what I have so far. Initially the get_factors function just
> iterated over the entire range(2, n + 1), but since a number can't
> have a factor greater than half of itself, I tried to shorten the
> range by doing range(2, n //2), but that still leaves 300 billion
> numbers to go through.

Without giving you the answer, I will note that the range can be
further reduced by quite a lot (I'm talking orders of magnitude, not
just by half).

> def get_primes(number_list):
>    primes = number_list[:]
>
>    for n in number_list:
>        for x in range(2, n):
>            if n % x == 0:
>                primes.remove(n)
>                break
>
>    return primes

Also, primality testing and factorization are very similar problems,
and the same range optimization could be applied here as well.

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

How can I speed up a script that iterates over a large range (600 billion)?

[John Salerno]

I'm working on the Project Euler exercises and I'm stumped on problem
3:

"What is the largest prime factor of the number 600851475143 ?"

Now, I've actually written functions to get a list of the factors of
any given number, and then another function to get the prime numbers
from that list. It works fine with small numbers, but when I try to
feed my get_factors function with the above number (600 billion),
naturally it takes forever! But according to the Project Euler
website:

"I've written my program but should it take days to get to the answer?

Absolutely not! Each problem has been designed according to a "one-
minute rule", which means that although it may take several hours to
design a successful algorithm with more difficult problems, an
efficient implementation will allow a solution to be obtained on a
modestly powered computer in less than one minute."

But it definitely takes more than a minute, and I still haven't gotten
it to end yet without just canceling it myself.

Here is what I have so far. Initially the get_factors function just
iterated over the entire range(2, n + 1), but since a number can't
have a factor greater than half of itself, I tried to shorten the
range by doing range(2, n //2), but that still leaves 300 billion
numbers to go through.

def get_factors(number):
factors = [number]

for n in range(2, number // 2):
if number % n == 0:
factors.append(n)

return factors


def get_primes(number_list):
primes = number_list[:]

for n in number_list:
for x in range(2, n):
if n % x == 0:
primes.remove(n)
break

return primes


print(max(get_primes(get_factors(600851475143


Also, I want to make it clear that I DO NOT WANT THE ANSWER. I really
want to solve this myself, but the reason I'm asking about it is to
see if there really is some way to change this code so that it can get
an answer in less than one minute, as the website says should be
possible. A hint about what I need to do would be nice, but not an
answer. I just don't see any way to get the factors without iterating
over the entire range of values, though.

Thanks!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to get return values of a forked process

[Ian Kelly]

> Where did you find the Unix docs you pasted in?  I didn't find it in
> the man pages.  Thank you.  Based on what you say, I will change my
> os._exit() to sys.exit().

http://docs.python.org/library/os.html#os.wait
http://docs.python.org/library/os.html#os.waitpid

I don't know what man pages you were looking at, but the Unix system
call does work the same way; to extract the exit status in C you're
supposed to use the WEXITSTATUS(status) macro, which is typically
implemented as ((status >> 8) & 0377).  See:

http://linux.die.net/man/2/waitpid
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to get return values of a forked process

[Miki Tebeka]

One way is to use pipes, have a look at 
http://code.activestate.com/recipes/576709/ for example.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: parse date string having "EDT"

[Miki Tebeka]

You might consider trying dateutil.parser.parse 
(http://labix.org/python-dateutil#head-c0e81a473b647dfa787dc11e8c69557ec2c3ecd2)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to get return values of a forked process

[Ian]

On Jun 21, 1:54 pm, Ian Kelly  wrote:
> On Tue, Jun 21, 2011 at 12:26 PM, Ian  wrote:
> > myForkedScript has code like this:
> > if fail:
> >    os._exit(1)
> > else:
> >    os._exit(os.EX_OK)
>
> > Is using os._exit() the correct way to get a return value back to the
> > main process?
>
> sys.exit() is the preferred way.
>
> > I thought the value 'n', passed in os._exit(n) would be the value I
> > get returned.  In the case of a failure, I get 256 returned rather
> > than 1.
>
> According to the docs, on Unix:
>
> """
> Wait for completion of a child process, and return a tuple containing
> its pid and exit status indication: a 16-bit number, whose low byte is
> the signal number that killed the process, and whose high byte is the
> exit status (if the signal number is zero); the high bit of the low
> byte is set if a core file was produced.
> """
>
> And on Windows:
>
> """
> Wait for completion of a process given by process handle pid, and
> return a tuple containing pid, and its exit status shifted left by 8
> bits (shifting makes cross-platform use of the function easier).
> """
>
> (256 >> 8) == 1
>
> However, I would advise using the subprocess module for this instead
> of the os module (which is just low-level wrappers around system
> calls).

Where did you find the Unix docs you pasted in?  I didn't find it in
the man pages.  Thank you.  Based on what you say, I will change my
os._exit() to sys.exit().
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Don't understand SequenceMatcher from difflib

[Terry Reedy]

On 6/21/2011 9:43 AM, Antoon Pardon wrote:


   matcher = SequenceMatcher(ls1, ls2)

...

What am I doing wrong?


Read the doc, in particular, the really stupid signature of the class:

"class difflib.SequenceMatcher(isjunk=None, a='', b='', autojunk=True)"
You are passing isjunk = ls1, a = ls2, and by default, b=''. So there 
are no matches, len(a) = 36, len(b) = 0, and the dummy match is (36,0,0) 
as you got.


There are also several example in the doc, all like
>>> s = SequenceMatcher(None, " abcd", "abcd abcd") # or
>>> s = SequenceMatcher(lambda x: x==" ", " abcd", "abcd abcd")

So you will get better results with
matcher = SequenceMatcher(None, ls1, ls2) # or
matcher = SequenceMatcher(a=ls1, b=ls2)

In the future, please try to simply examples before posting for help.

print(list(SequenceMatcher('a','abc').get_matching_blocks()))

shows the problem you posted in one easily read line of input and 
output. I only waded through the distracting code and output you posted 
to find that problem because I patched SequenceMatcher last fall.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: How to get return values of a forked process

[Ian Kelly]

On Tue, Jun 21, 2011 at 12:26 PM, Ian  wrote:
> myForkedScript has code like this:
> if fail:
>os._exit(1)
> else:
>os._exit(os.EX_OK)
>
> Is using os._exit() the correct way to get a return value back to the
> main process?

sys.exit() is the preferred way.

> I thought the value 'n', passed in os._exit(n) would be the value I
> get returned.  In the case of a failure, I get 256 returned rather
> than 1.

According to the docs, on Unix:

"""
Wait for completion of a child process, and return a tuple containing
its pid and exit status indication: a 16-bit number, whose low byte is
the signal number that killed the process, and whose high byte is the
exit status (if the signal number is zero); the high bit of the low
byte is set if a core file was produced.
"""

And on Windows:

"""
Wait for completion of a process given by process handle pid, and
return a tuple containing pid, and its exit status shifted left by 8
bits (shifting makes cross-platform use of the function easier).
"""

(256 >> 8) == 1

However, I would advise using the subprocess module for this instead
of the os module (which is just low-level wrappers around system
calls).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: running an existing script

[Ethan Furman]

Adam Chapman wrote:

Thanks Ethan

No way could I have worked that out in my state of stress!

For your second idea, would I need to type that into the python command 
line interface (the one that looks like a DOS window?


If you are actually in a python CLI, at the top of that screen does it 
say something like


Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit 
(Intel)] on win32

Type "help", "copyright", "credits" or "license" for more information.

?

If yes, then what I wrote earlier should actually work (I downloaded 
jBoost and looked at the nfold.py script).  Here it is again:


--> import os
--> os.chdir('path/to/nfold.py') # don't include nfold.py  ;)
--> import nfold
--> import sys
--> sys.argv = ["nfold.py", "--folds=5", "--data=spambase.data",
... "--spec=spambase.spec", "--rounds=500", "--tree=ADD_ALL",
... "--generate" ]
...
--> nfold.main()

I fixed the sys.argv line from last time.

Good luck!

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Simple question about loading a .glade UI file

[Anthony Papillion]

Hi Everyone,

So I'm tackling designing a non-CLI program using Glade. I went through some
tutorials and it seems like I'm doing things right but I'm my UI won't load.
Python keeps griping about "could not create glade XML object".

I have a .glade file called MainWindow.glade and my main window is called
(predictably) winMain. Here is the code I'm using to load it:

#!/usr/bin/env python
import sys
try:
import pygtk
pygtk.require("2.0")
except:
pass

try:
import gtk
import gtk.glade
except:
print "GTK could not be loaded."
sys.exit(1)

class GMB:

def __init__(self):
self.gladefile = "MainWindow.glade"
 self.wTree = gtk.glade.XML(self.gladefile)
self.wTree.signal_autoconnect(self)
 self.window = self.wTree.get_widget("winMain")
if(self.window):
self.window.connect("destroy", gtk.main_quit)

def on_winMain_delete(self, widget, dummy):
gtk.main_quit()

if __name__ == "__main__":
myGui = GMB()
gtk.main()

Is there any reason why I'd be getting this error from the code above? Both
the UI file and the source code file are in the same directory.

Thanks!
Anthony
-- 
http://mail.python.org/mailman/listinfo/python-list

How to get return values of a forked process

[Ian]

Hello all,

I need some helped with forking.  In my script, I fork a process.  I
want to get return values from the child process.

This is the script that does the forking:
for x in (mylist):
   pid = os.fork()
   if pid:
 pidList.append(pid)
   else:
os.execv('/usr/bin/python',('/usr/bin/
python',myForkedScript))

for pid in pidList:
childPid, status = os.waitpid(pid,0)
# I think status should be the return value of the forked
process; I would expect status to be a 1 or a 0

myForkedScript has code like this:
if fail:
   os._exit(1)
else:
   os._exit(os.EX_OK)


Is using os._exit() the correct way to get a return value back to the
main process?

I thought the value 'n', passed in os._exit(n) would be the value I
get returned.  In the case of a failure, I get 256 returned rather
than 1.

Thanks for the assistance!
IL
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: running an existing script

[Benjamin Kaplan]

On Tue, Jun 21, 2011 at 10:45 AM, Adam Chapman
 wrote:
> Hi,
>
> I'm trying to put together a lot of pieces of source code in matlab,
> java, perl and python.
>
> Im an expert when it comes to matlab, but novice in all the others
> listed above. However, I have integrated the java and perl code so
> they can be called from matlab.
>
> I know that there is a toolbox out there called Pymat but i think that
> uses 32bit activex so rules my configuration out.
>
> However I think I can hack in to the python command prompt from
> matlab.
>
> Basically I just want to run a single script from the python command
> window. Once I know how to do that I can be off on my way to perform
> the matlab interfacing.
>
> there is an example of the command I need in the python prompt at
> http://jboost.sourceforge.net/doc.html#cv .
>
> however, I can't seem to run the python script by typing the command
> on that link in the python prompt.
>

That command they show isn't run from a Python shell. It's run from
either a Unix shell (bash and friends) or a Windows command prompt
(cmd). If you want to run a script, you have to give the path to that
script. ./ means the current directory and .. is the parent directory
if you want to give relative paths, or you can just write out the
whole file path.

> Can I please ask how to set the current duirectory in python?
>

os.chdir changes the current directory, but you probably don't need to do that.

> the script I want to run is in a different directory to the one python
> is installed to
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Better way to iterate over indices?

[Ethan Furman]

Billy Mays wrote:
I have always found that iterating over the indices of a list/tuple is 
not very clean:


for i in range(len(myList)):
doStuff(i, myList[i])


Definitely not beautiful.  ;)


I know I could use enumerate:

for i, v in enumerate(myList):
doStuff(i, myList[i])


If you actually need the index, then this is the way to do it.  Note 
that in most cases, you don't need the index and can iterate directly:


for v in myList:
doStuff(v)

From your sample code (assuming you don't need i) this does the same thing.

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Better way to iterate over indices?

[Benjamin Kaplan]

On Tue, Jun 21, 2011 at 11:05 AM, Billy Mays  wrote:
> I have always found that iterating over the indices of a list/tuple is not
> very clean:
>
> for i in range(len(myList)):
>    doStuff(i, myList[i])
>
>
>
>
> I know I could use enumerate:
>
> for i, v in enumerate(myList):
>    doStuff(i, myList[i])
>
> ...but that stiff seems clunky.

Why does enumerate seem clunky, other than the fact that you should
probably have
doStuff(i,v)
instead of myList[i] in there? It's a bit more awkward than C's
syntax, but since the typical use case is just iteration anyway, it's
not a huge deal for those few cases where you actually need the
indices.

>
> Are there any better ways to iterate over the indices of a list /tuple?
>
> --Bill
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Better way to iterate over indices?

[Noah Hall]

On Tue, Jun 21, 2011 at 7:05 PM, Billy Mays  wrote:
> I have always found that iterating over the indices of a list/tuple is not
> very clean:
>
> for i in range(len(myList)):
>    doStuff(i, myList[i])

> I know I could use enumerate:
>
> for i, v in enumerate(myList):
>    doStuff(i, myList[i])
>
> ...but that stiff seems clunky.

You're not using it properly. Think about it. You're giving two names
- i and v. You've forgotten about v -

>>> for i, v in enumerate('fish'):
... print i, v
...
0 f
1 i
2 s
3 h


HTH.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: running an existing script

[Ethan Furman]

Adam Chapman wrote:

Hi,


Howdy!


I'm trying to put together a lot of pieces of source code in matlab,
java, perl and python.


[snippety]


Basically I just want to run a single script from the python command
window. Once I know how to do that I can be off on my way to perform
the matlab interfacing.

there is an example of the command I need in the python prompt at
http://jboost.sourceforge.net/doc.html#cv .


That looks like a shell prompt, not a Python prompt


however, I can't seem to run the python script by typing the command
on that link in the python prompt.

Can I please ask how to set the current duirectory in python?


nfold.py is a python script -- you can't just type in the name once 
inside python and have it work.  It would require something like


--> import os
--> os.chdir('path/to/nfold.py') # don't include nfold.py ;)
--> import nfold
--> import sys
--> sys.argv = ["--folds=5", "--data=spambase.data",
... "--spec=spambase.spec", "--rounds=500", "--tree=ADD_ALL",
... "--generate" ]
...
--> nfold.main()  # assuming it has a main function that can be called
  # in this manner

and that probably won't work.  What you probably want to do is execute 
the command "python /path/to/nfold.py --fold=5 ..." (include the 
nfold.py this time ).  I have no idea how to do that from Matlab.


Good luck!

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Better way to iterate over indices?

[Ian Kelly]

On Tue, Jun 21, 2011 at 12:05 PM, Billy Mays  wrote:
> I know I could use enumerate:
>
> for i, v in enumerate(myList):
>    doStuff(i, myList[i])
>
> ...but that stiff seems clunky.

Why not:

for i, v in enumerate(myList):
doStuff(i, v)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: parse date string having "EDT"

[Junaid P V]

Thanks,

My script should be platform independent, so I think filtering out time zone 
info is better.
-- 
http://mail.python.org/mailman/listinfo/python-list

Better way to iterate over indices?

[Billy Mays]

I have always found that iterating over the indices of a list/tuple is 
not very clean:


for i in range(len(myList)):
doStuff(i, myList[i])




I know I could use enumerate:

for i, v in enumerate(myList):
doStuff(i, myList[i])

...but that stiff seems clunky.

Are there any better ways to iterate over the indices of a list /tuple?

--Bill
--
http://mail.python.org/mailman/listinfo/python-list

running an existing script

[Adam Chapman]

Hi,

I'm trying to put together a lot of pieces of source code in matlab,
java, perl and python.

Im an expert when it comes to matlab, but novice in all the others
listed above. However, I have integrated the java and perl code so
they can be called from matlab.

I know that there is a toolbox out there called Pymat but i think that
uses 32bit activex so rules my configuration out.

However I think I can hack in to the python command prompt from
matlab.

Basically I just want to run a single script from the python command
window. Once I know how to do that I can be off on my way to perform
the matlab interfacing.

there is an example of the command I need in the python prompt at
http://jboost.sourceforge.net/doc.html#cv .

however, I can't seem to run the python script by typing the command
on that link in the python prompt.

Can I please ask how to set the current duirectory in python?

the script I want to run is in a different directory to the one python
is installed to
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Do we still need to inherit from "object" to create new-style classes?

[Terry Reedy]

On 6/20/2011 9:26 PM, John Salerno wrote:

I can't quite seem to find the answer to this anywhere. The book I'm
reading right now was written for Python 3.1 and doesn't use (object),
so I'm thinking that was just a way to force new-style classes in 2.x
and is no longer necessary in 3.x. Is that right?

(The documentation doesn't mention object anymore,


Lib ref 2. builtin functions:
"object()
Return a new featureless object. object is a base for all classes. It 
has the methods that are common to all instances of Python classes. This 
function does not accept any arguments.


Note
object does not have a __dict__, so you can’t assign arbitrary 
attributes to an instance of the object class.

"


but elsewhere on
the Python website it says the documentation hasn't been updated for
new-style classes yet, hence my confusion.)


What page? *It* may need updating ;-).

The core 3.x docs have been updated by removing all reference to 
old-style classes and the modifier 'new-style'. The concept 'new-style' 
only exists in opposition to 'old-style'. 3.x just has classes, and all 
are subclasses of object.


--
Terry Jan Reedy


--
http://mail.python.org/mailman/listinfo/python-list

Re: Is the mailing list to usenet gateway borked?

[Peter Pearson]

On 20 Jun 2011 23:49:11 GMT, Steven D'Aprano wrote:
[snip]
> I will treat this as a bug in Pan, and take it to the appropriate forums, 
> but for anyone who cares, here's one example:
>
> From:   Steven D'Aprano 
> Subject:   Re: What is this syntax ?
> Newsgroups:   comp.lang.python
> References:   <4dfdfc99$0$715$426a3...@news.free.fr> <4dfe10d1$0$28053
> $426a3...@news.free.fr>  
>  dd0c35.16204819062...@news.panix.com>  l...@python.org>
> MIME-Version:   1.0
> Content-Type:   text/plain; charset=UTF-8
> Content-Transfer-Encoding:   8bit
> Date:   19 Jun 2011 23:19:56 GMT
> Lines:   36
> Message-ID:   <4dfe841c$0$30002$c3e8da3$54964...@news.astraweb.com>

As you probably expected, that message is displayed normally
by slrn.

-- 
To email me, substitute nowhere->spamcop, invalid->net.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rant on web browsers

[ron_m]

This might help

http://blog.stevenlevithan.com/archives/date-time-format
-- 
http://mail.python.org/mailman/listinfo/python-list

[OT] One click, one (buggy) life...

[DavCori80]

Hi everyone,
I would like to share a youtube clip...one click costs nothing while can 
save lives sometimes (especially mine).


http://www.youtube.com/watch?v=PiCeqtGHpJI

Thanks a lot and cheers.


DavCori
http://www.ar4tro.com/welcome.html
--
http://mail.python.org/mailman/listinfo/python-list

[ANN] Shed Skin 0.8

[Mark Dufour]

Hi all,

I have just released version 0.8 of Shed Skin, an experimental
(restricted-)Python-to-C++ compiler.

Please see my blog for the full announcement:

http://shed-skin.blogspot.com

The Shed Skin homepage can be found here:

http://shedskin.googlecode.com


Thanks,
Mark Dufour
-- 
http://www.youtube.com/watch?v=E6LsfnBmdnk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to iterate on a changing dictionary

[MRAB]

On 21/06/2011 12:51, Gurpreet Singh wrote:

Perhaps this is the simplest and best solution which appears in this case. Just 
copy the desired items to a new dictionary and discard the original one.

import re
myDict={'a':'alpha','b':'beta','c':'charley','d':'disney'}
myNewDict={}
for k,v in myDict.iteritems():
 if re.search("a",v)!=None:
myNewDict[k]=v
print myDict
print myNewDict


Using regex is overkill. Try this instead:

if "a" in v:
--
http://mail.python.org/mailman/listinfo/python-list

Don't understand SequenceMatcher from difflib

[Antoon Pardon]

I have the following code I wrote.

==

from difflib import SequenceMatcher

import sys
write = sys.stdout.write
warn = sys.stderr.write

def program(argv):
  ls1 = open(argv[1]).readlines()
  ls2 = open(argv[2]).readlines()
  matcher = SequenceMatcher(ls1, ls2)
  s1 = 0
  s2 = 0
  print ls1
  print ls2
  warn("*** %d %d \n" % (len(ls1), len(ls2)))
  for e1, e2, lg in matcher.get_matching_blocks():
warn("*** %d %d %d\n" % (e1, e2, lg))
for i in xrange(s1, e1):
  write('- ')
  write(ls1[i])
for i in xrange(s2, e2):
  write('+ ')
  write(ls2[i])
for i in xrange(e1, e1+lg):
  write('  ')
  write(ls1[i])
s1, s2 = e1 + lg, e2 + lg

if __name__ == '__main__':
  program(sys.argv)

===

Now when I run it I get the following result:

python diff.py map.0 map.1
['\n', 'begin\n', '  a1\n', '  a2\n', '  a3\n', '  a4\n', '  a5\n', 'end\n', 
'\n', 'begin\n', '  c1\n', '  c2\n', '  c3\n', '  c4\n', '  c5\n', '  c6\n', '  
c7\n', 'end\n', '\n', 'begin\n', '  e1\n', '  e2\n', '  e3\n', '  e4\n', '  
e5\n', '  e6\n', '  e7\n', '  e8\n', '  e9\n', 'end\n']
['\n', 'begin\n', '  a1\n', '  a2\n', '  a3\n', '  a4\n', '  a5\n', 'end\n', 
'\n', 'begin\n', '  c1\n', '  c2\n', '  c3\n', '  c4\n', '  c5\n', '  c6\n', '  
c7\n', 'end\n', '\n', 'begin\n', '  d1\n', '  d2\n', '  d3\n', 'end\n', '\n', 
'begin\n', '  e1\n', '  e2\n', '  e3\n', '  e4\n', '  e5\n', '  e6\n', '  
e7\n', '  e8\n', '  e9\n', 'end\n']
*** 30 36 
*** 36 0 0
- 
- begin
-   a1
-   a2
-   a3
-   a4
...
- Traceback (most recent call last):
  File "diff.py", line 31, in 
program(sys.argv)
  File "diff.py", line 21, in program
write(ls1[i])
IndexError: list index out of range

What I don't understand is: The first list is 30 items long and the second 36.
But the first match I get after calling get_matching_blocks says the match 
starts
at item 36 of the first list.

Yes I noticed it is the sepcial last match with 0 siza,e but even if that would 
be
correct because there would be no common items, the first number of the match
shouldn't be more than the length of the first list.

What am I doing wrong?
-- 
http://mail.python.org/mailman/listinfo/python-list

Emails backup in python 3.2

[TheSaint]

Hello,
I'm looking for an idea how to backup emails retrieved by poplib and save 
them into mailbox.mbox file.
The problem is the received message which is a list of bytes streams, 
mailbox.mbox don't expect a list.
What conversion should I do?
A file type io.StringIO ?
decoding every bytes stream which might not have any declared codec?

As far as python moved into unicode, why doesn't it handle these undecoded 
bytes as it was with strings before?

-- 
goto /dev/null
-- 
http://mail.python.org/mailman/listinfo/python-list

Web design services | website designing | hire a website designer | creative web design

[sravan kumar]

web design services, XHTML Conversion Services, Offshore outsourcing
web design & SEO Expert Acedezines provides best services for your
website design, search engine optimization, web design services,
brochure design, flash intro animation, website designing, wordpress
themes.
for more details http://www.acedezines.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: break in a module

[Cameron Simpson]

On 21Jun2011 20:04, I wrote:
| So the caller does an:
| 
|   import foo
| 
| as normal, with no special wrapping. And the module goes:
| 
|   spam()
|   if condition:
| raise StopIteration
|   ham()
|   cheese()

Of course, that should be StopImport, not StopIteration.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

Agree, for Law is costly. -- Very good advice to litigious Persons, founded
upon Reason and Experience; for many Times the Charges of a Suit exceed the
Value of the Thing in Dispute. - Bailey's dictionary, 1736
-- 
http://mail.python.org/mailman/listinfo/python-list

PSD to XHTML Conversion Services and PSD to HTML CSS Conversion Services, PSD to Joomla, Drupal, Wordpress Conversion

[xhtml champs]

PSD to XHTML Conversion, PSD to HTML CSS, Joomla, Wordpress, Drupal,
CMS, VBULLETIN, PHPBB and  includes convert to XHTML like PSD to
XHTML, web designing services, logos and banner design, website
building, animation,presentations.
for more details: http://www.xhtmlchamps.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to iterate on a changing dictionary

[Gurpreet Singh]

Perhaps this is the simplest and best solution which appears in this case. Just 
copy the desired items to a new dictionary and discard the original one.

import re
myDict={'a':'alpha','b':'beta','c':'charley','d':'disney'}
myNewDict={}
for k,v in myDict.iteritems():
if re.search("a",v)!=None:
   myNewDict[k]=v
print myDict
print myNewDict
-- 
http://mail.python.org/mailman/listinfo/python-list