Re: running Python2 Python3 parallel concurrent

[rusi]

On Mar 31, 6:05 am, harrismh777  wrote:
> Greetings,
>
>      The purpose of this communique is to document a process for
> installing python2.7.1 in parallel with python3.2 on a concurrent
> desktop with independent idle and python path structure.

Not sure if this is relevant...
python-mode (emacs mode for python development) is broken for python 3
I submitted a 1-line patch which makes python-mode work for 2.x and
3.x
Here https://bugs.launchpad.net/python-mode/+bug/450552

-- 
http://mail.python.org/mailman/listinfo/python-list

Jobs

[Mukunda Lohani]

If you want to get the innovative information on the Job & job Market,
please click on this website:

www.111jobs.blogspot.com

This website is based on the job technique and idea.
-- 
http://mail.python.org/mailman/listinfo/python-list

News

[Mukunda Lohani]

For getting the effective information on the News, please click on
this website:

www.ml97712.blogspot.com

This website will provide you the knowledge about the best news.
-- 
http://mail.python.org/mailman/listinfo/python-list

Thinking and Analysis based on Marxism & Literature

[Mukunda Lohani]

For getting good knowledge about Marxism and Literature, Marx & his
Opinions to sharpen thinking analytically, don't forget to click on
the website given below:-
www.analyzethink.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fun python 3.2 one-liner

[Giacomo Boffi]

Chris Angelico  writes:

>> def f(x,n,w): return x if n==1 else\
>>    (lambda x0=f(x[::2],n/2,w[::2]),\
>>            x1=f(x[1::2],n/2,w[::2]): reduce(lambda a,b: a+b ,\
>>                                      zip(*[(x0[k]+w[k]*x1[k],\
>>                                             x0[k]-w[k]*x1[k])\
>>                                             for k in range(n/2)])))()

> What sort of parameters does this take? So far all I can figure out
> is that n is an integer and x and w are sliceables, but I'm not sure
> whether x and w should be strings or arrays.

def direct_fft(x,n):
  return f(x,n,[exp(-2*pi*1j*k/n) for k in range(n/2)]) 
def inverse_fft(x,n):
  return [x/n for x in f(x,n,[exp(+2*pi*1j*k/n) for k in range(n/2)])] 
-- 
le mie sacrosante questioni di principio
  VS gli sciocchi puntigli di quel cretino del mio vicino
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Testing for performance regressions

[Steven D'Aprano]

On Mon, 04 Apr 2011 20:59:52 -0700, geremy condra wrote:

> On Mon, Apr 4, 2011 at 7:45 PM, Steven D'Aprano
>  wrote:

>> * The disclaimers about timing code snippets that can be found in the
>> timeit module apply. If possible, use timeit rather than roll-you-own
>> timers.
> 
> Huh. In looking into timing attacks actually one of the biggest lessons
> I learned was *not* to use timeit- that the overhead and variance
> involved in using it will wind up consuming small changes in behavior in
> ways that are fairly opaque until you really take it apart.

Do you have more details?

I follow the advice in the timeit module, and only ever look at the 
minimum value, and never try to calculate a mean or variance. Given the 
number of outside influences ("What do you mean starting up a browser 
with 200 tabs at the same time will affect the timing?"), I wouldn't 
trust a mean or variance to be meaningful.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Terry Reedy]

On 4/4/2011 9:16 PM, harrismh777 wrote:


Another item that would be nice as an IDLE enhancement would be a menu
option that applies the fixers (either direction depending on version
2.7 <--> 3.2) right in the IDE. Entries that could not be fixed could be
flagged for manual update.


I have had the same idea, so naturally I agree ;-).
I might even work on it eventually.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Paddy]

On Tuesday, April 5, 2011 2:16:07 AM UTC+1, harrismh777 wrote:
> Steven D'Aprano wrote:
> > I prefer to consider Python 2.7 and Python 3.x as different dialects of
> > the same language. There are a very few handful of incompatibilities,
> > most of which can be automatically resolved by the 2to3 fixers.
> 
> Yes, I am actually finding this to be consistent with my experience of 
> trying to come up to speed with 3.2.  I have been relieved to find that 
> less has changed than the fear-mongering and bickering was leading me to 
> believe.
> 
> Another item that would be nice as an IDLE enhancement would be a menu 
> option that applies the fixers (either direction depending on version 
> 2.7 <--> 3.2) right in the IDE. Entries that could not be fixed could be 
> flagged for manual update.
> 
> If there are good tools for app developers to use to make the transition 
> smoother then the development community won't get their ear chewed off 
> so ragged, I'm supposing.

Hats off and three cheers to the developers and python community as a whole, as 
some are down to sugesting easier access to 2<->3 rather than OMG! How do I 
port!!

Now that is an excellent sign that Python works.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Testing for performance regressions

[Paddy]

In an extended case when you try and capture how a function works over a range 
of inputs, you might want to not assume some relationship between input size 
and time, as this mnight limit your ability to change algorithms and still have 
acceptable performance. 

I.e. instead of this:

input_range = (MIN, AVERAGE, MAX)
for i in inpute_range:
..baseline = Timer(simple_func(i)).timeit()
..time_taken = Timer(my_func(i)).timeit()
..assert time_taken <= simple_relation(i) * baseline

It might be better to do this:

input_range = (MIN_R, AVERAGE_R, MAX_R)
time_ranges = (MIN_T, AVERAGE_T, MAX_T)
for i,t in zip(inpute_range, time_ranges):
..baseline = Timer(simple_func(i)).timeit()
..time_taken = Timer(my_func(i)).timeit()
..assert time_taken <= t * baseline

This comes from electronic circuit design where designs must be proven to work 
over a range for different values for example voltage ranges and temperature 
ranges. 

The action of the function being timed might not be simple, for example if 
my_func swapped algorithms depending on its input to favour the average case. 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Testing for performance regressions

[geremy condra]

On Mon, Apr 4, 2011 at 7:45 PM, Steven D'Aprano
 wrote:
> I'm writing some tests to check for performance regressions (i.e. you
> change a function, and it becomes much slower) and I was hoping for some
> guidelines or hints.
>
> This is what I have come up with so far:
>
>
> * The disclaimers about timing code snippets that can be found in the
> timeit module apply. If possible, use timeit rather than roll-you-own
> timers.

Huh. In looking into timing attacks actually one of the biggest
lessons I learned was *not* to use timeit- that the overhead and
variance involved in using it will wind up consuming small changes in
behavior in ways that are fairly opaque until you really take it
apart.

> * Put performance tests in a separate test suite, because they're
> logically independent of regression tests and functional tests, and
> therefore you might not want to run them all the time.
>
> * Never compare the speed of a function to some fixed amount of time,
> since that will depend on the hardware you are running on, but compare it
> relative to some other function's running time. E.g.:
>
> # Don't do this:
> time_taken = Timer(my_func).timeit()  # or similar
> assert time_taken <= 10
>    # This is bad, since the test is hardware dependent, and a change
>    # in environment may cause this to fail even if the function
>    # hasn't changed.
>
> # Instead do this:
> time_taken = Timer(my_func).timeit()
> baseline = Timer(simple_func).timeit()
> assert time_taken <= 2*baseline
>    # my_func shouldn't be more than twice as expensive as simple_func
>    # no matter how fast or slow they are in absolute terms.
>
>
> Any other lessons or hints I should know?

If you can get on it, emulab is great for doing network performance
and correctness testing, and even if you can't it might be worth
running a small one at your company. I wish I'd found out about it
years ago.

Geremy Condra
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Testing for performance regressions

[Dan Stromberg]

On Mon, Apr 4, 2011 at 7:45 PM, Steven D'Aprano <
steve+comp.lang.pyt...@pearwood.info> wrote:

> I'm writing some tests to check for performance regressions (i.e. you
> change a function, and it becomes much slower) and I was hoping for some
> guidelines or hints.
>
> This is what I have come up with so far:
>
>
> * The disclaimers about timing code snippets that can be found in the
> timeit module apply. If possible, use timeit rather than roll-you-own
> timers.
>
> * Put performance tests in a separate test suite, because they're
> logically independent of regression tests and functional tests, and
> therefore you might not want to run them all the time.
>
> * Never compare the speed of a function to some fixed amount of time,
> since that will depend on the hardware you are running on, but compare it
> relative to some other function's running time. E.g.:
>
> # Don't do this:
> time_taken = Timer(my_func).timeit()  # or similar
> assert time_taken <= 10
># This is bad, since the test is hardware dependent, and a change
># in environment may cause this to fail even if the function
># hasn't changed.
>
> # Instead do this:
> time_taken = Timer(my_func).timeit()
> baseline = Timer(simple_func).timeit()
> assert time_taken <= 2*baseline
># my_func shouldn't be more than twice as expensive as simple_func
># no matter how fast or slow they are in absolute terms.
>
>
> Any other lessons or hints I should know?
>
> If it helps, my code will be targeting Python 3.1, and I'm using a
> combination of doctest and unittest for the tests.
> 


Interesting topic.

I suppose you could compare to a pystone result times some constant.
http://code.activestate.com/recipes/440700-performance-testing-with-a-pystone-measurement-dec/

FWIW, doctest is a cool idea, but it kind of limits your options, as it
enshrines little details that'll cause your tests to fail if you move to
Pypy or Jython or IronPython or whatever.

I tend to lump my performance-related tests in with my other tests, but
perhaps this is a personal preference thing.  So of course, I try to keep my
performance tests brief - sometimes with the non-default option of doing a
more thorough test.  Because the time to know that things have suddenly
slowed way down is during development, not right before a new release.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: iterable argument required

[eryksun ()]

On Monday, April 4, 2011 9:40:33 AM UTC-4, Νικόλαος Κούρας wrote:

In one of your messages you wrote the following:

> cursor.execute( '''INSERT INTO users(mail, comment) VALUES(%s,
> %s)''', (mail, comment) )
> except MySQLdb.Error:
> print ( "Error %d: %s" % (e.args[0], e.args[1]) )

Is this a typo in your message or the actual code? If 'e' is unassigned you 
should be getting a NameError. The standard Python2 syntax (before version 2.6) 
is the following:

except MySQLdb.Error, e:
print("Error %d: %s" % (e.args[0], e.args[1]))

You also wrote:

> mail = None, comment = None

This should cause a SyntaxError because it's trying to assign None = None. 
That's assuming it's on line 167, after the cursor.execute(...) on line 166.

> Whats was the problem as i have written it?
> Your solution is the same as mine except the fact that you assign
> values and statements into variables.

I was just rewriting it to make sure I was parsing it right.

> I tried it but iam getting an Internal Server Error.

Yes, I made a mistake. It should have been `cursor.execute(SQL_COMMENT_FORM, 
(mail, comment))`, using a comma instead of a '%' to have it generate SQL 
string literals from the tuple.

> Also i noticed that if i append a query string in the end of a url
> with the varibble mail attached like
> 
> http://superhost.gr/hosting.html?mail=test
> 
> then page hosting.html does load without any problem.
> 
> If i remove the query string from the ned of the URL then i'am getting
> the error message i posted.
> 
> So its not that the if condition is wrong but something happens with
> the form variable 'mail' .

Insert a test to print out the type and value of 'mail' for various inputs. 

Regarding the message itself, on Python 2.7.1, I get the following TypeError 
message if I try iterate None:

TypeError: argument of type 'NoneType' is not iterable

Python 2.5.2 on "http://shell.appspot.com"; yields the same error. Version 2.5 
improved the error messages to include the type of the object (see issue 
1507676). The message "iterable argument required" looks like an older version 
of CPython.
-- 
http://mail.python.org/mailman/listinfo/python-list

Testing for performance regressions

[Steven D'Aprano]

I'm writing some tests to check for performance regressions (i.e. you 
change a function, and it becomes much slower) and I was hoping for some 
guidelines or hints.

This is what I have come up with so far:


* The disclaimers about timing code snippets that can be found in the 
timeit module apply. If possible, use timeit rather than roll-you-own 
timers.

* Put performance tests in a separate test suite, because they're 
logically independent of regression tests and functional tests, and 
therefore you might not want to run them all the time.

* Never compare the speed of a function to some fixed amount of time, 
since that will depend on the hardware you are running on, but compare it 
relative to some other function's running time. E.g.:

# Don't do this:
time_taken = Timer(my_func).timeit()  # or similar
assert time_taken <= 10
# This is bad, since the test is hardware dependent, and a change 
# in environment may cause this to fail even if the function 
# hasn't changed.

# Instead do this:
time_taken = Timer(my_func).timeit()
baseline = Timer(simple_func).timeit()
assert time_taken <= 2*baseline
# my_func shouldn't be more than twice as expensive as simple_func
# no matter how fast or slow they are in absolute terms.


Any other lessons or hints I should know?

If it helps, my code will be targeting Python 3.1, and I'm using a 
combination of doctest and unittest for the tests.


Thanks in advance,



-- 
Steven.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[harrismh777]

Steven D'Aprano wrote:

I prefer to consider Python 2.7 and Python 3.x as different dialects of
the same language. There are a very few handful of incompatibilities,
most of which can be automatically resolved by the 2to3 fixers.


Yes, I am actually finding this to be consistent with my experience of 
trying to come up to speed with 3.2.  I have been relieved to find that 
less has changed than the fear-mongering and bickering was leading me to 
believe.


Another item that would be nice as an IDLE enhancement would be a menu 
option that applies the fixers (either direction depending on version 
2.7 <--> 3.2) right in the IDE. Entries that could not be fixed could be 
flagged for manual update.


If there are good tools for app developers to use to make the transition 
smoother then the development community won't get their ear chewed off 
so ragged, I'm supposing.




kind regards,
m harris

--
http://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing, shared memory vs. pickled copies

[Philip Semanchuk]

On Apr 4, 2011, at 9:03 PM, Dan Stromberg wrote:

> On Mon, Apr 4, 2011 at 4:34 PM, Philip Semanchuk wrote:
> 
>> So if you're going to use multiprocessing, you're going to use pickle, and
>> you need pickleable objects.
>> 
> 
> http://docs.python.org/library/multiprocessing.html#sharing-state-between-processes


Thank you, Dan. My reading comprehension skills need work.

Cheers
Philip

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyThreadState_Swap crash

[Philip Semanchuk]

On Apr 4, 2011, at 9:08 AM, Wiktor Adamski wrote:

> I have 2 threads in C code using python 2.5.2. First thread creates
> new interpreter (i need several interpreters but those 2 threads use
> only one) like that:
> 
> PyEval_AcquireLock();
> threadState = Py_NewInterpreter();
> PyThreadState_Swap(threadState);
> 
> // calling python API
> 
> PyThreadState_Swap(NULL);
> PyEval_ReleaseLock();
> 
> Second thread uses interpreter created in first thread:
> 
> PyEval_AcquireLock();
> PyThreadState_Swap(threadState);
> 
> and sometimes PyThreadState_Swap crashes in debug build
> (PyGILState_GetThisThreadState() returns garbage). In release build
> that code doesn't run and so far no other problem was found.
> I call PyEval_InitThreads() at the begining of program and every
> PyEval_AcquireLock() has PyEval_ReleaseLock().
> 
> Am I allowed to use the same threadState in different threads?
> If I am, is there another problem in my code?
> Or maybe it's a bug in python - acording to documentation "Python
> still supports the creation of additional interpreters (using
> Py_NewInterpreter()), but mixing multiple interpreters and the
> PyGILState_*() API is unsupported." - I don't use PyGILState_ but it's
> used internally in PyThreadState_Swap(). I also don't use
> PyEval_RestoreThread() - comment sugests that crashing code is present
> because possibility of calling from PyEval_RestoreThread().

Hi Wiktor,
I'm sorry I don't have a solution or even a suggestion for you. I just wanted 
to point out that PyEval_AcquireLock() and PyEval_ReleaseLock() were recently 
deprecated:
http://bugs.python.org/issue10913

Obviously they'll be around for quite a while longer but given the 
ominous-but-vague warning in issue10913's description, you might want to stay 
away from them. It's frustrating for me because I've got code I can't get to 
work without them.

Good luck
Philip



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing, shared memory vs. pickled copies

[Dan Stromberg]

On Mon, Apr 4, 2011 at 4:34 PM, Philip Semanchuk wrote:

> So if you're going to use multiprocessing, you're going to use pickle, and
> you need pickleable objects.
>

http://docs.python.org/library/multiprocessing.html#sharing-state-between-processes
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Steven D'Aprano]

On Tue, 05 Apr 2011 07:41:37 +1000, Chris Angelico wrote:

> On Tue, Apr 5, 2011 at 7:10 AM, rantingrick 
> wrote:
>> olks who have vast libraries of Python code that have been rendered
>> useless because a few elites have spent too much time lamenting
>> minutia.
> 
> How is the code "rendered useless" when (a) Python 2.7 is still the
> default Python in many places, and (b) in any place where it's not, it's
> going to be about one command to install a python2?

Please don't feed the troll.

Check the archives. rantingrick isn't interested in good-faith debate. 
He's trying to rally followers to lead on his crusade to save Python from 
itself (which *entirely* consists of him declaring that there is a 
problem, and everyone else dropping what they're doing to do the actual 
work of fixing it).


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing, shared memory vs. pickled copies

[Robert Kern]

On 4/4/11 3:20 PM, John Ladasky wrote:

Hi folks,

I'm developing some custom neural network code.  I'm using Python 2.6,
Numpy 1.5, and Ubuntu Linux 10.10.  I have an AMD 1090T six-core CPU,
and I want to take full advantage of it.  I love to hear my CPU fan
running, and watch my results come back faster.


You will want to ask numpy questions on the numpy mailing list.

  http://www.scipy.org/Mailing_Lists


When I'm training a neural network, I pass two numpy.ndarray objects
to a function called evaluate.  One array contains the weights for the
neural network, and the other array contains the input data.  The
evaluate function returns an array of output data.

I have been playing with multiprocessing for a while now, and I have
some familiarity with Pool.  Apparently, arguments passed to a Pool
subprocess must be able to be pickled.  Pickling is still a pretty
vague progress to me, but I can see that you have to write custom
__reduce__ and __setstate__ methods for your objects.  An example of
code which creates a pickle-friendly ndarray subclass is here:

http://www.mail-archive.com/numpy-discussion@scipy.org/msg02446.html


Note that numpy arrays are already pickle-friendly. This message is telling you 
how, *if* you are already subclassing, how to make your subclass pickle the 
extra information it holds.



Now, I don't know that I actually HAVE to pass my neural network and
input data as copies -- they're both READ-ONLY objects for the
duration of an evaluate function (which can go on for quite a while).
So, I have also started to investigate shared-memory approaches.  I
don't know how a shared-memory object is referenced by a subprocess
yet, but presumably you pass a reference to the object, rather than
the whole object.   Also, it appears that subprocesses also acquire a
temporary lock over a shared memory object, and thus one process may
well spend time waiting for another (individual CPU caches may
sidestep this problem?) Anyway, an implementation of a shared-memory
ndarray is here:

https://bitbucket.org/cleemesser/numpy-sharedmem/src/3fa526d11578/shmarray.py

I've added a few lines to this code which allows subclassing the
shared memory array, which I need (because my neural net objects are
more than just the array, they also contain meta-data).


Honestly, you should avoid subclassing ndarray just to add metadata. It never 
works well. Make a plain class, and keep the arrays as attributes.



But I've run
into some trouble doing the actual sharing part.  The shmarray class
CANNOT be pickled.


Please never just *say* that something doesn't work. Show us what you tried, and 
show us exactly what output you got. I assume you tried something like this:


[Downloads]$ cat runmp.py
from multiprocessing import Pool
import shmarray


def f(z):
return z.sum()

y = shmarray.zeros(10)
z = shmarray.ones(10)

p = Pool(2)
print p.map(f, [y, z])


And got output like this:

[Downloads]$ python runmp.py
Exception in thread Thread-2:
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/threading.py", 
line 530, in __bootstrap_inner

self.run()
  File 
"/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/threading.py", 
line 483, in run

self.__target(*self.__args, **self.__kwargs)
  File 
"/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/multiprocessing/pool.py", 
line 287, in _handle_tasks

put(task)
PicklingError: Can't pickle 'multiprocessing.sharedctypes.c_double_Array_10'>: attribute lookup 
multiprocessing.sharedctypes.c_double_Array_10 failed



Now, the sharedctypes is supposed to implement shared arrays. Underneath, they 
have some dynamically created types like this c_double_Array_10 type. 
multiprocessing has a custom pickler which has a registry of reduction functions 
for types that do not implement a __reduce_ex__() method. For these dynamically 
created types that cannot be imported from a module, this dynamic registry is 
the only way to do it. At least at one point, the Connection objects which 
communicate between processes would use this custom pickler to serialize objects 
to bytes to transmit them.


However, at least in Python 2.7, multiprocessing seems to have a C extension 
module defining the Connection objects. Unfortunately, it looks like this C 
extension just imports the regular pickler that is not aware of these custom 
types. That's why you get this error. I believe this is a bug in Python.


So what did you try, and what output did you get? What version of Python are you 
using?



I think that my understanding of multiprocessing
needs to evolve beyond the use of Pool, but I'm not sure yet.  This
post suggests as much.

http://mail.scipy.org/pipermail/scipy-user/2009-February/019696.html


Maybe. If the __reduce_ex__() method is implemented properly (and 
multiprocessing bugs aren't getting in the way), you ought to be able to pass 
them to a Pool just fine. You just need to make

Re: Multiprocessing, shared memory vs. pickled copies

[Philip Semanchuk]

On Apr 4, 2011, at 4:20 PM, John Ladasky wrote:

> I have been playing with multiprocessing for a while now, and I have
> some familiarity with Pool.  Apparently, arguments passed to a Pool
> subprocess must be able to be pickled.  

Hi John,
multiprocessing's use of pickle is not limited to Pool. For instance, objects 
put into a multiprocessing.Queue are also pickled, as are the args to a 
multiprocessing.Process. So if you're going to use multiprocessing, you're 
going to use pickle, and you need pickleable objects. 


> Pickling is still a pretty
> vague progress to me, but I can see that you have to write custom
> __reduce__ and __setstate__ methods for your objects.

Well, that's only if one's objects don't support pickle by default. A lot of 
classes do without any need for custom __reduce__ and __setstate__ methods. 
Since you're apparently not too familiar with pickle, I don't want you to get 
the false impression that it's a lot of trouble. I've used pickle a number of 
times and never had to write custom methods for it.



> Now, I don't know that I actually HAVE to pass my neural network and
> input data as copies -- they're both READ-ONLY objects for the
> duration of an evaluate function (which can go on for quite a while).
> So, I have also started to investigate shared-memory approaches.  I
> don't know how a shared-memory object is referenced by a subprocess
> yet, but presumably you pass a reference to the object, rather than
> the whole object.   Also, it appears that subprocesses also acquire a
> temporary lock over a shared memory object, and thus one process may
> well spend time waiting for another (individual CPU caches may
> sidestep this problem?) Anyway, an implementation of a shared-memory
> ndarray is here:

There's no standard shared memory implementation for Python. The mmap module is 
as close as you get. I wrote & support the posix_ipc and sysv_ipc modules which 
give you IPC primitives (shared memory and semaphores) in Python. They work 
well (IMHO) but they're *nix-only and much lower level than multiprocessing. If 
multiprocessing is like a kitchen well stocked with appliances, posix_ipc (and 
sysc_ipc) is like a box of sharp knives.

Note that mmap and my IPC modules don't expose Python objects. They expose raw 
bytes in memory. YOu're still going to have to jump through some hoops (...like 
pickle) to turn your Python objects into a bytestream and vice versa.


What might be easier than fooling around with boxes of sharp knives is to 
convert your ndarray objects to Python lists. Lists are pickle-friendly and 
easy to turn back into ndarray objects once they've crossed the pickle 
boundary. 


> When should one pickle and copy?  When to implement an object in
> shared memory?  Why is pickling apparently such a non-trivial process
> anyway?  And, given that multi-core CPU's are apparently here to stay,
> should it be so difficult to make use of them?

My answers to these questions:

1) Depends
2) In Python, almost never unless you're using a nice wrapper like shmarray.py
3) I don't think it's non-trivial =)
4) No, definitely not. Python will only get better at working with multiple 
cores/CPUs, but there's plenty of room for improvement on the status quo.

Hope this helps
Philip





-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Steven D'Aprano]

On Mon, 04 Apr 2011 17:09:07 -0500, harrismh777 wrote:

> Python(3) is a new language. It has many of the same characteristics
> of Python2, but will be more consistent, cleaner, leaner, more robust...
> and certainly loved more universally by more people the world over for
> centuries to come;-)

Only if you define "language" so narrowly that Python 2.1 and Python 2.2 
are also different languages, or CPython and Jython or IronPython.

I prefer to consider Python 2.7 and Python 3.x as different dialects of 
the same language. There are a very few handful of incompatibilities, 
most of which can be automatically resolved by the 2to3 fixers. To 
describe them as different "languages" leaves no term to describe the 
differences between (say) Python and Cobra:

http://cobra-language.com/docs/python/

let alone something like Python vs Forth.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fun python 3.2 one-liner

[Emile van Sebille]

On 4/4/2011 3:16 PM Gregory Ewing said...

Chris Angelico wrote:


(Remind me how it is that Python code is more readable than line noise
or Perl code?)


Crazy thought: I wonder if Perl programmers have "multi
line Perl" competitions where they laugh their heads off
at how readable the code is, and how nobody in their
right mind would ever write Perl code that way?-)



I don't know if I should be laughing or nodding in agreement... :)

Emile


--
http://mail.python.org/mailman/listinfo/python-list

Re: Fun python 3.2 one-liner

[Chris Angelico]

On Tue, Apr 5, 2011 at 8:16 AM, Gregory Ewing
 wrote:
> Chris Angelico wrote:
>
> Crazy thought: I wonder if Perl programmers have "multi
> line Perl" competitions where they laugh their heads off
> at how readable the code is, and how nobody in their
> right mind would ever write Perl code that way?-)

Ha!! Okay, now I have to explain to my fellow bus passengers what it
is that I just cracked up laughing at...

You know what? I don't think I can.

I do like readable code, but quite a few of my favorite language
features are the ones commonly (ab)used to make unreadable code. C's
?: operator, Pike's interpretation of || for defaults, Python's lambda
functions... all great ways to shorten your code, but so easily used
for evil.

I think I like things that can be used for evil.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Toronto PyCamp 2011

[Chris Calloway]

The University of Toronto Department of Physics brings PyCamp to Toronto 
on Monday, June 27 through Thursday, June 30, 2011.


Register today at http://trizpug.org/boot-camp/torpy11/

For beginners, this ultra-low-cost Python Boot Camp makes you productive 
so you can get your work done quickly. PyCamp emphasizes the features 
which make Python a simpler and more efficient language. Following along 
with example Python PushUps™ speeds your learning process. Become a 
self-sufficient Python developer in just four days at PyCamp! PyCamp is 
conducted on the campus of the University of Toronto in a state of the 
art high technology classroom.


--
Sincerely,

Chris Calloway http://nccoos.org/Members/cbc
office: 3313 Venable Hall   phone: (919) 599-3530
mail: Campus Box #3300, UNC-CH, Chapel Hill, NC 27599
--
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Chris Angelico]

On Tue, Apr 5, 2011 at 8:16 AM, rantingrick  wrote:
>> and (b) in any place where it's not,
>> it's going to be about one command to install a python2?
>
> Oh thanks Chris for revealing the simplicity of 2 to 3 code porting.
> And might ask where you will begin to volunteer your expertise to the
> gazillion lines of code that need porting? hmm?

That is specifically NOT porting. Sorry, I'm not a 2to3-on-call. I
just code for one platform.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fun python 3.2 one-liner

[Gregory Ewing]

Chris Angelico wrote:


(Remind me how it is that Python code is more readable than line noise
or Perl code?)


Crazy thought: I wonder if Perl programmers have "multi
line Perl" competitions where they laugh their heads off
at how readable the code is, and how nobody in their
right mind would ever write Perl code that way?-)

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[rantingrick]

On Apr 4, 4:41 pm, Chris Angelico  wrote:
> How is the code "rendered useless" when (a) Python 2.7 is still the
> default Python in many places,

That's the point. We are going to see Python2.x around for a very long
time. A *very* long time Chris. Sadly if this conversion was planned a
wee bit better, we could have seen it happen rather quickly instead.
We could have reined in the multiplicity instead of propagating it!

> and (b) in any place where it's not,
> it's going to be about one command to install a python2?

Oh thanks Chris for revealing the simplicity of 2 to 3 code porting.
And might ask where you will begin to volunteer your expertise to the
gazillion lines of code that need porting? hmm?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Gregory Ewing]

Paul Rubin wrote:


Vector processors are back, they just call them GPGPU's now.


Also present to some extent in the CPU, with
MMX, Altivec, etc.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing, shared memory vs. pickled copies

[Dan Stromberg]

On Mon, Apr 4, 2011 at 1:20 PM, John Ladasky  wrote:

> When should one pickle and copy?  When to implement an object in
> shared memory?  Why is pickling apparently such a non-trivial process
> anyway?  And, given that multi-core CPU's are apparently here to stay,
> should it be so difficult to make use of 
> them?


Pickle and copy when your distinct processes will benefit from  having
multiple distinct copies of the data - that is, copies that can diverge
from  each other if written to in one or more of the processes.

Use shared memory if your distinct processes need to be able to see a single
copy of the same data - EG, because one needs to see the result of the
changes made in another.

Pickling's not a big deal -  it's just turning structured data (from an int
to a nested series of objects) into a stream of bytes.  Not all extension
types support it without extra work though.

Yes, multicore is increasingly  important, and Python needs to support it
well.  multiprocessing is one good way of doing so.  Also, Python 3.2 has a
new module facilitating this,  and an improved GIL situation.  Then there
are things like greenlets and stackless python.  Oh, and if you move to
jython or ironpython, you get better multithreading.

Multithreading is generally a bit lighter-weight than multiprocessing, but
it also gives a little tighter coupling between different parts of the
program and a greater probability of race conditions.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[harrismh777]

rantingrick wrote:

Yes and whilst that was a brilliant display of bombastic arrogance
your statements miss the point completely.



And what about the large steaming pile of elephant dung in
the room your nose seems to be unable to smell?



As we all know Python has experienced an explosion of usage over the
past years, however i would say with the confusions of the Python2 to
Python3 conversion, old tutorials, Ruby's competition, and just plain
mis information in the wild we have cast deep into the throes of a
internal Pythonic rejection-ism and now find ourselves teetering on
the brink of full blown world wide Pythonic recession-ism unless we
get this runaway ship under control very quickly.


oooh, ouch.


Sadly, there may be some truth in there...

... to play the advocate for a moment, as the client community we need 
to get our heads around the motives of the development community. The 
overall goal (it seems) is to make the Python language an 'ideal' that 
is admittantly evolving over time. The frustrating thing for the rest of 
us is that we now must develop code for two versions--- as well we must 
not make assumptions about whether our code will port nicely to other 
systems/platforms.  While this is frustrating, it is not an 
insurmountable mountain of elephant dung... although, I found the 
metaphor funny.   :)


   Python(3) is a new language. It has many of the same characteristics 
of Python2, but will be more consistent, cleaner, leaner, more robust... 
and certainly loved more universally by more people the world over for 
centuries to come;-)


   "Bring out yer dead...,""Bring out yer dead..."

:)



--
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Gregory Ewing]

geremy condra wrote:


I'd be interested in seeing the performance impact of this, although I
wonder if it'd be feasible.


A project I have in the back of my mind goes something
like this:

1) Design an instruction set for a Python machine and
a microcode architecture to support it

2) Write a simulator for it

3) Use the simulator to evaluate how effective it would
be if actually implemented, e.g. in an FPGA.

And if I get that far:

4) (optional) Get hold of a real FPGA and implement it

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Determining version of OpenSSL

[Adam Mercer]

On Mon, Apr 4, 2011 at 16:44, Martin v. Loewis  wrote:

>> is there a way that this can be done in python2.4? It's annoying but I
>> need to support python2.4 for a while yet :-(
>
> ldd /usr/lib/python2.4/lib-dynload/_ssl.so
> [...]
>        libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0x7f6a5a9b7000)
> [...]

Thanks but I was hoping to do this using python as this code needs to
run on both Linux and Mac OS X, I know I can run the appropriate tools
depending on the platform. This is an option at least...

Cheers

Adam
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Gregory Ewing]

Terry Reedy wrote:
So a 
small extension to array with .map, .filter, .reduce, and a wrapper 
class would be more useful than I thought.


Also useful would be some functions for doing elementwise
operations between arrays. Sometimes you'd like to just do
a bit of vector arithmetic, and pulling in the whole of
numpy as a dependency seems like overkill.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Fun python 3.2 one-liner

[Chris Angelico]

On Tue, Apr 5, 2011 at 6:09 AM, gb  wrote:
> harrismh777  writes:
>
>> Seriously, these little one liners teach me more about the python
>> language in less time than [...]
>
> def f(x,n,w): return x if n==1 else\
>    (lambda x0=f(x[::2],n/2,w[::2]),\
>            x1=f(x[1::2],n/2,w[::2]): reduce(lambda a,b: a+b ,\
>                                      zip(*[(x0[k]+w[k]*x1[k],\
>                                             x0[k]-w[k]*x1[k])\
>                                             for k in range(n/2)])))()
>
> it was a joke of sort played on it.comp.lang.python

(Remind me how it is that Python code is more readable than line noise
or Perl code?)

What sort of parameters does this take? So far all I can figure out is
that n is an integer and x and w are sliceables, but I'm not sure
whether x and w should be strings or arrays.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Determining version of OpenSSL

[Martin v. Loewis]

> import ssl
> ssl.OPENSSL_VERSION
> 
> is there a way that this can be done in python2.4? It's annoying but I
> need to support python2.4 for a while yet :-(

ldd /usr/lib/python2.4/lib-dynload/_ssl.so
[...]
libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0x7f6a5a9b7000)
[...]

HTH,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Chris Angelico]

On Tue, Apr 5, 2011 at 7:10 AM, rantingrick  wrote:
> olks who have vast libraries of Python
> code that have been rendered useless because a few elites have spent
> too much time lamenting minutia.

How is the code "rendered useless" when (a) Python 2.7 is still the
default Python in many places, and (b) in any place where it's not,
it's going to be about one command to install a python2?

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: proposal to allow to set the delimiter in str.format to something other than curly bracket

[Alia Khouri]

Peter Otten wrote:

> You could automatically convert from a custom format like that in your
> original post...



Here's a class wrapping your functionality:

import re

class Template(object):
'''uses double brackets e.g [[ob.attr]] as delims to get
  around curly bracket ({}) collisions when generating code
'''
_pattern = re.compile(r"\[\[|]]|\{|\}")
_lookup = {
"[[": "{",
"]]": "}",
"{": "{{",
"}": "}}",
}
def __init__(self, txt):
self.txt = txt
self.type = type

def _substitute(self, m):
return self._lookup[m.group()]

def render(self, *args, **kwds):
return self._pattern.sub(
self._substitute, self.txt).format(*args, **kwds)

def test_Template():
class P: pass
p = P()
p.name = 'peter'
txt = 'hello there [[o.name]]'
t = Template(txt)
assert t.render(o=p) == 'hello there peter'

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tkSimpleDialog.Dialog.ok calls Cancel

[rantingrick]

On Apr 4, 2:06 pm, Matt H  wrote:
> I am subclassing tkSimpleDialog.Dialog as a (sqlite) database front-
> end. This parent dialog opens a number of child dialogs to propagate
> sub-tables (like sub-forms in OOo). These subforms write their input
> into the database (INSERT...)
>
> The parent dialog then either commits or rolls-back transactions based
> on whether the user clicks ok or cancel. This ensures that if
> something is written to the db in a subform, that it doesn't stay in
> the database if the user chooses cancel (the rollback transaction code
> is in cancel).
>
> I have everything working except for the fact that python
> tkSimpleDialog.ok calls tkSimpleDialog.cancel to clean up, which means
> all of my transactions get cancelled (even if the user clicks ok). Is
> there a way to do this without re-implementing or copying the
> tkSimpleDialog.Dialog ok and cancel methods entirely?
>
> In other words, how do I make OK not execute cancel?

The ok and cancel methods are just a few lines of code for crying out
loud. And besides, you must have inherited from this class anyway to
create your custom dialog. If copy paste is not working for you then i
would suggest building a better light bulb. Tesla did it (and one-
upped that old fart and cheat Edison) so why can't you?

PS: And besides, the tkSimpleDialog class was ill conceived from day
one. Search the archives for my discussions on the pitfalls of this
code and my suggestions for curing (some) the many problems contained
within.

PPS: Not to say that a modal dialog that self closes is a design flaw
BTW!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[rantingrick]

On Apr 2, 5:13 am, Steven D'Aprano  wrote:

> "Should I use the list, list2, sortable_list, sortable_list2,
> sortable_lost3, [note spelling, which we're stuck with forever],
> heterogeneous_list, heterogeneous_list_without_stooge_sort, new_list,
> fancy_list, fancy_list2, fancy_list_with_extra_oomph, newer_than_new_list
> or list3?"
>
> Each and every interface carries a cost. Even if it is small, that cost
> must be weighed up against the benefits, rather than declaring that all
> interfaces are sacred once published.

Yes and whilst that was a brilliant display of bombastic arrogance
your statements miss the point completely. And to respond i'll pull a
sentence from your own post...

> None of those assumptions are even *remotely* true in the real world.

Moving on...

> Removing a published interface imposes a one-time cost on those using
> that interface, but it has an on-going benefit for all.

Oh really? And what about the large steaming pile of elephant dung in
the room your nose seems to be unable to smell?  You just *assume*
that more "new hands" are hopping on board the old Python vessel to
hell than "old hands" are dangling on to the rudders for dear life.
What a naive assumption Mr. D'Aprano!

As we all know Python has experienced an explosion of usage over the
past years, however i would say with the confusions of the Python2 to
Python3 conversion, old tutorials, Ruby's competition, and just plain
mis information in the wild we have cast deep into the throes of a
internal Pythonic rejection-ism and now find ourselves teetering on
the brink of full blown world wide Pythonic recession-ism unless we
get this runaway ship under control very quickly.

Now whilst i agree with most of the changes in Python3 i wonder if
some of them could have waited until Python4. We seemed to have become
so pedantic as to render ourselves blind to the cries of others. And
when i say "others" i am not speaking general "others". No, i am
speaking of fellow tried and true Pythonistas who have many years of
investment in this language. Folks who have vast libraries of Python
code that have been rendered useless because a few elites have spent
too much time lamenting minutia.

Where is the call from on high to rid the web of old Python2.x tuts
and ring in the era of Python3.x compatibility? Where is the energy to
re-ignite the flames of collaborative community? Where is the
leadership this community desperately needs? Where is the excitement
for the future of Python in the old hats? Oh yes i forgot... they're
too busy porting 2.x code to give a flying fig!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: proposal to allow to set the delimiter in str.format to something other than curly bracket

[Alia Khouri]

Peter Otten wrote:
> You could automatically convert from a custom format like that in your
> original post:
>
> import re
>
> _lookup = {
>     "[[": "{",
>     "]]": "}",
>     "{": "{{",
>     "}": "}}",
>
> }
>
> def _substitute(m):
>     return _lookup[m.group()]
>
> def custom_format(template, *args, **kw):
>     return (re.compile(r"\[\[|]]|\{|\}")
>             .sub(_substitute, template)
>             .format(*args, **kw))
>
> code = "class [[0]]Model { public bool IsModel(){ return a[42] || true; } }"
> print custom_format(code, "My")

Nice! I didn't think of that. I guess I could get some additional
performance by taking the re.compile step out of the function. Thanks
for the tip!

AK

AK


-- 
http://mail.python.org/mailman/listinfo/python-list

Determining version of OpenSSL

[Adam Mercer]

Hi

I'm trying to determine the version of OpenSSL that a given python is
compiled against, with python2.7 I can do this with:

import ssl
ssl.OPENSSL_VERSION

is there a way that this can be done in python2.4? It's annoying but I
need to support python2.4 for a while yet :-(

Cheers

Adam
-- 
http://mail.python.org/mailman/listinfo/python-list

Multiprocessing, shared memory vs. pickled copies

[John Ladasky]

Hi folks,

I'm developing some custom neural network code.  I'm using Python 2.6,
Numpy 1.5, and Ubuntu Linux 10.10.  I have an AMD 1090T six-core CPU,
and I want to take full advantage of it.  I love to hear my CPU fan
running, and watch my results come back faster.

When I'm training a neural network, I pass two numpy.ndarray objects
to a function called evaluate.  One array contains the weights for the
neural network, and the other array contains the input data.  The
evaluate function returns an array of output data.

I have been playing with multiprocessing for a while now, and I have
some familiarity with Pool.  Apparently, arguments passed to a Pool
subprocess must be able to be pickled.  Pickling is still a pretty
vague progress to me, but I can see that you have to write custom
__reduce__ and __setstate__ methods for your objects.  An example of
code which creates a pickle-friendly ndarray subclass is here:

http://www.mail-archive.com/numpy-discussion@scipy.org/msg02446.html

Now, I don't know that I actually HAVE to pass my neural network and
input data as copies -- they're both READ-ONLY objects for the
duration of an evaluate function (which can go on for quite a while).
So, I have also started to investigate shared-memory approaches.  I
don't know how a shared-memory object is referenced by a subprocess
yet, but presumably you pass a reference to the object, rather than
the whole object.   Also, it appears that subprocesses also acquire a
temporary lock over a shared memory object, and thus one process may
well spend time waiting for another (individual CPU caches may
sidestep this problem?) Anyway, an implementation of a shared-memory
ndarray is here:

https://bitbucket.org/cleemesser/numpy-sharedmem/src/3fa526d11578/shmarray.py

I've added a few lines to this code which allows subclassing the
shared memory array, which I need (because my neural net objects are
more than just the array, they also contain meta-data).  But I've run
into some trouble doing the actual sharing part.  The shmarray class
CANNOT be pickled.  I think that my understanding of multiprocessing
needs to evolve beyond the use of Pool, but I'm not sure yet.  This
post suggests as much.

http://mail.scipy.org/pipermail/scipy-user/2009-February/019696.html

I don't believe that my questions are specific to numpy, which is why
I'm posting here, in a more general Python forum.

When should one pickle and copy?  When to implement an object in
shared memory?  Why is pickling apparently such a non-trivial process
anyway?  And, given that multi-core CPU's are apparently here to stay,
should it be so difficult to make use of them?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Paul Rubin]

John Nagle  writes:
> That sort of thing was popular in the era of the early
> Cray machines.  Once superscalar CPUs were developed,
> the overhead on tight inner loops went down, and several
> iterations of a loop could be in the pipeline at one time,

Vector processors are back, they just call them GPGPU's now.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: running Python2 Python3 parallel concurrent

[harrismh777]

John Roth wrote:

You might want to look at PEP 394, which is tentatively scheduled for
Python 3.3 and the next maintenance release of 2.7. As far as I can
tell, this pretty much solves the problem for Unixoid and MacOS
systems.


Thanks John.  I finally read PEP 394 and 397. Yes, these handle the 
problem on the execute end. The solution to the problem on the 
application development end (my stuff, see previous) is concerning the 
requirement to be able to build apps for both versions in parallel--- 
from the development end.
Ultimately, it would be nice to have support in IDLE for multiple 
versions from under the umbrella of one IDE. I have decided that rather 
than try to port 32 -> 2.7  ( or 27. -> 3.2 ) that I will be developing 
for both (in parallel) concurrently. When I am done with the app two 
versions will be able, one sh-banged for 2.7 and the other sh-banged for 
3.2/
At the moment I am using two instances of IDLE running in parallel, one 
using 2.7.1 and the other using 3.2 /I would like to see them 
combined under one IDE with configuration options for multiple version 
environments (perhaps in addition to 2.7 , 3.2).
So, I'm playing a bit with IDLE to see what I can come up with. I don't 
know if anyone wants to see a PEP on that or not.
And, while I'm at it, I think I'll look into a way to provide a clear 
screen option for the IDLE interactive window...   :)


kind regards,
m harris

--
http://mail.python.org/mailman/listinfo/python-list

Re: Fun python 3.2 one-liner

[gb]

harrismh777  writes:

> Seriously, these little one liners teach me more about the python
> language in less time than [...]

def f(x,n,w): return x if n==1 else\
(lambda x0=f(x[::2],n/2,w[::2]),\
x1=f(x[1::2],n/2,w[::2]): reduce(lambda a,b: a+b ,\
  zip(*[(x0[k]+w[k]*x1[k],\
 x0[k]-w[k]*x1[k])\
 for k in range(n/2)])))()

it was a joke of sort played on it.comp.lang.python

[thanks to marco (zip(*...))  and antonio (lambda with default
arguments)]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[John Nagle]

On 4/4/2011 12:47 AM, Gregory Ewing wrote:

John Nagle wrote:


A tagged machine might make Python faster. You could have
unboxed ints and floats, yet still allow values of other types,
with the hardware tagging helping with dispatch. But it probably
wouldn't help all that much. It didn't in the LISP machines.


What might help more is having bytecodes that operate on
arrays of unboxed types -- numpy acceleration in hardware.


That sort of thing was popular in the era of the early
Cray machines.  Once superscalar CPUs were developed,
the overhead on tight inner loops went down, and several
iterations of a loop could be in the pipeline at one time,
if they didn't conflict.  Modern superscalar machines have
register renaming, so the same program-visible register on
two successive iterations can map to different registers within
the CPU, allowing two iterations of the same loop to execute
simultaneously.  This eliminates the need for loop unrolling and
Duff's device.

John Nagle

--
http://mail.python.org/mailman/listinfo/python-list

Re: XML header with lxml

[Chroma Key]

On 2011-04-04 18:54:40 +0200, Jabba Laci said:


I want to construct an XML file with lxml but I don't find how to add
the '' header.

from lxml import etree as ET

html = ET.Element("html")
print ET.tostring(html)


Add the "xml_declaration=True" as an argument of etree.tostring().

--
C-K

--
http://mail.python.org/mailman/listinfo/python-list

Re: integer multiplication

[casevh]

On Apr 4, 9:41 am, Terry Reedy  wrote:
> On 4/4/2011 1:51 AM, Paul Rubin wrote:
>
> > I didn't realize Python used Karatsuba.  The main issue is probably that
> > Python uses a straightforward portable C implementation that's not
> > terribly efficient,
>
> but relatively easy for a couple of people to maintain. For (C)Python 3,
> which no longer has a C int type, I believe changes were focused on
> making calculations with small integers almost as fast as in 2.x.
>
> (I believe that retaining two implementations internally was considered
> but rejected. Could be wrong.)
>
>  >If you look for the gmpy module, it gives you a way to use gmp from
>  >Python.  In crypto code (lots of 1024 bit modular exponentials) I think
>  >I found gmpy to be around 4x faster than Python longs.
>
> For specialized use, specialized gmpy is the way to go.
>
> I am curious how gmpy compares to 3.x ints (longs) with small number
> calculations like 3+5 or 3*5.
>
> --
> Terry Jan Reedy

(Disclaimer: I'm the current maintainer of gmpy.)

A quick comparison between native integers and gmpy.mpz() on Python
3.2, 64-bit Linux and gmpy 1.14.

For multiplication of single digit numbers, native integers are faster
by ~25%. The breakeven threshold for multiplication occurs around 12
digits and by 20 digits, gmpy is almost 2x faster.

I've made some improvements between gmpy 1.04 and 1.14 to decrease the
overhead for gmpy operations. The breakeven point for older versions
will be higher so if you are running performance critical code with
older versions of gmpy, I'd recommend upgrading to 1.14.

casevh




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: iterable argument required

[Νικόλαος Κούρας]

On 4 Απρ, 17:38, Mel  wrote:
> Íéêüëáïò Êïýñáò wrote:
> >> > iam getting the following error which i dont understand
>
> >> > **
> >> > 163         # insert guest comments into database if form was
> >> > submitted
> >> >   164         if "@" in mail and comment not in ("Ó÷ïëéÜóôå Þ ñùôÞóôå
> >> > ìå ó÷åôéêÜ", ""):
> >> >   165                 try:
> >> >   166                         cursor.execute( '''INSERT INTO
> >> > users(mail, comment) VALUES(%s, %s)''', (mail, comment) )
> >> > mail = None, comment = None
>
> >> > TypeError: iterable argument required
> >> >       args = ('iterable argument required',)
>
> > In my original question can you explain to me what the meaning of the
> > following error is?
>
> > 
> > mail = None, comment = None
> > TypeError: iterable argument required
> >       args = ('iterable argument required',)
> > *
>
> > Also i noticed that if i append a query string in the end of a url
> > with the varibble mail attached like
>
> >http://superhost.gr/hosting.html?mail=test
>
> > then page hosting.html does load without any problem.
>
> > If i remove the query string from the ned of the URL then i'am getting
> > the error message i posted.
>
> > So its not that the if condition is wrong but something happens with
> > the form variable 'mail' .
>
> My wild guess is that the trouble is in `if "@" in mail` .  You can only
> test somthing `in` something if the second thing is iterable.  So when
> you don't supply a value via `?mail=' -- maybe the code that sets the
> value of `mail` from the URL (code you don't show us) sets `mail=None`,
> your server-side code blows up.  Things would be simpler if you included
> a traceback in your error logging.
>
>         Mel.

I think you and Peter are right. Somehow mail variable which is taken
its value from a form field is assigned a None value.

Here is the hosting.html part of the code that include the form
variable mail





Σχολιάστε ή ρωτήστε με σχετικά







But as you can see it has a default value of "Ποιό είναι το email
σου?"
So how can be of a NoneType ?

>  Things would be simpler if you included a traceback in your error logging.

You mean the web server's last error log messages for the error.log
file?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Terry Reedy]

O


fix exactly, and there is always worry that permanance enhancements may
have unforseen side effects. I will let Raymond make the call on this.


/permanance/performance/, /unforseen/unforeseen/


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: [pyplot] using f1=figure(1)

[Giacomo Boffi]

"eryksun ()"  writes:

> figure(fig1.number)
> plot(...)

that's already much better than figure(1);...;figure(2);...

> Alternatively, you can use the plot methods of a particular axes:
>
> fig1 = figure()
> ax1 = axes()
> fig2 = figure()
> ax2 = axes()
>
> ax1.plot(...)
> ax2.plot(...)

that's nicer

> It works the same for subplots: [...]

thanks a lot
-- 
le mie sacrosante questioni di principio
  VS gli sciocchi puntigli di quel cretino del mio vicino
-- 
http://mail.python.org/mailman/listinfo/python-list

tkSimpleDialog.Dialog.ok calls Cancel

[Matt H]

I am subclassing tkSimpleDialog.Dialog as a (sqlite) database front-
end. This parent dialog opens a number of child dialogs to propagate
sub-tables (like sub-forms in OOo). These subforms write their input
into the database (INSERT...)

The parent dialog then either commits or rolls-back transactions based
on whether the user clicks ok or cancel. This ensures that if
something is written to the db in a subform, that it doesn't stay in
the database if the user chooses cancel (the rollback transaction code
is in cancel).

I have everything working except for the fact that python
tkSimpleDialog.ok calls tkSimpleDialog.cancel to clean up, which means
all of my transactions get cancelled (even if the user clicks ok). Is
there a way to do this without re-implementing or copying the
tkSimpleDialog.Dialog ok and cancel methods entirely?

In other words, how do I make OK not execute cancel?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: integer multiplication

[Terry Reedy]

On 4/4/2011 1:20 PM, geremy condra wrote:

On Mon, Apr 4, 2011 at 9:41 AM, Terry Reedy  wrote:



(I believe that retaining two implementations internally was considered but
rejected. Could be wrong.)


There are two implementations, grade school multiplication and
karatsuba, which kicks in after a given cutoff.


I meant internally retaining the 2.7 machine int and unbounded long types.


I am curious how gmpy compares to 3.x ints (longs) with small number
calculations like 3+5 or 3*5.


I have this data somewhere, if you're interested I'll try to dredge it up.


My question is whether gmpy ints could be a complete substitute for 3.x 
ints, or whether speed for bit (1000 digit) ints came at the expense of 
extra overhead making small int calculations slower. That is separate 
from the issue of whether gmpy ints implement the entire int interface, 
or whether they currently inter-operate with other types as well.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Terry Reedy]

On 4/4/2011 1:14 PM, Terry Reedy wrote:

On 4/4/2011 5:23 AM, Paul Rubin wrote:

Gregory Ewing writes:

What might help more is having bytecodes that operate on
arrays of unboxed types -- numpy acceleration in hardware.


That is an interesting idea as an array or functools module patch.
Basically a way to map or fold arbitrary functions over arrays, with a
few obvious optimizations to avoid refcount churning. It could have
helped with a number of things I've done over the years.


For map, I presume you are thinking of an array.map(func) in system code
(C for CPython) equivalent to

def map(self,func):
for i,ob in enumerate(self):
self[i] = func(ob)

The question is whether it would be enough faster. Of course, what would
really be needed for speed are wrapped system-coded funcs that map would
recognize and pass and received unboxed array units to and from. At that
point, we just about invented 1-D numpy ;-).

I have always thought the array was underutilized, but I see now that it
only offers Python code space saving at a cost of interconversion time.
To be really useful, arrays of unboxed data, like strings and bytes,
need system-coded functions that directly operate on the unboxed data,
like strings and bytes have. Array comes with a few, but very few,
generic sequence methods, like .count(x) (a special-case of reduction).


After posting this, I realized that ctypes makes it easy to find and 
wrap functions in a shared library as a Python object (possibly with 
parameter annotations) that could be passed to array.map, etc. No 
swigging needed, which is harder than writing simple C functions. So a 
small extension to array with .map, .filter, .reduce, and a wrapper 
class would be more useful than I thought.



--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Terry Reedy]

On 4/4/2011 9:35 AM, Lie Ryan wrote:

On 04/04/11 19:34, Antoon Pardon wrote:

On Fri, Apr 01, 2011 at 10:21:33PM -0400, Terry Reedy wrote:


rewriting cmp_to_key in C is underway

http://bugs.python.org/issue11707


Nice to know! Any chance this wil get into 2.7.x?


Python 2.7 still have list.sort(cmp=...)/sorted(cmp=...), so cmp_to_key
is not much use there. Just pass your comparison function to the cmp
argument.


.cmp_to_key was added to ease moving to 3.x or maintaining code usable 
on both. A faster version would encourage such use. But it is not a bug 
fix exactly, and there is always worry that permanance enhancements may 
have unforseen side effects. I will let Raymond make the call on this.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Terry Reedy]

On 4/4/2011 5:34 AM, Antoon Pardon wrote:

On Fri, Apr 01, 2011 at 10:21:33PM -0400, Terry Reedy wrote:


rewriting cmp_to_key in C is underway

http://bugs.python.org/issue11707


Nice to know! Any chance this wil get into 2.7.x?


I posted the question to the issue.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: integer multiplication

[geremy condra]

On Mon, Apr 4, 2011 at 9:41 AM, Terry Reedy  wrote:
> On 4/4/2011 1:51 AM, Paul Rubin wrote:
>
>> I didn't realize Python used Karatsuba.  The main issue is probably that
>> Python uses a straightforward portable C implementation that's not
>> terribly efficient,
>
> but relatively easy for a couple of people to maintain. For (C)Python 3,
> which no longer has a C int type, I believe changes were focused on making
> calculations with small integers almost as fast as in 2.x.
>
> (I believe that retaining two implementations internally was considered but
> rejected. Could be wrong.)

There are two implementations, grade school multiplication and
karatsuba, which kicks in after a given cutoff.

>>If you look for the gmpy module, it gives you a way to use gmp from
>>Python.  In crypto code (lots of 1024 bit modular exponentials) I think
>>I found gmpy to be around 4x faster than Python longs.
>
> For specialized use, specialized gmpy is the way to go.
>
> I am curious how gmpy compares to 3.x ints (longs) with small number
> calculations like 3+5 or 3*5.

I have this data somewhere, if you're interested I'll try to dredge it up.

Geremy Condra
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Terry Reedy]

On 4/4/2011 5:23 AM, Paul Rubin wrote:

Gregory Ewing  writes:

What might help more is having bytecodes that operate on
arrays of unboxed types -- numpy acceleration in hardware.


That is an interesting idea as an array or functools module patch.
Basically a way to map or fold arbitrary functions over arrays, with a
few obvious optimizations to avoid refcount churning.  It could have
helped with a number of things I've done over the years.


For map, I presume you are thinking of an array.map(func) in system code 
(C for CPython) equivalent to


def map(self,func):
  for i,ob in enumerate(self):
self[i] = func(ob)

The question is whether it would be enough faster. Of course, what would 
really be needed for speed are wrapped system-coded funcs that map would 
recognize and pass and received unboxed array units to and from. At that 
point, we just about invented 1-D numpy ;-).


I have always thought the array was underutilized, but I see now that it 
only offers Python code space saving at a cost of interconversion time. 
To be really useful, arrays of unboxed data, like strings and bytes, 
need system-coded functions that directly operate on the unboxed data, 
like strings and bytes have. Array comes with a few, but very few, 
generic sequence methods, like .count(x) (a special-case of reduction).


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

XML header with lxml

[Jabba Laci]

Hi,

I want to construct an XML file with lxml but I don't find how to add
the '' header.

from lxml import etree as ET

html = ET.Element("html")
print ET.tostring(html)

simply prints


Thanks,

Laszlo
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: integer multiplication

[Terry Reedy]

On 4/4/2011 1:51 AM, Paul Rubin wrote:


I didn't realize Python used Karatsuba.  The main issue is probably that
Python uses a straightforward portable C implementation that's not
terribly efficient,


but relatively easy for a couple of people to maintain. For (C)Python 3, 
which no longer has a C int type, I believe changes were focused on 
making calculations with small integers almost as fast as in 2.x.


(I believe that retaining two implementations internally was considered 
but rejected. Could be wrong.)


>If you look for the gmpy module, it gives you a way to use gmp from
>Python.  In crypto code (lots of 1024 bit modular exponentials) I think
>I found gmpy to be around 4x faster than Python longs.

For specialized use, specialized gmpy is the way to go.

I am curious how gmpy compares to 3.x ints (longs) with small number 
calculations like 3+5 or 3*5.



--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[geremy condra]

On Mon, Apr 4, 2011 at 12:47 AM, Gregory Ewing
 wrote:
> John Nagle wrote:
>
>>    A tagged machine might make Python faster.  You could have
>> unboxed ints and floats, yet still allow values of other types,
>> with the hardware tagging helping with dispatch.   But it probably
>> wouldn't help all that much.  It didn't in the LISP machines.
>
> What might help more is having bytecodes that operate on
> arrays of unboxed types -- numpy acceleration in hardware.

I'd be interested in seeing the performance impact of this, although I
wonder if it'd be feasible.

Geremy Condra
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie -> NameError: getopt

[R. Tyler Croy]

On Mon, 04 Apr 2011 05:14:46 -0700, Mustafa Cayci wrote:

> Hello,
> 
> I followed several postings in Google and came up with the beginning of
> the following code:
> During the execution, I am getting
> 
> Problem invoking WLST - Traceback (innermost last):
>   File "/home/oracle/wlsuserconfigfiles/./Health_Check_Servers.py", line
>   80, in
> ?
>   File "/home/oracle/wlsuserconfigfiles/./Health_Check_Servers.py", line
>   22, in
> main
> NameError: getopt
> 
> I thought I was importing the "getopt" as shown below.

Try bringing your imports up out of the "if __name__" block. Where you are 
referencing getopt (the main() function), the getopt module hasn't yet 
been imported.

You can do something like this:

import getopt

def main():
# ...


-- 
- R. Tyler Croy
--
Code: http://github.com/rtyler
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: proposal to allow to set the delimiter in str.format to something other than curly bracket

[Peter Otten]

Alia Khouri wrote:

> Terry Reedy wrote:
> 
>> Just double the brackets, just as one doubles '\\' to get '\' in a
>> string.
>>
>> >>> "class {0}Model {{ public bool IsModel(){{ returntrue;
>> ".format('My')
>>
>> 'class MyModel { public bool IsModel(){ returntrue; } }'
>>
> 
> Indeed, I tried that, but it means I have to double bracket all the
> csharp code which just creates more work for me.

You could automatically convert from a custom format like that in your 
original post:

import re

_lookup = {
"[[": "{",
"]]": "}",
"{": "{{",
"}": "}}",
}

def _substitute(m):
return _lookup[m.group()]

def custom_format(template, *args, **kw):
return (re.compile(r"\[\[|]]|\{|\}")
.sub(_substitute, template)
.format(*args, **kw))

code = "class [[0]]Model { public bool IsModel(){ return a[42] || true; } }"
print custom_format(code, "My")


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: better way to do this in python

[nn]

On Apr 3, 8:06 am, Mag Gam  wrote:
> Thanks for the responses.
>
> Basically, I have a large file with this format,
>
> Date INFO username command srcipaddress filename
>
> I would like to do statistics on:
> total number of usernames and who they are
> username and commands
> username and filenames
> unique source ip addresses
> unique filenames
>
> Then I would like to bucket findings with days (date).
>
> Overall, I would like to build a log file analyzer.
>
>
>
>
>
>
>
> On Sat, Apr 2, 2011 at 10:59 PM, Dan Stromberg  wrote:
>
> > On Sat, Apr 2, 2011 at 5:24 PM, Chris Angelico  wrote:
>
> >> On Sun, Apr 3, 2011 at 9:58 AM, Mag Gam  wrote:
> >> > I suppose I can do something like this.
> >> > (pseudocode)
>
> >> > d={}
> >> > try:
> >> >  d[key]+=1
> >> > except KeyError:
> >> >  d[key]=1
>
> >> > I was wondering if there is a pythonic way of doing this? I plan on
> >> > doing this many times for various files. Would the python collections
> >> > class be sufficient?
>
> >> I think you want collections.Counter. From the docs: "Counter objects
> >> have a dictionary interface except that they return a zero count for
> >> missing items instead of raising a KeyError".
>
> >> ChrisA
>
> > I realize you (Mag) asked for a Python solution, but since you mention
> > awk... you can also do this with "sort < input | uniq -c" - one line of
> > "code".  GNU sort doesn't use as nice an algorithm as a hashing-based
> > solution (like you'd probably use with Python), but for a sort, GNU sort's
> > quite good.
>
> > --
> >http://mail.python.org/mailman/listinfo/python-list

Take a look at:
http://code.activestate.com/recipes/577535-aggregates-using-groupby-defaultdict-and-counter/

for some ideas of how to group and count things.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: proposal to allow to set the delimiter in str.format to something other than curly bracket

[Alia Khouri]

Terry Reedy wrote:

> Just double the brackets, just as one doubles '\\' to get '\' in a string.
>
>  >>> "class {0}Model {{ public bool IsModel(){{ returntrue; ".format('My')
>
> 'class MyModel { public bool IsModel(){ returntrue; } }'
>

Indeed, I tried that, but it means I have to double bracket all the
csharp code which just creates more work for me.

Here is my solution (aka Cheetah ultra-light) ripped from
stdlib.string: it involves hacking the string.Template class so that
you get attribute lookups. I know it uses eval, and it's probably
highly insecure, but it's for one off batch jobs, and I hope it
doesn't hurt anyone:


import re

class _multimap:
"""Helper class for combining multiple mappings.

Used by .substitute() to combine the mapping and keyword
arguments.
"""
def __init__(self, primary, secondary):
self._primary = primary
self._secondary = secondary

def __getitem__(self, key):
try:
return self._primary[key]
except KeyError:
return self._secondary[key]


class _TemplateMetaclass(type):
pattern = r"""
%(delim)s(?:
  (?P%(delim)s) |   # Escape sequence of two delimiters
  (?P%(id)s)  |   # delimiter and a Python identifier
  {(?P%(id)s)}   |   # delimiter and a braced identifier
  (?P)  # Other ill-formed delimiter exprs
)
"""

def __init__(cls, name, bases, dct):
super(_TemplateMetaclass, cls).__init__(name, bases, dct)
if 'pattern' in dct:
pattern = cls.pattern
else:
pattern = _TemplateMetaclass.pattern % {
'delim' : re.escape(cls.delimiter),
'id': cls.idpattern,
}
cls.pattern = re.compile(pattern, re.IGNORECASE | re.VERBOSE)


class Template:
"""A string class for supporting $-substitutions."""
__metaclass__ = _TemplateMetaclass

delimiter = '$'
idpattern = r'[_a-z][_a-z0-9\.]*'

def __init__(self, template):
self.template = template

# Search for $$, $identifier, ${identifier}, and any bare $'s

def _invalid(self, mo):
i = mo.start('invalid')
lines = self.template[:i].splitlines(True)
if not lines:
colno = 1
lineno = 1
else:
colno = i - len(''.join(lines[:-1]))
lineno = len(lines)
raise ValueError('Invalid placeholder in string: line %d, col
%d' %
 (lineno, colno))

def substitute(self, *args, **kws):
if len(args) > 1:
raise TypeError('Too many positional arguments')
if not args:
mapping = kws
elif kws:
mapping = _multimap(kws, args[0])
else:
mapping = args[0]
# Helper function for .sub()
def convert(mo):
# Check the most common path first.
named = mo.group('named') or mo.group('braced')
if named is not None:
# We use this idiom instead of str() because the
latter will
# fail if val is a Unicode containing non-ASCII
characters.
#  here is the probably dangerous eval hack XX
if '.' in named:
return eval(named, kws)
else:
val = mapping[named]
return '%s' % (val,)

if mo.group('escaped') is not None:
return self.delimiter
if mo.group('invalid') is not None:
self._invalid(mo)
raise ValueError('Unrecognized named group in pattern',
 self.pattern)
return self.pattern.sub(convert, self.template)


def test_templates():
class P: pass
p = P()
p.name = 'ak'
txt = 'hello there ${o.name}'
t = Template(txt)
assert t.substitute(o=p) == 'hello there ak'

if __name__ == '__main__': test_templates()



-- 
http://mail.python.org/mailman/listinfo/python-list

Pydev 2.0 Released

[Fabio Zadrozny]

Hi All,

PyDev 2.0 has been released

Details on PyDev: http://pydev.org
Details on its development: http://pydev.blogspot.com

Release Highlights:
===

Major
-

* TDD actions on Ctrl+1

* Improved code coverage support

See video at: http://pydev.org/video_pydev_20.html with these improvements


Noteworthy


PyUnit

 * It's possible to pin a test run and restore it later.
 * Errors that occur while importing modules are properly shown.
 * It's possible to override the test runner configurations for a given launch.
 * The Nose test runner works properly when there's an error in a fixture.

Editor

 * When there's some text selected and ' or " is entered, the content
is converted to a string.
 * Handling literals with ui linking.
 * Creating ui link in the editor after entering (,[,{ when it is auto-closed.
 * On hover, when there's a name defined in another module, the
statement containing the name is shown.
 * It's possible to launch an editor with a file not in the workspace
(a project must be selected in this case)
 * If a line starts with __version__ no import is added above it.
 * When doing assign to attributes, if there's a pass in the line the
assign will be added, it's removed.
 * When Ctrl+1 is used to add an import on an unresolved variable, if
Ctrl is pressed on apply a local import is done.

Interactive console (options)

 * Focus on creation
 * When created the selection may be directly sent to the console

The DJANGO_SETTINGS_MODULE environment var is passed when making a launch.

The outline page now has a filter.

The input() method properly works in Python 3.2 (last "\r" no longer shown).

**LOTS** of other adjustments and bug fixes



What is PyDev?
---

PyDev is a plugin that enables users to use Eclipse for Python, Jython
and IronPython development -- making Eclipse a first class Python IDE
-- It comes with many goodies such as code completion, syntax
highlighting, syntax analysis, refactor, debug and many others.


Cheers,

-- 
Fabio Zadrozny
--
Software Developer

Appcelerator
http://appcelerator.com/

Aptana
http://aptana.com/

PyDev - Python Development Environment for Eclipse
http://pydev.org
http://pydev.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: string to path problem

[ecu_jon]

On Apr 4, 5:06 am, Kushal Kumaran 
wrote:
> On Mon, Apr 4, 2011 at 9:48 AM, ecu_jon  wrote:
> > On Apr 4, 12:17 am, Chris Rebert  wrote:
> >> On Sun, Apr 3, 2011 at 8:30 PM, ecu_jon  wrote:
> >> > i am writing a basic backup program for my school. so they wanted the
> >> > possibility to be able to set source/destination from a config file.
> >> > my source/destination was fine before, i would build it up with
> >> > functions, like 1 that got the user-name, and put it all together with
> >> > os.path.join. but if they set a source in the config file to something
> >> > like c:\users\jon\backup  python tries to read from c:\\users\\jon\
> >> > \backup, and throws out a read permission (because it doesn't
> >> > exist ...).
>
> >> Please give the exact error message and full exception traceback that
> >> you're getting.
>
> >> Cheers,
> >> Chris
> > Traceback (most recent call last):
> >  File "I:\college\spring11\capstone-project\testing1.py", line 39, in
> > 
> >    shutil.copy2(source1, destination)
> >  File "C:\Python27\lib\shutil.py", line 127, in copy2
> >    copyfile(src, dst)
> >  File "C:\Python27\lib\shutil.py", line 81, in copyfile
> >    with open(src, 'rb') as fsrc:
> > IOError: [Errno 13] Permission denied: 'c:\\users\\jon\\backup'
>
> > i have permission to c:\users\jon\*
> > but c:\\* obviously does not exist.
>
> The extra backslashes in the string literal are there to "escape" the
> required backslashes.  This is required because the backslash
> character is used to introduce certain special characters in strings,
> such as tabs and newlines.  The actual string does not contain the
> extra backslashes.  This is documented in extensive detail in the
> language 
> reference:http://docs.python.org/reference/lexical_analysis.html#string-literals.
>  But you might want to start with the 
> tutorial:http://docs.python.org/tutorial/introduction.html#strings
>
> Example:
>
>
>
> >>> s = 'c:\\users\\jon\\backup'
> >>> print s
> c:\users\jon\backup
>
> Is c:\users\jon\backup a directory?  The shutil.copyfile function will
> only copy a file.  There is a shutil.copytree that will copy an entire
> directory tree.
>
> --
> regards,
> kushal

well i changed a few minor things in the bigger problem, and c:\users
\jon\backup as the source worked fine. now to test it on winxp, where
the default has a space in the name.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: iterable argument required

[Mel]

Νικόλαος Κούρας wrote:

>> > iam getting the following error which i dont understand
>>
>> > **
>> > 163         # insert guest comments into database if form was
>> > submitted
>> >   164         if "@" in mail and comment not in ("Σχολιάστε ή ρωτήστε
>> > με σχετικά", ""):
>> >   165                 try:
>> >   166                         cursor.execute( '''INSERT INTO
>> > users(mail, comment) VALUES(%s, %s)''', (mail, comment) )
>> > mail = None, comment = None
>>
>> > TypeError: iterable argument required
>> >       args = ('iterable argument required',)
>>

> In my original question can you explain to me what the meaning of the
> following error is?
>
> 
> mail = None, comment = None
> TypeError: iterable argument required
>   args = ('iterable argument required',)
> *
>
> Also i noticed that if i append a query string in the end of a url
> with the varibble mail attached like
>
> http://superhost.gr/hosting.html?mail=test
>
> then page hosting.html does load without any problem.
>
> If i remove the query string from the ned of the URL then i'am getting
> the error message i posted.
>
> So its not that the if condition is wrong but something happens with
> the form variable 'mail' .

My wild guess is that the trouble is in `if "@" in mail` .  You can only
test somthing `in` something if the second thing is iterable.  So when
you don't supply a value via `?mail=' -- maybe the code that sets the
value of `mail` from the URL (code you don't show us) sets `mail=None`,
your server-side code blows up.  Things would be simpler if you included
a traceback in your error logging.

Mel.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: iterable argument required

[Peter Otten]

Νικόλαος Κούρας wrote:

>> if "@" in mail and comment not in INVALID_COMMENTS:

> In my original question can you explain to me what the meaning of the
> following error is?
> 
> 
> mail = None, comment = None
> TypeError: iterable argument required
> args = ('iterable argument required',)
> *

That's not the standard format for a traceback in Python and you don't 
provide enough context like: 

- your python version
- the framework that produces the non-standard traceback
- a significant portion of your code

That makes it harder than necessary to find out what's going on.

Python 2.4.6 (#2, Jan 21 2010, 23:45:25)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> "@" in None
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: iterable argument required

So "mail = None" means exactly that, you somehow assigned None to the mail 
variable.

Note that newer Python versions give a slightly improved error message:

$ python2.5 -c '"@" in None'
Traceback (most recent call last):
  File "", line 1, in 
TypeError: argument of type 'NoneType' is not iterable

Background: the 'in' operator tries hard to produce a meaningful result 
before it gives up:

>>> class A:
... def __getattr__(self, name):
... print name
... raise AttributeError
...
>>> 42 in A()
__contains__
__iter__
__getitem__
Traceback (most recent call last):
  File "", line 1, in 
TypeError: argument of type 'instance' is not iterable

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using python to post data to a form

[Littlefield, Tyler]

>Sending POST data can be done as follows (I'm changing bar=foo to
Thanks for this, and the links.

On 4/4/2011 12:24 AM, Chris Rebert wrote:

On Sun, Apr 3, 2011 at 10:36 PM, Littlefield, Tyler  wrote:

Hello:
I have some data that needs to be fed through a html form to get validated
and processed and the like. How can I use python to send data through that
form, given a specific url? the form says it uses post, but I"m not really
sure what the difference is.

They're different HTTP request methods:
http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_methods

The key upshot in this case is that GET requests place the parameters
in the URL itself, whereas POST requests place them in the body of the
request.


would it just be:
http://mysite.com/bla.php?foo=bar&bar=foo?

No, that would be using GET.


If so, how do I do that with python?

Sending POST data can be done as follows (I'm changing bar=foo to
bar=qux for greater clarity):

from urllib import urlopen, urlencode

form_data = {'foo' : 'bar', 'bar' : 'qux'}
encoded_data = urlencode(form_data)
try:
 # 2nd argument to urlopen() is the POST data to send, if any
 f = urlopen('http://mysite.com/bla.php', encoded_data)
 result = f.read()
finally:
 f.close()

Relevant docs:
http://docs.python.org/library/urllib.html

Cheers,
Chris
--
http://blog.rebertia.com




--

Thanks,
Ty

--
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: iterable argument required

[Νικόλαος Κούρας]

On 3 Απρ, 17:01, "eryksun ()"  wrote:
> On Saturday, April 2, 2011 12:26:18 PM UTC-4, Νικόλαος Κούρας wrote:
> > Hello, after inserting this line if "@" in mail and comment not in
> > ("Σχολιάστε ή ρωτήστε με σχετικά", ""):
>
> > iam getting the following error which i dont understand
>
> > **
> > 163         # insert guest comments into database if form was
> > submitted
> >   164         if "@" in mail and comment not in ("Σχολιάστε ή ρωτήστε
> > με σχετικά", ""):
> >   165                 try:
> >   166                         cursor.execute( '''INSERT INTO
> > users(mail, comment) VALUES(%s, %s)''', (mail, comment) )
> > mail = None, comment = None
>
> > TypeError: iterable argument required
> >       args = ('iterable argument required',)
>
> Here's how I parse what you've written so far:
>
> INVALID_COMMENTS = ("Σχολιάστε ή ρωτήστεμε σχετικά", "")
> SQL_COMMENT_FORM = "INSERT INTO users(mail, comment) VALUES(%s, %s)"
>
> # insert guest comments into database if form was submitted
> mail = form.getvalue('mail')
> comment = form.getvalue('comment')
> if "@" in mail and comment not in INVALID_COMMENTS:
>     try:        
>         cursor.execute(SQL_COMMENT_FORM % (mail, comment))
>     except MySQLdb.Error as e:
>         print("Error %d: %s" % (e.args[0], e.args[1]))
>     else:
>         mail = comment = None

Whats was the problem as i have written it?
Your solution is the same as mine except the fact that you assign
values and statements into variables.
I tried it but iam getting an Internal Server Error.

In my original question can you explain to me what the meaning of the
following error is?


mail = None, comment = None
TypeError: iterable argument required
  args = ('iterable argument required',)
*

Also i noticed that if i append a query string in the end of a url
with the varibble mail attached like

http://superhost.gr/hosting.html?mail=test

then page hosting.html does load without any problem.

If i remove the query string from the ned of the URL then i'am getting
the error message i posted.

So its not that the if condition is wrong but something happens with
the form variable 'mail' .
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Lie Ryan]

On 04/04/11 19:34, Antoon Pardon wrote:
> On Fri, Apr 01, 2011 at 10:21:33PM -0400, Terry Reedy wrote:
>>
>> rewriting cmp_to_key in C is underway
>>
>> http://bugs.python.org/issue11707
>>
> Nice to know! Any chance this wil get into 2.7.x?

Python 2.7 still have list.sort(cmp=...)/sorted(cmp=...), so cmp_to_key
is not much use there. Just pass your comparison function to the cmp
argument.
-- 
http://mail.python.org/mailman/listinfo/python-list

PyThreadState_Swap crash

[Wiktor Adamski]

I have 2 threads in C code using python 2.5.2. First thread creates
new interpreter (i need several interpreters but those 2 threads use
only one) like that:

PyEval_AcquireLock();
threadState = Py_NewInterpreter();
PyThreadState_Swap(threadState);

// calling python API

PyThreadState_Swap(NULL);
PyEval_ReleaseLock();

Second thread uses interpreter created in first thread:

PyEval_AcquireLock();
PyThreadState_Swap(threadState);

and sometimes PyThreadState_Swap crashes in debug build
(PyGILState_GetThisThreadState() returns garbage). In release build
that code doesn't run and so far no other problem was found.
I call PyEval_InitThreads() at the begining of program and every
PyEval_AcquireLock() has PyEval_ReleaseLock().

Am I allowed to use the same threadState in different threads?
If I am, is there another problem in my code?
Or maybe it's a bug in python - acording to documentation "Python
still supports the creation of additional interpreters (using
Py_NewInterpreter()), but mixing multiple interpreters and the
PyGILState_*() API is unsupported." - I don't use PyGILState_ but it's
used internally in PyThreadState_Swap(). I also don't use
PyEval_RestoreThread() - comment sugests that crashing code is present
because possibility of calling from PyEval_RestoreThread().
-- 
http://mail.python.org/mailman/listinfo/python-list

Newbie -> NameError: getopt

[Mustafa Cayci]

Hello,

I followed several postings in Google and came up with the beginning of the 
following code:
During the execution, I am getting 

Problem invoking WLST - Traceback (innermost last):
  File "/home/oracle/wlsuserconfigfiles/./Health_Check_Servers.py", line 80, in 
?
  File "/home/oracle/wlsuserconfigfiles/./Health_Check_Servers.py", line 22, in 
main
NameError: getopt

I thought I was importing the "getopt" as shown below.

Thanks for you help in advance,
Mustafa



#Conditionally import wlstModule only when script is executed with jython
if __name__ == '__main__':

from wlstModule import *#@UnusedWildImport
import sys
import os
from java.lang import System
import getopt

serverName = ''
portNumber = ''
ucf = ''
ukf = ''
adminURL = ''
protocolName = ''
portNumber = ''

def main(argv):

try:
opts, args = getopt.getopt(sys.argv[1:], "s:p:t:u:k:")
except getopt.GetoptError, err:
print str(err)
usage()
sys.exit(2)
...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using python to post data to a form

[Corey Richardson]

On 04/04/2011 01:36 AM, Littlefield, Tyler wrote:
> Hello:
> I have some data that needs to be fed through a html form to get 
> validated and processed and the like. How can I use python to send data 
> through that form, given a specific url? the form says it uses post, but 
> I"m not really sure what the difference is. would it just be:
> http://mysite.com/bla.php?foo=bar&bar=foo?
> If so, how do I do that with python?
> 

import urllib
import urllib2

url = "http://www.foo.com/";
data = {"name": "Guido", "status": "BDFL"}

data = urllib.urlencode(data)
request = urllib2.Request(url, data)
response = urllib2.urlopen(request)

page = response.read()

So yeah, passing in a Request object to urlopen that has some
urlencode'ed data in it.
-- 
Corey Richardson
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Guido rethinking removal of cmp from sort method

[Antoon Pardon]

On Fri, Apr 01, 2011 at 10:21:33PM -0400, Terry Reedy wrote:
> 
> rewriting cmp_to_key in C is underway
> 
> http://bugs.python.org/issue11707
> 
Nice to know! Any chance this wil get into 2.7.x?

-- 
Antoon Pardon
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Paul Rubin]

Gregory Ewing  writes:
> What might help more is having bytecodes that operate on
> arrays of unboxed types -- numpy acceleration in hardware.

That is an interesting idea as an array or functools module patch.
Basically a way to map or fold arbitrary functions over arrays, with a
few obvious optimizations to avoid refcount churning.  It could have
helped with a number of things I've done over the years.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: string to path problem

[Kushal Kumaran]

On Mon, Apr 4, 2011 at 9:48 AM, ecu_jon  wrote:
> On Apr 4, 12:17 am, Chris Rebert  wrote:
>> On Sun, Apr 3, 2011 at 8:30 PM, ecu_jon  wrote:
>> > i am writing a basic backup program for my school. so they wanted the
>> > possibility to be able to set source/destination from a config file.
>> > my source/destination was fine before, i would build it up with
>> > functions, like 1 that got the user-name, and put it all together with
>> > os.path.join. but if they set a source in the config file to something
>> > like c:\users\jon\backup  python tries to read from c:\\users\\jon\
>> > \backup, and throws out a read permission (because it doesn't
>> > exist ...).
>>
>> Please give the exact error message and full exception traceback that
>> you're getting.
>>
>> Cheers,
>> Chris
> Traceback (most recent call last):
>  File "I:\college\spring11\capstone-project\testing1.py", line 39, in
> 
>    shutil.copy2(source1, destination)
>  File "C:\Python27\lib\shutil.py", line 127, in copy2
>    copyfile(src, dst)
>  File "C:\Python27\lib\shutil.py", line 81, in copyfile
>    with open(src, 'rb') as fsrc:
> IOError: [Errno 13] Permission denied: 'c:\\users\\jon\\backup'
>
> i have permission to c:\users\jon\*
> but c:\\* obviously does not exist.

The extra backslashes in the string literal are there to "escape" the
required backslashes.  This is required because the backslash
character is used to introduce certain special characters in strings,
such as tabs and newlines.  The actual string does not contain the
extra backslashes.  This is documented in extensive detail in the
language reference:
http://docs.python.org/reference/lexical_analysis.html#string-literals.
 But you might want to start with the tutorial:
http://docs.python.org/tutorial/introduction.html#strings

Example:

>>> s = 'c:\\users\\jon\\backup'
>>> print s
c:\users\jon\backup
>>>

Is c:\users\jon\backup a directory?  The shutil.copyfile function will
only copy a file.  There is a shutil.copytree that will copy an entire
directory tree.

-- 
regards,
kushal
-- 
http://mail.python.org/mailman/listinfo/python-list

Have a nice day.

[Ashraf Ali]

Hello friends.
 What are you looking for. You can find everything
what you want.
just visit: www.hothitsbollywood.blogspot.com
www.aishwaryaraismile.blogspot.com
www.bollywoodhotpik.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Gregory Ewing]

John Nagle wrote:


A tagged machine might make Python faster.  You could have
unboxed ints and floats, yet still allow values of other types,
with the hardware tagging helping with dispatch.   But it probably
wouldn't help all that much.  It didn't in the LISP machines.


What might help more is having bytecodes that operate on
arrays of unboxed types -- numpy acceleration in hardware.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python CPU

[Gregory Ewing]

Paul Rubin wrote:


You can order 144-core Forth chips right now,

   http://greenarrays.com/home/products/index.html

They are asynchronous cores running at around 700 mhz, so you get an
astounding amount of raw compute power per watt and per dollar.  But for
me at least, it's not that easy to figure out applications where their
weird architecture fits well.


Hmmm... Maybe compiling Python to Forth would make sense?-)

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list