ANN: wxPython 2.8.9.2 release

[Robin Dunn]

The wxWidgets team is in the early stages of preparing for a 2.8.10
release, but I already had a set of 2.8.9.2 release candidate files
that I made a few days ago.  Since it's still possible that there
could be delays in the 2.8.10 release I thought that it would be nice
to go ahead and release the 2.8.9.2 binaries.  So...


Announcing
--

The 2.8.9.2 release of wxPython is now available for download at
http://wxpython.org/download.php.  This release adds the wx.lib.agw
package, adds an event watcher to the widget inspection tool, and
fixes a bunch of bugs.  A summary of changes is listed below and also
at http://wxpython.org/recentchanges.php.

Source code is available as a tarball and a source RPM, as well as
binaries for Python 2.4, 2.5 and 2.6[1], for Windows and Mac, as well
some packages for various Linux distributions.


[1] If installing the Python 2.6 version of wxPython on 64-bit XP or
Vista then please read the README presented by the installer for
instructions on how to enable the themed controls.  (If anybody has a
better solution for this please let Robin know.)



What is wxPython?
-

wxPython is a GUI toolkit for the Python programming language. It
allows Python programmers to create programs with a robust, highly
functional graphical user interface, simply and easily. It is
implemented as a Python extension module that wraps the GUI components
of the popular wxWidgets cross platform library, which is written in
C++.

wxPython is a cross-platform toolkit. This means that the same program
will usually run on multiple platforms without modifications.
Currently supported platforms are 32-bit and 64-bit Microsoft Windows,
most Linux or other Unix-like systems using GTK2, and Mac OS X 10.4+.
In most cases the native widgets are used on each platform to provide
a 100% native look and feel for the application.



Changes in 2.8.9.2
--

Added the wx.lib.agw package, which contiains most of the widgets from
http://xoomer.alice.it/infinity77/main/freeware.html written by Andrea
Gavana.  Andrea's widgets that were already in wx.lib were also moved
to the wx.lib.agw package, with a small stub module left in wx.lib.
As part of this addition the demo framework was given the ability to
load demo modules from a sub-folder of the demo directory, to make it
easier to maintain collections of demo samples as a group.

Added the wx.PyPickerBase class which can be used to derive new picker
classes in Python.  Used it to implement a color picker for Mac that
uses a wx.BitmapButton instead of a normal wx.Button.  This makes the
color picker look and behave lots better on Mac than before.

You can now pass the handler function to the Unbind method.  If it is
given then Unbind will only disconenct the event handler that uses the
same handler function, so if there are multiple bindings for the same
event type you'll now be able to selectively unbind specific
instances.

Added a new tool to the Widget Inspection Tool that allows you to watch
the events passing through a widget.  It can also be used
independently, see wx.lib.eventwatcher.



--
Robin Dunn
Software Craftsman
http://wxPython.org  Java give you jitters?  Relax with wxPython!

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Scott David Daniels]

Maxim Khitrov wrote:

... Here's the function that I'll be using from now on. It gives me
exactly the behavior I need, with an int initializer being treated as
array size. Still not as efficient as it could be if supported
natively by array (one malloc instead of two + memmove + extra
function call), but very good performance nevertheless:

from array import array as _array
array_null = dict((tc, '\0' * _array(tc).itemsize) for tc in 'cbBuhHiIlLfd')

How about:
  array_null = dict((tc, _array(tc, (0,)).tostring() for tc in 
'cbBuhHiIlLfd')

...
(some ancient floating points did not use all-0 bytes for 0.0).

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: CSV readers and UTF-8 files

[Chris Rebert]

On Thu, Feb 19, 2009 at 8:11 AM, Falcolas  wrote:
> On Feb 19, 7:21 am, mk  wrote:
>> Hello everyone,
>>
>> Is it just me or CSV reader/DictReader and UTF-8 files do not work
>> correctly in Python 2.6.1 (Windows)?
>
> I would point out in the CSV module documentation (http://
> docs.python.org/library/csv.html) it explicitly mentions that it can't
> handle unicode.
>
> You can use their workaround in the examples section for UTF-8, or
> with another form of encoding (I used MIME) for UTF-16.
>
> ~G

This really ought to be fixed for 3.0+ (seems to still be ASCII-only
according to the 3.0 docs...)

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[Steven D'Aprano]

andrew cooke wrote:

> 
> i wonder what fraction of people posting with "bug?" in their titles here
> actually find bugs?

About 99.99%.

Unfortunately, 99.98% have found bugs in their code, not in Python.


-- 
Steven

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steven D'Aprano]

Steve Holden wrote:

> What Guido doesn't seem to have accepted yet is that slowing [C]Python
> down by 50% on a single-processor CPU will actually be a worthwhile
> tradeoff in ten years time, when nothing will have less than eight cores
> and the big boys will be running at 64 kilo-cores.

Ten years?

There's no guarantee that Python will still even be around in ten years. It
probably will be, I see no reason why it won't, but who knows? Maybe we'll
have mandatory software warranties tailored to suit the Microsofts and
Apples, and Guido and the PSF will be forced to abandon the language.

I think a design mistake would be to hamstring Python now for a hypothetical
benefit in a decade. But, well, in five years time, or three? Don't know.


-- 
Steven

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steven D'Aprano]

Paul Rubin wrote:

> How old is your computer, why did you buy it, and is it the first one
> you ever owned?
> 
> For most of us, I suspect, it is not our first one, and we bought it
> to get a processing speedup relative to the previous one. 

My computer is about eight months old, and I bought it because the previous
one died.


> If such 
> speedups were useless or unimportant, we would not have blown our hard
> earned cash replacing perfectly good older hardware, 

Oh the assumptions in that statement...

"Blowing hard-earned cash" assumes that people buy computers only when they
need to. That's certainly not true -- there's a lot of irrational purchases
involved. I have a friend who has just recently spent $2000 on storage so
he can store more games and videos, which he cheerfully admits he'll never
play or watch. He describes it as his "dragon's horde": it is for knowing
it's there, not for using.

Often hardware is upgraded because it's broken, or because you can't get
parts, or because the software you need will only run on newer machines. I
say *newer* rather than *faster*, because speed is only sometimes a factor
in why software won't run on old machines. My old Mac running a 68030 in
1990 ran Microsoft Word perfectly fast enough for even advanced word
processing needs, and nearly twenty years later, there's nothing I need
from a word processor that I couldn't do in 1990.


> so we have to 
> accept the concept that speed matters and ignore those platitudes that
> say otherwise.

The *perception* that speed matters, matters. The reality is that the
majority of computing tasks outside of certain specialist niches are I/O
bound, not CPU. Office server software is rarely CPU bound, and when it is,
in my experience there's one rogue process using all the CPU: the software
is broken, and a faster CPU would just let it be broken at a faster speed.

Gamers need better graphics cards and more memory, not faster CPUs. Internet
surfers need faster ethernet, more bandwidth and more memory, not faster
CPUs. Graphics designers need bigger hard drives and more memory, not
faster CPUs. (Hmm. There seems to be a pattern there...)

Of course, there are a few niches that do require faster CPUs: video
editing, some (but by no means all) Photoshop filters, number crunching,
etc. But even for them, you can often get more bang-for-your-buck
performance increase by adding more memory.

Speaking for myself, I'd happily take a 20% slower CPU for more reliable,
faster DVD/CD burning. What do I care if it takes my computer 120ms to open
a window instead of 100ms, but I care a lot if it takes me 10 minutes to
make a coaster instead of 7 minutes to make a good disc.



-- 
Steven

--
http://mail.python.org/mailman/listinfo/python-list

Re: Levenshtein word comparison -performance issue

[S.Selvam Siva]

On Sat, Feb 14, 2009 at 3:01 PM, Peter Otten <__pete...@web.de> wrote:

> Gabriel Genellina wrote:
>
> > En Fri, 13 Feb 2009 08:16:00 -0200, S.Selvam Siva <
> s.selvams...@gmail.com>
> > escribió:
> >
> >> I need some help.
> >> I tried to find top n(eg. 5) similar words for a given word, from a
> >> dictionary of 50,000 words.
> >> I used python-levenshtein module,and sample code is as follow.
> >>
> >> def foo(searchword):
> >> disdict={}
> >> for word in self.dictionary-words:
> >>distance=Levenshtein.ratio(searchword,word)
> >>disdict[word]=distance
> >> """
> >>  sort the disdict dictionary by values in descending order
> >> """
> >> similarwords=sorted(disdict, key=disdict.__getitem__, reverse=True)
> >>
> >> return similarwords[:5]
> >
> > You may replace the last steps (sort + slice top 5) by heapq.nlargest -
> at
> > least you won't waste time sorting 49995 irrelevant words...
> > Anyway you should measure the time taken by the first part (Levenshtein),
> > it may be the most demanding. I think there is a C extension for this,
> > should be much faster than pure Python calculations.
> >
>
> [I didn't see the original post]
>
> You can use the distance instead of the ratio and put the words into bins
> of
> the same length. Then if you find enough words with a distance <= 1 in the
> bin with the same length as the search word you can stop looking.
>
> You might be able to generalize this approach to other properties that are
> fast to calculate and guarantee a minimum distance, e. g. set(word).
>
> Peter
> --
> http://mail.python.org/mailman/listinfo/python-list
>


Thank you all for your response,

[sorry,I was away for a while.]
I used functools,heapq modules but that helped me a little,
then i categorized the words depending on the length and
compares with a small set(each set 5/4=12,500),
so now its taking quarter of time as compared to older method.

Further, can i use Thread to achieve parallel comparison ?,as i have little
knowledge on python-thread.
Will the following code achive parallelism?

thread1= threading.Thread(target=self.findsimilar,
args=("1",searchword,dic-word-set1)
   thread2= threading.Thread(target=self.findsimilar,
args=("2",searchword,dic-word-set1)
   thread3= threading.Thread(target=self.findsimilar,
args=("3",searchword,dic-word-set1)
   thread1.start()
   thread2.start()
   thread3.start()
   thread1.join()
   thread2.join()
   thread3.join()

I would like to hear suggestion.
Note:The issue is i am developing spell checker for my local languge,i may
use more than 2.5 lakh words,so i need to have a best way to find out
alternative wordlist
-- 
Yours,
S.Selvam
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steven D'Aprano]

Steve Holden wrote:

>> It's only an error if it gets in the way. It's the experience of a lot
>> of programmers that it doesn't, so it's not an error.
>> 
> And it's not a feature of the language, rather of one or two
> implementations. Neither JPython not IronPython use a GIL to the best of
> my knowledge, so you are still quite at liberty to use them.

I found this interesting benchmark on the relative speeds of CPython 2.3,
IronPython 0.1 and Jython 2.1. It's from six years ago, so not exactly
reporting on the state of the art, but it suggests to me that IronPython is
faster at the fundamentals but much slower at some things.

http://www.python.org/~jeremy/weblog/031209a.html


-- 
Steven

--
http://mail.python.org/mailman/listinfo/python-list

Re: Which Version of wxPython for Win XP

[Tim Roberts]

"W. eWatson"  wrote:
>
>I'm going to try out wxPython 2.8.92 for py25. It seems like the ansi 
>version is the choice for me. The other choice has unicode. Do I care?

Well, there's a subtle point that gives me an opportunity to point out a
lesser-known "feature" of the NT-based systems (XP, Vista, etc).

The core Win32 APIs on the NT-based systems are all Unicode.  When you call
an ANSI API, the system converts all of your ANSI strings to Unicode, then
calls the Unicode API, then converts any output strings back to ANSI.

So, there is, actually, a (small) performance advantage in using the
Unicode APIs instead of the ANSI APIs.

Unless you need your wx programs to run on Windows 98, I recommend you use
the Unicode version of wxPython.
-- 
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list

py test

[ssd]

test

b 


--
http://mail.python.org/mailman/listinfo/python-list

Re: Keeping the Console Open with IDLE

[Matimus]

On Feb 19, 8:06 pm, "W. eWatson"  wrote:
> I'm using IDLE for editing, but execute programs directly. If there are
> execution or "compile" errors, the console closes before I can see what it
> contains. How do I prevent that?
> --
>                                 W. eWatson
>
>               (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
>                Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet
>
>                      Web Page: 

Open a console window and type in the name of the script rather than
just double clicking on it. Or, you can terminate your script with a
'raw_input("press enter to quit")'.

Matt
--
http://mail.python.org/mailman/listinfo/python-list

Re: "metaclass conflict" error: where is noconflict ?

[Michele Simionato]

On Feb 19, 9:58 pm, Chris Rebert  wrote:
> On Thu, Feb 19, 2009 at 5:01 AM, Barak, Ron  wrote:
> > Hi,
>
> > I have a class derived from two parents (in blue below), which gives me the
> > following error:
>
> > $ python -u ./failover_pickle_demo09.py
> > Traceback (most recent call last):
> >   File "./failover_pickle_demo09.py", line 291, in 
> >     class ListControl(wx.Frame, CopyAndPaste):
> > TypeError: Error when calling the metaclass bases
> >     metaclass conflict: the metaclass of a derived class must be a
> > (non-strict) subclass of the metaclasses of all its bases
> > Googling suggested I should add
> > from noconflict import classmaker
> > and
>
> > __metaclass__=classmaker()
> > to this class.
>
> > However, I don't seem able to find where to get the noconflict module from.
>
> > Do any of you where noconflict  could be downloaded/installed from ?


It refers to this cookbook recipe:
http://code.activestate.com/recipes/204197/

See also the printed cookbook:

http://books.google.it/books?id=1Shx_VXS6ioC&pg=PA786&lpg=PA786&dq=python+cookbook+noconflict&source=bl&ots=BB413AZ8R7&sig=cnAB-E9rNFolHBEZQTIm_d4Mj3o&hl=it&ei=GT2eSfCtBdWa_gadppHYCw&sa=X&oi=book_result&resnum=2&ct=result#PPA786,M1

However, there is no module to download, and this is on purpose:
instead of blindly apply a "fix" one should
understand what the conflict is: then it is quite easy to
solve it by hand. The recipe discussion explains it all.
--
http://mail.python.org/mailman/listinfo/python-list

Keeping the Console Open with IDLE

[W. eWatson]

I'm using IDLE for editing, but execute programs directly. If there are 
execution or "compile" errors, the console closes before I can see what it 
contains. How do I prevent that?

--
   W. eWatson

 (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
  Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet

Web Page: 

--
http://mail.python.org/mailman/listinfo/python-list

Re: Which Version of wxPython for Win XP

[W. eWatson]

Mike Driscoll wrote:

On Feb 19, 11:29 am, "W. eWatson"  wrote:

W. eWatson wrote:

eric_dex...@msn.com wrote:

On Feb 19, 8:22 am, "W. eWatson"  wrote:

I'm going to try out wxPython 2.8.92 for py25. It seems like the ansi
version is the choice for me. The other choice has unicode. Do I care?
--
W. eWatson
  (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
   Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet
 Web Page: 

There is a good chance you don't care...  I think the other is for
internationalization...  best to look..  It would be easier to answer
the question if you said what you are going to do with it, and who
needed to use your software.  It's even possible that you might want
to try pygame depending on what you want to use it for and who the
audience is (and how good thier computers are)
http://pypi.python.org/simple/Dex%20Tracker/

Thanks. I think I'll take the chance. Somewhat simple programming.
Off I go to install it. It can always be un-installed.

Well, that was an interesting experience. It appears to have recompiled a
lot of stuff, and probably some libraries. IDLE is still with me as the way
to open py files. Apparently, if I want to get around IDLE, I may have to
install some other editor.

--
W. eWatson



The compiling you are referring to is just making the py files into
pyc files. This is normal and is a speed improvement. In fact, a lot
of the scripts you write will create a pyc file when first run.

In other words, wxPython does not affect your Python install in any
way other than making itself available as a 3rd party package (i.e.
adding itself to the path), just like any other good 3rd party
package. I'm not sure why you're even talking about IDLE...wxPython is
a GUI toolkit, not an IDE.

Mike
Thanks. Yes, I finally discovered that the need here was not what I wanted 
afterall. Un-installed now.


--
   W. eWatson

 (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
  Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet

Web Page: 

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Tim Roberts]

Paul Rubin  wrote:
>
>C is basically never appropriate.  C should be outlawed by Congress
>with the ban enforced by roving pie-throwing squads .

One of my favorite quotes:

   The last good thing written in C was Schubert's Ninth Symphony.
-- 
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list

Matplotlib change xticks and retain xticks changing during zoom

[dean . wheatley]

Hello,

I execute the following code:

try:
from math import *
import pylab as p
except:
print "Couldn't import all dependent libraries"
sys.exit()
dataLength = 100
data = [sin(2*pi*x/dataLength) for x in range(0,dataLength)]
p.plot(data)
p.show()


This produces a figure window. When I zoom in regions of the plot, the
xticks change correctly to the zoomed region.

This is in contrast to the effect of the following code:

try:
from math import *
import pylab as p
except:
print "Couldn't import all dependent libraries"
sys.exit()
dataLength = 100
data = [sin(2*pi*x/dataLength) for x in range(0,dataLength)]
p.plot(data)
newXTicks= [str(x/2.0) for x in p.xticks()[0]]
p.xticks(p.xticks()[0], newXTicks)
p.show()

This code produces tick marks [0, 10, 20, 30, 40, 50]. However when
zooming, the xtick marks do not adjust correctly to the zoomed
regions.


Your help on this issue would be great, thanks!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Maxim Khitrov]

On Thu, Feb 19, 2009 at 10:06 PM, David Cournapeau  wrote:
> On Fri, Feb 20, 2009 at 11:43 AM, Maxim Khitrov  wrote:
>>
>> Yes, I may have a case where one thread is still sending data, while
>> another tries to close the connection, or two threads trying to close
>> the connection at the same time. In both cases, I need some parts of
>> the code to be atomic to prevent errors.
>
> That does not sound like the right approach, then. Matlab engine is
> not thread safe in that sense:
>
> http://www.mathworks.fr/support/solutions/data/1-YR98I.html?product=ML&solution=1-YR98I
>
> cheers,
>
> David
>

"One option is to use MUTEXes around every call into the MATLAB
Engine" - that's exactly what I'm doing. Some setup work, like
creating mxArrays is done without a lock, but all eng* calls are
mutually exclusive. I've been doing a lot of my own testing, and so
far have seen no problems with this approach.

- Max
--
http://mail.python.org/mailman/listinfo/python-list

Re: To unicode or not to unicode

[Benjamin Peterson]

Ron Garret  flownet.com> writes:

> 
> I'm writing a little wiki that I call µWiki.  That's a lowercase Greek 
> mu at the beginning (it's pronounced micro-wiki).  It's working, except 
> that I can't actually enter the name of the wiki into the wiki itself 
> because the default unicode encoding on my Python installation is 
> "ascii".  So I'm trying to decide on a course of action.  There seem to 
> be three possibilities:

You should never have to rely on the default encoding. You should explicitly
decode and encode data.

> 
> 1.  Change the code to properly support unicode.  Preliminary 
> investigations indicate that this is going to be a colossal pain in the 
> ass.

Properly handling unicode may be painful at first, but it will surely pay off in
the future.




--
http://mail.python.org/mailman/listinfo/python-list

Re: Fwd: IDLE error on 3.0.1

[Ned Deily]

In article <3f795ab3-9ba5-4833-971c-9a442cef7...@talktalk.net>,
 John Forse  wrote:
> > Occaisionally, if I mistype in python shell or try to copy & paste a  
> > line there, I get a permanent hang with the error below. It isn't  
> > corrected by quitting IDLE, but is put right with a restart.  I am  
> > running 3.0.1 on Mac 10.5.6. Is this a bug, as it never happened on  
> > the same system with 2.5 &2.6.1 ?
> > Is there another way to prevent this?
> > Regards
> > John Forse
> >
> > IDLE cant bind TCP/IP port 8833 which is necessary to communicate  
> > with it's python execution server. Either no networking is installed  
> > on this computer or another process(another IDLE?) is using the  
> > port. Run IDLE with the -n command line switch to start without a  
> > sub-process and refer to help/IDLE Help 'Running without a sub- 
> > process' for further details
> >
> PS I AM connected to the network at the time and Help/Idle help is  
> greyed-out on my IDLE. I've tried entering -n to IDLE on the initial  
> blank window, but it seems to make no difference to the problem. I  
> don't understand what is 
> happening.

The error message means that there is another IDLE still running so I 
assume you get this when you try to launch IDLE again after the first 
one hangs?  If the first IDLE is hung, there should be either an IDLE or 
Python icon in your dock and, if IDLE is not responding to menu items or 
type-ins, you should be able to cause it to quit by pressing Option 
while clicking on the icon in the dock and selecting Force Quit.

It would be very helpful if you could outline the steps to reproduce 
this (in particular, how you launch IDLE) and open an issue on the 
Python bug tracker: http://bugs.python.org/

-- 
 Ned Deily,
 n...@acm.org

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problems with OS X 10.5.6 and Python 2.5 and GDAL 1.6

[Helly John J.]

Thanks, Ned. Despite the problems, your help has given me something to  
work with.

Cheers.
--
John Helly, University of California, San Diego
San Diego Supercomputer Center
Scripps Institution of Oceanography 9500 Gilman Dr. Mail Code, La  
Jolla CA 92093
Phone: Voice +01 760 840 8660 mobile / stonesteps (Skype) /  
stonesteps7 (iChat)

http://www.sdsc.edu/~hellyj



On Feb 19, 2009, at 7:02 PM, Ned Deily wrote:

In article <2fe1a73a-5bdd-45cd-8323-ccdf68df3...@ucsd.edu>,
"Helly John J."  wrote:

[...]
3.  I changed the gdal_merge.py code so it uses the correct syntax
even after the error but then find that I get the original 'no module'
error, I think, because gdal_merge.py starts with the line:

#!/usr/bin/env python

I don't understand this usage but when I run it alone (i.e., /usr/bin/
env) I see that it forces the PATH, and the environment in general, to
be something different than what I have set in my .bash_profile.  For
example, it puts

PATH=/Library/Frameworks/Python.framework/Versions/Current/bin:

in the beginning in front of /usr/bin which I had moved up in the path
in my .bash_profile.  I overcame this by getting rid of /usr/bin/env
and substituting /usr/bin/python.


See .

The idea behind /usr/bin/env usage is to allow scripts to get around
absolute paths and to be able to find the interpreter for the script by
using the search order of $PATH.  So, if the script with /usr/bin/env
python was invoking the wrong python, it sounds like your $PATH wasn't
really what you thought it was, despite changing your .bash_profile.
Did you start a new terminal session or login shell?  What does "echo
$PATH" say?


So, there are three issues:

1. the malloc error
2. the interaction with the gdal_merge.py error handling


No suggestions on the first two, other than to perhaps install
easy_install and GDAL and friends using the python.org 2.5 and/or to ask
in a more specialized forum.


Otherwise, everything's peachy.


Good luck!

--
Ned Deily,
n...@acm.org

--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[David Cournapeau]

On Fri, Feb 20, 2009 at 11:43 AM, Maxim Khitrov  wrote:
>
> Yes, I may have a case where one thread is still sending data, while
> another tries to close the connection, or two threads trying to close
> the connection at the same time. In both cases, I need some parts of
> the code to be atomic to prevent errors.

That does not sound like the right approach, then. Matlab engine is
not thread safe in that sense:

http://www.mathworks.fr/support/solutions/data/1-YR98I.html?product=ML&solution=1-YR98I

cheers,

David
--
http://mail.python.org/mailman/listinfo/python-list

Re: Problems with OS X 10.5.6 and Python 2.5 and GDAL 1.6

[Ned Deily]

In article <2fe1a73a-5bdd-45cd-8323-ccdf68df3...@ucsd.edu>,
 "Helly John J."  wrote:
>[...]
> 3.  I changed the gdal_merge.py code so it uses the correct syntax  
> even after the error but then find that I get the original 'no module'  
> error, I think, because gdal_merge.py starts with the line:
> 
> #!/usr/bin/env python
> 
> I don't understand this usage but when I run it alone (i.e., /usr/bin/ 
> env) I see that it forces the PATH, and the environment in general, to  
> be something different than what I have set in my .bash_profile.  For  
> example, it puts
> 
> PATH=/Library/Frameworks/Python.framework/Versions/Current/bin:
> 
> in the beginning in front of /usr/bin which I had moved up in the path  
> in my .bash_profile.  I overcame this by getting rid of /usr/bin/env  
> and substituting /usr/bin/python.

See .

The idea behind /usr/bin/env usage is to allow scripts to get around 
absolute paths and to be able to find the interpreter for the script by 
using the search order of $PATH.  So, if the script with /usr/bin/env 
python was invoking the wrong python, it sounds like your $PATH wasn't 
really what you thought it was, despite changing your .bash_profile.  
Did you start a new terminal session or login shell?  What does "echo 
$PATH" say?

> So, there are three issues:
> 
> 1. the malloc error
> 2. the interaction with the gdal_merge.py error handling

No suggestions on the first two, other than to perhaps install 
easy_install and GDAL and friends using the python.org 2.5 and/or to ask 
in a more specialized forum.

> Otherwise, everything's peachy.

Good luck!

-- 
 Ned Deily,
 n...@acm.org

--
http://mail.python.org/mailman/listinfo/python-list

Fwd: IDLE error on 3.0.1

[John Forse]

Regards
John



Begin forwarded message:


From: John Forse 
Date: 20 February 2009 02:46:19 GMT
To: python-list@python.org
Subject: IDLE error on 3.0.1

Occaisionally, if I mistype in python shell or try to copy & paste a  
line there, I get a permanent hang with the error below. It isn't  
corrected by quitting IDLE, but is put right with a restart.  I am  
running 3.0.1 on Mac 10.5.6. Is this a bug, as it never happened on  
the same system with 2.5 &2.6.1 ?

Is there another way to prevent this?
Regards
John Forse

IDLE cant bind TCP/IP port 8833 which is necessary to communicate  
with it's python execution server. Either no networking is installed  
on this computer or another process(another IDLE?) is using the  
port. Run IDLE with the -n command line switch to start without a  
sub-process and refer to help/IDLE Help 'Running without a sub- 
process' for further details


PS I AM connected to the network at the time and Help/Idle help is  
greyed-out on my IDLE. I've tried entering -n to IDLE on the initial  
blank window, but it seems to make no difference to the problem. I  
don't understand what is happening.--
http://mail.python.org/mailman/listinfo/python-list

To unicode or not to unicode

[Ron Garret]

I'm writing a little wiki that I call µWiki.  That's a lowercase Greek 
mu at the beginning (it's pronounced micro-wiki).  It's working, except 
that I can't actually enter the name of the wiki into the wiki itself 
because the default unicode encoding on my Python installation is 
"ascii".  So I'm trying to decide on a course of action.  There seem to 
be three possibilities:

1.  Change the code to properly support unicode.  Preliminary 
investigations indicate that this is going to be a colossal pain in the 
ass.

2.  Change the default encoding on my Python installation to be latin-1 
or UTF8.  The disadvantage to this is that no one else will be able to 
run my code without making the same change to their installation, since 
you can't change default encodings once Python has started.

3.  Punt and spell it 'uwiki' instead.

I'm feeling indecisive so I thought I'd ask other people's opinion.  
What should I do?

rg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Maxim Khitrov]

On Thu, Feb 19, 2009 at 9:15 PM, John Machin  wrote:
> On Feb 20, 6:53 am, Maxim Khitrov  wrote:
>> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern  wrote:
>> > On 2009-02-19 12:52, Maxim Khitrov wrote:
>>
>> >> Hello all,
>>
>> >> I'm currently writing a Python<->  MATLAB interface with ctypes and
>> >> array.array class, using which I'll need to push large amounts of data
>> >> to MATLAB.
>>
>> > Have you taken a look at mlabwrap?
>>
>> >  http://mlabwrap.sourceforge.net/
>>
>> > At the very least, you will probably want to use numpy arrays instead of
>> > array.array.
>>
>> >  http://numpy.scipy.org/
>>
>> I have, but numpy is not currently available for python 2.6, which is
>> what I need for some other features, and I'm trying to keep the
>> dependencies down in any case. Mlabwrap description doesn't mention if
>> it is thread-safe, and that's another one of my requirements.
>>
>> The only feature that I'm missing with array.array is the ability to
>> quickly pre-allocate large chunks of memory. To do that right now I'm
>> using array('d', (0,) * size).
>
> It would go somewhat faster if you gave it a float instead of an int.
>
>> It would be nice if array accepted an
>> int as the second argument indicating how much memory to allocate and
>> initialize to 0.
>
> While you're waiting for that to happen, you'll have to use the
> fromstring trick, or another gimmick that is faster and is likely not
> to use an extra temp 8Mb for a 1M-element array, as I presume the
> fromstring does.
>
> [Python 2.6.1 on Windows XP SP3]
> [Processor: x86 Family 15 Model 36 Stepping 2 AuthenticAMD ~1994 Mhz]
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d',(0,)*
> 100)"
> 10 loops, best of 3: 199 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d',(0.,)*100)"
> 10 loops, best of 3: 158 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d');x.fromstring('\0'*8*100)"
> 10 loops, best of 3: 36 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
> ('d','\0'*8*100)"
> 10 loops, best of 3: 35.7 msec per loop
>
> C:\junk>\python26\python -mtimeit -s"from array import array" "array
> ('d',(0.,))*100"
> 10 loops, best of 3: 19.5 msec per loop

Interesting, though I'm not able to replicate that last outcome. The
string method is still the fastest on my machine. Furthermore, it
looks like the order in which you do the multiplication also matters -
(8 * size * '\0') is faster than ('\0' * 8 * size). Here is my test
and outcome:

---
from array import array
from timeit import repeat

print repeat(lambda: array('d', (0,) * 10), number = 100)
print repeat(lambda: array('d', (0.0,) * 10), number = 100)
print repeat(lambda: array('d', (0.0,)) * 10, number = 100)
print repeat(lambda: array('d', '\0' * 10 * 8), number = 100)
print repeat(lambda: array('d', '\0' * 8 * 10), number = 100)
print repeat(lambda: array('d', 8 * 10 * '\0'), number = 100)
---

[0.91048107424534941, 0.88766983642377162, 0.88312824645684618]
[0.72164595848486179, 0.72038338197219343, 0.72346024633711981]
[0.10763947529894136, 0.1047547164728595, 0.10461521722863232]
[0.05856873793382178, 0.058508825334111947, 0.058361838698573365]
[0.057632016342657799, 0.057521392119007864, 0.057227118035289237]
[0.056006643320014149, 0.056331811311153501, 0.05618743321510]

The array('d', (0.0,)) * 10 method is a good compromise between
performance and amount of memory used, so maybe I'll use that instead.

- Max
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Benjamin Kaplan]

On Thu, Feb 19, 2009 at 7:16 PM, Paul Rubin  wrote:

> Terry Reedy  writes:
> > > The slowness itself can be addressed by technical means, such as
> > > native-code compilation and eliminating the GIL.
> >
> > Given that the GIL remains to make Python run faster in the usual (up
> > to now, at least) case of 1 processor, that seems a strange statement.
>
> We've had this discussion before.  The 1-processor slowdown that you
> refer to comes from replacing the GIL with the blunt instrument of a
> lock around each reference count operation.  That has the advantage of
> not breaking CPython in a million places, but has the drawback of
> taking a big performance hit.  The long term fix is to replace
> reference counts with a tracing GC.  That is apparently not feasible
> in the framework of CPython and the many extension modules that have
> been written for it, so it would have to be accompanied by an
> implementation switch (e.g. PyPy).
>
> Steve Holden has mentioned Jython and Ironpython a few times in this
> thread.  Those are reasonable proofs of the concept of a GIL-less
> Python, but for various reasons (spelled J-V-M and W-i-n-d-o-w-s) are
> not all that suitable for many current Python users.


Actually, Mono supports IronPython so it will work on M-a-c, L-i-n-u-x,
S-o-l-a-r-i-s, and possibly even W-i-i and P-l-a-y-s-t-a-t-i-o-n-3.




> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Maxim Khitrov]

On Thu, Feb 19, 2009 at 9:34 PM, David Cournapeau  wrote:
> On Fri, Feb 20, 2009 at 4:53 AM, Maxim Khitrov  wrote:
>> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern  wrote:
>>> On 2009-02-19 12:52, Maxim Khitrov wrote:

 Hello all,

 I'm currently writing a Python<->  MATLAB interface with ctypes and
 array.array class, using which I'll need to push large amounts of data
 to MATLAB.
>>>
>>> Have you taken a look at mlabwrap?
>>>
>>>  http://mlabwrap.sourceforge.net/
>>>
>>> At the very least, you will probably want to use numpy arrays instead of
>>> array.array.
>>>
>>>  http://numpy.scipy.org/
>>
>> I have, but numpy is not currently available for python 2.6, which is
>> what I need for some other features, and I'm trying to keep the
>> dependencies down in any case. Mlabwrap description doesn't mention if
>> it is thread-safe, and that's another one of my requirements.
>
> What do you mean by thread-safe ? Different threads calling the same
> matlab engine handle ?

Yes, I may have a case where one thread is still sending data, while
another tries to close the connection, or two threads trying to close
the connection at the same time. In both cases, I need some parts of
the code to be atomic to prevent errors.

- Max
--
http://mail.python.org/mailman/listinfo/python-list

Re: Problems with OS X 10.5.6 and Python 2.5 and GDAL 1.6

[Helly John J.]

Hi.

You are correct about finding the module, Ned.  Thanks for that  
although I don't understand why it works.  However, there are  
complications.


1. Using '/usr/bin/python' and 'from osgeo import gdal' works but  
throws an error the first time it's invoked.


NeptuneDesk.local:/Volumes/B2_160GB/PALSAR_200902/Geotiffs>/usr/bin/ 
python

Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from osgeo import gdal
Python(21061) malloc: *** error for object 0xa00696d8: Non-aligned  
pointer being freed

*** set a breakpoint in malloc_error_break to debug
>>> from osgeo import gdal
>>>

2. Since I'm trying to use gdal_merge.py this causes a second attempt  
within gdal_merge.py, after the error above, to be made with the old  
(now wrong) syntax:


try:
from osgeo import gdal
except ImportError:
osgeo import gdal

3.  I changed the gdal_merge.py code so it uses the correct syntax  
even after the error but then find that I get the original 'no module'  
error, I think, because gdal_merge.py starts with the line:


#!/usr/bin/env python

I don't understand this usage but when I run it alone (i.e., /usr/bin/ 
env) I see that it forces the PATH, and the environment in general, to  
be something different than what I have set in my .bash_profile.  For  
example, it puts


PATH=/Library/Frameworks/Python.framework/Versions/Current/bin:

in the beginning in front of /usr/bin which I had moved up in the path  
in my .bash_profile.  I overcame this by getting rid of /usr/bin/env  
and substituting /usr/bin/python.


So, there are three issues:

1. the malloc error
2. the interaction with the gdal_merge.py error handling
3. the environment setup and where that's coming from.

Otherwise, everything's peachy.

Cheers.
--
John Helly, University of California, San Diego
San Diego Supercomputer Center,  Mail Code 0527
Scripps Institution of Oceanography, Climate, Atmospheric Science, and  
Physical Oceanography, Mail Code 0224

9500 Gilman Dr., La Jolla CA 92093
+01 760 840 8660 mobile / stonesteps (Skype) / stonesteps7 (iChat) /  
URL (http://www.sdsc.edu/~hellyj)


On Feb 19, 2009, at 11:16 AM, Ned Deily wrote:

In article ,
Ned Deily  wrote:
It looks like you have installed GDAL to the site-packages directory  
of

the Apple-supplied python 2.5 (which, for 10.5, is 2.5.1, not 2.4).
That site-packages directory is /Library/Python/2.5. The Apple- 
supplied

python comes with a sym link from /usr/bin/python.  If you launch it,
you'll probably find GDAL is there as expected.

If you do want to use the python.org python, which is somewhat newer,
you need to install its own version of setuptools/easy_install and use
it to install GDAL to the site-packages directory of that python which
is located here:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- 
pack

ages.  The two python instances co-exist.  To install another

 easy_install, ensure the python.org python comes first on your

PATH, then follow the instructions here:
  


BTW, that easy_install will itself be installed into the bin directory
of the python.org python framework.  So to ensure you are using that
easy_install and that python, make sure that bin directory is on your
PATH before "/usr/bin", so, if necessary, something like:

export PATH="/Library/Frameworks/Python.framework/Versions/2.5/bin: 
$PATH"


--
Ned Deily,
n...@acm.org

--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

Mensanator wrote:
> On Feb 19, 7:18 pm, Steve Holden  wrote:
>> Mensanator wrote:
>>> On Feb 19, 2:18 pm, Paul Rubin  wrote:
>> [...]
>>> When I run I Python program, the Windows task manager shows both
>>> cores running (usually a 60/40 split) for an overall 50% usage.
>>> What am I actually seeing? If Python only uses one of the cores,
>>> why do both light up? Is everything much more complicated (due to
>>> OS scheduling, etc.) than the simple explanations of GIL?
>> You are seeing your Python program running on one core, and the usual
>> Windows crap keeping the other one busy.
> 
> I thought of that, but the usual Windows crap accounts for only a
> couple percent prior to the Python program running. Christian Heimes
> answer sounds more realistic.
> 
> But what do I know?
> 
At least as much as I do, probably.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[David Cournapeau]

On Fri, Feb 20, 2009 at 4:53 AM, Maxim Khitrov  wrote:
> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern  wrote:
>> On 2009-02-19 12:52, Maxim Khitrov wrote:
>>>
>>> Hello all,
>>>
>>> I'm currently writing a Python<->  MATLAB interface with ctypes and
>>> array.array class, using which I'll need to push large amounts of data
>>> to MATLAB.
>>
>> Have you taken a look at mlabwrap?
>>
>>  http://mlabwrap.sourceforge.net/
>>
>> At the very least, you will probably want to use numpy arrays instead of
>> array.array.
>>
>>  http://numpy.scipy.org/
>
> I have, but numpy is not currently available for python 2.6, which is
> what I need for some other features, and I'm trying to keep the
> dependencies down in any case. Mlabwrap description doesn't mention if
> it is thread-safe, and that's another one of my requirements.

What do you mean by thread-safe ? Different threads calling the same
matlab engine handle ?

David
--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[John Machin]

On Feb 20, 6:53 am, Maxim Khitrov  wrote:
> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern  wrote:
> > On 2009-02-19 12:52, Maxim Khitrov wrote:
>
> >> Hello all,
>
> >> I'm currently writing a Python<->  MATLAB interface with ctypes and
> >> array.array class, using which I'll need to push large amounts of data
> >> to MATLAB.
>
> > Have you taken a look at mlabwrap?
>
> >  http://mlabwrap.sourceforge.net/
>
> > At the very least, you will probably want to use numpy arrays instead of
> > array.array.
>
> >  http://numpy.scipy.org/
>
> I have, but numpy is not currently available for python 2.6, which is
> what I need for some other features, and I'm trying to keep the
> dependencies down in any case. Mlabwrap description doesn't mention if
> it is thread-safe, and that's another one of my requirements.
>
> The only feature that I'm missing with array.array is the ability to
> quickly pre-allocate large chunks of memory. To do that right now I'm
> using array('d', (0,) * size).

It would go somewhat faster if you gave it a float instead of an int.

> It would be nice if array accepted an
> int as the second argument indicating how much memory to allocate and
> initialize to 0.

While you're waiting for that to happen, you'll have to use the
fromstring trick, or another gimmick that is faster and is likely not
to use an extra temp 8Mb for a 1M-element array, as I presume the
fromstring does.

[Python 2.6.1 on Windows XP SP3]
[Processor: x86 Family 15 Model 36 Stepping 2 AuthenticAMD ~1994 Mhz]

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d',(0,)*
100)"
10 loops, best of 3: 199 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d',(0.,)*100)"
10 loops, best of 3: 158 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d');x.fromstring('\0'*8*100)"
10 loops, best of 3: 36 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d','\0'*8*100)"
10 loops, best of 3: 35.7 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "array
('d',(0.,))*100"
10 loops, best of 3: 19.5 msec per loop

HTH,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Christian Heimes]

Mensanator wrote:
> I thought of that, but the usual Windows crap accounts for only a
> couple percent prior to the Python program running. Christian Heimes
> answer sounds more realistic.
> 
> But what do I know?

Be happy that your program makes use of both cores? :]

You can restrict your program from using more than one core by setting
the cpu affinity. On Windows the pywin32 packages wraps all necessary calls:


phandle = win32process.GetCurrentProcess()
win32process.SetProcessAffinityMask(phandle, mask)


Past week I've written a wrapper for the Linux syscalls
sched_setaffinity and friends. I may be able and allowed to release the
stuff in a few weeks.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Maxim Khitrov]

On Thu, Feb 19, 2009 at 7:01 PM, Scott David Daniels
 wrote:
> Maxim Khitrov wrote:
>>
>> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern 
>> wrote:
>> I have, but numpy is not currently available for python 2.6, which is
>> what I need for some other features, and I'm trying to keep the
>> dependencies down in any case
>> The only feature that I'm missing with array.array is the ability to
>> quickly pre-allocate large chunks of memory. To do that right now I'm
>> using array('d', (0,) * size). It would be nice if array accepted an
>> int as the second argument indicating how much memory to allocate and
>> initialize to 0.
>
> In the meantime, you could write a function (to ease the shift to numpy)
> and reduce your interface problem to a very small set of lines:
>def zeroes_d(n):
>'''Allocate a n-element vector of 'd' elements'''
>vector = array.array('d') # fromstring has no performance bug
>vector.fromstring(n * 8 * '\0')
>return vector
> Once numpy is up and running on 2.6, this should be easy to convert
> to a call to zeroes.

Here's the function that I'll be using from now on. It gives me
exactly the behavior I need, with an int initializer being treated as
array size. Still not as efficient as it could be if supported
natively by array (one malloc instead of two + memmove + extra
function call), but very good performance nevertheless:

from array import array as _array
array_null = dict((tc, '\0' * _array(tc).itemsize) for tc in 'cbBuhHiIlLfd')

def array(typecode, init):
if isinstance(init, int):
return _array(typecode, array_null[typecode] * init)
return _array(typecode, init)

- Max
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Mensanator]

On Feb 19, 7:18 pm, Steve Holden  wrote:
> Mensanator wrote:
> > On Feb 19, 2:18 pm, Paul Rubin  wrote:
> [...]
> > When I run I Python program, the Windows task manager shows both
> > cores running (usually a 60/40 split) for an overall 50% usage.
>
> > What am I actually seeing? If Python only uses one of the cores,
> > why do both light up? Is everything much more complicated (due to
> > OS scheduling, etc.) than the simple explanations of GIL?
>
> You are seeing your Python program running on one core, and the usual
> Windows crap keeping the other one busy.

I thought of that, but the usual Windows crap accounts for only a
couple percent prior to the Python program running. Christian Heimes
answer sounds more realistic.

But what do I know?

>
> regards
>  Steve
> --
> Steve Holden        +1 571 484 6266   +1 800 494 3119
> Holden Web LLC              http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Newby Question for reading a file

[Steve Holden]

alex23 wrote:
> On Feb 20, 5:38 am, Peter Otten <__pete...@web.de> wrote:
>> Yes you can get portions of the line by slicing:
>>
>> for line in open("infile"):
>> if line[8:18] != line[18:28]:
>> print line,
> 
> You can also name the slices, which makes the code more clear (IMO):
> 
> endda = slice(8,18)
> begda = slice(18,28)
> for line in open("infile"):
> if line[endda] != line[begda]:
> print line,
> 
>> Or you can use the struct module:
> 
> But this is the approach I'd take.

This is one of the lesser-appreciated aspects of Python. I anticipate a
flood of "Named Slices" blog posts round about now. Wonder if I'll be
the first?

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Maxim Khitrov]

On Thu, Feb 19, 2009 at 7:01 PM, Scott David Daniels
 wrote:
> Maxim Khitrov wrote:
>>
>> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern 
>> wrote:
>> I have, but numpy is not currently available for python 2.6, which is
>> what I need for some other features, and I'm trying to keep the
>> dependencies down in any case
>> The only feature that I'm missing with array.array is the ability to
>> quickly pre-allocate large chunks of memory. To do that right now I'm
>> using array('d', (0,) * size). It would be nice if array accepted an
>> int as the second argument indicating how much memory to allocate and
>> initialize to 0.
>
> In the meantime, you could write a function (to ease the shift to numpy)
> and reduce your interface problem to a very small set of lines:
>def zeroes_d(n):
>'''Allocate a n-element vector of 'd' elements'''
>vector = array.array('d') # fromstring has no performance bug
>vector.fromstring(n * 8 * '\0')
>return vector
> Once numpy is up and running on 2.6, this should be easy to convert
> to a call to zeroes.

If I do decide to transition at any point, it will require much
greater modification. For example, to speed-up retrieval of data from
Matlab, which is returned to me as an mxArray structure, I allocate an
array.array for it and then use ctypes.memmove to copy data directly
into the array's buffer (address obtained through buffer_info()).

Same thing for sending data, rather than allocate a separate mxArray,
copy data, and then send, I create an empty mxArray and set its data
pointer to the array's buffer. I'm sure that there are equivalents in
numpy, but the point is that the transition, which currently would not
benefit my code in any significant way, will not be a quick change.

On the other hand, I have to thank you for the fromstring example. For
some reason, it never occurred to me that creating a string of nulls
would be much faster than a tuple of zeros. In fact, you can pass the
string to the constructor and it calls fromstring automatically. For
an array of 1 million elements, using a string to initialize is 18x
faster. :)

- Max
--
http://mail.python.org/mailman/listinfo/python-list

Re: Problems with OS X 10.5.6 and Python 2.5 and GDAL 1.6

[Helly John J.]

Thanks very much.  I'll give it a go.
Cheers.
--
John Helly, University of California, San Diego
San Diego Supercomputer Center,  Mail Code 0527
Scripps Institution of Oceanography, Climate, Atmospheric Science, and  
Physical Oceanography, Mail Code 0224

9500 Gilman Dr., La Jolla CA 92093
+01 760 840 8660 mobile / stonesteps (Skype) / stonesteps7 (iChat) /  
URL (http://www.sdsc.edu/~hellyj)


On Feb 19, 2009, at 11:16 AM, Ned Deily wrote:

In article ,
Ned Deily  wrote:
It looks like you have installed GDAL to the site-packages directory  
of

the Apple-supplied python 2.5 (which, for 10.5, is 2.5.1, not 2.4).
That site-packages directory is /Library/Python/2.5. The Apple- 
supplied

python comes with a sym link from /usr/bin/python.  If you launch it,
you'll probably find GDAL is there as expected.

If you do want to use the python.org python, which is somewhat newer,
you need to install its own version of setuptools/easy_install and use
it to install GDAL to the site-packages directory of that python which
is located here:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- 
pack

ages.  The two python instances co-exist.  To install another

 easy_install, ensure the python.org python comes first on your

PATH, then follow the instructions here:
  


BTW, that easy_install will itself be installed into the bin directory
of the python.org python framework.  So to ensure you are using that
easy_install and that python, make sure that bin directory is on your
PATH before "/usr/bin", so, if necessary, something like:

export PATH="/Library/Frameworks/Python.framework/Versions/2.5/bin: 
$PATH"


--
Ned Deily,
n...@acm.org

--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

Mensanator wrote:
> On Feb 19, 2:18 pm, Paul Rubin  wrote:
[...]
> When I run I Python program, the Windows task manager shows both
> cores running (usually a 60/40 split) for an overall 50% usage.
> 
> What am I actually seeing? If Python only uses one of the cores,
> why do both light up? Is everything much more complicated (due to
> OS scheduling, etc.) than the simple explanations of GIL?

You are seeing your Python program running on one core, and the usual
Windows crap keeping the other one busy.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Newby Question for reading a file

[alex23]

On Feb 20, 5:38 am, Peter Otten <__pete...@web.de> wrote:
> Yes you can get portions of the line by slicing:
>
> for line in open("infile"):
>     if line[8:18] != line[18:28]:
>             print line,

You can also name the slices, which makes the code more clear (IMO):

endda = slice(8,18)
begda = slice(18,28)
for line in open("infile"):
if line[endda] != line[begda]:
print line,

> Or you can use the struct module:

But this is the approach I'd take.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

MRAB wrote:
> Tim Wintle wrote:
> [snip]
>> Yes, we're coming to a point where we're going to have tens of cores in
>> a chip, but by that time someone far cleverer than me (possibly someone
>> who's on this list) will have solved that problem. The top experts in
>> many fields use Python, and if they weren't able to make use of multiple
>> core chips, then there wouldn't be any demand for them.
>>
> Perhaps we should ask Guido who it is; after all, he's the one with the
> time machine! :-)

But he clearly hasn't been using it lately.

Perhaps it's time Python stopped being a dictatorship?

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

Paul Rubin wrote:
> Tim Rowe  writes:
>>> I really believe that GIL is a design error.
>> It's only an error if it gets in the way. It's the experience of a lot
>> of programmers that it doesn't, so it's not an error.
> 
> [...]  Denying the
> problem's existence is not sensible.

And if wishes were horses then beggars would ride :-)
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Paul Rubin]

Terry Reedy  writes:
> > The slowness itself can be addressed by technical means, such as
> > native-code compilation and eliminating the GIL.
> 
> Given that the GIL remains to make Python run faster in the usual (up
> to now, at least) case of 1 processor, that seems a strange statement.

We've had this discussion before.  The 1-processor slowdown that you
refer to comes from replacing the GIL with the blunt instrument of a
lock around each reference count operation.  That has the advantage of
not breaking CPython in a million places, but has the drawback of
taking a big performance hit.  The long term fix is to replace
reference counts with a tracing GC.  That is apparently not feasible
in the framework of CPython and the many extension modules that have
been written for it, so it would have to be accompanied by an
implementation switch (e.g. PyPy).

Steve Holden has mentioned Jython and Ironpython a few times in this
thread.  Those are reasonable proofs of the concept of a GIL-less
Python, but for various reasons (spelled J-V-M and W-i-n-d-o-w-s) are
not all that suitable for many current Python users.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

Terry Reedy wrote:
> Paul Rubin wrote:
> 
>> I would say, slow execution is a drawback that we put up with in order
>> to gain benefits of Python programming that are mostly unrelated to
>> the causes of the slowness.  The slowness itself can be addressed by
>> technical means, such as native-code compilation and eliminating the
>> GIL.
> 
> Given that the GIL remains to make Python run faster in the usual (up to
> now, at least) case of 1 processor, that seems a strange statement.
> 
>From a historical perspective, that's going to seem like a very
parochial PoV in (say) twenty years.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Scott David Daniels]

Maxim Khitrov wrote:

On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern  wrote:
I have, but numpy is not currently available for python 2.6, which is
what I need for some other features, and I'm trying to keep the
dependencies down in any case
The only feature that I'm missing with array.array is the ability to
quickly pre-allocate large chunks of memory. To do that right now I'm
using array('d', (0,) * size). It would be nice if array accepted an
int as the second argument indicating how much memory to allocate and
initialize to 0.


In the meantime, you could write a function (to ease the shift to numpy)
and reduce your interface problem to a very small set of lines:
def zeroes_d(n):
'''Allocate a n-element vector of 'd' elements'''
vector = array.array('d') # fromstring has no performance bug
vector.fromstring(n * 8 * '\0')
return vector
Once numpy is up and running on 2.6, this should be easy to convert
to a call to zeroes.

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Terry Reedy]

Paul Rubin wrote:


I would say, slow execution is a drawback that we put up with in order
to gain benefits of Python programming that are mostly unrelated to
the causes of the slowness.  The slowness itself can be addressed by
technical means, such as native-code compilation and eliminating the
GIL.


Given that the GIL remains to make Python run faster in the usual (up to 
now, at least) case of 1 processor, that seems a strange statement.


--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[MRAB]

Tim Wintle wrote:
[snip]

Yes, we're coming to a point where we're going to have tens of cores in
a chip, but by that time someone far cleverer than me (possibly someone
who's on this list) will have solved that problem. The top experts in
many fields use Python, and if they weren't able to make use of multiple
core chips, then there wouldn't be any demand for them.


Perhaps we should ask Guido who it is; after all, he's the one with the
time machine! :-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

Paul Rubin wrote:
> Tim Rowe  writes:
>>> That Python is so much slower than C is yet another area where Python
>>> can use improvement.
>> No, because we don't use Python where C would be more appropriate.
> 
> C is basically never appropriate.  C should be outlawed by Congress
> with the ban enforced by roving pie-throwing squads .  If a
> Python program is too slow, making Python faster is far preferable to
> switching to C.
> 
Unfortunately it's also far more difficult. I speak with the experience
of the "Need for Speed" sprint behind me: the accumulated brainpower at
that event should have been able to pick all the low-hanging fruit, and
yet the resultant net speedup was worthwhile, but definitely not immense.

>> Sure nobody would complain if Python were faster, but it's not for
>> speed that we choose Python. Not speed of /execution/ that is.
> 
> I would say, slow execution is a drawback that we put up with in order
> to gain benefits of Python programming that are mostly unrelated to
> the causes of the slowness.  The slowness itself can be addressed by
> technical means, such as native-code compilation and eliminating the
> GIL.  I believe (for example) that the PyPy project is doing both of
> these.
> 
And IronPython and JPython already have. (How many times do I have to
say this before somebody with access to decent multi-processor hardware
runs some actual benchmarks?  Where's snakebite.org when you need it? ;-)

>> Different languages have different trade-offs. Python's trade-offs
>> suit us. If they don't suit you, use a language with trade-offs that
>> do.
> 
> I don't think Python's slowness is inherent in the language.  It's
> mostly a shortcoming of the implementation that should be fixed like
> any other such shortcoming.

Reasonable, and true. Some people talk about the GIL as though it were
something other than an implementation artifact.

What Guido doesn't seem to have accepted yet is that slowing [C]Python
down by 50% on a single-processor CPU will actually be a worthwhile
tradeoff in ten years time, when nothing will have less than eight cores
and the big boys will be running at 64 kilo-cores.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Christian Heimes]

Mensanator wrote:
> When I run I Python program, the Windows task manager shows both
> cores running (usually a 60/40 split) for an overall 50% usage.
> 
> What am I actually seeing? If Python only uses one of the cores,
> why do both light up? Is everything much more complicated (due to
> OS scheduling, etc.) than the simple explanations of GIL?

A Python *program* can utilize more than one core, just Python *code*
can't run on multiple cores in parallel. Everytime a C function calls
code that doesn't use the Python API it can chose to release the GIL.
That way a Python program can wrap a library and use as many cores as
possible.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

Tim Wintle wrote:
> On Thu, 2009-02-19 at 12:18 -0800, Paul Rubin wrote:
>> If such
>> speedups were useless or unimportant, we would not have blown our hard
>> earned cash replacing perfectly good older hardware, so we have to
>> accept the concept that speed matters and ignore those platitudes that
>> say otherwise.
> 
> Kind of agree (although I use a netbook at lot at the moment, and I
> don't use that because of speed-ups!)
[...]
> (3)
> Multiple cores scale processing power linearly at best with the number
> of cores (since you're normally going to be IO based at some point).
> Perhaps the GIL will be relaxed a bit, but it's not going to have a
> massive effect.
> 
There are significant classes of problems which *are* compute bound, and
as computers are applied more and more to planning and design problems
it seems likely that kind of application will become more significant.

In twenty years time our laptops will probably be continually optimizing
aspects of our lives using advanced linear algebra algorithms over
matrices with hundreds or thousands of millions of elements. I wouldn't
like Python to be excluded from solving such problems, and others we
currently fail to foresee.

Though my own interest does tend to lie in the areas where interaction
of various kinds dominates the need for large chunks of processing
power, I can't ignore the obvious. Many compute-bound problems do exist,
and they are important.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Paul Rubin]

Tim Rowe  writes:
> > I really believe that GIL is a design error.
> It's only an error if it gets in the way. It's the experience of a lot
> of programmers that it doesn't, so it's not an error.

It does get in the way of quite a few of us, but I wouldn't exactly
call it an error.  It was a sensible decision at the time it was made.
Changing technology and changing requirements have turned it into a
problem since then.  

A sensible decision later becoming a problem is a fairly normal
occurrence, not just in software but in just about every area of human
endeavor.  The sensible response to the changed landscape is to
recognize the problem and figure out what to do about it.  Denying the
problem's existence is not sensible.
--
http://mail.python.org/mailman/listinfo/python-list

Re: os.fdopen giving "Invalid Argument" from pipes

[Scott David Daniels]

Tim Wintle wrote:

On Thu, 2009-02-19 at 11:50 -0500, Jean-Paul Calderone wrote: ...

(also, i before e *except* after c et cetera).  Flip 'em around and
all is well.


Thanks - never was great at speling :-)


I used to be great at spelling until I became a programmer.  Programming
rewards consistency in spelling over correctness.  Spelling "recievers"
the same way 1000 times is better than getting it correct 999 times and
"wrong" once. :-)

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Falcolas]

On Feb 19, 3:11 pm, Paul Rubin  wrote:
> Falcolas  writes:
> > It's a proposition that used to bother me, until I did some actual
> > programming of real world problems in Python. I've yet to really find
> > a case where the application was slow enough to justify the cost of
> > using multiple Python processes.
>
> Right, that's basically the issue here: the cost of using multiple
> Python processes is unnecessarily high.  If that cost were lower then
> we could more easily use multiple cores to make oru apps faster.

I was actually referring to the time cost of researching or developing
a parallel execution algorithm which would be suitable for multiple
processes.

The system overhead of using the Python multiprocess module is fairly
negligible for the systems I work on.

> Different languages have different trade-offs. Python's trade-offs
> suit us. If they don't suit you, use a language with trade-offs that
> do.

+1

~G
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Paul Rubin]

Tim Rowe  writes:
> > That Python is so much slower than C is yet another area where Python
> > can use improvement.
> 
> No, because we don't use Python where C would be more appropriate.

C is basically never appropriate.  C should be outlawed by Congress
with the ban enforced by roving pie-throwing squads .  If a
Python program is too slow, making Python faster is far preferable to
switching to C.

> Sure nobody would complain if Python were faster, but it's not for
> speed that we choose Python. Not speed of /execution/ that is.

I would say, slow execution is a drawback that we put up with in order
to gain benefits of Python programming that are mostly unrelated to
the causes of the slowness.  The slowness itself can be addressed by
technical means, such as native-code compilation and eliminating the
GIL.  I believe (for example) that the PyPy project is doing both of
these.

> Different languages have different trade-offs. Python's trade-offs
> suit us. If they don't suit you, use a language with trade-offs that
> do.

I don't think Python's slowness is inherent in the language.  It's
mostly a shortcoming of the implementation that should be fixed like
any other such shortcoming.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Mensanator]

On Feb 19, 2:18 pm, Paul Rubin  wrote:
> sturlamolden  writes:
> > Yes, the GIL prevents Python threads from being used in a certain way.
> > But do you really need to use threads like that? Or do you just think
> > you do?
>
> How old is your computer, why did you buy it, and is it the first one
> you ever owned?
>
> For most of us, I suspect, it is not our first one, and we bought it
> to get a processing speedup relative to the previous one.  If such
> speedups were useless or unimportant, we would not have blown our hard
> earned cash replacing perfectly good older hardware, so we have to
> accept the concept that speed matters and ignore those platitudes that
> say otherwise.
>
> It used to be that new computers were faster than the old ones because
> they ran at higher clock rates.  That was great, no software changes
> at all were required to benefit from the higher speed.  Now, they get
> the additional speed by having more cores.  That's better than nothing
> but making use of it requires fixing the GIL.

When I run I Python program, the Windows task manager shows both
cores running (usually a 60/40 split) for an overall 50% usage.

What am I actually seeing? If Python only uses one of the cores,
why do both light up? Is everything much more complicated (due to
OS scheduling, etc.) than the simple explanations of GIL?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Threading and tkinter

[Scott David Daniels]

gert wrote:

Hope you do not mind ignoring part of answers, so I can figure out
more why things work the way they are.
This two examples work, what i do not understand is that in function
display i do not have to declare root, v or x ?

...


x=0
def weegbrug():
global x
while True:
x=x+1
sleep(0.5)
start_new_thread(weegbrug,())

def display():
v.set(x)
root.after(500, lambda:display())

1) Note that x, v, and root are read and manipulated, but not written.
   So, they do not need to be declared global (global is only needed
   to declare writable global names).
2) The last statement could more easily be written:
   root.after(500, display)
...

root.after(500, lambda:display())

Again you can write this:
   root.after(500, display)

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Tim Wintle]

On Thu, 2009-02-19 at 12:18 -0800, Paul Rubin wrote:
> If such
> speedups were useless or unimportant, we would not have blown our hard
> earned cash replacing perfectly good older hardware, so we have to
> accept the concept that speed matters and ignore those platitudes that
> say otherwise.

Kind of agree (although I use a netbook at lot at the moment, and I
don't use that because of speed-ups!)

> It used to be that new computers were faster than the old ones because
> they ran at higher clock rates.  That was great, no software changes
> at all were required to benefit from the higher speed.  Now, they get
> the additional speed by having more cores.  That's better than nothing
> but making use of it requires fixing the GIL.

My opinion on this (when talking about personal computers rather than
servers) is that:

(1)
Computers *appear* faster now because they have more cores - you can
have one doing the fancy GUI effects of Compiz etc. in the background,
while the other core actually does the work.

(2) 
Actual speedups aren't that related to either clock speed or cores at
the moment, they're related to the switch to 64-bit processors, the
massive increases in RAM and the increase in system bus speeds and other
IO (and of course graphics cards). I suspect that the next common
increase will be solid state hard disks.

e.g. I used to expect my computer to be paging all the time, although
I'd try to reduce it - these days I'm really upset when I see I've had
to page *anything* to disk!

Another massive increase (which I am willing to pay more for with the
work I do) is the processor cache - at first it was amazing when we got
significant level2 cache advertised on pc equipment, now I can fit
massive amounts of code into my 4mb level-2 cache *on my laptop*! That's
a huge impact for numerical work.

(3)
Multiple cores scale processing power linearly at best with the number
of cores (since you're normally going to be IO based at some point).
Perhaps the GIL will be relaxed a bit, but it's not going to have a
massive effect.

Tim Wintle


--
http://mail.python.org/mailman/listinfo/python-list

Pydev 1.4.3 Released

[Fabio Zadrozny]

Hi All,

Pydev and Pydev Extensions 1.4.3 have been released

Details on Pydev Extensions: http://www.fabioz.com/pydev
Details on Pydev: http://pydev.sf.net
Details on its development: http://pydev.blogspot.com

Release Highlights in Pydev Extensions:
-

* Fixed racing conditions in the context-insensitive indexing which
could corrupt the index.
* Search references working on Eclipse 3.3 and Eclipse 3.4
* Lambda properly treated as a new context for code-analysis
* Analysis is now recognizing __name__
* Analysis now marks variables used when accessed with augstore (+=)
* Some definitions were not properly indexed on some assign cases


Release Highlights in Pydev:
--

* Interactive console The interpreter to be used can be chosen
* New modules can be created from templates
* Interpreter configuration improved!
  o Environment variables can be specified for a given interpreter
  o Canceling operation now works correctly

* Debugger
  o Variables correctly gotten on Jython 2.1 / 2.2
  o Using globals as an union of original globals+locals so that
generator expressions can be evaluated
  o Breakpoints only opened on double-click (no longer on select)

* The project preferences are now applied even if the page to
configure the project is not visible.
* Jython 2.5b1 working (problem with sitecustomize)
* Wrap paragraph fixed
* Set comprehension working on Python 3.0 parsing
* Find definition working when a module outside of the known pythonpath is found
* Source folders were not properly found sometimes -- when workspace
was not properly refreshed
* Invalid modules could get in the memory
* Getting the grammar version for a project could be wrong (and could
loose its indexing at that time)
* Multiple external zip files can be added at once to the pythonpath
* nonlocal added to keywords
* Fixed annoying problem where cursor was jumping when it shouldn't (outline)
* Fixed problem where the breakpoint could be lost (now, playing safe
and matching anything in the file if the context cannot be gotten)
* Ctrl + 2 + --reindex can be used to reindex all the opened projects
if the indexing becomes corrupt
* Changing nothing on project config and pressing OK no longer
reanalyzes the modules



What is PyDev?
---

PyDev is a plugin that enables users to use Eclipse for Python and
Jython development -- making Eclipse a first class Python IDE -- It
comes with many goodies such as code completion, syntax highlighting,
syntax analysis, refactor, debug and many others.


Cheers,

-- 
Fabio Zadrozny
--
Software Developer

Aptana
http://aptana.com/python

Pydev Extensions
http://www.fabioz.com/pydev

Pydev - Python Development Enviroment for Eclipse
http://pydev.sf.net
http://pydev.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: iterating through files

[Mensanator]

On Feb 19, 4:56 pm, Steve Holden  wrote:
> Chris Rebert wrote:
> > On Thu, Feb 19, 2009 at 2:28 PM, Mike Driscoll  wrote:
> >> On Feb 19, 4:22 pm, Mike Driscoll  wrote:
> >>> On Feb 19, 3:56 pm, oamram  wrote:
>
>  Hi Pythonist,
>  new to python. i have a directory with about 50 text file and i need to
>  iterate through them and get
>  line 7 to 11 from each file and write those lines into another file(one 
>  file
>  that will contain all lines).
>  Cheers, Omer.
>  --
>  View this message in 
>  context:http://www.nabble.com/iterating-through-files-tp22048070p22048070.html
>  Sent from the Python - python-list mailing list archive at Nabble.com.
> >>> I would recommend using the glob module to grab a list of the files
> >>> you want or you can just create your own list. Then use a loop to grab
> >>> the lines you want. Something like this:
>
> >>> f = open(textFile)
> >>> newFile = open(newFileName, "a")
> >>> x = 1
> >>> for line in f.readlines():
> >>>     if x >=7 and x <=11:
> >>>          newFile.write(line + "\n")
>
> >>> You could even put the above inside a loop that loops over the list of
> >>> files. Anyway, that's one approach. I'm sure there are many others.
>
> >>> Mike
> >> Oops...I forgot to iterate the counter. The code should look like
> >> this:
>
> >> 
>
> >> f = open(textFile)
> >> newFile = open(newFileName, "a")
> >> x = 1
> >> for line in f.readlines():
> >>    if x >=7 and x <=11:
> >>         newFile.write(line + "\n")
> >>    x +=1
>
> >> 
>
> > Or you could use enumerate(); also, readlines() isn't necessary:
>
> > f = open(textFile)
> > newFile = open(newFileName, "a")
> > for x, line in enumerate(f):
> >     if x >=7 and x <=11:
> >         newFile.write(line + "\n")
>
> > Sounds a bit like homework to me though...
>
> But all these solutions read the whole file.  You should really read six
> lines throwing them away then read five lines to keep.

It might be useful not to assume every file has at least 11 lines
and program accordingly.

>
> Since this might be homework I'll not write the code, but it will
> involve f.next() ...
>
> regards
>   Steve
> --
> Steve Holden        +1 571 484 6266   +1 800 494 3119
> Holden Web LLC              http://www.holdenweb.com/- Hide quoted text -
>
> - Show quoted text -

--
http://mail.python.org/mailman/listinfo/python-list

Re: iterating through files

[Steve Holden]

Chris Rebert wrote:
> On Thu, Feb 19, 2009 at 2:28 PM, Mike Driscoll  wrote:
>> On Feb 19, 4:22 pm, Mike Driscoll  wrote:
>>> On Feb 19, 3:56 pm, oamram  wrote:
>>>
 Hi Pythonist,
 new to python. i have a directory with about 50 text file and i need to
 iterate through them and get
 line 7 to 11 from each file and write those lines into another file(one 
 file
 that will contain all lines).
 Cheers, Omer.
 --
 View this message in 
 context:http://www.nabble.com/iterating-through-files-tp22048070p22048070.html
 Sent from the Python - python-list mailing list archive at Nabble.com.
>>> I would recommend using the glob module to grab a list of the files
>>> you want or you can just create your own list. Then use a loop to grab
>>> the lines you want. Something like this:
>>>
>>> f = open(textFile)
>>> newFile = open(newFileName, "a")
>>> x = 1
>>> for line in f.readlines():
>>> if x >=7 and x <=11:
>>>  newFile.write(line + "\n")
>>>
>>> You could even put the above inside a loop that loops over the list of
>>> files. Anyway, that's one approach. I'm sure there are many others.
>>>
>>> Mike
>> Oops...I forgot to iterate the counter. The code should look like
>> this:
>>
>> 
>>
>> f = open(textFile)
>> newFile = open(newFileName, "a")
>> x = 1
>> for line in f.readlines():
>>if x >=7 and x <=11:
>> newFile.write(line + "\n")
>>x +=1
>>
>> 
> 
> Or you could use enumerate(); also, readlines() isn't necessary:
> 
> f = open(textFile)
> newFile = open(newFileName, "a")
> for x, line in enumerate(f):
> if x >=7 and x <=11:
> newFile.write(line + "\n")
> 
> Sounds a bit like homework to me though...
> 
But all these solutions read the whole file.  You should really read six
lines throwing them away then read five lines to keep.

Since this might be homework I'll not write the code, but it will
involve f.next() ...

regards
  Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

Tim Rowe wrote:
> 2009/2/19  :
>> Hi again
>>
>> I really want to imply that i am not in search of a perfect language.
>> Python for programming productivity is a great language but there are
>> some real world facts. Some people want a language that provides great
>> flexibility. A language can provide threads and processes and
>> programmer choose the way. I really believe that GIL is a design
>> error.
> 
> It's only an error if it gets in the way. It's the experience of a lot
> of programmers that it doesn't, so it's not an error.
> 
And it's not a feature of the language, rather of one or two
implementations. Neither JPython not IronPython use a GIL to the best of
my knowledge, so you are still quite at liberty to use them.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Tim Rowe]

2009/2/19  :
> Hi again
>
> I really want to imply that i am not in search of a perfect language.
> Python for programming productivity is a great language but there are
> some real world facts. Some people want a language that provides great
> flexibility. A language can provide threads and processes and
> programmer choose the way. I really believe that GIL is a design
> error.

It's only an error if it gets in the way. It's the experience of a lot
of programmers that it doesn't, so it's not an error.


-- 
Tim Rowe
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Tim Rowe]

2009/2/19 Paul Rubin :

> That Python is so much slower than C is yet another area where Python
> can use improvement.

No, because we don't use Python where C would be more appropriate.
Sure nobody would complain if Python were faster, but it's not for
speed that we choose Python. Not speed of /execution/ that is.
Different languages have different trade-offs. Python's trade-offs
suit us. If they don't suit you, use a language with trade-offs that
do.


-- 
Tim Rowe
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Tim Wintle]

On Thu, 2009-02-19 at 12:48 -0800, Falcolas wrote:
> That's fair, but by using a high level language in the first place,
> you've already made the conscious decision to sacrifice speed for ease
> of programming. Otherwise, you would probably be programming in C.
My parents would have gone mad at me for saying that when I was young -
C is just about the highest-level language they ever used - Assembly/hex
all the way!

So if you really want speed then why don't you write your code in
assembly? That's the only "perfect language" - it's capable of doing
everything in the most efficient way possible on your machine.

Of course that's a hassle, so I guess you may as well use C, since
that's almost certainly only linearly worse than using assembly, and it
takes far less time to use.

Oh, but then you may as well use python, since (again) that's probably
only linearly worse than C, and well-programmed C at that - I certainly
wouldn't end up with some of the optimisations that have gone into the
python interpreter!

That's basically what my mind goes through whenever I choose a language
to use for a task - and why I almost always end up with Python.

> It's a proposition that used to bother me, until I did some actual
> programming of real world problems in Python. I've yet to really find
> a case where the application was slow enough to justify the cost of
> using multiple Python processes.

I deal with those situations a fair bit - but the solutions are normally
easy - if it's limited by waiting for IO then I use threads, if it's
limited by waiting for CPU time then I use multiple processes, or share
the job over another application (such as MySQL), or run a task over a
cluster.

If you have a task where the linear optimisation offered by multiple
cores is really important then you can either:
 * Run it over multiple processes, or multiple machines in Python
or
 * spend a year writing it in C or assembly, by which time you can buy a
new machine that will run it fine in Python.


Yes, we're coming to a point where we're going to have tens of cores in
a chip, but by that time someone far cleverer than me (possibly someone
who's on this list) will have solved that problem. The top experts in
many fields use Python, and if they weren't able to make use of multiple
core chips, then there wouldn't be any demand for them.

Tim Wintle


> 
> ~G
> --
> http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: iterating through files

[Chris Rebert]

On Thu, Feb 19, 2009 at 2:28 PM, Mike Driscoll  wrote:
> On Feb 19, 4:22 pm, Mike Driscoll  wrote:
>> On Feb 19, 3:56 pm, oamram  wrote:
>>
>> > Hi Pythonist,
>> > new to python. i have a directory with about 50 text file and i need to
>> > iterate through them and get
>> > line 7 to 11 from each file and write those lines into another file(one 
>> > file
>> > that will contain all lines).
>>
>> > Cheers, Omer.
>> > --
>> > View this message in 
>> > context:http://www.nabble.com/iterating-through-files-tp22048070p22048070.html
>> > Sent from the Python - python-list mailing list archive at Nabble.com.
>>
>> I would recommend using the glob module to grab a list of the files
>> you want or you can just create your own list. Then use a loop to grab
>> the lines you want. Something like this:
>>
>> f = open(textFile)
>> newFile = open(newFileName, "a")
>> x = 1
>> for line in f.readlines():
>> if x >=7 and x <=11:
>>  newFile.write(line + "\n")
>>
>> You could even put the above inside a loop that loops over the list of
>> files. Anyway, that's one approach. I'm sure there are many others.
>>
>> Mike
>
> Oops...I forgot to iterate the counter. The code should look like
> this:
>
> 
>
> f = open(textFile)
> newFile = open(newFileName, "a")
> x = 1
> for line in f.readlines():
>if x >=7 and x <=11:
> newFile.write(line + "\n")
>x +=1
>
> 

Or you could use enumerate(); also, readlines() isn't necessary:

f = open(textFile)
newFile = open(newFileName, "a")
for x, line in enumerate(f):
if x >=7 and x <=11:
newFile.write(line + "\n")

Sounds a bit like homework to me though...

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: iterating through files

[Mike Driscoll]

On Feb 19, 4:22 pm, Mike Driscoll  wrote:
> On Feb 19, 3:56 pm, oamram  wrote:
>
> > Hi Pythonist,
> > new to python. i have a directory with about 50 text file and i need to
> > iterate through them and get
> > line 7 to 11 from each file and write those lines into another file(one file
> > that will contain all lines).
>
> > Cheers, Omer.
> > --
> > View this message in 
> > context:http://www.nabble.com/iterating-through-files-tp22048070p22048070.html
> > Sent from the Python - python-list mailing list archive at Nabble.com.
>
> I would recommend using the glob module to grab a list of the files
> you want or you can just create your own list. Then use a loop to grab
> the lines you want. Something like this:
>
> f = open(textFile)
> newFile = open(newFileName, "a")
> x = 1
> for line in f.readlines():
>     if x >=7 and x <=11:
>          newFile.write(line + "\n")
>
> You could even put the above inside a loop that loops over the list of
> files. Anyway, that's one approach. I'm sure there are many others.
>
> Mike

Oops...I forgot to iterate the counter. The code should look like
this:



f = open(textFile)
newFile = open(newFileName, "a")
x = 1
for line in f.readlines():
if x >=7 and x <=11:
 newFile.write(line + "\n")
x +=1



Sorry about that.

Mike
--
http://mail.python.org/mailman/listinfo/python-list

Re: iterating through files

[Mike Driscoll]

On Feb 19, 3:56 pm, oamram  wrote:
> Hi Pythonist,
> new to python. i have a directory with about 50 text file and i need to
> iterate through them and get
> line 7 to 11 from each file and write those lines into another file(one file
> that will contain all lines).
>
> Cheers, Omer.
> --
> View this message in 
> context:http://www.nabble.com/iterating-through-files-tp22048070p22048070.html
> Sent from the Python - python-list mailing list archive at Nabble.com.

I would recommend using the glob module to grab a list of the files
you want or you can just create your own list. Then use a loop to grab
the lines you want. Something like this:

f = open(textFile)
newFile = open(newFileName, "a")
x = 1
for line in f.readlines():
if x >=7 and x <=11:
 newFile.write(line + "\n")

You could even put the above inside a loop that loops over the list of
files. Anyway, that's one approach. I'm sure there are many others.

Mike
--
http://mail.python.org/mailman/listinfo/python-list

Tkinter: stopping a background thread on demand

[coldpizza]

I am writing a Tk GUI for a console script. I start a background
thread and pass it a callback function which writes debug info to a
Text widget in the main Tk loop.

I tried to find what is the established way to cleanly abort a child
thread when the user clicks the Stop button, but apparently Python
does not allow the parent process to explicitly stop a child thread.
Also I could not figure out how a parent can send a signal for a child
to stop and let it cleanup and exit cleanly.

I finally figured out a solution which was to set a 'TERMINATE' flag
on the `func_dict` property of the callback function, and then check
for the flag in the child module, but I still have doubts that this is
the proper way to do this sort of thing.

This works fine but it looks more like a dirty hack to me so my
question is: what is the proper way to send a signal to a child thread
to gracefully terminate, assuming the user decided to cancel an
operation?

from Tkinter import *
import threading

root = Tk()

class ConverterThread(threading.Thread):
def __init__(self, source, target, callbackFunc):
threading.Thread.__init__(self, name='converterThread')
self.infile = srcName
self.outfile = destName
self.callbackFunc = callbackFunc
def run(self):
import myconverter
myconverter.convert(self.infile, self.outfile,
self.callbackFunc)

def stopConversion(event=None):
global job
if (job and job.isAlive()):
callbackFunc.func_dict['TERMINATE'] = True
job = None

def updateLogCallback(data=None):
log.insert(END, str(data))
log.see('end')

job = ConverterThread(src='file1', dest='file2, updateLogCallback)
job.start()

stopBtn = Button(root, text="Stop", command=stopConversion)
stopBtn.pack()
log = Text(root)
root.mainloop()


And the callbackFunc in myConverter.py looks like this:

def printCallback(data):
if (callback):
callback(data)
if callback.func_dict['TERMINATE'] == True:
clean_up()
sys.exit()
else:
print data


--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[rushenaly]

Hi again

I really want to imply that i am not in search of a perfect language.
Python for programming productivity is a great language but there are
some real world facts. Some people want a language that provides great
flexibility. A language can provide threads and processes and
programmer choose the way. I really believe that GIL is a design
error.

Thanks.

Rushen
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Paul Rubin]

Falcolas  writes:
> That's fair, but by using a high level language in the first place,
> you've already made the conscious decision to sacrifice speed for ease
> of programming. Otherwise, you would probably be programming in C.

That Python is so much slower than C is yet another area where Python
can use improvement.  

> It's a proposition that used to bother me, until I did some actual
> programming of real world problems in Python. I've yet to really find
> a case where the application was slow enough to justify the cost of
> using multiple Python processes.

Right, that's basically the issue here: the cost of using multiple
Python processes is unnecessarily high.  If that cost were lower then
we could more easily use multiple cores to make oru apps faster.
--
http://mail.python.org/mailman/listinfo/python-list

Re:

[Chris Rebert]

On Thu, Feb 19, 2009 at 2:07 PM, Gary Wood  wrote:
>  I'm stuck on a tutorial Hands on Python3
>
> Exercise 1.13.7.3. ** Complete the following function. This starting code is
> in joinAllStub.py. Save it to the new name joinAll.py. Note the way an
> example is given in the documentation string. It simulates the use of the
> function in the Shell. This is a common convention:
>
> import string
> '''exercise to complete and test this function'''
> def joinStrings(sentance):
> '''Join all the strings in stringList into one string,
> and return the result. For example:
> >>> print joinStrings(['very', 'hot', 'day'])
> 'veryhotday'
> '''
>  # finish the code for this function
> # i tried this but not sure whats mising
> for i in sentance:
> print(i)

You're not supposed to output the strings, you've supposed to combine
them and return the single combined string (it's nearly identical to
how you summed the numbers). The printing is done by the calls to
print() outside of this function.

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

[no subject]

[Gary Wood]

 I'm stuck on a tutorial Hands on Python3
Exercise 1.13.7.3. ** Complete the following function. This starting code is in 
joinAllStub.py. Save it to the new name joinAll.py. Note the way an example is 
given in the documentation string. It simulates the use of the function in the 
Shell. This is a common convention: 

import string
'''exercise to complete and test this function'''
def joinStrings(sentance):
'''Join all the strings in stringList into one string,
and return the result. For example:
>>> print joinStrings(['very', 'hot', 'day'])
'veryhotday'
'''
 # finish the code for this function
# i tried this but not sure whats mising
for i in sentance:
print(i)


   #this worked for numbers  
   # def sumList(nums):#1 
   # '''Return the sum of the numbers in nums.'''   
#sum = 0   #2 
   # for num in nums:  #3 
   # sum = sum + num   #4 
  #  return sum   

   
   



def main():
print(joinStrings(['very','hot' ,'day']))
print(joinStrings(['this', 'is', 'it']))
print(joinStrings(['1', '2', '3', '4', '5']))

main()
--
http://mail.python.org/mailman/listinfo/python-list

iterating through files

[oamram]

Hi Pythonist,
new to python. i have a directory with about 50 text file and i need to
iterate through them and get 
line 7 to 11 from each file and write those lines into another file(one file
that will contain all lines).

Cheers, Omer.
-- 
View this message in context: 
http://www.nabble.com/iterating-through-files-tp22048070p22048070.html
Sent from the Python - python-list mailing list archive at Nabble.com.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Regression test test_site failed on current trunk

[Andy]

On 19 Feb., 21:47, Steve Holden  wrote:

> Do you mean you can't get previous lines in your Python command history
> to show up? This sounds as though you may not have built the readline
> support in to your experimental build. I can't remember exactly what the
> deal is, but I know that readline is GPL licensed, so it may not come as
> part of the standard distribution.

Thanks for your reply.
I just want to move left and right with the cursor keys.
Thats it.
I installed libreadline5-dev and rerun ./configure and make
...now it works.


--
http://mail.python.org/mailman/listinfo/python-list

Re: numpy.memmap advice?

[Lionel]

On Feb 19, 12:26 pm, Carl Banks  wrote:
> On Feb 19, 10:36 am, Lionel  wrote:
>
>
>
>
>
> > On Feb 19, 9:51 am, Carl Banks  wrote:
>
> > > On Feb 19, 9:34 am, Lionel  wrote:
>
> > > > On Feb 18, 12:35 pm, Carl Banks  wrote:
>
> > > > > On Feb 18, 10:48 am, Lionel  wrote:
>
> > > > > > Thanks Carl, I like your solution. Am I correct in my understanding
> > > > > > that memory is allocated at the slicing step in your example i.e. 
> > > > > > when
> > > > > > "reshaped_data" is sliced using "interesting_data = reshaped_data[:,
> > > > > > 50:100]"? In other words, given a huge (say 1Gb) file, a memmap 
> > > > > > object
> > > > > > is constructed that memmaps the entire file. Some relatively small
> > > > > > amount of memory is allocated for the memmap operation, but the bulk
> > > > > > memory allocation occurs when I generate my final numpy sub-array by
> > > > > > slicing, and this accounts for the memory efficiency of using 
> > > > > > memmap?
>
> > > > > No, what accounts for the memory efficienty is there is no bulk
> > > > > allocation at all.  The ndarray you have points to the memory that's
> > > > > in the mmap.  There is no copying data or separate array allocation.
>
> > > > Does this mean that everytime I iterate through an ndarray that is
> > > > sourced from a memmap, the data is read from the disc? The sliced
> > > > array is at no time wholly resident in memory? What are the
> > > > performance implications of this?
>
> > > Ok, sorry for the confusion.  What I should have said is that there is
> > > no bulk allocation *by numpy* at all.  The call to mmap does allocate
> > > a chunk of RAM to reflect file contents, but the numpy arrays don't
> > > allocate any memory of their own: they use the same memory as was
> > > allocated by the mmap call.
>
> > > Carl Banks- Hide quoted text -
>
> > > - Show quoted text -
>
> > Thanks for the explanations Carl. I'm sorry, but it's me who's the
> > confused one here, not anyone else :-)
>
> > I hate to waste everyone's time again, but something is just not
> > "clicking" in that black-hole I call a brain. So..."numpy.memmap"
> > allocates a chunk off the heap to coincide with the file contents. If
> > I memmap the entire 1 Gb file, a corresponding amount (approx. 1 Gb)
> > is allocated? That seems to contradict what is stated in the numpy
> > documentation:
>
> > "class numpy.memmap
> > Create a memory-map to an array stored in a file on disk.
>
> > Memory-mapped files are used for accessing small segments of large
> > files on disk, without reading the entire file into memory."
>
> Yes, it allocates room for the whole file in your process's LOGICAL
> address space.  However, it doesn't actually reserve any PHYSICAL
> memory, or read in any data from the disk, until you've actually
> access the data.  And then it only reads small chunks in, not the
> whole file.
>
> So when you mmap your 1GB file, the OS sets aside a 1 GB chunk of
> address to use for your memory map.  That's all it does: it doesn't
> read anything from disk, it doesn't reserve any physical RAM.  Later,
> when you access a byte in the mmap via a pointer, the OS notes that it
> hasn't yet loaded the data at that address, so it grabs a small chunk
> of physical ram and reads in the a small amount of data from the disk
> containing the byte you are accessing.  This all happens automatically
> and transparently to you.
>
> > In my previous example that we were working with (100x100 data file),
> > you used an offset to memmap the "lower-half" of the array. Does this
> > mean that in the process of memmapping that lower half, RAM was set
> > aside for 50x100 32-bit complex numbers? If so, and I decide to memmap
> > an entire file, there is no memory benefit in doing so.
>
> The mmap call sets aside room for all 100x100 32-bit complex numbers
> in logical address space, regardless of whether you use the offset
> parameter or not.  However, it might only read in part of the file in
> from disk, and will only reserve physical RAM for the parts it reads
> in.
>
> > At this point do you (or anyone else) recommend I just write a little
> > function for my class that takes the coords I intend to load and "roll
> > my own" function? Seems like the best way to keep memory to a minimum,
> > I'm just worried about performance. On the other hand, the most I'd be
> > loading would be around 1k x 1k worth of data.-
>
> No, if your file is not too large to mmap, just do it the way you've
> been doing it.  The documentation you've been reading is pretty much
> correct, even if you approach it naively.  It is both memory and I/O
> efficient.  You're overthinking things here; don't try to outsmart the
> operating system.  It'll take care of the performance issues
> satisfactorily.
>
> The only thing you have to worry about is if the file is too large to
> fit into your process's logical address space, which on a typical 32-
> bit system is 2-3 GB (depending on configuration) minus the space
> occupied by P

Re: Regular expression bug?

[Ron Garret]

In article ,
 Albert Hopkins  wrote:

> On Thu, 2009-02-19 at 10:55 -0800, Ron Garret wrote:
> > I'm trying to split a CamelCase string into its constituent components.  
> > This kind of works:
> > 
> > >>> re.split('[a-z][A-Z]', 'fooBarBaz')
> > ['fo', 'a', 'az']
> > 
> > but it consumes the boundary characters.  To fix this I tried using 
> > lookahead and lookbehind patterns instead, but it doesn't work:
> 
> That's how re.split works, same as str.split...

I think one could make the argument that 'foo'.split('') ought to return 
['f','o','o']

> 
> > >>> re.split('((?<=[a-z])(?=[A-Z]))', 'fooBarBaz')
> > ['fooBarBaz']
> > 
> > However, it does seem to work with findall:
> > 
> > >>> re.findall('(?<=[a-z])(?=[A-Z])', 'fooBarBaz')
> > ['', '']
> 
> 
> Wow!
> 
> To tell you the truth, I can't even read that...

It's a regexp.  Of course you can't read it.  ;-)

rg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[Ron Garret]

In article ,
 "andrew cooke"  wrote:

> i wonder what fraction of people posting with "bug?" in their titles here
> actually find bugs?

IMHO it ought to be an invariant that len(r.split(s)) should always be 
one more than len(r.findall(s)).

> anyway, how about:
> 
> re.findall('[A-Z]?[a-z]*', 'fooBarBaz')
> 
> or
> 
> re.findall('([A-Z][a-z]*|[a-z]+)', 'fooBarBaz')

That will do it.  Thanks!

rg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Explanation for trailing comma in example from Learning Python?

[Steve Holden]

alex goretoy wrote:
> Thank you for clerification Christian,
> when using trailing comma with print statement/function, does it not
> mean to output newline after printed data?
> 
It does (in Python before 3.0) - but that has nothing to do with the
original question. You will find you get quite different results for

print 1, # print integer 1 followed by no newline

and

print (1, )  # print the one-element tuple whose element
 # is the integer 1, followed by a newline.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[Ron Garret]

In article ,
 Peter Otten <__pete...@web.de> wrote:

> Ron Garret wrote:
> 
> > I'm trying to split a CamelCase string into its constituent components.
> 
> How about
> 
> >>> re.compile("[A-Za-z][a-z]*").findall("fooBarBaz")
> ['foo', 'Bar', 'Baz']

That's very clever.  Thanks!

> > (BTW, I tried looking at the source code for the re module, but I could
> > not find the relevant code.  re.split calls sre_compile.compile().split,
> > but the string 'split' does not appear in sre_compile.py.  So where does
> > this method come from?)
> 
> It's coded in C. The source is Modules/sremodule.c.

Ah.  Thanks!

rg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[Ron Garret]

In article ,
 MRAB  wrote:

> Ron Garret wrote:
> > I'm trying to split a CamelCase string into its constituent components.  
> > This kind of works:
> > 
>  re.split('[a-z][A-Z]', 'fooBarBaz')
> > ['fo', 'a', 'az']
> > 
> > but it consumes the boundary characters.  To fix this I tried using 
> > lookahead and lookbehind patterns instead, but it doesn't work:
> > 
>  re.split('((?<=[a-z])(?=[A-Z]))', 'fooBarBaz')
> > ['fooBarBaz']
> > 
> > However, it does seem to work with findall:
> > 
>  re.findall('(?<=[a-z])(?=[A-Z])', 'fooBarBaz')
> > ['', '']
> > 
> > So the regular expression seems to be doing the Right Thing.  Is this a 
> > bug in re.split, or am I missing something?
> > 
> > (BTW, I tried looking at the source code for the re module, but I could 
> > not find the relevant code.  re.split calls sre_compile.compile().split, 
> > but the string 'split' does not appear in sre_compile.py.  So where does 
> > this method come from?)
> > 
> > I'm using Python2.5.
> > 
> I, amongst others, think it's a bug (or 'misfeature'); Guido thinks it
> might be intentional, but changing it could break some existing code.

That seems unlikely.  It would only break where people had code invoking 
re.split on empty matches, which at the moment is essentially a no-op.  
It's hard to imagine there's a lot of code like that around.  What would 
be the point?

> You could do this instead:
> 
>  >>> re.sub('(?<=[a-z])(?=[A-Z])', '@', 'fooBarBaz').split('@')
> ['foo', 'Bar', 'Baz']

Blech!  ;-)  But thanks for the suggestion.

rg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Explanation for trailing comma in example from Learning Python?

[Chris Rebert]

> On Thu, Feb 19, 2009 at 2:54 PM, Christian Heimes  wrote:
>>
>> Carl Schumann wrote:
>> > I could see the logic in always or never having a trailing comma.   What
>> > I don't understand here is why only the single element case has a
>> > trailing comma.   Any explanations please?
>>
>> Does this code shad some light on the trailing comma? :)
>>
>> >>> (1) == 1
>> True
>> >>> (1,) == 1
>> False
>> >>> type((1))
>> 
>> >>> type((1,))
>> 
>>
>> >>> a = 1
>> >>> a
>> 1
>> >>> a = (1)
>> >>> a
>> 1
>> >>> a = (1,)
>> >>> a
>> (1,)
>> >>> a = 1,
>> >>> a
>> (1,)
>>
On Thu, Feb 19, 2009 at 12:57 PM, alex goretoy
 wrote:
> Thank you for clerification Christian,
> when using trailing comma with print statement/function, does it not mean to
> output newline after printed data?
>
> -Alex Goretoy
> http://www.goretoy.com

Yes it does (it outputs a space instead of a newline), but only with
the print-statement, not the print() function.

Cheers,
Chris
-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Explanation for trailing comma in example from Learning Python?

[ma]

A comma is what generates a tuple. It's not the parenthesis;)
http://docs.python.org/library/stdtypes.html#typesseq

"A single item tuple must have a trailing comma, such as (d,)."

On Thu, Feb 19, 2009 at 3:57 PM, alex goretoy
wrote:

> Thank you for clerification Christian,
> when using trailing comma with print statement/function, does it not mean
> to output newline after printed data?
>
> -Alex Goretoy
> http://www.goretoy.com
>
>
>
>
> On Thu, Feb 19, 2009 at 2:54 PM, Christian Heimes wrote:
>
>> Carl Schumann wrote:
>> > I could see the logic in always or never having a trailing comma.   What
>> > I don't understand here is why only the single element case has a
>> > trailing comma.   Any explanations please?
>>
>> Does this code shad some light on the trailing comma? :)
>>
>> >>> (1) == 1
>> True
>> >>> (1,) == 1
>> False
>> >>> type((1))
>> 
>> >>> type((1,))
>> 
>>
>> >>> a = 1
>> >>> a
>> 1
>> >>> a = (1)
>> >>> a
>> 1
>> >>> a = (1,)
>> >>> a
>> (1,)
>> >>> a = 1,
>> >>> a
>> (1,)
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: "metaclass conflict" error: where is noconflict ?

[Chris Rebert]

On Thu, Feb 19, 2009 at 5:01 AM, Barak, Ron  wrote:
> Hi,
>
> I have a class derived from two parents (in blue below), which gives me the
> following error:
>
> $ python -u ./failover_pickle_demo09.py
> Traceback (most recent call last):
>   File "./failover_pickle_demo09.py", line 291, in 
> class ListControl(wx.Frame, CopyAndPaste):
> TypeError: Error when calling the metaclass bases
> metaclass conflict: the metaclass of a derived class must be a
> (non-strict) subclass of the metaclasses of all its bases
> Googling suggested I should add
> from noconflict import classmaker
> and
>
> __metaclass__=classmaker()
> to this class.
>
> However, I don't seem able to find where to get the noconflict module from.
>
> Do any of you where noconflict  could be downloaded/installed from ?

>From what I could google, you should in theory be able to fix the
problem (without using any 3rd party module) by doing:

class ListControlMeta(type(wx.Frame), type(CopyAndPaste)):
pass

class ListControl(wx.Frame, CopyAndPaste):
__metaclass__ = ListControlMeta
#rest of class...

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Explanation for trailing comma in example from Learning Python?

[alex goretoy]

Thank you for clerification Christian,
when using trailing comma with print statement/function, does it not mean to
output newline after printed data?

-Alex Goretoy
http://www.goretoy.com



On Thu, Feb 19, 2009 at 2:54 PM, Christian Heimes  wrote:

> Carl Schumann wrote:
> > I could see the logic in always or never having a trailing comma.   What
> > I don't understand here is why only the single element case has a
> > trailing comma.   Any explanations please?
>
> Does this code shad some light on the trailing comma? :)
>
> >>> (1) == 1
> True
> >>> (1,) == 1
> False
> >>> type((1))
> 
> >>> type((1,))
> 
>
> >>> a = 1
> >>> a
> 1
> >>> a = (1)
> >>> a
> 1
> >>> a = (1,)
> >>> a
> (1,)
> >>> a = 1,
> >>> a
> (1,)
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3D CAD -- need collaborators, or just brave souls :)

[r]

On Feb 19, 2:29 am, Lie  wrote:
> On Feb 18, 8:02 pm, r  wrote:
> Blender's UI is designed for effective and efficient 3D workflow, not
> for low learning curve.

And that will be it's downfall!

I know what what Blenders UI is designed for. However not too many
people get religious about learning a UI, which is the only way you
will ever learn Blenders UI. This is why although Blender has a cult
following, it will never grow into a tool that the mainstream world
knows about, or for that matter cares about.

I have been using Blender on and off for almost a year and i still
cannot get comfortable with the UI. For instance adding a material to
a object involves multiple steps between multiple windows (WHAT!). Go
take a look at SketchUp's UI and you will see how the future will
look. Even 3DS or Maya is easier to learn that Blender.




--
http://mail.python.org/mailman/listinfo/python-list

Newbie request assistance....Python - PAMIE - Java

[frankrentef]

Greetings all...

Newbie here to the Python world.  I've written some basic script with
the purpose of automating testing of an application via Python /
Pamie.  I've had pretty good success until a spot I'm testing displays
a Java pop up.  I need to have Pamie access the fields in the window
(i.e. add a value and activate the "enter" button.)  Can someone
assist.

Some of the code I have currently is as follows...when the script I've
written gets to the end the java window pops up automaticallyI
don't understand how to interface with it.

THNX

ie.textBoxSet('_ctl0_ContentPlaceHolder1_Bocarc1_txtMAX', '5000.00')
time.sleep(2)
ie.textBoxSet('_ctl0_ContentPlaceHolder1_Bocarc1_txtMIN', '0')
time.sleep(2)
ie.buttonClick('_ctl0_ContentPlaceHolder1_Bocarc1_ckCheckMicr')
time.sleep(2)
ie.buttonClick('_ctl0_ContentPlaceHolder1_Bocarc1_ckCheckMicr')
ie.timesleep(2)
ie.textBoxSet('_ctl0_ContentPlaceHolder1_Bocarc1_txtMaxMicrRejects',
'1')
time.sleep(2)
ie.buttonClick('_ctl0_ContentPlaceHolder1_arc1_btnUpdate')
time.sleep2
--
http://mail.python.org/mailman/listinfo/python-list

Re: Explanation for trailing comma in example from Learning Python?

[Christian Heimes]

Carl Schumann wrote:
> I could see the logic in always or never having a trailing comma.   What
> I don't understand here is why only the single element case has a
> trailing comma.   Any explanations please?

Does this code shad some light on the trailing comma? :)

>>> (1) == 1
True
>>> (1,) == 1
False
>>> type((1))

>>> type((1,))


>>> a = 1
>>> a
1
>>> a = (1)
>>> a
1
>>> a = (1,)
>>> a
(1,)
>>> a = 1,
>>> a
(1,)

--
http://mail.python.org/mailman/listinfo/python-list

Re: Regression test test_site failed on current trunk

[Steve Holden]

Andy wrote:
> Hi,
> 
> I checked out the python trunk (curently 2.7a0), compiled it on my
> linux machine and run the regression test suit. Below is the output of
> the failed part:
> 
> test_site
> [14871 refs]
> test test_site failed -- Traceback (most recent call last):
>   File "/mybook/storage/python_lnx/Lib/test/test_site.py", line 105,
> in test_s_option
> self.assertEqual(rc, 1)
> AssertionError: 0 != 1
> 
> Is this something serious.
> 
I'd say any test failure without an obvious explanation is serious
enough to report as a bug. See http://bugs.python.org/

> I compiled python with:
> ./configure --prefix=/dev/null --with-pydebug
> make -s
> 
> Further I've got problems with moving the cursor when I started python
> (2.7a0). There are only esc sequences displayed. My Local installation
> (2.5.2) works fine.

Do you mean you can't get previous lines in your Python command history
to show up? This sounds as though you may not have built the readline
support in to your experimental build. I can't remember exactly what the
deal is, but I know that readline is GPL licensed, so it may not come as
part of the standard distribution.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Falcolas]

On Feb 19, 1:18 pm, Paul Rubin  wrote:
> ...  If such
> speedups were useless or unimportant, we would not have blown our hard
> earned cash replacing perfectly good older hardware, so we have to
> accept the concept that speed matters and ignore those platitudes that
> say otherwise.

That's fair, but by using a high level language in the first place,
you've already made the conscious decision to sacrifice speed for ease
of programming. Otherwise, you would probably be programming in C.

The question really is "Is it fast enough", and the answer usually is
"Yes". And when the answer is "No", there are many things which can be
done before the need to multi-thread the whole script comes about.

It's a proposition that used to bother me, until I did some actual
programming of real world problems in Python. I've yet to really find
a case where the application was slow enough to justify the cost of
using multiple Python processes.

~G
--
http://mail.python.org/mailman/listinfo/python-list

Explanation for trailing comma in example from Learning Python?

[Carl Schumann]

Hi,

I am surprised at the trailing comma in the following example from 
"Learning Python":

Python 2.3.4 (#1, Dec 10 2007, 15:05:56)
[GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(*args): print args
...
>>> f()
()
>>> f(1)
(1,)
>>> f(1,2)
(1, 2)
>>> f(1,2,3)
(1, 2, 3)
>>> f(1,2,3,4)
(1, 2, 3, 4)
>>>
I could see the logic in always or never having a trailing comma.   What 
I don't understand here is why only the single element case has a 
trailing comma.   Any explanations please?


Thanks,
Carl Schumann


--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Grant Edwards]

On 2009-02-19, Steve Holden  wrote:

> By the way, since you have chosen Java you might be interested
> to know that the JPython implementation (also open source)
> generates JVM bytecode, and allows you to freely mix Java and
> Python classes.
>
> There is no Global Interpreter Lock in JPython ...

I think somebody should propose adding one.

It would be a nice change of pace from the standard
never-ending thread(s) on the GIL.

-- 
Grant Edwards   grante Yow! I'm into SOFTWARE!
  at   
   visi.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Steve Holden]

rushen...@gmail.com wrote:
> Thank you Steve,
> 
> I really wanted to learn python, but as i said i don't want to make a
> dead investment. I hope someone can fix these design errors and maybe
> can write an interpreter in python :)
> 
> Thank you so much great community...
> Rushen

By the way, since you have chosen Java you might be interested to know
that the JPython implementation (also open source) generates JVM
bytecode, and allows you to freely mix Java and Python classes.

There is no Global Interpreter Lock in JPython ...

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: reading binary data from a 32 bit machine on 64 bit machine

[Gabriel Genellina]

En Thu, 19 Feb 2009 16:51:39 -0200, harijay  escribió:


Hi I am very confused with the use of the struct module to read binary
data from a file.
( I have only worked with ascii files so far)

I have a file spec for a Data-logger (http://www.dataq.com/support/
techinfo/ff.htm)


That format is rather convoluted -- due to historical reasons, I imagine...


I am collecting some voltage , time traces on one channel and they are
written to the binary file on a 32 bit windows machine

The file spec says that the number of header bytes in the data file
header is stored as a 16 bit  eye "I" at bits 6-7


If it says "at *byte* positions 6-7" you need a seek(6) to start reading  
from there, not seek(5).



Now I want to get at that number. When I try format !h I get a
meaningful number
If f is my file handle opened with "rb" mode


f.seek(5)
(Integer,) = struct.unpack('!h',f.read(2))
(Integer,)

(9348,)

I am assuming that means that there are 9348 header bytes . Can
someone look at the format spec and tell me if I am on the right
track.


Not exactly. Why '!' (network byte order)? The spec doesn't say about byte  
order, but since it's a Windows program we can assume little endian, '<'  
or just '=' (native).
But instead of multiple seeks + micro-reads I'd read the whole header and  
decode it at once (the fixed part is only 110 bytes long):


fixed_header_fmt = struct.Struct("Now, elements[4] is the fourth row in the table, "Number of bytes in each  
channel info entry"


The format is built from the Type column: UI -> H, I -> h, B -> B, UL ->  
L, L -> l, D -> d, F -> f.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: numpy.memmap advice?

[Carl Banks]

On Feb 19, 10:36 am, Lionel  wrote:
> On Feb 19, 9:51 am, Carl Banks  wrote:
>
>
>
>
>
> > On Feb 19, 9:34 am, Lionel  wrote:
>
> > > On Feb 18, 12:35 pm, Carl Banks  wrote:
>
> > > > On Feb 18, 10:48 am, Lionel  wrote:
>
> > > > > Thanks Carl, I like your solution. Am I correct in my understanding
> > > > > that memory is allocated at the slicing step in your example i.e. when
> > > > > "reshaped_data" is sliced using "interesting_data = reshaped_data[:,
> > > > > 50:100]"? In other words, given a huge (say 1Gb) file, a memmap object
> > > > > is constructed that memmaps the entire file. Some relatively small
> > > > > amount of memory is allocated for the memmap operation, but the bulk
> > > > > memory allocation occurs when I generate my final numpy sub-array by
> > > > > slicing, and this accounts for the memory efficiency of using memmap?
>
> > > > No, what accounts for the memory efficienty is there is no bulk
> > > > allocation at all.  The ndarray you have points to the memory that's
> > > > in the mmap.  There is no copying data or separate array allocation.
>
> > > Does this mean that everytime I iterate through an ndarray that is
> > > sourced from a memmap, the data is read from the disc? The sliced
> > > array is at no time wholly resident in memory? What are the
> > > performance implications of this?
>
> > Ok, sorry for the confusion.  What I should have said is that there is
> > no bulk allocation *by numpy* at all.  The call to mmap does allocate
> > a chunk of RAM to reflect file contents, but the numpy arrays don't
> > allocate any memory of their own: they use the same memory as was
> > allocated by the mmap call.
>
> > Carl Banks- Hide quoted text -
>
> > - Show quoted text -
>
> Thanks for the explanations Carl. I'm sorry, but it's me who's the
> confused one here, not anyone else :-)
>
> I hate to waste everyone's time again, but something is just not
> "clicking" in that black-hole I call a brain. So..."numpy.memmap"
> allocates a chunk off the heap to coincide with the file contents. If
> I memmap the entire 1 Gb file, a corresponding amount (approx. 1 Gb)
> is allocated? That seems to contradict what is stated in the numpy
> documentation:
>
> "class numpy.memmap
> Create a memory-map to an array stored in a file on disk.
>
> Memory-mapped files are used for accessing small segments of large
> files on disk, without reading the entire file into memory."

Yes, it allocates room for the whole file in your process's LOGICAL
address space.  However, it doesn't actually reserve any PHYSICAL
memory, or read in any data from the disk, until you've actually
access the data.  And then it only reads small chunks in, not the
whole file.

So when you mmap your 1GB file, the OS sets aside a 1 GB chunk of
address to use for your memory map.  That's all it does: it doesn't
read anything from disk, it doesn't reserve any physical RAM.  Later,
when you access a byte in the mmap via a pointer, the OS notes that it
hasn't yet loaded the data at that address, so it grabs a small chunk
of physical ram and reads in the a small amount of data from the disk
containing the byte you are accessing.  This all happens automatically
and transparently to you.


> In my previous example that we were working with (100x100 data file),
> you used an offset to memmap the "lower-half" of the array. Does this
> mean that in the process of memmapping that lower half, RAM was set
> aside for 50x100 32-bit complex numbers? If so, and I decide to memmap
> an entire file, there is no memory benefit in doing so.

The mmap call sets aside room for all 100x100 32-bit complex numbers
in logical address space, regardless of whether you use the offset
parameter or not.  However, it might only read in part of the file in
from disk, and will only reserve physical RAM for the parts it reads
in.


> At this point do you (or anyone else) recommend I just write a little
> function for my class that takes the coords I intend to load and "roll
> my own" function? Seems like the best way to keep memory to a minimum,
> I'm just worried about performance. On the other hand, the most I'd be
> loading would be around 1k x 1k worth of data.-

No, if your file is not too large to mmap, just do it the way you've
been doing it.  The documentation you've been reading is pretty much
correct, even if you approach it naively.  It is both memory and I/O
efficient.  You're overthinking things here; don't try to outsmart the
operating system.  It'll take care of the performance issues
satisfactorily.

The only thing you have to worry about is if the file is too large to
fit into your process's logical address space, which on a typical 32-
bit system is 2-3 GB (depending on configuration) minus the space
occupied by Python and other heap objects, which is probably only a
few MB.



Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[Paul Rubin]

sturlamolden  writes:
> Yes, the GIL prevents Python threads from being used in a certain way.
> But do you really need to use threads like that? Or do you just think
> you do?

How old is your computer, why did you buy it, and is it the first one
you ever owned?

For most of us, I suspect, it is not our first one, and we bought it
to get a processing speedup relative to the previous one.  If such
speedups were useless or unimportant, we would not have blown our hard
earned cash replacing perfectly good older hardware, so we have to
accept the concept that speed matters and ignore those platitudes that
say otherwise.

It used to be that new computers were faster than the old ones because
they ran at higher clock rates.  That was great, no software changes
at all were required to benefit from the higher speed.  Now, they get
the additional speed by having more cores.  That's better than nothing
but making use of it requires fixing the GIL.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Newby Question for reading a file

[steven.oldner]

On Feb 19, 1:44 pm, Curt Hash  wrote:
> On Thu, Feb 19, 2009 at 12:07 PM, steven.oldner  
> wrote:
>
> > On Feb 19, 12:40 pm, Mike Driscoll  wrote:
> > > On Feb 19, 12:32 pm, "steven.oldner"  wrote:
>
> > > > Simple question but I haven't found an answer.  I program in ABAP, and
> > > > in ABAP you define the data structure of the file and move the file
> > > > line into the structure, and then do something to the fields.  That's
> > > > my mental reference.
>
> > > > How do I separate or address each field in the file line with PYTHON?
> > > > What's the correct way of thinking?
>
> > > > Thanks!
>
> > > I don't really follow what you mean since I've never used ABAP, but
> > > here's how I typically read a file in Python:
>
> > > f = open("someFile.txt")
> > > for line in f:
> > >     # do something with the line
> > >     print line
> > > f.close()
>
> > > Of course, you can read just portions of the file too, using something
> > > like this:
>
> > > f.read(64)
>
> > > Which will read 64 bytes. For more info, check the following out:
>
> > >http://www.diveintopython.org/file_handling/file_objects.html
>
> > >  - Mike
>
> > Hi Mike,
>
> > ABAP is loosely based on COBOL.
>
> > Here is what I was trying to do, but ended up just coding in ABAP.
>
> > Read a 4 column text file of about 1,000 lines and compare the 2
> > middle field of each line.  If there is a difference, output the line.
>
> > The line's definition in ABAP is PERNR(8) type c, ENDDA(10) type c,
> > BEGDA(10) type c, and LGART(4) type c.
> > In ABAP the code is:
> > LOOP AT in_file.
> >  IF in_file-endda <> in_file-begda.
> >    WRITE:\ in_file. " that's same as python's print
> >  ENDIF.
> > ENDLOOP.
>
> > I can read the file, but didn't know how to look st the fields in the
> > line.  From what you wrote, I need to read each segment/field of the
> > line?
>
> > Thanks,
>
> > Steve
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
> You could do something like this:
>
> f = open('file.txt', 'r')
> for line in f:
>     a,b = line.split()[1:-1]   # tokenize the string into sequence of
> length 4 and store two middle values in a and b
>     if a != b:
>         print line
> f.close()- Hide quoted text -
>
> - Show quoted text -

Peter, you are correct, just fields glued together.  I did not know
there was a struc module and that code looks real good.  This is
something I will use in the future. Thanks!

Curt, that looks good also.  I just need to test the 2 middle values.
I didn't know how to store line.split values into variables and this
is simple. Thanks!

Again, thanks Mike, Peter and Curt.  Now if you ever need to know
ABAP... ;)




--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[MRAB]

Ron Garret wrote:
I'm trying to split a CamelCase string into its constituent components.  
This kind of works:



re.split('[a-z][A-Z]', 'fooBarBaz')

['fo', 'a', 'az']

but it consumes the boundary characters.  To fix this I tried using 
lookahead and lookbehind patterns instead, but it doesn't work:



re.split('((?<=[a-z])(?=[A-Z]))', 'fooBarBaz')

['fooBarBaz']

However, it does seem to work with findall:


re.findall('(?<=[a-z])(?=[A-Z])', 'fooBarBaz')

['', '']

So the regular expression seems to be doing the Right Thing.  Is this a 
bug in re.split, or am I missing something?


(BTW, I tried looking at the source code for the re module, but I could 
not find the relevant code.  re.split calls sre_compile.compile().split, 
but the string 'split' does not appear in sre_compile.py.  So where does 
this method come from?)


I'm using Python2.5.


I, amongst others, think it's a bug (or 'misfeature'); Guido thinks it
might be intentional, but changing it could break some existing code.
You could do this instead:

>>> re.sub('(?<=[a-z])(?=[A-Z])', '@', 'fooBarBaz').split('@')
['foo', 'Bar', 'Baz']
--
http://mail.python.org/mailman/listinfo/python-list

Re: "Maximum recursion depth exceeded"...why?

[Thomas Allen]

On Feb 18, 10:15 pm, rdmur...@bitdance.com wrote:
> Thomas Allen  wrote:
> > On Feb 18, 4:51 am, alex23  wrote:
> > > On Feb 18, 7:34 pm, rdmur...@bitdance.com wrote:
>
> > > > Yeah, but wget -r -k will do that bit of it, too.
>
> > > Wow, nice, I don't know why I never noticed that. Cheers!
>
> > Hm...doesn't do that over here. I thought it may have been because of
> > absolute links (not to site root), but it even leaves things like  > href="/">. Does it work for you guys?
>
> It works for me.  The sample pages I just tested on it don't use
> any href="/" links, but my 'href="/about.html"' got properly
> converted to 'href="../about.html"'.  (On the other hand my '/contact.html'
> got converted to a full external URL...but that's apparently because the
> contact.html file doesn't actually exist :)
>
> --RDM

Thanks for the help everyone. The idea of counting the slashes was the
linchpin of this little script, and with a little trial and error, I
successfully generated a local copy of the site. I don't think my
colleague knows what went into this, but he seemed appreciative :^)

Thomas
--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[Peter Otten]

Ron Garret wrote:

> I'm trying to split a CamelCase string into its constituent components.

How about

>>> re.compile("[A-Za-z][a-z]*").findall("fooBarBaz")
['foo', 'Bar', 'Baz']

> This kind of works:
> 
 re.split('[a-z][A-Z]', 'fooBarBaz')
> ['fo', 'a', 'az']
> 
> but it consumes the boundary characters.  To fix this I tried using
> lookahead and lookbehind patterns instead, but it doesn't work:
> 
 re.split('((?<=[a-z])(?=[A-Z]))', 'fooBarBaz')
> ['fooBarBaz']
> 
> However, it does seem to work with findall:
> 
 re.findall('(?<=[a-z])(?=[A-Z])', 'fooBarBaz')
> ['', '']
> 
> So the regular expression seems to be doing the Right Thing.  Is this a
> bug in re.split, or am I missing something?

IRC the split pattern must consume at least one character, but I can't find
the reference.
 
> (BTW, I tried looking at the source code for the re module, but I could
> not find the relevant code.  re.split calls sre_compile.compile().split,
> but the string 'split' does not appear in sre_compile.py.  So where does
> this method come from?)

It's coded in C. The source is Modules/sremodule.c.

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: reading binary data from a 32 bit machine on 64 bit machine

[MRAB]

harijay wrote:

Hi I am very confused with the use of the struct module to read binary
data from a file.
( I have only worked with ascii files so far)

I have a file spec for a Data-logger (http://www.dataq.com/support/
techinfo/ff.htm)

I am collecting some voltage , time traces on one channel and they are
written to the binary file on a 32 bit windows machine

The file spec says that the number of header bytes in the data file
header is stored as a 16 bit  eye "I" at bits 6-7

Now I want to get at that number. When I try format !h I get a
meaningful number
If f is my file handle opened with "rb" mode


f.seek(5)
(Integer,) = struct.unpack('!h',f.read(2))
(Integer,)

(9348,)

I am assuming that means that there are 9348 header bytes . Can
someone look at the format spec and tell me if I am on the right
track.

Also if a binary guru can point me in the direction of a single line
format pythonic way to get at the header that will be great


The number of header bytes is stored in bytes 6-7, so you need seek(6),
and the 16-bit value is little-endian as far as I can tell:

f.seek(6)
header_bytes, = struct.unpack('http://mail.python.org/mailman/listinfo/python-list

Re: Strange array.array performance

[Maxim Khitrov]

On Thu, Feb 19, 2009 at 2:23 PM, Gabriel Genellina
 wrote:
> En Thu, 19 Feb 2009 16:52:54 -0200, Maxim Khitrov 
> escribió:
>
>> input = array('B', range(256) * 1)
>>
>> # Case 1
>> start = clock()
>> data1 = array('B', input)
>> print format(clock() - start, '.10f')
>
>> That seems very wrong. In the end, all arrays have the same data, but
>> by specifying it in the constructor the creation process takes over
>> 350x longer than the other two methods. Is this a bug, or is there
>> some valid reason for it?
>
> It's a known issue: http://bugs.python.org/issue5109

I see, thanks.
--
http://mail.python.org/mailman/listinfo/python-list