Merging pdf files based on a value in a field

[MrJean1]

Try PyPDF2, see the merge example.



/Jean
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: packaging python code

[MrJean1]

> Is there any way to pack my .py with all required libraries and create a self 
> running package? Something like building exe file with static libraries. 
> Therefore, the user won't install any thing manually.


Try Py2exe  to create a standalone .exe file (but for 
Windows only though).

/Jean
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Program uses twice as much memory in Python 3.6 than in Python 3.5

[MrJean1]

Similarly, on  macOS 10.12.3 Sierra:

% python3.5
Python 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 08:49:46) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> s = set(range(10))
>>> sys.getsizeof(s)
736
>>> sys.getsizeof(set(s))
736
>>> sys.getsizeof(set(set(s)))
736
>>>

% python3
Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> s = set(range(10))
>>> sys.getsizeof(s)
736
>>> sys.getsizeof(set(s))
1248
>>> sys.getsizeof(set(set(s)))
1248
>>>

% python2
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> s = set(range(10))
>>> sys.getsizeof(s)
744
>>> sys.getsizeof(set(s))
744
>>> sys.getsizeof(set(set(s)))
744
>>> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Quick way to calculate lines of code/comments in a collection of Python scripts?

[MrJean1]

On Wednesday, October 5, 2016 at 1:57:14 PM UTC-4, Malcolm Greene wrote:
> Looking for a quick way to calculate lines of code/comments in a
> collection of Python scripts. This isn't a LOC per day per developer
> type analysis - I'm looking for a metric to quickly judge the complexity
> of a set of scripts I'm inheriting.
> 
> Thank you,
> Malcolm

Here is a basic LOC counter for Python source files

best text editor for programming Python on a Mac

[MrJean1]

Try TextWrangler from BareBones 
.  I've been using that for 
years on MacOS X.

/Jean
-- 
https://mail.python.org/mailman/listinfo/python-list

How to get all IP addresses in python?

[MrJean1]

Try function getIPs from this module



/Jean
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fork/exec close file descriptors

[MrJean1]

The subprocess module uses upper bound MAXFD which is defined as

code
try:
MAXFD = os.sysconf(SC_OPEN_MAX)
except:
MAXFD = 256
/code

/Jean
-- 
https://mail.python.org/mailman/listinfo/python-list

Create on Win and run on Win or Mac

[MrJean1]

See this thread for possible answers

http://stackoverflow.com/questions/2933/how-can-i-create-a-directly-executable-cross-platform-gui-app-using-python

/Jean
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fixing Python install on the Mac after running 'CleanMyMac'

[MrJean1]

FWIW, I recently upgraded an older MacBook to Mac OS X 10.7.5 and there are 3 
different versions of Python in /System/Library/Frameworks/Python.framework, 
see:

$ ls /System/Library/Frameworks/Python.framework/Versions/
2.5 2.6 2.7 Current


It is unclear whether MacOS X 10.7.5 installed all four of these Python 
versions.  The older one(s) may be left over from the previous MacOS X release 
on the machine.  But all are Apple builds of the Python version, for example:

$ python2.7
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type help, copyright, credits or license for more information.
 ^D


In addition, there is a link in /usr/bin to the python2.X, pythonw2.X and 
python2.X-config file for each version, see

$ ls -la /usr/bin/python*
-rwxr-xr-x  2 root  wheel  62752 May 21 19:06 /usr/bin/python
-rwxr-xr-x  6 root  wheel925 May 21 19:06 /usr/bin/python-config
lrwxr-xr-x  1 root  wheel 75 May 21 19:06 /usr/bin/python2.5 - 
../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python2.5
lrwxr-xr-x  1 root  wheel 82 May 21 19:06 /usr/bin/python2.5-config - 
../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python2.5-config
lrwxr-xr-x  1 root  wheel 75 May 21 19:06 /usr/bin/python2.6 - 
../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
lrwxr-xr-x  1 root  wheel 82 May 21 19:06 /usr/bin/python2.6-config - 
../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6-config
lrwxr-xr-x  1 root  wheel 75 May 21 19:06 /usr/bin/python2.7 - 
../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
lrwxr-xr-x  1 root  wheel 82 May 21 19:06 /usr/bin/python2.7-config - 
../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config
-rwxr-xr-x  2 root  wheel  62752 May 21 19:06 /usr/bin/pythonw
lrwxr-xr-x  1 root  wheel 76 May 21 19:06 /usr/bin/pythonw2.5 - 
../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/pythonw2.5
lrwxr-xr-x  1 root  wheel 76 May 21 19:06 /usr/bin/pythonw2.6 - 
../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/pythonw2.6
lrwxr-xr-x  1 root  wheel 76 May 21 19:06 /usr/bin/pythonw2.7 - 
../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw2.7


Perhaps you can reinstall these Pythons from the CD which came with the Mac.  
Let me know if you need  more info.

/Jean




On Thursday, May 28, 2015 at 11:47:52 PM UTC-4, Laura Creighton wrote:
 webmas...@python.org just got some mail from some poor embarrased
 soul who ran this program and broke their Python install.
 
 They are running Mac OSX 10.7.5
 
 They are getting:
 
  Utility has encountered a fatal error, and will now terminate.  A
  Python runtime could not be located. You may need to install a
  framework build of Python or edit the PyRuntimeLocations array in this
  applications info.plist file.  Then there are two oblong circles. One
  says Open Console. The other says Terminate.
 
 So https://docs.python.org/2/using/mac.html says:
 
The Apple-provided build of Python is installed in
/System/Library/Frameworks/Python.framework and /usr/bin/python,
respectively. You should never modify or delete these, as they are
Apple-controlled and are used by Apple- or third-party software.
 
 So, I assume this poor soul has done precisely that.
 
 What do I tell her to do now?
 
 Laura
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fixing Python install on the Mac after running 'CleanMyMac'

[MrJean1]

Correct, the Current version is just a link to 2.7.  Also, the binaries 
'/usr/bin/python{,w} seem to be copies of the 
/System/Library/.../Versions/2.7/bin/python{,w} files.

/Jean


On Friday, May 29, 2015 at 2:01:05 PM UTC-4, Ned Deily wrote:
 In article bffd4736-f9ff-492e-9069-836f6e786...@googlegroups.com,
  MrJean1 mrje...@gmail.com wrote:
  FWIW, I recently upgraded an older MacBook to Mac OS X 10.7.5 and there are 
  3 
  different versions of Python in 
  /System/Library/Frameworks/Python.framework, 
  see:
  
  $ ls /System/Library/Frameworks/Python.framework/Versions/
  2.5 2.6 2.7 Current
   
  It is unclear whether MacOS X 10.7.5 installed all four of these Python 
  versions.  The older one(s) may be left over from the previous MacOS X 
  release on the machine
 
 For compatibility with applications linked on older OS X releases, Apple 
 customarily ships more than one version of Python frameworks with OS X.  
 There are actually three versions of the system Python frameworks there: 
 you should see that Current is actually a symbolic link to 2.7.  All 
 of those versions are shipped as part of OS X 10.7.
 
 -- 
  Ned Deily,
  n...@acm.org

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Measure the amount of memory used?

[MrJean1]

Take a look it this recipe (for Linux only):

http://code.activestate.com/recipes/286222/

/Jean

On Aug 18, 8:08 am, Jack Bates ms...@freezone.co.uk wrote:
 I wrote a content filter for Postfix with 
 Python,https://github.com/jablko/cookie

 It should get started once, and hopefully run for a long time - so I'm
 interested in how it uses memory:

  1) How does the amount of memory used change as it runs?

  2) How does the amount of memory used change as I continue to hack on
 it, and change the code?

 My naive thought was that I'd periodically append to a file, the virtual
 memory size from /proc/[pid]/stat and a timestamp. From this a could
 make a graph of the amount of memory used as my content filter runs, and
 I could compare two graphs to get a clue whether this amount changed as
 I continue to hack

  - but some Googling quickly revealed that measuring memory is actually
 quite complicated? Neither the virtual memory size nor the resident set
 size accurately measure the amount of memory used by a process

 Has anyone else measured the memory used by a Python program? How did
 you do it?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Get the IP address of WIFI interface

[MrJean1]

Perhaps, this recipe works for your case:

  http://code.activestate.com/recipes/577191

It does parse ifconfig and ipconfig, if found.

/Jean



On May 15, 2:14 pm, Andrew Berg bahamutzero8...@gmail.com wrote:
 On 2011.05.15 06:12 AM, Tim Golden wrote: ... and for Windows:

  code
  import wmi

  for nic in wmi.WMI ().Win32_NetworkAdapterConfiguration (IPEnabled=1):
     print nic.Caption, nic.IPAddress

  /code

 One thing I found out about Win32_NetworkAdapterConfiguration is that it
 only contains /current/ information and not the stored info that it uses
 when making an initial connection (you can see and edit this info in the
 Network and Sharing Center applet). The difference is that if you're
 offline, that WMI object will have no useful info at all. You can find
 the info in the registry if you know what the UUID (or whatever it is)
 of (or assigned to) the interface (it's in
 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters\Inter 
 faces).
 The OP said the card would be connected, so it might not be an issue,
 but I think it's important to know that. Wouldn't want you to suddenly
 get blank strings or exceptions and not know why. ;-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython on the Web

[MrJean1]

FireFox 3.6.13 on MacOS X Tiger (10.4.11) fails:

  Error: too much recursion
  Error: Modules is not defined
  Source File: http://synthensity.com/static/python.html

/Jean

On Jan 2, 11:26 pm, Wolfgang Strobl ne...@mystrobl.de wrote:
 azakai alonmozi...@gmail.com:

 On Jan 2, 4:58 pm, pyt...@bdurham.com wrote:
  Azakai/Gerry,

   Errors when using Firefox 3.6.3:

  I'm running Firefox 3.6.1.3 and the interpreter is running fine.

 I guess that meant FIrefox 3.6.13 (without the last dot), the current
 stable version.

 I'm using Firefox 3.6.13 (german) on Windowx XP (32bit, german) here,
 and the interpreter is running fine, too.  Same for Chrome 8.0.552.224.

 --
 Wir danken f r die Beachtung aller Sicherheitsbestimmungen

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython on the Web

[MrJean1]

FYI,

The example

  http://syntensity.com/static/python.html

works fine in Safari 4.1.3 on MacOS X Tiger (10.4.11).

/Jean


On Jan 3, 5:59 pm, azakai alonmozi...@gmail.com wrote:
 On Jan 3, 12:23 pm, Gerry Reno gr...@verizon.net wrote:





  On 01/03/2011 03:10 PM, azakai wrote:

   On Jan 2, 5:55 pm, Gerry Reno gr...@verizon.net wrote:

   I tried printing sys.path and here is the output:

   ['', '/usr/local/lib/python27.zip', '/usr/local/lib/python2.7/',
   '/usr/local/lib/python2.7/plat-linux2',
   '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old',
   '/usr/local/lib/lib-dynload']

   Now, those paths must be on your machine because they are not on my
   client machine.  But the interpreter is now running on MY machine.  Well
   in a sandbox really.  So how is that going to work?

   Yeah, those are the paths on the machine where the binary was compiled
   (so, they are the standard paths on ubuntu).

   Anyhow the filesystem can't (and shouldn't) be accessed from inside a
   browser page.

  Well, the local filesystem could be accessible with the user's
  permission and this should be an option.

 Hmm, I think this might be possible with the HTML5 File API. Would
 definitely be useful here.

 - azakai

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compile on SunOS?

[MrJean1]

These command lines used to build 32-bit Python 2.4 and 2.5 on Solaris
10 Opteron using SUN's compilers:

  setenv LD_LIBRARY_PATH 
  env CCSHARED=-KPIC LDSHARED=cc -xtarget=native -G LDFLAGS=-
xtarget=native CC=cc \
   CPP=cc-xtarget=native -E BASECFLAGS=-xtarget=native OPT=-xO5
CFLAGS=-xtarget=native \
   CXX=CC -xtarget=native ./configure --enable-shared --without-gcc
--disable-ipv6 --prefix=
  make

Replace the 's accordingly.  Use BASECFLAGS=-xtarget=native -
xlibmieee for IEEE floating point.

Disclaimer, I have not tried building Python 2.6 nor 2.7.

/Jean


On Dec 31, 8:34 am, Alex Zhang cheungti...@gmail.com wrote:
 Hi All,
 I'm trying to build Python 2.7.1 on Sun Solaris 10 amd64, however end up
 with:

 Python build finished, but the necessary bits to build these modules
 were not found:
 _bsddb             _tkinter           bsddb185
 gdbm               linuxaudiodev      ossaudiodev
 To find the necessary bits, look in setup.py in detect_modules() for the
 module's name.

 Failed to build these modules:
 _bisect            _codecs_cn         _codecs_hk
 _codecs_iso2022    _codecs_jp         _codecs_kr
 _codecs_tw         _collections       _csv
 _ctypes            _ctypes_test       _curses
 _curses_panel      _elementtree       _functools
 _hashlib           _heapq             _hotshot
 _io                _json              _locale
 _lsprof            _multibytecodec    _multiprocessing
 _random            _socket            _sqlite3
 _ssl               _struct            _testcapi
 array              audioop            binascii
 bz2                cmath              cPickle
 crypt              cStringIO          datetime
 dbm                dl                 fcntl
 future_builtins    grp                imageop
 itertools          math               mmap
 nis                operator           parser
 pyexpat            resource           select
 spwd               strop              sunaudiodev
 syslog             termios            time
 unicodedata        zlib

 running build_scripts

 I am using cc provided in Solaris 10, readline downloaded from GNU and
 compiled in 32bit. Also, I added this entry:

 readline readline.c -I/local32/include -L/local32/lib -R/local32/lib
 -lreadline -ltermcap

 to Modules/Setup.local in order to get readline running.
 Currently:

 dns# /opt/python/bin/python
 Python 2.7.1 (r271:86832, Dec 31 2010, 07:21:22) [C] on sunos5
 Type help, copyright, credits or license for more information.
   import hashlib
 Traceback (most recent call last):
    File stdin, line 1, in module
    File /opt/python/lib/python2.7/hashlib.py, line 136, in module
      globals()[__func_name] = __get_hash(__func_name)
    File /opt/python/lib/python2.7/hashlib.py, line 71, in
 __get_builtin_constructor
      import _md5
 ImportError: No module named _md5

 I can not use hashlib and many other modules however I can use the rest
 modules.
 Thanks for all your kind reply.
 --
 OSQDU::Alex

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to measure TCP congestion windows using python ??

[MrJean1]

FWIW, on CentOS 4.7, the ctypes version works fine, but the struct
version fails, because len(tcp_info) is only 100 bytes while
struct.calcsize('B...L') is 104.

However, if the format is changed to '7B23L', i.e. one 'L' shorter,
the struct version works and returns to same result as the ctypes
version.

/Jean


On Dec 19, 4:45 pm, plz u...@compgroups.net/ wrote:
 hi
 many thanks for helping me
 i also tried to manipulate it last night
 here is my code...

 import socket
 import struct

 sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
 tcp_info = sock.getsockopt(socket.SOL_TCP, socket.TCP_INFO, 
 struct.calcsize('BBB'))
 print struct.unpack('BBB', tcp_info)

 the result of struct.unpack of tcp_info is following
 in /usr/include/linux/tcp.h
 used Linux Redhat and Python 2.7

 anyway your code is very useful
 Thxs again ;)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ctypes question

[MrJean1]

Try again after changing line 16 to

  sn = SmiGetNode(None, 1.3.6.1.2.1.2.2)

Because, SmiGetNode is a Python function which accepts Python objects
as arguments.  Passing is a ctypes object oid is incorrect.

/Jean


On Dec 14, 10:36 am, News Wombat newswom...@gmail.com wrote:
 On Dec 11, 12:59 pm, MrJean1 mrje...@gmail.com wrote:

  In general, for shared libraries, you need to define those first as
  prototype using ctypes.CFUNCTYPE() and then instantiate each prototype
  once supplying the necessary parameter flags using
  prototype(func_spec, tuple_of_param_flags).  See sections 15.16.2.3
  and 4 of the ctypes docs*.

 I tried the cfuntype and proto steps, and it's not crashing now
 (that's good), but now i'm just left with null pointers as a return
 object.  I'm still working through all of the examples you sent.  They
 were extremely helpful.  Here's where I'm at now...

 What is strange is I can actually get smiGetNode to work if I don't
 cfunctype/proto it.  If i do, nada.  however, the smiGetNextNode fails
 no matter what, usually with a segfault, but depending on how i
 construct it, sometimes a null pointer.

 constants.py:http://pastebin.com/f3b4Wbf0
 libsmi.py:http://pastebin.com/XgtpG6gr
 smi.c (the actual function):http://pastebin.com/Pu2vabWM

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ctypes question

[MrJean1]

It is not entirely clear what the functions and especially what their
signatures are in that C library clibsmi.

In general, for shared libraries, you need to define those first as
prototype using ctypes.CFUNCTYPE() and then instantiate each prototype
once supplying the necessary parameter flags using
prototype(func_spec, tuple_of_param_flags).  See sections 15.16.2.3
and 4 of the ctypes docs*.

Take a look the Python bindings** for the VLC library, the file called
vlc.py***.  The function _Cfunction is used to create the Python
callable for each C function in that VLC library.  All the Python
callables are in the second half of the vlc.py file, starting at line
2600.

Hope this helps,

/Jean

*) http://docs.python.org/library/ctypes.html#foreign-functions

**) http://wiki.videolan.org/Python_bindings

***) http://git.videolan.org/?p=vlc/bindings/
python.git;a=tree;f=generated;b=HEAD


On Dec 10, 3:32 pm, News Wombat newswom...@gmail.com wrote:
 Hi everyone,

 I've been experimenting with the ctypes module and think it's great.
 I'm hitting a few snags though with seg faults.  I attached two links
 that holds the code.  The line i'm having problems with is this,

 sn=clibsmi.smiGetNextNode(pointer(sno),SMI_NODEKIND_ANY)

 It will work one time, and if I call it again with the result of the
 previous, even though the result (a c struct) looks ok, it will
 segfault.  I think it's a problem with pointers or maybe the function
 in the c library trying to change a string that python won't let it
 change.  I'm stuck, any tips would be appreciated.  Thanks, and Merry
 Christmas!

 constants.py:http://pastebin.com/HvngjzZN
 libsmi.py:http://pastebin.com/19C9kYEa

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleHTTPServer, external CSS, and Google Chrome

[MrJean1]

FWIW,

There is a blue text on a red background in all 4 browsers Google
Chrome 6.0.472.59, Safari 5.0.1 (7533.17.8), FireFox 3.6.9 and IE
6.0.2900.5512 with Python 2.7 serving that page on my Windows XP
SP 3 machine.

/Jean

On Sep 16, 11:59 pm, Justin Ezequiel justin.mailingli...@gmail.com
wrote:
 I am running python -m SimpleHTTPServer 80 on Windows XP Pro SP 3
 (Python 2.5.4)

 browsinghttp://localhost/using IE8 and FireFox 3.6, I get blue text
 on red background
 on Google Chrome 6.0 however, I get blue text on white background
 placing index.htm and styles.css (see below) under IIS, I get blue
 text on red background for all browsers,
 including Google Chrome 6.0.

 I get exactly the same results when browsing from another machine.
 what's wrong? what do I need to change in SimpleHTTPServer?

 index.htm
 --
 !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN
    http://www.w3.org/TR/html4/strict.dtd;
 html
 head
 titleyour title/title
 meta http-equiv=Content-Type content=text/
 html;charset=ISO-8859-1
 link rel=stylesheet href=styles.css type=text/css
 style type=text/css
 body {
     color: blue;}

 /style
 /head
 bodypfoo bar/p
 /body
 /html
 --

 styles.css
 --
 body {
     background-color: red;}

 --

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Windows: How to detect whether a Python app/script is running in console/GUI mode?

[MrJean1]

On Jul 27, 8:36 am, Tim Golden m...@timgolden.me.uk wrote:
 On 27/07/2010 15:58, Brian Curtin wrote:

  On Tue, Jul 27, 2010 at 09:36,pyt...@bdurham.com  wrote:

  Windows: How can I detect whether a Python app/script is running in
  console/GUI mode? By app I mean a script compiled to an exe via py2exe or
  similar.

  Thank you,
  Malcolm

  I don't remember much about py2exe, but you could check if
  ``os.path.split(sys.executable)[1]`` equals pythonw.exe (typical for GUIs)
  or just python.exe (regular console).

 Don't know whether it's foolproof, but I suspect that
 checking whether win32console.GetConsoleWindow ()
 returns zero is probably not a bad approach.

 TJG    

Executables built with py2exe have an attribute sys.frozen and its
value is 'console_exe' for console applications or 'windows_exe' for
GUI applications.  See for example http://www.py2exe.org/index.cgi/
Py2exeEnvironment.

/Jean
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What does this PyChecker warning mean?

[MrJean1]

Although PyChecker 0.8.18 is quite an improvement over previous
releases, it does have quirks.  The PyChecker postprocessor might be
helpful, see

  http://code.activestate.com/recipes/546532/

/Jean


On Jun 1, 4:48 am, Leo Breebaart l...@lspace.org wrote:
 When fed the following code:

  def Foo():

     class A(object):
         def __init__(self):
             pass

     class B(object):
         def __init__(self):
             pass

 PyChecker 0.8.18 warns:

   foo.py:9: Redefining attribute (__init__) original line (5)

 I do not understand what is meant by this warning. In fact, it
 simply seems wrong -- but I have learned not to jump to that
 conclusion too quickly, so I was hoping someone here could
 perhaps enlighten me...

 Many thanks in advance,

 --
 Leo Breebaart  l...@lspace.org

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Completely Deleting A Directory

[MrJean1]

Two comments:

1) Should delete_dir not be called instead of os.rmdir in this line

(os.rmdir, os.remove)[os.path.islink(item)](item)

2) Function rmtree in the shutil module considers symlinks to a
directory an error http://docs.python.org/library/shutil.html#module-
shutil since Python 2.6.

/Jean


On Apr 26, 2:09 am, Lawrence D'Oliveiro l...@geek-
central.gen.new_zealand wrote:
 It doesn’t seem to mention in the documentation for os.walk
 http://docs.python.org/library/os.html that symlinks to directories are
 returned in the list of directories, not the list of files. This will lead
 to an error in the os.rmdir call in the example directory-deletion routine
 on that page.

 This version fixes that problem.

 def delete_dir(dir) :
     deletes dir and all its contents.
     if os.path.isdir(dir) :
         for parent, dirs, files in os.walk(dir, topdown = False) :
             for item in files :
                 os.remove(os.path.join(parent, item))
             #end for
             for item in dirs :
                 item = os.path.join(parent, item)
                 (os.rmdir, os.remove)[os.path.islink(item)](item)
             #end for
         #end for
         os.rmdir(dir)
     #end if
 #end delete_dir

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Completely Deleting A Directory

[MrJean1]

The answer to 1) is no, due to topdown = False in the call to os.walk.

/Jean

On Apr 26, 8:31 am, MrJean1 mrje...@gmail.com wrote:
 Two comments:

 1) Should delete_dir not be called instead of os.rmdir in this line

                 (os.rmdir, os.remove)[os.path.islink(item)](item)

 2) Function rmtree in the shutil module considers symlinks to a
 directory an error http://docs.python.org/library/shutil.html#module-
 shutil since Python 2.6.

 /Jean

 On Apr 26, 2:09 am, Lawrence D'Oliveiro l...@geek-



 central.gen.new_zealand wrote:
  It doesn’t seem to mention in the documentation for os.walk
  http://docs.python.org/library/os.html that symlinks to directories are
  returned in the list of directories, not the list of files. This will lead
  to an error in the os.rmdir call in the example directory-deletion routine
  on that page.

  This version fixes that problem.

  def delete_dir(dir) :
      deletes dir and all its contents.
      if os.path.isdir(dir) :
          for parent, dirs, files in os.walk(dir, topdown = False) :
              for item in files :
                  os.remove(os.path.join(parent, item))
              #end for
              for item in dirs :
                  item = os.path.join(parent, item)
                  (os.rmdir, os.remove)[os.path.islink(item)](item)
              #end for
          #end for
          os.rmdir(dir)
      #end if
  #end delete_dir

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to monitor memory usage within Python? (Linux)

[MrJean1]

For Linux only

http://code.activestate.com/recipes/286222/

/Jean

On Feb 24, 2:35 pm, kj no.em...@please.post wrote:
 Is there some standard module for getting info about the process's
 memory usage, in a Linux/Unix system?

 (I want to avoid hacks that involve, e.g., scraping ps's output.)

 Thanks!

 ~K

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.6 ftplib has timeout parameter, but how to detect a timeout

[MrJean1]

Brendan,

The timeout argument of the FTP class (and the connect method) is used
only to establish the connection to FTP sockets.  If the timeout
expires, an error called socket.timeout is raised.

AFAIK, the timeout argument does not limit FTP transfers.  To impose a
time limit on FTP transfers, try using the timelimited function from
this recipe:

 http://code.activestate.com/recipes/576780/

/Jean



On Dec 14, 11:08 am, Brendan brendandetra...@yahoo.com wrote:
 I was quite happy to see that ftplib in Python 2.6 now has a timeout
 parameter. With large file downloads my script would often hang,
 presumably from timing out. Now that there is a timeout parameter, how
 would I detect when a timeout occurs?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to timeout when waiting for raw_input from user ?

[MrJean1]

Try using the function timelimited from this recipe

http://code.activestate.com/recipes/576780/

An (untested) example with a 60 second timeout would be:

try:
r = timelimited(60, raw_input, 'enter right or wrong: ')
except TimeLimitExpired:

except KeyboardInterrupt:


/Jean


On Dec 4, 3:52 pm, northof40 shearich...@gmail.com wrote:
 Hi - I'm writing a *very* simple program for my kids. It asks the user
 to give it the answer to a maths question and says right or wrong

 They now want a timed version where they would only get so long to
 respond to the question.

 I'm thinking of some logic where a raw_input call is executed and then
 if more than X seconds elapses before the prompt is replied to the
 process writes a message Sorry too slow (or similar).

 I can't see the wood for the trees here - what's the best way to do
 this given the rather simple environment it's needed within.

 Regards

 richard.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best strategy for overcoming excessive gethostbyname timeout.

[MrJean1]

Take a look at function timelimited in this recipe

 http://code.activestate.com/recipes/576780/

/Jean


On Nov 29, 8:08 am, r0g aioe@technicalbloke.com wrote:
 r0g wrote:
  r0g wrote:
  Gabriel Genellina wrote:
  En Fri, 27 Nov 2009 22:35:36 -0300, r0g aioe@technicalbloke.com
  escribió:

  gethostbyname ignores setdefaulttimeout.

  How big a job is it to use non-blocking sockets to write a DNS lookup
  function with a customisable timeout? A few lines? A few hundred? I'd
  snip

  As usual, everything is working beautifully until I try to make it work
  with windows!

  Turns out signals.SIGALRM is Unix only and I want to run on both
  platforms so I have done as the docs suggested and tried to convert the
  code to use threading.Timer to trigger an exception if the DNS lookup is
  taking too long.

 Actually none of that was necessary in the end. Digging into the pydns
 source to debug the 30 second pause I happened across the timeout parameter!

  result = ping.do_one( google.com, 5 )

 

 Phew, that simplifies thing a lot! :)

 Roger.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Looking for an efficient Python script to download and save a .zip file programmatically

[MrJean1]

Here are some examples using urllib.urlretrieve():

http://code.activestate.com/recipes/576530/

http://mail.python.org/pipermail/tutor/2005-May/038797.html

/Jean Brouwers


On Jan 10, 2:23 pm, Chris Rebert c...@rebertia.com wrote:
 On Sat, Jan 10, 2009 at 9:12 AM, David Shi davidg...@yahoo.co.uk wrote:
  I am looking for an efficient Python script to download and save a .zip file
  programmatically (from http or https call).

 You want 
 urllib.urlretrieve():http://docs.python.org/library/urllib.html#urllib.urlretrieve

 Cheers,
 Chris

 --
 Follow the path of the Iguana...http://rebertia.com

--
http://mail.python.org/mailman/listinfo/python-list

Re: trapping all method calls in a class...

[MrJean1]

The decorate_meths() function as given fails:

  TypeError: 'dictproxy' object does not support item assignment

But this version avoids that error (on Python 2.2 thru 2.6):

def decorate_meths(klass):
for nam, val in klass.__dict__.items():
if callable(val):
setattr(klass, nam, decorate(val))
return klass  # ?


/Jean Brouwers


On Dec 20, 11:32 pm, Chris Rebert c...@rebertia.com wrote:
 On Sat, Dec 20, 2008 at 11:12 PM, Piyush Anonymous

 piyush.subscript...@gmail.com wrote:
  hi,
  i need to trap all method calls in a class in order to update a counter
  which is increased whenever a method is called and decreased whenever method
  returns. in order to that i am trying to write a decorator for all the
  methods.

  see the code here with error.
  ---
 http://codepad.org/2w7JVvDB
  
  any suggestions? any other better way of doing it?

 I call unnecessary use of metaclasses! Here's my (untested) attempt at
 a simpler class decorator approach:

 def decorate_meths(klass):
     attrs = klass.__dict__.items()
         for name, val in attrs:
                 if callable(val):
                         klass.__dict__[name] = decorate(val)

 def decorate(method):
     #should be called for every method call in the class
     def decorated(self, *args, **kwds):
         print 2 Inside __call__()
         returnval = method(self, *args,**kwds)
         print 3 After self.f(*args)
         return returnval
     return decorated

 #...@decorate_meths -- this syntax requires a later Python version
 class Person(object):
         def testprint(self,val):
                 print blah blah
 Person = decorate_meths(Person)

 #rest of code after the class definition would be the same

 Sidenotes about your code:
 - `args` and `kwds` are the conventional names for the * and **
 special arguments
 - the `methodname` variable was actually getting method objects, not
 strings, as its values; this was probably part of the bug in your
 program

 Cheers,
 Chris

 --
 Follow the path of the Iguana...http://rebertia.com

--
http://mail.python.org/mailman/listinfo/python-list

Re: simplest way to strip a comment from the end of a line?

[MrJean1]

Using rsplit('#', 1) works for lines *with* comments:

 'this is a test'.rsplit('#', 1)
['this is a test']

 'this is a test #with a comment'.rsplit('#', 1)
['this is a test ', 'with a comment']

 this is a '#gnarlier' test #with a comment.rsplit('#', 1)
[this is a '#gnarlier' test , 'with a comment']


But not if # occurs in lines without comments:

 this is a '#gnarlier' test.rsplit('#', 1)
[this is a ', gnarlier' test]


/Jean Brouwers



On Dec 4, 7:50 am, Joe Strout [EMAIL PROTECTED] wrote:
 I have lines in a config file which can end with a comment (delimited  
 by # as in Python), but which may also contain string literals  
 (delimited by double quotes).  A comment delimiter within a string  
 literal doesn't count.  Is there any easy way to strip off such a  
 comment, or do I need to use a loop to find each # and then count the  
 quotation marks to its left?

 Thanks,
 - Joe

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python test case management system?

[MrJean1]

Perhaps Qmtest fits your needs

http://www.codesourcery.com/qmtest/index_html

/Jean

On Sep 4, 4:36 pm, Mudcat [EMAIL PROTECTED] wrote:
 I had originally planned on writing my own software for managing test
 cases; however new boss = new directive. This will make it more
 difficult to get the functionality I need for test cases that are
 automated and executed using python.   I've searched for alternatives
 but so far haven't come up with any good options.

 Does anyone know of a good test case management system written in
 python, or possibly another application (either open source or
 commercial) that can be extended using python?

 Thanks

--
http://mail.python.org/mailman/listinfo/python-list

Re: How to create a timer/scheduler in Python?

[MrJean1]

There is a module called sched in the standard Python library

  http://docs.python.org/lib/module-sched.html

/Jean Brouwers



John Dann wrote:
 I need what I'd call (in .Net) a timer, ie I need to run a function eg
 every 2 seconds - it doesn't need to be millisec accurate but it would
 be nice if it wasn't eg every 4 seconds or something.

 Rather surprisingly, Core Python (Chun) doesn't seem to index 'timer'
 or 'scheduler', which leaves me wondering whether this is an aspect of
 Python that isn't perhaps widely used?

 Looking around on the net I can see references to a thread timer, but
 I'm not really looking to start any new threads (I just want part of
 the GUI to update every 2 secs) and don't want to get into that sort
 of complication while still just learning Python.

 Is there really no simple timer/scheduler function available in
 Python?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Not Sure This Can Be Done...

[MrJean1]

In any script upon startup, sys.path[0] contains the full path of the
directory where the script is located.  See http://docs.python.org/
lib/module-sys.html under 'path'.

it should be straightforward from here (untested though).  In each
script, get the sys.path[0] string, split it using os.path, replace
the last item with 'tools' and join again with os.path.  If the
resulting string is not in sys.path, insert it after sys.path[0].

/Jean Brouwers


On Apr 1, 1:57 pm, gamename [EMAIL PROTECTED] wrote:
 Hi,

 I generally have several copies of the same development environment
 checked out from cvs at any one time.  Each development tree has a
 'tools' dir containing python modules.  Scattered in different places
 in the tree are various python scripts.

 What I want to do is force my scripts to always look in the closest
 'tools' dir for any custom modules to import. For example:

 tree1/tools
 tree1/my_scripts/foo.py

 tree2/tools
 tree2/my_scripts/foo.py

 How can I make 'foo.py' always look in '../../tools' for custom
 modules? My preference would be to avoid changing the 'foo.py' script
 and have some way to resolve it via the environment (like PYTHONPATH,
 or .pth files, etc.).

 Anyone have any ideas?
 TIA,
 -T

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting OS platform in Python

[MrJean1]

On Jan 10, 7:53 pm, Benjamin [EMAIL PROTECTED] wrote:
 On Jan 10, 8:37 pm, Devraj [EMAIL PROTECTED] wrote: Hi everyone,

  My Python program needs reliably detect which Operating System its
  being run on, infact it even needs to know which distribution of say
  Linux its running on. The reason being its a GTK application that
  needs to adapt itself to be a Hildon application if run on devices
  like the N800.

 platform.dist might help you. It's not very complete at all, though.
 (This is supposed to improve in 2.6, though)


Better yet, first use  sys.platform.  If that is 'linux2', then use
platform.dist().

/Jean Brouwers


  I have been searching around for an answer to this, and did find some
  messages on a lists that suggested the use of sys.platform to detect
  platform, with counter posts saying that it didn't work on Windows
  etc.

  Can anyone please shed some light on this?

  Thanks a lot.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TextWrangler and new Python version (Mac)

[MrJean1]

On Jan 4, 3:33 pm, cf29 [EMAIL PROTECTED] wrote:
 I installed Python 2.5 on my Mac (OS X Tiger). When running scripts
 with the TextWrangler Run command it is using the system installed
 version of Python (2.3). If I run the scripts with the Apple Terminal
 it uses the new version (2.5).

 Is there any way to ask TextWrangler to use the new version of Python?


Chapter 12 of the TextWrangler User Manual under Unix Scripting:
Perl, Python, Ruby, Shells and More has several paragraphs on this
very issue.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython and a C extension using Boehm GC

[MrJean1]

It depends on how the GC inside the extension is built.  If it is a
drop-in replacement for malloc, then GC *must* be loaded and
initialized upfront if possible.  There is no need to memcpy anything
between Python and the extension.

However, if GC does not replace malloc, etc., then GC-ed memory is
only used within the extension.  GC_INIT can be called when the
extension is loaded and memcpy-ing between Python and the extension is
mandatory.

There are other details to consider.  For example, on some platforms,
GC *must* be initialized from the main executable.  That may preclude
both scenarios, altogether.

/Jean Brouwers



On Dec 25, 7:35 pm, Andrew MacIntyre [EMAIL PROTECTED]
wrote:
 malkarouri wrote:
  Is it possible to write a Python extension that uses the Boehm garbage
  collector?
  I have a C library written that makes use of boehm-gc for memory
  management. To use that, I have to call GC_INIT() at the start of the
  program that uses the library. Now I want to encapsulate the library
  as a CPython extension. The question is really is that possible? And
  will there be conflicts between the boehm-gc and Python memory
  management? And when should I call GC_INIT?

 It probably should be possible with some caveats:
 - memory allocated by Python is never passed into the library such that
    it also ends up being subject to boehm-gc;
 - memory allocated by the library is never used by Python objects.

 So memcpy()ing between library allocated and Python allocated memory
 would seem to be a way to achieve this.

 I would call GC_INIT in the extension's import routine
 (initmodule_name()) for a C extension, and immediately after loading
 the library if using ctypes.

 --
 -
 Andrew I MacIntyre                     These thoughts are mine alone...
 E-mail: [EMAIL PROTECTED]  (pref) | Snail: PO Box 370
         [EMAIL PROTECTED]             (alt) |        Belconnen ACT 2616
 Web:    http://www.andymac.org/              |        Australia

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython and a C extension using Boehm GC

[MrJean1]

FWIIW, I built GC 6.7 on a RHEL 3 (Opteron) system using

  ./configure --prefix=... --enable-redirect-malloc --enable-
threads=posix --enable-thread-local-alloc
  make; make check; make install

Then, I tried running a few examples with 3 different, existing Python
binaries each pre-loaded with the libgc.so library

  env  LD_PRELOAD=.../libgc.so  python 

One is Python 2.2.3 included in RHEL 3, one is a Python 2.5.1 build
and is a Python 3.0a2 build, all 64-bit.  All seemed to work OK.

These are 3 existing Python binaries without any call to GC_INIT().
AFAICT, on Linux, GC_INIT is a no-op anyway.

/Jean Brouwers


On Dec 26, 7:14 am, MrJean1 [EMAIL PROTECTED] wrote:
 It depends on how the GC inside the extension is built.  If it is a
 drop-in replacement for malloc, then GC *must* be loaded and
 initialized upfront if possible.  There is no need to memcpy anything
 between Python and the extension.

 However, if GC does not replace malloc, etc., then GC-ed memory is
 only used within the extension.  GC_INIT can be called when the
 extension is loaded and memcpy-ing between Python and the extension is
 mandatory.

 There are other details to consider.  For example, on some platforms,
 GC *must* be initialized from the main executable.  That may preclude
 both scenarios, altogether.

 /Jean Brouwers

 On Dec 25, 7:35 pm, Andrew MacIntyre [EMAIL PROTECTED]
 wrote:

  malkarouri wrote:
   Is it possible to write a Python extension that uses the Boehm garbage
   collector?
   I have a C library written that makes use ofboehm-gcfor memory
   management. To use that, I have to call GC_INIT() at the start of the
   program that uses the library. Now I want to encapsulate the library
   as a CPython extension. The question is really is that possible? And
   will there be conflicts between theboehm-gcand Python memory
   management? And when should I call GC_INIT?

  It probably should be possible with some caveats:
  - memory allocated by Python is never passed into the library such that
     it also ends up being subject toboehm-gc;
  - memory allocated by the library is never used by Python objects.

  So memcpy()ing between library allocated and Python allocated memory
  would seem to be a way to achieve this.

  I would call GC_INIT in the extension's import routine
  (initmodule_name()) for a C extension, and immediately after loading
  the library if using ctypes.

  --
  -
  Andrew I MacIntyre                     These thoughts are mine alone...
  E-mail: [EMAIL PROTECTED]  (pref) | Snail: PO Box 370
          [EMAIL PROTECTED]             (alt) |        Belconnen ACT 2616
  Web:    http://www.andymac.org/             |        Australia

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython and a C extension using Boehm GC

[MrJean1]

Perhaps, you can pre-load the extension library when Python is
invoked. It is probably trying.

Pre-loading is commonly done done for memory management and profiling
libraries and it may (or may not) work for libraries including the
Boehm-GC.  And if it does work, call GC_INIT inside the initialization
function of the extension library.  The latter will be called just
before Python's main is.

If you are using the GNU C, writing the initialization function could
be as simple as

void __attribute__((constructor))
_initializer (void)  /* any name */
{
   call GC_INIT();
}

For more details, see http://gcc.gnu.org/onlinedocs/gcc/Function-
Attributes.html under 'constructor'.  Other compilers may support a
#pragma like init for this purpose.

Pre-loading a (shared) library on Linux is typically done using the
env command, e.g.:

  $ env  LD_PRELOAD=path_to_the_library  python 

Some command shells support other ways and the name LD_PRELOAD may be
different on other O/S's.

HTH, /Jean Brouwers



On Dec 25, 3:34 am, malkarouri [EMAIL PROTECTED] wrote:
 Hi everyone,

 Is it possible to write a Python extension that uses the Boehm garbage
 collector?
 I have a C library written that makes use of boehm-gc for memory
 management. To use that, I have to call GC_INIT() at the start of the
 program that uses the library. Now I want to encapsulate the library
 as a CPython extension. The question is really is that possible? And
 will there be conflicts between the boehm-gc and Python memory
 management? And when should I call GC_INIT?

 Best Regards,

 Muhammad Alkarouri

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython and a C extension using Boehm GC

[MrJean1]

Correction.  The second line should be ... It is probably worth
trying.

/Jean Brouwers

On Dec 25, 12:19 pm, MrJean1 [EMAIL PROTECTED] wrote:
 Perhaps, you can pre-load the extension library when Python is
 invoked. It is probably trying.

 Pre-loading is commonly done done for memory management and profiling
 libraries and it may (or may not) work for libraries including the
 Boehm-GC.  And if it does work, call GC_INIT inside the initialization
 function of the extension library.  The latter will be called just
 before Python's main is.

 If you are using the GNU C, writing the initialization function could
 be as simple as

     void __attribute__((constructor))
     _initializer (void)  /* any name */
     {
        call GC_INIT();
     }

 For more details, see http://gcc.gnu.org/onlinedocs/gcc/Function-
 Attributes.html under 'constructor'.  Other compilers may support a
 #pragma like init for this purpose.

 Pre-loading a (shared) library on Linux is typically done using the
 env command, e.g.:

   $ env  LD_PRELOAD=path_to_the_library  python 

 Some command shells support other ways and the name LD_PRELOAD may be
 different on other O/S's.

 HTH, /Jean Brouwers

 On Dec 25, 3:34 am, malkarouri [EMAIL PROTECTED] wrote:

  Hi everyone,

  Is it possible to write a Python extension that uses the Boehm garbage
  collector?
  I have a C library written that makes use of boehm-gc for memory
  management. To use that, I have to call GC_INIT() at the start of the
  program that uses the library. Now I want to encapsulate the library
  as a CPython extension. The question is really is that possible? And
  will there be conflicts between the boehm-gc and Python memory
  management? And when should I call GC_INIT?

  Best Regards,

  Muhammad Alkarouri

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: It's ok to __slots__ for what they were intended

[MrJean1]

My milage does vary, see this older post

  http://mail.python.org/pipermail/python-list/2004-May/261985.html

Similar figures are shown with Python 2.5, both for 32- and 64-bit.

/Jean Brouwers


On Dec 21, 12:07 pm, Fredrik Lundh [EMAIL PROTECTED] wrote:
 John Nagle wrote:
       I'd like to hear more about what kind of performance gain can be
  obtained from __slots__.  I'm looking into ways of speeding up
  HTML parsing via BeautifulSoup.  If a significant speedup can be
  obtained when navigating large trees of small objects, that's worth
  quite a bit to me.

 The following micro-benchmarks are from Python 2.5 on a Core Duo
 machine.  C0 is an old-style class, C1 is a new-style class, C2 is a
 new-style class using __slots__:

 # read access
 $ timeit -s import q; o = q.C0(); o.attrib = 1 o.attrib
 1000 loops, best of 3: 0.133 usec per loop
 $ timeit -s import q; o = q.C1(); o.attrib = 1 o.attrib
 1000 loops, best of 3: 0.184 usec per loop
 $ timeit -s import q; o = q.C2(); o.attrib = 1 o.attrib
 1000 loops, best of 3: 0.161 usec per loop

 # write access
 $ timeit -s import q; o = q.C0(); o.attrib = 1 o.attrib = 1
 1000 loops, best of 3: 0.15 usec per loop
 $ timeit -s import q; o = q.C1(); o.attrib = 1 o.attrib = 1
 100 loops, best of 3: 0.217 usec per loop
 $ timeit -s import q; o = q.C2(); o.attrib = 1 o.attrib = 1
 100 loops, best of 3: 0.209 usec per loop

 $ more q.py
 class C0:
      pass

 class C1(object):
      pass

 class C2(object):
      __slots__ = [attrib]

 Your mileage may vary.

   I'm looking into ways of speeding up HTML parsing via BeautifulSoup.

 The solution to that is spelled lxml.

 /F

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: It's ok to __slots__ for what they were intended

[MrJean1]

You are correct.  Mea culpa.

/Jean Brouwers



On Dec 21, 1:41 pm, Fredrik Lundh [EMAIL PROTECTED] wrote:
 MrJean1 wrote:
  My milage does vary, see this older post

    http://mail.python.org/pipermail/python-list/2004-May/261985.html

  Similar figures are shown with Python 2.5, both for 32- and 64-bit.

 unless I'm missing something, you're measuring object creation time.

 I'm measuring attribute access time (the topic was navigating large
 trees of small objects, not building them).

 /F

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Catching a segfault in a Python library

[MrJean1]

Try catching SIGSEGV using the Python signal module

  http://docs.python.org/lib/module-signal.html

An example (for SIGALRM) is on the next page

  http://docs.python.org/lib/node546.html

However, it may not work since a SIGSEGV fault is pretty much the end
of everything :-(

/Jean Brouwers



On Nov 24, 6:22 am, Donn Ingle [EMAIL PROTECTED] wrote:
  I think the idea is, certain fonts in his collection may be corrupt,
  and he wants to just scan through and load them, ignoring the ones
  that make the program crash.  

 Ya got me! Sheesh, I can't hide anywhere :D

  The bug in this case lies with a third
  party and isn't something he can easily fix (although he can file
  reports to the third party (PIL)).

 I've a bad memory and can't recall what I told PIL at the time. It might
 have been a case of waiting to see what new versions can do.

  not nice for the application to just crash when that happens, asking
  them if they want to debug it.  

 Zigactly! You can wrap try/except around the calls that (by debugging) you
 know are the culprits, but a segfault is a segfault and bam! you are at the
 command line again.

  I haven't really found a solution,
  just have tried to prevent corrupted files in the system for now.  Let
  me know if you get this solved

 I'll certainly pop a note. I think, though, that the answer may reside in
 the basic theme of this thread:

 runapp
  result = runActualApp( )
  while True:
   if result == allokay: break
   else:
Start handling the horror

 Unless a segfault goes through that too, like Krypton through Superman.
 \d

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Populating a dictionary, fast

[MrJean1]

On MacOS** your version

#!/usr/bin/python
v = {}
for line in open('keys.txt'):
v[long(line.strip())] = True

using these http://www.sendspace.com/file/9530i7 keys takes

24.9 secs with ActivePython 2.5.1
and
212.3 secs with Apple's Python 2.3.5.


However, this version

#!/usr/bin/python
v = {}
for line in open('keys.txt'):
v[line] = True

takes
16.5 secs with ActivePython 2.5.1
and
18.7 secs with Apple's Python 2.3.5.


/Jean Brouwers

**) MacOS X 10.4.10 on Mini Mac, 1.83 GHz Intel Core Duo, 2 GB RAM.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Solaris 10 + Sun Studio 12 Pyrhon 2.4.4 64-bit build problem

[MrJean1]

Building 64-bit Python 2.4.4 on Solaris 10 and SUC C/C++ using the
instructions from

  http://ccnuma.anu.edu.au/~wpc/blog/programming/building-
python.html

worked just fine on Ultra 20 Opteron machine.  The test result summary
is below.

/Jean Brouwers


249 tests OK.
2 tests failed:
test_cmath test_pty
40 tests skipped:
test_aepack test_al test_applesingle test_bsddb test_bsddb185
test_bsddb3 test_bz2 test_cd test_cl test_codecmaps_cn
test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr
test_codecmaps_tw test_crypt test_curses test_dl test_gdbm test_gl
test_imageop test_imgfile test_linuxaudiodev test_macfs
test_macostools test_nis test_normalization test_ossaudiodev
test_pep277 test_plistlib test_rgbimg test_scriptpackages
test_socket_ssl test_socketserver test_tcl test_timeout
test_unicode_file test_urllib2net test_urllibnet test_winreg
test_winsound
5 skips unexpected on sunos5:
test_tcl test_bz2 test_crypt test_dl test_nis


plumb and tree wrote:
 I've been trying for days to build 64 bit python with Solaris 10 + Sun
 Studio 12.

 Can anyone helpl please.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Solaris 10 + Sun Studio 12 Pyrhon 2.4.4 64-bit build problem

[MrJean1]

On final comment.  For 64-bit usage, Python 2.5.1 is the better
choice.

More on that here http://docs.python.org/whatsnew/pep-353.html.

/Jean Brouwers


 On Oct 30, 10:15 am, MrJean1 [EMAIL PROTECTED] wrote:
 Building 64-bit Python 2.4.4 on Solaris 10 and SUC C/C++ using the
 instructions from

   http://ccnuma.anu.edu.au/~wpc/blog/programming/building-
 python.html

 worked just fine on Ultra 20 Opteron machine.  The test result summary
 is below.

 /Jean Brouwers

 
 249 tests OK.
 2 tests failed:
 test_cmath test_pty
 40 tests skipped:
 test_aepack test_al test_applesingle test_bsddb test_bsddb185
 test_bsddb3 test_bz2 test_cd test_cl test_codecmaps_cn
 test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr
 test_codecmaps_tw test_crypt test_curses test_dl test_gdbm test_gl
 test_imageop test_imgfile test_linuxaudiodev test_macfs
 test_macostools test_nis test_normalization test_ossaudiodev
 test_pep277 test_plistlib test_rgbimg test_scriptpackages
 test_socket_ssl test_socketserver test_tcl test_timeout
 test_unicode_file test_urllib2net test_urllibnet test_winreg
 test_winsound
 5 skips unexpected on sunos5:
 test_tcl test_bz2 test_crypt test_dl test_nis

 plumb and tree wrote:
  I've been trying for days to build 64 bit python with Solaris 10 + Sun
  Studio 12.

  Can anyone helpl please.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: why did these companies choose Tcl over Python

[MrJean1]

That is correct.  Tcl has it roots at UC Berkeley and was originally
used
to provide a command line interface for electronic design automation
(EDA)
tools.

Most commercial EDA vendors at that time were using their own,
proprietary
command language.  Only later became Tcl widely adopted among EDA
vendors
and -even more important- among users, electronic designers.

There is a large base of Tcl scripts in existence among users.
Replacing
that is practically and commercially simply not feasible regardless of
the strengths or weaknesses of Tcl vs any other language.

Recently, Python is getting more traction inside EDA, especially for
internal use and internal development and in some cases as the design
language of newer EDA tools.

/Jean Brouwers


On Oct 30, 3:47 pm, Martin v. Löwis [EMAIL PROTECTED] wrote:
  I would prefer to use Python but can't deny how popular Tcl is,  as
  mentioned above,  so my question is why wasn't Python selected by
  these companies as the choice of scripting languages for their
  product?

 I think this question needs to be answered on a case-by-case basis,
 but my guess is that it is in most cases historical. Work on Tcl
 started in 1988, and it was the first (major?) embeddable scripting
 language (that is also free software etc). Python wasn't released
 until 1991, and wasn't first recognized as being just as easily
 embeddable (and I think early releases weren't as easily embeddable
 as today's Python is).

 Tcl's original objective was to support circuit design, so people
 in that field clearly knew that Tcl worked, but they were likely
 unaware of any alternatives (or else the future of these alternatives
 may have been uncertain).

 So at that time, Tcl would have been the obvious (because only)
 choice. Now these products are stuck with Tcl, and redoing all
 the work (including the existing extension modules!) in a different
 programming language would be a lot of work.

 Regards,
 Martin


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Solaris 10 + Sun Studio 12 Pyrhon 2.4.4 64-bit build problem

[MrJean1]

Maybe this helps.  Using ./configure --without-gcc  in Python
2.5.1 on Solaris 10 gives a different message.  That message indicates
that C++ compiler 'c++' will be used but the Studio C++ compiler on
Solaris is 'CC'.

Using ./configure  --without-gcc  --with-cxx-main=CC  made that
message go away.  Python did build after that and the tests are
running.

But it is a 32-bit binary, not 64-bit. Another ./configure option will
be needed to build that.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Solaris 10 + Sun Studio 12 Pyrhon 2.4.4 64-bit build problem

[MrJean1]

Building 64-bit Python is still elusive. I tried various ways to add  -
xtarget=opteron -xarch-amd64  to the C/C++ flags but that still fails
to produce a 64-bit build.  Changing the Makefile is not sufficient
since that causes compilation errors.

It looks like the ./configure command must run with those additional
flags.  Passing the flags in environment variables defining CC and CXX
only works for CXX.  The additional flags do not appear on the C
compiler command lines.

/Jean Brouwers



On Oct 28, 11:03 pm, plumb and tree [EMAIL PROTECTED] wrote:
 I've been trying for days to build 64 bit python with Solaris 10 + Sun
 Studio 12.

 Can anyone helpl please.

 This is how I tried to do build:

 # ./configure --prefix=/opt/python2.4 --without-gcc --enable-shared
 checking MACHDEP... sunos5
 checking EXTRAPLATDIR...
 checking for --without-gcc... yes
 checking for --with-cxx=compiler... no
 checking for c++... /opt/SUNWspro/bin/CC
 checking for C++ compiler default output file name... a.out
 checking whether the C++ compiler works... yes
 checking whether we are cross compiling... no
 checking for suffix of executables...
 checking for gcc... cc
 checking for C compiler default output file name... a.out
 checking whether the C compiler works... yes
 checking whether we are cross compiling... no
 checking for suffix of executables...
 checking for suffix of object files... o
 checking whether we are using the GNU C compiler... no
 checking whether cc accepts -g... yes
 checking for cc option to accept ANSI C... none needed
 checking how to run the C preprocessor... /opt/SUNWspro/bin/CC
 configure: error: C preprocessor /opt/SUNWspro/bin/CC fails sanity
 check
 See `config.log' for more details.

 (sorry for the long log file but I'm not sure which part are
 important)
 config.log:
 This file contains any messages produced by compilers while
 running configure, to aid debugging if configure makes a mistake.

 It was created by python configure 2.4, which was
 generated by GNU Autoconf 2.59.  Invocation command line was

   $ ./configure --prefix=/opt/python2.4 --without-gcc --enable-shared

 ## - ##
 ## Platform. ##
 ## - ##

 hostname = zone2
 uname -m = sun4u
 uname -r = 5.10
 uname -s = SunOS
 uname -v = Generic_118833-36

 /usr/bin/uname -p = sparc
 /bin/uname -X = System = SunOS
 Node = zone2
 Release = 5.10
 KernelID = Generic_118833-36
 Machine = sun4u
 BusType = unknown
 Serial = unknown
 Users = unknown
 OEM# = 0
 Origin# = 1
 NumCPU = 1

 /bin/arch  = sun4
 /usr/bin/arch -k   = sun4u
 /usr/convex/getsysinfo = unknown
 hostinfo   = unknown
 /bin/machine   = unknown
 /usr/bin/oslevel   = unknown
 /bin/universe  = unknown

 PATH: /usr/local/bin
 PATH: /usr/xpg4/bin
 PATH: /usr/sfw/bin
 PATH: /opt/SUNWspro/bin
 PATH: /usr/local/mysql/bin
 PATH: /usr/ccs/bin
 PATH: /usr/local/bin
 PATH: /usr/xpg4/bin
 PATH: /usr/sfw/bin
 PATH: /opt/SUNWspro/bin
 PATH: /usr/local/mysql/bin
 PATH: /usr/ccs/bin
 PATH: /usr/local/bin
 PATH: /usr/xpg4/bin
 PATH: /usr/sfw/bin
 PATH: /opt/SUNWspro/bin
 PATH: /usr/local/mysql/bin
 PATH: /usr/ccs/bin
 PATH: /usr/sbin
 PATH: /usr/bin
 PATH: /usr/openwin/bin
 PATH: /usr/sfw/bin
 PATH: /usr/openwin/bin
 PATH: /usr/sfw/bin
 PATH: /usr/openwin/bin
 PATH: /usr/sfw/bin

 ## --- ##
 ## Core tests. ##
 ## --- ##

 configure:1510: checking MACHDEP
 configure:1655: result: sunos5
 configure:1661: checking EXTRAPLATDIR
 configure:1676: result:
 configure:1697: checking for --without-gcc
 configure:1746: result: yes
 configure:1752: checking for --with-cxx=compiler
 configure:1773: result: no
 configure:1792: checking for c++
 configure:1818: result: /opt/SUNWspro/bin/CC
 configure:1858: checking for C++ compiler default output file name
 configure:1861: /opt/SUNWspro/bin/CC   -L/opt/SUNWmlib/lib -lrt -lm
 conftest.cc  5
 configure:1864: $? = 0
 configure:1910: result: a.out
 configure:1915: checking whether the C++ compiler works
 configure:1921: ./a.out
 configure:1924: $? = 0
 configure:1941: result: yes
 configure:1948: checking whether we are cross compiling
 configure:1950: result: no
 configure:1953: checking for suffix of executables
 configure:1955: /opt/SUNWspro/bin/CC -o conftest   -L/opt/SUNWmlib/lib
 -lrt -lm conftest.cc  5
 configure:1958: $? = 0
 configure:1983: result:
 configure:2057: checking for gcc
 configure:2083: result: cc
 configure:2327: checking for C compiler version
 configure:2330: cc -V /dev/null 5
 cc: Sun C 5.9 SunOS_sparc Patch 124867-01 2007/07/12
 usage: cc [ options] files.  Use 'cc -flags' for details
 configure:2333: $? = 1
 configure:2356: checking for C compiler default output file name
 configure:2359: cc -O2  -L/opt/SUNWmlib/lib -lrt -lm conftest.c  5
 configure:2362: $? = 0
 configure:2408: result: a.out
 configure:2413: checking whether the C compiler works
 configure:2419: ./a.out
 configure:2422: $? = 0
 configure:2439: result: yes
 configure:2446: checking 

Re: Solaris 10 + Sun Studio 12 Pyrhon 2.4.4 64-bit build problem

[MrJean1]

Here is one way to build a 64-bit Python binary on Solaris 10 using
SUN's compilers**.  This is probably not the recommended procedure but
it does create a partial, expected result.

Here are the 4 steps I used:

1 - Run  ./configure --without-gcc  --with-cxx-main=CC  --enable-64-
bit  

2 - Edit the generated Makefile and change 3 lines

CC = cc -xtarget=opteron -xarch=amd64
CXX= CC -xtarget=opteron -xarch=amd64
MAINCC = CC -xtarget=opteron -xarch=amd64

The -xtarget and -xarch flags must be adjusted for UltraSparc.

3 - Edit python source file ./Include/pyport.h before line 728 (in
Python 2.5.1) as follows.

Add these 3 lines:

#ifndef __GNUC__  /* non-GNU C/C++, like SUN */
# undef LONG_BIT
#endif

such that LONG_BIT is #define'd next

#ifndef LONG_BIT
#define LONG_BIT (8 * SIZEOF_LONG)
#endif

before this error message can occur

#if LONG_BIT != 8 * SIZEOF_LONG
/* 04-Oct-2000 LONG_BIT is apparently (mis)defined as 64 on some
recent
 * 32-bit platforms using gcc.  We try to catch that here at
compile-time
 * rather than waiting for integer multiplication to trigger bogus
 * overflows.
 */
#error LONG_BIT definition appears wrong for platform (bad gcc/
glibc config?).
#endif

4 - Run make and check the resulting Python binary with  file ./
python.

However, make test still fails and that will require further
investigation.

/Jean Brouwers


**) Sun C/C++ 5.8 2005/10/14 on an Ultra 20 Opteron machine


On Oct 29, 12:06 pm, MrJean1 [EMAIL PROTECTED] wrote:
 Building 64-bit Python is still elusive. I tried various ways to add  -
 xtarget=opteron -xarch-amd64  to the C/C++ flags but that still fails
 to produce a 64-bit build.  Changing the Makefile is not sufficient
 since that causes compilation errors.

 It looks like the ./configure command must run with those additional
 flags.  Passing the flags in environment variables defining CC and CXX
 only works for CXX.  The additional flags do not appear on the C
 compiler command lines.

 /Jean Brouwers

 On Oct 28, 11:03 pm, plumb and tree [EMAIL PROTECTED] wrote:
.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Solaris 10 + Sun Studio 12 Pyrhon 2.4.4 64-bit build problem

[MrJean1]

Correction: the number of tests which pass must be 276.

/Jean Brouwers


On Oct 29, 10:29 pm, MrJean1 [EMAIL PROTECTED] wrote:
 Here is a better way to build Python on Solaris, both 32- and 64-
 bit.

  http://ccnuma.anu.edu.au/~wpc/blog/programming/building-python.html

 since it builds both the binary and make test on my Ultra 20 machine.
 In my case 26 tests pass, 1 failed, 44 are skipped and there are 6 u
 expected skips.  The latter are expected and due to missing packages,
 like Tcl.

 Use the instructions from this site and forget mine.  But it may still
 be necessary to patch the ./Include/pyport.h file as shown in my
 previous message.

 /Jean Brouwers

 On Oct 29, 9:15 pm, MrJean1 [EMAIL PROTECTED] wrote:

  Here is one way to build a 64-bit Python binary on Solaris 10 using
  SUN's compilers**.  This is probably not the recommended procedure but
  it does create a partial, expected result.

  Here are the 4 steps I used:

  1 - Run  ./configure --without-gcc  --with-cxx-main=CC  --enable-64-
  bit  

  2 - Edit the generated Makefile and change 3 lines

  CC = cc -xtarget=opteron -xarch=amd64
  CXX= CC -xtarget=opteron -xarch=amd64
  MAINCC = CC -xtarget=opteron -xarch=amd64

  The -xtarget and -xarch flags must be adjusted for UltraSparc.

  3 - Edit python source file ./Include/pyport.h before line 728 (in
  Python 2.5.1) as follows.

  Add these 3 lines:

  #ifndef __GNUC__  /* non-GNU C/C++, like SUN */
  # undef LONG_BIT
  #endif

  such that LONG_BIT is #define'd next

  #ifndef LONG_BIT
  #define LONG_BIT (8 * SIZEOF_LONG)
  #endif

  before this error message can occur

  #if LONG_BIT != 8 * SIZEOF_LONG
  /* 04-Oct-2000 LONG_BIT is apparently (mis)defined as 64 on some
  recent
   * 32-bit platforms using gcc.  We try to catch that here at
  compile-time
   * rather than waiting for integer multiplication to trigger bogus
   * overflows.
   */
  #error LONG_BIT definition appears wrong for platform (bad gcc/
  glibc config?).
  #endif

  4 - Run make and check the resulting Python binary with  file ./
  python.

  However, make test still fails and that will require further
  investigation.

  /Jean Brouwers

  **) Sun C/C++ 5.8 2005/10/14 on an Ultra 20 Opteron machine

  On Oct 29, 12:06 pm, MrJean1 [EMAIL PROTECTED] wrote: Building 64-bit 
  Python is still elusive. I tried various ways to add  -
   xtarget=opteron -xarch-amd64  to the C/C++ flags but that still fails
   to produce a 64-bit build.  Changing the Makefile is not sufficient
   since that causes compilation errors.

   It looks like the ./configure command must run with those additional
   flags.  Passing the flags in environment variables defining CC and CXX
   only works for CXX.  The additional flags do not appear on the C
   compiler command lines.

   /Jean Brouwers

   On Oct 28, 11:03 pm, plumb and tree [EMAIL PROTECTED] wrote:

  .


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Solaris 10 + Sun Studio 12 Pyrhon 2.4.4 64-bit build problem

[MrJean1]

Here is a better way to build Python on Solaris, both 32- and 64-
bit.

 http://ccnuma.anu.edu.au/~wpc/blog/programming/building-python.html

since it builds both the binary and make test on my Ultra 20 machine.
In my case 26 tests pass, 1 failed, 44 are skipped and there are 6 u
expected skips.  The latter are expected and due to missing packages,
like Tcl.

Use the instructions from this site and forget mine.  But it may still
be necessary to patch the ./Include/pyport.h file as shown in my
previous message.

/Jean Brouwers



On Oct 29, 9:15 pm, MrJean1 [EMAIL PROTECTED] wrote:
 Here is one way to build a 64-bit Python binary on Solaris 10 using
 SUN's compilers**.  This is probably not the recommended procedure but
 it does create a partial, expected result.

 Here are the 4 steps I used:

 1 - Run  ./configure --without-gcc  --with-cxx-main=CC  --enable-64-
 bit  

 2 - Edit the generated Makefile and change 3 lines

 CC = cc -xtarget=opteron -xarch=amd64
 CXX= CC -xtarget=opteron -xarch=amd64
 MAINCC = CC -xtarget=opteron -xarch=amd64

 The -xtarget and -xarch flags must be adjusted for UltraSparc.

 3 - Edit python source file ./Include/pyport.h before line 728 (in
 Python 2.5.1) as follows.

 Add these 3 lines:

 #ifndef __GNUC__  /* non-GNU C/C++, like SUN */
 # undef LONG_BIT
 #endif

 such that LONG_BIT is #define'd next

 #ifndef LONG_BIT
 #define LONG_BIT (8 * SIZEOF_LONG)
 #endif

 before this error message can occur

 #if LONG_BIT != 8 * SIZEOF_LONG
 /* 04-Oct-2000 LONG_BIT is apparently (mis)defined as 64 on some
 recent
  * 32-bit platforms using gcc.  We try to catch that here at
 compile-time
  * rather than waiting for integer multiplication to trigger bogus
  * overflows.
  */
 #error LONG_BIT definition appears wrong for platform (bad gcc/
 glibc config?).
 #endif

 4 - Run make and check the resulting Python binary with  file ./
 python.

 However, make test still fails and that will require further
 investigation.

 /Jean Brouwers

 **) Sun C/C++ 5.8 2005/10/14 on an Ultra 20 Opteron machine

 On Oct 29, 12:06 pm, MrJean1 [EMAIL PROTECTED] wrote: Building 64-bit 
 Python is still elusive. I tried various ways to add  -
  xtarget=opteron -xarch-amd64  to the C/C++ flags but that still fails
  to produce a 64-bit build.  Changing the Makefile is not sufficient
  since that causes compilation errors.

  It looks like the ./configure command must run with those additional
  flags.  Passing the flags in environment variables defining CC and CXX
  only works for CXX.  The additional flags do not appear on the C
  compiler command lines.

  /Jean Brouwers

  On Oct 28, 11:03 pm, plumb and tree [EMAIL PROTECTED] wrote:

 .


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: good example of C extension for Mac OS X

[MrJean1]

There is a C template in file Modules/xxmodule.c in Python 2.5, maybe
earlier.

/Jean Brouwers


On Oct 27, 8:11 am, chewie54 [EMAIL PROTECTED] wrote:
 Hi All,

 Does anyone now of a good example to use as template for a  C program
 extension that needs to be built on the Mac OS X.

 Thanks,


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: good example of C extension for Mac OS X

[MrJean1]

There is a C template in file Modules/xxmodule.c in Python 2.5, maybe
earlier.

/Jean Brouwers

On Oct 27, 8:11 am, chewie54 [EMAIL PROTECTED] wrote:
 Hi All,

 Does anyone now of a good example to use as template for a  C program
 extension that needs to be built on the Mac OS X.

 Thanks,


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: good example of C extension for Mac OS X

[MrJean1]

You will need to download the MacOS X version from

  http://www.python.org/download/releases/2.5.1/.

It contains the source code.

/Jean Brouwers


On Oct 27, 12:31 pm, chewie54 [EMAIL PROTECTED] wrote:
 On Oct 27, 2:59 pm, MrJean1 [EMAIL PROTECTED] wrote:

  There is a C template in file Modules/xxmodule.c in Python 2.5, maybe
  earlier.

  /Jean Brouwers

  On Oct 27, 8:11 am, chewie54 [EMAIL PROTECTED] wrote:

   Hi All,

   Does anyone now of a good example to use as template for a  C program
   extension that needs to be built on the Mac OS X.

   Thanks,

 I have looked at the docs on the python.org but specifically looking
 for
 examples showing how to deal with linked lists and more about when to
 use
 references Py_INCREF and Py_DECREF

 I didn't find Modules/xxmodule.c on my Mac OS X installation.

 Thanks again


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Not Working on Solaris

[MrJean1]

FWIIW, on my Solaris 10 (Opteron) machine, there are no import
subprocess errors, not with Python 2.4.3 nor with Python 2.5.

/Jean Brouwers

PS) Python 2.4.3 is the ActivePython Solaris build 11 from
ActivState.  Python 2.5 was built from source using SUN compilers, not
GNU.  There is an ActivePython Solaris build for 2.5.1 but I have not
tried that one yet.  See http://www.activestate.com/store/
download.aspx?prdGUID=b08b04e0-6872-4d9d-a722-7a0c2dea2758



On Jun 15, 8:00 am, Lee [EMAIL PROTECTED] wrote:
 Has anyone ran into this problem? I've done extensive googling and
 research and I cannot seem to find the answer.

 I downloaded the source for 2.5.1 from python.org compiled and
 installed it on a Solaris box,

 uname -a returns

 SunOS unicom5 5.8 Generic_117350-26 sun4u sparc SUNW,Sun-Fire-V210

 When I launch the python interpreter, I try the following:

 
 Python 2.5.1 (r251:54863, Jun 13 2007, 13:40:52)
 [GCC 3.2.3] on sunos5
 Type help, copyright, credits or license for more information. 
 import subprocess

 Traceback (most recent call last):
   File stdin, line 1, in module
   File /usr/local/lib/python2.5/subprocess.py, line 401, in module
 import select
 ImportError: No module named select
 

 The subprocess module works just fine on a BSD box I have and after
 doing a 'find' I notice there is a select.so module. This does not
 exist on the Solaris box, only subprocess exists. Within subprocess.py
 on the Solaris box, exists the following:

 
 else:
 import select
 import errno
 import fcntl
 import pickle
 

 Any ideas? I'd like to get the subprocess module working...

 -Lee

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 32 OS on 64-bit machine

[MrJean1]

$ python
Python 2.5c1 (r25c1:51305, Sep 12 2006, 08:39:50)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-54)] on linux2
Type help, copyright, credits or license for more information.
 import platform
 print platform.processor()
x86_64
 print platform.architecture()
('64bit', 'ELF')


This is an Opteron box running 64-bit RedHat Enterprise Lunix release
3 update 7, not 32-bit.

$ uname -a
Linux localhost.localdomain 2.4.21-40.EL #1 Thu Feb 2 22:20:41 EST
2006 x86_64 x86_64 x86_64 GNU/Linux

/Jean Brouwers



On May 3, 2:10 am, SamG [EMAIL PROTECTED] wrote:
 If anyone has a x86_64 machine and is running a 32bit OS on top of
 that could you tell me what output would you get for the following
 program

 #==
 import platform
 print platform.processor()
 print platform.architecture()
 #==

 Thanks in advance
 : )~

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to create pid.lock via python?

[MrJean1]

May this works for your case

  http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/498171.

/Jean Brouwers


On Mar 5, 3:12 am, Marco [EMAIL PROTECTED] wrote:
 Hello,

 I write a program control a device via RS232, I hope to add some code
 to let the program canNOT be used by other people when one people
 using.

 Can you tell me how to create pid.lock file in python?
 Thank you!!

 --
 LinuX Power

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best Free and Open Source Python IDE

[MrJean1]

On Feb 8, 6:03 am, Srikanth [EMAIL PROTECTED] wrote:
 Yes,

 All I need is a good IDE, I can't find something like Eclipse (JDT).
 Eclipse has a Python IDE plug-in but it's not that great. Please
 recommend.

 Thanks,
 Srikanth


Here is a list http://wiki.python.org/moin/
IntegratedDevelopmentEnvironments.

My preference is SciTE http://www.scintilla.org/SciTE.html on Linux
and Windows and BBebit on MacOS http://www.barebones.com.  None are
quite IDEs though.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Match 2 words in a line of file

[MrJean1]

Without using re, this may work (untested ;-):

def lines_with_words(file, word1, word2):
Print all lines in file that have both words in it.
for line in file:
words = line.split()
if word1 in words and word2 in words:
print line


/Jean Brouwers


Rickard Lindberg wrote:
 [EMAIL PROTECTED] wrote:
  Hi
 
  Am pretty new to python and hence this question..
 
  I have file with an output of a process. I need to search this file one
  line at a time and my pattern is that I am looking for the lines that
  has the word 'event' and the word 'new'.
 
  Note that i need lines that has both the words only and not either one
  of them..
 
  how do i write a regexp for this.. or better yet shd i even be using
  regexp or is there a better way to do this
 
  thanks

 Maybe something like this would do:

 import re

 def lines_with_words(file, word1, word2):
 Print all lines in file that have both words in it.
 for line in file:
 if re.search(r\b + word1 + r\b, line) and \
re.search(r\b + word2 + r\b, line):
 print line

 Just call the function with a file object and two strings that
 represent the words that you want to find in each line.

 To match a word in regex you write \bWORD\b.

 I don't know if there is a better way of doing this, but I believe that
 this should at least work.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Asyncore select statement problem

[MrJean1]

Try using another ascyncore example and see if that works for you.
Maybe, first one without threading, like


http://effbot.org/lib/asyncore.html#asyncore-example-basic-http-client

Asyncore worked fine on my application on Linux, but haven't tried that
on MacOS.

/Jean Brouwers



JamesHoward wrote:
 I have a problem with python's asyncore module throwing a bad file
 descriptor error.  The code might be difficult to copy here, but the
 problem is essentially:

 The server wants to sever the connection of an open Asyncore socket.
 Calling the socket.close() nor the socket.shutdown(2) calls seem to
 work.  The only way I can close the connection without creating the
 error below is to have the client close the connection.

 I have the asyncore.loop() as the last line of a thread that is spawned
 within the applications mainframe.py or gui thread.  It doesn't seem
 to me like this would make a difference, but I am unfamiliar with the
 specifics of how the asyncore module works.

 Any thoughts people have would be greatly appreciated.  If needed I may
 be able to create a small version of the problem to post for people to
 see.

 Thanks,
 Jim Howard


 Exception in thread Thread-1:
 Traceback (most recent call last):
   File
 /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py,
 line 460, in __bootstrap
 self.run()
   File
 /Users/jwhoward2/Documents/Projects/LJServer/LJDeviceServer/DeviceServer.py,
 line 23, in run
 asyncore.loop()
   File
 /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py,
 line 191, in loop
 poll_fun(timeout, map)
   File
 /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py,
 line 121, in poll
 r, w, e = select.select(r, w, e, timeout)
 error: (9, 'Bad file descriptor')

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython help wxSashWindow

[MrJean1]

Take a look at the wxSashWindow example in the wxPython demo.   That
is located under the Core Windows/Controls item.

/Jean Brouwers


MatthewWarren wrote:
 Hi, I'm wondering if anyone can tell me here, or point to a specific
 tutorial (  I have searched for 1/2hour, but can find only reference
 style docs or not-quite-what-im-after help) on how to build a
 wxSashWindow in wxPython. I'm just starting out with wxPython, and the
 first thing i need to do is use 3 sash windows, 1 split vertically, and
 on the left and right of that a SashWindow splitting horizontally. I
 can make frames, buttons, use sizers etc.. as they are fairly
 intuitive, but SashWindow and SashLayoutWindow have stumped me.
 
 Thanks,
 
 Matt.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compile for 64bit RHEL

[MrJean1]

Both Python 2.4.4 and 2.5 built just fine on my Opteron box with RHEL
(release 3 update 7).  There are no test failures, only 35 skipped
tests (which are expected on Linux).

/Jean Brouwers


Christopher Taylor wrote:
 Has anyone been able to get python 2.4 to compile properly for x86_64 RHEL?
 
 Respectfully,
 Christopher Taylor

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compile python on Solaris 64bit

[MrJean1]

I did build Python 2.5 on a Solaris 10 Ultra 20 machine (Opteron) but
using the SUN compilers.  Two changes were needed in the Makefile:
CXX=CC and CCSHARED=-KPIC.

There are a few issues which I have not investigated further (like Tcl
and ffitarget.h), but the tests pass except for 3 unexpected skips.
Other than that, Python 2.5 seems to be running fine, so far.

Also, I am using the Python 2.4 build for Solaris build from
ActiveState as well as Python 2.3.3 which is distributed with Solaris
in /usr/sfw/bin.

/Jean Brouwers


Martijn de Munnik wrote:
 Hi,

 I want to compile python on my solaris 10 system (amd 64 bit).

 I did the following:

 ./configure --prefix=/opt/64/python
 make

 which resulted in this error:
 Include/pyport.h, line 730: #error: LONG_BIT definition appears wrong
 for platform (bad gcc/glibc config?).

 so I edited the file and removed the error line and did a configure and
 make again after a make distclean.

 after a new make I get this error complaining about 32 and 64 bit.

 ld: fatal: file Parser/acceler.o: wrong ELF class: ELFCLASS32

 when I look to the cc statements none of my CFLAGS are passed while these
 are in my environment;

 CC=cc
 CFLAGS=-xO3 -mt -fsimple=1 -ftrap=%none -nofstore -xbuiltin=%all -xlibmil
 -xlibmopt -xtarget=opteron -xarch=amd64 -xregs=no%frameptr
 CXX=CC
 CXXFLAGS=-xO3 -mt -fsimple=1 -ftrap=%none -nofstore -xbuiltin=%all
 -xlibmil -xlibmopt -xtarget=opteron -xarch=amd64 -xregs=no%frameptr
 LDFLAGS=-xtarget=opteron -xarch=amd64


 anybody succesfully compiled python on solaris?
 
 thanks,
 martijn

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: print time comparison: IDLE versus terminal on ultra 20

[MrJean1]

With the binaries /usr/sfw/bin/python and /usr/sfw/bin/idle the results
are 0.002279 resp. 0.222831 secs for the same print test.

The Python version is 2.3.3 and IDLE version 1.0.2 on the same Ultra 20
Opteron box running Solaris 10.

/Jean Brouwers




sam wrote:
 i forgot to mention that i'm running a version of python 2.3 (think
 it's 2.3.5), as that's what was installed and i'm not hooked up to the
 internet with the ultra 20 yet. that may account for some of the
 difference.
 
 sam

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dumping the state of a deadlocked process

[MrJean1]

[EMAIL PROTECTED] wrote:
 MrJean1 wrote:
  Did you try using the signal module?  If not, a basic example is here
  http://docs.python.org/lib/node546.html which may need to be
  extended.

 I looks useful. I gave it a try, and the only weakness it has is that
 when my process locks, it locks so badly that it doesn't respond to
 CTRL-C, or any other signal. But by sending it a SIGQUIT which causes
 it to dump the current state, and then kill it, I get the dump I need.

The Ctrl-C signal SIGINT is caught by Python by default and the signal
handler
raises a KeyboardInterrupt exception.  For any other signals, the
signal is caught
but the signal handler is not called until Python returns to the main
loop.

Therefore, if some extension -like Postgresql in this case- is busy or
hangs, nothing
will happen until Python regains control.



 This is actually not a multi-threaded app. It's an application which
 uses a SQL DB. The problem I was having was that I had a cursor which
 started a transaction, and then never finished. Then some other cursor
 came along and tried to perform a delete table, and they both locked
 up. The cursor isn't ending it's transaction, and the transaction
 prevents the delete table from being executed. Luckily Postgresql
 allows me to list current activity, otherwise I would have been
 scratching my head still.

 Using logging or print statements to debug this sort of things is
 highly unsatisfactory. I think the way Java uses SIGQUIT is pretty
 neat, are there any reasons why Python can't adopt something similar?

I can not anwer that.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: print time comparison: IDLE versus terminal on ultra 20

[MrJean1]

On my Ultra 20 box, the test program takes 0.00039982 secs in a
terminal window and 0.236839 secs in IDLE, i.e. about 600x slower.
This is ActivePython 2.4.3 for Solaris 10 on a 2+ GHz Opteron.

A partial explanation for the difference in run time between terminal
and IDLE is that IDLE uses thru a Tkinter/Tcl/Tk window which
postprocesses the output, like colorizing, etc.

 /Jean Brouwers

PS) On MacOS X 10.3.9 and a 400 MHz G4 the terminal time is 0.172556
secs and IDLE takes between 4.04731 and 4.87804 secs (depending on the
options selected) or 24-28x longer .  This is Python 2.4.3, IDLE 1.1.3
and Tcl/Tk 8.4.13 as distributed with MacOS X.


sam wrote:
 hi all,

 i continue to footle around on my spanking new ultra 20 (1.8GHz /
 Opteron Model 144), gradually trying to get to grips with python and
 unix both.

 the slow print time in IDLE had already struck me as rather odd.
 running programs with heavy print requirements from the terminal was a
 major discovery though, printing being so fast as to make everything
 appear at once except in the case of literally thousands of items to be
 printed.

 test case:

 import time

 time1 = time.time()
 for i in range(100):
 print 'go get \'em, son!'
 time2 = time.time()

 print time2-time1


 in IDLE:
 4.433 seconds

 in terminal:
 0.001 seconds

 a difference of between 3 and 4 orders of magnitude is rather striking.
 anyone know what's going on here? is it a python, a unix thing, or
 something else?
 
 sam

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: profiling memory usage

[MrJean1]

The latest Python release 2.5 includes improvements to the memory
manager and *may* work better for you, it all depends on the root cause
of the problem.  For more details, see the 5th bullet on this page
http://docs.python.org/whatsnew/ports.html.

/Jean Brouwers


Eddie wrote:
 Hi,

 I am looking for a method to profile memory usage in my python program.
 The program provides web service and therefore is intended to run for a
 long time. However, the memory usage tends to increase all the time,
 until in a day or two the system cannot handle it any more and starts
 to do constant swapping. Is there a way to look at which objects or
 variables are taking the huge amount of memory space?
 
 Thanks,
 
 Eddie

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dumping the state of a deadlocked process

[MrJean1]

Did you try using the signal module?  If not, a basic example is here
http://docs.python.org/lib/node546.html which may need to be
extended.

/Jean Brouwers


[EMAIL PROTECTED] wrote:
 Hi all

 I'm currently having some issues with a process getting deadlocked. The
 problem is that the only way I can seem to find information about where
 it deadlocks is by making a wild guess, insert a pdb.set_trace() before
 this point, and then step until it locks up, hoping that I've guessed
 right.

 The frustrating part is that most of the time my guesses are wrong.

 It would be really nice if I could send the python process some signal
 which would cause it to print the current stacktrace and exit
 immediately. That way I would quickly be able to pinpoint where in the
 code the deadlock happens. Java has a somewhat similar feature where
 you can send a running VM process a SIGQUIT, to which it will respond
 by dumping all current threads and lots of other information on stdout.
 
 Is this possible somehow?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: releasing memory to malloc

[MrJean1]

The memory manager in the latest Python release 2.5 does return freed
memory to the underlying system, if possible. For more details, see the
5th bullet on this page

  http://docs.python.org/whatsnew/ports.html.

/Jean Brouwers



[EMAIL PROTECTED] wrote:
 The workaround I went with made use of the shelve module and
 calls to gc.collect() to cap the memory consumed by the Python
 allocator. It was a bit intrusive but it got the job done.

 Would a method in the gc module that released memory to malloc
 be something that could get added to Python? Or are there some
 reasons why allowing that would be a bad idea?

 Regards,
 Iker

 P.S.
 This may be a repeat of an earlier message - it seems that
 google groups may have discarded my earlier post.


 [EMAIL PROTECTED] wrote:
  Is there any way to get Python to release memory back to the
  C allocator? I'm currently running a script that goes through
  the following steps:
 
  1) Creates a very large number of Python objects to produce
  a relatively small data structure that sits in a C extension.
  The Python objects consume quite a bit of memory.
 
  2) Releases all the Python objects.
 
  3) Invokes a function of said C extension for further
  processing. This step needs as much memory as possible.
 
  I'd like step 2 to return memory to the C allocator so that it
  is available to the extension in step 3 (which uses malloc).
  
  Regards,
  Iker Arizmendi

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python interpreter on solaris 10

[MrJean1]

FWIIW,

On my stock Ultra 20 / Solaris 10 / Opteron box, python, idle, etc.
just run fine provided /usr/sfw/bin is in your PATH environment
variable.  That is Python 2.3.3, however.

In addition, I installed the Python 2.4.3 build for Solaris from
ActiveState and python, idle, etc. run without any glitch as well.
More at http://www.activestate.com/Products/ActivePython/?mp=1

Lastly, I downloaded the very latest Python 2.5 release and tried to
build that using the SUN compilers.  There are some issues with that
since this is an Opteron and not a SPARC processor.

/Jean Brouwers


sam wrote:
 dear all,

 having spent the last couple of weeks getting to grips with python on
 windows, i am in the position of trying to make the transition to my
 newly arrived ultra 20.

 however, although there are plenty of files with idle somewhere in the
 title, idle.py (which i assume is the interpreter) will not open, as
 there is 'no installed viewer.' surely sun have not installed python
 only for it to unusable? i may well be completely off base here, as
 using unix is still very much a mystery to me.

 i have tried /usr/sfw/lib/python at the command prompt. it worked
 yesterday, but having now moved and restored the python2.3 file to its
 original position, i find the same command prompt failing to work. have
 i messed it up by moving it around?

 my workstation remains a shiny box which does naught but hum
 malevolently. is there hope?
 
 thank you,
 
 sam

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to evaluate the memory usage of a python program?

[MrJean1]

Iff you are using Python on Linux, here is one option which may work
for you

  http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222

/Jean


Daniel Mark wrote:
 Hello all:

 I have a python program and would like to find out the maximum memory
 used
 by this program.

 Does Python provide such module so I could check it easily?
 
 
 
 
 Thank you
 -Daniel

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Detecting 64bit vs. 32bit Linux

[MrJean1]

Try function architecture() from the platform module in Python 2.3 and
2.4.  The first item of the returned tuple shows whether the underlying
system is 64-bit capable.

Here is what it returns on RedHat Fedora Core 2 Linux on Opteron:

 platform.architecture()
('64bit', 'ELF')
 platform.uname()
('Linux', '', '2.6.16.14', '#1 SMP Sat Jul 1 14:09:18 CDT 2006',
'x86_64', 'x86_64')


On RedHat Fedora Core 2 on Pentium 4:

 platform.architecture()
('32bit', 'ELF')
 platform.uname()
('Linux', '', '2.6.10-1771-FC2', '#1 Mon Mar 28 00:50:14 EST 2005',
'i686', 'i686')


And on MacOS X 10.3.9 G4:

 platform.architecture()
('32bit', '')
 platform.uname()
('Darwin', '', '7.9.0', 'Darwin Kernel Version 7.9.0: Wed Mar 30
20:11:17 PST 2005; root:xnu/xnu-517.12.7.obj~1/RELEASE_PPC ', 'Power
Macintosh', 'powerpc')


/Jean Brouwers



dwelch91 wrote:
 I need to detect whether the operating system I am running on (not the
 Python version) is 64bit or 32bit. One requirement is that I need to
 include support for non-Intel/AMD architectures.

 The 2 ways I have thought detecting 64bit are:

 1. struct.calcsize(P) == 8
 2. '64' in os.uname()[4]

 I'm not convinced that either one of these is really adequate. Does
 anybody have any other ideas on how to do this?
 
 Thanks,
 
 Don

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: calculating system clock resolution

[MrJean1]

Depends  iff you are using Linux, print

  cat  /proc/cpuinfo

and look for the line cpu ...Hz:   Parsing that would be
straightforward.

Keep in mind, the time.time() function reports the wall clock time,
which usually has up to a millisecond resolution, regardless of the CPU
speed.

There is also time.clock(), more about that on
http://docs.python.org/lib/module-time.html

/Jean Brouwers


[EMAIL PROTECTED] wrote:
 Hello all

 I have the problem of how to calculate the resolution of the system
 clock.
 Its now two days of head sratching and still there is nothing more than
 these few lines on my huge white sheet of paper stiring at me. Lame I
 know.

 import time

 t1 = time.time()
 while True:
 t2 = time.time()
 if t2  t1:
 print t1, t2
 # start calculating here
 break


 BTW t1 and t2 print out equal up to the fraction on my machine. What
 does
 python know that I don't? A pointer to the source lines where this is
 implemented
 would even be helpfull to clear this out. Can't seem to find it.
 
 Anyone any ideas?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Use of __slots__

[MrJean1]

An example of the RARE use case may be this particular one

  http://mail.python.org/pipermail/python-list/2004-May/220513.html

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PySizeof: almost useful

[MrJean1]

Check the sizeof() method in the mxTools package from eGenix:

  http://www.egenix.com/files/python/mxTools.html

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best way to determine if a certain PID is still running

[MrJean1]

Take look at the poll() methods in the subprocess.py source file of
your Python install.

It shows how to use the os.wait_pid(pid, os.WNOHANG) to check whether a
process is still running or has terminated (and how, from the returned
status value).

Btw, on *nix you must call os.wait_pid(pid, ...) to avoid creating
zombie processes.

Using the subprocess module hides and handles all those details for
you, both on *nix and Windows.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best way to determine if a certain PID is still running

[MrJean1]

Take look at the poll() methods in the subprocess.py source file of
your Python install.

It shows how to use the os.wait_pid(pid, os.WNOHANG) to check whether a
process is still running or has terminated (and how, from the returned
status value).

Btw, on *nix you must call os.wait_pid(pid, ...) to avoid creating
zombie processes.

Using the subprocess module hides and handles all those details for
you, both on *nix and Windows.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to calculate the CPU time consumption and memory consuption of any python program in Linux

[MrJean1]

For CPU time usage, see the standard time module

  http://docs.python.org/lib/module-time.html

specifically the time.clock() function.  For memory usage see

  http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222


/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

[MrJean1]

Did you try the function I posted on Nov 15?  It returns the high water
mark, like sbrk(0) and works for RH Linux (which is dlmalloc, AFAIK).

/Jean Brouwers

PS) Here is that code again (for RH Linux only!)

size_t hiwm (void) {
/*  info.arena - number of bytes allocated
 *  info.hblkhd - size of the mmap'ed space
 *  info.uordblks - number of bytes used (?)
 */
struct mallinfo info = mallinfo();
size_t s = (size_t) info.arena + (size_t) info.hblkhd;
return (s);
}

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Estimating memory use?

[MrJean1]

There is a function mx_sizeof() in the mx.Tools module from eGenix
which may be helpful.  More at


http://www.egenix.com/files/python/eGenix-mx-Extensions.html#mxTools

/Jean Brouwers


PS) This is an approximation for memory usage which is useful in
certain, simple cases.

Each built-in type has an attribute __basicsize__ which is the size in
bytes needed to represent the basic type.  For example
str.__basicsize__ returns 24 and int.__basictype__ returns 12.

However, __basicsize__ does not include the space needed to store the
object value.  For a string, the length of the string has to be added
(times the character width).  For example, the size of string abcd
would at least approximately str.__basicsize__ + len(abcd) bytes,
assuming single byte characters.

In addition, memory alignment should be taken into account by rounding
the size up to the next multiple of 8 (or maybe 16, depending on
platform, etc.).

An approximation for the amount of memory used by a string S (of single
byte characters) aligned to A bytes would be

   (str.__basicsize__  +  len(S)  +  A - 1)A

Things are more complicated for types like list, tuple and dict and
instances of a class.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Estimating memory use?

[MrJean1]

The name of the function in mx.Tools is sizeof() and not mx_sizeof().
My apologies.

Also, it turns out that the return value of mx.Tools.sizeof() function
is non-aligned.  For example mx.Tools.sizeof(abcde) returns 29 which
is fine, but not entirely accurate.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what happens when the file begin read is too big for all lines to be read with readlines()

[MrJean1]

Just try it, it is not that hard ... ;-)

/Jean Brouwers

PS) Here is what happens on Linux:

  $ limit vmemory 1
  $ python
  ...
   s = file(bugfile).readlines()
  Traceback (most recent call last):
File stdin, line 1 in ?
  MemoryError
  

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

[MrJean1]

For some more details on Linux' mallinfo, see
ftp://gee.cs.oswego.edu/pub/misc/malloc.h and maybe function mSTATs()
in glibc/malloc/malloc.c (RedHat).

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

[MrJean1]

For some more details on Linux' mallinfo, see
ftp://gee.cs.oswego.edu/pub/misc/malloc.h and maybe function mSTATs()
in glibc/malloc/malloc.c (RedHat).

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

[MrJean1]

This may work on MacOS X.  An initial, simple test does yield credible
values.

However, I am not a MacOS X expert.  It is unclear which field of the
malloc_statistics_t struct to use and how malloc_zone_statistics with
zone NULL accumulates the stats for all zones.

/Jean Brouwers

#if _MACOSX
#include malloc/malloc.h
/* typedef struct malloc_statistics_t {
   unsigned  blocks_in_use;
   size_tsize_in_use;
   size_tmax_size_in_use; -- high water mark of touched memory
   size_tsize_allocated;  -- reserved in memory
   } malloc_statistics_t;
*/
size_t hiwm (
size_t since)
{
size_t s;
malloc_statistics_t t;
/* get cummulative (?) stats for all zones */
malloc_zone_statistics(NULL, t);
s = t.size_allocated;  /* or t.max_size_in_use? */
return (s - since);
}
#endif

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: So what are __slots__ and when should I use them?

[MrJean1]

Here is an example of the difference between a class with __slots__ and
__dict__

  http://mail.python.org/pipermail/python-list/2004-May/220513.html

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

[MrJean1]

My suggestion would also be to use sbrk() as it provides a high-water
mark for the memory usage of the process.

Below is the function hiwm() I used on Linux (RedHat).  MacOS X and
Unix versions are straigthforward.  Not sure about Windows.

/Jean Brouwers

#if _LINUX
#include malloc.h

size_t hiwm (void) {
/*  info.arena - number of bytes allocated
 *  info.hblkhd - size of the mmap'ed space
 *  info.uordblks - number of bytes used (?)
 */
struct mallinfo info = mallinfo();
size_t s = (size_t) info.arena + (size_t) info.hblkhd;
return (s);
}

#elif _MAXOSX || _UNIX
#include unistd.h

size_t hiwm (void) {
size_t s = (size_t) sbrk(0);
return (s);
}

#elif _WINDOWS
size_t hiwm (void) {
size_t s = (size_t) 0; /* ??? */
return (s);
}

#endif

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what does 0 mean in MyApp(0)

[MrJean1]

See the documentation for the __init__() method here

  http://www.wxpython.org/docs/api/wx.App-class.html

Btw, this is wxPython 2.6, AFAIK.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parser suggestion

[MrJean1]

My recommendation for a project like this would be SimpleParse

  http://simpleparse.sourceforge.net

Some examples are here

  http://simpleparse.sourceforge.net/simpleparse_grammars.html

and

  http://www-128.ibm.com/developerworks/linux/library/l-simple.html
 
/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Human readable number formatting

[MrJean1]

Here is another function for human formatting:

pre

def sistr(value, prec=None, K=1024.0, k=1000.0, sign='', blank=' '):
'''
Convert value to a signed string with an SI prefix.

The 'prec' value specifies the number of fractional
digits to be included.  Use 'prec=0' to omit any
fraction.  If 'prec' is not specified or None, the
precision is adjusted to make the returned string 6
characters (without the sign).

The 'sign' character is used for positive values.
Negative values are always prefixed with '-'.

Uppercase 'K' is the scale factor for values above
1.0 and lowercase 'k' scales values below 1.0.

The 'blank' character is used as the SI prefix for
values between k and K, i.e. value without an SI
prefix.  Set 'blank' to None, False or '' if no
alignment is required.

name symbol   10**   symbol name
=
deca   da+  1 - d   deci
hecto  h +  2 - c   centi
- - - - - - - - - - - - - - - - -
Kilo   K +  3 - m   milli
Mega   M +  6 -/u   micro
Giga   G +  9 - n   nano
Tera   T + 12 - p   pico
Peta   P + 15 - f   femto
ExaE + 18 - a   atto
Zetta  Z + 21 - z   zepto
Yotta  Y + 24 - y   yocto
-
Xona   X + 27 - x   xonto
Weka   W + 30 - w   wekto
Vunda  V + 33 - v   vunkto
UdaU + 36 - u*  unto
Treda  TD*   + 39 -td   trekto
Sorta  S + 42 - s   sotro
Rinta  R + 45 - r   rimto
Quexa  Q + 48 - q   quekto
Pepta  PP+ 51 -pk   pekro
Ocha   O + 54 - o   otro
Nena   N + 57 -nk   nekto
MInga  MI+ 60 -mk   mikto
Luma   L + 63 - l   lunto

The prefixes below the line are non-sanctioned SI
and are only used until the symbols marked * to
avoid ambiguity.  The symbols above the dotted
line are not used and '/u' is returned as 'u'.

See http://en.wikipedia.org/wiki/Binary_prefix or
http://www.bipm.org/en/si/prefixes.html and maybe
http://jimvb.home.mindspring.com/unitsystem.htm
'''
s, v, p = sign, float(value), None
if v  0.0:
   s, v = '-', -v
if v  K:
   if v = 1.0:
  p = blank
   elif k  10.0:
  for f in iter('munpfazyxwv'):  # no unto, ...
  v *= k  # scale up
  if v = 1.0:
 p = f
 break
elif K  10.0:
   for f in iter('KMGTPEZYXWVU'):  # no Treda, ...
   v /= K  # scale down
   if v  K:
  p = f
  break
 # format value
if p is None:  # too large, small or invalid K, k
   return %.0e* % value
elif prec is None:
   if v  100.0:
  if v  10.0:
 prec = 3
  else:
 prec = 2
   else:
  if v  1000.0:
 prec = 1
  else:
 prec = 0
elif prec  0:
   prec = 0 # rounds
return %s%0.*f%s % (s, prec, v, p)


if __name__ == '__main__':
x = 17
while x  1.0e18:
print sistr(x), x
x *= 17
x = 0.12
while x  1.0e-18:
print sistr(x), x
x *= 0.12

/pre

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Human readable number formatting

[MrJean1]

No, I didn't.  See the references at the bottom.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: memoru usage of process

[MrJean1]

What do you mean by 'Python way' and 'not related to Python'?

How is parsing the output of 'ps' different from the method used in
recipe 286222?

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: memoru usage of process

[MrJean1]

On Linux, this may work for you

  http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222

/Jean Brouwers



Jacek Poplawski wrote:
 I need to know how much memory uses child process (after
 subprocess.Popen), so I want function:

 get_memory_usage(pid)

 I found two ways:

 - call ps and analyze its output - this is not portable (different
 output on Linux, Cygwin and QNX)

 - use resource.getrusage - but it works for self/children, not for
 single process with given pid
 
 Do you know any ideas how to do it in Python?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxPython StyledTextCtrl and tabs?

[MrJean1]

Maybe the SetUseTabs() method helps.  See this page

  http://www.yellowbrain.com/stc/guides.html#setusetabs

and the summary at the top.  Disclaimer: untested.

/Jean Brouwers


Lonnie Princehouse wrote:
 Does anyone know of a way to make the wxPython StyledTextCtrl expand
 tabs into spaces?   (yes, I'm trying to use it to edit Python code :P)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: File parser

[MrJean1]

Take a closer look at SimpleParse/mxTextTools

//www.python.org/pypi/SimpleParse/2.0.1a3

We have used these to parse log files of several 100 MB with simple and
complex grammars up to 250+ productions.  Highly recommended.

/Jean Brouwers

PS) For an introduction see also this story
http://www-128.ibm.com/developerworks/linux/library/l-simple.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Speed quirk: redundant line gives six-fold speedup

[MrJean1]

Two observations:

1 - The difference in run time with and without the dummy* globals is
due to a difference in the number of invokations of the search()
function: 1,140 resp. 27,530 in my environment.

To verify, just change the line

  def search():
  

to

  searches = 0
  def search():
  global searches
  searches += 1
  

and add at the very end

  print searches, searches


2 - The run times with and without the dummy* variables is equal(ly
slow) if the LLentry() class and min() function call are modified to be
independent of the object value.

Change line

  class LLentry: pass

to

  LLinst = 0
  class LLentry(object):
  def __init__(self):
  global LLinst
  LLinst += 1
  self.I = LLinst

and change line

mm = min((c.S, c) for c in rowitems(h))[1].D

to

mm = min((c.S, c.I, c) for c in rowitems(h))[2].D


/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Suppressing checking of modules with pychecker

[MrJean1]

FWIIW,

We use PyChecker all the time with Python files using importing wx plus
wx.grid or wx.stc, etc. and the run times vary between 5 and 15
seconds.

This is Python 2.4 with wxPython 2.4.2.4 and RedHat Fedora Core 2 Linux
running on a 1.2 GHz Pentium 4 M laptop.

/Jean Brouwers

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: whitespace , comment stripper, and EOL converter

[MrJean1]

Attached is another version of the stripper.py file.  It contains my
change which seem to handle docstring correctly (at least on itself).


/Jean Brouwers

pre

##
# Python source stripper / cleaner ;)
##

import os
import sys
import token
import keyword
import StringIO
import tokenize
import traceback
__credits__ = \
'''
J¸rgen Hermann
M.E.Farmer
Jean Brouwers
'''
__version__ = '.8'
__author__ = 'M.E.Farmer'
__date__ =  'Apr 16, 2005,' \
'Jan 15 2005,' \
'Oct 24 2004' \

'''this docstring should be removed
'''

##

class Stripper:
Python source stripper / cleaner

def __init__(self, raw):
self.raw = raw

def format(self, out=sys.stdout, comments=0, docstrings=0,
spaces=1, untabify=1, eol='unix'):
 strip comments,
strip docstrings,
strip extra whitespace and lines,
convert tabs to spaces,
convert EOL's in Python code.

# Store line offsets in self.lines
self.lines = [0, 0]
pos = 0
self.temp = StringIO.StringIO()
# Strips the first blank line if 1
self.lasttoken = 1
self.spaces = spaces
# 0  = no change, 1 = strip 'em
self.comments = comments # yep even these
# 0  = no change, 1 = strip 'em, 8 or 'pep8'= strip all but
's
self.docstrings = docstrings

if untabify:
   self.raw = self.raw.expandtabs()
self.raw = self.raw.rstrip()+' '
self.out = out

# Have you ever had a multiple line ending script?
# They can be nasty so lets get them all the same.
self.raw = self.raw.replace('\r\n', '\n')
self.raw = self.raw.replace('\r', '\n')
self.lineend = '\n'

# Gather lines
while 1:
pos = self.raw.find(self.lineend, pos) + 1
if not pos: break
self.lines.append(pos)

self.lines.append(len(self.raw))
self.pos = 0
self.lastOP = ''

# Wrap text in a filelike object
text = StringIO.StringIO(self.raw)

# Parse the source.
## Tokenize calls the __call__
## method for each token till done.
try:
tokenize.tokenize(text.readline, self)
except tokenize.TokenError, ex:
traceback.print_exc()

# Ok now we write it to a file
# but we also need to clean the whitespace
# between the lines and at the ends.
self.temp.seek(0)

# All this should be written into the
# __call__ method just haven't yet...

# Mac CR
if eol == 'mac':
   self.lineend = '\r'
# Windows CR LF
elif eol == 'win':
   self.lineend = '\r\n'
# Unix LF
else:
   self.lineend = '\n'

for line in self.temp.readlines():
if spaces == -1:
self.out.write(line.rstrip()+self.lineend)
else:
if not line.isspace():
self.lasttoken=0
self.out.write(line.rstrip()+self.lineend)
else:
self.lasttoken+=1
if self.lasttoken=self.spaces and self.spaces:
self.out.write(self.lineend)

def __call__(self, toktype, toktext, (srow,scol), (erow,ecol),
line):
 Token handler.

# calculate new positions
oldpos = self.pos
newpos = self.lines[srow] + scol
self.pos = newpos + len(toktext)

  ##print *token: %s  text: %r  line: %r % \
   (token.tok_name[toktype], toktext, line)

# kill comments
if self.comments:
if toktype == tokenize.COMMENT:
return

# kill doc strings
if self.docstrings:
# a STRING must be a docstring
# if the most recent OP was ':'
if toktype == tokenize.STRING and self.lastOP == ':':
# pep8 frowns on triple single quotes
if (self.docstrings == 'pep8' or
self.docstrings == 8):
if not toktext.endswith(''):
return
else:
return
elif toktype == token.OP:
# remember most recent OP
self.lastOP = toktext
elif self.lastOP == ':':
# newline and indent are OK inside docstring
if toktype not in [token.NEWLINE, token.INDENT]:
   # otherwise the docstring ends
   self.lastOP = ''
elif toktype == token.NEWLINE:
# consider any string starting
# on a new line as a docstring
self.lastOP = ':'

# 

Re: whitespace , comment stripper, and EOL converter

[MrJean1]

There is an issue with both my and your code: it only works if doc
strings are triple quoted and if there are no other triple quoted
strings in the Python code.

A triple quoted string used in an assignment will be removed, for
example this case

  s  =  '''this string should not be removed'''


It is still unclear how to distinguish doc strings from other strings.
Also, I have not checked the precise Python syntax, but doc strings do
not need to be enclosed by triple quotes.  A single quote may be
allowed too.

Maybe this rule will work: a doc string is any string preceded by a
COLON token followed by zero, one or more INDENT or NEWLINE tokens.
Untested!

/Jean Brouwers



M.E.Farmer wrote:
 Thanks Jean,
 I have thought about adding docstrings several times, but I was
stumped
 at how to determine a docstring from a regular tripleqoted string ;)
 I have been thinking hard about the problem and I think I have an
idea.
 If the line has nothing before the start of the string it must be a
 docstring.
 Sounds simple enough but in Python there are 12 or so 'types' of
 strings .
 Here is my crack at it feel free to improve it ;)
 I reversed  the logic on the comments and docstrings so I could add a
 special mode to docstring stripping ...pep8 mode .
 Pep8 mode only strips double triple quotes from your source code
 leaving the offending single triple quotes behind. Probably just
stupid
 but someone might find it usefull.

##
 # Python source stripper

##

 import os
 import sys
 import token
 import keyword
 import StringIO
 import tokenize
 import traceback
 __credits__ = '''
 Jürgen Hermann
 M.E.Farmer
 Jean Brouwers
 '''
 __version__ = '.8'
 __author__ = 'M.E.Farmer'
 __date__ =  'Apr 16, 2005,' \
 'Jan 15 2005,' \
 'Oct 24 2004' \



##

 class Stripper:
 Python source stripper
 
 def __init__(self, raw):
 self.raw = raw

 def format(self, out=sys.stdout, comments=0, docstrings=0,
 spaces=1, untabify=1, eol='unix'):
  strip comments,
 strip docstrings,
 strip extra whitespace and lines,
 convert tabs to spaces,
 convert EOL's in Python code.
 
 # Store line offsets in self.lines
 self.lines = [0, 0]
 pos = 0
 # Strips the first blank line if 1
 self.lasttoken = 1
 self.temp = StringIO.StringIO()
 self.spaces = spaces
 self.comments = comments
 self.docstrings = docstrings

 if untabify:
self.raw = self.raw.expandtabs()
 self.raw = self.raw.rstrip()+' '
 self.out = out

 # Have you ever had a multiple line ending script?
 # They can be nasty so lets get them all the same.
 self.raw = self.raw.replace('\r\n', '\n')
 self.raw = self.raw.replace('\r', '\n')
 self.lineend = '\n'

 # Gather lines
 while 1:
 pos = self.raw.find(self.lineend, pos) + 1
 if not pos: break
 self.lines.append(pos)

 self.lines.append(len(self.raw))
 self.pos = 0

 # Wrap text in a filelike object
 text = StringIO.StringIO(self.raw)

 # Parse the source.
 ## Tokenize calls the __call__
 ## method for each token till done.
 try:
 tokenize.tokenize(text.readline, self)
 except tokenize.TokenError, ex:
 traceback.print_exc()

 # Ok now we write it to a file
 # but we also need to clean the whitespace
 # between the lines and at the ends.
 self.temp.seek(0)

 # All this should be written into the
 # __call__ method just haven't yet...

 # Mac CR
 if eol == 'mac':
self.lineend = '\r'
 # Windows CR LF
 elif eol == 'win':
self.lineend = '\r\n'
 # Unix LF
 else:
self.lineend = '\n'

 for line in self.temp.readlines():
 if spaces == -1:
 self.out.write(line.rstrip()+self.lineend)
 else:
 if not line.isspace():
 self.lasttoken=0
 self.out.write(line.rstrip()+self.lineend)
 else:
 self.lasttoken+=1
 if self.lasttoken=self.spaces and self.spaces:
 self.out.write(self.lineend)

 def __call__(self, toktype, toktext,
  (srow,scol), (erow,ecol), line):
  Token handler.
 
 # calculate new positions
 oldpos = self.pos
 newpos = self.lines[srow] + scol
 self.pos = newpos + len(toktext)

 # kill comments
 if self.comments:
 if toktype == 

Re: whitespace , comment stripper, and EOL converter

[MrJean1]

Great tool, indeed!  But doc strings stay in the source text.

If you do need to remove doc strings as well, add the following into
the __call__ method.

...  # kill doc strings
...  if not self.docstrings:
...  if toktype == tokenize.STRING and len(toktext) = 6:
...  t = toktext.lstrip('rRuU')
...  if ((t.startswith(''') and t.endswith(''')) or
...  (t.startswith('') and t.endswith(''))):
...  return

as shown in the original post below.  Also, set self.docstrings in the
format method, similar to self.comments as shown below in lines
starting with '...'.


/Jean Brouwers



M.E.Farmer wrote:
 qwweeeit wrote:
  Thanks! If you answer to my posts one more time I could  consider
you
 as
  my tutor...
 
  It was strange to have found a bug...! In any case I will not go
 deeper
  into the matter, because for me it's enough your explanatiom.
  I corrected the problem by hand removing the tokens spanning
multiple
 lines
  (there were only 8 cases...).
 
  Instead I haven't understood your hint about comments...
  I succeded  in realizing a python script which removes comments.
 
  Here it. is (in all its cumbersome and criptic appearence!...):
 
  # removeCommentsTok.py
  import tokenize
  Input = pippo1
  Output = pippo2
  f = open(Input)
  fOut=open(Output,w)
 
  nLastLine=0
  for i in tokenize.generate_tokens(f.readline):
  .   if i[0]==52 and nLastLine != (i[2])[0]:
  .   .   fOut.write((i[4].replace(i[1],'')).rstrip()+'\n')
  .   .   nLastLine=(i[2])[0]
  .   elif i[0]==4 and nLastLine != (i[2])[0]:
  .   .   fOut.write((i[4]))
  .   .   nLastLine=(i[2])[0]
  f.close()
  fOut.close()
 
  Some explanations for the guys like me...:
  - 52 and 4 are the arbitrary codes for comments and NEWLINE
 respectively
  - the comment removing is obtained by clearing the comment (i[1])
in
 the
input line (i[4])
  - I also right trimmed the line to get rid off the remaining
blanks.
 Tokenizer sends multiline strings and comments as a single token.


##
 # python comment and whitespace stripper :)

##

 import keyword, os, sys, traceback
 import StringIO
 import token, tokenize
 __credits__ = 'just another tool that I needed'
 __version__ = '.7'
 __author__ = 'M.E.Farmer'
 __date__ =  'Jan 15 2005, Oct 24 2004'


##

 class Stripper:
 python comment and whitespace stripper :)
 
 def __init__(self, raw):
 self.raw = raw

...   def format(self, out=sys.stdout, comments=0, docstrings=0,
spaces=1,
 untabify=1, eol='unix'):
 ''' strip comments, strip extra whitespace,
 convert EOL's from Python code.
 '''
 # Store line offsets in self.lines
 self.lines = [0, 0]
 pos = 0
 # Strips the first blank line if 1
 self.lasttoken = 1
 self.temp = StringIO.StringIO()
 self.spaces = spaces
 self.comments = comments
...   self.docstrings = docstrings

 if untabify:
self.raw = self.raw.expandtabs()
 self.raw = self.raw.rstrip()+' '
 self.out = out

 self.raw = self.raw.replace('\r\n', '\n')
 self.raw = self.raw.replace('\r', '\n')
 self.lineend = '\n'

 # Gather lines
 while 1:
 pos = self.raw.find(self.lineend, pos) + 1
 if not pos: break
 self.lines.append(pos)

 self.lines.append(len(self.raw))
 # Wrap text in a filelike object
 self.pos = 0

 text = StringIO.StringIO(self.raw)

 # Parse the source.
 ## Tokenize calls the __call__
 ## function for each token till done.
 try:
 tokenize.tokenize(text.readline, self)
 except tokenize.TokenError, ex:
 traceback.print_exc()

 # Ok now we write it to a file
 # but we also need to clean the whitespace
 # between the lines and at the ends.
 self.temp.seek(0)

 # Mac CR
 if eol == 'mac':
self.lineend = '\r'
 # Windows CR LF
 elif eol == 'win':
self.lineend = '\r\n'
 # Unix LF
 else:
self.lineend = '\n'

 for line in self.temp.readlines():
 if spaces == -1:
 self.out.write(line.rstrip()+self.lineend)
 else:
 if not line.isspace():
 self.lasttoken=0
 self.out.write(line.rstrip()+self.lineend)
 else:
 self.lasttoken+=1
 if self.lasttoken=self.spaces and self.spaces:
 self.out.write(self.lineend)


 def __call__(self, toktype, toktext,
 (srow,scol), (erow,ecol), line):
 '''