[ANN] wxPython 2.8.1.1
Announcing -- The 2.8.1.1 release of wxPython is now available for download at http://wxpython.org/download.php. This release adds a few minor enhancements and a number of bug fixes designed to further stabalize the 2.8.x release series. Source code is available, as well as binaries for both Python 2.4 and 2.5, for Windows and Mac, as well some pacakges for various Linux distributions. A summary of changes is listed below and also at http://wxpython.org/recentchanges.php. What is wxPython? - wxPython is a GUI toolkit for the Python programming language. It allows Python programmers to create programs with a robust, highly functional graphical user interface, simply and easily. It is implemented as a Python extension module that wraps the GUI components of the popular wxWidgets cross platform library, which is written in C++. wxPython is a cross-platform toolkit. This means that the same program will usually run on multiple platforms without modifications. Currently supported platforms are 32-bit Microsoft Windows, most Linux or other Unix-like systems using GTK2, and Mac OS X 10.3+, in most cases the native widgets are used on each platform to provide a 100% native look and feel for the application. Changes in 2.8.1.1 -- wxMSW: Fix lack of spin control update event when control lost focus Added a typeId property to the PyEventBinder class that holds the eventType ID used for that event. So when you need the eventType (such as when sending your own instance of standard events) you can use, for example, wx.EVT_BUTTON.typeId instead of wx.wxEVT_COMMAND_BUTTON_CLICKED. Note that there are a few composite events, such as EVT_MOUSE and EVT_SCROLL, that will actually bind multiple event types at once, and in these cases the typeId property may not give you what you want. You should use te component events in these cases. PyCrust now has an option for showing/hiding the notebook. wxMSW: Corrected drawing of bitmaps for disabled menu items. Enhanced the wx.lib.mixins.inspect module. In addition to showing a PyCrust window it is now a widget browser, which provides a tree loaded up with all the widgets in the app, optionally with the sizers too, and also a panel displaying the properties of the selected window. Run the demo and type Ctrl-Alt-I keystroke (or Cmd-Alt-I on the Mac) to see how it works. You can add this to your own apps with just a few lines of code. Added wx.SearchCtrl.[Get|Set]DescriptiveText wxMac: Added support for the wx.FRAME_FLOAT_ON_PARENT style. wxMac: the popups used for call tips and autocomplete lists in StyledTextCtrl (such as in PyShell) are now top-level float-on-parent windows so they are no longer clipped by the bounds of the stc window. Added EVT_TASKBAR_CLICK and use it to show taskbar icon menu on right button release, not press, under MSW (bug 1623761) Added wx.TreeCtrl.CollapseAll[Children]() and IsEmpty() methods Fix wx.MDIChidFrame.GetPosition() (patch 1626610) Fix attribute memory leak in wx.grid.Grid::ShowCellEditControl() (patch 1629949) wxGTK: Fix for controls on a toolbar being the full height of the toolbar instead of their natural height. wx.lib.customtreectrl patches from Andrea Gavana. wxMac: Applied patch #1622389, fixing two memory leaks in GetPartialTextExtents. More fixes for the native wx.ListCtrl on Mac. Added wx.aui.AuiNotebook.GetAuiManager(). Added wx.aui.AuiMDIParentFrame and wx.aui.AuiMDIChildFrame, which essentially implement the MDI interface using a normal wx.Frame and a wx.aui.AuiNotebook. -- Robin Dunn Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython! -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
SOAPpy and ArrayOfString
Hello all,
I am trying to use a web services API with Python and SOAPpy as a
client. Some of the method paramters in this web service's WSDL are
asking for an ArrayOfString type. Here is my code so far:
#!/usr/bin/python
from SOAPpy import WSDL
wsdlFile =
'http://localhost:8080/axis/services/USD_R11_WebService?WSDL'
server = WSDL.Proxy(wsdlFile)
server.soapproxy.config.dumpSOAPOut = 1
server.soapproxy.config.dumpSOAPIn = 1
SID = server.login('srvcdesk', '')
contacts = server.doSelect(SID, 'cnt', userid = 'abeju01', 1,
['last_name', 'first_name', 'userid'])
I tried using a simple Python list for the argument that wants
ArrayOfString. Python generates the following XML for the last
argument of the doSelect command above:
v5 SOAP-ENC:arrayType=xsd:string[3] xsi:type=SOAP-ENC:Array
itemlast_name/item
itemfirst_name/item
itemuserid/item
/v5
The web service does not like this at all and returns the following
error:
SOAPpy.Types.faultType: Fault soapenv:Server.userException:
org.xml.sax.SAXException: Bad types (class [Ljava.lang.Object; - class
usdjws65.ArrayOfString):
I know in Perl I had to do some special gyrations to pass the right
structue for ArrayOfString to this same web service. Is there any way
to force a type in Python or any other ideas on how to do this?
--
http://mail.python.org/mailman/listinfo/python-list
mmap caching
I've been trying to track down a memory leak (which I initially attributed erroneously to numpy) and it turns out to be caused by a memory mapped file. It seems that mmap caches without limit the chunks it reads, as the memory usage grows to several hundreds MBs according to the Windows task manager before it dies with a MemoryError. I'm positive that these chunks are not referenced anywhere else; in fact if I change the mmap object to a normal file, memory usage remains constant. The documentation of mmap doesn't mention anything about this. Can the caching strategy be modified at the user level ? George -- http://mail.python.org/mailman/listinfo/python-list
Re: A solution to the MSVCRT vs MSVCR71 problem?
At Sunday 21/1/2007 00:07, sturlamolden wrote: Solution to problem 1: Compile with your compiler or choice, never mind which CRT are used. Use a 'dumpbin' program, find all references to msvcrt in the binary DLL file. Create a dll with name py_crt that exports these functions but redirects them to msvcrt71. That is, in the file py_crt.def we put something like EXPORTS malloc=msvcr71.malloc free=msvcr71.free etc. Compile the DLL py_crt.dll and then open your pyd file in binary mode. Exchange all occurances of the string msvcrt (or any other CRT name) with py_crt. Now your binary should work just fine. What you need to make sure is just that the name of the proxy has the same number of letters as the CRT your compiler linked. So if it is msvcrt81.dll, e.g. use something like py_crt81.dll instead of py_crt.dll. Solution to problem 2: [modify external references inside python25.dll to use a different runtime] This would only work, if runtime dll's were compatibles between them, and they are not. You can't blindly redirect a call to msvcr71.__xyz to msvcr80.__xyz and expect that to work magically - it may have a different number of arguments, or different types, or even may not exist anymore. (And what about any symbol exported by ordinal?) -- Gabriel Genellina Softlab SRL __ Preguntá. Respondé. Descubrí. Todo lo que querías saber, y lo que ni imaginabas, está en Yahoo! Respuestas (Beta). ¡Probalo ya! http://www.yahoo.com.ar/respuestas -- http://mail.python.org/mailman/listinfo/python-list
Re: A solution to the MSVCRT vs MSVCR71 problem?
Gabriel Genellina schrieb: This would only work, if runtime dll's were compatibles between them, and they are not. You can't blindly redirect a call to msvcr71.__xyz to msvcr80.__xyz and expect that to work magically - it may have a different number of arguments, or different types, or even may not exist anymore. Actually, the libraries *are* binary-compatible (on the ABI level). You just can't mix two libraries in a single program easily. (And what about any symbol exported by ordinal?) That doesn't happen for msvcrt, as the import library links by name. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
***** See my Cam *****
http://clippay.blogspot.com True Pooper... -- http://mail.python.org/mailman/listinfo/python-list
Re: A solution to the MSVCRT vs MSVCR71 problem?
At Sunday 21/1/2007 05:38, Martin v. Löwis wrote: Gabriel Genellina schrieb: This would only work, if runtime dll's were compatibles between them, and they are not. You can't blindly redirect a call to msvcr71.__xyz to msvcr80.__xyz and expect that to work magically - it may have a different number of arguments, or different types, or even may not exist anymore. Actually, the libraries *are* binary-compatible (on the ABI level). You just can't mix two libraries in a single program easily. That's a good thing - but is this just by accident, or is documented somewhere? I remember that I tried something like that in the past, and failed. (Perhaps earlier versions where not fully backwards compatible - or I didn't try hard enough that time). -- Gabriel Genellina Softlab SRL __ Preguntá. Respondé. Descubrí. Todo lo que querías saber, y lo que ni imaginabas, está en Yahoo! Respuestas (Beta). ¡Probalo ya! http://www.yahoo.com.ar/respuestas -- http://mail.python.org/mailman/listinfo/python-list
Re: Number methods
Nick Maclaren wrote: I can't find any description of these. Most are obvious, but some are not. Note that this is from the point of view of IMPLEMENTING them, not USING them. Specifically: The Python equivalents of these methods are described in the reference manual: http://docs.python.org/ref/numeric-types.html More details can be founf in various PEPs: http://www.python.org/dev/peps/ Does Python use classic division (nb_divide) and inversion (nb_invert) or are they entirely historical? Note that I can very easily provide the latter. Python uses classic divison by default. True divison is used only when the division __future__ directive is in effect. See PEP 238 for details: http://www.python.org/dev/peps/pep-0238/ The nb_invert method is used for the implementation of the bitwise inverse unary operator (~). I don't think that it is deprecated. See: http://docs.python.org/lib/bitstring-ops.html for details. Is there any documentation on the coercion function (nb_coerce)? It seems to have unusual properties. It is used for old style Python classes and extension types that don't have Py_TPFLAGS_CHECKTYPES in their tp_flags. See: http://docs.python.org/ref/coercion-rules.html and http://www.python.org/dev/peps/pep-0208/ for details. Ziga -- http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
Gary Jefferson wrote: Thanks for the reply Vinay. I had been reading those docs prior to posting, but addFilter/removeFilter only work on individual logger instances and handlers. I want to globally enable/disable multiple logger instances, based on their namespaces (see the example I provided). I cannot find a way to do this. I can certainly call addFilter on every instance of every logger throughout the source code, but this is more than inconvenient -- it breaks when new modules are added with their own loggers in their own namespaces (until fixed by manually adding addFilter() to those instances). e.g., suppose you have a source tree with a several components that do network access, and several components that do other things like saving state to disk, accessing the filesystem, etc. And suppose you find a bug that you think is in the networking code. It would be nice to be able to GLOBALLY enable just the loggers that belong in the networking namespace, and disable all others (similar to how Filters are supposed to work for individual loggers). And then to be able to do this with any component, by providing, for example, a command line switch or environment variable. Otherwise, the poor programmer is forced to go and edit every module in the source tree to selectively turn on/off their respecitve loggers. Or am I missing something really obvious about how this is done with the logging module? I don't know enough about your target environment and application to necessarily give you the best advice, but I'll make some general comments which I hope are useful. You seem to be thinking that loggers are binary - i.e. you turn them on or off. But they can be controlled more finely than that; you should be able to get the effect that you want by using the different logging levels available judiciously, as well as using the fact that loggers inhabit a named hierarchy. Note that loggers, by default, inherit the level of the first ancestor logger which has an explicitly set level (by ancestor, I mean in the name hierarchy). The root logger's default level is WARNING, so by default all loggers will work at this level. [N.B. You can also set levels for individual handlers, e.g. to ensure that only CRITICAL conditions are emailed to a specified email address using SMTPHandler, or that ERROR conditions and above are written to file but not to console.] So, for your networking scenario, let's suppose you do the following: Have all network loggers live in the hierarchy namespace below network (e.g. network, network.tcp, network.http etc.). By default, all of these will only log events of severity WARNING and above. Also, suppose you log events in your application code, which are sometimes but not always of interest, at level DEBUG or level INFO. Then, these events will never show up in the logging output, since they are below WARNING in severity. Subsequently, if you want to turn on logging verbosity for the network code only, you can arrange, via a command-line switch or environment variable or configuration file, to do logging.getLogger(network).setLevel(logging.DEBUG) whereupon you will start seeing events from the networking code at severity DEBUG and INFO. This will affect all loggers in the network hierarchy whose levels you have not explicitly set (so that they will get the effective level of the first ancestor which has a level explicitly set - the logger named network). If all you are interested in is turning on verbosity based on different event severities (levels), you should not need to use or set Filters. Filters are for use only when levels don't meet your use case requirements. Best regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Is any python like linux shell?
I learned some python in windows. And now I've turned to linux. I read a book and it teaches how to write shell script with bash, but I don't feel like the grammar of bash. Since I know about python, I want to get a linux shell which use python grammar. I searched by google and I found pysh, which is not maintained any more. There's another script named pyshell, which is not likely what I'm searching for. So, will somebody please tell me if there are any python like shells for linux? -- http://mail.python.org/mailman/listinfo/python-list
Re: Is any python like linux shell?
Frank Potter wrote: I learned some python in windows. And now I've turned to linux. I read a book and it teaches how to write shell script with bash, but I don't feel like the grammar of bash. Since I know about python, I want to get a linux shell which use python grammar. I searched by google and I found pysh, which is not maintained any more. There's another script named pyshell, which is not likely what I'm searching for. So, will somebody please tell me if there are any python like shells for linux? IPython is actually more than a simple shell. Give it a try: http://ipython.scipy.org/. George -- http://mail.python.org/mailman/listinfo/python-list
Re: A solution to the MSVCRT vs MSVCR71 problem?
Gabriel Genellina wrote: This would only work, if runtime dll's were compatibles between them, and they are not. It is standard C, defined by ANSI and ISO. -- http://mail.python.org/mailman/listinfo/python-list
Re: A solution to the MSVCRT vs MSVCR71 problem?
Gabriel Genellina schrieb: That's a good thing - but is this just by accident, or is documented somewhere? It's documented somewhere (although I can't find the documentation right now - it explains how you can link object files from a static library compiled with an older compiler version against a new version of the C library). I remember that I tried something like that in the past, and failed. (Perhaps earlier versions where not fully backwards compatible - or I didn't try hard enough that time). It's been that way for ages, atleast since they started to support 32-bit code. So if you had problems, they might have had a different source. It's a different thing for the C++ libraries, though. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: A solution to the MSVCRT vs MSVCR71 problem?
sturlamolden schrieb: This would only work, if runtime dll's were compatibles between them, and they are not. It is standard C, defined by ANSI and ISO. ANSI and ISO don't define the ABI, though. For example, the definition of the FILE type might (and does) vary across compilers, even if all these compilers implement C99. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: mmap caching
George Sakkis [EMAIL PROTECTED] wrote: I've been trying to track down a memory leak (which I initially attributed erroneously to numpy) and it turns out to be caused by a memory mapped file. It seems that mmap caches without limit the chunks it reads, as the memory usage grows to several hundreds MBs according to the Windows task manager before it dies with a MemoryError. I'm positive that these chunks are not referenced anywhere else; in fact if I change the mmap object to a normal file, memory usage remains constant. The documentation of mmap doesn't mention anything about this. Can the caching strategy be modified at the user level ? I'm not familiar with mmap() on windows, but assuming it works the same way as unix... The point of mmap() is to map files into memory. It is completely up to the OS to bring pages into memory for you to read / write to, and completely up to the OS to get rid of them again. What you would expect is that the file is demand paged into memory as you access bits of it. These pages will remain in memory until the OS feels some memory pressure when the pages will be written out if dirty and then dropped. The OS will try to keep hold of pages as long as possible just in case you need them again. The pages dropped should be the least recently used pages. I wouldn't have expected a MemoryError though... Did you do mmap.flush() after writing? -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list
Python Windows Editors
I downloaded python-2.5.msi and installed it. I believe its editor is IDE. I understand there's a Win editor called pythonwin. I believe it's in the download pywin32-210.win32-py2.5.exe, but I'm not sure if this exe file has just the editor or all of Python. Comments? If not how do I get the PythonWin editor by itself? BTW, one of the features I did not like of IDE is the limited file Print command. It puts everything in 16pt type, and gives no choice over what pages should be printed. Maybe there's an option? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- Web Page: home.earthlink.net/~mtnviews -- http://mail.python.org/mailman/listinfo/python-list
Beginners Tutorial in PDF Format?
I'd like to print a tutorial in one fell swoop, but it seems most on the various sites are page by page embedded descriptions in the page. Any available as a pdf? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- Web Page: home.earthlink.net/~mtnviews -- http://mail.python.org/mailman/listinfo/python-list
Accessing class variables in staticmethods.
Hi, I want to access a static variable in a staticmethod. The variable can be redefined by derived classes and that should be reflected in base's staticmethod. Consider this trivial example- class Base: staticvar = 'Base' @staticmethod def printname(): # this doesn't work # print staticvar # this does work but derived classes wouldn't behave as I want print Base.staticvar class Derived(Base): staticvar = 'Derived' Base.printname() # should print 'Base' Derived.printname() # should print 'Derived' Any idea on how to go about this? Also from a staticmethod how can I find out other attributes of the class (not objects)? Do static methods get some classinfo via some implicit argument(s)? Thanks in advance, Ram -- http://mail.python.org/mailman/listinfo/python-list
Re: Accessing class variables in staticmethods.
On 21 Jan 2007 12:49:17 -0800, Ramashish Baranwal [EMAIL PROTECTED] wrote: class Base: staticvar = 'Base' @staticmethod def printname(): # this doesn't work # print staticvar # this does work but derived classes wouldn't behave as I want print Base.staticvar class Derived(Base): staticvar = 'Derived' Base.printname() # should print 'Base' Derived.printname() # should print 'Derived' Any idea on how to go about this? Also from a staticmethod how can I find out other attributes of the class (not objects)? Do static methods get some classinfo via some implicit argument(s)? No, staticmethods get told nothing about the class they're being defined in. What you want is a classmethod, which gets passed the class to work with. Using classmethods, your code becomes: #untested, bear in mind class Base: staticvar = 'Base' @classmethod def printname(cls): print cls.staticvar class Derived(Base): staticvar = 'Derived' Base.printname() #prints 'Base' Derived.printname() #prints 'Derived' Incidentally, you can also use cls.__name__ for this purpose, but I guess that your actual motivation for this is more complicated than class names. -- http://mail.python.org/mailman/listinfo/python-list
Re: mmap caching
Nick Craig-Wood wrote: George Sakkis [EMAIL PROTECTED] wrote: I've been trying to track down a memory leak (which I initially attributed erroneously to numpy) and it turns out to be caused by a memory mapped file. It seems that mmap caches without limit the chunks it reads, as the memory usage grows to several hundreds MBs according to the Windows task manager before it dies with a MemoryError. I'm positive that these chunks are not referenced anywhere else; in fact if I change the mmap object to a normal file, memory usage remains constant. The documentation of mmap doesn't mention anything about this. Can the caching strategy be modified at the user level ? I'm not familiar with mmap() on windows, but assuming it works the same way as unix... The point of mmap() is to map files into memory. It is completely up to the OS to bring pages into memory for you to read / write to, and completely up to the OS to get rid of them again. What you would expect is that the file is demand paged into memory as you access bits of it. These pages will remain in memory until the OS feels some memory pressure when the pages will be written out if dirty and then dropped. The OS will try to keep hold of pages as long as possible just in case you need them again. The pages dropped should be the least recently used pages. I wouldn't have expected a MemoryError though... Did you do mmap.flush() after writing? The file is written once and then opened as read-only, there's no flushing. So if caching is completely up to the OS, I take it that my options are either (1) modify my algorithms so that they work in fixed-size batches instead of arbitrarily long sequences or (2) implement my own memory-mapping scheme to fit my algorithms. I guess (1) would be the less trouble overall, or is there a way to give a hint to the OS on how large cache can it use ? George -- http://mail.python.org/mailman/listinfo/python-list
Re: Beginners Tutorial in PDF Format?
W. Watson wrote: I'd like to print a tutorial in one fell swoop, but it seems most on the various sites are page by page embedded descriptions in the page. Any available as a pdf? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- Web Page: home.earthlink.net/~mtnviews http://docs.python.org/download.html http://www.diveintopython.org/ HTH Robert -- http://mail.python.org/mailman/listinfo/python-list
Re: mmap caching
George Sakkis schrieb: I've been trying to track down a memory leak (which I initially attributed erroneously to numpy) and it turns out to be caused by a memory mapped file. It seems that mmap caches without limit the chunks it reads, as the memory usage grows to several hundreds MBs according to the Windows task manager before it dies with a MemoryError. You must be misinterpreting what you are seeing. It's the operating system that decides what part of a memory-mapped file are held in memory, and that is certainly not without limits. Notice that there are several values that can be called memory usage (such as the size of the committed address space, the working set size, etc); you don't mention which of these values grows several hundreds MB. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Windows Editors
W. Watson wrote: I downloaded python-2.5.msi and installed it. I believe its editor is IDE. I understand there's a Win editor called pythonwin. I believe it's in the download pywin32-210.win32-py2.5.exe, but I'm not sure if this exe file has just the editor or all of Python. Comments? If not how do I get the PythonWin editor by itself? BTW, one of the features I did not like of IDE is the limited file Print command. It puts everything in 16pt type, and gives no choice over what pages should be printed. Maybe there's an option? Print your files from Notepad, after all, they're just text files. But IDLE has more usefull Formatting options. Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- Web Page: home.earthlink.net/~mtnviews -- http://mail.python.org/mailman/listinfo/python-list
Re: SOAPpy and ArrayOfString
Figured it out. This works:
#!/usr/bin/python
import SOAPpy
wsdlFile =
'http://localhost:8080/axis/services/USD_R11_WebService?WSDL'
server = SOAPpy.WSDL.Proxy(wsdlFile)
server.soapproxy.config.dumpSOAPOut = 1
server.soapproxy.config.dumpSOAPIn = 1
SID = server.login('srvcdesk', 'x')
attr = SOAPpy.structType()
attr._addItem('string', 'userid')
attr._addItem('string', 'last_name')
attr._addItem('string', 'first_name')
contacts = server.doSelect(SID, 'cnt', userid = 'srvcdesk', 1, attr)
print contacts
server.logout(SID)
The XML generated is very similar, but my web service seems to like
this for some reason...weird.
icius wrote:
Hello all,
I am trying to use a web services API with Python and SOAPpy as a
client. Some of the method paramters in this web service's WSDL are
asking for an ArrayOfString type. Here is my code so far:
#!/usr/bin/python
from SOAPpy import WSDL
wsdlFile =
'http://localhost:8080/axis/services/USD_R11_WebService?WSDL'
server = WSDL.Proxy(wsdlFile)
server.soapproxy.config.dumpSOAPOut = 1
server.soapproxy.config.dumpSOAPIn = 1
SID = server.login('srvcdesk', '')
contacts = server.doSelect(SID, 'cnt', userid = 'abeju01', 1,
['last_name', 'first_name', 'userid'])
I tried using a simple Python list for the argument that wants
ArrayOfString. Python generates the following XML for the last
argument of the doSelect command above:
v5 SOAP-ENC:arrayType=xsd:string[3] xsi:type=SOAP-ENC:Array
itemlast_name/item
itemfirst_name/item
itemuserid/item
/v5
The web service does not like this at all and returns the following
error:
SOAPpy.Types.faultType: Fault soapenv:Server.userException:
org.xml.sax.SAXException: Bad types (class [Ljava.lang.Object; - class
usdjws65.ArrayOfString):
I know in Perl I had to do some special gyrations to pass the right
structue for ArrayOfString to this same web service. Is there any way
to force a type in Python or any other ideas on how to do this?
--
http://mail.python.org/mailman/listinfo/python-list
Reading Fortran Data
Hello All: After trying to find an open source alternative to Matlab (or IDL), I am currently getting acquainted with Python and, in particular SciPy, NumPy, and Matplotlib. While I await the delivery of Travis Oliphant's NumPy manual, I have a quick question (hopefully) regarding how to read in Fortran written data. The data files are not binary, but ASCII text files with no formatting and mixed data types (strings, integers, floats). For example, I have the following write statements in my Fortran code: I write the files as such: WRITE(90,'(A30)') fgeo_name WRITE(90,'(A30)') fmed_name WRITE(90,*) nfault,npoint WRITE(90,*) (xpt(n), n=1,npoint) WRITE(90,*) (ypt(n), n=1,npoint) and, WRITE(10,'(A30)') fname DO i=1,nfault WRITE(10,*) dbn(i),dtn(i),xfwnt(i),yfwnt(i),xfent(i),yfent(i), slpvlS(i),slpvlD(i),slpvlT(i),segdp1(i) END DO I then respectively read them into Fortran as: READ(70,'(A30)') fgeo_name READ(70,'(A30)') fmed_name READ(70,*) nfault,npoint READ(70,*) (x(n), n=1,npoint) READ(70,*) (y(n), n=1,npoint) and, READ(20,'(A30)') fname DO i=1,nfault READ(20,*) dbn(i),dtn(i),xfwnt(i),yfwnt(i),xfent(i),yfent(i), slpvlS(i),slpvlD(i),slpvlT(i),segdp1(i) END DO I also read them into IDL for visualization using the READF command. I was wondering how I might go about reading this into Python using NumPy. If this is not trivial, let me know and I'll just wait until the NumPy manual arrives. Cheers, t. -- http://mail.python.org/mailman/listinfo/python-list
instancemethod
import MySQLdb
class Db:
_db=-1
_cursor=-1
@classmethod
def __init__(self,server,user,password,database):
self._db=MySQLdb.connect(server , user , password , database)
self._cursor=self._db.cursor()
@classmethod
def excecute(self,cmd):
self._cursor.execute(cmd)
self._db.commit()
@classmethod
def rowcount(self):
return int(self._cursor.rowcount)
@classmethod
def fetchone(self):
return self._cursor.fetchone()
@classmethod
def close(self):
self._cursor.close()
self._db.close()
if __name__ == '__main__':
gert=Db('localhost','root','**','gert')
gert.excecute('select * from person')
for x in range(0,gert.rowcount):
print gert.fetchone()
gert.close()
[EMAIL PROTECTED]:~$ python ./Desktop/svn/db/Py/db.py
Traceback (most recent call last):
File ./Desktop/svn/db/Py/db.py, line 35, in module
for x in range(0,gert.rowcount):
TypeError: range() integer end argument expected, got instancemethod.
[EMAIL PROTECTED]:~$
Can anybody explain what i must do in order to get integer instead of
a instance ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: pyparsing Combine without merging sub-expressions
Steven Bethard wrote:
Within a larger pyparsing grammar, I have something that looks like::
wsj/00/wsj_0003.mrg
When parsing this, I'd like to keep around both the full string, and the
AAA_ substring of it, so I'd like something like::
foo.parseString('wsj/00/wsj_0003.mrg')
(['wsj/00/wsj_0003.mrg', 'wsj_0003'], {})
How do I go about this? I was using something like::
digits = pp.Word(pp.nums)
alphas = pp.Word(pp.alphas)
wsj_name = pp.Combine(alphas + '_' + digits)
wsj_path = pp.Combine(alphas + '/' + digits + '/' + wsj_name +
... '.mrg')
But of course then all I get back is the full path::
wsj_path.parseString('wsj/00/wsj_0003.mrg')
(['wsj/00/wsj_0003.mrg'], {})
The tokens are what the tokens are, so if you want to replicate a
sub-field, then you'll need a parse action to insert it into the
returned tokens. BUT, if all you want is to be able to easily *access*
that sub-field, then why not give it a results name? Like this:
wsj_name = pp.Combine(alphas + '_' + digits).setResultsName(name)
Leave everything else the same, but now you can access the name field
independently from the rest of the combined tokens.
result = wsj_path.parseString('wsj/00/wsj_0003.mrg')
print result.dump()
print result.name
print result.asList()
-- Paul
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python Windows Editors
The pywin32 extension does not contain the python runtime itself, but comes with many (very useful) classes you can use when interacting with the windows universe. The editor is based on a component called scintilla (http://www.scintilla.org/). The referenced site also offers an editor SciTE based around the component, which comes with some nice features and syntax files for a number of languages. If you are going to make use of python on the windows platform, I would highly recommend the pywin32 extension. Regards, Ant W. Watson wrote: I downloaded python-2.5.msi and installed it. I believe its editor is IDE. I understand there's a Win editor called pythonwin. I believe it's in the download pywin32-210.win32-py2.5.exe, but I'm not sure if this exe file has just the editor or all of Python. Comments? If not how do I get the PythonWin editor by itself? BTW, one of the features I did not like of IDE is the limited file Print command. It puts everything in 16pt type, and gives no choice over what pages should be printed. Maybe there's an option? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- Web Page: home.earthlink.net/~mtnviews -- http://mail.python.org/mailman/listinfo/python-list
Re: instancemethod
if __name__ == '__main__':
gert=Db('localhost','root','**','gert')
gert.excecute('select * from person')
for x in range(0,gert.rowcount):
print gert.fetchone()
gert.close()
[EMAIL PROTECTED]:~$ python ./Desktop/svn/db/Py/db.py
Traceback (most recent call last):
File ./Desktop/svn/db/Py/db.py, line 35, in module
for x in range(0,gert.rowcount):
TypeError: range() integer end argument expected, got instancemethod.
[EMAIL PROTECTED]:~$
Can anybody explain what i must do in order to get integer instead of
a instance ?
Gert,
for x in range(0,gert.rowcount):
gert.rowcount is the method (and not a data attribute).
gert.rowcount() is the method call, which get the return value from
method.
So try this.
for x in range( 0,gert.rowcount() ):
-N
--
http://mail.python.org/mailman/listinfo/python-list
Re: instancemethod
On 21 Jan 2007 14:35:19 -0800, Nanjundi [EMAIL PROTECTED] wrote:
if __name__ == '__main__':
gert=Db('localhost','root','**','gert')
gert.excecute('select * from person')
for x in range(0,gert.rowcount):
print gert.fetchone()
gert.close()
[EMAIL PROTECTED]:~$ python ./Desktop/svn/db/Py/db.py
Traceback (most recent call last):
File ./Desktop/svn/db/Py/db.py, line 35, in module
for x in range(0,gert.rowcount):
TypeError: range() integer end argument expected, got instancemethod.
[EMAIL PROTECTED]:~$
Can anybody explain what i must do in order to get integer instead of
a instance ?
Gert,
for x in range(0,gert.rowcount):
gert.rowcount is the method (and not a data attribute).
gert.rowcount() is the method call, which get the return value from
method.
So try this.
for x in range( 0,gert.rowcount() ):
Doh! :)
thx
--
http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
I don't know if this is helpfull or not but (or for that matter current). http://cens.ioc.ee/projects/f2py2e/ offers some suggestions and it looks like you can use it with c code also. Tyler wrote: Hello All: After trying to find an open source alternative to Matlab (or IDL), I am currently getting acquainted with Python and, in particular SciPy, NumPy, and Matplotlib. While I await the delivery of Travis Oliphant's NumPy manual, I have a quick question (hopefully) regarding how to read in Fortran written data. The data files are not binary, but ASCII text files with no formatting and mixed data types (strings, integers, floats). For example, I have the following write statements in my Fortran code: I write the files as such: WRITE(90,'(A30)') fgeo_name WRITE(90,'(A30)') fmed_name WRITE(90,*) nfault,npoint WRITE(90,*) (xpt(n), n=1,npoint) WRITE(90,*) (ypt(n), n=1,npoint) and, WRITE(10,'(A30)') fname DO i=1,nfault WRITE(10,*) dbn(i),dtn(i),xfwnt(i),yfwnt(i),xfent(i),yfent(i), slpvlS(i),slpvlD(i),slpvlT(i),segdp1(i) END DO I then respectively read them into Fortran as: READ(70,'(A30)') fgeo_name READ(70,'(A30)') fmed_name READ(70,*) nfault,npoint READ(70,*) (x(n), n=1,npoint) READ(70,*) (y(n), n=1,npoint) and, READ(20,'(A30)') fname DO i=1,nfault READ(20,*) dbn(i),dtn(i),xfwnt(i),yfwnt(i),xfent(i),yfent(i), slpvlS(i),slpvlD(i),slpvlT(i),segdp1(i) END DO I also read them into IDL for visualization using the READF command. I was wondering how I might go about reading this into Python using NumPy. If this is not trivial, let me know and I'll just wait until the NumPy manual arrives. Cheers, t. -- http://mail.python.org/mailman/listinfo/python-list
Re: Py 2.5 on Language Shootout
On 20 Jan 2007 14:19:12 -0800, Isaac Gouy [EMAIL PROTECTED] wrote: Ramon Diaz-Uriarte wrote: On 20 Jan 2007 11:34:46 -0800, Isaac Gouy [EMAIL PROTECTED] wrote: (...) And that's why the existence of CINT is such a stark reminder of the separation between the language and the implementation. When people think of C what do they think of - gcc? tiny-c? intel c? microsoft c? some mythical C implementation? Really, this ain't my war. Sure, there are two things: the language and the implementation. But, for practical purposes, when most people today say Python they mean CPython, whereas if the say Scheme, they certainly need to say _which_ Scheme (I think only PLT is in the official shootout page; there are others in the beta tests). If people want to mean Jython or Stackless, they just say that. As for C, I think people will need to qualify what exactly they mean. I think all these issues do not really lead to confusion for most of us; certainly not if you go to the shootout page. But as I said, this ain't my war. I was simply pointing out that correcting one poster for talking about languages when referring to python was hair splitting. And I think we are all running in circles, because I guess we all agree. This is turning into what in Spain (a country of catholic tradition) we call a discussion about the sex of the angels (el sexo de los angeles), i.e., whether angels are male or female or something else. Since sexing angels is not my area of expertise, I'll just shut up (I actually don't really know why I even said anything about this issue; please, forgive my chatiness). In England the corresponding expression is Counting Angels on a Pinhead http://dannyayers.com/2001/misc/angels.htm Thanks, that is neat. I find the discussion on the sex of the angels, well, sexier. But we are probably a few hundred years late to start a catholic-protestant religious war here :-). R. Best, R. I think readers understood the previous poster. Best, R. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list -- Ramon Diaz-Uriarte Statistical Computing Team Structural Biology and Biocomputing Programme Spanish National Cancer Centre (CNIO) http://ligarto.org/rdiaz -- http://mail.python.org/mailman/listinfo/python-list -- Ramon Diaz-Uriarte Statistical Computing Team Structural Biology and Biocomputing Programme Spanish National Cancer Centre (CNIO) http://ligarto.org/rdiaz -- http://mail.python.org/mailman/listinfo/python-list -- Ramon Diaz-Uriarte Statistical Computing Team Structural Biology and Biocomputing Programme Spanish National Cancer Centre (CNIO) http://ligarto.org/rdiaz -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Windows Editors
Hi, I guess you could take a look at pydev extension for eclipse and if you wan't an almost straight forward installation, you should consider this link : http://www.easyeclipse.org/site/distributions/python.html Regards, Laurent W. Watson a écrit : I downloaded python-2.5.msi and installed it. I believe its editor is IDE. I understand there's a Win editor called pythonwin. I believe it's in the download pywin32-210.win32-py2.5.exe, but I'm not sure if this exe file has just the editor or all of Python. Comments? If not how do I get the PythonWin editor by itself? BTW, one of the features I did not like of IDE is the limited file Print command. It puts everything in 16pt type, and gives no choice over what pages should be printed. Maybe there's an option? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
[EMAIL PROTECTED] wrote: I don't know if this is helpfull or not but (or for that matter current). http://cens.ioc.ee/projects/f2py2e/ offers some suggestions and it looks like you can use it with c code also. f2py has been folded into numpy. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: mmap caching
Martin v. Löwis wrote: George Sakkis schrieb: I've been trying to track down a memory leak (which I initially attributed erroneously to numpy) and it turns out to be caused by a memory mapped file. It seems that mmap caches without limit the chunks it reads, as the memory usage grows to several hundreds MBs according to the Windows task manager before it dies with a MemoryError. You must be misinterpreting what you are seeing. It's the operating system that decides what part of a memory-mapped file are held in memory, and that is certainly not without limits. Sure; what I meant was that that whatever the limit is, it's high enough that a MemoryError is raised before the limit is reached. Notice that there are several values that can be called memory usage (such as the size of the committed address space, the working set size, etc); you don't mention which of these values grows several hundreds MB. It's the one in the 'Processes' tab of the Windows task manager (XP proffesional). By the way, I ran the same program on a box with more physical memory and the mem. usage stops growing at around 430MB, by which time the whole file is most likely cached. I'd be interested in any suggestions other than buy more RAM :) (these are not my machines anyway). Thanks, George -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Tyler wrote: Hello All: After trying to find an open source alternative to Matlab (or IDL), I am currently getting acquainted with Python and, in particular SciPy, NumPy, and Matplotlib. While I await the delivery of Travis Oliphant's NumPy manual, I have a quick question (hopefully) regarding how to read in Fortran written data. The data files are not binary, but ASCII text files with no formatting and mixed data types (strings, integers, floats). For example, I have the following write statements in my Fortran code: Konrad Hinsen has a module for reading this kind of file. http://dirac.cnrs-orleans.fr/ScientificPython/ Specifically, Scientific.IO.FortranFormat . -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
[ANN] wxPython 2.8.1.1
Announcing -- The 2.8.1.1 release of wxPython is now available for download at http://wxpython.org/download.php. This release adds a few minor enhancements and a number of bug fixes designed to further stabalize the 2.8.x release series. Source code is available, as well as binaries for both Python 2.4 and 2.5, for Windows and Mac, as well some pacakges for various Linux distributions. A summary of changes is listed below and also at http://wxpython.org/recentchanges.php. What is wxPython? - wxPython is a GUI toolkit for the Python programming language. It allows Python programmers to create programs with a robust, highly functional graphical user interface, simply and easily. It is implemented as a Python extension module that wraps the GUI components of the popular wxWidgets cross platform library, which is written in C++. wxPython is a cross-platform toolkit. This means that the same program will usually run on multiple platforms without modifications. Currently supported platforms are 32-bit Microsoft Windows, most Linux or other Unix-like systems using GTK2, and Mac OS X 10.3+, in most cases the native widgets are used on each platform to provide a 100% native look and feel for the application. Changes in 2.8.1.1 -- wxMSW: Fix lack of spin control update event when control lost focus Added a typeId property to the PyEventBinder class that holds the eventType ID used for that event. So when you need the eventType (such as when sending your own instance of standard events) you can use, for example, wx.EVT_BUTTON.typeId instead of wx.wxEVT_COMMAND_BUTTON_CLICKED. Note that there are a few composite events, such as EVT_MOUSE and EVT_SCROLL, that will actually bind multiple event types at once, and in these cases the typeId property may not give you what you want. You should use te component events in these cases. PyCrust now has an option for showing/hiding the notebook. wxMSW: Corrected drawing of bitmaps for disabled menu items. Enhanced the wx.lib.mixins.inspect module. In addition to showing a PyCrust window it is now a widget browser, which provides a tree loaded up with all the widgets in the app, optionally with the sizers too, and also a panel displaying the properties of the selected window. Run the demo and type Ctrl-Alt-I keystroke (or Cmd-Alt-I on the Mac) to see how it works. You can add this to your own apps with just a few lines of code. Added wx.SearchCtrl.[Get|Set]DescriptiveText wxMac: Added support for the wx.FRAME_FLOAT_ON_PARENT style. wxMac: the popups used for call tips and autocomplete lists in StyledTextCtrl (such as in PyShell) are now top-level float-on-parent windows so they are no longer clipped by the bounds of the stc window. Added EVT_TASKBAR_CLICK and use it to show taskbar icon menu on right button release, not press, under MSW (bug 1623761) Added wx.TreeCtrl.CollapseAll[Children]() and IsEmpty() methods Fix wx.MDIChidFrame.GetPosition() (patch 1626610) Fix attribute memory leak in wx.grid.Grid::ShowCellEditControl() (patch 1629949) wxGTK: Fix for controls on a toolbar being the full height of the toolbar instead of their natural height. wx.lib.customtreectrl patches from Andrea Gavana. wxMac: Applied patch #1622389, fixing two memory leaks in GetPartialTextExtents. More fixes for the native wx.ListCtrl on Mac. Added wx.aui.AuiNotebook.GetAuiManager(). Added wx.aui.AuiMDIParentFrame and wx.aui.AuiMDIChildFrame, which essentially implement the MDI interface using a normal wx.Frame and a wx.aui.AuiNotebook. -- Robin Dunn Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Windows Editors
W. Watson wrote: I downloaded python-2.5.msi and installed it. I believe its editor is IDE. I understand there's a Win editor called pythonwin. I believe it's in the download pywin32-210.win32-py2.5.exe, but I'm not sure if this exe file has just the editor or all of Python. Comments? If not how do I get the PythonWin editor by itself? BTW, one of the features I did not like of IDE is the limited file Print command. It puts everything in 16pt type, and gives no choice over what pages should be printed. Maybe there's an option? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. there a too many (good) IDE's for Python ;-) maybe we should give them a ranking, I tried 2 of them, and only was stable enough: PyScripter http://mmm-experts.com/Products.aspx?ProductID=4 cheers, Stef Mientki -- http://mail.python.org/mailman/listinfo/python-list
Re: mmap caching
George Sakkis schrieb: You must be misinterpreting what you are seeing. It's the operating system that decides what part of a memory-mapped file are held in memory, and that is certainly not without limits. Sure; what I meant was that that whatever the limit is, it's high enough that a MemoryError is raised before the limit is reached. The operating system will absolutely, definitely, certainly release any cached data it can purge before reporting it is out of memory. So if you get a MemoryError, it is *not* because the operating system has cached too much data. In fact, memory that is read in because of mmap should *never* cause a MemoryError. Python calls MapViewOfFile when mmap.mmap is invoked, at which point the operating commits to providing that much address space to the application, along with backing storage on disk (typically, from the file being mapped, unless it is an anonymous map). Later access to the mapped range cannot fail (except for hardware errors), and if it would, you wouldn't see a MemoryError. It's the one in the 'Processes' tab of the Windows task manager (XP proffesional). By the way, I ran the same program on a box with more physical memory and the mem. usage stops growing at around 430MB, by which time the whole file is most likely cached. I'd be interested in any suggestions other than buy more RAM :) (these are not my machines anyway). As a starting point, try understanding better what is really happening. Turn on Virtual Memory Size in View/Select Columns also, and perhaps a few additional counters as well. Also take a look at the Commit Charge, which takes into account swap file usage as well. Try increasing the size of the swap file. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: PyMeld for html templates?
Thanks Richie! That's exactly the reply I was hoping for. Hooray! Sean On Jan 20, 2007, at 9:50 PM, Richie Hindle wrote: Hi Sean, Thanks Richie -- but actually, what I had in mind was slightly different. I want for my CONTENT pages to only contain the content. So to modify your example: LAYOUT = htmlheadtitle id='title'The Title/title/head bodydiv id='info'The page information goes here./div p id='footerCopyright Me 2007./p/body/html INFO = div id='info'pHere is the info, which would in the real world be read from a file./p/div Can I do this just as easily? Yes, of course, just the same way: from PyMeld import Meld LAYOUT = htmlheadtitle id='title'The Title/title/head bodydiv id='info'The page information goes here./div p id='footerCopyright Me 2007./p/body/html INFO = div id='info'pHere is the info, which would in the real world be read from a file./p/div page = Meld(LAYOUT) info = Meld(INFO) page.info = info.info._content print page There's nothing that says a Meld has to be a complete HTML document. One other unrelated question... Let's say I have a table with a template row with id=myRow. I want to use PyMeld to clone that row n number of times (depending on database results). So here's my stoopid question: Does that mean that my XHTML will contain n rows all with the same id=myRow thereby making my XHTML invalid? Yes, it does, but you can simply delete the 'id' attributes from the rows, with `del row.id`, before adding them back into the document. -- Richie Hindle [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Tyler wrote: Hello All: After trying to find an open source alternative to Matlab (or IDL), I am currently getting acquainted with Python and, in particular SciPy, NumPy, and Matplotlib. While I await the delivery of Travis Oliphant's NumPy manual, I have a quick question (hopefully) regarding how to read in Fortran written data. The data files are not binary, but ASCII text files with no formatting and mixed data types (strings, integers, floats). For example, I have the following write statements in my Fortran code: In plain Python, you can read each line in to a string, break the string into words using split, and then convert the words into variables of the desired types. If you are new to Python, this is an important idiom to learn. I don't know if NumPy has facilities to do this more easily. I write the files as such: WRITE(90,'(A30)') fgeo_name WRITE(90,'(A30)') fmed_name Let me comment on the Fortran code. For the following lines using list-directed output, the compiler has considerable freedom in how it writes the output. I guess you expect the integers nfault and npoint to be written on one line and the vectors xpt and ypt to each be written on separate lines. The compiler could print each number on a separate line and be standard-conforming. This does not matter if you are going to use a Fortran list-directed read to read the file, but it will matter if you are using other languages. I suggest that you use format strings to get more control over the ouptput format before you think about reading the output files in Python. Otherwise you will be trying to hit a moving target. snip -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Tyler wrote: Hello All: After trying to find an open source alternative to Matlab (or IDL), I am currently getting acquainted with Python and, in particular SciPy, NumPy, and Matplotlib. While I await the delivery of Travis Oliphant's NumPy manual, I have a quick question (hopefully) regarding how to read in Fortran written data. I think you made a good choice, if for no other reason than the fact that Python and numpy absolutely rock when it comes to interfacing with Fortran (and any C code that can be called by Fortran). I suggest having a look at pyfort and/or f2py to see if they can be useful to you. If your data files are temporary (that is, they only exist to pass data from one program to another), you might not even need them. For completeness, I'll answer how to write as well as read Fortran data. It turns out that all of this can be done without numpy, but there is one very convenient numpy function. The data files are not binary, but ASCII text files with no formatting and mixed data types (strings, integers, floats). For example, I have the following write statements in my Fortran code: I write the files as such: WRITE(90,'(A30)') fgeo_name WRITE(90,'(A30)') fmed_name Fortran pads it's output when using a width field. A good way to do this in Python is to use string formating with a given width. If fgeo_name is a Python string, they you'd write something like this: f.write(%-30s\n % fgeo_name) The negeative sign is to pad the name on the left, which is how Fortran does it (by default, Python string formatting pads on the right). You could also use the .ljust method of string objects: f.write(fgeo_name.ljust(30) + \n) Don't forget the newline on the end. WRITE(90,*) nfault,npoint Fortran writes this as two arbitrary integers separated by a space. To do this in Python, use string formating. This also adds a space to the beginning as most Fortran implementations seem to do. f.write( %d %d\n % (nfault,npoint)) WRITE(90,*) (xpt(n), n=1,npoint) WRITE(90,*) (ypt(n), n=1,npoint) Now, an array. I think you'd be safest writing one value per line. You would do that like this: for x in xpt: f.write( %#g\n % x) The # sign is there to force the number to be formatted with a decimal point. You'll probably want to tweak the format string in other ways (to specify a precision, for instance). and, WRITE(10,'(A30)') fname DO i=1,nfault WRITE(10,*) dbn(i),dtn(i),xfwnt(i),yfwnt(i),xfent(i),yfent(i), slpvlS(i),slpvlD(i),slpvlT(i),segdp1(i) END DO I'd write this as one number per line. f.write(%-30s\n % fname) for i in range(nfault): f.write( %#g\n % dbn[i]) f.write( %#g\n % dtn[i]) # and so on If you know Python well, there are more Pythonic ways to do this, but this is straightforward and works well enough. Now for the reading part. I then respectively read them into Fortran as: READ(70,'(A30)') fgeo_name READ(70,'(A30)') fmed_name Once you've opened a file, then: fgeo_name = f.readline().strip() fmed_name = f.readline().strip() Note that this strips the padding off the name. If the first line of the file is ABC followed by 27 spaces, the result will be ABC with the spaces stripped. READ(70,*) nfault,npoint Python doesn't have any built in input formating (a la READ in Fortran or scanf in C), so one usually does this kind of thing by hand. Fortunately, Python makes this quite easy. The following will do what you want: s = f.readline().split() nfault = int(s[0]) npoint = int(s[1]) Here's what happens: it reads in a line, and splits the line on whitespace into substrings. It assigns the list of substrings to s. Then it converts the first substring (s[0]) to an int and assigns it to nfault; the second (s[1]) to npoint. With more advanced knowledge of Python, you could write it in one line like this: nfault,npoint = (int(ss) for ss in f.readline().split()) READ(70,*) (x(n), n=1,npoint) READ(70,*) (y(n), n=1,npoint) Fortran programs seem to wrap free-form output to 80 columns, inserting newlines whenever. Because of this, you can't really free-form read data in a line-by-line way (unless it's short, like the above example), but now have to read data number-by-number, which is not so straightforward. Fortunately, numpy has a function, fromfile, which reads data number-by-number. If f is the file object you're reading from, then you could read in the array like this: x = numpy.fromfile(f,sep= ,count=npoint,dtype=numpy.Float) According to my tests, this leaves f pointing right past the end of the array, so that you can still read in other arrays and variables. Don't forget to specify the type. and, READ(20,'(A30)') fname DO i=1,nfault READ(20,*) dbn(i),dtn(i),xfwnt(i),yfwnt(i),xfent(i),yfent(i), slpvlS(i),slpvlD(i),slpvlT(i),segdp1(i) END DO Because this line has a very good chance of being split up upon output, and because they're all
Re: Code reformater?
On Sun, 21 Jan 2007 14:15:46 +1100, Steven D'Aprano [EMAIL PROTECTED] wrote: Still, it is better not to lose the indentation in the first place. Thanks for the tips. But it does happen when copy/pasting code from either a web page or an e-mail that TABs are messed up, which is not a problem with other languages, but is a problem with Python. Too bad. -- http://mail.python.org/mailman/listinfo/python-list
OpenOffice 2.0 UNO update Links; need help
I've got a ~100 page document I assemble from ~30 OOo .odt files with some search and replace functions. I then produce a PDF. So far so good. Now I need to get a barcode from our internal website and insert that. The barcode will vary based on some parameters. Our internal site provides a .jpg image (or .gif) based on those parameters. Can anyone provide a snippet or some pointers on how to do this. I've googled it to death and checked the OOo DevGuide to no avail. Either python or java solution is acceptable. --- The information contained in this message may be privileged and / or confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and deleting the material from any computer. --- -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Windows Editors
[EMAIL PROTECTED] wrote: W. Watson wrote: I downloaded python-2.5.msi and installed it. I believe its editor is IDE. I understand there's a Win editor called pythonwin. I believe it's in the download pywin32-210.win32-py2.5.exe, but I'm not sure if this exe file has just the editor or all of Python. Comments? If not how do I get the PythonWin editor by itself? BTW, one of the features I did not like of IDE is the limited file Print command. It puts everything in 16pt type, and gives no choice over what pages should be printed. Maybe there's an option? Print your files from Notepad, after all, they're just text files. But IDLE has more usefull Formatting options. Thanks. Already did it, but used Word. I can get line numbers in that way. -- http://mail.python.org/mailman/listinfo/python-list
Re: instancemethod
Gert Cuykens [EMAIL PROTECTED] escribió en el mensaje
news:[EMAIL PROTECTED]
class Db:
_db=-1
_cursor=-1
@classmethod
def __init__(self,server,user,password,database):
self._db=MySQLdb.connect(server , user , password , database)
self._cursor=self._db.cursor()
@classmethod
def excecute(self,cmd):
self._cursor.execute(cmd)
self._db.commit()
if __name__ == '__main__':
gert=Db('localhost','root','**','gert')
gert.excecute('select * from person')
for x in range(0,gert.rowcount):
print gert.fetchone()
gert.close()
Besides your specific question that was already answered, why are you using
classmethods at all?
You are constructing a Db instance and using it as if all were normal
instance methods... Just remove all those @classmethod declarations and use
it in a standard way.
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
Re: Beginners Tutorial in PDF Format?
Robert Hicks wrote: W. Watson wrote: I'd like to print a tutorial in one fell swoop, but it seems most on the various sites are page by page embedded descriptions in the page. Any available as a pdf? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- Web Page: home.earthlink.net/~mtnviews http://docs.python.org/download.html http://www.diveintopython.org/ HTH Robert Thanks, but the first url takes me to a web page that contains the complete tutorial. Does choosing pdf (USA) in the table download a file that contains all the documents on the list below and to the left? Wayne T. Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Two laws Newton and Einstein didn't discover: 1. Time is money. 2. Matter will be damaged in direct proportion to its value. -- Web Page: home.earthlink.net/~mtnviews -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Windows Editors
I prefer PyScripter too, but would like to know if I can have 'indentation guides' enabled like PythonWin allows. On 1/21/07, Stef Mientki [EMAIL PROTECTED] wrote: I tried 2 of them, and only was stable enough: PyScripter http://mmm-experts.com/Products.aspx?ProductID=4 -- http://mail.python.org/mailman/listinfo/python-list
Re: Beginners Tutorial in PDF Format?
W. Watson [EMAIL PROTECTED] escribió en el mensaje news:[EMAIL PROTECTED] Robert Hicks wrote: W. Watson wrote: I'd like to print a tutorial in one fell swoop, but it seems most on the various sites are page by page embedded descriptions in the page. Any available as a pdf? http://docs.python.org/download.html http://www.diveintopython.org/ Thanks, but the first url takes me to a web page that contains the complete tutorial. Does choosing pdf (USA) in the table download a file that contains all the documents on the list below and to the left? Try again. The first url goes to this page: Download Python 2.5 Documentation (19 September 2006): To download an archive containing all the documents for this version of Python in one of various formats [pdf included, you can choose between Letter/A4]. Below, it says These documents are not available for download individually. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: instancemethod
On 1/22/07, Gabriel Genellina [EMAIL PROTECTED] wrote:
Gert Cuykens [EMAIL PROTECTED] escribió en el mensaje
news:[EMAIL PROTECTED]
class Db:
_db=-1
_cursor=-1
@classmethod
def __init__(self,server,user,password,database):
self._db=MySQLdb.connect(server , user , password , database)
self._cursor=self._db.cursor()
@classmethod
def excecute(self,cmd):
self._cursor.execute(cmd)
self._db.commit()
if __name__ == '__main__':
gert=Db('localhost','root','**','gert')
gert.excecute('select * from person')
for x in range(0,gert.rowcount):
print gert.fetchone()
gert.close()
Besides your specific question that was already answered, why are you using
classmethods at all?
You are constructing a Db instance and using it as if all were normal
instance methods... Just remove all those @classmethod declarations and use
it in a standard way.
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
i thought @classmethod was the normal method ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Carl Banks wrote: snip WRITE(90,*) nfault,npoint Fortran writes this as two arbitrary integers separated by a space. I wrote a paragraph in my reply explaining why this is wrong. A Fortran list-directed write can print results in an almost arbitrary format, depending on the compiler. Many compilers will separate integers by several spaces, not just one, and they could use commas instead of spaces if they wanted. The number of items printed before a new line is started is also compiler-dependent. For more control, one uses a formatted write, for example write (90,(2(1x,i0))) nfault,npoint snip -- http://mail.python.org/mailman/listinfo/python-list
Re: instancemethod
On 1/22/07, Gert Cuykens [EMAIL PROTECTED] wrote:
On 1/22/07, Gabriel Genellina [EMAIL PROTECTED] wrote:
Gert Cuykens [EMAIL PROTECTED] escribió en el mensaje
news:[EMAIL PROTECTED]
class Db:
_db=-1
_cursor=-1
@classmethod
def __init__(self,server,user,password,database):
self._db=MySQLdb.connect(server , user , password , database)
self._cursor=self._db.cursor()
@classmethod
def excecute(self,cmd):
self._cursor.execute(cmd)
self._db.commit()
if __name__ == '__main__':
gert=Db('localhost','root','**','gert')
gert.excecute('select * from person')
for x in range(0,gert.rowcount):
print gert.fetchone()
gert.close()
Besides your specific question that was already answered, why are you using
classmethods at all?
You are constructing a Db instance and using it as if all were normal
instance methods... Just remove all those @classmethod declarations and use
it in a standard way.
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
i thought @classmethod was the normal method ?
did some reading and i think i know why now :)
http://www.faqts.com/knowledge_base/view.phtml/aid/16824
python always seems to amaze me how other languages make a mess of
things that suppose to be simple
--
http://mail.python.org/mailman/listinfo/python-list
pylab, matplotlib ... roots function question
Hello NG, given this call to roots funtion from pylab In [342]: roots([0,2,2]) Out[342]: array([-1.]) as far as I understand it [a0,a1,a2] stands for a0+a1*x+a2*x^2 in the above case it yields 2x^2+2x = 2x(1+x) and the roots are 0 and -1 I am wondering why roots function gives me only the -1 second try In [343]: roots([1,0,0]) Out[343]: array([], dtype=float64) ok, as it should be In [344]: roots([0,0,1]) Out[344]: array([], dtype=float64) here again, 0 is the root of x^2 Do I miss something important? Regards, Daniel -- http://mail.python.org/mailman/listinfo/python-list
Re: instancemethod
import MySQLdb
class Db:
_db=-1
_cursor=-1
rowcount=-1
def __init__(self,server,user,password,database):
self._db=MySQLdb.connect(server , user , password , database)
self._cursor=self._db.cursor()
def excecute(self,cmd):
self._cursor.execute(cmd)
self._db.commit()
self.rowcount=int(self._cursor.rowcount())
def fetchone(self):
return self._cursor.fetchone()
def close(self):
self._cursor.close()
self._db.close()
if __name__ == '__main__':
gert=Db('localhost','root','***','gert')
gert.excecute('select * from person')
for x in range(0,gert.rowcount):
print gert.fetchone()
gert.close()
Traceback (most recent call last):
File Desktop/svn/db/Py/db.py, line 28, in module
gert.excecute('select * from person')
File Desktop/svn/db/Py/db.py, line 17, in excecute
self.rowcount=int(self._cursor.rowcount())
TypeError: 'long' object is not callable
I guess i did something wrong again ? :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: instancemethod
never mind i think i need some sleep lol i did the exact opposite this time .rowcount() - .rowcount -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Beliavsky wrote: Carl Banks wrote: snip WRITE(90,*) nfault,npoint Fortran writes this as two arbitrary integers separated by a space. I wrote a paragraph in my reply explaining why this is wrong. It's a safe assumption for a line of two integers. It might not exactly produce what a Fortran program would, but it would work in a read statement. A Fortran list-directed write can print results in an almost arbitrary format, depending on the compiler. Many compilers will separate integers by several spaces, not just one, and they could use commas instead of spaces if they wanted. 1. Hardly any compiler will produce a line of two integers, or reals, that another compiler couldn't read back. 2. The number of spaces separating the numbers isn't important when reading back free-form data. 3. Fear that a Fortran compiler might use commas or wrap lines at ten columns or whatever, because it's not based on a standard, is misguided paranoia. The number of items printed before a new line is started is also compiler-dependent. For more control, one uses a formatted write, for example write (90,(2(1x,i0))) nfault,npoint I think it's just more work to guard against something isn't very relevant in practice. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: PyMeld for html templates?
I'm trying to get PyMeld happening but I'm a bit stumped as to how to make it work with mod_python. The pymeld docs show examples only for the Python command line interpreter and show using the print statement to output stuff. But using mod_python.apache, I think you need to use req.write(something) format. And of course, this fails when you feed it output from Meld. Any tips? Sean On Jan 20, 2007, at 6:18 AM, Richie Hindle wrote: [Sean] I wonder if anyone has any thoughts on PyMeld as a template system for churning out general websites? I'm doing that (but then I would be wouldn't I? 8-) http://www.mandant.net is an example - the content of each page comes from a file containing just the content, the layout and sidebar are defined in a template HTML file, and the navigation is built by a Python script. All that is pulled together using PyMeld into a set of HTML files and deployed to the web server (there's no need to it on the fly for that site, but you certainly could). I want for a URI request to mysite.com/info to pull in a layout.html template which will in turn be populated by the info.html template. [...] how about PyMeld, any ideas on how to do it or whether it will even work as I've described? You'd do something like this: from PyMeld import Meld LAYOUT = htmlheadtitle id='title'The Title/title/head bodydiv id='info'The page information goes here./div p id='footerCopyright Me 2007./p/body/html INFO = htmlheadtitle id='title'The real title/title/head body id='info'pHere is the info, which would in the real world be read from a file./p/body/html page = Meld(LAYOUT) info = Meld(INFO) page.title = info.title._content page.info = info.info._content print page Is that the sort of thing you had in mind? -- Richie Hindle [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Carl Banks wrote: snip A Fortran list-directed write can print results in an almost arbitrary format, depending on the compiler. Many compilers will separate integers by several spaces, not just one, and they could use commas instead of spaces if they wanted. 1. Hardly any compiler will produce a line of two integers, or reals, that another compiler couldn't read back. Yes, but for more than three numbers, the statement is wrong. Intel Fortran prints four double precision random n as 0.555891433847495 0.591161642339424 0.888434673900224 0.487293557925127 but g95 prints them on a single line. I advise against using list-directed Fortran writes to create files that other programs will read, and I think most experienced Fortran programmers would agree. -- http://mail.python.org/mailman/listinfo/python-list
Help with 'popen'
Can someone let me know why this won't work? Thanks.
from os import popen
popen('export asdfasdf=hello').read()
''
popen('echo $asdfasdf').read()
'\n'
Thanks.
Stephen
--
http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
Vinay Sajip wrote: I don't know enough about your target environment and application to necessarily give you the best advice, but I'll make some general comments which I hope are useful. You seem to be thinking that loggers are binary - i.e. you turn them on or off. But they can be controlled more finely than that; you should be able to get the effect that you want by using the different logging levels available judiciously, as well as using the fact that loggers inhabit a named hierarchy. Note that loggers, by default, inherit the level of the first ancestor logger which has an explicitly set level (by ancestor, I mean in the name hierarchy). The root logger's default level is WARNING, so by default all loggers will work at this level. [N.B. You can also set levels for individual handlers, e.g. to ensure that only CRITICAL conditions are emailed to a specified email address using SMTPHandler, or that ERROR conditions and above are written to file but not to console.] So, for your networking scenario, let's suppose you do the following: Have all network loggers live in the hierarchy namespace below network (e.g. network, network.tcp, network.http etc.). By default, all of these will only log events of severity WARNING and above. Also, suppose you log events in your application code, which are sometimes but not always of interest, at level DEBUG or level INFO. Then, these events will never show up in the logging output, since they are below WARNING in severity. Subsequently, if you want to turn on logging verbosity for the network code only, you can arrange, via a command-line switch or environment variable or configuration file, to do logging.getLogger(network).setLevel(logging.DEBUG) whereupon you will start seeing events from the networking code at severity DEBUG and INFO. This will affect all loggers in the network hierarchy whose levels you have not explicitly set (so that they will get the effective level of the first ancestor which has a level explicitly set - the logger named network). If all you are interested in is turning on verbosity based on different event severities (levels), you should not need to use or set Filters. Filters are for use only when levels don't meet your use case requirements. Best regards, Vinay Sajip Vinay, okay, I think what you described will work out for me -- thank you very much for the explanation. I am still a bit confused about Filters, though. It seems they are a bit of an anomoly in the hierarchical view of loggers that the API supports elsewhere, i.e., filters don't seem to inherit... Or am I missing something again? Here's a quick example: import logging log1 = logging.getLogger(top) log2 = logging.getLogger(top.network) log3 = logging.getLogger(top.network.tcp) log4 = logging.getLogger(top.network.http) log5 = logging.getLogger(top.config) log6 = logging.getLogger(top.config.file) logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(levelname)s %(message)s') filter = logging.Filter(top.network) log1.addFilter(filter) # only affects log1, do this for each of log2-7 too? log1.debug(I'm top) log2.debug(I'm top.network) log3.debug(I'm top.network.tcp) log4.debug(I'm top.network.http) log5.debug(I'm top.config) log6.debug(I'm top.config.file) This is only for the binary case (and I think if I ignore the binary case and filters altogether as you suggested), but it really would be nice to be able to squelch /all/ output from loggers that belong to certain parts of the namespace by using a filter as above (which I can't get to work). Perhaps if I set up a basicConfig with a loglevel of nothing, I can get this to approximate squelching of everything but that which I explicitly setLevel (which does inherit properly). In other words, the addFilter/removeFilter part of the API seems rather impotent if it can't be inherited in the logging namespaces. In fact, I can't really figure out a use case where I could possibly want to use it without it inheriting. Obviously I'm missing something. I'm sure I've consumed more attention that I deserve already in this thread, but, do you have any pointers which can enlighten me as to how to effectively use addFilter/removeFilter? many thanks, Gary -- http://mail.python.org/mailman/listinfo/python-list
Re: Is any python like linux shell?
George Sakkis wrote: Frank Potter wrote: I learned some python in windows. And now I've turned to linux. I read a book and it teaches how to write shell script with bash, but I don't feel like the grammar of bash. Since I know about python, I want to get a linux shell which use python grammar. I searched by google and I found pysh, which is not maintained any more. There's another script named pyshell, which is not likely what I'm searching for. So, will somebody please tell me if there are any python like shells for linux? IPython is actually more than a simple shell. Give it a try: http://ipython.scipy.org/. George Frank, IPython is great, but it is not a replacement for a shell like bash. If you have a Linux system then you still need to know the rudiments of bash - Paddy. -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Beliavsky wrote: Carl Banks wrote: snip A Fortran list-directed write can print results in an almost arbitrary format, depending on the compiler. Many compilers will separate integers by several spaces, not just one, and they could use commas instead of spaces if they wanted. 1. Hardly any compiler will produce a line of two integers, or reals, that another compiler couldn't read back. Yes, but for more than three numbers, the statement is wrong. Intel Fortran prints four double precision random n as 0.555891433847495 0.591161642339424 0.888434673900224 0.487293557925127 but g95 prints them on a single line. Did you try to use one compiler's program's output as the other's input? How did it work? I advise against using list-directed Fortran writes to create files that other programs will read, and I think most experienced Fortran programmers would agree. It's been awhile, but I'd consider myself an experienced Fortran programner. I don't really agree. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: html + javascript automations = [mechanize + ?? ] or something else?
I tried to install pamie (but I have mostly used python on cygwin on windows). In the section What will you need to run PAMIE, it says I will need Mark Hammonds Win32 All which I can not find. Can anyone tell me how do I install PAMIE? Do I need python for windows that is different from cygwin's python? Thanks, --j ina wrote: John wrote: I have to write a spyder for a webpage that uses html + javascript. I had it written using mechanize but the authors of the webpage now use a lot of javascript. Mechanize can no longer do the job. Does anyone know how I could automate my spyder to understand javascript? Is there a way to control a browser like firefox from python itself? How about IE? That way, we do not have to go thru something like mechanize? Thanks in advance for your help/comments, --j You want pamie, iec or ishybrowser. Pamie is probably the best choice since it gets patches and updates on a regular basis. http://pamie.sourceforge.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: pyparsing Combine without merging sub-expressions
Dennis Lee Bieber wrote:
On Sat, 20 Jan 2007 13:49:52 -0700, Steven Bethard
[EMAIL PROTECTED] declaimed the following in comp.lang.python:
Within a larger pyparsing grammar, I have something that looks like::
wsj/00/wsj_0003.mrg
When parsing this, I'd like to keep around both the full string, and the
AAA_ substring of it, so I'd like something like::
foo.parseString('wsj/00/wsj_0003.mrg')
(['wsj/00/wsj_0003.mrg', 'wsj_0003'], {})
If working file name/paths, why not use the functions in os.path?
Two reasons. First, as I mentioned, this is within a larger pyparsing
grammar so it's not as easy to switch back and forth between the two.
Second, I do want to do some data validation (e.g. the name of the file
needs to be in a particular format) so I either need to post-process the
os.path approach or just do it in pyparsing.
But that then allows whitespace between the pieces of the path, which
there shouldn't be::
If you didn't have whitespace coming in, there shouldn't be any
going out. If you do, you likely have malformed data and probably should
detect it earlier...
Well that's the intention of using pyparsing here. With a proper
grammar, pyparsing can detect the malformed data for me and throw an error.
STeVe
--
http://mail.python.org/mailman/listinfo/python-list
Re: pyparsing Combine without merging sub-expressions
Paul McGuire wrote:
Steven Bethard wrote:
Within a larger pyparsing grammar, I have something that looks like::
wsj/00/wsj_0003.mrg
When parsing this, I'd like to keep around both the full string, and the
AAA_ substring of it, so I'd like something like::
foo.parseString('wsj/00/wsj_0003.mrg')
(['wsj/00/wsj_0003.mrg', 'wsj_0003'], {})
How do I go about this? I was using something like::
digits = pp.Word(pp.nums)
alphas = pp.Word(pp.alphas)
wsj_name = pp.Combine(alphas + '_' + digits)
wsj_path = pp.Combine(alphas + '/' + digits + '/' + wsj_name +
... '.mrg')
[snip]
BUT, if all you want is to be able to easily *access*
that sub-field, then why not give it a results name? Like this:
wsj_name = pp.Combine(alphas + '_' + digits).setResultsName(name)
Leave everything else the same, but now you can access the name field
independently from the rest of the combined tokens.
Works great. Thanks!
STeVe
--
http://mail.python.org/mailman/listinfo/python-list
Re: Reading Fortran Data
Wow! Thanks for the help everyone. I will look into each of your comments in more detail in the morning, but I'll mention a few things I guess. The first is, the list-directed output was not necesarily my choice as some of the code is also used by my supervisor (and her colleagues) and I had to keep the IO similar. However, for non-legacy codes (even though is was upgraded to Fortran90), I can ensure you that I will employ a more portable format. Also, I am very interested in the Scientific.IO module mentioned and will look into it further along with f2py. Once again, I thank all of you for your replies, I think I even learnt a few things about Fortran here too Cheers, t. -- http://mail.python.org/mailman/listinfo/python-list
Re: html + javascript automations = [mechanize + ?? ] or something else?
My python2.5 installation on windows did not come with win32com.
How do I install/get this module for windows?
Thanks,
--j
Duncan Booth wrote:
John [EMAIL PROTECTED] wrote:
Is there a way
to control a browser like firefox from python itself? How about IE?
IE is easy enough to control and you have full access to the DOM:
import win32com
win32com.client.gencache.EnsureModule('{EAB22AC0-30C1-11CF-A7EB-
C05BAE0B}', 0, 1, 1)
module 'win32com.gen_py.EAB22AC0-30C1-11CF-A7EB-C05BAE0Bx0x1x1' from
'C:\Python25\lib\site-packages\win32com\gen_py\EAB22AC0-30C1-11CF-A7EB-
C05BAE0Bx0x1x1.py'
IE = win32com.client.DispatchEx('InternetExplorer.Application.1')
dir(IE)
['CLSID', 'ClientToWindow', 'ExecWB', 'GetProperty', 'GoBack', 'GoForward',
'GoHome', 'GoSearch', 'Navigate', 'Navigate2', 'PutProperty',
'QueryStatusWB', 'Quit', 'Refresh', 'Refresh2', 'ShowBrowserBar', 'Stop',
'_ApplyTypes_', '__call__', '__cmp__', '__doc__', '__getattr__',
'__init__', '__int__', '__module__', '__repr__', '__setattr__', '__str__',
'__unicode__', '_get_good_object_', '_get_good_single_object_', '_oleobj_',
'_prop_map_get_', '_prop_map_put_', 'coclass_clsid']
IE.Visible=True
IE.Navigate(http://plone.org;)
while IE.Busy: pass
print IE.Document.getElementById(portlet-news).innerHTML
DT class=portletHeaderA class=feedButton link-plain
href=feed://plone.org/news/newslisting/RSSIMG title=RSS subscription
feed for news items alt=RSS src=http://plone.org/rss.gif; /AA
href=http://plone.org/news;News/A /DT
... and so on ...
See
http://msdn.microsoft.com/workshop/browser/webbrowser/reference/objects/int
ernetexplorer.asp
for the documentation.
--
http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
Gary Jefferson wrote: I am still a bit confused about Filters, though. It seems they are a bit of an anomoly in the hierarchical view of loggers that the API supports elsewhere, i.e., filters don't seem to inherit... Or am I missing something again? Here's a quick example: import logging log1 = logging.getLogger(top) log2 = logging.getLogger(top.network) log3 = logging.getLogger(top.network.tcp) log4 = logging.getLogger(top.network.http) log5 = logging.getLogger(top.config) log6 = logging.getLogger(top.config.file) logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(levelname)s %(message)s') filter = logging.Filter(top.network) log1.addFilter(filter) # only affects log1, do this for each of log2-7 too? log1.debug(I'm top) log2.debug(I'm top.network) log3.debug(I'm top.network.tcp) log4.debug(I'm top.network.http) log5.debug(I'm top.config) log6.debug(I'm top.config.file) This is only for the binary case (and I think if I ignore the binary case and filters altogether as you suggested), but it really would be nice to be able to squelch /all/ output from loggers that belong to certain parts of the namespace by using a filter as above (which I can't get to work). Perhaps if I set up a basicConfig with a loglevel of nothing, I can get this to approximate squelching of everything but that which I explicitly setLevel (which does inherit properly). In other words, the addFilter/removeFilter part of the API seems rather impotent if it can't be inherited in the logging namespaces. In fact, I can't really figure out a use case where I could possibly want to use it without it inheriting. Obviously I'm missing something. I'm sure I've consumed more attention that I deserve already in this thread, but, do you have any pointers which can enlighten me as to how to effectively use addFilter/removeFilter? I don't really think there's a problem with the logging API - it went through a fair amount of peer review on python-dev before making it into the Python distribution. You need to use what's there rather than shoehorn it into how you think it ought to be. Filters are for more esoteric requirements - you can see some examples of filters in the old (out of date) standalone distribution at http://www.red-dove.com/python_logging.html (download and examine some of the test scripts). Loggers aren't binary - levels are there to be used. Most people get by with judicious use of levels, using Filters on loggers only for unusual cases. Since Filters are only meant to be used in unusual situations, there is no need to think about inheriting them. Filters can be set on Handlers so that even if Loggers log the events, they don't go to any output if filtered out at the handlers for that output. BTW I would also advise reading PEP-282 to understand more about the logging approach. Best regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Xah's Edu Corner: Introduction to 3D Graphics Programing
Xah Lee wrote in 2006-12-22: Of Interest: Introduction to 3D Graphics Programing http://xahlee.org/3d/index.html Folks, i have expanded my tutorial to several pages in the past nearly two months, and thank you very much for those who have given encouragement. I had plans to write pages that actually contained Python or Perl and elisp code to demonstrate the mingled use of common languages with POV-RAY and other tools for doing algorithmic mathematical art... However, i'm posting now for one in-credible discovery that amazed me, and believe it is a revolutionary both with respect to technology, as well as its social impact. Following is my brief introduction. The web version is at: Introduction to Second Life http://xahlee.org/sl/sl.html --- Introduction to Second Life Xah Lee, 2007-01-09 above: A screenshot of a location in Second Life. This place is called “The Future”, it is a place built by Henry Segerman ↗ built. Second Life is a online virtual world with 2 million accounts as of 2006-12, and reportedly 10 thousand users logged in at any moment. Basically, you operate a software that is a 3D world much like 3D games, but everything in the “game” is built by users, and you can interact with other users, including buying and selling virtual land with real money. What people do inside Second Life is entirely up to them (as in real life). And, as it happens, what people do mostly in Second Life are pretty much what people do in real life. The major activities are: sex, shopping, socializing, dancing. My mathematician friend Henry Segerman introduced me to 2nd life. (also because it appeared in Time Mag in 2006-12.) Henry has a 2nd life page that contains many math objects made in Second Life: http://www.stanford.edu/~segerman/2ndlife.html. Part of my interest in Second Life is to build geometric models. (See Introduction to 3D Graphics Programing) I have for the past couple years sought for a software platform/system where i can build 3D objects, with abilities to do interactive adjustment (such as moving a slide to change a surface's parameter), dynamic rotation (viewing from different angles), animations (such as morphing that shows geometric processes), and most of all, walk-thru in it as if it is a building. As far as my experiences goes, no software platform for 3D graphics are close to the ideal of what i need to do. They lack one feature or the other, or otherwise requires the programer to be a specialist with years of dedication in learning the tool. For example, Mathematica↗ has great collection of math functions but no dynamic graphics. POV-RAY↗ and 3D-modelers like AutoCAD↗ can do great in building 3D objects but they are not designed for interactivity, animations, or walk-thru. Java the programming language↗ allows one to write applets that does rotation and interactive manipulation but programing in Java is extremely unnecessarily complex and yet it is still not possible to do walk-thrus. The one class of platform that does all these, is 3D game engines. But alas, they take a dedicated game programing specialist to be able to use it. Second Life changed all this. (Note: Just for completness, my requirement for a 3D-graphics programing software system is this: • easy to use for average programers or scientists. • Easy to build geometry models, such as basic shapes like spheres and blocks, as well as surfaces or meshes of triangles. • designed for interactivity. That is, the programer can easily build buttons and sliders that changes parameters and have the object reflect these changes visually right away. • the user can easily change viewing angles or rotate the object. • The programer can easily do animations. For example, morph a sphere into a cube, or a bunch of spheres flying in space as a swarm of flies. • The ability to do walk-thru (or fly-thru). For example, if i build a fancy 3D-maze or architecture, i should be able to — say — become a ant, and walk inside the object, so as to view the object from inside and as well as getting the real experience of perceiving such a building. (Before my discover of Second Life, the platform that i was considering nearly ideal and preparing to learn, are VPython↗ and Macromedia Flash↗ (i do still plan to learn these technologies) )) Second Life is not just the answer to my mathematical fantasies, but because of its Real-World nature, being run and build by real people, and with scarily real money market of the virtual dollars it uses, is nothing but a technological revolution with great social impact. It is essentially the virtual-3D-world-wide-web dream of VRML↗ envisioned around 1995. Anshe Chung↗ is famouly known for being the first to become a (real world) millionaire by selling entirely virtual items and virtual services inside Second Life. For a encyclopedic introduction, see: Second Life↗ Here are some articles about Second Life. (note that there are a lot linked at the bottom of the Wikipedia article.) * Time
Re: html + javascript automations = [mechanize + ?? ] or somethingelse?
John [EMAIL PROTECTED] escribió en el mensaje news:[EMAIL PROTECTED] My python2.5 installation on windows did not come with win32com. How do I install/get this module for windows? Look for the pywin32 package at sourceforge.net -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Help with 'popen'
stephen_b wrote:
Can someone let me know why this won't work? Thanks.
from os import popen
popen('export asdfasdf=hello').read()
''
popen('echo $asdfasdf').read()
'\n'
Thanks.
Stephen
Python starts a new shell for each command, so your environment variable
is lost. You probably want this, which makes the variable permanent.
py import os
py os.environ['asdfasdf'] = 'hello'
py os.popen('echo $asdfasdf').read()
'hello\n'
James
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is any python like linux shell?
Paddy wrote: George Sakkis wrote: Frank Potter wrote: I learned some python in windows. And now I've turned to linux. I read a book and it teaches how to write shell script with bash, but I don't feel like the grammar of bash. Since I know about python, I want to get a linux shell which use python grammar. I searched by google and I found pysh, which is not maintained any more. There's another script named pyshell, which is not likely what I'm searching for. So, will somebody please tell me if there are any python like shells for linux? IPython is actually more than a simple shell. Give it a try: http://ipython.scipy.org/. George Frank, IPython is great, but it is not a replacement for a shell like bash. If you have a Linux system then you still need to know the rudiments of bash Or better yet, csh. ;) James -- http://mail.python.org/mailman/listinfo/python-list
Re: html + javascript automations = [mechanize + ?? ] or somethingelse?
I tried it, didnt work with the python25 distribution msi file that is on python.org But activestate python worked. Now I can open IE using COM. What I am trying to figure out is how to click an x,y coordinate on a page in IE automatically using COM. How about typing something automatically...Any ideas? Thanks, --j Gabriel Genellina wrote: John [EMAIL PROTECTED] escribió en el mensaje news:[EMAIL PROTECTED] My python2.5 installation on windows did not come with win32com. How do I install/get this module for windows? Look for the pywin32 package at sourceforge.net -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Is any python like linux shell?
Frank Potter wrote: I learned some python in windows. And now I've turned to linux. I read a book and it teaches how to write shell script with bash, but I don't feel like the grammar of bash. Since I know about python, I want to get a linux shell which use python grammar. I searched by google and I found pysh, which is not maintained any more. There's another script named pyshell, which is not likely what I'm searching for. So, will somebody please tell me if there are any python like shells for linux? Best is to do simple things (ls, mv, rm, cp) via bash or tcsh and to use python for things of any greater complexity (even slightly more complex). My strongest advice would be to let go of the idea of shell scripting altogether and begin to write programs built from libraries to solve your problems. The mere act of beginning to write a shell script (in, say, bash) will tie you to that language, even when you realize your task was much more complex than you had originally imagined. It has happened to me time and time again. James -- http://mail.python.org/mailman/listinfo/python-list
Re: Accessing class variables in staticmethods.
Sam wrote: On 21 Jan 2007 12:49:17 -0800, Ramashish Baranwal [EMAIL PROTECTED] wrote: class Base: staticvar = 'Base' @staticmethod def printname(): # this doesn't work # print staticvar # this does work but derived classes wouldn't behave as I want print Base.staticvar class Derived(Base): staticvar = 'Derived' Base.printname() # should print 'Base' Derived.printname() # should print 'Derived' Any idea on how to go about this? Also from a staticmethod how can I find out other attributes of the class (not objects)? Do static methods get some classinfo via some implicit argument(s)? No, staticmethods get told nothing about the class they're being defined in. What you want is a classmethod, which gets passed the class to work with. Using classmethods, your code becomes: #untested, bear in mind class Base: staticvar = 'Base' @classmethod def printname(cls): print cls.staticvar class Derived(Base): staticvar = 'Derived' Base.printname() #prints 'Base' Derived.printname() #prints 'Derived' Incidentally, you can also use cls.__name__ for this purpose, but I guess that your actual motivation for this is more complicated than class names. Thanks Sam, using classmethod works. You guessed it correctly, my actual motivation is more complicated but on the same line. -Ram -- http://mail.python.org/mailman/listinfo/python-list
Re: pylab, matplotlib ... roots function question
Schüle Daniel wrote: Hello NG, given this call to roots funtion from pylab It's actually from numpy and numpy questions are best asked (and best answered!) on numpy-discussion. http://www.scipy.org/Mailing_Lists In [342]: roots([0,2,2]) Out[342]: array([-1.]) as far as I understand it [a0,a1,a2] stands for a0+a1*x+a2*x^2 in the above case it yields 2x^2+2x = 2x(1+x) and the roots are 0 and -1 I am wondering why roots function gives me only the -1 No, it's the other way around. In [1]: from numpy import * In [2]: roots? Type: function Base Class: type 'function' Namespace: Interactive File: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.2.dev3507-py2.5-macosx-10.4-i386.egg/numpy/lib/polynomial.py Definition: roots(p) Docstring: Return the roots of the polynomial coefficients in p. The values in the rank-1 array p are coefficients of a polynomial. If the length of p is n+1 then the polynomial is p[0] * x**n + p[1] * x**(n-1) + ... + p[n-1]*x + p[n] So you were really solving 2*x + 2 = 0, the single root of which is -1. second try In [343]: roots([1,0,0]) Out[343]: array([], dtype=float64) ok, as it should be No, that's actually wrong. What version of numpy are you using? With a recent SVN checkout of numpy, I get the correct answer: In [3]: roots([1,0,0]) Out[3]: array([ 0., 0.]) -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
class explorer for automating IE
I found this class which was written in 2003. http://xper.org/wiki/seminar/InternetExplorerAutomation Is there a better/more complete version around that someone knows of. Thanks, --j -- http://mail.python.org/mailman/listinfo/python-list
Re: class explorer for automating IE
Is there an analogue of IE Mechanize in python? http://search.cpan.org/src/ABELTJE/Win32-IE-Mechanize-0.009/README Thanks, --j On Jan 22, 1:48 am, John [EMAIL PROTECTED] wrote: I found this class which was written in 2003. http://xper.org/wiki/seminar/InternetExplorerAutomation Is there a better/more complete version around that someone knows of. Thanks, --j -- http://mail.python.org/mailman/listinfo/python-list
Re: Is any python like linux shell?
Frank Potter wrote: I learned some python in windows. And now I've turned to linux. I read a book and it teaches how to write shell script with bash, but I don't feel like the grammar of bash. Since I know about python, I want to get a linux shell which use python grammar. I searched by google and I found pysh, which is not maintained any more. There's another script named pyshell, which is not likely what I'm searching for. So, will somebody please tell me if there are any python like shells for linux? You could in principle use Python as a general-purpose interactive shell, but it wouldn't make much sense. Bash is more convenient for basic operations such as changing directories, listing directories, copying files, etc. For more advanced scripting, I use both bash and Python. I use bash typically as an executive for scripting Python programs. For example, I use bash for stepping through a list of directories and running a set of python programs in each directory. Sure, I could use Python for the whole thing, but simply starting programs and moving files around is more straightforward in bash than in Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: class explorer for automating IE
John wrote: Is there an analogue of IE Mechanize in python? http://www.google.com/search?q=python%20mechanizebtnI=I%27m+Feeling+Lucky -- http://mail.python.org/mailman/listinfo/python-list
[ python-Bugs-1603907 ] subprocess: error redirecting i/o from non-console process
Bugs item #1603907, was opened at 2006-11-27 18:20
Message generated for change (Comment added) made by astrand
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=105470aid=1603907group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Oren Tirosh (orenti)
Assigned to: Peter Åstrand (astrand)
Summary: subprocess: error redirecting i/o from non-console process
Initial Comment:
In IDLE, PythonWin or other non-console interactive Python under Windows:
from subprocess import *
Popen('cmd', stdout=PIPE)
Traceback (most recent call last):
File pyshell#11, line 1, in -toplevel-
Popen('', stdout=PIPE)
File C:\python24\lib\subprocess.py, line 533, in __init__
(p2cread, p2cwrite,
File C:\python24\lib\subprocess.py, line 593, in _get_handles
p2cread = self._make_inheritable(p2cread)
File C:\python24\lib\subprocess.py, line 634, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required
The same command in a console windows is successful.
Why it happens:
subprocess assumes that GetStdHandle always succeeds but when there is no
console it returns None. DuplicateHandle then complains about getting a
non-integer. This problem does not happen when redirecting all three standard
handles.
Solution:
Replace None with -1 (INVALID_HANDLE_VALUE) in _make_inheritable.
Patch attached.
--
Comment By: Peter Åstrand (astrand)
Date: 2007-01-21 16:31
Message:
Logged In: YES
user_id=344921
Originator: NO
This the suggested patches are not ready for commit, I'm moving this issue
to bugs instead.
--
Comment By: Oren Tirosh (orenti)
Date: 2007-01-07 19:13
Message:
Logged In: YES
user_id=562624
Originator: YES
Oops. The new patch does not solve it in all cases in the win32api
version, either...
--
Comment By: Oren Tirosh (orenti)
Date: 2007-01-07 19:09
Message:
Logged In: YES
user_id=562624
Originator: YES
If you duplicate INVALID_HANDLE_VALUE you get a new valid handle to
nothing :-) I guess the code really should not rely on this undocumented
behavior. The reason I didn't return INVALID_HANDLE_VALUE directly is
because DuplicateHandle returns a _subprocess_handle object, not an int.
It's expected to have a .Close() method elsewhere in the code.
Because of subtle difference between in the behavior of the _subprocess
and win32api implementations of GetStdHandle in this case solving this
issue this gets quite messy!
File Added: subprocess-noconsole2.patch
--
Comment By: Peter Åstrand (astrand)
Date: 2007-01-07 11:58
Message:
Logged In: YES
user_id=344921
Originator: NO
This patch looks very interesting. However, it feels a little bit strange
to call DuplicateHandle with a handle of -1. Is this really allowed? What
will DuplicateHandle return in this case? INVALID_HANDLE_VALUE? In that
case, isn't it better to return INVALID_HANDLE_VALUE directly?
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=105470aid=1603907group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 16:46 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1634739group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Closed Resolution: Invalid Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Peter Åstrand (astrand) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py extract_frame.py.output 21 ./extract_frame.py extract_frame.py.output 2 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at (demuxer.c). Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame extract_frame.output2 21 ./extract_frame extract_frame.output2 2 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py extract_frame.py.output2 21 ./extract_frame.py extract_frame.py.output2 2 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print %x % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- Comment By: Peter Åstrand (astrand) Date: 2007-01-21 16:37 Message: Logged In: YES user_id=344921 Originator: NO That's the only thing I managed to get with the C version. But with the Python version, if I don't list the contents of /proc/pid/fd immediately after the transcode process started, I find it very hard to believe that just listing the contents of a kernel-virtual directory can change the behaviour of an application. I think it's much more likely that you have a timing issue. Since nothing indicates that there's actually a problem with the
[ python-Bugs-1598181 ] subprocess.py: O(N**2) bottleneck
Bugs item #1598181, was opened at 2006-11-17 07:40 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1598181group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Closed Resolution: Fixed Priority: 5 Private: No Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Peter Åstrand (astrand) Summary: subprocess.py: O(N**2) bottleneck Initial Comment: subprocess.py (Python 2.5, current SVN, probably all versions) contains this O(N**2) code: bytes_written = os.write(self.stdin.fileno(), input[:512]) input = input[bytes_written:] For large but reasonable input the second line is rate limiting. Luckily, it is very easy to remove this bottleneck. I'll upload a simple patch. Below is a small script that demonstrates the huge speed difference. The output on my machine is: creating input 0.888417959213 slow slicing input 61.1553330421 creating input 0.863168954849 fast slicing input 0.0163860321045 done The numbers are times in seconds. This is the source: import time import sys size = 100 t0 = time.time() print creating input input = \n.join([str(i) for i in xrange(size)]) print time.time()-t0 t0 = time.time() print slow slicing input n_out_slow = 0 while True: out = input[:512] n_out_slow += 1 input = input[512:] if not input: break print time.time()-t0 t0 = time.time() print creating input input = \n.join([str(i) for i in xrange(size)]) print time.time()-t0 t0 = time.time() print fast slicing input n_out_fast = 0 input_done = 0 while True: out = input[input_done:input_done+512] n_out_fast += 1 input_done += 512 if input_done = len(input): break print time.time()-t0 assert n_out_fast == n_out_slow print done -- Comment By: Peter Åstrand (astrand) Date: 2007-01-21 16:45 Message: Logged In: YES user_id=344921 Originator: NO Backported to 2.5, in rev. 53513. -- Comment By: Neal Norwitz (nnorwitz) Date: 2007-01-17 08:00 Message: Logged In: YES user_id=33168 Originator: NO Peter this is fine for 2.5.1. Please apply and update Misc/NEWS. Thanks! -- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2007-01-07 16:15 Message: Logged In: YES user_id=71407 Originator: YES Thanks for the fixes! -- Comment By: Peter Åstrand (astrand) Date: 2007-01-07 15:36 Message: Logged In: YES user_id=344921 Originator: NO Fixed in trunk revision 53295. Is this a good candidate for backporting to 25-maint? -- Comment By: Mike Klaas (mklaas) Date: 2007-01-04 19:20 Message: Logged In: YES user_id=1611720 Originator: NO I reviewed the patch--the proposed fix looks good. Minor comments: - input_done sounds like a flag, not a count of written bytes - buffer() could be used to avoid the 512-byte copy created by the slice -- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2006-11-17 07:43 Message: Logged In: YES user_id=71407 Originator: YES Sorry, I didn't know the tracker would destroy the indentation. I'm uploading the demo source as a separate file. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1598181group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1634739group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Closed Resolution: Works For Me Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Peter Åstrand (astrand) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py extract_frame.py.output 21 ./extract_frame.py extract_frame.py.output 2 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at (demuxer.c). Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame extract_frame.output2 21 ./extract_frame extract_frame.output2 2 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py extract_frame.py.output2 21 ./extract_frame.py extract_frame.py.output2 2 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print %x % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- Comment By: Florent Rougon (frougon) Date: 2007-01-21 16:24 Message: Logged In: YES user_id=310088 Originator: YES I never wrote that it was the listing of /proc/pid/fd that was changing the behavior of transcode. Please don't put words in my mouth. I wrote that some fds are open soon after the transcode process is started, and quickly closed afterwards, when run from the Python test script. The rest of your answer again shows that you didn't read the bug report. I'll repeat a last time. The title of this bug report is Problem running a subprocess. It is
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Settings changed) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1634739group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Closed Resolution: Invalid Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Peter Åstrand (astrand) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py extract_frame.py.output 21 ./extract_frame.py extract_frame.py.output 2 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at (demuxer.c). Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame extract_frame.output2 21 ./extract_frame extract_frame.output2 2 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py extract_frame.py.output2 21 ./extract_frame.py extract_frame.py.output2 2 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print %x % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- Comment By: Florent Rougon (frougon) Date: 2007-01-21 16:24 Message: Logged In: YES user_id=310088 Originator: YES I never wrote that it was the listing of /proc/pid/fd that was changing the behavior of transcode. Please don't put words in my mouth. I wrote that some fds are open soon after the transcode process is started, and quickly closed afterwards, when run from the Python test script. The rest of your answer again shows that you didn't read the bug report. I'll repeat a last time. The title of this bug report is Problem running a subprocess. It is
[ python-Bugs-1546442 ] subprocess.Popen can't read file object as stdin after seek
Bugs item #1546442, was opened at 2006-08-25 07:52
Message generated for change (Comment added) made by astrand
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=105470aid=1546442group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: GaryD (gazzadee)
Assigned to: Peter Åstrand (astrand)
Summary: subprocess.Popen can't read file object as stdin after seek
Initial Comment:
When I use an existing file object as stdin for a call
to subprocess.Popen, then Popen cannot read the file if
I have called seek on it more than once.
eg. in the following python code:
import subprocess
rawfile = file('hello.txt', 'rb')
rawfile.readline()
'line 1\n'
rawfile.seek(0)
rawfile.readline()
'line 1\n'
rawfile.seek(0)
process_object = subprocess.Popen([cat],
stdin=rawfile, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
process_object.stdout now contains nothing, implying
that nothing was on process_object.stdin.
Note that this only applies for a non-trivial seek (ie.
where the file-pointer actually changes). Calling
seek(0) multiple times in a row does not change
anything (obviously).
I have not investigated whether this reveals a problem
with seek not changing the underlying file descriptor,
or a problem with Popen not handling the file
descriptor properly.
I have attached some complete python scripts that
demonstrate this problem. One shows cat working after
calling seek once, the other shows cat failing after
calling seek twice.
Python version being used:
Python 2.4.2 (#1, Nov 3 2005, 12:41:57)
[GCC 3.4.3-20050110 (Gentoo Linux 3.4.3.20050110,
ssp-3.4.3.20050110-0, pie-8.7 on linux2
--
Comment By: Peter Åstrand (astrand)
Date: 2007-01-21 20:43
Message:
Logged In: YES
user_id=344921
Originator: NO
It's not obvious that the subprocess module is doing anything wrong here.
Mixing streams and file descriptors is always problematic and should best
be avoided
(http://ftp.gnu.org/gnu/Manuals/glibc-2.2.3/html_node/libc_232.html).
However, the subprocess module *does* accept a file object (based on a
libc stream), for convenience. For things to work correctly, the
application and the subprocess module needs to cooperate. I admit that the
documentation needs improvement on this topic, though.
It's quite easy to demonstrate the problem, you don't need to use seek at
all. Here's a simple test case:
import subprocess
rawfile = file('hello.txt', 'rb')
rawfile.readline()
p = subprocess.Popen([cat], stdin=rawfile, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
print File contents from Popen() call to cat:
print p.stdout.read()
p.wait()
The descriptor offset is at the end, since the stream buffers.
http://ftp.gnu.org/gnu/Manuals/glibc-2.2.3/html_node/libc_233.html
describes the need for cleaning up a stream, when you switch from stream
functions to descriptor functions. This is described at
http://ftp.gnu.org/gnu/Manuals/glibc-2.2.3/html_node/libc_235.html#SEC244.
The documentation recommends the fclean() function, but it's only available
on GNU systems and not in Python. As I understand it, fflush() works good
for cleaning an output stream.
For input streams, however, things are difficult. fflush() might work
sometimes, but to be sure, you must set the file pointer as well. And,
this does not work for files that are not random access, since there's no
way of move the buffered data back to the operating system.
So, since subprocess cannot reliable deal with this situation, I believe
it shouldn't try. I think it makes more sense that the application
prepares the file object for low-level operations. There are many other
Python modules that uses the .fileno() method, for example the select()
module, and as far as I understand, this module doesn't try to clean
streams or anything like that.
To summarize: I'm leaning towards a documentation solution.
--
Comment By: lplatypus (ldeller)
Date: 2006-08-25 09:13
Message:
Logged In: YES
user_id=1534394
I found the cause of this bug:
A libc FILE* (used by python file objects) may hold a
different file offset than the underlying OS file
descriptor. The posix version of Popen._get_handles does
not take this into account, resulting in this bug.
The following patch against svn trunk fixes the problem. I
don't have permission to attach files to this item, so I'll
have to paste the patch here:
Index: subprocess.py
===
--- subprocess.py (revision 51581)
+++ subprocess.py (working copy)
@@ -907,6 +907,12 @@
else:
# Assuming file-like object
p2cread =
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 16:46 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1634739group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Closed Resolution: Invalid Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Peter Åstrand (astrand) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame extract_frame.output 21 ./extract_frame extract_frame.output 2 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py extract_frame.py.output 21 ./extract_frame.py extract_frame.py.output 2 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at (demuxer.c). Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame extract_frame.output2 21 ./extract_frame extract_frame.output2 2 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py extract_frame.py.output2 21 ./extract_frame.py extract_frame.py.output2 2 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print %x % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- Comment By: Peter Åstrand (astrand) Date: 2007-01-21 21:22 Message: Logged In: YES user_id=344921 Originator: NO I never wrote that it was the listing of /proc/pid/fd that was changing the behavior of transcode. Please don't put words in my mouth. I wrote that some fds are open soon after the transcode process is started, and quickly closed afterwards, when run from the Python test script. Sorry about that, I did misunderstand you. I wrote and attached minimal example programs that reproduce the bug. The problem is that although extract_frame.py is short
[ python-Bugs-1599254 ] mailbox: other programs' messages can vanish without trace
Bugs item #1599254, was opened at 2006-11-19 16:03 Message generated for change (Comment added) made by baikie You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1599254group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 9 Private: No Submitted By: David Watson (baikie) Assigned to: A.M. Kuchling (akuchling) Summary: mailbox: other programs' messages can vanish without trace Initial Comment: The mailbox classes based on _singlefileMailbox (mbox, MMDF, Babyl) implement the flush() method by writing the new mailbox contents into a temporary file which is then renamed over the original. Unfortunately, if another program tries to deliver messages while mailbox.py is working, and uses only fcntl() locking, it will have the old file open and be blocked waiting for the lock to become available. Once mailbox.py has replaced the old file and closed it, making the lock available, the other program will write its messages into the now-deleted old file, consigning them to oblivion. I've caused Postfix on Linux to lose mail this way (although I did have to turn off its use of dot-locking to do so). A possible fix is attached. Instead of new_file being renamed, its contents are copied back to the original file. If file.truncate() is available, the mailbox is then truncated to size. Otherwise, if truncation is required, it's truncated to zero length beforehand by reopening self._path with mode wb+. In the latter case, there's a check to see if the mailbox was replaced while we weren't looking, but there's still a race condition. Any alternative ideas? Incidentally, this fixes a problem whereby Postfix wouldn't deliver to the replacement file as it had the execute bit set. -- Comment By: David Watson (baikie) Date: 2007-01-21 22:10 Message: Logged In: YES user_id=1504904 Originator: YES Hold on, I have a plan. If _toc is only regenerated on locking, or at the end of a flush(), then the only way self._pending can be set at that time is if the application has made modifications before calling lock(). If we make that an exception-raising offence, then we can assume that self._toc is a faithful representation of the last known contents of the file. That means we can preserve the existing message keys on a reread without any of that _user_toc nonsense. Diff attached, to apply on top of mailbox-unified2. It's probably had even less review and testing than the previous version, but it appears to pass all the regression tests and doesn't change any existing semantics. File Added: mailbox-update-toc-new.diff -- Comment By: A.M. Kuchling (akuchling) Date: 2007-01-21 03:16 Message: Logged In: YES user_id=11375 Originator: NO sigh I'm starting to lose track of all the variations on the bug. Maybe we should just add more warnings to the documentation about locking the mailbox when modifying it and not try to fix this at all. -- Comment By: David Watson (baikie) Date: 2007-01-20 18:20 Message: Logged In: YES user_id=1504904 Originator: YES Hang on. If a message's key changes after recreating _toc, that does not mean that another process has modified the mailbox. If the application removes a message and then (inadvertently) causes _toc to be regenerated, the keys of all subsequent messages will be decremented by one, due only to the application's own actions. That's what happens in the broken locking test case: the program intends to remove message 0, flush, and then remove message 1, but because _toc is regenerated in between, message 1 is renumbered as 0, message 2 is renumbered as 1, and so the program deletes message 2 instead. To clear _toc in such code without attempting to preserve the message keys turns possible data loss (in the case that another process modified the mailbox) into certain data loss. That's what I'm concerned about. -- Comment By: A.M. Kuchling (akuchling) Date: 2007-01-19 15:24 Message: Logged In: YES user_id=11375 Originator: NO After reflection, I don't think the potential changing actually makes things any worse. _generate() always starts numbering keys with 1, so if a message's key changes because of lock()'s, re-reading, that means someone else has already modified the mailbox. Without the ToC clearing, you're already fated to have a corrupted mailbox because the new mailbox will be written using outdated file offsets. With the ToC clearing, you delete the wrong message. Neither outcome is good, but data is lost either way. The
[ python-Bugs-1641109 ] 2.3.6.4 Error in append and extend descriptions
Bugs item #1641109, was opened at 2007-01-21 23:34 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1641109group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: ilalopoulos (arafin) Assigned to: Nobody/Anonymous (nobody) Summary: 2.3.6.4 Error in append and extend descriptions Initial Comment: 2.3.6.4 Mutable Sequence Types (2.4.4 Python Doc) Error in the table describing append and extend operations for the list type. specificaly: s.append(x) same as s[len(s):len(s)] = [x] (2) s.extend(x) same as s[len(s):len(s)] = x (3) should be: s.append(x) same as s[len(s):len(s)] = x (2) s.extend(x) same as s[len(s):len(s)] = [x] (3) -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1641109group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1546442 ] subprocess.Popen can't read file object as stdin after seek
Bugs item #1546442, was opened at 2006-08-25 15:52
Message generated for change (Comment added) made by ldeller
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=105470aid=1546442group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: GaryD (gazzadee)
Assigned to: Peter Åstrand (astrand)
Summary: subprocess.Popen can't read file object as stdin after seek
Initial Comment:
When I use an existing file object as stdin for a call
to subprocess.Popen, then Popen cannot read the file if
I have called seek on it more than once.
eg. in the following python code:
import subprocess
rawfile = file('hello.txt', 'rb')
rawfile.readline()
'line 1\n'
rawfile.seek(0)
rawfile.readline()
'line 1\n'
rawfile.seek(0)
process_object = subprocess.Popen([cat],
stdin=rawfile, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
process_object.stdout now contains nothing, implying
that nothing was on process_object.stdin.
Note that this only applies for a non-trivial seek (ie.
where the file-pointer actually changes). Calling
seek(0) multiple times in a row does not change
anything (obviously).
I have not investigated whether this reveals a problem
with seek not changing the underlying file descriptor,
or a problem with Popen not handling the file
descriptor properly.
I have attached some complete python scripts that
demonstrate this problem. One shows cat working after
calling seek once, the other shows cat failing after
calling seek twice.
Python version being used:
Python 2.4.2 (#1, Nov 3 2005, 12:41:57)
[GCC 3.4.3-20050110 (Gentoo Linux 3.4.3.20050110,
ssp-3.4.3.20050110-0, pie-8.7 on linux2
--
Comment By: lplatypus (ldeller)
Date: 2007-01-22 12:23
Message:
Logged In: YES
user_id=1534394
Originator: NO
Fair enough, that's probably cleaner and more efficient than playing games
with fflush and lseek anyway. If file objects are not supported properly
then maybe they shouldn't be accepted at all, forcing the application to
call fileno() if that's what is wanted. That might break a lot of
existing code though. Then again it may be beneficial to get everyone to
review code which passes file objects to Popen in light of this behaviour.
--
Comment By: Peter Åstrand (astrand)
Date: 2007-01-22 06:43
Message:
Logged In: YES
user_id=344921
Originator: NO
It's not obvious that the subprocess module is doing anything wrong here.
Mixing streams and file descriptors is always problematic and should best
be avoided
(http://ftp.gnu.org/gnu/Manuals/glibc-2.2.3/html_node/libc_232.html).
However, the subprocess module *does* accept a file object (based on a
libc stream), for convenience. For things to work correctly, the
application and the subprocess module needs to cooperate. I admit that the
documentation needs improvement on this topic, though.
It's quite easy to demonstrate the problem, you don't need to use seek at
all. Here's a simple test case:
import subprocess
rawfile = file('hello.txt', 'rb')
rawfile.readline()
p = subprocess.Popen([cat], stdin=rawfile, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
print File contents from Popen() call to cat:
print p.stdout.read()
p.wait()
The descriptor offset is at the end, since the stream buffers.
http://ftp.gnu.org/gnu/Manuals/glibc-2.2.3/html_node/libc_233.html
describes the need for cleaning up a stream, when you switch from stream
functions to descriptor functions. This is described at
http://ftp.gnu.org/gnu/Manuals/glibc-2.2.3/html_node/libc_235.html#SEC244.
The documentation recommends the fclean() function, but it's only available
on GNU systems and not in Python. As I understand it, fflush() works good
for cleaning an output stream.
For input streams, however, things are difficult. fflush() might work
sometimes, but to be sure, you must set the file pointer as well. And,
this does not work for files that are not random access, since there's no
way of move the buffered data back to the operating system.
So, since subprocess cannot reliable deal with this situation, I believe
it shouldn't try. I think it makes more sense that the application
prepares the file object for low-level operations. There are many other
Python modules that uses the .fileno() method, for example the select()
module, and as far as I understand, this module doesn't try to clean
streams or anything like that.
To summarize: I'm leaning towards a documentation solution.
--
Comment By: lplatypus (ldeller)
Date: 2006-08-25 17:13
Message:
Logged In: YES
user_id=1534394
I found the cause of this bug:
A libc FILE* (used by
[ python-Bugs-654766 ] asyncore.py and quot;handle_exptquot;
Bugs item #654766, was opened at 2002-12-16 10:42 Message generated for change (Comment added) made by sf-robot You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=654766group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.2 Status: Closed Resolution: Out of Date Priority: 5 Private: No Submitted By: Jes�s Cea Avi�n (jcea) Assigned to: Josiah Carlson (josiahcarlson) Summary: asyncore.py and quot;handle_exptquot; Initial Comment: Python 2.2.2 here. Asyncore.py doesn't invoke quot;handle_exptquot; ever (quot;handle_exptquot; is documented in docs). Managing OOB data is imprescindible to handle quot;connection refusedquot; errors in Windows, for example. -- Comment By: SourceForge Robot (sf-robot) Date: 2007-01-21 19:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-06 22:18 Message: Logged In: YES user_id=341410 Originator: NO According to the most recent Python trunk, handle_expt() is called when an error is found within a .select() or .poll() call. Is this still an issue for you in Python 2.4 or Python 2.5? Setting status as Pending, Out of Date as I believe this bug is fixed. -- Comment By: Alexey Klimkin (klimkin) Date: 2004-03-04 00:24 Message: Logged In: YES user_id=410460 Patch #909005 fixes the problem. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=654766group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1579370 ] Segfault provoked by generators and exceptions
Bugs item #1579370, was opened at 2006-10-18 04:23 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detailatid=105470aid=1579370group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.5 Status: Open Resolution: None Priority: 9 Private: No Submitted By: Mike Klaas (mklaas) Assigned to: Nobody/Anonymous (nobody) Summary: Segfault provoked by generators and exceptions Initial Comment: A reproducible segfault when using heavily-nested generators and exceptions. Unfortunately, I haven't yet been able to provoke this behaviour with a standalone python2.5 script. There are, however, no third-party c extensions running in the process so I'm fairly confident that it is a problem in the core. The gist of the code is a series of nested generators which leave scope when an exception is raised. This exception is caught and re-raised in an outer loop. The old exception was holding on to the frame which was keeping the generators alive, and the sequence of generator destruction and new finalization caused the segfault. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-22 08:51 Message: Logged In: YES user_id=21627 Originator: NO I don't like mklaas' patch, since I think it is conceptually wrong to have PyTraceBack_Here() use the frame's thread state (mklaas describes it as dirty, and I agree). I'm proposing an alternative patch (tr.diff); please test this as well. File Added: tr.diff -- Comment By: Neal Norwitz (nnorwitz) Date: 2007-01-17 08:01 Message: Logged In: YES user_id=33168 Originator: NO Bumping priority to see if this should go into 2.5.1. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-04 11:42 Message: Logged In: YES user_id=21627 Originator: NO Why do frame objects have a thread state in the first place? In particular, why does PyTraceBack_Here get the thread state from the frame, instead of using the current thread? Introduction of f_tstate goes back to r7882, but it is not clear why it was done that way. -- Comment By: Andrew Waters (awaters) Date: 2007-01-04 10:35 Message: Logged In: YES user_id=1418249 Originator: NO This fixes the segfault problem that I was able to reliably reproduce on Linux. We need to get this applied (assuming it is the correct fix) to the source to make Python 2.5 usable for me in production code. -- Comment By: Mike Klaas (mklaas) Date: 2006-11-27 19:41 Message: Logged In: YES user_id=1611720 Originator: YES The following patch resets the thread state of the generator when it is resumed, which prevents the segfault for me: Index: Objects/genobject.c === --- Objects/genobject.c (revision 52849) +++ Objects/genobject.c (working copy) @@ -77,6 +77,7 @@ Py_XINCREF(tstate-frame); assert(f-f_back == NULL); f-f_back = tstate-frame; +f-f_tstate = tstate; gen-gi_running = 1; result = PyEval_EvalFrameEx(f, exc); -- Comment By: Eric Noyau (eric_noyau) Date: 2006-11-27 19:07 Message: Logged In: YES user_id=1388768 Originator: NO We are experiencing the same segfault in our application, reliably. Running our unit test suite just segfault everytime on both Linux and Mac OS X. Applying Martin's patch fixes the segfault, and makes everything fine and dandy, at the cost of some memory leaks if I understand properly. This particular bug prevents us to upgrade to python 2.5 in production. -- Comment By: Tim Peters (tim_one) Date: 2006-10-28 07:18 Message: Logged In: YES user_id=31435 I tried Tim's hope.py on Linux x86_64 and Mac OS X 10.4 with debug builds and neither one crashed. Tim's guess looks pretty damn good too. Neal, note that it's the /Windows/ malloc that fills freed memory with dangerous bytes in a debug build -- this really has nothing to do with that it's a debug build of /Python/ apart from that on Windows a debug build of Python also links in the debug version of Microsoft's malloc. The valgrind report is pointing at the same thing. Whether this leads to a crash is purely an accident of when and how the system malloc happens to reuse the freed memory. -- Comment By: Neal Norwitz (nnorwitz) Date: 2006-10-28 06:56 Message:
