How can I intentionally crash my lappy?
I am no programmer, but know the rudiments [the rudi'est of rudiments] of working with Python. I have a simple need, to have a simple script/app I can run that will crash my PC. On my desktops, I can always hit the reset, but that is not an option with my laptop. Can anyone tell me of an easy way to guarantee a program will instantly kill/BSOD my windows7/64bit laptop? Any help is much appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Dynamic list name from a string
Hi, I need to create a list with a dynamic name, something like this: '%s_list' %my_dynamic list = [] It's this possible? -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I intentionally crash my lappy?
On Mon, 20 Dec 2010 02:45:18 -0600, mechtheist wrote: I am no programmer, but know the rudiments [the rudi'est of rudiments] of working with Python. I have a simple need, to have a simple script/app I can run that will crash my PC. On my desktops, I can always hit the reset, but that is not an option with my laptop. Can anyone tell me of an easy way to guarantee a program will instantly kill/BSOD my windows7/64bit laptop? Any help is much appreciated. You realise that the operating system (yes, even Windows!) is designed to *not* crash no matter what the application does? (The OS may live up to that requirement imperfectly, but still, it tries.) You'll probably find that task easier from C than from Python. But if I were to attempt such a strange thing, I'd try to shut down some critical processes used by the OS. I'm not much of a Windows person, but if you call up the task manager (ctrl-alt-del) and inspect the process list, then find one that causes the machine to crash if you shut it down, you can probably do the same thing from a script. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic list name from a string
On Mon, 20 Dec 2010 02:08:57 -0800, MarcoS wrote: Hi, I need to create a list with a dynamic name, something like this: '%s_list' %my_dynamic list = [] It's this possible? I'm pretty sure you don't *need* to, you just think you do. It is highly unlikely that there is anything you can do with such a variable-variable- name that you can't do by a more sensible technique. But since you ask: name = 'variable' variable Traceback (most recent call last): File stdin, line 1, in module NameError: name 'variable' is not defined globals()[name] = this is a bad idea, don't do it variable this is a bad idea, don't do it Here's another way: del variable exec(%s = 'this is an even worse idea, be careful with this' % name) variable 'this is an even worse idea, be careful with this' Seriously, you probably shouldn't do this. The first is harmful because it leads to confusing, complicated, unclear code that is hard to maintain; the second is harmful for the same reasons, *plus* it is slower, AND could lead to serious security bugs. Google on code injection attack for more information. (There are uses for these techniques, or at least similar techniques, but they are rare, fairly advanced, and quite specialised.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I intentionally crash my lappy?
http://pcsupport.about.com/od/tipstricks/ht/makebsodxp.htm Am Mo, 20.12.2010, 09:45 schrieb mechtheist: I am no programmer, but know the rudiments [the rudi'est of rudiments] of working with Python. I have a simple need, to have a simple script/app I can run that will crash my PC. On my desktops, I can always hit the reset, but that is not an option with my laptop. Can anyone tell me of an easy way to guarantee a program will instantly kill/BSOD my windows7/64bit laptop? Any help is much appreciated. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic list name from a string
Hello,
On 12/20/2010 11:08 AM, MarcoS wrote:
Hi, I need to create a list with a dynamic name, something like this:
'%s_list' %my_dynamic list = []
It's this possible?
I would suggest you use a dictionary to store your lists like this:
lists = {}
lists[my_dynamic_list] = []
Maybe you tell us some more what you want to achieve...
Regards
Andre
--
http://mail.python.org/mailman/listinfo/python-list
encoding attribute codecs.getwriter-produced streams
Try this code:
# foo.py
import sys, codecs
stream = codecs.getwriter('utf-8')(sys.stdout)
print stream.encoding
$ python foo.py | cat
None
I expected the `encoding' attribute to be UTF-8, since the stream
otherwise correctly functions as a utf-8 encoding stream.
Is this a bug in the stream factory returned by codecs.getwriter(...)?
If not, is there another way to determine a stream's output encoding
that would work for both default and codecs-created streams?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic list name from a string
Ok, thanks Steven and Andre for yours reply. This is my situation: I've a list of objects of differents classes, with a common attribute name Something like this: class Object1: obj1_name other fields date class Object2: obj2_name other fields date ... class ObjectN: objN_name other fields date the lists are created dynamically at runtime, depending or not if there one or more correspondig Objects, ex. list_object1 = [object1_1, object1_2 ... object1_n] ... list_objectN = [object2_1, object2_2 ... object2_n] Then i need to include all into one global list sorting the various objects by date ex global_list = [object1_1, object2_n, object1_2 ... etc] I think that Andre solution it's the best way to solve my dynamics lists problem, now I'll verify how to create the global list and sort it -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I intentionally crash my lappy?
On 12/20/2010 4:22 AM, Steven D'Aprano wrote: On Mon, 20 Dec 2010 02:45:18 -0600, mechtheist wrote: I am no programmer, but know the rudiments [the rudi'est of rudiments] of working with Python. I have a simple need, to have a simple script/app I can run that will crash my PC. On my desktops, I can always hit the reset, but that is not an option with my laptop. Can anyone tell me of an easy way to guarantee a program will instantly kill/BSOD my windows7/64bit laptop? Any help is much appreciated. You realise that the operating system (yes, even Windows!) is designed to *not* crash no matter what the application does? (The OS may live up to that requirement imperfectly, but still, it tries.) You'll probably find that task easier from C than from Python. But if I were to attempt such a strange thing, I'd try to shut down some critical processes used by the OS. I'm not much of a Windows person, but if you call up the task manager (ctrl-alt-del) and inspect the process list, then find one that causes the machine to crash if you shut it down, you can probably do the same thing from a script. Thank you, good approach ideas there. Windows has gotten a lot better about crashing, but it is definitely still way the f 'imperfect'. Now I just have to figure a way to make it imperfecter. -- http://mail.python.org/mailman/listinfo/python-list
Re: True lists in python?
Vito 'ZeD' De Tullio zak.mc.kra...@libero.it wrote: Steven D'Aprano wrote: I can't see any way to go from this linked list: node1 - node2 - node3 - node4 - node5 - node6 - node7 to this: node1 - node6 - node5 - node4 - node3 - node2 - node7 in constant time. You have to touch each of the nodes being reversed. very crude example: crude example snipped No, you just showed how you can reverse an entire list in constant time. The original question and Steven's example were asking to reverse just part of the list. I guess you might be able to do it with a double-linked list provided that when traversing the list you always keep two nodes around to determine the direction. e.g. instead of asking for node6.nextNode() you ask for node6.nextNode(previous=node1) and then the code can return whichever sibling wasn't given. That would make reversal (assuming you have both nodes) O(1), but would make traversing the list slower. Also, I don't think it would help if you wanted to implement John Nagle's algorithm for the travelling salesman problem: you could hold a separate (array based) list of nodes in order to make the random selection O(1), but you wouldn't know which direction to follow from each node when it comes to cutting the list into 3. -- Duncan Booth http://kupuguy.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I intentionally crash my lappy?
On 20/12/2010 10:45, mechtheist wrote: I am no programmer, but know the rudiments [the rudi'est of rudiments] of working with Python. I have a simple need, to have a simple script/app I can run that will crash my PC. On my desktops, I can always hit the reset, but that is not an option with my laptop. Can anyone tell me of an easy way to guarantee a program will instantly kill/BSOD my windows7/64bit laptop? You could try a forkbomb and execute that via a system call. Not sure why anyone would want to do such a thing, but there you go... -- -- Paul http://www.paulscott.za.net http://twitter.com/paulscott56 http://avoir.uwc.ac.za All Email originating from UWC is covered by disclaimer http://www.uwc.ac.za/portal -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I intentionally crash my lappy?
Thanks, that is exactly what I was looking for. Unfortunately, it is only for /non/-USB keyboards, I may be able to figure something out from there with hotkey or something. On 12/20/2010 3:31 AM, Stefan Sonnenberg-Carstens wrote: http://pcsupport.about.com/od/tipstricks/ht/makebsodxp.htm Am Mo, 20.12.2010, 09:45 schrieb mechtheist: I am no programmer, but know the rudiments [the rudi'est of rudiments] of working with Python. I have a simple need, to have a simple script/app I can run that will crash my PC. On my desktops, I can always hit the reset, but that is not an option with my laptop. Can anyone tell me of an easy way to guarantee a program will instantly kill/BSOD my windows7/64bit laptop? Any help is much appreciated. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I intentionally crash my lappy?
Am 20.12.2010 09:45, schrieb mechtheist: I am no programmer, but know the rudiments [the rudi'est of rudiments] of working with Python. I have a simple need, to have a simple script/app I can run that will crash my PC. On my desktops, I can always hit the reset, but that is not an option with my laptop. Can anyone tell me of an easy way to guarantee a program will instantly kill/BSOD my windows7/64bit laptop? Do you want to crash it or are you looking for a way to restart Windows? Every modern operating system tries very hard (and successful) to prevent users from messing with the system. You have two choices. Either you ask the OS politely to restart (e.g. shutdown -r -f -t 1) or you need some sort of STONITH device (e.g. IPMI card, watchdog, other node fencing device, multi-plug with network interface). Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating custom types from C code
Thanks for the reply.
I remember reading about named tuples when they were back-ported to
the 2.X series but I never played with them.
Is there a way to instantiate a named tuple from C code?
Maybe I'm over-thinking this whole thing.
Is there a simple way that I can define a class in Python and
instantiate that type from C?
On Sat, Dec 18, 2010 at 1:18 AM, Stefan Behnel stefan...@behnel.de wrote:
Eric Frederich, 17.12.2010 23:58:
I have an extension module for a 3rd party library in which I am
wrapping some structures.
My initial attempt worked okay on Windows but failed on Linux.
I was doing it in two parts.
The first part on the C side of things I was turning the entire
structure into a char array.
The second part in Python code I would unpack the structure.
Anyway, I decided I should be doing things a little cleaner so I read
up on Defining New Types
http://docs.python.org/extending/newtypes.html
I got it to work but I'm not sure how to create these new objects from C.
You may want to take a look at Cython. It makes writing C extensions easy.
For one, it will do all sorts of type conversions for you, and do them
efficiently and safely (you get an exception on overflow, for example). It's
basically Python, so creating classes and instantiating them is trivial.
Also note that it's generally not too much work to rewrite an existing C
wrapper in Cython, but it's almost always worth it. You immediately get more
maintainable code that's much easier to extend and work on. It's also often
faster than hand written code.
http://cython.org
My setup is almost exactly like the example on that page except
instead of 2 strings and an integer I have 5 unsigned ints.
I do not expect to ever be creating these objects in Python. They
will only be created as return values from my wrapper functions to the
3rd party library.
In Cython 0.14, you can declare classes as final and internal using a
decorator, meaning that they cannot be subtyped from Python and do not show
up in the module dict. However, note that there is no way to prevent users
from getting their hands at the type once you give them an instance.
I could return a tuple from those functions but I want to use dot
notation (e.g. somestruct.var1).
Then __getattr__ or properties are your friend.
So, question number 1:
Is defining my own type like that overkill just to have an object
to access using dots?
Creating wrapper objects is totally normal.
Also note that recent Python versions have named tuples, BTW.
I'll never create those objects from Python.
Is there a shortcut to creating objects and setting attributes
from within C?
The Cython code for instantiating classes is identical to Python.
In any case, I was able to create my own custom object from C code like
so...
PyObject *foo(SomeCStruct bar){
PyObject *ret;
ret = _PyObject_New(mymodule_SomeStructType);
PyObject_SetAttrString(ret, var1 , Py_BuildValue(I, bar.var1
));
PyObject_SetAttrString(ret, var2 , Py_BuildValue(I, bar.var2
));
PyObject_SetAttrString(ret, var3 , Py_BuildValue(I, bar.var3
));
PyObject_SetAttrString(ret, var4 , Py_BuildValue(I, bar.var4
));
PyObject_SetAttrString(ret, var5 , Py_BuildValue(I, bar.var5
));
return ret;
}
When using _PyObject_New I notice that neither my new or init function
are ever called.
I verified that they are getting called when creating the object from
Python
Things often work a little different in Python and C. Directly calling
_PyObject_New() is a lot less than what Python does internally. The
canonical way is to PyObject_Call() the type (or to use one of the other
call functions, depending on what your arguments are).
(which I would never do anyway).
Your users could do it, though, so you should make sure that won't crash the
interpreter that way by leaving internal data fields uninitialised.
Question number 2:
Do I need to be calling PyObject_SetAttrString or is there a way
to set the unsigned ints on the structure direcly?
It seems overkill to create a Python object for an unsigned int
just to set it as an attribute on a custom defined type.
You will have to do it at some point, though, either at instantiation time
or at Python access time. Depending on the expected usage, either of the two
can be more wasteful.
Question number 3:
In the above code, is there a memory leak? Should I be
Py_DECREF'ing the return value from Py_BuildValue after I'm done using
it.
You can look that up in the C-API docs. If a function doesn't say that it
steals a reference, you still own the reference when it returns and have
to manually decref it (again, a thing that you won't usually have to care
about in Cython). So, yes, the above leaks one reference for each call to
Py_BuildValue().
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
--
Re: Creating custom types from C code
Eric Frederich, 20.12.2010 16:23: I remember reading about named tuples when they were back-ported to the 2.X series but I never played with them. Is there a way to instantiate a named tuple from C code? There isn't a dedicated C-API for named tuples, but you can instantiate any Python type from C code. Maybe I'm over-thinking this whole thing. Is there a simple way that I can define a class in Python and instantiate that type from C? Sure, see my previous comments to your original mail. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python-list Digest, Vol 87, Issue 122
ZS On Dec 19, 2010 6:05 AM, python-list-requ...@python.org wrote: Send Python-list mailing list submissions to python-list@python.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-list or, via email, send a message with subject or body 'help' to python-list-requ...@python.org You can reach the person managing the list at python-list-ow...@python.org When replying, please edit your Subject line so it is more specific than Re: Contents of Python-list digest... -- http://mail.python.org/mailman/listinfo/python-list
Re: Python-list Digest, Vol 87, Issue 122
as On Dec 19, 2010 6:05 AM, python-list-requ...@python.org wrote: Send Python-list mailing list submissions to python-list@python.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-list or, via email, send a message with subject or body 'help' to python-list-requ...@python.org You can reach the person managing the list at python-list-ow...@python.org When replying, please edit your Subject line so it is more specific than Re: Contents of Python-list digest... -- http://mail.python.org/mailman/listinfo/python-list
Re: Catching user switching and getting current active user from root on linux
I give up, I will never try to use a usenet group again. For the ones of you who tried to help thank you. You helped to identify some of my troubles, as for you @usernet, you are a troll -- http://mail.python.org/mailman/listinfo/python-list
Re: how to measure TCP congestion windows using python ??
FWIW, on CentOS 4.7, the ctypes version works fine, but the struct
version fails, because len(tcp_info) is only 100 bytes while
struct.calcsize('B...L') is 104.
However, if the format is changed to '7B23L', i.e. one 'L' shorter,
the struct version works and returns to same result as the ctypes
version.
/Jean
On Dec 19, 4:45 pm, plz u...@compgroups.net/ wrote:
hi
many thanks for helping me
i also tried to manipulate it last night
here is my code...
import socket
import struct
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
tcp_info = sock.getsockopt(socket.SOL_TCP, socket.TCP_INFO,
struct.calcsize('BBB'))
print struct.unpack('BBB', tcp_info)
the result of struct.unpack of tcp_info is following
in /usr/include/linux/tcp.h
used Linux Redhat and Python 2.7
anyway your code is very useful
Thxs again ;)
--
http://mail.python.org/mailman/listinfo/python-list
Questions
Folks, 1. As a free lance developer I need to use Python with Web Content Services using IIS7 (Windows 7). However, I don't have a lot of time to learn IIS7. Can someone tell me how to configure IIS7 to be used by Python CGI scripts? 2. I noticed that the latest distribution from ActiveState has Jason with it; but the documentation does not go into Ajax or the use of Jason with Python. Any suggestions on where I can find info? 3. I have used Tk in the past with Tcl and IncrTcl. Where can I find a lot of Python/Tk examples so that I can save some time in developing GUIs? Thanks, Lew -- http://mail.python.org/mailman/listinfo/python-list
**** httplib.InvalidURL: nonnumeric port ****
All,
When i try to open a URL of the form
http://joule:8041/DteEnLinea/ws/EnvioGuia.jws, I am getting an error as
below complaining for some non-numeric port.
wm (wmosds) [zibal] - /u01/home/apli/wm/app/gdd :python
Python 2.4.1 (#2, May 30 2005, 09:40:30) [C] on aix5
Type help, copyright, credits or license for more information.
import httplib, mimetypes
h = httplib.HTTP('http://joule:8041/DteEnLinea/ws/EnvioGuia.jws')
Traceback (most recent call last):
File stdin, line 1, in ?
File /u01/home/apli/wm/python241/lib/python2.4/httplib.py, line 1093, in
__init__
self._setup(self._connection_class(host, port, strict))
File /u01/home/apli/wm/python241/lib/python2.4/httplib.py, line 582, in
__init__
self._set_hostport(host, port)
File /u01/home/apli/wm/python241/lib/python2.4/httplib.py, line 594, in
_set_hostport
raise InvalidURL(nonnumeric port: '%s' % host[i+1:])
httplib.InvalidURL: nonnumeric port: '8041/DteEnLinea/ws/EnvioGuia.jws'
How could we avoid this problem?
Regards,
Anurag
--
http://mail.python.org/mailman/listinfo/python-list
Sending XML to a WEB Service and Getting Response Back
Dear Python Mates, I have a requirement to send a XML Data to a WEB Service whose URL is of the form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws I also have to read back the response returned as a result of sending this data to this WebService. This web service implements the following operations: sendNCR This web service has no callbacks. I have pasted the complete WSDL for this WEB Service below my email. I would appreciate if someone could guide me with sample code using a Python Library suitable to fulfill this requirement of mine. Regards, Anurag ?xml version=1.0 encoding=utf-8? wsdl:definitions xmlns:conv= http://www.openuri.org/2002/04/soap/conversation/; xmlns:cw= http://www.openuri.org/2002/04/wsdl/conversation/; xmlns:http= http://schemas.xmlsoap.org/wsdl/http/; xmlns:jms= http://www.openuri.org/2002/04/wsdl/jms/; xmlns:mime= http://schemas.xmlsoap.org/wsdl/mime/; xmlns:s= http://www.w3.org/2001/XMLSchema; xmlns:s0=http://www.openuri.org/; xmlns:soap=http://schemas.xmlsoap.org/wsdl/soap/; xmlns:soapenc= http://schemas.xmlsoap.org/soap/encoding/; xmlns:wsdl= http://schemas.xmlsoap.org/wsdl/; targetNamespace=http://www.openuri.org/; wsdl:types s:schema xmlns:s=http://www.w3.org/2001/XMLSchema; elementFormDefault=qualified targetNamespace=http://www.openuri.org/; s:element name=sendNCR s:complexType s:sequence s:element name=xml type=s:string minOccurs=0/ /s:sequence /s:complexType /s:element s:element name=sendNCRResponse s:complexType s:sequence s:element name=sendNCRResult type=s:int/ /s:sequence /s:complexType /s:element s:element name=int type=s:int/ /s:schema /wsdl:types wsdl:message name=sendNCRSoapIn wsdl:part name=parameters element=s0:sendNCR/ /wsdl:message wsdl:message name=sendNCRSoapOut wsdl:part name=parameters element=s0:sendNCRResponse/ /wsdl:message wsdl:message name=sendNCRHttpGetIn wsdl:part name=xml type=s:string/ /wsdl:message wsdl:message name=sendNCRHttpGetOut wsdl:part name=Body element=s0:int/ /wsdl:message wsdl:message name=sendNCRHttpPostIn wsdl:part name=xml type=s:string/ /wsdl:message wsdl:message name=sendNCRHttpPostOut wsdl:part name=Body element=s0:int/ /wsdl:message wsdl:portType name=EnvioGuiaSoap wsdl:operation name=sendNCR wsdl:input message=s0:sendNCRSoapIn/ wsdl:output message=s0:sendNCRSoapOut/ /wsdl:operation /wsdl:portType wsdl:portType name=EnvioGuiaHttpGet wsdl:operation name=sendNCR wsdl:input message=s0:sendNCRHttpGetIn/ wsdl:output message=s0:sendNCRHttpGetOut/ /wsdl:operation /wsdl:portType wsdl:portType name=EnvioGuiaHttpPost wsdl:operation name=sendNCR wsdl:input message=s0:sendNCRHttpPostIn/ wsdl:output message=s0:sendNCRHttpPostOut/ /wsdl:operation /wsdl:portType wsdl:binding name=EnvioGuiaSoap type=s0:EnvioGuiaSoap soap:binding transport=http://schemas.xmlsoap.org/soap/http; style=document/ wsdl:operation name=sendNCR soap:operation soapAction=http://www.openuri.org/sendNCR; style=document/ wsdl:input soap:body use=literal/ /wsdl:input wsdl:output soap:body use=literal/ /wsdl:output /wsdl:operation /wsdl:binding wsdl:binding name=EnvioGuiaHttpGet type=s0:EnvioGuiaHttpGet http:binding verb=GET/ wsdl:operation name=sendNCR http:operation location=/sendNCR/ wsdl:input http:urlEncoded/ /wsdl:input wsdl:output mime:mimeXml part=Body/ /wsdl:output /wsdl:operation /wsdl:binding wsdl:binding name=EnvioGuiaHttpPost type=s0:EnvioGuiaHttpPost http:binding verb=POST/ wsdl:operation name=sendNCR http:operation location=/sendNCR/ wsdl:input mime:content type=application/x-www-form-urlencoded/ /wsdl:input wsdl:output mime:mimeXml part=Body/ /wsdl:output /wsdl:operation /wsdl:binding wsdl:service name=EnvioGuia wsdl:port name=EnvioGuiaSoap binding=s0:EnvioGuiaSoap soap:address location=http://joule:8041/DteEnLinea/ws/EnvioGuia.jws / /wsdl:port wsdl:port name=EnvioGuiaHttpGet binding=s0:EnvioGuiaHttpGet http:address location=http://joule:8041/DteEnLinea/ws/EnvioGuia.jws / /wsdl:port wsdl:port name=EnvioGuiaHttpPost binding=s0:EnvioGuiaHttpPost http:address location=http://joule:8041/DteEnLinea/ws/EnvioGuia.jws / /wsdl:port /wsdl:service /wsdl:definitions -- http://mail.python.org/mailman/listinfo/python-list
Re: **** httplib.InvalidURL: nonnumeric port ****
On Tue, 21 Dec 2010 00:01:44 +0530, Anurag Chourasia wrote: import httplib help(httplib.HTTP) Help on class HTTP in module httplib: class HTTP | Compatibility class with httplib.py from 1.5. | | Methods defined here: | | __init__(self, host='', port=None, strict=None) The constructor doesn't take a complete URL as an argument. Also, shouldn't you be using httplib.HTTPConnection? The docs state that httplib.HTTP is for backward compatibility only. Kev -- http://mail.python.org/mailman/listinfo/python-list
help with link parsing?
Hello all, I have a question. I guess this worked pre 2.6; I don't remember the last time I used it, but it was a while ago, and now it's failing. Anyone mind looking at it and telling me what's going wrong? Also, is there a quick way to match on a certain site? like links from google.com and only output those? #!/usr/bin/env python #This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published #by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. #This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of #MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. # #You should have received a copy of the GNU General Public License along with this program. If not, see #http://www.gnu.org/licenses/. This script will parse out all the links in an html document and write them to a textfile. import sys,optparse import htmllib,formatter #program class declarations: class Links(htmllib.HTMLParser): def __init__(self,formatter): htmllib.HTMLParser.__init__(self, formatter) self.links=[] def start_a(self, attrs): if (len(attrs)0): for a in attrs: if a[0]==href: self.links.append(a[1]) print a[1] break def main(argv): if (len(argv)!=3): print(Error:\n+argv[0]+ input output.\nParses input for all links and saves them to output.) return 1 lcount=0 format=formatter.NullFormatter() html=Links(format) print Retrieving data: page=open(argv[1],r) print Feeding data to parser: html.feed(page.read()) page.close() print Writing links: output=open(argv[2],w) for i in (html.links): output.write(i+\n) lcount+=1 output.close() print(Wrote +str(lcount)+ links to +argv[2]+.); print(done.) if (__name__ == __main__): #we call the main function passing a list of args, and exit with the return code passed back. sys.exit(main(sys.argv)) -- Thanks, Ty -- http://mail.python.org/mailman/listinfo/python-list
Re: **** httplib.InvalidURL: nonnumeric port ****
Hi Kevin,
Thanks for the response.
I tried with HTTPConnection but the same problem.
h1 = httplib.HTTPConnection('
http://joule:8041/DteEnLinea/ws/EnvioGuia.jws')
Traceback (most recent call last):
File stdin, line 1, in ?
File /u01/home/apli/wm/python241/lib/python2.4/httplib.py, line 582, in
__init__
self._set_hostport(host, port)
File /u01/home/apli/wm/python241/lib/python2.4/httplib.py, line 594, in
_set_hostport
raise InvalidURL(nonnumeric port: '%s' % host[i+1:])
httplib.InvalidURL: nonnumeric port: '8041/DteEnLinea/ws/EnvioGuia.jws'
Do i need to use some other library to be able to send XML Data (and get a
response back) to this Kind of Web Service address i.e.
http://joule:8041/DteEnLinea/ws/EnvioGuia.jws ?
Regards,
Anurag
On Tue, Dec 21, 2010 at 12:42 AM, Kev Dwyer kevin.p.dw...@gmail.com wrote:
On Tue, 21 Dec 2010 00:01:44 +0530, Anurag Chourasia wrote:
import httplib
help(httplib.HTTP)
Help on class HTTP in module httplib:
class HTTP
| Compatibility class with httplib.py from 1.5.
|
| Methods defined here:
|
| __init__(self, host='', port=None, strict=None)
The constructor doesn't take a complete URL as an argument.
Also, shouldn't you be using httplib.HTTPConnection? The docs
state that httplib.HTTP is for backward compatibility only.
Kev
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Trying to parse a HUGE(1gb) xml file
Hi c.l.p folks
This is a rather long post, but i wanted to include all the details
everything i have tried so far myself, so please bear with me read the entire
boringly long post.
I am trying to parse a ginormous ( ~ 1gb) xml file.
0. I am a python xml n00b, s have been relying on the excellent beginner
book DIP(Dive_Into_Python3 by MP(Mark Pilgrim) Mark , if u are readng this,
you are AWESOME so is your witty humorous writing style)
1. Almost all exmaples pf parsing xml in python, i have seen, start off with
these 4 lines of code.
import xml.etree.ElementTree as etree
tree = etree.parse('*path_to_ginormous_xml*')
root = tree.getroot() #my huge xml has 1 root at the top level
print root
2. In the 2nd line of code above, as Mark explains in DIP, the parse function
builds returns a tree object, in-memory(RAM), which represents the entire
document.
I tried this code, which works fine for a small ( ~ 1MB), but when i run this
simple 4 line py code in a terminal for my HUGE target file (1GB), nothing
happens.
In a separate terminal, i run the top command, i can see a python process,
with memory (the VIRT column) increasing from 100MB , all the way upto 2100MB.
I am guessing, as this happens (over the course of 20-30 mins), the tree
representing is being slowly built in memory, but even after 30-40 mins,
nothing happens.
I dont get an error, seg fault or out_of_memory exception.
My hardware setup : I have a win7 pro box with 8gb of RAM intel core2 quad
cpuq9400.
On this i am running sun virtualbox(3.2.12), with ubuntu 10.10 as guest os,
with 23gb disk space 2gb(2048mb) ram, assigned to the guest ubuntu os.
3. I also tried using lxml, but an lxml tree is much more expensive, as it
retains more info about a node's context, including references to it's parent.
[http://www.ibm.com/developerworks/xml/library/x-hiperfparse/]
When i ran the same 4line code above, but with lxml's elementree ( using the
import below in line1of the code above)
import lxml.etree as lxml_etree
i can see the memory consumption of the python process(which is running the
code) shoot upto ~ 2700mb then, python(or the os ?) kills the process as it
nears the total system memory(2gb)
I ran the code from 1 terminal window (screenshot :http://imgur.com/ozLkB.png)
ran top from another terminal (http://imgur.com/HAoHA.png)
4. I then investigated some streaming libraries, but am confused - there is
SAX[http://en.wikipedia.org/wiki/Simple_API_for_XML] , the iterparse
interface[http://effbot.org/zone/element-iterparse.htm]
Which one is the best for my situation ?
Any all code_snippets/wisdom/thoughts/ideas/suggestions/feedback/comments/ of
the c.l.p community would be greatly appreciated.
Plz feel free to email me directly too.
thanks a ton
cheers
ashish
email :
ashish.makani
domain:gmail.com
p.s.
Other useful links on xml parsing in python
0. http://diveintopython3.org/xml.html
1.
http://stackoverflow.com/questions/1513592/python-is-there-an-xml-parser-implemented-as-a-generator
2. http://codespeak.net/lxml/tutorial.html
3.
https://groups.google.com/forum/?hl=enlnk=gstq=parsing+a+huge+xml#!topic/comp.lang.python/CMgToEnjZBk
4. http://www.ibm.com/developerworks/xml/library/x-hiperfparse/
5.http://effbot.org/zone/element-index.htm
http://effbot.org/zone/element-iterparse.htm
6. SAX : http://en.wikipedia.org/wiki/Simple_API_for_XML
--
http://mail.python.org/mailman/listinfo/python-list
Re: Questions
On 12/20/2010 12:10 PM, lewi...@verizon.net wrote: 3. I have used Tk in the past with Tcl and IncrTcl. Where can I find a lot of Python/Tk examples so that I can save some time in developing GUIs? Do a search, or possibly a google codesearch, on tkinter or Tkinter, which is Python's tk inter(face). In fact, search the archive of python-list (or gmane's mirror) for posted examples. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: **** httplib.InvalidURL: nonnumeric port ****
On Tue, Dec 21, 2010 at 12:42 AM, Kev Dwyer kevin.p.dw...@gmail.com wrote:
On Tue, 21 Dec 2010 00:01:44 +0530, Anurag Chourasia wrote:
import httplib
help(httplib.HTTP)
Help on class HTTP in module httplib:
class HTTP
| Compatibility class with httplib.py from 1.5.
|
| Methods defined here:
|
| __init__(self, host='', port=None, strict=None)
The constructor doesn't take a complete URL as an argument.
Also, shouldn't you be using httplib.HTTPConnection? The docs
state that httplib.HTTP is for backward compatibility only.
On Mon, Dec 20, 2010 at 11:30 AM, Anurag Chourasia
anurag.choura...@gmail.com wrote:
Hi Kevin,
Thanks for the response.
I tried with HTTPConnection but the same problem.
h1 =
httplib.HTTPConnection('http://joule:8041/DteEnLinea/ws/EnvioGuia.jws')
Traceback (most recent call last):
File stdin, line 1, in ?
File /u01/home/apli/wm/python241/lib/python2.4/httplib.py, line 582, in
__init__
self._set_hostport(host, port)
File /u01/home/apli/wm/python241/lib/python2.4/httplib.py, line 594, in
_set_hostport
raise InvalidURL(nonnumeric port: '%s' % host[i+1:])
httplib.InvalidURL: nonnumeric port: '8041/DteEnLinea/ws/EnvioGuia.jws'
Do i need to use some other library to be able to send XML Data (and get a
response back) to this Kind of Web Service address
i.e. http://joule:8041/DteEnLinea/ws/EnvioGuia.jws ?
Regards,
Anurag
(1) Please avoid top-posting
(2) Did you read the other part of Kev's comment? The constructor
doesn't take a complete URL as an argument. You must pass only the
domain to the constructor and then call .request() on the
HTTPConnection object.
(3) If you're using SOAP, try https://fedorahosted.org/suds/
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: help with link parsing?
On Mon, Dec 20, 2010 at 11:14 AM, Littlefield, Tyler ty...@tysdomain.com wrote: Hello all, I have a question. I guess this worked pre 2.6; I don't remember the last time I used it, but it was a while ago, and now it's failing. Anyone mind looking at it and telling me what's going wrong? Please describe /exactly/ how it is failing for you, including the full exception traceback (if any). The script seems to work fine for me under both Python v2.6.6 and v2.7.1. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: **** httplib.InvalidURL: nonnumeric port ****
On Tue, 21 Dec 2010 01:00:36 +0530, Anurag Chourasia wrote:
Anurag,
HTTPConnection takes a host and a port number as arguments, not just a URL.
So you could construct your connection request like this:
conn = httplib.HTTPConnection('joule', 8041)
then use the request() method on the connection to make a PUT or GET request:
conn.request('PUT', url, body, headers)
Please read the documentation for the httplib module, and perhaps some basic
material on how HTTP requests work.
Cheers,
Kev
--
http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
[Wrapped to meet RFC1855 Netiquette Guidelines] On 2010-12-20, spaceman-spiff ashish.mak...@gmail.com wrote: This is a rather long post, but i wanted to include all the details everything i have tried so far myself, so please bear with me read the entire boringly long post. I am trying to parse a ginormous ( ~ 1gb) xml file. [SNIP] 4. I then investigated some streaming libraries, but am confused - there is SAX[http://en.wikipedia.org/wiki/Simple_API_for_XML] , the iterparse interface[http://effbot.org/zone/element-iterparse.htm] I have made extensive use of SAX and it will certainly work for low memory parsing of XML. I have never used iterparse; so, I cannot make an informed comparison between them. Which one is the best for my situation ? Your posed was long but it failed to tell us the most important piece of information: What does your data look like and what are you trying to do with it? SAX is a low level API that provides a callback interface allowing you to processes various elements as they are encountered. You can therefore do anything you want to the information, as you encounter it, including outputing and discarding small chunks as you processes it; ignoring most of it and saving only what you want to memory data structures; or saving all of it to a more random access database or on disk data structure that you can load and process as required. What you need to do will depend on what you are actually trying to accomplish. Without knowing that, I can only affirm that SAX will work for your needs without providing any information about how you should be using it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Sending XML to a WEB Service and Getting Response Back
I recommend you use the urllib.request in the library of python says everything that you want to know. 2010/12/20, Anurag Chourasia anurag.choura...@gmail.com: Dear Python Mates, I have a requirement to send a XML Data to a WEB Service whose URL is of the form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws I also have to read back the response returned as a result of sending this data to this WebService. This web service implements the following operations: sendNCR This web service has no callbacks. I have pasted the complete WSDL for this WEB Service below my email. I would appreciate if someone could guide me with sample code using a Python Library suitable to fulfill this requirement of mine. Regards, Anurag ?xml version=1.0 encoding=utf-8? wsdl:definitions xmlns:conv= http://www.openuri.org/2002/04/soap/conversation/; xmlns:cw= http://www.openuri.org/2002/04/wsdl/conversation/; xmlns:http= http://schemas.xmlsoap.org/wsdl/http/; xmlns:jms= http://www.openuri.org/2002/04/wsdl/jms/; xmlns:mime= http://schemas.xmlsoap.org/wsdl/mime/; xmlns:s= http://www.w3.org/2001/XMLSchema; xmlns:s0=http://www.openuri.org/; xmlns:soap=http://schemas.xmlsoap.org/wsdl/soap/; xmlns:soapenc= http://schemas.xmlsoap.org/soap/encoding/; xmlns:wsdl= http://schemas.xmlsoap.org/wsdl/; targetNamespace=http://www.openuri.org/; wsdl:types s:schema xmlns:s=http://www.w3.org/2001/XMLSchema; elementFormDefault=qualified targetNamespace=http://www.openuri.org/; s:element name=sendNCR s:complexType s:sequence s:element name=xml type=s:string minOccurs=0/ /s:sequence /s:complexType /s:element s:element name=sendNCRResponse s:complexType s:sequence s:element name=sendNCRResult type=s:int/ /s:sequence /s:complexType /s:element s:element name=int type=s:int/ /s:schema /wsdl:types wsdl:message name=sendNCRSoapIn wsdl:part name=parameters element=s0:sendNCR/ /wsdl:message wsdl:message name=sendNCRSoapOut wsdl:part name=parameters element=s0:sendNCRResponse/ /wsdl:message wsdl:message name=sendNCRHttpGetIn wsdl:part name=xml type=s:string/ /wsdl:message wsdl:message name=sendNCRHttpGetOut wsdl:part name=Body element=s0:int/ /wsdl:message wsdl:message name=sendNCRHttpPostIn wsdl:part name=xml type=s:string/ /wsdl:message wsdl:message name=sendNCRHttpPostOut wsdl:part name=Body element=s0:int/ /wsdl:message wsdl:portType name=EnvioGuiaSoap wsdl:operation name=sendNCR wsdl:input message=s0:sendNCRSoapIn/ wsdl:output message=s0:sendNCRSoapOut/ /wsdl:operation /wsdl:portType wsdl:portType name=EnvioGuiaHttpGet wsdl:operation name=sendNCR wsdl:input message=s0:sendNCRHttpGetIn/ wsdl:output message=s0:sendNCRHttpGetOut/ /wsdl:operation /wsdl:portType wsdl:portType name=EnvioGuiaHttpPost wsdl:operation name=sendNCR wsdl:input message=s0:sendNCRHttpPostIn/ wsdl:output message=s0:sendNCRHttpPostOut/ /wsdl:operation /wsdl:portType wsdl:binding name=EnvioGuiaSoap type=s0:EnvioGuiaSoap soap:binding transport=http://schemas.xmlsoap.org/soap/http; style=document/ wsdl:operation name=sendNCR soap:operation soapAction=http://www.openuri.org/sendNCR; style=document/ wsdl:input soap:body use=literal/ /wsdl:input wsdl:output soap:body use=literal/ /wsdl:output /wsdl:operation /wsdl:binding wsdl:binding name=EnvioGuiaHttpGet type=s0:EnvioGuiaHttpGet http:binding verb=GET/ wsdl:operation name=sendNCR http:operation location=/sendNCR/ wsdl:input http:urlEncoded/ /wsdl:input wsdl:output mime:mimeXml part=Body/ /wsdl:output /wsdl:operation /wsdl:binding wsdl:binding name=EnvioGuiaHttpPost type=s0:EnvioGuiaHttpPost http:binding verb=POST/ wsdl:operation name=sendNCR http:operation location=/sendNCR/ wsdl:input mime:content type=application/x-www-form-urlencoded/ /wsdl:input wsdl:output mime:mimeXml part=Body/ /wsdl:output /wsdl:operation /wsdl:binding wsdl:service name=EnvioGuia wsdl:port name=EnvioGuiaSoap binding=s0:EnvioGuiaSoap soap:address location=http://joule:8041/DteEnLinea/ws/EnvioGuia.jws / /wsdl:port wsdl:port name=EnvioGuiaHttpGet binding=s0:EnvioGuiaHttpGet http:address location=http://joule:8041/DteEnLinea/ws/EnvioGuia.jws / /wsdl:port wsdl:port name=EnvioGuiaHttpPost binding=s0:EnvioGuiaHttpPost http:address location=http://joule:8041/DteEnLinea/ws/EnvioGuia.jws / /wsdl:port /wsdl:service /wsdl:definitions -- Enviado desde mi dispositivo móvil Diego I. Hidalgo D. -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
On Mon, 2010-12-20 at 11:34 -0800, spaceman-spiff wrote:
Hi c.l.p folks
This is a rather long post, but i wanted to include all the details
everything i have tried so far myself, so please bear with me read
the entire boringly long post.
I am trying to parse a ginormous ( ~ 1gb) xml file.
Do that hundreds of times a day.
0. I am a python xml n00b, s have been relying on the excellent
beginner book DIP(Dive_Into_Python3 by MP(Mark Pilgrim) Mark , if
u are readng this, you are AWESOME so is your witty humorous
writing style)
1. Almost all exmaples pf parsing xml in python, i have seen, start off with
these 4 lines of code.
import xml.etree.ElementTree as etree
tree = etree.parse('*path_to_ginormous_xml*')
root = tree.getroot() #my huge xml has 1 root at the top level
print root
Yes, this is a terrible technique; most examples are crap.
2. In the 2nd line of code above, as Mark explains in DIP, the parse
function builds returns a tree object, in-memory(RAM), which
represents the entire document.
I tried this code, which works fine for a small ( ~ 1MB), but when i
run this simple 4 line py code in a terminal for my HUGE target file
(1GB), nothing happens.
In a separate terminal, i run the top command, i can see a python
process, with memory (the VIRT column) increasing from 100MB , all the
way upto 2100MB.
Yes, this is using DOM. DOM is evil and the enemy, full-stop.
I am guessing, as this happens (over the course of 20-30 mins), the
tree representing is being slowly built in memory, but even after
30-40 mins, nothing happens.
I dont get an error, seg fault or out_of_memory exception.
You need to process the document as a stream of elements; aka SAX.
3. I also tried using lxml, but an lxml tree is much more expensive,
as it retains more info about a node's context, including references
to it's parent.
[http://www.ibm.com/developerworks/xml/library/x-hiperfparse/]
When i ran the same 4line code above, but with lxml's elementree
( using the import below in line1of the code above)
import lxml.etree as lxml_etree
You're still using DOM; DOM is evil.
Which one is the best for my situation ?
Any all
code_snippets/wisdom/thoughts/ideas/suggestions/feedback/comments/ of
the c.l.p community would be greatly appreciated.
Plz feel free to email me directly too.
http://docs.python.org/library/xml.sax.html
http://coils.hg.sourceforge.net/hgweb/coils/coils/file/62335a211fda/src/coils/foundation/standard_xml.py
--
http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
Hi Usernet First up, thanks for your prompt reply. I will make sure i read RFC1855, before posting again, but right now chasing a hard deadline :) I am sorry i left out what exactly i am trying to do. 0. Goal :I am looking for a specific element..there are several 10s/100s occurrences of that element in the 1gb xml file. The contents of the xml, is just a dump of config parameters from a packet switch( although imho, the contents of the xml dont matter) I need to detect them then for each 1, i need to copy all the content b/w the element's start end tags create a smaller xml file. 1. Can you point me to some examples/samples of using SAX, especially , ones dealing with really large XML files. 2.This brings me to another q. which i forgot to ask in my OP(original post). Is simply opening the file, using reg ex to look for the element i need, a *good* approach ? While researching my problem, some article seemed to advise against this, especially since its known apriori, that the file is an xml since regex code gets complicated very quickly is not very readable. But is that just a style/elegance issue, for my particular problem (detecting a certain element, then creating(writing) a smaller xml file corresponding to, each pair of start end tags of said element), is the open file regex approach, something you would recommend ? Thanks again for your super-prompt response :) cheers ashish -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
On Mon, 2010-12-20 at 12:29 -0800, spaceman-spiff wrote: I need to detect them then for each 1, i need to copy all the content b/w the element's start end tags create a smaller xml file. Yep, do that a lot; via iterparse. 1. Can you point me to some examples/samples of using SAX, especially , ones dealing with really large XML files. SaX is equivalent to iterparse (iterpase is a way, to essentially, do SaX-like processing). I provided an iterparse example already. See the Read_Rows method in http://coils.hg.sourceforge.net/hgweb/coils/coils/file/62335a211fda/src/coils/foundation/standard_xml.py 2.This brings me to another q. which i forgot to ask in my OP(original post). Is simply opening the file, using reg ex to look for the element i need, a *good* approach ? No. -- http://mail.python.org/mailman/listinfo/python-list
Why do my list go uni-code by itself?
I'm reading a fixed format text file, line by line. I hereunder present the code. I have snipped out part not related to the file reading. Only relevant detail left out is the lstCutters. It looks like this: [[1, 9], [11, 21], [23, 48], [50, 59], [61, 96], [98, 123], [125, 150]] It specifies the first and last character position of each token in the fixed format of the input line. All this works fine, and is only to explain where I'm going. The code, in the function definition, is broken up in more lines than necessary, to be able to monitor the variables, step by step. --- Code start -- import codecs snip def CutLine2List(strIn,lstCut): strIn = strIn.strip() print 'InNextLine',strIn # skip if line is empty if len(strIn)1: return False lstIn = list() for cc in lstCut: strSubline =strIn[cc[0]-1:cc[1]-1].strip() lstIn.append(strSubline) print 'InSubline2'+lstIn[len(lstIn)-1]+'' del strIn, lstCut,cc print 'InReturLst',lstIn return lstIn snip filIn = codecs.open( strFileNameIn, mode='r', encoding='utf-8', errors='strict', buffering=1) for linIn in filIn: lstIn = CutLine2List(linIn,lstCutters) --- Code end -- A sample output, representing one line from the input file looks like this: InNextLine I 30 2002-12-11 20:01:19.280 563FANØ 2001-12-12-15.46.12.734502 2001-12-12-15.46.12.734502 InSubline2I InSubline230 InSubline22002-12-11 20:01:19.280 InSubline2563 InSubline2FANØ InSubline22001-12-12-15.46.12.73450 InSubline22001-12-12-15.46.12.73450 InReturLst [u'I', u'30', u'2002-12-11 20:01:19.280', u'563', u'FAN\xd8', u'2001-12-12-15.46.12.73450', u'2001-12-12-15.46.12.73450'] Question: In the last printout, tagged InReturLst all entries turn into uni-code. What happens here? Look for the word 'FANØ'. This word changes from 'FANØ' to u'FAN\xd8' -- That's a problem to me, and I don't want it to change like this. What do I do to stop this behavior? Best Regards Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
On 12/20/2010 2:49 PM, Adam Tauno Williams wrote: Yes, this is a terrible technique; most examples are crap. Yes, this is using DOM. DOM is evil and the enemy, full-stop. You're still using DOM; DOM is evil. For serial processing, DOM is superfluous superstructure. For random access processing, some might disagree. Which one is the best for my situation ? Any all code_snippets/wisdom/thoughts/ideas/suggestions/feedback/comments/ of the c.l.p community would be greatly appreciated. Plz feel free to email me directly too. http://docs.python.org/library/xml.sax.html http://coils.hg.sourceforge.net/hgweb/coils/coils/file/62335a211fda/src/coils/foundation/standard_xml.py For Python (unlike Java), wrapping module functions as class static methods is superfluous superstructure that only slows things down. raise Exception(...) # should be something specific like raise ValueError(...) -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
On 2010-12-20, spaceman-spiff ashish.mak...@gmail.com wrote: 0. Goal :I am looking for a specific element..there are several 10s/100s occurrences of that element in the 1gb xml file. The contents of the xml, is just a dump of config parameters from a packet switch( although imho, the contents of the xml dont matter) Then you need: 1. To detect whenever you move inside of the type element you are seeking and whenever you move out of it. As long as these elements cannot be nested inside of each other, this is an easy binary task. If they can be nested, then you will need to maintain some kind of level count or recursively decompose each level. 2. Once you have obtained a complete element (from its start tag to its end tag) you will need to test whether you have the single correct element that you are looking for. Something like this (untested) will work if the target tag cannot be nested in another target tag: - import xml.sax - class tagSearcher(xml.sax.ContentHandler): - - def startDocument(): - self.inTarget = False - - def startElement(name, attrs): - if name == targetName: - self.inTarget = True - elif inTarget = True: - # save element information - - def endElement(name): - if name == targetName: - self.inTarget = False - # test the saved information to see if you have the - # one you want: - # - #if its the peice you are looking for, then - #you can process the information - #you have saved - # - #if not, disgard the accumulated - # information and move on - - def characters(content): - if self.inTarget == True: - # save the content - - yourHandler = tagSearcher() - yourParser = xml.sax.make_parser() - yourParser.parse(inputXML, yourHandler) Then you just walk through the document picking up and discarding each target element type until you have the one that you are looking for. I need to detect them then for each 1, i need to copy all the content b/w the element's start end tags create a smaller xml file. Easy enough; but, with SAX you will have to recreate the tags from the information that they contain because they will be skipped by the character() events; so you will need to save the information from each tag as you come across it. This could probably be done more automatically using saxutils.XMLGenerator; but, I haven't actually worked with it before. xml.dom.pulldom also looks interesting 1. Can you point me to some examples/samples of using SAX, especially , ones dealing with really large XML files. There is nothing special about large files with SAX. Sax is very simple. It walks through the document and calls the the functions that you give it for each event as it reaches varius elements. Your callback functions (methods of a handler) do everthing with the information. SAX does nothing more then call your functions. There are events for reaching a starting tag, an end tag, and characters between tags; as well as some for beginning and ending a document. 2.This brings me to another q. which i forgot to ask in my OP(original post). Is simply opening the file, using reg ex to look for the element i need, a *good* approach ? While researching my problem, some article seemed to advise against this, especially since its known apriori, that the file is an xml since regex code gets complicated very quickly is not very readable. But is that just a style/elegance issue, for my particular problem (detecting a certain element, then creating(writing) a smaller xml file corresponding to, each pair of start end tags of said element), is the open file regex approach, something you would recommend ? It isn't an invalid approach if it works for your situatuation. I have used it before for very simple problems. The thing is, XML is a context free data format which makes it difficult to generate precise regular expressions, especially where where tags of the same type can be nested. It can be very error prone. Its really easy to have a regex work for your tests and fail, either by matching too much or failing to match, because you didn't anticipate a given piece of data. I wouldn't consider it a robust solution. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why do my list go uni-code by itself?
I'm reading a fixed format text file, line by line. I hereunder present the code. I have snipped out part not related to the file reading. Only relevant detail left out is the lstCutters. It looks like this: [[1, 9], [11, 21], [23, 48], [50, 59], [61, 96], [98, 123], [125, 150]] It specifies the first and last character position of each token in the fixed format of the input line. All this works fine, and is only to explain where I'm going. The code, in the function definition, is broken up in more lines than necessary, to be able to monitor the variables, step by step. --- Code start -- import codecs snip def CutLine2List(strIn,lstCut): strIn = strIn.strip() print 'InNextLine',strIn # skip if line is empty if len(strIn)1: return False More Pythonic would be: if not strIn: lstIn = list() for cc in lstCut: strSubline =strIn[cc[0]-1:cc[1]-1].strip() The start index is inclusive; the end index is exclusive. lstIn.append(strSubline) print 'InSubline2'+lstIn[len(lstIn)-1]+'' del strIn, lstCut,cc Not necessary to del the names. They exist only within the function, which you're about to leave. print 'InReturLst',lstIn return lstIn Sometimes it returns a list and sometimes False. That's a bad idea; try to be consistent. snip filIn = codecs.open( strFileNameIn, mode='r', encoding='utf-8', errors='strict', buffering=1) You're decoding from UTF-8 to Unicode, so all the strings you're working on are Unicode strings. for linIn in filIn: lstIn = CutLine2List(linIn,lstCutters) -- http://mail.python.org/mailman/listinfo/python-list
Re: Why do my list go uni-code by itself?
On Mon, Dec 20, 2010 at 2:08 PM, Martin Hvidberg mar...@hvidberg.net wrote: Question: In the last printout, tagged InReturLst all entries turn into uni-code. What happens here? Actually, they were all unicode to begin with. You're using codecs.open to read the file, which transparently decodes the data using the supplied encoding (in this case, utf-8). If you wanted to preserve the original bytes, you would just use the open() function to open the file instead. Look for the word 'FANØ'. This word changes from 'FANØ' to u'FAN\xd8' – That's a problem to me, and I don't want it to change like this. This happens because you're printing a list instead of a unicode string. When you print the unicode string, it tries to print the actual characters. When you print the list, it constructs the repr of the list, which uses the repr of each of the items in the list, and the repr of the unicode string is u'FAN\xd8'. If you don't want this to happen, then you will need to format the list as a string yourself instead of relying on print to do what it thinks you might want. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Modifying an existing excel spreadsheet
I have a user supplied 'template' Excel spreadsheet. I need to create a new excel spreadsheet based on the supplied template, with data filled in. I found the tools here http://www.python-excel.org/, and http://sourceforge.net/projects/pyexcelerator/. I have been trying to use the former, since the latter seems to be devoid of documentation (not even any docstrings). My first thought was to copy the template, open the copy, modify it and save the modifications. But it looks like if I open an existing spreadsheet it must be read only. So I tried to open the template, copy it to a new spreadsheet and write the new spreadsheet, but I can't seem to copy the images, and it looks like copying the formatting is going to be difficult. Can anyone give me any tips or advice? Thanks in advance, -EdK Ed Keith e_...@yahoo.com Blog: edkeith.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Why do my list go uni-code by itself?
On 01/-10/-28163 02:59 PM, Martin Hvidberg wrote: I'm reading a fixed format text file, line by line. I hereunder present the code. I have snipped out part not related to the file reading. Only relevant detail left out is the lstCutters. It looks like this: [[1, 9], [11, 21], [23, 48], [50, 59], [61, 96], [98, 123], [125, 150]] It specifies the first and last character position of each token in the fixed format of the input line. All this works fine, and is only to explain where I'm going. The code, in the function definition, is broken up in more lines than necessary, to be able to monitor the variables, step by step. --- Code start -- import codecs snip def CutLine2List(strIn,lstCut): strIn = strIn.strip() print 'InNextLine',strIn # skip if line is empty if len(strIn)1: return False lstIn = list() for cc in lstCut: strSubline =strIn[cc[0]-1:cc[1]-1].strip() lstIn.append(strSubline) print 'InSubline2'+lstIn[len(lstIn)-1]+'' del strIn, lstCut,cc print 'InReturLst',lstIn return lstIn snip filIn = codecs.open( strFileNameIn, mode='r', encoding='utf-8', errors='strict', buffering=1) for linIn in filIn: lstIn = CutLine2List(linIn,lstCutters) --- Code end -- A sample output, representing one line from the input file looks like this: InNextLine I 30 2002-12-11 20:01:19.280 563 FANØ 2001-12-12-15.46.12.734502 2001-12-12-15.46.12.734502 InSubline2I InSubline230 InSubline22002-12-11 20:01:19.280 InSubline2563 InSubline2FANØ InSubline22001-12-12-15.46.12.73450 InSubline22001-12-12-15.46.12.73450 InReturLst [u'I', u'30', u'2002-12-11 20:01:19.280', u'563', u'FAN\xd8', u'2001-12-12-15.46.12.73450', u'2001-12-12-15.46.12.73450'] Question: In the last printout, tagged InReturLst all entries turn into uni-code. What happens here? Look for the word 'FANØ'. This word changes from 'FANØ' to u'FAN\xd8' -- That's a problem to me, and I don't want it to change like this. What do I do to stop this behavior? Best Regards Martin If you don't want Unicode, why do you specify that the file is encoded as utf-8 ? If it's ASCII, just open the file, without using a utf-8 codec. Of course, then you'll have to fix the input file to make it ASCII. The character in the input file following the letters FAN is not a zero, it's some other character, apparently 00D8 in the Unicode table, not 0030. It didn't change in the InRturLst line. You were reading Unicode strings from the file. When you print Unicode, it encodes it in whatever your console device specifies. But when you print a list, it uses repr() on the elements, so you get to see what their real type is. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Sending XML to a WEB Service and Getting Response Back
On Mon, Dec 20, 2010 at 11:58 AM, Anurag Chourasia anurag.choura...@gmail.com wrote: Dear Python Mates, I have a requirement to send a XML Data to a WEB Service whose URL is of the form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws I also have to read back the response returned as a result of sending this data to this WebService. This web service implements the following operations: sendNCR This web service has no callbacks. I have pasted the complete WSDL for this WEB Service below my email. I would appreciate if someone could guide me with sample code using a Python Library suitable to fulfill this requirement of mine. The ZSI or SOAPpy library should do what you need. -- http://mail.python.org/mailman/listinfo/python-list
Class-override of a sort-key method?
Hi all - it would seem that these days, all the cool kids use the sort function's 'key' kwarg in order to sort a list of custom objects quickly. Unfortunately, as opposed to using 'cmp', where you can implent __cmp__ to get 'automatic sorting' in a similar fashion, there doesn't seem to be a direct analogue for a class-overridable method for providing a sort key. (ie, something like '__sortkey__' or '__key__'). Is there one, and I'm just missing it? If not, are there any plans to add one? (I did a quick search of the PEP list, and the only hits for 'sort' I saw had to do with sorting dictionaries by value). - Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Class-override of a sort-key method?
On Mon, Dec 20, 2010 at 3:23 PM, pythonlist.cali...@spamgourmet.com wrote: Hi all - it would seem that these days, all the cool kids use the sort function's 'key' kwarg in order to sort a list of custom objects quickly. Really? They don't bother to define __cmp__ or similar? Sounds lazy and poorly structured. Unfortunately, as opposed to using 'cmp', where you can implent __cmp__ to get 'automatic sorting' in a similar fashion, there doesn't seem to be a direct analogue for a class-overridable method for providing a sort key. (ie, something like '__sortkey__' or '__key__'). Just simply delegate to key comparison in your __cmp__ (or similar) method(s). Is there one, and I'm just missing it? If not, are there any plans to add one? (I did a quick search of the PEP list, and the only hits for 'sort' I saw had to do with sorting dictionaries by value). If you know at class-definiton-time how you want instances to be sorted, then just define __cmp__ (or the rich comparison methods) appropriately, possibly even delegating to a comparison of keys (as the class defines them). For example: from functools import total_ordering @total_ordering class Person(object): def __init__(self, first_name, last_name): self.first_name = first_name self.last_name = last_name @property def _key(self): Identifying key for a Person, by which they are sorted return (self.last_name, self.first_name) def __eq__(self, other): return isinstance(other, Person) and self._key == other._key def __lt__(self, other): return self._key other._key If you want to abstract even this away, then just write a class decorator; there's no need to add yet another (rather complicated due to all the interactions with the existing comparison methods) special method. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Catching user switching and getting current active user from root on linux
On 12/20/2010 12:54 PM, mpnordland wrote: I give up, I will never try to use a usenet group again. For the ones of you who tried to help thank you. You helped to identify some of my troubles, as for you @usernet, you are a troll Don't give up after one experience. Usenet can be really useful as long as you know who to listen to and who to ignore ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Redundant importing of modules
When writing a function that uses a module such as NumPy, it is tempting to include the statement import numpy or import numpy as np in the definition of the function, in case the function is used in a script that hasn't already imported NumPy. That could lead to the script issuing the import numpy command more than once. Does Python know to disregard redundant import commands? -- http://mail.python.org/mailman/listinfo/python-list
Re: Class-override of a sort-key method?
On 12/20/2010 7:10 PM, Chris Rebert wrote: On Mon, Dec 20, 2010 at 3:23 PM, pythonlist.cali...@spamgourmet.com wrote: Hi all - it would seem that these days, all the cool kids use the sort function's 'key' kwarg in order to sort a list of custom objects quickly. Really? They don't bother to define __cmp__ or similar? Sounds lazy and poorly structured. That sounds to me like a potentially ill-informed response. Unfortunately, as opposed to using 'cmp', where you can implent __cmp__ to get 'automatic sorting' in a similar fashion, there doesn't seem to be a direct analogue for a class-overridable method for providing a sort key. (ie, something like '__sortkey__' or '__key__'). Why do you talk about implementing __cmp__? Why should this be necessary? Just simply delegate to key comparison in your __cmp__ (or similar) method(s). Assuming, of course, that you are conveniently sorting only object over which you have complete control ... Is there one, and I'm just missing it? If not, are there any plans to add one? (I did a quick search of the PEP list, and the only hits for 'sort' I saw had to do with sorting dictionaries by value). If you know at class-definiton-time how you want instances to be sorted, then just define __cmp__ (or the rich comparison methods) appropriately, possibly even delegating to a comparison of keys (as the class defines them). For example: from functools import total_ordering @total_ordering class Person(object): def __init__(self, first_name, last_name): self.first_name = first_name self.last_name = last_name @property def _key(self): Identifying key for a Person, by which they are sorted return (self.last_name, self.first_name) def __eq__(self, other): return isinstance(other, Person) and self._key == other._key def __lt__(self, other): return self._key other._key If you want to abstract even this away, then just write a class decorator; there's no need to add yet another (rather complicated due to all the interactions with the existing comparison methods) special method. But the *real* point is (as the documentation attempts to point out) that by providing the key argument it gets called only once per element, whereas the cmp argument (or the objects' __cmp__() method if you insist on letting sort delegate to that) gets called every time two objects have to be compared. If cmp is a Python function (or equivalently if __cmp__() is a Python method) then calling it will take much longer than calling hte built-in default routines. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Redundant importing of modules
On 12/20/2010 8:36 PM, Jshgwave wrote: When writing a function that uses a module such as NumPy, it is tempting to include the statement import numpy or import numpy as np in the definition of the function, in case the function is used in a script that hasn't already imported NumPy. That could lead to the script issuing the import numpy command more than once. Does Python know to disregard redundant import commands? Oh, yes (as long as they are imported by the same name each time). There's a dict at sys.modules that has a key for each loaded module's name. When an attempt is made to import a module the first thing the interpreter does is to look at sys.modules. If it has the correct key in it then the assumption is that the module has already been imported, and its namespace is made available as the module name immediately. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Bug in fixed_point?!
I cannot figure out what I'm doing wrong. The following does not return a fixed point: from scipy import optimize xxroot= optimize.fixed_point(lambda xx: exp(-2.0*xx)/2.0, 1.0, args=(), xtol=1e-12, maxiter=500) print ' %f solves fixed point, ie f(%f)=%f ?'% (xxroot,xxroot,exp(-2.0*xxroot)/2.0) Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Modifying an existing excel spreadsheet
Am 20.12.2010 22:56, schrieb Ed Keith: I have a user supplied 'template' Excel spreadsheet. I need to create a new excel spreadsheet based on the supplied template, with data filled in. I found the tools here http://www.python-excel.org/, and http://sourceforge.net/projects/pyexcelerator/. I have been trying to use the former, since the latter seems to be devoid of documentation (not even any docstrings). My first thought was to copy the template, open the copy, modify it and save the modifications. But it looks like if I open an existing spreadsheet it must be read only. Could you post some code ? Did you try a simple file copy or do you iterate over all the cells ? So I tried to open the template, copy it to a new spreadsheet and write the new spreadsheet, but I can't seem to copy the images, and it looks like copying the formatting is going to be difficult. Can anyone give me any tips or advice? Thanks in advance, -EdK Ed Keith e_...@yahoo.com Blog: edkeith.blogspot.com As long as your program only needs to run under windows, COM automation is IMHO the best solution. Python tells Excel what to do. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bug in fixed_point?!
On 12/20/2010 10:03 PM, C Barrington-Leigh wrote: I cannot figure out what I'm doing wrong. The following does not return a fixed point: What did it do? For nearly all such questions, cut and paste actual output or traceback. from scipy import optimize xxroot= optimize.fixed_point(lambda xx: exp(-2.0*xx)/2.0, 1.0, args=(), xtol=1e-12, maxiter=500) print ' %f solves fixed point, ie f(%f)=%f ?'% (xxroot,xxroot,exp(-2.0*xxroot)/2.0) from math import exp x = 1.0 for i in range(70): print(repr(x)) x = exp(-2.0*x)/2.0 converges to 0.2835716452048919 Did you cut and paste what you actually ran? -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Google AI challenge: planet war. Lisp won.
Wasn't that the challenge where they wouldn't even accept solutions written in many other languages (including both OCaml and F#)? Cheers, Jon. Xah Lee xah...@gmail.com wrote in message news:44a8f48f-e291-463e-a042-d0cbc31a2...@z17g2000prz.googlegroups.com... discovered this rather late. Google has a AI Challenge: planet wars. http://ai-contest.com/index.php it started sometimes 2 months ago and ended first this month. the winner is Gábor Melis, with his code written in lisp. Congrats lispers! Gábor wrote a blog about it here http://quotenil.com/Planet-Wars-Post-Mortem.html (not sure if this has been mentioned here but quick search didn't find it) Xah ∑ http://xahlee.org/ ☄ -- http://mail.python.org/mailman/listinfo/python-list
Re: Sending XML to a WEB Service and Getting Response Back
On 12/20/2010 12:14 PM, Hidura wrote: I recommend you use the urllib.request in the library of python says everything that you want to know. 2010/12/20, Anurag Chourasiaanurag.choura...@gmail.com: Dear Python Mates, I have a requirement to send a XML Data to a WEB Service whose URL is of the form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws I also have to read back the response returned as a result of sending this data to this WebService. This web service implements the following operations: sendNCR This web service has no callbacks. I have pasted the complete WSDL for this WEB Service below my email. I would appreciate if someone could guide me with sample code using a Python Library suitable to fulfill this requirement of mine. Regards, Anurag If you're writing the client side, the service talks SOAP, and you have a WSDL file, use the suds module. SOAPpy is way out of date. The last update on SourceForge was in 2001. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Google AI challenge: planet war. Lisp won.
On Mon, Dec 20 2010, Jon Harrop wrote: Wasn't that the challenge where they wouldn't even accept solutions written in many other languages (including both OCaml and F#)? Cheers, Jon. http://ai-contest.com/faq.php Question: There is no starter package for my favorite language. What shall I do? Answer: You don't know C++, Java, or Python? Okay fine. Tell the forums what starter package you want to see, and we will try our best to make it for you. The folks working on Lisp submissions apparently created their own starter package. I am sure that something similar could have been done for OCaml or F#. That's probably too bad. These types of competitions are good publicity for less popular languages (assuming that the bots to well). Jason -- http://mail.python.org/mailman/listinfo/python-list
Re: Sending XML to a WEB Service and Getting Response Back
On 12/20/2010 11:34 PM, John Nagle wrote: SOAPpy is way out of date. The last update on SourceForge was in 2001. 2007, actually: http://sourceforge.net/projects/pywebsvcs/files/ And there is repository activity within the past 9 months. Still, point taken. -- http://mail.python.org/mailman/listinfo/python-list
[issue10461] Use with statement throughout the docs
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: I've already made the change, Terry, just holding off committing it because Georg has frozen the py3k branch until 3.2b2 is released. There are a few other changes I'm making, will commit these soon after 3.2b2 is released. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10461 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4761] create Python wrappers for openat() and others
Changes by Matt Joiner anacro...@gmail.com: -- nosy: +anacrolix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4761 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8863] Display Python backtrace on SIGSEGV, SIGFPE and fatal error
STINNER Victor victor.stin...@haypocalc.com added the comment:
Le lundi 20 décembre 2010 07:55:08, vous avez écrit :
+#define NFAULT_SIGNALS (sizeof(fault_signals) / sizeof(fault_signals[0]))
+static fault_handler_t fault_handlers[4];
, should use NFAULT_SIGNALS instead of 4.
Ah yes, NFAULT_SIGNALS is a better choice than the maximum of possible signals
(4).
However, this bit of code bothers me a lot:
+const int fd = 2; /* should be fileno(stderr) */
To assume that fd=2 is the write place to be writing bytes is assuming
quite a bit about the state of the application. It is not unusual at all
to close 0,1,2 when writing daemons, which frees them up to be assigned to
*anything*.
Write into a closed file descriptor just does nothing. Closed file descriptors
are not a problem.
For all you know, fd=2 currently is a network socket that you
will be throwing gibberish at, or worse it could be a block device that
you are writing gibberish on.
The GNU libc has also a fault handler (source code: debug/segfault.c). It uses
the file descriptor 2, except if SEGFAULT_OUTPUT_NAME environment variable is
set (value stored in fname variable): open the specified file.
/* This is the name of the file we are writing to. If none is given
or we cannot write to this file write to stderr. */
fd = 2;
if (fname != NULL)
{
fd = open (fname, O_TRUNC | O_WRONLY | O_CREAT, 0666);
if (fd == -1)
fd = 2;
}
The GNU libc installs handlers for SIGSEGV, SIGILL, SIGBUS, SIGSTKFLT,
SIGABBRT and SIGFPE signals. The handler restores the default handler and re-
raise the signal:
/* Pass on the signal (so that a core file is produced). */
sa.sa_handler = SIG_DFL;
sigemptyset (sa.sa_mask);
sa.sa_flags = 0;
sigaction (signal, sa, NULL);
raise (signal);
Where raise(signal); is something like kill(getpid(), signal).
It only uses an alternate stack if SEGFAULT_USE_ALTSTACK environment variable
is set.
The handler can display registers, the backtrace and the memory mappings,
depending on the compilation options and the operating system.
The closest discussion I could find on this subject was on the libstdc++
mailing-list with regard to their verbose termination code:
http://gcc.gnu.org/ml/gcc-patches/2004-02/msg02388.html
AFAICT, their conclusion was that the only reasonable solution was to write
to the stderr FILE since it was the only thing that is guaranteed to make
sense always (even though it may fail). Their situation is different in
that they are handling a C++ exception, so they don't have to stick to
async-safe functions, but absent that extra difficulty, I believe the
reasoning is the same.
The FILE* type cannot be used because fprintf(), fputs(), ... are not signal-
safe.
I'm not sure there is a safe way to know what the fileno for sys.stderr
is because it can be anything, including an object whose fileno changes
over time
The signal handler cannot access the Python object. Eg. if sys.stderr is a
StringIO() (which has no file descriptor), it cannot be used.
However, I think it would be fair to support only built-in io
types that are obviously safe, since you could cache the fileno() value at
assignment to use in your fault handler.
The problem is to detect that stderr file descriptor changes (eg. closed,
duplicated, reopened, etc.). I don't think that it's possible to detect such
changes (with a portable function).
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8863
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10739] Subprocess behavior on Windows
R. David Murray rdmur...@bitdance.com added the comment: This might be an example of the general problem that on windows, sockets and files don't mix well. You can't use a file in a select call, either. I think there are two possibilities here: either makefile doesn't produce anything very useful on windows, or there's a bug in either makefile or subprocess. If the former, the proper place to document it would be in the makefile docs. -- assignee: d...@python - nosy: +brian.curtin, gregory.p.smith, r.david.murray, tim.golden versions: +Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10739 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10694] zipfile.py end of central directory detection not robust
Kevin Hendricks kevin.hendri...@sympatico.ca added the comment: I have not looked at how other tools handle this. They could simply ignore what comes after a valid endrecdata is found, they could strip it out (truncate it) or make it into a final comment. I guess for simply unpacking a zip archive, all of these are equivalent (it goes unused). But if you are copying a zip archive to another archive then ignoring it and truncating it may be safer in some sense (since you have no idea what this extra data is for and why it might be attached) but then you are not being faithful to the original but at the same time you do not want to create improper zip archives. If you change the extra data into a final comment, then at least none of the original data is actually lost (just moved slightly in the copied zip and protected as a comment) and could be recovered if it turns out to have been useful. With so many things using/making the zip archive format (jars, normal zips, epubs, etc) you never know what might have been left over at the end of the zip file and if it was important. So I am not really sure how to deal with this properly. Also I know nothing about _EndRecData64 and if it needs to somehow be handled in a different way. So someone who is very familiar with the code should look at this and tell us what is the right thing to do and even if the approach I took is correct (it works fine for me and I have taken to including my own zipfile.py in my own projects until this gets worked out) but it might not be the right thing to do. As for a test case, I know nothing about that but will look at test_zipfile.py. I am a Mac OS X user/developer so all of my code is targeted to running on both Python 2.5 (Mac OS X 10.5.X) and Python 2.6 (Mac OS 10.6.X). Python 3.X and even Python 2.7 are not on my horizon and not even on my build machine (trying to get Mac OS X users to install either would be an experience in frustration). I simply looked at the source in Python 2.7 and Python 3.1.3 (from the official Python releases from python.org) to see that the problem still exists (and it does). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10694 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8863] Display Python backtrace on SIGSEGV, SIGFPE and fatal error
STINNER Victor victor.stin...@haypocalc.com added the comment: The fault handler is unable to retrieve the thread state if the GIL is released. I will try to fix that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8863 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10739] Subprocess behavior on Windows
Ross Lagerwall rosslagerw...@gmail.com added the comment: Since the code in subprocess gets the underlying fileno of the file-like object (line 819 of subprocess.py), I presume it is an example of the general problem of files and sockets not mixing very well on Windows. So, I have attached a patch to document this in socket.makefile(). However, I also see that *it* is documented in select.select() that it will not work with Windows file-like objects. Maybe there should still be a note in subprocess.Popen() as per the first patch? -- Added file: http://bugs.python.org/file20117/socketdoc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10739 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10694] zipfile.py end of central directory detection not robust
R. David Murray rdmur...@bitdance.com added the comment: It's pretty easy, really, to do an SVN checkout of python and compile it on a mac, if you are at all familiar with the unix command line. If you don't have the time or desire for that, though, someone will eventually get to it, we just don't know when ;) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10694 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10694] zipfile.py end of central directory detection not robust
Kevin Hendricks kevin.hendri...@sympatico.ca added the comment: Been programming on unix/vax and then linux since the mid 80s and on punch cards in the late 70s. Grew my first beard writing 8080 and Z80 assembler. All of that was over 30 years ago. All I want to do is report a damn bug! Then I get nudged for a test case (although how to recreate the bug was already described) so I add the two lines to create a test case. Then I get nudged for a patch, so I give a patch even though there are many ways to deal with the issue. Then I get nudged for patches for other branches, then I get nudged for official test_zipfile.py patches. All of this **before** the damn owner has even bothered to look at it and say if he/she even wants it or if the patch is even correct. I have my own working code for the epub ebook stuff I am working on so this issue no longer impacts me. How did I go from a simple bug report to having to build python's latest checkout just to get someone to look at the bug. You have got to be kidding me! I am done here, do what you want with the bug. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10694 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10087] HTML calendar is broken
Changes by Ron Adam ron_a...@users.sourceforge.net: -- nosy: +ron_adam ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10087 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10740] sqlite3 module should allow DDL statements in transactions
New submission from Scott Urban scott.ur...@isilon.com: The python sqlite module automatically commits open transactions when it encounters a DDL statement. This is unnecessary; DDL is transactional in my testing (see attached). Attached patch addresses the issue. Patch is against 2.6.1, but looking at Trunk in svn, it seems like the patch is needed and would apply. One issue I could foresee is that this behavior might depend on the sqlite version in use (I'm on 3.6.10). Patch also allows pragma statement. -- components: Library (Lib) files: pysql-transactions.2.diff keywords: patch messages: 124392 nosy: scott.urban priority: normal severity: normal status: open title: sqlite3 module should allow DDL statements in transactions type: behavior versions: Python 2.6 Added file: http://bugs.python.org/file20118/pysql-transactions.2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10740 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10740] sqlite3 module should allow DDL statements in transactions
Scott Urban scott.ur...@isilon.com added the comment: Here are some tests. -- Added file: http://bugs.python.org/file20119/test_sqlite_ddl.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10740 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10741] PyGILState_GetThisThreadState() lacks a doc entry
New submission from Antoine Pitrou pit...@free.fr: PyGILState_GetThisThreadState() is documented in Include/pystate.h but not in the official docs. It should be documented along PyGILState_Ensure() and friends. -- assignee: d...@python components: Documentation messages: 124394 nosy: d...@python, haypo, pitrou priority: normal severity: normal status: open title: PyGILState_GetThisThreadState() lacks a doc entry versions: Python 2.7, Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10741 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10739] Subprocess behavior on Windows
R. David Murray rdmur...@bitdance.com added the comment: I think I'll leave that decision up to the doc crew. My thought was that makefile was supposedly returning a file, therefore it was appropriate to document there that it wasn't really a file on windows, whereas subprocess docs are only talking about files, and it only just so happens that you can pass it a socket on unix by using makefile. -- assignee: - d...@python stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10739 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10694] zipfile.py end of central directory detection not robust
R. David Murray rdmur...@bitdance.com added the comment: Sorry, I thought I was being clear that if you *wanted* to help further here was how you could, but if you didn't then we'd get to it eventually. We're all volunteers here, just like you, so every bit of help...helps, and we thank you sincerely for taking the time to report the bug and provide the preliminary patch and test. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10694 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8863] Display Python backtrace on SIGSEGV, SIGFPE and fatal error
Scott Dial sc...@scottdial.com added the comment: On 12/20/2010 8:30 AM, STINNER Victor wrote: Write into a closed file descriptor just does nothing. Closed file descriptors are not a problem. My issue not with a closed file descriptor, it is with an open file descriptor that is not what you think it is. For all you know, fd=2 currently is a network socket that you will be throwing gibberish at, or worse it could be a block device that you are writing gibberish on. The GNU libc has also a fault handler (source code: debug/segfault.c). It uses the file descriptor 2, except if SEGFAULT_OUTPUT_NAME environment variable is set (value stored in fname variable): open the specified file. The GNU libc segfault handler is *opt-in* via the SEGFAULT_SIGNALS environment variable. I do not know of a system where SEGFAULT_SIGNALS is a part of the default environment. I suspect the only time anyone uses that variable and code is if they are using the catchsegv tool, which comes with glibc. In any case, that developer should be aware of the implication of closing fd 2. The FILE* type cannot be used because fprintf(), fputs(), ... are not signal- safe. My point was that in C++, they have an object that an application developer more readily associates with stderr than fd 2. That writing to stderr leads to a write to fd 2 is incidental, because it's possible that stderr does *not* lead to a write to fd 2 and that writing to fd 2 would be a bad thing to do blindly. For instance, you can call freopen(path, mode, stderr) and fd 2 will end up closed and another fd will be the target of stderr. I don't believe POSIX guarantees that open() will not re-use fd 2. The problem is to detect that stderr file descriptor changes (eg. closed, duplicated, reopened, etc.). I don't think that it's possible to detect such changes (with a portable function). When I said that, I hadn't fully investigated the intricacies of the io types. I was unaware that you could assign to sys.stderr.buffer.raw and change out the target fd. I assumed a BufferedWriter could not have the target stream changed out from beneath it. So, I don't have a solution to the problem of knowing the correct (if any) file descriptor to write to. If the argument is made that fd 2 is the right place for most Python applications, then there needs to be a programmatic way to disable this feature and/or tell it where to write, so that programs that daemonize can do the right thing. -- title: Display Python backtrace on SIGSEGV, SIGFPE and fatal error - Display Python backtrace on SIGSEGV, SIGFPE and fatal error ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8863 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10740] sqlite3 module should allow DDL statements in transactions
R. David Murray rdmur...@bitdance.com added the comment: See also Issue 8145. It would be nice if someone could sort all this out, but I'm not knowledgeable enough to do so. For this patch, it would be a significant change it behaviour. Therefore it would have to be a new feature controlled by a flag of some sort (which is part of the reason for the reference to issue 8145). -- nosy: +ghaering, r.david.murray type: behavior - feature request versions: +Python 3.3 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10740 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8863] Display Python backtrace on SIGSEGV, SIGFPE and fatal error
Antoine Pitrou pit...@free.fr added the comment: The problem is to detect that stderr file descriptor changes (eg. closed, duplicated, reopened, etc.). I don't think that it's possible to detect such changes (with a portable function). When I said that, I hadn't fully investigated the intricacies of the io types. I was unaware that you could assign to sys.stderr.buffer.raw and change out the target fd. I assumed a BufferedWriter could not have the target stream changed out from beneath it. AFAICT, this is not deliberate (not documented, and not tested for). It should probably be fixed, actually, because there's no code that I know of that ensures it does something meaningful. -- title: Display Python backtrace on SIGSEGV, SIGFPE and fatal error - Display Python backtrace on SIGSEGV, SIGFPE and fatal error ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8863 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10557] Malformed error message from float()
Changes by Alexander Belopolsky belopol...@users.sourceforge.net: -- versions: +Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10557 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10715] uninformative error message
Changes by Éric Araujo mer...@netwok.org: -- components: +IO nosy: +eric.araujo versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10715 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10728] argparse.ArgumentParser.print_help uses sys.stdout
Éric Araujo mer...@netwok.org added the comment: Alright, I didn’t know you were doing mass merges. I personally prefer to leave reports open until backported, now I’ll know your habits. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10728 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8145] Documentation about sqlite3 isolation_level
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file17940/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8145 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10730] add .svgz to mimetypes.suffix_map and .svg to types_map
Éric Araujo mer...@netwok.org added the comment: +1 on adding SVG types in 3.2 or 3.3. -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10730 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10254] unicodedata.normalize('NFC', s) regression
Alexander Belopolsky belopol...@users.sourceforge.net added the comment: Attached patch, issue10254a.diff, adds the OP's cases to test_unicodedata and changes the code as I suggested in msg124173 because ISTM that comb = comb1 matches the pr-29 definition: D2'. In any character sequence beginning with a starter S, a character C is blocked from S if and only if there is some character B between S and C, and either B is a starter or it has the same or higher combining class as C. http://www.unicode.org/review/pr-29.html Unfortunately, all tests pass with either comb = comb1 or comb == comb1, so before I commit, I would like to figure out the test case that would properly exercise this code. -- Added file: http://bugs.python.org/file20120/issue10254a.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10254 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3871] cross and native build of python for mingw32 with distutils
Changes by alesko askon...@gmail.com: -- nosy: +alesko ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3871 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10742] memoryview.readonly attribute is not documented
New submission from flashk fla...@gmail.com: The 'readonly' attribute is not explicitly described, even though it is used in the sample code for the memoryview type. I've attached a patch that adds a description of the 'readonly' attribute. -- assignee: d...@python components: Documentation files: stdtypes.diff keywords: patch messages: 124403 nosy: d...@python, flashk priority: normal severity: normal status: open title: memoryview.readonly attribute is not documented versions: Python 2.7 Added file: http://bugs.python.org/file20121/stdtypes.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10742 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1243654] Faster output if message already has a boundary
R. David Murray rdmur...@bitdance.com added the comment: Turns out there's a bug in my version of the patch, and no test in the email test suite traversed that code path. Attached patch fixes this; I'll commit and backport after trunk unfreezes. Note that the backport contains a second bug (calls self._make_boundary when it should be just _make_boundary) -- status: closed - open Added file: http://bugs.python.org/file20122/boundar_fix_fix.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1243654 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10743] 3.2's sysconfig doesn't work with virtualenv
New submission from Sridhar Ratnakumar sridh...@activestate.com:
From http://code.google.com/p/virtualenv5/issues/detail?id=6 - it seems that
the `sysconfig` module is looking for Makefile in wrong directory, while
ideally it must be looking into the base Python install.
import sysconfig; sysconfig.get_paths('purelib')
Traceback (most recent call last):
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 332, in _init_posix
_parse_makefile(makefile, vars)
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 220, in _parse_makefile
with open(filename, errors=surrogateescape) as f:
IOError: [Errno 2] No such file or directory:
'/tmp/e/lib/python3.2/config-3.2m/Makefile'
--
assignee: tarek
components: Distutils, Macintosh
messages: 124405
nosy: eric.araujo, srid, tarek
priority: normal
severity: normal
status: open
title: 3.2's sysconfig doesn't work with virtualenv
type: behavior
versions: Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10743
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10744] ctypes arrays have incorrect buffer information (PEP-3118)
New submission from Pauli Virtanen p...@iki.fi: Ctypes arrays have invalid buffer interface information (on Python 3.1.2): import ctypes x = (ctypes.c_double*2)() y = memoryview(x) y.shape (2,) y.format '(2)d' This implies that the array contains 2 items, each consisting of 2 floats, which is incorrect -- the shape information is duplicated. A suggested fix is attached. (From http://projects.scipy.org/numpy/ticket/1699) -- assignee: theller components: ctypes files: 001-ctypes-fix-pep-3118-format-strings-for-arrays.patch keywords: patch messages: 124406 nosy: pv, theller priority: normal severity: normal status: open title: ctypes arrays have incorrect buffer information (PEP-3118) type: behavior versions: Python 3.1, Python 3.2, Python 3.3 Added file: http://bugs.python.org/file20123/001-ctypes-fix-pep-3118-format-strings-for-arrays.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10744 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10740] sqlite3 module should allow DDL statements in transactions
Scott Urban scott.ur...@isilon.com added the comment: I find the way that the sqlite3 module handles transactions pretty surprising in general, but I agree that someone who got used to DDL not rolling back could in theory find this patch surprising. We will apply this patch to our python build because having DDL covered by a transactions is convenient for certain tasks. If anyone can think of a problem with this simple patch, I'd like to hear it. Thanks Scott -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10740 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10745] --user option, per user site-packages undocumented in Installing Python Modules document
New submission from Chris Lasher chris.las...@gmail.com: Python 2.6 saw the introduction of per user site-packages directory for easy installation of Python packages into a guaranteed location in which the user has appropriate permissions. http://bugs.python.org/issue1799 http://www.python.org/dev/peps/pep-0370/ http://docs.python.org/whatsnew/2.6.html#pep-370-per-user-site-packages-directory With it came a new option available in distutils-powered setup.py scripts, --user. It has been a year since this feature was introduced, yet no documentation has appeared in the official Python Documentation other than in the What's New document. Specifically, this option should appear and be documented in the Installing Python Modules document. http://docs.python.org/install/ It would be very helpful if the documentation described the advantages of using this option over --home and --prefix. I am not the first user to notice this gap in the documentation, e.g., http://www.devx.com/opensource/Article/42353/1763 however, I couldn't find any bugs open for this issue so I have created this new one. -- assignee: tarek components: Distutils, Documentation messages: 124408 nosy: eric.araujo, gotgenes, tarek priority: normal severity: normal status: open title: --user option, per user site-packages undocumented in Installing Python Modules document versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10745 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10087] HTML calendar is broken
Ron Adam ron_a...@users.sourceforge.net added the comment:
The problem is in the following line...
return ''.join(v).encode(encoding, xmlcharrefreplace)
The .encode(encoding, xmlcharrefreplace) is returning a bytes object.
Here is the simplest change to resolve the problem.
return str(''.join(v).encode(encoding, xmlcharrefreplace),
encoding=encoding)
But maybe a different way to implement xmlcharrefreplace is what is needed.
Would it be better if it were a function or method in the xml (or html) module?
Just because it can be implemented as an encoding doesn't mean it should be.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10087
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10745] setup.py install --user option undocumented
Changes by Éric Araujo mer...@netwok.org: -- assignee: tarek - eric.araujo components: +Distutils2 -Distutils title: --user option, per user site-packages undocumented in Installing Python Modules document - setup.py install --user option undocumented versions: +3rd party -Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10745 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10087] HTML calendar is broken
Chris Lambacher ch...@kateandchris.net added the comment:
Sorry in advance for the long winded response.
Ron, have you looked at my patch?
The underlying issue is that the semantics for print() between Python 1 and 3.
print() does not accept a bytes type in Python 3. In Python 2 str was a bytes
type and so print happily sent encoded strings to stdout.
This presents an issue for both --type=html and the text version if an encoding
is asked for. Just using print() will result in repr being called on the byte
string and you get either an invalid HTML file or a text file with extra junk
in it (same junk in both).
If you ask for an encoding, you are going to get bytes. Changing it back into a
string to mask that effect does not actually fix things for you because once
you do print() you are back to a default encoding and therefore more broken
because you are not doing what the user asked for (which is a particular
encoding).
In order for:
return str(''.join(v).encode(encoding, xmlcharrefreplace),
encoding=encoding)
to solve the issue, you would also need to take away the ability for the user
to specify an encoding (at the command line and via the API). It's already a
string, why make it a byte and then a string again? If you don't want to deal
with encoding, then return a string and leave it up to the consumer of the API
to handle the desired encoding (and the xmlcharrefreplace, maybe with a note
in the docs).
If you do want to deal with encoding (which I think we are stuck with), then
solve the real issue by not using print() (see my patch).
I think the only reason that my patch was not accepted, and why this is still
languishing is that I said I would provide tests and have not had time to do
so.
Please feel free to correct me if I am wrong about any of the above.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10087
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10746] ctypes c_long c_bool have incorrect PEP-3118 type codes
New submission from Pauli Virtanen p...@iki.fi:
Currently on Python 3.x:
import ctypes
memoryview(ctypes.c_long()).format
'l'
This is invalid on 64-bit platforms: the above means 32-bit little-endian
float. The '' endian specification turns on the standard size mode
(similarly as for the struct module), which makes type character have a
platform-independent meaning.
Unfortunately, the struct module format syntax does *not* allow specifying
native-size non-native-endian items. So just replacing '' by '^' cannot be in
general be done.
Suggested fix attached. It adds a converter function that maps the
platform-dependent ctypes codes to struct module standard-size codes; handling
c_long and c_bool specially.
***
After this patch (and the one in http://bugs.python.org/issue10744 ):
import numpy as np
from ctypes import *
class Point(Structure):
... _fields_ = [(x, c_long), (y, c_long)]
...
class StructWithArrays(Structure):
... _fields_ = [(x, c_long * 3 * 2), (y, Point * 4)]
...
x = StructWithArrays()
y = np.asarray(x)
y.dtype
dtype([('x', 'i8', (2, 3)), ('y', [('x', 'i8'), ('y', 'i8')], (4,))])
y['x'] = [[1,2,3],[4,5,6]]
y['y']['x'] = np.arange(4) + 10
y['y']['y'] = np.arange(4) + 20
x.x[0][0]
1
x.x[0][1]
2
x.x[0][2]
3
x.y[0].x
10
x.y[1].x
11
x.y[0].y
20
x.y[1].y
21
--
files: 001-ctypes-fix-pep-3118-type-codes-for-c-long-and-c-bool.patch
keywords: patch
messages: 124411
nosy: pv
priority: normal
severity: normal
status: open
title: ctypes c_long c_bool have incorrect PEP-3118 type codes
Added file:
http://bugs.python.org/file20124/001-ctypes-fix-pep-3118-type-codes-for-c-long-and-c-bool.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10746
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10746] ctypes c_long c_bool have incorrect PEP-3118 type codes
Changes by Pauli Virtanen p...@iki.fi: -- assignee: - theller components: +ctypes nosy: +theller type: - behavior versions: +Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10746 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10743] 3.2's sysconfig doesn't work with virtualenv
Ned Deily n...@acm.org added the comment:
That does seem to be a regression since distutils.sysconfig works correctly in
a virtualenv:
$ python3.2 -c 'import
distutils.sysconfig;print(distutils.sysconfig.get_makefile_filename())'
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/config-3.2m/Makefile
$ python3.2 -c 'import sysconfig;print(sysconfig.get_makefile_filename())'
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/config-3.2m/Makefile
but in a virtualenv:
$ source t/bin/activate
$ python3.2 -c 'import
distutils.sysconfig;print(distutils.sysconfig.get_makefile_filename())'
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/config-3.2m/Makefile
$ python3.2 -c 'import sysconfig;print(sysconfig.get_makefile_filename())'
Traceback (most recent call last):
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 332, in _init_posix
_parse_makefile(makefile, vars)
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 220, in _parse_makefile
with open(filename, errors=surrogateescape) as f:
IOError: [Errno 2] No such file or directory:
'/Users/nad/t/lib/python3.2/config-3.2m/Makefile'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File string, line 1, in module
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 322, in get_makefile_filename
return os.path.join(get_path('stdlib'),
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 451, in get_path
return get_paths(scheme, vars, expand)[name]
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 442, in get_paths
return _expand_vars(scheme, vars)
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 168, in _expand_vars
_extend_dict(vars, get_config_vars())
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 487, in get_config_vars
_init_posix(_CONFIG_VARS)
File
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/sysconfig.py,
line 337, in _init_posix
raise IOError(msg)
BTW, your example will fail in any case since get_paths takes a scheme name;
purelib is a path name.
--
nosy: +barry, ned.deily
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10743
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10743] 3.2's sysconfig doesn't work with virtualenv
Ned Deily n...@acm.org added the comment: Setting to deferred blocker for 3.2 release manager evaluation. -- nosy: +georg.brandl priority: normal - deferred blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10743 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10740] sqlite3 module should allow DDL statements in transactions
Changes by Andrew Svetlov andrew.svet...@gmail.com: -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10740 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10087] HTML calendar is broken
Ron Adam ron_a...@users.sourceforge.net added the comment: Oops. You're right. I miss understood how the encode method works in this particular case. ;-/ I agree with your comments as well. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10087 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10715] uninformative error message
Phillip M. Feldman phillip.m.feld...@gmail.com added the comment: I eventually determined that a call to `subprocess.Popen` was responsible for the message, but could have determined this much more quickly if the message had included the name of the file that could not be opened (executed). Phillip On Mon, Dec 20, 2010 at 11:20 AM, Ãric Araujo rep...@bugs.python.orgwrote: Changes by Ãric Araujo mer...@netwok.org: -- components: +IO nosy: +eric.araujo versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10715 ___ -- nosy: +phillip.m.feld...@gmail.com Added file: http://bugs.python.org/file20125/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10715 ___I eventually determined that a call to `subprocess.Popen` was responsible for the message, but could have determined this much more quickly if the message had included the name of the file that could not be opened (executed).br brPhillipbrbrdiv class=gmail_quoteOn Mon, Dec 20, 2010 at 11:20 AM, Ãric Araujo span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/span wrote:brblockquote class=gmail_quote style=margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex; br Changes by Ãric Araujo lt;a href=mailto:mer...@netwok.org;mer...@netwok.org/agt;:br br br --br components: +IObr nosy: +eric.araujobr versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.6br divdiv/divdiv class=h5br ___br Python tracker lt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;br lt;a href=http://bugs.python.org/issue10715; target=_blankhttp://bugs.python.org/issue10715/agt;br ___br /div/div/blockquote/divbr ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10715] uninformative error message
Changes by Alexander Belopolsky belopol...@users.sourceforge.net: Removed file: http://bugs.python.org/file20125/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10715 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10715] uninformative error message
Phillip M. Feldman phillip.m.feld...@gmail.com added the comment: Why was this removed? On Mon, Dec 20, 2010 at 8:30 PM, Alexander Belopolsky rep...@bugs.python.org wrote: Changes by Alexander Belopolsky belopol...@users.sourceforge.net: Removed file: http://bugs.python.org/file20125/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10715 ___ -- Added file: http://bugs.python.org/file20126/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10715 ___Why was this removed?brbrdiv class=gmail_quoteOn Mon, Dec 20, 2010 at 8:30 PM, Alexander Belopolsky span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/span wrote:brblockquote class=gmail_quote style=margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex; br Changes by Alexander Belopolsky lt;a href=mailto:belopol...@users.sourceforge.net;belopol...@users.sourceforge.net/agt;:br br br Removed file: a href=http://bugs.python.org/file20125/unnamed; target=_blankhttp://bugs.python.org/file20125/unnamed/abr divdiv/divdiv class=h5br ___br Python tracker lt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;br lt;a href=http://bugs.python.org/issue10715; target=_blankhttp://bugs.python.org/issue10715/agt;br ___br /div/div/blockquote/divbr ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10254] unicodedata.normalize('NFC', s) regression
Alexander Belopolsky belopol...@users.sourceforge.net added the comment: On Mon, Dec 20, 2010 at 2:50 PM, Alexander Belopolsky rep...@bugs.python.org wrote: .. Unfortunately, all tests pass with either comb = comb1 or comb == comb1, so before I commit, I would like to figure out the test case that would properly exercise this code. After some more thought, I've realized that the comb comb1 case is impossible if comb1 != 0 (due to canonical reordering step) and if comb1 == 0, the comb1 to comb comparison is not reached. In other words, it does not matter whether comparison is done as Martin suggested in msg120018 or as it is done in the latest patch. The fact that comb comb1 case is impossible if comb1 != 0 is actually mentioned in PR 29 itself. See Table 1: Differences at http://www.unicode.org/review/pr-29.html. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10254 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
