Re: [Python-Dev] sys.setdefaultencoding() vs. csv module + unicode
> The csv module says it's not unicode safe but the 2.5 docs [3] have a
> workaround for this. While the workaround says nothing about
> sys.setdefaultencoding() it simply does not work with the default
> encoding, "ascii." Is this _the_ problem with the csv module? Should
> I give up and use XML? Below is code that works vs. code that
> doesn't. Am I interpretting the workaround from the docs wrong?
These questions are off-topic for python-dev; please ask them on
comp.lang.python instead. python-dev is for the development *of*
Python, not for the development *with* Python.
> kumar$ python2.5
> Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
> [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
import sys, csv, codecs
f = codecs.open('unicsv.csv','wb','utf-8')
w = csv.writer(f)
w.writerow([u'lang', u'espa\xa4ol'])
What you should do here is
def encoderow(r):
return [s.encode("utf-8") for s in r])
f = open('unicsv.csv', 'wb', 'utf-8')
w = csv.writer(f)
w.writerow(encoderow([u'lang', u'espa\xa4ol'])
IOW, you need to encode *before* passing the strings
to the CSV module, not afterwards.
If it is too tedious for you to put in the encoderow
calls all the time, you can write a wrapper for CSV
writers which transparently encodes all Unicode
strings.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RFC] urlparse - parse query facility
On Tuesday 12 June 2007, Senthil Kumaran wrote: > This mail is a request for comments on changes to urlparse module. We > understand that urlparse returns the 'complete query' value as the query > component and does not > provide the facilities to separate the query components. User will have to > use the cgi module (cgi.parse_qs) to get the query parsed. I agree with the comments Jim provided. > Below method implements the urlparse_qs(url, > keep_blank_values,strict_parsing) that will help in parsing the query > component of the url. It behaves same as the cgi.parse_qs. Except that it takes a URL, not only a query string. > def urlparse_qs(url, keep_blank_values=0, strict_parsing=0): ... > scheme, netloc, url, params, querystring, fragment = urlparse(url) I see no reason to incorporate the URL splitting into the function; the existing function signatures for cgi.parse_qs and cgi.parse_qsl are sufficient. It may be convenient to add methods to the urlparse.BaseResult class providing access to the parsed version of the query on the instance. -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RFC] urlparse - parse query facility
* Jim Jewett <[EMAIL PROTECTED]> [2007-06-13 19:27:24]: > > a) import cgi and call cgi module's query_ps. [circular imports] > > or > > > b) Implement a stand alone query parsing facility in urlparse *AS IN* > > cgi module. > > Assuming (b), please remove the (code for the) parsing from the cgi > module, and just import it back from urlparse (or urllib). Since cgi > already imports urllib (which imports urlparse), this isn't adding any > dependencies -- but it keeps the code in a single location. Sure, thats a good idea as I see it. It wont break anything as well. Thanks, -- O.R.Senthil Kumaran http://uthcode.sarovar.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
> > Probably I found one reason comparation for classic style class is
slower on
> > Python2.5.
> > Comparation function instance_compare() calls
PyErr_GivenExceptionMatches(),
> > and it was just flag operation on 2.4. But on 2.5, probably related to
> > introduction of BaseException,
> > it checks inherited type tuple. (ie: PyExceptionInstance_Check)
>
> I'm curious about the speed of 2.6 (trunk). I think this should have
> become faster due to the introduction of fast subtype checks (he says
> without looking at the code).
>
> n
>
Yes, I confirmed trunk is faster than 2.5.
///
// Code
import timeit
t = timeit.Timer("""
f1 < f2
""", """
class Foo:
pass
f1 = Foo()
f2 = Foo()
""")
print t.timeit(1)
///
// Result
release-maint24 0.337sec
release-maint25 0.625sec
trunk 0.494sec
//
// Result of plex_test2.py
release-maint24 2.944sec
release-maint25 4.026sec
trunk 3.625sec
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RFC] urlparse - parse query facility
> a) import cgi and call cgi module's query_ps. [circular imports] or > b) Implement a stand alone query parsing facility in urlparse *AS IN* > cgi module. Assuming (b), please remove the (code for the) parsing from the cgi module, and just import it back from urlparse (or urllib). Since cgi already imports urllib (which imports urlparse), this isn't adding any dependencies -- but it keeps the code in a single location. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
On 6/13/07, ocean <[EMAIL PROTECTED]> wrote:
> > Meanwhile I tried to replace the parsing I did with Plex by re.Scanner.
> And
> > again there is a remarkable speed difference. Again python2.5 is slower:
> >
> > try:
> > from re import Scanner
> > except:
> > from sre import Scanner
> >
> > pars = {}
> > order = []
> > count = 0
> >
> > def par(scanner,name):
> > global count, order, pars
> >
> > if name in ['caller','e','pi']:
> > return name
> > if name not in pars.keys():
> > pars[name] = ('ns', count)
> > order.append(name)
> > ret = 'a[%d]'%count
> > count += 1
> > else:
> > ret = 'a[%d]'%(order.index(name))
> > return ret
> >
> > scanner = Scanner([
> > (r"x", lambda y,x: x),
> > (r"[a-zA-Z]+\.", lambda y,x: x),
> > (r"[a-z]+\(", lambda y,x: x),
> > (r"[a-zA-Z_]\w*", par),
> > (r"\d+\.\d*", lambda y,x: x),
> > (r"\d+", lambda y,x: x),
> > (r"\+|-|\*|/", lambda y,x: x),
> > (r"\s+", None),
> > (r"\)+", lambda y,x: x),
> > (r"\(+", lambda y,x: x),
> > (r",", lambda y,x: x),
> > ])
> >
> > import profile
> > import pstats
> >
> > def run():
> > arg = '+amp*exp(-(x-pos)/fwhm)'
> > for i in range(100):
> > scanner.scan(arg)
> >
> > profile.run('run()','profscanner')
> > p = pstats.Stats('profscanner')
> > p.strip_dirs()
> > p.sort_stats('cumulative')
> > p.print_stats()
>
> Well, I tried this script, there was no big difference.
> Python2.4 0.772sec
> Python2.5 0.816sec
>
> Probably I found one reason comparation for classic style class is slower on
> Python2.5.
> Comparation function instance_compare() calls PyErr_GivenExceptionMatches(),
> and it was just flag operation on 2.4. But on 2.5, probably related to
> introduction of BaseException,
> it checks inherited type tuple. (ie: PyExceptionInstance_Check)
I'm curious about the speed of 2.6 (trunk). I think this should have
become faster due to the introduction of fast subtype checks (he says
without looking at the code).
n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] sys.setdefaultencoding() vs. csv module + unicode
I'm seeing conflicting opinions on whether to put
sys.setdefaultencoding('utf-8') in sitecustomize.py or not ([1] vs.
[2]) and frankly I'm confused.
The csv module says it's not unicode safe but the 2.5 docs [3] have a
workaround for this. While the workaround says nothing about
sys.setdefaultencoding() it simply does not work with the default
encoding, "ascii." Is this _the_ problem with the csv module? Should
I give up and use XML? Below is code that works vs. code that
doesn't. Am I interpretting the workaround from the docs wrong? If
so, can someone please give me a hint ;) I should also point out that
I've tried this with the StringIO queued approach (from the
workaround) but that doesn't solve anything.
1) with the default encoding :
kumar$ python2.5
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
>>> import sys, csv, codecs
>>> f = codecs.open('unicsv.csv','wb','utf-8')
>>> w = csv.writer(f)
>>> w.writerow([u'lang', u'espa\xa4ol'])
Traceback (most recent call last):
File "", line 1, in
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa4' in
position 4: ordinal not in range(128)
>>>
2) with custom encoding :
kumar$ python2.5 -S
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
>>> import sys, csv, codecs
>>> sys.setdefaultencoding('utf-8')
>>> f = codecs.open('unicsv.csv','wb','utf-8')
>>> w = csv.writer(f)
>>> w.writerow([u'lang', u'espa\xa4ol'])
>>> f.close()
thanks, Kumar
[1] http://mail.python.org/pipermail/python-dev/2007-June/073593.html
[2] http://diveintopython.org/xml_processing/unicode.html
[3] http://docs.python.org/lib/csv-examples.html#csv-examples
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
> Meanwhile I tried to replace the parsing I did with Plex by re.Scanner.
And
> again there is a remarkable speed difference. Again python2.5 is slower:
>
> try:
> from re import Scanner
> except:
> from sre import Scanner
>
> pars = {}
> order = []
> count = 0
>
> def par(scanner,name):
> global count, order, pars
>
> if name in ['caller','e','pi']:
> return name
> if name not in pars.keys():
> pars[name] = ('ns', count)
> order.append(name)
> ret = 'a[%d]'%count
> count += 1
> else:
> ret = 'a[%d]'%(order.index(name))
> return ret
>
> scanner = Scanner([
> (r"x", lambda y,x: x),
> (r"[a-zA-Z]+\.", lambda y,x: x),
> (r"[a-z]+\(", lambda y,x: x),
> (r"[a-zA-Z_]\w*", par),
> (r"\d+\.\d*", lambda y,x: x),
> (r"\d+", lambda y,x: x),
> (r"\+|-|\*|/", lambda y,x: x),
> (r"\s+", None),
> (r"\)+", lambda y,x: x),
> (r"\(+", lambda y,x: x),
> (r",", lambda y,x: x),
> ])
>
> import profile
> import pstats
>
> def run():
> arg = '+amp*exp(-(x-pos)/fwhm)'
> for i in range(100):
> scanner.scan(arg)
>
> profile.run('run()','profscanner')
> p = pstats.Stats('profscanner')
> p.strip_dirs()
> p.sort_stats('cumulative')
> p.print_stats()
Well, I tried this script, there was no big difference.
Python2.4 0.772sec
Python2.5 0.816sec
Probably I found one reason comparation for classic style class is slower on
Python2.5.
Comparation function instance_compare() calls PyErr_GivenExceptionMatches(),
and it was just flag operation on 2.4. But on 2.5, probably related to
introduction of BaseException,
it checks inherited type tuple. (ie: PyExceptionInstance_Check)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] TLSAbruptCloseError
On Tue, Jun 05, 2007 at 12:55:07PM -0400, Todd Hopfinger wrote: >I am using TLS Lite and J2ME SecureConnection for the purposes of >encrypting traffic to/from a Java Midlet client and a multithreaded Python >server. However, I encounter a TLSAbruptCloseError. I have tried to >determine the cause of the exception to no avail. I understand that it has >to do with close_notify alerts. My abbreviated code follows. It may or may not be your specific problem, but Microsoft SSL servers tend to just drop the TCP connection when they're done, rather than do a proper SSL shutdown. This tends to make errors such as the above, which you must then ignore. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Instance variable access and descriptors
Hi, On Tue, Jun 12, 2007 at 08:10:26PM +1200, Greg Ewing wrote: > Rather than spend time tinkering with the lookup order, > it might be more productive to look into implementing > a cache for attribute lookups. See patch #1700288. Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
