Re: [Zope] unicodedecoderror in page templates
According to Andreas Jung:
> [...]
> >> --On 20. Mai 2007 14:20:38 +0200 Einar Næss Jensen
> >> <[EMAIL PROTECTED]> wrote:
> >>
> >> > When I get errors like:
> >> > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
> >> > 2940: ordinal not in range(128)
> >> > how can I locate the character which causes the error?
> >> > In particular: where is the position (2940 in the example)?
>
> [...]
>
> As Peter wrote: you are likely mixing somewhere different encodings with
> unicode or something similar. Which Zope version are you using?
> If 2.10 then you might check at the UnicodeConflictResolver as introduced
> in Zope 2.10 (check the release notes). Otherwise ensure that you are not
> mixing unicode strings with non-unicode strings.
With Zope < 2.10 you can try the following recipe to get right (ie
wrong) part of the template:
http://zwiki.org/UnicodeDecodeErrors
\wlang{}
--
[EMAIL PROTECTED]Fax: +43/1/31336/9207
Zentrum fuer Informatikdienste, Wirtschaftsuniversitaet Wien, Austria
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] POST sucks.
According to Paul Winkler:
> Been doing web services (XML-RPC) stuff lately.
> What suggestions do people have for troubleshooting a zope-based
> XML-RPC application?
>
> Access and trace logs are totally useless, since even the method name is
> hidden in the POST payload, so I feel like I'm flying blind; when a
> client developer tells me he gets occasional hangs, I have no idea what
> kind of request triggers the hang.
>
> I may end up sprinkling my code with a lot more logging calls than I'm used
> to. Any other suggestions?
I found it very useful to log every xml-rpc call (call and return
value). For this we have patched ZPublisher/xmlrpc.py (and one line in
ZPublisher/HTTPRequest.py). See attachment (tested against 2.9.3).
The log entries look like that:
Sep 10 06:58:40 bach-s10 xmlrpc id=-177841652, client=137.208.89.23, \
user=stupl, inst=bach-s10.wu-wien.ac.at:10082, path=/bach/stupl/RPC, \
call=get_adreptl3(2002,)
Sep 10 06:58:40 bach-s10 xmlrpc id=-177841652, client=137.208.89.23, \
user=stupl, inst=bach-s10.wu-wien.ac.at:10082, time=88.27ms, out-ok, \
ret=[('E', 19007, None, 7066351, 1, None), ('E', 19010, None, ...
\wlang{}
--
[EMAIL PROTECTED]Fax: +43/1/31336/9207
Zentrum fuer Informatikdienste, Wirtschaftsuniversitaet Wien, Austria
-included file follows-
--- xmlrpc.py.ori 2006-05-12 13:53:45.0 +0200
+++ xmlrpc.py 2006-09-12 18:52:39.094041280 +0200
@@ -21,11 +21,56 @@
import re
import sys, types
+import time
from HTTPResponse import HTTPResponse
import xmlrpclib
+import logging
+logger = logging.getLogger('event.xmlrpc')
+logger.setLevel(logging.INFO)
+hdlr = logging.handlers.SysLogHandler('/dev/log')
+logger.addHandler(hdlr)
from zExceptions import Unauthorized
+def log_before(request, method, response):
+# PATH_INFO, or PATH_TRANSLATED ???
+#url = request.base + request.environ['PATH_INFO']
+path = request.environ.get('PATH_INFO')
+host_port = request.environ.get('HTTP_HOST')
+user = request._authUserPW() # returns (user,pw) tuple or None
+if user:
+user = user[0]
+response.xmlrpc_timestamp = time.time()
+idstring = 'xmlrpc id=%u, client=%s, user=%s, inst=%s' % (
+ id(response), request._client_addr, user, host_port)
+args = '%s' % (request.args,)
+if len(args) > 200: # syslog line limit: 1024
+sargs = []
+# XXX
+logger.info(
+ "%s, path=%s, call=%s%s" %
+ (idstring, path, method, request.args))
+response.xmlrpc_idstring = idstring
+
+def get_xmlrpc_info(response):
+ts = getattr(response, 'xmlrpc_timestamp', None)
+if ts:
+t = '%.2fms' % ((time.time() - ts) * 1000,)
+else:
+t = '-'
+idstring = (getattr(response, 'xmlrpc_idstring', None) or
+ 'xmlrpc %u' % (id(response),))
+return idstring,t
+
+def log_ok(response, ret):
+idstring, t = get_xmlrpc_info(response)
+logger.info("%s, time=%s, out-ok, ret=%s" % (idstring, t, ret))
+
+def log_exception(response, errortext):
+idstring, t = get_xmlrpc_info(response)
+logger.info(
+"%s, time=%s, out-error, exception=%s" % (idstring, t, errortext))
+
def parse_input(data):
"""Parse input data and return a method path and argument tuple
@@ -97,8 +142,11 @@
def setBody(self, body, title='', is_error=0, bogus_str_search=None):
if isinstance(body, xmlrpclib.Fault):
+txt = str(body)
+log_exception(self, txt)
# Convert Fault object to XML-RPC response.
-body=xmlrpclib.dumps(body, methodresponse=1, allow_none=True)
+body=xmlrpclib.dumps(body, methodresponse=1, allow_none=True,
+ encoding='iso-8859-1')
else:
if type(body) == types.InstanceType:
# Avoid disclosing private members. Private members are
@@ -117,8 +165,13 @@
# was a Python None. This is now patched in xmlrpclib to
# allow Nones nested inside data structures too.
try:
+ret = '%s' % (body,)
+if len(ret) > 80:
+ret = ret[:80] + '...'
body = xmlrpclib.dumps(
-(body,), methodresponse=1, allow_none=True)
+(body,), methodresponse=1, allow_none=True,
+encoding='iso-8859-1')
+log_ok(self, ret)
except:
self.exception()
return
@@ -141,6 +194,9 @@
isinstance(t, types.ClassType) and issubclass(t, Unauthorized)
):
+txt = '%s: %s' % (t, v)
+log_exception(self, txt)
+
return self._real.exception(fatal=fatal, info=info)
# Create an appropriate Fault object. Containing error information
--- HTTPRequest.py.orig 2006-05-12 13:53:45.0 +0200
+++ HTTPRequest.py 2006-06-01 10:23:57.272282896 +0200
@@ -394,6
Re: [Zope] converting bytestreams from iso-8859-1 to utf-8
According to Giuseppe Bonelli:
> sorry if this is not zope specific, but can someone please explain
> to me the following behaviour when trying to convert an iso-8859-1 string
> read from a file to an utf-8 encoded one?
>
> s='\x93test\x94' #an iso-8859-1 string
> #\x93 and \x94 are left and right
> #double quotation marks,
> #as seen in a browser set to iso-8859-1
\x93 and \x94 are *not* iso-8859-1 quotation marks. See for example
http://en.wikipedia.org/wiki/ISO_8859-1
Instead they seem to be from the Windows-125X (X=0,1,...) codepage:
http://www.microsoft.com/globaldev/reference/sbcs/1250.mspx
> ss=unicode(s,'iso-8859-1').encode('utf-8')
> gives
> ss='\xc2\x93test\xc2\x94'
> which is wrong (as seen in a browser set to utf-8)!
but:
>>> unicode(s,'cp1250').encode('utf-8')
'\xe2\x80\x9ctest\xe2\x80\x9d'
is right.
> Do I have to explicitly replace all characters above \x7F ?
No, you have to use the right encodings ;-)
\wlang{}
--
[EMAIL PROTECTED]Fax: +43/1/31336/9207
Zentrum fuer Informatikdienste, Wirtschaftsuniversitaet Wien, Austria
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Python Classes and Zope.
According to Dario Lopez-Kästen:
> But in order to even display it in a zpt I must transmogrify it into a
> special zope-object, and *that* is not so easy as I have discovered.
>
> In my case I am not so interested in importing the moduels or classes
> into a Script(Python) - I have allready passed the objects in question
> thtough my product, but still I get some "Zope Does Not Allow That" error.
You could add
__allow_access_to_unprotected_subobjects__ = 1
to your class. Then all attributes and methods of its instances should
be accessible from Zope. However, this can be a security problem, so
be careful...
\wlang{}
--
[EMAIL PROTECTED]Fax: +43/1/31336/9207
Zentrum fuer Informatikdienste, Wirtschaftsuniversitaet Wien, Austria
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Visibility and Copy&Paste support
According to Peter Bengtsson:
> Some of these objects I don't want to encourage people to create via
> the ZMI and I don't want to clutter up the "Add new object drop down"
> with stuff that can't be added.
>
> If I register the class, but set 'visibility=None' I still can't
> copy/cut & paste the instanciated objects. Why? What's the point of
> the 'visibility=None' parameter?
As this has bitten me the last time...
The method
CopySupport.py: CopyContainer._verifyObjectPaste
is responsible for that. It refuses to paste objects with meta_types,
which are not found in the "all_meta_types" attribute of the target
object.
And yes, I find it (at least) irritating, that an object cannot be
pasted, only because it isn't in the product add list
(visibility=None). And the error message (when pasting) does not help
either:
The object ... does not support this operation.
Is there a way to hide products from the product add list without
losing the ability to paste its corresponding objects?
\wlang{}
--
[EMAIL PROTECTED]Fax: +43/1/31336/9207
Zentrum fuer Informatikdienste, Wirtschaftsuniversitaet Wien, Austria
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO troubles on RedHat EL4 Linux
According to Tim Peters:
> I don't know. Dieter asked whether you ran the tests via "zopectl
> test", but I didn't see an answer to that.
Ok, here some data points...
bender:~/Zope-2.7.7-final$ cat /proc/version
Linux version 2.6.9-11.ELsmp ([EMAIL PROTECTED]) (gcc version 3.4.3 20050227
(Red Hat 3.4.3-22)) #1 SMP Fri May 20 18:26:27 EDT 2005
bender:~/Zope-2.7.7-final$ python2.3
Python 2.3.5 (#1, Apr 19 2005, 14:53:39)
[GCC 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Running one single test:
bender:~/Zope-2.7.7-final$ python2.3 test.py testConnection
checkNoVerificationOnServerRestart\$
Running unit tests from /home/wlang/Zope-2.7.7-final/lib/python
==
ERROR: checkNoVerificationOnServerRestart
(ZEO.tests.testConnection.FileStorageReconnectionTests)
--
Traceback (most recent call last):
File
"/home/wlang/Zope-2.7.7-final/lib/python/ZEO/tests/ConnectionTests.py", line
121, in tearDown
os.waitpid(pid, 0)
OSError: [Errno 10] No child processes
--
Ran 1 test in 0.689s
FAILED (errors=1)
After some retries, the same test passes:
bender:~/Zope-2.7.7-final$ python2.3 test.py testConnection
checkNoVerificationOnServerRestart\$
Running unit tests from /home/wlang/Zope-2.7.7-final/lib/python
--
Ran 1 test in 0.691s
OK
Interesstingly, if i run the test with strace, i never see the test
fail (i tried at least 30 times):
bender:~/Zope-2.7.7-final$ strace -e trace=signal -o /var/tmp/zeotest.trc
python2.3 test.py testConnection checkNoVerificationOnServerRestart\$
Running unit tests from /home/wlang/Zope-2.7.7-final/lib/python
--
Ran 1 test in 0.710s
OK
(Obviously a Heisenberg effect -- the observation influences the
behaviour ;-)
If anyone is interessted in the trace file -- it can be found at:
http://slime.wu-wien.ac.at/misc/zeotest.trc
(However, it would be way more interessting to see the syscalls while
the test is failing...)
Also, i debugged the whole test with the python debugger. Unfortunatly
(as with strace), i was not able to reproduce the failing of the test
in the debugger.
> the ZEO tests spawn processes directly via Python's
> os.spawnve(), and later waits for them to end, via the waitpid() code
> shown earlier. It doesn't muck around with signals, forks, or
> anything else that should be platform-dependent (the same ZEO-test
> process code is used on both Linux and Windows, BTW -- for this
> reason, it can't rely on any fancy signal or process gimmicks;
> spawnve+watipid is the entire story here).
Yes, its as simple as that: zeo ist started, zeo is stopped, and when
the parent calls waitpid, we get the "No child processes" error most of
the time :-(
Any ideas what we can try to narrow this down?
> All the failures you showed were in test teardown. If that's all the
> failures you got, then all the test bodies actually passed. Of course
> you have to be wary that normal methods of detecting child-process
> termination aren't working as hoped on this box, because all the test
> failures you reported were exactly failures to detect child-process
> termination.
Sure -- we could just make this change:
bender:.../ZEO/tests$ diff ConnectionTests.py.ori ConnectionTests.py
121c121,124
< os.waitpid(pid, 0)
---
> try:
> os.waitpid(pid, 0)
> except OSError:
> pass
then all tests will pass. But then we will not know why the zeo zombie
vanishes before the waitpid can reap the exit code ;-)
\wlang{}
PS: i'am afraid it turns out to be a python thread / signals / race
problem -- yuck!
--
[EMAIL PROTECTED]Fax: +43/1/31336/9207
Zentrum fuer Informatikdienste, Wirtschaftsuniversitaet Wien, Austria
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
