Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?
On 9 Jul 2004 at 21:17, Dieter Maurer wrote: > Check whether the connection is maintained in a "_v_" attribute > (in this case, it is per thread). > Otherwise, the connection would need to be maintained in a module > global registry indexed with a DA id (e.g. its ZODB path). This is the deep magic of which my previous note speaks. Where are these details documented (in general terms, not just about DA's), so that I can figure this out without bugging the list too much? -- Brad Clements,[EMAIL PROTECTED] (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.7.0 and Pythom Image Library
John Ziniti wrote at 2004-7-6 20:44 -0400: > ... >I've had problems using PIL-based products in Zope because Zope >has its own module called "ImageFile" and Zope does some sys.path >munging so that when certain PIL modules do "import ImageFile", >they import the Zope version, instead of the ImageFile.py that >comes with PIL. This causes lots of hard-to-track errors that >have "weird" error messages ( I always get "BadFont" ). I do not think this is a Zope fault... We can use Zope and PIL together without any problem and without the need to change either Zope nor PIL. >I have "fixed" this problem before by editing the module for >the type of image I am using to do a more explicit import. For >example, for JPEG images, I edit PIL/JpegImagePlugin.py to >read: > >from PIL import ImageFile Provided that "PIL.JpegImagePlugin" was imported via the "PIL" package, "import ImageFile" will work as well -- without any danger to clash with any other "ImageFile"... The problem almost surely is that PIL provides for accessing PIL modules without going through the PIL package *AND* your application makes use of this possibility. Always access PIL modules in your application code via "PIL.module" and never via "module". -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] How to add sections to zope.conf?
Chris Withers wrote at 2004-7-4 09:19 +0100: >Say I have a process which needs some configuration but which is based on zope. >How do I add arbitary sections into zope.conf at the top level and then how do I >get hold of the configuration? > >What I'd like to do (and tried!) is to write a component.xml, put it in my >package and then add a section to the top-level of my zope.conf something like: > >%import mypackage > >somekey somevalue > > >...and then get hold of it with: > >from App.config import getConfiguration >config = getConfiguration().mypackage_config > >...but this doesn't seem to work. Am I doing something wrong or do their need to >be changes to zopeschema.xml to allow this? You must extend the "zopeschema.xml" or let your process use a schema derived from "zopeschema.xml" (schema support some form of inheritance). -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?
Brad Clements wrote at 2004-7-9 09:35 -0400: >I am using ZsapdbDA with SAPDB (now MaxDB) and have run into problems when Zope >has more than one thread. > >The folks at SAP suggest that there should be one database connection per thread, >rather >than sharing one connection across threads. This is the default for DAs as it is much easier to implement one connection per thread than one connection per DA instance... >I haven't yet looked at the Zsapdb source, but I think it has only one connection for >all >threads. Then do look at the source. I would be astonished when you were right... Check whether the connection is maintained in a "_v_" attribute (in this case, it is per thread). Otherwise, the connection would need to be maintained in a module global registry indexed with a DA id (e.g. its ZODB path). -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: "hasattr" geddon
On Fri, 9 Jul 2004 16:22:17 +0200 Dieter Maurer <[EMAIL PROTECTED]> wrote: [..] > The "hasattr" replacement in Python's "__builtin__" could look like: > > _marker = [] > def hasattr(obj, attr): > return getattr(obj, attr, _marker) is not _marker > > > Opinions? +1 Another advantage to a global patch is that 3rd party products (such as CMF) would benefit automatically. I noticed yesterday some hasattr calls in DCWorkflow that could cause inaccurate results if the hasattr returned false due to a ConflictError. It might be worth profiling this and an alternate implementation in C to see if the latter is worthwhile. I can't imagine the C implementation would be all that complex. That would dispense with any performance disadvantage of the patch. -Casey ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: cStringIO strange errors
Tor Oskar Wilhelmsen wrote: Hi I'm using the code below to fetch an image from a database, and for inserting the image as an Image object in ZOPE. The strange thing is that the code causes python to crash, but sometimes it work (for 1 of 10 or something like that). Im using Python 2.3.3 for Windows and Zope 2.7.0 for windows. I don't get any indication about what causing the error, but testing leaves me with an error on the code below. Does anyone know what I have done wrong? imthumb = PIL.Image.open(StringIO(pic[3]))#pic[3] is the blob from the db imthumb.thumbnail(tsize) btfil = StringIO() imthumb.save(btfil,"JPEG") btfil.seek(0) #thumbnail title thumbnailtitle = str(pic[1]) self.manage_addProduct['OFSP'].thumbnails.manage_addImage(thumbnailid,btfil,thumbnailtitle,"","image/jpeg") btfil.close() I suggest you run Zope under gdb, as per: http://zope.org/Members/matt/StabilityHOWTO which should give you some sense of what in particular is crashing. I also don't know what that '.thumbnails' bit is doing in your invocation of the product / factory dispatcher. If the intent is to have the image created in the 'thumbnails' subfolder of 'self', then I would spell your invocation as: dispatcher = self.thumbnails.manage_addProduct['OFSP'] dispatcher.manage_addImage( ) Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation "Zope Dealers" http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] "hasattr" geddon
Yesterday, I upgraded to Zope 2.7.1. Because several of our extensions broke, I have to look into detail at the changed and I saw that the "hasattr geddon" was started. Unfortunately, in a way that I feel as not optimal and error prone... In the module I saw, "if hasattr(obj, attr)" has been replaced by "if getattr(obj, attr, None)". This is neither general (in the special case, it was at least not false), nor optimal (in the special case, the boolean check was very expensive) nor clear. I propose to implement the "hasattr geddon" instead by a monkey patch for Python's builtin "hasattr". This has the following advantages: * no code changes needed with the exception of the monkey patch installation * the code is as clear as possible * the implementation can be made efficient and safe * no danger of missed "hasattr" occurrences Disadvantages: * Occasionally, a "hasattr" call might become slightly more inefficient than necessary (when "hasattr" operates on an object which is not persistent). * the usual problems with monkey patches (if someone else tries to monkey patch "hasattr", too) The "hasattr" replacement in Python's "__builtin__" could look like: _marker = [] def hasattr(obj, attr): return getattr(obj, attr, _marker) is not _marker Opinions? -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] cStringIO strange errors
Hi I'm using the code below to fetch an image from a database, and for inserting the image as an Image object in ZOPE. The strange thing is that the code causes python to crash, but sometimes it work (for 1 of 10 or something like that). Im using Python 2.3.3 for Windows and Zope 2.7.0 for windows. I don't get any indication about what causing the error, but testing leaves me with an error on the code below. Does anyone know what I have done wrong? imthumb = PIL.Image.open(StringIO(pic[3])) #pic[3] is the blob from the dbimthumb.thumbnail(tsize) btfil = StringIO()imthumb.save(btfil,"JPEG")btfil.seek(0)#thumbnail titlethumbnailtitle = str(pic[1])self.manage_addProduct['OFSP'].thumbnails.manage_addImage(thumbnailid,btfil,thumbnailtitle,"","image/jpeg")btfil.close() - Tor Oskar Wilhelmsen ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Streaming large files
thanks a lot for your answeres! I think switching to Zope 2.7.1 and using IStreamIterators did the job for me :-) thilo Thilo Staebler wrote: hi! I have to stream large mpeg-files over http with range support. this works quite nice until the client seeks in the (media)player and the files aren't to large, because zope caches the whole file... ...so each seek means streaming the whole file from the requested position till end *g* -> my RESPONSE object doesn't even know, that the player has seeked e.g. back to the beginning... is there a way to bypass the zope caching? and how do I know if the player has disconnected, so I can stop streaming? thanks in advance thilo ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Streaming large files
On Thursday, July 8, 2004, at 01:07 PM, Thilo Staebler wrote: hi! I have to stream large mpeg-files over http with range support. this works quite nice until the client seeks in the (media)player and the files aren't to large, because zope caches the whole file... ...so each seek means streaming the whole file from the requested position till end *g* -> my RESPONSE object doesn't even know, that the player has seeked e.g. back to the beginning... is there a way to bypass the zope caching? and how do I know if the player has disconnected, so I can stop streaming? you mean cacheing in memory? Anything based on "File" does not do that. thanks in advance thilo ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?
sathya wrote: we have been looking at psycopg for postgres. zope opens atleast one connection per zope thread. I have obeserved the same for mysql as well. Are you creating your own threads within zope ? in which case you have to do the locking yourself Brad Clements wrote: I am using ZsapdbDA with SAPDB (now MaxDB) and have run into problems when Zope has more than one thread. The folks at SAP suggest that there should be one database connection per thread, rather than sharing one connection across threads. I haven't yet looked at the Zsapdb source, but I think it has only one connection for all threads. Is there a document somewhere that explains how to make variables "thread private" in Zope? I vaguely recall something about one cache per thread, or that variables are read- only shared unless updated by a thread, and then they're copied.. This is probably quite wrong, but obviously I just don't understand how data and threading works in Zope. So, I'm looking for a document that explains this clearly. How to have data that is not shared between threads (perhaps a kind of connection pool thingy). Also, I have other projects where I really want to have just one object of it's kind no matter how many threads. For example, gvibDA uses Thunked_TM for this, but that doesn't really work with ZSQLDA because I've seen that multiple connect calls are made, even with the THUNKED_TM. Is there a "Zen Of" document about this somewhere? Thanks ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Making a ZSQL.DA fully multi-threaded?
I am using ZsapdbDA with SAPDB (now MaxDB) and have run into problems when Zope has more than one thread. The folks at SAP suggest that there should be one database connection per thread, rather than sharing one connection across threads. I haven't yet looked at the Zsapdb source, but I think it has only one connection for all threads. Is there a document somewhere that explains how to make variables "thread private" in Zope? I vaguely recall something about one cache per thread, or that variables are read- only shared unless updated by a thread, and then they're copied.. This is probably quite wrong, but obviously I just don't understand how data and threading works in Zope. So, I'm looking for a document that explains this clearly. How to have data that is not shared between threads (perhaps a kind of connection pool thingy). Also, I have other projects where I really want to have just one object of it's kind no matter how many threads. For example, gvibDA uses Thunked_TM for this, but that doesn't really work with ZSQLDA because I've seen that multiple connect calls are made, even with the THUNKED_TM. Is there a "Zen Of" document about this somewhere? Thanks -- Brad Clements,[EMAIL PROTECTED] (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] TALParser barfing on byte-order marked utf8 XML files.
Yo,
We are using TAL for things other than ZPT. but are having problems with
files that include a BOM preamble.
the problem is that althought the underlying XML parser is capable of
parsing these kind of files, TALParser initialises his parent without
encoding (XMLParser.__init__(self) in TALParser.py line 27)
Anyway,
I have attached a small example (test.py + test.ml) that illustrates
the problem with Zope 2.7.1.
running the test gives:
UnicodeEncodeError: 'ascii' codec can't encode character u'\ufeff' in
position 0: ordinal not in range(128)
which is perfectly logical: feff (the start of the bom preamble) is not
ascii.
chipping away the preamble (data=data[4:] ) gives problems further on in
the file as the test example has some german characters (ä)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in
position 50: ordinal not in range(128) which is also perfectly logical:
ä has code 132.
My question is simply: why is TALParser not taking the encoding into
acount ? Is this deliberate, or is it an oversight ?
Romain Slootmaekers.

Archäologe Paläo-Anthropologie
#
#
#
from xml.dom.minidom import parseString
import sys
from TAL.TALParser import TALParser
from TAL.TALInterpreter import TALInterpreter
from TAL.DummyEngine import DummyEngine
import StringIO
import codecs
print sys.getdefaultencoding()
def readData():
f = open('test.xml','r')
readerClass = codecs.getreader('utf8')
print readerClass
reader = readerClass(f)
data = reader.read()
f.close()
print "size = %s" % len(data)
return data
def expand(xml):
parser = TALParser()
xml = xml[4:]
parser.parseString(xml)
program, macros = parser.getCode()
engine = DummyEngine(0)
out = StringIO.StringIO()
interpreter = TALInterpreter(program,macros,engine,stream=out)
interpreter()
result = out.getvalue()
return result
data = readData()
expanded = expand(data)
document = parseString(expanded)
print "ok"___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Streaming large files
Thilo Staebler wrote: hi! I have to stream large mpeg-files over http with range support. this works quite nice until the client seeks in the (media)player and the files aren't to large, because zope caches the whole file... ...so each seek means streaming the whole file from the requested position till end *g* -> my RESPONSE object doesn't even know, that the player has seeked e.g. back to the beginning... is there a way to bypass the zope caching? and how do I know if the player has disconnected, so I can stop streaming? thanks in advance thilo You may want to take a look at Railroad project from Infrae. http://infrae.com/products/railroad/ -- Godefroid Chapelle (aka __gotcha)http://bubblenet.be ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
