[Zope-CMF] Action expression context
Hello! A long time ago, I proposed a change so that applications could customise the expression context. http://www.mail-archive.com/zope-...@lists.zope.org/msg06079.html I've only just got round to making the change, and was going to check it in - as it was so long ago, I thought I'd flag it up first in case opinions have changed. Also, I have an opportunity to play a bit with zope 3, which I haven't looked at before. I've only looked at it for a few minutes, but was wondering - is there is a library that basically provides the same APIs/interfaces as CMF does in zope 2? I'm basically just grasping around for some way to get going quickly with updating some existing apps ... but suspect there aren't any (shortcuts that is, I have plenty of existing apps) :-/ Cheers, Miles ___ Zope-CMF maillist - Zope-CMF@zope.org https://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-dev] HA storage for zope
Is there anything else worth considering? Dedicated HA-hardware. We are using a SUN Cluster for all of our backend service like ZEO, Oracle, Postgres and LDAP. I'm interested in how that works specifically with zeo. how does it ensure the clustered Zeos don't try to write conflicting transactions at the same time? Or is it set up for another service to start on failover, rather than a load-balanced arrangement? Thanks Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] HA storage for zope
Hi Jens, * ** I'm looking at a HA setup for a project, and was wondering what the ** current best way forward would be. There seem to be a few potential ** options around for the storage end of the setup, and I'm wondering ** what's now considered the current best practice for this sort of ** setup. *[...] * Is there anything else worth considering? * the classics: DRBD and Heartbeat (for the zeo-server). Are you using this at the moment? My only worry would be that it can take zeo a while to start up. do you ever have any problems when it fails over, as the instances can't connect to anything? (in the past we've had some issues when error messages like these make it into the cache layer for some reason) Thanks, Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] HA storage for zope
Hi, I'm looking at a HA setup for a project, and was wondering what the current best way forward would be. There seem to be a few potential options around for the storage end of the setup, and I'm wondering what's now considered the current best practice for this sort of setup. Specifically, I've come up with three options: 1. RelStorage Using this, I think I can then take care of replication/mirroring as I have access to a database that is already clustered in a HA environment. My questions are: + Are the connections opened only when zope is started? Say I unplugged a network cable and then plugged it back in again (breaking the database connection) - will it be re-opened? + How does RelStorage take care of the blob storage? + Are there any details of big sites out there that use RelStorage (particularly on Oracle)? 2. ZeoRAID I could use ZeoRAID to write to maintain several independent storages subject to stopping the ZeoRAID server being the single point of failure (I posted some questions about this separately). 3. ZRS If I'm right, ZRS still has a limitation in that there is a single server for writes. Also, budget may be an issue. Having read the factsheet, I'm a bit unsure as to how it is functionally different from ZeoRAID - can anyone explain? I also thought of using DirectoryStorage to take care of replication/mirroring at the hardware level, for example using a clustered SAN, but I think this would not work because the actual directory tree can only be writeable by one zope at a time. Am I right about this? Is there anything else worth considering? I'd be grateful if anyone operating a HA-setup for their zope site, could post your setup and experiences? To me, out of the above options, RelStorage looks to be the most promising, simply because it will involve the least setup and the database clustering and tools are pretty robust and well-understood by the sql experts here. Thanks in advance for your help, I will post a write-up once it's done, Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope] ZeoRAID usage in production
Hi, This is mainly a question for Christian, but I'm cc'ing the list as it will be useful knowledge to be generally available. I was looking at this product and thinking about using it in an upcoming assignment. From the docs: Keep backups of your data. ZEORaid has good unit test coverage but hasn't seen live action yet, so keep this in mind. - is this still the case? Is anyone else using it at the moment at all for demanding sites? Could you post some stats, experiences and any gotchas at all? How does ZeoRAID cope with the blob storage? Also, what's the timeout for connecting to the zeo storages? Also, I read this bit in the docs: The RAID server itself remains the last single point of failure in the system. This problem is solved as the RAID server does not maintain any persistent state (except the configuration data: it's listening ip and port and the list of storages). The RAID server can be made reliable by providing a hot-spare server using existing HA tools (taking over the IP when a host goes down) and the existing ZEO ClientStorage behaviour. Is it possible to provide a pool of ZeoRAID processes, each talking to the same zeo storages, and put the list of ZeoRAID processes into the client storage configuration so it will round-robin if one is not available? Many thanks for your help, Miles ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Conflict error
Hi I have a PAS plugin that pulls user properties from a sql database. Intermittently, I get conflicterrors from the plugin when zope commits its transaction - even though no db changes have been made. Frustratingly, there is very little information in the traceback of actual use, other than the name of the class itself. Is there an easy way to identify what's keeping the database in the transaction so I can correct it? Thanks Miles ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope-dev] zodb session errors
Hi, We have a site that we've recently upgraded from 2.7.6 to 2.8.8 which has started exhibiting higher-than-average session errors. The site consists of 2 zeo clients on 2 separate machines, with a shared session tempstorage mounted via the zeo server. I expect some errors with sessions as a matter of course, but was trying to make the user experience a little less 'severe'. Traceback (innermost last): * Module ZPublisher.Publish, line 114, in publish snip template-rendering related traceback * Module DocumentTemplate.DT_Util, line 197, in eval __traceback_info__: portal_basket * Module string, line 1, in expression * Module Products.WFPlugins.ShopTool.BasketTool, line 74, in oneLineDetails * Module Products.WFPlugins.ShopTool.BasketTool, line 54, in getBasket line 54 reads: sess = self.session_data_manager.getSessionData(create=create) * Module Products.Sessions.SessionDataManager, line 93, in getSessionData * Module Products.Sessions.SessionDataManager, line 180, in _getSessionDataObject * Module Products.Transience.Transience, line 846, in new_or_existing * Module Products.Transience.Transience, line 449, in __setitem__ KeyError: 1163679740 I don't understand is what's happening here (and in turn what to do about it). It looks as if the session machinery detects there is not any sessiondata set up for this particular browser, goes to set one up and then can't find the current timeslice to put the new sessiondata into. Does this mean that the timeslice has changed in-between working it out and assigning the sessiondata to it? Or that the timeslice has been garbage-collected in-between? Also, can the problem be solved (without side-effects!) by changing the period to match the timeout time - i.e. so it can always find the current timeslice? Thanks for any wisdom! Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: zodb session errors
Hi, More info: I have a hunch that this might have been due to out-of-sync clocks on the two machines (turns out they were 5 minutes out). I have adjusted the clocks and will check the error logs. Specifically, I am imagining that this would cause session data created on one machine to be mysteriously lost at certain times on the other machine when it looks in the wrong timeslice. I'm not sure why it wouldn't be possible to create new sessiondata as below though - is this explanation a possibility? Miles Miles Waller wrote: Hi, We have a site that we've recently upgraded from 2.7.6 to 2.8.8 which has started exhibiting higher-than-average session errors. The site consists of 2 zeo clients on 2 separate machines, with a shared session tempstorage mounted via the zeo server. I expect some errors with sessions as a matter of course, but was trying to make the user experience a little less 'severe'. Traceback (innermost last): * Module ZPublisher.Publish, line 114, in publish snip template-rendering related traceback * Module DocumentTemplate.DT_Util, line 197, in eval __traceback_info__: portal_basket * Module string, line 1, in expression * Module Products.WFPlugins.ShopTool.BasketTool, line 74, in oneLineDetails * Module Products.WFPlugins.ShopTool.BasketTool, line 54, in getBasket line 54 reads: sess = self.session_data_manager.getSessionData(create=create) * Module Products.Sessions.SessionDataManager, line 93, in getSessionData * Module Products.Sessions.SessionDataManager, line 180, in _getSessionDataObject * Module Products.Transience.Transience, line 846, in new_or_existing * Module Products.Transience.Transience, line 449, in __setitem__ KeyError: 1163679740 I don't understand is what's happening here (and in turn what to do about it). It looks as if the session machinery detects there is not any sessiondata set up for this particular browser, goes to set one up and then can't find the current timeslice to put the new sessiondata into. Does this mean that the timeslice has changed in-between working it out and assigning the sessiondata to it? Or that the timeslice has been garbage-collected in-between? Also, can the problem be solved (without side-effects!) by changing the period to match the timeout time - i.e. so it can always find the current timeslice? Thanks for any wisdom! Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Moving ZopeVersionControl to subversion?
I noticed that Tres was making changes in CVS and preparing a 0.3.2 release, so there is at least some activity. That was me ... I badgered Tres into incorporating a patch I'd put into the tracker. Before that there was no activity. It definitely doesn't have a maintainer. There are clues that there are people out there using it, but noone has admitted so 'in the open'! Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Collector issue 2137
Hi, I reported a bug back in the summer, and submitted patches and tests for it: http://www.zope.org/Collectors/Zope/2137 The bug was in ZopeVersionControl, which is in the CVS repository. Does anyone know who is responsible for this code? I think there are a few users of the product (Plone Versioning) and it is a dependency of CMFStaging which is in the SVN repository. I'd really like to get this fixed, as there's no easy workaround for it other than manually changing the code (which I want to avoid because I forget to do it). I'm a bit frustrated as the only progress the issue has generated so far was a discussion about whether or not the issue belonged in the Zope Collector. Please, please, please - is anyone able to assist? Thanks, Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope] upgrade from 2.7.5 to 2.8.6
Hi, I have just upgraded the staging/test version of a large site from 2.7.5 to 2.8.6. When restarting zeo, it seemed to take a 'long' (in the most subjective sense) time to come up, and I noticed this entry in the logs: 2006-09-14T17:36:23 WARNING ZODB.FileStorage Failed to load database index: exceptions.AttributeError: type object 'BTrees._fsBTree.fsBTree' has no attribute '__basicnew__' Does anyone know what this means? Is it important/indicative of any sort of problem at all that I should be aware of? Does the storage do anything in response to this at all? When the live version is upgraded, downtime has to be at a minimum, or I will get phonecalls ... Thanks, Miles ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope-dev] port number for HTTPServer
Hi, I am trying to find out the port number for the HTTP service running in a zope instance. I tried using socket_map as used in the Control_Panel interface to report on running network services, but abandoned this because I need to get the same information when running in a debug instance as through the web. My next approach uses App.config.getConfiguration().servers to get the information fromn zope.conf directly. But a few things seem not quite right: - getConfiguration().servers gives me different lists through the debug prompt and through the web. why is this? - what is the most robust way to determine which of the listed servers is the HTTPServer? - is it ever possible that the servers list would contain more than one HTTPServer? The reason for doing this is so that zope can call out to an external program, and give it a notification url that the external program can call back on. Unfortunately I can't change this method of integration. Thanks for your help! Miles ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope] Changing class of objects in the ZODB
Hi, Does anyone have a good approach to changing the class of an object already stored in the ZODB? A site is using classes defined and customised in a product on the filesystem, and I want to split the product into two to separate the unmodified vanilla product, and customisations. I found a script from shane when I googled which purported to do this, but I did not understand exactly what it did and so am a bit nervous of using it. Also, it appeared to do it all in one transaction, which would not be practical for this site (too many objects). I had originally been envisaging doing something in __getstate__ for these objects, so when they were retrieved, they were updated and marked as having been changed so they would be committed at the end of the transaction. But I don't really know exactly what! I don't think it will work to use module_aliases because there will still be a class that can be imported from the original location. Any help much appreciated, Miles ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Changing class of objects in the ZODB
To confirm, I *just* want to change the class - attribute values, name and state information can remain exactly the same. XML import/export is a good approach which I need to think about a bit more. On the face of it, the data.fs is well over 1Gb in size after packing, and so is likely to create some issues in itself, but there may be ways to make this more straightforward. Doyon, Jean-Francois wrote: Assuming you're not changing attribute values, but *just* the class, I've had success with doing an XML export, a find/replace of the class name, and a re-import. The XML is fairly readable, you can just look for example for Products.MyProduct.OldClass and replace it with Products.MyProduct.NewClass. J.F. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: relevance ranking in ZCTextIndex or equivalent
Hi,
Thanks for the help. From my investigations, it seems it's not possible
to meet the requirements in a super-straightforward way - a query that
uses several text indexes adds each individual score together, so the
only output available is the total score.
Trying to separate the scores out (for example so it's a tuple
(title_score, description_score, body_text_score) that I can sort on)
looks quite hard - it looks like it would mean changing the indexes to
return the scores in this different format.
My latest approach is to do something like the following (untested):
from BTrees.IIBTree import difference
def specialSearch(words):
# i'm going to manipulate the indexes directly
getIndex = portal_catalog._catalog.getIndex
r1, id1 = getIndex('Title')._apply_index( {'Title':words} )
r2, id2 = getIndex('Description')._apply_index( {'Description':words} )
r3, id3 = getIndex('SearchableText')._apply_index(
{'SearchableText':words} )
# de-dupe this set of results
r3 = difference(r3, r2)
r2 = difference(r2, r1)
# now i have 3 IIBuckets, consisting of (docid, score) tuples
# i sort them into order on score
r1 = r1.byValue(0)
r2 = r2.byValue(0)
r3 = r3.byValue(0)
# concatenate them, preserving the order
res = r1 + r2 + r3
# return something catalog brain-like
return LazyMap(catalog.__getitem__, rs, len(rs))
My debug-prompt tests seem to indicate that this should work. I don't
know if anyone who knows more about lists and btrees can comment if
there's a better way to do the sorting and concatenation of the
different result sets.
Thanks,
Miles
Jonathan wrote:
- Original Message - From: Miles Waller
[EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Wednesday, May 31, 2006 10:59 AM
Subject: [Zope] relevance ranking in ZCTextIndex or equivalent
Hi,
I'm planning to implement a text search where
(match against the title)
ranks more highly than
(match in the description)
ranks more highly than
(matches against the body text).
Titles and descriptions are short bits of text, so results in these
categories can be ranked just by the frequency that the word appears in
that part of the text. Matches against the body text should ideally be
ranked more like ZCTextIndex rather than plain frequency.
My ideas are:
- do three separate searches, and then concatenate the result sets
together.
problem: making sure there are no duplicates in the list without parsing
all the results in their entirety.
- hijack the 'scoring' part of the index, so those results with matches
in the title can have their scores artificially heightened to achieve
the ordering i want
problem: it's compleletely opaque without a lot of study whether this
would achieve what i want. i'd also need to index the items so the
index knew what was in the title, which could be a problem.
- index title, description and text separately, and then use dieter's
AdvancedQuery product to do the query and combine results
problem: is it possible to get at the scores when the documents are
returned from the index to be able to order them? are the scores
returned separately, or will each query overwrite the last one?
Has anyone ever tried to do this - or got any pointers - at all?
A definitely non-trivial task, but here are some ideas to get you
pointed in the right (I hope) direction:
Try googling, or looking in the zope source for:
data_record_normalized_score_
BaseIndex.py
OkapiIndex.py
SetOps.py
okascore.c
Good Luck!
Jonathan
___
Zope maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] relevance ranking in ZCTextIndex or equivalent
Hi, I'm planning to implement a text search where (match against the title) ranks more highly than (match in the description) ranks more highly than (matches against the body text). Titles and descriptions are short bits of text, so results in these categories can be ranked just by the frequency that the word appears in that part of the text. Matches against the body text should ideally be ranked more like ZCTextIndex rather than plain frequency. My ideas are: - do three separate searches, and then concatenate the result sets together. problem: making sure there are no duplicates in the list without parsing all the results in their entirety. - hijack the 'scoring' part of the index, so those results with matches in the title can have their scores artificially heightened to achieve the ordering i want problem: it's compleletely opaque without a lot of study whether this would achieve what i want. i'd also need to index the items so the index knew what was in the title, which could be a problem. - index title, description and text separately, and then use dieter's AdvancedQuery product to do the query and combine results problem: is it possible to get at the scores when the documents are returned from the index to be able to order them? are the scores returned separately, or will each query overwrite the last one? Has anyone ever tried to do this - or got any pointers - at all? Thanks in advance, Miles ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
