Re: [Zope-dev] Catalog improvements
Andreas Jung wrote: I think the software MG from the book Managing Gigabytes is GPLed and currently released as mg-1.21. Walking through the TOC of the book, it seems to be a very detailed sources about text processing and gives very much informations about different indexes types. But I miss some explanations about current data structures like suffix arrays or suffix tree that have several advantages for text processing compared to B-Trees. Hmmm... looks like it's time ot go buy a book :-) cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Searching/Indexing/ZODB/SQL/BerkleyDB
Casey Duncan wrote: I'm not sure I want to store the indexes in the ZODB, just index ZODB data at a low level. Ah, okay, and yes, in that case, I am in complete agreement ;-) (the level I'm aiming at is just to be able to index python objects, I'll leave plugging that into the ZODB architecture up to someone who understands it better...) Yup, I think I have a solution, but it'll involve some coding ;^) Ooo...care to explain? :-) I would rather avoid having to use a relational database unless I have to. Perhaps the index pluggability could be made to support different backends (like FileStorage et al does). Yeah, unfortunately, the difficult bit is combining queries: gimme the results where index1=='fish' and index2 is between 2 and 5kg. if index1 is in SQL and index2 is in ZODB, for example, how would you go about efficiently combining results? That said, I wasn't aware of Matt's work up until very recently. I'd love to see an Indexer that didn't require an RDB (or BerkleyDB :-P) and scaled to GigaBytes of Data... Yup, me too. Well, I'm just purchasing my copy of Managing Gigabytess now ;-) OK, I'm available all this week, but I'm not as available the next two weeks. Lets find a good time. I'm available any time and date, just as long as I get a coupla days notice... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Fw: [Exuserfolder-devel] Zope 2.5b1 release
Dario Lopez-Kästen wrote: Anyway. In case you were wondering what the previous email meant, there's the actual meaning d8) I'm sure Andy's got a point, but we haven't got the context of those points. Is he complaining about the change in UserFolder API in Zope 2.5? cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Searching/Indexing/ZODB/SQL/BerkleyDB
On Thu, 29 Nov 2001, Chris Withers wrote: I would rather avoid having to use a relational database unless I have to. Perhaps the index pluggability could be made to support different backends (like FileStorage et al does). Yeah, unfortunately, the difficult bit is combining queries: gimme the results where index1=='fish' and index2 is between 2 and 5kg. if index1 is in SQL and index2 is in ZODB, for example, how would you go about efficiently combining results? Is there not a set datatype in python that could be used? Admittedly, most of the stuff in MG is about textual searches rather than exact searches (it can do boolean searches too, but the book is mainly about ranking). It uses an algorithm called the 'Cosine Ranking Algorithm'. Basically if you imagine an N-dimensional space, where N is the number of terms in your vocabulary and represent a document as a vector in that space whose direction is the composite of the terms that appear in it. You then represent a query string as a vector in the same space, the similarity between the document and the query is the angle between the two vectors... the smaller the angle the greater the similarity. Still with me? :) -Matt -- Matt Hamilton [EMAIL PROTECTED] Netsight Internet Solutions, Ltd. Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Hosting | Web Design | Domain Names | Co-location | DB Integration ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Fw: [Exuserfolder-devel] Zope 2.5b1 release
Yikes, it was pointed out to me that I typed amk instead of akm. Though I knew it was Andrew Milton and not Andrew Kuchling, I mixed up the letters. (In fact, as I was typing it, I was thinking wow, that looks like Andrew Kuchling's monogram.) --Paul Paul Everitt wrote: Whew, that email (and the preceding one in the thread) is quite a whopper. In substance, amk raises some pretty serious issues that we need to come to grips with very quickly. I don't have enough information to respond right now, but trust that we'll get a good response back today. Neo-moderator note: if the discussion _does_ start over here, everyone is advised to leave the anger at the door. Problems like this can usually be solved pretty easily by direct communication and open minds. --Paul Dario Lopez-Kästen wrote: I thought this might be intresting for discussion on zope-dev as well... /dario - Original Message - From: Andrew Kenneth Milton [EMAIL PROTECTED] To: Andrew Kenneth Milton [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Thursday, November 29, 2001 8:45 AM Subject: Re: [Exuserfolder-devel] Zope 2.5b1 release It has been pointed out that perhaps the dripping sarcasm drowned out the acutal points of the email, sorry I was particularly pissed off at how half-assed it all was. If you are writing a product that needs to create a user, you go look in User.py and find out what the API since that tends to be the only place the API is documented. If you look at how UserFolder is implemented; a) The change to manage_* seems to be completely arbitrary, since we already had _do* methods that meant you didn't have to call manage_users with fake submit buttons. So what is the point of having manage_ ? b) manage_* methods usually indicate web callable methods, which these clearly aren't (no docstring, no REQUEST parameter). In fact they don't return a status at all, so they seem to doubly useless. c) If it's an internal API for applications then the methods should probably be protected by being prefixed with an _ to indicate they're not for calling from the web or from within Documents/Templates. d) Even if you conformed to the API by calling e.g. manage_addUser() this seemed to be incorrect, since it didn't do things like encrypt the password. This meant you would have to either a) encrypt the password yourself (*very bad for XUF*), or b) call _addUser() which is not part of a defined API but looks like it should be (i.e. manage_addUser and _addUser seem to be reversed in functionality). ii) This is compounded by the fact you would have to query the UserFolder itself to find out if you should be encrypting passwords (what no UserFolder capability API?). e) s/_do/manage_/ doesn't constitute a new API. It's just the old one with the names changed. f) We've done a lot of work to make a flexible UF product that is I18Ned (well -able), and they still use the value of the submit button. I would have expected to see an alternative manage_users called from the ZMI that behaved much better, with manage_users calls raising warnings. g) Someone got paid to do it, and that just pisses me off, given the quality of the stuff we release for free (and I'm still looking for a job btw d8). We're not perfect, the 0.10.1 release shows that, but, at least we spent more than three minutes including thinking time on it, and I at least try not to change things just for the sake of it. Anyway. In case you were wondering what the previous email meant, there's the actual meaning d8) -- Totally Holistic Enterprises Internet| | Andrew Milton The Internet (Aust) Pty Ltd | | ACN: 082 081 472 ABN: 83 082 081 472 | M:+61 416 022 411 | Carpe Daemon PO Box 837 Indooroopilly QLD 4068|[EMAIL PROTECTED]| ___ Exuserfolder-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/exuserfolder-devel ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists -
Re: [Zope-dev] Searching/Indexing/ZODB/SQL/BerkleyDB
Chris Withers wrote: Toby Dickenson wrote: FileStorage is 'damn fast', so almost anything is going to be slower. Indeed, until it runs out of RAM for its indexes ;-) I wish you would finish testing the change I made for you. It should reduce the memory consumption by an order of magnitude. I took an afternoon out of a rather busy schedule to put this together for you. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (888) 344-4332http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] ZODBC bug
I found ZODBC bug at max_rows. file name: db.py line: 230 Current version is getting all rows regardless of max_rows. Fixed version is getting rows until rows are smaller than max_rows. Current version: while status==SQL_SUCCESS: . r.append(rd) status=SQLFetch(stmt) . Fixed version: cnt = 0 while status==SQL_SUCCESS and cnt max_rows: ... r.append(rd) cnt = cnt + 1 status=SQLFetch(stmt) ... Have a nice day !!
Re: [Zope-dev] Fw: [Exuserfolder-devel] Zope 2.5b1 release
A K Milton wrote: a) The change to manage_* seems to be completely arbitrary, since we already had _do* methods that meant you didn't have to call manage_users with fake submit buttons. So what is the point of having manage_ ? They were added in response to this fishbowl proposal: http://dev.zope.org/Wikis/DevSite/Proposals/UserFolderXmlRpcQuickFix It was a quick fix intended to help people doing user management over XML-RPC. If there are problems in maintaining compatibility with the previous API, and products that rely on that, well that's a bug and it needs Collecting and sorting out before 2.5 final. I'm concerned about this too, and I'm glad it's reached Zope-Dev, as I've got some LoginManager user folders in use, and I don't want these to break when I start using Zope 2.5 on those systems. In the fishbowl proposal comments, Brial Lloyd wrote: This is so long overdue that I've just checked this in for the Zope 2.5 line in CVS. (It is slated for 2.5 because it is an API change and has documentation impact, plus I would like to follow up and clean up some of the old form dispatch code and want to make sure we have an upgrade cycle to make sure other implementations of user folders don't break). I see an intention not to break other user folder products. Given that the fishbowl proposal in question is supposed to make for a very small change, any breakage in existing products is a bug in its implementation. In summary: I want to make sure that things are no worse in Zope 2.5 final than in Zope 2.5. Any breakage caused by this API change is a bug, and needs to be sorted out by Zope 2.5 final. I can offer some help in fixing these bugs, especially if they find their way into the Collector, so I can take ownership of them. Improvements to the user folder API that fall outside getting it working with XML-RPC bring up larger issues, which I see are being discussed here: http://dev.zope.org/Wikis/DevSite/Proposals/BetterUserManagement -- Steve Alexander Software Engineer Cat-Box limited ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Fw: [Exuserfolder-devel] Zope 2.5b1 release
Steve Alexander wrote: In summary: I want to make sure that things are no worse in Zope 2.5 final than in Zope 2.5. Any breakage caused by this API change is a bug, and needs to be sorted out by Zope 2.5 final. That should have read: I want to make sure that the user management API is no worse in Zope 2.5 than in Zope 2.4. -- Steve Alexander ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Searching/Indexing/ZODB/SQL/BerkleyDB
Jim Fulton wrote: Chris Withers wrote: Toby Dickenson wrote: FileStorage is 'damn fast', so almost anything is going to be slower. Indeed, until it runs out of RAM for its indexes ;-) I wish you would finish testing the change I made for you. Sorry, to be clear, my comment was in the context of using a FileStorage to exclusively store searching and indexing information. Jim has provided a patch which I was trying to test, sadly, for whatever reason, it went wrong and killed the box I was testing on. There are some issues preventing me resurrecting the box (location, staff, etc) but I will let you guys know as soon as I get some information... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Fw: [Exuserfolder-devel] Zope 2.5b1 release
a) The change to manage_* seems to be completely arbitrary, since we already had _do* methods that meant you didn't have to call manage_users with fake submit buttons. So what is the point of having manage_ ? They were added in response to this fishbowl proposal: http://dev.zope.org/Wikis/DevSite/Proposals/UserFolderXmlRpcQuickFix It was a quick fix intended to help people doing user management over XML-RPC. Right - the idea was to _add to_ the existing API, in a completely backward-compatible way, so that: - untrusted code (DTML, Python scripts, other code managed by security constraints) could be used to do user mgmt (if the caller has appropriate rights, of course). Previously, the only code accessible to these was unwieldy (the build-a-fake- request approach), and the corresponding _ methods were not accessible because _ methods can't be called from Web code. - trusted code (external methods, Python products) would have a clearer and easier API for doing the same. While they could have used the _ methods, they are not documented as if they are a part of the official API, which is a point of confusion. - An XML-RPC (given appropriate rights) call could be used to do user management work. If there are problems in maintaining compatibility with the previous API, and products that rely on that, well that's a bug and it needs Collecting and sorting out before 2.5 final. I'm concerned about this too, and I'm glad it's reached Zope-Dev, as I've got some LoginManager user folders in use, and I don't want these to break when I start using Zope 2.5 on those systems. Nor do I - my goal for this was (and remains) 100% backward compatibility, and to make people's lives easier, not harder. It looks like I've muffed the implementation, for which I certainly take full responsibility (and which I'll rectify today). I think I also failed to adequately express the goal for this - I'll need to update the docstrings as well. The goal was not to change the (admitted ancient and crummy) way that the Web interfaces to user folders interact with the API (the dispatching based on submit button lameness), as I'm sure that many, many implementations would break. The goal was not to deprecate _that_ usage of the 'manage_users' method. The idea was to deprecate the use of 'manage_users' from Web-based or product code in favor of the new (and hopefully easier) APIs. That allows us to address the lameness of the 'manage_users' method re: user folder UI as a separate issue in the future, while still being able to give scripters, product authors and XML-RPC users something that they can use now. Brian Lloyd[EMAIL PROTECTED] Software Engineer 540.361.1716 Zope Corporation http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] User folders, access rights, security, collector item 47
Sorry to post here. I posted last week on the zope list about what appears to be a new/strange security related problem in 2.4.3 I thought it was a problem in LdapUserFolder, but jens spent more time debugging than I did and he feels it's somewhere in the Zope security machinery. http://collector.zope.org/Zope/47 Is it security related when security doesn't do what you want (though nothing has been exposed, rather, what we want exposed isn't)? I'd try to tackle this myself, but I really have no idea where to start. I know everyone is busy, but I was wondering if anyone else has been able to reproduce the condition so that we can say for certain if its a bug, or a mis-configuration. Thanks, Brad Clements,[EMAIL PROTECTED] (315)268-1000 http://www.murkworks.com (315)268-9812 Fax netmeeting: ils://ils.murkworks.com AOL-IM: BKClements ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
