[Zope3-Users] Re: RFC 4122 UUIDs
Fred Drake wrote: On 4/6/07, Philipp von Weitershausen <[EMAIL PROTECTED]> wrote: How is there a ZCML pain? Simply register the adapter for all IAnnotatable objects. Typically your content objects are annotatable anyway because you want DublinCore stuff etc. This sounds like this will result in huge numbers of UUIDs being generated for objects that aren't "syndicatable content" in this application. The subscriber should probably be registered for objects that can be syndicated, not everything, to avoid useless CPU load and database bloat. Of course, as the OP suggested there's already an annotation for syndication-related information, I'd just add it to that, and not use a separate annotation. Another common pattern is to invent a new marker interface, e.g. IUUIDAware, which perhaps inherits from IAnnotatable so that it's clear there are annotations involved. Then you register adapters from that, and use a generic ZCML statement to give various objects this marker. In a CMF world, for example, we may do: Martin ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Stephan Richter wrote: On Thursday 05 April 2007 15:28, Jürgen Kartnaller wrote: Please do not use an annotation! This is a performance issue. Your data is very small so do not create a new object in the database store it directly as a property on the instance. Every object lookup is expensive, especially if you use ZEO! Hey Juergen, come on! I know you guys are on an optimization trip at Lovely, but this is great information to store on an annotation. The UUID does not usually need to be looked up and is mainly for external world reference, such as feeds like Derek metioned he wanted. The package is also an add-on. You don't need it, then do not use it. As further optimization, Derek mainly needs this id for a very specific set of content types, so I would narrow the subscriber to only give those special types a UUID. I know, everything starts simple, I remember last year when we used our sample data with 12 users and 500 books, now we have 1200 users and 10 books (35 objects registered in the intid utility) in the production system and this is just the beginning. The simple thing is now a complex one and performance is an issue everywhere. Jürgen ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: RFC 4122 UUIDs
On 4/6/07, Philipp von Weitershausen <[EMAIL PROTECTED]> wrote: How is there a ZCML pain? Simply register the adapter for all IAnnotatable objects. Typically your content objects are annotatable anyway because you want DublinCore stuff etc. This sounds like this will result in huge numbers of UUIDs being generated for objects that aren't "syndicatable content" in this application. The subscriber should probably be registered for objects that can be syndicated, not everything, to avoid useless CPU load and database bloat. Of course, as the OP suggested there's already an annotation for syndication-related information, I'd just add it to that, and not use a separate annotation. -Fred -- Fred L. Drake, Jr. "Every sin is the result of a collaboration." --Lucius Annaeus Seneca ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Derek Richardson wrote: Look at zope.intid; it should be IObjectCreatedEvent or IObjectAddedEvent. It's zope.app.intid, btw. I have started coding a uuid package for Zope. zope.intid uses IObjectAddedEvent; I plan to as well. Yes. However, the uuid differs from the intid in that I want to store the uuid on the object, rather than in a utility. This will ensure that uuids travel with their objects without having to modify import/export or other code. My initial idea is to use an annotation, but I'm not sure this is right. This is what annotations were made for, so yes, use annotations. I'll have to annotate all the content classes in the system, which seems to be a *huge* zcml pain. Any advice for avoiding this? How is there a ZCML pain? Simply register the adapter for all IAnnotatable objects. Typically your content objects are annotatable anyway because you want DublinCore stuff etc. -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Stephan Richter wrote: On Wednesday 04 April 2007 13:06, Derek Richardson wrote: I am hoping that Zope 3 assigns an RFC 4122 UUID to each content item. If not, I am hoping there is a third-party product to do this. No there is neither. We have an intid utility that guarantees System-wide unique ids. This is a misconception which occurs througout this thread. Integer IDs are only unique within the reach of a *particular* integer ID utility. They're not unique throughout the whole system (which I interpret as "global") nor through the whole ZODB. In fact, problems can arise when you have a site with an intid utility, populate it with objects and then create another intid utility in a subsite. -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Stephan Richter wrote: On Thursday 05 April 2007 15:28, Jürgen Kartnaller wrote: Please do not use an annotation! This is a performance issue. Your data is very small so do not create a new object in the database store it directly as a property on the instance. Every object lookup is expensive, especially if you use ZEO! Hey Juergen, come on! I know you guys are on an optimization trip at Lovely, but this is great information to store on an annotation. The UUID does not usually need to be looked up and is mainly for external world reference, such as feeds like Derek metioned he wanted. The package is also an add-on. You don't need it, then do not use it. As further optimization, Derek mainly needs this id for a very specific set of content types, so I would narrow the subscriber to only give those special types a UUID. Regards, Stephan My use case is very constrained - I need UUIDs for syndicated objects only. So, potentially, I could just generate and assign them when content is syndicated and keep them in an annotation I'm already using for syndication. However, as Martin Aspeli mentioned, this functionality seems ripe for a general-purpose package. If I split it out into its own add-on, it's more work for me but perhaps generally useful. I've seen past buzz about UUIDs in Zope, but it seems nothing has ever come of it. My question is: are there potential use cases, outside of my own, that justify the extra work to make it a separate add-on? And, if so, what are these use cases and what do I need to do to support them? I could see that there might be performance-sensitive applications that would need UUIDs on many objects, but I don't know of one. Does anyone else? The scenario I'm most trying to avoid is several different packages implementing UUIDs and objects getting multiple UUIDs, one from each package, all implemented different ways, because of the lack of a general solution. Derek ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: RFC 4122 UUIDs
On Thursday 05 April 2007 15:28, Jürgen Kartnaller wrote: > Please do not use an annotation! > > This is a performance issue. Your data is very small so do not create a > new object in the database store it directly as a property on the > instance. Every object lookup is expensive, especially if you use ZEO! Hey Juergen, come on! I know you guys are on an optimization trip at Lovely, but this is great information to store on an annotation. The UUID does not usually need to be looked up and is mainly for external world reference, such as feeds like Derek metioned he wanted. The package is also an add-on. You don't need it, then do not use it. As further optimization, Derek mainly needs this id for a very specific set of content types, so I would narrow the subscriber to only give those special types a UUID. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Derek Richardson wrote: Martin Aspeli wrote: Derek Richardson-2 wrote: Martin Aspeli wrote: Stephan Richter-2 wrote: On Wednesday 04 April 2007 13:06, Derek Richardson wrote: I am hoping that Zope 3 assigns an RFC 4122 UUID to each content item. If not, I am hoping there is a third-party product to do this. No there is neither. We have an intid utility that guarantees System-wide unique ids. This utility is used at several places most notably the catalog. I would like RFC 4122 UUIDs to provide standard Atom feeds of Zope content [1]. It will be better if Zope itself assigns UUIDs, so that there is a single source and not a possibility of multiple packages assigning different UUIDs to the same content item. You have to write your own utility to generate the UUID. I checked the RFC quickly and our IntIds are certainly not of the format requested by RFC 4122. Presumably, the intid implementation would be a useful reference for such a utility, and this would a good canidate for a general, re-usable package. Martin Good. I would like to write it as a general, re-usable package. Generating RFC 4122 UUIDs is easy - there's a routine for it in the python 2.5 standard libraries and a python 2.3+ version here: http://zesty.ca/python/uuid.html What seems more difficult is how to plug the routine in so that all content items receive one and only one UUID upon creation. Maybe listen for IObjectCreatedEvent and annotate then? Or is there a potential race condition that means events are not the way to go? (Unsure of the depths of Zope 3's architecture) Look at zope.intid; it should be IObjectCreatedEvent or IObjectAddedEvent. Martin I have started coding a uuid package for Zope. zope.intid uses IObjectAddedEvent; I plan to as well. However, the uuid differs from the intid in that I want to store the uuid on the object, rather than in a utility. This will ensure that uuids travel with their objects without having to modify import/export or other code. My initial idea is to use an annotation, but I'm not sure this is right. I'll have to annotate all the content classes in the system, which seems to be a *huge* zcml pain. Any advice for avoiding this? Please do not use an annotation! This is a performance issue. Your data is very small so do not create a new object in the database store it directly as a property on the instance. Every object lookup is expensive, especially if you use ZEO! Jürgen ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Martin Aspeli wrote: Derek Richardson-2 wrote: Martin Aspeli wrote: Stephan Richter-2 wrote: On Wednesday 04 April 2007 13:06, Derek Richardson wrote: I am hoping that Zope 3 assigns an RFC 4122 UUID to each content item. If not, I am hoping there is a third-party product to do this. No there is neither. We have an intid utility that guarantees System-wide unique ids. This utility is used at several places most notably the catalog. I would like RFC 4122 UUIDs to provide standard Atom feeds of Zope content [1]. It will be better if Zope itself assigns UUIDs, so that there is a single source and not a possibility of multiple packages assigning different UUIDs to the same content item. You have to write your own utility to generate the UUID. I checked the RFC quickly and our IntIds are certainly not of the format requested by RFC 4122. Presumably, the intid implementation would be a useful reference for such a utility, and this would a good canidate for a general, re-usable package. Martin Good. I would like to write it as a general, re-usable package. Generating RFC 4122 UUIDs is easy - there's a routine for it in the python 2.5 standard libraries and a python 2.3+ version here: http://zesty.ca/python/uuid.html What seems more difficult is how to plug the routine in so that all content items receive one and only one UUID upon creation. Maybe listen for IObjectCreatedEvent and annotate then? Or is there a potential race condition that means events are not the way to go? (Unsure of the depths of Zope 3's architecture) Look at zope.intid; it should be IObjectCreatedEvent or IObjectAddedEvent. Martin I have started coding a uuid package for Zope. zope.intid uses IObjectAddedEvent; I plan to as well. However, the uuid differs from the intid in that I want to store the uuid on the object, rather than in a utility. This will ensure that uuids travel with their objects without having to modify import/export or other code. My initial idea is to use an annotation, but I'm not sure this is right. I'll have to annotate all the content classes in the system, which seems to be a *huge* zcml pain. Any advice for avoiding this? Thanks, Derek ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Fred Drake wrote: On 4/5/07, David Pratt <[EMAIL PROTECTED]> wrote: The UUID generator that I have been looking at is with the Chandler project (currently under Apache license) that could be easily wrapped: Python 2.5 comes with a "uuid" module that works just fine with Python 2.4; that comes in handy as well, and doesn't need wrapping. -Fred Yes, at the cheesehop (uuid, not pyuuid). That's what I am starting to use for my uuid package for zope. Derek ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: RFC 4122 UUIDs
On 4/5/07, David Pratt <[EMAIL PROTECTED]> wrote: The UUID generator that I have been looking at is with the Chandler project (currently under Apache license) that could be easily wrapped: Python 2.5 comes with a "uuid" module that works just fine with Python 2.4; that comes in handy as well, and doesn't need wrapping. -Fred -- Fred L. Drake, Jr. "Every sin is the result of a collaboration." --Lucius Annaeus Seneca ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: RFC 4122 UUIDs
I also like the idea of UUIDs the way they are meant to be - which should be independent of the instance. I can imagine a further possibility to synchronize objects from independent zodb sources if this existed. The UUID generator that I have been looking at is with the Chandler project (currently under Apache license) that could be easily wrapped: http://svn.osafoundation.org/chandler/trunk/internal/chandlerdb/chandlerdb/util/uuid.c Regards, David Marius Gedminas wrote: On Wed, Apr 04, 2007 at 03:12:26PM -0400, Stephan Richter wrote: On Wednesday 04 April 2007 14:34, Derek Richardson wrote: I believe that that will not guarantee a *universally* unique id, but only an id unique within that ZODB. Am I wrong? Well, intid guarantees to be unique for this Zope instance, even accross multiple databases conencted to this Zope instance. All you have to do is to id the Zope instance, for example, ip+port should suffice. I'd imagine collisions are likely when ip+port is 192.168.1.1:8080. Marius Gedminas ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: RFC 4122 UUIDs
On Wed, Apr 04, 2007 at 03:12:26PM -0400, Stephan Richter wrote: > On Wednesday 04 April 2007 14:34, Derek Richardson wrote: > > I believe that that will not guarantee a *universally* unique id, but > > only an id unique within that ZODB. Am I wrong? > > Well, intid guarantees to be unique for this Zope instance, even accross > multiple databases conencted to this Zope instance. All you have to do is to > id the Zope instance, for example, ip+port should suffice. I'd imagine collisions are likely when ip+port is 192.168.1.1:8080. Marius Gedminas -- I dont know about madness (and anyway, the little green martians dancing around me tell me not to worry...), but I've implemented something not unlike this just now. -- Peter Sabaini signature.asc Description: Digital signature ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Stephan Richter wrote: On Wednesday 04 April 2007 14:34, Derek Richardson wrote: I believe that that will not guarantee a *universally* unique id, but only an id unique within that ZODB. Am I wrong? Well, intid guarantees to be unique for this Zope instance, even accross multiple databases conencted to this Zope instance. All you have to do is to id the Zope instance, for example, ip+port should suffice. One problem we see in Plone (which has similar types of extrinsic ids) is that if you export/import into a new site, ids are not stable (unless you exported the entire site, including the id). When used for e.g. references between objects, these break. If UUIDs were essentially hashes that'd make this problem managable. I'm not sure if this particular approach solves that though. Martin ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: RFC 4122 UUIDs
On Wednesday 04 April 2007 14:34, Derek Richardson wrote: > I believe that that will not guarantee a *universally* unique id, but > only an id unique within that ZODB. Am I wrong? Well, intid guarantees to be unique for this Zope instance, even accross multiple databases conencted to this Zope instance. All you have to do is to id the Zope instance, for example, ip+port should suffice. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Re: RFC 4122 UUIDs
Derek Richardson wrote:
I believe that that will not guarantee a *universally* unique id, but
only an id unique within that ZODB. Am I wrong?
The RFC prescribes a specific algorithm for generating universally
unique IDs.
Of course, they are only "universally unique" in a probabilistic way.
It wouldn't be difficult to salt the integer IDs such that they generate
UUIDs that are as likely as any other to be unique. Something like this
would work:
import uuid
int_id = 42
salt = 0x32352352353243263235235235324326
print uuid.UUID('%X' % (int_id ^ salt))
Another option would be to create an IOBtree to map each object's intid
to a randomly generated UUID (generated by the uuid.uuid1 method from
the earlier referenced module).
--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
I believe that that will not guarantee a *universally* unique id, but only an id unique within that ZODB. Am I wrong? The RFC prescribes a specific algorithm for generating universally unique IDs. Derek Benji York wrote: Derek Richardson wrote: I am hoping that Zope 3 assigns an RFC 4122 UUID to each content item. If not, I am hoping there is a third-party product to do this. I would use the standard intid utility and simply encode the generated integer as a UUID. ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Derek Richardson and I were just discussing this matter on freenode #vice ... I have no idea at this point if the suggested event model is better, but it seems like an implementation that safely handles the missing annotation on request, would be simpler and more efficient. In other words, rather than listening for the creation event, why not provide an interface that can be adapted to any object - where the implementation assumes objects may not yet have a UUID stored as an annotation. When the first request comes in, if none is found, you generate it and annotate and persist the object then. I would think as long as this implementation catches the potential write conflict error, and handles by re-requesting the UUID annotation, the race case should also be eliminated. Flaws with this line of thinking? Cheers, Paul Derek Richardson-2 wrote: Martin Aspeli wrote: Stephan Richter-2 wrote: On Wednesday 04 April 2007 13:06, Derek Richardson wrote: I am hoping that Zope 3 assigns an RFC 4122 UUID to each content item. If not, I am hoping there is a third-party product to do this. No there is neither. We have an intid utility that guarantees System-wide unique ids. This utility is used at several places most notably the catalog. I would like RFC 4122 UUIDs to provide standard Atom feeds of Zope content [1]. It will be better if Zope itself assigns UUIDs, so that there is a single source and not a possibility of multiple packages assigning different UUIDs to the same content item. You have to write your own utility to generate the UUID. I checked the RFC quickly and our IntIds are certainly not of the format requested by RFC 4122. Presumably, the intid implementation would be a useful reference for such a utility, and this would a good canidate for a general, re-usable package. Martin Good. I would like to write it as a general, re-usable package. Generating RFC 4122 UUIDs is easy - there's a routine for it in the python 2.5 standard libraries and a python 2.3+ version here: http://zesty.ca/python/uuid.html What seems more difficult is how to plug the routine in so that all content items receive one and only one UUID upon creation. Maybe listen for IObjectCreatedEvent and annotate then? Or is there a potential race condition that means events are not the way to go? (Unsure of the depths of Zope 3's architecture) ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] Re: RFC 4122 UUIDs
Martin Aspeli wrote: Stephan Richter-2 wrote: On Wednesday 04 April 2007 13:06, Derek Richardson wrote: I am hoping that Zope 3 assigns an RFC 4122 UUID to each content item. If not, I am hoping there is a third-party product to do this. No there is neither. We have an intid utility that guarantees System-wide unique ids. This utility is used at several places most notably the catalog. I would like RFC 4122 UUIDs to provide standard Atom feeds of Zope content [1]. It will be better if Zope itself assigns UUIDs, so that there is a single source and not a possibility of multiple packages assigning different UUIDs to the same content item. You have to write your own utility to generate the UUID. I checked the RFC quickly and our IntIds are certainly not of the format requested by RFC 4122. Presumably, the intid implementation would be a useful reference for such a utility, and this would a good canidate for a general, re-usable package. Martin Good. I would like to write it as a general, re-usable package. Generating RFC 4122 UUIDs is easy - there's a routine for it in the python 2.5 standard libraries and a python 2.3+ version here: http://zesty.ca/python/uuid.html What seems more difficult is how to plug the routine in so that all content items receive one and only one UUID upon creation. Maybe listen for IObjectCreatedEvent and annotate then? Or is there a potential race condition that means events are not the way to go? (Unsure of the depths of Zope 3's architecture) Derek ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
