Re: [Zope3-dev] Re: Community opinion about search+filter
Hello, Somehow relevant to the subject I just found an article on Wickert's site: http://www.wiggy.net/ , Using a seperate Data.fs for the catalog -- Best regards, Groszer Adam -- Quote of the day: Keep your fears for yourself, but share your courage with others. - Robert Louis Stevenson ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: IKeyReference for files
Martijn Faassen [EMAIL PROTECTED] schreibt: Yeah, and I do need to be cross-platform with this, so inodes are not ideal. Hi, it depends on how cross-platform you need to be and whether you have another solution that is ideal;) For Win32 you can use the low value of the FileIndex number of the filesystems master file table. This is the last value of the (10-)tuple returned by the Win32 API method win32file.GetFileInformationByHandle() Note that the new Blob-support also uses win32file Concerning the problem of the different filesystems and devices I can only guess whether your use cases really need to take different devices into account. In our use case we simply assumed that the files are on a single device and remain were they are. If you follow Christian's suggestion of using hard links with the oid of a shadow object in the ZODB a similar restriction applies. In any case I'm also interested in IKeyReferences for files, so I hope that there is a better solution that is cross platform, allows an efficient lookup, works across devices, and offers also an easy way to detect catalog relevant changes. I fear there is no ideal solution to all these problems and we need to consider trade-offs. Regards, Uwe Dr. Uwe Oestermeier Institut für Wissensmedien Knowledge Media Research Center Konrad-Adenauer-Str. 40 D-72072 Tuebingen Germany [EMAIL PROTECTED] Tel. +49 7071 979-208 Fax +49 7071 979-100 ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Community opinion about search+filter
Adam Groszer wrote: Somehow relevant to the subject I just found an article on Wickert's site: http://www.wiggy.net/ , Using a seperate Data.fs for the catalog The win here is actually partitioning the object cache... Similar wins could be achieved without making backup/pack/etc more complicated by making the object cache more intelligent: - based on size of objects, not just number - memory limiting (including intra-transaction) cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] buildbot failure in Zope3 trunk 2.4 Linux zc-buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 Linux zc-buildbot. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 709 Blamelist: dobe BUILD FAILED: failed svn sincerely, -The Buildbot ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 709 Blamelist: dobe BUILD FAILED: failed svn sincerely, -The Buildbot ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: IKeyReference for files
Hey Christian, Christian Theune wrote: Am Mittwoch, den 28.03.2007, 01:12 +0200 schrieb Martijn Faassen: [snip] Quick shot on integrating the suggestion from Uwe: - Reserve an OID from the ZODB storage, maybe create a 'shadow' object in the ZODB. - Use a directory that maintains a hard link to the real file with the oid as its name. Voila. Btw: This is a good bit like blob support works in ZODB. ;) This is an interesting idea; I hadn't considered looking at the blob support. This would allow one to use the PersistentKeyReference, right? This hard-link directory, is that portable? I guess the blob support does it, so that should be possible. I guess there is another approach if you go for ZODB shadow objects anyway, though - you could simply store the path in the shadow object (or the filename or directory name, along with a reference to its parent). If an object changes place, the key reference would still work, and only the internal path would need to be changed. No need for a directory with hard-links. I have been taking the approach where no shadow objects are created so far, as this makes it easier to browse a filesystem that may be changing. I will have to consider how a shadow structure in the ZODB affects this. Then of course there is the 'svn up' situation. I imagine the changes triggered by an SVN up can be translated into various added, moved and deleted events, but that is going to be tricky in the face of moved directories and the like. Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: IKeyReference for files
Hi, Am Mittwoch, den 28.03.2007, 18:05 +0200 schrieb Martijn Faassen: Hey Christian, Christian Theune wrote: Am Mittwoch, den 28.03.2007, 01:12 +0200 schrieb Martijn Faassen: [snip] Quick shot on integrating the suggestion from Uwe: - Reserve an OID from the ZODB storage, maybe create a 'shadow' object in the ZODB. - Use a directory that maintains a hard link to the real file with the oid as its name. Voila. Btw: This is a good bit like blob support works in ZODB. ;) This is an interesting idea; I hadn't considered looking at the blob support. This would allow one to use the PersistentKeyReference, right? This hard-link directory, is that portable? I guess the blob support does it, so that should be possible. I guess there is another approach if you go for ZODB shadow objects anyway, though - you could simply store the path in the shadow object (or the filename or directory name, along with a reference to its parent). If an object changes place, the key reference would still work, and only the internal path would need to be changed. No need for a directory with hard-links. I have been taking the approach where no shadow objects are created so far, as this makes it easier to browse a filesystem that may be changing. I will have to consider how a shadow structure in the ZODB affects this. Then of course there is the 'svn up' situation. I imagine the changes triggered by an SVN up can be translated into various added, moved and deleted events, but that is going to be tricky in the face of moved directories and the like. Right. I'm not sure whether I missed anything, but I don't quite have your whole use case on my radar. Blobs have the advantage that they can assume to be the only source of changes to the physical storage location. Having changes to a (cached) data structure from the outside is always troublesome. Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
