[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 542
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 543
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 547
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Lennart Regebro

On 3/26/07, Dieter Maurer [EMAIL PROTECTED] wrote:

When IncrementalSearch does it, it works roughly as Jim described it
under 1 above (although there is no need that it works the the primary
key directly).


Right.


Unless the indexes are able to return sorted (partial results), we
need to determine the results first and then sort them. And that
takes time at least linear in the number of hits (to determine the sort
values for the documents).


OK, at least this avoids the big intermediate results when searching
over several indexes. But you still have to get all of the results,
and sort them before you can return the X first. I have the impression
that Lucene somehow solves this with their sorting indexes, but I'm
not sure, and I haven't tried to understand the code.

--
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 551
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 Linux zc-buildbot

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 Linux zc-buildbot.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 551
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Jim Fulton


On Mar 26, 2007, at 8:45 PM, Martijn Faassen wrote:
...

I think we can safely conclude that:

* there is no silver bullet in all this (your point)


Yes


* there is probably room for improvement.


Yes

...


My hopes:

* is that there is some low-hanging fruit in improving things.


Possibly, including through education.

...


* that we may be able to provide some more infrastructure to help
developers in scaling particular catalog usage scenarios (batching
with sorting).


Possibly, although if this is really a goal, it will require:

- Educating people on how hard this is and

- Providing infrastructure that let's people record use alternate  
document ids or supplementary document ids.  (Not even thinking about  
relevance ranks.)  I suspect that such an infrastructure will be too  
hard to use for many people.


I still get the impression that you think that batching is more of an  
opportunity than it really is.  Yes, it does reduce time  
significantly, but still not enough to achieve scalability.  (It  
really only lets you deal with somewhat larger small result sets.)



I argue these points as you initially gave me the strong impression
saying that there's no point in even talking about all this.


I never said any such thing, In fact, I spent quite a bit of effort  
talking about it.



After
that I thought we were actually going somewhere with this discussion,
but you now strengthen this impression by apparently giving up in
exasparation. That is what is making *me* slightly exasparated. :)


Fine. I gave up because you dismiss literature and theory. You seem  
to imply that in practice there is some kind of magic that will  
somehow make sorts go fast despite any theoretical basis. This  
convinced me that further discussion was a waste of time.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 556
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 557
Blamelist: andreasjung,dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 Linux zc-buildbot

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 Linux zc-buildbot.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 557
Blamelist: andreasjung,dobe

BUILD FAILED: failed test_2

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Jim Washington
Hi, Martijn

I have a suggestion, only because I have played around with the idea a bit.

Google for python factoradics, and you will get my blog entry about
factoradics.

I see the problem statement as How to obtain batching without
re-sorting multiple times.

If you see a sort order as one permutation of a list, the factoradic
technique provides a key to that permutation.  So, in theory, one would
sort the list, and store the factoradic index for that permutation.  The
next time the particular sort order is requested, it's a simple matter
of unpacking the factoradic. I have not done any tests to see whether
unpacking a factoradic is significantly less expensive than re-sorting. 
Intuitively, it should be.  In practice, I am not so sure.

Anyway, this is FWIW.  :)

-Jim Washington

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 Linux zc-buildbot

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 Linux zc-buildbot.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 565
Blamelist: andreasjung,dobe

BUILD FAILED: failed test_2

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Martijn Faassen

Hey Jim,

On 3/27/07, Jim Fulton [EMAIL PROTECTED] wrote:
[snip]

 After
 that I thought we were actually going somewhere with this discussion,
 but you now strengthen this impression by apparently giving up in
 exasparation. That is what is making *me* slightly exasparated. :)

Fine. I gave up because you dismiss literature and theory. You seem
to imply that in practice there is some kind of magic that will
somehow make sorts go fast despite any theoretical basis. This
convinced me that further discussion was a waste of time.


I can see how you interpreted my statements that way. My apologies; I
did not intend to dismiss literature and theory nor did I wish to give
the impression that I do.

All I tried to express is that algorithmic scalability is just one
aspect of scalability issues, and lower-level design choices and
constant factors do matter in the final execution. I don't know how
much - getting a rough idea requires the comparisons we talked about,
and I intend to do this at some point. :) And theory clearly puts
upper limits on what is possible, and the upper limits (which are not
as high as people would naively expect). I was assuming you understood
that I understood the fact that theory provides a limit as a given. :)

I realize that there is no way to improve things fundamentally in all
cases, and I realize that writing an easy-enough API that helps
matters in at least some cases will be a challenge.

Regards,

Martijn
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 570
Blamelist: dobe,mj

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 571
Blamelist: baijum

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 Linux zc-buildbot

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 Linux zc-buildbot.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 571
Blamelist: baijum

BUILD FAILED: failed test_2

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Zope3 Standalone Page Templates

2007-03-27 Thread Fred Drake

On 3/26/07, Jim Fulton [EMAIL PROTECTED] wrote:

We have a lot of work to do on dependencies. :(


Fortunately, a number of community members are working on this right
now.  We should see some real improvements very soon.


 -Fred

--
Fred L. Drake, Jr.fdrake at gmail.com
Every sin is the result of a collaboration. --Lucius Annaeus Seneca
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 576
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 577
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Disabling the Zope 3 Collector today

2007-03-27 Thread Jim Fulton


Heads up!

Assuming that I can figure out how to do so, I'm going to disable the  
Zope 3 collector today in preparation for switching to Launchpad.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread Benji York

[EMAIL PROTECTED] wrote:

The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.


This is caused by a ZODB test failure regarding blobs.  zodb-dev has 
been alerted.

--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 580
Blamelist: dobe

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Dieter Maurer
Jim Washington wrote at 2007-3-27 08:24 -0400:
 ...
If you see a sort order as one permutation of a list, the factoradic
technique provides a key to that permutation.  So, in theory, one would
sort the list, and store the factoradic index for that permutation.  The
next time the particular sort order is requested, it's a simple matter
of unpacking the factoradic. I have not done any tests to see whether
unpacking a factoradic is significantly less expensive than re-sorting. 
Intuitively, it should be.  In practice, I am not so sure.

In what way is it different from caching?
The packed factoradic seems like a cache to me.


-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Dieter Maurer
Lennart Regebro wrote at 2007-3-27 11:59 +0200:
 ...
OK, at least this avoids the big intermediate results when searching
over several indexes. But you still have to get all of the results,
and sort them before you can return the X first. I have the impression
that Lucene somehow solves this with their sorting indexes, but I'm
not sure, and I haven't tried to understand the code.

The ZCatalog, too, can use sorting indexes (and AdvancedQuery
which stole the idea from ZCatalog).

However, this approach is only efficient when the sort index size
is small compared to the result size.

If the result size is large, the sorting index can only help you
to get the sorting values quite fast as they are stored compactly
together.



-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Dieter Maurer
Jim Fulton wrote at 2007-3-26 15:55 -0400:
 ...

On Mar 26, 2007, at 3:28 PM, Dieter Maurer wrote:

 Jim Fulton wrote at 2007-3-25 09:53 -0400:

 On Mar 25, 2007, at 3:01 AM, Adam Groszer wrote:
 MF I think one of the main limitations of the current catalog (and
 MF hurry.query) is efficient support for sorting and batching the
 query
 MF results. The Zope 3 catalog returns all matching results, which
 can then
 MF be sorted and batched. This will stop being scalable for large
 MF collections. A relational database is able to do this
 internally, and is
 MF potentially able to use optimizations there.

 What evidence to you have to support this assertion?  We did some
 literature search on this a few years ago and found no special trick
 to avoid sorting costs.

 I know of 2 approaches to reducing sort cost:

 1. Sort your results based on the primary key and therefore, pick
 your primary key to match your sort results.  In terms of the Zope
 catalog framework, the primary keys are the document IDs, which are
 traditionally chosen randomly.  You can pick your primary keys based
 on a desired sort order instead. A variation on this theme is to use
 multiple sets of document ids,  storing multiple sets of ids in each
 index.  Of course, this approach doesn't help with something like
 relevance ranks.

 2. Use an N-best algorithm.  If N is the size of the batch and M is
 the corpus size, then this is O(M*ln(N)) rather than O(M*ln(M)) which
 is a significant improvement if N  M, but still quite expensive.

 The major costs in sorting are usually not the log(n) but
 the very high linear costs fetching the sort keys (although for  
 huge n,
 we will reach the asymptotic limits).

Right. The problem is the N not the log(N). :)


 Under normal conditions, a relational database can be far more  
 efficient
 to fetch values either from index structures or the data records
 than Zope -- as

   * its data representation is much more compact

   * it often supports direct access

   * the server itself can access and process all data.


 With the ZODB, the data is hidden in pickles (less compact), there is
 no direct access (instead the complete pickle need to be decoded)

The catalog sort index mechanism uses the un-index data structure in  
the sort index to get sort keys. This is a pretty compact data  
structure.

The data usually is in IOBuckets which contain 45 values on the average.
In a corresponding relational index structure, you could have several hundreds
of values.

 and
 all operations are done in the client (rather than in the server).

Which is often fine if the desired data are in the client cache.  It  
avoids making the storage a bottleneck.

Yes, but the if is important. Quite often, some operations flush
almost all objects from the cache (partly because the cache is controlled
by the number of objects and not by their size) and after that,
filling the cache again takes ages.

Moreover, a relational database can (and usually does) use caching as
well. It is not restricted to a cliend side only technique.

I know that most relational database backends have a different architecture
than ZEO: use one process (or at least one thread) per connection
such that several activities can interleave the high IO wait times.
The one thread ZEO architecture must take more care not to become the
bottleneck.



-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter

2007-03-27 Thread Jim Washington
Dieter Maurer wrote:
 Jim Washington wrote at 2007-3-27 08:24 -0400:
   
 ...
 If you see a sort order as one permutation of a list, the factoradic
 technique provides a key to that permutation.  So, in theory, one would
 sort the list, and store the factoradic index for that permutation.  The
 next time the particular sort order is requested, it's a simple matter
 of unpacking the factoradic. I have not done any tests to see whether
 unpacking a factoradic is significantly less expensive than re-sorting. 
 Intuitively, it should be.  In practice, I am not so sure.
 

 In what way is it different from caching?
 The packed factoradic seems like a cache to me.


   
Hi, Dieter

A factoradic index is representable as a long integer.  Given that
integer and the canonical list, you can regenerate the permutation
represented by that integer.  So, instead of caching the sorted list
itself, you find and keep this integer, which is all the information
needed to algorithmically re-obtain the sorted list.

So, this would be slower than caching, but (I think) faster than re-sorting.

-Jim Washington

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: IKeyReference for files

2007-03-27 Thread Martijn Faassen

Wichert Akkerman wrote:

Previously Uwe Oestermeier wrote:

Martijn Faassen [EMAIL PROTECTED] schreibt:
Now I'm hoping I'm missing some kind of strategy and perhaps someone 
will have a luminous idea to make this work without the creation of a 
separate index. Or if not, at least I can give up looking and just go 
and write that index. Does anyone have any suggestions?

Hi Martijn,

I have experimented with the inodes of files, which are a good candidate
for IKeyReferences for files. Using inodes solves the problem that the ids
should remain stable across moves. They are also a good basis for a
detection of moves which happen outside the control of Zope.


Unfortuantely there are filesystems without usable inode numbers (or
inodes). You also need to take multiple filesystems into account, which
means you need to include the device major and minor number in your key.
This leads to another problem: those numbers may not be stable across
reboots. 


Not stable across reboot would be bad. This needs to work minimally 
under Linux and Windows.


How does one get to inodes on Linux from Python? Does Windows have an 
equivalent?


Regards,

Martijn

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: IKeyReference for files

2007-03-27 Thread Wichert Akkerman
Previously Martijn Faassen wrote:
 Wichert Akkerman wrote:
 Previously Uwe Oestermeier wrote:
 Martijn Faassen [EMAIL PROTECTED] schreibt:
 Now I'm hoping I'm missing some kind of strategy and perhaps someone 
 will have a luminous idea to make this work without the creation of a 
 separate index. Or if not, at least I can give up looking and just go 
 and write that index. Does anyone have any suggestions?
 Hi Martijn,
 
 I have experimented with the inodes of files, which are a good candidate
 for IKeyReferences for files. Using inodes solves the problem that the ids
 should remain stable across moves. They are also a good basis for a
 detection of moves which happen outside the control of Zope.
 
 Unfortuantely there are filesystems without usable inode numbers (or
 inodes). You also need to take multiple filesystems into account, which
 means you need to include the device major and minor number in your key.
 This leads to another problem: those numbers may not be stable across
 reboots. 
 
 Not stable across reboot would be bad. This needs to work minimally 
 under Linux and Windows.

Linux makes no strong guarantees. Especially for devices which can
appear on different busses or bus slots (USB, firewire, iSCSI) the block
numbers can change.

Wichert.

-- 
Wichert Akkerman [EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/   It is hard to make things simple.
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: IKeyReference for files

2007-03-27 Thread Christian Theune
Hi,

Am Mittwoch, den 28.03.2007, 01:12 +0200 schrieb Martijn Faassen:
 Wichert Akkerman wrote:
  Previously Uwe Oestermeier wrote:
  Martijn Faassen [EMAIL PROTECTED] schreibt:
  Now I'm hoping I'm missing some kind of strategy and perhaps someone 
  will have a luminous idea to make this work without the creation of a 
  separate index. Or if not, at least I can give up looking and just go 
  and write that index. Does anyone have any suggestions?
  Hi Martijn,
 
  I have experimented with the inodes of files, which are a good candidate
  for IKeyReferences for files. Using inodes solves the problem that the ids
  should remain stable across moves. They are also a good basis for a
  detection of moves which happen outside the control of Zope.
  
  Unfortuantely there are filesystems without usable inode numbers (or
  inodes). You also need to take multiple filesystems into account, which
  means you need to include the device major and minor number in your key.
  This leads to another problem: those numbers may not be stable across
  reboots. 
 
 Not stable across reboot would be bad. This needs to work minimally 
 under Linux and Windows.
 
 How does one get to inodes on Linux from Python? Does Windows have an 
 equivalent?

Quick shot on integrating the suggestion from Uwe:

- Reserve an OID from the ZODB storage, maybe create a 'shadow' object
in the ZODB.
- Use a directory that maintains a hard link to the real file with the
oid as its name.

Voila.

Btw: This is a good bit like blob support works in ZODB. ;)

Christian

-- 
gocept gmbh  co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development


signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] buildbot failure in Zope3 trunk 2.4 FreeBSD tmiddleton

2007-03-27 Thread buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 FreeBSD tmiddleton.

Buildbot URL: http://buildbot.zope.org/

Build Reason: changes
Build Source Stamp: 659
Blamelist: andreasjung,baijum,dobe,mj,srichter

BUILD FAILED: failed test

sincerely,
 -The Buildbot

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com