[Zope3-dev] Re: .xpt extension for explicit xml processing

2005-10-13 Thread Michel Pelletier

Julien Anguenot wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi there,

We are working with Fred on the tal processing mixed modes issue I
raised in a previous thread on this list.

The development branch for this has been created over there :
http://svn.zope.org/Zope3/branches/tal_mixed_mode/

We thought about adding an .xpt extension meaning explicit xml
processing to avoid having a more complex input type sniffer and thus
decrease the performances of page template...


+1

We'd find this very useful in one of our projects going on at the moment.

-Michel

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: RDFLib and Zope 3

2005-08-31 Thread Michel Pelletier
On Wed, 2005-08-31 at 10:07 -0400, Gary Poster wrote:

  I'm interested in contemplating RDF as a full catalog solution for
  Zope, at least as a thought experiment.  

Great!

  Note that the use of bound variables also removes the need for brains.
 
 We actually don't have catalog brains in Zope 3 anyway, but yes,  
 maybe it would let you only wake up objects when you need to, which  
 is what you are getting at.  It might make sorting easier too.  Need  
 to think this through.

Activation is one issue I actually wasn't getting at, but it's
definitely something to think about.  What I was getting at was that the
result is completely unrelated to any object.  A ZCatalog result object
had the duel role of also being a handle back to the original object (in
addition to preventing activation).  When I work on these things I tend
to disregard the presence of any kind of Python object that the data is
associated with.  I think though that we are well on our way to coming
up with a way to formalize the rdf/object bridge and we'll eventually
have the right solution and terminology here.

  Sure, there are some notes from me on the z3lab site that might be of
  interest for thinking about zemantic and catalog integration.
 
 Could you send a URL?
 
snip
 I think the RDF spec can be used for interpretation too, given the  
 rich spellings that RDF allows for predicates, and the typing of its  
 nodes.  I do think that the RDF component should only deal with RDF,  
 so I agree with your general desire.  But RDF is very, very rich: a  
 *lot* of functionality could be in a pure RDF library like RDFlib,  
 including support for predicate constraints and join indexes for  
 instance.  That would be useful whatever your RDF use, if you needed  
 efficiency for common searches.

Ah I didn't get that you were going for such a full solution until this
email.  You're right, it's a bold proposal but I think with some more
fleshing out it can become reality.  This could completely merge the use
cases of both rdf graphs and zope catalogs.

As Dan is interested in componentizing everything, this is a good
oppertunity to think about how to make these changes.

 
 The extra RDF features could also be a layer on top of a simpler  
 graph implementation, if that were a desired design.  I'm not saying  
 that RDFlib should have all the features, but that a pure-RDF library  
 *could* have all the features.

Yep, I'm on your frequency now.  I think it's a good idea.

-Michel

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: RDFLib and Zope 3

2005-08-30 Thread Michel Pelletier
On Mon, 2005-08-29 at 23:24 -0400, Gary Poster wrote:

  Right.  Well in this case we would provide just a very simple  
  interface
  facade that had no effect when run in an environment with no
  zope.interface (ie, catch the ImportError, null-out the facade) or  
  hook
  into zope.interface if it is available.  This way rdflib can be  
  still be
  used with or without zope.interface.
 
  Sounds good.
 
 OK, cool.

Stub interfaces (from Zemantic) are now checked into the rdflib trunk.
Dan and I also had a discussion on what changed to the lib would be made
if we actually depended on zope.interface.  Definately the plugin
framework would go, but there were some other unanswered questions we
had.


  Yep, feel free to stop by anytime.
 
 OK, cool, I plan to again. :-)

Please do, we need some help sketching out what kinds of changes would
be required moving over to a component model.

 I'm interested in contemplating RDF as a full catalog solution for  
 Zope, at least as a thought experiment.  The SPARQL work seems  
 interesting, in regards to some of the recent discussion on the Zope  
 3 list; and the ability to seamlessly and simultaneously query both  
 relationship information between objects and more traditional catalog  
 data seems compelling.

And I think SPARQL is up to the task to query a catalog for the most
part, there might be some holes in the language that don't suite the
catalog exactly (text index querying, for example, is out of spec), but
for field and keyword indexes SPARQL would make a great query language,
one would just need to teach the catalog to assign well know predicates
to index names, possibly via adaptation.  For example, I think the
following queries and many like it are easily possible against a
catalog:

PREFIX  dc: http://purl.org/dc/elements/1.1/
SELECT  ?title
WHERE
{ ?book dc:title ?title }

Note that the use of bound variables also removes the need for brains.
Given that SPARQL also has CONSTRUCT (to build a graph of results) and
DESCRIBE (generate results describe one or more resources) clauses, I
think it makes a better object query language than OQL.

Note that the sparql support needs help!  We have query logic and half a
parser, the parser needs completion and to be glued to the logic.

 
 It seems to me that allowing a back end to index particular  
 relationships--and joins, particularly through blank nodes--would be  
 a big step in letting RDF actually be a front end to a full catalog.   
 Another step would be having a front end that honored rdfs range and  
 domain constraints.
 
 I plan to get on IRC and bother you all again as soon as I have time  
 to do so. :-)

Sure, there are some notes from me on the z3lab site that might be of
interest for thinking about zemantic and catalog integration.  THis may
be completely the wrong direction, bust most of my experiments so far
have been to keep a strong separation between how zemantic stores
searchable data (without interpretation) and how the catalog stores it
(with interpretation).  

For example, rdflib doesn't interpret a dc:modification_data value as a
date at all.  It's just one element in a graph that rdflib stores, not
interprets.  We (meaning Dan and I, as we've discussed this) want to
keep as strong barrier as possible between a Graph and its
interpretation, ie, we don't want to encapsulate or imply any
interpretation on any graphs, and if we do, we want to make sure that
interpretation is pluggable and that a graph can be transposed between
interpretations easily.  Interpreting dc:modification_date as a date
certainly does make sense, but its an assumption we can't make globally,
although I'm not against sensible defaults.

The catalog, on the other hand, makes these assumptions by design.  It
knows dc:modification_date is a date because the user instructs that
value to go into a date range searchable index.  This is not bad, it's
good, I think that rdflib and a catalog can leverage the distinction
between uninterpreted and interpreted data.

So far most of my thoughts have been that content (or whatever)
describes itself as RDF.  This is stored in a Graph with no
interpretation, the rdflib object only holds the shape of the graph.
Events get fired that make decisions on how that graph should be
interpreted, and the data is then cataloged.  I see a one to many
relationship possible, one interpreting catalog can derive its
interpretations on many graphs, and one graph can have many interpreting
catalogs, all queriable via SPARQL.

-Michel



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: Zope3-dev Digest, Vol 25, Issue 37

2005-08-25 Thread Michel Pelletier
 From: Olivier Grisel [EMAIL PROTECTED]
 Subject: [Zope3-dev] Re: [DRAFT] local portlets and perspectives

  
  (SellerURI, merchandiseURI, BuyerURI, Sale Transaction)
  
 Let's rewrite this relation to a prolog equivalent fact:
 transaction(SellerURI, merchandiseURI, BuyerURI, Sale Transaction).
 
 There is no primary key in that relation. We could add one such as 
 TransactionURI for instance :
 
 transaction(TransactionURI, SellerURI, merchandiseURI, BuyerURI, Sale 
 Transaction).
 
 In this case we could rewrite this fact without loss of information 
 using only binary relations/predicates by projecting it:
 
 transactionSeller(transactionURI, SellerURI), 
 transactionMerchandise(transactionURI, merchandiseURI), 
 transactionBuyer(transactionURI, buyerURI), 
 transactionType(transactionURI, Sale Transaction).
 
 This description is then equivalent and RDF friendly. However it 
 requires the introduction of a primary key which was implicit in the 
 first description and is much more verbose.

I think the original triadic statement can be encoded with one rdf
triple + context in rdflib:

Graph.add((SellerURI, merchandiseURI, BuyerURI), TransactionURI)

Where TransactionURI is the context.  I'm still feeling that these two
models are either equivalent, or there is some property of triadic
relations that I just don't get yet.  This paper:
http://www.cs.indiana.edu/pub/techreports/TR606.pdf also appears (from
reading the first five pages) to state that reified RDF triples *are*
Peirce relations, which seems to make sense as they are statements about
statements, although I can't claim to really understand the guts of the
paper.

I want to reiterate to you guys that I'm not trying to be argumentative
or prove that one thing is better than the other, I take a deep interest
in these subjects and I want to know as much as possible about them.
But unfortunately I think we've strayed off the topic for this list.

-Michel

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: RDFLib and Zope 3

2005-08-25 Thread Michel Pelletier
On Thu, 2005-08-25 at 14:32 -0400, Gary Poster wrote:

  see what he thinks.  I wonder how lite the component kernel can go.
 
 The only thing I have in mind is the interface package, which is what  
 Twisted uses.  That's all we would need.  zope.component needs  
 zope.interface, zope.testing, and zope.exceptions, according to its  
 DEPENDENCIES.cfg.

Right.  Well in this case we would provide just a very simple interface
facade that had no effect when run in an environment with no
zope.interface (ie, catch the ImportError, null-out the facade) or hook
into zope.interface if it is available.  This way rdflib can be still be
used with or without zope.interface.

 
  In the mean time the adapters can live inside Zemantic, which is an
  rdflib to zope bridge anyway.  Let me know if you want to send  
  patches,
  otherwise I'll probably get around to adding functionality like this
  soon.
 
 I'm actually interested in trying to hook this up, but have very  
 limited time.  I might play with it just within RDFLib alone during  
 some hobby time tonight, but otherwise may  need to toss this off to  
 you if you'll catch it.
 
 I also kind of want to hear Dan's reaction before I spend too much time.

#redfoot on freenode is a good place to catch him, and me.

 I thought I read that an RDF triad was itself something that could be  
 a node in another RDF triad, but I can't find that anywhere now.  Can  
 you confirm or deny? :-)

Yes, it's called reification, making a statement about a statement.

http://www.w3.org/TR/rdf-primer/#reification

http://en.wikipedia.org/wiki/Resource_Description_Framework#Statement_reification_and_context

http://www.w3.org/TR/rdf-mt/#Reif

-Michel



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Florent's O-R blog entry

2005-08-24 Thread Michel Pelletier

This is a general reply, Martijn just summed up many of the points so
nicely (as usual) that I'm using his email as a starting point...

 From: Martijn Faassen [EMAIL PROTECTED]
 Subject: Re: [Zope3-dev] Florent's O-R blog entry

 I have had some opportunity to work with the Zope 3 catalog recently, 
 and I have a few comments. First of all, I agree with the main idea that 
 the Zope 3 catalog is not a hack, and is clean and flexible. I believe 
 the catalog should be invested in, as I think it's cool.

agreed.

 Now as to where I see areas where features are lacking in the Zope 3 
 catalog:
 
 Underfeatured query API
 ---
 
 I do think that currently the API to query it is woefully underfeatured.
 
 I've tried to work on this problem and am sitting on some code that just 
 needs a bit of time to polish and release that allows a simple query 
 language on top of the catalog. It's just building up a tree of python 
 objects for queries, nothing special, but it is a lot higher level than 
 what's already there.

This is an important feature that the catalog needs, query logic.  The
argument that Python (without at least a query API) is its query
language no longer holds water for me.  Good query languages should
allow you to define what you are looking for, now how you look for it,
and your code, inspired by something like Dieter's AdvancedQuery, is
absolutely necessary.

But... I think we are missing another layer here.  I pointed this out in
my first reply to Gary's post about Zemantic.  I think a three-tiered
approach needs to be taken to searching, just like it has been taken
with many other aspects of Zope 3.  I think this removes the whole
argument of catalog vs. rdbms for enterprise systems and make the
answer even better.  

1) The model tier contains the searchable sources.  These sources
provide a simple search source interface.  I do not think it's possible
and I do not propose to have a consistent query interface across various
source implementation, but it's certainly possible to have a consistent
source management interface and it's possible for sources to describe
themselves and their searchable content using a common schema language.
Sources can be local components (catalogs, rdbms) or remote (google,
wikipedia, etc)

2) The controller tier provides query logic (agents is a common term
here).  Agents are components that implement a particular query
interface, and know how to query registered searchable sources or other
agents based on that source or agent's description.  Agents take care of
the dirty work (like result merging and joins) and provide a clean
interface to submit a query and retrieve results.  What the query
looks like and which search source is used depends on the agent and its
configuration.  If they know how to, agents can query agents to delegate
their work.

3) The view tier is a simple, high-level interface that the user (in
this case the typical third-party developer) primarily interacts with.
It is used to discover, query and manage sources and agents.  It doesn't
hide so much the underlying complexity as it provides a way to manage
the complexity.

Consider your typical CMF/Plone based application today.  They have a
portal_catalog that is the sole searchable source for the application
framework.  To add another source, one must, in general, hack the
application or the framework (note Archetypes gets around this, to a
degree, but only for catalogs, and only on values that map to an AT
field).  

If a third-party product wanted to plug into the search framework, they
need to either hack the existing portal_catalog, or if that is not
possible (which it very often is not due to name and usage conflicts at
the index level) they must create their own catalog.  But now the two
searchable sources are completely disconnected and unrelated and various
interfaces and things have to be hacked to include both sources in the
application's logic.

-Michel


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] [DRAFT] local portlets and perspectives

2005-08-24 Thread Michel Pelletier
On Wed, 2005-08-24 at 15:28 -0400, [EMAIL PROTECTED] wrote:

 Message: 5
 Date: Wed, 24 Aug 2005 20:24:30 +0200
 From: Jean-Marc Orliaguet [EMAIL PROTECTED]
 Subject: [Zope3-dev] [DRAFT] local portlets and perspectives


 It is built on the notion of Perspective (see the link) and on the
 idea of querying the catalog using triadic relations instead of joining
 sets of query results based on dyadic predicates (such as with RDF).

I've been looking for more info on Peirce relations since you mentioned
them to me in Paris, but I haven't found much.  This is a good one I've
found so far:

http://www.angelfire.com/md2/timewarp/peirce.html

I like:

According to Peirce, ‘meaning’ is a triadic relation between a sign,
an object, and an interpretant. This triadic relation is not reducible
to a set of dyadic relations between a sign and an object or between an
object and an interpretant (CP 1.345). Meaning is never reducible to
Firstness or Secondness, but can only be a ‘genuine’ Thirdness. A
general meaning can always be found in ‘genuine’ triadic relations, but
can never be found in ‘degenerate’ triadic relations which have lost
their Thirdness.

Word.  But it sounds to me from the description that follows that RDF is
a triadic relation, but I'm no semiotic phenomenologist.  Do you know of
more references on this subject?

Another thought, can a Peirce relation be meta-encoded in RDF?

-Michel

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] [DRAFT] local portlets and perspectives

2005-08-24 Thread Michel Pelletier
On Thu, 2005-08-25 at 04:10 +0200, Jean-Marc Orliaguet wrote:

 You would need a quadruple store in that case (see the previous mail) -

rdflib.Graph is actually a quadruple store.  Many users still say
TripleStore because that was the old class name, but most RDF systems,
rdflib included, have grown a concept called context  a quad-element
that lets you assign context to a triple, sparql calls this the
source, and some people say that it is used to track the provenance
of a triple.  Perhaps this is similar to the triadiac relations.

 if you have (Subject, Predicate, Object) in RDF
 
 here you have:
 - monadic relation: (first, predicate)
 - dyadic relation: (first, second, predicate)
 - triadic relation: (first, second, third, predicate)

Ok, so in RDF a dyadic relationship is (DocumentURI, dc:title, Expense
Report), what would a similar triadic relationship be?


-Michel


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: RDFLib and Zope 3

2005-08-23 Thread Michel Pelletier
On Tue, 2005-08-23 at 12:49 -0400, Gary Poster wrote:
 Michel (and anyone else with experience with RDFLib on the list), I  
 recently looked at RDFLib (http://rdflib.net/) and came away (after  
 an hour or so) with a good first impression.

Great.  I've cc:ed Dan Krech, the lead rdflib developer on this mail.
For his benefit I might explain things that you obviously know.

 My biggest disappointment was that, from the perspective of a Zope 3  
 developer, using it alongside other Zope 3 indexes (and other intid- 
 based data structures) meant that I would have to externally convert  
 to and from RDF in order to merge results and convert the RDF URIs to  
 objects. 

Correct.  A specific and important optimization in Zope-style cataloging
is that objects have a cheap unique integer to reduce catalog footprint
and significantly improve result merging and joining.  These intergers
are exposed as a utility component in Zope.

  It would be much more efficient if I could have an RDF  
 resource class that represented an intid, and even more efficient if  
 I could get IFBTrees back directly from searches that somehow  
 included the intids.  

Yes, this is a problem that needs to be solved, and your suggestion is
one way to solve it.  I've discussed this a few time with Florent at the
paris and EUpy sprints and he had a similar suggestion.  

I'm  uncomfortable with it for a few reasons, 1) because intids are such
a Zope-catalog-optimization specific thing.  I know why they are
exposed, so that catalog results can be efficiently merged, but they
don't have anything to do with RDF, so 2) rdflib can't really change its
interface to accomodate them.  Also, 3) they are backend specific, for
example rdflib has a URI - integer mapping for its in-meomory and ZODB
backends to reduce footprint, but a sql backend would need no such
integer, you would in fact have to *add* a column to hold that value
just so the data would merge efficiently with a catalog.  This seems
antithetical to Zope 3's philosophy in general as it violates the
concept of not requiring third party libs and data to change themselves
significantly just to work with Zope.  Of course, this isn't a problem
of the catalog, it's a problem in general merging search results from
anywhere.

I'd like to make the optimization available so that searches on a graph
can be efficiently merged with searches on a catalog, but I don't think
it can be done by pushing intids down into rdflib, or for that matter
any other third party component you want to play with the catalog
efficiently.  Perhaps instead of pushing the integers down we could push
URIs up, Zope's cataloging could grown another layer of indirection on
top of intids and provide a URI utility that maps to intids.  Of course
you might object to that for the same reasons I'm objecting to this. ;)
But at least URIs are a well known standard.

Somewhat at right angles to this, I think Zope needs to grow another
search interface, a higher level one that hides all of this integer id
stuff from the user.  I proposed something incomplete along these lines
to the z3labs site, an interface that could aggregate searches across
multiple registered search sources, whether catalogs, rdflib Graphs,
relational databases, remote systems, google, etc.  

With something like this, no need to worry about intersecting two
floating point result sets efficiently, the underlying search framework
performs that optimization if it is available.  Note that the primary
benefit of such an interface is not necessarily merging results across
multiple sources, but instead providing a consistent interface
regardless of the search source.

 Then I could leverage the relationship and  
 keyword capabilities of RDFLib while also merging results efficiently  
 with other index-like data structures in Zope 3.  The intid-specific  
 resources could even have stable URI representations without too much  
 trouble, so that they could be exported and imported with RDFLib, if  
 desired.

Hmm so these resource objects you are suggesting, they would be
persistent objects?  I don't quite have the picture of what you suggest.
Perhaps these resource classes can be managed by a utility?

 Have you thought about that use case?  If one used a variation of  
 your back end that assigned intids to non-intid-based resources like  
 URIs and Literals and stored the relationships via intids, 

One doesn't need a variation, this is exactly the way the in-memory and
ZODB backends work now as an optimization.  But they are internal
details of the implementation of those backends.

 you could  
 store the data as IFBTrees and offer up an API to get raw IFBTree  
 results.  Any obvious ways that would be a problem?  Does it feel  
 reasonable to you?  Any suggestions?

Well not any good ones yet, although I know it's an important problem.
I'll have to think about it a bit more.  Do you understand my
objections?  Does anyone else have any suggestions out there?  

Re: [Zope3-dev] Re: RDFLib and Zope 3

2005-08-23 Thread Michel Pelletier
On Tue, 2005-08-23 at 18:04 -0400, Gary Poster wrote:

 The relationship between ZODB content objects, their int id as  
 provided by the pertinent intid utility, and a (theoretical)  
 corresponding RDF URI is what I'm having a hard time not making hacky  
 in my mind.  I'll think about it some more.

They might not be that hacky, this might be the wrong direction to take
but URI's don't have to be visually meaningful, blank nodes, for
example, are usually just '_:' concatenated to a random opaque string.
If the URI were 'zope:' (maybe path/to/intid/util:) that
would work just as well, it would also be trival to transform into a
feasible join key if the URI was also a URL that looked up, instead of
some network resource, an intid.  

Actually being able to trivially transform an intid to an rdflib URI
might be something to think about.  Thinking about it more, the current
Zemantic uses the physical path of the object as the rdf:about= URI
when an object adds itself, because honestly I could think of no other
URI in Zope.  This is obviously wrong, but I didn't have a better answer
in paris.  Why not use the `intid` plus some URI sugar?  If the URI and
the intid can be easily converted from one to the other then that should
solve the whole problem, no?

 
 Another difficulty is that I like the RDF data model and the RDFLib  
 implementation, but I haven't found a compelling reason to care much  
 about the actual RDF format input and output.  Is there a practically  
 compelling defense of RDF as a format somewhere to which you can  
 point me?

I'm sure you're aware of this but for others: RDF does not specify a
syntax, only the data model.  The most popular syntax, RDF/XML, is
pretty bad, but rdflib also supports the NT syntax, which is a plain
text format.  There are some other triple languages out there that may
look even better, and support for them in rdflib would require writing
only a parser and maybe a serializer if you want that format back out.
I like SLIP but the parser needs some work:

http://www.scottsweeney.com/projects/slip/

and lastly I've been kicking around a new syntax based on SLIP I call
SLIPR.  Unlike SLIP which is for any XML, SLIPR is RDF only.

https://svn.cignex.com/public/slipr/data/pyinrdf.slpr

I only have the syntax outlined right now, I'm still working on the
parser.  This is my attempt to mix RDF with indented syntax.  It looks
great in python-mode. ;)  Unfortunately this is low priority.  The good
news is the high priority is a SPARQL parser, which is coming along
nicely.  Kudos to the fabulous pyparsing library. Hopefully we should
have full sparql support by 2.4.

-Michel

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] KeywordIndex

2005-07-18 Thread Michel Pelletier
On Mon, 2005-07-18 at 12:00 -0400, [EMAIL PROTECTED] wrote:

 Date: Mon, 18 Jul 2005 09:14:16 -0600
 From: Jeff Shell [EMAIL PROTECTED]
 Subject: [Zope3-dev] KeywordIndex
 To: zope3-dev@zope.org
 Message-ID: [EMAIL PROTECTED]
 Content-Type: text/plain; charset=ISO-8859-1
 
 I'm working on a simple application which is the first time I get to
 use the catalog in Zope 3. I'm writing against Zope 3.1b1. I was
 dismayed not to see KeywordIndex in the main catalog set, but then I
 found it in zope.index.keyword. But it seems to be a bit behind. I
 have it somewhat working through subclassing, etc, but it's been
 purely guess work on my part to get things this far. In my product
 package, I have the following:

snip
 1. 
I'm unable to help you directly with your problem, although Gary's post
about the SetIndex looks very promising and I would like to see that
code as well.  As you said, something like a keyword index is exactly
what your application is designed around, but if I could digress from
the topic a little I'd like to suggest another solution, rdflib.

rdflib covers some of the same use cases as a keyword index.  Your
objects (content, whatever) and your keywords would be assigned unique
identifiers.  You then add relations to an rdflib.Graph that associate
the keyword with your objects:

 dc = rdflib.Namespace('http://purl.org/dc/elements/1.1/')
 blue = rdflib.BNode()  # creates a URI for you
 g = rdflib.Graph()
 g.add((object_uri, dc.keywords, blue))

and then query the data back out with either a low-level g.triples((s,
p, o)) pattern or a sparql query.  For example, print a list of all
object URIs that have the blue_uri keyword:

 print [s for (s, p, o) in g.triples((rdflib.Any, dc.keywords, blue))]

or sparql:

 sg = rdflib.sparqlGraph(g)
 select = (?object_uri,)
 where = rdflib.GraphPattern([(?object_uri, dc.keywords, blue)])
 for object_uri in sg.query(select, where): print object_uri

is the same query, but longer.  With sparql you can do more complex sql
like queries however against other relations.  

Hope this helps,

-Michel

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: relational data

2005-05-04 Thread Michel Pelletier

 Message: 7
 Date: Wed, 04 May 2005 08:17:09 +0200
 From: Achim Domma (Procoders) [EMAIL PROTECTED]
 Subject: [Zope3-dev] relational data
 To: zope3-dev@zope.org
 Message-ID: [EMAIL PROTECTED]
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed
 
 Hi,
 
 I'm evaluating ZOPE for some projects we might have to implement. ZODB 
 looks very nice for storing object hierarchies but I don't understand 
 how to store relational data.
 
 For example, f I store people and qualified realations between them, I 
 usally have a table 'person' which hold the person records, 
 'relation_type' which hold the possible relations and 
 'person_person_relation' which holds the ids of the related persons and 
 the id of the type of the relation.
 
 I don't see how I would store something like this in ZODB and I want to 
 understand if it's better to stick with a relational database for such a 
 case. Or is there a solution, which I might not see at the moment?
 
 This should be no criticism, I only want to understand what Zope is good 
 for and where something else might suite my needs better.

ZODB does not strictly require hierarchical object graphs, that's a
Zopeism, ZODB will allow you to construct arbitrary object graphs.
Given that it can store relations as well as it can store any other
object. Storing references like you describe can be done easily in ZODB.
There are three existing systems like this to my knowledge.  

Although the terminology differs depending on what system you are
looking at, I'll use RDF terms here, subject (referer), predicate
(reference) and object (referent).  So if your relationship is Bob
likes Mary then 'Bob' is the subject, 'likes' is the predicate, and
'Mary' is the object.  This subject/predicate/object model goes by many
names but for the sake of discussion lets call it SPO.

Storing these references can be done any number of ways, but in ZODB I
have seen two ways, the first is from Zemantic (http://zemantic.org/)
which is a very early Zope 3 product that stores RDF.  Zemantic lets you
store and query SPO relations (called triples in RDF).  Each subject,
predicate, and object is stored in a multi-dimensional ZODB BTree.

The second is Archetype references (for Zope 2) that use a ZCatalog to
store basically a very similar data structure to what Zemantic uses, a
three column table (constructed of ZCatalog indexes) that index the
relationships in each dimension.  Archetype references are pretty mature
and offer more features than Zemantic that go beyond RDF, but as of now
it's Zope 2 only.

The third framework comes with the SchoolBell Zope 3 product I have no
experience with and I don't know exactly how it stores its data, but I
suspect it is less bleeding edge than Zemantic (I can say that with
some confidence because I am in the middle of upgrading Zemantic to
rdflib 2.1, which is pretty radically different than 2.0).

Or, of course, you can just use a relational database which might give
you more piece of mind.

Hope that helps,

-Michel



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Vocabularies, sources and relationships

2005-04-07 Thread Michel Pelletier
On Thursday 07 April 2005 11:24 am, Michel Pelletier wrote:

 I haven't looked at the ST library much, I get the impressions it's more
 focused on similar uses to that of Archetype references, which is great.
 There are features in AT refs that would be a bit clumsy to implement in
 Zemantic (like relationships being persistent objects with behavior and
 data) although not at all impossible.

 Clumsy unless you need your relationships to span systems and applications.
 The your relationship model must be based on something like RDF or its
 equivalent (something that does not apply application specifc
 interpretation to the stored references).  The other upshot of using
 something like RDF is that you are not inventing a whole relational model
 that does not necessiarly play natively with others.

I should probably be more clear by saying that I don't know if ST's package 
has these same benefits as RDF, Kapil and Ben told me it's based on an XML 
linking standard (XLink?) so it's quite possible. Kapil also pointed out to 
me that exporting one relational set onto another is, one way or the other, 
pretty straightforward if one wants to span systems and applications.  RDF 
is just one standardized way to do this with minimal pain, and as I 
understand it there are others (topic maps, A. Water's xsdb, probably many 
more...)

-Michel
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com