"Great minds think alike." :grin: --- A. Soroka The University of Virginia Library
> On Jan 11, 2017, at 9:52 AM, Claude Warren <cla...@xenei.com> wrote: > > To be honest, I was thinking that if the graph notification for the > cassandra graph needed to notify all the attached graph implementations of > changes I would use Kafka or other similar pluggable Queue to do the > notification. > > Claude > > On Wed, Jan 11, 2017 at 2:48 PM, A. Soroka <aj...@virginia.edu> wrote: > >>> Perhaps we should look at a design for distributed update notification? >> >> If this is something that seems interesting, it's worth noting that there >> is a bunch of work on which to rely for this kind of problem. For example, >> colleagues of mine have had success with Apache Kafka. No need for Jena to >> reinvent any wheels. >> >> --- >> A. Soroka >> The University of Virginia Library >> >>> On Jan 11, 2017, at 7:24 AM, Claude Warren <cla...@xenei.com> wrote: >>> >>> Cassandra does not support transactions. However, I can see the use for >> a >>> bounding construct that indicates "all these triples were added/deleted." >>> >>> The original driver for the Cassandra implementation was that we needed >> one >>> for a use case at work. However, Cassandra is generally about speed of >>> writes and scale. So expect that the Cassandra implementation will be >>> viewed as large scale. This leads me to some concerns about hash joins >> but >>> that is a side issue to be addressed in the Op stuff for the Cassandra >>> implementation. >>> >>> Perhaps we should look at a design for distributed update notification? >>> >>> Claude >>> >>> On Wed, Jan 11, 2017 at 11:53 AM, Andy Seaborne <a...@apache.org> wrote: >>> >>>> Hi Claude, >>>> >>>> On 09/01/17 19:22, Claude Warren wrote: >>>> >>>>> Greetings, >>>>> >>>>> Given that the Cassandra server can host multiple client and those >>>>> clients can open the same graph on the server simultaneously. >> Basically >>>>> two updatable synchronized views on one data set. >>>>> >>>>> Assume graph A is opened on client X and client Y and applications at X >>>>> and >>>>> Y both register listeners on the graph. >>>>> >>>>> If application at X deletes a triple should the listener at Y be >> notified? >>>>> >>>>> I have been thinking about adding a queue based (JMS 1.1?) listener >>>>> implementation so that distributed system would be notified of changes >>>>> from >>>>> remote systems. >>>>> >>>> >>>> >>>> The second question deals with reasoners. If the reasoners are using >> the >>>>> distributed graph store then I don't think there is an issue so perhaps >>>>> this one goes away but..... >>>>> >>>>> Reasoners do not like it (don't respond to) data that is written into >> the >>>>> graph behind the scenes. In a distributed environment does it make >> sense >>>>> to somehow utilize the graph listen messages (as noted above) to fire >>>>> update rules? >>>>> >>>> >>>> The current reasoners are written assuming changes happen via their API. >>>> >>>> >>>>> Are there other issues that I have missed? >>>>> >>>> >>>> The current event mechanism provides synchronous events - when the "add" >>>> has returned the event has been fired. >>>> >>>> >>>> The Cassandra graph is about scale? >>>> >>>> Per triple events are very fine grained - what if a million triples are >>>> added - will there be a million events? And the whole update is sent to >>>> all listeners by JMS? >>>> >>>> 100 million triples? >>>> >>>> >>>> Where I'm going with Delta is that changes get recorded (RDF Patch) so >>>> that they can be applied elsewhere but also inspected. The granularity >> is >>>> different - the grain size is the transaction - a bunch of adds and >> deletes. >>>> >>>> So a different design is that an even on change at the end of >> transaction >>>> and that event is just notification something has happened and the event >>>> listeners can choose to inspect the inspect the change or not. >>>> >>>> I've also done where the patch drives sending an SPARQL Update to a >> remote >>>> copy - generally pure-push systems rather ones that pull the bulk >> changes >>>> because of issues like whether the client is ready to process the >> event, is >>>> actually running, there is a comms glitch or whether the client is >> actually >>>> interested in the content - they may, like the reasoners, be interested >>>> that a change has happened but their action is not simply on the >> content of >>>> just the change. >>>> >>>> Andy >>>> >>> >>> >>> >>> -- >>> I like: Like Like - The likeliest place on the web >>> <http://like-like.xenei.com> >>> LinkedIn: http://www.linkedin.com/in/claudewarren >> >> > > > -- > I like: Like Like - The likeliest place on the web > <http://like-like.xenei.com> > LinkedIn: http://www.linkedin.com/in/claudewarren
