[Neo4j] hi.
hi! I find a good website：www.ebisri.com On this website ,you can find many new and origianl electronic products .Now they are holding sales promotion activity, all the product are sold at a low cost and good quality ,and the delivery is on time . It is a good chance that you should not lose. If you need some, visit this website . Let us enjoy the happiness of shopping. Greetings! ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [Neo] single Neo4J database, access from multiple nodes
+1 Thrift. One implementation solves all the client issues in all languages. Plus its pretty quick (more than quick enough for CPU / IO bound tasks such as Neo). Al On 4 June 2010 10:26, Peter Neubauer peter.neuba...@neotechnology.comwrote: Eelco, Tobias has been playing around with implementing the RMI http://components.neo4j.org/neo4j-remote-graphdb/ with protobuffers instead, but I think the RMI based variant is the most stable right now. Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sat, May 22, 2010 at 10:23 PM, Eelco Hillenius eelco.hillen...@gmail.com wrote: You might also take a look at Thrift, which would let you define your own service API and expose it via RPC (Thrift binary protocol). There's some work here, but it might be easier than messing around with your own REST API. I do have some code laying around that will just generate automatic proxies doing REST calls from interface definitions. So it wouldn't really be a very big deal. However, the main problem I would have is that I would prefer to use the same client code on whatever the underlying server is. It looks like this is supported for local vs RMI, but not (yet, aside from experimental rest-graphdb) for the rest server. I was mainly wondering if there are plans in that direction. If you're already familiar with a messaging tier in Java -- e.g. JMS, AMQP, etc. -- that's another approach. I've used this and it works very well. But there's a steep adoption curve. I think I'd prefer to keep this synchronous tbh. I'm comfortable working with messaging, but for this purpose I think that it would complicate things too much (debugging can be a pain, having to write async code for what is logically sync, etc). Thanks! Eelco ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Dr Alastair James CTO James Media Group www.jamesmedia.net www.adnet-media.net www.worldreviewer.com www.thehotelguru.com 'Inspiring Travel' IATA 96012851 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] PHP REST API client
Hi there Anders, Thanks for spotting the bug! It appears that the Neo REST API (the HTTP URIs the server expects) changed since the initial version I coded it for. All fixed now. I also included the set NULL to remove the property function. New version on github. Cheers Al ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] PHP REST API client
Hi! I tried it out and have a suggestion: to make the semantics for storing NULLs consistent you could change the PropertyContainer::__set method to remove the property if it exists when trying to set it to NULL. Excellent idea! I will add ASAP. For some reason calling Node::save twice gives me an exception, so I can't update a node after the first save and save it again with new property values. Maybe a bug? Looks like it. I will fix ASAP! Alastair /anders On 06/02/2010 01:00 AM, Alastair James wrote: Hi there! Sorry, been a bit quiet on the PHP REST API front for a few weeks. I will be added some features this week (traversals etc...), but in the mean time, I have (finally) written up a little blog post detailing how the current version works! http://onewheeledbicycle.com/2010/06/01/getting-started-with-neo4j-rest-api-and-php/ Stay tuned for more! Alastair ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Dr Alastair James CTO James Media Group www.jamesmedia.net www.adnet-media.net www.worldreviewer.com www.thehotelguru.com 'Inspiring Travel' IATA 96012851 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] PHP REST API client
Hi there! Sorry, been a bit quiet on the PHP REST API front for a few weeks. I will be added some features this week (traversals etc...), but in the mean time, I have (finally) written up a little blog post detailing how the current version works! http://onewheeledbicycle.com/2010/06/01/getting-started-with-neo4j-rest-api-and-php/ Stay tuned for more! Alastair ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] REST blog post
Almost missed this! http://blog.neo4j.org/2010/04/neo4j-rest-server-part1-get-it-going.html http://blog.neo4j.org/2010/04/neo4j-rest-server-part1-get-it-going.htmlGood write up. I will write a 'getting started in PHP' post using the client I wrote. However, somebody raised a good point on hacker news, how does this affect licensing? Ok, if Neo is a core component I can see it making no difference. But what if Neo4J was just on of several possible graph backends (others being FlockDB / custom mysql) and you could choose the one that best works for you and your scale? Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] REST blog post
It's going to depend on which backend you use at every specific moment in time. When you use Neo4j, you must respects the terms of Neo4j's license. When you use MySQL, you must respect MySQL's license. I mean more from the point of view from the author of a 3rd party bit of software. If I make a software package that uses Neo I need to release it under the same license as Neo or pay a fee. So, can I release a closed source or other licensed package that 'can' use Neo via a de-coupled REST API? I am not thinking of doing this, just interested to know! Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] PHP REST API client
Hi...
On 14 April 2010 08:16, Mattias Persson matt...@neotechnology.com wrote:
Cool, I just verified that you can create a relationship with properties in
the REST API, f.ex:
{
to: http://localhost:/node/123;,
type: knows,
data: {
foo: bar,
baz: 43
}
}
Arh! I thought the properties were stored in a 'properties' attribute not
'data'.
There may be a bug in the functional test for that feature.
In CreateRelationshipFunctionalTest.java :
String jsonString = {\to\ : \ + WebServer.BASE_URI + node/ + endNode
+ \, \type\ : \LOVES\, \properties\ : {\foo\ :
\bar\}};
You can see its saved with 'properties' which means a) the test is out of
date and b) that the test does not test the presence of the attribute when
reloading!
Al
--
Dr Alastair James
CTO James Publishing Ltd.
http://www.linkedin.com/pub/3/914/163
www.worldreviewer.com
WINNER Travolution Awards Best Travel Information Website 2009
WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008
WINNER Travolution Awards Best New Online Travel Company 2008
WINNER Travel Weekly Magellan Award 2008
WINNER Yahoo! Finds of the Year 2007
Noli nothis permittere te terere!
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] PHP REST API client
On 14 April 2010 08:20, Peter Neubauer neubauer.pe...@gmail.com wrote: Very cool! In case you are up to it, maybe write some additions to the http://wiki.neo4j.org/content/PHP ? I think this may easily become the preferred way to interact with Neo4j from PHP! Cheers Peter. I will write something on there (and a blog post) when I get a moment! Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] PHP REST API client
Thanks! On 14 April 2010 10:53, Mattias Persson matt...@neotechnology.com wrote: I saw that and fixed that as well! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Dispell the myth? FlockDB vs. Neo4j
(not sure why they didn't use a proven high speed graph DB myself - but then I am biased!). I am guessing its because they only need 1st order relations (there are little friend of a friend or higher operations on twitter) and were very worried about scalability and sharding (as far as I am aware there is no way for a Neo Db to be bigger than one machine at the moment). Not to mention that they probably have considerable mysql infrastructure and expertise I think its pretty obvious that FlockDB will be very slow for 'deep' traversals (network protocol speed alone will ensure this as FlockDb will need to talk to mysql on each node visited), but for order 1 traversals (e.g. what boils down to a join across a mapping table) mysql is probably a lot faster (no transactional overhead for read ops). Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] PHP REST API client
Hi all. Just got a basic PHP REST client API up and running. Features: * Node creation * Node loading by id * Node delete * Create relationship between two nodes (does not store properties in relationships yet. Might be a REST API bug) * List / filter relationships on a node Pretty rough at the moment, and no documentation or tests! http://github.com/onewheelgood/Neo4J-REST-PHP-API-client The demo.php file should give an indication of how to use it (I have tried to stick as closely to the Java API). I hope someone finds it useful! Cheers Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Where to submit REST API bug reports?
Hi there... Where do I file REST API bug reports? Cheers Al -- Dr Alastair James CTO James Publishing Ltd. http://www.linkedin.com/pub/3/914/163 www.worldreviewer.com WINNER Travolution Awards Best Travel Information Website 2009 WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 WINNER Travolution Awards Best New Online Travel Company 2008 WINNER Travel Weekly Magellan Award 2008 WINNER Yahoo! Finds of the Year 2007 Noli nothis permittere te terere! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] RST API character encoding + unicode
Hi! The example you mentioned: \u2018Hello world\u2018, is probably properly escaped since 2018 is LEFT SINGLE QUOTATION MARK [2], and quotation marks should be escaped, but is this the sequence you are experiencing problems with? If that gets returned unescaped, that could be a problem, but if it's another char, like \u00f6, then it should be allowed to be represented unescaped as a proper unicode char. Yes, that is an example string that is failing. However, \u2018 is a fancy quote (http://www.fileformat.info/info/unicode/char/2018/index.htm) not one of the quotes that needs to be encoded, so it should not matter if its returned unencoded or not (its part of the JSON encoding). It might still be a bug though, so if you could provide some more details that would be great. Also could you check if the truncation of messages occurs in your client due to decoding issues, on the server or on the wire. The web service returns a HTTP CONTENT LENGTH header of the same length and is received by my client, so I think its a bug. I tell you what I think the issue is, I noticed the enclosed example contains two unicode charcters, each of which is a three byte sequence in UTF-8. The response is exactly 4 characters too short (mising ut}), so its almost like the output length is the number of characters not the number of bytes. Looking at GenericWebService.java - line 86 - method addHeaders builder = builder.header( HttpHeaders.CONTENT_LENGTH, String.valueOf( builder.clone().build().getEntity().toString().length() ) ); Won't that set the content length to the number of characters NOT bytes? Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Traversers in the REST API
Since in manycases the results of a query will need to be reformed into their associated domain objects Unlikely to be the case over the HTTP API. Its unlikely people will create domain objects in (e.g.) PHP they will just use the data directly. Pagination is kinda tricky if the data changes between subsequent requests for pages. Since pagination is generally used for UIs, a common approach is to place the entire dataset (or a cursor, depending on where the data is coming from) in a session object. Regardless of where it is kept, if you want to deal with data changes, you either have to a) invalidate the cached dataset if data changes or b) keep a copy of the whole dataset around in its as queried state so that subsequent paging requests are consistent. Either case involves keeping a fairly big duplicate data structure on the server or middle tier and violates one of the objectives of REST-ful APIs, which is that of statelessness. For that reason, I personally think the REST-ful API shouldn't deal with paging. It should probably be done at some intermediate level as needed by applications. We can certainly build a separate API that we can all leverage if needed, but I don't think it should be in the core REST-ful layer. Well, I think for my use cases (websites), its likely that users dont flick between pages that often. For example, on may sites, users will view page 1 and select an item, any very view move on to page 2. Its a very different usage pattern compared to a traditional desktop UI, so there is absolutely no need to hold the sorted set on the server in a cursor type way. A typical use case for me would be 1000+ matching rows, with 90%+ of page views for the first 10, 5% for the next 10 etc...! You can clearly see that sending the entire results set of 1000+ rows over HTTP/JSON is inefficient. Of course, caching between the web server and the neo HTTP API can help, but not in all cases, and it seems silly to rely on this. Al -- Dr Alastair James CTO James Publishing Ltd. http://www.linkedin.com/pub/3/914/163 www.worldreviewer.com WINNER Travolution Awards Best Travel Information Website 2009 WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 WINNER Travolution Awards Best New Online Travel Company 2008 WINNER Travel Weekly Magellan Award 2008 WINNER Yahoo! Finds of the Year 2007 Noli nothis permittere te terere! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] How to efficiently query in Neo4J?
On 9 April 2010 14:21, Max De Marzi Jr. maxdema...@gmail.com wrote: On first traversal, add a relationship to a found node to each node that would return, and check for this relationship on the second traversal? Maybe create a unique id, set a property or add a node property with the unique id on the first traversal, and check for this property on the second traversal? I suspect that would lead to pretty poor performance given that I am altering the data on each node visited. Also, I would have to remember to remove these properties / relationships later on otherwise the database will balloon in size. That said, it would be suitable for queries that I could generate (say) every hour. So adding relations to the graph would be acceptable for a index that is built using a 'slow query', but not suitable for realtime results. ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Traversers in the REST API
What I want to avoid is keeping state on the server while waiting for the client to request the next page. You are quite right. However, I think for many use cases (e.g. generating a paginated list of results on a webpage) it would not be necessary to store state on the server. That would be more similar to a SQL cursor, what I am talking about is simply SQL LIMIT, OFFSET and ORDER BY. Cheers Al On 8 April 2010 17:23, Tobias Ivarsson tobias.ivars...@neotechnology.comwrote: What I want to avoid is keeping state on the server while waiting for the client to request the next page. -- Dr Alastair James CTO James Publishing Ltd. http://www.linkedin.com/pub/3/914/163 www.worldreviewer.com WINNER Travolution Awards Best Travel Information Website 2009 WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 WINNER Travolution Awards Best New Online Travel Company 2008 WINNER Travel Weekly Magellan Award 2008 WINNER Yahoo! Finds of the Year 2007 Noli nothis permittere te terere! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] How to efficiently query in Neo4J?
Hi... On 8 April 2010 22:35, Michael Ludwig mil...@gmx.de wrote: After giving this some thought, it looks to me as if there is nothing particularly graphy in your example. I know, most everything is a graph, but here the data is more regular: Your hierarchical catalog of tags immediately made me think of Joe Celko's nested sets, which is a very efficient way to represent trees in terms of sets, as found in SQL databases. (Heresy again, I know, but well.) And the relationship of posts to tags is simply N-M, and that's it. We are currently using something similar to model this is SQL. However, having to maintain the nested set model is quite complex and something I really want to avoid in user code. There aren't any real links (edges) between posts, which arguably would make your data model more graphy. In your model, related posts are related by virtue of their attributes (they share some tags, or are posted by the same user), and not eis ipsis. So I'd say there is not much in the way of graphiness. It was a simplified example, in reality there are relations between posts, posts and authors, tags and tags etc... It is exactly because we want 'anything to be relatable to anything' that the graph database model works so well. Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] How to efficiently query in Neo4J?
As always, it really isn't that simple. Comparing cold queries is
probably not a good indicator of steady state performance, since
RDBMS's and Graph DB's have different models for file system access and
caching. Even different RDBMS's have dramatically different behaviors
in common queries (ever try to use MySQL for set operations - yuck.).
Factor in a wide range of SLAs needed for performance vs
availability vs affordability vs scalability vs adminstration costs,
and the equation gets a whole lot more complicated.
Exactly.
From experience its possible to build a post/tag system in SQL that performs
very well. However, the SQL model is inherently less flexible than the graph
database model (what if I want to introduce a new relationship type, in a
traditional SQL schema that would require new join tables etc...).
I'm sure there's a graphy-model for the tag/post example that could be
made smoking fast with Neo also.
Hopefully! I suppose my question is I would like to be able to harness a
graph database to give flexibility and eloquence to our data model. However,
can I query it efficiently without domain specific hacks and extra layers of
code?.
Al
Original Message
Subject: Re: [Neo] How to efficiently query in Neo4J?
From: Michael Ludwig mil...@gmx.de
Date: Thu, April 08, 2010 6:02 pm
To: Neo user discussions user@lists.neo4j.org
Max De Marzi Jr. schrieb am 08.04.2010 um 16:48:18 (-0500)
[Re: [Neo] How to efficiently query in Neo4J?]:
You know this is something that I think needs to be made clear...
using just the graph is not the right way to go unless you have a
very
special application.
Some things are better not done in the graph. So I decided to keep
that in tables, and just move the person relationships to the graph
(works with, manages, knows, friends, etc).
I treat the graph like a specialized index. Makes a lot more sense
now, and I get the best of both worlds.
Exactly what I think. An iterable index, and a great one for the kind
of
graphy queries that cannot be done efficiently using sets and joins.
Any thoughts on what constitutes *graphiness*, if I may venture this
term?
--
Michael Ludwig
___
Neo mailing list
User@lists.neo4j.org
[1]https://lists.neo4j.org/mailman/listinfo/user
References
1. https://lists.neo4j.org/mailman/listinfo/user
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Dr Alastair James
CTO James Publishing Ltd.
http://www.linkedin.com/pub/3/914/163
www.worldreviewer.com
WINNER Travolution Awards Best Travel Information Website 2009
WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008
WINNER Travolution Awards Best New Online Travel Company 2008
WINNER Travel Weekly Magellan Award 2008
WINNER Yahoo! Finds of the Year 2007
Noli nothis permittere te terere!
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Traversers in the REST API
On 8 April 2010 21:17, Michael Ludwig mil...@gmx.de wrote: Limiting and paging usually go hand in hand with sorting, in my experience. Why would anyone want to page through an unsorted collection? Its quite possible that you might want the nodes in the order they were found (e.g. the closest matching nodes first), however, I agree, sorting by an arbitrary property is very useful! Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] RESTful HTML representation
Hi there! Good work! I suppose the best example I can think of is the couchdb default HTML interface... Looks nice, but most importantly works very well. Some embedded javascript to do things like expandable lists etc... Cheers Al On 6 April 2010 10:28, Mattias Persson matt...@neotechnology.com wrote: Hi all, We've just added a little (just super-plain and w/o css, so ugly) read-only HTML representation of the REST api. Do you guys know of any examples of nice layouts for these kinds of contents we could look at for inspiration? -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Dr Alastair James CTO James Publishing Ltd. http://www.linkedin.com/pub/3/914/163 www.worldreviewer.com WINNER Travolution Awards Best Travel Information Website 2009 WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 WINNER Travolution Awards Best New Online Travel Company 2008 WINNER Travel Weekly Magellan Award 2008 WINNER Yahoo! Finds of the Year 2007 Noli nothis permittere te terere! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Traversers in the REST API
These two ways of traversing a graph complement each other, it's not that
one is better than the other. Would you agree on this?
I think I agree. I would hope to be able to use XPath/Gremlin style querying
for most things, and a more programatic system for more complex ones.
a JSON document describing the traverser, like:
{ order: depth first,
uniquness: node,
return evaluator:
{ language: javascript,
body: function shouldReturn( pos ) {...} },
prune evaluator:
{ language: javascript,
body: function },
relationships: [
{ direction: outgoing,
type: KNOWS },
{ type: LOVES }
],
max depth: 4 }
Looks good for my needs. Using javascript in this form looks sensible.
Any idea about the performance implications of using a javax.scripting
language here? I guess not too severe.
Is there any need for a shared context between calls to the evaluators? So I
could store custom information and access it again when traversing further
nodes. So, you could passing a 'context' object (with its initial values)
that gets passed in as a second parameter to each evaluator function. Then
again, this is probably bad practice
Any idea how you will handle pagination? Obviously sorting is an issue as
you are unlikely to want the nodes in the traversal order. In my mind it
would be nice to allow the return evaluator to return a 'sorting value' that
indicates that nodes rank in the result set. E.g. sorting on a score
attribute of the node:
function shouldReturn(pos)
{
if (!some_condition) return false;
return pos.currentNode().score;
}
But I guess this is a comment on the Neo API as a whole?
Cheers
Al
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Traversers in the REST API
Cheers guys. All sounds good. One comment: As for sorting: yes, that is a comment on the API as a whole. We have opted at not providing sorting, since there are good sorting facilities available in the JRE already. Since that makes it easy for the user to implement their own sorting it would be sub optimal for Neo4j to provide sorting. Since sorting is a costly operation (both in time and space) it should be done as late in the process as possible, probably with a lot of user code in between the traversal and the place where the sorting actually takes place. This has been our thinking in the REST API as well, meaning that sorting will be left to the client. It is possible that we will return to this decision and add sorting to the REST API, and that it might trickle down to the core API. Features like this are however much easier to add than to remove, which is why it is not implemented at the moment. Well, thats the case when Neo is running in the same JVM as the user code, but when we start talking about returning 1000s of nodes in JSON over HTTP just to get the first 10 this is clearly sub-optimal (as I build websites this is a very common use case). So, as you say, sorting and limiting can wait, but I suspect the HTTP API would benefit from offering it. Limiting need not require changes to the core API, it could be implemented as a second stage in the HTTP API code prior to output encoding. Cheers Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] How to efficiently query in Neo4J?
Hi there... I am looking at moving a website to a model based on Neo4J, however, I am having trouble seeing how to optimise the 'main query' type for Neo4J. Briefly, the site consists of posts, each tagged with various attributes, e.g. (its a travel site) location, theme, cost etc... Also the tags are hierarchical. So, for location we have (say) 'tuscany' inside 'italy' inside 'europe'. For theme we have (say) 'cycling' inside 'activity'. I can beautifully model the parent child relationships of these 'tags' using a graph Db as objects with a 'CHILD_OF' relationship type. Then the posts go in and have related 'tags' related to them with a 'TAGGED_WITH' relationship types. Fine. However, here is the complex bit (well to me): I need to be able to find all posts tagged with a set of tags (AND operation so each post must be tagged with each tag). Whats more, these queries need to respect the parent relationships in the tags. So if I search for 'activity' in 'france' it needs to traverse the CHILD_OF relationships to find things tagged with any child of 'activity' AND any child of 'France'. It seems pretty easy to write a traversal class that would find all posts in any child of ether 'tag' node, simply follow 'CHILD_OF' and 'TAGGED_WITH' backwards and return nodes of type post. However how to do the AND bit? The only way I can see is to return both lists and union them in user code... however that seems inelegant and may not scale brilliantly. Any ideas how to optimise? Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Map reduce style pagination [WAS: Traversers in the REST API]
Hi there... This is a branch of the thread on Traversers in the REST API. I just want to run an idea by you all. Tell me to shut up if I am being stupid. Quote from Tobias Ivarsson: As for sorting: yes, that is a comment on the API as a whole. We have opted at not providing sorting, since there are good sorting facilities available in the JRE already. Since that makes it easy for the user to implement their own sorting it would be sub optimal for Neo4j to provide sorting. Since sorting is a costly operation (both in time and space) it should be done as late in the process as possible, probably with a lot of user code in between the traversal and the place where the sorting actually takes place. This has been our thinking in the REST API as well, meaning that sorting will be left to the client. It is possible that we will return to this decision and add sorting to the REST API, and that it might trickle down to the core API. Features like this are however much easier to add than to remove, which is why it is not implemented at the moment. Thinking about this, it must be possible for the core engine to do a better job at returning a sorted and limited result set compared to doing it in user code. Doing it in user code requires all matching nodes to be passed to user code and a full sort to take place before pagination. Thats time complexity O(n log n) (worst case given a merge sort in the JRE), plus the memory implications of passing all the node objects. Doing it with a map reduce style system in the core engine would allow you to maintain sorted result set during traversal that optimised for (what must be) the most common case of wanting 'LIMIT N OFFSET M' items, when N+M is small (e.g. the first few pages of a query). Ok, so say that the 'return evaluator' returns some sortable object instead of a boolean, or null if that node is not returned. During a traversal for LIMIT N OFFSET M items, you maintain a sorted list of N+M slots. As each comparable 'sort' value is emitted from the return evaluator its is inserted in the correct place in the list. Once the list is full, you can maintain a pointer to the lowest value and quickly throw away the majority (as long as N+M is small relative to the number of items). I think this algorithm has complexity O(n z) where z is the time complexity of inserting an element in the sorted list (I suppose N+M). So this method would be very fast at returning a sorted list of of the first few pages of a result set for a query that touches lots of nodes. It would, of course, degrade as N+M approaches the size of all the nodes visited. I suspect that many real life large scale applications use this technique. Take for example google what will return page 1 much faster than page 10 of a search. Any thoughts? Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] How to efficiently query in Neo4J?
H... I am guessing the most efficient way might be to have a two stage return evaluator. E.g. The custom return evaluator class has a hash table of node id = count pairs. Each time 'isReturnableNode' is called, it increments the count for that node id in the hash. If count = total number of tags to check for return the node. Thus, if you run a traversal starting from each tag using the same instance of the custom return evaluator, none of the traversals will emit any nodes (good for memory!) until the last one, where the ones matching all the tags will be emitted. Does that sound like it would work? Al On 7 April 2010 16:20, Max De Marzi Jr. maxdema...@gmail.com wrote: I've had similar issues and they way I've done it (which may not be the right way) is to run the first traversal and store the returned nodes. Then run the second traversal and return only if it is contained in the set of returned nodes in the first traversal. The traverses hit each node only once, and since we want to return only if they are found twice, I don't think there is a clean way to do it in a single traversal. On Wed, Apr 7, 2010 at 9:53 AM, Alastair James al.ja...@gmail.com wrote: Hi there... I am looking at moving a website to a model based on Neo4J, however, I am having trouble seeing how to optimise the 'main query' type for Neo4J. Briefly, the site consists of posts, each tagged with various attributes, e.g. (its a travel site) location, theme, cost etc... Also the tags are hierarchical. So, for location we have (say) 'tuscany' inside 'italy' inside 'europe'. For theme we have (say) 'cycling' inside 'activity'. I can beautifully model the parent child relationships of these 'tags' using a graph Db as objects with a 'CHILD_OF' relationship type. Then the posts go in and have related 'tags' related to them with a 'TAGGED_WITH' relationship types. Fine. However, here is the complex bit (well to me): I need to be able to find all posts tagged with a set of tags (AND operation so each post must be tagged with each tag). Whats more, these queries need to respect the parent relationships in the tags. So if I search for 'activity' in 'france' it needs to traverse the CHILD_OF relationships to find things tagged with any child of 'activity' AND any child of 'France'. It seems pretty easy to write a traversal class that would find all posts in any child of ether 'tag' node, simply follow 'CHILD_OF' and 'TAGGED_WITH' backwards and return nodes of type post. However how to do the AND bit? The only way I can see is to return both lists and union them in user code... however that seems inelegant and may not scale brilliantly. Any ideas how to optimise? Al ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Dr Alastair James CTO James Publishing Ltd. http://www.linkedin.com/pub/3/914/163 www.worldreviewer.com WINNER Travolution Awards Best Travel Information Website 2009 WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 WINNER Travolution Awards Best New Online Travel Company 2008 WINNER Travel Weekly Magellan Award 2008 WINNER Yahoo! Finds of the Year 2007 Noli nothis permittere te terere! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Neo4j RESTful traversal API
HI there. Loving Neo4j, but been putting off using because was no * straightforward* way to use from PHP. I even started writing my own RESTful API, but never found the time to finish it off! However, I am excited that there is now a (semi?) official one in the works, even if it is early days. I have started a PHP library for accessing neo4j via the REST API... I will share when its ready. I can see how most of the functionality works from the Java functional tests and the application.wadl index. However, there is no mention of any way to access neo4j's traversal features. I was wondering how this is planned to work? In my API I had something like: node/1/traverse?relationship_type=KNOWS Which would traverse from node 1 along relationship type 'KNOWS' etc... However, it started to get complex very quickly as you need separate return and stop evaluators. I suppose, the most beautiful method would to to traverse via Gremlin: /node/1/outE/i...@age27] (obviously URI encoded) That would be dead sexy. However, I know the gremlin guys seem a little reluctant to offer a RESTful API (they seem to be stuck in their Java ways)... Whats the plan? Cheers Alastair ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Neo4j RESTful traversal API
Cheers Peter. Al On 22 March 2010 15:26, Peter Neubauer neubauer.pe...@gmail.com wrote: Btw, the wiki etc is up again - sorry for the inconvenience! Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. On Mon, Mar 22, 2010 at 10:59 AM, Peter Neubauer neubauer.pe...@gmail.com wrote: Guys, we are working on getting the site up again - our provider changed servers on us! Sorry for the inconvenience! Regarding REST, we are back on it starting this week, shortly Mattias will outline our way forward and ask for input from you all, so tune in and contribute! Good to see some early testers emerging here :) Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.tinkerpop.com - Processing for Internet-scale graphs. http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party. On Mon, Mar 22, 2010 at 10:36 AM, andrew.pot...@gmx.net wrote: Alastair, that sounds very cool. I'm very interested in getting Neo4j to workwith PHP as well. And like you I've had to put development onto the back burner for a while. I'd like to help somehow but with your specific question about traversal, I'm not sure I can be of much help. I don't know if you've considered the other alternatives. I've added quite a bit of info to the Neo-Wiki about using PHP. Maybe you have seen it already. I'd give you the direct link but the neo4j website is currently down :( Mainly I did some testing to show that it is feasible to access ne4j from PHP using javabridge. I thionk Peter has developed this further (there is a post somewhere in the mailing list). Javabride is one alternative. there are two others. One the RESTful API and another would be direct integration into the PHP language similar to the way MongoDB is integrated. I wanted to experiment with both of theses but haven't gotten around to it. I'm glad your'e working in this direction also. Andy Potter On Monday 22 March 2010 10:12:38 Alastair James wrote: HI there. Loving Neo4j, but been putting off using because was no * straightforward* way to use from PHP. I even started writing my own RESTful API, but never found the time to finish it off! However, I am excited that there is now a (semi?) official one in the works, even if it is early days. I have started a PHP library for accessing neo4j via the REST API... I will share when its ready. I can see how most of the functionality works from the Java functional tests and the application.wadl index. However, there is no mention of any way to access neo4j's traversal features. I was wondering how this is planned to work? In my API I had something like: node/1/traverse?relationship_type=KNOWS Which would traverse from node 1 along relationship type 'KNOWS' etc... However, it started to get complex very quickly as you need separate return and stop evaluators. I suppose, the most beautiful method would to to traverse via Gremlin: /node/1/outE/i...@age27] (obviously URI encoded) That would be dead sexy. However, I know the gremlin guys seem a little reluctant to offer a RESTful API (they seem to be stuck in their Java ways)... Whats the plan? Cheers Alastair ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Dr Alastair James CTO James Publishing Ltd. http://www.linkedin.com/pub/3/914/163 www.worldreviewer.com WINNER Travolution Awards Best Travel Information Website 2009 WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 WINNER Travolution Awards Best New Online Travel Company 2008 WINNER Travel Weekly Magellan Award 2008 WINNER Yahoo! Finds of the Year 2007 Noli nothis permittere te terere! ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
