Re: [Neo4j] Neo4j starter Question. Please help.
2011/3/29 PALASH JHABAK palash.jha...@gmail.com Hi, I am totally new to the Neo4j graph database. I am using embedded Neo4j with Python. I have learned to make graphs using the Neorj-python API. Next I wish to calculate, in the big graph, things like Shortest path between nodes, if any, list of all paths between them etc. I was wondering if there is an API similar to the REST api which I could in python for embedded Neo4j to calculate above and other similar things. Hi Palash, There's a new Neo4j python binding at the works and I don't know if any graph algorithm stuff will be included natively in there... but things like shortest path is written in java and exists in the graph-algo componenthttp://components.neo4j.org/neo4j-graph-algo/milestone/(included with the neo4j distribution http://neo4j.org/download). Some of its functionality (shortest path, all paths, all simple paths, dijkstra) is exposed via the REST API http://components.neo4j.org/neo4j-server/milestone/rest.html also. These things could also be exposed in the python bindings, but if that wouldn't be the case you can use java libraries from within Jython seamlessly as a solution, right? Kindly help. Thanks and Regards, Palash Jhabak Junior Undergraduate, Department. of Electrical Engineering IIT Bombay +91-9930490752 m...@palashjhabak.com sms palashjhabak to 56070 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] the update of the index
2011/3/30 孤竹 ho...@foxmail.com Hi,all I am wonder what's the principle of the update of Neo4j ? I think Neo4j use Lucene, but the Lucene after version 2.9, some little part change may cause all change of the index . Is this a problem for Neo4j? Could you provide more information here? Is there a particular change between lucene version 2.9 and 3.0 which makes lucene modify the entire index? Why and in what scenarios would that matter? Also the use of lucene is confined to indexes, and not used in any part of the graph itself. Best, Mattias ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Thanks for the answers.
I'll take here point 5 and 6.
5) does neo4j have a replication tool? I.e. is it possible to sync a
remote and a embedded database instance? Hibernate used to help here.
Are there tools to help?
Yes. There are two. There is a hot backup option for making backups of live
running databases. And there is the relatively new HA (high availability)
infrastructure for keeping clusters of databases synchronized. The various
databases can be running as embedded or as server, it does not matter, they
can still be synchronized. There are rules, however as to which is master,
and which is best to write to. See
http://wiki.neo4j.org/content/High_Availability_Cluster
That sounds nice. My scenario is something like: I have a centralized
database. On the Desktop side I have a workstation on which I do GIS
analysis. People want to get a chunk of data of interest, so they can
pollute them with their analyses until they are happy. So it is a
bit the concept of a distributed versioning system. I have my local
workspace, on which I play, then I could push back that result I
liked.
Anything like that around? :)
Another way is doing online backups,
http://docs.neo4j.org/chunked/milestone/operations-backup.html
6) Timeseries. The only way to hande then seems to be the timeline, right?
[...]
I have recently added a TimelineIndex (with a LuceneTimeline implementation
class) which uses an Index as backend instead of the graph and may be
faster.
See
https://github.com/neo4j/graphdb/blob/master/lucene-index/src/test/java/org/neo4j/index/timeline/TestTimeline.javafor
an example of the source and
http://components.neo4j.org/neo4j/1.3.M05/apidocs/org/neo4j/index/lucene/TimelineIndex.htmlfor
javadoc of the interface.
I am not sure if I do not see it or if it is not there.
I see how a timeline index is build. But then, how do I get that
timeline from a just opened database instance?
It would be really great to have s simple example like the matrix ones
on the wiki, in which a timeline is created, the database is closed,
then reopened and the timeline is queried.
If I am just blinded by too much info at one time, feel free to smash
me with a link/hint.
Thanks,
Andrea
I have no experience with the timeline class. I have always rolled my own
time index, and it was fast.
However, the most likely issue you are facing is with too many
transactions.
Group your commits. The easiest way to do this is every 1000, or 1
(pick
a number between these two :-), just do
tx.success();tx.finish();tx=db.beginTx();
The above number of 9000 could be added in one single commit. So move the
try{]finally{} around the entire loop. Then add the intermediate commits as
described above if you think you will get more data than that. I personally
commit every 1000. I found going to a bigger number helped, but not that
much. Most of the performance gains are achieved in the first 1000.
And how do I run to query it? I couldn't find any docs and testcase to
get the timeline from a database instance and to query it.
I don't know that class well enough to answer, sorry.
7) Foreign keys. Do they always simply get Relationships?
A foreign key is an rdbms concept. You could do it the rdbms way by storing
a property on one node that can be looked up in an index to get the other
node, but that would be like deliberately not using the graph. So it is
generally correct to model a 'foreign key' as a relationship. But perhaps
I'm not understanding what you mean by 'do they get relationships'?
(there is not, as far as I remember, any mention of the term 'foreign key'
in the neo4j docs, so perhaps you should give an example of what you mean)
Well, that is it for a first round. If you find the time to answer this,
thanks.
Hope the answers helped.
Regards, Craig
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Hi Craig, I am cc'ing the list with these answers, since I think there are questions here others know much more about. I will answer all with what I know, or think I know :-) thanks :) 1) what can I put into the node? I see that the superclass proposes Property types. So I was wondering if blogs are completely out of question. You can put any java primitive, strings and arrays of primitives and strings. Since anything can be serialized to byte[], you can store anything you want really. But practically it is considered no-ideal to store very large blobs or strings due to reduced performance. But my understanding is that blogs are typically small enough to not be a problem for this. So just store the blog contents as a string. Ok, in fact it shouldn't be a performance downgrade even with large blobs, right? It just depends on whether the queried part refers to an id or similar and that node then is simply connected to the blob. I.e. I extract the blob only when I am sure it is the wanted. Is that what you meant? 2) could neoclipse act like a kind query engine for udig? Or perhaps be easily adapted? I am wondering how an advanced user could browse the database. There are a lot of ways of browsing the database, especially for web user interfaces. See http://wiki.neo4j.org/content/Visualization_options_for_graphs I have a modified and embedded version of neoclipse inside my AWE application (see the online videos on youtube and vimeo). I use it to allow advanced users to browse the database :-) Well, your users must be very advanced and I am not :) Neoclipe is really a very nice tool, but I tried to open a created timeline with it and it almost exploded, which is understandable. I am allowed to choose the depth, but am not allowed to choose what data to see. I can't say show me the data between 5 and 6 in the morning or whatever else. Or maybe am I missing something? Probably because of the database type the old way of having a look at the data is not possible any longer. But then which is the right way? Having a console and let Gremlin shine? [...] 4) Indexes in rdbm are done on a table basis and every new record gets inserted. Now you have to add the values to the index? It looks to me as if you index only what you want to, right? But while not indexing in an rdbm leads to slover results, in this case the result is missing? This is only due to there being two search API's. If you search using the index, you get answers from the index. If you search using the graph you get answers from the graph. Many questions are actually faster using the graph, so you should not be too quick to use an index at all. In fact, dare I say it, if you need an index perhaps you have not modeled the graph correctly :-) (having said that, I do use the index myself, but less often than the graph). In an RDBMS the normal, non-indexed search is extremely slow (brute force search), and the index is a drop-in replacement for that. But in the graph, the only brute force search would be a full database scan, which no-one sane would do... Instead the graph search requires knowledge of the graph structure, and therefor the search query can be complex, and by definition completely different to an index search. So the graph search and index search are far too different to be placed behind the same API. However, I could imagine that some object db wrappers like neo4j.rb and jo4neo might be able to do this, since they have influence over, and knowledge of, the graph structure they create. Ok, I change my question. What do you do when you have two big types of data, one that does perfectly fit in the graph concept, and one the really doesn't have anything to do with it? I guess you put everything into the neo4j db and then query one with the graph traverser and the other one with the lucene indexer? My questions might seem a bit dummy, I apologize for that, I am trying to understand why and how I should make use of a graph database. [...] 7) Foreign keys. Do they always simply get Relationships? A foreign key is an rdbms concept. You could do it the rdbms way by storing a property on one node that can be looked up in an index to get the other node, but that would be like deliberately not using the graph. So it is generally correct to model a 'foreign key' as a relationship. But perhaps I'm not understanding what you mean by 'do they get relationships'? (there is not, as far as I remember, any mention of the term 'foreign key' in the neo4j docs, so perhaps you should give an example of what you mean) Sorry, this was a language glitch: do they get relationships == are they always modeled to relationships when migrating to a graph concept And your answer is what I expected. Well, that is it for a first round. If you find the time to answer this, thanks. Hope the answers helped. Regards, Craig Thanks, Andrea ___ Neo4j mailing list
Re: [Neo4j] question
6) Timeseries. The only way to hande then seems to be the timeline, right? [...] I have recently added a TimelineIndex (with a LuceneTimeline implementation class) which uses an Index as backend instead of the graph and may be faster. See https://github.com/neo4j/graphdb/blob/master/lucene-index/src/test/java/org/neo4j/index/timeline/TestTimeline.javafor an example of the source and http://components.neo4j.org/neo4j/1.3.M05/apidocs/org/neo4j/index/lucene/TimelineIndex.htmlfor javadoc of the interface. I am not sure if I do not see it or if it is not there. I see how a timeline index is build. But then, how do I get that timeline from a just opened database instance? It would be really great to have s simple example like the matrix ones on the wiki, in which a timeline is created, the database is closed, then reopened and the timeline is queried. If I am just blinded by too much info at one time, feel free to smash me with a link/hint. Such a timeline is just a thin wrapper around a normal Index, so just do: TimelineIndex timeline = new LuceneTimeline( graphDb, graphDb.index().forNodes( myTimeline ) ); irregardless of you've used/populated it before. No difference. Thanks, Andrea ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Hi! See http://wiki.neo4j.org/content/Visualization_options_for_graphs I have a modified and embedded version of neoclipse inside my AWE application (see the online videos on youtube and vimeo). I use it to allow advanced users to browse the database :-) Well, your users must be very advanced and I am not :) Neoclipe is really a very nice tool, but I tried to open a created timeline with it and it almost exploded, which is understandable. I am allowed to choose the depth, but am not allowed to choose what data to see. I can't say show me the data between 5 and 6 in the morning or whatever else. Or maybe am I missing something? You can choose which relationship types and directions to view, but that's it. Try decreasing the maximum number of nodes to show in the preferences, if you're OK with some missing data in the visualization. To just browse a graph and look at the structure it can be useful to bring the maximum down to 20 or so. /anders Probably because of the database type the old way of having a look at the data is not possible any longer. But then which is the right way? Having a console and let Gremlin shine? [...] 4) Indexes in rdbm are done on a table basis and every new record gets inserted. Now you have to add the values to the index? It looks to me as if you index only what you want to, right? But while not indexing in an rdbm leads to slover results, in this case the result is missing? This is only due to there being two search API's. If you search using the index, you get answers from the index. If you search using the graph you get answers from the graph. Many questions are actually faster using the graph, so you should not be too quick to use an index at all. In fact, dare I say it, if you need an index perhaps you have not modeled the graph correctly :-) (having said that, I do use the index myself, but less often than the graph). In an RDBMS the normal, non-indexed search is extremely slow (brute force search), and the index is a drop-in replacement for that. But in the graph, the only brute force search would be a full database scan, which no-one sane would do... Instead the graph search requires knowledge of the graph structure, and therefor the search query can be complex, and by definition completely different to an index search. So the graph search and index search are far too different to be placed behind the same API. However, I could imagine that some object db wrappers like neo4j.rb and jo4neo might be able to do this, since they have influence over, and knowledge of, the graph structure they create. Ok, I change my question. What do you do when you have two big types of data, one that does perfectly fit in the graph concept, and one the really doesn't have anything to do with it? I guess you put everything into the neo4j db and then query one with the graph traverser and the other one with the lucene indexer? My questions might seem a bit dummy, I apologize for that, I am trying to understand why and how I should make use of a graph database. [...] 7) Foreign keys. Do they always simply get Relationships? A foreign key is an rdbms concept. You could do it the rdbms way by storing a property on one node that can be looked up in an index to get the other node, but that would be like deliberately not using the graph. So it is generally correct to model a 'foreign key' as a relationship. But perhaps I'm not understanding what you mean by 'do they get relationships'? (there is not, as far as I remember, any mention of the term 'foreign key' in the neo4j docs, so perhaps you should give an example of what you mean) Sorry, this was a language glitch: do they get relationships == are they always modeled to relationships when migrating to a graph concept And your answer is what I expected. Well, that is it for a first round. If you find the time to answer this, thanks. Hope the answers helped. Regards, Craig Thanks, Andrea ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Filter by relationship Property
Hi,
I am trying traverse paths, instead of Nodes.. I am trying the below traverse
to get the paths filter by the relationship property customerId.
The filter not working, please let me know is there a way to get the filter
working...
{order: breadth first,return filter: {
language: javascript,
body: position.length() 0
position.lastRelationship().getProperty('customerId',0) == 1
},max depth: 1,algorithm: allPaths,to:
http://localhost:7474/db/data/node/492}
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Filter by relationship Property
It looks like you're combining a request to a traversal with a request for
paths (using a graph algorithm) and that won't work. But since allPaths is
a very simple traverser with a return filter I think you'll get desired
results with this traversal:
{
order: breadth first,
max depth: 1,
return filter :
{
language: javascript,
body: position.length()0 position.endNode().getId()==492
position.lastRelationship().getProperty('customerId',0)==1
}
}
Hmm, when I write this I see that max depth is 1... are you just listing
relationships between two nodes with a certain property, customerId=1? I
would like to say that you can get
relationshipshttp://components.neo4j.org/neo4j-server/milestone/rest.html#Get_relationships_on_nodeinstead,
but I realize that you can't supply a filter to it.
2011/3/30 Justine Mathews justine.math...@megree.com
Hi,
I am trying traverse paths, instead of Nodes.. I am trying the below
traverse to get the paths filter by the relationship property customerId.
The filter not working, please let me know is there a way to get the filter
working...
{order: breadth first,return filter: {
language: javascript,
body: position.length() 0
position.lastRelationship().getProperty('customerId',0) == 1
},max depth: 1,algorithm: allPaths,to:
http://localhost:7474/db/data/node/492}
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Hi Anders, [...] Well, your users must be very advanced and I am not :) Neoclipe is really a very nice tool, but I tried to open a created timeline with it and it almost exploded, which is understandable. I am allowed to choose the depth, but am not allowed to choose what data to see. I can't say show me the data between 5 and 6 in the morning or whatever else. Or maybe am I missing something? You can choose which relationship types and directions to view, but that's it. Try decreasing the maximum number of nodes to show in the preferences, if you're OK with some missing data in the visualization. To just browse a graph and look at the structure it can be useful to bring the maximum down to 20 or so. Yes, I understand it, but that just helps the visualization overload. There is no way to view all the aliens that have blue skin and age 10 years from a neo4j database, if not in a lucene search engine, right? Perhaps I am thinking too rdbm, but how do I overview my data without lines of code? How do I do a simple count on the blue aliens? Tanks, Andrea /anders [...] ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
2011/3/30 andrea antonello andrea.antone...@gmail.com Thanks Mattias, [...] Such a timeline is just a thin wrapper around a normal Index, so just do: TimelineIndex timeline = new LuceneTimeline( graphDb, graphDb.index().forNodes( myTimeline ) ); irregardless of you've used/populated it before. No difference. Ok, with the lucene index it worked for me. So one more questions. Assume you have a set of timeseries that represent precipitation data from different meteo stations. Usually (in rdbm) I would hold them in a table and index station id and time). Then I would query over a time interval for a certain station. With the timeline index it has to be done the way around, right? I.e. I create an timelineindex for each station that will hold all the data for that station? But that way I will have to know how the various indexes are named? What if another user gave me the database. Once I simply would have looked over the tables to see were teh needed data were. Now I have to understand where the data are and which indexes are handling them? I am a bit confused, sorry if I am pushing this. Thanks, Andrea I think for that the TimelineIndex interface would have to be extended to be able to hold additional data so that you can do compound querieshttp://docs.neo4j.org/chunked/milestone/indexing-lucene-extras.html#indexing-lucene-compoundto it and get exactly the functionality you're asking for with only one index. Another way is to just copy the LuceneTimeline code and roll this yourself, it's really small, mostly one-liners for each implemented method. Thanks, Andrea ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
I think for that the TimelineIndex interface would have to be extended to be able to hold additional data so that you can do compound queries http://docs.neo4j.org/chunked/milestone/indexing-lucene-extras.html#indexing-lucene-compound to it and get exactly the functionality you're asking for with only one index. Another way is to just copy the LuceneTimeline code and roll this yourself, it's really small, mostly one-liners for each implemented method. Alternatively just role your own graph-tree structure that provides the same capabilities. Then you can index any combination of properties together, to suite your planned queries. This is obviously much more work than Mattias suggestion, and does require that you know more about your domain (ie. less general). But it does allow you to inspect the index itself with graph traversals, gremlin or neoclipse, which is not possible with lucene. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Hi Andrea! Yes, I understand it, but that just helps the visualization overload. There is no way to view all the aliens that have blue skin and age 10 years from a neo4j database, if not in a lucene search engine, right? Neoclipse actually can use the integrated Lucene indices. Try the search button on top! You can't use it for filtering, but to get starting points for traversal in the graph. However, for now it's only able to perform Lucene queries on a single key (note: set the Search mode to Query), not compound queries. I'll add compound queries to my backlog, by it might take a week until I get around fixing it. Perhaps I am thinking too rdbm, but how do I overview my data without lines of code? How do I do a simple count on the blue aliens? I think Gremlin could be useful here. /anders Tanks, Andrea /anders [...] ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
2011/3/30 Craig Taverner cr...@amanzi.com I think for that the TimelineIndex interface would have to be extended to be able to hold additional data so that you can do compound queries http://docs.neo4j.org/chunked/milestone/indexing-lucene-extras.html#indexing-lucene-compound to it and get exactly the functionality you're asking for with only one index. Another way is to just copy the LuceneTimeline code and roll this yourself, it's really small, mostly one-liners for each implemented method. Alternatively just role your own graph-tree structure that provides the same capabilities. Then you can index any combination of properties together, to suite your planned queries. This is obviously much more work than Mattias suggestion, and does require that you know more about your domain (ie. less general). But it does allow you to inspect the index itself with graph traversals, gremlin or neoclipse, which is not possible with lucene. It is inspectable via tools such as Luke http://code.google.com/p/luke/but you can't do that runtime. I don't know what kind of inspection you'd had in mind though. Indexes have support for indexing of any combination of properties as well and is already implemented, so it's probably an easier solution, as you stated Craig :) ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
My experience with using large graph trees for indexes has been mixed, with performance issues under heavy read/write load, perhaps due to the many potential locks required during insertions. We switched to the timeline index, fwiw. - Reply message - From: Craig Taverner cr...@amanzi.com Date: Wed, Mar 30, 2011 7:43 am Subject: [Neo4j] question To: Neo4j user discussions user@lists.neo4j.org I think for that the TimelineIndex interface would have to be extended to be able to hold additional data so that you can do compound queries http://docs.neo4j.org/chunked/milestone/indexing-lucene-extras.html#indexing-lucene-compound to it and get exactly the functionality you're asking for with only one index. Another way is to just copy the LuceneTimeline code and roll this yourself, it's really small, mostly one-liners for each implemented method. Alternatively just role your own graph-tree structure that provides the same capabilities. Then you can index any combination of properties together, to suite your planned queries. This is obviously much more work than Mattias suggestion, and does require that you know more about your domain (ie. less general). But it does allow you to inspect the index itself with graph traversals, gremlin or neoclipse, which is not possible with lucene. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Ok, in fact it shouldn't be a performance downgrade even with large blobs, right? It just depends on whether the queried part refers to an id or similar and that node then is simply connected to the blob. I.e. I extract the blob only when I am sure it is the wanted. Is that what you meant? If the blob is a property of a node, it is not loaded when you access that node, but when you access its properties. I do not know enough about the implications on performance with large blobs, only that it has been mentioned many times before that for really large blogs, rather store them somewhere else (eg. filesystem) and reference them from the graph (eg. path to file, url, etc.). But I still believe that blogs are not big enough to really be a concern, but perhaps someone more knowledgable can correct me here? Probably because of the database type the old way of having a look at the data is not possible any longer. But then which is the right way? Having a console and let Gremlin shine? Filtering the neoclipse view with relationship types and directions helps. Limiting the number of nodes returns helps a lot. I use 100 max. But use neoclipse as a visualization tool mostly for visualizing the structure, not for analytics. Ok, I change my question. What do you do when you have two big types of data, one that does perfectly fit in the graph concept, and one the really doesn't have anything to do with it? I guess you put everything into the neo4j db and then query one with the graph traverser and the other one with the lucene indexer? My questions might seem a bit dummy, I apologize for that, I am trying to understand why and how I should make use of a graph database. When I'm deciding between using a graph or using lucene, the size of the data is not really a factor, but its 'graphiness' :-) For example, if I have a property of very high diversity, like peoples names, then lucene is a natural choice. If you have a property with structure, like categories or tags, or inheritance, or other relationship concepts, then the graph is best. There are cases in the middle, for example I generally model numerical properties in the graph, but I think most others would use lucene. I use the graph because it naturally leads to statistics data. For example, if we use the time property, and collect all events in the same second and connect them to the same 1s time node, we now know the number of events in that second from the structure of the graph. Connect each 1s node to a 1min node, and we know how many seconds in that minute contained data, etc. Obviously this is a very simple special case, and usually I keep more statistical metadata in the graph tree than mere counters, but the result is that your index now contains lots of statistics you can query without even touching the original data nodes (ie. very fast statistics queries). ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
That sounds nice. My scenario is something like: I have a centralized database. On the Desktop side I have a workstation on which I do GIS analysis. People want to get a chunk of data of interest, so they can pollute them with their analyses until they are happy. So it is a bit the concept of a distributed versioning system. I have my local workspace, on which I play, then I could push back that result I liked. Anything like that around? :) Not that I know of, but the issue I believe has been tackled by many users of neo4j. It is quote domain specific, and so not easy to generalize, but probably not that hard to implement for a limited, specific domain. For example, I have a product that has three components, an Android client collecting data and posting JSON packets at a central neo4j server, which adds them to a graph. Then my desktop app, just like yours, queries the server for a subset of the data, duplicates that in its internal, local neo4j database, and performs statistical calculations on that. I do not (yet) publish these results back to the central server, so I have not dealt with any versioning or conflict resolution, but have thought about it (at least within the scope of my domain). ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Agreed, Rick. My opinion is the main reason to role your own index is to make use of domain specific optimizations not available with generic indices. In my case the main win is the combination of statistics result and index that is possible. But I have to confess, the real reason I started using graphs as indexes was just that I thought the graph concept so cool, I did not want to pollute it with something non-graphy. Foolish ideology, I know, and I grew out of that more than a year ago, but it did influence many of my early neo4j decisions :-) On Wed, Mar 30, 2011 at 1:49 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: My experience with using large graph trees for indexes has been mixed, with performance issues under heavy read/write load, perhaps due to the many potential locks required during insertions. We switched to the timeline index, fwiw. - Reply message - From: Craig Taverner cr...@amanzi.com Date: Wed, Mar 30, 2011 7:43 am Subject: [Neo4j] question To: Neo4j user discussions user@lists.neo4j.org I think for that the TimelineIndex interface would have to be extended to be able to hold additional data so that you can do compound queries http://docs.neo4j.org/chunked/milestone/indexing-lucene-extras.html#indexing-lucene-compound to it and get exactly the functionality you're asking for with only one index. Another way is to just copy the LuceneTimeline code and roll this yourself, it's really small, mostly one-liners for each implemented method. Alternatively just role your own graph-tree structure that provides the same capabilities. Then you can index any combination of properties together, to suite your planned queries. This is obviously much more work than Mattias suggestion, and does require that you know more about your domain (ie. less general). But it does allow you to inspect the index itself with graph traversals, gremlin or neoclipse, which is not possible with lucene. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
2011/3/30 Craig Taverner cr...@amanzi.com Agreed, Rick. My opinion is the main reason to role your own index is to make use of domain specific optimizations not available with generic indices. In my case the main win is the combination of statistics result and index that is possible. But I have to confess, the real reason I started using graphs as indexes was just that I thought the graph concept so cool, I did not want to pollute it with something non-graphy. Foolish ideology, I know, and I grew out of that more than a year ago, but it did influence many of my early neo4j decisions :-) I know that feeling :) On Wed, Mar 30, 2011 at 1:49 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: My experience with using large graph trees for indexes has been mixed, with performance issues under heavy read/write load, perhaps due to the many potential locks required during insertions. We switched to the timeline index, fwiw. - Reply message - From: Craig Taverner cr...@amanzi.com Date: Wed, Mar 30, 2011 7:43 am Subject: [Neo4j] question To: Neo4j user discussions user@lists.neo4j.org I think for that the TimelineIndex interface would have to be extended to be able to hold additional data so that you can do compound queries http://docs.neo4j.org/chunked/milestone/indexing-lucene-extras.html#indexing-lucene-compound to it and get exactly the functionality you're asking for with only one index. Another way is to just copy the LuceneTimeline code and roll this yourself, it's really small, mostly one-liners for each implemented method. Alternatively just role your own graph-tree structure that provides the same capabilities. Then you can index any combination of properties together, to suite your planned queries. This is obviously much more work than Mattias suggestion, and does require that you know more about your domain (ie. less general). But it does allow you to inspect the index itself with graph traversals, gremlin or neoclipse, which is not possible with lucene. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Anyone going to NextConf?
Hi there, seems http://nextconf.eu/next11/ is something to go for this summer and talk graphs - anyone attending from this community? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Anyone going to NextConf?
Hi! thanks you for noticing!! As a new Berliner I will probably try to go. Pere 2011/3/30 Peter Neubauer peter.neuba...@neotechnology.com Hi there, seems http://nextconf.eu/next11/ is something to go for this summer and talk graphs - anyone attending from this community? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Anyone going to NextConf?
I correct myself, too much money!! Pere If anyone come to town, we can share some beers! 2011/3/30 Pere Urbon Bayes p...@moviepilot.com Hi! thanks you for noticing!! As a new Berliner I will probably try to go. Pere 2011/3/30 Peter Neubauer peter.neuba...@neotechnology.com Hi there, seems http://nextconf.eu/next11/ is something to go for this summer and talk graphs - anyone attending from this community? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Graph visualization in the web admin tool
We opted to change from graphdracula to arbor.js though (today, actually), because the dracula lib was buggy. My initial reaction to arbor.js is very positive, although it wasn't quite as straight-forward as dracula to use. I'd recommend downloading arbor.js and tweaking their example apps to get a feel for how the lib works. It's been fairly unpainful to work this into the webadmin app, together with neo4js you can rather quickly get up to speed visualizing data from a neo4j server on the web :) /Jacob On Mon, Mar 28, 2011 at 3:50 PM, Clement Honore honor...@gmail.com wrote: Thanks for your answer Peter. I will take a look at dracula. I've already check this link http://wiki.neo4j.org/content/Visualization_options_for_graphs# (this is the on this page on my first post but I forget the link... ^^) and there is so many solutions that I don't know which to choose... 2011/3/28 Peter Neubauer peter.neuba...@neotechnology.com Clement, the current iteration is building on http://www.graphdracula.net/ which is built on top of RaphaelJS I think. There are many others, look at http://wiki.neo4j.org/content/Visualization_options_for_graphs# and pick your choice. Let us know your experiences and feel free to contribute to the webadmin! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Mar 28, 2011 at 1:54 PM, Clement Honore honor...@gmail.com wrote: Hi! Since the 1.0 M5 release, it's possible to get a visual representation of the graph within the web administration tool. I've almost the same kind of representation to do in a web application and I wonder which library did you use. Is it something referenced on this page or something completely written by yourselves ? Thanks for the tips! Clément. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Graph visualization in the web admin tool
Thanks for the update. For my part, I think I'll give a try to Javascript Infovis Toolkit (http://thejit.org). I think you should take a look at it. The API seems very complete and the project is well documented (and activ). The objects are very customizable, edge should be oriented or not, you can add events on nodes or edges, change styles... and I like the RGrpah visualization ^^ (but ForceDirected is maybe more what you need). Kind regards 2011/3/30 Jacob Hansson jacob.hans...@neotechnology.com We opted to change from graphdracula to arbor.js though (today, actually), because the dracula lib was buggy. My initial reaction to arbor.js is very positive, although it wasn't quite as straight-forward as dracula to use. I'd recommend downloading arbor.js and tweaking their example apps to get a feel for how the lib works. It's been fairly unpainful to work this into the webadmin app, together with neo4js you can rather quickly get up to speed visualizing data from a neo4j server on the web :) /Jacob On Mon, Mar 28, 2011 at 3:50 PM, Clement Honore honor...@gmail.comwrote: Thanks for your answer Peter. I will take a look at dracula. I've already check this link http://wiki.neo4j.org/content/Visualization_options_for_graphs# (this is the on this page on my first post but I forget the link... ^^) and there is so many solutions that I don't know which to choose... 2011/3/28 Peter Neubauer peter.neuba...@neotechnology.com Clement, the current iteration is building on http://www.graphdracula.net/ which is built on top of RaphaelJS I think. There are many others, look at http://wiki.neo4j.org/content/Visualization_options_for_graphs# and pick your choice. Let us know your experiences and feel free to contribute to the webadmin! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Mar 28, 2011 at 1:54 PM, Clement Honore honor...@gmail.com wrote: Hi! Since the 1.0 M5 release, it's possible to get a visual representation of the graph within the web administration tool. I've almost the same kind of representation to do in a web application and I wonder which library did you use. Is it something referenced on this page or something completely written by yourselves ? Thanks for the tips! Clément. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Google Summer of Code 2011
All, please feel free to join the fun, it will be awesome! And if you know others that might be interested, please spread the word! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Mar 29, 2011 at 11:46 PM, Craig Taverner cr...@amanzi.com wrote: Hi all, Last year Neo4j was represented in the Google Summer of Code with two successful projects, in collaboration with Gephi and OSGeo. This year we are again interested in supporting GSoC projects within other open source organizations interested in integrating with Neo4j. The OSGeohttp://osgeo.orghas already welcomed our offer to mentor Neo4j Spatial http://components.neo4j.org/neo4j-spatial/snapshot/neo4j-spatial/projects within the OSGeo umbrella. We will be updating the Neo4j, uDig and possibly GeoTools and GeoServer wikis where necessary. If you have ideas that are not related to Neo4j Spatial, contact us on the mailing list and suggest which accepted organization we could partner with. If the idea is interesting enough, we should be able to find a mentor for it. For further information, here are some links to follow: - GSoC ideashttp://udig.refractions.net/confluence/display/HACK/Summer+of+Codeon the uDig wiki (focusing on Neo4j Spatial) - Neo4j GSoC Ideashttp://wiki.neo4j.org/content/Google_Summer_of_Code_Ideason the Neo4j wiki, with a wide range of interesting ideas - List of accepted GSoChttp://socghop.appspot.com/gsoc/program/accepted_orgs/google/gsoc2010organizations we could consider partnering with for GSoC projects Regards, Craig ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] cycle detection
Ok, I wrote a class to do the work without changing the graph
http://dl.dropbox.com/u/7137905/SCC_neo4j_tarjan.jar
It surely needs some improvements, for example it cannot manage too
large graphs (e.g. more than 10^6 nodes), but should work.
A note: it looks for Strongly Connected Components, subsets of the graph
where any node can be reached by any other node of the set without going
out of the set itself. A SCC bigger than one node implies the existence
of at least one cycle.
Cheers,
Jacopo
P.S.: the code create a set of nodes called s, that is truncated when
it's sure it contains a SCC. Would be a good idea to remove
corresponding nodes from the maps indexes and lowlinks, since they will
never be used, in order to consume less memory.
Il giorno mar, 29/03/2011 alle 10.19 +0200, Mattias Persson ha scritto:
2011/3/28 Jacopo jacopo.far...@email.it
Uh, I may create a node and use relationships with it instead of node
properties, to delete it after the work, but it doesn't sound a good
solution.
Letting a graph algorithm set temporary properties and creating other
entities (thus holding write locks and memory) seems like a bad idead. Hold
these things in a Set or Map instead.
Jacopo
Il giorno lun, 28/03/2011 alle 21.23 +0200, Jacopo ha scritto:
There's no problem with it!
The only issue is that it needs to add properties to visited nodes in
order to be able to detect cycles. It's possible to delete them after
the work, but in case the graph already uses that properties name or the
program is interrupted before finishing it would be a problem. Is there
a way to create temporary properties?
Jacopo
Il giorno lun, 28/03/2011 alle 10.16 +0200, Peter Neubauer ha scritto:
Cool!
Would be great to maybe add this to the graph-algo package, if you
don't mind? Just fork and add it from
https://github.com/neo4j/graphdb/tree/master/graph-algo and do a merge
request ...
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph
database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing
party.
On Sun, Mar 27, 2011 at 10:56 PM, Jacopo jacopo.far...@email.it
wrote:
In case you are interested, I implemented cycle detection by using
Tarjan algorithm, not the traversal.
The code is visible in the Italian Wikipedia, I hope it's
intelligible
although the language.
http://it.wikipedia.org/wiki/Algoritmo_di_Tarjan_per_le_componenti_fortemente_connesse#Implementazione_in_Java
Hi
Jacopo Farina
Il giorno ven, 25/03/2011 alle 13.51 +0100, Mattias Persson ha
scritto:
I think you could implement this using RELATIONSHIP_GLOBAL
uniqueness, like
(from the top of my head):
Traversal.description().uniqueness( Uniqueness.RELATIONSHIP_GLOBAL
)
.evaluator(new Evaluator() {
public Evaluation(Path path) {
return path.length() 0
endNodeOccursPreviouslyInPath(
path ) ?
Evaluation.INCLUDE_AND_CONTINUE :
Evaluation.EXCLUDE_AND_CONTINUE;
}
private boolean endNodeOccursPreviouslyInPath(Path path) {
Node endNode = path.endNode();
int counter = 0;
for ( Node node : path.nodes() ) {
if ( counter++ path.length() node.equals(
endNode ) )
return true;
}
return false;
}
} ).traverse(...);
This should (if it even compiles :) ) return paths containing
cycles.
Something like this you're looking for?
2011/3/25 Wouter De Borger w.debor...@gmail.com
Hi all,
I would like to detect all cycles in a traversal.
I know the traversal framework has cycle avoidance built-in, but
there
doesn't seem to be an API for cycle detection!
Has anyone already implemented a cycle detector for traversals?
Thanks in advance,
Wouter
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Caselle da 1GB,
[Neo4j] Can I get a Neo4J server to participate in Spring managed transactions?
All, I'm in the process of evaluating some components for a new architecture and we're thinking about using Neo4J for the persistence solution. We have a couple of Spring 3 MVC applications running that host our RESTful API and would like them to transactionally communicate with a Neo4J server. Prototypes work fine when embedding Neo4J inside the Spring application but now we want to move the data server out into its own process. Is that possible? I've only seen discussions about Neo4J's RESTful API as a means of accessing the data in the server and that is not a transactional solution. Is it possible to do what I want or are we out of luck? Many Thanks, Ron Ron Kurr http://www.google.com/profiles/kurron/ ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Lucene index: Too many open files
Hi, during stress tests, I got a Too many open files message (see detailed stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, Core i7 which handles the load with ease. cat /proc/sys/fs/file-max 2399285 The parameter fs.file-max was already set very high, but I'll increase it further and see if it will happen again. fs.file-max = 6815744 fs.aio-max-nr = 1048576 Are there recommendations of linux kernel parameters? Or/and would you recommend other things to circumvent that, f.e. using multiple indexes instead of one? And it seems to me that finishing a Neo4j transaction will always trigger a LuceneTransaction.doCommit, even if there was no update on index (I try to avoid for the log nodes). Can this be switched off? Thanks and greetings Axel [1] Stack trace java.lang.RuntimeException: java.io.FileNotFoundException: /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too many open files) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) at org.structr.core.log.LogService.flushQueue(LogService.java:101) at org.structr.core.log.LogService.run(LogService.java:73) Caused by: java.io.FileNotFoundException: /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too many open files) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:233) at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.init(SimpleFSDirectory.java:180) at org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) at org.apache.lucene.index.FieldsWriter.init(FieldsWriter.java:86) at org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66) at org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144) at org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:193) at org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1460) at org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1479) at org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1099) at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:777) at org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:752) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1932) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1906) at org.neo4j.index.impl.lucene.LuceneTransaction.applyDocuments(LuceneTransaction.java:299) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:260) ... 12 more javax.transaction.xa.XAException: Unknown xid[GlobalId[NEOKERNL|5131995998687892543|61442], BranchId[ 52 49 52 49 52 49 ]] at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.rollback(XaResourceManager.java:470) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.rollback(XaResourceHelpImpl.java:111) at org.neo4j.kernel.impl.transaction.TransactionImpl.doRollback(TransactionImpl.java:533) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:648) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) at org.structr.core.log.LogService.flushQueue(LogService.java:101) at org.structr.core.log.LogService.run(LogService.java:73) Mar 30, 2011 6:31:53 PM org.neo4j.kernel.impl.transaction.TxManager commit SEVERE: Unable to rollback transaction. Some resources may be commited others not. Neo4j kernel should be SHUTDOWN for resource maintance
[Neo4j] Query paths of certain lengths...
Suppose I have the following node/paths n - n1 n - n2 n - n4 n2 - n1 n2 - n3 n2 - n4 I want to find all paths of depth two (for example): n - n2 - n3 n - n2 - n4 and filter out those paths where a shorter path exists to a given node (n) leaving just n - n2 - n3 since n - n4 is shallower than n - n2 - n4 Is this possible? I see GraphAlgoFactory.pathsWithLength(expander, length). But unclear on how to proceed beyond that. Jeff ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Filter by relationship Property
The solution is not really working for me...
Actually I am trying find paths between the Nodes, based on following criteria,
1) Combination of Relationships (which is working).
2) Maxdepth
3) Algoritham - allpaths
4) And also I only need paths of certain property value of relationship(In
our case the Property is customerId)
That is when we create relationship we add property(customerId) to it, based on
user group.
If I cannot filter, I will be pulling out more unwanted paths, which will not
really work that way for us..
Can all the above combination, will work together?
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
It looks like you're combining a request to a traversal with a request for
paths (using a graph algorithm) and that won't work. But since allPaths is a
very simple traverser with a return filter I think you'll get desired results
with this traversal:
{
order: breadth first,
max depth: 1,
return filter :
{
language: javascript,
body: position.length()0 position.endNode().getId()==492
position.lastRelationship().getProperty('customerId',0)==1
}
}
Hmm, when I write this I see that max depth is 1... are you just listing
relationships between two nodes with a certain property, customerId=1? I would
like to say that you can get
relationshipshttp://components.neo4j.org/neo4j-server/milestone/rest.html#Get_relationships_on_nodeinstead,
but I realize that you can't supply a filter to it.
2011/3/30 Justine Mathews
justine.math...@megree.commailto:justine.math...@megree.com
Hi,
I am trying traverse paths, instead of Nodes.. I am trying the below
traverse to get the paths filter by the relationship property customerId.
The filter not working, please let me know is there a way to get the
filter working...
{order: breadth first,return filter: {
language: javascript,
body: position.length() 0
position.lastRelationship().getProperty('customerId',0) == 1
},max depth: 1,algorithm: allPaths,to:
http://localhost:7474/db/data/node/492}
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/http://www.justinemathews.com%3chttp:/www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
___
Neo4j mailing list
User@lists.neo4j.orgmailto:User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] A picture worth a 170'000 nodes and 650'000 edges
Hi. Nice visualization! How long did it take to render this image? For 3m edges and 1m nodes it takes around 20min on a AMD PhenomII X4. I see. I think it's not bad for this size of network. Offtopic for Neo4j - does Cytoscape provide any Sparql support (as is advertised on the front page :-). I could not find any. There is a plugin called RDFScape, but it needs some updates to support new versions of Cytoscape: http://www.bioinformatics.org/rdfscape/wiki/Main/HomePage Handling huge RDF data sets in Cytoscape is an item on our TODO list, and that's one of the reasons we want to support Neo4j to appeal Cytoscape to semantic web community. Thanks, Kei http://chihuahua.ucsf.edu/cgi-bin/wikigsoc.pl (Idea #4) Thanks, Kei Cytoscape Core Developer: http://cytoscape.org/ 2011/3/24 Tobias Ivarsson tobias.ivars...@neotechnology.com: Nice to see my visualization code being put to good use. The most recent version has moved to github, along with the rest of Neo4j, since this is an unofficial tool it is on my personal github: https://github.com/thobe/neoviz Cheers, Tobias On Thu, Mar 24, 2011 at 8:09 PM, Alfredas Chmieliauskas al.fre...@gmail.com wrote: For fun: This is a visualization of the graph resulting from a simulation (interacting energy markets). http://test.eeni.tbm.tudelft.nl/~alfredas/d13n-graph.png After 10 simulation ticks we have ~170'000 entities (agents, markets, power plants, bids, substances, technologies, etc) and ~650'000 relations between these entities. Different colors represent different types of relations. These relations are created at a rate of ~60'000 per tick, as a result of agents trading, investing and energy flows. By the end of the simulation we get ~1m nodes and ~4m edges. The visualization was made using neo-graphviz: https://svn.neo4j.org/laboratory/components/neo-graphviz/ The crazy part is that we can still make sense of what's happening there, using a few pipes and traversals :-) Alfredas ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Keiichiro Ono http://www.keiono.net/ ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Keiichiro Ono http://www.keiono.net/ ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] question
Woot guys. That was a long reading :) Thanks for all the answers, I'll go back to prototyping. If i come up with something usefull, I would like to wrap up this discussion in a wiki page. Thanks, Andrea On Wed, Mar 30, 2011 at 2:14 PM, Mattias Persson matt...@neotechnology.com wrote: 2011/3/30 Craig Taverner cr...@amanzi.com Agreed, Rick. My opinion is the main reason to role your own index is to make use of domain specific optimizations not available with generic indices. In my case the main win is the combination of statistics result and index that is possible. But I have to confess, the real reason I started using graphs as indexes was just that I thought the graph concept so cool, I did not want to pollute it with something non-graphy. Foolish ideology, I know, and I grew out of that more than a year ago, but it did influence many of my early neo4j decisions :-) I know that feeling :) On Wed, Mar 30, 2011 at 1:49 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: My experience with using large graph trees for indexes has been mixed, with performance issues under heavy read/write load, perhaps due to the many potential locks required during insertions. We switched to the timeline index, fwiw. - Reply message - From: Craig Taverner cr...@amanzi.com Date: Wed, Mar 30, 2011 7:43 am Subject: [Neo4j] question To: Neo4j user discussions user@lists.neo4j.org I think for that the TimelineIndex interface would have to be extended to be able to hold additional data so that you can do compound queries http://docs.neo4j.org/chunked/milestone/indexing-lucene-extras.html#indexing-lucene-compound to it and get exactly the functionality you're asking for with only one index. Another way is to just copy the LuceneTimeline code and roll this yourself, it's really small, mostly one-liners for each implemented method. Alternatively just role your own graph-tree structure that provides the same capabilities. Then you can index any combination of properties together, to suite your planned queries. This is obviously much more work than Mattias suggestion, and does require that you know more about your domain (ie. less general). But it does allow you to inspect the index itself with graph traversals, gremlin or neoclipse, which is not possible with lucene. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene index: Too many open files
Sounds like a bug to me. - Reply message - From: Axel Morgner a...@morgner.de Date: Wed, Mar 30, 2011 1:23 pm Subject: [Neo4j] Lucene index: Too many open files To: Neo4j user discussions user@lists.neo4j.org Hi, during stress tests, I got a Too many open files message (see detailed stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, Core i7 which handles the load with ease. cat /proc/sys/fs/file-max 2399285 The parameter fs.file-max was already set very high, but I'll increase it further and see if it will happen again. fs.file-max = 6815744 fs.aio-max-nr = 1048576 Are there recommendations of linux kernel parameters? Or/and would you recommend other things to circumvent that, f.e. using multiple indexes instead of one? And it seems to me that finishing a Neo4j transaction will always trigger a LuceneTransaction.doCommit, even if there was no update on index (I try to avoid for the log nodes). Can this be switched off? Thanks and greetings Axel [1] Stack trace java.lang.RuntimeException: java.io.FileNotFoundException: /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too many open files) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) at org.structr.core.log.LogService.flushQueue(LogService.java:101) at org.structr.core.log.LogService.run(LogService.java:73) Caused by: java.io.FileNotFoundException: /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too many open files) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:233) at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.init(SimpleFSDirectory.java:180) at org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) at org.apache.lucene.index.FieldsWriter.init(FieldsWriter.java:86) at ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [Blog Post] Global vs. Local Graph Ranking
Good stuff Marko. This is why Neo4j was done. And Tinkerpop. /peter Send from my mobile device, please excuse typos and brevity. On Mar 30, 2011 11:57 PM, Marko Rodriguez okramma...@gmail.com wrote: Hi people of the guild, Thought many of you might be interested in a new blog post... This is a concept that few appreciate and that has been huge in my interpretation of graph algorithms. http://markorodriguez.com/2011/03/30/global-vs-local-graph-ranking/ Enjoy, Marko. http://markorodriguez.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Filter by relationship Property
Justine,
in order to get really full control over how you are using the algos,
would you be able to write a simple plugin that can handle this in
Java, along the lines of
http://docs.neo4j.org/chunked/snapshot/server-plugins.html ?
Otherwise, you can use any scripting language and send script code to
a plugin. We are right now doing a very simple one in order to execute
Gremlin (or any Groovy code) through the REST API, see
https://github.com/peterneubauer/neo4j-gremlin-plugin/blob/master/src/main/java/org/neo4j/server/plugin/gremlin/GremlinPlugin.java
for a very early draft.
Would a Java plugin work for you to start with? In there, you can use
the full API, like in
http://components.neo4j.org/neo4j-graph-algo/stable/
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Mar 30, 2011 at 8:04 PM, Justine Mathews
justine.math...@megree.com wrote:
The solution is not really working for me...
Actually I am trying find paths between the Nodes, based on following
criteria,
1) Combination of Relationships (which is working).
2) Maxdepth
3) Algoritham - allpaths
4) And also I only need paths of certain property value of
relationship(In our case the Property is customerId)
That is when we create relationship we add property(customerId) to it, based
on user group.
If I cannot filter, I will be pulling out more unwanted paths, which will not
really work that way for us..
Can all the above combination, will work together?
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
It looks like you're combining a request to a traversal with a request for
paths (using a graph algorithm) and that won't work. But since allPaths is
a very simple traverser with a return filter I think you'll get desired
results with this traversal:
{
order: breadth first,
max depth: 1,
return filter :
{
language: javascript,
body: position.length()0 position.endNode().getId()==492
position.lastRelationship().getProperty('customerId',0)==1
}
}
Hmm, when I write this I see that max depth is 1... are you just listing
relationships between two nodes with a certain property, customerId=1? I
would like to say that you can get
relationshipshttp://components.neo4j.org/neo4j-server/milestone/rest.html#Get_relationships_on_nodeinstead,
but I realize that you can't supply a filter to it.
2011/3/30 Justine Mathews
justine.math...@megree.commailto:justine.math...@megree.com
Hi,
I am trying traverse paths, instead of Nodes.. I am trying the below
traverse to get the paths filter by the relationship property customerId.
The filter not working, please let me know is there a way to get the
filter working...
{order: breadth first,return filter: {
language: javascript,
body: position.length() 0
position.lastRelationship().getProperty('customerId',0) == 1
},max depth: 1,algorithm: allPaths,to:
http://localhost:7474/db/data/node/492}
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/http://www.justinemathews.com%3chttp:/www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
___
Neo4j mailing list
User@lists.neo4j.orgmailto:User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Regards,
Justine K Mathews, MCSD.NET
Mob: +44-(0) 7795268546
http://www.justinemathews.comhttp://www.justinemathews.com/
http://uk.linkedin.com/in/justinemathews
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Can I get a Neo4J server to participate in Spring managed transactions?
You don't really need neo4j-server for your needs. If you just want to have multiple clients accessing a single neo4j-embedded DB you can just have Neo4j-Embedded run in HA mode within each of your spring apps. So you get higher throughput through read caching and have still transactional Neo4j-DB access. If you are able to use cache sharding you can also improve performance for certain scenarios: http://jim.webber.name/2011/02/23/abe72f61-27fb-4c1b-8ce1-d0db7583497b.aspx Cheers Michael Am 30.03.2011 um 19:07 schrieb Ronald Kurr: All, I'm in the process of evaluating some components for a new architecture and we're thinking about using Neo4J for the persistence solution. We have a couple of Spring 3 MVC applications running that host our RESTful API and would like them to transactionally communicate with a Neo4J server. Prototypes work fine when embedding Neo4J inside the Spring application but now we want to move the data server out into its own process. Is that possible? I've only seen discussions about Neo4J's RESTful API as a means of accessing the data in the server and that is not a transactional solution. Is it possible to do what I want or are we out of luck? Many Thanks, Ron Ron Kurr http://www.google.com/profiles/kurron/ ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
