[Neo4j] Fwd: Sync databases
I recently posted the following topic on the Gremlin users list and I've been directed here. Apparently Michael Hunger is the man I want to talk to :) It seems there are a few other people interested in finding out my results as well. Forwarded conversation Subject: Sync databases From: *Eddy Respondek* eddy.respon...@gmail.com Date: Wed, Sep 14, 2011 at 11:30 AM To: gremlin-us...@googlegroups.com This may be a little off topic but maybe someone has done something similar before. Basically I have a separate Wordpress site (php/mysql) which I've been extending significantly and I've setup another server on the same network for graph db testing (neo4j/tinkerpop/python-bulbs). I'm confident with my graph setup now and would like to attempt to get something small into development so I can monitor the results. I want to do a simple like relationship between users and articles. That means I need to keep an identical index of user ids and article ids in the graph db. I know how to update the id's when a new user or article is created, deleted, etc. What I don't know is the correct way to ensure data integrity in case something goes wrong like the graph db server crashes, etc. Does anyone have any thoughts on the best way to do this? -- From: *Marko Rodriguez* okramma...@gmail.com Date: Wed, Sep 14, 2011 at 11:44 AM To: gremlin-us...@googlegroups.com Hey Eddy, Someone might be able to help you here, but the guy who will give you the two page rattle on such matters is Michael Hunger on the Neo4j users list. I've read him talking about similar things --- cross db transactions-style stuff. You might want to post your thoughts to that list. Marko. http://markorodriguez.com -- From: *James Thornton* james.thorn...@gmail.com Date: Wed, Sep 14, 2011 at 12:34 PM To: gremlin-us...@googlegroups.com Hi Eddy - You should definitely post this question to the Neo4j list as well because I would be interested in Michael's ideas on this. One approach would be to use a message-passing library like ZeroMQ ( http://www.zeromq.org/) to set up a communication channel between PHP and Python. This will allow you to write to both MySQL and Neo4j when you create a new user. ZeroMQ is stupid fast. You can send millions of requests per second ( http://www.zeromq.org/results:10gbe-tests-v031), and it supports pub/sub and muticast so you can write to multiple devices/programs at once ( http://zguide.zeromq.org/page:all). Here are the PHP and Python bindings: * http://www.zeromq.org/bindings:php * http://www.zeromq.org/bindings:python JSON is probably the easiest way to serialize data, or you could use a binary serialization library like MessagePack. I'm working on creating a batch loader for Bulbs that uses ZeroMQ to send requests to a Java/Jython server running the Neo4jBatchGraph implementation Marko just added ( https://groups.google.com/d/topic/gremlin-users/muuylAEZKrQ/discussionZeroMQ) -- I'll post an example soon. - James ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Fwd: Sync databases
HI Eddy, the way I do it in java with JPA (relational) and graph: * store the row-primary key + table in a property of the node/relationship * index the table:key for the node I didn't change the relational schema at all: Then I hooked into events: * when a node is loaded I get table + id from the field and load the relational data too and connect the two * when a relational row is loaded I lookup the table:id in the index to get the node and connect the two * nodes are attached to relational rows, so whenever a row loaded that has no node, then one is created and the lookup information is stored as above That of course depends if you have to use the data together. Cheers Michael Am 14.09.2011 um 08:59 schrieb Eddy Respondek: I recently posted the following topic on the Gremlin users list and I've been directed here. Apparently Michael Hunger is the man I want to talk to :) It seems there are a few other people interested in finding out my results as well. Forwarded conversation Subject: Sync databases From: *Eddy Respondek* eddy.respon...@gmail.com Date: Wed, Sep 14, 2011 at 11:30 AM To: gremlin-us...@googlegroups.com This may be a little off topic but maybe someone has done something similar before. Basically I have a separate Wordpress site (php/mysql) which I've been extending significantly and I've setup another server on the same network for graph db testing (neo4j/tinkerpop/python-bulbs). I'm confident with my graph setup now and would like to attempt to get something small into development so I can monitor the results. I want to do a simple like relationship between users and articles. That means I need to keep an identical index of user ids and article ids in the graph db. I know how to update the id's when a new user or article is created, deleted, etc. What I don't know is the correct way to ensure data integrity in case something goes wrong like the graph db server crashes, etc. Does anyone have any thoughts on the best way to do this? -- From: *Marko Rodriguez* okramma...@gmail.com Date: Wed, Sep 14, 2011 at 11:44 AM To: gremlin-us...@googlegroups.com Hey Eddy, Someone might be able to help you here, but the guy who will give you the two page rattle on such matters is Michael Hunger on the Neo4j users list. I've read him talking about similar things --- cross db transactions-style stuff. You might want to post your thoughts to that list. Marko. http://markorodriguez.com -- From: *James Thornton* james.thorn...@gmail.com Date: Wed, Sep 14, 2011 at 12:34 PM To: gremlin-us...@googlegroups.com Hi Eddy - You should definitely post this question to the Neo4j list as well because I would be interested in Michael's ideas on this. One approach would be to use a message-passing library like ZeroMQ ( http://www.zeromq.org/) to set up a communication channel between PHP and Python. This will allow you to write to both MySQL and Neo4j when you create a new user. ZeroMQ is stupid fast. You can send millions of requests per second ( http://www.zeromq.org/results:10gbe-tests-v031), and it supports pub/sub and muticast so you can write to multiple devices/programs at once ( http://zguide.zeromq.org/page:all). Here are the PHP and Python bindings: * http://www.zeromq.org/bindings:php * http://www.zeromq.org/bindings:python JSON is probably the easiest way to serialize data, or you could use a binary serialization library like MessagePack. I'm working on creating a batch loader for Bulbs that uses ZeroMQ to send requests to a Java/Jython server running the Neo4jBatchGraph implementation Marko just added ( https://groups.google.com/d/topic/gremlin-users/muuylAEZKrQ/discussionZeroMQ) -- I'll post an example soon. - James ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Multi-master server architecture
Hi, I'm interested by the Multi-master feature like explained here http://www.assembla.com/spaces/orientdb/wiki/Multi-master_server_architecture. i'm also interested to discuss this proposition... What do you think ? What re you remarks ? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Multi-master-server-architecture-tp3335268p3335268.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene Index view tool
Which version are you using for luke? You might need to use the same version as your Lucene core. The latest one I think it's 3.xx Sent from my BlackBerry® wireless device -Original Message- From: consultpreet [via Neo4j Community Discussions] ml-node+s438527n3335822...@n3.nabble.com Date: Wed, 14 Sep 2011 06:43:17 To: noppanitnoppani...@gmail.com Subject: Lucene Index view tool Hi All, could some one please suggest a tool to view and query Lucene Index, i tried Luke http://www.getopt.org/luke/ http://www.getopt.org/luke/ , but it wouldn't recognize index. ___ If you reply to this email, your message will be added to the discussion below: http://neo4j-community-discussions.438527.n3.nabble.com/Lucene-Index-view-tool-tp3335822p3335822.html To start a new topic under Neo4j Community Discussions, email ml-node+s438527n438527...@n3.nabble.com To unsubscribe from Neo4j Community Discussions, visit http://neo4j-community-discussions.438527.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=438527code=bm9wcGFuaXQuY0BnbWFpbC5jb218NDM4NTI3fDExOTIzNzAyNjk= -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Lucene-Index-view-tool-tp3335822p3335829.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Fwd: Sync databases
From: *Eddy Respondek* eddy.respon...@gmail.com Date: Wed, Sep 14, 2011 at 11:30 AM To: gremlin-us...@googlegroups.com This may be a little off topic but maybe someone has done something similar before. Basically I have a separate Wordpress site (php/mysql) which I've been extending significantly and I've setup another server on the same network for graph db testing (neo4j/tinkerpop/python-bulbs). I'm confident with my graph setup now and would like to attempt to get something small into development so I can monitor the results. I want to do a simple like relationship between users and articles. That means I need to keep an identical index of user ids and article ids in the graph db. I know how to update the id's when a new user or article is created, deleted, etc. What I don't know is the correct way to ensure data integrity in case something goes wrong like the graph db server crashes, etc. Does anyone have any thoughts on the best way to do this? I store the last loaded IDs as properties on the root node, then every time my sync script is run it loads everything from those IDs forward, checking as it goes that it doesn't create duplicates. It's not the fastest way, but it's robust. If the graph DB becomes corrupt, you roll back to the latest back up and rerun the sync. (We have the advantage that our data is immutable - you'll need some extra changes if that isn't the case for you, but can use the same general technique) Xavier ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j low-level data storage
Hello everybody, I have some questions regarding data storage in neo4j. How does neo4j store the data on the physical level? Are elements which have a close relationship in the graph stored in an adjacent way on disk? Any special binary format? How do you treat SSDs compared to common harddisks? What are your measures to improve I/O? Is there a technical documention which describes this? regards, Daniel -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-low-level-data-storage-tp3336483p3336483.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Behaviour of LuceneIndex vs. BatchInserterIndex for multiple adds
Hi all, as far as i know, the Batch Inserter Index has a slightly different behaviour than the default LuceneIndex: 1. It supports the addition of multiple Key/Value pairs at once and 2. Multiple calls to add() will result in multiple Index entries The default Lucene Index implementation does not support this. What seems to be undesired behaviour mainly introduced to gain a little bit of performance has a huge benefit: It allows indexing of M:N-Relations at the node level. Imagine a graph with 2 Persons and 4 kinds of Vehicles: Person 1, Person 2, yellow buses, yellow bicyles, red buses and red bicycles. Let the graph be the following: Person 1 -OWNS- Yellow Bus Person 2 -OWNS- Red Bicycle Person 2 -OWNS- Red Bus Person 2 -OWNS- Yellow Bicyle Using the batch inserter one can now call personNodeIndex.add(1, color-yellow, vehicle-bus) personNodeIndex.add(1, color-red, vehicle-bicyle) personNodeIndex.add(1, color-red, vehicle-bus) personNodeIndex.add(1, color-yellow, vehicle-bicycle) and personNodeIndex.query(color:yellow vehicle:bus) will return *only* person 1. Since this behaviour is not present in the LuceneIndex, i fear that you will probably try to fix the behaviour to be the same as with LuceneIndex. Are there plans to do so? From my perspective, it would be a cooler thing to have the BatchInserterIndexes behaviour within the LuceneIndex to. Do you plan something like this? Best regards, Benjamin ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j low-level data storage
Hi, shameless plug explanation=my blog You can find a decent explanation for pre-1.4 storage layout here: http://digitalstain.blogspot.com/2010/10/neo4j-internals-file-storage.html It has not been updated however to explain extended addressing for 4B+ entities in a db (which do not change the layout, just the semantics of some bits). Nevertheless, the basic structures (fixed size records, doubly linked list etc) are still there. For the way I/O is performed (and minimized/improved upon) you can look at the way memory mapping is used here: http://digitalstain.blogspot.com/2010/10/neo4j-internals-persistence-and-memory.html And of course, there is object caching explained here: http://digitalstain.blogspot.com/2010/10/neo4j-internals-caching.html / shameless plug The organization in general is greatly favored by SSDs which are more friendly to linked lists on disk. However, there is no optimization in place yet targeting a specific storage technology - as in most applications, the better the I/O performance, the better Neo4j does. I'd be happy to help dive into more specific questions as they come to you. cheers, CG On Wed, Sep 14, 2011 at 8:09 PM, danielb danielbercht...@gmail.com wrote: Hello everybody, I have some questions regarding data storage in neo4j. How does neo4j store the data on the physical level? Are elements which have a close relationship in the graph stored in an adjacent way on disk? Any special binary format? How do you treat SSDs compared to common harddisks? What are your measures to improve I/O? Is there a technical documention which describes this? regards, Daniel -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-low-level-data-storage-tp3336483p3336483.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j low-level data storage
shameless plug explanation=my blog There is no shame in producing good work and sharing it with others. Marko. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Spring Data Graph + Eclipse + AJDT : Editor errors
First of hi to all Graphistas especially the Springraphistas (Michael the gang) I want to congratulate you all guys for SDG - I finally got to write more than a few lines, and it simply rules! Release of 1.1 has added a lot of great functionality, especially in terms of querying (Cypher, Gremlin etc) which is extremely welcomed! Now to the problem : eclipse (3.6/3.7) editor reports errors such as Type mismatch : Cannot convert from ClassXXX to Class? extends RelationshipBacked or The method relateTo(World, String) is undefined for the type XXX / The method persist() is undefined for the type XXX or Bound mismatch: The type XXX is not a valid substitute for the bounded parameter T extends GraphBacked? of the type GraphRepositoryT when it shouldn't. Projects compile and run perfectly :-) But its an old problem that's been mentioned a lot - and unfortunately it persists(), verified :-( I've started a discussion with all details on SpringSource ( http://forum.springsource.org/showthread.php?114627-Spring-Data-Graph-Eclipse-AJDT-Editor-errors) a JIRA issue (https://jira.springsource.org/browse/DATAGRAPH-104) hoping it will be finally get solved for good by the wonderfull spring (graph) people! Best regards Agelos ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Recommended way to index and lookup paired properties
Can anyone help w/ this question? =) Thanks! Aseem On Thu, Sep 8, 2011 at 9:58 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey guys, quick question on indexing. We track Amazon products in our db, and the way Amazon identifies its products is with an Amazon Standard Identification Number (ASIN). The only catch is that ASINs are only unique within a particular locale, and Amazon has a few different locales (e.g. us, uk, de, jp, etc.). Because of this, when we index and lookup Amazon products, we need to index and lookup with *both* properties together. E.g. we really want to index the pair. As such -- and because we use Neo4j's auto-indexing -- we created a shadow locale+asin property that sits alongside the regular locale and asin properties, and it's a concatenation of the two values, e.g. us+A123456789. We thus index and perform lookups via this shadow property. Recently, though, I saw that you can query even exact indexes with complex Lucene syntax, so in theory, we could ditch the shadow property and query something like (locale:us AND asin:A123456789). Will this be slower though, if there many items w/ the same locale? Generalizing, then, which of these two routes is the recommended way of indexing paired properties? (Or is there a third way?) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] lock or not?
hi all, I have implemented a function UserFactory.get_or_create_with_external_id(long external_id). the function basically search the index, see if a node already created and has the property external_id equals to external_id. if there is such a node, return, if not, create one, assign property and index it. this function will be called inside of transactions, from unmanaged extensions. so it's multi-threaded scenario. my question is if i need to mark the function synchronized. my guess is true, and i feel it'll hurt performance badly. if my guess is wrong, why? thanks -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
We finally decided that it should be added. http://docs.neo4j.org/chunked/snapshot/query-function.html#functions-id It's in snapshot, and will be part of our next stable release. Andrés On Thu, Aug 25, 2011 at 2:39 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey guys, Is there any way I can have Cypher return/print node IDs? If I specify for it to return a node, it returns/prints all data about that node, which can be excessive. But the problem is, the node's ID isn't a property on that node. I've tried both node~ID and node~SELF, like rel~TYPE, but neither works. If this isn't possible today, consider it a feature request. =) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
Nice! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Sep 14, 2011 at 9:56 PM, Andres Taylor andres.tay...@neotechnology.com wrote: We finally decided that it should be added. http://docs.neo4j.org/chunked/snapshot/query-function.html#functions-id It's in snapshot, and will be part of our next stable release. Andrés On Thu, Aug 25, 2011 at 2:39 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey guys, Is there any way I can have Cypher return/print node IDs? If I specify for it to return a node, it returns/prints all data about that node, which can be excessive. But the problem is, the node's ID isn't a property on that node. I've tried both node~ID and node~SELF, like rel~TYPE, but neither works. If this isn't possible today, consider it a feature request. =) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Recommended way to index and lookup paired properties
I think this falls into the category of best to try it. I would simulate a couple million items and see what kind of performance you get in both scenarios. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Wednesday, September 14, 2011 3:41 PM To: Neo4j user discussions Subject: Re: [Neo4j] Recommended way to index and lookup paired properties Can anyone help w/ this question? =) Thanks! Aseem On Thu, Sep 8, 2011 at 9:58 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey guys, quick question on indexing. We track Amazon products in our db, and the way Amazon identifies its products is with an Amazon Standard Identification Number (ASIN). The only catch is that ASINs are only unique within a particular locale, and Amazon has a few different locales (e.g. us, uk, de, jp, etc.). Because of this, when we index and lookup Amazon products, we need to index and lookup with *both* properties together. E.g. we really want to index the pair. As such -- and because we use Neo4j's auto-indexing -- we created a shadow locale+asin property that sits alongside the regular locale and asin properties, and it's a concatenation of the two values, e.g. us+A123456789. We thus index and perform lookups via this shadow property. Recently, though, I saw that you can query even exact indexes with complex Lucene syntax, so in theory, we could ditch the shadow property and query something like (locale:us AND asin:A123456789). Will this be slower though, if there many items w/ the same locale? Generalizing, then, which of these two routes is the recommended way of indexing paired properties? (Or is there a third way?) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
Awesome! Great to hear. Thanks Andrés! Aseem On Wed, Sep 14, 2011 at 12:58 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nice! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Sep 14, 2011 at 9:56 PM, Andres Taylor andres.tay...@neotechnology.com wrote: We finally decided that it should be added. http://docs.neo4j.org/chunked/snapshot/query-function.html#functions-id It's in snapshot, and will be part of our next stable release. Andrés On Thu, Aug 25, 2011 at 2:39 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, Is there any way I can have Cypher return/print node IDs? If I specify for it to return a node, it returns/prints all data about that node, which can be excessive. But the problem is, the node's ID isn't a property on that node. I've tried both node~ID and node~SELF, like rel~TYPE, but neither works. If this isn't possible today, consider it a feature request. =) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: how would you query first- and second-degree friends?
And now it's in code. http://docs.neo4j.org/chunked/snapshot/query-match.html#match-optional-relationship Hope that helps, Andrés On Thu, Aug 25, 2011 at 10:34 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Good and valid requests, Aseem. Consider them heard. We'll see how soon we can get it to you. Andrés On Thu, Aug 25, 2011 at 12:57 PM, Aseem Kishore aseem.kish...@gmail.comwrote: Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: how would you query first- and second-degree friends?
Awesome! Thanks again! Aseem On Wed, Sep 14, 2011 at 12:59 PM, Andres Taylor andres.tay...@neotechnology.com wrote: And now it's in code. http://docs.neo4j.org/chunked/snapshot/query-match.html#match-optional-relationship Hope that helps, Andrés On Thu, Aug 25, 2011 at 10:34 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Good and valid requests, Aseem. Consider them heard. We'll see how soon we can get it to you. Andrés On Thu, Aug 25, 2011 at 12:57 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] lock or not?
If a second thread reads that there is no node with external_id 123 in between the time that a first thread finds no node and elects to create it, you will get 2 nodes with external_id 123. So yes, you need to introduce a lock and synchronize. You can create read locks in the graph database but you probably will not need to do that. In your case you are reading from an index and I am not sure what locks are available for indexes. If you are running an embedded database and you can be sure that only your one JVM will access the database then you can just lock in the Java thread/object space and elect that other threads must wait to read if the node exists until the first thread has created it. If you need to communicate concurrency via the database, you will have to do all the same locking in the Java space and add a read lock on some control node in Neo4j. If you want to create a read lock on a node you can do the following: lockManager = graphDb.getConfig().getLockManager(); lockManager.getReadLock(someNode); ... lockManager.releaseReadLock(someNode, null); Threading and synchronization in Java will perform great. Depending on your model, most of the time your node will exist and two or more threads will just queue up and get the same reference. That waiting is inefficient by nature, but if your business rules require it then you have to do it. Cheers, McKinley On Wed, Sep 14, 2011 at 12:45 PM, Linan Wang tali.w...@gmail.com wrote: hi all, I have implemented a function UserFactory.get_or_create_with_external_id(long external_id). the function basically search the index, see if a node already created and has the property external_id equals to external_id. if there is such a node, return, if not, create one, assign property and index it. this function will be called inside of transactions, from unmanaged extensions. so it's multi-threaded scenario. my question is if i need to mark the function synchronized. my guess is true, and i feel it'll hurt performance badly. if my guess is wrong, why? thanks -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] lock or not?
McKinley Thank you very much for the detailed explanation. however, I don't get the part about only one JVM will access database. neo4j doesn't support multiple JVMs has write access to the same db, right? On Wed, Sep 14, 2011 at 9:14 PM, McKinley mckinley1...@gmail.com wrote: If a second thread reads that there is no node with external_id 123 in between the time that a first thread finds no node and elects to create it, you will get 2 nodes with external_id 123. So yes, you need to introduce a lock and synchronize. You can create read locks in the graph database but you probably will not need to do that. In your case you are reading from an index and I am not sure what locks are available for indexes. If you are running an embedded database and you can be sure that only your one JVM will access the database then you can just lock in the Java thread/object space and elect that other threads must wait to read if the node exists until the first thread has created it. If you need to communicate concurrency via the database, you will have to do all the same locking in the Java space and add a read lock on some control node in Neo4j. If you want to create a read lock on a node you can do the following: lockManager = graphDb.getConfig().getLockManager(); lockManager.getReadLock(someNode); ... lockManager.releaseReadLock(someNode, null); Threading and synchronization in Java will perform great. Depending on your model, most of the time your node will exist and two or more threads will just queue up and get the same reference. That waiting is inefficient by nature, but if your business rules require it then you have to do it. Cheers, McKinley On Wed, Sep 14, 2011 at 12:45 PM, Linan Wang tali.w...@gmail.com wrote: hi all, I have implemented a function UserFactory.get_or_create_with_external_id(long external_id). the function basically search the index, see if a node already created and has the property external_id equals to external_id. if there is such a node, return, if not, create one, assign property and index it. this function will be called inside of transactions, from unmanaged extensions. so it's multi-threaded scenario. my question is if i need to mark the function synchronized. my guess is true, and i feel it'll hurt performance badly. if my guess is wrong, why? thanks -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] lock or not?
I mean that if you are not running the REST server or high availability then you can assume that even if you only put the read lock in the Java thread/object world, the database will not change. No other process exists that could change it. You do not need to bother with a read lock in the database. Cheers, McKinley On Wed, Sep 14, 2011 at 3:07 PM, Linan Wang tali.w...@gmail.com wrote: McKinley Thank you very much for the detailed explanation. however, I don't get the part about only one JVM will access database. neo4j doesn't support multiple JVMs has write access to the same db, right? On Wed, Sep 14, 2011 at 9:14 PM, McKinley mckinley1...@gmail.com wrote: If a second thread reads that there is no node with external_id 123 in between the time that a first thread finds no node and elects to create it, you will get 2 nodes with external_id 123. So yes, you need to introduce a lock and synchronize. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] lock or not?
Got it. i really appreciate your help. On Wed, Sep 14, 2011 at 11:14 PM, McKinley mckinley1...@gmail.com wrote: I mean that if you are not running the REST server or high availability then you can assume that even if you only put the read lock in the Java thread/object world, the database will not change. No other process exists that could change it. You do not need to bother with a read lock in the database. Cheers, McKinley On Wed, Sep 14, 2011 at 3:07 PM, Linan Wang tali.w...@gmail.com wrote: McKinley Thank you very much for the detailed explanation. however, I don't get the part about only one JVM will access database. neo4j doesn't support multiple JVMs has write access to the same db, right? On Wed, Sep 14, 2011 at 9:14 PM, McKinley mckinley1...@gmail.com wrote: If a second thread reads that there is no node with external_id 123 in between the time that a first thread finds no node and elects to create it, you will get 2 nodes with external_id 123. So yes, you need to introduce a lock and synchronize. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java PaaS with Neo4j (slightly off topic)
At CFoundry you can access local storage but it is not persistent. No access to local storage at Heroku, don't know about the others. Sorry. Only IaaS as AWS. Cheers Michael Am 09.09.2011 um 06:40 schrieb Christopher Schmidt: Hi all, I am using (the embedded version of) Neo4j together with a webapplication (WAR file for Tomcat). Does anyone know a PaaS provider (like CloudBees) that allow a local file storage? (This would be a simple solution beside implementing the Neo4j Server REST interface and f.e. using a Neo4j Heroku Addon) -- Christopher twitter: @fakod blog: http://blog.fakod.eu ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] May I know the cache mechanism of Neo4j?
Hello everyone, Anybody can tell me more about the cache mechanism of Neo4j? When use this code get both fans of two user. The result will be auto cache? or Not? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-May-I-know-the-cache-mechanism-of-Neo4j-tp3337895p3337895.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java PaaS with Neo4j (slightly off topic)
You can run Java on Windows Azure with persistent file storage. (Microsoft even ship Eclipse tooling support and everything for Azure.) That's how we're hosting neo4j on my current project. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Christopher Schmidt Sent: Friday, 9 September 2011 2:41 PM To: user@lists.neo4j.org Subject: [Neo4j] Java PaaS with Neo4j (slightly off topic) Hi all, I am using (the embedded version of) Neo4j together with a webapplication (WAR file for Tomcat). Does anyone know a PaaS provider (like CloudBees) that allow a local file storage? (This would be a simple solution beside implementing the Neo4j Server REST interface and f.e. using a Neo4j Heroku Addon) -- Christopher twitter: @fakod blog: http://blog.fakod.eu ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java PaaS with Neo4j (slightly off topic)
A bit more info ... The individual Azure machines will get spun up and down as you do new deployments, etc. The storage is called Cloud Drive and is an Azure page blob which gets mounted as a drive letter. (It's the equivalent of uploading a VHD (Virtual Hard Drive) file to Amazon's S3, then mounting it as a drive letter with efficient block level access.) The blob itself has a full internal file system. -- Tatham -Original Message- From: Tatham Oddie Sent: Thursday, 15 September 2011 3:38 PM To: user@lists.neo4j.org Subject: RE: [Neo4j] Java PaaS with Neo4j (slightly off topic) You can run Java on Windows Azure with persistent file storage. (Microsoft even ship Eclipse tooling support and everything for Azure.) That's how we're hosting neo4j on my current project. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Christopher Schmidt Sent: Friday, 9 September 2011 2:41 PM To: user@lists.neo4j.org Subject: [Neo4j] Java PaaS with Neo4j (slightly off topic) Hi all, I am using (the embedded version of) Neo4j together with a webapplication (WAR file for Tomcat). Does anyone know a PaaS provider (like CloudBees) that allow a local file storage? (This would be a simple solution beside implementing the Neo4j Server REST interface and f.e. using a Neo4j Heroku Addon) -- Christopher twitter: @fakod blog: http://blog.fakod.eu ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java PaaS with Neo4j (slightly off topic)
On a note, we are testing Neo4j HA installations on Azure, so there will be samples coming out soon for that. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Sep 15, 2011 at 7:40 AM, Tatham Oddie tat...@oddie.com.au wrote: A bit more info ... The individual Azure machines will get spun up and down as you do new deployments, etc. The storage is called Cloud Drive and is an Azure page blob which gets mounted as a drive letter. (It's the equivalent of uploading a VHD (Virtual Hard Drive) file to Amazon's S3, then mounting it as a drive letter with efficient block level access.) The blob itself has a full internal file system. -- Tatham -Original Message- From: Tatham Oddie Sent: Thursday, 15 September 2011 3:38 PM To: user@lists.neo4j.org Subject: RE: [Neo4j] Java PaaS with Neo4j (slightly off topic) You can run Java on Windows Azure with persistent file storage. (Microsoft even ship Eclipse tooling support and everything for Azure.) That's how we're hosting neo4j on my current project. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Christopher Schmidt Sent: Friday, 9 September 2011 2:41 PM To: user@lists.neo4j.org Subject: [Neo4j] Java PaaS with Neo4j (slightly off topic) Hi all, I am using (the embedded version of) Neo4j together with a webapplication (WAR file for Tomcat). Does anyone know a PaaS provider (like CloudBees) that allow a local file storage? (This would be a simple solution beside implementing the Neo4j Server REST interface and f.e. using a Neo4j Heroku Addon) -- Christopher twitter: @fakod blog: http://blog.fakod.eu ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Best practice for User Authentication and Authorization
Hi, What design pattern do you recommend for user authentication and authorization in neo4j graph?I'm searching a simple and flexible way to restrict the access to certain part of graph which is dynamically depends on the user and role...etc.How to avoid excessive traversing? Any idea is appreciated! Regards, Brendan ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
