Re: [Neo4j] Big index solutions?
Hi Peter, I finished my testing. I tried jdbm tree and map, HSQL, and jboss cache as a wrapper around both HSQL and jdbm. I found that jboss cache doesn't necessarily persist to disk at the end of a transaction, so it fails the acid test. HSQL is super fast in memory but was terrible when forced to commit every transaction. (I tested 1.8, which doesn't support transactions, only each update is a transaction. Maybe 2.0 is better.) So that leave jdbm. The tree (surprisingly) was much faster than the map. I know from experience that jdbm doesn't scale well withy multiple threads, yet, in this application I was thinking it may still be a good fit. It would be nice though if they at least used a ReentrantReadWriteLock rather than method synchronization to allow concurrent reads. Hope that helps. Thanks, -Paul Paul Jackson, Principal Software Engineer Pitney Bowes Business Insight 4200 Parliament Place | Suite 600 | Lanham, MD 20706-1844 USA O: 301.918.0850 | M: 703.862.0120 | www.pb.com paul.jack...@pb.com Every connection is a new opportunityT Please consider the environment before printing or forwarding this email. If you do print this email, please recycle the paper. This email message may contain confidential, proprietary and/or privileged information. It is intended only for the use of the intended recipient(s). If you have received it in error, please immediately advise the sender by reply email and then delete this email message. Any disclosure, copying, distribution or use of the information contained in this email message to or by anyone other than the intended recipient is strictly prohibited. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of the Company. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer Sent: Tuesday, December 21, 2010 11:12 AM To: rick.bullo...@burningskysoftware.com Cc: Neo4j user discussions Subject: Re: [Neo4j] Big index solutions? Mmh, we are looking at JDBM now, and it seems to be promising. Will inform you on the progress of that! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Dec 21, 2010 at 12:19 PM, rick.bullo...@burningskysoftware.com wrote: > That should fit in RAM just fine, except for the effect of the string > block/page size probably. What about a btree backed by neo relationships? > Not fast enough? > > - Reply message - > From: "Peter Neubauer" > Date: Mon, Dec 20, 2010 3:54 pm > Subject: [Neo4j] Big index solutions? > To: "Neo4j user discussions" > > Hi folks, > I wonder if any of you has seen a fast exact index solution that works > for the batchinserter (FAST) and over big indexes (like 100M strings > of length 20characters) that don't fit in RAM. > > Lucene is unable to cache such indexes and gets slow. > > Does anybody have experiences with other reverse lookup solutions like > Berkeley DB, Ehcache or others? Would be great to combine them with > the batchinserter to be able to fast insert big edge-lists with > node-index-lookups into Neo4j ... > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > > > > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Load last 10 created nodes from neo4j
BTW, neo's ID is reused, so the solution only valid in single context case. 致敬 向雅 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [Commits] [neo4j/server] 846d17: Added Dijkstra as an algorithm of choice when quer...
Great! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sun, Mar 6, 2011 at 3:01 PM, wrote: > Branch: refs/heads/master > Home: https://github.com/neo4j/server > > Commit: 846d17e502a18cdf875b7804f10cd8a59c4cd8bd > > https://github.com/neo4j/server/commit/846d17e502a18cdf875b7804f10cd8a59c4cd8bd > Author: Mattias Persson > Date: 2011-03-06 (Sun, 06 Mar 2011) > > Changed paths: > M > server/src/functionaltest/java/org/neo4j/server/rest/PathsFunctionalTest.java > M server/src/main/java/org/neo4j/server/rest/repr/PathRepresentation.java > M server/src/main/java/org/neo4j/server/rest/web/DatabaseActions.java > M server/src/site/apt/rest.apt > M server/src/test/java/org/neo4j/server/rest/web/DatabaseActionsTest.java > > Log Message: > --- > Added Dijkstra as an algorithm of choice when querying for paths between two > nodes > > > ___ > Commits mailing list > comm...@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/commits > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Load last 10 created nodes from neo4j
You could keep a linked list for the last created nodes, like: (list ref node) ---> (list item node) ---> (list item node) / / / / vv (last node) (next to last node) ... New nodes gets added first in that list and you can purge it so that it doesn't grow over a specific threshold. Look https://svn.neo4j.org/components/utils/trunk/src/main/java/org/neo4j/util/FixedLengthNodeList.javafor such a piece of code. So whenever you'd like to get the N latest nodes you just start at the (list ref node) and traverse back N steps and get all the nodes which the list item "place holder" nodes points to. 2011/3/6 mike_t > Hi, thanks Craig and Rick for your answers! > > I will try put the last nodes and relationships to an index. Maybe this > solution works for me. > > I think the better approach would be if neo4j-db would offer a function > where i can read out the node or relationship id-counter! > > -- > View this message in context: > http://neo4j-user-list.438527.n3.nabble.com/Load-last-10-created-nodes-from-neo4j-tp2633139p2641470.html > Sent from the Neo4J User List mailing list archive at Nabble.com. > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neoclipse Wish List
Hi, Craig. Thanks for the info. I've tried to deselect relationships but the checkboxes seem to be disabled. Like the idea of being able to specify a traversal. Loading and saving these various filtered "views" would also be helpful I think. Cheers, Rick - Reply message - From: "Craig Taverner" Date: Sat, Mar 5, 2011 6:23 pm Subject: [Neo4j] Neoclipse Wish List To: "Neo4j user discussions" You can filter the number of nodes in the preferences settings (click the gear icon on the top left, and then click neo4j, and change maximum number of nodes). You can filter the relationships by types in the relationships view, deselect the types (also by incoming/outgoing). But as far as I know there is not filter on properties of nodes or relationships. Should be easy to add though. My wish list includes: - Menu of possible alternative roots (places in the database to jump directly to, possible based on a lucene query). - List of existing lucene indices available to query - visualize sub-graph not by a fixed depth, but by a traversal query (possibly using the syntax of the REST API, since that is dynamically interpreted) On Sat, Mar 5, 2011 at 10:38 PM, Rick Bullotta < rick.bullo...@burningskysoftware.com> wrote: > Neoclipse is an awesome tool, but here are a few items that would greatly > increase the utility and usability: > > > > 1) Provide a limit on the # of nodes/relationships that are displayed (and > a > warning that additional nodes and relationships were not shown) > > 2) Provide display filters based on node and/or relationship property > values > and relationship types > > > > I think these would greatly improve the situation when very large or > complex > graphs are involved, since now the visualizations become far too "busy" to > do anything. > > > > Thoughts? > > > > Rick > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Load last 10 created nodes from neo4j
Hi, thanks Craig and Rick for your answers! I will try put the last nodes and relationships to an index. Maybe this solution works for me. I think the better approach would be if neo4j-db would offer a function where i can read out the node or relationship id-counter! -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/Load-last-10-created-nodes-from-neo4j-tp2633139p2641470.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neoclipse errors
On Sat, Mar 5, 2011 at 6:58 PM, Anders Nawroth wrote: > Hi! > > I just uploaded Neoclipse 1.3.M03, you can download it from here: > http://neo4j.org/download/ > > From this version, the search function uses the new integrated index. > At the moment it can only search for exact matches in node indices. It's > more of a proof of concept so far, expect it to get better soon! Thanks so much! ... Really appreciated, kudos to you and the whole neo4j team! Best regards! -- Massimo http://meridio.blogspot.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neoclipse Wish List
Hi! 2011-03-06 00:23, Craig Taverner: > You can filter the number of nodes in the preferences settings (click the > gear icon on the top left, and then click neo4j, and change maximum number > of nodes). But it still doesn't warn you when you hit the limit. > But as far as I know there is not filter on properties of nodes or > relationships. Should be easy to add though. How would you like that to look? > My wish list includes: > > - Menu of possible alternative roots (places in the database to jump > directly to, possible based on a lucene query). Maybe an ability to bookmark search results would be enough? > - List of existing lucene indices available to query The new search dialog now lists the node indices from the integrated index. The relationship indices will soon come, but there's more refactoring needed to add relationship search. The next step is then to add Lucene query support, not only lookup for exact matches. > - visualize sub-graph not by a fixed depth, but by a traversal query > (possibly using the syntax of the REST API, since that is dynamically > interpreted) Nice idea to use the REST API syntax! /anders > > > On Sat, Mar 5, 2011 at 10:38 PM, Rick Bullotta< > rick.bullo...@burningskysoftware.com> wrote: > >> Neoclipse is an awesome tool, but here are a few items that would greatly >> increase the utility and usability: >> >> >> >> 1) Provide a limit on the # of nodes/relationships that are displayed (and >> a >> warning that additional nodes and relationships were not shown) >> >> 2) Provide display filters based on node and/or relationship property >> values >> and relationship types >> >> >> >> I think these would greatly improve the situation when very large or >> complex >> graphs are involved, since now the visualizations become far too "busy" to >> do anything. >> >> >> >> Thoughts? >> >> >> >> Rick >> >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Different build errors
Maybe we could change to the bundle-packaged one that is on maven central? http://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles.netty/3.2.3.Final_1/ It's also a more recent version. /anders 2011-03-05 19:50, Andreas Kollegger skrev: > Did `sbt update` fail? Because it did for me when I just now set up a sample > project, failing to find "org.jboss.netty#netty;3.2.1.Final", which I believe > is only available from the JBoss repository. > > We should look into making that an optional dependency since it isn't > available from maven central and probably isn't needed unless you're using > HighAvailablity. > > For now, you could either include the JBoss repository -- > http://repository.jboss.org/nexus/content/groups/public -- or skip the Neo4j > dependency jar, instead specifying each individual artifact. > > -Andreas > > On Mar 5, 2011, at 7:34 PM, Nolan Darilek wrote: > >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 03/05/2011 12:27 PM, Andreas Kollegger wrote: >>> >>> What build tool are you using: maven, sbt, ant, gradle, or other? >>> >> >> SBT. >> -BEGIN PGP SIGNATURE- >> Version: GnuPG v1.4.10 (GNU/Linux) >> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ >> >> iEUEARECAAYFAk1ygkkACgkQIaMjFWMehWJ6qwCfYPtD4Wi0gVKSaMVUpTYFDv9a >> V2UAmNfMu+haW5i1v+DEuC6gxFTCP/E= >> =9TAx >> -END PGP SIGNATURE- >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
