Re: [Neo4j] neoclipse ubuntu 10.04 problem
Matej, Are you sure your database is shut down properly from your java code before opening it with Neoclipse ? Try to just open and close it and make sure there are no exceptions. HTH /peter On Saturday, March 19, 2011, Matěj Plch wrote: > Hi, > I'm running Ubuntu 10.04 (2.6.32-30) 32bit and have problem with > neoclipse. I downloaded neoclipse. I wanted to open some db made from > Java application, but I got some errors. So I set path to new db and > clicked to connect/start database. And got: > ./neoclipse > 19.3.2011 2:02:27 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo > INFO: Starting GraphDbServiceManager > 19.3.2011 2:02:41 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo > INFO: trying to start/connect ... > 19.3.2011 2:02:41 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo > INFO: connected to embedded neo4j > java.util.concurrent.ExecutionException: > java.lang.IllegalStateException: Can't start new database: the old one > isn't shutdown properly. > > In the info window: > java.lang.IllegalStateException: Can't start new database: the old one > isn't shutdown properly.: Can't start new database: the old one isn't > shutdown properly. > > There are no errors in messages.log. The db was created. > > How can I get rid of this problem? > > Running neo4j and also neoclipse version 1.2. > > $ java -version > java version "1.6.0_24" > Java(TM) SE Runtime Environment (build 1.6.0_24-b07) > Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode) > > Best regards > Matěj Plch. > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Fans of Neo4j From Chinese
Hi there, Without having too much insight in the concurrency of things, Neo4j locks on node level. That means, if two transactions in different threads are trying to modify the same nodes or relationships, an exception will be thrown or the transactions will be queued. Also, I am not sure how the lucene index is behaving in multiple threads, since we are wrapping it to be able to provide transactional support, which is not built into Lucene itself. Mattias, do you have any details on that? /peter On Friday, March 18, 2011, 孤竹 wrote: > hi, > > >Sorry for disturb you , I am a chinese engineer , Excused for my bad > english :) . > > >Recently, I am learning Neo4j and trying to use it in my project . But > When I make a Pressure on neo4j with 5 theads , 10 theads, 20 and 30, I found > the nodes inserted to the Neo4J is not change obvious (sometimes not change ~ > ~! ). Does it not matter with threads ? the kenerl will make it Serial ? Is > there any documents or something about The performance of Neo4j ? thanks for > your help > > > >The program as follows: >I put this function in ExecutorService ,with 5/10/30 threads. then test > for the nodes inserted into at same time .(The counts have not changed > obviously) > > > Transaction tx = null; > Node before = null; > try { > for (int i = 0; i < 100; i++) { > if(stop == true){ > return; > } > if (graphDb == null) { > return; > } > try { > if (tx == null) { > tx = graphDb.beginTx(); > } > // 引用计数加1 > writeCount.addAndGet(1); > int startNodeString = > name.addAndGet(1); > Node start = > getOrCreateNodeWithOutIndex("" > + startNodeString); > if (before == null) { > // 根节点.哈哈哈 I got U > Node root = > graphDb.getNodeById(0); > > root.createRelationshipTo(start, LEAD); > } > if (before != null) { > > before.createRelationshipTo(start, LOVES); > } > int endNodeName = name.addAndGet(1); > Node end = > getOrCreateNodeWithOutIndex("" + endNodeName); > start.createRelationshipTo(end, > KNOWS); > before = end; > // 每一千次 commit一次 > if (i % 100 == 0) { > tx.success(); > tx.finish(); > tx = null; > } > } catch (Exception e) { > System.out.println("write : = " + e); > } > } > } catch (Exception e) { > } finally { > tx.finish(); > } > } > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Fans of Neo4j From Chinese
Neo4j serializes commits. I.e. at most one thread is committing a transaction at once. For the actual work of building up the data to be committed, Neo4j supports multiple concurrent threads. This fact alone, that there is a single congestion point, means that if an application, like in your case, is very write centric, it is unlikely for it to scale beyond two threads, with one building up the next commit while the other is commiting its data. It might scale to a few more threads than that if the buildup time is significantly larger than the commit time. It is simple time slicing, "only one train can be at the station at once", then you have to do the maths on how many "trains can be out on the track" during that time. It is also worth keeping in mind, that for CPU bound operation, an application doesn't scale much further than the number of CPUs in the computer. The threads that are not in commit mode - i.e. the ones that are building up the data for their next commit - are CPU bound, and contending for the same CPU resources. This means that your application is not going to scale much further than the number of CPUs in your computer, and few desktop/laptop computers have more than 4 CPUs these days, which makes 5 threads about the most you can squeeze out of it, anything more than that is just going to add contention, and possibly even slow things down. Finally, the (CPU bound) threads that create the graph might be contending on the same resources. As Peter said. If multiple threads modify the same node or relationship, i.e. if they create relationships to the same node (the root node for example), they are all going to block on that resource. Neo4j only allows one transaction to modify each entity at a time. This means that to get maximum concurrency out of your data creation, each thread should be creating each own disconnected subgraph. And if they have connected parts, the connections to the "global data" should be made last in the transaction (in a predictable order to avoid deadlocks[1]), to maximize the time the thread is operational before hitting the "congestion point" that is the (potentially) contended data. Cheers, Tobias [1] Neo4j will detect if a deadlock has occurred and throw a DeadlockDetectedException in that case. 2011/3/18 孤竹 > hi, > > > Sorry for disturb you , I am a chinese engineer , Excused for my bad > english :) . > > > Recently, I am learning Neo4j and trying to use it in my project . But > When I make a Pressure on neo4j with 5 theads , 10 theads, 20 and 30, I > found the nodes inserted to the Neo4J is not change obvious (sometimes not > change ~ ~! ). Does it not matter with threads ? the kenerl will make it > Serial ? Is there any documents or something about The performance of Neo4j > ? thanks for your help > > > > The program as follows: > I put this function in ExecutorService ,with 5/10/30 threads. then test > for the nodes inserted into at same time .(The counts have not changed > obviously) > > > Transaction tx = null; >Node before = null; >try { >for (int i = 0; i < 100; i++) { >if(stop == true){ >return; >} >if (graphDb == null) { >return; >} >try { >if (tx == null) { >tx = graphDb.beginTx(); >} >// 引用计数加1 >writeCount.addAndGet(1); >int startNodeString = > name.addAndGet(1); >Node start = > getOrCreateNodeWithOutIndex("" >+ startNodeString); >if (before == null) { >// 根节点.哈哈哈 I got U >Node root = > graphDb.getNodeById(0); > > root.createRelationshipTo(start, LEAD); >} >if (before != null) { > > before.createRelationshipTo(start, LOVES); >} >int endNodeName = name.addAndGet(1); >Node end = > getOrCreateNodeWithOutIndex("" + endNodeName); >start.createRelationshipTo(end, > KNOWS); >before = end; >// 每一千次 commit一次 >if (i % 100 == 0) { >tx.success(); >
Re: [Neo4j] neoclipse ubuntu 10.04 problem
Sure, ok when Im working with db created with Java API, just call: private static void getNode() { int ticketId1 = 1; GraphDatabaseService graphDb = new EmbeddedGraphDatabase("var1/graphdb"); Transaction tx = graphDb.beginTx(); try { Node node = graphDb.getNodeById(ticketId1); int id = Integer.parseInt(node.getProperty("id").toString()); System.out.println("node1 id: " + id); } finally { tx.finish(); graphDb.shutdown(); } } Get right result... Wanted to open it with neoclipse, get that error. Wanted to open it with Gephi with neo4j support get error with version of neo4j library problem (tried with 1.03.M4 and the same error)... But it I get error with also when I'm creating absilutelz new database from Neoclipse... There can be only one neo4j db instance in system? How can I see, if its open or not? Netstat, ps aux? Thank you for your support... Dne 19.3.2011 09:15, Peter Neubauer napsal(a): > Matej, > Are you sure your database is shut down properly from your java code > before opening it with Neoclipse ? > > Try to just open and close it and make sure there are no exceptions. > > > HTH > > /peter > > On Saturday, March 19, 2011, Matěj Plch wrote: >> Hi, >> I'm running Ubuntu 10.04 (2.6.32-30) 32bit and have problem with >> neoclipse. I downloaded neoclipse. I wanted to open some db made from >> Java application, but I got some errors. So I set path to new db and >> clicked to connect/start database. And got: >> ./neoclipse >> 19.3.2011 2:02:27 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo >> INFO: Starting GraphDbServiceManager >> 19.3.2011 2:02:41 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo >> INFO: trying to start/connect ... >> 19.3.2011 2:02:41 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo >> INFO: connected to embedded neo4j >> java.util.concurrent.ExecutionException: >> java.lang.IllegalStateException: Can't start new database: the old one >> isn't shutdown properly. >> >> In the info window: >> java.lang.IllegalStateException: Can't start new database: the old one >> isn't shutdown properly.: Can't start new database: the old one isn't >> shutdown properly. >> >> There are no errors in messages.log. The db was created. >> >> How can I get rid of this problem? >> >> Running neo4j and also neoclipse version 1.2. >> >> $ java -version >> java version "1.6.0_24" >> Java(TM) SE Runtime Environment (build 1.6.0_24-b07) >> Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode) >> >> Best regards >> Matěj Plch. >> >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neoclipse ubuntu 10.04 problem
Matej, Try tx.success() before finish()? /peter Send from my mobile device, please excuse typos and brevity. On Mar 19, 2011 11:45 AM, "Matěj Plch" wrote: > Sure, > ok when Im working with db created with Java API, just call: > > private static void getNode() { > int ticketId1 = 1; > > GraphDatabaseService graphDb = new > EmbeddedGraphDatabase("var1/graphdb"); > Transaction tx = graphDb.beginTx(); > try { > Node node = graphDb.getNodeById(ticketId1); > int id = Integer.parseInt(node.getProperty("id").toString()); > System.out.println("node1 id: " + id); > } finally { > tx.finish(); > graphDb.shutdown(); > } > } > > Get right result... > > Wanted to open it with neoclipse, get that error. Wanted to open it with > Gephi with neo4j support get error with version of neo4j library problem > (tried with 1.03.M4 and the same error)... > > But it I get error with also when I'm creating absilutelz new database > from Neoclipse... There can be only one neo4j db instance in system? How > can I see, if its open or not? Netstat, ps aux? > Thank you for your support... > > Dne 19.3.2011 09:15, Peter Neubauer napsal(a): >> Matej, >> Are you sure your database is shut down properly from your java code >> before opening it with Neoclipse ? >> >> Try to just open and close it and make sure there are no exceptions. >> >> >> HTH >> >> /peter >> >> On Saturday, March 19, 2011, Matěj Plch wrote: >>> Hi, >>> I'm running Ubuntu 10.04 (2.6.32-30) 32bit and have problem with >>> neoclipse. I downloaded neoclipse. I wanted to open some db made from >>> Java application, but I got some errors. So I set path to new db and >>> clicked to connect/start database. And got: >>> ./neoclipse >>> 19.3.2011 2:02:27 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo >>> INFO: Starting GraphDbServiceManager >>> 19.3.2011 2:02:41 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo >>> INFO: trying to start/connect ... >>> 19.3.2011 2:02:41 org.neo4j.neoclipse.graphdb.GraphDbServiceManager logInfo >>> INFO: connected to embedded neo4j >>> java.util.concurrent.ExecutionException: >>> java.lang.IllegalStateException: Can't start new database: the old one >>> isn't shutdown properly. >>> >>> In the info window: >>> java.lang.IllegalStateException: Can't start new database: the old one >>> isn't shutdown properly.: Can't start new database: the old one isn't >>> shutdown properly. >>> >>> There are no errors in messages.log. The db was created. >>> >>> How can I get rid of this problem? >>> >>> Running neo4j and also neoclipse version 1.2. >>> >>> $ java -version >>> java version "1.6.0_24" >>> Java(TM) SE Runtime Environment (build 1.6.0_24-b07) >>> Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode) >>> >>> Best regards >>> Matěj Plch. >>> >>> ___ >>> Neo4j mailing list >>> User@lists.neo4j.org >>> https://lists.neo4j.org/mailman/listinfo/user >>> >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Webadmin improvements
Hey all! Just wanted to update you on some changes being made to the web admin tool. The last two weeks or so, webadmin development has lived in a nice, quiet branch, but it is now merged back into the server master. *First, a TL;DR:* I've made a bunch of big changes to webadmin, it is now more awesome, but there are probably lots of bugs, please give feedback :) *And then, giant detailed email:* *The merge means some good technical things have happened:* - We now use Backbone.js, which has lead to much better code structure ( http://documentcloud.github.com/backbone/) - All webadmin code is written in Coffeescript, which looks like a blend of Python, Ruby and JS. This has really sped up development, and I hope this will make it easier for non-JS fans to interact with the webadmin code base. (http://jashkenas.github.com/coffee-script/) - We use require.js to define dependencies between coffeescript files. This gets rid off a ton of stuff in the server POM, makes debugging easier, and allows us to use require.js tools to aggregate and optimize both JS and CSS for production packaging. This will also make it easy to dynamically load JS plugins into webadmin if we want that functionality later on. - All templates are written in HAML, which is a major improvement to the JST templates we used before. - Based on Andreas work on the web-visualization project, the webadmin integration tests can now be triggered to run in both IE, FF and Chrome. They pass in FF, there seems to be bugs/inconsistencies in how webdriver handles keyboard emulation in chrome, and I haven't had time to run the tests in IE (they won't pass, see further down). - All webdriver tests are written as Cucumber tests, allowing us to write easy-to-read tests that can be direct translations of features and related acceptance criteria from our user stories. See https://github.com/neo4j/server/blob/master/server/features/webadmin/databrowser-node.featurefor an example. * It also means some new features are now in place:* - A new and more polished UI - A complete rewrite of the data browser, hopefully making it easier, more straight forward and faster to use - The new data browser features both a tabular and a visualized view of the data. The visualized view is so far only a two-day hack to see that it works, but it will get lots of caressing and love before the 1.3 release. - Our first keyboard shortcut, and a simple framework to cleanly put more of them in place. Press "s" to get to the databrowser search bar from any page. *And there are a few things I need to work on before the 1.3 release*: - The webdriver tests are not currently run by teamcity. Andrés/Thomas: To have it run them, change the mvn call for the webdriver step to be "mvn integration-test -Dtests=web". Note that b/c of the cucumber stuff, running them has the same requirements as running the server-QA branch. - IE currently does not work. I've resolved some issues, but have a few related to plotting and searching in the data browser left. Oh, and it looks like absolute bollocks and vomits exceptions. Will adress ASAP. - The build pipeline for webadmin is not included in the server pom. Yeah. Don't panic, doing mvn package will produce a server artifact with a working webadmin. What I mean is that the build system does not currently propagate changes in the coffeescript or HAML files (under src/main/coffeescript) into "compiled" js files in the webadmin-html directory. I wrote a mashup maven plugin that does exactly that ( https://github.com/jakewins/brew), but because I got sidetracked and because the general consensus appears to be maven will be replaced soon, I never put the effort in to get the plugin to maven central. Thus, currently and reluctantly, the toolchain for working with webadmin is disconnected from the normal build system. I'm really sorry about that, this will be my first priority to fix. If anyone is wanting to do any work that would require this toolchain before this is resolved, drop me an email and I'll explain how to set it up. Phew. Sorry about the giant email. I'm off for a week in Austria in a few hours, before the last snow melts, will be back to coding Monday evening the 28'th and will make the issues listed above, together with visualization, my top priorities. I would really love to hear your comments and see bug reports, so that we can deliver a seriously awesome, beautiful and stable web admin tool in the 1.3 release. Cheers! Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user