I posted this on StackOverflow but thought I would put here as well in case
anybody has any ideas. Creating 2 nodes and one relationship in an empty
database is taking 150ms to 170ms. This is making it really hard to run
unit tests efficiently so I'm looking for any suggestions to speed things
up. Is there a "don't write to disk because I'm just testing" mode for
Neo4j?
I have Neo4j server running inside a virtual machine using Ubuntu 13.10 and
I am accessing via REST using Cypher queries. The virtual machine has 4 GB
of memory allocated to it. I've changed the open file count to 40000, set
the initial JVM heap to 1G and my `neo4j.properties` file is as follows:
neostore.nodestore.db.mapped_memory=250M
neostore.relationshipstore.db.mapped_memory=100M
neostore.propertystore.db.mapped_memory=100M
neostore.propertystore.db.strings.mapped_memory=100M
neostore.propertystore.db.arrays.mapped_memory=100M
keep_logical_logs=3 days
node_auto_indexing=true
node_keys_indexable=id
I've also updated sysctl based on the Neo4j Linux tuning guide:
vm.dirty_background_ratio = 50
vm.dirty_ratio = 80
Since I am testing queries, the basic routine is to run my suite of tests
and then delete all of the nodes and run them all again. At the start of
each test run, the database has 0 nodes in it. My suite of tests of about
100 queries is taking 22 seconds to run. Basic parameterized creates such
as:
CREATE (x:user { email: {param0},
name: {param1},
displayname: {param2},
id: {param3},
href: {param4},
object: {param5} })
CREATE x-[:LOGIN]->(:login { password: {param6},
salt: {param7} } )
are currently taking over 170ms to execute (and that's the average, first
query time before execution plan is cached is 700ms). During a test run,
the CPU in the VM never exceeds 50% and memory usage is at a steady 1.4Gb.
Why would creating a single node in an empty database take 170ms? At this
point unit testing is becoming almost impossible since it is so slow. This
is my first time trying to tune Neo4j so I'm not really sure how to figure
out where the problem is or what changes should be made.
**Additional Details**
I'm using Go 1.2 to make REST calls to the `cypher` endpoint
(`http://localhost:7474/db/data/cypher`) of a locally installed Neo4j
instance. I'm setting the request headers for content-type to
"application/json", accept to "application/json" and "X-Stream" to true. I
always return either an array of maps or nothing depending on the query.
It seems like the creates are the problem and are taking forever. For
example:
2014/01/15 11:35:51 NewUser took 123.314938ms
2014/01/15 11:35:51 NewUser took 156.101784ms
2014/01/15 11:35:52 NewUser took 167.439442ms
2014/01/15 11:35:52 ValidatePassword took 4.287416ms
NewUser creates two new nodes and one relationship and is taking 167ms,
while ValidatePassword is a read-only operation and it completes in 4ms.
Also note that the three calls to NewUser are identical parameterized
queries. While the creates are the big problem, I'm also a little concerned
that Neo4j is taking 4ms to just find a labeled node when there are only
100 nodes in the database.
I do not restart the server in between test runs or delete the database. I
issue a single delete all nodes query `MATCH (n) OPTIONAL MATCH (n)-[r]-()
DELETE n,r` at the end of the test run. Running the same test suite
multiple times back to back does not improve the query times.
Anybody have any thoughts on how to improve the performance I'm seeing?
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to neo4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
