Re: [Neo4j] Access Node's property block will happening??

2011-09-26 Thread McKinley
I would like to see your test and help. Can you post the most narrow test
possible to show the behavior you see?

Thanks,

McKinley

On Mon, Sep 26, 2011 at 8:13 PM, iamyuanlong  wrote:

> In EmbeddedGraphDatabase access Node's property block will happening?
> When multiple threads access one node's property . Sometimes will cost a
> lost of time.
> Tomorrow is the last time. If i can't solve this problem. My team will give
> up Neo4j . Then use Mogondb instead. But I like neo4j. And I hope I can use
> it in my project. So could you help me?
> Then EmbeddedGraphDatabase  is Singleton.I will try
> EmbeddedReadOnlyGraphDatabase , it will be useful?
>
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Traversal performance

2011-09-26 Thread David Montag
Also, try running it 100 times. Then you should see some JVM
optimizations/JIT kick in.

David

On Mon, Sep 26, 2011 at 9:24 PM, Rick Devinsus wrote:

> That was it- the cache wasn't warmed.  I tried running the same test twice,
> that increased the speed around 7x (450K traversals per second).  Thanks
> for
> the help.
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Traversal-performance-tp3371038p3371546.html
> Sent from the Neo4j Community Discussions mailing list archive at
> Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
David Montag 
Neo Technology, www.neotechnology.com
Cell: 650.556.4411
Skype: ddmontag
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Traversal performance

2011-09-26 Thread Rick Devinsus
That was it- the cache wasn't warmed.  I tried running the same test twice,
that increased the speed around 7x (450K traversals per second).  Thanks for
the help.

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/Traversal-performance-tp3371038p3371546.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Turning Off or Configuring log rotation

2011-09-26 Thread Romiko Derbynew
Hi Peter,

Tatham has created a new thread in this regards ".Net client for Neo4j", We 
have CI Build server as he mentions and code is always up to date.

Source Code
http://hg.readify.net/neo4jclient

NuGet (Package Installer)
http://nuget.org/List/Packages/Neo4jClient

Cheers

-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On 
Behalf Of Peter Neubauer
Sent: Tuesday, 27 September 2011 2:19 AM
To: Neo4j user discussions
Cc: Tatham Oddie
Subject: Re: [Neo4j] Turning Off or Configuring log rotation

Very cool,
glad to hear things work out. Got any code for the .NET bindings? I think we 
should start pulling client libraries into our build system, so there are 
stable versions of the most used clients for Neo4j available as part of the QA 
process.

WDYT?

Cheers,

/peter neubauer

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org   - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 1:24 PM, Romiko Derbynew  
wrote:
> Hi Peter,
>
> Indeed our desired result is now achieved. Tatham added batching to our .net 
> client as well. We are more than happy to assist with HA as this is one of 
> the last hurdles we need to overcome. If you can let us know in advance when 
> the release is scheduled then we will incorporate it into our sprint for HA 
> featuring.
>
> Much appreciated and thx for 1.5 release, could not have timed it better.
>
> Sent from my iPhone
>
> On 26/09/2011, at 5:33 PM, "Peter Neubauer" 
>  wrote:
>
>> Romiko,
>> soudns good. Is this solving your current problems or is there stuff
>> left to take care of for the moment? Also, there will be some code
>> coming out on deploying Neo4j HA on Azure. Would you be interested in
>> trying that out and maybe take it a bit further?
>>
>> Cheers,
>>
>> /peter neubauer
>>
>> GTalk:  neubauer.peter
>> Skype   peter.neubauer
>> Phone   +46 704 106975
>> LinkedIn   http://www.linkedin.com/in/neubauer
>> Twitter  http://twitter.com/peterneubauer
>>
>> http://www.neo4j.org   - Your high performance graph database.
>> http://startupbootcamp.org/- Öresund - Innovation happens HERE.
>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>>
>>
>>
>> On Mon, Sep 26, 2011 at 4:16 AM, Romiko Derbynew
>>  wrote:
>>> Hi Guys,
>>>
>>> We managed to fix the performance issue.
>>>
>>> * Used Paramterised Queries - increased performance by 1 extra node
>>> per second, reduced frequency usage of heap thus no jvm memory
>>> errors
>>> * Configured our .Net client to use 70 threads for rest calls (I chose this 
>>> number as Neo4j reserves 80, so I decided 70) 5 TIMES INCREASE IN 
>>> PERFORMANCE
>>>
>>>
>>> We are now going to implement the batching to improve performance
>>> even more :)
>>>
>>> I have updated the blog post with these findings.
>>> http://romikoderbynew.com/2011/09/23/neo4j-java-jvm-garbage-collecti
>>> on-profilingheap-sizelog-rotationlucene-rest-api-performance-analysi
>>> s/
>>>
>>>
>>> I will write a separate post on the improvements from the batching.
>>>
>>> MyStory Welcome, Temporary from MyStory Administration. Logout?
>>> Agency Import Job 301
>>>
>>> Log output:
>>>
>>> [01:44:58] [Thread   10] Migration message received at: 26/09/2011
>>> 1:44:58 AM +00:00
>>>
>>> [01:44:59] [Thread   10] Migration message payload: JobUniqueId:
>>> 301, AgencyUniqueId: 201
>>>
>>> [01:44:59] [Thread   10] Executing sql script:
>>> e:\AzureTemp\Resources\4bb24cfd-0b35-4c91-83de-97f335159a49\temp\tem
>>> p\RoleTemp\634526342992784034\output.sql
>>>
>>> [01:45:01] [Thread   10] Processing script command line 1/37620
>>>
>>> [01:45:03] [Thread   10] Processing script command line 2/37620
>>>
>>> [01:45:05] [Thread   10] Processing script command line 3/37620
>>>
>>> [01:45:06] [Thread   10] All sql script commands executed
>>> 37620/37620
>>>
>>> [01:45:13] [Thread   10] Importing 37619 clients into the database
>>>
>>> [01:45:44] [Thread   12] Processed 169/37619 clients, 5.62 clients
>>> per second, 01:51:06 remaining
>>>
>>> [01:46:16] [Thread   29] Processed 360/37619 clients, 5.80 clients
>>> per second, 01:47:02 remaining
>>>
>>> [01:46:29] [Thread   69] Processed 1000/37619 clients, 13.30 clients
>>> per second, 00:45:52 remaining
>>>
>>> [01:46:50] [Thread   28] Processed 2000/37619 clients, 20.78 clients
>>> per second, 00:28:34 remaining
>>>
>>> [01:47:11] [Thread   30] Processed 3000/37619 clients, 25.47 clients
>>> per second, 00:22:39 remaining
>>>
>>> [01:47:33] [Thread   32] Processed 4000/37619 clients, 28.71 clients
>>> per second, 00:19:31 remaining
>>>
>>> [01:47:55] [Thread   76] Processed 5000/37619 clients, 30.95 client

Re: [Neo4j] Traversal performance

2011-09-26 Thread Bryce
It wont make any difference if the memory mapping settings are just larger
than the file sizes, or a lot larger therefore fiddling with those
settings wont make any difference from your original test.

Generally when people see very high performance it is because a lot of the
data they are traversing over is already in memory, i.e. the caches are
warmed.  So is this test you are running just from a cold start, and if so
can you try the test twice, within the same vm that is.

On Tue, Sep 27, 2011 at 3:48 PM, Rick Devinsus wrote:

> I took a look at the files and none were larger than 500MB, however it
> makes
> a lot of sense to change the memory as you suggested so I altered the
> options as shown below.  I also started eclipse with different memory
> options than the defaults (eclipse -vmargs -Xmx2000m -server).  The changes
> didn't make it any faster though.
>
> I had read about people getting 2M traversals per second, since I'm only
> seeing around 65000/sec  I'm starting to think that represented the number
> of nodes searched through not the number returned based on the traversal's
> criteria.
>
>
> neostore.nodestore.db.mapped_memory=1.5G
> neostore.relationshipstore.db.mapped_memory=1.5G
> neostore.propertystore.db.mapped_memory=1.5G
> neostore.propertystore.db.index.mapped_memory=1.5G
> neostore.propertystore.db.index.keys.mapped_memory=50M
> neostore.propertystore.db.strings.mapped_memory=50M
> neostore.propertystore.db.arrays.mapped_memory=50M
>
> my file sizes:
> neostore.relationshipstore.db 500MB
> neostore.propertystore.db 383MB
> neostore.nodestore.db 137MB
> (others are all less than 1MB)
> the largest lucene node is 367MB
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Traversal-performance-tp3371038p3371379.html
> Sent from the Neo4j Community Discussions mailing list archive at
> Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo4j] Access Node's property block will happening??

2011-09-26 Thread iamyuanlong
In EmbeddedGraphDatabase access Node's property block will happening? 
When multiple threads access one node's property . Sometimes will cost a
lost of time.
Tomorrow is the last time. If i can't solve this problem. My team will give
up Neo4j . Then use Mogondb instead. But I like neo4j. And I hope I can use
it in my project. So could you help me?
Then EmbeddedGraphDatabase  is Singleton.I will try
EmbeddedReadOnlyGraphDatabase , it will be useful?

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Access-Node-s-property-block-will-happening-tp3371419p3371419.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Traversal performance

2011-09-26 Thread Rick Devinsus
I took a look at the files and none were larger than 500MB, however it makes
a lot of sense to change the memory as you suggested so I altered the
options as shown below.  I also started eclipse with different memory
options than the defaults (eclipse -vmargs -Xmx2000m -server).  The changes
didn't make it any faster though.

I had read about people getting 2M traversals per second, since I'm only
seeing around 65000/sec  I'm starting to think that represented the number
of nodes searched through not the number returned based on the traversal's
criteria.


neostore.nodestore.db.mapped_memory=1.5G
neostore.relationshipstore.db.mapped_memory=1.5G
neostore.propertystore.db.mapped_memory=1.5G
neostore.propertystore.db.index.mapped_memory=1.5G
neostore.propertystore.db.index.keys.mapped_memory=50M
neostore.propertystore.db.strings.mapped_memory=50M
neostore.propertystore.db.arrays.mapped_memory=50M

my file sizes:
neostore.relationshipstore.db 500MB
neostore.propertystore.db 383MB
neostore.nodestore.db 137MB
(others are all less than 1MB)
the largest lucene node is 367MB

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/Traversal-performance-tp3371038p3371379.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Traversal performance

2011-09-26 Thread Bryce
One initial suggestion would be that your memory mapped settings are
probably not very near optimal.  If you have a look at the file sizes in
your graph data directory then the closer you can get to covering each db
files entire size the better.  I would assume that some of the files will be
bigger than others and in fact you will probably find a few of them are
very small, so you are wasting memory on them that you could assign to
another memory mapping.

So in one of mine I have:
5807428  neostore.nodestore.db
335536170  neostore.relationshipstore.db
398675470  neostore.propertystore.db
1208  neostore.propertystore.db.index
6906  neostore.propertystore.db.index.keys
1112428784  neostore.propertystore.db.strings
158  neostore.propertystore.db.arrays

In which case there is no point in me assigning much if any memory to:
neostore.propertystore.db.arrays.mapped_memory
neostore.propertystore.db.index.keys.mapped_memory
neostore.propertystore.db.index.mapped_memory

The other thing to take into account is that the
neostore.nodestore.db.mapped_memory and
neostore.relationshipstore.db.mapped_memory
settings have a lot more impact on traversal than the property story
settings.  The property store settings will help when you are reading
properties from nodes or relationships.

So if you can assign memory mapping settings for nodes and relationships to
fit it all in memory map that would be good, otherwise still best to assign
more to those, and definitely don't give the ones like arrays much memory
(unless you are using them a lot).

On Tue, Sep 27, 2011 at 12:52 PM, Rick  wrote:

> Looking for help on how to tune traversals, this is a great product with
> the
> best API and I want to make sure Im getting the most from it.  I'm trying
> to
> understand if 62,500 traversals per second is the best I can do given the
> following scenario:
>
> - 15.6M nodes
> - 15.6M relationships
> - Data is structured as shown below so that the root has 250 children, each
> of its children have 250 children, and each of their children have 250
> children
> - If i get the entire list of children and grandchildren for a top node
> (max
> 3 levels deep), I get 62,500 nodes, and this takes about 800-1000ms
> - The server is a dual core quad 3.2ghz Xeon with 16gb ram
> - The neo4j.props settings are:
>
> neostore.nodestore.db.mapped_memory=1G
> neostore.relationshipstore.db.mapped_memory=1G
> neostore.propertystore.db.mapped_memory=1G
> neostore.propertystore.db.index.mapped_memory=1G
> neostore.propertystore.db.index.keys.mapped_memory=1G
> neostore.propertystore.db.strings.mapped_memory=1G
> neostore.propertystore.db.arrays.mapped_memory=1G
>
>
> - The code that does the traversal is
>
>Traverser trav = user.traverse( Order.BREADTH_FIRST,
> new StopEvaluator()
> { public boolean isStopNode(TraversalPosition pos) { return pos.depth() >=
> 3; }  },
>new ReturnableEvaluator() { public
> boolean
> isReturnableNode(TraversalPosition pos) { return pos.depth()< 3; }
>   }, KNOWS, Direction.BOTH );
>
>for ( Node node : trav )
>{
>// Do something with node...
>i++;
>}
>
>
> Data example
>  root
>node 0-0-0
>  node 0-0-1
>  node 0-0-2
>  ...
>   node 0-1-0
>  node 0-1-1
>  node 0-1-2
>  ...
>
>
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Traversal-performance-tp3371038p3371038.html
> Sent from the Neo4j Community Discussions mailing list archive at
> Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo4j] Traversal performance

2011-09-26 Thread Rick
Looking for help on how to tune traversals, this is a great product with the
best API and I want to make sure Im getting the most from it.  I'm trying to
understand if 62,500 traversals per second is the best I can do given the
following scenario:

- 15.6M nodes
- 15.6M relationships
- Data is structured as shown below so that the root has 250 children, each
of its children have 250 children, and each of their children have 250
children
- If i get the entire list of children and grandchildren for a top node (max
3 levels deep), I get 62,500 nodes, and this takes about 800-1000ms
- The server is a dual core quad 3.2ghz Xeon with 16gb ram
- The neo4j.props settings are:

neostore.nodestore.db.mapped_memory=1G
neostore.relationshipstore.db.mapped_memory=1G
neostore.propertystore.db.mapped_memory=1G
neostore.propertystore.db.index.mapped_memory=1G
neostore.propertystore.db.index.keys.mapped_memory=1G
neostore.propertystore.db.strings.mapped_memory=1G
neostore.propertystore.db.arrays.mapped_memory=1G


- The code that does the traversal is 

Traverser trav = user.traverse( Order.BREADTH_FIRST, 
new StopEvaluator()
{ public boolean isStopNode(TraversalPosition pos) { return pos.depth() >=
3; }  }, 
new ReturnableEvaluator() { public 
boolean
isReturnableNode(TraversalPosition pos) { return pos.depth()< 3; }
   }, KNOWS, Direction.BOTH );

for ( Node node : trav )
{   
// Do something with node...
i++;
}   


Data example
  root
node 0-0-0
  node 0-0-1
  node 0-0-2
  ...
   node 0-1-0
  node 0-1-1
  node 0-1-2
  ...



--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/Traversal-performance-tp3371038p3371038.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo4j] .NET client for neo4j

2011-09-26 Thread Tatham Oddie
Hi Peter,

We publish our client on NuGet: http://nuget.org/List/Packages/Neo4jClient

(NuGet is .NET's package manager.)

The source is at http://hg.readify.net/neo4jclient

It supports:

  * all the basic CRUD operations

  * index management

  * Gremlin queries via a fluent interface: you use C# lambdas and get full 
code completion in the IDE as you type; we only compile it down to Gremlin and 
send it over the wire when you enumerate it or call a scalar method like 
.Count(). See http://dl.dropbox.com/u/5216304/FluentGremlin.png

This week we moved to parametized Gremlin queries over the wire (eg, 
g.v(p0).out[[label:p1]]) and saw a good performance improvement in our 
downstream app. This change was implemented completely within Neo4jClient's 
query processor so downstream apps just got the improvement automatically.

We've also started optimizing some of our internal methods to use batching. For 
example, creating, relating and indexing a node all happens in a single REST 
call. We'll probably end up exposing the batching API outside of the client too 
via some type of unit-of-work pattern.

As for build, our CI triggers on a code push and publishes a new package to 
NuGet upon a successful build. If the tests pass, it gets published. (Our unit 
tests mock out all the REST calls so we have a high level of confidence in 
them. Risk is low anyway, as NuGet makes it easy to grab any previous version 
of the package.)


-- Tatham

-Original Message-
From: neubauer.pe...@gmail.com [mailto:neubauer.pe...@gmail.com] On Behalf Of 
Peter Neubauer
Sent: Tuesday, 27 September 2011 2:19 AM
To: Neo4j user discussions
Cc: Tatham Oddie
Subject: Re: [Neo4j] Turning Off or Configuring log rotation

Very cool,
glad to hear things work out. Got any code for the .NET bindings? I think we 
should start pulling client libraries into our build system, so there are 
stable versions of the most used clients for Neo4j available as part of the QA 
process.

WDYT?

Cheers,

/peter neubauer

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org   - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 1:24 PM, Romiko Derbynew  
wrote:
> Hi Peter,
>
> Indeed our desired result is now achieved. Tatham added batching to our .net 
> client as well. We are more than happy to assist with HA as this is one of 
> the last hurdles we need to overcome. If you can let us know in advance when 
> the release is scheduled then we will incorporate it into our sprint for HA 
> featuring.
>
> Much appreciated and thx for 1.5 release, could not have timed it better.
>
> Sent from my iPhone
>
> On 26/09/2011, at 5:33 PM, "Peter Neubauer" 
>  wrote:
>
>> Romiko,
>> soudns good. Is this solving your current problems or is there stuff
>> left to take care of for the moment? Also, there will be some code
>> coming out on deploying Neo4j HA on Azure. Would you be interested in
>> trying that out and maybe take it a bit further?
>>
>> Cheers,
>>
>> /peter neubauer
>>
>> GTalk:  neubauer.peter
>> Skype   peter.neubauer
>> Phone   +46 704 106975
>> LinkedIn   http://www.linkedin.com/in/neubauer
>> Twitter  http://twitter.com/peterneubauer
>>
>> http://www.neo4j.org   - Your high performance graph database.
>> http://startupbootcamp.org/- Öresund - Innovation happens HERE.
>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>>
>>
>>
>> On Mon, Sep 26, 2011 at 4:16 AM, Romiko Derbynew
>>  wrote:
>>> Hi Guys,
>>>
>>> We managed to fix the performance issue.
>>>
>>> * Used Paramterised Queries - increased performance by 1 extra node
>>> per second, reduced frequency usage of heap thus no jvm memory
>>> errors
>>> * Configured our .Net client to use 70 threads for rest calls (I chose this 
>>> number as Neo4j reserves 80, so I decided 70) 5 TIMES INCREASE IN 
>>> PERFORMANCE
>>>
>>>
>>> We are now going to implement the batching to improve performance
>>> even more :)
>>>
>>> I have updated the blog post with these findings.
>>> http://romikoderbynew.com/2011/09/23/neo4j-java-jvm-garbage-collecti
>>> on-profilingheap-sizelog-rotationlucene-rest-api-performance-analysi
>>> s/
>>>
>>>
>>> I will write a separate post on the improvements from the batching.
>>>
>>> MyStory Welcome, Temporary from MyStory Administration. Logout?
>>> Agency Import Job 301
>>>
>>> Log output:
>>>
>>> [01:44:58] [Thread   10] Migration message received at: 26/09/2011
>>> 1:44:58 AM +00:00
>>>
>>> [01:44:59] [Thread   10] Migration message payload: JobUniqueId:
>>> 301, AgencyUniqueId: 201
>>>
>>> [01:44:59] [Thread   10] Executing sql script:
>>> e:\AzureTemp\Resources\4bb24cfd-0b35-4c91-83de-97f335159a49\temp\tem
>>>

Re: [Neo4j] .NET client for neo4j

2011-09-26 Thread Tatham Oddie
Some more usage examples:

http://hg.readify.net/neo4jclient/src/tip/Test/ApiUsageIdeas.cs


-- Tatham


-Original Message-
From: Tatham Oddie
Sent: Tuesday, 27 September 2011 7:41 AM
To: 'Peter Neubauer'; Neo4j user discussions
Subject: .NET client for neo4j

Hi Peter,

We publish our client on NuGet: http://nuget.org/List/Packages/Neo4jClient

(NuGet is .NET's package manager.)

The source is at http://hg.readify.net/neo4jclient

It supports:

  * all the basic CRUD operations

  * index management

  * Gremlin queries via a fluent interface: you use C# lambdas and get full 
code completion in the IDE as you type; we only compile it down to Gremlin and 
send it over the wire when you enumerate it or call a scalar method like 
.Count(). See http://dl.dropbox.com/u/5216304/FluentGremlin.png

This week we moved to parametized Gremlin queries over the wire (eg, 
g.v(p0).out[[label:p1]]) and saw a good performance improvement in our 
downstream app. This change was implemented completely within Neo4jClient's 
query processor so downstream apps just got the improvement automatically.

We've also started optimizing some of our internal methods to use batching. For 
example, creating, relating and indexing a node all happens in a single REST 
call. We'll probably end up exposing the batching API outside of the client too 
via some type of unit-of-work pattern.

As for build, our CI triggers on a code push and publishes a new package to 
NuGet upon a successful build. If the tests pass, it gets published. (Our unit 
tests mock out all the REST calls so we have a high level of confidence in 
them. Risk is low anyway, as NuGet makes it easy to grab any previous version 
of the package.)


-- Tatham

-Original Message-
From: neubauer.pe...@gmail.com [mailto:neubauer.pe...@gmail.com] On Behalf Of 
Peter Neubauer
Sent: Tuesday, 27 September 2011 2:19 AM
To: Neo4j user discussions
Cc: Tatham Oddie
Subject: Re: [Neo4j] Turning Off or Configuring log rotation

Very cool,
glad to hear things work out. Got any code for the .NET bindings? I think we 
should start pulling client libraries into our build system, so there are 
stable versions of the most used clients for Neo4j available as part of the QA 
process.

WDYT?

Cheers,

/peter neubauer

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org   - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 1:24 PM, Romiko Derbynew  
wrote:
> Hi Peter,
>
> Indeed our desired result is now achieved. Tatham added batching to our .net 
> client as well. We are more than happy to assist with HA as this is one of 
> the last hurdles we need to overcome. If you can let us know in advance when 
> the release is scheduled then we will incorporate it into our sprint for HA 
> featuring.
>
> Much appreciated and thx for 1.5 release, could not have timed it better.
>
> Sent from my iPhone
>
> On 26/09/2011, at 5:33 PM, "Peter Neubauer" 
>  wrote:
>
>> Romiko,
>> soudns good. Is this solving your current problems or is there stuff
>> left to take care of for the moment? Also, there will be some code
>> coming out on deploying Neo4j HA on Azure. Would you be interested in
>> trying that out and maybe take it a bit further?
>>
>> Cheers,
>>
>> /peter neubauer
>>
>> GTalk:  neubauer.peter
>> Skype   peter.neubauer
>> Phone   +46 704 106975
>> LinkedIn   http://www.linkedin.com/in/neubauer
>> Twitter  http://twitter.com/peterneubauer
>>
>> http://www.neo4j.org   - Your high performance graph database.
>> http://startupbootcamp.org/- Öresund - Innovation happens HERE.
>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>>
>>
>>
>> On Mon, Sep 26, 2011 at 4:16 AM, Romiko Derbynew
>>  wrote:
>>> Hi Guys,
>>>
>>> We managed to fix the performance issue.
>>>
>>> * Used Paramterised Queries - increased performance by 1 extra node
>>> per second, reduced frequency usage of heap thus no jvm memory
>>> errors
>>> * Configured our .Net client to use 70 threads for rest calls (I chose this 
>>> number as Neo4j reserves 80, so I decided 70) 5 TIMES INCREASE IN 
>>> PERFORMANCE
>>>
>>>
>>> We are now going to implement the batching to improve performance
>>> even more :)
>>>
>>> I have updated the blog post with these findings.
>>> http://romikoderbynew.com/2011/09/23/neo4j-java-jvm-garbage-collecti
>>> on-profilingheap-sizelog-rotationlucene-rest-api-performance-analysi
>>> s/
>>>
>>>
>>> I will write a separate post on the improvements from the batching.
>>>
>>> MyStory Welcome, Temporary from MyStory Administration. Logout?
>>> Agency Import Job 301
>>>
>>> Log output:
>>>
>>> [01:44:58] [Thread   10] Migration message received a

Re: [Neo4j] Clarification on read and write locks

2011-09-26 Thread McKinley
Bryce,

Have you looked at LockManage.releaseReadLock(...)?

https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/impl/transaction/LockManager.java#L143

You can call that method with a null argument for the transaction which will
take you here:

https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/impl/transaction/RWLock.java#L190

Which creates a

https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/impl/transaction/RWLock.java#L469

I haven't thought through what you are describing, but I just wanted to make
sure you saw that part of the API.

Cheers,

McKinley
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo4j] Neo4j Events This Week

2011-09-26 Thread Allison Sparrow
***

Hi Everyone,

Here are the events coming up for this week. Check it out below.

We are adding more events on the regular, so be sure to check back at
http://plancast.com/neo4j for updates throughout the month. If you have any
questions regarding participation, feel free to email me at
alli...@neotechnology.com.

*LONDON Meetup: Neo4j and Spring - Practical Primer*
Wednesday, 28 September 2011 @18:30 GMT

In this talk for the Neo4j User Group, Aleksa Vukotic will introduce how you
can integrate Neo4j with Spring - the popular Java enterprise framework.

http://skillsmatter.com/podcast/nosql/neo4j-spring-integration/js-1603
*
LINZ (AUSTRIA) Conference Talk: Spring Data Graph Presentation @eJUG in
Austria *
Thursday, 29 September

Michael Hunger speaks on Spring Data Graph at eJUG in Austria

ejug.at/

best,

*Allison Sparrow* *
**Marketing Manager | Neo Technology*
+19499036091 | @ayeeson 

***
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] add property to all relationships

2011-09-26 Thread Bryce
I do similar processing a lot, i.e. changes across the entire graph database
(either relationships or nodes).  Code along the lines of:

public void process() {
int count = 0;
Transaction tx = null;
for (Node node : graphDb.getAllNodes()) {
if ((count % 1) == 0) {
if (tx != null) {
System.err.println("Processed: " + count);
tx.success();
tx.finish();
}
tx = graphDb.beginTx();
}

// do some processing
}
if (tx != null) {
tx.success();
tx.finish();
}
}

will split processing of all nodes up into transaction batches of 10,000
changes.

On Mon, Sep 26, 2011 at 11:37 PM, sometime  wrote:

> Can you please tell me where to read about "transaction batches" ?
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368893.html
> Sent from the Neo4j Community Discussions mailing list archive at
> Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Clarification on read and write locks

2011-09-26 Thread Bryce
Hi Tobias,

Thanks for the response, these are exactly the kind of details that I was
wanting to get a better understanding of.  Would be good to get a public api
lock system working, though I know you guys have lots of time pressures :-)

>From what you have said my understanding is that there is no supported
public api way of getting a read lock at present.  Doing the best with the
tools at hand at present, and understanding this could break in the future
if the implementation changes, would the following fix the acquireLock
method I posted earlier:
private void acquireLock( LockType lockType )
{
GraphDatabaseService graphDb = baseNode.getGraphDatabase();
if ( lockType == LockType.READ && graphDb instanceof
AbstractGraphDatabase )
{
Config config = ((AbstractGraphDatabase)
baseNode.getGraphDatabase()).getConfig();
config.getLockManager().getReadLock( baseNode );
config.getLockReleaser().addLockToTransaction( baseNode,
LockType.READ );
}
else {
// default to write lock if read locks unavailable
baseNode.removeProperty( "___dummy_property_to_acquire_lock___"
);
}
}

My thoughts on the questions you have posed would be:

1) Should there be release() methods for locks as well? or should
locks always be tied to the transaction?
I like the idea of having the flexibility of locking a certain node for only
a part of a transaction, especially in the case of read locks. write
locks I am not so sure about, maybe being able to release a write lock only
if you didn't actually change the locked element.

I am not sure if you have looked at the graph collections stuff, and
particularly the UnrolledLinkedList implementation I recently added, but
that is the specific situation I am thinking of.  My thinking of how an
ideal locking situation would work here would be, for reading:

   - get read lock on the node representing the first page of the ULL
   - iterate over the value relationships associated with this page
   - get read lock on the node representing the second page
   - release the read lock on the first page
   - any read locks left at the end would need to be tied to transaction,
   therefore would need to be able to attach and detach locks from transactions

For writing:

   - get write lock on the base node (representing the entry point to the
   data structure)
   - do processing to find where the write is going to occur within the data
   structure (i.e. which page(s) are changed)
   - get write lock on these page nodes
   - release write lock on base node (since it hasn't changed then this
   would be instant release)
   - change the page nodes
   - the write locks associated with the page nodes would be released with
   the transaction commit

With this as little of the data structure is being locked at a given
time but enough to stop concurrency issues (I think, this is all off the
top of my head right now).

2) Where should the methods live? On the entities themselves?
i.e. node.acquireReadLock() / node.aquireWriteLock() or on the Transaction
object to signal that locks are tied to the life cycle of the transaction?
i.e. tx.readLock(node) / tx.writeLock(node)

Bit less sure about this but something like this might work:
Lock lock = node.acquireReadLock(); // or node.aquireWriteLock()
tx.addLock(lock); // now lock is tied to transaction
tx.removeLock(lock); // no longer tied to transaction

Cheers
Bryce

On Tue, Sep 27, 2011 at 12:12 AM, Tobias Ivarsson <
tobias.ivars...@neotechnology.com> wrote:

> On Mon, Sep 26, 2011 at 2:28 AM, Bryce  wrote:
>
> > Is there any reason that access to the lock manager is a little
> difficult?
> >  Is there any issue with using it (both single server and HA)?  Are their
> > any issues to look out for?  And does the above code look workable?
> >
>
> The reason the LockManager is "hidden away" is because it is considered an
> implementation detail.
> We reserve the right to change it at any point without prior notice.
> From one version to another, the API of this class could potentially look
> completely different, or have very different semantics.
> If you write code that depend on implementation internal classes it will at
> best cause you compilation errors when upgrading to a later version of
> Neo4j, at worst break in subtle ways after you've deployed your
> application.
>
> There is another important thing to note about your sample code:
>
> On Mon, Sep 26, 2011 at 2:28 AM, Bryce  wrote:
> >
> >private void acquireLock( LockType lockType )
> >{
> >GraphDatabaseService graphDb = baseNode.getGraphDatabase();
> >if ( lockType == LockType.READ && graphDb instanceof
> > AbstractGraphDatabase )
> >{
> >((AbstractGraphDatabase)
> > baseNode.getGraphDatabase()).getConfig().getLockManager().getReadLock(
> > baseNode );
> >}
> >else {
> >// default to write lock if read locks un

Re: [Neo4j] class org.springframework.data.neo4j.config.Neo4jConfiguration is not an enhanced class

2011-09-26 Thread Wim Rijnders
HI Peter,

Whichi neo-friendly branch of what component do you mean? I am not sure
about what item you are talking about.

I found a reference
here to
a bundle-friendly classloader, is that what you mean? Unfortunately, the
corresponding link does not work.

Could you point me to the code/component in question?

Regards and thanks,

Wim.


On Thu, Sep 22, 2011 at 4:43 PM, Peter Neubauer <
peter.neuba...@neotechnology.com> wrote:

> Wim,
> you probably should try out the neo4j-friendly branch instead, but I
> hear you are already in contact with Michael on this - might actually
> be a Spring - issue ...
>
> Cheers,
>
> /peter neubauer
>
> GTalk:  neubauer.peter
> Skype   peter.neubauer
> Phone   +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter  http://twitter.com/peterneubauer
>
> http://www.neo4j.org   - Your high performance graph database.
> http://startupbootcamp.org/- Öresund - Innovation happens HERE.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>
>
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Turning Off or Configuring log rotation

2011-09-26 Thread Peter Neubauer
Very cool,
glad to hear things work out. Got any code for the .NET bindings? I
think we should start pulling client libraries into our build system,
so there are stable versions of the most used clients for Neo4j
available as part of the QA process.

WDYT?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 1:24 PM, Romiko Derbynew
 wrote:
> Hi Peter,
>
> Indeed our desired result is now achieved. Tatham added batching to our .net 
> client as well. We are more than happy to assist with HA as this is one of 
> the last hurdles we need to overcome. If you can let us know in advance when 
> the release is scheduled then we will incorporate it into our sprint for HA 
> featuring.
>
> Much appreciated and thx for 1.5 release, could not have timed it better.
>
> Sent from my iPhone
>
> On 26/09/2011, at 5:33 PM, "Peter Neubauer" 
>  wrote:
>
>> Romiko,
>> soudns good. Is this solving your current problems or is there stuff
>> left to take care of for the moment? Also, there will be some code
>> coming out on deploying Neo4j HA on Azure. Would you be interested in
>> trying that out and maybe take it a bit further?
>>
>> Cheers,
>>
>> /peter neubauer
>>
>> GTalk:      neubauer.peter
>> Skype       peter.neubauer
>> Phone       +46 704 106975
>> LinkedIn   http://www.linkedin.com/in/neubauer
>> Twitter      http://twitter.com/peterneubauer
>>
>> http://www.neo4j.org               - Your high performance graph database.
>> http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>>
>>
>>
>> On Mon, Sep 26, 2011 at 4:16 AM, Romiko Derbynew
>>  wrote:
>>> Hi Guys,
>>>
>>> We managed to fix the performance issue.
>>>
>>> * Used Paramterised Queries - increased performance by 1 extra node per 
>>> second, reduced frequency usage of heap thus no jvm memory errors
>>> * Configured our .Net client to use 70 threads for rest calls (I chose this 
>>> number as Neo4j reserves 80, so I decided 70) 5 TIMES INCREASE IN 
>>> PERFORMANCE
>>>
>>>
>>> We are now going to implement the batching to improve performance even more 
>>> :)
>>>
>>> I have updated the blog post with these findings.
>>> http://romikoderbynew.com/2011/09/23/neo4j-java-jvm-garbage-collection-profilingheap-sizelog-rotationlucene-rest-api-performance-analysis/
>>>
>>>
>>> I will write a separate post on the improvements from the batching.
>>>
>>> MyStory Welcome, Temporary from MyStory Administration. Logout?
>>> Agency Import Job 301
>>>
>>> Log output:
>>>
>>> [01:44:58] [Thread   10] Migration message received at: 26/09/2011 1:44:58 
>>> AM +00:00
>>>
>>> [01:44:59] [Thread   10] Migration message payload: JobUniqueId: 301, 
>>> AgencyUniqueId: 201
>>>
>>> [01:44:59] [Thread   10] Executing sql script: 
>>> e:\AzureTemp\Resources\4bb24cfd-0b35-4c91-83de-97f335159a49\temp\temp\RoleTemp\634526342992784034\output.sql
>>>
>>> [01:45:01] [Thread   10] Processing script command line 1/37620
>>>
>>> [01:45:03] [Thread   10] Processing script command line 2/37620
>>>
>>> [01:45:05] [Thread   10] Processing script command line 3/37620
>>>
>>> [01:45:06] [Thread   10] All sql script commands executed 37620/37620
>>>
>>> [01:45:13] [Thread   10] Importing 37619 clients into the database
>>>
>>> [01:45:44] [Thread   12] Processed 169/37619 clients, 5.62 clients per 
>>> second, 01:51:06 remaining
>>>
>>> [01:46:16] [Thread   29] Processed 360/37619 clients, 5.80 clients per 
>>> second, 01:47:02 remaining
>>>
>>> [01:46:29] [Thread   69] Processed 1000/37619 clients, 13.30 clients per 
>>> second, 00:45:52 remaining
>>>
>>> [01:46:50] [Thread   28] Processed 2000/37619 clients, 20.78 clients per 
>>> second, 00:28:34 remaining
>>>
>>> [01:47:11] [Thread   30] Processed 3000/37619 clients, 25.47 clients per 
>>> second, 00:22:39 remaining
>>>
>>> [01:47:33] [Thread   32] Processed 4000/37619 clients, 28.71 clients per 
>>> second, 00:19:31 remaining
>>>
>>> [01:47:55] [Thread   76] Processed 5000/37619 clients, 30.95 clients per 
>>> second, 00:17:33 remaining
>>>
>>> [01:48:18] [Thread   87] Processed 6000/37619 clients, 32.52 clients per 
>>> second, 00:16:12 remaining
>>>
>>> [01:48:46] [Thread   13] Processed 7000/37619 clients, 32.95 clients per 
>>> second, 00:15:29 remaining
>>>
>>> [01:49:10] [Thread   70] Processed 8000/37619 clients, 33.84 clients per 
>>> second, 00:14:35 remaining
>>>
>>> [01:49:36] [Thread   54] Processed 9000/37619 clients, 34.25 clients per 
>>> second, 00:13:55 remaining
>>>
>>> [01:50:02] [Thread   64] Processed 1/37619 clients, 34.69 clients per 

Re: [Neo4j] Killing the Wiki

2011-09-26 Thread McKinley
My top topic is http://wiki.neo4j.org/content/Servlets_with_Neo4j but it
needs to be rounded out with more general concurrency instructions too.

Thanks,

McKinley

On Sun, Sep 25, 2011 at 11:01 PM, Peter Neubauer <
peter.neuba...@neotechnology.com> wrote:

> Hi everyone,
> Anders and me are going to try to kill the wiki and put the material
> into the manual, http://docs.neo4j.org . However, some pages are more
> interesting to pull over than others.
>
> If you are interested, could you please list what your top 3 sections
> from http://wiki.neo4j.org/content/Main_Page are so we can start at
> the right spots?
>
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Killing the Wiki

2011-09-26 Thread Alexandre de Assis Bento Lima
Hi,

My top 3:

1 - Design Guide
2 - IMDB Example
3 - Configuration Settings

Cheers,
Alexandre.

On Mon, 26 Sep 2011 08:01:52 +0200, Peter Neubauer wrote
> Hi everyone,
> Anders and me are going to try to kill the wiki and put the material
> into the manual, http://docs.neo4j.org . However, some pages are more
> interesting to pull over than others.
> 
> If you are interested, could you please list what your top 3 sections
> from http://wiki.neo4j.org/content/Main_Page are so we can start at
> the right spots?
> 
> Cheers,
> 
> /peter neubauer
> 
> GTalk:      neubauer.peter
> Skype       peter.neubauer
> Phone       +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter      http://twitter.com/peterneubauer
> 
> http://www.neo4j.org               - Your high performance graph database.
> http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user


Alexandre A. B. Lima.
---
COPPE / Computer Science
Federal University of Rio de Janeiro
Rio de Janeiro, Brazil.
+55-21-2562-8690

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] java.util.logging

2011-09-26 Thread Peter Neubauer
No problem,
let us know how things go!

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 3:11 PM, yulio jordio
 wrote:
> Thanks for you fast reply Peter. I was just interested to know, as i've
> switched my app to use the slf4j api, and the logback implementation. i'll
> just add the bridge for java.util.logging.
>
> Thanks again,
>
> --
> View this message in context: 
> http://neo4j-community-discussions.438527.n3.nabble.com/java-util-logging-tp3369174p3369195.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] java.util.logging

2011-09-26 Thread yulio jordio
Thanks for you fast reply Peter. I was just interested to know, as i've
switched my app to use the slf4j api, and the logback implementation. i'll
just add the bridge for java.util.logging.

Thanks again,

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/java-util-logging-tp3369174p3369195.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] java.util.logging

2011-09-26 Thread Peter Neubauer
Yulio,
that is exactly the case. Java Logging is not the best API there is,
but we get rid of a dependency, especially for the Neo4j core libs. Do
you have any special requirement that needs to be satisfied?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 3:03 PM, yulio jordio
 wrote:
> Hi,
>
> I'm just tidying up the logging in my application and I've noticed that
> neo4j uses java.util.logging to do its logging. Is there any reason why
> that's been used instead of SLF4J? I'm guessing its probably to do with
> minimising the number of dependencies?
>
> Thanks,
>
> --
> View this message in context: 
> http://neo4j-community-discussions.438527.n3.nabble.com/java-util-logging-tp3369174p3369174.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo4j] java.util.logging

2011-09-26 Thread yulio jordio
Hi,

I'm just tidying up the logging in my application and I've noticed that
neo4j uses java.util.logging to do its logging. Is there any reason why
that's been used instead of SLF4J? I'm guessing its probably to do with
minimising the number of dependencies?

Thanks,

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/java-util-logging-tp3369174p3369174.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] neo4jphp and some cypher question

2011-09-26 Thread F. De Haes
Hi Michael, Adriano, Andres, Peter,

Thanks for your answers. I've got it working.

Greetings,
Filip

2011/9/26 Andres Taylor 

> On Sun, Sep 25, 2011 at 11:13 PM, Adriano Henrique de Almeida <
> adrianoalmei...@gmail.com> wrote:
>
> > Hi Filip,
> >
> > if you need only those nodes who have the Geemente property, you can
> > specify
> > it on the where clause. Example:
> >
> > where a.Geemente and a.Geemente = 'Antwerpen'
> >
>
> Another alternative, depending on what you are looking for, is to do this:
> WHERE a.Geemente? = 'Antwerpen'
>
> This means - if a has a property Geemente, it should be Antwerpen. But
> nodes
> without the property are still returned.
>
> Both are valid, it's just a matter of what you need.
>
> Andrés
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] neo4jphp and some cypher question

2011-09-26 Thread Andres Taylor
On Sun, Sep 25, 2011 at 11:13 PM, Adriano Henrique de Almeida <
adrianoalmei...@gmail.com> wrote:

> Hi Filip,
>
> if you need only those nodes who have the Geemente property, you can
> specify
> it on the where clause. Example:
>
> where a.Geemente and a.Geemente = 'Antwerpen'
>

Another alternative, depending on what you are looking for, is to do this:
WHERE a.Geemente? = 'Antwerpen'

This means - if a has a property Geemente, it should be Antwerpen. But nodes
without the property are still returned.

Both are valid, it's just a matter of what you need.

Andrés
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Turning Off or Configuring log rotation

2011-09-26 Thread Romiko Derbynew
Hi Peter,

Indeed our desired result is now achieved. Tatham added batching to our .net 
client as well. We are more than happy to assist with HA as this is one of the 
last hurdles we need to overcome. If you can let us know in advance when the 
release is scheduled then we will incorporate it into our sprint for HA 
featuring.

Much appreciated and thx for 1.5 release, could not have timed it better.

Sent from my iPhone

On 26/09/2011, at 5:33 PM, "Peter Neubauer"  
wrote:

> Romiko,
> soudns good. Is this solving your current problems or is there stuff
> left to take care of for the moment? Also, there will be some code
> coming out on deploying Neo4j HA on Azure. Would you be interested in
> trying that out and maybe take it a bit further?
>
> Cheers,
>
> /peter neubauer
>
> GTalk:  neubauer.peter
> Skype   peter.neubauer
> Phone   +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter  http://twitter.com/peterneubauer
>
> http://www.neo4j.org   - Your high performance graph database.
> http://startupbootcamp.org/- Öresund - Innovation happens HERE.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>
>
>
> On Mon, Sep 26, 2011 at 4:16 AM, Romiko Derbynew
>  wrote:
>> Hi Guys,
>>
>> We managed to fix the performance issue.
>>
>> * Used Paramterised Queries - increased performance by 1 extra node per 
>> second, reduced frequency usage of heap thus no jvm memory errors
>> * Configured our .Net client to use 70 threads for rest calls (I chose this 
>> number as Neo4j reserves 80, so I decided 70) 5 TIMES INCREASE IN 
>> PERFORMANCE
>>
>>
>> We are now going to implement the batching to improve performance even more 
>> :)
>>
>> I have updated the blog post with these findings.
>> http://romikoderbynew.com/2011/09/23/neo4j-java-jvm-garbage-collection-profilingheap-sizelog-rotationlucene-rest-api-performance-analysis/
>>
>>
>> I will write a separate post on the improvements from the batching.
>>
>> MyStory Welcome, Temporary from MyStory Administration. Logout?
>> Agency Import Job 301
>>
>> Log output:
>>
>> [01:44:58] [Thread   10] Migration message received at: 26/09/2011 1:44:58 
>> AM +00:00
>>
>> [01:44:59] [Thread   10] Migration message payload: JobUniqueId: 301, 
>> AgencyUniqueId: 201
>>
>> [01:44:59] [Thread   10] Executing sql script: 
>> e:\AzureTemp\Resources\4bb24cfd-0b35-4c91-83de-97f335159a49\temp\temp\RoleTemp\634526342992784034\output.sql
>>
>> [01:45:01] [Thread   10] Processing script command line 1/37620
>>
>> [01:45:03] [Thread   10] Processing script command line 2/37620
>>
>> [01:45:05] [Thread   10] Processing script command line 3/37620
>>
>> [01:45:06] [Thread   10] All sql script commands executed 37620/37620
>>
>> [01:45:13] [Thread   10] Importing 37619 clients into the database
>>
>> [01:45:44] [Thread   12] Processed 169/37619 clients, 5.62 clients per 
>> second, 01:51:06 remaining
>>
>> [01:46:16] [Thread   29] Processed 360/37619 clients, 5.80 clients per 
>> second, 01:47:02 remaining
>>
>> [01:46:29] [Thread   69] Processed 1000/37619 clients, 13.30 clients per 
>> second, 00:45:52 remaining
>>
>> [01:46:50] [Thread   28] Processed 2000/37619 clients, 20.78 clients per 
>> second, 00:28:34 remaining
>>
>> [01:47:11] [Thread   30] Processed 3000/37619 clients, 25.47 clients per 
>> second, 00:22:39 remaining
>>
>> [01:47:33] [Thread   32] Processed 4000/37619 clients, 28.71 clients per 
>> second, 00:19:31 remaining
>>
>> [01:47:55] [Thread   76] Processed 5000/37619 clients, 30.95 clients per 
>> second, 00:17:33 remaining
>>
>> [01:48:18] [Thread   87] Processed 6000/37619 clients, 32.52 clients per 
>> second, 00:16:12 remaining
>>
>> [01:48:46] [Thread   13] Processed 7000/37619 clients, 32.95 clients per 
>> second, 00:15:29 remaining
>>
>> [01:49:10] [Thread   70] Processed 8000/37619 clients, 33.84 clients per 
>> second, 00:14:35 remaining
>>
>> [01:49:36] [Thread   54] Processed 9000/37619 clients, 34.25 clients per 
>> second, 00:13:55 remaining
>>
>> [01:50:02] [Thread   64] Processed 1/37619 clients, 34.69 clients per 
>> second, 00:13:16 remaining
>>
>> [01:50:29] [Thread   50] Processed 11000/37619 clients, 34.85 clients per 
>> second, 00:12:43 remaining
>>
>> [01:50:57] [Thread   82] Processed 12000/37619 clients, 34.97 clients per 
>> second, 00:12:12 remaining
>>
>> [01:51:25] [Thread   27] Processed 13000/37619 clients, 35.02 clients per 
>> second, 00:11:42 remaining
>>
>> [01:51:54] [Thread   10] Processed 14000/37619 clients, 34.99 clients per 
>> second, 00:11:15 remaining
>>
>> [01:52:24] [Thread   76] Processed 15000/37619 clients, 34.89 clients per 
>> second, 00:10:48 remaining
>>
>> [01:52:54] [Thread   68] Processed 15970/37619 clients, 34.70 clients per 
>> second, 00:10:23 remaining
>>
>> [01:52:55] [Thread   17] Processed 16000/37619 clients, 34.69 clients per 
>> second, 00:10:23 remaining
>>
>> [01:53:25] [Thread   59] Processed 16918/37619 c

Re: [Neo4j] Clarification on read and write locks

2011-09-26 Thread Tobias Ivarsson
On Mon, Sep 26, 2011 at 2:28 AM, Bryce  wrote:

> Is there any reason that access to the lock manager is a little difficult?
>  Is there any issue with using it (both single server and HA)?  Are their
> any issues to look out for?  And does the above code look workable?
>

The reason the LockManager is "hidden away" is because it is considered an
implementation detail.
We reserve the right to change it at any point without prior notice.
>From one version to another, the API of this class could potentially look
completely different, or have very different semantics.
If you write code that depend on implementation internal classes it will at
best cause you compilation errors when upgrading to a later version of
Neo4j, at worst break in subtle ways after you've deployed your application.

There is another important thing to note about your sample code:

On Mon, Sep 26, 2011 at 2:28 AM, Bryce  wrote:
>
>private void acquireLock( LockType lockType )
>{
>GraphDatabaseService graphDb = baseNode.getGraphDatabase();
>if ( lockType == LockType.READ && graphDb instanceof
> AbstractGraphDatabase )
>{
>((AbstractGraphDatabase)
> baseNode.getGraphDatabase()).getConfig().getLockManager().getReadLock(
> baseNode );
>}
>else {
>// default to write lock if read locks unavailable
>baseNode.removeProperty( "___dummy_property_to_acquire_lock___"
> );
>}
>}
>

The two different code paths in this code have very different semantics. The
first path (if it is a read lock and the db is and AbstractGraphDatabase)
will acquire a read lock and hold it indefinitely. The second path will try
to remove a dummy property, resulting in a write lock that gets tied to the
current transaction, and thus gets released when the transaction completes.

There is another class involved in the lock management in Neo4j, it is
called LockReleaser. All locks that are acquired through the public APIs are
registered with the LockReleaser. This will tie the lock to the transaction
to ensure that the lock is released when the transaction completes. If the
lock is only acquired through the LockManager, and not registered with the
LockReleaser, you need to manually release the lock.


The reason for the remove-non-existing-property-hack is that we haven't
decided on a good public API for lock management. The questions around this
are:
* Should there be release() methods for locks as well? or should locks
always be tied to the transaction?
* Where should the methods live? On the entities themselves? i.e.
node.acquireReadLock() / node.aquireWriteLock() or on the Transaction object
to signal that locks are tied to the life cycle of the transaction? i.e.
tx.readLock(node) / tx.writeLock(node)

Cheers,
-- 
Tobias Ivarsson 
Hacker, Neo Technology
www.neotechnology.com
Cellphone: +46 706 534857 (Swe); +1 650 450 3806 (US)
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] add property to all relationships

2011-09-26 Thread Rick Bullotta
Or you could simply call the alternate version of getProperty with a default 
value of zero...

On Sep 26, 2011, at 5:44 AM, "sometime"  wrote:

> Hi!
> 
> I have 1.2M nodes and 5M relationships. Around 20 000 relationships are two
> properties: prop1 prop2, the remaining relationships have only one property:
> prop1.
> I've got all the relationships that have only one property - add prop2 with
> value "0".
> 
> It should run on all relationships and add this property?
> 
> If yes - then I have to run the robot and it will run at least 3 days.
> Can be somehow easier to solve this problem so as not to spend too much
> time?
> 
> --
> View this message in context: 
> http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368779.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] add property to all relationships

2011-09-26 Thread sometime
Can you please tell me where to read about "transaction batches" ?

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368893.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] add property to all relationships

2011-09-26 Thread Peter Neubauer
Yes,
I am just thinking that doing the property updates in transaction
batches of 10.000 relationship per transaction should not let this
take 3 days...

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 12:09 PM, sometime  wrote:
> Not 10 000, 5 000 000 !
>
> --
> View this message in context: 
> http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368830.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] add property to all relationships

2011-09-26 Thread sometime
Not 10 000, 5 000 000 !

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368830.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] add property to all relationships

2011-09-26 Thread Peter Neubauer
Viktor,
even if you do a transaction size of 10.000 updates, it runs for 3 days?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 11:44 AM, sometime  wrote:
> Hi!
>
> I have 1.2M nodes and 5M relationships. Around 20 000 relationships are two
> properties: prop1 prop2, the remaining relationships have only one property:
> prop1.
> I've got all the relationships that have only one property - add prop2 with
> value "0".
>
> It should run on all relationships and add this property?
>
> If yes - then I have to run the robot and it will run at least 3 days.
> Can be somehow easier to solve this problem so as not to spend too much
> time?
>
> --
> View this message in context: 
> http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368779.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo4j] add property to all relationships

2011-09-26 Thread sometime
Hi!

I have 1.2M nodes and 5M relationships. Around 20 000 relationships are two
properties: prop1 prop2, the remaining relationships have only one property:
prop1.
I've got all the relationships that have only one property - add prop2 with
value "0".

It should run on all relationships and add this property?

If yes - then I have to run the robot and it will run at least 3 days.
Can be somehow easier to solve this problem so as not to spend too much
time?

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368779.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Looking or help on with a design and logo making

2011-09-26 Thread Peter Neubauer
Cool Pere,
I for one am always up to new stuff, let's Beerlin soon!

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 9:48 AM, Pere Urbón Bayes  wrote:
> Hi!
>   my name is Pere Urbon, some of you already know me. Now I am preparing a
> project where I am creating a website, and some "product" image. The project
> is all about coding dojos, graph, databases, hadoop, etc... but as I am
> really bad doing design tasks, I look for help, someone who can provide the
> project a high impulse and make it beautiful.
> This project will always remain free, with and for the community, so I can
> not promisse you any salary. Obviously you will get the 100% recognition of
> your work, being a co founder of the project, links, whatever you prefer.
> So if you can help me, please drop me an email and I'll give you more
> insights, or if you are in Berlin, or Barcelona, we share a Beer and talk
> about it.
> --
> /purbon
> - @purbon
> - http://www.purbon.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "neo4jrb" group.
> To post to this group, send email to neo4...@googlegroups.com.
> To unsubscribe from this group, send email to
> neo4jrb+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/neo4jrb?hl=en.
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo4j] Looking or help on with a design and logo making

2011-09-26 Thread Pere Urbón Bayes
Hi!
  my name is Pere Urbon, some of you already know me. Now I am preparing a
project where I am creating a website, and some "product" image. The project
is all about coding dojos, graph, databases, hadoop, etc... but as I am
really bad doing design tasks, I look for help, someone who can provide the
project a high impulse and make it beautiful.

This project will always remain free, with and for the community, so I can
not promisse you any salary. Obviously you will get the 100% recognition of
your work, being a co founder of the project, links, whatever you prefer.

So if you can help me, please drop me an email and I'll give you more
insights, or if you are in Berlin, or Barcelona, we share a Beer and talk
about it.

-- 
/purbon
- @purbon
- http://www.purbon.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo4j] Turning Off or Configuring log rotation

2011-09-26 Thread Peter Neubauer
Romiko,
soudns good. Is this solving your current problems or is there stuff
left to take care of for the moment? Also, there will be some code
coming out on deploying Neo4j HA on Azure. Would you be interested in
trying that out and maybe take it a bit further?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Mon, Sep 26, 2011 at 4:16 AM, Romiko Derbynew
 wrote:
> Hi Guys,
>
> We managed to fix the performance issue.
>
> * Used Paramterised Queries - increased performance by 1 extra node per 
> second, reduced frequency usage of heap thus no jvm memory errors
> * Configured our .Net client to use 70 threads for rest calls (I chose this 
> number as Neo4j reserves 80, so I decided 70) 5 TIMES INCREASE IN 
> PERFORMANCE
>
>
> We are now going to implement the batching to improve performance even more :)
>
> I have updated the blog post with these findings.
> http://romikoderbynew.com/2011/09/23/neo4j-java-jvm-garbage-collection-profilingheap-sizelog-rotationlucene-rest-api-performance-analysis/
>
>
> I will write a separate post on the improvements from the batching.
>
> MyStory Welcome, Temporary from MyStory Administration. Logout?
> Agency Import Job 301
>
> Log output:
>
> [01:44:58] [Thread   10] Migration message received at: 26/09/2011 1:44:58 AM 
> +00:00
>
> [01:44:59] [Thread   10] Migration message payload: JobUniqueId: 301, 
> AgencyUniqueId: 201
>
> [01:44:59] [Thread   10] Executing sql script: 
> e:\AzureTemp\Resources\4bb24cfd-0b35-4c91-83de-97f335159a49\temp\temp\RoleTemp\634526342992784034\output.sql
>
> [01:45:01] [Thread   10] Processing script command line 1/37620
>
> [01:45:03] [Thread   10] Processing script command line 2/37620
>
> [01:45:05] [Thread   10] Processing script command line 3/37620
>
> [01:45:06] [Thread   10] All sql script commands executed 37620/37620
>
> [01:45:13] [Thread   10] Importing 37619 clients into the database
>
> [01:45:44] [Thread   12] Processed 169/37619 clients, 5.62 clients per 
> second, 01:51:06 remaining
>
> [01:46:16] [Thread   29] Processed 360/37619 clients, 5.80 clients per 
> second, 01:47:02 remaining
>
> [01:46:29] [Thread   69] Processed 1000/37619 clients, 13.30 clients per 
> second, 00:45:52 remaining
>
> [01:46:50] [Thread   28] Processed 2000/37619 clients, 20.78 clients per 
> second, 00:28:34 remaining
>
> [01:47:11] [Thread   30] Processed 3000/37619 clients, 25.47 clients per 
> second, 00:22:39 remaining
>
> [01:47:33] [Thread   32] Processed 4000/37619 clients, 28.71 clients per 
> second, 00:19:31 remaining
>
> [01:47:55] [Thread   76] Processed 5000/37619 clients, 30.95 clients per 
> second, 00:17:33 remaining
>
> [01:48:18] [Thread   87] Processed 6000/37619 clients, 32.52 clients per 
> second, 00:16:12 remaining
>
> [01:48:46] [Thread   13] Processed 7000/37619 clients, 32.95 clients per 
> second, 00:15:29 remaining
>
> [01:49:10] [Thread   70] Processed 8000/37619 clients, 33.84 clients per 
> second, 00:14:35 remaining
>
> [01:49:36] [Thread   54] Processed 9000/37619 clients, 34.25 clients per 
> second, 00:13:55 remaining
>
> [01:50:02] [Thread   64] Processed 1/37619 clients, 34.69 clients per 
> second, 00:13:16 remaining
>
> [01:50:29] [Thread   50] Processed 11000/37619 clients, 34.85 clients per 
> second, 00:12:43 remaining
>
> [01:50:57] [Thread   82] Processed 12000/37619 clients, 34.97 clients per 
> second, 00:12:12 remaining
>
> [01:51:25] [Thread   27] Processed 13000/37619 clients, 35.02 clients per 
> second, 00:11:42 remaining
>
> [01:51:54] [Thread   10] Processed 14000/37619 clients, 34.99 clients per 
> second, 00:11:15 remaining
>
> [01:52:24] [Thread   76] Processed 15000/37619 clients, 34.89 clients per 
> second, 00:10:48 remaining
>
> [01:52:54] [Thread   68] Processed 15970/37619 clients, 34.70 clients per 
> second, 00:10:23 remaining
>
> [01:52:55] [Thread   17] Processed 16000/37619 clients, 34.69 clients per 
> second, 00:10:23 remaining
>
> [01:53:25] [Thread   59] Processed 16918/37619 clients, 34.43 clients per 
> second, 00:10:01 remaining
>
> [01:53:27] [Thread   46] Processed 17000/37619 clients, 34.42 clients per 
> second, 00:09:59 remaining
>
> [01:53:58] [Thread   58] Processed 17998/37619 clients, 34.33 clients per 
> second, 00:09:31 remaining
>
> [01:53:58] [Thread   34] Processed 18000/37619 clients, 34.32 clients per 
> second, 00:09:31 remaining
>
> [01:54:28] [Thread   21] Processed 18900/37619 clients, 34.07 clients per 
> second, 00:09:09 remaining
>
> [01:54:34] [Thread   13] Processed 19000/37619 clients, 33.90 clients per 
> second, 00:09:09 remaining
>
> [01:55:04] [Thread   35] Pro