Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J

2010-12-04 Thread Peter Neubauer 
Rick,
HA is totally transparent to any client of the embedded API after
setup. That means, if you have a number of Neo4j Server (or any other
thing like various bindings etc) the only difference is in the
instantiation of the GraphDatabaseService.

So, from the REST (or your own app) API you are not seeing HA at all.
It just looks like one DB running. To scale, you can set up one
endpoint to every HA slave in your cluster, and build your custom load
balancing (e.g. with AKKA etc) on top of that. See
http://docs.neo4j.org/html/milestone/#_high_availability for more
info.

Will be happy to help you give it a spin!

Where the REST layer might come into play is the upcoming sharding
effort - going to start working on that after 1.2, that is, pretty
soon :)

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Sat, Dec 4, 2010 at 6:32 PM, Rick Bullotta
 wrote:
> Hi, Peter.
>
> That makes sense.  I was anticipating roughly 10X-100X difference.  Good to
> see we're near the lower end of that, though I'm not surprised.  You guys
> make fast stuff! ;-)
>
> Another question: Is the REST API integral to the HA scenario or do you
> envision that at some point the HA model be implemented with embedded
> instances also?  I'm just a bit unclear on how the HA/REST interaction (or
> more specifically, the HA architecture) works.  Any Wiki page I should look
> at?
>
> Thanks!
>
> Rick
>
> -Original Message-
> From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
> Behalf Of Peter Neubauer
> Sent: Saturday, December 04, 2010 12:17 PM
> To: Neo4j user discussions
> Subject: Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J
>
> Well,
> we REST having the serialization overhead, plus the crossing of system
> boundaries regarding caching, and the problem with transaction
> handling, it is no doubt much more performant to use the embedded
> Neo4j scenario when you want optimum performance and can control all
> aspects of your system.
>
> However, REST is great for integration and easy start, and access
> outside Java. The third approach is a native driver protocol, maybe
> built on Protobuffer, Thrift or others, that should in performance be
> somewhere in between.
>
> The Neo4j REST API is not on par with the native approach especially
> regarding many (small) operations, we first need to enable better
> server side support with either scripts being executed serverside
> (dynamic via sending them or static via server extensions). Rexster is
> simply exposing Jersey and JAXRS for Java classes, taking away some of
> the burden of building your own server. We are right now debating on
> how to best make server extensions both live near the Neo4j instance
> and at the same time be isolated from the server, so the server can
> perform without custom extension destroying things, much like stored
> procedures in RDBMS. It looks like OSGi might perform good services
> here ...
>
> Anyway, benchmarks are hard to do since the range of functionality is
> very different. In traversal scenarios though I think it is, and the
> difference should not be too big, since most of the job is done on the
> server side, the main overhead is the serialization of the request and
> the results. Again, it depend on how your usage scenarios look like.
>
> For small transactions, I have been running an insert of 1 nodes
> via addNode() over REST with https://github.com/maxdemarzi/neography
> in about 10sec, while the same takes less than 1sec on my machine in
> embedded mode (IIRC) ... small transactions needing to hit disc.
>
> Cheers,
>
> /peter neubauer
>
> GTalk:      neubauer.peter
> Skype       peter.neubauer
> Phone       +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter      http://twitter.com/peterneubauer
>
> http://www.neo4j.org               - Your high performance graph database.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>
>
>
> On Sat, Dec 4, 2010 at 3:22 PM, Rick Bullotta
>  wrote:
>> Has anyone done any rough performance comparisons of the two approaches?
>  I
>> have to think we're looking at 1 or 2 orders of magnitude difference, but
>> would like to know if there is any hard data yet.
>>
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>

Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J

2010-12-04 Thread Rick Bullotta 
Hi, Peter.

That makes sense.  I was anticipating roughly 10X-100X difference.  Good to
see we're near the lower end of that, though I'm not surprised.  You guys
make fast stuff! ;-)

Another question: Is the REST API integral to the HA scenario or do you
envision that at some point the HA model be implemented with embedded
instances also?  I'm just a bit unclear on how the HA/REST interaction (or
more specifically, the HA architecture) works.  Any Wiki page I should look
at?

Thanks!

Rick

-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Peter Neubauer
Sent: Saturday, December 04, 2010 12:17 PM
To: Neo4j user discussions
Subject: Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J

Well,
we REST having the serialization overhead, plus the crossing of system
boundaries regarding caching, and the problem with transaction
handling, it is no doubt much more performant to use the embedded
Neo4j scenario when you want optimum performance and can control all
aspects of your system.

However, REST is great for integration and easy start, and access
outside Java. The third approach is a native driver protocol, maybe
built on Protobuffer, Thrift or others, that should in performance be
somewhere in between.

The Neo4j REST API is not on par with the native approach especially
regarding many (small) operations, we first need to enable better
server side support with either scripts being executed serverside
(dynamic via sending them or static via server extensions). Rexster is
simply exposing Jersey and JAXRS for Java classes, taking away some of
the burden of building your own server. We are right now debating on
how to best make server extensions both live near the Neo4j instance
and at the same time be isolated from the server, so the server can
perform without custom extension destroying things, much like stored
procedures in RDBMS. It looks like OSGi might perform good services
here ...

Anyway, benchmarks are hard to do since the range of functionality is
very different. In traversal scenarios though I think it is, and the
difference should not be too big, since most of the job is done on the
server side, the main overhead is the serialization of the request and
the results. Again, it depend on how your usage scenarios look like.

For small transactions, I have been running an insert of 1 nodes
via addNode() over REST with https://github.com/maxdemarzi/neography
in about 10sec, while the same takes less than 1sec on my machine in
embedded mode (IIRC) ... small transactions needing to hit disc.

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Sat, Dec 4, 2010 at 3:22 PM, Rick Bullotta
 wrote:
> Has anyone done any rough performance comparisons of the two approaches?
 I
> have to think we're looking at 1 or 2 orders of magnitude difference, but
> would like to know if there is any hard data yet.
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J

2010-12-04 Thread Peter Neubauer 
Well,
we REST having the serialization overhead, plus the crossing of system
boundaries regarding caching, and the problem with transaction
handling, it is no doubt much more performant to use the embedded
Neo4j scenario when you want optimum performance and can control all
aspects of your system.

However, REST is great for integration and easy start, and access
outside Java. The third approach is a native driver protocol, maybe
built on Protobuffer, Thrift or others, that should in performance be
somewhere in between.

The Neo4j REST API is not on par with the native approach especially
regarding many (small) operations, we first need to enable better
server side support with either scripts being executed serverside
(dynamic via sending them or static via server extensions). Rexster is
simply exposing Jersey and JAXRS for Java classes, taking away some of
the burden of building your own server. We are right now debating on
how to best make server extensions both live near the Neo4j instance
and at the same time be isolated from the server, so the server can
perform without custom extension destroying things, much like stored
procedures in RDBMS. It looks like OSGi might perform good services
here ...

Anyway, benchmarks are hard to do since the range of functionality is
very different. In traversal scenarios though I think it is, and the
difference should not be too big, since most of the job is done on the
server side, the main overhead is the serialization of the request and
the results. Again, it depend on how your usage scenarios look like.

For small transactions, I have been running an insert of 1 nodes
via addNode() over REST with https://github.com/maxdemarzi/neography
in about 10sec, while the same takes less than 1sec on my machine in
embedded mode (IIRC) ... small transactions needing to hit disc.

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Sat, Dec 4, 2010 at 3:22 PM, Rick Bullotta
 wrote:
> Has anyone done any rough performance comparisons of the two approaches?  I
> have to think we're looking at 1 or 2 orders of magnitude difference, but
> would like to know if there is any hard data yet.
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J

2010-12-04 Thread Javier de la Rosa 
I have the same doubts.

On Sat, Dec 4, 2010 at 09:50, Alex Averbuch  wrote:

> "Rexster  is a RESTful graph shell
> that exposes any Blueprints graph through a standalone HTTP server"
>

It's better Rexster or the Neo4 REST native component?



-- 
Javier de la Rosa
http://versae.es
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] [SPAM] Re: Performance benchmarks of RES T API vs embedded Neo4J

2010-12-04 Thread rick.bullo...@burningskysoftware.com 
Everything. ;-)

Seriously though, it would good to measure small and large sets of node inserts 
and deletes, with varying numbers and types of properties and relationships, as 
well as a spectrum of queries/traversals, etc.



- Reply message -
From: "Alex Averbuch" 
Date: Sat, Dec 4, 2010 9:50 am
Subject: [SPAM] Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J
To: "Neo4j user discussions" 

Hi Rick,
You could get a ballpark comparison by using
Blueprints,
and comparing the performance of
Neo4jGraph
with
the performance of
RexsterGraph
..

"Rexster  is a RESTful graph shell
that exposes any Blueprints graph through a standalone HTTP server"

Using Neo4jGraph is straight forward.
For a discussion on using RexsterGraph go
here
..

If you know the type of graph operations that you would like to test,
GraphDB-Bench  can be used to
compare performance of the two Graph implementations (Embedded vs REST).

What type of graph operations would you like to test?
I might be able to help by setting up the specific GraphDB-Bench benchmark
for you.

Cheers,
Alex

On Sat, Dec 4, 2010 at 3:22 PM, Rick Bullotta <
rick.bullo...@burningskysoftware.com> wrote:

> Has anyone done any rough performance comparisons of the two approaches?  I
> have to think we're looking at 1 or 2 orders of magnitude difference, but
> would like to know if there is any hard data yet.
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Performance benchmarks of REST API vs embedded Neo4J

2010-12-04 Thread Alex Averbuch 
Hi Rick,
You could get a ballpark comparison by using
Blueprints,
and comparing the performance of
Neo4jGraph
with
the performance of
RexsterGraph
.

"Rexster  is a RESTful graph shell
that exposes any Blueprints graph through a standalone HTTP server"

Using Neo4jGraph is straight forward.
For a discussion on using RexsterGraph go
here
.

If you know the type of graph operations that you would like to test,
GraphDB-Bench  can be used to
compare performance of the two Graph implementations (Embedded vs REST).

What type of graph operations would you like to test?
I might be able to help by setting up the specific GraphDB-Bench benchmark
for you.

Cheers,
Alex

On Sat, Dec 4, 2010 at 3:22 PM, Rick Bullotta <
rick.bullo...@burningskysoftware.com> wrote:

> Has anyone done any rough performance comparisons of the two approaches?  I
> have to think we're looking at 1 or 2 orders of magnitude difference, but
> would like to know if there is any hard data yet.
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] Performance benchmarks of REST API vs embedded Neo4J

2010-12-04 Thread Rick Bullotta 
Has anyone done any rough performance comparisons of the two approaches?  I
have to think we're looking at 1 or 2 orders of magnitude difference, but
would like to know if there is any hard data yet.

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user