Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-26 Thread Peter Neubauer
Cool,
let us know how things work out with the full dataset!

Cheers,

/peter neubauer

COO and Sales, Neo Technology

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org - Your high performance graph database.
http://www.tinkerpop.com  - Processing for Internet-scale graphs.
http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



On Thu, Mar 25, 2010 at 8:58 PM, Laurent Laborde kerdez...@gmail.com wrote:
 my new ssd (Intel X25-V, 40GB, 114€) is finally installed.
 i continue my collatz number crunching from 90 millions to 100 millions.
 Result : i'm cpu-bound again (1 core at 100%)

 when the base was small enough to fit in ram it took ~10s to compute a
 block of 10k.
 now the db is ~22Go and it took more than a mn to compute a block of
 10k on the raptor 10krpm.

 i moved the base to the ssd, then started the computation again.
 I'm back to ~10s per 10k.

 i'm running the windows monitor, while computing i have a mix of :
 - 1.5 MB/s read
 - 2.5 MB/s write

 The queue length on the ssd is at ~10% (while it was, of course, 100%
 on the raptor).

 A side note :
 When moving the DB from the raptor to the SSD, the ssd was the
 bottlenack, but it make sense :
 It's faster to sequentially read from a 10krpm than sequentially write
 on a cheap SSD. :)

 conclusion :
 TOTALLY ROCKS \o/


 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-25 Thread Laurent Laborde
my new ssd (Intel X25-V, 40GB, 114€) is finally installed.
i continue my collatz number crunching from 90 millions to 100 millions.
Result : i'm cpu-bound again (1 core at 100%)

when the base was small enough to fit in ram it took ~10s to compute a
block of 10k.
now the db is ~22Go and it took more than a mn to compute a block of
10k on the raptor 10krpm.

i moved the base to the ssd, then started the computation again.
I'm back to ~10s per 10k.

i'm running the windows monitor, while computing i have a mix of :
- 1.5 MB/s read
- 2.5 MB/s write

The queue length on the ssd is at ~10% (while it was, of course, 100%
on the raptor).

A side note :
When moving the DB from the raptor to the SSD, the ssd was the
bottlenack, but it make sense :
It's faster to sequentially read from a 10krpm than sequentially write
on a cheap SSD. :)

conclusion :
TOTALLY ROCKS \o/


-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-23 Thread Peter Neubauer
Laurent,
have you looked at the BatchInserter for doing big initial
populations? It is MUCH faster than the normal Neo4j transactional
approach, see http://wiki.neo4j.org/content/Indexing_with_BatchInserter

Would that help?

Cheers,

/peter neubauer

COO and Sales, Neo Technology

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org - Your high performance graph database.
http://www.tinkerpop.com  - Processing for Internet-scale graphs.
http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



On Mon, Mar 22, 2010 at 11:35 PM, Laurent Laborde kerdez...@gmail.com wrote:
 On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com wrote:
 thank you peter.
 Following the various link, i found http://www.cytoscape.org/
 Look very promising !! i'll try this weekend :)

 3 days later, i'm still populating neo4j like a crazy, trying to
 compute the collatz conjecture up to 100 millions.
 i'm at 63.5 millions, but it's now much much slower than at the beginning.

 probably because of the insane numbers of index.getSingleNode on
 lucene on a growing index.
 i'm expecting something like a billion of node when it will reach 100 
 millions.

 the index is 6GB and the base is 9GB.
 The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6
 GB of data in the neo4j drectory with only 8GB on ram on my windows 7
 i'm now (random) IO bound.

 My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad;
 considering it's highly mixed R/W and purely random IO)

 time to buy a 32GB SSD :)

 --
 Keru


 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-23 Thread Laurent Laborde
nope.
The LuceneIndexBatchInserter is designed for being performant when
inserting large amounts of data with minimal lookups from the index
during that time.

problem : i do a lot of lookup while inserting :)

-- 
Ker2x


On Tue, Mar 23, 2010 at 10:43 AM, Peter Neubauer
neubauer.pe...@gmail.com wrote:
 Laurent,
 have you looked at the BatchInserter for doing big initial
 populations? It is MUCH faster than the normal Neo4j transactional
 approach, see http://wiki.neo4j.org/content/Indexing_with_BatchInserter

 Would that help?

 Cheers,

 /peter neubauer

 COO and Sales, Neo Technology

 GTalk:      neubauer.peter
 Skype       peter.neubauer
 Phone       +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter      http://twitter.com/peterneubauer

 http://www.neo4j.org             - Your high performance graph database.
 http://www.tinkerpop.com      - Processing for Internet-scale graphs.
 http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



 On Mon, Mar 22, 2010 at 11:35 PM, Laurent Laborde kerdez...@gmail.com wrote:
 On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 thank you peter.
 Following the various link, i found http://www.cytoscape.org/
 Look very promising !! i'll try this weekend :)

 3 days later, i'm still populating neo4j like a crazy, trying to
 compute the collatz conjecture up to 100 millions.
 i'm at 63.5 millions, but it's now much much slower than at the beginning.

 probably because of the insane numbers of index.getSingleNode on
 lucene on a growing index.
 i'm expecting something like a billion of node when it will reach 100 
 millions.

 the index is 6GB and the base is 9GB.
 The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6
 GB of data in the neo4j drectory with only 8GB on ram on my windows 7
 i'm now (random) IO bound.

 My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad;
 considering it's highly mixed R/W and purely random IO)

 time to buy a 32GB SSD :)

 --
 Keru


 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-23 Thread Peter Neubauer
Awh,
kinda not very fast inserting then. From looking at the code, this
kinda sux for fast inserting since you need to go in and out of
indexes. Not sure how to speed that up. Could you keep some of the
index in memory maybe?

Cheers,

/peter neubauer

COO and Sales, Neo Technology

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org - Your high performance graph database.
http://www.tinkerpop.com  - Processing for Internet-scale graphs.
http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



On Tue, Mar 23, 2010 at 10:49 AM, Laurent Laborde kerdez...@gmail.com wrote:
 nope.
 The LuceneIndexBatchInserter is designed for being performant when
 inserting large amounts of data with minimal lookups from the index
 during that time.

 problem : i do a lot of lookup while inserting :)

 --
 Ker2x


 On Tue, Mar 23, 2010 at 10:43 AM, Peter Neubauer
 neubauer.pe...@gmail.com wrote:
 Laurent,
 have you looked at the BatchInserter for doing big initial
 populations? It is MUCH faster than the normal Neo4j transactional
 approach, see http://wiki.neo4j.org/content/Indexing_with_BatchInserter

 Would that help?

 Cheers,

 /peter neubauer

 COO and Sales, Neo Technology

 GTalk:      neubauer.peter
 Skype       peter.neubauer
 Phone       +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter      http://twitter.com/peterneubauer

 http://www.neo4j.org             - Your high performance graph database.
 http://www.tinkerpop.com      - Processing for Internet-scale graphs.
 http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



 On Mon, Mar 22, 2010 at 11:35 PM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 thank you peter.
 Following the various link, i found http://www.cytoscape.org/
 Look very promising !! i'll try this weekend :)

 3 days later, i'm still populating neo4j like a crazy, trying to
 compute the collatz conjecture up to 100 millions.
 i'm at 63.5 millions, but it's now much much slower than at the beginning.

 probably because of the insane numbers of index.getSingleNode on
 lucene on a growing index.
 i'm expecting something like a billion of node when it will reach 100 
 millions.

 the index is 6GB and the base is 9GB.
 The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6
 GB of data in the neo4j drectory with only 8GB on ram on my windows 7
 i'm now (random) IO bound.

 My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad;
 considering it's highly mixed R/W and purely random IO)

 time to buy a 32GB SSD :)

 --
 Keru


 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-23 Thread Craig Taverner
Why not use a graph based index? Properly structured it should provide fast
reads all the time (property structured kind-of means you have the most
likely reads in cache, which is dependant on the type of data).

On Tue, Mar 23, 2010 at 10:56 AM, Peter Neubauer
neubauer.pe...@gmail.comwrote:

 Awh,
 kinda not very fast inserting then. From looking at the code, this
 kinda sux for fast inserting since you need to go in and out of
 indexes. Not sure how to speed that up. Could you keep some of the
 index in memory maybe?

 Cheers,

 /peter neubauer

 COO and Sales, Neo Technology

 GTalk:  neubauer.peter
 Skype   peter.neubauer
 Phone   +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter  http://twitter.com/peterneubauer

 http://www.neo4j.org - Your high performance graph database.
 http://www.tinkerpop.com  - Processing for Internet-scale graphs.
 http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



 On Tue, Mar 23, 2010 at 10:49 AM, Laurent Laborde kerdez...@gmail.com
 wrote:
  nope.
  The LuceneIndexBatchInserter is designed for being performant when
  inserting large amounts of data with minimal lookups from the index
  during that time.
 
  problem : i do a lot of lookup while inserting :)
 
  --
  Ker2x
 
 
  On Tue, Mar 23, 2010 at 10:43 AM, Peter Neubauer
  neubauer.pe...@gmail.com wrote:
  Laurent,
  have you looked at the BatchInserter for doing big initial
  populations? It is MUCH faster than the normal Neo4j transactional
  approach, see http://wiki.neo4j.org/content/Indexing_with_BatchInserter
 
  Would that help?
 
  Cheers,
 
  /peter neubauer
 
  COO and Sales, Neo Technology
 
  GTalk:  neubauer.peter
  Skype   peter.neubauer
  Phone   +46 704 106975
  LinkedIn   http://www.linkedin.com/in/neubauer
  Twitter  http://twitter.com/peterneubauer
 
  http://www.neo4j.org - Your high performance graph
 database.
  http://www.tinkerpop.com  - Processing for Internet-scale graphs.
  http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.
 
 
 
  On Mon, Mar 22, 2010 at 11:35 PM, Laurent Laborde kerdez...@gmail.com
 wrote:
  On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com
 wrote:
  thank you peter.
  Following the various link, i found http://www.cytoscape.org/
  Look very promising !! i'll try this weekend :)
 
  3 days later, i'm still populating neo4j like a crazy, trying to
  compute the collatz conjecture up to 100 millions.
  i'm at 63.5 millions, but it's now much much slower than at the
 beginning.
 
  probably because of the insane numbers of index.getSingleNode on
  lucene on a growing index.
  i'm expecting something like a billion of node when it will reach 100
 millions.
 
  the index is 6GB and the base is 9GB.
  The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6
  GB of data in the neo4j drectory with only 8GB on ram on my windows 7
  i'm now (random) IO bound.
 
  My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad;
  considering it's highly mixed R/W and purely random IO)
 
  time to buy a 32GB SSD :)
 
  --
  Keru
 
 
  --
  Laurent ker2x Laborde
  Sysadmin  DBA at http://www.over-blog.com/
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
 
 
 
  --
  Laurent ker2x Laborde
  Sysadmin  DBA at http://www.over-blog.com/
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-23 Thread Laurent Laborde
i will try and benchmark :)

-- 
Ker2x

On Tue, Mar 23, 2010 at 11:09 AM, Craig Taverner cr...@amanzi.com wrote:
 Why not use a graph based index? Properly structured it should provide fast
 reads all the time (property structured kind-of means you have the most
 likely reads in cache, which is dependant on the type of data).

 On Tue, Mar 23, 2010 at 10:56 AM, Peter Neubauer
 neubauer.pe...@gmail.comwrote:

 Awh,
 kinda not very fast inserting then. From looking at the code, this
 kinda sux for fast inserting since you need to go in and out of
 indexes. Not sure how to speed that up. Could you keep some of the
 index in memory maybe?

 Cheers,

 /peter neubauer

 COO and Sales, Neo Technology

 GTalk:      neubauer.peter
 Skype       peter.neubauer
 Phone       +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter      http://twitter.com/peterneubauer

 http://www.neo4j.org             - Your high performance graph database.
 http://www.tinkerpop.com      - Processing for Internet-scale graphs.
 http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



 On Tue, Mar 23, 2010 at 10:49 AM, Laurent Laborde kerdez...@gmail.com
 wrote:
  nope.
  The LuceneIndexBatchInserter is designed for being performant when
  inserting large amounts of data with minimal lookups from the index
  during that time.
 
  problem : i do a lot of lookup while inserting :)
 
  --
  Ker2x
 
 
  On Tue, Mar 23, 2010 at 10:43 AM, Peter Neubauer
  neubauer.pe...@gmail.com wrote:
  Laurent,
  have you looked at the BatchInserter for doing big initial
  populations? It is MUCH faster than the normal Neo4j transactional
  approach, see http://wiki.neo4j.org/content/Indexing_with_BatchInserter
 
  Would that help?
 
  Cheers,
 
  /peter neubauer
 
  COO and Sales, Neo Technology
 
  GTalk:      neubauer.peter
  Skype       peter.neubauer
  Phone       +46 704 106975
  LinkedIn   http://www.linkedin.com/in/neubauer
  Twitter      http://twitter.com/peterneubauer
 
  http://www.neo4j.org             - Your high performance graph
 database.
  http://www.tinkerpop.com      - Processing for Internet-scale graphs.
  http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.
 
 
 
  On Mon, Mar 22, 2010 at 11:35 PM, Laurent Laborde kerdez...@gmail.com
 wrote:
  On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com
 wrote:
  thank you peter.
  Following the various link, i found http://www.cytoscape.org/
  Look very promising !! i'll try this weekend :)
 
  3 days later, i'm still populating neo4j like a crazy, trying to
  compute the collatz conjecture up to 100 millions.
  i'm at 63.5 millions, but it's now much much slower than at the
 beginning.
 
  probably because of the insane numbers of index.getSingleNode on
  lucene on a growing index.
  i'm expecting something like a billion of node when it will reach 100
 millions.
 
  the index is 6GB and the base is 9GB.
  The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6
  GB of data in the neo4j drectory with only 8GB on ram on my windows 7
  i'm now (random) IO bound.
 
  My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad;
  considering it's highly mixed R/W and purely random IO)
 
  time to buy a 32GB SSD :)
 
  --
  Keru
 
 
  --
  Laurent ker2x Laborde
  Sysadmin  DBA at http://www.over-blog.com/
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
 
 
 
  --
  Laurent ker2x Laborde
  Sysadmin  DBA at http://www.over-blog.com/
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-23 Thread Laurent Laborde
i ordered a SSD Intel X25-V (114€)
According to http://www.anandtech.com/storage/showdoc.aspx?i=3773 it's
a very good one, considering the very low price (for a SSD).
Not the best (of course), but still faster than my HDD Velociraptor 10krpm.

40GB (~30GB formated) should be enough for all my personnal neo4j needs.

-- 
Ker2x

On Tue, Mar 23, 2010 at 11:26 AM, Laurent Laborde kerdez...@gmail.com wrote:
 i will try and benchmark :)

 --
 Ker2x

 On Tue, Mar 23, 2010 at 11:09 AM, Craig Taverner cr...@amanzi.com wrote:
 Why not use a graph based index? Properly structured it should provide fast
 reads all the time (property structured kind-of means you have the most
 likely reads in cache, which is dependant on the type of data).

 On Tue, Mar 23, 2010 at 10:56 AM, Peter Neubauer
 neubauer.pe...@gmail.comwrote:

 Awh,
 kinda not very fast inserting then. From looking at the code, this
 kinda sux for fast inserting since you need to go in and out of
 indexes. Not sure how to speed that up. Could you keep some of the
 index in memory maybe?

 Cheers,

 /peter neubauer

 COO and Sales, Neo Technology

 GTalk:      neubauer.peter
 Skype       peter.neubauer
 Phone       +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter      http://twitter.com/peterneubauer

 http://www.neo4j.org             - Your high performance graph database.
 http://www.tinkerpop.com      - Processing for Internet-scale graphs.
 http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



 On Tue, Mar 23, 2010 at 10:49 AM, Laurent Laborde kerdez...@gmail.com
 wrote:
  nope.
  The LuceneIndexBatchInserter is designed for being performant when
  inserting large amounts of data with minimal lookups from the index
  during that time.
 
  problem : i do a lot of lookup while inserting :)
 
  --
  Ker2x
 
 
  On Tue, Mar 23, 2010 at 10:43 AM, Peter Neubauer
  neubauer.pe...@gmail.com wrote:
  Laurent,
  have you looked at the BatchInserter for doing big initial
  populations? It is MUCH faster than the normal Neo4j transactional
  approach, see http://wiki.neo4j.org/content/Indexing_with_BatchInserter
 
  Would that help?
 
  Cheers,
 
  /peter neubauer
 
  COO and Sales, Neo Technology
 
  GTalk:      neubauer.peter
  Skype       peter.neubauer
  Phone       +46 704 106975
  LinkedIn   http://www.linkedin.com/in/neubauer
  Twitter      http://twitter.com/peterneubauer
 
  http://www.neo4j.org             - Your high performance graph
 database.
  http://www.tinkerpop.com      - Processing for Internet-scale graphs.
  http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.
 
 
 
  On Mon, Mar 22, 2010 at 11:35 PM, Laurent Laborde kerdez...@gmail.com
 wrote:
  On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com
 wrote:
  thank you peter.
  Following the various link, i found http://www.cytoscape.org/
  Look very promising !! i'll try this weekend :)
 
  3 days later, i'm still populating neo4j like a crazy, trying to
  compute the collatz conjecture up to 100 millions.
  i'm at 63.5 millions, but it's now much much slower than at the
 beginning.
 
  probably because of the insane numbers of index.getSingleNode on
  lucene on a growing index.
  i'm expecting something like a billion of node when it will reach 100
 millions.
 
  the index is 6GB and the base is 9GB.
  The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6
  GB of data in the neo4j drectory with only 8GB on ram on my windows 7
  i'm now (random) IO bound.
 
  My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad;
  considering it's highly mixed R/W and purely random IO)
 
  time to buy a 32GB SSD :)
 
  --
  Keru
 
 
  --
  Laurent ker2x Laborde
  Sysadmin  DBA at http://www.over-blog.com/
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
 
 
 
  --
  Laurent ker2x Laborde
  Sysadmin  DBA at http://www.over-blog.com/
  ___
  Neo mailing list
  User@lists.neo4j.org
  https://lists.neo4j.org/mailman/listinfo/user
 
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/




-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-22 Thread Laurent Laborde
On Fri, Mar 19, 2010 at 10:20 AM, Laurent Laborde kerdez...@gmail.com wrote:
 thank you peter.
 Following the various link, i found http://www.cytoscape.org/
 Look very promising !! i'll try this weekend :)

3 days later, i'm still populating neo4j like a crazy, trying to
compute the collatz conjecture up to 100 millions.
i'm at 63.5 millions, but it's now much much slower than at the beginning.

probably because of the insane numbers of index.getSingleNode on
lucene on a growing index.
i'm expecting something like a billion of node when it will reach 100 millions.

the index is 6GB and the base is 9GB.
The cpu usage is now ~10% instead of 25%, i have a grand total of 9+6
GB of data in the neo4j drectory with only 8GB on ram on my windows 7
i'm now (random) IO bound.

My SATA velociraptor 10krpm is fighting at 1MB/s (not so bad;
considering it's highly mixed R/W and purely random IO)

time to buy a 32GB SSD :)

-- 
Keru


-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-19 Thread Peter Neubauer
Very cool!

A fast search gave me this thread:
http://groups.google.com/group/view-theinfo/browse_thread/thread/731b82b961fd557a
that might have some hints.

Otherwise, we have collected a few options on
http://wiki.neo4j.org/content/Visualization_options_for_graphs, feel
free to add - specially Ubigraph seems to render larger graphs well
from what they say, but it's not OSS.

Cheers,

/peter neubauer

COO and Sales, Neo Technology

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org - Your high performance graph database.
http://www.tinkerpop.com  - Processing for Internet-scale graphs.
http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



On Thu, Mar 18, 2010 at 10:59 PM, Laurent Laborde kerdez...@gmail.com wrote:
 i quickly hacked an app to populate the neo4j database with the
 collatz conjecture graph.
 here it is http://github.com/ker2x/Collatz4Neo/blob/master/collatz4neo.java
 and the code is ugly, but it just works(c)(r)(tm)

 now the problem is to find a way to graph millions of nodes. :/
 something like this could be nice :
 http://www.race.u-tokyo.ac.jp/~uchida/blogdata/dataset1_community_normal.png

 *hugs*

 --
 Ker2x

 On Thu, Mar 18, 2010 at 3:20 PM, Peter Neubauer
 neubauer.pe...@gmail.com wrote:
 Still,
 very cool, and processing rocks!

 Cheers,

 /peter neubauer

 COO and Sales, Neo Technology

 GTalk:      neubauer.peter
 Skype       peter.neubauer
 Phone       +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter      http://twitter.com/peterneubauer

 http://www.neo4j.org             - Your high performance graph database.
 http://www.tinkerpop.com      - Processing for Internet-scale graphs.
 http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



 On Tue, Mar 16, 2010 at 12:21 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 i totally forgot to say that it's jsut a proof of concept, this small
 app don't use neo4j

 --
 Ker2x

 On Tue, Mar 16, 2010 at 12:19 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 i used processing : http://www.processing.org/
 + the traer physic lib : http://www.cs.princeton.edu/~traer/physics/

 --
 Ker2x


 On Tue, Mar 16, 2010 at 12:14 AM, Mattias Persson
 matt...@neotechnology.com wrote:
 Cool :) What library do you use to visualize the graph?

 2010/3/16 Laurent Laborde kerdez...@gmail.com:
 On Mon, Mar 15, 2010 at 12:13 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 thanks to xkcd, it just made me notice that exploring the collatz
 conjecture is just exploring a huge graph.
 http://xkcd.com/710/

 If i have some spare-spare-spare-spare time, i'll try that. (and draw 
 the graph)

 i built a small version in processing (java applet) that do not scale
 to huge number, but it's fun :
 http://ker.endofinternet.net/img/collatz/

 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Mattias Persson, [matt...@neotechnology.com]
 Neo Technology, www.neotechnology.com
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-19 Thread Laurent Laborde
thank you peter.
Following the various link, i found http://www.cytoscape.org/
Look very promising !! i'll try this weekend :)

-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-18 Thread Peter Neubauer
Still,
very cool, and processing rocks!

Cheers,

/peter neubauer

COO and Sales, Neo Technology

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org - Your high performance graph database.
http://www.tinkerpop.com  - Processing for Internet-scale graphs.
http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



On Tue, Mar 16, 2010 at 12:21 AM, Laurent Laborde kerdez...@gmail.com wrote:
 i totally forgot to say that it's jsut a proof of concept, this small
 app don't use neo4j

 --
 Ker2x

 On Tue, Mar 16, 2010 at 12:19 AM, Laurent Laborde kerdez...@gmail.com wrote:
 i used processing : http://www.processing.org/
 + the traer physic lib : http://www.cs.princeton.edu/~traer/physics/

 --
 Ker2x


 On Tue, Mar 16, 2010 at 12:14 AM, Mattias Persson
 matt...@neotechnology.com wrote:
 Cool :) What library do you use to visualize the graph?

 2010/3/16 Laurent Laborde kerdez...@gmail.com:
 On Mon, Mar 15, 2010 at 12:13 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 thanks to xkcd, it just made me notice that exploring the collatz
 conjecture is just exploring a huge graph.
 http://xkcd.com/710/

 If i have some spare-spare-spare-spare time, i'll try that. (and draw the 
 graph)

 i built a small version in processing (java applet) that do not scale
 to huge number, but it's fun :
 http://ker.endofinternet.net/img/collatz/

 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Mattias Persson, [matt...@neotechnology.com]
 Neo Technology, www.neotechnology.com
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-18 Thread Laurent Laborde
i quickly hacked an app to populate the neo4j database with the
collatz conjecture graph.
here it is http://github.com/ker2x/Collatz4Neo/blob/master/collatz4neo.java
and the code is ugly, but it just works(c)(r)(tm)

now the problem is to find a way to graph millions of nodes. :/
something like this could be nice :
http://www.race.u-tokyo.ac.jp/~uchida/blogdata/dataset1_community_normal.png

*hugs*

-- 
Ker2x

On Thu, Mar 18, 2010 at 3:20 PM, Peter Neubauer
neubauer.pe...@gmail.com wrote:
 Still,
 very cool, and processing rocks!

 Cheers,

 /peter neubauer

 COO and Sales, Neo Technology

 GTalk:      neubauer.peter
 Skype       peter.neubauer
 Phone       +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter      http://twitter.com/peterneubauer

 http://www.neo4j.org             - Your high performance graph database.
 http://www.tinkerpop.com      - Processing for Internet-scale graphs.
 http://www.thoughtmade.com - Scandinavias coolest Bring-a-Thing party.



 On Tue, Mar 16, 2010 at 12:21 AM, Laurent Laborde kerdez...@gmail.com wrote:
 i totally forgot to say that it's jsut a proof of concept, this small
 app don't use neo4j

 --
 Ker2x

 On Tue, Mar 16, 2010 at 12:19 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 i used processing : http://www.processing.org/
 + the traer physic lib : http://www.cs.princeton.edu/~traer/physics/

 --
 Ker2x


 On Tue, Mar 16, 2010 at 12:14 AM, Mattias Persson
 matt...@neotechnology.com wrote:
 Cool :) What library do you use to visualize the graph?

 2010/3/16 Laurent Laborde kerdez...@gmail.com:
 On Mon, Mar 15, 2010 at 12:13 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 thanks to xkcd, it just made me notice that exploring the collatz
 conjecture is just exploring a huge graph.
 http://xkcd.com/710/

 If i have some spare-spare-spare-spare time, i'll try that. (and draw 
 the graph)

 i built a small version in processing (java applet) that do not scale
 to huge number, but it's fun :
 http://ker.endofinternet.net/img/collatz/

 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Mattias Persson, [matt...@neotechnology.com]
 Neo Technology, www.neotechnology.com
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-15 Thread Mattias Persson
Cool :) What library do you use to visualize the graph?

2010/3/16 Laurent Laborde kerdez...@gmail.com:
 On Mon, Mar 15, 2010 at 12:13 AM, Laurent Laborde kerdez...@gmail.com wrote:
 thanks to xkcd, it just made me notice that exploring the collatz
 conjecture is just exploring a huge graph.
 http://xkcd.com/710/

 If i have some spare-spare-spare-spare time, i'll try that. (and draw the 
 graph)

 i built a small version in processing (java applet) that do not scale
 to huge number, but it's fun :
 http://ker.endofinternet.net/img/collatz/

 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Mattias Persson, [matt...@neotechnology.com]
Neo Technology, www.neotechnology.com
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-15 Thread Laurent Laborde
i used processing : http://www.processing.org/
+ the traer physic lib : http://www.cs.princeton.edu/~traer/physics/

-- 
Ker2x


On Tue, Mar 16, 2010 at 12:14 AM, Mattias Persson
matt...@neotechnology.com wrote:
 Cool :) What library do you use to visualize the graph?

 2010/3/16 Laurent Laborde kerdez...@gmail.com:
 On Mon, Mar 15, 2010 at 12:13 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 thanks to xkcd, it just made me notice that exploring the collatz
 conjecture is just exploring a huge graph.
 http://xkcd.com/710/

 If i have some spare-spare-spare-spare time, i'll try that. (and draw the 
 graph)

 i built a small version in processing (java applet) that do not scale
 to huge number, but it's fun :
 http://ker.endofinternet.net/img/collatz/

 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Mattias Persson, [matt...@neotechnology.com]
 Neo Technology, www.neotechnology.com
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


Re: [Neo] Something fun to try in neo4j : collatz conjecture

2010-03-15 Thread Laurent Laborde
i totally forgot to say that it's jsut a proof of concept, this small
app don't use neo4j

-- 
Ker2x

On Tue, Mar 16, 2010 at 12:19 AM, Laurent Laborde kerdez...@gmail.com wrote:
 i used processing : http://www.processing.org/
 + the traer physic lib : http://www.cs.princeton.edu/~traer/physics/

 --
 Ker2x


 On Tue, Mar 16, 2010 at 12:14 AM, Mattias Persson
 matt...@neotechnology.com wrote:
 Cool :) What library do you use to visualize the graph?

 2010/3/16 Laurent Laborde kerdez...@gmail.com:
 On Mon, Mar 15, 2010 at 12:13 AM, Laurent Laborde kerdez...@gmail.com 
 wrote:
 thanks to xkcd, it just made me notice that exploring the collatz
 conjecture is just exploring a huge graph.
 http://xkcd.com/710/

 If i have some spare-spare-spare-spare time, i'll try that. (and draw the 
 graph)

 i built a small version in processing (java applet) that do not scale
 to huge number, but it's fun :
 http://ker.endofinternet.net/img/collatz/

 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Mattias Persson, [matt...@neotechnology.com]
 Neo Technology, www.neotechnology.com
 ___
 Neo mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




 --
 Laurent ker2x Laborde
 Sysadmin  DBA at http://www.over-blog.com/




-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user


[Neo] Something fun to try in neo4j : collatz conjecture

2010-03-14 Thread Laurent Laborde
thanks to xkcd, it just made me notice that exploring the collatz
conjecture is just exploring a huge graph.
http://xkcd.com/710/

If i have some spare-spare-spare-spare time, i'll try that. (and draw the graph)

-- 
Laurent ker2x Laborde
Sysadmin  DBA at http://www.over-blog.com/
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user