Re: [Neo4j] Configuring Neo4J via Python bindings

2011-10-18 Thread Jacob Hansson 
It looks like our build pipeline is clogged up right now. We're looking into
it, but it might take a while..

/jake

On Mon, Oct 17, 2011 at 9:56 PM, Chris Diehl di...@alumni.cmu.edu wrote:

 Hey Jake,

 Awesome! Thanks so much for making this happen! On my end, it doesn't look
 like the docs are up yet. I'll check back...

 Thanks again, Chris

 On Mon, Oct 17, 2011 at 4:12 AM, Jacob Hansson 
 jacob.hans...@neotechnology.com wrote:

  Hey Chris,
 
  I've just added support for passing in extra JVM args, it's available as
 of
  this commit:
 
 
 https://github.com/neo4j/python-embedded/commit/d41950f8c7e6c65d67f60868976e1864a496a2ea
 
  Documentation will pop up in an hour or so here:
 
 http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html
 
  This feature will be available as of the 1.5 release, due later this
 week.
  If you want to get access to it right away, you can build neo4j-embedded
  directly from source, the code is available here:
  https://github.com/neo4j/python-embedded
 
  See the readme for how to build from source.
 
 
  Thanks for bringing this up!
  Jake
 
  On Fri, Oct 14, 2011 at 7:47 PM, Chris Diehl di...@alumni.cmu.edu
 wrote:
 
   Hi Michael,
  
   I added a print statement to my code to output how many nodes have been
   inserted. The script zips along until it gets up around 18.6K nodes.
 Then
   the program halts when a Java exception is thrown, as shown below.
  
   Inserting user 18629
   Inserting user 18630
   Inserting user 18631
   Inserting user 18632
   Inserting user 18633
   Inserting user 18634
   Inserting user 18635
   Traceback (most recent call last):
File jive_app_db_graph_construction.py, line 53, in module
  node_idx['userid'][user_node['userid']] = user_node
File
  
  
 
 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/contextlib.py,
   line 34, in __exit__
  self.gen.throw(type, value, traceback)
File string, line 11, in transaction
   RuntimeError: An unknown error occured while handling a Java Exception
  
   Strangely enough, the script returned a more explicit Out of Heap
 Memory
   error once when I reran the script. It's bizarre that the failure
 output
  is
   not consistent, given I start with a clean DB each time. Increasing the
   amount of available heap memory seems like the reasonable place to
 start.
   If
   you can let me know how to specify the amount of heap memory for the
 JVM
   instantiated by the bindings, that'd be great.
  
   I'm running on a MacBook Pro with 8 gigs of memory running OSX 10.6.8.
  
   Here's the contents of the messages.log file.
  
   2011-10-14 10:34:17.336-0700: Creating new db @
  /Users/chris.diehl/Graphs/
   JiveCommunityV5.0.ng/neostore
   2011-10-14 10:34:17.419-0700: Opened [/Users/chris.diehl/Graphs/
   JiveCommunityV5.0.ng/nioneo_logical.log.1] clean empty log, version=0,
   lastTxId=1
   2011-10-14 10:34:17.441-0700: Opened [/Users/chris.diehl/Graphs/
   JiveCommunityV5.0.ng/index/lucene.log.1] clean empty log, version=0,
   lastTxId=1
   2011-10-14 10:34:17.443-0700: TM new log: tm_tx_log.1
   2011-10-14 10:34:17.451-0700: --- CONFIGURATION START ---
   2011-10-14 10:34:17.451-0700: Physical mem: 8192MB, Heap size: 123MB
   2011-10-14 10:34:17.452-0700: Kernel version: Neo4j - Graph Database
  Kernel
   1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8)
   2011-10-14 10:34:17.453-0700: Neo4j - Graph Database Kernel
 1.5-SNAPSHOT
   (revision: 1.5.M02-4-g3d1e9d8)
   2011-10-14 10:34:17.456-0700: Operating System: Mac OS X; version:
  10.6.8;
   arch: x86_64; cpus: 8
   2011-10-14 10:34:17.456-0700: VM Name: Java HotSpot(TM) 64-Bit Server
 VM
   2011-10-14 10:34:17.456-0700: VM Vendor: Apple Inc.
   2011-10-14 10:34:17.456-0700: VM Version: 20.1-b02-384
   2011-10-14 10:34:17.456-0700: Boot Class Path:
  
  
 
 /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsfd.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/ui.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/laf.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/sunrsasign.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsse.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jce.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/charsets.jar
   2011-10-14 10:34:17.457-0700: Class Path:

Re: [Neo4j] Configuring Neo4J via Python bindings

2011-10-17 Thread Chris Diehl 
Hey Jake,

Awesome! Thanks so much for making this happen! On my end, it doesn't look
like the docs are up yet. I'll check back...

Thanks again, Chris

On Mon, Oct 17, 2011 at 4:12 AM, Jacob Hansson 
jacob.hans...@neotechnology.com wrote:

 Hey Chris,

 I've just added support for passing in extra JVM args, it's available as of
 this commit:

 https://github.com/neo4j/python-embedded/commit/d41950f8c7e6c65d67f60868976e1864a496a2ea

 Documentation will pop up in an hour or so here:
 http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html

 This feature will be available as of the 1.5 release, due later this week.
 If you want to get access to it right away, you can build neo4j-embedded
 directly from source, the code is available here:
 https://github.com/neo4j/python-embedded

 See the readme for how to build from source.


 Thanks for bringing this up!
 Jake

 On Fri, Oct 14, 2011 at 7:47 PM, Chris Diehl di...@alumni.cmu.edu wrote:

  Hi Michael,
 
  I added a print statement to my code to output how many nodes have been
  inserted. The script zips along until it gets up around 18.6K nodes. Then
  the program halts when a Java exception is thrown, as shown below.
 
  Inserting user 18629
  Inserting user 18630
  Inserting user 18631
  Inserting user 18632
  Inserting user 18633
  Inserting user 18634
  Inserting user 18635
  Traceback (most recent call last):
   File jive_app_db_graph_construction.py, line 53, in module
 node_idx['userid'][user_node['userid']] = user_node
   File
 
 
 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/contextlib.py,
  line 34, in __exit__
 self.gen.throw(type, value, traceback)
   File string, line 11, in transaction
  RuntimeError: An unknown error occured while handling a Java Exception
 
  Strangely enough, the script returned a more explicit Out of Heap Memory
  error once when I reran the script. It's bizarre that the failure output
 is
  not consistent, given I start with a clean DB each time. Increasing the
  amount of available heap memory seems like the reasonable place to start.
  If
  you can let me know how to specify the amount of heap memory for the JVM
  instantiated by the bindings, that'd be great.
 
  I'm running on a MacBook Pro with 8 gigs of memory running OSX 10.6.8.
 
  Here's the contents of the messages.log file.
 
  2011-10-14 10:34:17.336-0700: Creating new db @
 /Users/chris.diehl/Graphs/
  JiveCommunityV5.0.ng/neostore
  2011-10-14 10:34:17.419-0700: Opened [/Users/chris.diehl/Graphs/
  JiveCommunityV5.0.ng/nioneo_logical.log.1] clean empty log, version=0,
  lastTxId=1
  2011-10-14 10:34:17.441-0700: Opened [/Users/chris.diehl/Graphs/
  JiveCommunityV5.0.ng/index/lucene.log.1] clean empty log, version=0,
  lastTxId=1
  2011-10-14 10:34:17.443-0700: TM new log: tm_tx_log.1
  2011-10-14 10:34:17.451-0700: --- CONFIGURATION START ---
  2011-10-14 10:34:17.451-0700: Physical mem: 8192MB, Heap size: 123MB
  2011-10-14 10:34:17.452-0700: Kernel version: Neo4j - Graph Database
 Kernel
  1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8)
  2011-10-14 10:34:17.453-0700: Neo4j - Graph Database Kernel 1.5-SNAPSHOT
  (revision: 1.5.M02-4-g3d1e9d8)
  2011-10-14 10:34:17.456-0700: Operating System: Mac OS X; version:
 10.6.8;
  arch: x86_64; cpus: 8
  2011-10-14 10:34:17.456-0700: VM Name: Java HotSpot(TM) 64-Bit Server VM
  2011-10-14 10:34:17.456-0700: VM Vendor: Apple Inc.
  2011-10-14 10:34:17.456-0700: VM Version: 20.1-b02-384
  2011-10-14 10:34:17.456-0700: Boot Class Path:
 
 
 /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsfd.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/ui.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/laf.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/sunrsasign.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsse.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jce.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/charsets.jar
  2011-10-14 10:34:17.457-0700: Class Path:

Re: [Neo4j] Configuring Neo4J via Python bindings

2011-10-14 Thread Chris Diehl 
Hi Michael,

I added a print statement to my code to output how many nodes have been
inserted. The script zips along until it gets up around 18.6K nodes. Then
the program halts when a Java exception is thrown, as shown below.

Inserting user 18629
Inserting user 18630
Inserting user 18631
Inserting user 18632
Inserting user 18633
Inserting user 18634
Inserting user 18635
Traceback (most recent call last):
  File jive_app_db_graph_construction.py, line 53, in module
node_idx['userid'][user_node['userid']] = user_node
  File
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/contextlib.py,
line 34, in __exit__
self.gen.throw(type, value, traceback)
  File string, line 11, in transaction
RuntimeError: An unknown error occured while handling a Java Exception

Strangely enough, the script returned a more explicit Out of Heap Memory
error once when I reran the script. It's bizarre that the failure output is
not consistent, given I start with a clean DB each time. Increasing the
amount of available heap memory seems like the reasonable place to start. If
you can let me know how to specify the amount of heap memory for the JVM
instantiated by the bindings, that'd be great.

I'm running on a MacBook Pro with 8 gigs of memory running OSX 10.6.8.

Here's the contents of the messages.log file.

2011-10-14 10:34:17.336-0700: Creating new db @ /Users/chris.diehl/Graphs/
JiveCommunityV5.0.ng/neostore
2011-10-14 10:34:17.419-0700: Opened [/Users/chris.diehl/Graphs/
JiveCommunityV5.0.ng/nioneo_logical.log.1] clean empty log, version=0,
lastTxId=1
2011-10-14 10:34:17.441-0700: Opened [/Users/chris.diehl/Graphs/
JiveCommunityV5.0.ng/index/lucene.log.1] clean empty log, version=0,
lastTxId=1
2011-10-14 10:34:17.443-0700: TM new log: tm_tx_log.1
2011-10-14 10:34:17.451-0700: --- CONFIGURATION START ---
2011-10-14 10:34:17.451-0700: Physical mem: 8192MB, Heap size: 123MB
2011-10-14 10:34:17.452-0700: Kernel version: Neo4j - Graph Database Kernel
1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8)
2011-10-14 10:34:17.453-0700: Neo4j - Graph Database Kernel 1.5-SNAPSHOT
(revision: 1.5.M02-4-g3d1e9d8)
2011-10-14 10:34:17.456-0700: Operating System: Mac OS X; version: 10.6.8;
arch: x86_64; cpus: 8
2011-10-14 10:34:17.456-0700: VM Name: Java HotSpot(TM) 64-Bit Server VM
2011-10-14 10:34:17.456-0700: VM Vendor: Apple Inc.
2011-10-14 10:34:17.456-0700: VM Version: 20.1-b02-384
2011-10-14 10:34:17.456-0700: Boot Class Path:
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsfd.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/ui.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/laf.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/sunrsasign.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsse.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jce.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/charsets.jar
2011-10-14 10:34:17.457-0700: Class Path:
/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/antlr-2.7.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-analysis-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-commons-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-tree-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-util-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/blueprints-core-0.9.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/geronimo-jta_1.1_spec-1.1.1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages

[Neo4j] Configuring Neo4J via Python bindings

2011-10-13 Thread Chris Diehl 
Hey All,

I just tested a Python script I've been working on to construct a
multi-relational graph in Neo4j. I am using the new Python bindings for
embedded Neo4j. After creating about 18K nodes, the database ground to a
halt.

In the past, when using Gremlin, I would up the heap size for the JVM when
encountering problems. I'm not sure how to tweak settings for the JVM now
that I'm using the Python bindings to fire up the database. Can I do that
from the GraphDatabase() call? Or do I need to look elsewhere to tweak the
JVM parameters?

If you have any other suggestions about parameters I should change, I'm all
ears.

Thanks!

Chris

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/Configuring-Neo4J-via-Python-bindings-tp3419961p3419961.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Configuring Neo4J via Python bindings

2011-10-13 Thread Michael Hunger 
Hi Chris,

What do you mean by ground to a halt ? Is it using all available CPU or heavy 
IO-waits (top, iostat, vmstat)?

A thread dump (via jps + jstack ) would be great.

You can probably pass in the config map into the  Graph database call.
Current config is logged to messages.log, please share that too as well as your 
system parameters (OS, CPU, RAM, Type of disk).

Michael

Am 13.10.2011 um 23:52 schrieb Chris Diehl:

 Hey All,
 
 I just tested a Python script I've been working on to construct a
 multi-relational graph in Neo4j. I am using the new Python bindings for
 embedded Neo4j. After creating about 18K nodes, the database ground to a
 halt.
 
 In the past, when using Gremlin, I would up the heap size for the JVM when
 encountering problems. I'm not sure how to tweak settings for the JVM now
 that I'm using the Python bindings to fire up the database. Can I do that
 from the GraphDatabase() call? Or do I need to look elsewhere to tweak the
 JVM parameters?
 
 If you have any other suggestions about parameters I should change, I'm all
 ears.
 
 Thanks!
 
 Chris
 
 --
 View this message in context: 
 http://neo4j-community-discussions.438527.n3.nabble.com/Configuring-Neo4J-via-Python-bindings-tp3419961p3419961.html
 Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user