date:20110717

I like Neo4j and am interested in using it in a number of projects (once I
figure out a good way to make Neo play nice in Glassfish) ... but I am
confused about something.   I apologize if this is a stupid question.

My understanding is that Lucene does *not* support transactions.  Supposedly
that lack of transaction support was a prime motivator for the Compass
project, and is a selling point for the Lucene competitor, Sphinx.

So, how can Neo4j provide real transaction support if it relies on Lucene? 
I am confused.

Thanks

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/How-can-you-have-transactions-with-Lucene-tp3176918p3176918.html
Sent from the Neo4J Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] Officially supported mechanism for J2EE

I really like Neo, but am concerned that there is so little official company
support for J2EE.  I want to use Neo as a standard data source in J2EE, with
container managed transactions, etc.

I don't want to use the REST interface - I want a standard data source
interface.  There does not appear to be an officially provided mechanism to
use Neo as a data source in J2EE.  I've seen a couple of people post their
code to help someone get started, but shouldn't Neo provide a real official
sanctioned mechanism?

P.S.  I don't want this thread to turn into a religious war over REST.  If
you like REST and want to defend it then please start a thread for that
purpose.  Please just accept that I don't want to use the REST interface - I
want a standard DB interface for my DB.

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/Officially-supported-mechanism-for-J2EE-tp3176938p3176938.html
Sent from the Neo4J Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] How can you have transactions with Lucene?

Neo4j wraps Lucene in a thin layer which makes it transactional. It's fully
ACID and can participate in the same transactions as neo4j operations. It's
modeled as its own data source in an XA environment (the neo4j graph is
another XA data source) with a logical log for commands so that it's safe
for crashes and will participate in the recovery process just like any other
XA data source, f.ex. the neo4j graph.

2011/7/17 eialbur ronalb...@gmail.com

I like Neo4j and am interested in using it in a number of projects (once I
figure out a good way to make Neo play nice in Glassfish) ... but I am
confused about something. I apologize if this is a stupid question.

My understanding is that Lucene does *not* support transactions.
Supposedly
that lack of transaction support was a prime motivator for the Compass
project, and is a selling point for the Lucene competitor, Sphinx.

So, how can Neo4j provide real transaction support if it relies on Lucene?
I am confused.

Thanks

--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/How-can-you-have-transactions-with-Lucene-tp3176918p3176918.html
Sent from the Neo4J Community Discussions mailing list archive at
Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Index returns all nodes for value=undefined!

No, there's no such undefined constant which will return everything. It
definitely sounds like a JavaScript issue, and that it turns nulls into
undefined. And perhaps your code which tries to index null values.

2011/7/16 Aseem Kishore aseem.kish...@gmail.com

 We're seeing this crazy behavior where our exact index is returning all
 nodes if we pass a lookup value of undefined! E.g.:

 http://localhost:7474/db/data/index/node/nodes/username/undefined

 Is this a bug in Neo4j / REST API? Or is this a symptom of some bug we have
 in our own code?

 It could very well be our code, since we're using JS, which turns undefined
 values into the string undefined if you try to add it to a string
 (definitely a wtfjs candidate), but I wanted to see if others are seeing
 this in Neo4j too!

 Thanks,

 Aseem
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] How can you have transactions with Lucene?

Sorry, but how does this thin layer control how Lucene interacts with the
disk (since Lucene isn't transactional).  Does it somehow buffer the disk
I/O?  Does it coerce Lucene to write to a different file and then update the
real file at the end of the transaction?

I'm not trying to be difficult, but I am interested in Lucene for my project
as well, and need it to behave properly regarding transactions.

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/How-can-you-have-transactions-with-Lucene-tp3176918p3176951.html
Sent from the Neo4J Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] NoSuchElementException

get/query methods are only guaranteed to return stuff added before the
latest call to flush() (it's all there in the javadocs). I think that's why
you probably get stale results from get/query. flush(), or at least the
first get/query after a flush() is expensive so try to minimize the number
of calls to flush(). Do this by having different steps in your batch
insertion, f.ex. one where you insert all nodes and index them... call
flush() and then insert relationships.

2011/7/15 sulabh choudhury sula...@gmail.com

 Well while inserting the nodes I keep a
 check batchInserter.nodeExists(node1) so I would guess a node would not be
 duplicated.
 Otherwise during a traversal I guess duplicate nodes. IIs there a way I can
 look into the BtachInsert data before I flush so that I can make sure no
 duplicate data has been inserted ?

 On Sat, Jun 25, 2011 at 10:49 AM, sulabh choudhury sula...@gmail.comwrote:

 Thank you Jim.
 I will wait for 1.5 and hope it resolves the issue :)


 On Fri, Jun 24, 2011 at 7:11 PM, sulabh choudhury sula...@gmail.comwrote:

 Hi,

 I just downloaded the neo4j-community-1.4.M04. I stumbled into the
 java.util.NoSuchElementException: More than one element in
 org.neo4j.index.impl.lucene.LuceneIndex$1@396cbd97. First element is
 'Node[3]' and the second element is 'Node[2]'
  at
 org.neo4j.helpers.collection.IteratorUtil.singleOrNull(IteratorUtil.java:114)
 ~[working_graphGen.jar:na]
 at
 org.neo4j.index.impl.lucene.IdToEntityIterator.getSingle(IdToEntityIterator.java:88)
 ~[working_graphGen.jar:na]
  at
 org.neo4j.index.impl.lucene.IdToEntityIterator.getSingle(IdToEntityIterator.java:32)
 ~[working_graphGen.jar:na]

 I looked up and found that this is a bug and has been fixed. I was
 wondering if the fix has been incorporated in the latest Milestone or not?




 --

 --
 Thanks and Regards,
 Sulabh Choudhury




 --

 --

 Thanks and Regards,
 Sulabh Choudhury





-- 
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] How can you have transactions with Lucene?

It just waits to write stuff until the transaction committs. Application
crashes in between are guarded by the logical log, because all operations
are written in the prepare-phase to a logical log (before the commit
starts). Neo4j doesn't do any magical tweaking of lucene at all because it
isn't necessary... it's fully ACID and transactional with the graph as it
is.

2011/7/17 eialbur ronalb...@gmail.com

 Sorry, but how does this thin layer control how Lucene interacts with the
 disk (since Lucene isn't transactional).  Does it somehow buffer the disk
 I/O?  Does it coerce Lucene to write to a different file and then update
 the
 real file at the end of the transaction?

 I'm not trying to be difficult, but I am interested in Lucene for my
 project
 as well, and need it to behave properly regarding transactions.

 --
 View this message in context:
 http://neo4j-community-discussions.438527.n3.nabble.com/How-can-you-have-transactions-with-Lucene-tp3176918p3176951.html
 Sent from the Neo4J Community Discussions mailing list archive at
 Nabble.com.
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] How can you have transactions with Lucene?

Lucene itself says it doesn't properly support ACID or XA transaction
support.
(Bug 1149 - Major - New Feature - Reopened - Unresolved).

--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/How-can-you-have-transactions-with-Lucene-tp3176918p3176981.html
Sent from the Neo4J Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] OSGi and Neo4j - superbundle approach

Hi Nicolas,
Have you been able to run
https://github.com/njouanin/neo4j-osgi/blob/master/examples/src/test/java/org/neo4j/examples/osgi/OSGiTest.java
properly? It actually is running on Equinox, so let me try it with
Felix ... back in a moment.

Also, in order to pull your changes, could you sign up as a committer,
and send a mail according to
http://wiki.neo4j.org/content/About_Contributor_License_Agreement ?


Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Sun, Jul 17, 2011 at 3:02 PM, Nicolas Jouanin
nicolas.joua...@gmail.com wrote:
 Hi Peter,

 Thanks for initiating this work on OSGi bundle.
 I've forked you repository and tried to make is work using this sample 
 activator [1]. My goal is to make my activator work when deployed on Felix 
 with the neo4j superbundle.
 First of all, I had troubles to mvn clean install the bundle module. 
 maven-bundle-plugin is complaining about bundle.namespace property not set. 
 I've fixed it.
 Then when deploying it on Felix I had an error saying that javax.transaction 
 couldn't be imported. Because this package is already included in 
 superbundle, I've removed it explicitally from the Import-Package directive. 
 You can see the changes here [2].
 Then I tried to make my bundle activated using felix and the following 
 bundles installed :
 g! lb
 START LEVEL 1
   ID|State      |Level|Name
    0|Active     |    0|System Bundle (3.2.2)
    1|Active     |    1|Neo4j OSGi default bundle (0.1.0.SNAPSHOT)
    2|Active     |    1|Apache Felix Bundle Repository (1.6.2)
    3|Active     |    1|Apache Felix Gogo Command (0.8.0)
    4|Active     |    1|Apache Felix Gogo Runtime (0.8.0)
    5|Active     |    1|Apache Felix Gogo Shell (0.8.0)
    6|Resolved   |    1|EscapeK - Neo4j OSGi how-to - Embedded client 
 (0.0.1.SNAPSHOT)
 There are only felix core, neo4j superbundle and my test bundle.
 So when starting felix I get the following trace :
 
 Welcome to Apache Felix Gogo

 g! Opening database in embedded mode: 17 juil. 2011 14:57:08 
 org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog doInternalRecovery
 INFO: Non clean shutdown detected on log [graphdb/nioneo_logical.log.1]. 
 Recovery started ...
 OK
 Populating it ... OK
 Hello, brave Neo4j world!
 ERROR: Bundle org.escapek.neo4j-osgi-howto.embedded-client [6] Error starting 
 file:/Users/nico/Dev/felix-framework-3.2.2/bundle/org.escapek.neo4j-osgi-howto.embedded-client_0.0.1-SNAPSHOT.jar
  (org.osgi.framework.BundleException: Activator start error in bundle 
 org.escapek.neo4j-osgi-howto.embedded-client [6].)
 java.lang.IllegalArgumentException: No index provider 'lucene' found
        at 
 org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:76)
        at 
 org.neo4j.kernel.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java:116)
        at 
 org.neo4j.kernel.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerImpl.java:178)
        at 
 org.neo4j.kernel.IndexManagerImpl.getOrCreateNodeIndex(IndexManagerImpl.java:267)
        at 
 org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:255)
        at 
 org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:249)
        at 
 org.escapek.neo4jOsgiHowto.embeddedClient.EmbeddedClientActivator.start(EmbeddedClientActivator.java:65)
        at 
 org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:629)
        at org.apache.felix.framework.Felix.activateBundle(Felix.java:1904)
        at org.apache.felix.framework.Felix.startBundle(Felix.java:1822)
        at 
 org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1192)
        at 
 org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:266)
        at java.lang.Thread.run(Thread.java:680)

 So everything works fine (open database, populate it and register service) 
 until the activator tries to register IndexProvider service. I then get the 
 previous exception which reminds me of something ...

 May be you can have a look at the stack trace to look for the error cause.

 NJ


 [1] - 
 https://github.com/njouanin/neo4j-osgi-howto/blob/master/embedded-client/src/main/java/org/escapek/neo4jOsgiHowto/embeddedClient/EmbeddedClientActivator.java
 [2] - 
 https://github.com/njouanin/neo4j-osgi/commit/4ec3eaf58b514d806fde3e48dd942c00fdfddf70

 Le 5 juil. 2011 à 22:41, Peter Neubauer a écrit :

 Hi all,
 I just pushed an initial working bundle-build to
 https://github.com/neo4j/neo4j-osgi. You can see the working test in
 https://github.com/neo4j/neo4j-osgi/blob/master/examples/src/test/java/org/neo4j/examples/osgi/OSGiTest.java
 that

Re: [Neo4j] OSGi and Neo4j - superbundle approach

Nicolas,
just pushed some more initialization along your lines, and have the
test running with the output:

[~/code/neo/neo4j-osgi/examples] $mvn clean test
[INFO] Scanning for projects...
[INFO] 
[INFO] Building Neo4j OSGi examples
[INFO]task-segment: [clean, test]
[INFO] 
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting /Users/peterneubauer/code/neo/neo4j-osgi/examples/target
[WARNING] POM for 'biz.aQute:bndlib:pom:1.43.0:test' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[WARNING] POM for 'biz.aQute:bndlib:pom:1.15.0:test' is invalid.

Its dependencies (if any) will NOT be available to the current build.
[INFO] [enforcer:enforce {execution: enforce-maven}]
[INFO] [license:check {execution: check-licenses}]
[INFO] Checking licenses...
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/peterneubauer/code/neo/neo4j-osgi/examples/src/main/resources
[INFO] skip non existing resourceDirectory
/Users/peterneubauer/code/neo/neo4j-osgi/examples/src/main/resources/META-INF
[INFO] Copying 0 resource to META-INF
[INFO] [compiler:compile {execution: default-compile}]
[INFO] No sources to compile
[INFO] [bundle:manifest {execution: bundle-manifest}]
[WARNING] Warning in manifest for
org.neo4j.examples:neo4j-osgi-examples:jar:0.1-SNAPSHOT : Superfluous
export-package instructions: [null.*]
[WARNING] Warning in manifest for
org.neo4j.examples:neo4j-osgi-examples:jar:0.1-SNAPSHOT : Did not find
matching referal for *
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/peterneubauer/code/neo/neo4j-osgi/examples/src/test/resources
[INFO] Copying 0 resource to META-INF
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Compiling 3 source files to
/Users/peterneubauer/code/neo/neo4j-osgi/examples/target/test-classes
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory:
/Users/peterneubauer/code/neo/neo4j-osgi/examples/target/surefire-reports

---
 T E S T S
---
Running org.neo4j.examples.osgi.OSGiTest
300 [main] INFO org.ops4j.pax.exam.spi.DefaultExamSystem - Pax Exam
System (Version: 2.1.0) created.
Jul 17, 2011 5:52:07 PM
org.ops4j.pax.url.mvn.internal.AetherBasedResolver resolve
INFO: Resolved (org.ops4j.pax.exam:pax-exam-container-rbc:jar:2.1.0)
as 
/Users/peterneubauer/.m2/repository/org/ops4j/pax/exam/pax-exam-container-rbc/2.1.0/pax-exam-container-rbc-2.1.0.jar
Jul 17, 2011 5:52:07 PM
org.ops4j.pax.url.mvn.internal.AetherBasedResolver resolve
INFO: Resolved (org.ops4j.pax.exam:pax-exam-extender-service:jar:2.1.0)
as 
/Users/peterneubauer/.m2/repository/org/ops4j/pax/exam/pax-exam-extender-service/2.1.0/pax-exam-extender-service-2.1.0.jar
Jul 17, 2011 5:52:07 PM
org.ops4j.pax.url.mvn.internal.AetherBasedResolver resolve
INFO: Resolved (org.osgi:org.osgi.compendium:jar:4.2.0) as
/Users/peterneubauer/.m2/repository/org/osgi/org.osgi.compendium/4.2.0/org.osgi.compendium-4.2.0.jar
Jul 17, 2011 5:52:07 PM
org.ops4j.pax.url.mvn.internal.AetherBasedResolver resolve
INFO: Resolved (org.ops4j.pax.logging:pax-logging-api:jar:1.6.2) as
/Users/peterneubauer/.m2/repository/org/ops4j/pax/logging/pax-logging-api/1.6.2/pax-logging-api-1.6.2.jar
Jul 17, 2011 5:52:07 PM
org.ops4j.pax.url.mvn.internal.AetherBasedResolver resolve
INFO: Resolved (org.ops4j.pax.logging:pax-logging-service:jar:1.6.2)
as 
/Users/peterneubauer/.m2/repository/org/ops4j/pax/logging/pax-logging-service/1.6.2/pax-logging-service-1.6.2.jar
Jul 17, 2011 5:52:07 PM
org.ops4j.pax.url.mvn.internal.AetherBasedResolver resolve
INFO: Resolved (org.neo4j:neo4j-osgi-bundle:jar:0.1-SNAPSHOT) as
/Users/peterneubauer/.m2/repository/org/neo4j/neo4j-osgi-bundle/0.1-SNAPSHOT/neo4j-osgi-bundle-0.1-SNAPSHOT.jar
[org.ops4j.pax.exam.rbc.internal.Activator] : Name, port or host is
null. So this RBC remains inactive.
org.ops4j.pax.logging.pax-logging-api[org.ops4j.pax.logging.internal.Activator]
: Enabling SLF4J API support.
org.ops4j.pax.logging.pax-logging-api[org.ops4j.pax.logging.internal.Activator]
: Enabling Jakarta Commons Logging API support.
org.ops4j.pax.logging.pax-logging-api[org.ops4j.pax.logging.internal.Activator]
: Enabling Log4J API support.
org.ops4j.pax.logging.pax-logging-api[org.ops4j.pax.logging.internal.Activator]
: Enabling Avalon Logger API support.
org.ops4j.pax.logging.pax-logging-api[org.ops4j.pax.logging.internal.Activator]
: Enabling JULI Logger API support.
[Framework Event Dispatcher] DEBUG
org.ops4j.pax.logging.pax-logging-service - BundleEvent STARTED
[Framework Event

[Neo4j] How to create a graph database out of a huge dataset?

2011-07-17 Thread st3ven

Hi all,

I'm new to neo4j and graph databases.
To create my graph database I got two questions for you:

1.
I want to create a graph database out of a huge CSV file.
The problem is, that i need to index the nodes I have already created, so
that I don't create duplicate nodes.

My CSV file looks like this:

Article A, Timestamp, Author A
Article A, Timestamp, Author B
Article A, Timestamp, Author C
Article B, Timestamp, Author A
Article B, Timestamp, Author B
Article B, Timestamp, Author D

As you can see I need to access nodes I have already created and connect
them to the next Article.
Right now I'm using the LuceneIndex, but with around 6M authors this is
getting really slow.
Is there any other possibility to access nodes that I have already created?
BatchInserter also doesn't work, because there you can't access nodes which
you have created before.

2.
Is it possible to use neo4j in a distributed system?
If it is possible, are there any guides or tutorials how to realize that?

Thanks for your help,
Stephan


--
View this message in context: 
http://neo4j-community-discussions.438527.n3.nabble.com/How-to-create-a-graph-database-out-of-a-huge-dataset-tp3177076p3177076.html
Sent from the Neo4J Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Django + Neo4j seamless installation error: No module named CallbackFilter

2011-07-17 Thread Tobias Ivarsson

On Fri, Jul 15, 2011 at 2:33 AM, Richard Schwab endur...@gmail.com wrote:

 Hello,
 I'm having difficulty integrating Django and Neo4j. I've tried it on
 several
 different systems including Mac OSX 10.6.8 and Ubuntu 11.04. Perhaps
 someone
 can give me some feedback or suggestions? I'm good with Django but haven't
 played with VirtualEnv, Jython or Neo4j before.

 Here is the error I'm getting trying the steps at thobe.org's seamless
 neo4j
 integration in django:

 == Error ==
 ValueError: Unable to configure filter 'require_debug_false': Cannot
 resolve
 'django.utils.log.CallbackFilter': No module named CallbackFilter

 == Steps ==
 Mainly using
 http://journal.thobe.org/2009/12/seamless-neo4j-integration-in-django.html
 ,
 I'm following these steps and I apologize if these are unnecessary, I'm
 just
 trying to give more info to get to an answer quickly.

   1. VirtualEnv
  1. Install virtualenv ($ sudo apt-get install virtualenv)
  2. Set up a virtualenv ($ virtualenv TestEnv)
  3. Activate the virtualenv ($ source TestEnv/bin/activate)
   2. Jython - 2.5.2 (I've also tried 2.53b and no luck)
  1. Download jython (sourceforge.net)  uncompress
  2. Copy jython to /usr/lib, make symbolic link ($ ln -s
  /usr/lib/jython2.5.2/lib/jython /usr/bin/jython)
   3. Neo4j - 1.4
  1. Download Neo4j ($ svn co
  https://svn.neo4j.org/components/neo4j.py/trunk neo4j-python)
  2. Install Neo4j ($ sudo jython setup.py install)
  3. Tested: jython: import neo4j works
   4. Django - 1.3.0
  1. Download  extract Django (https://www.djangoproject.com/download/
 )
  2. Install Django ($ sudo jython setup.py install)
  3. Tested: jython: import django works
   5. Django-Jython - 1.2.0
  1. Download  extract Django-Jython (
  http://code.google.com/p/django-jython/downloads/list)
  2. Install Django-Jython ($ sudo jython setup.py install)
   6. Created a Django-Admin-Jy ln ($ alias django-admin-jy=jython
   /usr/lib/jython2.5.2/Lib/site-packages/django/bin/django-admin.py)
   7. Created a Django Project ($ django-admin-jy startproject neo4django)
   8. Created a Django App ($ jython neo4django/manage.py startapp business)
   9. Edited settings.py
  1.

  Edit settings.py

  NEO4J_RESOURCE_URI = '/var/neo4j/neo4django'

  # NEO4J_RESOURCE_URI should be the path to where

  #you want to store the Neo4j database.


  NEO4J_OPTIONS = {

  # this is optional and can be used to specify

  # extra startup parameters for Neo4j, such as

  # the classpath to load Neo4j from.

  }
  10. Edited business/models.py
  1.

  from neo4j.model import django_model as model


  class Person(model.NodeModel):

  first_name = model.Property()

  last_name = model.Property()

  def __unicode__(self):

  return u%s %s % (self.first_name, self.last_name)


  class Company(model.NodeModel):

  name = model.Property(indexed=True)

  owners = model.Relationship(Person,

  type=model.Outgoing.OWNED_BY,

  related_name=owns,

  )

  employees = model.Relationship(Person,

  type=model.Incoming.WORKS_AT,

  related_name=employer,

  related_single=True, # Only allow Persons to work at one
  Company

  )

  def __unicode__(self):

  return self.name
  11. Tested it out: $ jython neo4django/manage.py shell results in the
   error (same as written above)
  - ValueError: Unable to configure filter 'require_debug_false': Cannot
  resolve 'django.utils.log.CallbackFilter': No module named
 CallbackFilter

 So all that said if anyone has any ideas, I'd appreciate it immensely. I
 was
 wondering also if my versioning was correct, or if I should drop to some
 older versions of django perhaps? It doesn't seem like there's been much
 work on this since 2009.


That is a very good suspicion. That Django has changed since I wrote that
code, and that article.
Trying an older version of Django might do the trick. Sadly I can't find
which version of Django I used back then.

However, there is no use of CallbackFilter in my code, which seems to imply
that this would be a problem in Django.
Looking at the Django documentation:
https://docs.djangoproject.com/en/dev/topics/logging/#id3 the CallbackFilter
feature seems to not be released yet. How it got into your settings.py (I'm
guessing that is where 'require_debug_false' is configures) I don't know. If
I had not had your complete description of everything you did, I would have
guessed that you either:
a) Followed the documentation of the wrong version of Django from their
website (the dev version is easier to find then the ones for released
versions). Or
b) Used a newer version of Django to auto-generate the settings.py file, and
then an older version to run the code.

But since neither of those steps are in your

[Neo4j] neo4j in a distributed system

2011-07-17 Thread st3ven

Hi all,

I would like to know if it is possible to use neo4j in a distributed system.
If it is possible, are there any guides or tutorials how to realize that?
For my huge database it would be nice to use it in distributed system.

Thanks for your help,
Stephan
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] Fwd: RDF Examples not working - parent-pom



Anfang der weitergeleiteten E-Mail:

 Von: datablend i...@datablend.be
 Datum: 17. Juli 2011 17:40:39 MESZ
 An: Michael Hunger michael.hun...@neotechnology.com
 Betreff: Re: [Neo4j] RDF Examples not working - parent-pom
 
 Hi Michael,
 
 Thanks for the information. Wrote a new article on the use of Tinkerpop for 
 importing and querying RDF data: http://datablend.be/?p=554
 
 Greetings,
 
 Davy
 
 On 14 Jul 2011, at 17:50, Michael Hunger wrote:
 
 Davy,
 
 you used the neo4j-sail project for your examples, tools and demos. 
 
 Unfortunately that one is no longer under active development, so things like 
 the parent pom issue pop up.
 
 Would you mind having a look at the much more complete and up to date sail 
 implementation from tinkerpop to run on top of neo4j?
 
 Thanks so much and great work !
 
 
 Michael
 
 Anfang der weitergeleiteten E-Mail:
 
 Von: Thomas Fritz fritz...@gmail.com
 Datum: 14. Juli 2011 17:33:06 MESZ
 An: Neo4j user discussions user@lists.neo4j.org
 Betreff: Re: [Neo4j] RDF Examples not working - parent-pom
 Antwort an: Neo4j user discussions user@lists.neo4j.org
 
 Hi,
 
 No, it is of course not in my own pom.xml - the projects pom.xml - it
 is in the Neo4J dependencies (rdf, sparql, sail). I think it has to be
 fixed from Neo4J and has to be updated on maven central?
 
 Kind regards
 
 
 
 
 
 ---
 Thomas FRITZ
 web http://fritzthomas.com
 twitter http://twitter.com/thomasf
 
 
 
 2011/7/14 Michael Hunger michael.hun...@neotechnology.com:
 It should be directly in your pom.xml at the top, there is an xml section 
 with the parent element.
 
 Cheers
 
 Michael
 
 If that is not in your local pom, try to pull that project from svn remove 
 that section and build it locally.
 
 
 Am 14.07.2011 um 14:49 schrieb Thomas Fritz:
 
 Thanks for answering!
 How can i remove the parent section from the pom.xml? It is in the
 dependency project maven tries to resolve, right? So how and where can
 i edit it?
 
 Kind regards
 
 
 ---
 Thomas FRITZ
 web http://fritzthomas.com
 twitter http://twitter.com/thomasf
 
 
 
 2011/7/14 Michael Hunger michael.hun...@neotechnology.com:
 You can just try to remove the
 
 parent
 ...
 /parent
 
 block, that should work. Those are just meant for being built integrated 
 in a larger system build.
 
 Cheers
 
 Michael
 
 Am 14.07.2011 um 13:23 schrieb Thomas Fritz:
 
 Hi,
 
 Who can fix this? As i can see one of the neo4j-rdf dependencys pulls
 that dependency in.
 Or can i bypass and fix it locally until it is fixed?
 
 Thanks
 Kind regards
 
 
 
 ---
 Thomas FRITZ
 web http://fritzthomas.com
 twitter http://twitter.com/thomasf
 
 
 
 2011/7/14 Peter Neubauer neubauer.pe...@gmail.com:
 Not sure but parent - pom should maybe be parent - central?
 
 /peter
 
 Sent from my phone.
 On Jul 14, 2011 12:23 PM, Thomas Fritz fritz...@gmail.com wrote:
 I have tried the code example from here: http://datablend.be/?p=411 .
 I downloaded the source from github -
 https://github.com/datablend/neo4j-sail-test . I have also tried this
 example: https://github.com/aduric/Neo-SPARQL-Test
 They both can not be build with maven. The parent-pom can not be
 found by maven. I am using netbeans 7 with bundled maven 3.0.3:
 
 code
 
 Building Unnamed - neo4j-sail-test:neo4j-sail-test:jar:1.0
   task-segment: [clean, install]
 
 [clean:clean]
 Deleting directory
 /home/thomas/Projekte/Neo4J-Tests/neo4j-sail-test/target
 [resources:resources]
 Using platform encoding (UTF-8 actually) to copy filtered resources,
 i.e. build is platform dependent!
 Copying 1 resource
 snapshot org.neo4j:neo4j-rdf-sail:0.6-SNAPSHOT: checking for updates
 from central
 snapshot org.neo4j:neo4j-rdf-sail:0.6-SNAPSHOT: checking for updates
 from aduna-repo
 snapshot org.neo4j:neo4j-kernel:1.1-SNAPSHOT: checking for updates
 from aduna-repo
 snapshot org.neo4j:neo4j-kernel:1.1-SNAPSHOT: checking for updates
 from neo4j-snapshot-repository
 snapshot org.neo4j:neo4j-kernel:1.1-SNAPSHOT: checking for updates
 from sonatype-nexus-snapshots
 snapshot org.neo4j:neo4j-rdf:0.7-SNAPSHOT: checking for updates from
 aduna-repo
 snapshot org.neo4j:neo4j-rdf:0.7-SNAPSHOT: checking for updates from
 sonatype-nexus-snapshots
 snapshot org.neo4j:neo4j-meta-model:0.9-SNAPSHOT: checking for updates
 from aduna-repo
 snapshot org.neo4j:neo4j-meta-model:0.9-SNAPSHOT: checking for updates
 from sonatype-nexus-snapshots
 snapshot org.neo4j:parent-pom:7-SNAPSHOT: checking for updates from
 aduna-repo
 snapshot org.neo4j:parent-pom:7-SNAPSHOT: checking for updates from
 sonatype-nexus-snapshots
 Downloading:
 http://m2.neo4j.org/snapshots/org/neo4j/parent-pom/7-SNAPSHOT/parent-pom-7-20100729.145222-31.pom
 
 BUILD ERROR
 
 Error building POM (may

Re: [Neo4j] Officially supported mechanism for J2EE

Hi,

do you mean a javax.sql.DataSource or an JCA adaptor ?

Could you please explain your architecture and usage scenario where this would
be necessary.

You can use Neo4j in embedded mode inside any J2EE container without issues. If
the container
comes with its own XA-TM then you can integrate with that.

Cheers

Michael

Am 17.07.2011 um 17:03 schrieb eialbur:

I really like Neo, but am concerned that there is so little official company
support for J2EE. I want to use Neo as a standard data source in J2EE, with
container managed transactions, etc.

I don't want to use the REST interface - I want a standard data source
interface. There does not appear to be an officially provided mechanism to
use Neo as a data source in J2EE. I've seen a couple of people post their
code to help someone get started, but shouldn't Neo provide a real official
sanctioned mechanism?

P.S. I don't want this thread to turn into a religious war over REST. If
you like REST and want to defend it then please start a thread for that
purpose. Please just accept that I don't want to use the REST interface - I
want a standard DB interface for my DB.

--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/Officially-supported-mechanism-for-J2EE-tp3176938p3176938.html
Sent from the Neo4J Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Index returns all nodes for value=undefined!

Perhaps it is also an issue of your REST-client? If the value when rendered as 
part of a string evaluates to the 
empty string the resulting URL would be the same as the one that retrieves all 
nodes.

i.e. http://localhost:7474/db/data/index/node/nodes/username

Just an idea, could you verify the REST url that is used?


Am 17.07.2011 um 17:06 schrieb Mattias Persson:

 No, there's no such undefined constant which will return everything. It
 definitely sounds like a JavaScript issue, and that it turns nulls into
 undefined. And perhaps your code which tries to index null values.
 
 2011/7/16 Aseem Kishore aseem.kish...@gmail.com
 
 We're seeing this crazy behavior where our exact index is returning all
 nodes if we pass a lookup value of undefined! E.g.:
 
 http://localhost:7474/db/data/index/node/nodes/username/undefined
 
 Is this a bug in Neo4j / REST API? Or is this a symptom of some bug we have
 in our own code?
 
 It could very well be our code, since we're using JS, which turns undefined
 values into the string undefined if you try to add it to a string
 (definitely a wtfjs candidate), but I wanted to see if others are seeing
 this in Neo4j too!
 
 Thanks,
 
 Aseem
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user
 
 
 
 
 -- 
 Mattias Persson, [matt...@neotechnology.com]
 Hacker, Neo Technology
 www.neotechnology.com
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] How to create a graph database out of a huge dataset?

Stephan,

This is a common thing when inserting data.

You should be able to use lucene in both settings (6M authors is not that much).

Please have a look at your heap memory settings (and in transactional mode also
your memory-map settings for neo4j).

For batch inserter. You can query the index after you called flush on the
index. That should return a node-id which you then can use to create
relationship to that looked up node.

If you could share the code that you've already written we can have a look at
it.

The simplest approach is to just take a HashMap for the authors and their name
as key.
(while indexing them for later use anyway). Just make sure your heap is large
enough to hold the map and the objects created during the insert.

#2 can you describe your distributed system? You can run a cluster of neo4j-HA
instances that each of your distributed engines connects to. (HA instances can
run embedded or as Neo4j-servers depending on your needs). (see here:
http://docs.neo4j.org/chunked/stable/ha.html)

Cheers

Michael

Am 17.07.2011 um 18:31 schrieb st3ven:

Hi all,

I'm new to neo4j and graph databases.
To create my graph database I got two questions for you:

1.
I want to create a graph database out of a huge CSV file.
The problem is, that i need to index the nodes I have already created, so
that I don't create duplicate nodes.

My CSV file looks like this:

Article A, Timestamp, Author A
Article A, Timestamp, Author B
Article A, Timestamp, Author C
Article B, Timestamp, Author A
Article B, Timestamp, Author B
Article B, Timestamp, Author D

As you can see I need to access nodes I have already created and connect
them to the next Article.
Right now I'm using the LuceneIndex, but with around 6M authors this is
getting really slow.
Is there any other possibility to access nodes that I have already created?
BatchInserter also doesn't work, because there you can't access nodes which
you have created before.

2.
Is it possible to use neo4j in a distributed system?
If it is possible, are there any guides or tutorials how to realize that?

Thanks for your help,
Stephan

--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/How-to-create-a-graph-database-out-of-a-huge-dataset-tp3177076p3177076.html
Sent from the Neo4J Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] neo4j in a distributed system

2011-07-17 Thread Aliabbas Petiwala

Great Michael!
can you share with us the performance evaluation results with code of
the 32 billion nodes and relationships. This be a great proof for
neo4js high  performance.



On 7/17/11, Michael Hunger michael.hun...@neotechnology.com wrote:
 with huge database  you mean that you want to shard the graph?

 And how large is your database? Please note that neo4j can handle 32 billion
 nodes and relationships in a single instance.

 If your database is larger than that you have to apply some kind of domain
 level sharding.
 (See Jims article:
 http://jim.webber.name/2011/02/23/abe72f61-27fb-4c1b-8ce1-d0db7583497b.aspx
 and
 http://jim.webber.name/2011/02/16/3b8f4b3d-c884-4fba-ae6b-7b75a191fa22.aspx)



 Am 17.07.2011 um 18:50 schrieb st3...@web.de:

 Hi all,

 I would like to know if it is possible to use neo4j in a distributed
 system.
 If it is possible, are there any guides or tutorials how to realize that?
 For my huge database it would be nice to use it in distributed system.

 Thanks for your help,
 Stephan
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user



-- 
Aliabbas Petiwala
M.Tech CSE
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] RDF data wrapper/triple store options for fast traversal

2011-07-17 Thread Jan Bezget

Hello!
I'm new to Neo4j and need some advice regarding my project and how I should
proceed.
The purpose of the project is to find the shortest path between two things
(similar to RelFinder) as quickly as possible.

So far I've loaded a few million dbpedia triples using NxParser and
neo4j-rdf component as well as tried neo4j's pathfinding capabilities.
Everything works fine!
However, I have noticed that neo4j-rdf's DenseTripleStore stores object
literals and their predicates within the subject node as a set of
properties.

Since, above all, I'm after fast traversals I'm wondering, whether this
incurs any performance penalties to pathfinding algorithms? If I were to
store those literals as separate nodes, would _that_ incur penalties?
Is neo4j-rdf still in development or should I try out TinkerPop's Blueprints
(seems very popular lately!)?
Whilst I like the features these high-level wrappers provide, I'm really
only using pathfinding and indexing. Should I write my own lightweight
wrapper? But in the end is it really worth it, or should I just use existing
wrappers?

Thank you!
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] RDF data wrapper/triple store options for fast traversal

Jan,
I would suggest to use the Tinkerpop RDF wrapper, then examine the
resulting graph in neo4j, and se if raw traversals can find the
relationships for you, After all, you can always drop down to the raw
graph even while using the RDF implementation. That is one of the
points of Neo4j dealing with raw graphs.

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Sun, Jul 17, 2011 at 8:20 PM, Jan Bezget jan.bez...@uni-mb.si wrote:
 Hello!
 I'm new to Neo4j and need some advice regarding my project and how I should
 proceed.
 The purpose of the project is to find the shortest path between two things
 (similar to RelFinder) as quickly as possible.

 So far I've loaded a few million dbpedia triples using NxParser and
 neo4j-rdf component as well as tried neo4j's pathfinding capabilities.
 Everything works fine!
 However, I have noticed that neo4j-rdf's DenseTripleStore stores object
 literals and their predicates within the subject node as a set of
 properties.

 Since, above all, I'm after fast traversals I'm wondering, whether this
 incurs any performance penalties to pathfinding algorithms? If I were to
 store those literals as separate nodes, would _that_ incur penalties?
 Is neo4j-rdf still in development or should I try out TinkerPop's Blueprints
 (seems very popular lately!)?
 Whilst I like the features these high-level wrappers provide, I'm really
 only using pathfinding and indexing. Should I write my own lightweight
 wrapper? But in the end is it really worth it, or should I just use existing
 wrappers?

 Thank you!
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] How to create a graph database out of a huge dataset?

2011-07-17 Thread st3ven

Hi,

thanks for your fast answer.
Right now I'm using lucene for 6M authors, but my whole dataset consists of
nearly 25M authors.
Can i use lucene there also, because I think this getting really slow to
check if a user already exists.
How can I change my heap memory settings and my memory-map settings, cause
I'm using the transactional mode? 
Cause I think with 25M authors I will get a OutOfMemory Exception.

Here is my code that I have already written so far:

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Relationship;
import org.neo4j.graphdb.Transaction;
import org.neo4j.graphdb.index.Index;
import org.neo4j.graphdb.index.IndexHits;
import org.neo4j.graphdb.index.IndexManager;
import org.neo4j.kernel.EmbeddedGraphDatabase;

public class WikiGraphRegUser {

/**
 * @param args
 */
public static void main(String[] args) throws IOException {

BufferedReader bf = new BufferedReader(new FileReader(
E:/wiki0.csv));
WikiGraphRegUser wgru = new WikiGraphRegUser();
wgru.createGraphDatabase(bf);
}

private String articleName = ;
private GraphDatabaseService db;
private IndexManager index;
private IndexNode authorList;
private int transactionCounter = 0;
private Node article;
private boolean isFirstAuthor = false;
private Node author;
private Relationship relationship;
private int node;

private void createGraphDatabase(BufferedReader bf) {
db = new EmbeddedGraphDatabase(target/db);
index = db.index();
authorList = index.forNodes(Author);

String zeile;
Transaction tx = db.beginTx();

try {
// reads lines of CSV-file
while ((zeile = bf.readLine()) != null) {
if (transactionCounter++ % 5 == 0) {

tx.success();
tx.finish();
tx = db.beginTx();
}
// String[] looks like this: Article%;% 
Timestamp%;% Author
String[] artikelinfo = zeile.split(%;% );
if (artikelinfo.length != 3) {
System.out.println(ERROR: check CSV);
for (int i = 0; i  artikelinfo.length; 
i++) {

System.out.println(artikelinfo[i]);
}
return;
}

if (articleName == ) {
// create Article and connect with 
ReferenceNode
article = createArticle(artikelinfo[0],
db.getReferenceNode(), 
MyRelationshipTypes.ARTICLE);
articleName = artikelinfo[0];

isFirstAuthor = true;

} else if (!articleName.equals(artikelinfo[0])) 
{
// create Article and connect with 
ReferenceNode
article = createArticle(artikelinfo[0],
db.getReferenceNode(), 
MyRelationshipTypes.ARTICLE);
articleName = artikelinfo[0];
isFirstAuthor = true;
}
// checks if author already exists
IndexHitsNode hits = authorList.get(Author, 
artikelinfo[2]);
// if new author
if (hits.size() == 0) {
if (isFirstAuthor) {
// creates author and connects 
him with an article
author = 
createAndConnectNode(artikelinfo[2], article,

MyRelationshipTypes.WROTE, artikelinfo[1]);
isFirstAuthor = false;
} else {

author = 
createAndConnectNode(artikelinfo[2], article,

MyRelationshipTypes.EDIT, artikelinfo[1]);
}

}

Re: [Neo4j] OSGi and Neo4j - superbundle approach

2011-07-17 Thread Nicolas Jouanin

I've sent the CLA signup.
The OSGiTest is working fine for me too. Problems come when taking the felix 
framework out of the box and deploying manually neo4j-osgi bundle with another 
bundle containing the same activator class.
I made sure to deploy same bundles but I get 
java.lang.IllegalArgumentException: No index provider 'lucene' found when 
registring the IndexService.
I think i miss a bundle which is created automatically by pax during the test 
case. May be lucene is missing.
What do you think 

Le 17 juil. 2011 à 17:33, Peter Neubauer a écrit :

 Hi Nicolas,
 Have you been able to run
 https://github.com/njouanin/neo4j-osgi/blob/master/examples/src/test/java/org/neo4j/examples/osgi/OSGiTest.java
 properly? It actually is running on Equinox, so let me try it with
 Felix ... back in a moment.
 
 Also, in order to pull your changes, could you sign up as a committer,
 and send a mail according to
 http://wiki.neo4j.org/content/About_Contributor_License_Agreement ?
 
 
 Cheers,
 
 /peter neubauer
 
 GTalk:  neubauer.peter
 Skype   peter.neubauer
 Phone   +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter  http://twitter.com/peterneubauer
 
 http://www.neo4j.org   - Your high performance graph database.
 http://startupbootcamp.org/- Öresund - Innovation happens HERE.
 http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
 
 
 
 On Sun, Jul 17, 2011 at 3:02 PM, Nicolas Jouanin
 nicolas.joua...@gmail.com wrote:
 Hi Peter,
 
 Thanks for initiating this work on OSGi bundle.
 I've forked you repository and tried to make is work using this sample 
 activator [1]. My goal is to make my activator work when deployed on Felix 
 with the neo4j superbundle.
 First of all, I had troubles to mvn clean install the bundle module. 
 maven-bundle-plugin is complaining about bundle.namespace property not set. 
 I've fixed it.
 Then when deploying it on Felix I had an error saying that javax.transaction 
 couldn't be imported. Because this package is already included in 
 superbundle, I've removed it explicitally from the Import-Package directive. 
 You can see the changes here [2].
 Then I tried to make my bundle activated using felix and the following 
 bundles installed :
 g! lb
 START LEVEL 1
   ID|State  |Level|Name
0|Active |0|System Bundle (3.2.2)
1|Active |1|Neo4j OSGi default bundle (0.1.0.SNAPSHOT)
2|Active |1|Apache Felix Bundle Repository (1.6.2)
3|Active |1|Apache Felix Gogo Command (0.8.0)
4|Active |1|Apache Felix Gogo Runtime (0.8.0)
5|Active |1|Apache Felix Gogo Shell (0.8.0)
6|Resolved   |1|EscapeK - Neo4j OSGi how-to - Embedded client 
 (0.0.1.SNAPSHOT)
 There are only felix core, neo4j superbundle and my test bundle.
 So when starting felix I get the following trace :
 
 Welcome to Apache Felix Gogo
 
 g! Opening database in embedded mode: 17 juil. 2011 14:57:08 
 org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog doInternalRecovery
 INFO: Non clean shutdown detected on log [graphdb/nioneo_logical.log.1]. 
 Recovery started ...
 OK
 Populating it ... OK
 Hello, brave Neo4j world!
 ERROR: Bundle org.escapek.neo4j-osgi-howto.embedded-client [6] Error 
 starting 
 file:/Users/nico/Dev/felix-framework-3.2.2/bundle/org.escapek.neo4j-osgi-howto.embedded-client_0.0.1-SNAPSHOT.jar
  (org.osgi.framework.BundleException: Activator start error in bundle 
 org.escapek.neo4j-osgi-howto.embedded-client [6].)
 java.lang.IllegalArgumentException: No index provider 'lucene' found
at 
 org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:76)
at 
 org.neo4j.kernel.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java:116)
at 
 org.neo4j.kernel.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerImpl.java:178)
at 
 org.neo4j.kernel.IndexManagerImpl.getOrCreateNodeIndex(IndexManagerImpl.java:267)
at 
 org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:255)
at 
 org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:249)
at 
 org.escapek.neo4jOsgiHowto.embeddedClient.EmbeddedClientActivator.start(EmbeddedClientActivator.java:65)
at 
 org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:629)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1904)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1822)
at 
 org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1192)
at 
 org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:266)
at java.lang.Thread.run(Thread.java:680)
 
 So everything works fine (open database, populate it and register service) 
 until the activator tries to register IndexProvider service. I then get the 
 previous exception which reminds me of something ...
 
 May be you can have a look at the stack trace to

Re: [Neo4j] How can you have transactions with Lucene?

Exactly, that's what the neo4j integration layer adds to it.

2011/7/17 eialbur ronalb...@gmail.com

 Lucene itself says it doesn't properly support ACID or XA transaction
 support.
 (Bug 1149 - Major - New Feature - Reopened - Unresolved).

 --
 View this message in context:
 http://neo4j-community-discussions.438527.n3.nabble.com/How-can-you-have-transactions-with-Lucene-tp3176918p3176981.html
 Sent from the Neo4J Community Discussions mailing list archive at
 Nabble.com.
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] use one index per key or one index for all keys?

Hi Boris,

If you store them both in one index you get the benefit of asking queries
including both those keys... however the index will be bigger and
performance may drop slightly compared to having those two in separate
indices. But to get a more accurate answer you should really ask this in the
Lucene forums.

2011/7/15 Boris Kizelshteyn bo...@popcha.com

 I have an index called UniqueIndex, here I am storing emails and ids which
 you guessed it have to be unique. Is there any performance difference if I
 store both keys on the same index vs. storing them on two different
 indexes?

 Many thanks!
 ___
 Neo4j mailing list
 User@lists.neo4j.org
 https://lists.neo4j.org/mailman/listinfo/user




-- 
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] OSGi and Neo4j - superbundle approach

Nicolas,
the best would be to be able to emulate this in a test, too. Maybe
classloading issues? Maybe forking the PaxExam test will reproduce the
issue, or running you Felix setup as a testcase to trigger this?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Sun, Jul 17, 2011 at 9:21 PM, Nicolas Jouanin
nicolas.joua...@gmail.com wrote:
 I've sent the CLA signup.
 The OSGiTest is working fine for me too. Problems come when taking the felix 
 framework out of the box and deploying manually neo4j-osgi bundle with 
 another bundle containing the same activator class.
 I made sure to deploy same bundles but I get 
 java.lang.IllegalArgumentException: No index provider 'lucene' found when 
 registring the IndexService.
 I think i miss a bundle which is created automatically by pax during the test 
 case. May be lucene is missing.
 What do you think

 Le 17 juil. 2011 à 17:33, Peter Neubauer a écrit :

 Hi Nicolas,
 Have you been able to run
 https://github.com/njouanin/neo4j-osgi/blob/master/examples/src/test/java/org/neo4j/examples/osgi/OSGiTest.java
 properly? It actually is running on Equinox, so let me try it with
 Felix ... back in a moment.

 Also, in order to pull your changes, could you sign up as a committer,
 and send a mail according to
 http://wiki.neo4j.org/content/About_Contributor_License_Agreement ?


 Cheers,

 /peter neubauer

 GTalk:      neubauer.peter
 Skype       peter.neubauer
 Phone       +46 704 106975
 LinkedIn   http://www.linkedin.com/in/neubauer
 Twitter      http://twitter.com/peterneubauer

 http://www.neo4j.org               - Your high performance graph database.
 http://startupbootcamp.org/    - Öresund - Innovation happens HERE.
 http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



 On Sun, Jul 17, 2011 at 3:02 PM, Nicolas Jouanin
 nicolas.joua...@gmail.com wrote:
 Hi Peter,

 Thanks for initiating this work on OSGi bundle.
 I've forked you repository and tried to make is work using this sample 
 activator [1]. My goal is to make my activator work when deployed on Felix 
 with the neo4j superbundle.
 First of all, I had troubles to mvn clean install the bundle module. 
 maven-bundle-plugin is complaining about bundle.namespace property not set. 
 I've fixed it.
 Then when deploying it on Felix I had an error saying that 
 javax.transaction couldn't be imported. Because this package is already 
 included in superbundle, I've removed it explicitally from the 
 Import-Package directive. You can see the changes here [2].
 Then I tried to make my bundle activated using felix and the following 
 bundles installed :
 g! lb
 START LEVEL 1
   ID|State      |Level|Name
    0|Active     |    0|System Bundle (3.2.2)
    1|Active     |    1|Neo4j OSGi default bundle (0.1.0.SNAPSHOT)
    2|Active     |    1|Apache Felix Bundle Repository (1.6.2)
    3|Active     |    1|Apache Felix Gogo Command (0.8.0)
    4|Active     |    1|Apache Felix Gogo Runtime (0.8.0)
    5|Active     |    1|Apache Felix Gogo Shell (0.8.0)
    6|Resolved   |    1|EscapeK - Neo4j OSGi how-to - Embedded client 
 (0.0.1.SNAPSHOT)
 There are only felix core, neo4j superbundle and my test bundle.
 So when starting felix I get the following trace :
 
 Welcome to Apache Felix Gogo

 g! Opening database in embedded mode: 17 juil. 2011 14:57:08 
 org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog 
 doInternalRecovery
 INFO: Non clean shutdown detected on log [graphdb/nioneo_logical.log.1]. 
 Recovery started ...
 OK
 Populating it ... OK
 Hello, brave Neo4j world!
 ERROR: Bundle org.escapek.neo4j-osgi-howto.embedded-client [6] Error 
 starting 
 file:/Users/nico/Dev/felix-framework-3.2.2/bundle/org.escapek.neo4j-osgi-howto.embedded-client_0.0.1-SNAPSHOT.jar
  (org.osgi.framework.BundleException: Activator start error in bundle 
 org.escapek.neo4j-osgi-howto.embedded-client [6].)
 java.lang.IllegalArgumentException: No index provider 'lucene' found
        at 
 org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:76)
        at 
 org.neo4j.kernel.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java:116)
        at 
 org.neo4j.kernel.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerImpl.java:178)
        at 
 org.neo4j.kernel.IndexManagerImpl.getOrCreateNodeIndex(IndexManagerImpl.java:267)
        at 
 org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:255)
        at 
 org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:249)
        at 
 org.escapek.neo4jOsgiHowto.embeddedClient.EmbeddedClientActivator.start(EmbeddedClientActivator.java:65)

Re: [Neo4j] How to create a graph database out of a huge dataset?

Stephan,

can you perhaps share your csv file or give at least a few sample lines and a 
typical distribution (articles per author etc). You tested this with 20M 
arcticles and 6M authors? What is the current runtime of that import with which 
kind of hardware? (when working on a similar test I noticed that a lot of time 
was spent in the parsing, so it might be sensible to write a small converter to 
convert that csv file to a file with one entry per line and at the same time 
filter out invalid entries (wite them to a separate file). (If I read your code 
correctly you also have a number of authors per article on following lines 
duplicating the article name just with another author ? That might be also 
handled by the separate file, which could contain the # of author or have a 
blank line between each new article.

I don't know if your input data is presorted in any way. So if you might have 
recurring authors within a certain set you might also enable the caching of 
author index lookups.

((LuceneIndex)authorList).setCacheCapacity(Author,1);

you just specifiy your java heap memory with -Xmx3G or such. 

For your memory mapped files, it would be good to know how much memory your 
machine has. Otherwise see here: 
(http://docs.neo4j.org/chunked/1.4/configuration.html)

Something that might incur performance is creating the super-category nodes for 
articles and authors which then have 20M+ relationships. Perhaps it is better 
to split them by some sharding key to subnodes, or use an index for those too.

It might also be better to reduce your tx size a bit to 10k.

Something I noticed. You do the index get, check for the size and do 
getSingle() afterwards. I'm not sure if not closing the result iterator in the 
other case incurs any problems. So probably change that to 

Node author= authorList.get(Author, artikelinfo[2]).getSingle();

which automatically closes the result and returns null if there is none.

Regarding database size. What is the size and format of the timestamp string? 
It might be sensible to convert that to a long before storing it in the graph 
or at least make sure it
fits into the shortstring compression: 
(http://docs.neo4j.org/chunked/1.4/short-strings.html) depends on how you will 
use that in traversals.

I'll take your code and will run it locally to see where the performance issues 
are.

Cheers

Michael

Am 17.07.2011 um 20:53 schrieb st3ven:

 Hi,
 
 thanks for your fast answer.
 Right now I'm using lucene for 6M authors, but my whole dataset consists of
 nearly 25M authors.
 Can i use lucene there also, because I think this getting really slow to
 check if a user already exists.
 How can I change my heap memory settings and my memory-map settings, cause
 I'm using the transactional mode? 
 Cause I think with 25M authors I will get a OutOfMemory Exception.
 
 Here is my code that I have already written so far:
 
 import java.io.BufferedReader;
 import java.io.FileReader;
 import java.io.IOException;
 
 import org.neo4j.graphdb.GraphDatabaseService;
 import org.neo4j.graphdb.Node;
 import org.neo4j.graphdb.Relationship;
 import org.neo4j.graphdb.Transaction;
 import org.neo4j.graphdb.index.Index;
 import org.neo4j.graphdb.index.IndexHits;
 import org.neo4j.graphdb.index.IndexManager;
 import org.neo4j.kernel.EmbeddedGraphDatabase;
 
 public class WikiGraphRegUser {
 
   /**
* @param args
*/
   public static void main(String[] args) throws IOException {
 
   BufferedReader bf = new BufferedReader(new FileReader(
   E:/wiki0.csv));
   WikiGraphRegUser wgru = new WikiGraphRegUser();
   wgru.createGraphDatabase(bf);
   }
 
   private String articleName = ;
   private GraphDatabaseService db;
   private IndexManager index;
   private IndexNode authorList;
   private int transactionCounter = 0;
   private Node article;
   private boolean isFirstAuthor = false;
   private Node author;
   private Relationship relationship;
   private int node;
 
   private void createGraphDatabase(BufferedReader bf) {
   db = new EmbeddedGraphDatabase(target/db);
   index = db.index();
   authorList = index.forNodes(Author);
 
   String zeile;
   Transaction tx = db.beginTx();
 
   try {
   // reads lines of CSV-file
   while ((zeile = bf.readLine()) != null) {
   if (transactionCounter++ % 5 == 0) {
 
   tx.success();
   tx.finish();
   tx = db.beginTx();
   }
   // String[] looks like this: Article%;% 
 Timestamp%;% Author
   String[] artikelinfo = zeile.split(%;% );
   if (artikelinfo.length != 3) {

Re: [Neo4j] How can you have transactions with Lucene?