Re: [Neo4j] Lucene index: Too many open files
There are two things here: The error you're getting is because you probably forget to close some IndexHits somewhere, IndexHits that you haven't iterated all the way through... could you check your code for such mistakes? The other one that doCommit always seems to be invoked I just tried and cannot reproduce, so may be something on your end? could you provide a code snippet to reproduce this? Best, Mattias 2011/3/30 Axel Morgner > Hi, > > during stress tests, I got a "Too many open files" message (see detailed > stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, > Core i7 which handles the load with ease. > > cat /proc/sys/fs/file-max > 2399285 > > The parameter fs.file-max was already set very high, but I'll increase > it further and see if it will happen again. > > fs.file-max = 6815744 > fs.aio-max-nr = 1048576 > > Are there recommendations of linux kernel parameters? Or/and would you > recommend other things to circumvent that, f.e. using multiple indexes > instead of one? > > And it seems to me that finishing a Neo4j transaction will always > trigger a LuceneTransaction.doCommit, even if there was no update on > index (I try to avoid for the log nodes). Can this be switched off? > > Thanks and greetings > > Axel > > > [1] Stack trace > > java.lang.RuntimeException: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) > at > > org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) > at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) > at org.structr.core.log.LogService.flushQueue(LogService.java:101) > at org.structr.core.log.LogService.run(LogService.java:73) > Caused by: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.(RandomAccessFile.java:233) > at > > org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.(SimpleFSDirectory.java:180) > at > org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) > at org.apache.lucene.index.FieldsWriter.(FieldsWriter.java:86) > at > > org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66) > at > > org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144) > at > > org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:193) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1460) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1479) > at > > org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1099) > at > > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:777) > at > > org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:752) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1932) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1906) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.applyDocuments(LuceneTransaction.java:299) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:260) > ... 12 more > javax.transaction.xa.XAException: Unknown > xid[GlobalId[NEOKERNL|5131995998687892543|61442], BranchId[ 52 49 52 49 > 52 49 ]] > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.rollback(XaResourceManager.java:470) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.rollback(XaResourceHelpImpl.java:111) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.doRollback(TransactionImpl.java:533) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:648) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104
Re: [Neo4j] Lucene index: Too many open files
Is there a good way of ensuring the IndexHits, close? Especially if I give out the IndexHits (perhaps in an IteratorWrapper) to client code? Thanks Michael Am 31.03.2011 um 09:29 schrieb Mattias Persson: > There are two things here: > > The error you're getting is because you probably forget to close some > IndexHits somewhere, IndexHits that you haven't iterated all the way > through... could you check your code for such mistakes? > > The other one that doCommit always seems to be invoked I just tried and > cannot reproduce, so may be something on your end? could you provide a code > snippet to reproduce this? > > > Best, > Mattias > > 2011/3/30 Axel Morgner > >> Hi, >> >> during stress tests, I got a "Too many open files" message (see detailed >> stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, >> Core i7 which handles the load with ease. >> >> cat /proc/sys/fs/file-max >> 2399285 >> >> The parameter fs.file-max was already set very high, but I'll increase >> it further and see if it will happen again. >> >> fs.file-max = 6815744 >> fs.aio-max-nr = 1048576 >> >> Are there recommendations of linux kernel parameters? Or/and would you >> recommend other things to circumvent that, f.e. using multiple indexes >> instead of one? >> >> And it seems to me that finishing a Neo4j transaction will always >> trigger a LuceneTransaction.doCommit, even if there was no update on >> index (I try to avoid for the log nodes). Can this be switched off? >> >> Thanks and greetings >> >> Axel >> >> >> [1] Stack trace >> >> java.lang.RuntimeException: java.io.FileNotFoundException: >> /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too >> many open files) >>at >> >> org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) >>at >> >> org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) >>at >> >> org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) >>at >> >> org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) >>at >> >> org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) >>at >> org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) >>at >> org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) >>at >> >> org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) >>at >> org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) >>at >> >> org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) >>at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) >>at org.structr.core.log.LogService.flushQueue(LogService.java:101) >>at org.structr.core.log.LogService.run(LogService.java:73) >> Caused by: java.io.FileNotFoundException: >> /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too >> many open files) >>at java.io.RandomAccessFile.open(Native Method) >>at java.io.RandomAccessFile.(RandomAccessFile.java:233) >>at >> >> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.(SimpleFSDirectory.java:180) >>at >> org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) >>at org.apache.lucene.index.FieldsWriter.(FieldsWriter.java:86) >>at >> >> org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66) >>at >> >> org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144) >>at >> >> org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:193) >>at >> >> org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1460) >>at >> >> org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1479) >>at >> >> org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1099) >>at >> >> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:777) >>at >> >> org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:752) >>at >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1932) >>at >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1906) >>at >> >> org.neo4j.index.impl.lucene.LuceneTransaction.applyDocuments(LuceneTransaction.java:299) >>at >> >> org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:260) >>... 12 more >> javax.transaction.xa.XAException: Unknown >> xid[GlobalId[NEOKERNL|5131995998687892543|61442], BranchId[ 52 49 52 49 >> 52 49 ]] >>at >> >> org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.rollback(XaResourceManager.java:470) >>at >> >> org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.rollback(XaResourceHelpImpl.java:111) >
Re: [Neo4j] Lucene index: Too many open files
Thanks for the info re: calling close on the indexhits object. We will wrap that logic in a try/finally wherever we use index queries. - Reply message - From: "Mattias Persson" Date: Thu, Mar 31, 2011 3:29 am Subject: [Neo4j] Lucene index: Too many open files To: "Neo4j user discussions" There are two things here: The error you're getting is because you probably forget to close some IndexHits somewhere, IndexHits that you haven't iterated all the way through... could you check your code for such mistakes? The other one that doCommit always seems to be invoked I just tried and cannot reproduce, so may be something on your end? could you provide a code snippet to reproduce this? Best, Mattias 2011/3/30 Axel Morgner > Hi, > > during stress tests, I got a "Too many open files" message (see detailed > stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, > Core i7 which handles the load with ease. > > cat /proc/sys/fs/file-max > 2399285 > > The parameter fs.file-max was already set very high, but I'll increase > it further and see if it will happen again. > > fs.file-max = 6815744 > fs.aio-max-nr = 1048576 > > Are there recommendations of linux kernel parameters? Or/and would you > recommend other things to circumvent that, f.e. using multiple indexes > instead of one? > > And it seems to me that finishing a Neo4j transaction will always > trigger a LuceneTransaction.doCommit, even if there was no update on > index (I try to avoid for the log nodes). Can this be switched off? > > Thanks and greetings > > Axel > > > [1] Stack trace > > java.lang.RuntimeException: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) > at > > org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) > at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) > at org.structr.core.log.LogService.flushQueue(LogService.java:101) > at org.structr.core.log.LogService.run(LogService.java:73) > Caused by: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.(RandomAccessFile.java:233) > at > > org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.(SimpleFSDirectory.java:180) > at > org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) > at org.apache.lucene.index.FieldsWriter.(FieldsWriter.java:86) > at > > org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66) > at > > org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144) > at > > org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:193) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1460) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1479) > at > > org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1099) > at > > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:777) > at > > org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:752) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1932) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1906) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.applyDocuments(LuceneTransaction.java:299) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:260) > ... 12 more > javax.transaction.xa.XAException: Unknown > xid[GlobalId[NEOKERNL|5131995998687892543|61442], BranchId[ 52 49 52 49 > 52 49 ]] > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.rollback(XaResourceManager.java:470) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.rollback(XaResourceHelpImpl.java:111) > at > > org.neo4j.kernel.impl.transaction.Transa
Re: [Neo4j] Lucene index: Too many open files
I get that too, but sometimes. I got the feeling that it relates to the number of @Transactional annotations that get nested. I might be completely wrong. But I got rid of it by revising my annotations. Alfredas On Wed, Mar 30, 2011 at 11:40 PM, rick.bullo...@burningskysoftware.com wrote: > Sounds like a bug to me. > > - Reply message - > From: "Axel Morgner" > Date: Wed, Mar 30, 2011 1:23 pm > Subject: [Neo4j] Lucene index: Too many open files > To: "Neo4j user discussions" > > Hi, > > during stress tests, I got a "Too many open files" message (see detailed > stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, > Core i7 which handles the load with ease. > > cat /proc/sys/fs/file-max > 2399285 > > The parameter fs.file-max was already set very high, but I'll increase > it further and see if it will happen again. > > fs.file-max = 6815744 > fs.aio-max-nr = 1048576 > > Are there recommendations of linux kernel parameters? Or/and would you > recommend other things to circumvent that, f.e. using multiple indexes > instead of one? > > And it seems to me that finishing a Neo4j transaction will always > trigger a LuceneTransaction.doCommit, even if there was no update on > index (I try to avoid for the log nodes). Can this be switched off? > > Thanks and greetings > > Axel > > > [1] Stack trace > > java.lang.RuntimeException: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) > at > org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) > at > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) > at > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) > at > org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) > at > org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) > at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) > at org.structr.core.log.LogService.flushQueue(LogService.java:101) > at org.structr.core.log.LogService.run(LogService.java:73) > Caused by: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.(RandomAccessFile.java:233) > at > org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.(SimpleFSDirectory.java:180) > at > org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) > at org.apache.lucene.index.FieldsWriter.(FieldsWriter.java:86) > at > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4J Spatial - issue with bounding box indices in OSM Layer.
Hi Robert,
I took a look at this and the issue is that you are using the
OSMGeometryEncoder to decode the RTree nodes. And the GeometryEncoder is
designed to be specific to your data model, while the RTree internal data is
hard-coded into the RTree design. So there is no guarantee that any
particular data model's geometry encoder will store a bounding box in the
same way as the RTree does.
In this particular case the RTree uses the conventions of the JTS Envelope,
while the OSMGeometryEncoder uses the conventions of the GeoTools envelope.
I looked around and the WKBGeometryEncoder we use for ESRI Shapefile support
uses JTS, so your code would have worked there.
So you should definitely not use the OSM-specific geometry encoder for
looking at anything other than the OSM-specific geometries. The correct API
to get the bounding box from the index is, in fact, the
getLayerBoundingBox() method you already used.
So the only mistake was to grab the root node of the RTree and pass it to
the OSMGeometryEncoder as if it were an OSM Geometry Node, which it is not.
Does that clarify things?
Regards, Craig
On Tue, Mar 29, 2011 at 9:44 AM, Robert Boothby wrote:
> Sorry about dropping out at the end of last week - had some personal
> issues to deal with. I have the following unit test code that
> illustrates the breakdown in the envelope definition:
>
>@Test
>public void useLayer() {
> final OSMLayer osmLayer =
> (OSMLayer)spatialDB.getLayer("OSM-BUCKS");
>final GeometryFactory factory = osmLayer.getGeometryFactory();
> System.out.println("Unit of measure: " +
>
> CRS.getEllipsoid(osmLayer.getCoordinateReferenceSystem()).getAxisUnit().toString());
>
>final Point point = factory.createPoint(new
> Coordinate(-0.812988,51.796726));//51.808721,-0.689735));
>
>final Envelope layerBoundingBox =
> osmLayer.getIndex().getLayerBoundingBox();
>final Envelope usedEnvelope =
>
> osmLayer.getGeometryEncoder().decodeEnvelope(((RTreeIndex)osmLayer.getIndex()).getIndexRoot());
>System.out.println("Layer Bounding Box: " +
> layerBoundingBox.toString());
>System.out.println("Envelope used in search:" +
> usedEnvelope.toString());
>assertEquals(layerBoundingBox, usedEnvelope);
>
>SearchContain searchContain = new SearchContain(point);
>osmLayer.getIndex().executeSearch(searchContain);
>
>
>for(SpatialDatabaseRecord record: searchContain.getResults()){
>System.out.println("Container:" + record);
> for(String propertyName: record.getPropertyNames()){
>final Object propertyValue =
> record.getProperty(propertyName);
>if(propertyValue != null){
>System.out.println("\t" + propertyName + " : " +
> propertyValue);
>}
>}
>}
>}
>
> It throws an assertion failure at the 'assertEquals(layerBoundingBox,
> usedEnvelope)' - effectively I pull the layer index envelope out using
> the same code as the 'AbstractSearch.getEnvelope(Node geomNode)' does
> (used in AbstractSearchIntersection.needsToVisit()) and compare it
> with what the layer thinks the envelope should be. The same numeric
> values appear in the two envelopes but in different fields.
>
> Hopefully this gives you all you need to diagnose the problem - but if
> not let me know and we can work out how to drop my rather large test
> data set and project into a common place.
>
> Robert.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] undirected graph
how to create an undirected graph.please tell me the API. every time i create a graph,it is a directed graph,such as A--->B；but not B--->A thx~~ -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/undirected-graph-tp2758690p2758690.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] undirected graph
In Neo, all relationships can be traversed in either direction. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Sent: Thursday, March 31, 2011 9:53 AM To: user@lists.neo4j.org Subject: [Neo4j] undirected graph how to create an undirected graph.please tell me the API. every time i create a graph,it is a directed graph,such as A--->B；but not B--->A thx~~ -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/undirected-graph-tp2758690p2758690.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] undirected graph
Hi there,
you always create directed edges. However, if you don't want to take
directions into account, just ignore the directions in your traversals
using
for ( Path position : Traversal.description()
.depthFirst()
.relationships( LIKES, Direction.BOTH )
.prune( Traversal.pruneAfterDepth( 5 ) )
.traverse( myStartNode ) )
{
System.out.println( "Path from start node to current position is "
+ position );
}
So, the direction of the LIKES relationship is ignored. Is that ok for you?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
2011/3/31 见光分解 :
> how to create an undirected graph.please tell me the API.
> every time i create a graph,it is a directed graph,such as A--->B；but not
> B--->A
> thx~~
>
> --
> View this message in context:
> http://neo4j-user-list.438527.n3.nabble.com/undirected-graph-tp2758690p2758690.html
> Sent from the Neo4J User List mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] undirected graph
you mean that there are no differences between undirected graph and directed graph in neo4j? -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/undirected-graph-tp2758690p2758790.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] cycle detection
Jacopo,
do you have the source for this available somewhere? It's hard to
judge anything from the class files ...
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Mar 30, 2011 at 5:48 PM, Jacopo wrote:
> Ok, I wrote a class to do the work without changing the graph
> http://dl.dropbox.com/u/7137905/SCC_neo4j_tarjan.jar
>
> It surely needs some improvements, for example it cannot manage too
> large graphs (e.g. more than 10^6 nodes), but should work.
>
> A note: it looks for Strongly Connected Components, subsets of the graph
> where any node can be reached by any other node of the set without going
> out of the set itself. A SCC bigger than one node implies the existence
> of at least one cycle.
>
> Cheers,
> Jacopo
>
> P.S.: the code create a set of nodes called s, that is truncated when
> it's sure it contains a SCC. Would be a good idea to remove
> corresponding nodes from the maps indexes and lowlinks, since they will
> never be used, in order to consume less memory.
>
> Il giorno mar, 29/03/2011 alle 10.19 +0200, Mattias Persson ha scritto:
>> 2011/3/28 Jacopo
>>
>> > Uh, I may create a node and use relationships with it instead of node
>> > properties, to delete it after the work, but it doesn't sound a good
>> > solution.
>> >
>>
>> Letting a graph algorithm set temporary properties and creating other
>> entities (thus holding write locks and memory) seems like a bad idead. Hold
>> these things in a Set or Map instead.
>>
>>
>> > Jacopo
>> > Il giorno lun, 28/03/2011 alle 21.23 +0200, Jacopo ha scritto:
>> > > There's no problem with it!
>> > > The only issue is that it needs to add properties to visited nodes in
>> > > order to be able to detect cycles. It's possible to delete them after
>> > > the work, but in case the graph already uses that properties name or the
>> > > program is interrupted before finishing it would be a problem. Is there
>> > > a way to create temporary properties?
>> > >
>> > > Jacopo
>> > > Il giorno lun, 28/03/2011 alle 10.16 +0200, Peter Neubauer ha scritto:
>> > > > Cool!
>> > > >
>> > > > Would be great to maybe add this to the graph-algo package, if you
>> > > > don't mind? Just fork and add it from
>> > > > https://github.com/neo4j/graphdb/tree/master/graph-algo and do a merge
>> > > > request ...
>> > > >
>> > > > Cheers,
>> > > >
>> > > > /peter neubauer
>> > > >
>> > > > GTalk: neubauer.peter
>> > > > Skype peter.neubauer
>> > > > Phone +46 704 106975
>> > > > LinkedIn http://www.linkedin.com/in/neubauer
>> > > > Twitter http://twitter.com/peterneubauer
>> > > >
>> > > > http://www.neo4j.org - Your high performance graph
>> > database.
>> > > > http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
>> > > > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing
>> > party.
>> > > >
>> > > >
>> > > >
>> > > > On Sun, Mar 27, 2011 at 10:56 PM, Jacopo
>> > wrote:
>> > > > > In case you are interested, I implemented cycle detection by using
>> > > > > Tarjan algorithm, not the traversal.
>> > > > > The code is visible in the Italian Wikipedia, I hope it's
>> > intelligible
>> > > > > although the language.
>> > > > >
>> > http://it.wikipedia.org/wiki/Algoritmo_di_Tarjan_per_le_componenti_fortemente_connesse#Implementazione_in_Java
>> > > > >
>> > > > >
>> > > > > Hi
>> > > > > Jacopo Farina
>> > > > >
>> > > > > Il giorno ven, 25/03/2011 alle 13.51 +0100, Mattias Persson ha
>> > scritto:
>> > > > >> I think you could implement this using RELATIONSHIP_GLOBAL
>> > uniqueness, like
>> > > > >> (from the top of my head):
>> > > > >>
>> > > > >> Traversal.description().uniqueness( Uniqueness.RELATIONSHIP_GLOBAL
>> > )
>> > > > >> .evaluator(new Evaluator() {
>> > > > >> public Evaluation(Path path) {
>> > > > >> return path.length() > 0 &&
>> > endNodeOccursPreviouslyInPath(
>> > > > >> path ) ?
>> > > > >> Evaluation.INCLUDE_AND_CONTINUE :
>> > > > >> Evaluation.EXCLUDE_AND_CONTINUE;
>> > > > >> }
>> > > > >>
>> > > > >> private boolean endNodeOccursPreviouslyInPath(Path path) {
>> > > > >> Node endNode = path.endNode();
>> > > > >> int counter = 0;
>> > > > >> for ( Node node : path.nodes() ) {
>> > > > >> if ( counter++ < path.length() && node.equals(
>> > endNode ) )
>> > > > >> return true;
>> > > > >> }
>> > > > >> return false;
>> > > > >> }
>> > > > >> } ).traverse(...);
>> > > > >>
>> > > > >> This should (if it even com
Re: [Neo4j] undirected graph
Exactly. Just treat a directed graph undirected by ignoring the direction of relationships. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. 2011/3/31 见光分解 : > you mean that there are no differences between undirected graph and directed > graph in neo4j? > > -- > View this message in context: > http://neo4j-user-list.438527.n3.nabble.com/undirected-graph-tp2758690p2758790.html > Sent from the Neo4J User List mailing list archive at Nabble.com. > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Query paths of certain lengths...
Hi Jeff,
What you are describing is a Friends-of-a-friend (foaf) traversal, "which
are the friends of my friends, who are not my direct friends".
The best way to perform such a traversal is by using the traversal API:
n.traverse( Order.BREADTH_FIRST, // because we want to exclude the direct
friends early
new StopEvaluator() { // don't traverse deeper than depth two
boolean isStopNode(TraversalPosition pos) { return pos.depth() == 2; }
}, new ReturnableEvaluator() { // return the ones found on depth two
boolean isReturnableNode(TraversalPosition pos) { return pos.depth()
== 2; }
}, RelTypes.FRIEND, Direction.BOTH ); // Traverse Friend relationships,
in any direction
Since this traverses breadth first, it will find the direct friends first,
and exclude those, since this kind of traversal does not revisit nodes.
The traversal above almost does what you want, except that it only returns
nodes, and you wanted the paths. For that you can use our more capable
tentative traversal API to do the same traversal, but getting paths instead:
TraversalDescription foaf = Traversal.description()
.breadthFirst()
.relationships( RelTypes.FRIEND, Direction.OUTGOING )
.evaluator( new Evaluator() {
Evaluation evaluate( Path path )
{
return ( path.length() == 2 ) ? Evaluation.INCLUDE_AND_PRUNE :
Evaluation.EXCLUDE_AND_CONTINUE;
}
} );
Then you'd just iterate over the paths from the given node:
for (Path foafPath : foaf.traverse( n ) ) {
printPath( foafPath );
}
If you want that for ALL possible start-nodes, I guess you would have to
iterate through all nodes and apply the same traversal. But given just a
single start node, the above traversal is enough, and it is fast enough to
do in real time (show the friends of my friends on a social media website).
Cheers,
Tobias
On Wed, Mar 30, 2011 at 7:51 PM, jisenhart wrote:
>
> Suppose I have the following node/paths
>
> n -> n1
> n -> n2
> n -> n4
>
> n2 -> n1
> n2 -> n3
> n2 -> n4
>
> I want to find all paths of depth two (for example):
>
> n -> n2 -> n3
> n -> n2 -> n4
>
> and filter out those paths where a shorter path exists to a given node
> (n) leaving just
>
> n -> n2 -> n3
>
> since n -> n4 is "shallower" than n -> n2 -> n4
>
>
> Is this possible? I see GraphAlgoFactory.pathsWithLength(expander,
> length). But unclear on how to proceed beyond that.
>
> Jeff
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Tobias Ivarsson
Hacker, Neo Technology
www.neotechnology.com
Cellphone: +46 706 534857
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene index: Too many open files
On 31.03.2011 09:29, Mattias Persson wrote:
> There are two things here:
>
> The error you're getting is because you probably forget to close some
> IndexHits somewhere, IndexHits that you haven't iterated all the way
> through... could you check your code for such mistakes?
Hmm ... I checked the code. Iteration always goes through all hits I think:
IndexHits hits = index.query(new QueryContext(textualQueryString));
List result = nodeFactory.createNodes(hits);
The createNodes-Method looks like that:
public List createNodes(final Iterable hits) {
List nodes = new ArrayList();
if (hits != null && hits.iterator().hasNext()) {
for (Node node : hits) {
AbstractNode n = createNode(node);
nodes.add(n);
}
}
return nodes;
}
Maybe the IndexHits are not closed if thread gets interrupted?
Now I added a hits.close() after the createNodes line. More stress tests
will follow and reveal all flaws in my code ... ;-)
> The other one that doCommit always seems to be invoked I just tried and
> cannot reproduce, so may be something on your end? could you provide a code
> snippet to reproduce this?
Debugged it, but was not able to isolate or reproduce it. Will keep an
eye on it. I'm quite sure it has to do with my asynchronous logging
service, but it seems that it is kinda hard to track down ...
Thanks for your help, and best regards
Axel
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Connection Pool in Neo4j
Hi, I have just started using Neo4j. I was wondering if Connection Pooling has been implemented in Neo4j ? I could not find any helper classes or any sort of documentation on it. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Connection Pool in Neo4j
No need for connection pooling, since there is a single instance of the database connection that can be shared across threads. - Reply message - From: "sulabh choudhury" Date: Thu, Mar 31, 2011 8:35 pm Subject: [Neo4j] Connection Pool in Neo4j To: "user@lists.neo4j.org" Hi, I have just started using Neo4j. I was wondering if Connection Pooling has been implemented in Neo4j ? I could not find any helper classes or any sort of documentation on it. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Help for neo4j
Hello，I'm a newbie for Neo4j. Recently I visited wiki, got a lot of useful information, but there are some questions blocking me. 1. In neo4j's api，every method is thread-safe. I want to ask if every request for read or write to neo4j be queued，one by one processing. 2. config HA cluster by http://wiki.neo4j.org/content/High_Availability_Cluster，but when startup zookeeper，there is a error：invalid config server.1=localhost:2888:3888 server.2=localhost:2889:3889 server.3=localhost:2890:3890 I delete three rows as above，startup zookeeper successfully. run command：neo4j-shell -host localhost -port 1331，cannot connect. why? is there a more detail wiki about HA cluster？ 3. how to use RemoteGraphDatabase，can you give me a example？what protocol it support？ ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Filter by relationship Property
Have you tried my solution?
1) "relationships": [ { "type": "TYPE_A", "direction": "out" },
{ "type": "TYPE_B" } ]
2) "max depth": 5
3) A traversal is an "all paths" algorithm in itself, although with no given
end node. But this behaviour is mimicked by adding a condition in the return
filter.
4) I don't know if you're looking for a path between two given nodes, or
just all paths from a node to any node... but filtering/pruning like this
could work.
"return filter" : {
"language": "javascript",
"body": "position.length()>0 &&
position.lastRelationship().getProperty('customerId',0)==1 &&
position.endNode().getId()==492"
}, "prune evaluator": {
"language": "javascript",
"body": "position.length()>0 &&
position.lastRelationship().getProperty('customerId',0)!=1"
}
2011/3/30 Justine Mathews
> The solution is not really working for me...
>
> Actually I am trying find paths between the Nodes, based on following
> criteria,
>
>
> 1) Combination of Relationships (which is working).
>
> 2) Maxdepth
>
> 3) Algoritham - allpaths
>
> 4) And also I only need paths of certain property value of
> relationship(In our case the Property is customerId)
> That is when we create relationship we add property(customerId) to it,
> based on user group.
> If I cannot filter, I will be pulling out more unwanted paths, which will
> not really work that way for us..
> Can all the above combination, will work together?
> --
> Regards,
> Justine K Mathews, MCSD.NET
> Mob: +44-(0) 7795268546
> http://www.justinemathews.com http://www.justinemathews.com%3chttp:/www.justinemathews.com/>>
>
> > http://uk.linkedin.com/in/justinemathews
>
> >
>
> > ___
>
> > Neo4j mailing list
>
> > User@lists.neo4j.org
>
> > https://lists.neo4j.org/mailman/listinfo/user
>
> >
>
>
>
>
> --
> Regards,
> Justine K Mathews, MCSD.NET
> Mob: +44-(0) 7795268546
> http://www.justinemathews.com
