Re: [Neo4j] Category Nodes and Type Safety vs Performance and Scaling
Hi Guys, Small error. IEnumerable list = g.v(0).outE[[label:'HOSTS_COMPANY']].inV[[Key:'MyCompanyName']]. From: Romiko Derbynew Sent: Monday, 20 June 2011 3:56 PM To: Neo4j user discussions Cc: Tatham Oddie; Jamal Abreu (jab...@barnardos.org.au) Subject: Category Nodes and Type Safety vs Performance and Scaling Hi Guys, We currently thinking of how we can get type safety when we do queries, one thing we do is have a category nodes, so e.g. Here is a sample query IEnumerable list = g.v(0).outE[[label:'HOSTS']].inV[[Key:'MyCompanyName']].inE[[label:'IS_COMPANY']].inV So the part highlighted in bold leverages the category. However, we could also remove the category node (Agencies) and then just have more explicit relationships e.g. IEnumerable list = g.v(0).outE[[label:'HOSTS_AGENCY']].inV[[Key:'MyCompanyName']]. Notice, here we now have a relationship where from the name we can see that the types coming back will always be an Agency. This means relationship names have information about it's target Node type, making the amount of relationships proportional to the number of node types. What are the advantages/disadvantages of both approaches, which achieve the same result? Regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Category Nodes and Type Safety vs Performance and Scaling
Hi Guys, We currently thinking of how we can get type safety when we do queries, one thing we do is have a category nodes, so e.g. Here is a sample query IEnumerable list = g.v(0).outE[[label:'HOSTS']].inV[[Key:'MyCompanyName']].inE[[label:'IS_COMPANY']].inV So the part highlighted in bold leverages the category. However, we could also remove the category node (Agencies) and then just have more explicit relationships e.g. IEnumerable list = g.v(0).outE[[label:'HOSTS_AGENCY']].inV[[Key:'MyCompanyName']]. Notice, here we now have a relationship where from the name we can see that the types coming back will always be an Agency. This means relationship names have information about it's target Node type, making the amount of relationships proportional to the number of node types. What are the advantages/disadvantages of both approaches, which achieve the same result? Regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Multiple-source lowest-cost path search
On 6/19/2011 7:11 PM, Giacomo Bernardi wrote: > I'd like to build a second graph in which each e in (S-E) is connected From what i understood, connecting S-E to an arbitary node A1 and S with another arbitary node A2 and finding the lowest cost shortest path between A1 and A2 should give you the solution ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Multiple-source lowest-cost path search
Hello everyone, could you advice on the best way to do the following in neo4j? Given: - a graph G= on which each node has an associated positive cost - a subset of "source" nodes S in E. I'd like to build a second graph in which each e in (S-E) is connected to any one of the source nodes S via the lowest-cost path. I don't seem to find anything ready to use for this purpose in neo4j, am I right? Thanks! mino ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] More spatial questions
Hi Nolan, I think I can answer a few of your questions. Firstly, some background. The graph model of the OSM data is based largely on the XML formated OSM documents, and there you will find 'nodes', 'ways', 'relations' and 'tags' each as their own xml-tag, and as a consequence each will also have their own neo4j-node in the graph. Another point is that the geometry can be based on one or more nodes or ways, and so we always create another node for the geometry, and link it to the osm-node, way or relation that represents that geometry. What all this boils down to is that you cannot find the tags on the geometry node itself. You cannot even find the location on that node. If you want to use the graph model in a direct way, as you have been trying, you really do need to know how the OSM data is modeled. For example, for a LineString geometry, you would need to traverse from the geometry node to the way node and finally to the tags node (to get the tags). To get to the locations is even more complex. Rather than do that, I would suggest that you work with the OSM API we provided with the OSMLayer, OSMDataset and OSMGeometryEncoder classes. Then you do not need to know the graph model at all. For example, OSMDataset has a method for getting a Way object from a node, and the returned object can be queried for its nodes, geometry, etc. Currently we provide methods for returning neo4j-nodes as well as objects that make spatial sense. One minor issue here is the ambiguity inherent in the fact that both neo4j and OSM make use of the term 'node', but for different things. We have various solutions to this, sometimes replacing 'node' with 'point' and sometimes prefixing with 'osm'. The unit tests in TestsForDocs includes some tests for the OSM API. My first goal is to find the nearest OSM node to a given lat, lon. My > attempts seem to be made of fail thus far, however. Here's my code: > Most of the OSM dataset is converted into LineStrings, and what you really want to do is find the closest vertex of the closest LineString. We have a utility function 'findClosestEdges' in the SpatialTopologyUtils class for that. The unit tests in TestSpatialUtils, and the testSnapping() method in particular, show use of this. My thinking is that nodes should be represented as points, so I can't > see why this fails. When I run this in a REPL, I do get a node back. So > far so good. Next, I want to get the node's tags. So I run: > The spatial search will return 'geometries', which are spatial objects. In neo4j-spatial every geometry is represented by a unique node, but it is not required that that node contain coordinates or tags. That is up to the GeometryEncoder. In the case of the OSM model, this information is elsewhere, because of the nature of the OSM graph, which is a highly interconnected network of points, most of which do not represent Point geometries, but are part of much more complex geometries (streets, regions, buildings, etc.). n.getSingleRelationship(OSMRelation.TAGS, Direction.INCOMING) > The geometry node is not connected directly to the tags node. You need two steps to get there. But again, rather than figure out the graph yourself, use the API. In this case, instead of getting the geometry node from the SpatialDatabaseRecord, rather just get the properties using getPropertyNames and getProperty(String). This API works the same on all kinds of spatial data, and in the case of OSM data will return the TAGS, since those are interpreted as attributes of the geometries. n.getSingleRelationship(OSMRelationship.GEOM, > Direction.INCOMING).getOtherNode(n).getPropertyKeys > I see what appears to be a series of tags (oneway, name, etc.) Why are > these being returned for OSMRelation.GEOM rather than OSMRelation.TAGS? > These are not the tags. Now you have found the node representing an OSM 'Way'. This has a few properties on it that are relevant to the way, the name, whether the street is oneway or not, etc. Sometimes these are based on values in the tags, but they are not the tags themselves. This node is connected to the geometry node and the tags node, so you were half-way there (to the tags that is). You started at the geometry node, and stepped over to the way node, and one more step (this time with the TAGS relationship) would have got you to the tags. But again, I advise against trying to explore the OSM graph by itself. As you have already found, it is not completely trivial. What you should have done is access the attributes directly from the search results. Additionally, I see the property way_osm_id, which clearly isn't a tag. > It would also seem to indicate that this query returned a way rather > than a node like I'd hoped. This conclusion is further born out by the > tag names. So clearly I'm not getting the search correct. But beyond > that, the way being returned by this search isn't close to the lat,lon I > provided. What am I missing? > The lat/long values are quite a bit deeper in the graph
[Neo4j] ls command for trav in shell does not working
According to the neo4j-sh help, there is a -c option to the trav command that should allow for executing the ls command on each node visited in a traversal. -c Command to run for each returned node. Use $i for node/relationship id, example: -c "ls -f name $i". Multiple commands can be supplied with && in between. I'm trying to simply do an ls to see the properties present for each node, and this command does not work. trav -c "ls $i" This just prints out the same output as if I ran the trav command without any additional parameters (it prints the relationships, but not the properties of each node). Am I doing something wrong, or does the -c option not work? Thanks WP === Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html === ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] More spatial questions
Fortunately, recent changes seem to have made the memory leaks I was
experiencing a few weeks ago to vanish. Apologies for not playing a more
active part in these discussions, but I'm finding there to be a quite
steep learning curve here, and I don't have the time to make a major
push to overcome it at once. But I finally have an import that I can
play with in a Scala REPL, though I'm encountering things about it that
just don't make sense.
My first goal is to find the nearest OSM node to a given lat, lon. My
attempts seem to be made of fail thus far, however. Here's my code:
def layer = spatialService.getLayer("map").asInstanceOf[OSMLayer]
private def pointSearch(lat:Double, lon:Double) = new SearchClosest(
layer.getGeometryFactory.createPoint(
new Coordinate(lat, lon)
)
)
def nearestNode(lat:Double, lon:Double) = {
val query = pointSearch(lat, lon)
val l = layer
l.addSimpleDynamicLayer(Constants.GTYPE_POINT)
l.getIndex.executeSearch(query)
val results = query.getResults
results.headOption.map { r =>
r.getGeomNode
}
}
My thinking is that nodes should be represented as points, so I can't
see why this fails. When I run this in a REPL, I do get a node back. So
far so good. Next, I want to get the node's tags. So I run:
n.getSingleRelationship(OSMRelation.TAGS, Direction.INCOMING)
I get null. OK, so maybe this doesn't have tags. Yet, when I run:
n.getSingleRelationship(OSMRelationship.GEOM,
Direction.INCOMING).getOtherNode(n).getPropertyKeys
I see what appears to be a series of tags (oneway, name, etc.) Why are
these being returned for OSMRelation.GEOM rather than OSMRelation.TAGS?
Additionally, I see the property way_osm_id, which clearly isn't a tag.
It would also seem to indicate that this query returned a way rather
than a node like I'd hoped. This conclusion is further born out by the
tag names. So clearly I'm not getting the search correct. But beyond
that, the way being returned by this search isn't close to the lat,lon I
provided. What am I missing?
As an aside, in looking at the latest OSM import testcase, it seems like
the batch inserter may now be optional. Is this true, and what
benefits/disadvantages are there to its use? I tried importing the Texas
OSM data on my fairly powerful laptop, but gave up after 12 hours and
17 way imports (I think there are over a million in that dataset.)
Other geospatial formats seem to do the import in a matter of hours, but
this import seemed like it'd go on for days if I let it.
Thanks.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Rest API in the future
http://docs.neo4j.org/chunked/snapshot/rest-api-batch-ops.html Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sun, Jun 19, 2011 at 7:46 PM, Mattias Persson wrote: > 2011/6/19 Jim Webber > >> Hello Aniceto, >> >> > - EmbeddedGraphDatabase, which is good for tests, supports transactions >> > and has quick communications. Not good for medium sized or HA apps >> > because lacks separation between app and data >> >> The EmbeddedGraphDatabase is fine for production use - in fact it's the >> default choice! The difference between using an EmbeddedGraphDatabase and >> the REST API is where the instance of EmbeddedGraphDatabase resides. If you >> use the REST API, it lives in our server process, otherwise it lives in your >> process. >> >> Really the distinction is much less than that names imply. >> >> > - standalone server: data repository is set apart from application, even >> > more than one app can access it. If EE, HA is available, but has a REST >> > interface which does not support transactions >> >> It has transactions of course - Neo4j is *always* ACID transactional. But >> this interface does not expose those transactions to users. Instead a >> transaction is implicitly wrapped around every interaction. >> >> > Can we expect any new communication interface with TX and HA support for >> > standalone servers, at least for Java/JVM clients or has the roadmap >> > another target? >> >> HA is supported whether you choose to use Neo4j embedded within your >> process or whether its embedded within our process inside the REST API. >> >> We're starting to think about next generation protocol support. That will >> probably include transactional control on the client side, but we have >> nothing to release in the 1.4 timeframe. >> > > Apart from the fact that you can send "batch operations" to the REST API, > i.e. group a number of operations to be executed in one transaction, instead > of one by one. This has the benefint of those operations being atomically > applied and much faster as well due to less http requests and also less > transactional overhead (commiting each transaction has some overhead > otherwise). Maybe someone could supply a good link for further information > since I don't really have any. > >> >> Jim >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > > > > -- > Mattias Persson, [matt...@neotechnology.com] > Hacker, Neo Technology > www.neotechnology.com > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Rest API in the future
2011/6/19 Jim Webber > Hello Aniceto, > > > - EmbeddedGraphDatabase, which is good for tests, supports transactions > > and has quick communications. Not good for medium sized or HA apps > > because lacks separation between app and data > > The EmbeddedGraphDatabase is fine for production use - in fact it's the > default choice! The difference between using an EmbeddedGraphDatabase and > the REST API is where the instance of EmbeddedGraphDatabase resides. If you > use the REST API, it lives in our server process, otherwise it lives in your > process. > > Really the distinction is much less than that names imply. > > > - standalone server: data repository is set apart from application, even > > more than one app can access it. If EE, HA is available, but has a REST > > interface which does not support transactions > > It has transactions of course - Neo4j is *always* ACID transactional. But > this interface does not expose those transactions to users. Instead a > transaction is implicitly wrapped around every interaction. > > > Can we expect any new communication interface with TX and HA support for > > standalone servers, at least for Java/JVM clients or has the roadmap > > another target? > > HA is supported whether you choose to use Neo4j embedded within your > process or whether its embedded within our process inside the REST API. > > We're starting to think about next generation protocol support. That will > probably include transactional control on the client side, but we have > nothing to release in the 1.4 timeframe. > Apart from the fact that you can send "batch operations" to the REST API, i.e. group a number of operations to be executed in one transaction, instead of one by one. This has the benefint of those operations being atomically applied and much faster as well due to less http requests and also less transactional overhead (commiting each transaction has some overhead otherwise). Maybe someone could supply a good link for further information since I don't really have any. > > Jim > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Rest API in the future
Hello Aniceto, > - EmbeddedGraphDatabase, which is good for tests, supports transactions > and has quick communications. Not good for medium sized or HA apps > because lacks separation between app and data The EmbeddedGraphDatabase is fine for production use - in fact it's the default choice! The difference between using an EmbeddedGraphDatabase and the REST API is where the instance of EmbeddedGraphDatabase resides. If you use the REST API, it lives in our server process, otherwise it lives in your process. Really the distinction is much less than that names imply. > - standalone server: data repository is set apart from application, even > more than one app can access it. If EE, HA is available, but has a REST > interface which does not support transactions It has transactions of course - Neo4j is *always* ACID transactional. But this interface does not expose those transactions to users. Instead a transaction is implicitly wrapped around every interaction. > Can we expect any new communication interface with TX and HA support for > standalone servers, at least for Java/JVM clients or has the roadmap > another target? HA is supported whether you choose to use Neo4j embedded within your process or whether its embedded within our process inside the REST API. We're starting to think about next generation protocol support. That will probably include transactional control on the client side, but we have nothing to release in the 1.4 timeframe. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4J in Rails using C-Ruby
Hello, I plan to use Neo4J for a Ruby on Rails web app. However, I have not the possibility to use JRuby. Due to doubts of performance issues using the REST API, I'm considering to use the embedded graph database with means of the Ruby Java Bridge (RJB). The Ruby app then would talk to the JVM as a regular Java client. The problem is: Rails does not have an application scope like Java web frameworks or ASP.Net. However, this is (to my understanding) essential to hold only one instance of the EmbeddedGraphDatabase class because I've read that it's not possible to have multiple instances of this class at the same time. I can't use the singleton pattern because when using fastcgi or several Mongrel server instances each server would create its own singelton instance, resulting in multiple EmbeddedGraphDatabase instances. My question is: How is this problem solved in the JRuby-Neo4J implementation? How can several server processes share one instance of the EmbeddedGraphDatabase when using JRuby with Rails? Thanks so much in advance, Mathias ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Rest API in the future
I have downloaded neo4-rest-graphdb, a rest wrapper for Java. README file says this interface does not support transactions. So we have: - EmbeddedGraphDatabase, which is good for tests, supports transactions and has quick communications. Not good for medium sized or HA apps because lacks separation between app and data - standalone server: data repository is set apart from application, even more than one app can access it. If EE, HA is available, but has a REST interface which does not support transactions Can we expect any new communication interface with TX and HA support for standalone servers, at least for Java/JVM clients or has the roadmap another target? -- Aniceto ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
