Hi Craig,
I use my own id-mapping data structure (hashmap). What you said about
lucene is interesting, becuase yesterday I thought about doing so.
What I do now, first I create the nodes for point geometries, then I start
creating polygon nodes, extract the list of edges for each polygon, create
node for each edge, connect polygon to its edges using neo4j relationships
and connect edges to their corresponding points' nodes.
I thought maybe a top-down approach works better but I need to try it:
i.e., create polygon nodes, extract edges and create edge node, connect
polygon to edges,
create points and connect edges to points.
Any opinion?
Thanks,
Alireza
On Tuesday, October 14, 2014 12:17:07 PM UTC+2, Craig Taverner wrote:
>
> Hi Alireza,
>
> Sounds like you need a unique key to connect vertices during the edges
> import. You called this the 'vertix id'. We had the same problem importing
> OSM data in the OSMImporter class. In that code we used the lucene index to
> store the osm-id for the vertices. This works fine for moderate sized data,
> but does not scale that well. But it should be OK for 20000 vertices. Are
> you using lucene? Or some other way to lookup vertices by id?
>
> My vote for data as small as 20k vertices is to make your own id-mapping
> structure. For example, if the veritix id is in a well known range (eg.
> 0..20000) then make an array where the array index is the vertice id and
> the value is the neo4j node-id. If it is not in a well known range, use a
> HashMap. Both will be faster than lucene for small data sizes, but will not
> scale obviously.
>
> Regards, Craig
>
> On Tue, Oct 14, 2014 at 10:34 AM, Alireza Rezaei Mahdiraji <
> alire...@gmail.com <javascript:>> wrote:
>
>>
>> Hi Craig,
>>
>>
>> What would be the best way to import such data into Neo4j.
>> The sample file that I have first list all vertices with their
>> coordinates and then
>> all polygons with the list of their corresponding vertex id list.
>> The edges are inferred from this two list.
>>
>> My current script is extremely slow for verices around 20000 and polygon
>> around 35000.
>>
>> Thanks,
>> Alireza
>>
>> On Tuesday, September 9, 2014 1:23:44 PM UTC+2, Craig Taverner wrote:
>>>
>>> Hi Alireza,
>>>
>>> I opened your graph in the neo4j server and looked around, and then also
>>> write some Java test code (based on your snippet) to see what I could see.
>>> The main conclusion is that I see two nodes (not one) that are in the
>>> envelope, nodes with n.id=2 and n.id=10.
>>>
>>> I've attached screenshots of the Neo4j browser looking at the data, a
>>> graph view showing the nodes and the spatial layer connected to them, and a
>>> cypher query that lists the key properties. The cypher query I used was:
>>>
>>> MATCH (n) WHERE has(n.gtype) RETURN id(n), n.id, n.gtype, n.x, n.y,
>>> n.bbox, n.dim, n.eid
>>>
>>>
>>> Some interesting points I can see:
>>>
>>> - There are definitely two nodes with x=1, y=1 which should be
>>> inside Envelope(0.5,1.5,0.5,1.5).
>>> - The use of a property 'id' is allowed, but a little confusing
>>> because it is easy to think that might be the node id (see that I'm
>>> listing
>>> the node id and the node property 'id' just to be clear)
>>> - I'm surprised by the fact that there are three different gtype
>>> values: 1, 2, 3, which mean POINT, LINESTRING, POLYGON respectively, but
>>> clearly these are all points (they have only x,y information). This
>>> means
>>> that these objects were not created by the Neo4j Spatial library, but
>>> created by some other code? How did you create them? I suspect there is
>>> something not quite right there.
>>> - The BBox values for POINT objects should be points themselves (ie.
>>> (x1,y1) == (x2,y2)). And in fact I see for all objects with gtype==1,
>>> this
>>> is true. For other objects, you have bigger bounding boxes, which makes
>>> sense for the type, but since they are actually only points, it no
>>> longer
>>> makes sense.
>>> - When I look into the layer node, I see that the GeometryEncoder is
>>> the SimplePointEncoder, consistent with the observation that all data
>>> are
>>> x,y pairs (Points)
>>> - But I see the layer implementation is the EditableLayerImpl, which
>>> is more often used for WKT Geometries (allowing LineString and Polygon
>>> also).
>>>
>>> The test code that I wrote was in Java and based on your sample. The
>>> main part of the code looks like this:
>>> SpatialDatabaseService spatial = new SpatialDatabaseService(graphDb);
>>> Layer layer = spatial.getLayer(layerName);
>>> GeometryFactory gf = layer.getGeometryFactory();
>>> Envelope envelope = new Envelope(0.5, 1.5, 0.5, 1.5);
>>> Geometry geom = gf.toGeometry(envelope);
>>> try (Transaction tx = graphDb.beginTx()) {
>>> List<Node> nodes = GeoPipeline.startWithinSearch(layer,
>>> geom).toNodeList();
>>> for (Node node : nodes) {
>>> System.out.println("Node[" + node.getId() + ":" + node.getProperty("id")
>>> + "]: " + nodeXY(node) + " - "
>>> + nodeBBox(node));
>>> }
>>> assertEquals("Only one node should be in envelope", nodes.size(), 2);
>>> tx.success();
>>> }
>>>
>>> This test passes on the two nodes that look like Point(1,1) as expected.
>>> However, since you probably meant the second one to be a LineString
>>> (gtype=2), it probably was supposed to intersect the envelope, not be
>>> included. But since you use the SimplePointEncoder, it will be seen as a
>>> point at 1,1, so it will be included.
>>>
>>> My understanding is that you created the data incorrectly, using a
>>> SimplePointEncoder for data that you meant to be LineString and Polygon. I
>>> suggest that you rather use the WKT encoder. Do not create the nodes
>>> yourself, but use the layer.add(Geometry) to create them for you, and then
>>> add your own properties to them afterward, so you are sure the geoemtry
>>> information is correctly created.
>>>
>>> Regards, Craig
>>>
>>>
>>>
>>> On Thu, Sep 4, 2014 at 1:03 PM, Alireza Rezaei Mahdiraji <
>>> alire...@gmail.com> wrote:
>>>
>>>>
>>>> Hi Craig,
>>>>
>>>> I attached the neo4j database folder for the toy example I am using.
>>>>
>>>> The envelop I am using is as follow: xmin: 0.5, xmax: 1.5, ymin: -0.5,
>>>> ymax: 1.5
>>>>
>>>> This envelop only contains one graph vertex fully, the vertex with x=1
>>>> and y=1
>>>> but code returns several other results too which are partially
>>>> contained by the envelop.
>>>>
>>>> Thanks,
>>>> Best,
>>>> Alireza
>>>>
>>>> On Wednesday, September 3, 2014 5:08:36 PM UTC+2, Craig Taverner wrote:
>>>>>
>>>>> Hi Alireza,
>>>>>
>>>>> That code looks OK. Could you send me some sample data? I could try
>>>>> this myself and see what the problem is. The code underneath is using the
>>>>> JTS function Geometry.within(Geometry) and that is strictly defined. See
>>>>> http://tsusiatsoftware.net/jts/javadoc/com/vividsolutions/jts/geom/
>>>>> Geometry.html#within(com.vividsolutions.jts.geom.Geometry).
>>>>>
>>>>> But if I try your data I could double-check if there is perhaps an
>>>>> issue in how we use this.
>>>>>
>>>>> Regards, Craig
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Sep 3, 2014 at 3:15 PM, Alireza Rezaei Mahdiraji <
>>>>> alire...@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> I tried this snippet:
>>>>>>
>>>>>> GeometryFactory gf = layer.getGeometryFactory();
>>>>>> Geometry geom = gf.toGeometry(envelope);
>>>>>> List<Node> nodes = GeoPipeline.startWithinSearch(layer,
>>>>>> geom).toNodeList();
>>>>>>
>>>>>> but the result still contains nodes which are not fully inside the
>>>>>> envelope.
>>>>>>
>>>>>> What am I missing?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Alireza
>>>>>>
>>>>>>
>>>>>> On Tuesday, September 2, 2014 10:35:38 AM UTC+2, Craig Taverner wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I can see two ways in the Java API to do this. One is the 'within'
>>>>>>> filter in the GeoPipeline.
>>>>>>>
>>>>>>> - See example usage in the test code at
>>>>>>> - See the implementation using JTS in the code at
>>>>>>> https://github.com/neo4j-contrib/spatial/blob/master/src/mai
>>>>>>> n/java/org/neo4j/gis/spatial/pipes/filtering/FilterWithin.
>>>>>>> java#L47
>>>>>>>
>>>>>>> <https://github.com/neo4j-contrib/spatial/blob/master/src/main/java/org/neo4j/gis/spatial/pipes/filtering/FilterWithin.java#L47>
>>>>>>>
>>>>>>>
>>>>>>> And the other is the older, but still functional use of CQL for
>>>>>>> within queries:
>>>>>>>
>>>>>>> - See an example using dynamic layers at
>>>>>>> https://github.com/neo4j-contrib/spatial/blob/master/src/
>>>>>>> test/java/org/neo4j/gis/spatial/TestDynamicLayers.java#L138
>>>>>>>
>>>>>>> <https://github.com/neo4j-contrib/spatial/blob/master/src/test/java/org/neo4j/gis/spatial/TestDynamicLayers.java#L138>
>>>>>>>
>>>>>>> - And the implementation at https://github.com/neo4j-co
>>>>>>> ntrib/spatial/blob/master/src/main/java/org/neo4j/gis/spatia
>>>>>>> l/filter/SearchCQL.java#L56
>>>>>>>
>>>>>>> <https://github.com/neo4j-contrib/spatial/blob/master/src/main/java/org/neo4j/gis/spatial/filter/SearchCQL.java#L56>
>>>>>>>
>>>>>>> A direct usage of the SearchCQL class is also possible, but I did
>>>>>>> not see a test case for it. It was used by the GeoServer integration
>>>>>>> though.
>>>>>>>
>>>>>>> Regards, Craig
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Sep 1, 2014 at 5:15 PM, Alireza Rezaei Mahdiraji <
>>>>>>> alire...@gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I would like to find nodes of the graph which are completely (not
>>>>>>>> partially) contained in a given
>>>>>>>> Envelope. I tried several GeoPipeline methods but it seems they
>>>>>>>> all consider partial containment.
>>>>>>>> Any idea?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Best,
>>>>>>>> Alireza
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Neo4j" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to neo4j+un...@googlegroups.com.
>>>>>>>>
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Neo4j" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to neo4j+un...@googlegroups.com.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Neo4j" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to neo4j+un...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Neo4j" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to neo4j+un...@googlegroups.com <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to neo4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
