Re: [Neo4j] neo4j spatial - dynamic styling
Hi David, I think the problem on my side was simply lack of experience with SLD's. Since FOSS4G I've not spent any time on SLD's either, but instead on some updates to the OSM data model. However, I'm very keen to get the SLD work moving again, so right now I'm trying to turn your test code into a unit test in neo4j-spatial so I can start playing with it, and then integrate it with some code I got from Jesse for embedding the SLD in a uDig datastore so when we connect uDig to OSM in Neo4j it automatically gets great rendering :-) One question for you: the code has new dependencies for org.geotools.map and org.geotools.render. Where can I find these? A quick google search said org.geotools.map was found in gt-build, but that does not seem to exist. I'll look a little further, but if you have a quick answer, that would be great. And you last point about performance and style composition are very important. I'm assuming that the more advanced the styling we apply the slower the rendering, so I've pretty sure I'm going to be getting into deep water there and will certainly benefit from the advice of a real expert in SLD's. Of course, my plan has been to start by using the mapnik2geotools generated SLD's as a starting point, since I'm sure you did a great job with that :-) Regards, Craig On Thu, Oct 7, 2010 at 4:03 AM, David Winslow cdwins...@gmail.com wrote: Hi all, My name's David Winslow and I talked to Craig Taverner a few weeks ago at FOSS4G about Neo4j-spatial. I work with GeoTools and GeoServer and was interested in the demo I saw of OSM data stored and rendered from a Neo4j database instead of one of the more traditional storage systems. We also talked a bit about problems with styling in GeoTools. So, I played around with neo4j-spatial a bit last week to see what was going wrong and styling seems to work ok with the layers I tried. What I did was to run just the TestDynamicLayers suite and then write a small program to load it in GeoTools and render to PNG. http://gist.github.com/614421 is the precise code I used. Since I didn't run into any problems actually doing the styling, I guess the issues we discussed are a bit higher-level (performance, style composition, that kind of thing). Does anyone have the time to explain these problems in a bit more detail? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j spatial - dynamic styling
I found the correct geotools jar. It is gt-render. On Thu, Oct 7, 2010 at 10:18 AM, Craig Taverner cr...@amanzi.com wrote: Hi David, I think the problem on my side was simply lack of experience with SLD's. Since FOSS4G I've not spent any time on SLD's either, but instead on some updates to the OSM data model. However, I'm very keen to get the SLD work moving again, so right now I'm trying to turn your test code into a unit test in neo4j-spatial so I can start playing with it, and then integrate it with some code I got from Jesse for embedding the SLD in a uDig datastore so when we connect uDig to OSM in Neo4j it automatically gets great rendering :-) One question for you: the code has new dependencies for org.geotools.map and org.geotools.render. Where can I find these? A quick google search said org.geotools.map was found in gt-build, but that does not seem to exist. I'll look a little further, but if you have a quick answer, that would be great. And you last point about performance and style composition are very important. I'm assuming that the more advanced the styling we apply the slower the rendering, so I've pretty sure I'm going to be getting into deep water there and will certainly benefit from the advice of a real expert in SLD's. Of course, my plan has been to start by using the mapnik2geotools generated SLD's as a starting point, since I'm sure you did a great job with that :-) Regards, Craig On Thu, Oct 7, 2010 at 4:03 AM, David Winslow cdwins...@gmail.com wrote: Hi all, My name's David Winslow and I talked to Craig Taverner a few weeks ago at FOSS4G about Neo4j-spatial. I work with GeoTools and GeoServer and was interested in the demo I saw of OSM data stored and rendered from a Neo4j database instead of one of the more traditional storage systems. We also talked a bit about problems with styling in GeoTools. So, I played around with neo4j-spatial a bit last week to see what was going wrong and styling seems to work ok with the layers I tried. What I did was to run just the TestDynamicLayers suite and then write a small program to load it in GeoTools and render to PNG. http://gist.github.com/614421 is the precise code I used. Since I didn't run into any problems actually doing the styling, I guess the issues we discussed are a bit higher-level (performance, style composition, that kind of thing). Does anyone have the time to explain these problems in a bit more detail? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j spatial - dynamic styling
The most important thing you can do to improve performance is to ensure that your style renders an appropriate number of features (if you want render every driveway at state level it will take a while) and that your datastore can provide fast access to those features (ie, well-indexed). Beyond that, most styling options have negligible impact on rendering speed. (However, with as many different style rules as you have with OSM data things will start to add up.) wrt to mapnik2geotools - Mapnik allows styles to specify a postgres * subquery* in the table name field (so generated requests come out like select foo from (select foo, bar, baz from quux as roads) which is totally valid). GeoTools doesn't support this option, so my approach so far has been to inspect the datastore parameters and for every table that looks like a subquery, I generate an actual table - create table roads as (select foo, bar, baz from quux). While expedient, this approach doesn't translate well to other datastores. So, I think the first barrier to using mapnik2geotools with neo4j-spatial will be coming up with a solution to this problem that works with neo4j as the database. -- David Winslow On Thu, Oct 7, 2010 at 4:47 AM, Craig Taverner cr...@amanzi.com wrote: I found the correct geotools jar. It is gt-render. On Thu, Oct 7, 2010 at 10:18 AM, Craig Taverner cr...@amanzi.com wrote: Hi David, I think the problem on my side was simply lack of experience with SLD's. Since FOSS4G I've not spent any time on SLD's either, but instead on some updates to the OSM data model. However, I'm very keen to get the SLD work moving again, so right now I'm trying to turn your test code into a unit test in neo4j-spatial so I can start playing with it, and then integrate it with some code I got from Jesse for embedding the SLD in a uDig datastore so when we connect uDig to OSM in Neo4j it automatically gets great rendering :-) One question for you: the code has new dependencies for org.geotools.map and org.geotools.render. Where can I find these? A quick google search said org.geotools.map was found in gt-build, but that does not seem to exist. I'll look a little further, but if you have a quick answer, that would be great. And you last point about performance and style composition are very important. I'm assuming that the more advanced the styling we apply the slower the rendering, so I've pretty sure I'm going to be getting into deep water there and will certainly benefit from the advice of a real expert in SLD's. Of course, my plan has been to start by using the mapnik2geotools generated SLD's as a starting point, since I'm sure you did a great job with that :-) Regards, Craig On Thu, Oct 7, 2010 at 4:03 AM, David Winslow cdwins...@gmail.com wrote: Hi all, My name's David Winslow and I talked to Craig Taverner a few weeks ago at FOSS4G about Neo4j-spatial. I work with GeoTools and GeoServer and was interested in the demo I saw of OSM data stored and rendered from a Neo4j database instead of one of the more traditional storage systems. We also talked a bit about problems with styling in GeoTools. So, I played around with neo4j-spatial a bit last week to see what was going wrong and styling seems to work ok with the layers I tried. What I did was to run just the TestDynamicLayers suite and then write a small program to load it in GeoTools and render to PNG. http://gist.github.com/614421 is the precise code I used. Since I didn't run into any problems actually doing the styling, I guess the issues we discussed are a bit higher-level (performance, style composition, that kind of thing). Does anyone have the time to explain these problems in a bit more detail? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j spatial - dynamic styling
You may also find the styling section in the GeoServer manual useful while
getting up to speed on SLD:
http://docs.geoserver.org/stable/en/user/styling/index.html
wrt filtering out small features, I think that is generally done in the
stylesheet (GeoTools combines the filters of all the style rules together
and sends them to the database so that GeoTools fetches only what will
actually be rendered from the database.) This avoids having a hard-coded
decision about the size of the important features in the rendering code,
which as you note does not work very well for point data.
Is there documentation about the JSON query language beyond what is
available in the JavaDoc for DynamicLayer? In the short term, I am mainly
curious about (not) NULL tests, OR, and testing for specific values of
properties since these are the most used predicates in the MapQuest styles;
for example:
CREATE TABLE landmark AS
SELECT way, aeroway, amenity, landuse, leisure,
man_made, military, natural, power, shop, tourism, name,
religion
FROM planet_osm_polygon
WHERE landuse is not null or leisure is not null
or shop is not null
or aeroway in ('apron', 'aerodrome', 'taxiway', 'runway')
or amenity in ('parking', 'university', 'college', 'school',
'hospital',
'kindergarten', 'grave_yard')
or military in ('barracks', 'danger_area')
or natural in ('field', 'beach', 'heath', 'mud', 'wood')
or power in ('station', 'sub_station')
or tourism in ('attraction', 'camp_site', 'caravan_site',
'picnic_site', '
ORDER BY z_order, way_area DESC;
--
David Winslow
On Thu, Oct 7, 2010 at 3:52 PM, Craig Taverner cr...@amanzi.com wrote:
Hi David,
The most important thing you can do to improve performance is to ensure
that
your style renders an appropriate number of features (if you want
render
every driveway at state level it will take a while) and that your
datastore
can provide fast access to those features (ie, well-indexed). Beyond
that,
most styling options have negligible impact on rendering speed. (However,
with as many different style rules as you have with OSM data things will
start to add up.)
Seems like a useful option is for the DataStore to only return geometries
that are 'big enough' to matter at the resolution in question. This could
be
a clever modification to the spatial index itself, so that it is passed a
resolution hint during query time and simply ignores data that is 'too
small'. The concept of 'too small' is the tricky part, because it is not a
purely spatial concept. Some points are too small, while others are not,
depending on the tags. Points representing the location of cities are not
too small, while points representing the location of trees certainly. Of
course this can also be solved client side by putting such different
objects
in different layers and setting zoom level filters on the entire layer.
wrt to mapnik2geotools - Mapnik allows styles to specify a postgres *
subquery* in the table name field (so generated requests come out like
select
foo from (select foo, bar, baz from quux as roads) which is totally
valid).
GeoTools doesn't support this option, so my approach so far has been to
inspect the datastore parameters and for every table that looks like a
subquery, I generate an actual table - create table roads as (select
foo,
bar, baz from quux). While expedient, this approach doesn't translate
well
to other datastores.
So, I think the first barrier to using mapnik2geotools with neo4j-spatial
will be coming up with a solution to this problem that works with neo4j
as
the database.
My plan is for the DynamicLayers to support this kind of thing. You could
view these as database views, which is similar to your idea of generating
an
actual table, minus the data duplication, of course. Right now my dynamic
layers only filter on combinations of OSM tags, but there is not reason why
not to filter on much more complex things. Today the filter is, in fact, a
graph matching algorithms, specified in JSON. The JSON document describes
how to traverse the graph checking for attributes on various nodes. The
original reason for such a complex solution was because the OSM tags node
is
two steps away from the geometry node. However, now that it exists I can
hopefully use it for much more powerful queries of sub-graphs.
By the way, I added a modified version of your code to a unit test in
neo4j-spatial (not yet commited, but soon), and also played around with the
SLD to see if I could get some more interesting effects. I manage to
generate the attached image, which has three levels of road rendered with
different colors, all with dark borders. The bridges are highlighted, as
you
had, but now also with glowing red labels. I got the labeling from the SLD
tutorial on the geoserver wiki at
Re: [Neo4j] neo4j spatial - dynamic styling
So far the only docs for the JSON syntax are in the code. But you are
welcome to read the JSON parsing code itself. It is very short, albeit
cryptic.
I'm sure I support 'not null', but I am also sure I do not support OR. I
need to think a bit harder about how to support OR, since the current
compact code was designed around supporting AND. It should be possible. But
it is 1am now, and I definitely will not figure that out tonight ;-)
On Fri, Oct 8, 2010 at 1:18 AM, David Winslow cdwins...@gmail.com wrote:
You may also find the styling section in the GeoServer manual useful while
getting up to speed on SLD:
http://docs.geoserver.org/stable/en/user/styling/index.html
wrt filtering out small features, I think that is generally done in the
stylesheet (GeoTools combines the filters of all the style rules together
and sends them to the database so that GeoTools fetches only what will
actually be rendered from the database.) This avoids having a hard-coded
decision about the size of the important features in the rendering code,
which as you note does not work very well for point data.
Is there documentation about the JSON query language beyond what is
available in the JavaDoc for DynamicLayer? In the short term, I am mainly
curious about (not) NULL tests, OR, and testing for specific values of
properties since these are the most used predicates in the MapQuest styles;
for example:
CREATE TABLE landmark AS
SELECT way, aeroway, amenity, landuse, leisure,
man_made, military, natural, power, shop, tourism, name,
religion
FROM planet_osm_polygon
WHERE landuse is not null or leisure is not null
or shop is not null
or aeroway in ('apron', 'aerodrome', 'taxiway', 'runway')
or amenity in ('parking', 'university', 'college', 'school',
'hospital',
'kindergarten', 'grave_yard')
or military in ('barracks', 'danger_area')
or natural in ('field', 'beach', 'heath', 'mud', 'wood')
or power in ('station', 'sub_station')
or tourism in ('attraction', 'camp_site', 'caravan_site',
'picnic_site', '
ORDER BY z_order, way_area DESC;
--
David Winslow
On Thu, Oct 7, 2010 at 3:52 PM, Craig Taverner cr...@amanzi.com wrote:
Hi David,
The most important thing you can do to improve performance is to ensure
that
your style renders an appropriate number of features (if you want
render
every driveway at state level it will take a while) and that your
datastore
can provide fast access to those features (ie, well-indexed). Beyond
that,
most styling options have negligible impact on rendering speed.
(However,
with as many different style rules as you have with OSM data things
will
start to add up.)
Seems like a useful option is for the DataStore to only return geometries
that are 'big enough' to matter at the resolution in question. This could
be
a clever modification to the spatial index itself, so that it is passed a
resolution hint during query time and simply ignores data that is 'too
small'. The concept of 'too small' is the tricky part, because it is not
a
purely spatial concept. Some points are too small, while others are not,
depending on the tags. Points representing the location of cities are not
too small, while points representing the location of trees certainly. Of
course this can also be solved client side by putting such different
objects
in different layers and setting zoom level filters on the entire layer.
wrt to mapnik2geotools - Mapnik allows styles to specify a postgres *
subquery* in the table name field (so generated requests come out like
select
foo from (select foo, bar, baz from quux as roads) which is totally
valid).
GeoTools doesn't support this option, so my approach so far has been
to
inspect the datastore parameters and for every table that looks like a
subquery, I generate an actual table - create table roads as (select
foo,
bar, baz from quux). While expedient, this approach doesn't translate
well
to other datastores.
So, I think the first barrier to using mapnik2geotools with
neo4j-spatial
will be coming up with a solution to this problem that works with neo4j
as
the database.
My plan is for the DynamicLayers to support this kind of thing. You could
view these as database views, which is similar to your idea of generating
an
actual table, minus the data duplication, of course. Right now my dynamic
layers only filter on combinations of OSM tags, but there is not reason
why
not to filter on much more complex things. Today the filter is, in fact,
a
graph matching algorithms, specified in JSON. The JSON document describes
how to traverse the graph checking for attributes on various nodes. The
original reason for such a complex solution was because the OSM tags node
is
two steps away from the geometry node. However,
[Neo4j] neo4j spatial - dynamic styling
Hi all, My name's David Winslow and I talked to Craig Taverner a few weeks ago at FOSS4G about Neo4j-spatial. I work with GeoTools and GeoServer and was interested in the demo I saw of OSM data stored and rendered from a Neo4j database instead of one of the more traditional storage systems. We also talked a bit about problems with styling in GeoTools. So, I played around with neo4j-spatial a bit last week to see what was going wrong and styling seems to work ok with the layers I tried. What I did was to run just the TestDynamicLayers suite and then write a small program to load it in GeoTools and render to PNG. http://gist.github.com/614421 is the precise code I used. Since I didn't run into any problems actually doing the styling, I guess the issues we discussed are a bit higher-level (performance, style composition, that kind of thing). Does anyone have the time to explain these problems in a bit more detail? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
