On Tue, Feb 10, 2015 at 8:23 PM, Rich Fecher <rfec...@gmail.com> wrote:
> Yeah, so on the GeoWave project we started out with a simple sub-sampling
> approach that used a render transform to pass the pixel dimensions of the
> map request to our data store so that we could skip within our tablet
> servers at the equivalent of a pixel resolution on our space filling
> curve. This works well for basic point data but for polygons and lines -
> data with more complex stylization that does not well map what is actually
> rendered to the gridded representation on our space filling curve, we take
> advantage of actually doing the rendering within the tablet servers.
>
Hi,
so I've looked a bit into the code you have setup and tried to get a grip
on it.
Here is what I've gathered so far.
You have setup a modified renderer that has a "distributed rendering mode".
This is not really a different mode of operation of its own, but more of a
way to offload the
entire rendering process of a layer to the underlying store.
I see you basically setup a helper object that contains all the information
needed to
render the layer, styles, all rendering options, and so on, and you put
such object into
a query hint that is given to the store.
The store can handle this special hint, and in case it's found, it will
take the object
in the hint, and distribute it to all the nodes in your cluster, using a
typical distributed
processing pattern, where you move your processing algorithm towards your
data,
which is already suitably split among the various nodes.
Each node computes a RenderedImage using a local StreamingRenderer (which in
your case, it's really the object you put in the hint) and sends it back to
the store, which in turn
generates "fake" features with a special attribute containing one of images
we
got back from the nodes, the modified renderer will recognize them and
basically just
paint this fake feature collection as a set of images to blend on top of
the current map.
This sort of follows along existing ideas that stores willing to optimize
rendering
can actively participate in some steps of it, right now we have hints to
offload
geometry generalization
(Hints.GEOMETRY_DISTANCE/GEOMETRY_GENERALIZATION/GEOMETRY_SIMPLIFICATION)
and avoid repeated pixel fill at the store level (Hints.SCREENMAP).
I guess we could add a similar hint to GeoTools, although not the way
you've done it:
the hint is actually another StreamingRenderer clone that can be serialized
and send over
to the various nodes, I'd keep it purely descriptive instead, something
like (by broad strokes):
LayerRenderingContext {
ReferencedEnvelope renderingArea;
Rectangle paintArea;
RenderingHints java2dHints;
Map rendererHints;
List<FeatureTypeStyle> styles;
}
and it would be up to your store to eventually wrap it into a "active"
object, distribute it,
and return the images.
So this is one possible approach.... which makes me a bit nervous in that
there would be
no implementation in GeoTools to handle it, so it could get inadvertedly
broken over time,
and we would not notice.
There is another possible one, which is the DirectLayer approach Jody
mentioned,
and I believe this would work with little/no modifications.
Now, I understand you want to associate a normal store with a SLD style,
which
might be coming from your WMS client, but that per se is not a problem, in
your GeoServe plugin you could implement a org.geoserver.wms.GetMapCallback,
which gets notified of all GetMap request and its allowed to modify the
MapContent
before passing it down to the StreamingRenderer.
So, right there, you could build your own DirectLayer implementation that
uses
your store facilities for distributed rendering, and would have access to
the current style,
put it in the same place as the old layer using your store,
and as a result doing the distributed rendering without touching one bit of
StreamingRenderer.
One thing you'd miss would be the StreamingRenderer rendering hints, but I
guess
we could add those by modifying a bit the DirectLayer interface (not sure
if there
is anything significant using it nowadays).
Now, in another mail you also said that the nodes have a ScreenMap like
approach,
and that you stop reading the data once the current tile is full. ScreenMap
at the
moment can only tell you if a certain pixel is full, but we could easily
modify it to
add a full pixel counter and thus tell you if the target image is already
full, thus
getting the same optimization.
Jody, looking at this, I don't however see much of anything that could be
useful for
multithreaded/parallel rendering though.
Cheers
Andrea
PS: in any case we might have a problems with labelling, if the style has
any
label the distributed rendering thing cannot be applied... I guess your
GetMapCallback
could split the style in two parts and create two layers, a traditional one
for layers, and a distributed one for everything else.
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
-------------------------------------------------------
------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
