Re: [Geotools-devel] ImageMosaic API refactor proposal

[Jody Garnett]

Properly collectors are still used, think we are getting a small echo
chamber of uncertainty here. This is a refactor to use existing
infrastructure such as property accessors more effectively.

Importer is not quite the right place as additional granules can be added
over time via rest API right? And this is a GeoTools proposal 

On Tue, Jun 7, 2016 at 9:30 AM Daniele Romagnoli <
daniele.romagn...@geo-solutions.it> wrote:

> Hi,
> Providing some additional feedbacks on my side, too.
>
>
> On Mon, Jun 6, 2016 at 6:34 PM, Andrea Aime 
> wrote:
>
>> On Fri, Jun 3, 2016 at 1:19 AM, Devon Tucker 
>> wrote:
>>
>>> Here are a few things that might help understanding, section by section:

- *Description*: this is fine, makes sense, no questions (one
thing, the recursive file loading during indexing is already the default
behavior as far as I know)
- *Index Generation: *quite a bit lost here, how are
MosaicIndexConfiguration and GranuleCatalogManager going to be used by 
 the
existing classes? Or are they replacing it? I'd need to understand how 
 the
existing code is going to get reshuffled into a generic machiner plus
default implementations of the two above objects? Also, can you provide 
 an
example of an alternate implementation (being the refactor targeted to
extensibility)
- *Harvesting: *same as above, how is the code going to be moved
around, and examples of alternate implementations


>>> I hope to clarify these parts soon. To be honest, these parts were done
>>> earlier in the project and we've zig-zagged multiple times since then, so
>>> even I need to revisit to solidify and clarify this section. To give some
>>> perspective on the original motivations behind this part, at the time we
>>> were:
>>>
>>> - Performing some pre-processing on rasters before indexing them (adding
>>> overlays, creating alpha bands, generating elevation outlier footprints)
>>>
>>
>> Image mosaic looks like the wrong place to do this, the importer would be
>> much better (and already does some of those). See below for more
>> observations regarding this.
>>
> I would agree with Andrea's feedbacks on the pre-processing topic.
>
>
>>
>>
>>> - Collecting custom properties from rasters during indexing (resolution,
>>> date stamps, various tiff metadata, sample date, etc.)
>>>
>> Where this information will be stored? In the mentioned
> GranuleDescriptor's properties map? Shouldn't that achieved by
> PropertyCollector?
>
>>
>> This is done by property collectors, why not just extend that?
>>
>>
>>> - Looking at harvesting granules on demand either via. rest or the
>>> geoserver UI.
>>>
>>
>> The REST api already does that. There is no UI for it though.
>>
> Some more information about REST support for harvesting is available here
> with some examples:
>
> http://docs.geoserver.org/stable/en/user/rest/examples/curl.html#uploading-and-modifying-a-image-mosaic
>
>
>>
>>> - Exposing more of this configuration through a custom store and layer
>>> config in GeoServer.
>>>
>>
>> This can be arranged too, not sure it needs a refactor for that.
>>
>>
>>> - Index generation code is very isolated as it stands. It's tough to
>>> programmatically configure index generation.
>>>
>>
>> Ok, this one makes sense... the others do not seem to actually require a
>> refactor though.
>>
>>
>>>
>>>

- *Delegate coverage acceptance/rejection to a predicate object*:
makes sense I guess, so the reason to have this plugable is because you
might roll a collector that has less limitations than the default ones?

 Exactly.
>>>
>>
>> Ok but... why not keep the improved collectors in the mosaic instead?
>> They would benefit every user there (instead of just the gt-dem users)
>>
>>
>>>
- *Pre-process Granule Footprint Before Indexing*: makes sense,
thinking out loud is the footprint the only thing that needs 
 reprocessing?

 No, it's not really the only thing that need reprocessing. In fact
>>> PropertyCollectors have a tangentially related responsibility (and are used
>>> in roughly the same spot). Maybe this functionality could be combined
>>> somehow, and make monolithic some sort of monolithic visitor which would be
>>> responsible for:
>>>
>>> - Collecting properties from granules
>>> - Pre-processing the granule footprint if needed
>>> - Populating the index feature
>>>
>>> And so on. I'll have to think about this one a bit more, but as the
>>> proposal stands at least FootprintProcessor has a very clear responsibility.
>>>
>>
>>
>
>> See above about the usage of the importer. The indexing happens during
>> layer configuration in the UI, expensive operations are best done during
>> pre-processing.
>> Normally what you are describing here is done as part of a "ingest"
>> machinery, which is a off-line ETL of sorts 

Re: [Geotools-devel] ImageMosaic API refactor proposal

[Daniele Romagnoli]

Hi,
Providing some additional feedbacks on my side, too.


On Mon, Jun 6, 2016 at 6:34 PM, Andrea Aime 
wrote:

> On Fri, Jun 3, 2016 at 1:19 AM, Devon Tucker 
> wrote:
>
>> Here are a few things that might help understanding, section by section:
>>>
>>>- *Description*: this is fine, makes sense, no questions (one thing,
>>>the recursive file loading during indexing is already the default 
>>> behavior
>>>as far as I know)
>>>- *Index Generation: *quite a bit lost here, how are
>>>MosaicIndexConfiguration and GranuleCatalogManager going to be used by 
>>> the
>>>existing classes? Or are they replacing it? I'd need to understand how 
>>> the
>>>existing code is going to get reshuffled into a generic machiner plus
>>>default implementations of the two above objects? Also, can you provide 
>>> an
>>>example of an alternate implementation (being the refactor targeted to
>>>extensibility)
>>>- *Harvesting: *same as above, how is the code going to be moved
>>>around, and examples of alternate implementations
>>>
>>>
>> I hope to clarify these parts soon. To be honest, these parts were done
>> earlier in the project and we've zig-zagged multiple times since then, so
>> even I need to revisit to solidify and clarify this section. To give some
>> perspective on the original motivations behind this part, at the time we
>> were:
>>
>> - Performing some pre-processing on rasters before indexing them (adding
>> overlays, creating alpha bands, generating elevation outlier footprints)
>>
>
> Image mosaic looks like the wrong place to do this, the importer would be
> much better (and already does some of those). See below for more
> observations regarding this.
>
I would agree with Andrea's feedbacks on the pre-processing topic.


>
>
>> - Collecting custom properties from rasters during indexing (resolution,
>> date stamps, various tiff metadata, sample date, etc.)
>>
> Where this information will be stored? In the mentioned
GranuleDescriptor's properties map? Shouldn't that achieved by
PropertyCollector?

>
> This is done by property collectors, why not just extend that?
>
>
>> - Looking at harvesting granules on demand either via. rest or the
>> geoserver UI.
>>
>
> The REST api already does that. There is no UI for it though.
>
Some more information about REST support for harvesting is available here
with some examples:
http://docs.geoserver.org/stable/en/user/rest/examples/curl.html#uploading-and-modifying-a-image-mosaic


>
>> - Exposing more of this configuration through a custom store and layer
>> config in GeoServer.
>>
>
> This can be arranged too, not sure it needs a refactor for that.
>
>
>> - Index generation code is very isolated as it stands. It's tough to
>> programmatically configure index generation.
>>
>
> Ok, this one makes sense... the others do not seem to actually require a
> refactor though.
>
>
>>
>>
>>>
>>>- *Delegate coverage acceptance/rejection to a predicate object*:
>>>makes sense I guess, so the reason to have this plugable is because you
>>>might roll a collector that has less limitations than the default ones?
>>>
>>> Exactly.
>>
>
> Ok but... why not keep the improved collectors in the mosaic instead? They
> would benefit every user there (instead of just the gt-dem users)
>
>
>>
>>>- *Pre-process Granule Footprint Before Indexing*: makes sense,
>>>thinking out loud is the footprint the only thing that needs 
>>> reprocessing?
>>>
>>> No, it's not really the only thing that need reprocessing. In fact
>> PropertyCollectors have a tangentially related responsibility (and are used
>> in roughly the same spot). Maybe this functionality could be combined
>> somehow, and make monolithic some sort of monolithic visitor which would be
>> responsible for:
>>
>> - Collecting properties from granules
>> - Pre-processing the granule footprint if needed
>> - Populating the index feature
>>
>> And so on. I'll have to think about this one a bit more, but as the
>> proposal stands at least FootprintProcessor has a very clear responsibility.
>>
>
>

> See above about the usage of the importer. The indexing happens during
> layer configuration in the UI, expensive operations are best done during
> pre-processing.
> Normally what you are describing here is done as part of a "ingest"
> machinery, which is a off-line ETL of sorts that prepares the data for
> publication, but
> before it actually reaches GeoServer. The importer is one example of
> those, I've seen several others, all of them were asynch and offline.
>

I agree with this comment too.

>
>
>>
>>>- *Generalize Mosaicking per GranuleCollector and Update
>>>GranuleCollector to a tree-like hierarchy*: an example would be
>>>useful
>>>
>>> I can do an example for this. The motivation here is to delegate the
>> actual mosaicking to an object which may internally be delegating to other
>> mosaicking objects. For 

[Geotools-devel] [JIRA] (GEOT-5439) FeatureGraphGenerator with PolygonGraphGenerator problem

[HyungGyu Ryoo (JIRA)]

Title: Message Title


 
 
 
 

 
 
 

 
   
 HyungGyu Ryoo created an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 GeoTools /  GEOT-5439  
 
 
  FeatureGraphGenerator with PolygonGraphGenerator problem   
 

  
 
 
 
 

 
Issue Type: 
  Bug  
 
 
Affects Versions: 
 16-beta  
 
 
Assignee: 
 Unassigned  
 
 
Components: 
 graph extension  
 
 
Created: 
 07/Jun/16 2:49 PM  
 
 
Labels: 
 geotools  
 
 
Priority: 
  Medium  
 
 
Reporter: 
 HyungGyu Ryoo  
 

  
 
 
 
 

 
 => Test Code public void testIsolatedNodeAdd() throws Exception { gg = new PolygonGraphGenerator(new BasicGraphBuilder(), rel); fgg = new FeatureGraphGenerator( gg );  Polygon p1 = createPolygon("0 0,1 1,2 2,0 0"); Polygon p2 = createPolygon("3 3,4 4,5 5,3 3"); Polygon p3 = createPolygon("6 6,7 7,8 8,6 6");  Polygon[] ps =  { p1, p2, p3 } ; SimpleFeature[] features = features( ps );  Node n1 = (Node) fgg.add(features[0]); Node n2 = (Node) fgg.add(features[1]); Node n3 = (Node) fgg.add(features[2]);  assertNotNull(n1); assertEquals(n1.getObject(),features[0]);  assertNotNull(n2); assertEquals(n2.getObject(),features[1]);  assertNotNull(n3); assertEquals(n3.getObject(),features[2]);  Graph g = fgg.getGraph(); assertNotNull(g); } FeatureGraphGenerator decorates PolygonGraphGenerator. In case of adding node, FeatureGraphGenerator makes Graphable object with Polygon geometry by decorate generator. After making Graphable object, FeatureGraphGenerator changes it's object to feature(SimpleFeature). => Method in FeatureGraphGenerator public Graphable add( Object obj )  { SimpleFeature feature = (SimpleFeature) obj; Graphable g = decorated.add( feature.getDefaultGeometry() ); Geometry geom = (Geometry) g.getObject(); //Preserve geometry from Graphable, as it may be 

Re: [Geotools-devel] H2GIS new License

[Erwan Bocher]

Dear Jody,

2016-06-07 0:06 GMT+02:00 Jody Garnett :

> You may need to change your WIKI (https://github.com/orbisgis/h2gis/wiki)
> - "I wrote the code, it's mine, and I'm contributing it to H2GIS for
> distribution licensed under the GPL 3.0."
>

Good catch thanks.

>
> I also note that you have a package change and a license change to look
> forward for JTS (so you could actually license h2gis the same way as h2 now
> if you wish).
>

It was a hard and long discussion to change the license ;-)  so we'll leave
it at that for now.

Best regards.

Erwan

>
> --
> Jody Garnett
>
> On 6 June 2016 at 04:39, Erwan Bocher  wrote:
>
>> Dear Geotools community,
>>
>> To promote dissemination, H2GIS has changed its license from GPLv3 to
>> LGPLv3.
>> More information can be found here
>> http://www.h2gis.org/news/2016/06/03/h2gis-1.3.0-released/
>>
>> Best regards
>>
>>
>> --
>> Erwan Bocher
>> -
>> Ingénieur de Recherche CNRS,
>> Laboratoire Lab-STICC – CNRS UMR 6285
>> Equipe DECIDE
>> UNIVERSITÉ DE BRETAGNE-SUD
>> Institut Universitaire de Technologie de Vannes
>> 8, Rue Montaigne - BP 561 56017 Vannes Cedex
>> T: +33 2 97 62 64 92
>> W: https://cv.archives-ouvertes.fr/erwan-bocher
>> W: http://www.univ-ubs.fr 
>>
>> 
>>
>>
>> --
>> What NetFlow Analyzer can do for you? Monitors network bandwidth and
>> traffic
>> patterns at an interface-level. Reveals which users, apps, and protocols
>> are
>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>> planning reports.
>> https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
>> ___
>> GeoTools-Devel mailing list
>> GeoTools-Devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geotools-devel
>>
>>
>
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel