Re: [Geotools-devel] Proposal: Refactor ImageMosaic index management

[Devon Tucker]

The GranuleAcceptor stuff is done on that branch. I have the
PropertiesCollector stuff done locally, but there's some iffy stuff there.
Handling the granule envelope in a PropertiesCollector should be fine, but
there is an issue with how StructuredGridCoverages are handled:

https://github.com/dvntucker/geotools/blob/granule_acceptors/modules/plugin/imagemosaic/src/main/java/org/geotools/gce/imagemosaic/ImageMosaicConfigHandler.java#L512

For StructuredGCs all the feature attributes are copied right off the
feature from the input reader, then the properties collectors are called.
I'm not really sure how this would tie in to using a PropertiesCollector
the pre-process the incoming coverage envelope, which makes me wonder if
it's a good idea at all.



On Mon, Jun 20, 2016 at 2:53 PM, Jody Garnett 
wrote:

> Reviewing
> https://github.com/geotools/geotools/wiki/Refactor-ImageMosaic-Index-and-Catalog-management-for-improved-extensibility
> it appears up to date (with a remove CatalogManager heading). Are you going
> to verify the approach (on that branch) and then send us an email for the
> completed proposal?
>
> It is hard to leap into this proposal and provide any useful feedback as a
> PMC member, I think the productive conversation is with Simone as module
> maintainer.
> --
> Jody
>
>
> --
> Jody Garnett
>
> On 20 June 2016 at 13:48, Devon Tucker  wrote:
>
>> Hi all,
>>
>> Based on discussions with Simone and Jody we have made a few changes to
>> this proposal:
>>
>> - CatalogManager gets deleted, its methods moved either to
>> ImageMosaicConfigHandler, Utils, or the GranuleCatalog implementations
>>
>> - Instead of delegating granule acceptance to CatalogManager, as was
>> originally proposed, this functionality is broken out into its own
>> interface + factory SPI.
>>
>> - I changed the granule envelope handling to just use the existing
>> PropertiesCollector API, since it already sets attributes on the index
>> feature, I don't really see why not? Not sure if anyone would object to
>> this.
>>
>> Also there's a new branch with most of this implemented:
>>
>> 
>> 
>> https://github.com/dvntucker/geotools/tree/granule_acceptors
>>
>> As usual, let me know if there's any questions or feedback.
>>
>> On Wed, Jun 15, 2016 at 5:38 AM, Simone Giannecchini <
>> simone.giannecch...@geo-solutions.it> wrote:
>>
>>> Dear Devon,
>>> a few things:
>>>
>>> -0- we should really get rid of this CatalogManager as is (a class
>>> with only static methods as its state its spread over N other classes)
>>> -1- we can already mosaick images with different colormodels (to a
>>> certain extent, it does not make sense to mosaick a float raster with
>>> an RGB). RGB, Gray and Palette can be mosaicked as of today
>>> -4- I would add some UML diagrams to your proposal to show what we
>>> have now and what we have afterwards
>>>
>>> I noticed that you are half-way through with your refactor and its
>>> probably too early to look into it but I have two things I wanted to
>>> bring up.
>>> First of all, I believe you shuold look at the refactor of the
>>> indexing together with this, I would not split them otherwise your
>>> refactor in this case could be too narrow.
>>> Let me explain, your goal here is to improve the harvesting process
>>> allowing implementors to change how harvested elements are discarded
>>> as well as how they are preprocessed or manipulated and this is not
>>> isolated from the harvesting itself.
>>>
>>> I believe catalogmanager (although) the nice name, might go away and
>>> this kind of behavior could be isolated into smaller API likewise we
>>> did with the properties collector so that one can drop them inside the
>>> imagemosaic and ask it through the indexmanager to apply them.
>>> Ideally one could rather write its one harvester and completely
>>> customize how we put things inside the GranuleCatalog.
>>>
>>> Let me know what you think about this.
>>>
>>>
>>> Regards,
>>> Simone Giannecchini
>>> ==
>>> GeoServer Professional Services from the experts!
>>> Visit http://goo.gl/it488V for more information.
>>> ==
>>> Ing. Simone Giannecchini
>>> @simogeo
>>> Founder/Director
>>>
>>> GeoSolutions S.A.S.
>>> Via di Montramito 3/A
>>> 55054  Massarosa (LU)
>>> Italy
>>> phone: +39 0584 962313
>>> fax: +39 0584 1660272
>>> mob:   +39 333 8128928
>>>
>>> http://www.geo-solutions.it
>>> http://twitter.com/geosolutions_it
>>>
>>> ---
>>> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
>>> Le informazioni contenute in questo messaggio di posta elettronica e/o
>>> nel/i file/s allegato/i sono da considerarsi strettamente riservate.
>>> Il loro utilizzo è consentito esclusivamente al destinatario del
>>> messaggio, per le finalità indicate nel messaggio stesso. Qualora
>>> riceviate questo 

Re: [Geotools-devel] Proposal: Refactor ImageMosaic index management

[Jody Garnett]

Reviewing
https://github.com/geotools/geotools/wiki/Refactor-ImageMosaic-Index-and-Catalog-management-for-improved-extensibility
it appears up to date (with a remove CatalogManager heading). Are you going
to verify the approach (on that branch) and then send us an email for the
completed proposal?

It is hard to leap into this proposal and provide any useful feedback as a
PMC member, I think the productive conversation is with Simone as module
maintainer.
--
Jody


--
Jody Garnett

On 20 June 2016 at 13:48, Devon Tucker  wrote:

> Hi all,
>
> Based on discussions with Simone and Jody we have made a few changes to
> this proposal:
>
> - CatalogManager gets deleted, its methods moved either to
> ImageMosaicConfigHandler, Utils, or the GranuleCatalog implementations
>
> - Instead of delegating granule acceptance to CatalogManager, as was
> originally proposed, this functionality is broken out into its own
> interface + factory SPI.
>
> - I changed the granule envelope handling to just use the existing
> PropertiesCollector API, since it already sets attributes on the index
> feature, I don't really see why not? Not sure if anyone would object to
> this.
>
> Also there's a new branch with most of this implemented:
>
> 
> 
> https://github.com/dvntucker/geotools/tree/granule_acceptors
>
> As usual, let me know if there's any questions or feedback.
>
> On Wed, Jun 15, 2016 at 5:38 AM, Simone Giannecchini <
> simone.giannecch...@geo-solutions.it> wrote:
>
>> Dear Devon,
>> a few things:
>>
>> -0- we should really get rid of this CatalogManager as is (a class
>> with only static methods as its state its spread over N other classes)
>> -1- we can already mosaick images with different colormodels (to a
>> certain extent, it does not make sense to mosaick a float raster with
>> an RGB). RGB, Gray and Palette can be mosaicked as of today
>> -4- I would add some UML diagrams to your proposal to show what we
>> have now and what we have afterwards
>>
>> I noticed that you are half-way through with your refactor and its
>> probably too early to look into it but I have two things I wanted to
>> bring up.
>> First of all, I believe you shuold look at the refactor of the
>> indexing together with this, I would not split them otherwise your
>> refactor in this case could be too narrow.
>> Let me explain, your goal here is to improve the harvesting process
>> allowing implementors to change how harvested elements are discarded
>> as well as how they are preprocessed or manipulated and this is not
>> isolated from the harvesting itself.
>>
>> I believe catalogmanager (although) the nice name, might go away and
>> this kind of behavior could be isolated into smaller API likewise we
>> did with the properties collector so that one can drop them inside the
>> imagemosaic and ask it through the indexmanager to apply them.
>> Ideally one could rather write its one harvester and completely
>> customize how we put things inside the GranuleCatalog.
>>
>> Let me know what you think about this.
>>
>>
>> Regards,
>> Simone Giannecchini
>> ==
>> GeoServer Professional Services from the experts!
>> Visit http://goo.gl/it488V for more information.
>> ==
>> Ing. Simone Giannecchini
>> @simogeo
>> Founder/Director
>>
>> GeoSolutions S.A.S.
>> Via di Montramito 3/A
>> 55054  Massarosa (LU)
>> Italy
>> phone: +39 0584 962313
>> fax: +39 0584 1660272
>> mob:   +39 333 8128928
>>
>> http://www.geo-solutions.it
>> http://twitter.com/geosolutions_it
>>
>> ---
>> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
>> Le informazioni contenute in questo messaggio di posta elettronica e/o
>> nel/i file/s allegato/i sono da considerarsi strettamente riservate.
>> Il loro utilizzo è consentito esclusivamente al destinatario del
>> messaggio, per le finalità indicate nel messaggio stesso. Qualora
>> riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
>> cortesemente di darcene notizia via e-mail e di procedere alla
>> distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
>> Conservare il messaggio stesso, divulgarlo anche in parte,
>> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
>> diverse, costituisce comportamento contrario ai principi dettati dal
>> D.Lgs. 196/2003.
>>
>> The information in this message and/or attachments, is intended solely
>> for the attention and use of the named addressee(s) and may be
>> confidential or proprietary in nature or covered by the provisions of
>> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
>> Data Protection Code).Any use not in accord with its purpose, any
>> disclosure, reproduction, copying, distribution, or either
>> dissemination, either whole or partial, is strictly forbidden except
>> previous formal approval of the named addressee(s). If 

Re: [Geotools-devel] Proposal: Refactor ImageMosaic index management

[Devon Tucker]

Hi all,

Based on discussions with Simone and Jody we have made a few changes to
this proposal:

- CatalogManager gets deleted, its methods moved either to
ImageMosaicConfigHandler, Utils, or the GranuleCatalog implementations

- Instead of delegating granule acceptance to CatalogManager, as was
originally proposed, this functionality is broken out into its own
interface + factory SPI.

- I changed the granule envelope handling to just use the existing
PropertiesCollector API, since it already sets attributes on the index
feature, I don't really see why not? Not sure if anyone would object to
this.

Also there's a new branch with most of this implemented:



https://github.com/dvntucker/geotools/tree/granule_acceptors

As usual, let me know if there's any questions or feedback.

On Wed, Jun 15, 2016 at 5:38 AM, Simone Giannecchini <
simone.giannecch...@geo-solutions.it> wrote:

> Dear Devon,
> a few things:
>
> -0- we should really get rid of this CatalogManager as is (a class
> with only static methods as its state its spread over N other classes)
> -1- we can already mosaick images with different colormodels (to a
> certain extent, it does not make sense to mosaick a float raster with
> an RGB). RGB, Gray and Palette can be mosaicked as of today
> -4- I would add some UML diagrams to your proposal to show what we
> have now and what we have afterwards
>
> I noticed that you are half-way through with your refactor and its
> probably too early to look into it but I have two things I wanted to
> bring up.
> First of all, I believe you shuold look at the refactor of the
> indexing together with this, I would not split them otherwise your
> refactor in this case could be too narrow.
> Let me explain, your goal here is to improve the harvesting process
> allowing implementors to change how harvested elements are discarded
> as well as how they are preprocessed or manipulated and this is not
> isolated from the harvesting itself.
>
> I believe catalogmanager (although) the nice name, might go away and
> this kind of behavior could be isolated into smaller API likewise we
> did with the properties collector so that one can drop them inside the
> imagemosaic and ask it through the indexmanager to apply them.
> Ideally one could rather write its one harvester and completely
> customize how we put things inside the GranuleCatalog.
>
> Let me know what you think about this.
>
>
> Regards,
> Simone Giannecchini
> ==
> GeoServer Professional Services from the experts!
> Visit http://goo.gl/it488V for more information.
> ==
> Ing. Simone Giannecchini
> @simogeo
> Founder/Director
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob:   +39 333 8128928
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> ---
> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate.
> Il loro utilizzo è consentito esclusivamente al destinatario del
> messaggio, per le finalità indicate nel messaggio stesso. Qualora
> riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
> cortesemente di darcene notizia via e-mail e di procedere alla
> distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
> Conservare il messaggio stesso, divulgarlo anche in parte,
> distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
> diverse, costituisce comportamento contrario ai principi dettati dal
> D.Lgs. 196/2003.
>
> The information in this message and/or attachments, is intended solely
> for the attention and use of the named addressee(s) and may be
> confidential or proprietary in nature or covered by the provisions of
> privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
> Data Protection Code).Any use not in accord with its purpose, any
> disclosure, reproduction, copying, distribution, or either
> dissemination, either whole or partial, is strictly forbidden except
> previous formal approval of the named addressee(s). If you are not the
> intended recipient, please contact immediately the sender by
> telephone, fax or e-mail and delete the information in this message
> that has been received in error. The sender does not give any warranty
> or accept liability as the content, accuracy or completeness of sent
> messages and accepts no responsibility  for changes made after they
> were sent or for other risks which arise as a result of e-mail
> transmission, viruses, etc.
>
>
> On Tue, Jun 14, 2016 at 8:09 PM, Devon Tucker 
> wrote:
> > Hi all,
> >
> > After discussions about the ImageMosaic API proposal we have decided to
> > break it up into a few smaller pieces that 

[Geotools-devel] [JIRA] (GEOT-5445) YSLD Validator skips aliases

[Torben Barsballe (JIRA)]

Title: Message Title


 
 
 
 

 
 
 

 
   
 Torben Barsballe created an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 GeoTools /  GEOT-5445  
 
 
  YSLD Validator skips aliases   
 

  
 
 
 
 

 
Issue Type: 
  Bug  
 
 
Assignee: 
 Unassigned  
 
 
Components: 
 unsupported  
 
 
Created: 
 20/Jun/16 8:24 PM  
 
 
Priority: 
  Medium  
 
 
Reporter: 
 Torben Barsballe  
 

  
 
 
 
 

 
 The YSLDValidator for the ysld module does not run on aliases (i.e. variables). This can lead to a situation where it uses a 'value' validator when parsing a 'property' tag, causing erroneous parse errors. For example, given the following YSLD: 

 
define:  '#ff9900'
title: dark orange polygon
symbolizers:
- polygon:
stroke-width: 0.5
fill-color: *color
stroke-color: '#00'
 

 The validator reads the fill-color property, and gets a ColorValidator for validating the value. However, since the value is an alias, the validator skips it and uses the ColorValidator on the stroke-color property instead, leading to the error: noformat null; Invalid color, must be one of: '#RRGGBB', rgb(r,g,b), or _expression_; in 'reader', line 7, column 5: stroke-color: '#00' ^ noformat  
 

  
 
 
 
 

 
 
 

 

[Geotools-devel] [JIRA] (GEOT-5444) GradientColorMap generator uses a fixed offset of 0.01 for the before/after transparent ranges

[Andrea Aime [Administrator] (JIRA)]

Title: Message Title


 
 
 
 

 
 
 

 
   
 Andrea Aime [Administrator] created an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 GeoTools /  GEOT-5444  
 
 
  GradientColorMap generator uses a fixed offset of 0.01 for the before/after transparent ranges   
 

  
 
 
 
 

 
Issue Type: 
  Bug  
 
 
Assignee: 
 Andrea Aime [Administrator]  
 
 
Components: 
 render  
 
 
Created: 
 20/Jun/16 3:24 PM  
 
 
Fix Versions: 
 16-beta  
 
 
Priority: 
  Medium  
 
 
Reporter: 
 Andrea Aime [Administrator]  
 

  
 
 
 
 

 
 This is not valid in general, 0.01 can be a very a very small or a very large quantity depending on the nature of the data. Make it work with no offset at all instead (sharp jump from transparent to solid and vice-versa).  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment