[Geotools-devel] Reminder: GeoTools / GeoServer Meeting at 19:30 UTC on Tuesday

[Ben Caradoc-Davies]

GeoTools / GeoServer committee meeting on Skype at 19:30 UTC on Tuesday:
http://www.timeanddate.com/worldclock/fixedtime.html?msg=GeoTools+%2F+GeoServer+Meeting=2016=5=17=19=30=0=1

-- 
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand

--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Re: [Geotools-devel] Avoiding to scan postgreqsql type system in JDBCDataStore

[Justin Deoliveira]

Sounds like a good approach to me.
On Mon, May 16, 2016 at 11:37 AM Andrea Aime 
wrote:

> Hi,
> today I stumbled into a really weird issue, basically JDBCDataStore becomes
> extremely slow (minutes if the db is not co-located) if a database has
> tens of
> thousands of tables in it.
>
> The short version of it, is that to create a table we need to get from the
> java
> classes to the sql type names, using a mapping process that goes from
> "java class" to "jdbc type id" to "native sql type", and the second jump we
> do by first getting the full list of type ids and type names from the
> database
> (nice generic approach).
>
> However in postgresql even the tables we create are types, and for those,
> the jdbc driver has to make a special query for each table. So if you have
> 23k tables, the driver makes 1 + 23k queries, taking forever to return the
> list of id -> type mappings.
>
> More details can be found here:
> https://osgeo-org.atlassian.net/browse/GEOS-7533
>
> I checked if the driver had any knob or tunable to avoid scanning the
> tables, but it seems
> there are none (by code inspection).
>
> The solution I'd like to propose is to register in the postgis driver
> mappings for
> the most known types already, and do not hit JDBCMedata.getTypeInfo()
> unless
> we actually need to do that, that is, only if we end up with a type
> mapping that
> we cannot resolve using the short list (that is, a solution similar to
> what the jruby
> activerecord people applied, the ticket above contains a link).
>
> This would keep the solution for a postgresql specific issue in the postgis
> module.
>
> If you have objections could you please let me know? I'll proceed with
> the above approach otherwise
>
> Cheers
> Andrea
>
> --
> ==
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39  339 8844549
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in accord with its purpose, any disclosure, reproduction,
> copying, distribution, or either dissemination, either whole or partial, is
> strictly forbidden except previous formal approval of the named
> addressee(s). If you are not the intended recipient, please contact
> immediately the sender by telephone, fax or e-mail and delete the
> information in this message that has been received in error. The sender
> does not give any warranty or accept liability as the content, accuracy or
> completeness of sent messages and accepts no responsibility  for changes
> made after they were sent or for other risks which arise as a result of
> e-mail transmission, viruses, etc.
>
> ---
>
> --
> Mobile security can be enabling, not merely restricting. Employees who
> bring their own devices (BYOD) to work are irked by the imposition of MDM
> restrictions. Mobile Device Manager Plus allows you to control only the
> apps on BYO-devices by containerizing them, leaving personal data
> untouched!
> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
> ___
> GeoTools-Devel mailing list
> GeoTools-Devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geotools-devel
>
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!

[Geotools-devel] Avoiding to scan postgreqsql type system in JDBCDataStore

[Andrea Aime]

Hi,
today I stumbled into a really weird issue, basically JDBCDataStore becomes
extremely slow (minutes if the db is not co-located) if a database has tens
of
thousands of tables in it.

The short version of it, is that to create a table we need to get from the
java
classes to the sql type names, using a mapping process that goes from
"java class" to "jdbc type id" to "native sql type", and the second jump we
do by first getting the full list of type ids and type names from the
database
(nice generic approach).

However in postgresql even the tables we create are types, and for those,
the jdbc driver has to make a special query for each table. So if you have
23k tables, the driver makes 1 + 23k queries, taking forever to return the
list of id -> type mappings.

More details can be found here:
https://osgeo-org.atlassian.net/browse/GEOS-7533

I checked if the driver had any knob or tunable to avoid scanning the
tables, but it seems
there are none (by code inspection).

The solution I'd like to propose is to register in the postgis driver
mappings for
the most known types already, and do not hit JDBCMedata.getTypeInfo() unless
we actually need to do that, that is, only if we end up with a type mapping
that
we cannot resolve using the short list (that is, a solution similar to what
the jruby
activerecord people applied, the ticket above contains a link).

This would keep the solution for a postgresql specific issue in the postgis
module.

If you have objections could you please let me know? I'll proceed with
the above approach otherwise

Cheers
Andrea

-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility  for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

---
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Re: [Geotools-devel] Copyright header policy

[Andrea Aime]

Hi,
looked at the proposal, seems it got stuck?
Could the other PSC members provide a preference list too?

https://github.com/geotools/geotools/wiki/Updates-to-Copyright-Header-Policy

Cheers
Andrea


On Wed, Apr 27, 2016 at 10:15 AM, Andrea Aime 
wrote:

> Hi Justin,
> I made a bit of research as this is one topic that foundations probably
> have guidance on.
> Found these for the moment:
> https://www.gnu.org/prep/maintain/html_node/Copyright-Notices.html
>
> https://softwarefreedom.org/resources/2012/ManagingCopyrightInformation.html
> http://www.apache.org/legal/src-headers.html
>
> The official guidance for these foundations seem to suggest at least one
> year should show up in the headers.
> I found little evidence that our current approach has benefits, although
> some practices seem to imply
> a mass header change per (major) release... which does not make all that
> much sense to me, but
> would at least be easy to automate :-p
>
> Cheers
> Andrea
>
>
>
> On Wed, Apr 27, 2016 at 4:15 AM, Justin Deoliveira 
> wrote:
>
>> Thanks for the input everyone. I put together a proposal with all of the
>> options put forth on this thread. Apologies if I missed one, if I did just
>> let me know.
>>
>>
>> https://github.com/geotools/geotools/wiki/Updates-to-Copyright-Header-Policy
>>
>> Like I said before this proposal involves listing out in order of
>> preference the various options. Feel free to update the proposal or just
>> reply to this thread and I’ll happily make the update on your behalf.
>>
>> Thanks!
>>
>> -Justin
>>
>> On Tue, Apr 26, 2016 at 2:50 PM Jody Garnett 
>> wrote:
>>
>>> You asked where I was uncomfortable, not if it made sense :)
>>>
>>> --
>>> Jody Garnett
>>>
>>> On 25 April 2016 at 09:37, Justin Deoliveira  wrote:
>>>
 I can see how the original date could be useful for cases like this but
 have you ever seen a situation where something was marked (for example) (c)
 2010 and from 2010 to the current date of when you were doing the audit the
 copyright changed in some meaningful way? I am of course not counting any
 transfer of copyright to OSGeo, etc… since I would expect that to result in
 a new copyright timestamp.

 On Mon, Apr 25, 2016 at 12:12 AM Jody Garnett 
 wrote:

> The part that is making me uneasy is the few times I have done code
> audits on this codebase the headers were very useful. While I did have to
> duck out to git history on a few occasions it was a rare occurrence - in
> part because the team here has done a good job.
>
> --
> Jody Garnett
>
> On 24 April 2016 at 08:13, Justin Deoliveira 
> wrote:
>
>> What is it specifically that is making you uneasy? I thought Ben made
>> a pretty strong argument that any dates in the copyright don’t really 
>> mean
>> anything when it comes to actual legal stuff.
>>
>> Doing it periodically via a script does seem like a better
>> compromise… although from everything I have heard here I still don’t see
>> why initial creation date isn’t enough.
>>
>> Once this thread “quiets down” I’ll summarize all of the options that
>> folks have put forth and throw them into the proposal and we can vote for
>> the options we like best. Since there are a handful of options on the 
>> table
>> I think perhaps having people rank options 1,2,3 might work best.
>>
>>
>> On Sat, Apr 23, 2016 at 7:22 PM Jody Garnett 
>> wrote:
>>
>>> Yeah the longer I think about this one the more I am made
>>> uncomfortable.  We would be back to "significant" changes needing the
>>> header updated -
>>>
>>> As an alternative wow do you feel about updating the headers once
>>> each year (via a script) so that individual pull requests are not held 
>>> up?
>>>
>>> --
>>> Jody Garnett
>>>
>>> On 22 April 2016 at 12:05, Justin Deoliveira 
>>> wrote:
>>>
 Cool. I'll write up a quick one.

 On Fri, Apr 22, 2016 at 1:01 PM Jody Garnett <
 jody.garn...@gmail.com> wrote:

> If you would like to make that proposal we can update the
> developers guide and get it done ...
>
> --
> Jody Garnett
>
> On 22 April 2016 at 11:38, Justin Deoliveira 
> wrote:
>
>> Thanks Jody. That helps me understand.
>>
>> My thought is that the day to day overhead is not worth being
>> able to rest assured that in 75 years GeoTools will still be in good
>> standing with regard to copyright ;)
>>
>> My vote would be to change to a process where the copyright is
>> set to the current year that the file is created 

Re: [Geotools-devel] java.lang.ClassCastException in MySQLDataStore

[Ian Turton]

Thanks for this, what we need now is a Jira ticket, a unit test and a PR.
I'm on holiday with just my phone and limited bandwidth so I can't add the
links for those steps but have a look at the contribution guide.

Ian
On 15 May 2016 19:58, "Debasish Sahu"  wrote:

Hi Ian,

I am getting the following exception when using the Geotools MySQLDataStore
getBounds method.

Caused by: java.lang.ClassCastException: com.vividsolutions.jts.geom.Point
cannot be cast to com.vividsolutions.jts.geom.Polygon
at
org.geotools.data.mysql.MySQLDialectBasic.decodeGeometryEnvelope(MySQLDialectBasic.java:214)
at org.geotools.jdbc.JDBCDataStore.getBounds(JDBCDataStore.java:1322)

The following code is causing the exception

Polygon polygon = (Polygon) new WKBReader().read(wkb);

This can be solved by using

 Geometry geometry= new WKBReader().read(wkb);
 return geometry.getEnvelopeInternal();


Please can you help with the fix.


-- 
Thanks and Warm Regards
Debasish Sahu
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Geotools-devel] [JIRA] (GEOT-5420) Performance regression creating image mosaic reader against databases with thousands of tables

[Andrea Aime [Administrator] (JIRA)]

Title: Message Title


 
 
 
 

 
 
 

 
   
 Andrea Aime [Administrator] created an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 GeoTools /  GEOT-5420  
 
 
  Performance regression creating image mosaic reader against databases with thousands of tables   
 

  
 
 
 
 

 
Issue Type: 
  Bug  
 
 
Assignee: 
 Unassigned  
 
 
Components: 
 imagemosaic plugin  
 
 
Created: 
 16/May/16 11:25 AM  
 
 
Priority: 
  Medium  
 
 
Reporter: 
 Andrea Aime [Administrator]  
 

  
 
 
 
 

 
 Some new code scans all tables to check if they are valid mosaic indexes, but in fact, ends up using only the first one found, so stop at that one instead of marching on  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 

[Geotools-devel] [JIRA] (GEOT-5419) WorldFile reader don't release raster file lock

[Nuno Oliveira (JIRA)]

Title: Message Title


 
 
 
 

 
 
 

 
   
 Nuno Oliveira created an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 GeoTools /  GEOT-5419  
 
 
  WorldFile reader don't release raster file lock   
 

  
 
 
 
 

 
Issue Type: 
  Improvement  
 
 
Assignee: 
 Nuno Oliveira  
 
 
Attachments: 
 bugexample.zip  
 
 
Created: 
 16/May/16 10:41 AM  
 
 
Fix Versions: 
 14.4, 15.1, 16-beta  
 
 
Priority: 
  Medium  
 
 
Reporter: 
 Nuno Oliveira  
 

  
 
 
 
 

 
 Running the attached example will produce the following output: Reader successfully instantiated Reader successfully disposed Can delete pgw: true Can delete png: false The last line indicates that the PNG file cannot be deleted because of a lock remaining on the PNG file.  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment