Re: [Geotools-devel] Semantics of features aggregation and pagination

2014-06-04 Thread Mauro Bartolomeoli 
Hi,
I prepared a pull request for the subject:
https://github.com/geotools/geotools/pull/470

I used tha approach described in my last email.
As an additional note, I also added a preserveOrder flag to help pagination
be consistent.
With this flag the visitor set mantains the order in which items have been
inserted, so the elements are consistently ordered between each visitor
call.

Regards,
Mauro Bartolomeoli


2014-05-30 19:07 GMT+02:00 Andrea Aime :

> On Fri, May 30, 2014 at 6:58 PM, Mauro Bartolomeoli <
> mauro.bartolome...@geo-solutions.it> wrote:
>
>> Hi,
>> moving on on this, I tried the following approach:
>>  - created a new LimitingVisitor interface with getStartIndex and
>> getMaxFeatures methods, to be used by visitor aware stores
>>  - changed UniqueVisitor to implement this interface and added pagination
>> capabilities to the visitor if those two properties are defined (this will
>> handle stores that are not visitor aware)
>>  - made JDBCDataStore aware of LimitingVisitor, using it to get limits in
>> aggregated queries, if limits are defined in Visitor instead that in Query
>>
>> This way, the old behaviour is mantained if limits are defined in Query,
>> the new one (limits applied after aggregation) if they are defined in the
>> visitor.
>>
>> What do you think?
>> Can I move forward with it?
>>
>
> Seems a good approach to me, would be nice to hear from the other core
> devs (thinking Christian, Justin as jdbc module co-maintainer,
> and Jody since he designed the visitors in the first place, but of course
> anybody is warmly welcomed to provide feedback)
>
> Cheers
> Andrea
>
> --
> ==
> Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
> for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via Poggio alle Viti 1187
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39  339 8844549
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> ---
>



-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==

Dott. Mauro Bartolomeoli
@mauro_bart
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---
--
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
applications. Written by three acclaimed leaders in the field, 
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Re: [Geotools-devel] wfs-ng improvements

2014-06-04 Thread Justin Deoliveira 
Potentially but at this point we have to prioritize the tasks. If the other
issues brought up in this thread are blockers to upgrade to supported
status then I think its a better use of time to focus on those.


On Tue, Jun 3, 2014 at 4:09 PM, Jody Garnett  wrote:

> One more question - were you going to back port this work for the stable
> series?
>
> Usually we have at least a show of making a module available for public
> review prior to going to extension status. I would be happy to make a beta
> release (focused on Java 7 and a new wfs implementation) in order to make
> that happen.
> --
> Jody
>
>
>
>
> Jody Garnett
>
>
> On Tue, Jun 3, 2014 at 11:06 PM, Jody Garnett 
> wrote:
>
>> It seems pretty good, if anything can survive uDig testing :)
>>
>> Note we can also use the class path trick you did for the shapefile &
>> shapefile-ng handover. Allow new factory to work against the old
>> parameters, but perform a check to see if the old implementation is on the
>> class path first. If it is found it can just ignore the old parameters etc
>> ... if not found it is free to create its own WFS DataStore.
>> --
>> Jody
>>
>> Jody Garnett
>>
>>
>> On Tue, Jun 3, 2014 at 10:47 PM, Andrea Aime <
>> andrea.a...@geo-solutions.it> wrote:
>>
>>> On Tue, Jun 3, 2014 at 2:37 PM, Jody Garnett 
>>> wrote:
>>>
 I would recommend killing the old wfs module (it is unsupported for a
 reason - namely it has not attracted funding). With that "out of the way"
 wfs-ng is free to take its place.

>>>
>>> Yep, that's the simplest and cleaner approach, that's why I'm wondering
>>> to what level wfs-ng is a full replacement for the old one, given the
>>> module, despite being unsupported,
>>> has plenty of users.
>>>
>>> Cheers
>>> Andrea
>>>
>>> --
>>> ==
>>> Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
>>> for more information.
>>> ==
>>>
>>> Ing. Andrea Aime
>>> @geowolf
>>> Technical Lead
>>>
>>> GeoSolutions S.A.S.
>>> Via Poggio alle Viti 1187
>>> 55054  Massarosa (LU)
>>> Italy
>>> phone: +39 0584 962313
>>> fax: +39 0584 1660272
>>> mob: +39  339 8844549
>>>
>>> http://www.geo-solutions.it
>>> http://twitter.com/geosolutions_it
>>>
>>> ---
>>>
>>
>>
>
>
> --
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech
> ___
> GeoTools-Devel mailing list
> GeoTools-Devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geotools-devel
>
>


-- 
*Justin Deoliveira*
Vice President, Engineering | Boundless
jdeol...@boundlessgeo.com
@j_deolive 
--
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
applications. Written by three acclaimed leaders in the field, 
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Geotools-devel] [jira] (GEOT-4813) Add pagination support to UniqueVisitor

2014-06-04 Thread Mauro Bartolomeoli (JIRA) 
Title: Message Title










 

 Mauro Bartolomeoli created an issue











 






 GeoTools /  GEOT-4813



  Add pagination support to UniqueVisitor 










Issue Type:

  Improvement




Affects Versions:


 11.1




Assignee:

 Jody Garnett




Components:


 jdbc, main




Created:


 04/Jun/14 7:54 AM




Priority:

  Minor




Reporter:

 Mauro Bartolomeoli










We should add pagination support to the UniqueVisitor (startIndex, maxFeatures). We should also mantain order of items to allow splitting items in pages and optimize JDBCDataStore to do pagination directly on db where the visitor is supported by the dialect.












   

 Add Comment

Re: [Geotools-devel] wfs-ng breaking windows build

2014-06-04 Thread Jody Garnett 
Yeah, I put it in after asking Niels.

It can be added back after his pull request.

Jody Garnett


On Wed, Jun 4, 2014 at 7:53 PM, Andrea Aime 
wrote:

> Hi,
> the wfs-ng module has been breaking the windows build server for a while
> now.
> Shall we take it out of the build?
>
> Example failure:
>
>
>
> testInit_1_0(org.geotools.data.wfs.internal.WFSClientTest)  Time elapsed: 0 
> sec  <<< ERROR!
> java.io.FileNotFoundException: 
> C:\.jenkins\jobs\GeoTools-Master\workspace\modules\unsupported\wfs-ng\target\test-classes\org\geotools\data\wfs\impl\test-data\GeoServer_2.2.x\1.0.0\GetCapabilities.xml?
>  (The filename, directory name, or volume label syntax is incorrect)
>   at java.io.FileInputStream.open(Native Method)
>   at java.io.FileInputStream.(FileInputStream.java:146)
>   at java.io.FileInputStream.(FileInputStream.java:101)
>   at 
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
>   at org.geotools.data.ows.SimpleHttpClient.get(SimpleHttpClient.java:100)
>   at 
> org.geotools.data.ows.AbstractOpenWebService.internalIssueRequest(AbstractOpenWebService.java:424)
>   at 
> org.geotools.data.wfs.internal.WFSClient.internalIssueRequest(WFSClient.java:258)
>   at 
> org.geotools.data.wfs.internal.WFSClient.issueRequest(WFSClient.java:268)
>   at 
> org.geotools.data.wfs.internal.WFSClient.issueRequest(WFSClient.java:37)
>   at 
> org.geotools.data.ows.AbstractOpenWebService.negotiateVersion(AbstractOpenWebService.java:260)
>   at 
> org.geotools.data.ows.AbstractOpenWebService.(AbstractOpenWebService.java:110)
>   at org.geotools.data.wfs.internal.WFSClient.(WFSClient.java:51)
>   at org.geotools.data.wfs.internal.WFSClient.(WFSClient.java:45)
>   at 
> org.geotools.data.wfs.internal.WFSClientTest.newClient(WFSClientTest.java:54)
>   at 
> org.geotools.data.wfs.internal.WFSClientTest.testInit(WFSClientTest.java:43)
>   at 
> org.geotools.data.wfs.internal.WFSClientTest.testInit_1_0(WFSClientTest.java:72)
>
>
>
> Cheers
> Andrea
>
> --
> ==
> Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
> for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via Poggio alle Viti 1187
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39  339 8844549
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> ---
>
>
> --
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech
> ___
> GeoTools-Devel mailing list
> GeoTools-Devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geotools-devel
>
>
--
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
applications. Written by three acclaimed leaders in the field, 
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Geotools-devel] wfs-ng breaking windows build

2014-06-04 Thread Andrea Aime 
Hi,
the wfs-ng module has been breaking the windows build server for a while
now.
Shall we take it out of the build?

Example failure:



testInit_1_0(org.geotools.data.wfs.internal.WFSClientTest)  Time
elapsed: 0 sec  <<< ERROR!
java.io.FileNotFoundException:
C:\.jenkins\jobs\GeoTools-Master\workspace\modules\unsupported\wfs-ng\target\test-classes\org\geotools\data\wfs\impl\test-data\GeoServer_2.2.x\1.0.0\GetCapabilities.xml?
(The filename, directory name, or volume label syntax is incorrect)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:146)
at java.io.FileInputStream.(FileInputStream.java:101)
at 
sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
at org.geotools.data.ows.SimpleHttpClient.get(SimpleHttpClient.java:100)
at 
org.geotools.data.ows.AbstractOpenWebService.internalIssueRequest(AbstractOpenWebService.java:424)
at 
org.geotools.data.wfs.internal.WFSClient.internalIssueRequest(WFSClient.java:258)
at 
org.geotools.data.wfs.internal.WFSClient.issueRequest(WFSClient.java:268)
at 
org.geotools.data.wfs.internal.WFSClient.issueRequest(WFSClient.java:37)
at 
org.geotools.data.ows.AbstractOpenWebService.negotiateVersion(AbstractOpenWebService.java:260)
at 
org.geotools.data.ows.AbstractOpenWebService.(AbstractOpenWebService.java:110)
at org.geotools.data.wfs.internal.WFSClient.(WFSClient.java:51)
at org.geotools.data.wfs.internal.WFSClient.(WFSClient.java:45)
at 
org.geotools.data.wfs.internal.WFSClientTest.newClient(WFSClientTest.java:54)
at 
org.geotools.data.wfs.internal.WFSClientTest.testInit(WFSClientTest.java:43)
at 
org.geotools.data.wfs.internal.WFSClientTest.testInit_1_0(WFSClientTest.java:72)



Cheers
Andrea

-- 
==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---
--
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
applications. Written by three acclaimed leaders in the field, 
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Geotools-devel] Build failed in Jenkins: GeoTools-Master #281

2014-06-04 Thread winbuild 
See 

Changes:

[andrea aime] [GEOT-4812] Table hints not applied to aggregate queries

--
[...truncated 17453 lines...]
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools.ogc$net.opengis.wfs\builds\2014-06-04_11-10-19\archive\org.geotools.ogc\net.opengis.wfs\12-SNAPSHOT\net.opengis.wfs-12-SNAPSHOT.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools.ogc$net.opengis.wfs\builds\2014-06-04_11-10-19\archive\org.geotools.ogc\net.opengis.wfs\12-SNAPSHOT\net.opengis.wfs-12-SNAPSHOT-sources.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-coverage\builds\2014-06-04_11-10-19\archive\org.geotools\gt-coverage\12-SNAPSHOT\gt-coverage-12-SNAPSHOT.pom
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-coverage\builds\2014-06-04_11-10-19\archive\org.geotools\gt-coverage\12-SNAPSHOT\gt-coverage-12-SNAPSHOT.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-coverage\builds\2014-06-04_11-10-19\archive\org.geotools\gt-coverage\12-SNAPSHOT\gt-coverage-12-SNAPSHOT-sources.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-matfile5\builds\2014-06-04_11-10-19\archive\org.geotools\gt-matfile5\12-SNAPSHOT\gt-matfile5-12-SNAPSHOT.pom
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-matfile5\builds\2014-06-04_11-10-19\archive\org.geotools\gt-matfile5\12-SNAPSHOT\gt-matfile5-12-SNAPSHOT.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-matfile5\builds\2014-06-04_11-10-19\archive\org.geotools\gt-matfile5\12-SNAPSHOT\gt-matfile5-12-SNAPSHOT-sources.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-property-ng\builds\2014-06-04_11-10-19\archive\org.geotools\gt-property-ng\12-SNAPSHOT\gt-property-ng-12-SNAPSHOT.pom
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-property-ng\builds\2014-06-04_11-10-19\archive\org.geotools\gt-property-ng\12-SNAPSHOT\gt-property-ng-12-SNAPSHOT.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-property-ng\builds\2014-06-04_11-10-19\archive\org.geotools\gt-property-ng\12-SNAPSHOT\gt-property-ng-12-SNAPSHOT-sources.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-ogr-bridj\builds\2014-06-04_11-10-19\archive\org.geotools\gt-ogr-bridj\12-SNAPSHOT\gt-ogr-bridj-12-SNAPSHOT.pom
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-ogr-bridj\builds\2014-06-04_11-10-19\archive\org.geotools\gt-ogr-bridj\12-SNAPSHOT\gt-ogr-bridj-12-SNAPSHOT.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-ogr-bridj\builds\2014-06-04_11-10-19\archive\org.geotools\gt-ogr-bridj\12-SNAPSHOT\gt-ogr-bridj-12-SNAPSHOT-sources.jar
[JENKINS] Archiving 

 to 
C:\.jenkins\jobs\GeoTools-Master\modules\org.geotools$gt-graph\builds\201

[Geotools-devel] [jira] (GEOT-4812) Table hints not applied to aggregate queries

2014-06-04 Thread Andrea Aime (JIRA) 
Title: Message Title










 

 Andrea Aime created an issue











 






 GeoTools /  GEOT-4812



  Table hints not applied to aggregate queries 










Issue Type:

  Bug




Assignee:


 Unassigned




Components:


 jdbc




Created:


 04/Jun/14 3:30 AM




Priority:

  Major




Reporter:

 Andrea Aime










The recently introduced table hints are not applied when running aggregate queries, but they should












   

 Add Comment











 










 This message was sent by Atlassian JIRA (v6.1.6#6162-sha1:7af547c)

Re: [Geotools-devel] Upgrading our connection pool library

2014-06-04 Thread Andrea Aime 
On Wed, Jun 4, 2014 at 8:48 AM, Ben Caradoc-Davies <
ben.caradoc-dav...@csiro.au> wrote:

> Connections must be idle for at least 5 minutes (and up to 10) with these
> settings:
>
> minEvictableIdleTimeMillis="30"
> timeBetweenEvictionRunsMillis="30"
>
> If requests are very infrequent, then I agree that they will be much
> slower. For more frequent requests there should be no loss of performance.


That depends. The only way for no loss of performance regardless of the
pool policies
is that we get enough parallel requests over time to keep the pool at full
capacity.

If instead we get constant load of one requests (no parallel ones) the pool
will probably go down to just one connection,
 and it will have to grab new ones when the load goes back up.
That is, unless the connections are extracted in a round robin fashion...
but I'd expect the pool to extract objects using
a LRU policy instead (would make sense for performance considerations, the
data structures of the last used connection are more
likely to be in the CPU cache).

Also, to guarantee the connections are freed before the firewall timeout in
the worst case, the numTestsPerEvictionRun should
be as big as the max connection pool size (e..g, if you have 15
connections, and use numTestsPerEvictionRun=3,
it will take 5 runs of the evictor to free the pool, thus bringing the max
idle time before a connection is surely
evicted to 30 minutes)

Cheers
Andrea


-- 
==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---
--
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
applications. Written by three acclaimed leaders in the field, 
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel