Hi Andrea,
thanks for the feedback, please see my answers bellow :)
On 03/07/2018 03:25 PM, Andrea Aime wrote:
Hi Nuno,
not against but I have a couple of worries.
1) The where clause generated by GeoServer talks about attributes visible when the full query is
run, depending on the query
complexity one might decide to place the where somewhere that only has a subset of such attributes
computed (e..g, case with inner queries).
This will eventually break the query if the wrong attribute is filtered upon. I guess this migth
be dealt with at the documentation level:
"hey dude, we give you this opportunity, the mechanism works like this <lenghly explanation here>,
you can shoot yourself in the foot with it, don't blame us"
As I said:
/
//This comes handy (better performance) when we have extra operations that //
//can be done on top of the rows filtered with the GeoServer filter first./
The :where_clause: clause should always be applied at the point where we have
already all the final attributes (attributes GeoServer is aware of).
I understand your concern, a user may create the view and put the WHERE
place holder in a nested deep query where only a portion of the attributes are
available and then everything will be fine until a new request that uses a non
available attribute is requested ...
Unfortunately there is nothing we can do to prevent that to happen except warn
the user about the correct usage of the placeholder in the documentation.
2) The SQL View mechanism is generic, cross database... will whatever query location work along
with paging and query hints across
all databases? What I'm asking here is to consider all JDBCDataStore implementation, as this
is not "Oracle sql views", but "sql views" :-p
Again, this might be documented, e.g., "Sorry XYZ users, :where_clause: is not
for you"
Analytics operations (DENSE_RANK, FIRST_VALUE, LAST_VALUE, RANK, ROW_NUMBER)
are available in ANSI SQL (maybe I'm wrong). My example is a PostgreSQL query,
I didn't
test it with Oracle but I have similar queries that work that way in Oracle too
...
Cheers
Andera
Summarizing, this is an advanced feature that may come handy in some use cases
(common
enough to be handled by ANSI SQL) but if a user uses it wrong it can bite him
hard :)
On Wed, Mar 7, 2018 at 3:59 PM, Nuno Oliveira <nuno.olive...@geo-solutions.it
<mailto:nuno.olive...@geo-solutions.it>> wrote:
Hi all,
Sorry for the cross porting but this touches the two projects ...
I would like to extend the current support of SQL views (Virtual Tables) to
allow us to add a placeholder for the where clause where GeoServer will
add the needed filter.
This comes handy (better performance) when we have extra operations that
can be done on top of the rows filtered with the GeoServer filter first.
Consider the following use case: we have several meteorological stations
that have several measurements (wind speed, temperature, humidity,
etc ..) and for each one we are only interested in the last measurement for
each available measurement type.
We can create a query that returns exactly what we want:
SELECT STATION_NAME,
MEASUREMENT,
MEASUREMENT_TYPE,
LOCATION
FROM
(SELECT st.common_name AS STATION_NAME,
ob.value AS MEASUREMENT,
pr.param_name AS MEASUREMENT_TYPE,
st.position AS LOCATION,
*ROW_NUMBER() OVER(PARTITION BY st.id <http://st.id>, pr.param_name**
** ORDER BY ob.time DESC) AS RANK*
FROM meteo.meteo_stations st
LEFT JOIN meteo.meteo_observations ob ON st.id <http://st.id> =
ob.station_id
LEFT JOIN meteo.meteo_parameters pr ON ob.parameter_id = pr.id
<http://pr.id>) AS stations
WHERE RANK = 1;
The issue is that this query will NOT be as efficient as it can be when
used with GeoServer,
actually the performance will be quite bad. This happens because GeoServer
will add
is WHERE clause around the query above, which means that the ranking will
happen
for all the data set first and only after we will filter what we don't need.
The following query that uses the :where_clause: placeholder will do the
opposite, we will first
filter what we don't want and then do the ranking:
SELECT STATION_NAME,
MEASUREMENT,
MEASUREMENT_TYPE,
LOCATION
FROM
(SELECT STATION_NAME,
MEASUREMENT,
MEASUREMENT_TYPE,
LOCATION,
*ROW_NUMBER() OVER(PARTITION BY STATION_ID, MEASUREMENT_TYPE*
* ORDER BY TIME DESC) AS RANK*
FROM
(SELECT st.id <http://st.id> AS STATION_ID,
st.common_name AS STATION_NAME,
ob.value AS MEASUREMENT,
pr.param_name AS MEASUREMENT_TYPE,
ob.time AS TIME,
st.position AS LOCATION
FROM meteo.meteo_stations st
LEFT JOIN meteo.meteo_observations ob ON st.id <http://st.id> =
ob.station_id
LEFT JOIN meteo.meteo_parameters pr ON ob.parameter_id = pr.id
<http://pr.id>
WHERE 1 = 1 *:where_clause:*) AS stations_filtered) AS stations
WHERE RANK = 1;
This will make the usage of some SQL analytic functions affordable
(performance wise) with
GeoServer.
The changes in the code are quite small (a few lines), and this is an
additive change fully
backward compatible.
Any objection ? comments ?
Regards,
Nuno Oliveira
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts! Visithttp://goo.gl/it488V
for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone:+39 0584 962313 <tel:+39%200584%20962313>
fax:+39 0584 1660272 <tel:+39%200584%20166%200272>
http://www.geo-solutions.it
http://twitter.com/geosolutions_it <http://twitter.com/geosolutions_it>
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro
utilizzo è consentito esclusivamente al destinatario del messaggio, per le
finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio
senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via
e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal
Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
<mailto:GeoTools-Devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geotools-devel
<https://lists.sourceforge.net/lists/listinfo/geotools-devel>
--
Regards,
Andrea Aime
==GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more
information.==Ing. Andrea Aime @geowolfTechnical LeadGeoSolutions S.A.S.Via di Montramito 3/A55054
Massarosa (LU)phone: +39 0584 962313fax: +39 0584 1660272mob: +39 339
8844549http://www.geo-solutions.ithttp://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i
sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al
destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate
questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via
e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti,
copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi
dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use
of the named addressee(s) and may be confidential or proprietary in nature or covered by the
provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except previous formal
approval of the named addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the information in this message that
has been received in error. The sender does not give any warranty or accept liability as the
content, accuracy or completeness of sent messages and accepts no responsibility for changes made
after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel
