Re: [Qgis-developer] [Qgis-user] Iconset from Google

2014-10-22 Thread Paolo Cavallini
Il 21/10/2014 21:59, kimaidou ha scritto:
 Hi,
 
 I think we could use the same feature as in processing for sharing scripts and
 models. It is based on a git repository, and works quite well. It would be
 sraitforward for devs to add icons, and users would be able to choose which 
 icon set
 they want to download and then upgrade.

Hi Michael,
sounds interesting, as it avoids entirely the complexity of a web-based 
application,
and could be implemented soon, with minimal effort. This holds true for symbols 
and
styles as well.
Any comment? Any taker?
All the best.
-- 
Paolo Cavallini - www.faunalia.eu
Corsi QGIS e PostGIS: http://www.faunalia.eu/training.html
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


Re: [Qgis-developer] 1:many relation enhancements

2014-10-22 Thread Matthias Kuhn

Hi Régis

On 10/21/2014 09:44 PM, HAUBOURG wrote:

Hi Matthias,
Agregate functions could be provided by virtual table feature (qep is coming I 
think), I suppose we should chose one common way for advanced relationnal 
capabilities.


I agree that one common way is preferable, but actually this QEP itself 
is already duplicating functionality (of QgsExpression) and nobody was 
able yet to confirm that it is possible to optimize queries for 
execution on the database with sqlite virtual tables. Something that is 
a must from my point of view. I think it is a good initiative, but for 
the aforementioned reasons I am not yet completely convinced, that it's 
the one and only way.




About filtering, you're right,  we can't assume children table to be geographic.


... and neither for the parent ...


I was thinking of a search bar on top of the dialog showing new children 
candidates. That search bar could have a qgsexpression widget on its right to 
enable a permanent filter + a field chooser to choose on what field apply the 
user entries of search bar. why not have parent geometry accessible in 
expression builder and let the power user make spatial filters?
Yes, once there are expressions available the parent feature should be 
available for evaluation. Now I am not sure if I understood correctly. 
Are you referring to a filter to show only a subset of the related 
children or a filter to search for new children which you want to link 
to the current feature?
I was also thinking of improving the search field to link additional 
features (or change the parent of a feature) but that would mainly make 
sense (in my scenarios) if this could be evaluated server-side and that 
would require a project expression compiler for which I am currently 
also looking for funders :-)

..
Thanks for the tip on qtcreator, I worked on 2.4… now I know it's done ;-)
By now, no direct funding is possible for me (administrative blockers). I wish 
I could finance your work on unit tests first, but it is currently forbidden to 
french administrations..
I thought this barrier mainly exists for funding as such (like in unit 
tests, but thank you for mentioning it), but not for a particular 
contract work with a requirements document for a new feature? But you 
surely know better.


All the best,
Matthias

--
--

Please help taking QGIS to the next level of quality. Before November 15 !
http://blog.vitu.ch/10102014-1046/crowdfunding-initiative-automated-testing

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


Re: [Qgis-developer] [Qgis-user] Iconset from Google

2014-10-22 Thread Alexander Bruy
But we already have core implementation for this (I know, it is not merged
and also require server-side app). Maybe we should compare this two ways
and decide which one offers more functionality and will be more usable for
large symbol collections.

If I'm not wrong, core functionality offers tags, groups and some other nice
features. As reasult one can create complex search criteria. In other hand
solution available in Processing is very simple (AFAIK it was developed as
temporary solution until core functionality is not merged).

Just my 2c.


2014-10-22 9:38 GMT+03:00 Paolo Cavallini cavall...@faunalia.it:
 Il 21/10/2014 21:59, kimaidou ha scritto:
 Hi,

 I think we could use the same feature as in processing for sharing scripts 
 and
 models. It is based on a git repository, and works quite well. It would be
 sraitforward for devs to add icons, and users would be able to choose which 
 icon set
 they want to download and then upgrade.

 Hi Michael,
 sounds interesting, as it avoids entirely the complexity of a web-based 
 application,
 and could be implemented soon, with minimal effort. This holds true for 
 symbols and
 styles as well.
 Any comment? Any taker?
 All the best.
 --
 Paolo Cavallini - www.faunalia.eu
 Corsi QGIS e PostGIS: http://www.faunalia.eu/training.html
 ___
 Qgis-developer mailing list
 Qgis-developer@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/qgis-developer



-- 
Alexander Bruy
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


Re: [Qgis-developer] [Qgis-user] Iconset from Google

2014-10-22 Thread Alessandro Pasotti
2014-10-22 8:38 GMT+02:00 Paolo Cavallini cavall...@faunalia.it:
 Il 21/10/2014 21:59, kimaidou ha scritto:
 Hi,

 I think we could use the same feature as in processing for sharing scripts 
 and
 models. It is based on a git repository, and works quite well. It would be
 sraitforward for devs to add icons, and users would be able to choose which 
 icon set
 they want to download and then upgrade.

 Hi Michael,
 sounds interesting, as it avoids entirely the complexity of a web-based 
 application,
 and could be implemented soon, with minimal effort. This holds true for 
 symbols and
 styles as well.
 Any comment? Any taker?
 All the best.

Hi,

this topic has been addressed several times during the last few years,
the current situation is that a Django app skeleton for a symbology
and styles is already available from a GSOC work at
https://github.com/qgis/QGIS-Django/tree/gsoc_remote/qgis-app/symbols.

Unfortunately, after having examined the code during the HF in Wien we
decided that it was not production ready, and we have no time and
resources to improve the code. A rough estimate of the time required
to improve/complete the app and test/deploy it is 7 working days (for
the symbology application only).

But this would be only part of the development since we still would
require (in any case) the QGIS desktop integration part.

That said, I'm not against using a github based solution, even if I
don't generally like the idea that we depend more and more on external
sources.

An argument in favour of the Django app is that it would probably
allow for a tighter integration with the website, exactly like the
plugin website does.

I don't know anything about the amount of time/resource needed to
complete the github-based solutions though.

-- 
Alessandro Pasotti
w3:   www.itopen.it
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


Re: [Qgis-developer] 1:many relation enhancements

2014-10-22 Thread Hugo Mercier
Le 22/10/2014 08:39, Matthias Kuhn a écrit :
 Hi Régis
 
 On 10/21/2014 09:44 PM, HAUBOURG wrote:
 Hi Matthias,
 Agregate functions could be provided by virtual table feature (qep is
 coming I think), I suppose we should chose one common way for advanced
 relationnal capabilities.
 
 I agree that one common way is preferable, but actually this QEP itself
 is already duplicating functionality (of QgsExpression) and nobody was
 able yet to confirm that it is possible to optimize queries for
 execution on the database with sqlite virtual tables. Something that is
 a must from my point of view. I think it is a good initiative, but for
 the aforementioned reasons I am not yet completely convinced, that it's
 the one and only way.

Hi,

Since I'm the author of the mentioned QEP
(https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5), I have to
answer :)

What functionalities of expressions do you think it duplicates ?

I don't see anything that prevents virtual layer queries from being
optimized server-side. It has to be taken into account during the
implementation of virtual layers, of course, but it is hard to exhibit
the precise way it will work now without having an almost complete
implementation.
Anyway, it will try to complete my proposal with some more concrete
material about that when I find time.


___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


Re: [Qgis-developer] 1:many relation enhancements

2014-10-22 Thread Matthias Kuhn
Hi Hugo,

On 22.10.2014 09:13, Hugo Mercier wrote:
 Le 22/10/2014 08:39, Matthias Kuhn a écrit :
 Hi Régis

 On 10/21/2014 09:44 PM, HAUBOURG wrote:
 Hi Matthias,
 Agregate functions could be provided by virtual table feature (qep is
 coming I think), I suppose we should chose one common way for advanced
 relationnal capabilities.
 I agree that one common way is preferable, but actually this QEP itself
 is already duplicating functionality (of QgsExpression) and nobody was
 able yet to confirm that it is possible to optimize queries for
 execution on the database with sqlite virtual tables. Something that is
 a must from my point of view. I think it is a good initiative, but for
 the aforementioned reasons I am not yet completely convinced, that it's
 the one and only way.
 Hi,

 Since I'm the author of the mentioned QEP
 (https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5), I have to
 answer :)

 What functionalities of expressions do you think it duplicates ?

Almost all (IN, LIKE, ILIKE, *, +, / just to name a few)
:)


 I don't see anything that prevents virtual layer queries from being
 optimized server-side. It has to be taken into account during the
 implementation of virtual layers, of course, but it is hard to exhibit
 the precise way it will work now without having an almost complete
 implementation.
I was referring to this discussion here.

https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5#issuecomment-58148788

Sorry I did not respond again, but I don't think a complete optimization
is possible without having deep access to virtual table functionality in
sqlite (or having to parse the same SQL on our end and replace parts of
it which basically comes close to writing our own engine again).
But let's discuss this on the QEP PR instead of inside this ml thread.

 Anyway, it will try to complete my proposal with some more concrete
 material about that when I find time.
I'll make sure I'll have a look at it when I find time after you found
time :)

Best
Matthias

-- 
Help getting QGIS to the next level of quality before November 15!
http://blog.vitu.ch/10102014-1046/crowdfunding-initiative-automated-testing

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] [Qgis-user] Iconset from Google

2014-10-22 Thread Alexander Bruy
I remember Nathan said that desktop part is in good shape and can be easily
merged. But maybe with latest API breaks situation changed.

2014-10-22 9:54 GMT+03:00 Alessandro Pasotti apaso...@gmail.com:
 2014-10-22 8:38 GMT+02:00 Paolo Cavallini cavall...@faunalia.it:
 Il 21/10/2014 21:59, kimaidou ha scritto:
 Hi,

 I think we could use the same feature as in processing for sharing scripts 
 and
 models. It is based on a git repository, and works quite well. It would be
 sraitforward for devs to add icons, and users would be able to choose which 
 icon set
 they want to download and then upgrade.

 Hi Michael,
 sounds interesting, as it avoids entirely the complexity of a web-based 
 application,
 and could be implemented soon, with minimal effort. This holds true for 
 symbols and
 styles as well.
 Any comment? Any taker?
 All the best.

 Hi,

 this topic has been addressed several times during the last few years,
 the current situation is that a Django app skeleton for a symbology
 and styles is already available from a GSOC work at
 https://github.com/qgis/QGIS-Django/tree/gsoc_remote/qgis-app/symbols.

 Unfortunately, after having examined the code during the HF in Wien we
 decided that it was not production ready, and we have no time and
 resources to improve the code. A rough estimate of the time required
 to improve/complete the app and test/deploy it is 7 working days (for
 the symbology application only).

 But this would be only part of the development since we still would
 require (in any case) the QGIS desktop integration part.

 That said, I'm not against using a github based solution, even if I
 don't generally like the idea that we depend more and more on external
 sources.

 An argument in favour of the Django app is that it would probably
 allow for a tighter integration with the website, exactly like the
 plugin website does.

 I don't know anything about the amount of time/resource needed to
 complete the github-based solutions though.

 --
 Alessandro Pasotti
 w3:   www.itopen.it
 ___
 Qgis-developer mailing list
 Qgis-developer@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/qgis-developer



-- 
Alexander Bruy
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


Re: [Qgis-developer] 1:many relation enhancements

2014-10-22 Thread Even Rouault
Le mercredi 22 octobre 2014 09:46:23, Matthias Kuhn a écrit :
 Hi Hugo,
 
 On 22.10.2014 09:13, Hugo Mercier wrote:
  Le 22/10/2014 08:39, Matthias Kuhn a écrit :
  Hi Régis
  
  On 10/21/2014 09:44 PM, HAUBOURG wrote:
  Hi Matthias,
  Agregate functions could be provided by virtual table feature (qep is
  coming I think), I suppose we should chose one common way for advanced
  relationnal capabilities.
  
  I agree that one common way is preferable, but actually this QEP itself
  is already duplicating functionality (of QgsExpression) and nobody was
  able yet to confirm that it is possible to optimize queries for
  execution on the database with sqlite virtual tables. Something that is
  a must from my point of view. I think it is a good initiative, but for
  the aforementioned reasons I am not yet completely convinced, that it's
  the one and only way.
  
  Hi,
  
  Since I'm the author of the mentioned QEP
  (https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5), I have to
  answer :)
  
  What functionalities of expressions do you think it duplicates ?
 
 Almost all (IN, LIKE, ILIKE, *, +, / just to name a few)
 
 :)
 :
  I don't see anything that prevents virtual layer queries from being
  optimized server-side. It has to be taken into account during the
  implementation of virtual layers, of course, but it is hard to exhibit
  the precise way it will work now without having an almost complete
  implementation.
 
 I was referring to this discussion here.
 
 https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5#issuecomment-5814
 8788
 
 Sorry I did not respond again, but I don't think a complete optimization
 is possible without having deep access to virtual table functionality in
 sqlite (or having to parse the same SQL on our end and replace parts of
 it which basically comes close to writing our own engine again).
 But let's discuss this on the QEP PR instead of inside this ml thread.

Just to give you my feeback based on my implementation of SQLite SQL dialect 
in OGR ( 
https://github.com/OSGeo/gdal/blob/trunk/gdal/ogr/ogrsf_frmts/sqlite/ogrsqlitevirtualogr.cpp
 
), I also doubt that you could easily inform the server that a join should be 
done. AFAIR I could forward simple filtering to the server/backend (things like 
column_A = 5 AND column_A = 10 AND column_B != 10) by implementing the 
xFilter virtual method, but Virtual Tables are not informed of joins happening 
(the sqlite virtual table API hardy exposes SQL at all). So that would indeed 
require analyzing the SQL on QGIS side.

AFAIR, I've just verified that JOIN like t1.col1 = t2.col2 are seen by the 
virtual table implementation like successive filters t1.col1 = val1, t1.col1 
= val2, where val1, val2 are values from t2.col2.

And regarding xFilter, even filters with OR etc.. are hidden to the virtual 
table implementation and evaluated only by SQLite.

Even

 
  Anyway, it will try to complete my proposal with some more concrete
  material about that when I find time.
 
 I'll make sure I'll have a look at it when I find time after you found
 time :)
 
 Best
 Matthias

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] 1:many relation enhancements

2014-10-22 Thread Hugo Mercier
Le 22/10/2014 10:12, Even Rouault a écrit :
 Le mercredi 22 octobre 2014 09:46:23, Matthias Kuhn a écrit :
 Hi Hugo,

 On 22.10.2014 09:13, Hugo Mercier wrote:
 Le 22/10/2014 08:39, Matthias Kuhn a écrit :
 Hi Régis

 On 10/21/2014 09:44 PM, HAUBOURG wrote:
 Hi Matthias,
 Agregate functions could be provided by virtual table feature (qep is
 coming I think), I suppose we should chose one common way for advanced
 relationnal capabilities.

 I agree that one common way is preferable, but actually this QEP itself
 is already duplicating functionality (of QgsExpression) and nobody was
 able yet to confirm that it is possible to optimize queries for
 execution on the database with sqlite virtual tables. Something that is
 a must from my point of view. I think it is a good initiative, but for
 the aforementioned reasons I am not yet completely convinced, that it's
 the one and only way.

 Hi,

 Since I'm the author of the mentioned QEP
 (https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5), I have to
 answer :)

 What functionalities of expressions do you think it duplicates ?

 Almost all (IN, LIKE, ILIKE, *, +, / just to name a few)

Well, QgsExpression has been designed to mimick the SQL WHERE clause, so
yes I agree.
My point is that if you want to extend expression to support joins,
aggregates, window functions, etc. you end up recreating SQL, but
probably a less powerful one.

The difference I can see between expressions and SQL is that functions
used in expressions may have no direct equivalent SQLite-side. But it
can be resolved by using user-defined functions in SQLite (that would
just call code from QgsExpression for instance), so that the SQL dialect
would be seen as an extension of the current QgsExpression, not
something different.


 :)
 :
 I don't see anything that prevents virtual layer queries from being
 optimized server-side. It has to be taken into account during the
 implementation of virtual layers, of course, but it is hard to exhibit
 the precise way it will work now without having an almost complete
 implementation.

 I was referring to this discussion here.

 https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5#issuecomment-5814
 8788

 Sorry I did not respond again, but I don't think a complete optimization
 is possible without having deep access to virtual table functionality in
 sqlite (or having to parse the same SQL on our end and replace parts of
 it which basically comes close to writing our own engine again).
 But let's discuss this on the QEP PR instead of inside this ml thread.
 
 Just to give you my feeback based on my implementation of SQLite SQL dialect 
 in OGR ( 
 https://github.com/OSGeo/gdal/blob/trunk/gdal/ogr/ogrsf_frmts/sqlite/ogrsqlitevirtualogr.cpp
  
 ), I also doubt that you could easily inform the server that a join should be 
 done. AFAIR I could forward simple filtering to the server/backend (things 
 like 
 column_A = 5 AND column_A = 10 AND column_B != 10) by implementing the 
 xFilter virtual method, but Virtual Tables are not informed of joins 
 happening 
 (the sqlite virtual table API hardy exposes SQL at all). So that would indeed 
 require analyzing the SQL on QGIS side.
 
 AFAIR, I've just verified that JOIN like t1.col1 = t2.col2 are seen by the 
 virtual table implementation like successive filters t1.col1 = val1, 
 t1.col1 
 = val2, where val1, val2 are values from t2.col2.
 
 And regarding xFilter, even filters with OR etc.. are hidden to the virtual 
 table implementation and evaluated only by SQLite.
 

Thanks Even for your feedback.

The idea under virtual layers is that SQLite may be exposed to advanced
users if they want to use a complex query on some data sources.

It is very interesting to use SQLite to replace existing features from a
code point of view because it simplifies implementation.
But exposing a SQL query for already existing features such as joins may
not be desirable for standard users.

So for joins, since the information comes from QGIS (I want to join
tables t1 and t2 on this column predicate), the SQL query can be
deduced and some other metadata is also known, like the fact that t1 and
t2 may be from the very same database.
In that case the virtual layer may be created with some metadata saying
that this is in fact a join that must be optimized. And then the virtual
layer provider can ask the underlying provider (say postgis) if it can
process the whole join query.


___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] QEP/RFC sqlite virtual tables

2014-10-22 Thread Matthias Kuhn
Moving this to a new thread (from 1:many relation...

On 22.10.2014 10:38, Hugo Mercier wrote:
 Well, QgsExpression has been designed to mimick the SQL WHERE clause, so
 yes I agree.
 My point is that if you want to extend expression to support joins,
 aggregates, window functions, etc. you end up recreating SQL, but
 probably a less powerful one.
You can call it mimicking, you can also call it implementing it's own
SQL (subset) dialect.
Yes, you end up recreating SQL. How powerful it will be is hard to say
right now, right?

 The difference I can see between expressions and SQL is that functions
 used in expressions may have no direct equivalent SQLite-side. But it
 can be resolved by using user-defined functions in SQLite (that would
 just call code from QgsExpression for instance), so that the SQL dialect
 would be seen as an extension of the current QgsExpression, not
 something different.

 :)
 :
 I don't see anything that prevents virtual layer queries from being
 optimized server-side. It has to be taken into account during the
 implementation of virtual layers, of course, but it is hard to exhibit
 the precise way it will work now without having an almost complete
 implementation.
 I was referring to this discussion here.

 https://github.com/qgis/QGIS-Enhancement-Proposals/pull/5#issuecomment-5814
 8788

 Sorry I did not respond again, but I don't think a complete optimization
 is possible without having deep access to virtual table functionality in
 sqlite (or having to parse the same SQL on our end and replace parts of
 it which basically comes close to writing our own engine again).
 But let's discuss this on the QEP PR instead of inside this ml thread.
 Just to give you my feeback based on my implementation of SQLite SQL dialect 
 in OGR ( 
 https://github.com/OSGeo/gdal/blob/trunk/gdal/ogr/ogrsf_frmts/sqlite/ogrsqlitevirtualogr.cpp
  
 ), I also doubt that you could easily inform the server that a join should 
 be 
 done. AFAIR I could forward simple filtering to the server/backend (things 
 like 
 column_A = 5 AND column_A = 10 AND column_B != 10) by implementing the 
 xFilter virtual method, but Virtual Tables are not informed of joins 
 happening 
 (the sqlite virtual table API hardy exposes SQL at all). So that would 
 indeed 
 require analyzing the SQL on QGIS side.

 AFAIR, I've just verified that JOIN like t1.col1 = t2.col2 are seen by the 
 virtual table implementation like successive filters t1.col1 = val1, 
 t1.col1 
 = val2, where val1, val2 are values from t2.col2.

 And regarding xFilter, even filters with OR etc.. are hidden to the virtual 
 table implementation and evaluated only by SQLite.

 Thanks Even for your feedback.

 The idea under virtual layers is that SQLite may be exposed to advanced
 users if they want to use a complex query on some data sources.

 It is very interesting to use SQLite to replace existing features from a
 code point of view because it simplifies implementation.
 But exposing a SQL query for already existing features such as joins may
 not be desirable for standard users.

 So for joins, since the information comes from QGIS (I want to join
 tables t1 and t2 on this column predicate), the SQL query can be
 deduced and some other metadata is also known, like the fact that t1 and
 t2 may be from the very same database.
 In that case the virtual layer may be created with some metadata saying
 that this is in fact a join that must be optimized. And then the virtual
 layer provider can ask the underlying provider (say postgis) if it can
 process the whole join query.

I am not only referring to the current relations implementation here. I
don't think aggregate functions need necessarily be based on creating a
relation definition in the project file (Just like in a database you
also don't need a foreign key constraint to create join).

But if I want to create a new layer or an ad-hoc iterator that joins
data or makes use of other advanced relational database features, it
would be nice if QGIS could forward such requests to the database. Say I
want to create a symbology based on the amount of waste within
countries, QGIS should be able to execute this request on the database.
In this scenario you could likely want to define this request inside an
expression (e.g. a virtual column).
So basically, I think we need a functionality that is able to work as a
database abstraction layer. Forward wherever possible to the db, use a
(local) fallback implementation where not.

I am not opposed to use SQLite for this. There is no reason for us to
reinvent the wheel, the NIH syndrome should not be our driver. But I see
some limitations and I would rather prefer to have these properly
addressed. And I will be more than happy if you tell me that you
convinced the sqlite virtual table developers that they should extend
their API to expose the parsed SQL structure and allow to tweak it. Then
we have a fallback implementation which is well-tested but we still have
the 

Re: [Qgis-developer] QEP/RFC sqlite virtual tables

2014-10-22 Thread Hugo Mercier
Le 22/10/2014 11:21, Matthias Kuhn a écrit :
 On 22.10.2014 10:38, Hugo Mercier wrote:
 Well, QgsExpression has been designed to mimick the SQL WHERE clause, so
 yes I agree.
 My point is that if you want to extend expression to support joins,
 aggregates, window functions, etc. you end up recreating SQL, but
 probably a less powerful one.
 You can call it mimicking, you can also call it implementing it's own
 SQL (subset) dialect.

Sorry for my english, mimicking was not a negative term for me.

 Yes, you end up recreating SQL. How powerful it will be is hard to say
 right now, right?

Sure. But if you want it to be as powerful as other SQL engines, lots of
work is needed (parsing, planning, optimizing). Work that has already
been done, I think.


 So for joins, since the information comes from QGIS (I want to join
 tables t1 and t2 on this column predicate), the SQL query can be
 deduced and some other metadata is also known, like the fact that t1 and
 t2 may be from the very same database.
 In that case the virtual layer may be created with some metadata saying
 that this is in fact a join that must be optimized. And then the virtual
 layer provider can ask the underlying provider (say postgis) if it can
 process the whole join query.
 
 I am not only referring to the current relations implementation here. I
 don't think aggregate functions need necessarily be based on creating a
 relation definition in the project file (Just like in a database you
 also don't need a foreign key constraint to create join).

Not sure to understand. You mean there would be no need to create a
virtual layer just to execute an aggregate ? Exact.
But if the virtual layer concept is complete enough, then every single
layer can be seen as a special case of a (trivial) virtual layer, then
to create an aggregate, you won't need to create a new virtual layer.

 
 But if I want to create a new layer or an ad-hoc iterator that joins
 data or makes use of other advanced relational database features, it
 would be nice if QGIS could forward such requests to the database. Say I
 want to create a symbology based on the amount of waste within
 countries, QGIS should be able to execute this request on the database.
 In this scenario you could likely want to define this request inside an
 expression (e.g. a virtual column).
 So basically, I think we need a functionality that is able to work as a
 database abstraction layer. Forward wherever possible to the db, use a
 (local) fallback implementation where not.

I agree, but can you be more specific on your example ? How many tables
involved ? Which fields ? What kind of joins ?

 
 I am not opposed to use SQLite for this. There is no reason for us to
 reinvent the wheel, the NIH syndrome should not be our driver. But I see
 some limitations and I would rather prefer to have these properly
 addressed. And I will be more than happy if you tell me that you
 convinced the sqlite virtual table developers that they should extend
 their API to expose the parsed SQL structure and allow to tweak it. Then
 we have a fallback implementation which is well-tested but we still have
 the possibility to fine-tune requests for big databases.

Splitting the original query in local and remote queries seems to me
very hard in the general case. It would require not only the parsed SQL
AST, but some information about the execution plan. And the plan may
depends on statistics on the data. And even when you know the plan, I
think it is still hard to translate.

Generally speaking if the user tries to do a very complex query with
lots of different data sources, it is hard to guarantee good
performances, but I don't see it as a major problem. If you want
something complex AND fast, then use SQLite or PostGIS.

So, I would vote for optimizations that can be done for simple, defined
scenarios, like the join case given by the current join or relation
feature of QGIS : no nested queries, no group by, no order by, no CTE,
etc. That particular case could be detected when the join is created,
and optimizations can be created, like creating a (temporary) postgis
layer with sql=the join query.

I may be missing something. Are there other simple cases where the
optimization is well defined ?

To put it differently, I don't see the lack of knowledge about parsed
SQL query as a blocker for the implementation of this virtual layer
concept. A general cross-database SQL optimizer would need lots of work
(in QGIS and/or SQLite) and may come later (if really needed).

The other option would be to say we don't need everything SQLite
provides, and we won't never need it. So then we can define our own
subset of SQL and it would probably be easier to optimize. But I can't
see why we would want to limit the features :) it is usually the opposite.

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


Re: [Qgis-developer] [Qgis-user] Iconset from Google

2014-10-22 Thread Paolo Cavallini
Il 22/10/2014 10:03, Alexander Bruy ha scritto:
 I remember Nathan said that desktop part is in good shape and can be easily
 merged. But maybe with latest API breaks situation changed.

IMVHO we need this function a lot, and we need to be realistic: of course a 
fully
integrated solution, entirely under our control is by far preferable, but if 
nobody i
going to implement it anytime soon, a temporary solution would be good, in 
order to
start building our symbol collection, that can be migrated to the Great Final
Solution when ready.
All the best.

-- 
Paolo Cavallini - www.faunalia.eu
Corsi QGIS e PostGIS: http://www.faunalia.eu/training.html
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer