Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-05 Thread Nyall Dawson 
On Sat, 1 Sep 2018 at 19:56, Nyall Dawson  wrote:

> 2. I think there may be a real issue here - I've got at least one
> customer who is having WFS issues with 3.2 and master. BUT: on the
> other hand Travis has always been flaky with any test which uses
> threads, regardless of which area of code it's from. So it could just
> be Travis playing up again, in which case we'd need to disable these
> tests like we do most of the other thread-related tests...

For those following at home -- I think the underlying issue (a bug)
has been found and fixed... fingers crossed!

Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-04 Thread Tim Sutton 
Hi



> On 03 Sep 2018, at 22:54, Jonathan Moules  
> wrote:
> 
> I'd suggest based on Tom's post that the work should include some sort of 
> reliable way of testing the WFS client (and of course full test coverage) - I 
> don't know if GeoServer is Docker-happy these days
> 

Kartoza publishes geoserver images here:

https://hub.docker.com/r/kartoza/geoserver/tags/ 


https://github.com/kartoza/docker-geoserver 


Regards

Tim


> , but the default install in a VM should be sufficient for the task (it comes 
> with test layers) - although GeoServer doesn't support WFS-T for writing.
> Cheers,
> Jonathan
> 
> On 03/09/2018 12:04, Régis Haubourg wrote:
>> Hi all,
>> I very much think that the WFS client is an really bad state, and is not 
>> really reliable, especially in WFS-T context.
>> The good news is that we just have been funded to refactor it !
>> The work should start in september and land in 3.6. I will let our dev's 
>> come here with more technical details about the goals. I hope we will also 
>> be able to take benefit of this to this the OGC compliancy of the client 
>> here.
>> Best regards,
>> Régis
>> 
>> Le lun. 3 sept. 2018 à 11:36, Tom Chadwin > > a écrit :
>> I can't offer any helpful suggestions, but just to let you know I finally had
>> to disable all my plugin WFS tests. I used to cope, by rerunning failed
>> Travis runs, but by about three months ago, it seemed no longer usable -
>> failure after failure.
>> 
>> I was using a third-party WFS, and perhaps I could have got round this by
>> adding a WFS provider to the test docker image, but in this plugin's case, I
>> didn't think it worth the significant effort to do so. The WM(T)S tests also
>> use third-party sources and seem stable, so perhaps this wasn't the
>> underlying issue anyway.
>> 
>> If an improvement or solution could be found, it would be great to reinstate
>> these tests.
>> 
>> Thanks
>> 
>> Tom
>> 
>> 
>> 
>> -
>> Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
>> --
>> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html 
>> 
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org 
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
>> 
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org 
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
>> 
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

—








Tim Sutton

Co-founder: Kartoza
Ex Project chair: QGIS.org

Visit http://kartoza.com  to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux
IRC: timlinux on #qgis at freenode.net



signature.asc
Description: Message signed with OpenPGP
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-04 Thread Régis Haubourg 
Hi,
I just checked the goals of the work on the WFS provider. It is not a
refactor in fat, sorry for having used that word. The work consists in
having a closer look to the WFS and snapping cache interactions because
they cause lots of problems when the distant database has triggers that
would require WFS cache reloading and then snapping cache reloading.
Maybe Vincent can tell more about that, but it's not a major change then. I
should'nt have raised the hand here, sorry for the noise.
Regards,
Régis

Le mar. 4 sept. 2018 à 08:51, Jonathan Moules 
a écrit :

> I'd suggest based on Tom's post that the work should include some sort of
> reliable way of testing the WFS client (and of course full test coverage) -
> I don't know if GeoServer is Docker-happy these days, but the default
> install in a VM should be sufficient for the task (it comes with test
> layers) - although GeoServer doesn't support WFS-T for writing.
> Cheers,
> Jonathan
>
> On 03/09/2018 12:04, Régis Haubourg wrote:
>
> Hi all,
> I very much think that the WFS client is an really bad state, and is not
> really reliable, especially in WFS-T context.
> The good news is that we just have been funded to refactor it !
> The work should start in september and land in 3.6. I will let our dev's
> come here with more technical details about the goals. I hope we will also
> be able to take benefit of this to this the OGC compliancy of the client
> here.
> Best regards,
> Régis
>
> Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
> écrit :
>
>> I can't offer any helpful suggestions, but just to let you know I finally
>> had
>> to disable all my plugin WFS tests. I used to cope, by rerunning failed
>> Travis runs, but by about three months ago, it seemed no longer usable -
>> failure after failure.
>>
>> I was using a third-party WFS, and perhaps I could have got round this by
>> adding a WFS provider to the test docker image, but in this plugin's
>> case, I
>> didn't think it worth the significant effort to do so. The WM(T)S tests
>> also
>> use third-party sources and seem stable, so perhaps this wasn't the
>> underlying issue anyway.
>>
>> If an improvement or solution could be found, it would be great to
>> reinstate
>> these tests.
>>
>> Thanks
>>
>> Tom
>>
>>
>>
>> -
>> Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
>> --
>> Sent from:
>> http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> ___
> QGIS-Developer mailing listqgis-develo...@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-04 Thread Andrea Aime 
On Tue, Sep 4, 2018 at 8:51 AM Jonathan Moules 
wrote:

> I'd suggest based on Tom's post that the work should include some sort of
> reliable way of testing the WFS client (and of course full test coverage) -
> I don't know if GeoServer is Docker-happy these days, but the default
> install in a VM should be sufficient for the task (it comes with test
> layers) - although GeoServer doesn't support WFS-T for writing.
>
"GeoServer doesn't support WFS-T for writing" ... what??? :-)

The default data directory disables writes for any user but the
administrator, you can change it, or just authenticate
as the admin to run WFS-T

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information. == Ing. Andrea Aime @geowolf Technical Lead
GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39
0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
http://www.geo-solutions.it http://twitter.com/geosolutions_it
--- *Con riferimento
alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto, gli eventuali
allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
sarei comunque grato se potesse darmene notizia. This email is intended
only for the person or entity to which it is addressed and may contain
information that is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European Regulation 2016/679
“GDPR” - copying, dissemination or use of this e-mail or the information
herein by anyone other than the intended recipient is prohibited. If you
have received this email by mistake, please notify us immediately by
telephone or e-mail.*
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-04 Thread Jonathan Moules 
I'd suggest based on Tom's post that the work should include some sort 
of reliable way of testing the WFS client (and of course full test 
coverage) - I don't know if GeoServer is Docker-happy these days, but 
the default install in a VM should be sufficient for the task (it comes 
with test layers) - although GeoServer doesn't support WFS-T for writing.


Cheers,
Jonathan

On 03/09/2018 12:04, Régis Haubourg wrote:

Hi all,
I very much think that the WFS client is an really bad state, and is 
not really reliable, especially in WFS-T context.

The good news is that we just have been funded to refactor it !
The work should start in september and land in 3.6. I will let our 
dev's come here with more technical details about the goals. I hope we 
will also be able to take benefit of this to this the OGC compliancy 
of the client here.

Best regards,
Régis

Le lun. 3 sept. 2018 à 11:36, Tom Chadwin > a écrit :


I can't offer any helpful suggestions, but just to let you know I
finally had
to disable all my plugin WFS tests. I used to cope, by rerunning
failed
Travis runs, but by about three months ago, it seemed no longer
usable -
failure after failure.

I was using a third-party WFS, and perhaps I could have got round
this by
adding a WFS provider to the test docker image, but in this
plugin's case, I
didn't think it worth the significant effort to do so. The WM(T)S
tests also
use third-party sources and seem stable, so perhaps this wasn't the
underlying issue anyway.

If an improvement or solution could be found, it would be great to
reinstate
these tests.

Thanks

Tom



-
Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
--
Sent from:
http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org 
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer



___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer


___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Nyall Dawson 
On Tue, 4 Sep 2018 at 05:50, Even Rouault  wrote:
>
> Régis,
>
> Not that I'm against improvements, all the contrary, but just wanted to
> underline that the provider was seriously refactored already in 2.16. Clearly
> the lack of WFS-T support for 1.1 and 2.0 in the scope of those enhancements
> can be a source of confusion currently for users. What do you have in mind as
> refactoring exactly ?

I'm with Even here... please, proceed with extreme caution.

We may be seeing issues with the WFS provider since 3.2, but I do not
think a full rewrite is needed/wanted here, and potentially will cause
many regressions given how finicky WFS servers are and how many
server-specific fixes have been required in the current
implementation.

Nyall


>
> Even
>
> > Hi all,
> > I very much think that the WFS client is an really bad state, and is not
> > really reliable, especially in WFS-T context.
> > The good news is that we just have been funded to refactor it !
> > The work should start in september and land in 3.6. I will let our dev's
> > come here with more technical details about the goals. I hope we will also
> > be able to take benefit of this to this the OGC compliancy of the client
> > here.
> > Best regards,
> > Régis
> >
> > Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
> >
> > écrit :
> > > I can't offer any helpful suggestions, but just to let you know I finally
> > > had
> > > to disable all my plugin WFS tests. I used to cope, by rerunning failed
> > > Travis runs, but by about three months ago, it seemed no longer usable -
> > > failure after failure.
> > >
> > > I was using a third-party WFS, and perhaps I could have got round this by
> > > adding a WFS provider to the test docker image, but in this plugin's case,
> > > I
> > > didn't think it worth the significant effort to do so. The WM(T)S tests
> > > also
> > > use third-party sources and seem stable, so perhaps this wasn't the
> > > underlying issue anyway.
> > >
> > > If an improvement or solution could be found, it would be great to
> > > reinstate
> > > these tests.
> > >
> > > Thanks
> > >
> > > Tom
> > >
> > >
> > >
> > > -
> > > Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
> > > --
> > > Sent from:
> > > http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
> > > ___
> > > QGIS-Developer mailing list
> > > QGIS-Developer@lists.osgeo.org
> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Jeremy Palmer 
I agree WFS 3.0 is a much better implementation and it would be great if a
implementation is started soon to track the current standards development.
However, we still have (and will have for a long time) a user need to
support WFS 1.0 and 2.0 - so this still needs to be deal with.

On Tue, Sep 4, 2018 at 8:32 AM Carlo A. Bertelli (Charta s.r.l.) <
carlo.berte...@gmail.com> wrote:

> What about starting with/focusing on WFS 3? The new version is really
> cleaner and seems much more efficient.
> The current WFS implementation in QGIS is much better than previous
> versions, even if sometimes making a virtual OGR file il the only way to
> use some services. There are really bad server implementations out there,
> additionally an automated solution struggles against misleading and lazy
> XML informations. This is a broken idea or at least one that asks for close
> cooperation between server and client.
> Just my remaining cent.
> c
>
>
> On Mon, Sep 3, 2018 at 10:01 PM, Jeremy Palmer 
> wrote:
>
>> Can I also add that the refactoring that was funded in 2.16 added tests,
>> paging support, filter query builder and dynamic caching. There is now a
>> lot of complexity in the driver due to the complexity of WFS server
>> implementations and the standard, plus the multi-threading code. If a
>> refactor is proposed I would be against anything that doesn't deal current
>> use cases and edge cases which have already been implemented. Maybe some
>> analysis of the failed tests is the first place to start. WFS-T is a side
>> issue to that.
>>
>> Cheers,
>> Jeremy
>>
>>
>> On Tue, Sep 4, 2018 at 7:51 AM Régis Haubourg 
>> wrote:
>>
>>> Hi Even, thanks for pointing that, I missed that history.
>>> I'll ask the dev's for the detailed improvements planned, I dont' have
>>> any detail currently (sorry for that)
>>> Régis
>>>
>>> Le lun. 3 sept. 2018 à 21:29, Even Rouault 
>>> a écrit :
>>>
 Régis,

 Not that I'm against improvements, all the contrary, but just wanted to
 underline that the provider was seriously refactored already in 2.16.
 Clearly
 the lack of WFS-T support for 1.1 and 2.0 in the scope of those
 enhancements
 can be a source of confusion currently for users. What do you have in
 mind as
 refactoring exactly ?

 Even

 > Hi all,
 > I very much think that the WFS client is an really bad state, and is
 not
 > really reliable, especially in WFS-T context.
 > The good news is that we just have been funded to refactor it !
 > The work should start in september and land in 3.6. I will let our
 dev's
 > come here with more technical details about the goals. I hope we will
 also
 > be able to take benefit of this to this the OGC compliancy of the
 client
 > here.
 > Best regards,
 > Régis
 >
 > Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
 >
 > écrit :
 > > I can't offer any helpful suggestions, but just to let you know I
 finally
 > > had
 > > to disable all my plugin WFS tests. I used to cope, by rerunning
 failed
 > > Travis runs, but by about three months ago, it seemed no longer
 usable -
 > > failure after failure.
 > >
 > > I was using a third-party WFS, and perhaps I could have got round
 this by
 > > adding a WFS provider to the test docker image, but in this
 plugin's case,
 > > I
 > > didn't think it worth the significant effort to do so. The WM(T)S
 tests
 > > also
 > > use third-party sources and seem stable, so perhaps this wasn't the
 > > underlying issue anyway.
 > >
 > > If an improvement or solution could be found, it would be great to
 > > reinstate
 > > these tests.
 > >
 > > Thanks
 > >
 > > Tom
 > >
 > >
 > >
 > > -
 > > Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
 > > --
 > > Sent from:
 > > http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
 > > ___
 > > QGIS-Developer mailing list
 > > QGIS-Developer@lists.osgeo.org
 > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
 > > Unsubscribe:
 https://lists.osgeo.org/mailman/listinfo/qgis-developer


 --
 Spatialys - Geospatial professional services
 http://www.spatialys.com

>>> ___
>>> QGIS-Developer mailing list
>>> QGIS-Developer@lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>>
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>
>
>
> --
>

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Carlo A. Bertelli (Charta s.r.l.) 
What about starting with/focusing on WFS 3? The new version is really
cleaner and seems much more efficient.
The current WFS implementation in QGIS is much better than previous
versions, even if sometimes making a virtual OGR file il the only way to
use some services. There are really bad server implementations out there,
additionally an automated solution struggles against misleading and lazy
XML informations. This is a broken idea or at least one that asks for close
cooperation between server and client.
Just my remaining cent.
c


On Mon, Sep 3, 2018 at 10:01 PM, Jeremy Palmer  wrote:

> Can I also add that the refactoring that was funded in 2.16 added tests,
> paging support, filter query builder and dynamic caching. There is now a
> lot of complexity in the driver due to the complexity of WFS server
> implementations and the standard, plus the multi-threading code. If a
> refactor is proposed I would be against anything that doesn't deal current
> use cases and edge cases which have already been implemented. Maybe some
> analysis of the failed tests is the first place to start. WFS-T is a side
> issue to that.
>
> Cheers,
> Jeremy
>
>
> On Tue, Sep 4, 2018 at 7:51 AM Régis Haubourg 
> wrote:
>
>> Hi Even, thanks for pointing that, I missed that history.
>> I'll ask the dev's for the detailed improvements planned, I dont' have
>> any detail currently (sorry for that)
>> Régis
>>
>> Le lun. 3 sept. 2018 à 21:29, Even Rouault 
>> a écrit :
>>
>>> Régis,
>>>
>>> Not that I'm against improvements, all the contrary, but just wanted to
>>> underline that the provider was seriously refactored already in 2.16.
>>> Clearly
>>> the lack of WFS-T support for 1.1 and 2.0 in the scope of those
>>> enhancements
>>> can be a source of confusion currently for users. What do you have in
>>> mind as
>>> refactoring exactly ?
>>>
>>> Even
>>>
>>> > Hi all,
>>> > I very much think that the WFS client is an really bad state, and is
>>> not
>>> > really reliable, especially in WFS-T context.
>>> > The good news is that we just have been funded to refactor it !
>>> > The work should start in september and land in 3.6. I will let our
>>> dev's
>>> > come here with more technical details about the goals. I hope we will
>>> also
>>> > be able to take benefit of this to this the OGC compliancy of the
>>> client
>>> > here.
>>> > Best regards,
>>> > Régis
>>> >
>>> > Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
>>> >
>>> > écrit :
>>> > > I can't offer any helpful suggestions, but just to let you know I
>>> finally
>>> > > had
>>> > > to disable all my plugin WFS tests. I used to cope, by rerunning
>>> failed
>>> > > Travis runs, but by about three months ago, it seemed no longer
>>> usable -
>>> > > failure after failure.
>>> > >
>>> > > I was using a third-party WFS, and perhaps I could have got round
>>> this by
>>> > > adding a WFS provider to the test docker image, but in this plugin's
>>> case,
>>> > > I
>>> > > didn't think it worth the significant effort to do so. The WM(T)S
>>> tests
>>> > > also
>>> > > use third-party sources and seem stable, so perhaps this wasn't the
>>> > > underlying issue anyway.
>>> > >
>>> > > If an improvement or solution could be found, it would be great to
>>> > > reinstate
>>> > > these tests.
>>> > >
>>> > > Thanks
>>> > >
>>> > > Tom
>>> > >
>>> > >
>>> > >
>>> > > -
>>> > > Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
>>> > > --
>>> > > Sent from:
>>> > > http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
>>> > > ___
>>> > > QGIS-Developer mailing list
>>> > > QGIS-Developer@lists.osgeo.org
>>> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>
>>>
>>> --
>>> Spatialys - Geospatial professional services
>>> http://www.spatialys.com
>>>
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>



-- 
--
Carlo A. Bertelli
   Charta servizi e sistemi per il territorio e la storia ambientale srl
  Dipendenze del palazzo Doria,
  vc. alla Chiesa della Maddalena 9/2 16124  Genova (Italy)
  tel./fax +39(0)10 2475439  +39 0108566195  mobile:+39 393 1590711
   e-mail: berte...@chartasrl.eu  http://www.chartasrl.eu
--
___
QGIS-Developer mailing list

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Jeremy Palmer 
Can I also add that the refactoring that was funded in 2.16 added tests,
paging support, filter query builder and dynamic caching. There is now a
lot of complexity in the driver due to the complexity of WFS server
implementations and the standard, plus the multi-threading code. If a
refactor is proposed I would be against anything that doesn't deal current
use cases and edge cases which have already been implemented. Maybe some
analysis of the failed tests is the first place to start. WFS-T is a side
issue to that.

Cheers,
Jeremy

On Tue, Sep 4, 2018 at 7:51 AM Régis Haubourg 
wrote:

> Hi Even, thanks for pointing that, I missed that history.
> I'll ask the dev's for the detailed improvements planned, I dont' have any
> detail currently (sorry for that)
> Régis
>
> Le lun. 3 sept. 2018 à 21:29, Even Rouault  a
> écrit :
>
>> Régis,
>>
>> Not that I'm against improvements, all the contrary, but just wanted to
>> underline that the provider was seriously refactored already in 2.16.
>> Clearly
>> the lack of WFS-T support for 1.1 and 2.0 in the scope of those
>> enhancements
>> can be a source of confusion currently for users. What do you have in
>> mind as
>> refactoring exactly ?
>>
>> Even
>>
>> > Hi all,
>> > I very much think that the WFS client is an really bad state, and is not
>> > really reliable, especially in WFS-T context.
>> > The good news is that we just have been funded to refactor it !
>> > The work should start in september and land in 3.6. I will let our dev's
>> > come here with more technical details about the goals. I hope we will
>> also
>> > be able to take benefit of this to this the OGC compliancy of the client
>> > here.
>> > Best regards,
>> > Régis
>> >
>> > Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
>> >
>> > écrit :
>> > > I can't offer any helpful suggestions, but just to let you know I
>> finally
>> > > had
>> > > to disable all my plugin WFS tests. I used to cope, by rerunning
>> failed
>> > > Travis runs, but by about three months ago, it seemed no longer
>> usable -
>> > > failure after failure.
>> > >
>> > > I was using a third-party WFS, and perhaps I could have got round
>> this by
>> > > adding a WFS provider to the test docker image, but in this plugin's
>> case,
>> > > I
>> > > didn't think it worth the significant effort to do so. The WM(T)S
>> tests
>> > > also
>> > > use third-party sources and seem stable, so perhaps this wasn't the
>> > > underlying issue anyway.
>> > >
>> > > If an improvement or solution could be found, it would be great to
>> > > reinstate
>> > > these tests.
>> > >
>> > > Thanks
>> > >
>> > > Tom
>> > >
>> > >
>> > >
>> > > -
>> > > Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
>> > > --
>> > > Sent from:
>> > > http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
>> > > ___
>> > > QGIS-Developer mailing list
>> > > QGIS-Developer@lists.osgeo.org
>> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Régis Haubourg 
Hi Even, thanks for pointing that, I missed that history.
I'll ask the dev's for the detailed improvements planned, I dont' have any
detail currently (sorry for that)
Régis

Le lun. 3 sept. 2018 à 21:29, Even Rouault  a
écrit :

> Régis,
>
> Not that I'm against improvements, all the contrary, but just wanted to
> underline that the provider was seriously refactored already in 2.16.
> Clearly
> the lack of WFS-T support for 1.1 and 2.0 in the scope of those
> enhancements
> can be a source of confusion currently for users. What do you have in mind
> as
> refactoring exactly ?
>
> Even
>
> > Hi all,
> > I very much think that the WFS client is an really bad state, and is not
> > really reliable, especially in WFS-T context.
> > The good news is that we just have been funded to refactor it !
> > The work should start in september and land in 3.6. I will let our dev's
> > come here with more technical details about the goals. I hope we will
> also
> > be able to take benefit of this to this the OGC compliancy of the client
> > here.
> > Best regards,
> > Régis
> >
> > Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
> >
> > écrit :
> > > I can't offer any helpful suggestions, but just to let you know I
> finally
> > > had
> > > to disable all my plugin WFS tests. I used to cope, by rerunning failed
> > > Travis runs, but by about three months ago, it seemed no longer usable
> -
> > > failure after failure.
> > >
> > > I was using a third-party WFS, and perhaps I could have got round this
> by
> > > adding a WFS provider to the test docker image, but in this plugin's
> case,
> > > I
> > > didn't think it worth the significant effort to do so. The WM(T)S tests
> > > also
> > > use third-party sources and seem stable, so perhaps this wasn't the
> > > underlying issue anyway.
> > >
> > > If an improvement or solution could be found, it would be great to
> > > reinstate
> > > these tests.
> > >
> > > Thanks
> > >
> > > Tom
> > >
> > >
> > >
> > > -
> > > Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
> > > --
> > > Sent from:
> > > http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
> > > ___
> > > QGIS-Developer mailing list
> > > QGIS-Developer@lists.osgeo.org
> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Even Rouault 
Régis,

Not that I'm against improvements, all the contrary, but just wanted to 
underline that the provider was seriously refactored already in 2.16. Clearly 
the lack of WFS-T support for 1.1 and 2.0 in the scope of those enhancements 
can be a source of confusion currently for users. What do you have in mind as 
refactoring exactly ?

Even

> Hi all,
> I very much think that the WFS client is an really bad state, and is not
> really reliable, especially in WFS-T context.
> The good news is that we just have been funded to refactor it !
> The work should start in september and land in 3.6. I will let our dev's
> come here with more technical details about the goals. I hope we will also
> be able to take benefit of this to this the OGC compliancy of the client
> here.
> Best regards,
> Régis
> 
> Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
> 
> écrit :
> > I can't offer any helpful suggestions, but just to let you know I finally
> > had
> > to disable all my plugin WFS tests. I used to cope, by rerunning failed
> > Travis runs, but by about three months ago, it seemed no longer usable -
> > failure after failure.
> > 
> > I was using a third-party WFS, and perhaps I could have got round this by
> > adding a WFS provider to the test docker image, but in this plugin's case,
> > I
> > didn't think it worth the significant effort to do so. The WM(T)S tests
> > also
> > use third-party sources and seem stable, so perhaps this wasn't the
> > underlying issue anyway.
> > 
> > If an improvement or solution could be found, it would be great to
> > reinstate
> > these tests.
> > 
> > Thanks
> > 
> > Tom
> > 
> > 
> > 
> > -
> > Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
> > --
> > Sent from:
> > http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
> > ___
> > QGIS-Developer mailing list
> > QGIS-Developer@lists.osgeo.org
> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Régis Haubourg 
Hi all,
I very much think that the WFS client is an really bad state, and is not
really reliable, especially in WFS-T context.
The good news is that we just have been funded to refactor it !
The work should start in september and land in 3.6. I will let our dev's
come here with more technical details about the goals. I hope we will also
be able to take benefit of this to this the OGC compliancy of the client
here.
Best regards,
Régis

Le lun. 3 sept. 2018 à 11:36, Tom Chadwin  a
écrit :

> I can't offer any helpful suggestions, but just to let you know I finally
> had
> to disable all my plugin WFS tests. I used to cope, by rerunning failed
> Travis runs, but by about three months ago, it seemed no longer usable -
> failure after failure.
>
> I was using a third-party WFS, and perhaps I could have got round this by
> adding a WFS provider to the test docker image, but in this plugin's case,
> I
> didn't think it worth the significant effort to do so. The WM(T)S tests
> also
> use third-party sources and seem stable, so perhaps this wasn't the
> underlying issue anyway.
>
> If an improvement or solution could be found, it would be great to
> reinstate
> these tests.
>
> Thanks
>
> Tom
>
>
>
> -
> Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon
> --
> Sent from:
> http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-03 Thread Tom Chadwin 
I can't offer any helpful suggestions, but just to let you know I finally had
to disable all my plugin WFS tests. I used to cope, by rerunning failed
Travis runs, but by about three months ago, it seemed no longer usable -
failure after failure.

I was using a third-party WFS, and perhaps I could have got round this by
adding a WFS provider to the test docker image, but in this plugin's case, I
didn't think it worth the significant effort to do so. The WM(T)S tests also
use third-party sources and seem stable, so perhaps this wasn't the
underlying issue anyway.

If an improvement or solution could be found, it would be great to reinstate
these tests.

Thanks

Tom



-
Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon 
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-02 Thread Mathieu Pellerin 
Can these failures be reproduced locally if one repeatedly tests those
cases?

On Mon, Sep 3, 2018, 07:02 Nyall Dawson  wrote:

> On Sat, 1 Sep 2018 at 20:37, Richard Duivenvoorde 
> wrote:
>
> > And myself I also have the feeling that WFS is less stable
>
> This is my experience too, which is why I'm not going to 100%
> attribute these errors to Travis!
>
> Nyall
>
> > /usable then
> > in 2.18 (I release a list of national WFS/WMS/WCS's in one of my
> plugins).
> >
> > So I think it is not only CI that has problems, it looks like changes in
> > WFS made it more tricky to use?
> >
> > Regards,
> >
> > Richard Duivenvoorde
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-02 Thread Nyall Dawson 
On Sat, 1 Sep 2018 at 20:37, Richard Duivenvoorde  wrote:

> And myself I also have the feeling that WFS is less stable

This is my experience too, which is why I'm not going to 100%
attribute these errors to Travis!

Nyall

> /usable then
> in 2.18 (I release a list of national WFS/WMS/WCS's in one of my plugins).
>
> So I think it is not only CI that has problems, it looks like changes in
> WFS made it more tricky to use?
>
> Regards,
>
> Richard Duivenvoorde
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-01 Thread Richard Duivenvoorde 
On 09/01/2018 12:43 PM, Alessandro Pasotti wrote:

> Richard, please let's focus on Travis in this thread.
> 
> WFS UX issues on 3.x are a complete different topic.

Ok, sorry. What I just wanted to add: it looks like that 'default'-wfs
behaviour has changed, raising wfs problems .

I just wanted to raise awareness of this, thinking the CI failures could
have the same cause.

Regards,

Richard Duivenvoorde

___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-01 Thread Alessandro Pasotti 
On Sat, Sep 1, 2018 at 12:37 PM Richard Duivenvoorde 
wrote:

> On 09/01/2018 11:56 AM, Nyall Dawson wrote:
> > Hi devs,
> >
> > Just raising the question of what we should do about the constant WFS
> > test failures we get on Travis. I'd estimate 1 in 3 builds fails
> > because of WFS related tests hanging.
> >
> > I'm very reluctant to disable all these tests, because:
> >
> > 1. WFS is super important.
> > 2. I think there may be a real issue here - I've got at least one
> > customer who is having WFS issues with 3.2 and master. BUT: on the
> > other hand Travis has always been flaky with any test which uses
> > threads, regardless of which area of code it's from. So it could just
> > be Travis playing up again, in which case we'd need to disable these
> > tests like we do most of the other thread-related tests...
> >
> > The current situation is basically unworkable. So ideas?
>
> To add: QGIS recently has some issues and list messages concerning wfs
> too. I searched some:
>
> https://issues.qgis.org/issues/19702
>
> https://lists.osgeo.org/pipermail/qgis-user/2018-August/043237.html
>
> https://lists.osgeo.org/pipermail/qgis-user/2018-August/043241.html
>
> And myself I also have the feeling that WFS is less stable/usable then
> in 2.18 (I release a list of national WFS/WMS/WCS's in one of my plugins).
>
> So I think it is not only CI that has problems, it looks like changes in
> WFS made it more tricky to use?
>


Richard, please let's focus on Travis in this thread.

WFS UX issues on 3.x are a complete different topic.


-- 
Alessandro Pasotti
w3:   www.itopen.it
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-01 Thread Richard Duivenvoorde 
On 09/01/2018 11:56 AM, Nyall Dawson wrote:
> Hi devs,
> 
> Just raising the question of what we should do about the constant WFS
> test failures we get on Travis. I'd estimate 1 in 3 builds fails
> because of WFS related tests hanging.
> 
> I'm very reluctant to disable all these tests, because:
> 
> 1. WFS is super important.
> 2. I think there may be a real issue here - I've got at least one
> customer who is having WFS issues with 3.2 and master. BUT: on the
> other hand Travis has always been flaky with any test which uses
> threads, regardless of which area of code it's from. So it could just
> be Travis playing up again, in which case we'd need to disable these
> tests like we do most of the other thread-related tests...
> 
> The current situation is basically unworkable. So ideas?

To add: QGIS recently has some issues and list messages concerning wfs
too. I searched some:

https://issues.qgis.org/issues/19702

https://lists.osgeo.org/pipermail/qgis-user/2018-August/043237.html

https://lists.osgeo.org/pipermail/qgis-user/2018-August/043241.html

And myself I also have the feeling that WFS is less stable/usable then
in 2.18 (I release a list of national WFS/WMS/WCS's in one of my plugins).

So I think it is not only CI that has problems, it looks like changes in
WFS made it more tricky to use?

Regards,

Richard Duivenvoorde
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] What to do about WFS test failures?

2018-09-01 Thread Alessandro Pasotti 
On Sat, Sep 1, 2018 at 11:56 AM Nyall Dawson  wrote:

> Hi devs,
>
> Just raising the question of what we should do about the constant WFS
> test failures we get on Travis. I'd estimate 1 in 3 builds fails
> because of WFS related tests hanging.
>
> I'm very reluctant to disable all these tests, because:
>
> 1. WFS is super important.
> 2. I think there may be a real issue here - I've got at least one
> customer who is having WFS issues with 3.2 and master. BUT: on the
> other hand Travis has always been flaky with any test which uses
> threads, regardless of which area of code it's from. So it could just
> be Travis playing up again, in which case we'd need to disable these
> tests like we do most of the other thread-related tests...
>
> The current situation is basically unworkable. So ideas?
>
> Nyall
>


Sorry I do not have any solution,  the only consideration that comes up to
my mind is that if sum up all the time wasted by developers by
unrelated/random Travis failures we would probably be very badly surprised
(as I usually say Travis is basically a very slow binary entropy generator).

Talking about the tests, WFS tests, and (even for different reasons)
http-based integration tests (some auth tests, qgsfiledownloader, many
server OWS tests) are apparently more fragile than others, mainly because
they depend on (sometimes external) http services, but they have proven in
the past to be very effective in spotting out regressions on very important
feature likes WMS, WFS etc., some of them are also important because they
ensure that QGIS client can talk to its server component.

The very bad things about disabling tests are:

- their development costed a lot of time and efforts and disabling them is
not an incentive for the developers to write more tests [1]
- the tests are obviously important to prevent regressions
- before disabling a test we should make 100% sure that we are not
overlooking a real bug (or what are the tests written for in the first
place?)


So, my recommendation - not very useful in the short term I'm afraid -
would be to start exploring other more reliable options for our CI, even if
they are more expensive.

For the time being, as a temporary solution, there are no other options
than disabling though.

Maybe a good idea would be to allocate some money specifically for the
tests (a dedicated grant, a dedicated hackfest?) just an idea...

[1] of course sometimes it would just mean to write "better" and more
robust tests, but I suspect that this is not generally the case.

-- 
Alessandro Pasotti
w3:   www.itopen.it
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer