[QGIS-Developer] [SAGA] Raster values to points

[Niccolò Marchi]

Hi all,
updating from 2.18.11 to .13 the ‘Raster values to points’ disappeared to be 
substituted by  ‘Raster values to points (randomly)’. The same operation can be 
done with ‘v.what.rast.points’ but, even if well explained in some blogs, it 
turns to be pretty less user-friendly than the previous.
Is there a particular reason to this change?

Thanks in advance,

Niccolò
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] QGIS/OGR: FeatureIds reassigned on write to data provider?

[Sandro Mani]

On 20.09.2017 17:39, Even Rouault wrote:


On mercredi 20 septembre 2017 17:32:27 CEST Sandro Mani wrote:

> On 19.09.2017 19:15, Even Rouault wrote:

> > On mardi 19 septembre 2017 18:30:50 CEST Sandro Mani wrote:

> > > On 11.07.2017 14:31, Even Rouault wrote:

> > > > We could potentially imagine to defer the repacking at

> > > >

> > > > leaveUpdateMode() time, actually in the close() method which 
is called


> > > >

> > > > when the reference counter of the update mode drops to zero (*),

> > > >

> > > > instead of doing it at the end of each operation.

> > >

> > > To come back to this, I was looking at moving replack() to 
close(), but


> > >

> > > noticed that syncToDisc() has the logic "for shapefiles, remove 
spatial


> > >

> > > index files and create a new index" which calls close. And 
syncToDisk is


> > >

> > > still called after every change. So should syncToDisk be also 
moved to


> > >

> > > close(), or at least blocked if an updateMode is active?

> >

> > Yes that would probably make sense to block syncToDisk() while in

> > updateMode (so when mUpdateModeStackDepth > 0), but probably only if

> > enterUpdateMode() is called by QgsVectorLayer::startEditing(), and not

> > when called implicitly by doInitialActionsForEdition(), which might be

> > the case if someone directly works at the provider level

> >

> > bool QgsOgrProvider::doInitialActionsForEdition()

> >

> > {

> >

> > if ( !mValid )

> >

> > return false;

> >

> > if ( !mWriteAccess && mWriteAccessPossible && mDynamicWriteAccess )

> >

> > {

> >

> > QgsDebugMsg( "Enter update mode implictly" );

> >

> > if ( !enterUpdateMode() )

> >

> > return false;

> >

> > }

> >

> > return true;

> >

> > }

> >

> > So probably a mImplicitUpdateMode flag should be set above near the

> > QgsDebugMsg(), and the check to decide whether we can defer

> > syncToDisk() would be ( !mImplicitUpdateMode && 
mUpdateModeStackDepth > 0)


>

> Hm but will that flag never be cleared again once set? Since AFAICT a

> there is no leaveUpdateMode matching the enterUpdateMode called by

> doInitialActionsForEdition.

>

> As I understand, the approach to do changes with stable ids would be to

> first enterUpdateMode, then perform the changes, then leaveUpdateMode,

> at which point the changes are synced and the dataset repacked.

Yes

> With the

> mImplicitUpdateMode flag, if anyone worked directly at provider level

> before you come along, deferring syncToDisk would never happen since the

> mImplicitUpdateMode was set and remains set.

Yes my idea was to keep the current behaviour where repack() is done 
frequently, as you cannot know when the dataset should be in a 
consistant state.


But we could also decide to defer syncToDisk() at provider deletion 
(would make the code simpler)


I have no idea of the existing use cases behind and if that would 
break some.


Suppose something like [1] should both allow reliably deferring 
syncToDisk when an updateMode was explicitly entered while keeping the 
current behaviour of immediate syncToDisk for implicit update mode.


Sandro

[1] 
https://github.com/manisandro/QGIS/commit/13f451bdcbb653e9d69e55c79a5ba520d3f1712e
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] QGIS/OGR: FeatureIds reassigned on write to data provider?

[Even Rouault]

On mercredi 20 septembre 2017 17:32:27 CEST Sandro Mani wrote:
> On 19.09.2017 19:15, Even Rouault wrote:
> > On mardi 19 septembre 2017 18:30:50 CEST Sandro Mani wrote:
> > > On 11.07.2017 14:31, Even Rouault wrote:
> > > > We could potentially imagine to defer the repacking at
> > > > 
> > > > leaveUpdateMode() time, actually in the close() method which is called
> > > > 
> > > > when the reference counter of the update mode drops to zero (*),
> > > > 
> > > > instead of doing it at the end of each operation.
> > > 
> > > To come back to this, I was looking at moving replack() to close(), but
> > > 
> > > noticed that syncToDisc() has the logic "for shapefiles, remove spatial
> > > 
> > > index files and create a new index" which calls close. And syncToDisk is
> > > 
> > > still called after every change. So should syncToDisk be also moved to
> > > 
> > > close(), or at least blocked if an updateMode is active?
> > 
> > Yes that would probably make sense to block syncToDisk() while in
> > updateMode (so when mUpdateModeStackDepth > 0), but probably only if
> > enterUpdateMode() is called by QgsVectorLayer::startEditing(), and not
> > when called implicitly by doInitialActionsForEdition(), which might be
> > the case if someone directly works at the provider level
> > 
> > bool QgsOgrProvider::doInitialActionsForEdition()
> > 
> > {
> > 
> > if ( !mValid )
> > 
> > return false;
> > 
> > if ( !mWriteAccess && mWriteAccessPossible && mDynamicWriteAccess )
> > 
> > {
> > 
> > QgsDebugMsg( "Enter update mode implictly" );
> > 
> > if ( !enterUpdateMode() )
> > 
> > return false;
> > 
> > }
> > 
> > return true;
> > 
> > }
> > 
> > So probably a mImplicitUpdateMode flag should be set above near the
> > QgsDebugMsg(), and the check to decide whether we can defer
> > syncToDisk() would be ( !mImplicitUpdateMode && mUpdateModeStackDepth > 0)
> 
> Hm but will that flag never be cleared again once set? Since AFAICT a
> there is no leaveUpdateMode matching the enterUpdateMode called by
> doInitialActionsForEdition.
> 
> As I understand, the approach to do changes with stable ids would be to
> first enterUpdateMode, then perform the changes, then leaveUpdateMode,
> at which point the changes are synced and the dataset repacked.

Yes

> With the
> mImplicitUpdateMode flag, if anyone worked directly at provider level
> before you come along, deferring syncToDisk would never happen since the
> mImplicitUpdateMode was set and remains set.

Yes my idea was to keep the current behaviour where repack() is done 
frequently, as you 
cannot know when the dataset should be in a consistant state.
But we could also decide to defer syncToDisk() at provider deletion (would make 
the code 
simpler)
I have no idea of the existing use cases behind and if that would break some.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] QGIS/OGR: FeatureIds reassigned on write to data provider?

[Sandro Mani]

On 19.09.2017 19:15, Even Rouault wrote:


On mardi 19 septembre 2017 18:30:50 CEST Sandro Mani wrote:

> On 11.07.2017 14:31, Even Rouault wrote:

> > We could potentially imagine to defer the repacking at

> > leaveUpdateMode() time, actually in the close() method which is called

> > when the reference counter of the update mode drops to zero (*),

> > instead of doing it at the end of each operation.

>

> To come back to this, I was looking at moving replack() to close(), but

> noticed that syncToDisc() has the logic "for shapefiles, remove spatial

> index files and create a new index" which calls close. And syncToDisk is

> still called after every change. So should syncToDisk be also moved to

> close(), or at least blocked if an updateMode is active?

>

Yes that would probably make sense to block syncToDisk() while in 
updateMode (so when mUpdateModeStackDepth > 0), but probably only if 
enterUpdateMode() is called by QgsVectorLayer::startEditing(), and not 
when called implicitly by doInitialActionsForEdition(), which might be 
the case if someone directly works at the provider level


bool QgsOgrProvider::doInitialActionsForEdition()

{

if ( !mValid )

return false;

if ( !mWriteAccess && mWriteAccessPossible && mDynamicWriteAccess )

{

QgsDebugMsg( "Enter update mode implictly" );

if ( !enterUpdateMode() )

return false;

}

return true;

}

So probably a mImplicitUpdateMode flag should be set above near the 
QgsDebugMsg(), and the check to decide whether we can defer 
syncToDisk() would be ( !mImplicitUpdateMode && mUpdateModeStackDepth > 0)


Hm but will that flag never be cleared again once set? Since AFAICT a 
there is no leaveUpdateMode matching the enterUpdateMode called by 
doInitialActionsForEdition.


As I understand, the approach to do changes with stable ids would be to 
first enterUpdateMode, then perform the changes, then leaveUpdateMode, 
at which point the changes are synced and the dataset repacked. With the 
mImplicitUpdateMode flag, if anyone worked directly at provider level 
before you come along, deferring syncToDisk would never happen since the 
mImplicitUpdateMode was set and remains set.


Sandro
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Might snap change a 2D layer to 3D? discussion how to fix https://issues.qgis.org/issues/16927

[Luigi Pirelli]

The following PR should approach the above described issues:

https://github.com/qgis/QGIS/pull/5223

please give a try.

respect the request to add a config option to force or not conversion
I implemented the following rule:
1) as proposed by mathias... if there is only a conversion, do it!
2) if not conversion is possibile => raise error as soon as possibile
during editing => it's up to the use to adapt the receiving layer to
be compatible.

regards

Luigi Pirelli

**
* Boundless QGIS Support/Development: lpirelli AT boundlessgeo DOT com
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
* 
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
**


On 8 September 2017 at 17:05, Luigi Pirelli  wrote:
> On 8 September 2017 at 16:51, Matthias Kuhn  wrote:
>> I would try to avoid introducing the same logic in all tools like
>> snapping, pasting, GPS digitizing, processing, ...
>
> I strongly agree
>
> Luigi Pirelli
>
> **
> * Boundless QGIS Support/Development: lpirelli AT boundlessgeo DOT com
> * LinkedIn: https://www.linkedin.com/in/luigipirelli
> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
> * GitHub: https://github.com/luipir
> * Mastering QGIS 2nd Edition:
> * 
> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
> **
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Crash reading PostGIS raster

[pcav]

This is the ticket:
https://issues.qgis.org/issues/8073



--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] removing icons

[João Gaspar]

Hi Donn,
supporting the answer of Nathan of this subject you can see in
documentation:
http://docs.qgis.org/2.18/en/docs/user_manual/introduction/qgis_configuration.html#customization

Regards,
João


2017-09-20 14:41 GMT+01:00 Nathan Woodrow :

> Hey Donn,
>
> FIrst, your post is in the right place.  Maybe qgis-user is a better place
> but doesn't really matter for this.
>
> Second, you sure can do what you need.  The first thing you need is
> Settings -> Customization. Tick enable and you can disable and enable
> anything you need.  You can also Save To File to get an ini file which you
> can load QGIS with the following argument to start the UI with that
> interface: qgis -z yourfile.ini.  Pretty handy if you need different UIs
> for different use cases.
>
> Next thing to do after that is look for the "Customize ToolBars" plugin.
> That will let you make custom toolbars with what ever actions you need for
> your users.
>
> Regards,
> Nathan
>
>
> On Wed, Sep 20, 2017 at 11:24 PM, Donn Downey 
> wrote:
>
>> Hello, I develop database solutions for a niche industry that also has
>> significant GIS needs. I personally use QGIS and help others get going, but
>> even QGIS can be tough to teach to non-savvy computer users. As such, I
>> dream of a version of QGIS where we basically strip out 90% of the icons
>> from the half dozen relevant toolbars, in order to further simplify the
>> interface and make the necessary tasks more accessible.  I have no
>> programming experience- we would hire someone to do this- I’m just trying
>> to figure out options. Ultimately, we’d like to (freely) share a special
>> stripped down version of QGIS.
>>
>> First, is it technically possible to hide or remove icons from toolbars,
>> or develop a toolbar that is specific to our industry-specific purpose?
>> Along these lines, are there developers on this list that contract to do
>> work like this?
>>
>> I apologize if this isn’t an appropriate forum to post this topic- I’m
>> new to this. Please let me know. Thanks!
>>
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] removing icons

[Nathan Woodrow]

Hey Donn,

FIrst, your post is in the right place.  Maybe qgis-user is a better place
but doesn't really matter for this.

Second, you sure can do what you need.  The first thing you need is
Settings -> Customization. Tick enable and you can disable and enable
anything you need.  You can also Save To File to get an ini file which you
can load QGIS with the following argument to start the UI with that
interface: qgis -z yourfile.ini.  Pretty handy if you need different UIs
for different use cases.

Next thing to do after that is look for the "Customize ToolBars" plugin.
That will let you make custom toolbars with what ever actions you need for
your users.

Regards,
Nathan


On Wed, Sep 20, 2017 at 11:24 PM, Donn Downey  wrote:

> Hello, I develop database solutions for a niche industry that also has
> significant GIS needs. I personally use QGIS and help others get going, but
> even QGIS can be tough to teach to non-savvy computer users. As such, I
> dream of a version of QGIS where we basically strip out 90% of the icons
> from the half dozen relevant toolbars, in order to further simplify the
> interface and make the necessary tasks more accessible.  I have no
> programming experience- we would hire someone to do this- I’m just trying
> to figure out options. Ultimately, we’d like to (freely) share a special
> stripped down version of QGIS.
>
> First, is it technically possible to hide or remove icons from toolbars,
> or develop a toolbar that is specific to our industry-specific purpose?
> Along these lines, are there developers on this list that contract to do
> work like this?
>
> I apologize if this isn’t an appropriate forum to post this topic- I’m new
> to this. Please let me know. Thanks!
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

[QGIS-Developer] removing icons

[Donn Downey]

Hello, I develop database solutions for a niche industry that also has 
significant GIS needs. I personally use QGIS and help others get going, but 
even QGIS can be tough to teach to non-savvy computer users. As such, I dream 
of a version of QGIS where we basically strip out 90% of the icons from the 
half dozen relevant toolbars, in order to further simplify the interface and 
make the necessary tasks more accessible.  I have no programming experience- we 
would hire someone to do this- I’m just trying to figure out options. 
Ultimately, we’d like to (freely) share a special stripped down version of QGIS.

First, is it technically possible to hide or remove icons from toolbars, or 
develop a toolbar that is specific to our industry-specific purpose? Along 
these lines, are there developers on this list that contract to do work like 
this?

I apologize if this isn’t an appropriate forum to post this topic- I’m new to 
this. Please let me know. Thanks!
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Crash reading PostGIS raster

[pcav]

This is the stcktrace:

Thread 12 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff42e7d700 (LWP 3910)]
0x755da6b7 in QgsColorRampShader::shade(double, int*, int*, int*,
int*)
()
   from
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
(gdb) bt
#0  0x755da6b7 in QgsColorRampShader::shade(double, int*, int*,
int*, int*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#1  0x756015ed in QgsRasterShader::shade(double, int*, int*, int*,
int*) () at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#2  0x7561e543 in QgsSingleBandPseudoColorRenderer::block2(int,
QgsRectangle const&, int, int, QgsRasterBlockFeedback*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#3  0x75605fa7 in QgsBrightnessContrastFilter::block2(int,
QgsRectangle const&, int, int, QgsRasterBlockFeedback*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#4  0x7560a5b9 in QgsHueSaturationFilter::block2(int, QgsRectangle
const&, int, int, QgsRasterBlockFeedback*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#5  0x7561b777 in QgsRasterResampleFilter::block2(int, QgsRectangle
const&, int, int, QgsRasterBlockFeedback*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#6  0x755fee4a in QgsRasterProjector::block2(int, QgsRectangle
const&, int, int, QgsRasterBlockFeedback*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#7  0x755ec4cf in QgsRasterIterator::readNextRasterPart(int, int&,
int&, QgsRasterBlock**, int&, int&) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#8  0x7560fd7e in QgsRasterDrawer::draw(QPainter*,
QgsRasterViewPort*, QgsMapToPixel const*, QgsRenderContext const*,
QgsRasterBlockFeedback*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#9  0x755f7d69 in QgsRasterLayerRenderer::render() ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#10 0x7538371e in
QgsMapRendererParallelJob::renderLayerStatic(LayerRenderJob&) () at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#11 0x753844ce in QtConcurrent::FunctionWrapper1::operator()(LayerRenderJob&) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#12 0x7538422b in
QtConcurrent::MapKernel::iterator,
QtConcurrent::FunctionWrapper1
>::runIteration(QList::iterator, int, void*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#13 0x753842bb in
QtConcurrent::MapKernel::iterator,
QtConcurrent::FunctionWrapper1
>::runIterations(QList::iterator, int, int, void*) ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#14 0x7538470f in
QtConcurrent::IterateKernel::iterator,
void>::forThreadFunction() ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#15 0x75384438 in
QtConcurrent::IterateKernel::iterator,
void>::threadFunction() ()
at
/usr/local/src/qgis/QGIS/build_qgis2/output/lib/libqgis_core.so.2.18.13
#16 0x74818d5d in QtConcurrent::ThreadEngineBase::run() ()
at /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#17 0x7481adba in  () at /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#18 0x74827daa in  () at /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#19 0x7fffecafd494 in start_thread (arg=0x7fff42e7d700) at
pthread_create.c:333
#20 0x7fffed993abf in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:97



--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

[QGIS-Developer] Crash reading PostGIS raster

[Paolo Cavallini]

Hi all,
a freshly compiled QGIS 2.18 loads pg rasters correctly, but when
zooming segfaults miserably with:
Warning: RasterIO error: IReadBlock failed at X offset 0, Y offset 0
This is not new, but makes it pretty unusable.
Available for testing.
All the best.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

[QGIS-Developer] Refactor fields processing algorithm and expressions

[Andreas Neumann]

Hi,

I am running into troubles using expressions in the "Refactor fields" 
processing provider.


E.g. I have a date field and would like to reformat it in german format:

format_date("erfassungsdatum",'dd.MM.')

Although I am 100% certain that the above expression is syntactically 
correct, the expression builder insists that the expression is invalid 
and I can therefore not apply and save this expression.


Is this a known limitation because the model does not know which layer 
it is tied to?


Are there any workarounds?

Thanks,

Andreas

___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] OGR/GBD: Unrecognized field name OBJECTID

[Even Rouault]

> 
> Thanks for the detailed explanation.
> 
> Does it actually hurt to try with FID regardless of the contents of
> CPLGetLastErrorMsg if the previous attempt failed, instead of relying on
> the contents of an error message?

No, that shouldn't hurt.

> 
> PR is here:
> https://github.com/qgis/QGIS/pull/5224

Looks good to me

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] OGR/GBD: Unrecognized field name OBJECTID

[Sandro Mani]

On 20.09.2017 12:46, Even Rouault wrote:


On mercredi 20 septembre 2017 11:34:50 CEST Sandro Mani wrote:

> On 20.09.2017 08:45, Nyall Dawson wrote:

> > On 20 September 2017 at 10:18, Nyall Dawson 
 wrote:


> >> Hi all,

> >>

> >> I've just been trying to diagnose an issue with loading an existing

> >> 2.18 project in the new 2.18.13 release.

> >>

> >> Seems like something may be broken here with regards to loading OGR

> >> based layers which have a subset filter set.

> >>

> >> These layers trigger the "missing layers" dialog when loading the

> >> project and the layer link cannot be repaired without removing the

> >> filter clause.

> >>

> >> (specifically this is with .gdb layers on Windows - not sure yet if

> >> it's restricted to this combination).

> >>

> >> So just a heads up to test your existing projects before rolling out

> >> the 2.18.13 update.

> >

> > Failing unit test is here: https://github.com/qgis/QGIS/pull/5222,

> > regression bisected to https://github.com/qgis/QGIS/commit/8eaeaaa

> >

> > Sandro can you take a look at this?

>

> Argh.

>

> So the result of the subset query

>

> SELECT OBJECTID as orig_ogc_fid, * FROM "lines" WHERE "text" = 'shape 2'

>

> now is

>

> 305: OGR(1): OGR[3] error 1: Unrecognized field name OBJECTID.

> 305: OGR(1): Data source is invalid (Unrecognized field name OBJECTID.)

>

> but OBJECTID is what was returned by

>

> OGR_L_GetFIDColumn( layer )



Ah, that one is tricky. Let me try to explain the situation

Normally when you do ExecuteSQL(sql_statement, NULL, dialect) with 
dialect==NULL, OGR will forward sql_statement to the underlying SQL 
engine for a SQL compatible DBMS (Postgres, MySQL, SQLite, OCI, 
etc), and if there's no such engine, evaluate it with its own 
OGRSQL builtin engine.


For the FileGDB driver, the ESRI SDK has some SQL evaluation 
capability, but my testing showed that it was too limited to be 
reliable, so by default I fallback to OGRSQL.


You can force the use of the ESRI SQL engine by specifying a dialect 
== "FileGDB". If you do that with the above statement, you'll get


$ ogrinfo sampleFGDB_1.gdb -sql "select OBJECTID as foo, * FROM 
polygonArcsZM1" -dialect FileGDB -q


ERROR 1: Failed at executing 'select OBJECTID as foo, * FROM 
polygonArcsZM1' (An invalid SQL statement was used.)


So no better. Apparently, the ESRI SDK doesn't like selecting a column 
and "*".


So to come back to the bug, the SQL statement is evaluated with the 
OGRSQL engine. So this is a bug in it in the situation where the FID 
column name is an actual non-empty string. OGRSQL only understands 
"FID" as the shortcut for whatever the FID column name, and doesn't 
try to look if a column name matches OGR_L_GetFIDColumn() (when it was 
developped, all drivers that defaulted to OGRSQL had an implementation 
of OGR_L_GetFIDColumn() that returned a "" string). So this is a 
current shortcoming of the OGRSQL engine, that should probably be fixed


In the meantime you have to workaround it on the QGIS side, both for 
the "FileGDB" and "OpenFileGDB" drivers (. In that situation instead 
of using the result of OGR_L_GetFIDColumn() to build your SQL 
statement, hardcode FID.


So your statement will become:

SELECT FID as orig_ogc_fid, * FROM "lines" WHERE "text" = 'shape 2'

Actually when digging a bit more this could affect also the GeoJSON 
driver, but not in all situations. Only if you have a "id" member at 
the same level of "type": "Feature"


So something like

{ "type": "Feature", "id": 168, "properties": {...}, "geometry": {...} "

then id will be used as the FID and returned by OGR_L_GetFIDColumn()

And you'll hit the FileGDB bug

If you have something like

{ "type": "Feature", "properties": { "id": 168,...}, "geometry": {...} "

Then OGR_L_GetFIDColumn() will return "id" but you'll have also a 
regular id field, so "SELECT id as ogrig_ogc_fid, * FROM ..." will 
work then.






My (final!) suggestion is thus to first try:

SELECT {OGR_L_GetFIDColumn() or FID if it is empty} as orig_ogc_fid, * 
FROM


If that returns an error look for CPLGetLastErrorMsg() and if it 
contains OGR_L_GetFIDColumn() in it, retry with


SELECT FID as orig_ogc_fid, * FROM

Not pretty, but should be relatively robust




Thanks for the detailed explanation.

Does it actually hurt to try with FID regardless of the contents of 
CPLGetLastErrorMsg if the previous attempt failed, instead of relying on 
the contents of an error message?


PR is here:
https://github.com/qgis/QGIS/pull/5224


Sandro
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] OGR/GBD: Unrecognized field name OBJECTID (was: Re: Warning - possible project corruption in 2.18.13)

[Even Rouault]

On mercredi 20 septembre 2017 11:34:50 CEST Sandro Mani wrote:
> On 20.09.2017 08:45, Nyall Dawson wrote:
> > On 20 September 2017 at 10:18, Nyall Dawson  wrote:
> >> Hi all,
> >> 
> >> I've just been trying to diagnose an issue with loading an existing
> >> 2.18 project in the new 2.18.13 release.
> >> 
> >> Seems like something may be broken here with regards to loading OGR
> >> based layers which have a subset filter set.
> >> 
> >> These layers trigger the "missing layers" dialog when loading the
> >> project and the layer link cannot be repaired without removing the
> >> filter clause.
> >> 
> >> (specifically this is with .gdb layers on Windows - not sure yet if
> >> it's restricted to this combination).
> >> 
> >> So just a heads up to test your existing projects before rolling out
> >> the 2.18.13 update.
> > 
> > Failing unit test is here: https://github.com/qgis/QGIS/pull/5222,
> > regression bisected to https://github.com/qgis/QGIS/commit/8eaeaaa
> > 
> > Sandro can you take a look at this?
> 
> Argh.
> 
> So the result of the subset query
> 
> SELECT OBJECTID as orig_ogc_fid, * FROM "lines" WHERE "text" = 'shape 2'
> 
> now is
> 
> 305: OGR(1): OGR[3] error 1: Unrecognized field name OBJECTID.
> 305: OGR(1): Data source is invalid (Unrecognized field name OBJECTID.)
> 
> but OBJECTID is what was returned by
> 
> OGR_L_GetFIDColumn( layer )



Ah, that one is tricky. Let me try to explain the situation

Normally when you do ExecuteSQL(sql_statement, NULL, dialect) with 
dialect==NULL, OGR 
will forward sql_statement to the underlying SQL engine for a SQL compatible 
DBMS 
(Postgres, MySQL, SQLite, OCI, etc), and if there's no such engine, 
evaluate it with its own 
OGRSQL builtin engine.

For the FileGDB driver, the ESRI SDK has some SQL evaluation capability, but my 
testing 
showed that it was too limited to be reliable, so by default I fallback to 
OGRSQL.
You can force the use of the ESRI SQL engine by specifying a dialect == 
"FileGDB". If you do 
that with the above statement, you'll get

$ ogrinfo sampleFGDB_1.gdb -sql "select OBJECTID as foo, * FROM polygonArcsZM1" 
-
dialect FileGDB -q
ERROR 1: Failed at executing 'select OBJECTID as foo, * FROM polygonArcsZM1' 
(An invalid 
SQL statement was used.)

So no better. Apparently, the ESRI SDK doesn't like selecting a column and "*".

So to come back to the bug, the SQL statement is evaluated with the OGRSQL 
engine. So this 
is a bug in it in the situation where the FID column name is an actual 
non-empty string. 
OGRSQL only understands "FID" as the shortcut for whatever the FID column name, 
and 
doesn't try to look if a column name matches OGR_L_GetFIDColumn() (when it was 
developped, all drivers that defaulted to OGRSQL had an implementation of 
OGR_L_GetFIDColumn() that returned a "" string). So this is a current 
shortcoming of the 
OGRSQL engine, that should probably be fixed

In the meantime you have to workaround it on the QGIS side, both for the 
"FileGDB" and 
"OpenFileGDB" drivers (. In that situation instead of using the result of 
OGR_L_GetFIDColumn() to build your SQL statement, hardcode FID.

So your statement will become:

SELECT FID as orig_ogc_fid, * FROM "lines" WHERE "text" = 'shape 2'


Actually when digging a bit more this could affect also the GeoJSON driver, but 
not in all 
situations. Only if you have a "id" member at the same level of "type": 
"Feature"
So something like
{ "type": "Feature", "id": 168, "properties": {...}, "geometry": {...} "
then id will be used as the FID and returned by OGR_L_GetFIDColumn()

And you'll hit the FileGDB bug

If you have something like
{ "type": "Feature",  "properties": { "id": 168,...}, "geometry": {...} "

Then OGR_L_GetFIDColumn() will return "id" but you'll have also a regular id 
field, so 
"SELECT id as ogrig_ogc_fid, * FROM ..." will work then.






My (final!) suggestion is thus to first try:

SELECT {OGR_L_GetFIDColumn() or FID if it is empty} as orig_ogc_fid, * FROM 

If that returns an error look for CPLGetLastErrorMsg() and if it contains 
OGR_L_GetFIDColumn() in it, retry with

SELECT FID as orig_ogc_fid, * FROM 

Not pretty, but should be relatively robust




Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Refactor fields processing algorithm

[Andreas Neumann]

Hi Nyall,

Yes - changing to "feature source" brings back the "selected only" 
option for running the whole model.


Thanks!

Andreas


On 20.09.2017 12:08, Nyall Dawson wrote:

On 20 September 2017 at 19:57, Andreas Neumann  wrote:

Hi,

This is about QGIS master.

The refactor fields processing algorithm, when opened manually in the
toolbox, has an option to run the algorithm only on selected features.

However, when the same algorithm is included in a model, this option "only
selected features" is not available anymore.

Is there a workaround? How would I run this algorithm in the model only with
the selected features? In QGIS 2x there was a processing option to always
run processing with selected features only - but this option is now gone in
QGIS master.

Make sure your model uses "feature source" input, not a "vector layer"
input. "Vector layer" inputs should only be used in limited
circumstances now (specifically - when the model uses algorithms which
require a layer, such as those which modify a layer in place or those
which make a selection on a layer).

Nyall


___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Refactor fields processing algorithm

[Nyall Dawson]

On 20 September 2017 at 19:57, Andreas Neumann  wrote:
> Hi,
>
> This is about QGIS master.
>
> The refactor fields processing algorithm, when opened manually in the
> toolbox, has an option to run the algorithm only on selected features.
>
> However, when the same algorithm is included in a model, this option "only
> selected features" is not available anymore.
>
> Is there a workaround? How would I run this algorithm in the model only with
> the selected features? In QGIS 2x there was a processing option to always
> run processing with selected features only - but this option is now gone in
> QGIS master.

Make sure your model uses "feature source" input, not a "vector layer"
input. "Vector layer" inputs should only be used in limited
circumstances now (specifically - when the model uses algorithms which
require a layer, such as those which modify a layer in place or those
which make a selection on a layer).

Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

[QGIS-Developer] Refactor fields processing algorithm

[Andreas Neumann]

Hi,

This is about QGIS master.

The refactor fields processing algorithm, when opened manually in the 
toolbox, has an option to run the algorithm only on selected features.


However, when the same algorithm is included in a model, this option 
"only selected features" is not available anymore.


Is there a workaround? How would I run this algorithm in the model only 
with the selected features? In QGIS 2x there was a processing option to 
always run processing with selected features only - but this option is 
now gone in QGIS master.


Thanks for your help,

Andreas

___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

[QGIS-Developer] OGR/GBD: Unrecognized field name OBJECTID (was: Re: Warning - possible project corruption in 2.18.13)

[Sandro Mani]

On 20.09.2017 08:45, Nyall Dawson wrote:


On 20 September 2017 at 10:18, Nyall Dawson  wrote:

Hi all,

I've just been trying to diagnose an issue with loading an existing
2.18 project in the new 2.18.13 release.

Seems like something may be broken here with regards to loading OGR
based layers which have a subset filter set.

These layers trigger the "missing layers" dialog when loading the
project and the layer link cannot be repaired without removing the
filter clause.

(specifically this is with .gdb layers on Windows - not sure yet if
it's restricted to this combination).

So just a heads up to test your existing projects before rolling out
the 2.18.13 update.

Failing unit test is here: https://github.com/qgis/QGIS/pull/5222,
regression bisected to https://github.com/qgis/QGIS/commit/8eaeaaa

Sandro can you take a look at this?

Argh.

So the result of the subset query

SELECT OBJECTID as orig_ogc_fid, * FROM "lines" WHERE "text" = 'shape 2'

now is

305: OGR(1): OGR[3] error 1: Unrecognized field name OBJECTID.
305: OGR(1): Data source is invalid (Unrecognized field name OBJECTID.)

but OBJECTID is what was returned by

OGR_L_GetFIDColumn( layer )

Even, any ideas?


Sandro
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Warning - possible project corruption in 2.18.13

[Nyall Dawson]

On 20 September 2017 at 10:18, Nyall Dawson  wrote:
> Hi all,
>
> I've just been trying to diagnose an issue with loading an existing
> 2.18 project in the new 2.18.13 release.
>
> Seems like something may be broken here with regards to loading OGR
> based layers which have a subset filter set.
>
> These layers trigger the "missing layers" dialog when loading the
> project and the layer link cannot be repaired without removing the
> filter clause.
>
> (specifically this is with .gdb layers on Windows - not sure yet if
> it's restricted to this combination).
>
> So just a heads up to test your existing projects before rolling out
> the 2.18.13 update.

Failing unit test is here: https://github.com/qgis/QGIS/pull/5222,
regression bisected to https://github.com/qgis/QGIS/commit/8eaeaaa

Sandro can you take a look at this?

Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer