Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Antongeo76]

I've just tested in Master 9cae77f262,
It seems to works fine

many thanks for the explanation

Regards
Alessandro Pasotti-2 wrote
> On Tue, Oct 16, 2018 at 1:06 PM Antongeo76 

> antoviscomi@

>  wrote:
> 
>> Alessandro Pasotti-2 wrote
>> > May be related to https://issues.qgis.org/issues/20094
>> >
>> > See also: https://github.com/qgis/QGIS/pull/8200
>> >
>> > On Tue, Oct 16, 2018 at 12:50 PM Antongeo76 
>>
>> > antoviscomi@
>>
>> >  wrote:
>> >
>> >> Hi,
>> >>
>> >> This slowness is also present in QGIS 2.18 as well as in QGIS 3.2.3
>> >> and also occurs with fewer features in shapefiles (15000-16000 are
>> >> enough),
>> >> slowness increases as the number of unsaved editing operations
>> increases.
>> >>
>> >> With the "attribute table" opened, even "merge elements" or "split
>> >> feature"
>> >> operations,
>> >> cause some problems (i.e. the counting of the records and the
>> selection
>> >> of
>> >> the same does not seem synchronized with these editing operations)
>> >> and sometimes this cause freeze of QGIS
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> -
>> >> Antonio
>> >> --
>> >> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
>> >> ___
>> >> Qgis-user mailing list
>> >>
>>
>> > Qgis-user@.osgeo
>>
>> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> >
>> >
>> >
>> > --
>> > Alessandro Pasotti
>> > w3:   www.itopen.it
>> >
>> > ___
>> > Qgis-user mailing list
>>
>> > Qgis-user@.osgeo
>>
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
>> Yes, it can be related but it does not happen only when the "field
>> calculator" is used
>> it happens even if you perform several 'manual' editing operations and
>> save
>> them after about an hour.
>>
>>
> The bug I pointed to is not related to field calculator (the title is
> misleading): field calculator just triggers it because it does a lot of
> changes.
> 
> 
> 
>> It gives me the feeling that there is a buffer overflow
>>
> 
> Buffer overflow would most probably lead to a crash, and this is not the
> case.
> 
> 
> -- 
> Alessandro Pasotti
> w3:   www.itopen.it
> 
> ___
> Qgis-user mailing list

> Qgis-user@.osgeo

> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user





-
Antonio
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Alessandro Pasotti]

On Tue, Oct 16, 2018 at 1:06 PM Antongeo76  wrote:

> Alessandro Pasotti-2 wrote
> > May be related to https://issues.qgis.org/issues/20094
> >
> > See also: https://github.com/qgis/QGIS/pull/8200
> >
> > On Tue, Oct 16, 2018 at 12:50 PM Antongeo76 
>
> > antoviscomi@
>
> >  wrote:
> >
> >> Hi,
> >>
> >> This slowness is also present in QGIS 2.18 as well as in QGIS 3.2.3
> >> and also occurs with fewer features in shapefiles (15000-16000 are
> >> enough),
> >> slowness increases as the number of unsaved editing operations
> increases.
> >>
> >> With the "attribute table" opened, even "merge elements" or "split
> >> feature"
> >> operations,
> >> cause some problems (i.e. the counting of the records and the selection
> >> of
> >> the same does not seem synchronized with these editing operations)
> >> and sometimes this cause freeze of QGIS
> >>
> >>
> >>
> >>
> >>
> >> -
> >> Antonio
> >> --
> >> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
> >> ___
> >> Qgis-user mailing list
> >>
>
> > Qgis-user@.osgeo
>
> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >
> >
> >
> > --
> > Alessandro Pasotti
> > w3:   www.itopen.it
> >
> > ___
> > Qgis-user mailing list
>
> > Qgis-user@.osgeo
>
> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
> Yes, it can be related but it does not happen only when the "field
> calculator" is used
> it happens even if you perform several 'manual' editing operations and save
> them after about an hour.
>
>
The bug I pointed to is not related to field calculator (the title is
misleading): field calculator just triggers it because it does a lot of
changes.



> It gives me the feeling that there is a buffer overflow
>

Buffer overflow would most probably lead to a crash, and this is not the
case.


-- 
Alessandro Pasotti
w3:   www.itopen.it
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Luigi Pirelli]

nothing related with overflows as described in shown tikets... btw, please
give a check with master (not 2.18) nightly builds e.g. qgis-dev. Fix has
been merged and you can check if it works on on dev version not in 3.2.x.
Luigi Pirelli

**
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
*
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg
**


On Tue, 16 Oct 2018 at 13:06, Antongeo76  wrote:

> Alessandro Pasotti-2 wrote
> > May be related to https://issues.qgis.org/issues/20094
> >
> > See also: https://github.com/qgis/QGIS/pull/8200
> >
> > On Tue, Oct 16, 2018 at 12:50 PM Antongeo76 
>
> > antoviscomi@
>
> >  wrote:
> >
> >> Hi,
> >>
> >> This slowness is also present in QGIS 2.18 as well as in QGIS 3.2.3
> >> and also occurs with fewer features in shapefiles (15000-16000 are
> >> enough),
> >> slowness increases as the number of unsaved editing operations
> increases.
> >>
> >> With the "attribute table" opened, even "merge elements" or "split
> >> feature"
> >> operations,
> >> cause some problems (i.e. the counting of the records and the selection
> >> of
> >> the same does not seem synchronized with these editing operations)
> >> and sometimes this cause freeze of QGIS
> >>
> >>
> >>
> >>
> >>
> >> -
> >> Antonio
> >> --
> >> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
> >> ___
> >> Qgis-user mailing list
> >>
>
> > Qgis-user@.osgeo
>
> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >
> >
> >
> > --
> > Alessandro Pasotti
> > w3:   www.itopen.it
> >
> > ___
> > Qgis-user mailing list
>
> > Qgis-user@.osgeo
>
> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
> Yes, it can be related but it does not happen only when the "field
> calculator" is used
> it happens even if you perform several 'manual' editing operations and save
> them after about an hour.
>
> It gives me the feeling that there is a buffer overflow
>
>
>
> -
> Antonio
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[DelazJ]

Hi,

Any third party plugin enabled? There are some plugins that are/were
known/suspected to generate this overflow. You'll find related reports in
the bug tracker.
In 3.x, try to run on a clean user profile with no additional plugin.

DelazJ

Le mar. 16 oct. 2018 à 13:06, Antongeo76  a écrit :

> Alessandro Pasotti-2 wrote
> > May be related to https://issues.qgis.org/issues/20094
> >
> > See also: https://github.com/qgis/QGIS/pull/8200
> >
> > On Tue, Oct 16, 2018 at 12:50 PM Antongeo76 
>
> > antoviscomi@
>
> >  wrote:
> >
> >> Hi,
> >>
> >> This slowness is also present in QGIS 2.18 as well as in QGIS 3.2.3
> >> and also occurs with fewer features in shapefiles (15000-16000 are
> >> enough),
> >> slowness increases as the number of unsaved editing operations
> increases.
> >>
> >> With the "attribute table" opened, even "merge elements" or "split
> >> feature"
> >> operations,
> >> cause some problems (i.e. the counting of the records and the selection
> >> of
> >> the same does not seem synchronized with these editing operations)
> >> and sometimes this cause freeze of QGIS
> >>
> >>
> >>
> >>
> >>
> >> -
> >> Antonio
> >> --
> >> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
> >> ___
> >> Qgis-user mailing list
> >>
>
> > Qgis-user@.osgeo
>
> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >
> >
> >
> > --
> > Alessandro Pasotti
> > w3:   www.itopen.it
> >
> > ___
> > Qgis-user mailing list
>
> > Qgis-user@.osgeo
>
> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
> Yes, it can be related but it does not happen only when the "field
> calculator" is used
> it happens even if you perform several 'manual' editing operations and save
> them after about an hour.
>
> It gives me the feeling that there is a buffer overflow
>
>
>
> -
> Antonio
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Antongeo76]

Alessandro Pasotti-2 wrote
> May be related to https://issues.qgis.org/issues/20094
> 
> See also: https://github.com/qgis/QGIS/pull/8200
> 
> On Tue, Oct 16, 2018 at 12:50 PM Antongeo76 

> antoviscomi@

>  wrote:
> 
>> Hi,
>>
>> This slowness is also present in QGIS 2.18 as well as in QGIS 3.2.3
>> and also occurs with fewer features in shapefiles (15000-16000 are
>> enough),
>> slowness increases as the number of unsaved editing operations increases.
>>
>> With the "attribute table" opened, even "merge elements" or "split
>> feature"
>> operations,
>> cause some problems (i.e. the counting of the records and the selection
>> of
>> the same does not seem synchronized with these editing operations)
>> and sometimes this cause freeze of QGIS
>>
>>
>>
>>
>>
>> -
>> Antonio
>> --
>> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
>> ___
>> Qgis-user mailing list
>> 

> Qgis-user@.osgeo

>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> 
> 
> 
> -- 
> Alessandro Pasotti
> w3:   www.itopen.it
> 
> ___
> Qgis-user mailing list

> Qgis-user@.osgeo

> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Yes, it can be related but it does not happen only when the "field
calculator" is used
it happens even if you perform several 'manual' editing operations and save
them after about an hour.

It gives me the feeling that there is a buffer overflow



-
Antonio
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Alessandro Pasotti]

May be related to https://issues.qgis.org/issues/20094

See also: https://github.com/qgis/QGIS/pull/8200

On Tue, Oct 16, 2018 at 12:50 PM Antongeo76  wrote:

> Hi,
>
> This slowness is also present in QGIS 2.18 as well as in QGIS 3.2.3
> and also occurs with fewer features in shapefiles (15000-16000 are enough),
> slowness increases as the number of unsaved editing operations increases.
>
> With the "attribute table" opened, even "merge elements" or "split feature"
> operations,
> cause some problems (i.e. the counting of the records and the selection of
> the same does not seem synchronized with these editing operations)
> and sometimes this cause freeze of QGIS
>
>
>
>
>
> -
> Antonio
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user



-- 
Alessandro Pasotti
w3:   www.itopen.it
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Antongeo76]

Hi,

This slowness is also present in QGIS 2.18 as well as in QGIS 3.2.3
and also occurs with fewer features in shapefiles (15000-16000 are enough),
slowness increases as the number of unsaved editing operations increases.

With the "attribute table" opened, even "merge elements" or "split feature"
operations,
cause some problems (i.e. the counting of the records and the selection of
the same does not seem synchronized with these editing operations)
and sometimes this cause freeze of QGIS





-
Antonio
--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-User-f4125267.html
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[James Bambury]

Hello,

I've found the refactor fields process to be a reliable and fast backup
option (plus being a process it now works in the background on a separate
thread and shows a progress meter so you've a good idea if it's actually
working!) if you're happy to have entirely new fields added.

I've had similar issues recently, tried to add a virtual field onto a
geopackage layer of about ~5000 features consisting of a
attribute(get_feature()) and ended up freezing up with no sign of progress
after 15min. Refactor field did 6 new fields at once in about 20 seconds!

James

On Tue, 9 Oct 2018 at 22:32, Alexandre Neto  wrote:

> My first suggestion would be for, after importing from csv, save it to
> geopackages for the rest of the operations. Then, in the end export back to
> the necessary file format. I think working directly on text files is not
> optimal.
>
> Best regards,
>
> Alexandre Neto
>
>
> A ter, 9/10/2018, 20:02, Rebecca Bartlett <
> rebeccabartl...@cunet.carleton.ca> escreveu:
>
>> Hello,
>>
>>
>>
>> A couple of colleagues and I have been wrestling with a similar issue to
>> this and I’ve been hunting around to see if it’s documented, and happily
>> saw this email string. I’ve viewed the bug reports (both Kelley’s and the
>> related #19771) and while they capture most of what we’ve been running
>> into, it’s not an exact match. Here’s our situation and any input would be
>> greatly appreciated – including viable workarounds.
>>
>>
>>
>> 1.   Adding a .txt or .csv file to QGIS 3.2.1 as Delimited Text File
>> (5721 features; 9 fields; no geometry)
>>
>> 2.   Using Field Calculator via the attribute table to add a Decimal
>> field (length: 10; precision: 2 – the precision is important which is why
>> the virtual field workaround without opening the attribute table hasn’t
>> worked well with its precision of 3, plus the precision isn’t retained upon
>> table export although we may be missing an export option somewhere) and
>> input values from an already-existing text field
>>
>> a.   Copying text field to numeric field for a join
>>
>> 3.   Field calculation takes less than a second
>>
>> 4.   Saving the table doesn’t work. I left it running for 90 minutes
>> the first time and it has crashed QGIS on colleagues’ computers multiple
>> times
>>
>> a.   Notable that it also doesn’t work on 2.18.24 on Windows
>> machines but it’s fine on 2.18 with Macs (ran the process in a class of 20
>> after we had them use 2.18, and only the Windows users had issues)
>>
>> b.   Still takes a few moments to save on Mac 2.18 – but it works!
>>
>>
>>
>> We’ve tried several workarounds and the best one (a.k.a. the one that
>> works in 3.2.1 and 2.18.24 in Windows) is to use the Field Calculator via
>> the Source Fields tab in Properties, but it still takes >6 minutes to save
>> which seems quite long for 5700 features.
>>
>>
>>
>> I’m assuming this is the same issue but if there are any reliable
>> workarounds we’re all ears!
>>
>>
>>
>> Sincerely,
>>
>> Rebecca
>>
>>
>>
>> Rebecca Bartlett
>>
>> GIS and Digital Resources Librarian
>>
>> Research Support Services
>>
>> Carleton University Library
>>
>> 613-520-2600 x4084
>>
>> rebecca.bartl...@carleton.ca
>>
>>
>> --
>>
>> Okay, I've created an issue for this situation. Hopefully I've done it
>>
>> satisfactorily. Thanks again!
>>
>>
>>
>> On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann > > wrote:
>>
>>
>>
>> >* Hi,*
>>
>> >
>>
>> >* Glad to hear that it helped to run the operations on a closed table.*
>>
>> >* Nevertheless, as Luigi said, it would make sense to submit an issue 
>> >report*
>>
>> >* about this slowness.*
>>
>> >
>>
>> >* The other thing one has to keep on mind:*
>>
>> >
>>
>> >* The SQLite API is kind of limited. Only until recently, if you wanted to*
>>
>> >* rename a field, one had to copy the whole table with the renamed field*
>>
>> >* changed in the new table - then the old table had to be removed (all in 
>> >the*
>>
>> >* background), because there was API to rename columns in SQLite. I think*
>>
>> >* with the very recent version, which is not rolled out into most OS, this*
>>
>> >* has changed. You'd also need a very recent GDAL/OGR.*
>>
>> >* Greetings,*
>>
>> >* Andreas*
>>
>> >
>>
>> >* Am 03.10.2018 um 21:20 schrieb Kelley Jabr:*
>>
>> >
>>
>> >* Luigi, I'm pretty sure you just solved my problem. I tried a few*
>>
>> >* operations with the table closed and it was like night and day. That 400k*
>>
>> >* field calculator finished in mere moments! Thank you!!*
>>
>> >
>>
>> >* On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli > >> wrote:*
>>
>> >
>>
>> >>* can you test doing operations with field calculator without having the*
>>
>> >>* attribute table opened? just a workaround could be to use a virtual 
>> >>layer*
>>
>> >>* then save the entire layer later.*
>>
>> >>
>>
>> >>* Luigi Pirelli*
>>

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Alexandre Neto]

My first suggestion would be for, after importing from csv, save it to
geopackages for the rest of the operations. Then, in the end export back to
the necessary file format. I think working directly on text files is not
optimal.

Best regards,

Alexandre Neto


A ter, 9/10/2018, 20:02, Rebecca Bartlett 
escreveu:

> Hello,
>
>
>
> A couple of colleagues and I have been wrestling with a similar issue to
> this and I’ve been hunting around to see if it’s documented, and happily
> saw this email string. I’ve viewed the bug reports (both Kelley’s and the
> related #19771) and while they capture most of what we’ve been running
> into, it’s not an exact match. Here’s our situation and any input would be
> greatly appreciated – including viable workarounds.
>
>
>
> 1.   Adding a .txt or .csv file to QGIS 3.2.1 as Delimited Text File
> (5721 features; 9 fields; no geometry)
>
> 2.   Using Field Calculator via the attribute table to add a Decimal
> field (length: 10; precision: 2 – the precision is important which is why
> the virtual field workaround without opening the attribute table hasn’t
> worked well with its precision of 3, plus the precision isn’t retained upon
> table export although we may be missing an export option somewhere) and
> input values from an already-existing text field
>
> a.   Copying text field to numeric field for a join
>
> 3.   Field calculation takes less than a second
>
> 4.   Saving the table doesn’t work. I left it running for 90 minutes
> the first time and it has crashed QGIS on colleagues’ computers multiple
> times
>
> a.   Notable that it also doesn’t work on 2.18.24 on Windows machines
> but it’s fine on 2.18 with Macs (ran the process in a class of 20 after we
> had them use 2.18, and only the Windows users had issues)
>
> b.   Still takes a few moments to save on Mac 2.18 – but it works!
>
>
>
> We’ve tried several workarounds and the best one (a.k.a. the one that
> works in 3.2.1 and 2.18.24 in Windows) is to use the Field Calculator via
> the Source Fields tab in Properties, but it still takes >6 minutes to save
> which seems quite long for 5700 features.
>
>
>
> I’m assuming this is the same issue but if there are any reliable
> workarounds we’re all ears!
>
>
>
> Sincerely,
>
> Rebecca
>
>
>
> Rebecca Bartlett
>
> GIS and Digital Resources Librarian
>
> Research Support Services
>
> Carleton University Library
>
> 613-520-2600 x4084
>
> rebecca.bartl...@carleton.ca
>
>
> --
>
> Okay, I've created an issue for this situation. Hopefully I've done it
>
> satisfactorily. Thanks again!
>
>
>
> On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann  > wrote:
>
>
>
> >* Hi,*
>
> >
>
> >* Glad to hear that it helped to run the operations on a closed table.*
>
> >* Nevertheless, as Luigi said, it would make sense to submit an issue report*
>
> >* about this slowness.*
>
> >
>
> >* The other thing one has to keep on mind:*
>
> >
>
> >* The SQLite API is kind of limited. Only until recently, if you wanted to*
>
> >* rename a field, one had to copy the whole table with the renamed field*
>
> >* changed in the new table - then the old table had to be removed (all in 
> >the*
>
> >* background), because there was API to rename columns in SQLite. I think*
>
> >* with the very recent version, which is not rolled out into most OS, this*
>
> >* has changed. You'd also need a very recent GDAL/OGR.*
>
> >* Greetings,*
>
> >* Andreas*
>
> >
>
> >* Am 03.10.2018 um 21:20 schrieb Kelley Jabr:*
>
> >
>
> >* Luigi, I'm pretty sure you just solved my problem. I tried a few*
>
> >* operations with the table closed and it was like night and day. That 400k*
>
> >* field calculator finished in mere moments! Thank you!!*
>
> >
>
> >* On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli  >> wrote:*
>
> >
>
> >>* can you test doing operations with field calculator without having the*
>
> >>* attribute table opened? just a workaround could be to use a virtual layer*
>
> >>* then save the entire layer later.*
>
> >>
>
> >>* Luigi Pirelli*
>
> >>
>
> >>
>
> >>* 
> >>***
>
> >>* * LinkedIn: https://www.linkedin.com/in/luigipirelli 
> >>*
>
> >>* * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli 
> >>*
>
> >>* * GitHub: https://github.com/luipir *
>
> >>* * Mastering QGIS 2nd Edition:*
>
> >>* **
>
> >>* 
> >>https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
> >> 
> >>*
>
> >>* * Hire me: http://goo.gl/BYRQKg *
>
> >>
>
> >>* 
> 

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Rebecca Bartlett]

Hello,

A couple of colleagues and I have been wrestling with a similar issue to this 
and I've been hunting around to see if it's documented, and happily saw this 
email string. I've viewed the bug reports (both Kelley's and the related 
#19771) and while they capture most of what we've been running into, it's not 
an exact match. Here's our situation and any input would be greatly appreciated 
- including viable workarounds.


1.   Adding a .txt or .csv file to QGIS 3.2.1 as Delimited Text File (5721 
features; 9 fields; no geometry)

2.   Using Field Calculator via the attribute table to add a Decimal field 
(length: 10; precision: 2 - the precision is important which is why the virtual 
field workaround without opening the attribute table hasn't worked well with 
its precision of 3, plus the precision isn't retained upon table export 
although we may be missing an export option somewhere) and input values from an 
already-existing text field

a.   Copying text field to numeric field for a join

3.   Field calculation takes less than a second

4.   Saving the table doesn't work. I left it running for 90 minutes the 
first time and it has crashed QGIS on colleagues' computers multiple times

a.   Notable that it also doesn't work on 2.18.24 on Windows machines but 
it's fine on 2.18 with Macs (ran the process in a class of 20 after we had them 
use 2.18, and only the Windows users had issues)

b.   Still takes a few moments to save on Mac 2.18 - but it works!

We've tried several workarounds and the best one (a.k.a. the one that works in 
3.2.1 and 2.18.24 in Windows) is to use the Field Calculator via the Source 
Fields tab in Properties, but it still takes >6 minutes to save which seems 
quite long for 5700 features.

I'm assuming this is the same issue but if there are any reliable workarounds 
we're all ears!

Sincerely,
Rebecca

Rebecca Bartlett
GIS and Digital Resources Librarian
Research Support Services
Carleton University Library
613-520-2600 x4084
rebecca.bartl...@carleton.ca



Okay, I've created an issue for this situation. Hopefully I've done it

satisfactorily. Thanks again!



On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann https://lists.osgeo.org/mailman/listinfo/qgis-user>> wrote:



> Hi,

>

> Glad to hear that it helped to run the operations on a closed table.

> Nevertheless, as Luigi said, it would make sense to submit an issue report

> about this slowness.

>

> The other thing one has to keep on mind:

>

> The SQLite API is kind of limited. Only until recently, if you wanted to

> rename a field, one had to copy the whole table with the renamed field

> changed in the new table - then the old table had to be removed (all in the

> background), because there was API to rename columns in SQLite. I think

> with the very recent version, which is not rolled out into most OS, this

> has changed. You'd also need a very recent GDAL/OGR.

> Greetings,

> Andreas

>

> Am 03.10.2018 um 21:20 schrieb Kelley Jabr:

>

> Luigi, I'm pretty sure you just solved my problem. I tried a few

> operations with the table closed and it was like night and day. That 400k

> field calculator finished in mere moments! Thank you!!

>

> On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli  gmail.com> wrote:

>

>> can you test doing operations with field calculator without having the

>> attribute table opened? just a workaround could be to use a virtual layer

>> then save the entire layer later.

>>

>> Luigi Pirelli

>>

>>

>> **

>> * LinkedIn: https://www.linkedin.com/in/luigipirelli

>> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli

>> * GitHub: https://github.com/luipir

>> * Mastering QGIS 2nd Edition:

>> *

>> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition

>> * Hire me: http://goo.gl/BYRQKg

>>

>> **

>>

>>

>> On Wed, 3 Oct 2018 at 18:11, Kelley Jabr > indeed.com> wrote:

>>

>>> Hello,

>>>

>>> I'm working with a geopackage on ~530k features, and just about

>>> everything I try to do in the table takes way longer than expected.

>>>

>>>- Adding a field took 5 minutes (table now has 22 total fields, so

>>>not massive)

>>>- Field Calculator on ~32k selected records took 8 minutes

>>>(literally all I was putting in the cells was: N)

>>>- Field calculator on ~480k records took so many hours that I had to

>>>leave it running over the weekend and when I came back, the Field

>>>Calculator had "closed unexpectedly"

>>>

>>> (I'm running on a linux machine with an Intel Core i7 and 32GB of

>>> memory.)

>>>

>>> 

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Alexandre Neto]

Hi,

In SQL queries on databases like Postgresql it is common to use the Limit
operand to ask just a sample of the output to check if all looks OK without
reading the all table.

This situation made me wonder if this couldn't be a nice feature request to
the layers attribute table when working with very large tables: to only
show/load the first X features.

Obvious operations like selection or field calculator on the table should
affect the all table, and not just the visible layers.

I am not sure if it would have an impact on how fast working with the table
looks. What Kelley case seems to show, is that loading and showing the
table is where the problem is, not in the operations on the attributes.

What do u guy think?

Best regards,

Alex Neto



A qua, 3/10/2018, 21:54, Luigi Pirelli  escreveu:

> tnx Kelly, this is your issue and it's ok:
> https://issues.qgis.org/issues/20008
>
> to everyone, please add any comment or info that can help to
> replicate/solve
>
> Luigi Pirelli
>
>
> **
> * LinkedIn: https://www.linkedin.com/in/luigipirelli
> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
> * GitHub: https://github.com/luipir
> * Mastering QGIS 2nd Edition:
> *
> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
> * Hire me: http://goo.gl/BYRQKg
>
> **
>
>
> On Wed, 3 Oct 2018 at 22:40, Kelley Jabr  wrote:
>
>> Okay, I've created an issue for this situation. Hopefully I've done it
>> satisfactorily. Thanks again!
>>
>> On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann 
>> wrote:
>>
>>> Hi,
>>>
>>> Glad to hear that it helped to run the operations on a closed table.
>>> Nevertheless, as Luigi said, it would make sense to submit an issue report
>>> about this slowness.
>>>
>>> The other thing one has to keep on mind:
>>>
>>> The SQLite API is kind of limited. Only until recently, if you wanted to
>>> rename a field, one had to copy the whole table with the renamed field
>>> changed in the new table - then the old table had to be removed (all in the
>>> background), because there was API to rename columns in SQLite. I think
>>> with the very recent version, which is not rolled out into most OS, this
>>> has changed. You'd also need a very recent GDAL/OGR.
>>> Greetings,
>>> Andreas
>>>
>>> Am 03.10.2018 um 21:20 schrieb Kelley Jabr:
>>>
>>> Luigi, I'm pretty sure you just solved my problem. I tried a few
>>> operations with the table closed and it was like night and day. That 400k
>>> field calculator finished in mere moments! Thank you!!
>>>
>>> On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli  wrote:
>>>
 can you test doing operations with field calculator without having the
 attribute table opened? just a workaround could be to use a virtual layer
 then save the entire layer later.

 Luigi Pirelli


 **
 * LinkedIn: https://www.linkedin.com/in/luigipirelli
 * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
 * GitHub: https://github.com/luipir
 * Mastering QGIS 2nd Edition:
 *
 https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
 * Hire me: http://goo.gl/BYRQKg

 **


 On Wed, 3 Oct 2018 at 18:11, Kelley Jabr  wrote:

> Hello,
>
> I'm working with a geopackage on ~530k features, and just about
> everything I try to do in the table takes way longer than expected.
>
>- Adding a field took 5 minutes (table now has 22 total fields, so
>not massive)
>- Field Calculator on ~32k selected records took 8 minutes
>(literally all I was putting in the cells was: N)
>- Field calculator on ~480k records took so many hours that I had
>to leave it running over the weekend and when I came back, the Field
>Calculator had "closed unexpectedly"
>
> (I'm running on a linux machine with an Intel Core i7 and 32GB of
> memory.)
>
> Has anyone else experience similar issues? Any advice or
> recommendations for how to improve performance here?
>
> Thank you,
>
> --
>
> Kelley Jabr
>
> Analyst, Geographic Data Operations
>
> kj...@indeed.com
>
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user


>>>
>>> --
>>>
>>> Kelley Jabr
>>>
>>> Analyst, Geographic Data Operations
>>>
>>> kj...@indeed.com
>>>
>>> Indeed 

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Luigi Pirelli]

tnx Kelly, this is your issue and it's ok:
https://issues.qgis.org/issues/20008

to everyone, please add any comment or info that can help to replicate/solve

Luigi Pirelli

**
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
*
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg
**


On Wed, 3 Oct 2018 at 22:40, Kelley Jabr  wrote:

> Okay, I've created an issue for this situation. Hopefully I've done it
> satisfactorily. Thanks again!
>
> On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann 
> wrote:
>
>> Hi,
>>
>> Glad to hear that it helped to run the operations on a closed table.
>> Nevertheless, as Luigi said, it would make sense to submit an issue report
>> about this slowness.
>>
>> The other thing one has to keep on mind:
>>
>> The SQLite API is kind of limited. Only until recently, if you wanted to
>> rename a field, one had to copy the whole table with the renamed field
>> changed in the new table - then the old table had to be removed (all in the
>> background), because there was API to rename columns in SQLite. I think
>> with the very recent version, which is not rolled out into most OS, this
>> has changed. You'd also need a very recent GDAL/OGR.
>> Greetings,
>> Andreas
>>
>> Am 03.10.2018 um 21:20 schrieb Kelley Jabr:
>>
>> Luigi, I'm pretty sure you just solved my problem. I tried a few
>> operations with the table closed and it was like night and day. That 400k
>> field calculator finished in mere moments! Thank you!!
>>
>> On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli  wrote:
>>
>>> can you test doing operations with field calculator without having the
>>> attribute table opened? just a workaround could be to use a virtual layer
>>> then save the entire layer later.
>>>
>>> Luigi Pirelli
>>>
>>>
>>> **
>>> * LinkedIn: https://www.linkedin.com/in/luigipirelli
>>> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
>>> * GitHub: https://github.com/luipir
>>> * Mastering QGIS 2nd Edition:
>>> *
>>> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
>>> * Hire me: http://goo.gl/BYRQKg
>>>
>>> **
>>>
>>>
>>> On Wed, 3 Oct 2018 at 18:11, Kelley Jabr  wrote:
>>>
 Hello,

 I'm working with a geopackage on ~530k features, and just about
 everything I try to do in the table takes way longer than expected.

- Adding a field took 5 minutes (table now has 22 total fields, so
not massive)
- Field Calculator on ~32k selected records took 8 minutes
(literally all I was putting in the cells was: N)
- Field calculator on ~480k records took so many hours that I had
to leave it running over the weekend and when I came back, the Field
Calculator had "closed unexpectedly"

 (I'm running on a linux machine with an Intel Core i7 and 32GB of
 memory.)

 Has anyone else experience similar issues? Any advice or
 recommendations for how to improve performance here?

 Thank you,

 --

 Kelley Jabr

 Analyst, Geographic Data Operations

 kj...@indeed.com

 ___
 Qgis-user mailing list
 Qgis-user@lists.osgeo.org
 List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
 Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>>
>>>
>>
>> --
>>
>> Kelley Jabr
>>
>> Analyst, Geographic Data Operations
>>
>> kj...@indeed.com
>>
>> Indeed - We help people get jobs.
>>
>> Indeed.com 
>>
>>
>>
>> ___
>> Qgis-user mailing listqgis-u...@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
>>
>> ___
>> Qgis-user mailing list
>> Qgis-user@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
>
>
> --
>
> Kelley Jabr
>
> Analyst, Geographic Data Operations
>
> kj...@indeed.com
>
> Indeed - We help people get jobs.
>
> Indeed.com 
>
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: 

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Andreas Neumann]

Hi,

Glad to hear that it helped to run the operations on a closed table. 
Nevertheless, as Luigi said, it would make sense to submit an issue 
report about this slowness.


The other thing one has to keep on mind:

The SQLite API is kind of limited. Only until recently, if you wanted to 
rename a field, one had to copy the whole table with the renamed field 
changed in the new table - then the old table had to be removed (all in 
the background), because there was API to rename columns in SQLite. I 
think with the very recent version, which is not rolled out into most 
OS, this has changed. You'd also need a very recent GDAL/OGR.


Greetings,
Andreas

Am 03.10.2018 um 21:20 schrieb Kelley Jabr:
Luigi, I'm pretty sure you just solved my problem. I tried a few 
operations with the table closed and it was like night and day. That 
400k field calculator finished in mere moments! Thank you!!


On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli > wrote:


can you test doing operations with field calculator without having
the attribute table opened? just a workaround could be to use a
virtual layer then save the entire layer later.

Luigi Pirelli


**
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange:
http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
*

https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg

**


On Wed, 3 Oct 2018 at 18:11, Kelley Jabr mailto:kj...@indeed.com>> wrote:

Hello,

I'm working with a geopackage on ~530k features, and just
about everything I try to do in the table takes way longer
than expected.

  * Adding a field took 5 minutes (table now has 22 total
fields, so not massive)
  * Field Calculator on ~32k selected records took 8 minutes
(literally all I was putting in the cells was: N)
  * Field calculator on ~480k records took so many hours that
I had to leave it running over the weekend and when I came
back, the Field Calculator had "closed unexpectedly"

(I'm running on a linux machine with an Intel Core i7 and 32GB
of memory.)

Has anyone else experience similar issues? Any advice or
recommendations for how to improve performance here?

Thank you,

-- 


Kelley Jabr

Analyst, Geographic Data Operations

kj...@indeed.com 


___
Qgis-user mailing list
Qgis-user@lists.osgeo.org 
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user



--

Kelley Jabr

Analyst, Geographic Data Operations

kj...@indeed.com 


Indeed- We help people get jobs.

Indeed.com 




___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user


___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Luigi Pirelli]

may you open an issue about this slowness with all details to reproduce?
Only with opened issue the project can allocate resources to fix them...
and we are just in bugfix period before the 3.4 release. So your
contribution would be precious.

tnx

Luigi Pirelli

**
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
*
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg
**


On Wed, 3 Oct 2018 at 21:20, Kelley Jabr  wrote:

> Luigi, I'm pretty sure you just solved my problem. I tried a few
> operations with the table closed and it was like night and day. That 400k
> field calculator finished in mere moments! Thank you!!
>
> On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli  wrote:
>
>> can you test doing operations with field calculator without having the
>> attribute table opened? just a workaround could be to use a virtual layer
>> then save the entire layer later.
>>
>> Luigi Pirelli
>>
>>
>> **
>> * LinkedIn: https://www.linkedin.com/in/luigipirelli
>> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
>> * GitHub: https://github.com/luipir
>> * Mastering QGIS 2nd Edition:
>> *
>> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
>> * Hire me: http://goo.gl/BYRQKg
>>
>> **
>>
>>
>> On Wed, 3 Oct 2018 at 18:11, Kelley Jabr  wrote:
>>
>>> Hello,
>>>
>>> I'm working with a geopackage on ~530k features, and just about
>>> everything I try to do in the table takes way longer than expected.
>>>
>>>- Adding a field took 5 minutes (table now has 22 total fields, so
>>>not massive)
>>>- Field Calculator on ~32k selected records took 8 minutes
>>>(literally all I was putting in the cells was: N)
>>>- Field calculator on ~480k records took so many hours that I had to
>>>leave it running over the weekend and when I came back, the Field
>>>Calculator had "closed unexpectedly"
>>>
>>> (I'm running on a linux machine with an Intel Core i7 and 32GB of
>>> memory.)
>>>
>>> Has anyone else experience similar issues? Any advice or recommendations
>>> for how to improve performance here?
>>>
>>> Thank you,
>>>
>>> --
>>>
>>> Kelley Jabr
>>>
>>> Analyst, Geographic Data Operations
>>>
>>> kj...@indeed.com
>>>
>>> ___
>>> Qgis-user mailing list
>>> Qgis-user@lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
>>
>
> --
>
> Kelley Jabr
>
> Analyst, Geographic Data Operations
>
> kj...@indeed.com
>
> Indeed - We help people get jobs.
>
> Indeed.com 
>
>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Kelley Jabr]

Luigi, I'm pretty sure you just solved my problem. I tried a few operations
with the table closed and it was like night and day. That 400k field
calculator finished in mere moments! Thank you!!

On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli  wrote:

> can you test doing operations with field calculator without having the
> attribute table opened? just a workaround could be to use a virtual layer
> then save the entire layer later.
>
> Luigi Pirelli
>
>
> **
> * LinkedIn: https://www.linkedin.com/in/luigipirelli
> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
> * GitHub: https://github.com/luipir
> * Mastering QGIS 2nd Edition:
> *
> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
> * Hire me: http://goo.gl/BYRQKg
>
> **
>
>
> On Wed, 3 Oct 2018 at 18:11, Kelley Jabr  wrote:
>
>> Hello,
>>
>> I'm working with a geopackage on ~530k features, and just about
>> everything I try to do in the table takes way longer than expected.
>>
>>- Adding a field took 5 minutes (table now has 22 total fields, so
>>not massive)
>>- Field Calculator on ~32k selected records took 8 minutes (literally
>>all I was putting in the cells was: N)
>>- Field calculator on ~480k records took so many hours that I had to
>>leave it running over the weekend and when I came back, the Field
>>Calculator had "closed unexpectedly"
>>
>> (I'm running on a linux machine with an Intel Core i7 and 32GB of memory.)
>>
>> Has anyone else experience similar issues? Any advice or recommendations
>> for how to improve performance here?
>>
>> Thank you,
>>
>> --
>>
>> Kelley Jabr
>>
>> Analyst, Geographic Data Operations
>>
>> kj...@indeed.com
>>
>> ___
>> Qgis-user mailing list
>> Qgis-user@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
>

-- 

Kelley Jabr

Analyst, Geographic Data Operations

kj...@indeed.com

Indeed - We help people get jobs.

Indeed.com 
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Luigi Pirelli]

can you test doing operations with field calculator without having the
attribute table opened? just a workaround could be to use a virtual layer
then save the entire layer later.

Luigi Pirelli

**
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
*
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg
**


On Wed, 3 Oct 2018 at 18:11, Kelley Jabr  wrote:

> Hello,
>
> I'm working with a geopackage on ~530k features, and just about everything
> I try to do in the table takes way longer than expected.
>
>- Adding a field took 5 minutes (table now has 22 total fields, so not
>massive)
>- Field Calculator on ~32k selected records took 8 minutes (literally
>all I was putting in the cells was: N)
>- Field calculator on ~480k records took so many hours that I had to
>leave it running over the weekend and when I came back, the Field
>Calculator had "closed unexpectedly"
>
> (I'm running on a linux machine with an Intel Core i7 and 32GB of memory.)
>
> Has anyone else experience similar issues? Any advice or recommendations
> for how to improve performance here?
>
> Thank you,
>
> --
>
> Kelley Jabr
>
> Analyst, Geographic Data Operations
>
> kj...@indeed.com
>
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Nicolas Cadieux]

Hi,

I had similar problems with shp files in earlier version of QGIS.  I would 
reboot the Window machine and restart QGIS but the problem would start again 
after a number of edits.  I think it was a plugin that was causing a memory 
leak.  Try closing all the plugins and then re-starting Qgis.  Half a million 
is not that small.  Have you tried with a different files format just to check 
the speed?

Nicolas

> Le 3 oct. 2018 à 12:10, Kelley Jabr  a écrit :
> 
> Hello,
> 
> I'm working with a geopackage on ~530k features, and just about everything I 
> try to do in the table takes way longer than expected. 
> Adding a field took 5 minutes (table now has 22 total fields, so not massive)
> Field Calculator on ~32k selected records took 8 minutes (literally all I was 
> putting in the cells was: N)
> Field calculator on ~480k records took so many hours that I had to leave it 
> running over the weekend and when I came back, the Field Calculator had 
> "closed unexpectedly"
> (I'm running on a linux machine with an Intel Core i7 and 32GB of memory.)
> 
> Has anyone else experience similar issues? Any advice or recommendations for 
> how to improve performance here? 
> 
> Thank you,
> 
> -- 
> Kelley Jabr
> Analyst, Geographic Data Operations
> kj...@indeed.com
> 
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user