Darn ! I missed one of the postings about you setting the file in
read-only on the NAS. So this is an already known work-around. Sorry
about the noise.
-------- Videresendt meddelelse --------
Emne: Re: [Qgis-user] [EXTERNAL] Re: Geopackage slow on NAS if not
read-only
Dato: Thu, 16 Mar 2023 14:35:16 +0100
Fra: Bo Victor Thomsen <bo.victor.thom...@gmail.com>
Til: qgis-user@lists.osgeo.org
When any user on your network access the geopackage layer and (maybe
unnecessarily) puts the layer in edit mode, there will be created 2
extra files in the same directory as the gpkg file resides in. And QGIS
behaves different regarding /both/ read and write operations when these
file are present. This is probably the explanation of the longer reading
times, even if nobody actually is editing the layer. When the user stops
the editing mode for the layer, the 2 files disappear.
You can check if this is the explanation:
1. Open the gpkg layer in read mode on one computer, check the access
speed of the same layer on /another/ computer.
2. Set the layer in edit mode on the first computer, check the access
speed to the layer on the second computer
3. Revert the edit mode on the first computer, check the access speed
to the layer on the second computer.
I am guessing, that situations 1 and 3 are fast, while situation 2 is slow.
First of all: Never, ever try to implement some kind of multi-user
editing on a file based format, where the file resides on a networked
drive. It will never, ever work reliably. At some point 2 users will try
to edit the same layer at the same time and it will go kaboom (I can
read from your mails that you are aware about this). This goes for -
probably - every file based format on a network drive.
Secondly: What about making the gpkg file read-only at the network share
level for most of the users. And only granting write access to the gpkg
file for certain users that are instructed in /not/ setting the layer in
edit mode unless it's strictly necessary ?
The best solution: Install Postgres/PostGIS on your NAS
Med venlig hilsen / Best regards
Bo Victor Thomsen
Den 16-03-2023 kl. 13:49 skrev Árni Geirsson via QGIS-User:
Thank you for the various responses.
Let me be clear that what I am griping about is not the locking aspect
that would allow multiple users to edit without conflict. I fully
understand that the geopackage is not safe for that, nor is the
shapefile and perhaps also the file geodatabase. In my case, the
likelihood of two people wanting to edit the same geopackage is
negligible, so I take my chances. My only gripe is the rendering speed
of larger geopackages on a shared network, if they are not set
specifically to read-only. That is not a problem with the shapefile
and I think probably not with a file geodatabase. I am a long time
user of geopackages in a networked environment and I will continue to
use them - they are great - there is just this problem of rendering
speed over a network.
Árni
On Thu, 16 Mar 2023 at 12:33, Sebastian Gutwein <b...@rdgland.com> wrote:
This is all beyond my expertise but I have also struggled with the
effective use of gpkg’s. In my case over google drive.
Here is what SQLite (the underlying software for gpkgs) says about
using a SQLite database on a network:
https://www.sqlite.org/useovernet.html
I hope that someday someone implements a system that allows this
to work efficiently as gpkgs are better in many respects than
shapefiles. Perhaps Geodiff could be part of that solution.
https://github.com/MerginMaps/geodiff
On Thu, Mar 16, 2023 at 8:04 AM Árni Geirsson via QGIS-User
<qgis-user@lists.osgeo.org> wrote:
Hi Jorge
I understand what you are pointing at and I use databases such
as Postgres/PostGIS also with good results. The thing is that
working with data in files sometimes has advantages and that
is certainly how shapefiles have been used. The geopackage has
been suggested as a replacement for the shapefile in the
context of regular QGIS usage without any caveat saying that
geopackages only replace the storage and transfer role of
shapefiles and that geopackages should not be used for
editing in a shared environment, as is perfectly possible with
shapefiles and widely practiced. Not all users have access to
database systems such as Postgres. I'm just looking for some
clarity on best practices for the common scenario of working
with QGIS using file based data in a network environment.
Árni
On Thu, 16 Mar 2023 at 11:47, Jorge Gustavo Rocha via
QGIS-User <qgis-user@lists.osgeo.org> wrote:
Hi,
For data storage and manipulation databases are suitable.
Files are not.
Geopackages are wonderful to transfer data between systems
or to archive an entire project (snapshot of data, styles
and the project itself).
Regards,
Jorge
On 16/03/23 11:30, Árni Geirsson via QGIS-User wrote:
Thank you Jarosław. Isn't it strange that this was
discussed 5 years ago and SMB file sharing is very
common? Would a linux based NAS be able to use another
protocol? What are my options for file based data sharing
in QGIS? Abandoning geopackages is not a realistic option
for me, but I could get a different kind of NAS unit, if
that helps, what kind then? What amazes me is how little
I see this discussed. There was a message in this thread
this morning from Thomas Struller, but I am not sure it
is about the same root problem, maybe Thomas can elaborate.
Should this perhaps be discussed in another forum, closer
to the development of geopackage/sqlite?
Árni Geirsson
On Thu, 16 Mar 2023 at 09:52, Sadowski Jarosław
<jaroslaw.sadow...@cpk.pl> wrote:
Long story short: gpkg is bad idea for network drives
as SMB/NAS etc
Sources:
Write-Ahead Logging (sqlite.org)
<https://www.sqlite.org/wal.html#advantages>
writing gpkg and sqlite on samba shares fails · Issue
#628 · r-spatial/sf · GitHub
<https://github.com/r-spatial/sf/issues/628>
*_________________________________*
*Jarosław Sadowski*
Kierownik Zespołu ds. Ochrony Środowiska |/Biuro
Strategii i Planowania, Projektowania i Inżynierii
Podprogramu Kolejowego/
/Environmental Protection Team Leader //| Railway
Subprogramme Strategy & Planning, Design &
Engineering Department/
e: jaroslaw.sadow...@cpk.pl
m: +48 532 720 230
Centralny Port Komunikacyjny Sp. z o.o. z siedzibą w
Warszawie, Aleje Jerozolimskie 142B, 02-305 Warszawa
<https://www.google.com/maps/search/Aleje%0D%0A++++++++++++++++++++Jerozolimskie+142B,+02-305+Warszawa?entry=gmail&source=g>;
nr KRS 0000759991, Sąd Rejonowy dla m.st
<http://m.st>. Warszawy, XII Wydział Gospodarczy
Krajowego Rejestru Sądowego; NIP 701-08-94-497; REGON
381918620; kapitał zakładowy 1.277.500.000,00 zł
Administratorem danych osobowych przekazanych przez
Panią/Pana m.in. w korespondencji mailowej jest
Centralny Port Komunikacyjny Sp. z o.o. z siedzibą w
Warszawie. Przetwarzamy dane osobowe zgodnie z
przepisami ogólnego rozporządzenia o ochronie danych
(RODO), więcej informacji na ten temat znajduje się w
zakładce Polityka Prywatności oraz w Klauzuli
informacyjnej na stronie internetowej www.cpk.pl
<http://www.cpk.pl/>.
Treści zawarte w niniejszej wiadomości i załącznikach
do niej stanowią Tajemnicę Przedsiębiorstwa w
rozumieniu ustawy z dnia 16 kwietnia 1993 r. o
zwalczaniu nieuczciwej konkurencji. Jeśli otrzymałeś
tę wiadomość przez pomyłkę, bezzwłocznie skontaktuj
się z nadawcą wiadomości oraz usuń jej treść.
Centralny Port Komunikacyjny Sp. z o.o. with
headquarters in Warsaw, Aleje Jerozolimskie 142B,
02-305 Warsaw
<https://www.google.com/maps/search/Warsaw,+Aleje%0D%0A++++++++++++++++++++Jerozolimskie+142B,+02-305+Warsaw?entry=gmail&source=g>;
KRS No. 0000759991, District Court for the Capital
City of Warsaw, 12th Commercial Department of the
National Court Register; NIP 701-08-94-497; REGON
381918620; share capital of PLN 1.277.500.000,00.
The personal data controller of the personal data
provided by you, among others, in the e-mail
correspondence, is Centralny Port Komunikacyjny Sp. z
o. o. based in Warsaw. The personal data is processed
by us in accordance with the provisions of the
General Data Protection Regulation (GDPR), for
further information please read the Privacy Policy
tab and the Information Clause on the www.cpk.pl
<http://www.cpk.pl/>website.
The content of this message and its attachments
constitute the Business Secret within the meaning of
the Act of 16 April 1993 on combating unfair
competition. If you received this message by mistake,
contact the sender of the message immediately and
delete its content
*From:*QGIS-User <qgis-user-boun...@lists.osgeo.org>
*On Behalf Of *Árni Geirsson via QGIS-User
*Sent:* Thursday, March 16, 2023 10:18 AM
*To:* jhubbsl...@att.net
*Cc:* qgis-user@lists.osgeo.org
*Subject:* [EXTERNAL] Re: [Qgis-user] Geopackage slow
on NAS if not read-only
UWAGA: Ta wiadomość pochodzi spoza CPK Sp. z o.o.
Proszę pomóż zadbać o bezpieczeństwo naszej
organizacji. Zastanów się, zanim otworzysz link lub
załącznik.
Yes, as far as I know, the NAS unit uses SMB as the
file sharing protocol. I used another NAS from QNAP
before this one, also using SMB and with the same
problem. I thought pretty much all of the Linux based
NAS units were using SMB and if that is the problem,
it should be widespread, but I don't see any signs of
that. Is SMB a problem for geopackage?
I did a quick test in QGIS: A dataset of 178.000 line
features is rendered in about 1 second from a read
only geopackage. When I remove the read only flag, it
is rendered in about 5 seconds.
If I store the geopackage on the local hard drive,
the problem disappears, read-write or read-only does
not matter.
I have had suspicions about SMB being part of the
problem but I don't know enough about file access
deep down in the operating system to understand it.
Árni Geirsson
On Thu, 16 Mar 2023 at 01:45, jhubbslist--- via
QGIS-User <qgis-user@lists.osgeo.org
<mailto:qgis-user@lists.osgeo.org>> wrote:
Árni -
Are you using SMB/CIFS to access this NAS, and
are you using wifi or Ethernet to connect to it?
- Jeff
On 3/15/23 3:30 PM, Árni Geirsson via QGIS-User
wrote:
Hello all QGIS and geopackage users.
I store my geopackages on a Synology
RackStation NAS unit, like all other
documents that are kept on a shared drive in
the office. For larger datasets, the
rendering is very slow, unless I open the
properties dialog for the file in Windows and
check the read only box. After that, the
features are rendered blazingly fast. Nothing
else is changed to see the dramatic
difference in the rendering speed. Luckily, I
don't need to edit many of the larger
datasets, such as road networks and elevation
contours and the geopackage can be kept read
only. Shapefiles are not affected.
What explains this and does anyone know how
to solve the problem?
Do other users experience this?
Árni Geirsson
_______________________________________________
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info:https://lists.osgeo.org/mailman/listinfo/qgis-user
<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
Unsubscribe:
https://lists.osgeo.org/mailman/listinfo/qgis-user
<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
_______________________________________________
QGIS-User mailing list
QGIS-User@lists.osgeo.org
<mailto:QGIS-User@lists.osgeo.org>
List info:
https://lists.osgeo.org/mailman/listinfo/qgis-user
<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
Unsubscribe:
https://lists.osgeo.org/mailman/listinfo/qgis-user
<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
_______________________________________________
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info:https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-user
--
Jorge Gustavo Rocha
Geomaster, LDA
"Take the open source journey with us"
VAT/NIF: PT 510 906 109
Phone: +351 253 257 173 (landline)
Mobile: +351 910 333 888 (cellular)
Rua Afonso Palmeira, 31, 7
<https://www.google.com/maps/search/Rua+Afonso+Palmeira,+31,+7?entry=gmail&source=g>
D
4715-278 Braga
PORTUGAL
GPS 41.54627, -8.40432
_______________________________________________
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe:
https://lists.osgeo.org/mailman/listinfo/qgis-user
_______________________________________________
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
--
Sent from a phone. Amazing! But possibly garbled. Sebastian
Gutwein Regenerative Design Group 1 Chevalier Ave Greenfield MA
b...@rdgland.com (631) 241-1018
_______________________________________________
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info:https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-user
_______________________________________________
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
