Re: [Qgis-user] Re: File-based Mastermap

[maning sambale]

If you're also considering offline editing, spatialite and qgis is the
way to go.  You can create a parent spatialite and convert the data as
an offline spatialite which is what you will use for your field
laptops.  Back in the "office", you can sync the edited offline
spatialite db into the paraent db.

On Fri, Sep 30, 2011 at 3:05 AM, magerlin  wrote:
> I was considering suggesting SpatiaLite too but thougth I would just do a
> litte test before suggesting it.
>
> In Qgis 1.7 (win 7) loading and display of a shapefile with 600 000 features
> took 18 seconds.
>
> The same data converted to a SpatiaLite database took 30 seconds to load.
>
> So I am in doubt whether SpatiaLite is the thing to go for?



-- 
cheers,
maning
--
"Freedom is still the most radical idea of all" -N.Branden
wiki: http://esambale.wikispaces.com/
blog: http://epsg4253.wordpress.com/
--
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-user

[Qgis-user] Re: File-based Mastermap

[Jukka Rahkonen]

  writes:

> 
> I need to deploy four stand-alone laptops with
> a basic GIS (data viewing only) and have chosen QGIS for this. The data
> held is for the entire county of Warwickshire, and one of the datasets
> I need to put on there is MasterMap. Specifically the datasets I'm using
> are:Topographic line: 3.7 million featuresTopographic Area: 1.3 million
featuresCartographic Text: 250,000 featuresGiven the dataset size, obviously the
first
> choice is a database, but as these are stand-alone laptops that must have
> the simplest setup possible (they're going to be used out in the field
> away from tech-support) I've ruled that option out.I tried shapefile's but it
turns out that

Hi,

I think that the best method for browsing that much of vector data would be to
use OpenJUMP or copy the way how it handles PostGIS layers in to QGIS.

What is special with the OpenJUMP native PostGIS driver is that PostGIS layers
are dynamic. Only the features from withing the map view are queried from the
database. They are cached for some time but previously used features are thrown
away periodically which keeps the memory consumption small. You can have
millions of features in the database but when you zoom in OpenJUMP is having
only a handful of features on the layer. Application makes a new query when user
is panning so all the millions of features are available when needed and the
system is practically as fast as PostGIS is.

What the user must do is to set scale limits for the database layers. It does
not make sense ever to show million features on a map. When zoomed out the
application must switch off the detailed layer and show some genaralised layer
instead. That could be done by making such a into PostGIS and addind that as
another layer into GIS project.

-Jukka Rahkonen-

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-user

[Qgis-user] Re: File-based Mastermap

[magerlin]

I was considering suggesting SpatiaLite too but thougth I would just do a
litte test before suggesting it.

In Qgis 1.7 (win 7) loading and display of a shapefile with 600 000 features
took 18 seconds.

The same data converted to a SpatiaLite database took 30 seconds to load.

So I am in doubt whether SpatiaLite is the thing to go for?

Regards Morten


Rahkonen Jukka wrote:
> 
> Andreas Neumann  writes:
> 
>> 
>>  Perhaps you could also look into SpatiaLite.
>> 
>>  It comes without the limitations of Shapefiles (column names, file 
>>  sizes) and is fast and a database without installation (file based).
>> .


--
View this message in context: 
http://osgeo-org.1803224.n2.nabble.com/File-based-Mastermap-tp6839547p6845096.html
Sent from the qgis-user mailing list archive at Nabble.com.
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-user

[Qgis-user] Re: File-based Mastermap

[Jukka Rahkonen]

Andreas Neumann  writes:

> 
>  Perhaps you could also look into SpatiaLite.
> 
>  It comes without the limitations of Shapefiles (column names, file 
>  sizes) and is fast and a database without installation (file based).
> 
>  Also you need to copy only one file instead of multiple ones.
> 
>  SpatiaLite is not yet supported by FME (will be supported from FME 2013 
>  on I heard) - but you can use ogr2ogr for file translations.
> 
>  I believe the new DB-Manager Plugin from the last Google Summer of Code 
>  can also assist in converting data from Postgis to SpatiaLite, but I am 
>  not 100% sure.
> 
>  Andreas

GDAL SQLite driver does not make the most valid Spatialite databases. Alessandro
Furieri (man behind Spatialite)wrote:
"The current version of the spatialite's OGR driver 
has many severe issues.
Shortly said, you cannot safely use OGR to create a 
SpatiaLite own DB, because this will simply produce 
a broken (invalid, inconsistent) DB.
I've already developed a full patchset for OGR, and 
I hope to be able to release the code to the GDAL
project ASAP" 

While waiting for the patches there are two usable methods for making valid,
consistent DB:
1. Import shapefiles directly with Spatialite tools or with Spatialite-gui.
2. Use ogr2ogr first with options
   ogr2ogr -f "SQLite" -dsco SPATIALITE=yes -lco SPATIAL_INDEX=no temp.sqlite
source_data
3. Clean up the temporary spatialite database with Openlite utility by creating
a new empty database and copying the tables from the temp.sqlite into it.

Spatialite=yes option needs gdal 1.7.0 or higher
http://www.gdal.org/ogr/drv_sqlite.html

The troubles with Spatialite databases created by ogr2ogr are not very big. Some
foreign key constraints and triggers are not OK which makes it harder to make
properly working spatial views into database etc. For simple read-only access to
the tables the databases created with ogr2ogr are OK.

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-user