[Viking-devel] Viking cache directory
Hi everyone,
I started to use viking recently and I absolutely love it.
I'm also using JOSM heavily when contributing to OSM, and I'm trying to
save on bandwidth by trying to share the imagery cache between them.
I've already submitted the changes to JOSM's tracker, so that it simply
creates a directory hierarchy in the classical on-disk tile format
(Z/X/Y.png). JOSM adds some additional (useless) files into it, but
that's about it.
viking already loads the on-disk tile format, so I could already re-use
the imagery without downloading them twice, but that was inconvenient
when exploring new areas.
With the attached patch I just slightly modify the DIRSTRUCTURE so that
the default cache directory is basically also the same on-disk tile
format: ~/.viking-maps/t[ID]z[z]/(17-zoom)/x/y.png
This way I can just symlink the directories between the two programs,
which is a true boon.
I assume z here is actually used by someone? Because I only have 0
myself when browsing through Bing's imagery and mapnik. It would make
sense IMHO to just use the identical function/format between
DIRSTRUCTURE and DIRECTDIRACCESS, and make the cache sub-directories
just like any regular tile structure.
png is really a fat lie for aerial imagery, but then again both JOSM
and viking load it anyway. Specifying it in DIRSTRUCTURE like for
DIRECTDIRACCESS seemed superfluous, since png is hardcoded everywhere.
No user-visible changes, except that the old cache won't be used due to
the name change.
Any chance this or an improved version of this patch could get integrated?
Thanks.
diff -rud viking-1.5.Orig/src/vikmapslayer.c viking-1.5/src/vikmapslayer.c
--- viking-1.5.Orig/src/vikmapslayer.c 2014-03-12 20:51:25.065240587 +0100
+++ viking-1.5/src/vikmapslayer.c 2014-03-12 21:03:56.781208974 +0100
@@ -368,7 +368,7 @@
//
#define DIRECTDIRACCESS %s%d G_DIR_SEPARATOR_S %d G_DIR_SEPARATOR_S %d%s
-#define DIRSTRUCTURE %st%ds%dz%d G_DIR_SEPARATOR_S %d G_DIR_SEPARATOR_S %d
+#define DIRSTRUCTURE %st%dz%d G_DIR_SEPARATOR_S %d G_DIR_SEPARATOR_S %d G_DIR_SEPARATOR_S %d.png
#define MAPS_CACHE_DIR maps_layer_default_dir()
#ifdef WINDOWS
@@ -719,8 +719,8 @@
vml-cache_dir, (17 - mapcoord-scale), mapcoord-x, mapcoord-y, .png );
else
g_snprintf ( filename_buf, buf_len, DIRSTRUCTURE,
- vml-cache_dir, mode,
- mapcoord-scale, mapcoord-z, mapcoord-x, mapcoord-y );
+ vml-cache_dir, mode, mapcoord-z,
+ (17 - mapcoord-scale), mapcoord-x, mapcoord-y );
if ( g_file_test ( filename_buf, G_FILE_TEST_EXISTS ) == TRUE)
{
@@ -907,7 +907,7 @@
else
g_snprintf ( path_buf, max_path_len, DIRSTRUCTURE,
vml-cache_dir, mode,
- ulm.scale, ulm.z, ulm.x, ulm.y );
+ ulm.z, (17 - ulm.scale), ulm.x, ulm.y );
if ( g_file_test ( path_buf, G_FILE_TEST_EXISTS ) == TRUE ) {
GdkGC *black_gc = gtk_widget_get_style(GTK_WIDGET(vvp))-black_gc;
vik_viewport_draw_line ( vvp, black_gc, xx+tilesize_x_ceil, yy, xx, yy+tilesize_y_ceil );
@@ -1059,7 +1059,7 @@
gboolean need_download = FALSE;
g_snprintf ( mdi-filename_buf, mdi-maxlen, DIRSTRUCTURE,
mdi-cache_dir, vik_map_source_get_uniq_id(MAPS_LAYER_NTH_TYPE(mdi-maptype)),
- mdi-mapcoord.scale, mdi-mapcoord.z, x, y );
+ mdi-mapcoord.z, (17 - mdi-mapcoord.scale), x, y );
donemaps++;
int res = a_background_thread_progress ( threaddata, ((gdouble)donemaps) / mdi-mapstoget ); /* this also calls testcancel */
@@ -1148,7 +1148,7 @@
{
g_snprintf ( mdi-filename_buf, mdi-maxlen, DIRSTRUCTURE,
mdi-cache_dir, vik_map_source_get_uniq_id(MAPS_LAYER_NTH_TYPE(mdi-maptype)),
- mdi-mapcoord.scale, mdi-mapcoord.z, mdi-mapcoord.x, mdi-mapcoord.y );
+ mdi-mapcoord.z, (17 - mdi-mapcoord.scale), mdi-mapcoord.x, mdi-mapcoord.y );
if ( g_file_test ( mdi-filename_buf, G_FILE_TEST_EXISTS ) == TRUE)
{
g_remove ( mdi-filename_buf );
@@ -1204,8 +1204,8 @@
for ( b = mdi-y0; b = mdi-yf; b++ )
{
g_snprintf ( mdi-filename_buf, mdi-maxlen, DIRSTRUCTURE,
- vml-cache_dir, vik_map_source_get_uniq_id(map), ulm.scale,
- ulm.z, a, b );
+ vml-cache_dir, vik_map_source_get_uniq_id(map),
+ ulm.z, (17 - ulm.scale), a, b );
if ( g_file_test ( mdi-filename_buf, G_FILE_TEST_EXISTS ) == FALSE )
mdi-mapstoget++;
}
@@ -1290,8 +1290,8 @@
for (i = mdi-x0; i = mdi-xf; i++) {
for (j = mdi-y0; j = mdi-yf; j++) {
g_snprintf ( mdi-filename_buf, mdi-maxlen, DIRSTRUCTURE,
- vml-cache_dir, vik_map_source_get_uniq_id(map), ulm.scale,
- ulm.z, i, j );
+ vml-cache_dir, vik_map_source_get_uniq_id(map),
+ ulm.z, (17 - ulm.scale), i, j );
Re: [Viking-devel] Viking cache directory
I'm also using JOSM heavily when contributing to OSM, and I'm trying to save on bandwidth by trying to share the imagery cache between them. I've already submitted the changes to JOSM's tracker, so that it simply creates a directory hierarchy in the classical on-disk tile format (Z/X/Y.png). JOSM adds some additional (useless) files into it, but that's about it. I think sharing caches is a good idea. I am guessing viking is a relatively minor player in this area (but perhaps not). So the idea of letting JOSM be the defacto standard seems sensible. This way I can just symlink the directories between the two programs, which is a true boon. One could even have viking look in the josm default area, and save to it if possible, as default behavior. No user-visible changes, except that the old cache won't be used due to the name change. I'm not sure that matters all that much. But we could also make the new behavior be: look in josm cache using new format look in viking cache in old format when saving new files, save in josm cache in new format and that would provide a soft transition without a lot of code. pgp_RAOc29zKc.pgp Description: PGP signature -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech___ Viking-devel mailing list Viking-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/viking-devel Viking home page: http://viking.sf.net/
Re: [Viking-devel] Viking cache directory
On 03/21/2014 04:08 PM, Greg Troxel wrote: I've already submitted the changes to JOSM's tracker, so that it simply creates a directory hierarchy in the classical on-disk tile format (Z/X/Y.png). JOSM adds some additional (useless) files into it, but that's about it. I think sharing caches is a good idea. I am guessing viking is a relatively minor player in this area (but perhaps not). So the idea of letting JOSM be the defacto standard seems sensible. JOSM as of now, stores the data with a different flat hierarchy, and doesn't do pruning. More importantly, the choice of the default cache directory is /tmp/JOSM/, which for most distributions is nowdays tmpfs. My relevant path is here: https://josm.openstreetmap.de/ticket/9813 and just changes the hierarchy to match the on-disk tile format, without changing the default directory. I agree with the developers that it would be nice to collaborate on a common cache directory for tiles. ~/.cache/tiles/[name] would be a good choice. However, how do we choose [name]? We could propose a simple key=value value inside the cache directory (such as ~/.cache/files/[id]/cache.conf that has some known attributes: name=descriptive name url=base url type=TMS/WMS size=size limit for the cache (0=unlimited) and then we could actually run through the first level hierarchy to discover existing caches based on type/url (since any id would be arbitrary). I could definitely help with JOSM (to implement the discovery/format/pruning), but I don't have enough time to also tweak viking. The idea here is up for discussion. One could even have viking look in the josm default area, and save to it if possible, as default behavior. As I said, I wouldn't bother with the current JOSM cache. On linux is basically doesn't exist. On windows, people have been complaning to get slower and slower over time (due to the missing pruning and hierarchy), so it's better that with start with something decent. I would be personally more inclined to provide (and use) a simple migration script for those who are interested to simplify the code. -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech ___ Viking-devel mailing list Viking-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/viking-devel Viking home page: http://viking.sf.net/
