Re: [josm-dev] Image extraction from TMS cache

[Wiktor Niesiobedzki]

Hi Holger,

If you use josm code, there is a directory locking code, that prevents
from accessing cache simultaneously from different processes, as they
do not share memory structures that store current disk file layout and
may lead to corrupted reads/writes. So you should have JOSM shut down
first, before extracting the PNG.

There is no information stored about original file format. But you can
try to extract CacheEntry objects, which getContent() should return
original file content. But you will need to do file format
identification by your self using "magic" (like file), as uses this
magic to identify file format.

Cheers,

Wiktor

2015-12-20 18:33 GMT+01:00 Holger Mappt :
> Hi,
>
> I re-package josm-latest.jar into my cachextractor.jar to have a self
> contained archive. I missed to copy data/preferences.xsd, which killed my
> JOSM preferences. Now I copy directory data in addition to **/*.java and the
> cache read works.
>
> What do I need to do to shut down JOSM if it was started with the functions
> in JOSMFixture.init() (no GUI mode)? Looks like there is a background
> process. I call System.exit(0), but I would like to stop all running
> processes first.
>
> The other thing that caused me some headache was the export disk file
> format. The old cache that wrote tile files to disk wrote *.png files. I
> tried different PNG writers, but my files were always about 8 times bigger.
> It took me some time to figure out that even JPEG files were written with
> the .png file name extension. Now I save the tiles as JPEG and everything is
> fine. Does the current cache have information about the original tile file
> (format, data)?
>
> Regards,
> Holger
>
>
> Am 12.12.2015 um 20:09 schrieb Wiktor Niesiobedzki:
>>
>> As for your problem - this is kind of the problem we face also writing
>> the unit tests. It is solved by calling:
>> org.openstreetmap.josm.JOSMFixture.createUnitTestFixture().init();
>>
>> Though keep in mind, that this is part of the testing code, so it's
>> located here:
>>
>> http://josm.openstreetmap.de/browser/josm/trunk/test/unit/org/openstreetmap/josm/JOSMFixture.java
>>
>> This should give you a hint, how to initiate "headless" mode.
>>
>> Cheers,
>> Wiktor
>>
>> 2015-12-12 18:52 GMT+01:00 Holger Mappt:
>>>
>>> My idea was to write a small program that calls TMSLayer.getCache() to
>>> get
>>> the TMS cache. But to do that I need to initialize JOSM (preferences, TMS
>>> layer, and tile cache). How would I do that? Is there something like a
>>> headless mode where I can hook into? I tried to call some of the
>>> initialization methods, but there are too many dependencies and I would
>>> need
>>> a running JOSM to make that work.

___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev

Re: [josm-dev] Image extraction from TMS cache

[Holger Mappt]

Hi,

I re-package josm-latest.jar into my cachextractor.jar to have a self 
contained archive. I missed to copy data/preferences.xsd, which killed 
my JOSM preferences. Now I copy directory data in addition to **/*.java 
and the cache read works.


What do I need to do to shut down JOSM if it was started with the 
functions in JOSMFixture.init() (no GUI mode)? Looks like there is a 
background process. I call System.exit(0), but I would like to stop all 
running processes first.


The other thing that caused me some headache was the export disk file 
format. The old cache that wrote tile files to disk wrote *.png files. I 
tried different PNG writers, but my files were always about 8 times 
bigger. It took me some time to figure out that even JPEG files were 
written with the .png file name extension. Now I save the tiles as JPEG 
and everything is fine. Does the current cache have information about 
the original tile file (format, data)?


Regards,
Holger


Am 12.12.2015 um 20:09 schrieb Wiktor Niesiobedzki:

As for your problem - this is kind of the problem we face also writing
the unit tests. It is solved by calling:
org.openstreetmap.josm.JOSMFixture.createUnitTestFixture().init();

Though keep in mind, that this is part of the testing code, so it's
located here:
http://josm.openstreetmap.de/browser/josm/trunk/test/unit/org/openstreetmap/josm/JOSMFixture.java

This should give you a hint, how to initiate "headless" mode.

Cheers,
Wiktor

2015-12-12 18:52 GMT+01:00 Holger Mappt:

My idea was to write a small program that calls TMSLayer.getCache() to get
the TMS cache. But to do that I need to initialize JOSM (preferences, TMS
layer, and tile cache). How would I do that? Is there something like a
headless mode where I can hook into? I tried to call some of the
initialization methods, but there are too many dependencies and I would need
a running JOSM to make that work.


___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev

Re: [josm-dev] Image extraction from TMS cache

[Holger Mappt]

My idea was to write a small program that calls TMSLayer.getCache() to 
get the TMS cache. But to do that I need to initialize JOSM 
(preferences, TMS layer, and tile cache). How would I do that? Is there 
something like a headless mode where I can hook into? I tried to call 
some of the initialization methods, but there are too many dependencies 
and I would need a running JOSM to make that work.


Does it work to write a plugin that can be called from the command line, 
but without the GUI?


The other option is to re-implement the tile cache code without some of 
the dependencies. That seems to be a lot of work. It needs modifications 
every time the JOSM tile cache is modified.


BTW, was the tile cache code modified recently? My cache was reset 
(erased) yesterday.


Holger

Am 01.06.2015 um 22:20 schrieb Wiktor Niesiobedzki:

So what you could do, is to create small java program, that would
extract all the files from JOSM cache, based on josm.jar. And further
functionality just depends on what you want to do with cache content.

2015-06-01 20:45 GMT+02:00 Holger Mappt:

Is this something I can do on the command line or do I need to write a Java
program?  What would be a good starting point?


Am 28.05.2015 um 22:14 schrieb Wiktor Niesiobedzki:

This is +/- "propietary" standard of Apache Java Caching System
(https://commons.apache.org/proper/commons-jcs/). We are currently
using Indexed Disk Cache as store for tiles, so the easiest way
forward, is just to use JCS, to read the file, and then use keySet()
method, to get all the data that is stored in cache.


2015-05-28 20:30 GMT+02:00 Holger Mappt:

I assume the new TMS cache with the files TMS.data and TMS.key uses some
standard.  What tools can I use to extract the images from that cache? I
do
some post-processing for offline mapping with Mapserver.



___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev

Re: [josm-dev] Image extraction from TMS cache

[Wiktor Niesiobedzki]

Hi,

Your cache was reset probably due to this change:
http://josm.openstreetmap.de/changeset/9064/josm

We moved from IndexedDiskCache do BlockDiskCache.

As for your problem - this is kind of the problem we face also writing
the unit tests. It is solved by calling:
org.openstreetmap.josm.JOSMFixture.createUnitTestFixture().init();

Though keep in mind, that this is part of the testing code, so it's
located here:
http://josm.openstreetmap.de/browser/josm/trunk/test/unit/org/openstreetmap/josm/JOSMFixture.java

This should give you a hint, how to initiate "headless" mode.

Cheers,

Wiktor

2015-12-12 18:52 GMT+01:00 Holger Mappt :
> My idea was to write a small program that calls TMSLayer.getCache() to get
> the TMS cache. But to do that I need to initialize JOSM (preferences, TMS
> layer, and tile cache). How would I do that? Is there something like a
> headless mode where I can hook into? I tried to call some of the
> initialization methods, but there are too many dependencies and I would need
> a running JOSM to make that work.
>
> Does it work to write a plugin that can be called from the command line, but
> without the GUI?
>
> The other option is to re-implement the tile cache code without some of the
> dependencies. That seems to be a lot of work. It needs modifications every
> time the JOSM tile cache is modified.
>
> BTW, was the tile cache code modified recently? My cache was reset (erased)
> yesterday.
>
> Holger
>

___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev

Re: [josm-dev] Image extraction from TMS cache

[Holger Mappt]

Hi,

Is this something I can do on the command line or do I need to write a 
Java program?  What would be a good starting point?  I was assuming that 
there is some kind of cache explorer, but I couldn't find one.


Thanks,
Holger


Am 28.05.2015 um 22:14 schrieb Wiktor Niesiobedzki:

Hi,

This is +/- propietary standard of Apache Java Caching System
(https://commons.apache.org/proper/commons-jcs/). We are currently
using Indexed Disk Cache as store for tiles, so the easiest way
forward, is just to use JCS, to read the file, and then use keySet()
method, to get all the data that is stored in cache.

Cheers,

Wiktor

2015-05-28 20:30 GMT+02:00 Holger Mappt holger...@gmx.net:

Hi,

I assume the new TMS cache with the files TMS.data and TMS.key uses some
standard.  What tools can I use to extract the images from that cache? I do
some post-processing for offline mapping with Mapserver.

Thanks,
Holger


___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev

Re: [josm-dev] Image extraction from TMS cache

[Wiktor Niesiobedzki]

Hi,

There is no general general cache explorer for JCS, as it is
implementation dependant, what it stores in the cache. Generally -
what's in there, it's just plain Java objects.

So what you could do, is to create small java program, that would
extract all the files from JOSM cache, based on josm.jar. And further
functionality just depends on what you want to do with cache content.

Cheers,

Wiktor

2015-06-01 20:45 GMT+02:00 Holger Mappt holger...@gmx.net:
 Hi,

 Is this something I can do on the command line or do I need to write a Java
 program?  What would be a good starting point?  I was assuming that there is
 some kind of cache explorer, but I couldn't find one.

 Thanks,
 Holger



 Am 28.05.2015 um 22:14 schrieb Wiktor Niesiobedzki:

 Hi,

 This is +/- propietary standard of Apache Java Caching System
 (https://commons.apache.org/proper/commons-jcs/). We are currently
 using Indexed Disk Cache as store for tiles, so the easiest way
 forward, is just to use JCS, to read the file, and then use keySet()
 method, to get all the data that is stored in cache.

 Cheers,

 Wiktor

 2015-05-28 20:30 GMT+02:00 Holger Mappt holger...@gmx.net:

 Hi,

 I assume the new TMS cache with the files TMS.data and TMS.key uses some
 standard.  What tools can I use to extract the images from that cache? I
 do
 some post-processing for offline mapping with Mapserver.

 Thanks,
 Holger

___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev

Re: [josm-dev] Image extraction from TMS cache

[Wiktor Niesiobedzki]

Hi,

This is +/- propietary standard of Apache Java Caching System
(https://commons.apache.org/proper/commons-jcs/). We are currently
using Indexed Disk Cache as store for tiles, so the easiest way
forward, is just to use JCS, to read the file, and then use keySet()
method, to get all the data that is stored in cache.

Cheers,

Wiktor

2015-05-28 20:30 GMT+02:00 Holger Mappt holger...@gmx.net:
 Hi,

 I assume the new TMS cache with the files TMS.data and TMS.key uses some
 standard.  What tools can I use to extract the images from that cache? I do
 some post-processing for offline mapping with Mapserver.

 Thanks,
 Holger

 ___
 josm-dev mailing list
 josm-dev@openstreetmap.org
 https://lists.openstreetmap.org/listinfo/josm-dev

___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev

[josm-dev] Image extraction from TMS cache

[Holger Mappt]

Hi,

I assume the new TMS cache with the files TMS.data and TMS.key uses some 
standard.  What tools can I use to extract the images from that cache? 
I do some post-processing for offline mapping with Mapserver.


Thanks,
Holger

___
josm-dev mailing list
josm-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/josm-dev