Re: [mkgmap-dev] Embedding raster maps
Hi Steve, > mkgmap sets all of these to zero and seems to work. Of course it may > not work for your purpose with the images. I would try taking a > copy of a working map and zeroing out some of the unknown things > until it stops working. You will then know if something is essential > or not. Done that. Removing the "flags" seem to have no impact. Removing the size of this 5th RGN section does. Therefor it is important :) > Presumably it points into the extended polygon section, but I do not > have anything that displays that section at the moment so I can't > yet match anything up. Ok, I will think about it. Just from my understanding also the information how these raster tiles fit into the draw order is missing. Maybe it's hard coded or set by that section. I also noted that QMapShack is probably decoding the extended polylines completely wrong. (Just wonder why no one complained so far ^^ ). I saw in the code that there is some test code. Is there already some way to create a test img for extended polylines and polgons that I can use as a reference to fix my code? This would be of great help. Cheers Oliver ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Embedding raster maps
Hi Steve, thanks for the file. On my device nothing is visible at all. But that's kind of an information, too. There is no way to circle down bugs by simply skipping the extended part. Probably it is still some missing encoding. In the Isle of Man map (and others) there is much more information in the RGN header. Note the values in the "filling bytes". And there seems to be an additional section after the extended point2 section. I called it offset1/length2. --- RGN header --- size 007D type 47 41 52 4D 49 4E 20 52 47 4E | GARMIN RGN byte0x000C 01 flag 00 year 2010 month 7 day 21 hour 9 minute 53 second 6 offset1 D54E length1 offset_polyg2 D54E length_polyg2 1A1F byte0x0025_0x0038 02 00 00 00 00 00 00 00 FF 00 00 20 FD FC 03 00 00 00 00 00 offset_polyl2 EF6D length_polyl2 byte0x0041_0x0054 00 00 00 00 00 00 00 00 3F 00 00 20 FD 0F 00 00 00 00 00 00 offset_point2 EF6D length_point2 byte0x005D_0x0070 00 00 00 00 00 00 00 00 FF 07 00 20 3F F7 3F 00 00 00 00 00 offset2 EF6D length2 020B unknown 00E3 This section has a variable length in different sub-files. The unknown value E3 can be E5, too. It doesn't seem to be a record size. I had a look at several sections. They are not completely random, thus there seem to be no "compression". However the patterns do not ring a bell for me. I copied a couple of them into a file and loaded it into my cloud share: https://www.magentacloud.de/share/s429h1z-r6 Maybe one of you has an idea what it's about. I also uploaded my current version of the test map with a single raster map tile. It does not show. Top left coordinate is N46° 57.768 E011° 47.484. Sadly I am stuck here and I am running out of quality time to concentrate on the topic. I also have to update QMapShack's decoding. While reading the mkgmap source code I noticed a lot of new stuff that is not covered by the decoder. Cheers Oliver ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Embedding raster maps
Hi Steve, that was really helpful. I managed to get my map listed. However the raster layer did not show up :( Well I didn't really expect it. ;) > > I believe that the byte at 0x40 is the start block of the FAT so in > test.img it should be 2. Patching that byte did not make it load > though. Indeed, it seems to be an offset in number of 0x200 blocks. In all my example maps it's set to 8 and FAT starts at 0x1000. In your map it's set to 2 and FAT starts at 0x400. Another mystery solved. I guess the next step is to get a rectangular extended polygon without the raster data to show up. The polygon type for the raster map tiles used is 0x10613. Usually it only contains a single point as the extended data referencing the jpeg tile has the four boundaries coded as 32 bit integers. My goal is to remove the extended data and to code all 4 corner coordinates. This should give me a matrix of all rectangles. Again this raises the question about the minimum effort necessary to add a single polygon. In my naive point of view it would be just encoding it into the extended polygon section of the RGN file. Of course I tried it and it does not work. So I am missing something, or have a couple of more bugs in my code. Would it be possible to code another file with a single polygon QPolygonF( QPointF(11.7914,46.9628) QPointF(11.8021,46.9628) QPointF(11.8021,46.9555) QPointF(11.7914,46.9555) ) coded in three map levels: 0/24bit, 1/23bit and 2/22bit. This would give me a nice working reference to compare. Thanks & Cheers Oliver ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Embedding raster maps
Hi Steve, I tested your map. It's not listed either. I have a GPSMap64s and placed it into the internal as well as into the external memory. I removed all other maps on the device, except the basemap. The basemap is the only one listed. I made a minimum test.img, too. It contains no points, lines or areas. It has 3 zoom levels 22..24 with a sub-division each. I also uploaded a log from my test decoder. https://www.magentacloud.de/share/s429h1z-r6 I uploaded a second one with the version in the header set to 2. So far the difference to be another offset for the FAT. Maybe you can cross check if your device is willing to list them. Cheers, Oliver > Gesendet: Sonntag, 03. Januar 2021 um 23:08 Uhr > Von: "Steve Ratcliffe" > An: mkgmap-dev@lists.mkgmap.org.uk > Betreff: Re: [mkgmap-dev] Embedding raster maps > > > Hi Oliver > > > What is the minimum data an IMG file has to contain to be listed > > on the device? > > Its an interesting question. I don't remember having too many problems > getting a map recognised in the beginning. At r20 it worked > according to the commit comment but I only added the test map > builder at r24. > > However on my etrex 30 the map produced by r24 does not work. > > The first one that does work is r79. With a bit of experimenting > I discovered that adding a sufficiently large gap between the > end of the TRE header and the first data would make r78 work too. > > Going back to r24 and adding more padding also produced a map that > was recognized. I was able to remove the second copyright message > too and all map data. > > The required gap was between 40 and 101, depending on the rest of > the code. > > So this all makes very little sense and it is possible that this is > covering for something else that we don't understand. > > I've uploaded the minimal IMG file at: >http://files.mkgmap.org.uk/download/494/gmapsupp.img > > (After this I also managed to remove the first copyright message, so > you do not need any copyright messages and the label section can be > empty) > > With this version there is no message displayed on startup, which is > something that was fixed in r79 I think. > > There is just a single zoom and subdivision in the TRE section and > no points, line or polygons. > > Cheers, > Steve > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Embedding raster maps
Hi Andrzej yes this is the map that was used to get first information about the format. But it's still quite big and has a lot of stuff that's probably not related to the raster map layer. My question targets more to what is the minimum required to get a map listed on the device. Without any raster map etc. Is it ok to have sub files with no elements at all? Are there some elements required like the map boundary polygon? Is there the need of some other mandatory stuff that is not so obvious? Before I can concentrate on my raster map encoding I need something that works without. Cheers Oliver > Gesendet: Samstag, 02. Januar 2021 um 17:12 Uhr > Von: "Andrzej Popowski" > An: mkgmap-dev@lists.mkgmap.org.uk > Betreff: Re: [mkgmap-dev] Embedding raster maps > > Hi Oliver, > > that's an interesting subject, you are working on. You probably know > this free map, I think this is an example of required minimum: > http://static.garmin.com/shared/aus/HTML/_pages/isle-of-man.html > > -- > Best regards, > Andrzej > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] Embedding raster maps
Dear mailing list, let me introduce myself. Some might recognize my name. I am the main author of QLandkarte/QMapShack. I have quite some experience in decoding the Garmin formats. But little knowledge in encoding the format. My main focus is on raster maps rather than vector maps as they provide more details in alpine areas. Years ago I was in contact with Alex Whiter to decode and encode the Garmin Bird's Eye format that uses some kind of DRM. To make it work the DRM check on the devices had to be disabled. Alex provided a firmware patcher. That solution worked pretty well over the years. A couple of years ago we had another contact. This time it was decoding the raster maps embedded into the IMG format. At that time decoding was the goal. We never considered encoding the stuff as there is the JNX format as a much simpler solution. However life is not a constant and Alex died a couple of moons ago. See for details: https://www.gpspower.net/forum-announcements/359499-vale-alexwhiter.html And with him the knowledge about the firmware patcher. Right now it still works. But it's just a question of time that this will fail. Therefore I invested quite some days of spare time to consolidate my old decode code and to start encoding. Alex wrote some documentation in Russian and Wolfgang taking care of the QMapShack Wiki translated it into English: https://github.com/Maproom/qmapshack/wiki/RasterImg_AWhiter My current progress can be found here: https://github.com/kiozen/GarminImg Let me explain why I started to write my own encoder rather than diving into the code of mkgmap * I never wrote Java. And starting to learn Java with mkgmap is a bit frightening ;) Sorry :D * My target users are everyday Garmin users that do have raster maps but very little experience with map creation. Therefore I need a simple solution that can be shipped with QMapShack/QMapTools * All IMG files embedding raster maps that are available to me make use of the newer encoding via GMP part rather than the older format. I do not know if the last point is necessary. From my understanding it shouldn't matter. But as I am not sure I started with new newer format. The complete project reached the point, where I can create IMG files with a raster map embedded. And I can decode that map again. However - you surely guessed it - the device refuses to list the map. This is kind of expected. There are many details to get right and a single one can make it fail. Therefore I decided to reach out for help and inspiration contacting this list. My first question would be: What is the minimum data an IMG file has to contain to be listed on the device? Right now I try to create a file with more or less empty GMP section. From the comments in mkgmap I already learned that it needs at least two copyright strings. But does a map require to have points/lines and areas to be listed on the device? In other words: How can I make sure my header, FAT table and MPS section is ok? To have a map with no data listed on the device would be a huge step forward. Cheers Oliver ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
