Wow. Whoddu thunk we'd spend so much time and energy trying to make big pictures smaller.
Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -----Original Message----- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Friday, August 21, 2009 9:55 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File FormatsandProprietaryAlgorithms[SEC=UNCLASSIFIED] Not a stupid question -- but it doesn't work that way. The artifacts are due to the wavelet processing of the pixels near the tile boundaries, and the boundaries have to be treated reflectively within their individual tiles (in order to keep each tile separate), which means you can sometimes "see" where those boundaries are. "Overlapping" doesn't help because the wavelet footprint spans a large width, in order to handle the lower-resolution scales. Which in turn means you need to be able "reach" far away parts of the image at various (some might say "arbitrary") stages in the wavelet pipeline. Just trust me, it is a nontrivial problem to solve. Brighter minds than ours have spent a lot of energy on this problem -- a literature search would reveal a number of PhD theses and patents. -mpg -----Original Message----- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Landon Blake Sent: Friday, August 21, 2009 10:45 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats andProprietaryAlgorithms[SEC=UNCLASSIFIED] MPG wrote: "Tiling" essentially means you can take a large file and compress pieces of it independently. This avoids having to deal with the large memory footprint issues, but it can also lead to seam-line artifacts under certain conditions. Ideally, one would prefer to have the option of compressing large images without resorting to using tiles." This is probably a stupid question, since I know absolutely nothing about image compression, but couldn't you overlap the tiles slightly to avoid the seam lines? This would obviously result in a slightly larger file size because some pixels would be compressed twice. But that might be OK if you were trying to compress a huge image. What about reading chunks of the image off disk, instead of trying to put the whole image in memory? This would be slower, but might make an impossible task possible. We run into this problem with vector datasets to. Some datasets are just to stinking BIG. One of my tasks for OpenJUMP is to write a core module that displays vector data accessed directly from disk, instead of from memory. This will be slower, but it is better than crashing the program because there isn't enough RAM. Things must be more complicated than can be described in an e-mail, because we've got people a lot smarter than me working on these problems. I am just curious. (I tried reading about wavelet compression on Wikipedia yesterday and quickly got a headache.) :] Landon Office Phone Number: (209) 946-0268 Cell Phone Number: (209) 992-0658 -----Original Message----- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael P. Gerlek Sent: Friday, August 21, 2009 9:36 AM To: OSGeo Discussions Subject: RE: [OSGeo-Discuss] Open File Formats andProprietaryAlgorithms[SEC=UNCLASSIFIED] "Tiling" essentially means you can take a large file and compress pieces of it independently. This avoids having to deal with the large memory footprint issues, but it can also lead to seam-line artifacts under certain conditions. Ideally, one would prefer to have the option of compressing large images without resorting to using tiles. Note too that, in addition to the large image issue, many of the JP2 implementations out there are either not fully compliant or are not tuned for performance. A viable solution would need both of these. -mpg -----Original Message----- From: discuss-boun...@lists.osgeo.org [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Chris Puttick Sent: Friday, August 21, 2009 9:37 AM To: OSGeo Discussions Subject: Re: [OSGeo-Discuss] Open File Formats and ProprietaryAlgorithms[SEC=UNCLASSIFIED] Well, according to this page: http://jpeg2000.epfl.ch/ v.5.1, courtesy in part Eastman Kodak, provides "complete JP2 support at the decoding side" - not sure whether that covers the tiling or other geo needs, but doesn't it sound worth investigating? Chris ----- "Christopher Schmidt" <crschm...@crschmidt.net> wrote: > On Fri, Aug 21, 2009 at 08:27:13AM -0700, Landon Blake wrote: > > Thanks for the information Michael. I am downloading Opticks right > now. > > :] > > > > I also found this Java library for JP2, thought I'm not sure how > > complete/up-to-date it is: > > > > http://jj2000.epfl.ch/ > > > > Maybe we need a JPEG 2000 page on the OSGeo wiki. > > Note that "JPEG 2000 support" is different from "JPEG 2000 support > which > works on geo-sized images." The tiling (or 'paging'? as Michael calls > it) support that's supposed to be provided by OpenJPEG2000 has been > coming 'real soon now' for about 18 months now from my uneducated > observations, and until it's there, most tools using OpenJPEG for JP2s > are > going to suffering under much the same limitations. > > -- Chris > > > Landon > > Office Phone Number: (209) 946-0268 > > Cell Phone Number: (209) 992-0658 > > > > > > > > -----Original Message----- > > From: discuss-boun...@lists.osgeo.org > > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Considine, > Michael > > Sent: Friday, August 21, 2009 8:09 AM > > To: OSGeo Discussions > > Subject: RE: [OSGeo-Discuss] Open File Formats and > > ProprietaryAlgorithms[SEC=UNCLASSIFIED] > > > > All, > > > > Opticks is an open source remote sensing application and > development > > framework. We recently started the process to add JPEG 2000 support > to > > our framework. We picked OpenJpeg to add JPEG 2000 support to our > > application. They are also open source. We currently support > importing > > JPEG 2000 files but we are currently limited to the 4GB memory size > > after decoding. > > > > Our plan is to continue development and to upgrade to OpenJpeg 2.0 > once > > they have a stable release. That will allow Opticks to use a pager > to > > display and support much larger files. > > > > Michael Considine > > > > -----Original Message----- > > From: discuss-boun...@lists.osgeo.org > > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Bruce > Bannerman > > Sent: Thursday, August 20, 2009 8:15 PM > > To: 'OSGeo Discussions' > > Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary > > Algorithms[SEC=UNCLASSIFIED] > > > > > > IMO: > > > > > > Just another thought on this issue (though we do seem to be > recycling > > arguments over the years...): > > > > > > Assuming that I have a very large archive of spatial data, be it > imagery > > or any other spatial format and that I store my data in a variety > of > > proprietary formats: > > > > > > In ten years from now, can I be sure that: > > > > - the company that created, understands, and holds the IP in the > > data format will still be around? > > > > - there will still be software that runs on the then current > > operating environment, that can read and 'fully exploit' the data > > in the proprietary standard? > > > > - that this future software will work seamlessly with my then > current > > spatial environment? > > > > - if all of the above risks prove to eventuate, can I be sure that > I'll > > be able to salvage my data into another format, retaining its > complete > > > > semantic context? > > > > > > IMO, it is a high risk proposition to lock public (or private) > archives > > away in proprietary data formats. It makes more sense to use open > > standards and formats that are publically available. > > > > > > > > Bruce Bannerman > > > > > > > > > > > > > -----Original Message----- > > > From: discuss-boun...@lists.osgeo.org > > > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Michael > > > P. Gerlek > > > Sent: Friday, 21 August 2009 6:55 AM > > > To: OSGeo Discussions > > > Subject: RE: [OSGeo-Discuss] Open File Formats and > > > Proprietary Algorithms > > > > > > Some clarifications: > > > > > > > > > > > > - MrSID has both lossy and lossless modes > > > > > > - MrSID is not fractal based; it uses wavelets (and > > > arithmetic encoding) > > > > > > - you can't copyright algorithms; the MrSID source code > > > certainly is, however > > > > > > - MrSID relies on a number of patents, not all of which are > > > owned by LizardTech > > > > > > - reading MrSID does not require any fees; we have libraries > > > you can download, although they are not open source > > > > > > > > > > > > That said, some editorial comments (although I'm now wishing > > > I hadn't been so quick to rise to Landon's bait :-) > > > > > > > > > > > > - Some of you know the history of trying to open source > > > MrSID; I won't go into that here, except to say that > > > LizardTech doesn't own all of the required IP needed to make > > > that happen. > > > > > > - If we are speaking of the NAIP data, then no, it is not > > > exclusively available in MrSID format; it is also shipped as > GeoTIFFs. > > > > > > - JPEG 2000 is a very robust open standard alternative to > > > MrSID, and a number of players already support it (including > > > LizardTech), but not enough to make it viable for certain > > > domains like NAIP. > > > > > > - some of you also know the history on open JP2 support: > > > there is today no open source implementation of JP2 that is > > > suitable for geo work. Alas. > > > > > > > > > > > > -mpg > > > > > > > > > > > > > > > > > > From: discuss-boun...@lists.osgeo.org > > > [mailto:discuss-boun...@lists.osgeo.org] On Behalf Of Eric Wolf > > > Sent: Thursday, August 20, 2009 2:15 PM > > > To: OSGeo Discussions > > > Subject: Re: [OSGeo-Discuss] Open File Formats and > > > Proprietary Algorithms > > > > > > > > > > > > The MRSID format is a very special case - and perhaps an > > > opportunity for a new FOSS file format. MRSID is a lossless, > > > fractal-based, multi-scale raster compression format. > > > LizardTech has the algorithms to encode and decode MRSID > > > locked up in copyrights, and I believe, patents. Even > > > companies like ESRI shell out big bucks to LizardTech to be > > > able to read and write the MRSID format. > > > > > > > > > > > > I guess I missed the context of the discussion. Is the > > > government releasing certain data exclusively in this format? > > > If so, I think the argument can be made against this > > > practice. The different in compression between MRSID and > > > gziped TIFFs isn't really that great in this day of cheap > > > disks and fat pipes. > > > > > > > > > > > > -Eric > > > > > > > > > -=--=---=----=----=---=--=-=--=---=----=---=--=-=- > > > Eric B. Wolf New! 720-334-7734 > > > USGS Geographer > > > Center of Excellence in GIScience > > > PhD Student > > > CU-Boulder - Geography > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > Discuss mailing list > > Discuss@lists.osgeo.org > > http://lists.osgeo.org/mailman/listinfo/discuss > > > > > > > > This message and any enclosures are intended only for the > addressee. > > Please > > notify the sender by email if you are not the intended recipient. > If > > you are > > not the intended recipient, you may not use, copy, disclose, or > > distribute this > > message or its contents or enclosures to any other person and any > such > > actions > > may be unlawful. Ball reserves the right to monitor and review all > > messages > > and enclosures sent to or from this email address. > > _______________________________________________ > > Discuss mailing list > > Discuss@lists.osgeo.org > > http://lists.osgeo.org/mailman/listinfo/discuss > > > > > > Warning: > > Information provided via electronic media is not guaranteed against > defects including translation and transmission errors. If the reader > is not the intended recipient, you are hereby notified that any > dissemination, distribution or copying of this communication is > strictly prohibited. If you have received this information in error, > please notify the sender immediately. > > _______________________________________________ > > Discuss mailing list > > Discuss@lists.osgeo.org > > http://lists.osgeo.org/mailman/listinfo/discuss > > -- > Christopher Schmidt > Web Developer > _______________________________________________ > Discuss mailing list > Discuss@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/discuss ------ Files attached to this email may be in ISO 26300 format (OASIS Open Document Format). If you have difficulty opening them, please visit http://iso26300.info for more information. _______________________________________________ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss Warning: Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately. _______________________________________________ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss _______________________________________________ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss _______________________________________________ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
