[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764433#comment-17764433 ] Gary Lucas commented on IMAGING-316: Submitted Pull Request 318 to the Commons Imaging project on github. This enhancement will allow the Imaging library to read BigTIFF format files. The ability to write BigTIFF files will be subject of a future Jira issue. > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17763673#comment-17763673 ] Gary D. Gregory commented on IMAGING-316: - (y) > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17763473#comment-17763473 ] Gary Lucas commented on IMAGING-316: Have an updated version of the Commons Imaging library that successfully reads all of the sample images at the AWare Systems website. Am working on the JUnits tests. I expect to submit a pull request in the next few days. > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762356#comment-17762356 ] Gary Lucas commented on IMAGING-316: I requested and received permission from Joris Van Damme (of AWare Systems) to include the sample BigTIFF files I mentioned in the Commons Imaging distribution under the test resources files. These samples are a set of 8 BigTIFF files and one conventional TIFF file provided for reference purposes. The files are relatively small, making the whole collection come in at 158 KB (the "big" refers to the 64-bit address space, not the actual size of the files). This collection exercises all of the core functions that are affected by the 64-bit address space and will provide a solid basis for JUnit tests. I will, of course, include a proper citation and grateful acknowledgement of the data source in a README.TXT file. To learn more about BigTIFF, visit Mr. Van Damme's page at [https://www.awaresystems.be/imaging/tiff/bigtiff.html] > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762228#comment-17762228 ] Gary Lucas commented on IMAGING-316: I've located a great page with resources for the BigTIFF file format. AWare Systems has posted a page on the [The BigTIFF File format|https://www.awaresystems.be/imaging/tiff/bigtiff.html] that describes the enhancements required to support BigTIFF and provides a comprehensive set of test tiles to exercise any API that reads BigTIFF. These resources will be a great boost in developing this feature for Commons Imaging. > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437523#comment-17437523 ] Michael Osipov commented on IMAGING-316: Thank you, it was quite enlightening! > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437495#comment-17437495 ] Gary Lucas commented on IMAGING-316: Short answer, the BigTIFF specification simply calls for changes in the way file-positions are specified in a file. It doesn't change the internal representation of content. A TIFF file can contain multiple bundled images, and their individual formats would not change. However, to provide a bit of background... TIFF actually is an image file specification. And it also supports numeric grids (i.e. Earth surface elevation data sets). TIFF can support data using various data compression methods, of which JPEG is just one. In the case of TIFF, the file does not so much contain a JPEG image as it contains an image and uses JPEG-based data compression to store it. And, again, all images in a TIFF file (whether one or many) are all in a TIFF-specified data format. As an aside, right now there is a Jira item in place to upgrade Commons Imaging to handle TIFF files that contain JPEG-style images. I wish it was as simple as pumping the content through a conventional JPEG API. Although the TIFF format does call for JPEG methods to be used on TIFF files, the internal representation is different enough from the JPEG standard to mess things up. I have only just started looking at that one and have no idea what the level of effort is going to be. But, in answer to your question, I believe that particular issue would be independent of BigTIFF. > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IMAGING-316) Support the BigTIFF file format
[ https://issues.apache.org/jira/browse/IMAGING-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437444#comment-17437444 ] Michael Osipov commented on IMAGING-316: Forgive me my naive question: I know that TIFF is container, rather than a format/encoding itself. What does of encoded data is in that BigTIFF file in your case and would that work with plain JPEG, for example? > Support the BigTIFF file format > --- > > Key: IMAGING-316 > URL: https://issues.apache.org/jira/browse/IMAGING-316 > Project: Commons Imaging > Issue Type: New Feature >Affects Versions: 1.x >Reporter: Gary Lucas >Priority: Major > > Traditional TIFF files address file position in bytes using 32-bit integers. > This approach automatically limits the maximum size of a TIFF file to 4 GB. > The BigTIFF specification (formalized in 2011) uses 64-bit integers to > address file positions, and thus supports much larger files. I propose that > a future release of Commons Imaging would benefit from supporting BigTIFF. > The level of effort for this implementation may be large. > In terms of creating JUnit tests to support this effort, note that just > because a file uses the BigTIFF specification doesn't mean that the file has > to be super large. It should be possible to create BigTIFF test files that > are only a few kilobytes. Thus supporting BigTIFF does not necessarily mean > that massive files will need to be included in the Commons Imaging > distribution. > Finally, it is reasonable to ask if anyone would actually need images that > were so large that they couldn't fit within 4 GB. The short answer is that > some folks in the Geographic Information Systems (GIS) community do work with > images (or data sets) that large and, also, that some systems produce images > in BigTIFF format even when ordinary TIFF would suffice. > > P.S. It might be work investigating whether the existing Imaging library > actually supports the full 32-bit address space of a conventional TIFF. > Regrettably, Java doesn't support unsigned integer types. And it is > possible that a file address with the high bit set might be incorrectly > interpreted as a negative number. So I will be taking a look at the code to > make sure all file addresses are properly masked when they are handed over to > Java. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
