Re: [gdal-dev] Histograms without null pixels
Caleb, On Tue, Mar 5, 2013 at 11:53 AM, Caleb Hanger cdhan...@gmail.com wrote: On Tue, Mar 05, 2013 at 11:19:27AM +0530, Chaitanya kumar CH wrote: Caleb, gdalinfo uses the GetDefaultHistogram() method which can do this faster by fetching a precomputed histogram. What do you mean? The histogram is only precomputed if gdalinfo has been run previously and stored a cache of data (the XML file) for later reference; please correct me if I'm wrong. Otherwise, it seems to me that the data *must* be analyzed and the data computed; there is no way around that. Additionally, I don't think it makes sense to say that it is faster to assemble a histogram that includes out-of-range values than to assemble a histogram that does not, because the latter is a subset contained within the former. Some raster formats can store the histogram data as metadata. Also, there are is a shortcut; overviews can be used to get approximate values faster. gdalinfo is a command line program. Even a default histogram of 256 buckets is hard to visualize in text mode. A much better way is to use a desktop GIS like qgis to view the histogram. A desktop GIS is not practical for providing a shell script meant to automate a repetitive (and much larger) task, with a small chunk of data that it needs for a single step in the process. That is, I don't need to visualize the data -- I just need to get the information. The more primitive the format, the better, as I like things that are lean and mean, and I know how to instruct the script to do the required mathematics upon the text-only histogram. You can write a simple python script to get the histogram. You can get a good idea at http://trac.osgeo.org/gdal/browser/trunk/autotest/gcore/histogram.py#L103 Can you justify adding this extra functionality in gdalinfo? Yes: quite simply, a histogram that does not include illegitimate values makes more sense than a histogram that does include them. At least, this is the case in my experience; perhaps there are situations I'm unfamiliar with in which the histogram is desired to include values outside of the histogram's range, for some reason. I'd even go so far as to say that gdalinfo currently *lies* about the histogram, telling the user that the histogram shows 256 buckets from X to Y even though the histogram includes values which are *outside of that range*. Arguably an even better justification is that the machinery to exclude the illegitimate values is already present in GetHistogram, so *very* few lines of code would need to be added to gdalinfo, simply to provide a flag for the user and if that flag is specified, pass false for the right parameter to GetHistogram (bIncludeOutOfRange), instead of just calling GetDefaultHistogram. That sounds reasonable. If you can, you should submit a patch or a request at http://trac.osgeo.org/gdal/newticket Note that the functionality should include the facility to include options to mention the min/max values. On Tue, Mar 5, 2013 at 10:56 AM, Caleb Hanger cdhan...@gmail.com wrote: All, I am writing a script which is calling gdalinfo to get a histogram of pixel values in an image file. Apparently the mechanism for obtaining a default histogram enables the bIncludeOutOfRange flag to dump not-in-range pixels (including null pixels, in formats that support it) into whichever bin of the histogram is closest. I see that in the GetHistogram() function, the functionality is already coded to respond appropriately to either a true *or* false value in this flag. However, I can't seem to find any way to turn that flag off through any of the user-level programs (in particular, gdalinfo), rendering the parameter to the function useless, unless I'm just not looking hard enough. Hence, my questions: 1) Is there any standard user-level operation (preferably in gdalinfo) which disables, or at least *allows* for disabling of, the bIncludeOutOfRange flag? If so, I can't find it. 2) If not, how come? Has there been no apparent need for histograms that omit null values or other out of range situations? Thanks in advance for any insight, Caleb Hanger ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Best regards, Chaitanya kumar CH. +91-9494447584 17.2416N 80.1426E ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Best regards, Chaitanya kumar CH. +91-9494447584 17.2416N 80.1426E ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Histograms without null pixels
On Tue, Mar 05, 2013 at 01:31:01PM +0530, Chaitanya kumar CH wrote: On Tue, Mar 5, 2013 at 11:53 AM, Caleb Hanger cdhan...@gmail.com wrote: What do you mean? The histogram is only precomputed if gdalinfo has been run previously and stored a cache of data (the XML file) for later reference; please correct me if I'm wrong. Otherwise, it seems to me that the data *must* be analyzed and the data computed; there is no way around that. Additionally, I don't think it makes sense to say that it is faster to assemble a histogram that includes out-of-range values than to assemble a histogram that does not, because the latter is a subset contained within the former. Some raster formats can store the histogram data as metadata. Also, there are is a shortcut; overviews can be used to get approximate values faster. Understood, thanks for pointing that out. In that case, however, the responsibility to decide whether to include out-of-range values lies in the raster format, correct? You can write a simple python script to get the histogram. You can get a good idea at http://trac.osgeo.org/gdal/browser/trunk/autotest/gcore/histogram.py#L103 Right, of course I can construct my own utility to include the GDAL libraries and call GetHistogram in the manner I'd like, and I will probably end up doing so, but in C. My main goal at the moment is a bash shell script that accomplishes a bigger task, so with a Python script I'd still have to call out to the Python script externally. Either language would be suited well for the smaller purpose of getting and analyzing the histogram. I merely found it odd that gdalinfo does not allow the user to disable an option that just doesn't seem to make sense for most applications. Yes: quite simply, a histogram that does not include illegitimate values makes more sense than a histogram that does include them. At least, this is the case in my experience; perhaps there are situations I'm unfamiliar with in which the histogram is desired to include values outside of the histogram's range, for some reason. I'd even go so far as to say that gdalinfo currently *lies* about the histogram, telling the user that the histogram shows 256 buckets from X to Y even though the histogram includes values which are *outside of that range*. Arguably an even better justification is that the machinery to exclude the illegitimate values is already present in GetHistogram, so *very* few lines of code would need to be added to gdalinfo, simply to provide a flag for the user and if that flag is specified, pass false for the right parameter to GetHistogram (bIncludeOutOfRange), instead of just calling GetDefaultHistogram. That sounds reasonable. If you can, you should submit a patch or a request at http://trac.osgeo.org/gdal/newticket Note that the functionality should include the facility to include options to mention the min/max values. Thanks; I will post a patch when I have a chance. When requesting a histogram in gdalinfo, the min/max values are already mentioned, and this will just be an addition to, and modification of, that functionality. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Histograms without null pixels
The main usage of gdalinfo is manual inspection of the image. It was not intended for use within other scripts. The output is formatted for human reading. On Tue, Mar 5, 2013 at 9:00 PM, Caleb Hanger cdhan...@gmail.com wrote: On Tue, Mar 05, 2013 at 01:31:01PM +0530, Chaitanya kumar CH wrote: On Tue, Mar 5, 2013 at 11:53 AM, Caleb Hanger cdhan...@gmail.com wrote: What do you mean? The histogram is only precomputed if gdalinfo has been run previously and stored a cache of data (the XML file) for later reference; please correct me if I'm wrong. Otherwise, it seems to me that the data *must* be analyzed and the data computed; there is no way around that. Additionally, I don't think it makes sense to say that it is faster to assemble a histogram that includes out-of-range values than to assemble a histogram that does not, because the latter is a subset contained within the former. Some raster formats can store the histogram data as metadata. Also, there are is a shortcut; overviews can be used to get approximate values faster. Understood, thanks for pointing that out. In that case, however, the responsibility to decide whether to include out-of-range values lies in the raster format, correct? You can write a simple python script to get the histogram. You can get a good idea at http://trac.osgeo.org/gdal/browser/trunk/autotest/gcore/histogram.py#L103 Right, of course I can construct my own utility to include the GDAL libraries and call GetHistogram in the manner I'd like, and I will probably end up doing so, but in C. My main goal at the moment is a bash shell script that accomplishes a bigger task, so with a Python script I'd still have to call out to the Python script externally. Either language would be suited well for the smaller purpose of getting and analyzing the histogram. I merely found it odd that gdalinfo does not allow the user to disable an option that just doesn't seem to make sense for most applications. Yes: quite simply, a histogram that does not include illegitimate values makes more sense than a histogram that does include them. At least, this is the case in my experience; perhaps there are situations I'm unfamiliar with in which the histogram is desired to include values outside of the histogram's range, for some reason. I'd even go so far as to say that gdalinfo currently *lies* about the histogram, telling the user that the histogram shows 256 buckets from X to Y even though the histogram includes values which are *outside of that range*. Arguably an even better justification is that the machinery to exclude the illegitimate values is already present in GetHistogram, so *very* few lines of code would need to be added to gdalinfo, simply to provide a flag for the user and if that flag is specified, pass false for the right parameter to GetHistogram (bIncludeOutOfRange), instead of just calling GetDefaultHistogram. That sounds reasonable. If you can, you should submit a patch or a request at http://trac.osgeo.org/gdal/newticket Note that the functionality should include the facility to include options to mention the min/max values. Thanks; I will post a patch when I have a chance. When requesting a histogram in gdalinfo, the min/max values are already mentioned, and this will just be an addition to, and modification of, that functionality. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Best regards, Chaitanya kumar CH. +91-9494447584 17.2416N 80.1426E ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Newby: Compile gdal with perl on Windows / deployment
Hello again, I proceeded a little further now. I can run make generate and Makefile.PL now and dmake starts compiling until the error-message: osr_wrap.cpp:1581:24: fatal error: cpl_string.h: No such file or directory compilation terminated. dmake.exe: Error code 129, while making 'osr_wrap.o' The file is there - but how do I include the path - I altered the path-variable and user the perl -I ... switch to the same result ... Can anybody help? Thanks Helmut Am 01.03.2013 09:30, schrieb Helmut Seidel M.A.: Hello again, Thank you for your answers, which guided me through my first steps. Yesterday I was quite happy to succeed compiling mapserver with perl which is a first step. In compiling gdal I was not that lucky but got a few millimeters forward. I tried compiling with MinGW and mysys - and the --with-perl option. gdal itself compiles ok but the perl-portion breaks up with a file not found error (perl.h). I guess MinGW gets confused with the perl installations (it tries to use the one that comes with mysys not the one I installed in C:\Perl. Is there a way how to tell configure/make which Perl to use? I then tried the visualc option - which succeeded aswell, but without perl. So here are two more questions (sorry for that): 1. Is it possibible to tell Visualc to compile with perl (as in the --with-perl - option in MinGW)? 2. Is it possible to first build gdal without perl and then install perl later? Thanks again. Helmut Am 27.02.2013 14:57, schrieb Helmut Seidel M.A.: Hello everybody, in my comming up perl-project I need to use the Geo::Gdal - module. As I understand I need to compile gdal from sources first and then build the perl bindings. So far so good, but as I'm not a C/C++ developer I ran into a host of problems. The first thing I tried, was to install MinGW and compile gdal with the --with-perl option, which ran for about two hours but broke of with a message complaining that perl 5.8 was not found on my system (I have ActivePerl 5.16 installed) - as far as I could decipher the message - I tried gdal 1.90 and gdal 1.92. I then tried to compile it using VC 10 which seemed to work but I didn't find a possibility where to put the --with-perl option - anyway I tried to compile perl using perl Makefile.PL which imediately complained about being not able to find gdal-conf (which I couldn't find either). I guess there would be lots of complaints after that, but I only got that far. I'm sure my errors have something to do with setting up paths and compilation options, but I dont't know which and where to set them. So here's my question: has anybody a blow by blow account for C/C++ dummies how to set up Geo::Gdal? That would be great! Another question is how to deploy the finished application to other computers. I hope you could understand all of my English and my questions are not too stupid - but I spend two working days on it so far so I'm a little desperate now. Thanks and regards ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Creation/read format question
Great explanation, thanks! On 2013-03-04, at 9:14 PM, Chaitanya kumar CH wrote: Hi Tyler, 'ro' and 'rw' mean what they imply. The '+' symbol tells that the format supports the Create() method. So, we can use them to create a dataset from scratch and add metadata explicitly. Without it, we need a 'template' dataset. The 'v' means that the format driver supports the virtual system interface. Note that there may be drivers that support it but don't report that they do. On Tue, Mar 5, 2013 at 9:01 AM, Tyler Mitchell tyler.mitch...@actian.com wrote: Not sure how to describe it, but when gdal lists available formats, I see a few different ways that formats are marked as read, write, etc. The rw and ro items are easy enough for me to understand, but what about: rov rw+v rwv rw+ ? Does the v refer to support in VRTs perhaps? Do rw+v, rwv, rw+ all mean same thing? Any tips appreciated :) Tyler ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Best regards, Chaitanya kumar CH. +91-9494447584 17.2416N 80.1426E Tyler Mitchell Engineering Director Actian Corporation tyler.mitch...@actian.com MOBILE 250-303-1831 SKYPE spatialguru www.actian.com ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Newby: Compile gdal with perl on Windows / deployment
On 03/05/2013 07:40 PM, Helmut Seidel M.A. wrote: Hello again, I proceeded a little further now. I can run make generate and Makefile.PL now and dmake starts compiling until the error-message: osr_wrap.cpp:1581:24: fatal error: cpl_string.h: No such file or directory compilation terminated. dmake.exe: Error code 129, while making 'osr_wrap.o' The file is there - but how do I include the path - I altered the path-variable and user the perl -I ... switch to the same result ... Where cpl_string.h is in your system - did you install GDAL or are you building within the GDAL source tree? Makefile.PL should add the include path to the Makefile_Geo__OSR, which is used for compiling osr_wrap.cpp. You may need to edit Makefile.PL a bit. See how it tests where GDAL and gdal-config is. It adds to the $INC variable include paths depending on where is finds gdal-config. Ari Can anybody help? Thanks Helmut Am 01.03.2013 09:30, schrieb Helmut Seidel M.A.: Hello again, Thank you for your answers, which guided me through my first steps. Yesterday I was quite happy to succeed compiling mapserver with perl which is a first step. In compiling gdal I was not that lucky but got a few millimeters forward. I tried compiling with MinGW and mysys - and the --with-perl option. gdal itself compiles ok but the perl-portion breaks up with a file not found error (perl.h). I guess MinGW gets confused with the perl installations (it tries to use the one that comes with mysys not the one I installed in C:\Perl. Is there a way how to tell configure/make which Perl to use? I then tried the visualc option - which succeeded aswell, but without perl. So here are two more questions (sorry for that): 1. Is it possibible to tell Visualc to compile with perl (as in the --with-perl - option in MinGW)? 2. Is it possible to first build gdal without perl and then install perl later? Thanks again. Helmut Am 27.02.2013 14:57, schrieb Helmut Seidel M.A.: Hello everybody, in my comming up perl-project I need to use the Geo::Gdal - module. As I understand I need to compile gdal from sources first and then build the perl bindings. So far so good, but as I'm not a C/C++ developer I ran into a host of problems. The first thing I tried, was to install MinGW and compile gdal with the --with-perl option, which ran for about two hours but broke of with a message complaining that perl 5.8 was not found on my system (I have ActivePerl 5.16 installed) - as far as I could decipher the message - I tried gdal 1.90 and gdal 1.92. I then tried to compile it using VC 10 which seemed to work but I didn't find a possibility where to put the --with-perl option - anyway I tried to compile perl using perl Makefile.PL which imediately complained about being not able to find gdal-conf (which I couldn't find either). I guess there would be lots of complaints after that, but I only got that far. I'm sure my errors have something to do with setting up paths and compilation options, but I dont't know which and where to set them. So here's my question: has anybody a blow by blow account for C/C++ dummies how to set up Geo::Gdal? That would be great! Another question is how to deploy the finished application to other computers. I hope you could understand all of my English and my questions are not too stupid - but I spend two working days on it so far so I'm a little desperate now. Thanks and regards ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Creation/read format question
Selon Tyler Mitchell tyler.mitch...@actian.com: Great explanation, thanks! I knew this was documented somewhere. See --formats in the General Command Line Switches section of http://gdal.org/gdal_utilities.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Creation/read format question
On 2013-03-05, at 10:39 AM, Even Rouault wrote: Selon Tyler Mitchell tyler.mitch...@actian.com: Great explanation, thanks! I knew this was documented somewhere. See --formats in the General Command Line Switches section of http://gdal.org/gdal_utilities.html Ah, perfect, I had thought I saw it somewhere too! I'm writing about it in some tutorials and it helps a lot, thanks. Tyler ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Approaches for 3D and beyond
Hi guys, What are GEOS users and devs doing to enable M measures in their GEOS-backed solutions? I won't worry about specialised functions at this point, but at least being able to reliably handle and maintain all XYZM ordinates is important for me. Is it such a lot of work that it's not worth adding this to GEOS? I saw a few messages in the archives looking at how to add measures but it wasn't encouraging ;) Do you think GEOS will support all the SFSQL 1.2 types? Are people moving to Boost or CGAL instead? Would love to hear thoughts on it. Cheers, Tyler Tyler Mitchell Engineering Director Actian Corporation tyler.mitch...@actian.com MOBILE 250-303-1831 SKYPE spatialguru www.actian.com ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Approaches for 3D and beyond
oops wrong list :) On 2013-03-05, at 11:35 AM, Tyler Mitchell wrote: Hi guys, What are GEOS users and devs doing to enable M measures in their GEOS-backed solutions? I won't worry about specialised functions at this point, but at least being able to reliably handle and maintain all XYZM ordinates is important for me. Is it such a lot of work that it's not worth adding this to GEOS? I saw a few messages in the archives looking at how to add measures but it wasn't encouraging ;) Do you think GEOS will support all the SFSQL 1.2 types? Are people moving to Boost or CGAL instead? Would love to hear thoughts on it. Cheers, Tyler ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] GDAL fails on reading rset via Virtual File
Folks, I am using GDAL Virtual File mechanism to read data off a stream (something like what's shown below) VSILFILE* fpTemp = VSIFileFromMemBuffer(szTempFileName, (GByte*) s-file_size(), FALSE); this-gdal_data_ = (GDALDataset*) GDALOpen(szTempFileName, GA_ReadOnly) The code works for most of the file formats of varying size but when I tried my reader on the rset NITF files (for example foo.r0), it exited with an error below Image 29952x21760x1 File Format: File formats supported by GDAL Decoding view (0,0) (+29952,+21760)...ERROR 4: VSIFOpenL(/vsimem/tmp_0x1758cd0) failed unexpectedly in jpgdataset.cpp ERROR 1: /vsimem/tmp_0x1758cd0, band 1: IReadBlock failed at X offset 75, Y offset 2 ERROR 1: GetBlockRef failed at X block offset 75, Y block offset 2 vil_convert: /home/aashish/tools/vidtk/src.git/wrappers/vil_plugins/file_formats/vil_gdal_image.cxx:235: vil_image_view_base_sptr vil_gdal_image_impl::read_data(vil_memory_chunk_sptr, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int) [with TPixel = unsigned char]: Assertion `err == CE_None' failed. Any help is appreciated. Thanks. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Approaches for 3D and beyond
On 03/05/2013 09:36 PM, Tyler Mitchell wrote: oops wrong list :) Not really. XYZM issue is relevant for GDAL too http://trac.osgeo.org/gdal/wiki/GDAL20Changes Ari On 2013-03-05, at 11:35 AM, Tyler Mitchell wrote: Hi guys, What are GEOS users and devs doing to enable M measures in their GEOS-backed solutions? I won't worry about specialised functions at this point, but at least being able to reliably handle and maintain all XYZM ordinates is important for me. Is it such a lot of work that it's not worth adding this to GEOS? I saw a few messages in the archives looking at how to add measures but it wasn't encouraging ;) Do you think GEOS will support all the SFSQL 1.2 types? Are people moving to Boost or CGAL instead? Would love to hear thoughts on it. Cheers, Tyler ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
