Re: [gdal-dev] CSharp bindings question
Hi Ari, In my fork I've now added mono-mcs into the travis test machine and make test to CSharp. The build tests all work. https://travis-ci.org/OSGeo/gdal/builds/6445 However, one fix I did for the CSharp bindings is most probably wrong (convert return value of handle.AddrOfPinnedObject() to char *) https://github.com/ajolma/gdal/commit/6509ef06d6f89d99c446300e4f4a63b656139 11e Tamas, do you have an idea for this? There are a lot of #if ... #endif's in for example ogr.i to limit %feature(kwargs), this is due to a swig bug, which is fixed in 3.03 so we need to leave them in for now. https://github.com/swig/swig/issues/242 Good to know. I did a try with your branch and looks generally good to me as far as Python and Java are concerned. Thanks for your efforts on this. I found that the Python bindings no longer compile with SWIG 1.3.40, but that's not really an issue if can now move to later SWIG versions (and Python 3 support in SWIG 1.3.X needed an hack, so that would be great if we don't need it anymore). I tried with SWIG 2.0.10 and works good (tested with Python 3.1 since Travis already tests Python 2) I tried the Java bindings and looked at the generated code. I notice that 3 SWIGTYPE_p_ Java classes are generated due to missing typemaps : - SWIGTYPE_p_CPLErrorHandler: only used by a gdal.CPLPushErrorHandler() method that wasn't exposed before. We'd likely need to hide it. It would require otherwise defining the appropriate typemap for the callback of PushErrorHandler() but callback typemaps are generally involved. - SWIGTYPE_p_long_long: only used by Feature.SetFieldInteger64List() that wasn't exposed before. I guess I should be able to figure out the needed typemap - SWIGTYPE_p_unsigned_char : only used as the return type of gdal.HexToBinary(). This used to return byte[] before. Those are minor issues that I could fix once it is merged. There's a lot still to do to cleanup the common interface files but how do you feel, is there a chance that this is accepted into the trunk (and 2.1)? trunk as today is 2.0, so I think it would be more prudent if we wait for 2.0 to have been branched off from trunk before committing this. If we commit that early in 2.1 dev cycle, we should have enough time to discover and address potential issues. I'd also love to have a policy for developing the bindings and working test codes for all maintained languages. A rule could be that the use of #if ... #endif in common files needs a good justification I think one of the main pattern is that I want to map a C function to SWIG (for the need of the language of my interest) and realize that it needs a new typemap. So I add it for the language, and protect it by #if #endif to avoid breaking compilation for other languages (definitly justified), or expose new methods with the default and non-working SWIGTYPE_p_X types generated by SWIG. In this later case, we could question if #if #endif is justified: exposing non working methods isn't really great, but that shouldn't break users' workflow, since they won't be able to use them in practice, so they can be later fixed without really creating compatibility problems. As it might be difficult in practice to coordinate with all maintainers at the exact time were things are needed, perhaps we need some wiki page to document which new methods have been added and which languages are impacted, so that the maintainer can look at it when he has time and add the needed typemaps. and commits, which do not cause test codes to fail are ok per se. Seems reasonable to me. Even -- Spatialys - Geospatial professional services http://www.spatialys.com ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] CSharp bindings question
In my fork I've now added mono-mcs into the travis test machine and make test to CSharp. The build tests all work. https://travis-ci.org/OSGeo/gdal/builds/6445 However, one fix I did for the CSharp bindings is most probably wrong (convert return value of handle.AddrOfPinnedObject() to char *) https://github.com/ajolma/gdal/commit/6509ef06d6f89d99c446300e4f4a63b65613911e Tamas, do you have an idea for this? There are a lot of #if ... #endif's in for example ogr.i to limit %feature(kwargs), this is due to a swig bug, which is fixed in 3.03 so we need to leave them in for now. https://github.com/swig/swig/issues/242 There's a lot still to do to cleanup the common interface files but how do you feel, is there a chance that this is accepted into the trunk (and 2.1)? I'd also love to have a policy for developing the bindings and working test codes for all maintained languages. A rule could be that the use of #if ... #endif in common files needs a good justification and commits, which do not cause test codes to fail are ok per se. Best, Ari On 26.05.2015 13:53, Tamas Szekeres wrote: Is that a requirement that the bindings should work well with all SWIG versions or that the generated wrappers should work just fine? Formerly I have been thinking that we should support all versions, but it took large amount of extra efforts to work around all incompatible changes what SWIG introduces all the time even with the minor releases. Regarding to SWIG C# the earlier versions produced definitely wrong code and I had implement quite some generic stuff in the bindings (for example to work around the early garbage collection issues). I see some enhancements in the recent versions in this regard, but I'm not sure if I can completely remove these additions to get a stable and consistent build. Tamas 2015-05-26 11:09 GMT+02:00 Ari Jolma ari.jo...@gmail.com mailto:ari.jo...@gmail.com: 26.05.2015, 11:38, Even Rouault kirjoitti: Le mardi 26 mai 2015 10:13:49, Tamas Szekeres a écrit : Hi Ari, I haven't tried to compile that with mono for quite a long time. I'll give it a try. However we did not follow the latest changes in the SWIG implementation with the bindings, so I'd try with an earlier version (ie. 1.3.39) to generate the wrappers. I can confirm that I can compile the CSharp bindings on Linux with SWIG 1.3.40 (and run the tests), but I get the same error as Ari with SWIG 2.0.X As far as I know, Java and Python bindings build and run equaly well with SWIG 1.3.40 or 2.0.X (although there's a Unix makefile hack to have Python 3.2 compat, conditionnaly applied with SWIG 1.3.40, that is no longer needed with SWIG 2.0.4 or later) Swig 1.3.39 seems questionable. Just look at the download amounts at sourceforge. 1.3.39 one download and 1.3.40 148 downloads per week. However, 1.3.39 does *not* put the PVINVOKE() method twice into the PVINVOKE.cs file. May be we should consider including the generated wrappers in gdal instead of let the users to use different versions with different results. It would be good if we could have a common SWIG version that works for all the bindings. So currently it seems to be 1.3.40 ? Regarding putting the generated wrappers in SVN, that's already what we do for Python. We could also just include the generated wrappers in the tarballs. IMO users = people who use ready-made packages. Developers and packagers should be intelligent enough to use development tools. I don't like the idea of having generated files in source repositories. I'm also of the opinion that there should be a really good reason to use an old version of a common tool. And at least in my Linux (Mint, Maya - hmm, that seems already pretty old, I should upgrade) swig 2.0.11 is the current. But that's just me I guess. Ari Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gdal_translate and gdal_gdaladdo to create tiled pyrimidal tiffs: grey?
This post should now be mailed to the list as well. Sorry for the inconvenience. -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-translate-and-gdal-gdaladdo-to-create-tiled-pyrimidal-tiffs-grey-tp5208048p5208112.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] CSharp bindings question
Hi Ari, Which SWIG version have you been testing with? Converting IntPtr to string doesn't seem to be a good solution. We should do something like what we do for ReadRaster which also use AddrOfPinnedObject(). I'm trying to reproduce this. Best regards, Tamas 2015-05-29 9:11 GMT+02:00 Ari Jolma ari.jo...@gmail.com: In my fork I've now added mono-mcs into the travis test machine and make test to CSharp. The build tests all work. https://travis-ci.org/OSGeo/gdal/builds/6445 However, one fix I did for the CSharp bindings is most probably wrong (convert return value of handle.AddrOfPinnedObject() to char *) https://github.com/ajolma/gdal/commit/6509ef06d6f89d99c446300e4f4a63b65613911e Tamas, do you have an idea for this? There are a lot of #if ... #endif's in for example ogr.i to limit %feature(kwargs), this is due to a swig bug, which is fixed in 3.03 so we need to leave them in for now. https://github.com/swig/swig/issues/242 There's a lot still to do to cleanup the common interface files but how do you feel, is there a chance that this is accepted into the trunk (and 2.1)? I'd also love to have a policy for developing the bindings and working test codes for all maintained languages. A rule could be that the use of #if ... #endif in common files needs a good justification and commits, which do not cause test codes to fail are ok per se. Best, Ari On 26.05.2015 13:53, Tamas Szekeres wrote: Is that a requirement that the bindings should work well with all SWIG versions or that the generated wrappers should work just fine? Formerly I have been thinking that we should support all versions, but it took large amount of extra efforts to work around all incompatible changes what SWIG introduces all the time even with the minor releases. Regarding to SWIG C# the earlier versions produced definitely wrong code and I had implement quite some generic stuff in the bindings (for example to work around the early garbage collection issues). I see some enhancements in the recent versions in this regard, but I'm not sure if I can completely remove these additions to get a stable and consistent build. Tamas 2015-05-26 11:09 GMT+02:00 Ari Jolma ari.jo...@gmail.com: 26.05.2015, 11:38, Even Rouault kirjoitti: Le mardi 26 mai 2015 10:13:49, Tamas Szekeres a écrit : Hi Ari, I haven't tried to compile that with mono for quite a long time. I'll give it a try. However we did not follow the latest changes in the SWIG implementation with the bindings, so I'd try with an earlier version (ie. 1.3.39) to generate the wrappers. I can confirm that I can compile the CSharp bindings on Linux with SWIG 1.3.40 (and run the tests), but I get the same error as Ari with SWIG 2.0.X As far as I know, Java and Python bindings build and run equaly well with SWIG 1.3.40 or 2.0.X (although there's a Unix makefile hack to have Python 3.2 compat, conditionnaly applied with SWIG 1.3.40, that is no longer needed with SWIG 2.0.4 or later) Swig 1.3.39 seems questionable. Just look at the download amounts at sourceforge. 1.3.39 one download and 1.3.40 148 downloads per week. However, 1.3.39 does *not* put the PVINVOKE() method twice into the PVINVOKE.cs file. May be we should consider including the generated wrappers in gdal instead of let the users to use different versions with different results. It would be good if we could have a common SWIG version that works for all the bindings. So currently it seems to be 1.3.40 ? Regarding putting the generated wrappers in SVN, that's already what we do for Python. We could also just include the generated wrappers in the tarballs. IMO users = people who use ready-made packages. Developers and packagers should be intelligent enough to use development tools. I don't like the idea of having generated files in source repositories. I'm also of the opinion that there should be a really good reason to use an old version of a common tool. And at least in my Linux (Mint, Maya - hmm, that seems already pretty old, I should upgrade) swig 2.0.11 is the current. But that's just me I guess. Ari Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] CSharp bindings question
On 29.05.2015 15:22, Tamas Szekeres wrote: Hi Ari, Which SWIG version have you been testing with? Mostly with the one that's loaded by default into a new Ubuntu (travis) / Mint (I have version 17 which is based on Ubuntu Trusty), both are 2.0.11 I think. Converting IntPtr to string doesn't seem to be a good solution. We should do something like what we do for ReadRaster which also use AddrOfPinnedObject(). I'm trying to reproduce this. Yes I'm sure it is not, I just used it to make it compile. It is used only(?) by FileFromMemBuffer, which is not tested in CSharp tests (I'm not yet testing it in Perl bindings either). Ari Best regards, Tamas 2015-05-29 9:11 GMT+02:00 Ari Jolma ari.jo...@gmail.com mailto:ari.jo...@gmail.com: In my fork I've now added mono-mcs into the travis test machine and make test to CSharp. The build tests all work. https://travis-ci.org/OSGeo/gdal/builds/6445 However, one fix I did for the CSharp bindings is most probably wrong (convert return value of handle.AddrOfPinnedObject() to char *) https://github.com/ajolma/gdal/commit/6509ef06d6f89d99c446300e4f4a63b65613911e Tamas, do you have an idea for this? There are a lot of #if ... #endif's in for example ogr.i to limit %feature(kwargs), this is due to a swig bug, which is fixed in 3.03 so we need to leave them in for now. https://github.com/swig/swig/issues/242 There's a lot still to do to cleanup the common interface files but how do you feel, is there a chance that this is accepted into the trunk (and 2.1)? I'd also love to have a policy for developing the bindings and working test codes for all maintained languages. A rule could be that the use of #if ... #endif in common files needs a good justification and commits, which do not cause test codes to fail are ok per se. Best, Ari On 26.05.2015 13:53, Tamas Szekeres wrote: Is that a requirement that the bindings should work well with all SWIG versions or that the generated wrappers should work just fine? Formerly I have been thinking that we should support all versions, but it took large amount of extra efforts to work around all incompatible changes what SWIG introduces all the time even with the minor releases. Regarding to SWIG C# the earlier versions produced definitely wrong code and I had implement quite some generic stuff in the bindings (for example to work around the early garbage collection issues). I see some enhancements in the recent versions in this regard, but I'm not sure if I can completely remove these additions to get a stable and consistent build. Tamas 2015-05-26 11:09 GMT+02:00 Ari Jolma ari.jo...@gmail.com mailto:ari.jo...@gmail.com: 26.05.2015, 11:38, Even Rouault kirjoitti: Le mardi 26 mai 2015 10:13:49, Tamas Szekeres a écrit : Hi Ari, I haven't tried to compile that with mono for quite a long time. I'll give it a try. However we did not follow the latest changes in the SWIG implementation with the bindings, so I'd try with an earlier version (ie. 1.3.39) to generate the wrappers. I can confirm that I can compile the CSharp bindings on Linux with SWIG 1.3.40 (and run the tests), but I get the same error as Ari with SWIG 2.0.X As far as I know, Java and Python bindings build and run equaly well with SWIG 1.3.40 or 2.0.X (although there's a Unix makefile hack to have Python 3.2 compat, conditionnaly applied with SWIG 1.3.40, that is no longer needed with SWIG 2.0.4 or later) Swig 1.3.39 seems questionable. Just look at the download amounts at sourceforge. 1.3.39 one download and 1.3.40 148 downloads per week. However, 1.3.39 does *not* put the PVINVOKE() method twice into the PVINVOKE.cs file. May be we should consider including the generated wrappers in gdal instead of let the users to use different versions with different results. It would be good if we could have a common SWIG version that works for all the bindings. So currently it seems to be 1.3.40 ? Regarding putting the generated wrappers in SVN, that's already what we do for Python. We could also just include the generated wrappers in the tarballs. IMO users = people who use ready-made packages. Developers and packagers should be intelligent enough to use development tools. I don't like the idea of having generated files in source repositories. I'm also of the opinion that there
