I've been in C++ land for so long that I'm afraid I don't know the idioms that
modern C culture likes these days. Is what you describe OIIO_ImageInput_open or
OIIO_ImageBufAlgo_add... does that look like the way a 2020 C programmer would
try to "namespace" things in C that has no namespaces? Or would a C programmer
scoff at how wordy it is?
As far as how to name overloads... I think it might be instructive to look at a
few individual cases first and what feels right, then see if they generalize
into an overall rule. I'm hesitant to propose a rule first without examples to
know if we're going to hate it in practice.
Which C standard would you want to target?
-- lg
> On Oct 19, 2020, at 1:51 PM, Scott Wilson <[email protected]> wrote:
>
> Sounds good to me. Also, if you want to automate the c bindings to some
> degree, then you can look at https://github.com/EmbarkStudios/physx-rs
> <https://github.com/EmbarkStudios/physx-rs> for inspiration. There's a talk
> in the readme with their c bindings builder. Failing that, I'm fine with
> going through the legwork of making the c interface.
>
> If I may make a suggestion for the naming, I'd suggest more or less following
> what OIIO does in C++. So, for example, OIIO::ImageInput.open(...) would be
> OIIO_ImageInput_open(...) and classes/structs would be OIIOImageInput.
>
> The only question I have at the moment is how do we want to handle naming
> functions with overrides?
>
> On Mon., Oct. 19, 2020, 1:00 p.m. Anders Langlands,
> <[email protected] <mailto:[email protected]>> wrote:
> Branching this thread to talk specifics...
>
> So to summarize the different approaches we've each taken, in my repo
> (https://github.com/anderslanglands/oiio-rs/tree/master/coiio
> <https://github.com/anderslanglands/oiio-rs/tree/master/coiio>) I have a
> small shim library that creates a C interface to OIIO by wrapping with
> functions like:
>
> ImageBuf ImageBuf_create(const char* filename) {
> return new OIIO::ImageBuf(OIIO::string_view(filename));
> }
>
> When building the rust crate, this is compiled into a static library by Cargo
> using CMake, with the environment variable OIIO_ROOT specifying the path to
> the OIIO installation. Currently the build script errors out if OIIO_ROOT is
> not specified, but it would be trivial to have it default to /usr/local etc.
> I also at one point had it downloading OIIO and building it directly, but
> cmake-rs had some issues with always rebuilding OIIO on any change to the
> crate, which made development interminable, so I stripped that out. That was
> nearly two years ago so may have been fixed. A good alternative would be to
> provide a separate (bash or python) build script that would download deps and
> run the build manually as a pre-process.
>
> From what I can tell from Scott's repo, cxx is doing essentially the same
> thing, but cxx handles building the shim library internally, so there's no
> need for invoking CMake from build.rs <http://build.rs/>. Presumably you'd
> still need to specify the path to (and potentially build) OIIO and its deps
> using this method. If you compare the code in Scott's repo to mine, you can
> see they're very similar indeed.
>
> The issue I see here is that I don't think cxx actually saves you anything.
> In fact there's *more* code in the cxx case because you're specifying both
> the implementation (ffi.cpp) and the interface (ffi.h), whereas I only have
> to specify the implementation (coiio.cpp) and the interface is declared
> solely in ffi.rs <http://ffi.rs/>.
>
> Building and linking the static library is the easiest part of the process,
> and is also the part that would be more generally useful outside of just Rust
> - you could use that C interface to trivially bind any language you want, so
> I think we want to preserve that.
>
> That leaves the question of how do we generate the C interface in the first
> place. I've been doing it manually, which is a tedious process, but works.
> Since it's just calling the C++ directly I think it's also reasonably sturdy
> against changes to the underlying OIIO API, since the compiler should catch
> most misuses, although I'm sure there are plenty of opportunities for subtle
> bugs still. Not to mention bikeshedding about naming conventions and code
> styles :)
>
> My idea for how to make the binding generation more automatic was to try and
> leverage libclang to generate the C wrappers semi-automatically. This would
> still require a fair amount of code: first writing the binding generator,
> then writing the rules for how to wrap the C++ API, but should allow
> generating the C API automatically, rather than maintaining it manually for
> every release. The downside is obviously this tool doesn't exist yet, and it
> would add a (optional) dependency on clang to the project.
>
> I think the best course of action here would be to write a C wrapper in OIIO
> itself that could be maintained along with the rest of the project. This
> would build a little C99 library that could be installed alongside the C++
> library. This would unfortunately mean manually writing a header to go with
> it, although that part could probably be automated with a little scripting.
>
> Then oiio-rs should be a completely separate Rust crate that just uses
> bindgen to generate the unsafe API from the C library automatically, and
> provides a safe API on top of that. (Larry - Rust/Cargo kinda assumes that
> Cargo is the thing doing all the building and dependency management, so
> trying to provide a Rust artefact from the OIIO build process would be
> painful if it's possible at all).
>
> Let me know what you think.
>
>
> Cheers,
> Anders
>
>
>
>
>
> On Tue, 20 Oct 2020 at 02:53, Alvaro Castaneda <[email protected]
> <mailto:[email protected]>> wrote:
> Hi Larry and Anders
> I'm helping Scott with the Rust Wrapper.
> using CXX is been good, it is a very manual process and makes it only usable
> in Rust, Larry you mentioned a minimal C API, that would make it much simpler
> to wrap to Rust, that would also mean in can be wrapped to many other
> languages,
> So far we didn't want to go the C route, but that might not be a bad idea
> since it would open the library a lot more and it might make it simpler to
> automate, at least the bulk of it, for Rust.
>
> We need to discuss the approach.
>
> On Sun, Oct 18, 2020 at 8:40 PM Anders Langlands <[email protected]
> <mailto:[email protected]>> wrote:
> Hi Scott, yes please do add me to the repo I’d love to take a look and pitch
> in as time allows.
>
> On Mon, 19 Oct 2020 at 13:43, Scott Wilson <[email protected]
> <mailto:[email protected]>> wrote:
> Hey Anders,
>
> Cxx has so far been pretty okay. It's pretty manual (hopefully autocxx makes
> it better, but as far as I know, it's still nowhere ready for using it on
> OpenImageIO). Right now the process looks something like this:
>
> 1. Create a header/cpp file that contains all of your class methods as
> functions.
> 2. Create an unsafe Rust interferface that's a 1 to 1 copy of the C++ side.
> 3. Create a safe Rust interface.
>
> For me, the really nice thing is I don't need to worry about the C++ -> C ->
> Rust steps. It drops the C step, but I still need to write that C++ "ugly"
> interface.
>
> Also, if you want to join in on the fun, our repo is currently private while
> we get things to a working state. But, I can add you to the repo. Otherwise,
> I'm up for a discussion on how to take both designs and come up with the best
> one.
>
> On Sun, Oct 18, 2020 at 4:51 PM Anders Langlands <[email protected]
> <mailto:[email protected]>> wrote:
> Wrapping C in Rust is a two-stage process: first writing an "unsafe" FFI
> binding to the C API, which is usually almost completely automated with a
> crate called bindgen, then writing a "safe" crate that provides a Rust-y API
> using the unsafe FFI bindings. Wrapping C++ means writing a C API first, then
> binding that to Rust, which is what my crate does.
>
> I've been meaning to return to this (and OSL, OpenSubdiv and others) at some
> point and try to make a project-specific C-binding generator using libclang,
> as manually maintaining the C stubs is laborious and error-prone.
>
> Scott, I'd be curious to know how you're getting on with cxx, I've been
> meaning to look into that. I'd be happy to collaborate on something we could
> integrate into the main project as Larry suggests.
>
> On Mon, 19 Oct 2020 at 12:30, Larry Gritz <[email protected]
> <mailto:[email protected]>> wrote:
> Feel free to have the discussion on-list, I'm sure it would be of interest to
> many.
>
> If there was consensus on what the Rust APIs should look like, I would
> welcome adding a set of Rust bindings to the main OIIO distribution. Assuming
> that makes sense, I was thinking it would be much like we now have with the
> Python bindings. The advantage to making Rust bindings part of the main build
> would be that it could be built and tested as part of our CI, versioned along
> with the rest of OIIO, and essentially never allowed to break. Also, just
> like we would never accept a PR that added C++ functionality without making
> sure the Python bindings kept up, we could ensure that nothing is left out of
> the Rust bindings. While I can appreciate the cleanliness and independence of
> it being a separate project, I can't help but think that it will be a
> neverending nightmare to try to keep the bindings in sync with the main
> project.
>
> I don't know how automated it is to make Rust bindings for C (I know it's a
> PITA for C++), but if making Rust bindings is substantially easier if you had
> minimal plain C wrappers for the major C++ classes, I'm sure there would be a
> lot of happy consumers of that even outside the Rust interest group.
>
> I haven't had time to try Rust myself for any programming project, though
> I've followed it from afar and like the idea of helping that community. TBH,
> the main thing that keeps me from spending any time on Rust is just that I
> can't contemplate the hassle of trying to program without my favourite
> libraries, and having OIIO (and its many utilities that I reuse in basically
> everything I write) available in Rust will substantially lower the bar for me
> to dabble in it more.
>
> -- lg
>
>
>> On Oct 18, 2020, at 4:12 PM, Scott Wilson <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Hey Anders,
>>
>> We were inspired by what you did, and also decided to see if we can take
>> this in a slightly different direction/ use cxx. If you're interested in
>> discussing the wrapper more we can take it off the list.
>>
>> On Sun, Oct 18, 2020 at 3:35 PM Anders Langlands <[email protected]
>> <mailto:[email protected]>> wrote:
>> I also have a rust binding here if you're interested:
>> https://github.com/anderslanglands/oiio-rs
>> <https://github.com/anderslanglands/oiio-rs>
>> On Sun, 18 Oct 2020 at 04:43, Scott Wilson <[email protected]
>> <mailto:[email protected]>> wrote:
>> Awesome, thank you very much! I'll try this out and see how badly I break
>> things.
>>
>> On Sat., Oct. 17, 2020, 1:02 a.m. Larry Gritz, <[email protected]
>> <mailto:[email protected]>> wrote:
>> If you know the true legal extent of the memory allocation in which that
>> data pointer is located (in this case, the beginning and ending of the
>> vector, if you are passing a pointer to one of the elements of that vector),
>> then I think you could certainly consider it an error if any of these
>> addresses lay outside that buffer:
>>
>> data + xstride*width - 1
>> data + ystride*height - 1
>> data + ystride*(height - 1) + xstride*width - 1
>> data + zstride*depth
>> data + zstride*(depth - 1) + ystride*height - 1
>> data + zstride*(depth - 1) + ystride*(height - 1) + xstride*width - 1
>>
>> There may be a more succinct way to put that, but I think it covers all the
>> cases of + and - strides.
>>
>>
>>> On Oct 17, 2020, at 12:42 AM, Scott Wilson <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>> Thanks! I guess to come from this at a different angle, let's say I'm doing
>>> something like this:
>>>
>>> std::vector<uint8_t> pixels(10*10*3*1);
>>> ImageInput.read_image(TypeDesc::UINT8, @pixels[0])
>>>
>>> Would there be a case where I could pick a stride value that would fall
>>> outside the pixels vector?
>>>
>>> PS: Thanks! I'm working on this with a friend, and hope to have something
>>> released in the near future.
>>>
>>> On Fri., Oct. 16, 2020, 11:47 p.m. Larry Gritz, <[email protected]
>>> <mailto:[email protected]>> wrote:
>>> Oops, my math was wrong (in an unimportant detail): If you are making a
>>> mosaic of 16x5 of these 10x10 images, it is 80 small images you are
>>> assembling in total, not 40.
>>>
>>>
>>>> On Oct 16, 2020, at 11:43 PM, Larry Gritz <[email protected]
>>>> <mailto:[email protected]>> wrote:
>>>>
>>>> The strides don't describe the size of the image, they are the spacing in
>>>> memory of where you want the values to be placed upon being read (or taken
>>>> from in order to write). There is no invalid set of strides, because the
>>>> caller might want them to end up anywhere in memory.
>>>>
>>>> Or am I misunderstanding?
>>>>
>>>> For a fully "contiguous" memory buffer where you intend for every plane,
>>>> scanline, pixel, and channel immediately follows the previous one, then in
>>>> our example the strides would be xstride=3, ystride=30, zstride=300.
>>>> (Though for a 2D image, the zstride is not used.)
>>>>
>>>> Here's an example of where you might have a stride range that is wildly
>>>> outside this: Let's say that you have 40 of these 10 x 10 x 3 x uint8
>>>> image files and you are trying to read them in and assemble them into a
>>>> single RGBA mosaic image of 16x5 x 4 x uint8 (the additional channel is
>>>> alpha, which you will separately fill in as 1.0 [or 255 uint8] because
>>>> it's not in your RGB files). Here's a cartoon to illustrate this:
>>>>
>>>> +-----------------------------------------+
>>>> | | | | | | | | | | | | | | | | | | | | | |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
>>>> | | | | | | | | | | | | | | | | | | | | | |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
>>>> | | | | | | | | | |X| | | | | | | | | | | |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
>>>> | | | | | | | | | | | | | | | | | | | | | |
>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
>>>> | | | | | | | | | | | | | | | | | | | | | |
>>>> +-----------------------------------------+
>>>>
>>>> Each of my little grid cells is a 10x10 image. But that 10x10 image
>>>> denoted by the "X" needs to be placed in memory in the right portion of
>>>> the 16x10 x 5x10 mosaic. So what are the strides we use for the read?
>>>> Well, the xstride is 4 because we're making room for an alpha channel that
>>>> wasn't present in the file, the ystride is 640 (= 10*16*4), because each
>>>> scanline of the little 10x10 image that you read needs to be placed on the
>>>> proper scanline of the 160x50 mosaic you are assembling in memory.
>>>>
>>>> -- lg
>>>>
>>>>
>>>> P.S. Woo-hoo for making a Rust wrapper. I think that's a totally great
>>>> thing.
>>>>
>>>>
>>>>
>>>>> On Oct 16, 2020, at 10:46 PM, Scott Wilson <[email protected]
>>>>> <mailto:[email protected]>> wrote:
>>>>>
>>>>> I'm experimenting with a Rust wrapper for OIIO, and had some questions
>>>>> about the stride.
>>>>>
>>>>> Let's say I have an image that is 10x10 pixels, and 3 channels, and 1
>>>>> byte per channel. What strides would be invalid for that image? I'm
>>>>> guessing that anything between -10 * 10 * 3 * 1 to 10 * 10 * 3 * 1 and
>>>>> the AutoStride would be valid, and everything else may try to access
>>>>> memory that isn't initialized. Is this assumption correct, or am I
>>>>> missing something?
>>>>>
>>>>> Thanks!
>>>>> _______________________________________________
>>>>> Oiio-dev mailing list
>>>>> [email protected] <mailto:[email protected]>
>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>>>>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>>>>
>>>> --
>>>> Larry Gritz
>>>> [email protected] <mailto:[email protected]>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Oiio-dev mailing list
>>>> [email protected] <mailto:[email protected]>
>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>>>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>>>
>>> --
>>> Larry Gritz
>>> [email protected] <mailto:[email protected]>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Oiio-dev mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>>> _______________________________________________
>>> Oiio-dev mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>>
>> --
>> Larry Gritz
>> [email protected] <mailto:[email protected]>
>>
>>
>>
>>
>> _______________________________________________
>> Oiio-dev mailing list
>> [email protected] <mailto:[email protected]>
>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>> _______________________________________________
>> Oiio-dev mailing list
>> [email protected] <mailto:[email protected]>
>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>> _______________________________________________
>> Oiio-dev mailing list
>> [email protected] <mailto:[email protected]>
>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>> _______________________________________________
>> Oiio-dev mailing list
>> [email protected] <mailto:[email protected]>
>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
>
> --
> Larry Gritz
> [email protected] <mailto:[email protected]>
>
>
>
>
> _______________________________________________
> Oiio-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
> _______________________________________________
> Oiio-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
> _______________________________________________
> Oiio-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
> _______________________________________________
> Oiio-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
> _______________________________________________
> Oiio-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
> _______________________________________________
> Oiio-dev mailing list
> [email protected] <mailto:[email protected]>
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org>
> _______________________________________________
> Oiio-dev mailing list
> [email protected]
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
[email protected]
_______________________________________________
Oiio-dev mailing list
[email protected]
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
