I’d personally prefer to see if we could shorten the names a little so “oiio_ii_open” instead of “OIIO_ImageInput_open” if we could make sure that there wouldn’t be any abbreviation conflicts. The long form does have the advantage that it’s unambiguous though.
Regarding standards, Visual Studio has only just added support for C11 and C17 in preview and I don’t think we’d need anything beyond C99 for an API wrapper anyway? On Tue, 20 Oct 2020 at 10:18, Larry Gritz <[email protected]> wrote: > I've been in C++ land for so long that I'm afraid I don't know the idioms > that modern C culture likes these days. Is what you describe > OIIO_ImageInput_open or OIIO_ImageBufAlgo_add... does that look like the > way a 2020 C programmer would try to "namespace" things in C that has no > namespaces? Or would a C programmer scoff at how wordy it is? > > As far as how to name overloads... I think it might be instructive to look > at a few individual cases first and what feels right, then see if they > generalize into an overall rule. I'm hesitant to propose a rule first > without examples to know if we're going to hate it in practice. > > Which C standard would you want to target? > > -- lg > > > On Oct 19, 2020, at 1:51 PM, Scott Wilson <[email protected]> wrote: > > Sounds good to me. Also, if you want to automate the c bindings to some > degree, then you can look at https://github.com/EmbarkStudios/physx-rs > for inspiration. There's a talk in the readme with their c bindings > builder. Failing that, I'm fine with going through the legwork of making > the c interface. > > If I may make a suggestion for the naming, I'd suggest more or less > following what OIIO does in C++. So, for example, > OIIO::ImageInput.open(...) would be OIIO_ImageInput_open(...) and > classes/structs would be OIIOImageInput. > > The only question I have at the moment is how do we want to handle naming > functions with overrides? > > On Mon., Oct. 19, 2020, 1:00 p.m. Anders Langlands, < > [email protected]> wrote: > >> Branching this thread to talk specifics... >> >> So to summarize the different approaches we've each taken, in my repo ( >> https://github.com/anderslanglands/oiio-rs/tree/master/coiio) I have a >> small shim library that creates a C interface to OIIO by wrapping with >> functions like: >> >> ImageBuf ImageBuf_create(const char* filename) { >> return new OIIO::ImageBuf(OIIO::string_view(filename)); >> } >> >> When building the rust crate, this is compiled into a static library by >> Cargo using CMake, with the environment variable OIIO_ROOT specifying the >> path to the OIIO installation. Currently the build script errors out if >> OIIO_ROOT is not specified, but it would be trivial to have it default to >> /usr/local etc. I also at one point had it downloading OIIO and building it >> directly, but cmake-rs had some issues with always rebuilding OIIO on any >> change to the crate, which made development interminable, so I stripped >> that out. That was nearly two years ago so may have been fixed. A good >> alternative would be to provide a separate (bash or python) build script >> that would download deps and run the build manually as a pre-process. >> >> From what I can tell from Scott's repo, cxx is doing essentially the same >> thing, but cxx handles building the shim library internally, so there's no >> need for invoking CMake from build.rs. Presumably you'd still need to >> specify the path to (and potentially build) OIIO and its deps using this >> method. If you compare the code in Scott's repo to mine, you can see >> they're very similar indeed. >> >> The issue I see here is that I don't think cxx actually saves you >> anything. In fact there's *more* code in the cxx case because you're >> specifying both the implementation (ffi.cpp) and the interface (ffi.h), >> whereas I only have to specify the implementation (coiio.cpp) and the >> interface is declared solely in ffi.rs. >> >> Building and linking the static library is the easiest part of the >> process, and is also the part that would be more generally useful outside >> of just Rust - you could use that C interface to trivially bind any >> language you want, so I think we want to preserve that. >> >> That leaves the question of how do we generate the C interface in the >> first place. I've been doing it manually, which is a tedious process, but >> works. Since it's just calling the C++ directly I think it's also >> reasonably sturdy against changes to the underlying OIIO API, since the >> compiler should catch most misuses, although I'm sure there are plenty of >> opportunities for subtle bugs still. Not to mention bikeshedding about >> naming conventions and code styles :) >> >> My idea for how to make the binding generation more automatic was to try >> and leverage libclang to generate the C wrappers semi-automatically. This >> would still require a fair amount of code: first writing the binding >> generator, then writing the rules for how to wrap the C++ API, but should >> allow generating the C API automatically, rather than maintaining it >> manually for every release. The downside is obviously this tool doesn't >> exist yet, and it would add a (optional) dependency on clang to the project. >> >> I think the best course of action here would be to write a C wrapper in >> OIIO itself that could be maintained along with the rest of the project. >> This would build a little C99 library that could be installed alongside the >> C++ library. This would unfortunately mean manually writing a header to go >> with it, although that part could probably be automated with a little >> scripting. >> >> Then oiio-rs should be a completely separate Rust crate that just uses >> bindgen to generate the unsafe API from the C library automatically, and >> provides a safe API on top of that. (Larry - Rust/Cargo kinda assumes that >> Cargo is the thing doing all the building and dependency management, so >> trying to provide a Rust artefact from the OIIO build process would be >> painful if it's possible at all). >> >> Let me know what you think. >> >> >> Cheers, >> Anders >> >> >> >> >> >> On Tue, 20 Oct 2020 at 02:53, Alvaro Castaneda <[email protected]> wrote: >> >>> Hi Larry and Anders >>> I'm helping Scott with the Rust Wrapper. >>> using CXX is been good, it is a very manual process and makes it only >>> usable in Rust, Larry you mentioned a minimal C API, that would make it >>> much simpler to wrap to Rust, that would also mean in can be wrapped to >>> many other languages, >>> So far we didn't want to go the C route, but that might not be a bad >>> idea since it would open the library a lot more and it might make it >>> simpler to automate, at least the bulk of it, for Rust. >>> >>> We need to discuss the approach. >>> >>> On Sun, Oct 18, 2020 at 8:40 PM Anders Langlands < >>> [email protected]> wrote: >>> >>>> Hi Scott, yes please do add me to the repo I’d love to take a look and >>>> pitch in as time allows. >>>> >>>> On Mon, 19 Oct 2020 at 13:43, Scott Wilson <[email protected]> >>>> wrote: >>>> >>>>> Hey Anders, >>>>> >>>>> Cxx has so far been pretty okay. It's pretty manual (hopefully autocxx >>>>> makes it better, but as far as I know, it's still nowhere ready for using >>>>> it on OpenImageIO). Right now the process looks something like this: >>>>> >>>>> 1. Create a header/cpp file that contains all of your class methods as >>>>> functions. >>>>> 2. Create an unsafe Rust interferface that's a 1 to 1 copy of the C++ >>>>> side. >>>>> 3. Create a safe Rust interface. >>>>> >>>>> For me, the really nice thing is I don't need to worry about the C++ >>>>> -> C -> Rust steps. It drops the C step, but I still need to write that >>>>> C++ >>>>> "ugly" interface. >>>>> >>>>> Also, if you want to join in on the fun, our repo is currently private >>>>> while we get things to a working state. But, I can add you to the repo. >>>>> Otherwise, I'm up for a discussion on how to take both designs and come up >>>>> with the best one. >>>>> >>>>> On Sun, Oct 18, 2020 at 4:51 PM Anders Langlands < >>>>> [email protected]> wrote: >>>>> >>>>>> Wrapping C in Rust is a two-stage process: first writing an "unsafe" >>>>>> FFI binding to the C API, which is usually almost completely automated >>>>>> with >>>>>> a crate called bindgen, then writing a "safe" crate that provides a >>>>>> Rust-y >>>>>> API using the unsafe FFI bindings. Wrapping C++ means writing a C API >>>>>> first, then binding that to Rust, which is what my crate does. >>>>>> >>>>>> I've been meaning to return to this (and OSL, OpenSubdiv and others) >>>>>> at some point and try to make a project-specific C-binding generator >>>>>> using >>>>>> libclang, as manually maintaining the C stubs is laborious and >>>>>> error-prone. >>>>>> >>>>>> Scott, I'd be curious to know how you're getting on with cxx, I've >>>>>> been meaning to look into that. I'd be happy to collaborate on something >>>>>> we >>>>>> could integrate into the main project as Larry suggests. >>>>>> >>>>>> On Mon, 19 Oct 2020 at 12:30, Larry Gritz <[email protected]> wrote: >>>>>> >>>>>>> Feel free to have the discussion on-list, I'm sure it would be of >>>>>>> interest to many. >>>>>>> >>>>>>> If there was consensus on what the Rust APIs should look like, I >>>>>>> would welcome adding a set of Rust bindings to the main OIIO >>>>>>> distribution. >>>>>>> Assuming that makes sense, I was thinking it would be much like we now >>>>>>> have >>>>>>> with the Python bindings. The advantage to making Rust bindings part of >>>>>>> the >>>>>>> main build would be that it could be built and tested as part of our CI, >>>>>>> versioned along with the rest of OIIO, and essentially never allowed to >>>>>>> break. Also, just like we would never accept a PR that added C++ >>>>>>> functionality without making sure the Python bindings kept up, we could >>>>>>> ensure that nothing is left out of the Rust bindings. While I can >>>>>>> appreciate the cleanliness and independence of it being a separate >>>>>>> project, >>>>>>> I can't help but think that it will be a neverending nightmare to try to >>>>>>> keep the bindings in sync with the main project. >>>>>>> >>>>>>> I don't know how automated it is to make Rust bindings for C (I know >>>>>>> it's a PITA for C++), but if making Rust bindings is substantially >>>>>>> easier >>>>>>> if you had minimal plain C wrappers for the major C++ classes, I'm sure >>>>>>> there would be a lot of happy consumers of that even outside the Rust >>>>>>> interest group. >>>>>>> >>>>>>> I haven't had time to try Rust myself for any programming project, >>>>>>> though I've followed it from afar and like the idea of helping that >>>>>>> community. TBH, the main thing that keeps me from spending any time on >>>>>>> Rust >>>>>>> is just that I can't contemplate the hassle of trying to program >>>>>>> without my >>>>>>> favourite libraries, and having OIIO (and its many utilities that I >>>>>>> reuse >>>>>>> in basically everything I write) available in Rust will substantially >>>>>>> lower >>>>>>> the bar for me to dabble in it more. >>>>>>> >>>>>>> -- lg >>>>>>> >>>>>>> >>>>>>> On Oct 18, 2020, at 4:12 PM, Scott Wilson <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Hey Anders, >>>>>>> >>>>>>> We were inspired by what you did, and also decided to see if we can >>>>>>> take this in a slightly different direction/ use cxx. If you're >>>>>>> interested >>>>>>> in discussing the wrapper more we can take it off the list. >>>>>>> >>>>>>> On Sun, Oct 18, 2020 at 3:35 PM Anders Langlands < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> I also have a rust binding here if you're interested: >>>>>>>> https://github.com/anderslanglands/oiio-rs >>>>>>>> >>>>>>>> On Sun, 18 Oct 2020 at 04:43, Scott Wilson <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Awesome, thank you very much! I'll try this out and see how badly >>>>>>>>> I break things. >>>>>>>>> >>>>>>>>> On Sat., Oct. 17, 2020, 1:02 a.m. Larry Gritz, <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> If you know the true legal extent of the memory allocation in >>>>>>>>>> which that data pointer is located (in this case, the beginning and >>>>>>>>>> ending >>>>>>>>>> of the vector, if you are passing a pointer to one of the elements >>>>>>>>>> of that >>>>>>>>>> vector), then I think you could certainly consider it an error if >>>>>>>>>> any of >>>>>>>>>> these addresses lay outside that buffer: >>>>>>>>>> >>>>>>>>>> data + xstride*width - 1 >>>>>>>>>> data + ystride*height - 1 >>>>>>>>>> data + ystride*(height - 1) + xstride*width - 1 >>>>>>>>>> data + zstride*depth >>>>>>>>>> data + zstride*(depth - 1) + ystride*height - 1 >>>>>>>>>> data + zstride*(depth - 1) + ystride*(height - 1) + >>>>>>>>>> xstride*width - 1 >>>>>>>>>> >>>>>>>>>> There may be a more succinct way to put that, but I think it >>>>>>>>>> covers all the cases of + and - strides. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Oct 17, 2020, at 12:42 AM, Scott Wilson <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Thanks! I guess to come from this at a different angle, let's say >>>>>>>>>> I'm doing something like this: >>>>>>>>>> >>>>>>>>>> std::vector<uint8_t> pixels(10*10*3*1); >>>>>>>>>> ImageInput.read_image(TypeDesc::UINT8, @pixels[0]) >>>>>>>>>> >>>>>>>>>> Would there be a case where I could pick a stride value that >>>>>>>>>> would fall outside the pixels vector? >>>>>>>>>> >>>>>>>>>> PS: Thanks! I'm working on this with a friend, and hope to have >>>>>>>>>> something released in the near future. >>>>>>>>>> >>>>>>>>>> On Fri., Oct. 16, 2020, 11:47 p.m. Larry Gritz, < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Oops, my math was wrong (in an unimportant detail): If you are >>>>>>>>>>> making a mosaic of 16x5 of these 10x10 images, it is 80 small >>>>>>>>>>> images you >>>>>>>>>>> are assembling in total, not 40. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Oct 16, 2020, at 11:43 PM, Larry Gritz <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> The strides don't describe the size of the image, they are the >>>>>>>>>>> spacing in memory of where you want the values to be placed upon >>>>>>>>>>> being read >>>>>>>>>>> (or taken from in order to write). There is no invalid set of >>>>>>>>>>> strides, >>>>>>>>>>> because the caller might want them to end up anywhere in memory. >>>>>>>>>>> >>>>>>>>>>> Or am I misunderstanding? >>>>>>>>>>> >>>>>>>>>>> For a fully "contiguous" memory buffer where you intend for >>>>>>>>>>> every plane, scanline, pixel, and channel immediately follows the >>>>>>>>>>> previous >>>>>>>>>>> one, then in our example the strides would be xstride=3, ystride=30, >>>>>>>>>>> zstride=300. (Though for a 2D image, the zstride is not used.) >>>>>>>>>>> >>>>>>>>>>> Here's an example of where you might have a stride range that is >>>>>>>>>>> wildly outside this: Let's say that you have 40 of these 10 x 10 x >>>>>>>>>>> 3 x >>>>>>>>>>> uint8 image files and you are trying to read them in and assemble >>>>>>>>>>> them into >>>>>>>>>>> a single RGBA mosaic image of 16x5 x 4 x uint8 (the additional >>>>>>>>>>> channel is >>>>>>>>>>> alpha, which you will separately fill in as 1.0 [or 255 uint8] >>>>>>>>>>> because it's >>>>>>>>>>> not in your RGB files). Here's a cartoon to illustrate this: >>>>>>>>>>> >>>>>>>>>>> +-----------------------------------------+ >>>>>>>>>>> | | | | | | | | | | | | | | | | | | | | | | >>>>>>>>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| >>>>>>>>>>> | | | | | | | | | | | | | | | | | | | | | | >>>>>>>>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| >>>>>>>>>>> | | | | | | | | | |X| | | | | | | | | | | | >>>>>>>>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| >>>>>>>>>>> | | | | | | | | | | | | | | | | | | | | | | >>>>>>>>>>> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| >>>>>>>>>>> | | | | | | | | | | | | | | | | | | | | | | >>>>>>>>>>> +-----------------------------------------+ >>>>>>>>>>> >>>>>>>>>>> Each of my little grid cells is a 10x10 image. But that 10x10 >>>>>>>>>>> image denoted by the "X" needs to be placed in memory in the right >>>>>>>>>>> portion >>>>>>>>>>> of the 16x10 x 5x10 mosaic. So what are the strides we use for the >>>>>>>>>>> read? >>>>>>>>>>> Well, the xstride is 4 because we're making room for an alpha >>>>>>>>>>> channel that >>>>>>>>>>> wasn't present in the file, the ystride is 640 (= 10*16*4), because >>>>>>>>>>> each >>>>>>>>>>> scanline of the little 10x10 image that you read needs to be placed >>>>>>>>>>> on the >>>>>>>>>>> proper scanline of the 160x50 mosaic you are assembling in memory. >>>>>>>>>>> >>>>>>>>>>> -- lg >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> P.S. Woo-hoo for making a Rust wrapper. I think that's a totally >>>>>>>>>>> great thing. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Oct 16, 2020, at 10:46 PM, Scott Wilson < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> I'm experimenting with a Rust wrapper for OIIO, and had some >>>>>>>>>>> questions about the stride. >>>>>>>>>>> >>>>>>>>>>> Let's say I have an image that is 10x10 pixels, and 3 channels, >>>>>>>>>>> and 1 byte per channel. What strides would be invalid for that >>>>>>>>>>> image? I'm >>>>>>>>>>> guessing that anything between -10 * 10 * 3 * 1 to 10 * 10 * 3 * 1 >>>>>>>>>>> and the >>>>>>>>>>> AutoStride would be valid, and everything else may try to access >>>>>>>>>>> memory >>>>>>>>>>> that isn't initialized. Is this assumption correct, or am I missing >>>>>>>>>>> something? >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Oiio-dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Larry Gritz >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Oiio-dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Larry Gritz >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Oiio-dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Oiio-dev mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Larry Gritz >>>>>>>>>> [email protected] >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Oiio-dev mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Oiio-dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Oiio-dev mailing list >>>>>>>> [email protected] >>>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Oiio-dev mailing list >>>>>>> [email protected] >>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Larry Gritz >>>>>>> [email protected] >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Oiio-dev mailing list >>>>>>> [email protected] >>>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>>> >>>>>> _______________________________________________ >>>>>> Oiio-dev mailing list >>>>>> [email protected] >>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>> >>>>> _______________________________________________ >>>>> Oiio-dev mailing list >>>>> [email protected] >>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>> >>>> _______________________________________________ >>>> Oiio-dev mailing list >>>> [email protected] >>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>> >>> _______________________________________________ >>> Oiio-dev mailing list >>> [email protected] >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>> >> _______________________________________________ >> Oiio-dev mailing list >> [email protected] >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > > -- > Larry Gritz > [email protected] > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
