Re: gamut v0.0.7 ask for what you want
On Thursday, 11 August 2022 at 11:06:43 UTC, wjoe wrote:
I imagined you could allocate internal buffers for
encoding/decoding on the stack but your reply suggests
otherwise.
Yes.
For example, the QOI-10b codec needs an pallete of 256 16-bit
RGBA, that's 2 kb. Is that portable? There is probably a size at
which it isn't that portable anymore.
It also needs two scanlines of 16-bit RGBA, that is O(n) with the
image width, so can't go to the stack.
Now this encoder state is allocated at the end of the encoded
pixels, since it would be too large for a portable stack.
(Regular QOI can be all on the stack though.)
I mean, the primary thing you want is performance, complete
control over the memory is just a proxy for that.
However shouldn't a single function call back be enough?
Something like
``` D
void[] need_more_ram(size_t amount, void[] old_chunk, void*
user)
{
MyAllocator* a = cast(MyAllocator*)user;
void[] result = a.alloc(amount); // MyAllocator would return
an empty slice if amount == 0
if (amount && chunk.length) result[0..chunk.length] = chunk;
// it is assumed that on re-allocation amount > chunk.length
a.free(chunk.ptr);
return result;
}
```
Indeed, that would quite fitting as an API.
Re: gamut v0.0.7 ask for what you want
On Tuesday, 9 August 2022 at 22:02:37 UTC, Guillaume Piolat wrote:
On Monday, 8 August 2022 at 16:07:54 UTC, wjoe wrote:
your lib would not just be @nogc but @no_allocation.
All image decoders in gamut need to malloc more than just for
pixel data.
Even STB allocates for format conversion, zlib buffers, 16-bit
<-> 8-bit, etc. it's not just pixel data.
Single allocation pessimizes the size a lot because since you
haven't encoded yet, you need to prepare a buffer for a large
worst-case.
I imagined you could allocate internal buffers for
encoding/decoding on the stack but your reply suggests otherwise.
However shouldn't a single function call back be enough?
Something like
``` D
void[] need_more_ram(size_t amount, void[] old_chunk, void* user)
{
MyAllocator* a = cast(MyAllocator*)user;
void[] result = a.alloc(amount); // MyAllocator would return an
empty slice if amount == 0
if (amount && chunk.length) result[0..chunk.length] = chunk; //
it is assumed that on re-allocation amount > chunk.length
a.free(chunk.ptr);
return result;
}
```
Re: gamut v0.0.7 ask for what you want
On Monday, 8 August 2022 at 16:07:54 UTC, wjoe wrote: your lib would not just be @nogc but @no_allocation. All image decoders in gamut need to malloc more than just for pixel data. Even STB allocates for format conversion, zlib buffers, 16-bit <-> 8-bit, etc. it's not just pixel data. Single allocation pessimizes the size a lot because since you haven't encoded yet, you need to prepare a buffer for a large worst-case.
Re: gamut v0.0.7 ask for what you want
On Friday, 29 July 2022 at 10:59:02 UTC, Guillaume Piolat wrote: Ask for what you want... Since it's super easy to calculate the amount of memory required to hold the decompressed data... If you'd add a buffer parameter (akin to std.stdio.File.rawRead/Write), a slice to allocated memory that holds the decompressed data, people could use their allocator of choice to allocate the buffer and your lib would not just be @nogc but @no_allocation.
Re: gamut v0.0.7 ask for what you want
On Friday, 29 July 2022 at 14:28:55 UTC, ryuukk_ wrote: One suggestion, have a little struct with function pointers for malloc/free/realloc, so that we can plug our own allocator Hello, thanks for the heads-up! A few question about your use case (I believe you are using WebASM), to better understand. - Is it just for image pixel data, or all that could be allocated? - Are you OK with a global allocator struct that you assign in an initialization call to the library? Or you want each image to know about its own allocator potentially. - Are you asking that it is WebASM compatible? I'm not sure how feasible that is, because of the amount of core.stdc that is used.
Re: gamut v0.0.7 ask for what you want
That's a very cool library, i might ditch stb_image for yours, thanks for sharing! I went ahead and added it to the list of libraries that supports QOI file format here: https://github.com/phoboslab/qoi/pull/235 One suggestion, have a little struct with function pointers for malloc/free/realloc, so that we can plug our own allocator
gamut v0.0.7 ask for what you want
Using D and images I ended up with a problem. The problem was that parts of my code wanted to decode just specific image metadata, other just pixels. Others were interested in 10-bit, and others in this or that format. Finally, some were concerned about aligned layout and others just wanted "dense" pixels without gap between rows. The goal of Gamut is to cater to a reasonably large set of needs with D and images. I'll be at Dconf if you want to **ask for your particular use case.** The library is not super usable yet, it's still in the design phase. **Features:** - monomorphic image type, as opposed to a templated type, for all pixel format types The problem is that templated image library ends up taking compile-time when used at scale. The Image type will also act as "view". - loading images: PNG (8-bit and 10-bit) / JPEG (baseline and progressive) / QOI - writing images: PNG (8-bit) / JPEG (baseline) / BC7 in DDS / QOI - `nothrow @nogc`, all structs, no exceptions, standalone except `intel-intrinsics`. - (near future): control over the image layout for easy SIMD access. For example, you can ask for: * 64-bytes aligned row of pixels (for aligned SIMD access), * or a border of 2 pixels around the data (for resampling), * trailing pixels after each row (for unaligned SIMD access), * and that each row has a multiple of 4x pixels (for SIMD) - API inspired by FreeImage. Gamut has support for an evolving "QOIX" format that mostly improves upon QOI. Can be interesting if you have lots of PNG to distribute and want more compression than QOI. In the future I'd like to extend QOIX to support 10-bit. DUB: https://code.dlang.org/packages/gamut GitHub: https://github.com/AuburnSounds/gamut
