Re: How to do alligned allocation?
On Saturday, 1 October 2022 at 01:37:00 UTC, Steven Schveighoffer wrote: On 9/30/22 11:57 AM, Quirin Schroll wrote: Also, is the alignment of any type guaranteed to be a power of 2? In practice, it's not necessarily a power of 2, but it's *at least* 16 bytes. **Types** always require some power of 2 alignment (on any sensible platform, anyway), and it is usually *less* than 16 bytes - typically `size_t.sizeof`. The fact that the current GC implementation apparently has a minimum block size of 16 bytes, and that minimum size blocks are always size-aligned, is not guaranteed by the public API and *should not be* when requesting memory for something that the type system says only requires an alignment of `void.alignof == 1`. D and C both have formal ways to communicate alignment requirements to the allocator; people should use them and not constrain all future D GC development to conform to undocumented details of the current implementation. In general there are very few types (maybe vectors?) that need alignment more than 16 bytes. 256 bit SIMD (AVX/AVX2) and 512 bit SIMD (AVX512) `__vector`s should be `.sizeof` aligned (32 and 64 bytes, respectively). Memory used for inter-thread communication (such as mutexes) may perform significantly better if cache line aligned (typically 64 bytes, but CPU dependent). I don't know any other examples off the top of my head. The list of bit sizes is currently here: I'm pretty sure those are in **bytes** not **bits**. https://github.com/dlang/dmd/blob/82870e890f6f0e0dca3e8f0032a7819416319124/druntime/src/core/internal/gc/impl/conservative/gc.d#L1392-L1414 That's not a list of alignments, it is block sizes for some GC memory pools. The alignment of each block depends on the alignment of its pool, not just its size. It's not immediately obvious from the context, but I suspect the pools are actually page aligned, which would mean that the non power of 2 sized blocks are **not** consistently aligned to their own sizes. Regardless, it's not part of the public API, so it could change without warning.
Re: Interfacing with Rust
extern(C++)？ Why do you think Rust export C++ linkage? And why do you think Rust export some kind of OO object model linkage? Do it in plain C style, you may make it work. As said, check how it's done in: https://code.dlang.org/packages/rust_interop_d
Re: Interfacing with Rust
On Friday, 30 September 2022 at 06:25:33 UTC, Imperatorn wrote:
On Friday, 30 September 2022 at 00:18:42 UTC, Ruby The Roobster
wrote:
On Thursday, 29 September 2022 at 16:07:59 UTC, mw wrote:
On Thursday, 29 September 2022 at 16:02:43 UTC, Ruby The
Roobster wrote:
Is there any way one can interface with Rust, such as with a
struct, or a function?
I know that rust has an extern keyword, but I can't get it
to work.
https://code.dlang.org/packages/rust_interop_d
Read the notes on memory management:
Only pass pointers as u64 as value type.
This isn't the issue. I can't interface anything, period.
Show an example of exactly what you're trying to do. Maybe it's
some detail
lib.rs:
```rs
#[derive(Debug)]
struct Rectangle {
width: u32,
height: u32,
}
impl Rectangle {
extern "cdecl" fn area() -> u32 {
self.width * self.height
}
extern "cdecl" fn can_hold(, other: ) ->
bool {
self.width > other.width && self.height > other.height
}
}
```
main.d:
```d
struct Rectangle
{
uint width;
uint hight;
extern(C++) uint area();
extern(C++) bool can_hold(Rectangle rhs)
}
void main()
{
auto rect1 = Rectangle(1,1);
auto rect2 = Rectangle(0,0);
assert(rect1.area == 1 && rect2.area == 0 &&
rect1.canHold(rect2));
}
```
lib.rs was built as a "staticlib" using rustc, and the command I
used was:
ldc2 -m64 main.d lib.lib
The output:
```
main.obj : error LNK2019: unresolved external symbol "public:
unsigned int __cdecl Rectangle::area(void)"
(?area@Rectangle@@QEAAIXZ) referenced in function _Dmain
main.obj : error LNK2019: unresolved external symbol "public:
bool __cdecl Rectangle::can_hold(struct Rectangle)"
(?can_hold@Rectangle@@QEAA_NU1@@Z) referenced in function _Dmain
main.exe : fatal error LNK1120: 2 unresolved externals
Error: C:\Program Files (x86)\Microsoft Visual
Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\HostX64\x64\link.exe failed with status: 1120
```
Also, is it normal for the .lib file to be 11 megabytes?
Re: How to do alligned allocation?
On 9/30/22 11:57 AM, Quirin Schroll wrote: When I do `new void[](n)`, is that buffer allocated with an alignment of 1 or what are the guarantees? How can I set an alignment? Also, is the alignment of any type guaranteed to be a power of 2? In practice, it's not necessarily a power of 2, but it's *at least* 16 bytes. In general there are very few types (maybe vectors?) that need alignment more than 16 bytes. The list of bit sizes is currently here: https://github.com/dlang/dmd/blob/82870e890f6f0e0dca3e8f0032a7819416319124/druntime/src/core/internal/gc/impl/conservative/gc.d#L1392-L1414 -Steve
Re: How can I get the "owner" of a method?
On 9/30/22 14:11, solidstate1991 wrote:
> extern (C) public int registerDDelegate(alias Func,
That can be a lambda that takes ClassType. I wrote the following along
with place holder as I assumed them to be:
class C {
string s;
void foo() {
import std.stdio : writeln;
writeln("Look ma: It was ", s, " but I am using it in D!");
}
}
auto luaGetFromIndex(C)() {
auto c = new C();
c.s = "created in Lua";
return c;
}
extern (C)
int registerDDelegate(alias Func)() {
import std.traits : Parameters;
alias ClassType = Parameters!Func[0];
auto c = luaGetFromIndex!ClassType();
Func(c);
return 0;
}
void main() {
// Here: The lambda takes a C:
registerDDelegate!((C c) => c.foo);
}
Ali
Re: How to do alligned allocation?
On Saturday, 1 October 2022 at 00:32:28 UTC, tsbockman wrote:
alias Chunk = AliasSeq!(ubyte, ushort, uint, ulong,
Chunk16)[shift];
Oops, I forgot that `ulong.alignof` is platform dependent. It's
probably best to just go ahead and explicitly specify the
alignment for all `Chunk` types:
```D
private template Aligned(size_t alignment)
if(1 <= alignment && isPowerOf2(alignment))
{
enum int shift = bsr(alignment);
enum size_t mask = alignment - 1;
static if(alignment <= 16) {
enum chunkShift = shift, chunkMask = mask;
align(alignment) struct Chunk {
void[alignment] data;
}
} else {
enum chunkShift = Aligned!(16).shift, chunkMask =
Aligned!(16).mask;
alias Chunk = Aligned!(16).Chunk;
}
}
```
(This also eliminates the `std.meta : AliasSeq` dependency.)
Re: How to do alligned allocation?
On Friday, 30 September 2022 at 15:57:22 UTC, Quirin Schroll
wrote:
When I do `new void[](n)`, is that buffer allocated with an
alignment of 1 or what are the guarantees?
It is guaranteed an alignment of at least 1 because `void.alignof
== 1` (and because that is the lowest possible integer
alignment). When I last checked, `new T` guaranteed a minimum
alignment of `min(T.alignof, 16)`, meaning that all basic scalar
types (`int`, `double`, pointers, etc.), and SIMD `__vector`s up
to 128 bits will be correctly aligned, while 256 bit (for
example, AVX's `__vector(double[4])`) and 512 bit (AVX512) types
might not be.
Arrays and aggregate types (`struct`s and `class`es) by default
use the maximum alignment required by any of their elements or
fields (including hidden fields, like `__vptr` for `class`es).
This can be overridden manually using the `align` attribute,
which must be applied to the aggregate type as a whole. (Applying
`align` to an individual field does something else.)
How can I set an alignment?
If the desired alignment is `<= 16`, you can specify a type with
that `.alignof`.
However, if you may need higher alignment than the maximum
guaranteed to be available from the allocator, or you are not
writing strongly typed code to begin with, as implied by your use
of `void[]`, you can just align the allocation yourself:
```D
void[] newAligned(const(size_t) alignment)(const(size_t) size)
pure @trusted nothrow
if(1 <= alignment && isPowerOf2(alignment))
{
enum alignMask = alignment - 1;
void[] ret = new void[size + alignMask];
const misalign = (cast(size_t) ret.ptr) & alignMask;
const offset = (alignment - misalign) & alignMask;
ret = ret[offset .. offset + size];
return ret;
}
```
However, aligning memory outside of the allocator itself like
this does waste up to `alignment - 1` bytes per allocation, so
it's best to use as much of the allocator's internal alignment
capability as possible:
```D
import core.bitop : bsr;
import std.math : isPowerOf2;
import std.meta : AliasSeq;
void[] newAligned(const(size_t) alignment)(const(size_t) size)
pure @trusted nothrow
if(1 <= alignment && isPowerOf2(alignment))
{
alias Aligned = .Aligned!alignment;
void[] ret = new Aligned.Chunk[(size + Aligned.mask) >>
Aligned.chunkShift];
static if(Aligned.Chunk.alignof == alignment)
enum size_t offset = 0;
else {
const misalign = (cast(size_t) ret.ptr) & Aligned.mask;
const offset = (alignment - misalign) & Aligned.mask;
}
ret = ret[offset .. offset + size];
return ret;
}
private {
align(16) struct Chunk16 {
void[16] data;
}
template Aligned(size_t alignment)
if(1 <= alignment && isPowerOf2(alignment))
{
enum int shift = bsr(alignment);
enum size_t mask = alignment - 1;
static if(alignment <= 16) {
enum chunkShift = shift, chunkMask = mask;
alias Chunk = AliasSeq!(ubyte, ushort, uint, ulong,
Chunk16)[shift];
} else {
enum chunkShift = Aligned!(16).shift, chunkMask =
Aligned!(16).mask;
alias Chunk = Aligned!(16).Chunk;
}
}
}
@safe unittest {
static immutable(size_t[]) alignments =
[ 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024 ];
static immutable(size_t[]) sizes =
[ 9, 31, 4, 57, 369, 3358 ];
foreach(size; sizes) {
static foreach(alignment; alignments) { {
void[] memory = newAligned!alignment(size);
assert(memory.length == size);
assert((cast(size_t) &(memory[0])) % alignment == 0);
} }
}
}
```
Also, is the alignment of any type guaranteed to be a power of
2?
In practice, yes.
On Friday, 30 September 2022 at 16:23:00 UTC, mw wrote:
https://dlang.org/library/core/stdc/stdlib/aligned_alloc.html
It's the C func, so check C lib doc.
https://en.cppreference.com/w/c/memory/aligned_alloc
Note that common implementations place arbitrary restrictions on
the alignments and sizes accepted by `aligned_alloc`, so to
support the general case you would still need a wrapper function
like the one I provided above.
(If this all seems overly complicated, that's because it is. I
have no idea why allocators don't just build in the logic above;
it's extremely simple compared to the rest of what a good
general-purpose heap allocator does.)
Re: Interfacing with basic C++ class
On Thursday, 29 September 2022 at 12:49:06 UTC, Riccardo M wrote:
On Thursday, 29 September 2022 at 11:13:15 UTC, Ogi wrote:
So it turns out that D's structs are a much better match for
C++'s classes in this case. But why is this? Can you elaborate?
It must have to do with the fact that D structs are passed by
value?
In C++, class and struct are basically the same thing (AFAIK the
only difference is that struct members are public by default). In
D, they are two very different things: classes are reference
types — structs are value types, classes support inheritance —
structs only support `alias this`, and so on.
When interfacing to C++, disregard the keyword and look at the
implementation instead. If all its member functions are
non-virtual, map it to struct. Otherwise map it to class. If it
defines at least one pure virtual member function, map it to
abstract class. If all its member functions are either pure
virtual or non-virtual and it contains no fields, map it to
interface. Sounds complicated? Well, that’s because C++ is
complicated.
However the 'add' function only links correctly if C++ has the
function body defined outside of its class.
```
// C++
int MyClass::add(int asd) {
return field + asd;
}
```
If the function is defined inside its class, it is an undefined
reference at link time. Once again I ask for clarifications and
workarounds, if possible.
In C++, member functions defined inside its class are called
*inline* member functions. In contrast to normal functions which
must be defined once and only once in your program, inline
functions must be defined in every translation unit that uses
them. Let’s replicate your linking error in C++:
```C++
//c.h
class C {
public:
int foo() {
return 42;
}
void bar();
};
```
```C++
//c.cpp
#include "c.h"
void C::bar() { /* ... */ }
```
```C++
//main.cpp
#include
//Let’s see what happens if we forget `C::foo` definition:
class C {
public:
int foo();
void bar();
};
int main() {
auto c = new C();
printf("%d\n", c->foo());
c->bar();
return 0;
}
```
```
$ clang++ -c c.cpp
$ clang++ -c main.cpp
$ clang++ main.o c.o
main.o : error LNK2019: unresolved external symbol "public: int
__cdecl C::foo(void)" (?foo@C@@QEAAHXZ) referenced in function
main
a.exe : fatal error LNK1120: 1 unresolved externals
clang++: error: linker command failed with exit code 1120 (use -v
to see invocation)
```
Copying `C::foo` definition to `main.cpp` will fix this. Of
course, we could just include `c.h`.
Same goes for D. `MyClass.add` must be defined in your D module.
Re: How can I get the "owner" of a method?
On Friday, 30 September 2022 at 21:11:48 UTC, solidstate1991
wrote:
Let's say I have a class or an interface with a function, and I
want to pass that to a template to generate another function
(in this case one that passes it to a scripting engine, and
allows it to be called from there).
Maybe this will help you:
```d
template Bar(T)
{
void fun() {}
}
class Foo(T)
{
mixin Bar!T b;
alias fun = b.fun;
void test()
{
fun();
}
}
```
SDB@79
How can I get the "owner" of a method?
Let's say I have a class or an interface with a function, and I
want to pass that to a template to generate another function (in
this case one that passes it to a scripting engine, and allows it
to be called from there).
Currently I have the following issues:
1. I have to pass the class/interface type to the template, and I
cannot find anything about how to solve this issue.
2. It also doesn't work at the moment, as it wants to call the
member `Func` (name of the template argument) rather than the
Function described as such. (There's also probably in
std.functional I don't know about)
Code:
```d
extern (C) public int registerDDelegate(alias Func,
ClassType)(lua_State* state) nothrow
if(isSomeFunction!(Func)) {
import std.traits:Parameters, ReturnType;
Parameters!Func params;
int stackCounter = 0;
stackCounter--;
ClassType c = luaGetFromIndex!ClassType(state, stackCounter);
try {
foreach_reverse(ref param; params) {
stackCounter--;
param = luaGetFromIndex!(typeof(param))(state,
stackCounter);
}
} catch (Exception e) {
luaL_error(state, "Argument type mismatch with D functions!");
}
try {
static if(is(ReturnType!Func == void)) {
c.Func(params);
return 0;
} else static if(is(ReturnType!Func == struct)) {
auto retVal = c.Func(params);
static foreach (key ; retVal.tupleof) {
LuaVar(key).pushToLuaState(state);
}
return cast(int)retVal.tupleof.length;
} else {
LuaVar(c.Func(params)).pushToLuaState(state);
return 1;
}
} catch (Exception e) {
//luaPushVar(L, null);
lastLuaToDException = e;
try {
luaL_error(state, ("A D function threw:
"~e.toString~"!\0").ptr);
} catch(Exception e) {
luaL_error(state, "D threw when stringifying
exception!");
}
return 1;
}
}
```
Re: How to do alligned allocation?
On Friday, 30 September 2022 at 16:23:00 UTC, mw wrote: On Friday, 30 September 2022 at 15:57:22 UTC, Quirin Schroll wrote: When I do `new void[](n)`, is that buffer allocated with an alignment of 1 or what are the guarantees? How can I set an alignment? Also, is the alignment of any type guaranteed to be a power of 2? https://dlang.org/library/core/stdc/stdlib/aligned_alloc.html It's the C func, so check C lib doc. and then use emplace on the C-alloc-ed memory.
Re: How to do alligned allocation?
On Friday, 30 September 2022 at 15:57:22 UTC, Quirin Schroll wrote: When I do `new void[](n)`, is that buffer allocated with an alignment of 1 or what are the guarantees? How can I set an alignment? Also, is the alignment of any type guaranteed to be a power of 2? https://dlang.org/library/core/stdc/stdlib/aligned_alloc.html It's the C func, so check C lib doc.
How to do alligned allocation?
When I do `new void[](n)`, is that buffer allocated with an alignment of 1 or what are the guarantees? How can I set an alignment? Also, is the alignment of any type guaranteed to be a power of 2?
What are best practices around toString?
Dear Dlang experts, up until now I was perfectly happy with implementing `(override) string toString() const` or something to get nicely formatted (mostly debug) output for my structs, classes and exceptions. But recently I stumbled upon https://wiki.dlang.org/Defining_custom_print_format_specifiers and additionally https://github.com/dlang/dmd/blob/4ff1eec2ce7d990dcd58e5b641ef3d0a1676b9bb/druntime/src/object.d#L2637 which at first sight is great, because it provides the same customization of an objects representation with less memory allocations. When grepping through phobos, there are a bunch of "different" signatures implemented for this, e.g. ```d ... phobos/std/typecons.d:void toString(DG)(scope DG sink) const ... phobos/std/typecons.d:void toString(DG, Char)(scope DG sink, scope const ref FormatSpec!Char fmt) const ... phobos/std/typecons.d:void toString()(scope void delegate(const(char)[]) sink, scope const ref FormatSpec!char fmt) ... phobos/std/sumtype.d:void toString(this This, Sink, Char)(ref Sink sink, const ref FormatSpec!Char fmt); ... ``` to just show a few. Furthermore, when one works with instances of struct, objects or exceptions a `aInstance.toString()` does not "work" when one only implements the sink interface (which is to be expected), whereas a `std.conv.to!string` or a formatted write with `%s` always works (no matter what was used to implement the toString). So I wonder, what is best practice in the community and would it make sense to add something to dscanner, that "warns" on usages of `aInstance.toString()`? Kind regards, Christian
Re: Interfacing with Rust
On Friday, 30 September 2022 at 00:18:42 UTC, Ruby The Roobster wrote: On Thursday, 29 September 2022 at 16:07:59 UTC, mw wrote: On Thursday, 29 September 2022 at 16:02:43 UTC, Ruby The Roobster wrote: Is there any way one can interface with Rust, such as with a struct, or a function? I know that rust has an extern keyword, but I can't get it to work. https://code.dlang.org/packages/rust_interop_d Read the notes on memory management: Only pass pointers as u64 as value type. This isn't the issue. I can't interface anything, period. Show an example of exactly what you're trying to do. Maybe it's some detail
