Re: binary search
On Monday, 7 December 2020 at 06:24:27 UTC, drug wrote: Phobos provides this by SortedRange: https://dlang.org/phobos/std_range.html#.SortedRange Example of usage: https://run.dlang.io/is/WW2bn0 Thanks! :-)
Re: binary search
Phobos provides this by SortedRange: https://dlang.org/phobos/std_range.html#.SortedRange Example of usage: https://run.dlang.io/is/WW2bn0
Re: how to access record[0] of a csv row? Error: no [] operator overload for type CsvRecord!(int, cast(Malformed)1, string, dchar)
On Monday, 7 December 2020 at 04:38:07 UTC, Paul Backus wrote: On Monday, 7 December 2020 at 04:03:05 UTC, mw wrote: So my next question: given N, how do I create a Tuple!(double, double, ... n-double) type programmatically? import std.meta: Repeat; alias NDoubles = Tuple!(Repeat!(N, double)); Note that N must be a compile-time constant, since the number of elements in a Tuple is fixed at compile time. Yes, I just realized that Tuple (upper T, compile time) and tuple (lower t) are different things. Now, how to convert it to a native array: double[] row = record; Error: cannot implicitly convert expression record of type Tuple!(double, double, double, ..., double) to double[] (I know for tuple, we can do: double[] arr = [record];)
Re: Request assistance initializing struct instance at global scope
On Monday, 7 December 2020 at 05:28:41 UTC, user1234 wrote:
On Monday, 7 December 2020 at 04:13:16 UTC, Andrew Edwards
wrote:
Given:
===
extern(C):
char*[] hldr;
enum I = (1<<0);
struct S { char* ft; char** fm; int f; }
void main(){}
===
How do I initialize an instance of S at global scope?
You cant. At the global scope the initializers must be runnable
at compile time, i.e using CTFE. I've tried to simplify what
would be required:
---
extern(C):
char*[] hldr;
enum I = (1<<0);
struct S { char* ft; char** fm; int f; }
void main(){}
enum char[8] FirstLevel = ['0'];
enum DoubleLevel = &FirstLevel[0]; // here
S[] s = [
S(
FirstLevel.ptr,
DoubleLevel, // error is here actually,
interesting
0),
];
---
D is not able of that :
/tmp/temp_7F835402B0F0.d:9:28: Error: cannot use non-constant
CTFE pointer in an initializer `&['0', '\xff', '\xff', '\xff',
'\xff', '\xff', '\xff', '\xff'][0]`
sorry my example was bad, better one
---
extern(C):
char*[] hldr;
enum I = (1<<0);
struct S { char* ft; char** fm; int f; }
void main(){}
enum char[8] Stuff = ['0'];
enum FirstLevel = Stuff.ptr; // &Suffn so char* OK
S[] s = [
S(
FirstLevel, // char* OK
&FirstLevel, // error here char** NG at CTFE
0),
];
---
Re: Request assistance initializing struct instance at global scope
On Monday, 7 December 2020 at 04:13:16 UTC, Andrew Edwards wrote:
Given:
===
extern(C):
char*[] hldr;
enum I = (1<<0);
struct S { char* ft; char** fm; int f; }
void main(){}
===
How do I initialize an instance of S at global scope?
You cant. At the global scope the initializers must be runnable
at compile time, i.e using CTFE. I've tried to simplify what
would be required:
---
extern(C):
char*[] hldr;
enum I = (1<<0);
struct S { char* ft; char** fm; int f; }
void main(){}
enum char[8] FirstLevel = ['0'];
enum DoubleLevel = &FirstLevel[0]; // here
S[] s = [
S(
FirstLevel.ptr,
DoubleLevel, // error is here actually,
interesting
0),
];
---
D is not able of that :
/tmp/temp_7F835402B0F0.d:9:28: Error: cannot use non-constant
CTFE pointer in an initializer `&['0', '\xff', '\xff', '\xff',
'\xff', '\xff', '\xff', '\xff'][0]`
Re: how to access record[0] of a csv row? Error: no [] operator overload for type CsvRecord!(int, cast(Malformed)1, string, dchar)
On Monday, 7 December 2020 at 04:03:05 UTC, mw wrote: So my next question: given N, how do I create a Tuple!(double, double, ... n-double) type programmatically? import std.meta: Repeat; alias NDoubles = Tuple!(Repeat!(N, double)); Note that N must be a compile-time constant, since the number of elements in a Tuple is fixed at compile time.
Request assistance initializing struct instance at global scope
Given:
===
extern(C):
char*[] hldr;
enum I = (1<<0);
struct S { char* ft; char** fm; int f; }
void main(){}
===
How do I initialize an instance of S at global scope?
// Not sure how to do this... so try to keep as close to original as
possible
// Nope, does not work
S[] s = [
{
cast(char*)"c",
&hldr[0],
I
},
]
// Error: static variable C_HL_extensions cannot be read at compile time
S[] s = [ S(cast(char*)"c", &hldr[0], I) ];
// Error Deprecation: static constructor can only be of D linkage
S[] s;
static this() {
s = [ S(cast(char*)"c", &hldr[0], I) ];
}
Thanks,
Andrew
binary search
We have: // sorted values size_t lines = [20, 1755, 1756, 1757, 1798, 1824, 1825, 1839, 1840]; size_t search = 21; Goal: // Fast find index of the '21' in ordered array 'lines' auto found = lines.binarySearch( 20 ); // 0 - index auto low = lines.binarySearchLow( 21 ); // 0 - near lowest index Where is the implementation of "binary search", please?
Re: how to access record[0] of a csv row? Error: no [] operator overload for type CsvRecord!(int, cast(Malformed)1, string, dchar)
On Monday, 7 December 2020 at 03:51:02 UTC, Paul Backus wrote:
On Monday, 7 December 2020 at 02:25:23 UTC, mw wrote:
onlineapp.d(8): Error: no [] operator overload for type
CsvRecord!(int, cast(Malformed)1, string, dchar)
should `r`'s type be integer array? and how do I access each
elelment of the row?
Thanks.
The docs [1] say that csvReader returns an input range, not an
array. Input ranges don't support indexing, only iteration
(e.g. with `foreach`). If you want an array, you will have to
use `std.array.array` to create one.
[1] http://phobos.dpldocs.info/std.csv.csvReader.1.html
Thanks.
I saw the 1st example on this page:
https://dlang.org/phobos/std_csv.html
foreach (record; csvReader!(Tuple!(string, string, int))(text))
{
writefln("%s works as a %s and earns $%d per year",
record[0], record[1], record[2]);
}
So my next question: given N, how do I create a Tuple!(double,
double, ... n-double) type programmatically?
Re: how to access record[0] of a csv row? Error: no [] operator overload for type CsvRecord!(int, cast(Malformed)1, string, dchar)
On Monday, 7 December 2020 at 02:25:23 UTC, mw wrote: onlineapp.d(8): Error: no [] operator overload for type CsvRecord!(int, cast(Malformed)1, string, dchar) should `r`'s type be integer array? and how do I access each elelment of the row? Thanks. The docs [1] say that csvReader returns an input range, not an array. Input ranges don't support indexing, only iteration (e.g. with `foreach`). If you want an array, you will have to use `std.array.array` to create one. [1] http://phobos.dpldocs.info/std.csv.csvReader.1.html
how to access record[0] of a csv row? Error: no [] operator overload for type CsvRecord!(int, cast(Malformed)1, string, dchar)
Hi,
I'm trying this code: i.e. print out the 1st element of each row
https://run.dlang.io/is/pG921a
void main()
{
import std.csv;
import std.stdio: write, writeln, writef, writefln;
import std.algorithm.comparison : equal;
string text = "76,26,22";
auto records = text.csvReader!int;
foreach(r; records) {writeln(r[0]);} // line 8
assert(records.equal!equal([
[76, 26, 22],
]));
}
but I got a compile error:
onlineapp.d(8): Error: no [] operator overload for type
CsvRecord!(int, cast(Malformed)1, string, dchar)
should `r`'s type be integer array? and how do I access each
elelment of the row?
Thanks.
Re: converting D's string to use with C API with unicode
On Sunday, 6 December 2020 at 05:04:35 UTC, tsbockman wrote: On Sunday, 6 December 2020 at 02:07:10 UTC, Jack wrote: On Saturday, 5 December 2020 at 23:31:31 UTC, tsbockman wrote: On Saturday, 5 December 2020 at 21:55:13 UTC, Jack wrote: [...] `ws.length` is the length in `wchar`s, but `memcpy` expects the size in bytes. (This is because it takes `void*` pointers as inputs, and so does not know the element type or its size.) How do I get this size in bytes from wstring? `ws.length * wchar.sizeof` should do it. `wstring` is just an alias for `immutable(wchar[])`, and the `length` property is the number of `wchar` elements in the slice. makes sense, thanks! solved the memory corruption
Re: converting D's string to use with C API with unicode
On Sunday, 6 December 2020 at 04:41:56 UTC, Виталий Фадеев wrote:
On Saturday, 5 December 2020 at 19:51:14 UTC, Jack wrote:
So in D I have a struct like this:
struct ProcessResult
{
string[] output;
bool ok;
}
in order to use output from C WINAPI with unicode, I need to
convert each string to wchar* so that i can acess it from C
with wchar_t*. Is that right or am I missing anything?
struct ProcessResult
{
string[] output;
bool ok;
C_ProcessResult toCResult()
{
auto r = C_ProcessResult();
r.ok = this.ok; // just copy, no conversion needed
foreach(s; this.output)
r.output ~= cast(wchar*)s.ptr;
return r;
}
}
version(Windows) extern(C) export
struct C_ProcessResult
{
wchar*[] output;
bool ok;
}
Drawing string via WinAPI. As example.
// UTF-16. wchar*
wstring ws = "Abc"w;
ExtTextOutW( hdc, x, y, 0, &clipRect, cast( LPCWSTR ) ws.ptr,
cast( uint ) ws.length, NULL );
// UTF-8. char*
string s = "Abc";
import std.utf : toUTF16;
string ws = s.toUTF16;
ExtTextOutW( hdc, x, y, 0, &clipRect, cast( LPCWSTR ) ws.ptr,
cast( uint ) ws.length, NULL );
// UTF-32. dchar*
dstring ds = "Abc"d;
import std.utf : toUTF16;
string ws = ds.toUTF16;
ExtTextOutW( hdc, x, y, 0, &clipRect, cast( LPCWSTR ) ws.ptr,
cast( uint ) ws.length, NULL );
One char.
// UTF-16. wchar
wchar wc = 'A';
ExtTextOutW( hdc, x, y, 0, &clipRect, cast( LPCWSTR ) &wc, 1,
NULL );
// UTF-32. dchar
dchar dc = 'A';
import std.utf : encode;
wchar[ 2 ] ws;
auto l = encode( ws, dc );
ExtTextOutW( hdc, x, y, 0, &clipRect, cast( LPCWSTR ) &ws.ptr,
cast( uint ) l, NULL );
//
// Font API
string face = "Arial";
LOGFONT lf;
import std.utf : toUTF16;
lf.lfFaceName[ 0 .. face.length ] = face.toUTF16;
HFONT hfont = CreateFontIndirect( &lf );
// Common case
LPWSTR toLPWSTR( string s ) nothrow // wchar_t*. UTF-16
{
import std.utf : toUTFz, toUTF16z, UTFException;
try{ return toUTFz!( LPWSTR )( s );
}
catch ( UTFException e ) { return cast( LPWSTR )
"ERR"w.ptr; }
catch ( Exception e ) { return cast( LPWSTR )
"ERR"w.ptr; }
}
didn't know about toUTFz!( LPWSTR ), I'll save everything else
for futher reference, I'll be using WINAPI for a while. Thanks
alias toLPWSTR toPWSTR;
alias toLPWSTR toLPOLESTR;
alias toLPWSTR toPOLESTR;
that's interesting, I didn't about using multiples alias.
// WinAPI
string windowName = "Abc";
HWND hwnd =
CreateWindowEx(
...
windowName.toLPWSTR,
...
);
Re: low-latency GC
On Sunday, 6 December 2020 at 17:28:52 UTC, Bruce Carneal wrote: D is good for systems level work but that's not all. I use it for projects where, in the past, I'd have split the work between two languages (Python and C/C++). I much prefer working with a single language that spans the problem space. My impression from reading the forums is that people either use D as a replacement for C/C++ or Python/numpy, so I think your experience covers the essential use case scenario that is dominating current D usage? Any improvements have to improve both dimension, I agree. If there is a way to extend D's reach with zero or a near-zero complexity increase as seen by the programmer, I believe we should (as/when resources allow of course). ARC involves a complexity increase, to some extent. Library authors have to think a bit more principled about when objects should be phased out and destructed, which I think tend to lead to better programs. It would also allow for faster precise collection. So it could be beneficial for all.
Re: low-latency GC
On Sunday, 6 December 2020 at 17:35:19 UTC, IGotD- wrote: Is automatic atomic reference counting a contender for kernels? In kernels you want to reduce the increase/decrease of the counts. Therefore the Rust approach using 'clone' is better unless there is some optimizer that can figure it out. Performance is important in kernels, you don't want the kernel to steal useful CPU time that otherwise should go to programs. I am not sure if kernel authors want autmatic memory management, they tend to want full control and transparency. Maybe something people who write device drivers would consider. In general I think that reference counting should be supported in D, not only implicitly but also under the hood with fat pointers. This will make D more attractive to performance applications. Another advantage is the reference counting can use malloc/free directly if needed without any complicated GC layer with associated meta data. Yes, I would like to see it, just expect that there will be protests when people realize that they have to make ownership explicit. Also tracing GC in a kernel is my opinion not desirable. For the reason I previously mentioned, you want to reduce meta data, you want reduce CPU time, you want to reduce fragmentation. Special allocators for structures are often used. Yes, an ARC solution should support fixed size allocators for types that are frequently allocated to get better speed.
Re: low-latency GC
On Sunday, 6 December 2020 at 15:44:32 UTC, Ola Fosheim Grøstad wrote: It was more a hypothetical, as read barriers are too expensive. But write barriers should be ok, so a single-threaded incremental collector could work well if D takes a principled stance on objects not being 'shared' not being handed over to other threads without pinning them in the GC. Maybe a better option for D than ARC, as it is closer to what people are used to. In kernel programming there are plenty of atomic reference counted objects. The reason is that is you have kernel that supports SMP you must have it because you don't really know which CPU is working with a structure at any given time. These are often manually reference counted objects, which can lead to memory leaking bugs but they are not that hard to find. Is automatic atomic reference counting a contender for kernels? In kernels you want to reduce the increase/decrease of the counts. Therefore the Rust approach using 'clone' is better unless there is some optimizer that can figure it out. Performance is important in kernels, you don't want the kernel to steal useful CPU time that otherwise should go to programs. In general I think that reference counting should be supported in D, not only implicitly but also under the hood with fat pointers. This will make D more attractive to performance applications. Another advantage is the reference counting can use malloc/free directly if needed without any complicated GC layer with associated meta data. Also tracing GC in a kernel is my opinion not desirable. For the reason I previously mentioned, you want to reduce meta data, you want reduce CPU time, you want to reduce fragmentation. Special allocators for structures are often used.
Re: low-latency GC
On Sunday, 6 December 2020 at 16:42:00 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 14:44:25 UTC, Paulo Pinto wrote: And while on the subject of low level programming in JVM or .NET. https://www.infoq.com/news/2020/12/net-5-runtime-improvements/ Didnt say anything about low level, only simd intrinsics, which isnt really low level? It also stated "When it came to something that is pure CPU raw computation doing nothing but number crunching, in general, you can still eke out better performance if you really focus on "pedal to the metal" with your C/C++ code." So you must make the familiar "ease-of-programming" vs "x% of performance" choice, where 'x' is presumably much smaller than earlier. So it is more of a Go contender, and Go is not a systems level language... Apples and oranges. D is good for systems level work but that's not all. I use it for projects where, in the past, I'd have split the work between two languages (Python and C/C++). I much prefer working with a single language that spans the problem space. If there is a way to extend D's reach with zero or a near-zero complexity increase as seen by the programmer, I believe we should (as/when resources allow of course).
Re: low-latency GC
On Sunday, 6 December 2020 at 14:44:25 UTC, Paulo Pinto wrote: And while on the subject of low level programming in JVM or .NET. https://www.infoq.com/news/2020/12/net-5-runtime-improvements/ Didnt say anything about low level, only simd intrinsics, which isnt really low level? It also stated "When it came to something that is pure CPU raw computation doing nothing but number crunching, in general, you can still eke out better performance if you really focus on "pedal to the metal" with your C/C++ code." So it is more of a Go contender, and Go is not a systems level language... Apples and oranges. As I already mentioned in another thread, rebooting the language to pull in imaginary crowds will only do more damage than good, while the ones deemed unusable by the same imaginary crowd just keep winning market share, slowly and steady, even if takes yet another couple of years. A fair number of people here are in that imaginary crowd. So, I guess it isnt imaginary...
Re: low-latency GC
On Sunday, 6 December 2020 at 14:11:41 UTC, Max Haughton wrote: On Sunday, 6 December 2020 at 11:35:17 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 11:27:39 UTC, Max Haughton wrote: [...] No, unique doesnt need indirection, neither does ARC, we put the ref count at a negative offset. shared_ptr is a fat pointer with the ref count as a separate object to support existing C libraries, and make weak_ptr easy to implement. But no need for indirection. [...] I think you need a new IR, but it does not have to be used for code gen, it can point back to the ast nodes that represent ARC pointer assignments. One could probably translate the one used in Rust, even. https://gcc.godbolt.org/z/bnbMeY If you pass something as a parameter then there may or may not be an extra reference involved. Not specific for smart pointers, but ARC optimization should take care of that.
Re: low-latency GC
On Sunday, 6 December 2020 at 14:45:21 UTC, Bruce Carneal wrote: Well, you could in theory avoid putting owning pointers on the stack/globals or require that they are registered as gc roots. Then you don't have to scan the stack. All you need then is write barriers. IIRC 'shared' with teeth? It was more a hypothetical, as read barriers are too expensive. But write barriers should be ok, so a single-threaded incremental collector could work well if D takes a principled stance on objects not being 'shared' not being handed over to other threads without pinning them in the GC. Maybe a better option for D than ARC, as it is closer to what people are used to.
Re: low-latency GC
On Sunday, 6 December 2020 at 08:59:49 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 08:36:49 UTC, Bruce Carneal wrote: Yes, but they don't allow low level programming. Go also freeze to sync threads this has a rather profound impact on code generation. They have spent a lot of effort on sync instructions in code gen to lower the latency AFAIK. So, much of the difficulty in bringing low-latency GC to dlang would be the large code gen changes required. If it is a really big effort then that is all we need to know. Not worth it until we can see a big payoff and have more resources. Well, you could in theory avoid putting owning pointers on the stack/globals or require that they are registered as gc roots. Then you don't have to scan the stack. All you need then is write barriers. IIRC 'shared' with teeth?
Re: low-latency GC
On Sunday, 6 December 2020 at 08:12:58 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 07:45:17 UTC, Bruce Carneal wrote: GCs scan memory, sure. Lots of variations. Not germane. Not a rationale. We need to freeze the threads when collecting stacks/globals. D is employed at multiple "levels". Whatever level you call it, Go and modern JVMs employ low latency GCs in multi-threaded environments. Some people would like to use D at that "level". Yes, but they don't allow low level programming. Go also freeze to sync threads this has a rather profound impact on code generation. They have spent a lot of effort on sync instructions in code gen to lower the latency AFAIK. They surely do. Looking forward to see D achieve the same performance level as .NET 5 is capable of, beating Google's own gRPC C++ implementation, only Rust implementation beats it. https://www.infoq.com/news/2020/12/aspnet-core-improvement-dotnet-5/ And while on the subject of low level programming in JVM or .NET. https://www.infoq.com/news/2020/12/net-5-runtime-improvements/ Many of the performance improvements in the HTTP/2 implementation are related to the reimplementation from unmanaged C++ code to managed C# code. Lander notes that there "still is this kind of idea that managed languages are not quite up to the task for some of those low-level super performance sensitive components, Rich Lander being one of the main .NET architects, and upcoming Java 16 features, http://openjdk.java.net/jeps/389 (JNI replacement), http://openjdk.java.net/jeps/393 (native memory management). As I already mentioned in another thread, rebooting the language to pull in imaginary crowds will only do more damage than good, while the ones deemed unusable by the same imaginary crowd just keep winning market share, slowly and steady, even if takes yet another couple of years.
Re: low-latency GC
On Sunday, 6 December 2020 at 11:35:17 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 11:27:39 UTC, Max Haughton wrote: [...] No, unique doesnt need indirection, neither does ARC, we put the ref count at a negative offset. shared_ptr is a fat pointer with the ref count as a separate object to support existing C libraries, and make weak_ptr easy to implement. But no need for indirection. [...] I think you need a new IR, but it does not have to be used for code gen, it can point back to the ast nodes that represent ARC pointer assignments. One could probably translate the one used in Rust, even. https://gcc.godbolt.org/z/bnbMeY
Re: low-latency GC
On Sunday, 6 December 2020 at 12:58:44 UTC, IGotD- wrote: I was thinking about how to deal with this in D and the question is if it would be better to be able to control move as default per type basis. This way we can implement Rust style reference counting without intruding too much on the rest of the language. The question is if we want this or if we should go for a fully automated approach where the programmer doesn't need to worry about 'clone'. I dont know, but I suspect that people that use D want something more high level than Rust? But I dont use Rust, so...
Re: low-latency GC
On Sunday, 6 December 2020 at 11:07:50 UTC, Ola Fosheim Grostad wrote: ARC can be done incrementally, we can do it as a library first and use a modified version existing GC for detecting failed borrows at runtime during testing. But all libraries that use owning pointers need ownership to be made explicit. A static borrow checker an ARC optimizer needs a high level IR though. A lot of work though. The Rust approach is interesting as it doesn't need an ARC optimizer. Everything is a move so no increase/decrease is done when doing that. Increase is done first when the programmer decides to 'clone' the reference. This inherently becomes optimized without any compiler support. However, this requires that the programmer inserts 'clone' when necessary so it isn't really automatic. I was thinking about how to deal with this in D and the question is if it would be better to be able to control move as default per type basis. This way we can implement Rust style reference counting without intruding too much on the rest of the language. The question is if we want this or if we should go for a fully automated approach where the programmer doesn't need to worry about 'clone'.
Re: low-latency GC
On Sunday, 6 December 2020 at 11:27:39 UTC, Max Haughton wrote: ARC with a library will have overhead unless the compiler/ABI is changed e.g. unique_ptr in C++ has an indirection. No, unique doesnt need indirection, neither does ARC, we put the ref count at a negative offset. shared_ptr is a fat pointer with the ref count as a separate object to support existing C libraries, and make weak_ptr easy to implement. But no need for indirection. The AST effectively is a high-level IR. Not a good one, but good enough. The system Walter has built shows the means are there in the compiler already. I think you need a new IR, but it does not have to be used for code gen, it can point back to the ast nodes that represent ARC pointer assignments. One could probably translate the one used in Rust, even.
Re: low-latency GC
On Sunday, 6 December 2020 at 11:07:50 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 10:44:39 UTC, Max Haughton wrote: On Sunday, 6 December 2020 at 05:29:37 UTC, Ola Fosheim Grostad wrote: It has to be either some kind of heavily customisable small GC (i.e. with our resources the GC cannot please everyone), or arc. The GC as it is just hurts the language. Realistically, we probably need some kind of working group or at least serious discussion to really narrow down where to go in the future. The GC as it is now must go, we need borrowing to work with more than just pointers, etc. The issue is that it can't just be done incrementally, it needs to be specified beforehand. ARC can be done incrementally, we can do it as a library first and use a modified version existing GC for detecting failed borrows at runtime during testing. But all libraries that use owning pointers need ownership to be made explicit. A static borrow checker an ARC optimizer needs a high level IR though. A lot of work though. ARC with a library will have overhead unless the compiler/ABI is changed e.g. unique_ptr in C++ has an indirection. The AST effectively is a high-level IR. Not a good one, but good enough. The system Walter has built shows the means are there in the compiler already. As things are at the moment, the annotations we have for pointers like scope go a long way, but the language doesn't deal with things like borrowing structs (and the contents of structs i.e. making a safe vector) properly yet. That is what needs thinking about.
Re: low-latency GC
On Sunday, 6 December 2020 at 10:44:39 UTC, Max Haughton wrote: On Sunday, 6 December 2020 at 05:29:37 UTC, Ola Fosheim Grostad wrote: It has to be either some kind of heavily customisable small GC (i.e. with our resources the GC cannot please everyone), or arc. The GC as it is just hurts the language. Realistically, we probably need some kind of working group or at least serious discussion to really narrow down where to go in the future. The GC as it is now must go, we need borrowing to work with more than just pointers, etc. The issue is that it can't just be done incrementally, it needs to be specified beforehand. ARC can be done incrementally, we can do it as a library first and use a modified version existing GC for detecting failed borrows at runtime during testing. But all libraries that use owning pointers need ownership to be made explicit. A static borrow checker an ARC optimizer needs a high level IR though. A lot of work though.
Re: low-latency GC
On Sunday, 6 December 2020 at 05:29:37 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 05:16:26 UTC, Bruce Carneal wrote: How difficult would it be to add a, selectable, low-latency GC to dlang? Is it closer to "we cant get there from here" or "no big deal if you already have the low-latency GC in hand"? I've heard Walter mention performance issues (write barriers IIRC). I'm also interested in the GC-flavor performance trade offs but here I'm just asking about feasibility. The only reasonable option for D is single threaded GC or ARC. It has to be either some kind of heavily customisable small GC (i.e. with our resources the GC cannot please everyone), or arc. The GC as it is just hurts the language. Realistically, we probably need some kind of working group or at least serious discussion to really narrow down where to go in the future. The GC as it is now must go, we need borrowing to work with more than just pointers, etc. The issue is that it can't just be done incrementally, it needs to be specified beforehand.
Re: low-latency GC
On Sunday, 6 December 2020 at 08:59:49 UTC, Ola Fosheim Grostad wrote: Well, you could in theory avoid putting owning pointers on the stack/globals or require that they are registered as gc roots. Then you don't have to scan the stack. All you need then is write barriers. IIRC Abd read barriers... I assume. However with single threaded incremental, write barriers should be enough.
Re: low-latency GC
On Sunday, 6 December 2020 at 08:36:49 UTC, Bruce Carneal wrote: Yes, but they don't allow low level programming. Go also freeze to sync threads this has a rather profound impact on code generation. They have spent a lot of effort on sync instructions in code gen to lower the latency AFAIK. So, much of the difficulty in bringing low-latency GC to dlang would be the large code gen changes required. If it is a really big effort then that is all we need to know. Not worth it until we can see a big payoff and have more resources. Well, you could in theory avoid putting owning pointers on the stack/globals or require that they are registered as gc roots. Then you don't have to scan the stack. All you need then is write barriers. IIRC
Re: low-latency GC
On Sunday, 6 December 2020 at 08:12:58 UTC, Ola Fosheim Grostad wrote: On Sunday, 6 December 2020 at 07:45:17 UTC, Bruce Carneal wrote: GCs scan memory, sure. Lots of variations. Not germane. Not a rationale. We need to freeze the threads when collecting stacks/globals. OK. Low latency GCs exist. D is employed at multiple "levels". Whatever level you call it, Go and modern JVMs employ low latency GCs in multi-threaded environments. Some people would like to use D at that "level". Yes, but they don't allow low level programming. Go also freeze to sync threads this has a rather profound impact on code generation. They have spent a lot of effort on sync instructions in code gen to lower the latency AFAIK. So, much of the difficulty in bringing low-latency GC to dlang would be the large code gen changes required. If it is a really big effort then that is all we need to know. Not worth it until we can see a big payoff and have more resources. My question remains: how difficult would it be to bring such technology to D as a GC option? Is it precluded somehow by the language? Is it doable but quite a lot of effort because ...? Is it no big deal once you have the GC itself because you only need xyz hooks? Is it ...? Get rid of the system stack and globals. Use only closures and put in a restrictive memory model. Then maybe you can get a fully no freeze multi threaded GC. That would be a different language. It would be, but I don't think it is the only way to get lower latency GC. That said, if the code gen effort you mentioned earlier is a big deal, then no need to speculate/examine further. Also, I think Walter may have been concerned about read barrier overhead but, again, I'm looking for feasibility information. What would it take to get something that we could compare? Just add ARC + single threaded GC. And even that is quite expensive. Thanks for the feedback.
Re: low-latency GC
On Sunday, 6 December 2020 at 07:45:17 UTC, Bruce Carneal wrote: GCs scan memory, sure. Lots of variations. Not germane. Not a rationale. We need to freeze the threads when collecting stacks/globals. D is employed at multiple "levels". Whatever level you call it, Go and modern JVMs employ low latency GCs in multi-threaded environments. Some people would like to use D at that "level". Yes, but they don't allow low level programming. Go also freeze to sync threads this has a rather profound impact on code generation. They have spent a lot of effort on sync instructions in code gen to lower the latency AFAIK. My question remains: how difficult would it be to bring such technology to D as a GC option? Is it precluded somehow by the language? Is it doable but quite a lot of effort because ...? Is it no big deal once you have the GC itself because you only need xyz hooks? Is it ...? Get rid of the system stack and globals. Use only closures and put in a restrictive memory model. Then maybe you can get a fully no freeze multi threaded GC. That would be a different language. Also, I think Walter may have been concerned about read barrier overhead but, again, I'm looking for feasibility information. What would it take to get something that we could compare? Just add ARC + single threaded GC. And even that is quite expensive.
