Re: Directly compiling a D program with other libraries
On Monday, 13 March 2023 at 05:05:27 UTC, Jeremy wrote: Hello, I am new to this forum and to D. I am trying to compile a basic D program with libraries (`requests` which requires `cachetools` and `automem`) without using dub. I have never used dub before, only a compiler. The folders containing the libraries are in the same folder as main.d, the file I am trying to compile, and the command I am using to compile is `ldc2 -I. main.d`. When I compile my program, I just get linker errors such as: ``` /usr/lib/gcc/x86_64-pc-linux-gnu/12/../../../../x86_64-pc-linux-gnu/bin/ld: main.o: in function `_Dmain': main.d:(.text._Dmain+0x2f): undefined reference to `_D8requests10getContentFNcAyaZSQBd7streams__T6BufferThZQk' ``` Does anyone have any advice on how to solve my problem? That's a linker error, meaning the missing symbol isn't available to link into the executable. You need to compile the source of all the libraries you use and make sure the resultant binaries are available for the linker to link into the executable. The -I switch you've passed tells the compiler where to find imported modules. The compiler needs parse them to know which symbols are available for you to use when it's compiling your code. (The current working directory is the default anyway, so you don't need to pass `-I.` for that.) By default, the compiler does not compile imported modules. If you add `-i` to the command line, then it will compile all of the modules you import (as long as they're in the `-I` path), excluding the DRuntime and Phobos modules. It will then also pass all of the compiled object files to the linker, so then your linker error should go away. However, when you choose not to use dub, you need to also ensure that you are accounting for any special compiler flags the libraries you use may require (for example, specific `-version` values). If they're configured to compile as static or shared libraries, it may be easier just to store the source for each of them outside of your project's source tree, use dub to build each of them, and then pass the compiled libraries to the compiler when you build your program. In that case, you wouldn't use `-i`. Just make sure that `-I` is correctly configured in that case.
Re: Code organization, dub, etc.
On Monday, 13 March 2023 at 10:52:11 UTC, Joe wrote: months. Am I missing something on how to deal with multi-executable projects in dub (and I can think of many such projects)? Dub isn't very good at doing more than relatively basic things natively (which covers enough D projects for it to not matter in the general case). I personally like to use Meson with D. e.g. It [very easily allows](https://github.com/BradleyChatha/libbc/blob/master/meson.build#L84) multiple executables. Technically you can use the build command values in dub.json/sdl to make it spit out an extra build; but it's obviously a bit of a hack. You could also have a small script/makefile to call the right `dub build` twice. The issue with using Meson is that it makes using dub packages harder. Meson technically supports bringing in dub packages, but I've not managed to make it work right. For better or for worse we're stuck with dub as the standard package manager + build tool one-in-all for most of our open source libraries. I also tried to use `dfmt` and found it limited in comparison to `clang-format`. I do realize that dfmt is a community-supported tool. D's auxiliary tooling has been woeful in comparison to other languages for the last 8 years I've been using it. It's down to a few factors such as manpower; lack of use/inability to use the D compiler's frontend for tooling, etc. On the other hand the reason it feels bad to me is also because I mostly use C#; Typescript, and Go outside of using D - all of which have great (bar NPM) tooling and editor support so D may not actually be that bad overall. Glass Half Full or Half Empty kinda perspective. It's constantly getting better, but is always "behind" the more popular languages. If you were to write up your struggles with dfmt and friends, then I'm sure it'll be taken as welcome feedback. I find that D has much to offer in terms of language facilities and simplicity. About the only bump in the learning curve is the distinction between structs and classes. Have you met our lord and savior: D's metaprogramming? :D ([Shamless plug](https://www.youtube.com/watch?v=0lo-FOeWecA)) But yeah generally you might find D is a bit less "cohesive" and may feel kind of "splattered" in terms of... well, everything about it. It's a volunteer project with a dire lack of volunteers. Hope this helps, and hopefully I've kept misinformation to a minimum as I am very prone to working off of false memory.
Re: Code organization, dub, etc.
On Monday, 13 March 2023 at 12:56:57 UTC, Bradley Chatha wrote: For better or for worse we're stuck with dub as the standard package manager + build tool one-in-all for most of our open source libraries. Yeah, it seems like it's *only* for libraries (and a few single-exe utilities). Looking at code.dlang.org, under "Stand-alone applications/Server software", the top rated item is "handy-httpd" which according to its dub.json builds a library! And the second place "voxelman" is builds three libraries and one executable, which appears to be a "launcher" to access the libraries as plugins.
Re: Code organization, dub, etc.
On Monday, 13 March 2023 at 13:20:21 UTC, Joe wrote: Yeah, it seems like it's *only* for libraries (and a few single-exe utilities). Looking at code.dlang.org, under "Stand-alone applications/Server software", the top rated item is "handy-httpd" which according to its dub.json builds a library! And the second place "voxelman" is builds three libraries and one executable, which appears to be a "launcher" to access the libraries as plugins. The package registry is full of libraries, yes. That's what it's primarily for. There aren't a lot of executables uploaded there because they're usually better distributed in other ways. But plenty of people are using dub to build them. One way to handle multiple executables is to write a simple script that makes multiple calls to dub with the configurations you need. And I haven't looked into it yet, but it may be possible to use `preBuildCommands` to do the same thing. E.g., add a default with a `preBuildCommands` entry calling dub on multiple configurations.
Re: Code organization, dub, etc.
On Monday, 13 March 2023 at 13:20:21 UTC, Joe wrote: Yeah, it seems like it's *only* for libraries (and a few single-exe utilities). Looking at code.dlang.org, under "Stand-alone applications/Server software", the top rated item is "handy-httpd" which according to its dub.json builds a library! I'm not particularly interested in defending dub - i consider it a useless piece of crap that I only suffer through cuz some users demanded it - but you can use the subpackage thing to build multiple executables. My thing here does it: https://code.dlang.org/packages/adr-terminalemulator relevant config code here: https://github.com/adamdruppe/terminal-emulator/blob/master/dub.json#L31 So the mactabs exe and the attach exe and so on are all a bit different programs you can run from the code.
Re: Code organization, dub, etc.
On Monday, 13 March 2023 at 13:32:04 UTC, Mike Parker wrote: The package registry is full of libraries, yes. That's what it's primarily for. There aren't a lot of executables uploaded there because they're usually better distributed in other ways. But plenty of people are using dub to build them. One way to handle multiple executables is to write a simple script that makes multiple calls to dub with the configurations you need. That's essentially the same as using a Makefile (or CMake with custom commands) to build your project. What I had in mind when I mentioned "multi-executable projects" was something like Postgres or say, a tenth of that, e.g., a server executable, two or more client and utility executables and one or more libraries, all of it spread over a few directories, possibly to a depth of say, two from the root. Can dub handle something like that, e.g., can it handle nested dub.json's, or wouldn't it make much more sense to build such a thing with make, cmake, meson?
Preventing the Compiler from Optimizing Away Benchmarks
I was looking at [1] for ways to prevent the compiler from optimizing away code when trying to benchmark. It has the following C++ code as a simpler version: ``` inline BENCHMARK_ALWAYS_INLINE void DoNotOptimize(Tp& value) { asm volatile("" : "+r,m"(value) : : "memory"); } ``` I made an attempt to make a D version of it, but failed. Apparently DMD doesn't like the `""` in the first part of the asm instruction. I'm also not sure the `volatileLoad` command is right, but I didn't know of any other way to have volatile work in D (and I couldn't figure out how it actually worked from looking at the code). ``` void DoNotOptimize(T)(T* ptr) { import core.volatile: volatileLoad; T value = volatileLoad(ptr); asm {"" : "+r,m"(value) : : "memory";} } ``` [1] https://theunixzoo.co.uk/blog/2021-10-14-preventing-optimisations.html
Re: Code organization, dub, etc.
On Monday, 13 March 2023 at 13:58:29 UTC, Adam D Ruppe wrote: I'm not particularly interested in defending dub - i consider it a useless piece of crap that I only suffer through cuz some users demanded it For the record, I wasn't trying to attack dub (or dfmt). I was more interested in determining whether DLang (the Foundation and/or the community) would consider working with the CMake and clang-format communities to get them to support D in their products, or whether they prefer to stick with what they have in terms of tooling. I think I heard someone at DConf 22 ask about getting Intellij support for D and Walter said that JetBrains would ask for funding to do that. I don't think CMake support/development is in the same boat, is it?
Re: const in functions
On Sunday, 12 March 2023 at 19:09:13 UTC, Ali Çehreli wrote: --- In this case, using `ref` will increase performance while reducing the number of copies. Would it be wise to use `const ref` to protect the routine from ourselves or someone else? For example: ```d auto inConst( //const ref Int a, const ref Int b) { a.x += 1; return Int(a.x + b.x); } struct Foo(T) { T x; alias x this; this(T n) { x = n; } } alias Int = Foo!int; ``` SDB@79
Re: Preventing the Compiler from Optimizing Away Benchmarks
On Monday, 13 March 2023 at 14:17:57 UTC, jmh530 wrote: I was looking at [1] for ways to prevent the compiler from optimizing away code when trying to benchmark. It has the following C++ code as a simpler version: ``` inline BENCHMARK_ALWAYS_INLINE void DoNotOptimize(Tp& value) { asm volatile("" : "+r,m"(value) : : "memory"); } ``` I made an attempt to make a D version of it, but failed. Apparently DMD doesn't like the `""` in the first part of the asm instruction. I'm also not sure the `volatileLoad` command is right, but I didn't know of any other way to have volatile work in D (and I couldn't figure out how it actually worked from looking at the code). ``` void DoNotOptimize(T)(T* ptr) { import core.volatile: volatileLoad; T value = volatileLoad(ptr); asm {"" : "+r,m"(value) : : "memory";} } ``` [1] https://theunixzoo.co.uk/blog/2021-10-14-preventing-optimisations.html that's illegal code. You mix GCC/LLVM syntax with D asm block and the front-end wont recognize that. LDC recognizes a syntax similar to what is described in your link, see https://wiki.dlang.org/LDC_inline_assembly_expressions. GDC has it too (since that the syntax invented by GCC in first place) but I cant find the documentation ATM.
Re: const in functions
On 3/13/23 08:17, Salih Dincer wrote: > In this case, using `ref` will increase performance while reducing the > number of copies. I am not sure about that. Unless there is an expensive copy construction, most objects are simple data copies. To use 'ref' or not should be guided through semantics. Luckily, we have the -preview=in command line switch that removes the cost of copying from the equation: https://dlang.org/spec/function.html#in-params Just marking the input parameters 'in' should be sufficient in most cases. Ali
Re: const in functions
On Sunday, 12 March 2023 at 15:09:45 UTC, Salih Dincer wrote: Hi, [...] // A, we can get its to guarantee us that parameters // won't change: auto inConst(T)(T a, const T b) // const { // it's not needed --^ but ^-- why can't this be used Well you got the great answers to your questions already but I like to add a comment that is that **`const` parameters without `ref` are really moot.** I whish at least that they could give the guarantee to be rvalues (i.e have no address) but this is actually not the case. All the D compilers will actually copy `const` parameters to locals, making them lvalues, unless optimizations are requested(1). See codegen : https://godbolt.org/z/vev1PGWh3. This brings the idea that maybe a `rvalue` storage class could be added to the language. (1): Actually DMD does not drop the copy **at all**, even with -O.
Re: Best way to read/write Chinese (GBK/GB18030) files?
On 3/12/23 8:32 PM, zjh wrote: On Sunday, 12 March 2023 at 20:03:23 UTC, 0xEAB wrote: ... Thank you for your reply, but is there any way to output `gbk` code to the console? What is required is an addition to the `std.encoding` module, to allow such an encoding. Encodings are simply translating some encoding (e.g. utf) to another (e.g. gbk). If you look at `std.encoding` you can get an idea of what it might require. It will take some effort and especially some help from a knowledgeable user (such as yourself). -Steve
Re: Preventing the Compiler from Optimizing Away Benchmarks
On Monday, 13 March 2023 at 15:23:25 UTC, user1234 wrote: [snip] [1] https://theunixzoo.co.uk/blog/2021-10-14-preventing-optimisations.html that's illegal code. You mix GCC/LLVM syntax with D asm block and the front-end wont recognize that. LDC recognizes a syntax similar to what is described in your link, see https://wiki.dlang.org/LDC_inline_assembly_expressions. GDC has it too (since that the syntax invented by GCC in first place) but I cant find the documentation ATM. Thanks, that helps. Below seems to be working...(with LDC and -O) when I include the DoNotOptimize, it takes around 300-500us, but when I comment it out, then it takes about 5us. It would still take some work to figure out how to get it to work with DMD. ```d void DoNotOptimize(T)(T* ptr) { import ldc.llvmasm; import core.volatile: volatileLoad; T value = volatileLoad(ptr); __asm("", "*mr,~{memory}", &value, ); } void main() { import std.algorithm.iteration: sum; import std.array: uninitializedArray; import std.datetime.stopwatch; import std.random: uniform; import std.stdio: writeln; auto testData = uninitializedArray!(long[])(600_000); foreach(ref el; testData) el = uniform(0, 10); ulong seed = 0; ulong output = 0; StopWatch sw; sw.start(); DoNotOptimize(&seed); output = testData.sum(seed); DoNotOptimize(&output); sw.stop(); writeln("time: ", sw.peek); } ```
Re: Directly compiling a D program with other libraries
That's a linker error, meaning the missing symbol isn't available to link into the executable. You need to compile the source of all the libraries you use and make sure the resultant binaries are available for the linker to link into the executable. The -I switch you've passed tells the compiler where to find imported modules. The compiler needs parse them to know which symbols are available for you to use when it's compiling your code. (The current working directory is the default anyway, so you don't need to pass `-I.` for that.) By default, the compiler does not compile imported modules. If you add `-i` to the command line, then it will compile all of the modules you import (as long as they're in the `-I` path), excluding the DRuntime and Phobos modules. It will then also pass all of the compiled object files to the linker, so then your linker error should go away. Using `-i` solved my problem, thank you very much! However, when you choose not to use dub, you need to also ensure that you are accounting for any special compiler flags the libraries you use may require (for example, specific `-version` values). If they're configured to compile as static or shared libraries, it may be easier just to store the source for each of them outside of your project's source tree, use dub to build each of them, and then pass the compiled libraries to the compiler when you build your program. In that case, you wouldn't use `-i`. Just make sure that `-I` is correctly configured in that case. Okay, thanks.
Re: Best way to read/write Chinese (GBK/GB18030) files?
On Monday, 13 March 2023 at 15:50:37 UTC, Steven Schveighoffer wrote: What is required is an addition to the `std.encoding` module, to allow such an encoding. Thank you for your information.
DMD: what's the proper way to get a list of symbols from a Module object?
Hello, I am playing a little bit with DMD to get familiar with it (just to get a basic overview of it) I'm trying to come up with a proof of concept for https://github.com/dlang/DIPs/blob/master/DIPs/DIP1044.md ```D enum Tester { KNOWN = 1, WITHAUTO = 2 } void func(Tester a, Tester b) { } void main() { func(Tester.KNOWN, auto.WITHAUTO); } ``` The idea is to reuse auto, basically do like tuple, create a ``StructDeclaration`` and inside put a ``VarDeclaration`` just to remember what is the identifier, then once it tries to search for the symbol, we hijack it and try to search globally instead I came up with this: https://github.com/ryuukk/dmd/commit/cb86d398b68501fd334c090745e946db7b27ff97 It seems to follow the logic i have in mind, the problem is whenever i try to search for the symbol given the identifier i saved I had to set the module as parent of the ``StructDeclaration`` created to get a starting point The problem is the field ``members`` from ``Module`` only seems to list ``object`` module when it is trying to search for the symbol There should also be ``Tester`` with should be an ``EnumDeclaration``, why is it not listed? Am i doing something incorrect (that's probably the case) Anyway, i'm blind at this point, if someone could provide some guidance, that would be kind of you!
#define-like behavior
Hi, in C and C++ you can use #define to substitute a value in place of an identifier while preprocessing. If you initialize a new string and don't change its value after that, will the compiler substitute the string identifier with its value, like #define in C, or will it make a string in memory and refer to that?
Re: #define-like behavior
On Tuesday, 14 March 2023 at 05:47:35 UTC, Jeremy wrote: Hi, in C and C++ you can use #define to substitute a value in place of an identifier while preprocessing. If you initialize a new string and don't change its value after that, will the compiler substitute the string identifier with its value, like #define in C, or will it make a string in memory and refer to that? Manifest constants in D have a similar effect as #defined values, e.g.: ```d enum someVal = 10; writeln(someVal); ``` Here, the effect is the same as if you'd written `writeln(10)`. The difference is that with the preprocessor, it's a text replacement on the source before the compiler gets ahold of it, but in D the compiler handles the substitution internally. The same is true for string literals: ```d enum str = "Hello"; writeln(str); ``` String literals get special treatment from the compiler in that they are "interned". ```d auto s1 = "What's up?"; auto s2 = "What's up?"; ``` The literal "What's up?" *should* be stored in the binary only once, so both s1 and s2 will point to the same location. Substitute a manifest constant and the effect should be the same: ```d enum greeting = "What's up?"; auto s1 = greeting; auto s2 = greeting; ``` Just be aware that there's a consequence for array literals: ```d enum vals = [1, 2, 3]; auto a1 = vals; auto a2 = vals; ``` This allocates two dynamic arrays, not one. Everywhere you use `vals` it's just like using the literal directly, which usually means an allocation.
Re: #define-like behavior
On Tuesday, 14 March 2023 at 05:47:35 UTC, Jeremy wrote: Hi, in C and C++ you can use #define to substitute a value in place of an identifier while preprocessing. If you initialize a new string and don't change its value after that, will the compiler substitute the string identifier with its value, like #define in C, or will it make a string in memory and refer to that? In D, you can get #define-like behavior by declaring the string as a [manifest constant][1], like this: ```d enum myString = "hello"; ``` For more tips on how to translate C preprocessor idioms to D, take a look at ["The C Preprocessor vs D"][2] in the ["Articles" section][3] of dlang.org. [1]: https://dlang.org/spec/enum.html#manifest_constants [2]: https://dlang.org/articles/pretod.html [3]: https://dlang.org/articles/index.html