Re: GCC 10.1 Released

[Matthias Klumpp via Digitalmars-d-announce]

On Thursday, 14 May 2020 at 16:57:20 UTC, Iain Buclaw wrote:

As of last week (7th May), GCC 10.1 has now been released.

For the D language front-end, only a small number of 
incremental, but substantial changes have gone in.  Most 
notable of the lot has been the addition of `static foreach`, 
which makes the front-end (the C++ port of DMD) feature 
complete with DMD version 2.076.1.  There is also now a 
configurable separation between building Druntime and Phobos, 
which has allowed many targets to have gained library support 
for building a D runtime library by disabling the build of 
Phobos.


[...]


Hey Iain!
I love your summary and plans (can you do this more often? ^^), 
and just wanted to say kudos for the work you do on this! As far 
as the Linux world is concerned, getting GDC into GCC and keeping 
it well-integrated and up-to-date it is a major thing to get D 
established as first-class supported language.
Having that D binding generator will be even more useful for 
integrating D with existing, bigger codebases written in C or C++.
Keep up the great work! (And I'll get that dub package rebuilt 
against GDC 10.1 in Debian ASAP ^^)

Cheers,
Matthias

P.S: As a distribution developer, I would love LDC and GDC to be 
ABI-compatible, so we wouldn't have to choose a global default... 
But that's probably an unlikely thing to change anytime soon.

Re: meson and d

[Matthias Klumpp via Digitalmars-d-learn]

On Sunday, 9 June 2019 at 13:13:28 UTC, dkd wrote:

hi, i need some meson configuration that works with dmd and ldc

my setting don't work. because meson complains it cannot find 
the d compiler.


d compiler can be invoked on command line.

i use :
set dmd_path="path/to/d/base"
set PATH=%dmd_path%\bin;%PATH%;

mingw can be detected by setting GCC_HOME

i use win7 and x86
meson 0.50.1


Are you using Windows? Meson's D support has never been tested 
there.
In theory though you can point it at any D compiler by setting 
the "DC" environment variable.

Re: Meson support for Mir and Lubeck

[Matthias Klumpp via Digitalmars-d-announce]

[ Replying to lots of stuff in one message - evil, I know... ;-) ]

On Sunday, 9 September 2018 at 21:22:42 UTC, 9il wrote:

On Sunday, 5 August 2018 at 02:36:29 UTC, Matthias Klumpp wrote:

[...]


Looks like that only betterC projects are good enough to become 
Debian packages. Generally because of the have stable C ABI 
that does not depend on D compiler version at all.


Having a stable API/ABI is a *huge* plus, but we can also deal 
with pretty much any D library and D project in Debian, as long 
as it's build process is able to discover local dependencies and 
doesn't do evil stuff (like connecting to the network or writing 
to HOME).
It also has to build with LDC or have no D library dependencies 
and build with GDC (that's a bit oversimplified though).


In practice this means we can have dub projects with zero 
dependencies (except for the standard library) that build only 
executables in Debian packages with a bunch of hacks, or have any 
D project that uses Makefiles/Meson/CMake/Automake.

Mir has the latter now, so that's fine.


On Monday, 10 September 2018 at 06:31:44 UTC, Russel Winder wrote:
On Sun, 2018-09-09 at 21:22 +, 9il via 
Digitalmars-d-announce wrote:



[...] Many Debian packages depend
on specific versions of things like GCC runtime or LLVM. The 
Debian packaging
system allows for many versions of libraries to co-exists. Thus 
supporting

multiple versions of Druntime and Phobos is possible.


It is possible, but heavily discouraged, and also not something I 
would ever even try to do. Debian carries *one* version of 
Phobos/DRuntime per compiler that we support and that all D code 
in the archive has to build with.
There may be occasions where multiple versions are in the 
archive, but that happens only briefly during transitions.
For security reasons (only one thing to patch in stable) and 
maintainability reasons (the less to maintain the better) 
anything that doesn't build with the current version of D/Phobos 
will either be patched upstream or in Debian or dropped from 
Debian entirely.
So far this case has never happened though, all breakage of D 
code in Debian was usually addressed pretty quickly - except for 
strange compiler bugs on less common architectures.






[…]
> I only worry about potential name clashes with Mir (the 
> display server) in Ubuntu ^^


This is going to be a naming problem. Debian has many of these 
sort of naming clash and usually it is best for the smaller, 
newer project to accept that they need to choose a non-clashing 
name. Recent example the Mu editor, but there are many 
instances of this. D's Mir needs to choose a name that doesn't 
clash with Canonical's Mir.


This was more meant as a joke from my side ;-) Technically, 
nobody from Canonical bothered to upload a "mir" package to 
Debian yet, so technically the name is free to grab for us. 
However, introducing a src:mir package into Debian would either 
force Ubuntu to adapt and rename their display server source 
package (to mir-display-server) or not have D's mir for quite a 
while.
Socially I think claiming the src:mir name isn't the best idea, 
therefore - it's unfriendly to a derivative.
However, src:libmir is free and that's what we could go with as a 
pretty good compromise.
As for the binary package names, none of the D-mir names clash 
with Ubuntu's mir, so that's a non-issue.



On Monday, 10 September 2018 at 07:25:07 UTC, 9il wrote:
On Monday, 10 September 2018 at 06:31:44 UTC, Russel Winder 
wrote:

On Sun, 2018-09-09 at 21:22 +, 9il via [...]


Interesting, maybe we can go forward with D specific libraries 
in the future. Is there any D library that is used by 
application packages?


Quite a few. Debian uses GDC and LDC for compiling its D 
packages. LDC on the amd64, arm64, armhf and i386 architectures 
and GDC on all other archs.
The respective default D compiler is set via the 
default-d-compiler metapackage[1].


This means that whenever there's a new compiler release, ABI gets 
broken "thanks" to D's unstable ABI. Fortunately all new 
compilers also come with a new Phobos shared library that has had 
its SONAME bumped. Which means we can track which D compiler a 
package was built with via the phobos/druntime libraries they 
link to.
This enables us to rebuild dependencies in order via a regular 
library transition, coincidentally one is going on right now. 
See[2] and tick "Good" to see all D packages that are part of 
this transition.


This results in the fun conundrum though that as soon as we have 
D code that *doesn't* link against druntime or phobos, we can't 
track whether it was rebuilt with the latest compiler and it 
might fall through the cracks.
I wonder how much of an issue this is - other languages 
implemented hacks to deal with this issue by depending on 
artificial virtual ABI packages denoting the current compiler 
version.


Mir Optim can be easily used by other libraries and languages, 
developers don't need to know D at all as well as 

Re: GDC with D frontend 2.081.2

[Matthias Klumpp via Digitalmars-d-announce]

On Friday, 24 August 2018 at 08:50:22 UTC, Daniel Kozak wrote:
On Fri, Aug 24, 2018 at 10:40 AM Basile B. via 
Digitalmars-d-announce < digitalmars-d-announce@puremagic.com> 
wrote:



[...]
Great news. Best of luck for a wide adoption in the official 
package managers of the linux distributions.


I do not think this will be an issue. I believe as soon as gdc 
will be

part of gcc it will be part of many linux distribution


Indeed. Having GDC merged into GCC is *huge* for the D language. 
It means that pretty much every Linux distribution out there will 
have a supported D compiler available, and this includes 
enterprise distributions like RHEL. Even if enterprise distros 
might not give full support for GDC, it will still be available.
I know from experience that this is something people look at when 
deciding whether using a programming language is viable and can 
be long-time supported in their company, so for D and the public 
image of D, being in GCC is absolutely awesome.


FWIW, Debian already has an up-to-date GDC 8, so does Ubuntu. 
"Unfortunately"[1] for GDC though, the default compiler for D 
stuff in Debian is LDC on most architectures, because it still 
works quite a bit better (as in: more D code can be compiled with 
it).


[1]: Not really, LDC is really good too!

Re: D need an ORM library!

[Matthias Klumpp via Digitalmars-d-learn]

On Monday, 20 August 2018 at 15:58:37 UTC, Jesse Phillips wrote:

[...]

There are a number of things out there, but personally don't 
know there state or simplicity.


2016 there was a talk: http://dconf.org/2016/talks/nowak.html

But personally I preferred this one: 
http://dconf.org/2016/talks/smith.html

https://github.com/cruisercoder/dstddb


This all looks very dead. Is there any project still alive? The 
only effort that still seems to be actively developed as far as I 
can see is hunt-entity...


And I agree, SQLAlchemy is pretty great!

Re: LDC 1.11.0

[Matthias Klumpp via Digitalmars-d-announce]

On Sunday, 19 August 2018 at 12:56:28 UTC, Matthias Klumpp wrote:

[...]
Does that Git has thing work if there is no Git involved? ;-)
[...]


Ewww, looks like I can't write today...
At the moment, we get
```
LDC - the LLVM D compiler (1.8.0):
  based on DMD v2.078.3 and LLVM 5.0.2
  built with LDC - the LLVM D compiler (0.17.5)
```
unconditionally - if I switch to a ltsmaster snapshot, it will 
likely display 0.17.6 with no direct way of knowing which 
snapshot was taken.
(I could manually record that as an info file in the package, so 
it at least is recorded somewhere, but that's not easily 
accessible at all...)

Re: LDC 1.11.0

[Matthias Klumpp via Digitalmars-d-announce]

On Sunday, 19 August 2018 at 12:48:15 UTC, kinke wrote:

[...]
The used host compiler, incl. git hash if untagged, can be 
found in the --version output:


```
LDC - the LLVM D compiler (1.12.0git-1c87fd7):
  based on DMD v2.082.0 and LLVM 6.0.0
  built with LDC - the LLVM D compiler (0.17.6git-79d2284)
```


Does that Git has thing work if there is no Git involved? ;-)
To make this work I will have to download a tarball of ltsmaster, 
with no Git involved to get the hash from.

Re: LDC 1.11.0

[Matthias Klumpp via Digitalmars-d-announce]

On Sunday, 19 August 2018 at 12:23:37 UTC, kinke wrote:
On Saturday, 18 August 2018 at 21:05:42 UTC, Matthias Klumpp 
wrote:
Will we get a release of the ltsmaster branch as well? From 
the release notes it sounds like building with a more recent 
version is a good idea...


Using latest ltsmaster is always a good idea. I'm a bit 
reluctant to tag a new version, as it's a moving target (e.g., 
needed another adaptation yesterday in order to be able to 
bootstrap 2.082-based LDC, and will need more adaptations in a 
few weeks for LLVM 7...). 0.17.6 has been stuck in the pipeline 
for half a year now; perhaps we finally manage after LLVM 7 is 
supported.


The thing is, a release is actually tremendously helpful for the 
Debian packaging - we are using the latest tagged version there 
for ages.
I could, if you think that it is a good idea, just use a snapshot 
of the current ltsmaster Git branch, but that makes it harder to 
track with which ltsmaster state the Debian package was actually 
built with.

Re: LDC 1.11.0

[Matthias Klumpp via Digitalmars-d-announce]

On Saturday, 18 August 2018 at 16:47:35 UTC, kinke wrote:

Glad to announce LDC 1.11:

* Based on D 2.081.2.
* Prebuilt packages now using LLVM 6.0.1 and including 
additional cross-compilation targets (MIPS, MSP430, RISC-V and 
WebAssembly).
* Rudimentary support for compiling & linking directly to 
WebAssembly. See the dedicated Wiki page [1] for how to get 
started.
* AArch64 (64-bit ARM) now mostly working on Linux/glibc and 
Android.
* Some support for classes without TypeInfos, for -betterC 
and/or a minimal (d)runtime.


Full release log and downloads: 
https://github.com/ldc-developers/ldc/releases/tag/v1.11.0


Thanks to all contributors!

[1] https://wiki.dlang.org/Generating_WebAssembly_with_LDC


Awesome news! Will we get a release of the ltsmaster branch as 
well? From the release notes it sounds like building with a more 
recent version is a good idea...

Re: Dub support was added to Meson

[Matthias Klumpp via Digitalmars-d-announce]

On Friday, 10 August 2018 at 14:38:10 UTC, Russel Winder wrote:
On Sat, 2018-08-04 at 16:07 +, Filipe Laíns via 
Digitalmars-d-announce wrote:

Hello,

Dub support was finally merged to the Meson's upstream.
For the ones that don't know, Meson[1] is a fast build system
that uses ninja[2] as a backend. Until now it was impossible to
use dependencies via the Dub and many many packages only
supported Dub. Now you can import dependencies that already 
exist

in Dub's local repo. You still need to fetch and build them
though.


Sadly not in Meson 0.47.1 is a new release due soon?


There is no fixed release schedule, but from experience, this 
might take a few months.



First, you need to fetch and build the dependency.

   dub fetch vibe-d
   dub build vibe-d


This feels a little unsatisfactory. Dub (and soon SCons I hope) 
builds handle getting the dependency as well as building it and 
thence linking the project to it. Meson both depends on the dub 
executable and yet isn't doing the getting, it is assuming 
already got.


Of course Meson relies on all dependencies being present at 
build specification time, so nothing inconsistent, it just 
seems a wee bit incomplete.


This is intentional, see this comment from Jussi: 
https://github.com/mesonbuild/meson/pull/3592#issuecomment-390421754


Unfortunately, this makes the dub feature almost useless for 
Linux distro integration.
However, I hope we might be able to implement reading dub.json 
files that are locally installed and incorporate D sources from 
the dependency into the regular Meson build process.


This would mean that Meson's dlang plugin would have to implement 
quite a bit of dub and dub.json parsing, which is some effort.
The benefit would be the creation of a near-seamless bridge 
between the dub and Meson worlds (you'd only have to locally 
install a dub package), that is also useful for Linux 
distribution packaging.



[...]
I have been getting projects such as Unit-Threaded, DInotify, 
TinyEndian, and D_YAML to have Meson builds so as to create 
shared libraries prior to using pkg-config to handle D project 
being dependent on these libraries. It's working nicely (for me 
anyway) except Dub uses GitHub tags for versioning and Meson 
requires version numbers in the meson.build file. This 
duplication is getting quite annoying for all concerned. As is 
the Meson requirement to list all source files, rather than 
have a convention over configuration approach as Dub does – 
SCons has a different approach but still avoids explicit source 
lists. I can get round it with Meson, but I am informed it is 
naughty and should not be done.


Jup, deterministic source lists are useful for split-building 
stuff efficiently. And globbing sources is possible with Meson, 
but highly discouraged (and even looks ugly).


These builds create shared objects and allow for dynamic 
linking. Using Dub built archives means static linking. Unless 
Meson somehow manages to get shared objects out of the Dub 
build.


Unless Dub can generate shared objects, having the dependencies 
build locally and creating pkg-config files actually seems a 
better way that accessing Dub builds.

[...]


At the moment, I think for a final project, adding Meson files to 
dependencies still is the better way. However, for quickly 
testing out things and for building something locally, the new 
dub support is quite valuable.

Also, we can expand it in future, this is a first step :-)

Re: Meson support for Mir and Lubeck

[Matthias Klumpp via Digitalmars-d-announce]

On Saturday, 4 August 2018 at 18:23:15 UTC, 9il wrote:

[...]
The packages have been tested in a real world project on OSX 
with `$ meson configure -Ddefault_library=static/`.

[...]


Hmm, shouldn't this be `meson --default-library=static ..`?

Re: Meson support for Mir and Lubeck

[Matthias Klumpp via Digitalmars-d-announce]

On Saturday, 4 August 2018 at 18:23:15 UTC, 9il wrote:
The Meson Build system [13] support was added to the following 
Dub packages:


asdf [1] - JSON serialisation library
cblas [2] - CBLAS header for Dlang
lapack [3] - LAPACK header for Dlang
lubeck [4] - High level linear algebra library
mir-algorithm [5] - Core algorithm library and a home for Dlang 
multidimensional array package - ndslice

mir-blas [6] - ndslice wrapper for CBLAS
mir-lapack [7] - ndslice wrapper for LAPACK
mir-linux-kernel [8] - Linux headers for Dlang
mir-optim [9] - Optimisation framework
mir-random [10] - Advanced random number engines and random 
distributions

[...]


Neat! This will allow us to package Mir for Debian. At the 
moment, there is nothing using Mir in Debian (obviously) and 
nothing using it is planned for packaging, but adding it might be 
useful still, since it can also - with some limits - be used from 
non-D code.


I only worry about potential name clashes with Mir (the display 
server) in Ubuntu ^^

#dbugfix 16746 - Output Makefile-style depfiles for ninja and make

[Matthias Klumpp via Digitalmars-d]

Hi!

Since this issue came up again here just now, I would like to 
draw your attention to this particular bug.
It hinders integration with widely used build systems like 
Automake, CMake, Meson and build tools like ninja/make and in 
general results in very suprising results especially when using 
templates in code that is split-built.


A patch is available in a PR against dmd, but there was no new 
activity since last March, so it would be nice to get feedback 
and have the change finally merged in some form.


Bug: https://issues.dlang.org/show_bug.cgi?id=16746
PR:  https://github.com/dlang/dmd/pull/6961

Thank you and greetings from the Debian developer's conference in 
Taiwan (with more interest in D here than expected :-) )


Cheers,
Matthias

Re: GDC on Travis-CI

[Matthias Klumpp via Digitalmars-d-learn]

On Saturday, 2 June 2018 at 16:27:38 UTC, Seb wrote:

On Saturday, 2 June 2018 at 16:04:09 UTC, Matthias Klumpp wrote:

On Saturday, 2 June 2018 at 03:15:56 UTC, crimaniak wrote:
I started to work with Travis-CI, building packages using all 
three main compilers, and noticed that I have problems with 
gdc every time and need to tweak code because of many things 
missing. For example: 
https://travis-ci.org/crimaniak/json-patch/jobs/386963340

Why this situation with gdc and how best to deal with it?


The build log seems to indicate it uses gdc 4.8, while the 
current version of GDC is 8.1 which is based on the 2.076 
frontend.


Maybe just updating GDC on Travis will fix this.


Sadly that's not so easy as

1) the GDC download path changed and we are still waiting on 
feedback from Ian on this (since more than, see 
https://github.com/dlang/installer/pull/251)


2) AFAICT there are no public binaries that use anything later 
than 2.068.2 could be used (see 
https://gdcproject.org/downloads)


That's unfortunate. I hope Iain comments on this soon, because it 
means GDC will receive much less testing by other projects. For 
getting up-to-date binaries, in theory those could be extracted 
from GDC's Debian packages, which are usually very up-to-date 
(even Git snapshots are occasionally packaged) - problem there is 
that I am not sure whether they will work standalone and with the 
older GLibc on Travis.


@crimaniak: If you really want to build with all compilers, there 
is a workaround for this issue that does not involve you 
supporting ancient D versions, and that is to actually use 
Debian's GDC on Travis. I use this excessively for my own 
projects, mostly though because I need newer system libraries and 
because I explicitly want to build with the packaged compilers as 
well.
You can use a similar approach and limit it to GDC only, I 
created a PR for that: 
https://github.com/crimaniak/json-patch/pull/1
As you can see on 
https://travis-ci.org/crimaniak/json-patch/jobs/387197216 , your 
code builds fine with the latest GDC.
So, if you want that workaround, please take it, if not it may 
serve as a reference for others facing the same problem.

Re: GDC on Travis-CI

[Matthias Klumpp via Digitalmars-d-learn]

On Saturday, 2 June 2018 at 03:15:56 UTC, crimaniak wrote:
I started to work with Travis-CI, building packages using all 
three main compilers, and noticed that I have problems with gdc 
every time and need to tweak code because of many things 
missing. For example: 
https://travis-ci.org/crimaniak/json-patch/jobs/386963340

Why this situation with gdc and how best to deal with it?


The build log seems to indicate it uses gdc 4.8, while the 
current version of GDC is 8.1 which is based on the 2.076 
frontend.


Maybe just updating GDC on Travis will fix this.

Re: Is HibernateD dead?

[Matthias Klumpp via Digitalmars-d-learn]

On Monday, 7 May 2018 at 18:10:14 UTC, Matthias Klumpp wrote:

On Saturday, 5 May 2018 at 09:32:32 UTC, Brian wrote:

On Thursday, 3 May 2018 at 10:27:47 UTC, Pasqui23 wrote:

Last commit on https://github.com/buggins/hibernated
was almost a year ago

So what is the status of HibernateD?Should I use it if I need 
an ORM? Or would I risk unpatched security risks?


You can use Entity & Database library:

https://github.com/huntlabs/entity
https://github.com/huntlabs/database


I've tried both a while back, and they are still inferior to 
Hibernated (no surprise there, both projects are very new). 
[...]


I've looked at this again today, end Entity now seems to have 
OneToMany/ManyToMany relations (for 18 days), which is great news!
I might need to play with this a little again. In any case, if I 
do port my D code to another ORM, I want the next port to be the 
last time I ever do that, because it's a lot of work with quite 
some risk of breakage.


It's also really sad that the existing ORMs don't share a common 
database abstraction library, but well, different people do 
things differently.


In any case, many thanks to Vadim Lopatin for merging the 
existing PRs into ddbc and Hibernated for now! That makes life 
easier already :-)

Re: Is HibernateD dead?

[Matthias Klumpp via Digitalmars-d-learn]

On Monday, 7 May 2018 at 08:26:15 UTC, bauss wrote:

On Monday, 7 May 2018 at 06:12:19 UTC, Vadim Lopatin wrote:

On Thursday, 3 May 2018 at 20:49:35 UTC, Matthias Klumpp wrote:

On Thursday, 3 May 2018 at 18:52:34 UTC, singingbush wrote:

On Thursday, 3 May 2018 at 10:27:47 UTC, Pasqui23 wrote:

[...]
If someone is ready to maintain these projects, I can grant 
privileges for github repositories ddbc, hibernated.


Best regards,
Buggins


I wouldn't mind attempting maintaining them


I would help out as much as I can as well (but I lack the skill 
and time to properly maintain it, especially since I mostly care 
about my Postgres usecase, and not much about MySQL or ODBC).
Having ddbc in dlang-community might actually be a neat thing, to 
make it very easy for other contributors to merge stuff and easy 
the maintenance of it.
But ultimately that's Bauss' call, as potential future maintainer 
:-)


P.S: @Bauss: While you definitely will want ddbc as solid 
database abstraction, for Hibernated you might want to change it 
to be less Java-esque and maybe fit your existing ORM better.

Re: Is HibernateD dead?

[Matthias Klumpp via Digitalmars-d-learn]

On Saturday, 5 May 2018 at 09:32:32 UTC, Brian wrote:

On Thursday, 3 May 2018 at 10:27:47 UTC, Pasqui23 wrote:

Last commit on https://github.com/buggins/hibernated
was almost a year ago

So what is the status of HibernateD?Should I use it if I need 
an ORM? Or would I risk unpatched security risks?


You can use Entity & Database library:

https://github.com/huntlabs/entity
https://github.com/huntlabs/database


I've tried both a while back, and they are still inferior to 
Hibernated (no surprise there, both projects are very new). There 
was even a "fun" SQL injection issue initially.
So, while in the long run Entity might be a great option, if you 
want an ORM that is as complete as Hibernated is today, it's not 
a solution (I have high hopes for it though).

Re: Is HibernateD dead?

[Matthias Klumpp via Digitalmars-d-learn]

I've written an email to Vadim, maybe we get a reply on the 
status of both projects.



On Friday, 4 May 2018 at 07:18:09 UTC, bauss wrote:

[...]
Would it maybe be easier for you to base on ddbc[1] or another 
existing abstraction layer for database abstraction?
Ddbc is pretty neat, and even has support for reading structs 
directly from the database.




Perhaps, but it'd have to be a forked version as I don't really 
want to depend on something that isn't updated regularly.


ddbc's last commit was a year ago.


Yes, at the moment using ddbc and relying on it would mean taking 
over some maintenance of it. Ddbc is fairly complete though and 
might save you a lot of work, because it already abstracts a lot 
of (relational) database systems.



I can't seem to find a license for it though?


It's Boost licensed (BSL-1.0), but probably needs an explicit 
LICENSE file.
Maybe we can move ddbc to dlang-community, so more people can 
easily commit changes to it (provided it gets accepted there, and 
Vadim agrees with that move as well).


Perhaps I will end up having another "optional" dependency to 
it as a temporary until I can have a better implementation or 
something.


Taking over maintenance of it might be easier than reimplementing 
the database abstraction again though.
For Postgres, ddbc worked really well for me, and I assume its 
SQLite, MySQL and ODBC drivers are also still working well, 
meaning less work for you.
Without ddbc, you'd have to write new abstraction on top of some 
other libraries, like dpq2.


The frontend part of postgresql is almost finished, it's just 
having the postgresql driver working properly, which is where 
it's frozen right now.


Hmm... Does any public code for that exist already that I 
could play around with?
Unfortunately, I have a few more unusual requirements for 
Postgres, like:

 * UUIDs as primary keys, instead of integers


As far as I remember the implementation of @DbId in Diamond, 
then it supports whatever type.


Native support for std.uuid.UUID would be neat :-) For Hibernated 
I use a mixin to convert a UUID into strings transparently, for 
database insertion.



Diamond doesn't care much about what type your primary key is.

I will make sure that's how it function of course, if it 
currently doesn't behave like it, but I'm pretty sure it does.


 * Ability to register custom datatypes with the ORM (version 
numbers in this case, the ORM can view them as text, but the 
database has a special type for them)


That could be done with some attribute that lets you handle 
columns yourself.


Do you have a good name for it?

I was thinking @DbProxy and then the function would be 
something like:

[...]


Registering a new type with Postgres yields a new OID to identify 
the type, so I would need a function to tell the Postgres backend 
to treat OIDs of a certain number like "text" types. Ideally, I 
would also need to annotate entity to set a specific column type, 
like:


```
class Entity {
UUID uuid;

@ColumType("versionnumber")
string _version;
}
```
(With the result of CREATE TABLE not setting a "text" type for 
the new version column, but a "versionnumber" type instead)


That would do it. For Hibernated, I have Hibernated create the 
table initially, and then fire an ALTER TABLE at it afterwards to 
change the column type, while at the same time registering the 
new type OID with ddbc to be treated as text.
As said, this is a very specific requirement very few people will 
have ^^


Much more frequently people will ask for JSONB and JSON type 
support for a postgres driver though, I guess. For that, 
specifying the column type explicitly could be quite helpful as 
well, so switch between JSONB and JSON.


 * Obviously the usual ORM stuff, one-to-many, many-to-many, 
etc. relations




Yes, relations is one thing I haven't added and I have been 
wanting to do it for a while.


I will definitely look into having it added as well.


That's kind of key for an ORM :-) Handling relations manually was 
what made me abandon my "I just write raw SQL for everything" 
ways, because it gets quite complex and annoying in the long run.


(Obviously not a must-have list, I added support for custom 
datatypes to my ddbc fork as well, because it's not really a 
feature many people need)


Well, that's kind of the key to most of the development in 
Diamond.


I usually add functionality that isn't widely used and in most 
cases people implement it themselves ex. the whole diamond.seo 
is not usually something a framework has.


I keep an eye on it - at the moment, Vibe.d satisfies all 
requirements I have on a web framework, but that might change.
A well integrated ORM would certainly be a game changer, since 
Vibe.d is limited to Mongo and Redis only.


Diamond is a neat project, I played around with it about half 
a year ago, but didn't test the ORM part at all back then.


It wasn't that good back then and has improved a lot since, as 
well many other 

Re: Is HibernateD dead?

[Matthias Klumpp via Digitalmars-d-learn]

On Thursday, 3 May 2018 at 21:28:18 UTC, bauss wrote:

On Thursday, 3 May 2018 at 18:01:07 UTC, Matthias Klumpp wrote:
DiamondMVC looks nice, but I would need PostgreSQL support for 
sure.

Therefore, I think there are three options:
 1) Extend the DiamondMVC ORM to support missing features that 
Hibernated has (maybe make it use ddbc as backend?)
 2) Revive Hibernated - contacting Vadim Lopatin would be key 
for that, and maybe the project could be maintained in the 
dlang-community organization (although there are competing 
projects for it...)
 3) Find a different D ORM that does the job and expand it to 
include missing features.


Yes, I completely agree with PostgreSQL support. It's really 
important to me getting that working, as well MSSQL. I was 
hoping I could find time this weekend to actually do that.


Would it maybe be easier for you to base on ddbc[1] or another 
existing abstraction layer for database abstraction?
Ddbc is pretty neat, and even has support for reading structs 
directly from the database.



[1]: https://github.com/buggins/ddbc

Perhaps I will end up having another "optional" dependency to 
it as a temporary until I can have a better implementation or 
something.


The frontend part of postgresql is almost finished, it's just 
having the postgresql driver working properly, which is where 
it's frozen right now.


Hmm... Does any public code for that exist already that I could 
play around with?
Unfortunately, I have a few more unusual requirements for 
Postgres, like:

 * UUIDs as primary keys, instead of integers
 * Ability to register custom datatypes with the ORM (version 
numbers in this case, the ORM can view them as text, but the 
database has a special type for them)
 * Obviously the usual ORM stuff, one-to-many, many-to-many, etc. 
relations


(Obviously not a must-have list, I added support for custom 
datatypes to my ddbc fork as well, because it's not really a 
feature many people need)


Diamond is a neat project, I played around with it about half a 
year ago, but didn't test the ORM part at all back then.

Re: Is HibernateD dead?

[Matthias Klumpp via Digitalmars-d-learn]

On Thursday, 3 May 2018 at 18:52:34 UTC, singingbush wrote:

On Thursday, 3 May 2018 at 10:27:47 UTC, Pasqui23 wrote:

Last commit on https://github.com/buggins/hibernated
was almost a year ago

So what is the status of HibernateD?Should I use it if I need 
an ORM? Or would I risk unpatched security risks?


Both hibernated and ddbc have had open pull requests for 
months. It's really frustrating.


Oh hey :-) I applied your patches for ddbc and hibernated a to my 
copy while back, because they weren't merged and fix real issues.
There are also other patches floating around, for example people 
will really want 
https://github.com/KrzaQ/hibernated/commit/efa38c50effdd77e973b174feea89016b8d1fa1f applied when using hibernated.


If there is enough interest, we can maybe provide at least some 
basic level of maintenance for these projects together, maybe 
under the dlang-community umbrella or similar.
Per adoption guidelines[1], I think the projects are popular 
enough, but Hibernated is of course not the only D ORM (although 
a pretty complete one), and the continued maintenance is also not 
sure, even when PRs finally get reviewed and accepted faster (but 
that really depends on the library users).


In any case, we need to get in contact with buggins. I asked him 
ages ago about Hibernated on Gitter, but that was probably the 
worst way to contact him (as he is active on Github, but probably 
never read that message).



[1]: https://github.com/dlang-community/discussions

Re: Is HibernateD dead?

[Matthias Klumpp via Digitalmars-d-learn]

On Thursday, 3 May 2018 at 10:27:47 UTC, Pasqui23 wrote:

Last commit on https://github.com/buggins/hibernated
was almost a year ago

So what is the status of HibernateD?Should I use it if I need 
an ORM? Or would I risk unpatched security risks?


Hah!
I was just browsing the forums thinking about the same issue and 
whether I should ask a question about it.
I am using Hibernated in one bigger project, ripping it out at 
this point would be quite painful and I only ever want to do that 
if there is a sustainable and actively developed alternative that 
is comparable in features[1].


Truth is, so far I haven't found any D ORM that compares to 
Hibernated in terms of supported features and databases. 
Hibernated also has issues though, at the time I maintain a 
forked version with changes that I hope to upstream soon - 
unfortunately, the trivial open pull-request on the project 
doesn't look promising.


DiamondMVC looks nice, but I would need PostgreSQL support for 
sure.

Therefore, I think there are three options:
 1) Extend the DiamondMVC ORM to support missing features that 
Hibernated has (maybe make it use ddbc as backend?)
 2) Revive Hibernated - contacting Vadim Lopatin would be key for 
that, and maybe the project could be maintained in the 
dlang-community organization (although there are competing 
projects for it...)
 3) Find a different D ORM that does the job and expand it to 
include missing features.


I really don't want to write ORMs in D and I actually lack the 
skills to do it properly, but I rely pretty heavily on Hibernated 
and ddbc. So, if anyone has a solution for this, I would help 
with it for sure.
Asking Vadim (buggins) on the state of Hibernated would be the 
first thing to do, I think.


Cheers,
Matthias

[1]: In fact, when I switched the database backend once in the 
past from an attempt to not use an ORM to using Hibernated, I was 
very close to rewriting the whole thing in Python - in D, there 
are tons of ORMs and database abstraction layers written, but not 
a single one compares even remotely to the likes of SQLAlchemy. 
It would be awesome if instead of 5 70% completed projects, we 
had one 90% complete one.

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Friday, 20 April 2018 at 18:30:30 UTC, Matthias Klumpp wrote:
On Friday, 20 April 2018 at 05:32:32 UTC, Dmitry Olshansky 
wrote:
On Friday, 20 April 2018 at 00:11:25 UTC, Matthias Klumpp 
wrote:

On Thursday, 19 April 2018 at 18:45:41 UTC, kinke wrote:

[...]

[...]


I think the order of operations is wrong, here is an example 
from containers:


allocator.dispose(buckets);
static if (useGC)
GC.removeRange(buckets.ptr);

If GC triggers between dispose and removeRange, it will likely 
segfault.


Indeed! It's also the only place where this is shuffled around, 
all other parts of the containers library do this properly.
The thing I wonder about is though, that the crash usually 
appeared in an explicit GC.collect() call when the application 
was not running multiple threads. At that point, the GC - as 
far as I know - couldn't have triggered after the buckets were 
disposed of and the ranges were removed. But maybe I am wrong 
with that assumption.

This crash would be explained perfectly by that bug.


Turns out that was indeed the case! I created a small testcase 
which managed to very reliably reproduce the issue on all 
machines that I tested it on. After reordering the 
dispose/removeRange, the crashes went away completely.
I submitted a pull request to the containers library to fix this 
issue: https://github.com/dlang-community/containers/pull/107


I will also try to get the patch into the components in Debian 
and Ubuntu, so we can maybe have a chance of updating the 
software center metadata for Ubuntu before 18.04 LTS releases 
next week.
Since asgen uses HashMaps for pretty much everything, an most of 
the time with GC-managed elements, this should improve the 
stability of the application greatly.


Thanks a lot for the help in debugging this, I learned a lot 
about DRuntime internals in the process. Also, it is no 
exaggeration to say that the appstream-generator project would 
not be written in D (there was a Rust prototype once...) and I 
would probably not be using D as much (or at all) without the 
helpful community around it.

Thank you :-)

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Friday, 20 April 2018 at 05:32:32 UTC, Dmitry Olshansky wrote:

On Friday, 20 April 2018 at 00:11:25 UTC, Matthias Klumpp wrote:

On Thursday, 19 April 2018 at 18:45:41 UTC, kinke wrote:

[...]


Jup, I did that already, it just took a really long time to 
run because when I made the change to print errno I also 
enabled detailed GC profiling (via the PRINTF* debug options). 
Enabling the INVARIANT option for the GC is completely broken 
by the way, I enforced the compile to work by casting to 
shared, with the result of the GC locking up forever at the 
start of the program.


[...]


I think the order of operations is wrong, here is an example 
from containers:


allocator.dispose(buckets);
static if (useGC)
GC.removeRange(buckets.ptr);

If GC triggers between dispose and removeRange, it will likely 
segfault.


Indeed! It's also the only place where this is shuffled around, 
all other parts of the containers library do this properly.
The thing I wonder about is though, that the crash usually 
appeared in an explicit GC.collect() call when the application 
was not running multiple threads. At that point, the GC - as far 
as I know - couldn't have triggered after the buckets were 
disposed of and the ranges were removed. But maybe I am wrong 
with that assumption.

This crash would be explained perfectly by that bug.

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Friday, 20 April 2018 at 00:11:25 UTC, Matthias Klumpp wrote:

[...]
Jup, I did that already, it just took a really long time to run 
because when I made the change to print errno [...]


I forgot to mention that, the error code was 12, ENOMEM, so this 
is actually likely not a relevant issue afterall.

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Thursday, 19 April 2018 at 18:45:41 UTC, kinke wrote:
On Thursday, 19 April 2018 at 17:01:48 UTC, Matthias Klumpp 
wrote:
Something that maybe is relevant though: I occasionally get 
the following SIGABRT crash in the tool on machines which have 
the SIGSEGV crash:

```
Thread 53 "appstream-gener" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fdfe98d4700 (LWP 7326)]
0x75040428 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:54
54  ../sysdeps/unix/sysv/linux/raise.c: No such file or 
directory.

(gdb) bt
#0  0x75040428 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:54

#1  0x7504202a in __GI_abort () at abort.c:89
#2  0x00780ae0 in core.thread.Fiber.allocStack(ulong, 
ulong) (this=0x7fde0758a680, guardPageSize=4096, sz=20480) at 
src/core/thread.d:4606
#3  0x007807fc in 
_D4core6thread5Fiber6__ctorMFNbDFZvmmZCQBlQBjQBf 
(this=0x7fde0758a680, guardPageSize=4096, sz=16384, dg=...)

at src/core/thread.d:4134
#4  0x006f9b31 in 
_D3std11concurrency__T9GeneratorTAyaZQp6__ctorMFDFZvZCQCaQBz__TQBpTQBiZQBx (this=0x7fde0758a680, dg=...)
at 
/home/ubuntu/dtc/dmd/generated/linux/debug/64/../../../../../druntime/import/core/thread.d:4126
#5  0x006e9467 in 
_D5asgen8handlers11iconhandler5Theme21matchingIconFilenamesMFAyaSQCl5utils9ImageSizebZC3std11concurrency__T9GeneratorTQCfZQp (this=0x7fdea2747800, relaxedScalingRules=true, size=..., iname=...) at ../src/asgen/handlers/iconhandler.d:196
#6  0x006ea75a in 
_D5asgen8handlers11iconhandler11IconHandler21possibleIconFilenamesMFAyaSQCs5utils9ImageSizebZ9__lambda4MFZv (this=0x7fde0752bd00)

at ../src/asgen/handlers/iconhandler.d:392
#7  0x0082fdfa in core.thread.Fiber.run() 
(this=0x7fde07528580) at src/core/thread.d:4436
#8  0x0082fd5d in fiber_entryPoint () at 
src/core/thread.d:3665

#9  0x in  ()
```


You probably already figured that the new Fiber seems to be 
allocating its 16KB-stack, with an additional 4 KB guard page 
at its bottom, via a 20 KB mmap() call. The abort seems to be 
triggered by mprotect() returning -1, i.e., a failure to 
disallow all access to the the guard page; so checking `errno` 
should help.


Jup, I did that already, it just took a really long time to run 
because when I made the change to print errno I also enabled 
detailed GC profiling (via the PRINTF* debug options). Enabling 
the INVARIANT option for the GC is completely broken by the way, 
I enforced the compile to work by casting to shared, with the 
result of the GC locking up forever at the start of the program.


Anyway, I think for a chance I actually produced some useful 
information via the GC debug options:

Given the following crash:
```
#0  0x007f1d94 in 
_D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv (this=..., 
ptop=0x7fdfce7fc010, pbot=0x7fdfcdbfc010)

at src/gc/impl/conservative/gc.d:1990
p1 = 0x7fdfcdbfc010
p2 = 0x7fdfce7fc010
stackPos = 0
[...]
```
The scanned range seemed fairly odd to me, so I searched for it 
in the (very verbose!) GC debug output, which yielded:

```
235.25: 0xc4f090.Gcx::addRange(0x8264230, 0x8264270)
235.244460: 0xc4f090.Gcx::addRange(0x7fdfcdbfc010, 0x7fdfce7fc010)
235.253861: 0xc4f090.Gcx::addRange(0x8264300, 0x8264340)
235.253873: 0xc4f090.Gcx::addRange(0x8264390, 0x82643d0)
```
So, something is calling addRange explicitly there, causing the 
GC to scan a range that it shouldn't scan. Since my code doesn't 
add ranges to the GC, and I looked at the generated code from 
girtod/GtkD and it very much looks fine to me, I am currently 
looking into EMSI containers[1] as the possible culprit.
That library being the issue would also make perfect sense, 
because this issue started to appear with such a frequency only 
after containers were added (there was a GC-related crash before, 
but that might have been a different one).


So, I will look into that addRange call next.

[1]: https://github.com/dlang-community/containers

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Thursday, 19 April 2018 at 08:30:45 UTC, Kagamin wrote:
On Wednesday, 18 April 2018 at 22:24:13 UTC, Matthias Klumpp 
wrote:

size_t memSize = pooltable.maxAddr - minAddr;
(https://github.com/ldc-developers/druntime/blob/ldc/src/gc/impl/conservative/gc.d#L1982
 )
That wouldn't make sense for a pool size...

The machine this is running on has 16G memory, at the time of 
the crash the software was using ~2.1G memory, with 130G 
virtual memory due to LMDB memory mapping (I wonder what 
happens if I reduce that...)


If big LMDB mapping causes a problem, try a test like this:
---
import core.memory;
void testLMDB()
{
//how do you use it?
}
void test1()
{
void*[][] a;
foreach(i;0..10)a~=new void*[1];
void*[][] b;
foreach(i;0..10)b~=new void*[1];
b=null;
GC.collect();

testLMDB();

GC.collect();
foreach(i;0..10)a~=new void*[1];
foreach(i;0..10)b~=new void*[1];
b=null;
GC.collect();
}
---


I tried something similar, with no effect.
Something that maybe is relevant though: I occasionally get the 
following SIGABRT crash in the tool on machines which have the 
SIGSEGV crash:

```
Thread 53 "appstream-gener" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fdfe98d4700 (LWP 7326)]
0x75040428 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:54
54  ../sysdeps/unix/sysv/linux/raise.c: No such file or 
directory.

(gdb) bt
#0  0x75040428 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:54

#1  0x7504202a in __GI_abort () at abort.c:89
#2  0x00780ae0 in core.thread.Fiber.allocStack(ulong, 
ulong) (this=0x7fde0758a680, guardPageSize=4096, sz=20480) at 
src/core/thread.d:4606
#3  0x007807fc in 
_D4core6thread5Fiber6__ctorMFNbDFZvmmZCQBlQBjQBf 
(this=0x7fde0758a680, guardPageSize=4096, sz=16384, dg=...)

at src/core/thread.d:4134
#4  0x006f9b31 in 
_D3std11concurrency__T9GeneratorTAyaZQp6__ctorMFDFZvZCQCaQBz__TQBpTQBiZQBx (this=0x7fde0758a680, dg=...)
at 
/home/ubuntu/dtc/dmd/generated/linux/debug/64/../../../../../druntime/import/core/thread.d:4126
#5  0x006e9467 in 
_D5asgen8handlers11iconhandler5Theme21matchingIconFilenamesMFAyaSQCl5utils9ImageSizebZC3std11concurrency__T9GeneratorTQCfZQp (this=0x7fdea2747800, relaxedScalingRules=true, size=..., iname=...) at ../src/asgen/handlers/iconhandler.d:196
#6  0x006ea75a in 
_D5asgen8handlers11iconhandler11IconHandler21possibleIconFilenamesMFAyaSQCs5utils9ImageSizebZ9__lambda4MFZv (this=0x7fde0752bd00)

at ../src/asgen/handlers/iconhandler.d:392
#7  0x0082fdfa in core.thread.Fiber.run() 
(this=0x7fde07528580) at src/core/thread.d:4436
#8  0x0082fd5d in fiber_entryPoint () at 
src/core/thread.d:3665

#9  0x in  ()
```

This is in the constructor of a std.concurrency.Generator:
auto gen = new Generator!string (...)

I am not sure what to make of this yet though... This goes into 
DRuntime territory that I actually hoped to never have to deal 
with as much as I apparently need to now.

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 18 April 2018 at 22:12:12 UTC, kinke wrote:
On Wednesday, 18 April 2018 at 20:36:03 UTC, Johannes Pfau 
wrote:
Actually this sounds very familiar: 
https://github.com/D-Programming-GDC/GDC/pull/236


Interesting, but I don't think it applies here. Both start and 
end addresses are 16-bytes aligned, and both cannot be accessed 
according to the stack trace (`pbot=0x7fcf4d721010 Cannot access memory at address 0x7fcf4d721010>, 
ptop=0x7fcf4e321010 0x7fcf4e321010>`). That's quite interesting too: `memSize = 
209153867776`. Don't know what exactly it is, but it's a pretty 
large number (~194 GB).


size_t memSize = pooltable.maxAddr - minAddr;
(https://github.com/ldc-developers/druntime/blob/ldc/src/gc/impl/conservative/gc.d#L1982
 )
That wouldn't make sense for a pool size...

The machine this is running on has 16G memory, at the time of the 
crash the software was using ~2.1G memory, with 130G virtual 
memory due to LMDB memory mapping (I wonder what happens if I 
reduce that...)

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 18 April 2018 at 20:36:03 UTC, Johannes Pfau wrote:

[...]

Actually this sounds very familiar: 
https://github.com/D-Programming-GDC/GDC/pull/236


it took us quite some time to reduce and debug this:

https://github.com/D-Programming-GDC/GDC/pull/236/commits/ 
5021b8d031fcacac52ee43d83508a5d2856606cd


So I wondered why I couldn't find this in the upstream druntime 
code. Turns out our pull request has never been merged


https://github.com/dlang/druntime/pull/1678


Just to be sure, I applied your patch, but unfortunately I still 
get the same result...


On Wednesday, 18 April 2018 at 20:38:20 UTC, negi wrote:

On Monday, 16 April 2018 at 16:36:48 UTC, Matthias Klumpp wrote:

...


This reminds me of (otherwise unrelated) problems I had 
involving Linux 4.15.


If you feel out of ideas, I suggest you take a look at the 
kernels.  It might
be that Ubuntu is turning some security-related knob in a 
different direction
than Debian.  Or it might be some bug in 4.15 (I found it to be 
quite buggy,
specially during the first few point releases; 4.15 was the 
first upstream
release including large amounts of meltdown/spectre-related 
work).


All the crashes are happening on a 4.4 kernel though... I am 
currently pondering digging out a 4.4 kernel here to see if that 
makes me reproduce the crash locally.

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 18 April 2018 at 20:40:52 UTC, Matthias Klumpp 
wrote:

[...]

If possible, I'd give static linking a try.


I tried that, with at least linking druntime and phobos 
statically. I did not, however, link all the things statically.
That is something to try (at least statically linking all the D 
libraries).


No luck...
```
#0  0x007f10e8 in 
_D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv (this=..., 
ptop=0x7fcf6a11b010, pbot=0x7fcf6951b010)

at src/gc/impl/conservative/gc.d:1990
p1 = 0x7fcf6951b010
p2 = 0x7fcf6a11b010
stackPos = 0
stack =
{{pbot = 0x7fffcc60, ptop = 0x7f15af 
<_D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv+1403>}, {pbot = 
0xc22bf0 <_D2gc6configQhSQnQm6Config>, ptop = 0xc4cd28}, {pbot = 
0x87b4118, ptop = 0x87b4118}, {pbot = 0x0, ptop = 0xc4cda0}, 
{pbot = 0x7fffcca0, ptop = 0x7f15af 
<_D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv+1403>}, {pbot = 
0xc22bf0 <_D2gc6configQhSQnQm6Config>, ptop = 0xc4cd28}, {pbot = 
0x87af258, ptop = 0x87af258}, {pbot = 0x0, ptop = 0xc4cda0}, 
{pbot = 0x7fffcce0, ptop = 0x7f15af 
<_D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv+1403>}, {pbot = 
0xc22bf0 <_D2gc6configQhSQnQm6Config>, ptop = 0xc4cd28}, {pbot = 
0x87af158, ptop = 0x87af158}, {pbot = 0x0, ptop = 0xc4cda0}, 
{pbot = 0x7fffcd20, ptop = 0x7f15af 
<_D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv+1403>}, {pbot = 
0xc22bf0 <_D2gc6configQhSQnQm6Config>, ptop = 0xc4cd28}, {pbot = 
0x87af0d8, ptop = 0x87af0d8}, {pbot = 0x0, ptop = 0xc4cda0}, 
{pbot = 0x7fdf6b265000, ptop = 0x69b96a0}, {pbot = 0x28, ptop = 
0x7fcf5951b000}, {pbot = 0x309eab7000, ptop = 0x7fdf6b265000}, 
{pbot = 0x0, ptop = 0x0}, {pbot = 0x1381d00, ptop = 0x1c}, {pbot 
= 0x1d, ptop = 0x1c}, {pbot = 0x1a44100, ptop = 0x1a4410}, {pbot 
= 0x1a44, ptop = 0x4}, {pbot = 0x7fdf6b355000, ptop = 0x69b96a0}, 
{pbot = 0x28, ptop = 0x7fcf5951b000}, {pbot = 0x309eab7000, ptop 
= 0x4ac0}, {pbot = 0x4a, ptop = 0x0}, {pbot = 0x1381d00, ptop = 
0x1c}, {pbot = 0x1d, ptop = 0x1c}, {pbot = 0x4ac00, ptop = 
0x4ac0}, {pbot = 0x4a, ptop = 0x4}}

pcache = 0
pools = 0x69b96a0
highpool = 40
minAddr = 0x7fcf5951b000
memSize = 208820465664
base = 0xaef0
top = 0xae
p = 0x4618770
pool = 0x0
low = 110859936
high = 40
mid = 140528533483520
offset = 208820465664
biti = 8329709
pn = 142275872
bin = 1
offsetBase = 0
next = 0xc4cc80
next = {pbot = 0x7fffcbe0, ptop = 0x7f19ed 
<_D2gc4impl12conservativeQw3Gcx7markAllMFNbbZ14__foreachbody3MFNbKSQCm11gcinterface5RangeZi+57>}

__r292 = 0x7fffd320
__key293 = 8376632
rng = @0x0: 
#1  0x007f19ed in 
_D2gc4impl12conservativeQw3Gcx7markAllMFNbbZ14__foreachbody3MFNbKSQCm11gcinterface5RangeZi (this=0x7fffd360, __applyArg0=...)

at src/gc/impl/conservative/gc.d:2188
range = {pbot = 0x7fcf6951b010, ptop = 0x7fcf6a11b010, ti 
= 0x0}
#2  0x007fd161 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf7opApplyMFNbMDFNbKQBtZiZ9__lambda2MFNbKxSQCpQCpQCfZi (this=0x7fffd320, e=...) at src/rt/util/container/treap.d:47
#3  0x007fd539 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (dg=..., node=0x80396c0) at src/rt/util/container/treap.d:221

result = 0
#4  0x007fd565 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (dg=..., node=0x87c8140) at src/rt/util/container/treap.d:224

result = 0
#5  0x007fd516 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (dg=..., node=0x7fdfc8000950) at src/rt/util/container/treap.d:218

result = 16844032
#6  0x007fd516 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (dg=..., node=0x7fdfc8000a50) at src/rt/util/container/treap.d:218

result = 0
#7  0x007fd516 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (dg=..., node=0x7fdfc8000c50) at src/rt/util/container/treap.d:218

result = 0

[etc...]
#37 0x0077e889 in core.memory.GC.collect() () at 
src/core/memory.d:207
#38 0x006b4791 in asgen.engine.Engine.gcCollect() 
(this=0x77ee13c0) at ../src/asgen/engine.d:122

```

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 18 April 2018 at 18:55:48 UTC, kinke wrote:

On Wednesday, 18 April 2018 at 10:15:49 UTC, Kagamin wrote:
There's a number of debugging options for GC, though not sure 
which

ones are enabled in default debug build of druntime


Speaking for LDC, none are, they all need to be enabled 
explicitly. There's a whole bunch of them 
(https://github.com/dlang/druntime/blob/master/src/gc/impl/conservative/gc.d#L20-L31), so enabling most of them would surely help in tracking this down, but it's most likely still going to be very tedious.
I'm not really surprised that there are compilation errors when 
enabling the debug options, that's a likely fate of untested 
code unfortunately.


Yeah... Maybe making a CI build with "enable all the things" 
makes sense to combat that...



If possible, I'd give static linking a try.


I tried that, with at least linking druntime and phobos 
statically. I did not, however, link all the things statically.
That is something to try (at least statically linking all the D 
libraries).

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 18 April 2018 at 10:15:49 UTC, Kagamin wrote:
You can call GC.collect at some points in the program to see if 
they can trigger the crash


I already do that, and indeed I get crashes. I could throw those 
calls into every function though, or make a minimal pool size, 
maybe that yields something...



https://dlang.org/library/core/memory/gc.collect.html
If you link against debug druntime, GC can check invariants for 
correctness of its structures. There's a number of debugging 
options for GC, though not sure which ones are enabled in 
default debug build of druntime: 
https://github.com/ldc-developers/druntime/blob/ldc/src/gc/impl/conservative/gc.d#L1388


I get compile errors for the INVARIANT option, and I don't 
actually know how to deal with those properly:

```
src/gc/impl/conservative/gc.d(1396): Error: shared mutable method 
core.internal.spinlock.SpinLock.lock is not callable using a 
shared const object
src/gc/impl/conservative/gc.d(1396):Consider adding const 
or inout to core.internal.spinlock.SpinLock.lock
src/gc/impl/conservative/gc.d(1403): Error: shared mutable method 
core.internal.spinlock.SpinLock.unlock is not callable using a 
shared const object
src/gc/impl/conservative/gc.d(1403):Consider adding const 
or inout to core.internal.spinlock.SpinLock.unlock

```

Commenting out the locks (eww!!) yields no change in behavior 
though.


The crashes always appear in 
https://github.com/dlang/druntime/blob/master/src/gc/impl/conservative/gc.d#L1990


Meanwhile, I also tried to reproduce the crash locally in a 
chroot, with no result. All libraries used between the machine 
where the crashes occur and my local machine were 100% identical, 
the only differences I am aware of are obviously the hardware 
(AWS cloud vs. home workstation) and the Linux kernel (4.4.0 vs 
4.15.0)


The crash happens when built with LDC or DMD, that doesn't 
influence the result. Copying over a binary from the working 
machine to the crashing one also results in the same errors.


I am completely out of ideas here. Since I think I can rule out a 
hardware fault at Amazon, I don't even know what else would make 
sense to try.

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 17 April 2018 at 08:23:07 UTC, Kagamin wrote:

Other stuff to try:
1. run application compiled on debian against ubuntu libs
2. can you mix dependencies from debian and ubuntu?


I haven't tried that yet (next on my todo list), if I do run the 
program compiled with address sanitizer on Debian, I do get 
errors like:

```
AddressSanitizer:DEADLYSIGNAL
=
==25964==ERROR: AddressSanitizer: SEGV on unknown address 
0x7fac8db3f800 (pc 0x7fac9c433430 bp 0x0008 sp 
0x7ffc92be3dd0 T0)

==25964==The signal is caused by a READ memory access.
#0 0x7fac9c43342f in 
_D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv 
(/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xa142f)
#1 0x7fac9c433a2f in 
_D2gc4impl12conservativeQw3Gcx7markAllMFNbbZ14__foreachbody3MFNbKSQCm11gcinterface5RangeZi (/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xa1a2f)
#2 0x7fac9c459ad4 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xc7ad4)
#3 0x7fac9c459ac6 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xc7ac6)
#4 0x7fac9c459ac6 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xc7ac6)
#5 0x7fac9c459ac6 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf13opApplyHelperFNbxPSQDeQDeQDcQCv__TQCsTQCpZQDa4NodeMDFNbKxSQDiQDiQCyZiZi (/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xc7ac6)
#6 0x7fac9c459a51 in 
_D2rt4util9container5treap__T5TreapTS2gc11gcinterface5RangeZQBf7opApplyMFNbMDFNbKQBtZiZi (/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xc7a51)
#7 0x7fac9c430f26 in 
_D2gc4impl12conservativeQw3Gcx11fullcollectMFNbbZm 
(/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0x9ef26)
#8 0x7fac9c431226 in 
_D2gc4impl12conservativeQw14ConservativeGC__T9runLockedS_DQCeQCeQCcQCnQBs18fullCollectNoStackMFNbZ2goFNbPSQEaQEaQDyQEj3GcxZmTQvZQDfMFNbKQBgZm (/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0x9f226)
#9 0x7fac9c4355d0 in gc_term 
(/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xa35d0)
#10 0x7fac9c443ab2 in rt_term 
(/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xb1ab2)
#11 0x7fac9c443e65 in 
_D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZv 
(/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xb1e65)
#12 0x7fac9c443d0b in _d_run_main 
(/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xb1d0b)
#13 0x7fac9b9cfa86 in __libc_start_main 
(/lib/x86_64-linux-gnu/libc.so.6+0x21a86)
#14 0x55acd1dbe1d9 in _start 
(/home/matthias/Development/AppStream/generator/build/src/asgen/appstream-generator+0xba1d9)


AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV 
(/usr/lib/x86_64-linux-gnu/libdruntime-ldc-shared.so.78+0xa142f) 
in _D2gc4impl12conservativeQw3Gcx4markMFNbNlPvQcZv

==25964==ABORTING
```
So, I don't think this bug is actually limited to Ubuntu, it just 
shows up there more often for some reason.

Re: Issues with debugging GC-related crashes #2

[Matthias Klumpp via Digitalmars-d]

On Monday, 16 April 2018 at 16:36:48 UTC, Matthias Klumpp wrote:

[...]
The code uses std.typecons.scoped occasionally, does no GC 
allocations in destructors and does nothing to mess with the GC 
in general. There are a few calls to GC.add/removeRoot in the 
gir-to-d generated code (ObjectG.d), but those are very 
unlikely to cause issues (removing them did yield the same 
crash, and the same code is used by more projects).

[...]


Another thing to mention is that the software uses LMDB[1] and 
mmaps huge amounts of data into memory (gigabyte range).

Not sure if that information is relevant at all though.

[1]: https://symas.com/lmdb/technical/

Re: LDC 1.8.0

[Matthias Klumpp via Digitalmars-d-announce]

On Thursday, 29 March 2018 at 08:10:11 UTC, Martin Tschierschke 
wrote:
On Saturday, 24 March 2018 at 17:33:18 UTC, Matthias Klumpp 
wrote:
On Tuesday, 13 March 2018 at 01:52:48 UTC, Matthias Klumpp 
wrote:

[...]
Aww, just a little bit too late to easily get into Ubuntu 
18.04 LTS


Well It still made it, yay! (Even without me explicitly 
requesting it)
This means Ubuntu 18.04 will be pretty up-to-date when it 
comes to D stuff, only GDC 8 won't be the default (but still 
available).
The thing that is facilitating an up-to-date D stack in Debian 
and Ubuntu is software in the archive using D. The Tilix 
terminal emulator is at the forefront there, followed by my 
appstream-generator and the Laniakea archive management suite 
and all the bits and pieces those projects depend on (like 
GtkD in Tilix' case).


This is very cool! Will DMD become part of Ubuntu, too?


No, it's too late for that and adding DMD to Debian (and thereby 
Ubuntu) would be a significant amount of work.
For compiling D code in Debian/Ubuntu, we will pretty much always 
use LDC or GDC, because we want stronger optimization and better 
architecture support, while compilation time doesn't matter at 
all.
That being said, with DMD being under a completely free license 
now, the only thing that is preventing it from being in Debian is 
lack of manpower in the D team (having it would be very useful!).
So, if anyone is interested in helping out with maintaining 
compiler packages, D libraries or tools, please join the Debian D 
team![1].
You don't have to be a Debian developer to help, but you should 
ideally be somewhat familiar with Debian's policies and 
packaging. Me or others in the D team who are Debian developers 
can review the changes and sponsor them into the Debian and 
Ubuntu archives.


Since Ubuntu 18.04 LTS releases this April, getting DMD in that 
release will be almost impossible (feature freeze is very soon), 
but we could have it in Ubuntu 18.10, if someone creates packages 
for it.


[1]: https://salsa.debian.org/d-team/ - get a guest account: 
https://signup.salsa.debian.org/

Re: LDC 1.8.0

[Matthias Klumpp via Digitalmars-d-announce]

On Tuesday, 13 March 2018 at 01:52:48 UTC, Matthias Klumpp wrote:

[...]
Aww, just a little bit too late to easily get into Ubuntu 18.04 
LTS


Well It still made it, yay! (Even without me explicitly 
requesting it)
This means Ubuntu 18.04 will be pretty up-to-date when it comes 
to D stuff, only GDC 8 won't be the default (but still available).
The thing that is facilitating an up-to-date D stack in Debian 
and Ubuntu is software in the archive using D. The Tilix terminal 
emulator is at the forefront there, followed by my 
appstream-generator and the Laniakea archive management suite and 
all the bits and pieces those projects depend on (like GtkD in 
Tilix' case).

Re: LDC 1.8.0

[Matthias Klumpp via Digitalmars-d-announce]

On Tuesday, 13 March 2018 at 10:27:49 UTC, kinke wrote:
On Tuesday, 13 March 2018 at 01:52:48 UTC, Matthias Klumpp 
wrote:
The new and enforced "-shared" suffixes for the druntime and 
phobos shared libraries are a bit annoying (especially since 
this is a breaking change), but at least at Debian we'll 
follow upstream's defaults on that (it gives weird package 
names, but aside from that cosmetic issue the breakage caused 
by that change should be minimal).


Thank you for working on LDC! :-)


Thank you for maintaining the Debian/Ubuntu packages.

If you don't want the "-shared" suffix (and still only provide 
shared libs, no static ones), then getting rid of it should be 
as easy as patching


Unfortunately, we ship both - while Debian packages only use the 
shared libraries, and we want the shared libraries to be default, 
we also want to make the static ones available for people who 
want to opt into that for software they build.


1) 
https://github.com/ldc-developers/ldc/blob/v1.8.0/runtime/CMakeLists.txt#L116
2) 
https://github.com/ldc-developers/ldc/blob/v1.8.0/driver/main.cpp#L507


The only important thing is that `-link-defaultlib-shared` 
switch works (and that you error out for 
`-link-defaultlib-shared=false` if you don't ship with static 
libs).


I wonder if we could suffix the static libraries instead and 
whether that would cause any problems...


Cheers,
Matthias

Re: LDC 1.8.0

[Matthias Klumpp via Digitalmars-d-announce]

On Sunday, 4 March 2018 at 22:37:21 UTC, kinke wrote:

Hi everyone,

on behalf of the LDC team, I'm glad to announce LDC 1.8. The 
highlights of this version in a nutshell:


* Based on D 2.078.3.
* New switch `-link-defaultlib-shared` to link against shared 
druntime/Phobos.


Aww, just a little bit too late to easily get into Ubuntu 18.04 
LTS (since updating LDC will trigger a library transition, I am 
not sure if a case can be made to still include it).


The new and enforced "-shared" suffixes for the druntime and 
phobos shared libraries are a bit annoying (especially since this 
is a breaking change), but at least at Debian we'll follow 
upstream's defaults on that (it gives weird package names, but 
aside from that cosmetic issue the breakage caused by that change 
should be minimal).


Thank you for working on LDC! :-)

Re: Want to start a graphical Hello world.

[Matthias Klumpp via Digitalmars-d]

On Sunday, 21 January 2018 at 04:16:10 UTC, MHE wrote:

[...]
For this i have made a folder named GIT in the linux directory 
/usr/local/GIT

And Gitcloned in the GIT folder :
git clone https://github.com/nomad-software/x11
git clone https://github.com/nomad-software/tcltk.git
The clones are now in /usr/local/GIT/


Hmm, but you are using GtkD in your example... It looks like you 
compiled the wrong code.
What you actually want is GtkD from 
https://github.com/gtkd-developers/GtkD or installed from the 
Debian repositories (in case you want to use LDC) via `sudo apt 
install libgtkd-3-dev`



When i start the upper code in a cmd the result is as follows :
___
$ dmd hw_graphical.d
hw_graphical.d(1): Error: module MainWindow is in file 
'gtk/MainWindow.d' which cannot be read

import path[0] = /usr/include/dmd/phobos
import path[1] = /usr/include/dmd/druntime/import


What must be done that the code make a graphical window ?


If you want to use GtkD and achieve a quick result, you might 
want to use the Dub package manager.
GtkD also has a few demo applications that show you how to use it 
with dub, take a look at

https://github.com/gtkd-developers/GtkD/tree/master/demos/gtk

I want to make the D programming tkd module ready for GUI 
programming.

Need a step by step advice for BEGINNERS in D programming.


tkd is a project different from GtkD (which your example above is 
using). For "modern" UI that integrates well with GNOME, you will 
highly likely want to use GtkD. Tcl/Tk is of course an option as 
well, but the D bindings look less complete and well maintained.
In case you want to use Tk, you will need to change your code to 
actually use it, instead of GTK+ though ;-)


Any Internetlink for step by step instructions how to arrange D 
and TK would be helpful !


I can't help with Tk, but for GTK+ a quick Google search found 
https://sites.google.com/site/gtkdtutorial/ - this tutorial is 
rather old, but it might be useful as a reference.


This experience report by Gerald Nunn might also be an 
interesting read for you: 
https://gexperts.com/wp/learning-d-and-gtk/


Cheers,
Matthias

Re: Using Postgres connection functions

[Matthias Klumpp via Digitalmars-d-learn]

On Saturday, 13 January 2018 at 17:58:14 UTC, Joe wrote:
On Saturday, 13 January 2018 at 10:10:41 UTC, Jacob Carlborg 
wrote:
There's a native D library, ddb [1], for connecting to 
Postgres. Then you don't have to worry about null-terminated 
strings.


There are several D libraries that I would consider "native": 
derelict-pq, dpq, dpq2 and ddb. The latter perhaps has the 
distinction that it doesn't use libpq, but rather implements 
the Postgres FE/BE protocol. That's a bit *too* native for my 
taste. It means the library maintainer has to keep up with 
changes to the internal protocol, which although published, the 
Postgres group doesn't have to maintain compatibility from 
version to version. For example, they haven't dropped the 
PQsetdbLogin function even though the PQconnectdb and 
PQconnectdbParams functions are obviously preferred. OTOH, 
there used to be an AsciiRow message format in the protocol, 
that was dropped, unceremoniously (not even mentioned in the 
release notes).


If you are after a good way to use Postgres in a real-world 
application, I highly recommend ddbc[1] (which also supports 
other backends).
There are a lot of D Postgres bindings out there, and all of them 
are about 70% completed, but nobody really bothered to make one 
finished and really good (and well maintained) binding. DDBC is 
really close to being complete, and contains a few convenience 
features that make it nice to use in an application. It also is 
used by Hibernated[2] in case you want an ORM for your app at 
some point.
Both libraries aren't up to tools like SQLAlchemy & Co. from 
other programming languages, but they are decent.

For simple cases, dpq2 & Co. might work well enough as well.
In any case, please don't start another Postgres library and 
consider contributing to one of the existing ones, so that we 
maybe have one really awesome, 100% complete library at some 
point.


If, on the other hand, your goal is to learn about the low-level 
Postgres interface and not just to have a Postgres interface for 
an application you develop, by all means, play with it :-)


Cheers,
Matthias

[1]: https://github.com/buggins/ddbc
[2]: https://github.com/buggins/hibernated

Re: gdc is in

[Matthias Klumpp via Digitalmars-d]

On Sunday, 8 October 2017 at 08:38:15 UTC, Iain Buclaw wrote:
On 7 October 2017 at 19:42, Nordlöw via Digitalmars-d 
 wrote:

On Friday, 6 October 2017 at 15:21:05 UTC, jmh530 wrote:


I would think this would be bigger news...I mean LDC isn't 
even on 2.076 yet...


I very much so agree. This is fantastic news!

Are there any beta (alpha|beta|rc)-builds available for 
download?


No, but then again the poor little server is so constrained on 
resources at the moment, I'll be wanting to move to a new 
provider (replacing the single server with four) before such a 
thing happens.


I've had to add more swap because CI builds for mips, ppc and 
sparc started failing due to hitting the oomkiller.


Donating for the upkeep of our infrastructure is also welcome. 
;-)


This might be worthwhile. It is highly likely that when GDC is in 
GCC it will become the default D compiler in Red Hat Enterprise 
Linux (while real support in RHEL even maybe), other Linux 
distros like SUSE and even that we'll use it as the default D 
compiler in Debian, due to GCC being the default system compiler 
with strong optimizations and very good support for multiple 
architectures (not sure yet though, since LDC works well as well 
when it's not hit by the (suprisingly frequent) LLVM bugs).


So, in any case, GDC in GCC is *huge* and there is a high chance 
that people will come in contact with D through it first.
When it's in, I can try to get GDC a few more machines for CI (I 
can't promise anything at the moment, unfortunately).

Re: Released vibe.d 0.8.1

[Matthias Klumpp via Digitalmars-d-announce]

On Thursday, 31 August 2017 at 11:56:53 UTC, aberba wrote:
On Wednesday, 30 August 2017 at 16:53:40 UTC, Matthias Klumpp 
wrote:
On Wednesday, 30 August 2017 at 07:47:53 UTC, Sönke Ludwig 
wrote:
Apart from removing the old vibe-d:diet package in favor of 
diet-ng, this release most notably contains a number of 
performance improvements in the HTTP server, as well as 
improvements and fixes in the WebSocket code. Furthermore, 
initial OpenSSL 1.1.x support has been added and a few @safe 
related issues introduced in 0.8.0 have been fixed.


Change log:
https://vibed.org/blog/posts/vibe-release-0.8.1

DUB package:
https://code.dlang.org/packages/vibe-d/0.8.1


Debian packages are on their way too :-) (pending approval 
from our archive masters). Granted, this is most useful for 
Vibe.d-using software that wants to be in Debian.


Now, the only thing I am missing in Vibe is a good interface 
to PostgreSQL, because in some circumstances MongoDB is just a 
very bad choice.
(Postgres even outperforms it in my testcase, and PG supports 
JSON/BSON as well now).

DPQ2[1] looks very promising though :-)

Thank you for making Vibe.d!

[1]: https://github.com/denizzzka/dpq2


Even with mysql (using mysql-native), the absent of something 
like


struct User {
@optional int userName; //its ok if row doesn't have this 
column

@as("phone_number") string phoneNumber;

}

User[] users;

foreach(row; ...)
{
users ~= row.toStruct!User;
}



I would love that :-)

MongoDB has facilities for these stuff automatically thats why 
using it seem convenient and speeds up stuff.


Yeah, it speeds up development, but that doesn't help much if 
Mongo swallows your data or messes up replication, or if your 
data by its nature simply does not fit a document store well.
I've found Postgres to be very reliable and also very fast in the 
past, and - in large part thanks to Vibe.d - I am using Mongo now 
in a test deployment instead of it. I don't feel comfortable at 
all in continuing to use it though, which is unfortunate, since 
the development speed with it is quite high.

Re: Released vibe.d 0.8.1

[Matthias Klumpp via Digitalmars-d-announce]

On Wednesday, 30 August 2017 at 07:47:53 UTC, Sönke Ludwig wrote:
Apart from removing the old vibe-d:diet package in favor of 
diet-ng, this release most notably contains a number of 
performance improvements in the HTTP server, as well as 
improvements and fixes in the WebSocket code. Furthermore, 
initial OpenSSL 1.1.x support has been added and a few @safe 
related issues introduced in 0.8.0 have been fixed.


Change log:
https://vibed.org/blog/posts/vibe-release-0.8.1

DUB package:
https://code.dlang.org/packages/vibe-d/0.8.1


Debian packages are on their way too :-) (pending approval from 
our archive masters). Granted, this is most useful for 
Vibe.d-using software that wants to be in Debian.


Now, the only thing I am missing in Vibe is a good interface to 
PostgreSQL, because in some circumstances MongoDB is just a very 
bad choice.
(Postgres even outperforms it in my testcase, and PG supports 
JSON/BSON as well now).

DPQ2[1] looks very promising though :-)

Thank you for making Vibe.d!

[1]: https://github.com/denizzzka/dpq2

Re: D and Meson

[Matthias Klumpp via Digitalmars-d]

On Monday, 19 June 2017 at 12:21:24 UTC, Mike B Johnson wrote:

[...]
Funny: "The main design point of Meson is that every moment a 
developer spends writing or debugging build definitions is a 
second wasted. So is every second spent waiting for the build 
system to actually start compiling code."


Which is in direct contradiction to what Walter has said... and 
yet Walter is suppose to be all about fast cars and hot women.


Walter said you should invest a lot of time waiting for the build 
process? :P


On Wednesday, 14 June 2017 at 15:25:55 UTC, Russel Winder wrote:

[...]
If the person running the D support for Meson is on this list 
please contact me privately to tell me what I can do to help 
progress that support further.


Any changes that upstream Meson is happy with are fine :-)
The best thing to help with improving Meson support one can 
possibly do at the moment would be fixing this DMDFE feature 
request: https://issues.dlang.org/show_bug.cgi?id=16746
This will also benefit a lot of other build systems that use 
incremental builds.

Re: dmd debian installation conflicts with debian-goodies

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 28 June 2017 at 10:09:06 UTC, Ralph Amissah wrote:
Installing dmd if debian-goodies is installed fails. Both try 
to write a file named '/usr/bin/dman'


Debian Stretch is out, the freeze is over, perhaps now dmd will 
soon be
available as a package in Debian? Ldc2 does a great job but for 
testing
purposes and convenience it would be good to have the reference 
compiler.


Long-term, we will likely be using GDC in Debian as default D 
compiler, if that becomes viable. That GDC is in GCC now is a 
very big deal, which makes maintaining D in Debian and any Linux 
distribution (which uses GCC as system compiler) much easier.
Also, there is some company interest now, since it is expected 
that GCC/GDC will hit enterprise distributions such as RHEL as 
well, and thereby be widely available.


That being said, I want DMD to be available in Debian, and LDC is 
doing a very good job at the moment and is serving as our 
de-facto default D compiler.
Unfortunately now that the dman binary name is taken, DMD can't 
have it in Debian and that binary would have to be renamed, even 
if just temporarily in case we could convince the -goodies 
maintainer to change the name of the existing binary.


Is there likely to be D related activity at DebCamp and DebConf 
2017, Montreal?


Nothing is planned yet, but if there is interest in it, I would 
be happy to organize a BoF session there.


Cheers,
Matthias

Re: D support for the Meson build system

[Matthias Klumpp via Digitalmars-d-announce]

Btw, to make Meson and other build systems work really well, we 
would need this bug fixed in DMDFE: 
https://issues.dlang.org/show_bug.cgi?id=16746
At the moment, one needs to ninja clean way too often to get a 
good build.

Re: Persistent key-value-store for D?

[Matthias Klumpp via Digitalmars-d-learn]

On Wednesday, 26 April 2017 at 17:06:52 UTC, krylon wrote:

[...]
If I understand what I have read so far correctly, it is 
possible to access libraries written in C or C++ from D - in 
that case, I could just use Tokyocabinet directly, but I have 
not found any pointers on how to do this. Is this a feasible 
option, and if so, where can I find documentation on how to do 
this?


I can recommend using LMDB[1] which likely does all you want.
It also has D bindings[2]. We use it in the AppStream generator 
with multithreaded parallelization and it is working great so far 
(although for the usecase of asgen it's not an ideal database 
choice, but one which makes deployments of the generator really 
simple).


I explored other solutions including Tokyokabinet and LevelDB and 
LMDB was by far the fastest.


Cheers,
Matthias


[1]: 
https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database
[2]: 
https://github.com/ximion/appstream-generator/blob/master/src/asgen/bindings/lmdb.d

Re: htod for linux

[Matthias Klumpp via Digitalmars-d-learn]

You may also want to try dstep - I just recently used that tool 
for the first time, and it worked really well (required a bit of 
tweaking of the source and result files though, but nothing 
major).


=> https://github.com/jacob-carlborg/dstep

Re: [OT] Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 18:14:40 UTC, David Nadlinger wrote:
On Tuesday, 11 April 2017 at 12:03:27 UTC, Matthias Klumpp 
wrote:
On Monday, 10 April 2017 at 22:15:53 UTC, David Nadlinger 
wrote:
So do we need to put a reminder about the ABI being unstable 
into set of every release notes to make sure we won't get 
angry bug reports once users actually build their own D code 
against your packages? ;)


Nah, there are several options here, one would simply be to 
tell people not to use the distro packages with anything but 
the default D compiler used in the respective Debian release.


So as long as one sticks to packages in the official apt repos, 
all the libraries are guaranteed to be built with the 
distributed compiler as well?


Yes. Unfortunately there will be three of them which aren't 
compatible with each other, so we will kind of have to settle 
with one as default.


When you mentioned that you'd read the release notes regarding 
the ABI change, I got the impression that you had to manually 
rebuild the world for that to happen – hence my tongue-in-cheek 
remark about reminding you to do this in the release notes.


Well, it's a matter of telling the build admins or making a 
proper transition package (doesn't exit yet for D), but yeah, 
technically we'd need to rebuild all D stuff on ABI changes.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 15:31:46 UTC, David Nadlinger wrote:
On Tuesday, 11 April 2017 at 12:38:01 UTC, Matthias Klumpp 
wrote:
If you could change the SOVERSION with every one of these 
changes, or simply just tie it to the respective Phobos 
release, distributions would automatically do the right thing 
and compile all D code using Phobos against the new version.


As you mention, this is already done in LDC; not just the 
Debian packages, but also upstream. The soname will always be 
`libphobos2-ldc.so.74` or what have you.


(Thinking about it, it should probably include the patch 
version as well, although we haven't had a situation so far 
where we would have wanted to release multiple LDC versions for 
ABI-incompatible patch releases.)


Phobos is versioned properly in both GDC and LDC, and as long as 
that continues, no problems exist at all :-)

https://packages.debian.org/search?suite=stretch=phobos

This would - as said - also work for other D shared libraries, 
unless the D ABI in general breaks or someone tries to build a 
program with GDC and uses a library that was built with LDC or 
DMD (or any other possible compiler combination).

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 14:43:15 UTC, Russel Winder wrote:
On Tue, 2017-04-11 at 14:21 +, Matthias Klumpp via 
Digitalmars-d wrote:



[…]
At time I am playing around with the idea of using 
pkg-config[1]

files to enlist the sources a D library consists of.
By doing that, we would have a very build-system agnostic way 
of

doing storing that information that can be used by Automake
(native), Dub, Meson and even plain Makefiles. It's also very
widely used (although it is commonly not used to list sources).


And SCons.

What about CMake? (Which is the CLion build system using Make 
underneath, but they are rapidly moving to supporting Ninja.)


CMake supports this as well: 
https://cmake.org/cmake/help/v3.0/module/FindPkgConfig.html


The current idea is in case a library "foobar" would be packaged, 
to have a "foobar-src.pc" pkgconfig file (additionally to a 
potentially also existing "foobar.pc" file), containing something 
like this:


```
prefix=/usr/local
exec_prefix=${prefix}
includedir=${prefix}/include/d/foo

Name: foobar
Description: The foo library (sources)
Version: 1.0.0
Cflags: -I${includedir} -d-version=blahblub
Sources: ${includedir}/alpha.d ${includedir}/beta.d 
${includedir}/gamma.d

```

Build systems would then need to explicitly fetch the Sources 
field and add its expanded values to the project's sources.
(Using Cflags for this would be messy, as the build system might 
want to deal with flags and sources separately).


Alternatively dub could also define a layout and we write plugins 
for each build-system to make it work.


This will be really annoying with large libraries like GtkD 
though, which will require substantially longer to build. Maybe 
it's worth keeping some libraries precompiled (thereby locking 
all their reverse-dependencies to whatever D compiler was used to 
compile the library).


One problem with the pkg-config approach is that to support the 
precompiled case too, build systems would need to search for both 
"foobar" and "foobar-src" and only fail dependency detection if 
both of them are missing.


At least with CMake/Meson that's easy to do though (although it's 
a bit messy).

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 14:49:03 UTC, Russel Winder wrote:

[...]
Having played a bit with GtkD, you always want this as a shared 
library for development.


Yeah, GtkD is pretty massive and takes quite a large amount of 
time to compile... Redoing that for each software depending on it 
is pretty wasteful.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 14:26:37 UTC, rikki cattermole wrote:

[...]
The problem with /usr/include/d is that is where .di files 
would be located not .d. This would also match up with the 
c/c++ usage of it.


When I asked about this a while back, I was told to just install 
the sources into the include D as "almost nobody uses .di files 
except for proprietary libraries" (and do those even exist?).
But in any case, any path would be fine with me as long as people 
can settle on using it - `/usr/share/d` would be available ^^

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 14:04:44 UTC, rikki cattermole wrote:

[...]
/usr/share/source/D/package-name-version

Add a search path like that to Dub and create source only 
library packages and that is pretty much all the distribution 
we need for libraries I reckon.


It's more likely that the path would be `/usr/include/d` because 
that's the place the D sources seem to usually reside, especially 
if they are also used as "headers" for shared libraries.


At time I am playing around with the idea of using pkg-config[1] 
files to enlist the sources a D library consists of.
By doing that, we would have a very build-system agnostic way of 
doing storing that information that can be used by Automake 
(native), Dub, Meson and even plain Makefiles. It's also very 
widely used (although it is commonly not used to list sources).


This would also jive really well with the existing D packages in 
Debian and we could - if we go down the source-only route - still 
decide to precompile a few selected libraries, for example in 
case they are very time consuming to compile or really large.


This will *not* solve the issues with Phobos breakage though, as 
Phobos is a shared library. But doing that would pretty much work 
around all other problems, except for the cost of having 
statically linked binaries, but that seems to be inevitable 
anyway given the state of things.


(Disclaimer: This is not a D team policy draft yet, just an 
opinion at the moment that might be refined or changed entirely)


[1]: https://www.freedesktop.org/wiki/Software/pkg-config/

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 13:43:38 UTC, Jacob Carlborg wrote:

On 2017-04-11 02:47, Jonathan M Davis via Digitalmars-d wrote:

Honestly, I don't see how it really makes much sense to use 
shared libraries
with D except in cases where you have no choice. The lack of 
ABI

compatibility makes them almost useless.

Also, what are we even looking to distribute in debian? I 
would have thought
that the normal thing to do would be to build with dub, in 
which case,
having the compiler and dub be debian packages makes sense but 
not really
anything else. If you're looking to package an application 
that was written
in D, then that becomes another question, but then if you just 
statically
link it, the ABI compatibility problem goes away as does any 
need to package

any D library dependencies.


I agree, I don't see any point in distributing libraries. just 
applications. But I do know some people will refuse to install 
anything that doesn't come through the system package manager.


Every single bit of software that is available in the 
distribution needs to be packaged in it so you can replicate its 
build using only what is available in the distro. Fetching things 
from the internet is not allowed.
(That's actually a hard no in the policy, while other things like 
the use of static linking are rather "you shouldn't do it if you 
can avoid it")


The fundamental thing a distribution does is integrating software 
and creating a consistent whole out of many moving parts. In 
order to do that, you absolutely can not rely on site-specific 
package managers like pip, dub, npm, etc. as they are not built 
for that purpose and only see their own ecosystem.
You could still build with them though in case they integrate 
well with the distro.

Re: [OT] Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 12:42:13 UTC, Russel Winder wrote:
On Tue, 2017-04-11 at 12:03 +, Matthias Klumpp via 
Digitalmars-d wrote:



[…]
Nah, there are several options here, one would simply be to 
tell

people not to use the distro packages with anything but the
default D compiler used in the respective Debian release.
Go apparently tells people not to use Debian-shipped go code in
their own projects at all.


The vendoring systems that Go folk have invented are 
effectively mandatory for projects that want reproducible 
builds, and using platform specific code is not feasible. It 
suprises me that Debian and Fedora are going flat out trying to 
package Go stuff.


That's false. Debian is leading the effort on reproducible builds 
that many other projects (including Fedora) have joined, and a 
large chunk of packages is already reproducible[1].
It's actually quite the opposite: Build systems downloading 
random stuff from the internet make the system more likely to 
produce different build results.


But in any case, the primary use for Debian packages is to be 
used by the distribution.


[1]: 
https://tests.reproducible-builds.org/debian/reproducible.html

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Tuesday, 11 April 2017 at 00:47:34 UTC, Jonathan M Davis wrote:

On Monday, April 10, 2017 23:08:17 David Nadlinger via [...]
Also, what are we even looking to distribute in debian? I would 
have thought that the normal thing to do would be to build with 
dub, in which case, having the compiler and dub be debian 
packages makes sense but not really anything else. If you're 
looking to package an application that was written in D, then 
that becomes another question, but then if you just statically 
link it, the ABI compatibility problem goes away as does any 
need to package any D library dependencies.


You will have static-library packages which have the exact same 
ABI issues shared libraries have.
And yeah, this is obviously about stuff being built with D 
compilers in the distro, such as Tilix, BioD, AppStream Generator 
and all future things which might emerge and be useful to have in 
the OS.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 23:43:04 UTC, David Nadlinger wrote:

On Monday, 10 April 2017 at 23:27:35 UTC, Walter Bright wrote:
The next problem is that dmd occasionally changes the 
interface to the D runtime. […] I also do not know how the 
gdc/lds druntime

interfaces differ.


Just to make this very clear to everybody reading this thread: 
It's not even just that, but also the fact that we guarantee 
API-, but not ABI-stability for Phobos. Every time we continue 
to improve the pure/nothrow/@nogc situation by marking up some 
more code, we are breaking the ABI, because the mangled names 
of the involved symbols change. The ongoing work on `scope` 
also breaks the ABI when enabled.


If you could change the SOVERSION with every one of these 
changes, or simply just tie it to the respective Phobos release, 
distributions would automatically do the right thing and compile 
all D code using Phobos against the new version.
This might give Phobos a large soversion or and ugly one like 
"2.074", but it would work.
(LDC's Phobos and GDC's Phobos already have a soversion set in 
Debian...)

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 23:33:17 UTC, Walter Bright wrote:

On 4/10/2017 6:08 AM, Matthias Klumpp wrote:
I also want to stress that having a single C++ library like 
Boost compiled into
stuff and rolling dependency transitions when its API/ABI 
changes with a major
release is less of a problem than having the entire language 
give zero stability
and interoperability guarantees on anything that is compiled 
with it.


How is the g++/clang++ issue handled?


The C ABI is 100% compatible, the C++ ABI is "mostly" compatible, 
there is some deliberate breakage from the Clang guys though.
The issue isn't actually handled in Debian as all our code is 
always compiled with GCC, I am not aware of anything defaulting 
to Clang (although it might exist, but definitely not for library 
packages).


With only one dominant compiler, things are way easier ^^

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 23:27:35 UTC, Walter Bright wrote:

On 4/10/2017 5:59 AM, Matthias Klumpp wrote:
You need to see here that D is not the center of the world and 
we will need to
make it work nicely with the rest of the system. The technical 
policies work for
everything else, so there is nothing that really justifies an 
exception for D
here (if 10% of Debian's code was written in D and the Debian 
D team was really

large we could maybe get one, but not the way it is now).
And tbh, I think finding a good solution here is entirely 
possible.


I think it is possible, too, and thank you for your efforts 
helping us do this.


The ABI differences between the compilers is unfortunate, and 
is largely the result of historical accident.


The first problem is that my idea originally was for D to have 
its own function calling convention, which would free us to 
innovate to have a more efficient calling convention than the C 
one. This hasn't panned out in practice, and ldc/gdc decided to 
sensibly stick with the C ABI. At some point, we should just 
crowbar dmd to generate the C ABI, but this has its own 
problems - it'll break code that uses the inline assembler. No 
obvious solution there.


The next problem is that dmd occasionally changes the interface 
to the D runtime. Or more accurately, with about every release. 
This has not been an issue historically for us, as the two have 
always been a matched set. I'm a lot less sure how to deal with 
this. I also do not know how the gdc/lds druntime interfaces 
differ.


Thank you very much for this context! It's really good to know 
why things are the way they are to properly understand the 
problem (I am no compiler developer and ABIs are not my expertise 
- as outsider to this I see pages like 
https://dlang.org/spec/abi.html though and wonder whether the 
incompatibilities are seen as an issue in the D community and 
whether there is a chance to address them before putting work 
into setting up infrastructure to rebuild the world on compiler 
updates).

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 23:08:17 UTC, David Nadlinger wrote:

On Monday, 10 April 2017 at 17:27:28 UTC, Matthias Klumpp wrote:
That's why I have been writing a lot of Makefiles and Meson 
build definitions lately.


It seems like doing so without having a closer look at the 
realities of D software (no stable ABI, etc.) might not have 
been the best use of your time. How would Dub be such 
fundamentally unfixable that making the official language tools 
play nice with distro packaging (and the other way round) 
wouldn't be preferable over manually re-writing build systems 
all over the place?


Dub is not unfixable, but there are many more issues with it 
which make it very very hard to use in a distribution context - I 
tried that and it didn't work well.
I am treating D in Debian like I would treat C++, which is quite 
fair given the similar featuresets and challenges.


But with an unstable ABI the standard library is also 
affected, which would trigger us to do Haskell-style 
versioning (which mangles dependencies to depend on a virtual 
package containing a hash of the GHC version), and that not 
only sucks but also requires quite a lot of manpower.


So it sounds like there is a solution already for other 
languages. Could you elaborate some more on the problems with 
it? I suppose there is some wiki page documenting the process 
somewhere?


Haskell and OCaml permanently rebuild the whole stack on every 
new compiler release, which is why they have permanent transition 
trackers[1], so they basically continuously rebuild. I want to 
avoid this at all cost for D, as this is very very maintenance 
intensive and painful, and will require much more people to work 
on it than D has available in Debian.


[1]: https://release.debian.org/transitions/

Rust only has one compiler which strongly optimizes, so we 
don't have the problem of choosing the right one. Cargo is/was 
an issue but it's being worked on and seems to work well now: 
https://wiki.debian.org/Teams/RustPackaging/Cargo


Rust doesn't have a stable ABI either, and it doesn't look like 
there is any movement in that direction (not that I think that 
there should be). That the people driving the effort might not 
be aware of it yet doesn't mean it isn't an issue for them.


It's being worked on[2], but it's not a super-high priority.
There doesn't seem to be a definitive answer on how Rust is 
handled in Debian yet (but to know for certain, I would need to 
ask the Rust team).


[2]: https://github.com/rust-lang/rfcs/issues/600

IIRC OCaml is also very much a statically linked affair. And 
how does Debian distribute Go binaries? Is there any issue with 
those being linked statically? If not, let's just distribute D 
libraries as source and compile/link them statically when 
building binaries, and problem solved.


Surprisingly it looks like many Go packages are indeed provided 
as source-installations.
Doing something like this with D would require Makefiles and Dub 
to pick up sources from system locations properly which isn't 
really done yet...


Some of the compiler developers, myself included, understand 
the issues involving ABI stability and distro packaging quite 
well (although the latter admittedly only on a general level). 
In fact, one of my earliest open source memories is of some 
work in the trenches ensuring ABI stability of some bits of KDE 
across releases. Yet we are still going to tell you that the D 
ABI is going to remain unstable for the foreseeable future. 
This is not something that just requires a man-week or month to 
"fix" in the compiler, but would impact many other areas as 
well, for example language evolution.


If you somehow got the impression that this is just due to D 
developers "not getting it", just have a look at the other 
recent compiled languages. Go and Rust don't fare any 
differently, and even Swift, with all its development manpower, 
doesn't have a stable ABI yet [1]. And I believe header-only 
C++ template libraries have been mentioned already as well.


I can only speak for myself, of course, but I certainly see the 
strategic importance of integration into the Linux distribution 
ecosystem for D, and I'm very happy to work with packagers 
wherever possible. However, you also need to acknowledge the 
properties of the ecosystem *you* are working with. If you see 
a big stretch of difficult terrain in front of you, closing 
your eyes won't make it go away; you'll only lose time you 
could spend working around it. ABI instability is something 
you'll have to work around one way or the other.


That's the whole point of this thread, I want to find the best 
solution to deal with this issue and also one the D community can 
live with.
I am not set on any particular solution yet, at the moment I see 
the problem and I am thinking about how to deal with it in the 
best possible way (ideally not doing what OCaml/Haskell are 
doing, although pulling a "soft Haskell" where we rebuild 
everything 

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 22:36:39 UTC, Iain Buclaw wrote:
On 10 April 2017 at 23:52, David Nadlinger via Digitalmars-d 
 wrote:

On Monday, 10 April 2017 at 20:43:06 UTC, Iain Buclaw wrote:


Master sports Phobos 2.071.  Someone will have to see whether 
latter versions can be built using it.



… and some weird Frankensteinian mix of several frontend 
versions, I take it, maybe enough to build Phobos, but not 
necessarily compatible for user code? Or did you port all the 
changes since 2.068.2 back to C++?


 — David


All the regression fixes and none of the bugs!

The current situation is that it should be link-compatible with 
current upstream/stable.  Enough so that when someone has the 
time to test, it should just be a case switching the sources 
and building the D version.


First of all, thank you for your tremendous work on GDC! Fellow 
developers and me were also pretty stunned by you maintaining a 
quite large amount of different GDC versions in parallel without 
a huge team - that's some impressive work!


What is the thing that's blocking GDCs GCC inclusion? Just 
manpower? Also, you were talking about "bugs" on several 
occasions, what's the thing with that? Is it GCC or general 
Phobos bugs? It would probably be awesome to have a summary 
blogpost or similar on the state of GDC, that could potentially 
also attract volunteers.

Anyway, all a bit off-topic :-)

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 22:26:46 UTC, Joseph Rushton Wakeling 
wrote:

On Monday, 10 April 2017 at 13:20:00 UTC, Matthias Klumpp wrote:
This has worked nicely for every language. If you don't have 
templates in your API or don't change the templates between 
releases, you can survive with one library for a long time.


But the vast majority of D libraries _do_ have templates 
(starting with Phobos).  How should this situation be dealt 
with?


How does Debian deal with, e.g., fixes to the templated code in 
Boost, which impact on other packages built using those 
header-only libraries?


Boost's soversion is changed on every release, and the version is 
included in it's -dev package as well. That's why we have 
libboost1.62-dev: 
https://packages.debian.org/de/sid/libboost1.62-dev (and possibly 
more).
There is also a boost-defaults package setting the current 
default Boos version for packages to depend on.
If a new Boost comes out, it's soversion and -dev package name 
changes, triggering a package transition and subsequently a full 
rebuild of all stuff depending on Boost.


Doing something like this with D libraries would obviously be 
possible as well.

Re: [OT] Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 22:15:53 UTC, David Nadlinger wrote:

On Monday, 10 April 2017 at 17:50:08 UTC, Matthias Klumpp wrote:

I am reading release notes, so we rebuilt dependencies of LDC -


(I assume you mean reverse dependencies.)

[…] But since no bugs were reported, I assume no issues are 
present :-)


So do we need to put a reminder about the ABI being unstable 
into set of every release notes to make sure we won't get angry 
bug reports once users actually build their own D code against 
your packages? ;)


Nah, there are several options here, one would simply be to tell 
people not to use the distro packages with anything but the 
default D compiler used in the respective Debian release.
Go apparently tells people not to use Debian-shipped go code in 
their own projects at all.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 20:11:28 UTC, Wyatt wrote:

On Monday, 10 April 2017 at 18:46:31 UTC, H. S. Teoh wrote:


Hmm.  I guess there's no easy way to make dmd/ldc emit 
dependencies with modified SONAMEs?  So yeah, you're right, 
every software that depends on said libraries would have to 
explicitly depend on a different SONAME depending on what they 
were built with.  OK, crazy idea, nevermind. :-(


Doesn't sounds that crazy; you already do it with GCC versions, 
right?  (Debian _does_ have something like that, right?  Where 
you can pick your C compiler.)


Yes, that's why all packages need to honor the CFLAGS/CC env var 
somehow or get the default C compiler from dpkg at build time, so 
we can easily apply new C flags globally, and projects to build 
the distro with Clang work.
We do never change the SONAME of anything, however. We do track 
symbols and might change the SOVERSION occasionally if breakage 
is found (making 3 -> 3a etc.), but the soname isn't changed.
The good thing is that Clang and GCC are (with few exceptions) 
very compatible.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

I apologize in advance for the large amount of mail that will 
likely follow, but I want to address all comments.


On Monday, 10 April 2017 at 18:46:31 UTC, H. S. Teoh wrote:

[...]
One issue, though: if we standardize on compiling Debian 
packages with ldc, then what do we do with libraries that are 
not ABI-compatible with dmd?  Since users would expect that if 
they need libfoo, they'd just `apt-get install libfoo-dev` and 
then they should be able to just run` dmd -L-lfoo` and it 
should all magically "just work".


That's the problem I would like to see addressed (but given 
Walter's comment, it won't be feasible to resolve it in the near 
future).
We could simply do away with "don't use distro packages for your 
D programming", at least that's what Go recommends.

Re: D support for the Meson build system

[Matthias Klumpp via Digitalmars-d-announce]

On Monday, 10 April 2017 at 18:11:44 UTC, Russel Winder wrote:

[...]
I'll look to ensuring my facts are correct, and then find out 
where to put an issue about this – I am assuming a GitHub 
repository with issues .


Just file one at https://github.com/mesonbuild/meson/issues - it 
might even be that Meson supports this already, the project is 
moving so fast it's really hard to keep up with all the changes 
(Mesa and X investigating using it boosted it's development quite 
a bit, and it was really fast even before).

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 17:29:04 UTC, H. S. Teoh wrote:
On Mon, Apr 10, 2017 at 11:40:12AM +, Matthias Klumpp via 
Digitalmars-d wrote: [...]

[...]
If we do that, we will run into the D ABI trap: Libraries 
compiled with compiler X can not be used from software 
compiled with D compiler Y. There is actually no ABI stability 
guarantee even between DMD releases.  This will make 
integrating D a huge pain. Recompiling the dependency-chain of 
a software from source when compiling a package using the 
"right" compiler and statically adding the code is forbidden 
by distro policy.  Having static libraries in the dependencies 
doesn't solve the issue.  Compiling each library with all D 
compilers is highly impractical and not really feasible.


This is not a hard problem to solve, IMO.  Just build the 
library into two separate binaries, each with a SONAME that 
encodes the ABI it is compatible with.


This would require to hack each and every build system to support 
this *and* if there is a pkgconfig file for the shared library, 
to change all depending software to check for multiple library 
names which is a bit crazy...


The resulting two .so's can either be distributed as separate 
packages (for minimum bloat, if that's a concern), or as a 
single package that contains both binaries (since they have 
different SONAMEs this should not be a problem).


Since one library == one package in Debian, it would have to be 
multiple packages, otherwise we would need to override Lintian 
errors/warnings, which is always a bad idea.


Then if you compile some software X that depends on this 
library, it will pick up the correct version of the library 
depending on which compiler you compiled with.


Unfortunately not without special support in the software's build 
system :-/



[...]
DMD is unlikely to support other archs than amd64/ia32 in the 
foreseeable future, so the justification for dmd being 
unavailable for arch X would be that upstream DMD simply 
doesn't support it.  This, however, should not prevent us from 
using gdc/ldc on those other archs, so that we can still ship 
packages for those archs.  They will merely require ldc rather 
than dmd. And obviously, libraries built for that arch will 
only support the ldc SONAME, not the dmd one. (This may be an 
argument for bundling both SONAMEs in a single package -- it 
gets messy if we start shipping multiple libraries, some of 
which may not be available on all archs. By shipping a single 
package that includes both versions for ia32/amd64, we can 
simply omit the DMD-compiled version from other archs.)


Conditional build-dependencies are a bit annoying, but with a 
metapackage "d-compiler" or similar, using different D compilers 
on different architectures would definitely be possible.


Unfortunately, I realize that this means some packages that 
require the latest DMD would not be available on all archs, if 
they require features unavailable in gdc/ldc.  But this problem 
may not be a huge one, now that ldc is mostly up-to-date with 
dmd (at most 1 release behind, IIRC). GDC may lag behind a bit 
more because it is unfortunately tied to GCC releases, so we 
may have to forego using gdc for building newer D packages. But 
we should be able to ship most D packages compiled with both.


Compiling with multiple compilers is a really big effort with 
rather questionable gain, IMO.
But as far as LDCs compatibility with other D projects goes: That 
is really good, the only reason you sometimes can't compile some 
random D code with LDC might be bugs, but not old standard 
libraries.


Furthermore, I wonder if we should standardize on ldc for most 
D software instead of dmd, unless that software absolutely 
depends on features only available on dmd.  My reasons are:


- While dmd compiles very fast, it consistently (IME) produces 
code that
  runs about 20-30% slower than code produced by gdc (and I 
presume ldc
  as well). Since we're talking about building Debian packages 
on
  Debian's buildd's, which are background batch processes, 
compilation
  speed is no big deal, but the performance of the executable 
*is* a big
  deal. The last thing we want is to give a poor impression of 
D by
  shipping official Debian packages that have subpar 
performance.


- DMD is unlikely to target other archs than ia32/amd64 in the
  foreseeable future, AFAIK, unless the recent relicensing 
triumph of
  dmd's backend makes this future more likely.  There will 
definitely be
  resistance among DDs because of lack of support for other 
archs. LDC,

  in contrast, supports all of Debian's supported archs.

- LDC is already available in Debian, meaning that we can 
already start
  adding D packages built with ldc without needing to work 
through the

  red tape involved in adding a new compiler to the archive.


I agree with all of that, I think sticking with LDC might indeed 
be the least painful thing to do...


- The only case where I can see 

Re: D support for the Meson build system

[Matthias Klumpp via Digitalmars-d-announce]

On Monday, 10 April 2017 at 15:27:25 UTC, Russel Winder wrote:
On Mon, 2017-04-10 at 12:41 +, Matthias Klumpp via 
Digitalmars-d- announce wrote:



[…].


I am not buying the necessity of not-splitbuilding for 
optimizations yet. If that would be the case, how do 
optimizations work with projects using GCC/Clang where 
splitbuilding is the default and often only option (like Mesa, 
Linux, lots of scientific stuff).


I am investigating this architecture because Chapel code cannot 
really
be compiled separately as far as I can see. cf. 
http://chapel.cray.com/
 Chapel is a PGAS language (like X10) for use with Big 
Computation™ on
serious computers and also any computer. I'm interested as a 
way of
connecting Python visualisation to computation that NumPy can't 
really

handle.


That's pretty cool! One way to do this with Meson is to spawn a 
shell script as custom target, but that obviously sucks. It might 
be worth reporting this as issue upstream, with a concrete 
usecase like this, the Meson maintainers will highly likely add 
support for it.

One could also always write a plugin as a last resort.

It seems sensible therefore to offer this way of working for D 
since whether there is actually any optimisation benefit or not 
some people think there is and use it as a stick to beat you 
with if it isn't there.



Having some level of dub integration is Meson would be neat
indeed - maybe one could make a small helper binary Meson can
call to fetch things from the dub registry.
I wonder though how that would jive with Meson's own
subprojects/wrap system. Probably worth investigating.


My thought for SCons was to delegate the package fetching to 
Dub as a
subprocess or write some Python to use the Dub API. I'm not 
clear on
that as yet, the issue is whether the Dub local source repo is 
the
right way forward – using Dub for preparing the compiled 
artefact is
likely not the right way forward for SCons. This would then 
make it
easy to do something for Rust/Cargo – except that SCons doesn't 
really
support Rust yet, and with Cargo are there any Rust users not 
using

Cargo.

Having said all this SCons stuff, if there was Meson support 
for this
"get the source from the Dub repository, compile it and make it 
a

dependency" I'd likely stay with Meson for my codes.


SCons is considered evil, last time I checked ^^ => 
https://wiki.debian.org/UpstreamGuide#line867
(unless it's used right, which seems to be hard) - I have no idea 
though on whether the issues with it were fixed, the entry on 
SCons hasn't been updated in a while.

Re: [OT] Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 16:58:05 UTC, Johan Engelen wrote:

On Monday, 10 April 2017 at 13:20:00 UTC, Matthias Klumpp wrote:


Btw, at time we are just ignore the ABI issues, and 
surprisingly nothing broke yet, indicating that ABI breakage 
isn't very common or not affecting commonly used interfaces 
much.


One big ABI change was in 2.071: 
https://issues.dlang.org/show_bug.cgi?id=15644. And it involved 
interfaces. ;-)

Nothing broke because of that?


I am reading release notes, so we rebuilt dependencies of LDC - I 
have no idea about GDC-depending D code though. But since no bugs 
were reported, I assume no issues are present :-)

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 16:12:35 UTC, Iain Buclaw wrote:

[...]
Everyone should follow GDC's ABI, rather than trying to mimic 
DMD calling convention. ;-)


GDC is working very well, and using it would actually be the 
natural choice for us as GCC is the default compiler.

However, there are a few problems with that:
 1) GDC doesn't compile a large amount of D code currently out 
there due to having an outdated standard library and runtime. 
While we would like to use it, it doesn't help if it just can't 
compile the things.


 2) GDC isn't part of the official GCC. This has quite wide 
reaching implications, most importantly other distros not 
offering it - in Fedora / Red Hat, this is the reason why GDC 
isn't available. This means fewer people test with GDC and we're 
basically compiling upstream projects with a compiler they never 
ever tested, potentially creating new issues that we can't easily 
forward upstream. This is less of an issue with LDC as LDC is 
more widely available in other distributions.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 14:33:34 UTC, qznc wrote:

On Monday, 10 April 2017 at 11:40:12 UTC, Matthias Klumpp wrote:
  1) Is there some perspective on D getting a defined ABI that 
works with all major D compilers?
  2) What would the D community recommend on how to deal with 
the ABI issues currently? A Linux distribution is a bunch of 
tightly integrated software, and changing one piece in an 
incompatible way (e.g. by building it with LDC instead of DMD) 
will have consequences.
  3) Will DMD support more architectures in the near future? 
How should the architecture issue be handled?


My prediction for Walters reply:

1) No. Not worth it, because templates, ctfe, etc.


That's short-sighted IMHO, because if the template doesn't 
change, ABI/API doesn't change. Also, some projects use D as 
better C and don't expose this functionality. It should be up to 
the project to set the level of API/ABI stability, and not to the 
compiler to make everything unstable by default.



[...]
Tentative ping, but that Wiki page is not helpful. The linked 
svn repo is empty. Where and how do you work?


Yeah, the page is really poor, it was last touched in 2012. I 
made a few updates to at least link to the current Git repo.
We generally work on various Git repositories, but not all of 
them are run by the D team (e.g. libundead and libbiod as well as 
several games are things I am aware of that aren't D-team 
maintained but are part of other team's work).


One can find all stuff using D by testing the 
reverse-build-depends on the LDC and GDC compilers.


I guess the issue are still the same as you wrote here (except 
1. is solved):

https://gist.github.com/ximion/fe6264481319dd94c8308b1ea4e8207a

So, mostly dub needs work, I guess.


Yes, but since Meson is working well and Meson scripts are easy 
to write, it's not a super high priority item anymore.
As I said earlier, work as a distribution developer is pretty 
much always about reducing long-term maintenance cost, and not 
about less work short-term, which means we will gladly write 
Meson or Automake scripts to integrate software into Debian if 
there is a demand for it.


On Monday, 10 April 2017 at 15:11:01 UTC, Jack Stouffer wrote:

On Monday, 10 April 2017 at 11:40:12 UTC, Matthias Klumpp wrote:
  3) Will DMD support more architectures in the near future? 
How should the architecture issue be handled?


This can be definitively answered as "no", 
https://issues.dlang.org/show_bug.cgi?id=15108



*Walter Bright*:
Doing an ARM back end wouldn't be that hard. It's much less 
complex than x86. Most of the work would be deleting about half 
of the x86 code generator :-)


:D - doesn't sound like a flat-out no, much more like there just 
wasn't someone doing the work yet.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 14:21:43 UTC, Gerald wrote:

On Monday, 10 April 2017 at 11:40:12 UTC, Matthias Klumpp wrote:
There are a two issues though that we will be facing in Debian 
soon, and I would like to get some opinion and maybe 
perspective on from the D community on them.


First I would like to say thank you for all the work you did in 
getting Tilix packaged for Debian, it is very much appreciated. 
While I'm not an expert in this topic I'll throw in my two 
cents anyway :)


Thanks!

So to the topic at end, I tend to agree with Vladimir in that I 
just don't see it as feasible to make every dependency a shared 
library. Personally I would suggest that only libphobos and 
libdruntime be managed as shared libraries by a distro. 
Otherwise, any other dependencies used in DUB or elsewhere 
would be statically compiled.


This would raise the maintenance effort for D software, and 
require policy exceptions, which is something we won't do. It's 
more likely that we'll see less D in the distro than going down 
this route.
DUB is unusable for Debian packaging anyway, and alternatives 
like Automake or Meson handle shared libraries really well, so in 
itself there is no reason not to do it.


If in time a new D library ends up becoming a keystone 
foundation for many packages it could be considered for 
inclusion as a shared library. Otherwise, trying to manage 
every DUB dependency as a potential shared library is a huge 
amount of work and I don't feel most of them are mature or well 
maintained enough to support this approach.


We'll take care of that, it's what we do as a distro and we have 
a fair amount of experience in handling these kinds of upstream 
projects.
The goal is to reduce the maintenance cost for things once they 
are *in* the distro, any prior work to get it into the 
distribution is very well invested.
That's why I have been writing a lot of Makefiles and Meson build 
definitions lately.


If you just concentrate on compiler, libphobos and libdruntime, 
you can have separate packages for each compiler toolchain 
which was Arch does.


That's a necessity already and we do the same. But with an 
unstable ABI the standard library is also affected, which would 
trigger us to do Haskell-style versioning (which mangles 
dependencies to depend on a virtual package containing a hash of 
the GHC version), and that not only sucks but also requires quite 
a lot of manpower.


It has a libphobos package for DMD and a liblphobos for LDC. 
This then enables developers to specify the tool chain they 
prefer without interference.


Yes, but that's not feasible for use *in the distribution itself* 
as we can't just pick and choose the right compiler per package 
if something up in the dependency chain was compiled with the 
"wrong" compiler.


In addition to the question about D and C++. what do distros 
typically do for Rust and Cargo dependencies, or Java and 
Maven? Wouldn't that be a similar paradigm to D and DUB 
dependencies?


Maven dependencies are also separated out into smaller packages 
and maven (unlike dub) plays relatively well with Debian. It also 
has no ABI issues and interface stability can be tracked. Rust 
only has one compiler which strongly optimizes, so we don't have 
the problem of choosing the right one. Cargo is/was an issue but 
it's being worked on and seems to work well now: 
https://wiki.debian.org/Teams/RustPackaging/Cargo

(not that I am not involved in Haskell, Rust or Java packaging).

Just because it makes sense: We do have an upstream guide showing 
some best practices for upstream projects which help us to 
maintain software for long periods. It also contains 
language-specific advice and some general info which is valid for 
all languages:

https://wiki.debian.org/UpstreamGuide

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 12:59:58 UTC, qznc wrote:

[...]
How do Debian and C++ go along? There is no ABI compatibility 
between GCC and Clang afaik.


Clang offers compatibility for most basic features. There are 
some ABI compatibility issues though and you find them reported 
in the Clang/libc++ bugtrackers, and it's a pain (but the 
Clang/LLVM guys think they can do some ABI better/faster than 
what GCC offers, so some breakage is deliberate).
In Debian, GCC compiles everything as the system's default 
compiler, so at least inside the distribution we don't have to 
worry about potential incompatibilities. Since GCC also supports 
an enormous amount of architectures and has strong optimization, 
the case is different there.


In terms of "what happens when users use the OSes C++ libraries 
and compile with Clang instead of GCC" the situation is similar 
though: They might run into ABI issues (rarer though than with 
D). For the distro itself the problem doesn't exist though.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 13:07:22 UTC, Vladimir Panteleev 
wrote:

On Monday, 10 April 2017 at 12:59:37 UTC, Matthias Klumpp wrote:
Who came up with those policies and decided that they apply 
to D? Because I really don't think they should.

[...]
You need to see here that D is not the center of the world and 
we will need to make it work nicely with the rest of the 
system.


The opposite is also true: requiring a stable shared library 
API of every packaged D library is just as unreasonable. In 
fact, to make these rules useful and applicable to all D 
programs, you'd have to completely forbid templates in the 
library's public interface, which would immediately exclude 
Phobos for one.


There is a really easy way to fix this: SONAMEs. Whenever you 
change something in the library breaking ABI or API, you bump 
it's SOVERSION, which will force the distribution to perform a 
transition and rebuild the dependency chain. If you give 
absolutely zero stability guarantees, you just set the SOVERSION 
equal to the project's version and trigger a transition every 
time (incredibly annoying, but, well, okay).


This has worked nicely for every language. If you don't have 
templates in your API or don't change the templates between 
releases, you can survive with one library for a long time.


This is working really great on the level of individual 
libraries, but if the whole language is ABI-unstable, the issues 
are much bigger and harder to track.


Btw, at time we are just ignore the ABI issues, and surprisingly 
nothing broke yet, indicating that ABI breakage isn't very common 
or not affecting commonly used interfaces much.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 12:40:33 UTC, Vladimir Panteleev 
wrote:

[...]
Can we treat it more like an interpreted language instead?


An interpreted language would interpret the code on the target 
system at runtime. This is not what D does, so we can't really 
treat it like we treat Python (where it is possible no neatly 
separate Python modules into separate packages - transitions 
triggering rebuild cascades only exist when we jump to the next 
major CPython version, which is something distributions are well 
prepared for. Transitions are only an issue if they happen 
constantly).
At time, D is treated like C++, since it has much of the same 
challenges and we know how to deal with C++ - additionally to 
C++, D unfortunately though also has the unique issues outlined 
above, which complicate things.


I also want to stress that having a single C++ library like Boost 
compiled into stuff and rolling dependency transitions when its 
API/ABI changes with a major release is less of a problem than 
having the entire language give zero stability and 
interoperability guarantees on anything that is compiled with it.

Re: The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

On Monday, 10 April 2017 at 12:40:33 UTC, Vladimir Panteleev 
wrote:

On Monday, 10 April 2017 at 11:40:12 UTC, Matthias Klumpp wrote:
Recompiling the dependency-chain of a software from source 
when compiling a package using the "right" compiler and 
statically adding the code is forbidden by distro policy.


This is the part that I do not understand.

Who came up with those policies and decided that they apply to 
D? Because I really don't think they should.


They are the result of years of experience in building complex 
systems and keeping them secure.
If you have a dependency chain "X -> Y -> Z" (-> meaning "depends 
on"), and you find a security bug in Z, you the security team 
will just need to fix the bug in Z to resolve it in the whole 
distribution.
But if the code which has this issue is compiled into all of the 
packages that depend on them, you will need to rebuild the full 
dependency chain to actually fix the security issue, which is not 
only time intensive but also a huge maintenance effort.
In this simple example it doesn't look like much, but those 
dependency chains can grow massively large and complicated, and 
the only way to keep the large software stack maintainable and 
secure is by splitting pieces cleanly.


Embedded code copies are allowed in rare events, but in these 
cases the security team needs to be aware of them.
Sometimes, the licenses also explicitly prevent embedded code 
copies.


Aside from these issues, splitting things cleanly also makes 
general package maintenance much easier, and adds flexibility for 
our users who can mix and match parts of the distribution as they 
like and combine them with their own code.


You need to see here that D is not the center of the world and we 
will need to make it work nicely with the rest of the system. The 
technical policies work for everything else, so there is nothing 
that really justifies an exception for D here (if 10% of Debian's 
code was written in D and the Debian D team was really large we 
could maybe get one, but not the way it is now).
And tbh, I think finding a good solution here is entirely 
possible.

Re: D support for the Meson build system

[Matthias Klumpp via Digitalmars-d-announce]

On Monday, 10 April 2017 at 12:10:41 UTC, Russel Winder wrote:
On Mon, 2017-04-10 at 08:39 +, Atila Neves via 
Digitalmars-d- announce wrote:

[…]

As far as I know the only build system that does this by 
default for D is reggae.


I will be adding a new builder to the SCons D tools to do whole 
source and per package compiling – to add to the module at a 
time compiling. It would be good to add this for CMake-D and 
the D support in Meson.


I am not buying the necessity of not-splitbuilding for 
optimizations yet. If that would be the case, how do 
optimizations work with projects using GCC/Clang where 
splitbuilding is the default and often only option (like Mesa, 
Linux, lots of scientific stuff).


Having some level of dub integration is Meson would be neat 
indeed - maybe one could make a small helper binary Meson can 
call to fetch things from the dub registry.
I wonder though how that would jive with Meson's own 
subprojects/wrap system. Probably worth investigating.

The D ecosystem in Debian with free-as-in-freedom DMD

[Matthias Klumpp via Digitalmars-d]

Hi there!
These are probably questions directed mostly at Walter and others 
shaping D's goals, but this could be of general interest for many 
people, so to the forum it goes :-)


DMD is completely free software now and we can legally distribute 
it in Debian main - yay! This is an awesome achievement and will 
make D adoption in Linux distributions much easier (in fact, Red 
Hat is working on getting good support into Fedora too already).


There are a two issues though that we will be facing in Debian 
soon, and I would like to get some opinion and maybe perspective 
on from the D community on them.


Naturally, when the reference compiler is available in Debian, we 
would compile everything with that, as it is the development 
focus and the thing many people test with.
We do, however, have quite a bit of bioinformatics and other D 
software in the archive where performance matters - so our users 
and the developers of that software (like BioD, potentially Mir, 
maybe even Vibe.d) will want the fastest performance and will ask 
us to compile the libraries with LDC or GDC.


If we do that, we will run into the D ABI trap: Libraries 
compiled with compiler X can not be used from software compiled 
with D compiler Y. There is actually no ABI stability guarantee 
even between DMD releases.
This will make integrating D a huge pain. Recompiling the 
dependency-chain of a software from source when compiling a 
package using the "right" compiler and statically adding the code 
is forbidden by distro policy. Having static libraries in the 
dependencies doesn't solve the issue. Compiling each library with 
all D compilers is highly impractical and not really feasible.
So, how should we proceed here? We could make it "DMD is the only 
thing on the highway" compiling everything with DMD with zero 
exceptions, which would leave us with only DMD-internal ABI 
breakage and bad D code performance for some libraries. We could 
also continue using LDC for everything, but that comes with it's 
own issues (I am hitting quite a bunch of LDC bugs and upstream 
projects usually test with DMD or use features which are only in 
the latest DMD).


The other issue is architecture support. In Debian we are 
strongly encouraged to support as many architectures as possible, 
to the point of having to justify why arch X is not supported.
LDC runs on at least armhf and ppc64el additionally to 
amd64/ia32, while DMD says it's specifically only for ia32/amd64.
This means we might end up compiling stuff with different D 
compilers on different architectures, or we will need to drop 
architectures and request arch-specific package removals for 
things that currently build on architectures not supported by 
DMD, which will trigger resistance.


So, in summary:
  1) Is there some perspective on D getting a defined ABI that 
works with all major D compilers?
  2) What would the D community recommend on how to deal with the 
ABI issues currently? A Linux distribution is a bunch of tightly 
integrated software, and changing one piece in an incompatible 
way (e.g. by building it with LDC instead of DMD) will have 
consequences.
  3) Will DMD support more architectures in the near future? How 
should the architecture issue be handled?


I am interested in some feedback here, since I currently can't 
see a good way to address these issues.


Also: If you want to help out Debian's D team, feel free to ping 
me or any other D team member (we are very short handed and are 
only two active people right now). See https://wiki.debian.org/D 
(caution, wiki page is very outdated, last touched in 2012)

Re: GDC options

[Matthias Klumpp via Digitalmars-d-learn]

On Wednesday, 22 March 2017 at 05:47:59 UTC, Sebastien Alaiwan 
wrote:

On Monday, 13 March 2017 at 11:06:53 UTC, Russel Winder wrote:
It is a shame that dmd and ldc do not just use the standard 
GCC option set.

Totally agreed.

Moreover, funny stuff like "dmd -of" (instead of 
standard "-o ") breaks automatic Msys path conversion 
hack (the code translates Unix paths from the command line to 
Windows paths before the invocation of a non-msys program), 
which makes it impossible to use dmd under Msys without 
wrapping it first.


pkg-config also is a real pain to use with dmd (the 
pkg-config's output needs to be post-processed so it has the 
form "-L-lstuff" instead of "-lstuff").


This is an issue, because it makes it very hard to use write 
portable makefiles for programs containing D code. Too bad, 
because the D code is actually platform-independent, and 
there's been a lot of work in Phobos to make it easy to write 
such code.


D was designed to be binary compatible with the C ABI ; 
however, having a compiler whose command-line behaves so 
different from gcc makes it harder to actually work with 
existing C libs.


This is actually the main reason why I almost exclusively use 
gdc: to have one Makefile, for all platforms, allowing native 
and cross-compilation with no platform-specific special cases.


This is why most of my work in Meson to get D supported is adding 
weird hacks to translate compiler flags between GNU <-> non-GNU 
<-> DMD. It sucks quite badly, and every now and then I hit a 
weird corner case where things break.
For example: 
https://github.com/mesonbuild/meson/commit/d9cabe9f0ca6fb06808c1d5cf5206a7c5158517e


Would be amazing if all D compilers would support the GCC flags, 
like Clang does for C - that would help in removing a lot of 
hacks (and making Makefiles which work with all compilers).
The reason for not using a dmd wrapper is that one might want to 
use flags specific to a certain compiler.

Re: D-Apt package numbers

[Matthias Klumpp via Digitalmars-d]

On Monday, 20 March 2017 at 08:52:05 UTC, Russel Winder wrote:


I see that D-Apt has the Debian revision number on packages 
starting at

0. I had understood that the policy was to start at 1.


For stuff in *Debian* that is true, anything not in Debian should 
start at zero and add a "repository tag" to the Debian revision, 
if the package is new or has a new upstream version in their 
repository.
E.g. in Ubuntu, if the upstream version is "1.0", the revision 
Ubuntu chooses is "1.0-0ubuntu1". If a Debian package is 
modified, the "ubuntuX" tag is added to the Debian revision.
This ensures that Debian packages are preferred if they are 
available and that users as well as the Debian package maintainer 
knows where stuff was coming from when people report bugs (less 
of an issue on Ubuntu and other derivatives, there the tags are 
required to properly produce deltas between Debian and the 
derivative and to merge packages from Debian safely).


So, if d-apt is doing something like this, everything is fine :-)

Re: The end of curl (in phobos)

[Matthias Klumpp via Digitalmars-d]

On Saturday, 18 February 2017 at 22:48:53 UTC, Dmitry Olshansky 
wrote:

[...]
For some time I was a proponent of yanking stuff from Phobos 
into oblivion. Now I'm not. Stop breaking code. Yes, we should 
think harder before introducing libraries into Phobos but 
continuing on with removal of stuff just introduces the 
continuous churn for no adequate benefit.


What would anyone get from std.net.curl removal? Are you going 
to find all of D programs and patch them to use something else?


Yes, Phobos is full of historical accidents and cruft. I'm 
constantly tempted to propose Phobos v2 properly _designed_ 
(not *grown*) and without the junk. I really think it might be 
a good idea but only when we actually know what a proper design 
looks like.


Due to writing the AppStream metadata generator in D, which is an 
infrastructure piece of many Linux distributions, I have a fair 
bit of knowledge now about the problems people (especially 
newcomers who just want to scratch an itch and submit a quick 
patch) encounter when working with D code. The inconsistent 
standard library is - after compiler bugs - the biggest issue. 
Some people described Phobos as "PHP-esque" in terms of design, 
and I have to agree with them. Working with it is often 
unpleasant, and you can clearly see which parts of it were 
designed recently and which are "historical accidents".


Constantly shuffling stuff around in Phobos and adding/removing 
things will not solve the problem, it will just give newcomers a 
feeling of insecurity and make the language feel less mature.
Stunningly, a lot of projects write their own primitives instead 
of using Phobos (Vibe, lots of dub modules just providing 
containers, just now another general purpose utility library was 
announced on the forums, ...) which is a clear sign that Phobos 
isn't seen to be sufficient.


I think investigating to build a "Phobos2" standard library would 
be a very good idea - make it opt-in for a while, and then set a 
flag-date and switch, so people will only need to adjust their 
code once to jump on the new version, and don't constantly need 
to play catch with Phobos API breaks and riddle their code with 
version() and static if instructions to be able to compile with 
multiple Phobos and LDC/GDC/DMD versions.


Cheers,
Matthias

Re: D at FOSDEM this weekend

[Matthias Klumpp via Digitalmars-d]

On Friday, 3 February 2017 at 06:59:32 UTC, David Nadlinger wrote:

Hi all,

This year's FOSDEM is taking place Saturday–Sunday in Brussels 
(registration-less open source software event).


Any D heads in the area? Kai Nacke is going to give a talk on 
PGO in LDC in the LLVM dev room [1], and I'll also be around.


Too bad that I am not there this year :( (first FOSDEM to miss in 
a while).

Would have been nice to meet!

Enjoy the conference and the Belgian beer!

Re: It is still not possible to use D on debian/ubuntu

[Matthias Klumpp via Digitalmars-d]

On Saturday, 14 January 2017 at 23:24:18 UTC, Jack Applegame 
wrote:
On Saturday, 14 January 2017 at 18:41:21 UTC, Russel Winder 
wrote:
On Sat, 2017-01-14 at 17:28 +, Elronnd via Digitalmars-d 
wrote:
On Friday, 13 January 2017 at 11:50:25 UTC, Russel Winder 
wrote:


> LDC which is packaged by both Debian and Fedora is the
> only practically usable D compiler on both these platforms.

What's impractical about downloading and installing an 
rpm?  For that matter, downloading the source and compiling 
it isn't all that impractical either.


Downloading and installing an RPM outside of dnf.



What do you mean "outside"?
I use DMD on CentOS, and I installed it by command:

yum install 
http://downloads.dlang.org/releases/2.x/2.072.2/dmd-2.072.2-0.fedora.x86_64.rpm


From your point of view it is "outside" or "inside" of yum?


Still outside because it is not developed as part of Fedora.
One big reason for getting a language's toolchain into the main 
repositories of a distribution that Russel didn't mention yet is 
the additional QA and testing it will get.
For example, the PIE/PIC issue would not have happened at all if 
people were using the tools provided by the distribution, because 
we made sure that every tool we ship works with this change.
Using pieces that are part of the distribution is also way easier 
than getting them from external sources, also mainly because we 
can give a lot of guarantees about software we ship in a 
distribution.


And also, D software that is itself part of the distro will be 
compiled with one of the purely-free compilers anyway, so if you 
target one of those it just makes sense to primarily use LDC or 
GDC to ensure the software works well.

Re: Red Hat's issues in considering the D language

[Matthias Klumpp via Digitalmars-d]

On Friday, 23 December 2016 at 15:02:23 UTC, Ilya Yaroshenko 
wrote:

[...]
It is not true for Mir projects,  sometimes ICE occurs without 
any description while LDC just works. --Ilya


Bug report for ICEs requires to much efforts because code size 
should be reduced.


I found quite a few in LDC too ;-) In any case, Dustmite[1] helps 
to greatly reduce the time needed to create a minimal testcase to 
report as a bug, and the tool will soon be available as a Debian 
package as well (anything that doesn't use dub and is no library 
is rather easy to package).


[1]: https://github.com/CyberShadow/DustMite

Re: Terminix 1.4.0 Released

[Matthias Klumpp via Digitalmars-d-announce]

Congrats to the new release!

Could you maybe delete or rename the 1.20 tag? And maybe also the 
1.30 release? (see 
https://github.com/gnunn1/terminix/releases/tag/1.20 )
At least the tag messes up the new-release notifications at 
Debian, since 1.20 >> 1.4


Cheers,
   Matthias

Re: Red Hat's issues in considering the D language

[Matthias Klumpp via Digitalmars-d]

To clarify this point on the list:

On Thursday, 22 December 2016 at 10:40:32 UTC, Kagamin wrote:
On Tuesday, 20 December 2016 at 23:08:28 UTC, Andrei 
Alexandrescu wrote:

https://gist.github.com/ximion/77dda83a9926f892c9a4fa0074d6bf2b


Aren't requirements for packaging and recent versions mutually 
exclusive? The packaged version will undergo version freeze and 
will be older than the recent version no matter what you 
package.


This is true when the distribution is frozen, but there is a time 
when we will just get new software versions in there as soon as 
they are released.
But the much more important point for us is support and 
maintainability. The reference compiler will have a much bigger 
development team and higher focus of attention. Additionally, 
people will likely build their code with that compiler and might 
not test with other compilers.
So, if we then take D code and build it with a configuration 
upstream didn't test, and then encounter a bug, this will be an 
additional obstacle to overcome when communicating with upstream.
Furthermore, the compiler package - once frozen - will have to be 
supported for many years, and a bigger team behind it helps in 
finding issues and fixing them. Additionally, people learning D 
will told "use DMD" and won't find it in their distribution, 
which is annoying for them (they think D isn't well supported, 
while our LDC/GDC packages are less used).


Those are all points which make it useful to have a completely 
free compiler as reference compiler and in the distribution.


On the point of "free" being ideological: It of course is also an 
ideological issue, but that's not the only point.

See this excerpt from the DMD backend license:
```
The Software is copyrighted and comes with a single user license,
and may not be redistributed. If you wish to obtain a 
redistribution license,

please contact Digital Mars.
```

This alone makes it impossible for use and all our derivatives to 
legally redistribute the software. Adding it to a distro is a 
no-op.
Also, licenses restricting modification of software or 
proprietary software in general makes it impossible for use to 
deliver security fixes, and also makes integrating the software 
into the system much harder and sometimes impossible.


For GDC: Being part of GCC would be very awesome there, because 
then the Toolchain team of the respective distributions could 
easily make the D compiler available and maintain it (as done 
with e.g. gccgo for the Go language). At time we patch in GDC and 
Debian, but it looks like Red Hat will not go that way on 
RHEL/Fedora (and I completely understand why they don't want to 
do that).
Anyway, it's great to hear that the GDC Phobos isn't as old 
anymore as it was when I wrote the first version of the list :)


Confusing claim that he can't use dmd given that he says he 
uses it.


Huh? Where is this stated?

Re: Installing ldc breaks gdc

[Matthias Klumpp via Digitalmars-d]

Hi!

This issue should be fixed since LDC 1:1.1.0-2, which Xenial 
doesn't have.
Ideally, fetch a newer version from Debian or a PPA to solve this 
issue.


Cheers,
Matthias

Re: fPIC Error

[Matthias Klumpp via Digitalmars-d-learn]

On Saturday, 19 November 2016 at 21:14:47 UTC, Charles Hixson 
wrote:

[...]
IIRC, LDC didn't have that problem.  I don't remember testing 
gdc. But, yes, it is quite annoying.


That's because we can maintain those compilers with the 
distribution and configure them appropriately to compile with 
hardening flags and integrate properly with the distro.
GDC is an issue here, since AFAIK it doesn't support PIC yet, so 
for GDC - if it works - there will be some kind of workaround in 
place (telling the linker to not assume PIC code, I guess).
DMD has issues because it's a 3rd-party product - ideally, the 
compiler should be adjusted to output PIC code by default.
You can find more information on this change at 
https://wiki.debian.org/Hardening/PIEByDefaultTransition

Re: Cannot link with libphobos2.a with GCC 6.2 on Ubuntu 16.10

[Matthias Klumpp via Digitalmars-d-learn]

On Thursday, 13 October 2016 at 17:07:19 UTC, Nordlöw wrote:

On Thursday, 13 October 2016 at 17:02:32 UTC, Nordlöw wrote:

Am I using the wrong GCC version? Should I use GCC 5 instead?

GCC 6.2 is default on 16.10.


Compiling DMD with GCC 5 as

make -f posix.mak HOST_CXX=g++-5

also fails with same errors.


The new toolchains of Ubuntu (and Debian soon too) default to PIE 
code, so in order to link correctly, the project needs to be 
compiled with PIE/PIC to work.

Re: Required DMD changes for Mir and few thoughts about D future

[Matthias Klumpp via Digitalmars-d]

On Saturday, 8 October 2016 at 18:53:32 UTC, Andrei Alexandrescu 
wrote:

On 10/8/16 2:49 PM, Andrei Alexandrescu wrote:

On 10/8/16 1:22 PM, Martin Nowak wrote:
Integrating this with a pre-compiled ldc library is a 
fantastic idea

OTOH.
If we can make this work, it will be much less effort and 
yield the
fastest implementation. Also would speed up the development 
cycle a bit

b/c the kernels don't need to be recompiled/optimized.


You mean dmd/ldc/etc interop at binary level? Yes, that would 
be pretty

rad indeed! -- Andrei


(after thinking a bit more) ... but Mir seems to rely in good 
part on templates, which makes pre-compiled libraries less 
effective. -- Andrei


Independent from Mir, a stable ABI for D which all compilers 
follow would be a tremendous win, especially from the perspective 
of shipping D stuff in Linux distributions.

So maybe this is worth attempting?

Re: Examples Wanted: Usages of "body" as a Symbol Name

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 5 October 2016 at 16:57:42 UTC, Rory McGuire wrote:
On Wed, Oct 5, 2016 at 5:32 PM, angel via Digitalmars-d < 
digitalmars-d@puremagic.com> wrote:



On Wednesday, 5 October 2016 at 02:11:14 UTC, Meta wrote:
[...]
Really, why do we need a _body_ ?
We have pre-condition and post-condition (in and out), 
everything else is

a body.
It is simply inconsistent - a regular function with no in and 
out blocks
has no body block. Now one adds a pre-condition (and / or 
post-condition) -
whoop - one needs to wrap the whole function body ... well in 
a body

expression.



Recently I've had to use scope_ a lot more often than body_ but 
reserved keywords are really annoying, so the less we have the 
better :D


Agreed - I have exactly the same problem with "version", which is 
also really common for, well, to hold a version number of a 
component. Body is annoying too.


But, can keywords actually sanely be removed from the language 
without breaking the world?

How to debug (potential) GC bugs?

[Matthias Klumpp via Digitalmars-d-learn]

Hello!
I am working together with others on the D-based 
appstream-generator[1] project, which is generating software 
metadata for "software centers" and other package-manager 
functionality on Linux distributions, and is used by default on 
Debian, Ubuntu and Arch Linux.


For Ubuntu, some modifications on the code were needed, and 
apparently for them the code is currently crashing in the GC 
collection thread: http://paste.debian.net/840490/


The project is running a lot of stuff in parallel and is using 
the GC (if the extraction is a few seconds slower due to the GC 
being active, it doesn't matter much).


We also link against a lot of 3rd-party libraries and use a big 
amount of existing C code in the project.


So, I would like to know the following things:

1) Is there any caveat when linking to C libraries and using the 
GC in a project? So far, it seems to be working well, but there 
have been a few cases where I was suspicious about the GC 
actually doing something to malloc'ed stuff or C structs present 
in the bindings.


2) How can one debug issues like the one mentioned above 
properly? Since it seems to happen in the GC and doesn't give me 
information on where to start searching for the issue, I am a bit 
lost.


3) The tool seems to leak memory somewhere and OOMs pretty 
quickly on some machines. All the stuff using C code frees 
resources properly though, and using Valgrind on the project is a 
pain due to large amounts of data being mmapped. I worked around 
this a while back, but then the GC interfered with Valgrind, 
making information less useful. Is there any information on how 
to find memory leaks, or e.g. large structs the GC cannot free 
because something is still having a needless reference on it?


Unfortunately I can't reproduce the crash from 2) myself, it only 
seems to happen at Ubuntu (but Ubuntu is using some different 
codepaths too).


Any insights would be highly appreciated!
Cheers,
   Matthias

[1[: https://github.com/ximion/appstream-generator

Proper way to work around `Invalid memory operation`?

[Matthias Klumpp via Digitalmars-d-learn]

Hello!

I have a class similar to this one:
```
class Dummy
{

private:

string tmpDir;

public:

this (string fname)
{
tmpDir = buildPath ("/tmp", fname.baseName);
std.file.mkdirRecurse (tmpDir);
}

~this ()
{
close ();
}

void close ()
{
if (std.file.exists (tmpDir))
std.file.rmdirRecurse (tmpDir);
}
}
```

When the GC calls the classes destructor, I get a
`core.exception.InvalidMemoryOperationError@/<...>/ldc/runtime/druntime/src/core/exception.d(693):
 Invalid memory operation`

Looks like rmdirRecurse tries to allocate with the GC, and the GC 
doesn't like that.
Is there any good way to get the temporary directory deletet 
automatically when the object is freed?
At time, I work around this bug by calling close() manually at 
the appropriate time, but this feel like a rather poor solution.


Cheers,
Matthias

Re: Simple GEdit .lang & github color scheme

[Matthias Klumpp via Digitalmars-d-announce]

On Wednesday, 24 August 2016 at 14:38:48 UTC, floare wrote:

Dead link (dlang gedit color highlighting)

http://reign-studios.com/d-downloads/d.lang.tar.gz


Would you mind uploading on github for instance ?


Also, if someone would simply update this with GNOMEs default 
color scheme, I could push it upstream so we wouldn't need to use 
external stuff anymore for proper D support...

Re: D support for the Meson build system

[Matthias Klumpp via Digitalmars-d-announce]

On Monday, 22 August 2016 at 01:34:36 UTC, jkpl wrote:
On Sunday, 21 August 2016 at 19:08:59 UTC, Matthias Klumpp 
wrote:
for a project like Terminix, dub with LDC builds in 8.6s, 
while Meson and ninja take only 6s here.


Did you try to build with DUB but with WIFI or ethernet 
interface toggled off ?


No, but I obviously excluded any download times, and also 
dependency build-times to be fair to dub.


But, with network access toggled off:

Terminix:
  debug build:
dub: 0m8.60s
ninja: 0m6.36s
  release build (standard args):
dub: 0m12.923s
ninja: 0m11.740s

asgen:
 debug build:
dub: 0m13.00s
ninja: 0m13.04s
 release build:
dub: 0m25.72s
ninja: 0m19.19s

I ran these a few times, and the results were comparable, 
although I think if I would perform proper statistics the 
variance would be relatively high. Compiler was LDC 1.0.0 (on a 
Xeon E3-1231 v3).


One important thing about Ninja is that it will perform 
split-builds by default, so if you change something, only the 
changed piece needs to be recompiled. I *think* dub can do that 
too, but for some reason it never does it (even when using 
--parallel).
That behavior drastically reduces compile times when working on a 
project.
Also, Ninja knows about all the source files and generated files 
in advance, while dub needs to search for them on every run. That 
of course also has the drawback that one needs to specify the 
source files prior to building in the meson.build file.

Re: Terminix Stable 1.2.0 Released

[Matthias Klumpp via Digitalmars-d-announce]

On Saturday, 23 July 2016 at 17:00:45 UTC, Iain Buclaw wrote:
On 23 July 2016 at 16:24, Matthias Klumpp via 
Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> 
wrote:


3) Making LDC available for more architectures, or making GDC 
support a
higher version of the Phobos standard library and build shared 
libraries.
At time, LDC is the better fit because of shared library 
support and higher
Phobos version. Current D projects are hard to compile with 
GDC because of

the latter reason.
More architectures are not per-se essential, but would be 
awesome to have.
This feature request summarizes the status of arch support for 
D in free

compilers: https://github.com/ldc-developers/ldc/issues/1636



Well, as GDC is supporting the last C++ release, the only next 
logical step would be to get bootstrapping from 2.068 to 2.071 
or whatever version of the frontend has sufficiently ironed out 
all compatibility regressions.


I would love to use GDC for Debian, but a compiler is really 
useless if it doesn't compile 90% of the interesting D projects 
out there...

LDC however, can do that.

This means that backporting compiler fixes and the standard 
library from upstream is acceptably on the cards.  It's just 
that the feature-set will remain the same as 2.068.


API/ABI breaks in Phobos are really, really annoying - but GDC 
having an ancient Phobos version is even more annoying, since 
this basically ties us to using LDC.
GDC doesn't compile the majority of D projects ot there, and for 
my own I need to explicitly add support for it, e.g. by 
backporting standard library bits and shipping them with the 
source code.




5) Have hardening supported for the D compilers: 
https://wiki.debian.org/HardeningWalkthrough




As per the wiki, if you use GDC then there's nothing for you to 
do.


Since the normal toolchain of Linux distributions is GCC based 
and GCC has a pretty good backend with all the features we need, 
using GDC would be a good choice.
But LDCs shared-library support is a pretty big deal for distros, 
and together with the fact that GDC doesn't compile most of the 
interesting new projects, LDC is the way to go.


Furthermore, since GDC is out-of-tree, some distributions like 
Fedora don't have it / can't easily add it.


I would love to see this resolved - is this a manpower problem? 
Or are there other blockers?

Re: Terminix Stable 1.2.0 Released

[Matthias Klumpp via Digitalmars-d-announce]

On Saturday, 23 July 2016 at 11:40:20 UTC, qsdjlf wrote:

[...]
integration in distribution could be a good signal, what a say, 
Excellent signal, for the D language. I know that some people 
here are mostly starving at commercial usage...but for me such 
tools written in D and available in linux distros via their 
package manager would be a much more significant sign.


It is also something that brings people to D - a good toolchain 
and many projects using D and being established in Linux 
distributions is a *huge* influence.


Unfortunately, D is still a bit painful in Linux distros.
Those issues should be resolved to make integration easier:

1) Stable ABI for the D language which ideally also works across 
compilers


2) Fixing dub bugs and adding features - at time, dub is pretty 
unusable for distro packaging, which is why we use Terminix' 
Automake support.

Specifically, these issues need to be addressed:
 - Find & use system dub packages: 
https://github.com/dlang/dub/issues/838
 - Make dub compile by-file and not compile everything with one 
compiler command (for very large projects this makes 
parallelization hard and can also bring down less powerful build 
slaves)
 - Add "dub install" to install a project: 
https://github.com/dlang/dub/issues/839
 - "dub test" shouldn't override the main binary, otherwise we 
can't run tests properly in distros at compile-time: 
https://github.com/dlang/dub/issues/840


3) Making LDC available for more architectures, or making GDC 
support a higher version of the Phobos standard library and build 
shared libraries.
At time, LDC is the better fit because of shared library support 
and higher Phobos version. Current D projects are hard to compile 
with GDC because of the latter reason.
More architectures are not per-se essential, but would be awesome 
to have. This feature request summarizes the status of arch 
support for D in free compilers: 
https://github.com/ldc-developers/ldc/issues/1636


4) Resolving weird LDC bugs like this one: 
https://github.com/ldc-developers/ldc/issues/1618 - might 
actually be an LLVM issue, but I don't know enough to pin down 
the issue.


5) Have hardening supported for the D compilers: 
https://wiki.debian.org/HardeningWalkthrough


6) Ideally have the free D compilers agree on a common set of 
compiler options, like clang supports gcc compiler flags. That 
allows distros to apply compiler flags globally, e.g. to enable 
hardening.


Of these points 1, 2 and 4 are very important, 3 would be really 
nice to have and 5 and 6 would be going the extra mile to make D 
awesome in distros and us as distribution vendors really happy.

Re: Free the DMD backend

[Matthias Klumpp via Digitalmars-d]

On Wednesday, 1 June 2016 at 01:26:53 UTC, Eugene Wissner wrote:

On Tuesday, 31 May 2016 at 20:12:33 UTC, Russel Winder wrote:
On Tue, 2016-05-31 at 10:09 +, Atila Neves via 
Digitalmars-d wrote:

 […]

No, no, no, no. We had LDC be the default already on Arch 
Linux for a while and it was a royal pain. I want to choose 
to use LDC when and if I need performance. Otherwise, I want 
my projects to compile as fast possible and be able to use 
all the shiny new features.


So write a new backend for DMD the licence of which allows DMD 
to be in Debian and Fedora.


LDC shouldn't be the default compiler to be included in Debian 
or Fedora. Reference compiler and the default D compiler in a 
particular distribution are two independent things.


Exactly. But since we can legally distribute DMD in e.g. Debian, 
and DMD is the reference compiler, we will build software in 
Debian with a compiler that upstream might not have tested.
Additionally, new people usually try out a language with the 
default compiler found in their Linux distribution, and there is 
a chance that the reference compiler and default free compiler 
differ, which is just additional pain and plain weird in the 
Linux world.


E.g. think of Python. Everyone uses and tests with CPython, 
although there are other interpreters available. If CPython would 
be non-free, distros would need to compile with a free compiler, 
e.g. PyPy, which is potentially not feature complete, leading to 
a split in the Python ecosystem between what the reference 
compiler (CPython) does, and what people actually use in Linux 
distributions (PyPy). Those compilers might use different 
language versions, or have a different standard library or 
runtime, making the issue worse.
Fortunately, CPython is completely free, so we don't really have 
that issue ;-)

Re: Free the DMD backend

[Matthias Klumpp via Digitalmars-d]

On Sunday, 29 May 2016 at 10:56:57 UTC, Russel Winder wrote:
On Sun, 2016-05-29 at 04:08 +, Joakim via Digitalmars-d 
wrote:



[…]
It would be nice if that happened, but Walter has said 
Symantec isn't interested.  Aren't ldc and GDC enough?


This is why LDC should be seen in the D community as the main 
production toolchain, and Dub should default to LDC for 
compilation.


This is something which has been asked on my blog[1], and I do 
agree that having a completely free-as-in-freedom reference 
compiler would be an awesome win for the D ecosystem, and would 
pretty much kill most of the issues we have at distros to package 
D stuff. D is very unique with its half-proprietary compiler.
LDC seems to be a pretty good fit for replacing the backend. 
Shifting to LDC as reference compiler would basically mean to 
slowly give up DMD though, because other than being tested much, 
there wouldn't be a compelling reason to still use it when focus 
has shifted to LDC / GDC.
In any case, this is definitely something for Walter and Andrei 
to decide, and I do have a feeling that this question might have 
been raised already in the past...


[1]: 
http://blog.tenstral.net/2016/05/adventures-in-d-programming.html#comment-265879