Re: Release D 2.087.0
On Monday, 15 July 2019 at 19:20:27 UTC, Johannes Pfau wrote: I wonder how you can advertise this as a good idea: You have to manually keep declarations in sync, you have to be very careful to get the attributes right, module constructor evaluation order guarantees don't hold, no mangling (no type safety), you pollute the C namespace, no inlining, no templates. This is an established workaround at best, in no way it's a good solution. Was there an agreement that arbitrary amount of code imported in object.d doesn't impact compilation speed or such impact is unimportant?
Re: Release D 2.087.0
On Monday, 15 July 2019 at 19:56:29 UTC, David Nadlinger wrote: (Corollary: This should be fixed in a point release to unbreak various tooling and dependent build systems.) Fortunately, these changes still have not appeared in a release, so we can still fix them. The reason why this discussion occurred in this thread was that a Digger user tried to install the "master" version after a stable release, at which point the change had already occurred in the master branch. https://github.com/dlang/druntime/pull/2681
Re: Release D 2.087.0
On Monday, 15 July 2019 at 20:57:59 UTC, Johannes Pfau wrote:
I don't see how "should be made public" can be interpreted as
"should be installed", especially considering that templates
need source code installed (core.internal), but that's
completely orthogonal to what functions should be private
(core.internal) / public to users of druntime.
However, I'll open a PR to clarify that paragraph.
Yes, clarifying that paragraph is a good idea. The intended
meaning used to be unambiguous especially given this part:
"If a public module consists of a header file and an
implementation file (which are both maintained by hand) then it
is OK to import private modules in the implementation file."
This refers to the fact that some modules (object, core.thread)
used to have hand-written .di files to separate the public
interface from the implementations (which made use of rt.*
imports).
Now that those duplicate .di files have (fortunately!) been done
away with, I can see how the wording could be unclear without
that prior knowledge. And in either case, avoiding to use heavily
overloaded terms ("private"/"public") without further explanation
is a good idea.
This split has been in place since back in the D1/Tango days.
Sure, the core vs rt split did. But core.internal did not exist
in D1.
How core is organised internally has little do with whether rt is
public or not.
— David
Re: Release D 2.087.0
Am Mon, 15 Jul 2019 20:39:14 + schrieb David Nadlinger: > On Monday, 15 July 2019 at 20:27:16 UTC, Johannes Pfau wrote: >> I guess this should be documented somewhere then. > > See druntime/CONTRIBUTING.md: > > ``` > In general, only modules in the 'core' package should be made public. > The single exception is the 'object' module which is not in any package. > > The convention is to put private modules in the 'rt' or 'gc' packages > depending on what they do and only put public modules in 'core'. > > Also, always avoid importing private modules in public modules. […] > ``` > Well, this just opens the discussion on private vs. public modules again. The new array hooks are private as well, according to the definition above they would have to be in rt. And core.internal.* certainly aren't public modules. I don't see how "should be made public" can be interpreted as "should be installed", especially considering that templates need source code installed (core.internal), but that's completely orthogonal to what functions should be private (core.internal) / public to users of druntime. However, I'll open a PR to clarify that paragraph. > This split has been in place since back in the D1/Tango days. Sure, the core vs rt split did. But core.internal did not exist in D1. -- Johannes
Re: Release D 2.087.0
On Monday, 15 July 2019 at 20:27:16 UTC, Johannes Pfau wrote: I guess this should be documented somewhere then. See druntime/CONTRIBUTING.md: ``` In general, only modules in the 'core' package should be made public. The single exception is the 'object' module which is not in any package. The convention is to put private modules in the 'rt' or 'gc' packages depending on what they do and only put public modules in 'core'. Also, always avoid importing private modules in public modules. […] ``` This split has been in place since back in the D1/Tango days. — David
Re: Release D 2.087.0
Am Mon, 15 Jul 2019 19:52:57 + schrieb David Nadlinger: > On Monday, 15 July 2019 at 11:33:44 UTC, Mike Franklin wrote: >> My understanding is the `rt` is the language implementation >> and `core` is the low level library for users. > > This understanding would be mistaken. We haven't been shipping `rt` on > the import path for a long time. `core.internal` exists precisely to > fill this role, i.e. code that needs to be around at druntime import > time, but isn't supposed to be accessed directly by users. > > — David I guess this should be documented somewhere then. GDC has always shipped and still ships rt. and never had any problem with that. -- Johannes
Re: Release D 2.087.0
Am Mon, 15 Jul 2019 20:14:46 + schrieb David Nadlinger: > On Monday, 15 July 2019 at 14:00:23 UTC, Mike Franklin wrote: >> I'm sorry it broke digger, but digger is not how we typically build >> DMD, druntime, and Phobos. > > Either way, there is a simple resolution here: Put new template code or > other artefacts that are actually used via import in core.* (e.g. > core.internal.*). This also provides a natural boundary between legacy > code and the new runtime interface. If more and more code gets > template-ised, rt.* will slowly wither away, but there is nothing wrong > with that. At some point, it will just cease to exist naturally. Well, I guess if we all agree that rt. is basically deprecated, this may be a good way to move forward. -- Johannes
Re: Release D 2.087.0
On Monday, 15 July 2019 at 19:41:11 UTC, Johannes Pfau wrote: And duplicating extern(C) declarations, syncing them manually, ... is a safety liability and maintainance nightmare (see my other post). So in no way should we start to add more such functions interfacing rt to core.internal. core.internal.traits.externDFunc takes care of the safety aspect. This is a non-issue. (Interdependencies should be avoided regardless, of course; it is still just a crutch.) — David
Re: Release D 2.087.0
On Monday, 15 July 2019 at 14:00:23 UTC, Mike Franklin wrote: I'm sorry it broke digger, but digger is not how we typically build DMD, druntime, and Phobos. It also breaks the LDC build system. Just shipping rt.* too by itself would be simple, but as the frontend takes various libraries when referring to rt symbols internally (using types that don't match, etc.), compiling code that uses certain parts of rt.* at the same time as rt.* itself leads to non-obvious breakage. The easiest way of making sure this doesn't happen is to just not ship rt.*. Either way, there is a simple resolution here: Put new template code or other artefacts that are actually used via import in core.* (e.g. core.internal.*). This also provides a natural boundary between legacy code and the new runtime interface. If more and more code gets template-ised, rt.* will slowly wither away, but there is nothing wrong with that. At some point, it will just cease to exist naturally. — David
Re: Release D 2.087.0
On Monday, 15 July 2019 at 19:52:57 UTC, David Nadlinger wrote: On Monday, 15 July 2019 at 11:33:44 UTC, Mike Franklin wrote: My understanding is the `rt` is the language implementation and `core` is the low level library for users. This understanding would be mistaken. We haven't been shipping `rt` on the import path for a long time. `core.internal` exists precisely to fill this role, i.e. code that needs to be around at druntime import time, but isn't supposed to be accessed directly by users. (Corollary: This should be fixed in a point release to unbreak various tooling and dependent build systems.)
Re: Release D 2.087.0
On Monday, 15 July 2019 at 11:33:44 UTC, Mike Franklin wrote: My understanding is the `rt` is the language implementation and `core` is the low level library for users. This understanding would be mistaken. We haven't been shipping `rt` on the import path for a long time. `core.internal` exists precisely to fill this role, i.e. code that needs to be around at druntime import time, but isn't supposed to be accessed directly by users. — David
Re: Release D 2.087.0
Am Mon, 15 Jul 2019 17:59:25 + schrieb Seb: > On Monday, 15 July 2019 at 14:00:23 UTC, Mike Franklin wrote: >> On Monday, 15 July 2019 at 13:00:08 UTC, Vladimir Panteleev wrote: >> We are trying to implement many of those `extern(C)` runtime hooks as templates. Those templates need to be implicitly imported through object.d. That means code that was in `rt` is converted to a template, and then moved to object.d. However, as we do more and more of them object.d becomes unwieldy. I took the initiative to prevent object.d from turning into a more of a monstrosity that it already is, and moved those runtime templates (which used to reside in `rt`) back into `rt`. >>> >>> This is not a problem, and not at all related to the issue we're >>> discussing. The problem is that you chose to move them into `rt` >>> instead of somewhere under `core`, which would respect existing >>> conventions and avoid breakages like the ones we've seen reported in >>> this thread. >> >> It is related. If I follow your suggestion to move these >> implementations to `core.internal` and continue with the objective of >> converting all runtime hooks to templates, the vast majority of `rt` >> will end up being moved to `core.internal`. Is that what you're >> suggesting? >> >> `rt` is the language implementation. `core.internal` contains the >> utilities used internally by druntime and "privately" imported by >> Phobos. Following that established convention, I made the right >> decision. >> >> I'm sorry it broke digger, but digger is not how we typically build >> DMD, druntime, and Phobos. >> >> Mike > > The point is that we don't ship the sources of rt to the user. That's > the separation. With templates sources must be made available to the > user, s.t. the compiler can instantiate them. However, rt doesn't get > shipped to the user as it is compiled only. > But why is that? What's the benefit here? And do we skip to this convention forever, only for legacy reasons? We always shipped rt in gdc btw. Nobody ever complained. If we decide to move that code to core.internal, I'm with Mike that we should simply move all array code out of rt. In the long term, we may even end up moving anything out of rt: Modern D code is template heavy, template code needs the sources available. Inlining needs the sources as well. The more we get rid of TypeInfo and the more we modernize the compiler/runtime interface, this weill become an issue. And duplicating extern(C) declarations, syncing them manually, ... is a safety liability and maintainance nightmare (see my other post). So in no way should we start to add more such functions interfacing rt to core.internal. -- Johannes
Re: Release D 2.087.0
Am Mon, 15 Jul 2019 14:00:23 + schrieb Mike Franklin: > On Monday, 15 July 2019 at 13:00:08 UTC, Vladimir Panteleev wrote: > >>> We are trying to implement many of those `extern(C)` runtime hooks as >>> templates. Those templates need to be implicitly imported through >>> object.d. That means code that was in `rt` is converted to a >>> template, and then moved to object.d. However, as we do more and more >>> of them object.d becomes unwieldy. >>> >>> I took the initiative to prevent object.d from turning into a more of >>> a monstrosity that it already is, and moved those runtime templates >>> (which used to reside in `rt`) back into `rt`. >> >> This is not a problem, and not at all related to the issue we're >> discussing. The problem is that you chose to move them into `rt` >> instead of somewhere under `core`, which would respect existing >> conventions and avoid breakages like the ones we've seen reported in >> this thread. > > It is related. If I follow your suggestion to move these > implementations to `core.internal` and continue with the objective of > converting all runtime hooks to templates, the vast majority of `rt` > will end up being moved to `core.internal`. Is that what you're > suggesting? > > `rt` is the language implementation. `core.internal` contains the > utilities used internally by druntime and "privately" imported by > Phobos. Following that established convention, I made the right > decision. > > I'm sorry it broke digger, but digger is not how we typically build DMD, > druntime, and Phobos. > > Mike I agree here: rt is code deeply tied to the language / compiler. core.internal is largely code which is useful as standalone module (abort, convert, parts of dassert, lifetime, traits, string). However, the structure is not really clear: rt.util (older than core.internal) should probably rather be part of core.internal and some code in core.internal (arrayop, assert) should be in rt. Either way, dictating a code structure on druntime only because of build system aspects (these files are installed, these are not) seems to be a very bad idea. The code should be structured in a way to minimize cross- module dependencies and to seperate compiler specific from generic code. In addition, the build system shipped as part of druntime is the authoritative way to build the project. Even though digger is an important tool, we can't really compromise on code quality in druntime only to stay compatible to build systems using undocumented internals of the runtime build process. -- Johannes
Re: Release D 2.087.0
Am Mon, 15 Jul 2019 12:27:22 + schrieb Vladimir Panteleev: > On Monday, 15 July 2019 at 12:14:16 UTC, Mike Franklin wrote: >> Many of the implementations in `rt/array` require importing or >> referencing other implementations in `rt` (e.g. `rt.lifetime`). >> If they were moved to `core.internal` they would require >> importing `rt` or peeking into `rt` with various hacks, which exactly >> what you've said should not be done. > > This isn't exactly true. The restriction is that core should not > *import* rt. Have a look at all the extern(C) definitions in Druntime - > using extern(C) functions to communicate between the compiler and rt, as > well as core and rt, is not a "hack", but an established mechanism to > invoke the low-level implementations in Druntime. Grepping for extern in core.internal yields one result outside of pareoptions.d. If you count parseoptions.d, 6 results. I wonder how you can advertise this as a good idea: You have to manually keep declarations in sync, you have to be very careful to get the attributes right, module constructor evaluation order guarantees don't hold, no mangling (no type safety), you pollute the C namespace, no inlining, no templates. This is an established workaround at best, in no way it's a good solution. -- Johannes
Re: Release D 2.087.0
Am Mon, 15 Jul 2019 12:40:50 + schrieb Vladimir Panteleev: > On Monday, 15 July 2019 at 12:36:14 UTC, Mike Franklin wrote: >> Many of the implementations in `rt/array` are templates, so the entire >> implementation should be available through object.d, not just >> declarations. > > The amount of templated code is still finite, otherwise you would have > needed to include all of rt into the set of files to be made importable. Which is due to the limited time available in GSoC. Right now, the implementations simply call typeid and forward to the TypeInfo based implementations. But all of these implementations are ultimately meant to be fully templated, and then you'll need access to lots of rt functions. So why should we we now move the functions to core, so that we can later move them back to rt? > >> In `core.internal`, I see utilities that appear to be intended for use >> only within runtime, or "privately" imported by Phobos. >> I do not see implementations for fundamental language features >> as can be found in `rt`. The code in `rt/array` implementations for >> fundamental language features, not utilities to be used privately with >> druntime. > > Please have a closer look: > > - core.internal.hash contains the implementation of hashing routines > used for associative arrays. > - core.internal.arrayop contains the implementation of array vector > operations. This one doesn't seem to be too far from your work in > question. > Both don't have any real dependencies. Maybe all the array implementation stuff could be moved to core, but until that happens moving only parts of the implementation seems to be a bad idea. Especially considering this: >> "rt can import core, but core can't import rt." makes it pretty clear >> to me. It's probably a mistake that we even have both rt and core.internal. core.internal seems to be a much more recent addition (2013), probably inspired by std.internal. It's no wonder there's duplication and no clearly defined scope for the packages. And why would anyone think it's a good idea not to install the rt headers? What do you gain from this, except from a few KB saved disk space? -- Johannes
Re: Release D 2.087.0
On Monday, 15 July 2019 at 14:00:23 UTC, Mike Franklin wrote: On Monday, 15 July 2019 at 13:00:08 UTC, Vladimir Panteleev wrote: We are trying to implement many of those `extern(C)` runtime hooks as templates. Those templates need to be implicitly imported through object.d. That means code that was in `rt` is converted to a template, and then moved to object.d. However, as we do more and more of them object.d becomes unwieldy. I took the initiative to prevent object.d from turning into a more of a monstrosity that it already is, and moved those runtime templates (which used to reside in `rt`) back into `rt`. This is not a problem, and not at all related to the issue we're discussing. The problem is that you chose to move them into `rt` instead of somewhere under `core`, which would respect existing conventions and avoid breakages like the ones we've seen reported in this thread. It is related. If I follow your suggestion to move these implementations to `core.internal` and continue with the objective of converting all runtime hooks to templates, the vast majority of `rt` will end up being moved to `core.internal`. Is that what you're suggesting? `rt` is the language implementation. `core.internal` contains the utilities used internally by druntime and "privately" imported by Phobos. Following that established convention, I made the right decision. I'm sorry it broke digger, but digger is not how we typically build DMD, druntime, and Phobos. Mike The point is that we don't ship the sources of rt to the user. That's the separation. With templates sources must be made available to the user, s.t. the compiler can instantiate them. However, rt doesn't get shipped to the user as it is compiled only. As probably more tools and distributions are affected I suggest we move all source code that needs to be instantiated by the user to core.internal.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 13:00:08 UTC, Vladimir Panteleev wrote: We are trying to implement many of those `extern(C)` runtime hooks as templates. Those templates need to be implicitly imported through object.d. That means code that was in `rt` is converted to a template, and then moved to object.d. However, as we do more and more of them object.d becomes unwieldy. I took the initiative to prevent object.d from turning into a more of a monstrosity that it already is, and moved those runtime templates (which used to reside in `rt`) back into `rt`. This is not a problem, and not at all related to the issue we're discussing. The problem is that you chose to move them into `rt` instead of somewhere under `core`, which would respect existing conventions and avoid breakages like the ones we've seen reported in this thread. It is related. If I follow your suggestion to move these implementations to `core.internal` and continue with the objective of converting all runtime hooks to templates, the vast majority of `rt` will end up being moved to `core.internal`. Is that what you're suggesting? `rt` is the language implementation. `core.internal` contains the utilities used internally by druntime and "privately" imported by Phobos. Following that established convention, I made the right decision. I'm sorry it broke digger, but digger is not how we typically build DMD, druntime, and Phobos. Mike
Re: Release D 2.087.0
On Monday, 15 July 2019 at 13:27:39 UTC, Mike Franklin wrote: I asked for input from other developers before moving forward. They helped me understand that `rt` is where the core language features are implemented. Assuming it was the discussion linked in this thread, it did not seem like the finer details which we covered today have been considered back then. There are also two developers in this thread which are arguing the contrary, and it would be unfair to count only those participating in the prior discussion if you would make this a matter of numbers. I submitted PRs that were scrutinized by the reviewers and merged. Bad PRs get merged all the time. It happens. We need to recognize mistakes and do our best to address them. The code that I moved into `rt/array` initially came from `rt`, depends on implementations in `rt` and is the core implementation of D's built-in arrays, so it is perfectly logical for it to be in `rt`. That's a subjective assessment, and it would be perfectly fine to rely on subjective opinions to make decisions which would otherwise have no consequence, but it is not the case here. The decision was an avoidable breaking change. I'm sorry you disagree and don't find my explanations convincing. There are a few arguments remaining in this thread to which you have not responded.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 13:06:58 UTC, Vladimir Panteleev wrote: On Monday, 15 July 2019 at 12:42:57 UTC, Mike Franklin wrote: ... and are the exception, not the rule. I believe they should be moved to `rt`. BTW, from this discussion it seems to me that you did not have a good overview of the situation and made a bad decision based on that. No problem there, however, throughout this discussion, you've made a number of claims that have been refuted, and instead of re-evaluating the situation, you've stuck to your position while presenting continuously more radical counter-arguments. At this point, I apologize and would like to ask you to carefully consider whether you are trying to neutrally argue for the best outcome to make D better, or merely to continue supporting your position. I'm not thoughtless and clueless, and I've acted in the best interest of D. I asked for input from other developers before moving forward. They helped me understand that `rt` is where the core language features are implemented. The code that I moved into `rt/array` initially came from `rt`, depends on implementations in `rt` and is the core implementation of D's built-in arrays, so it is perfectly logical for it to be in `rt`. I submitted PRs that were scrutinized by the reviewers and merged. Even after considering your arguments, I still believe I made the right decision. I'm sorry you disagree and don't find my explanations convincing. Mike
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:42:57 UTC, Mike Franklin wrote: ... and are the exception, not the rule. I believe they should be moved to `rt`. BTW, from this discussion it seems to me that you did not have a good overview of the situation and made a bad decision based on that. No problem there, however, throughout this discussion, you've made a number of claims that have been refuted, and instead of re-evaluating the situation, you've stuck to your position while presenting continuously more radical counter-arguments. At this point, I apologize and would like to ask you to carefully consider whether you are trying to neutrally argue for the best outcome to make D better, or merely to continue supporting your position.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:57:46 UTC, Mike Franklin wrote: On Monday, 15 July 2019 at 12:27:22 UTC, Vladimir Panteleev wrote: This isn't exactly true. The restriction is that core should not *import* rt. Have a look at all the extern(C) definitions in Druntime - using extern(C) functions to communicate between the compiler and rt, as well as core and rt, is not a "hack", but an established mechanism to invoke the low-level implementations in Druntime. We are trying to implement many of those `extern(C)` runtime hooks as templates. Those templates need to be implicitly imported through object.d. That means code that was in `rt` is converted to a template, and then moved to object.d. However, as we do more and more of them object.d becomes unwieldy. I took the initiative to prevent object.d from turning into a more of a monstrosity that it already is, and moved those runtime templates (which used to reside in `rt`) back into `rt`. This is not a problem, and not at all related to the issue we're discussing. The problem is that you chose to move them into `rt` instead of somewhere under `core`, which would respect existing conventions and avoid breakages like the ones we've seen reported in this thread.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:27:22 UTC, Vladimir Panteleev wrote: This isn't exactly true. The restriction is that core should not *import* rt. Have a look at all the extern(C) definitions in Druntime - using extern(C) functions to communicate between the compiler and rt, as well as core and rt, is not a "hack", but an established mechanism to invoke the low-level implementations in Druntime. We are trying to implement many of those `extern(C)` runtime hooks as templates. Those templates need to be implicitly imported through object.d. That means code that was in `rt` is converted to a template, and then moved to object.d. However, as we do more and more of them object.d becomes unwieldy. I took the initiative to prevent object.d from turning into a more of a monstrosity that it already is, and moved those runtime templates (which used to reside in `rt`) back into `rt`. Mike
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:42:57 UTC, Mike Franklin wrote: On Monday, 15 July 2019 at 12:40:50 UTC, Vladimir Panteleev wrote: - core.internal.hash contains the implementation of hashing routines used for associative arrays. - core.internal.arrayop contains the implementation of array vector operations. This one doesn't seem to be too far from your work in question. ... and are the exception, not the rule. I believe they should be moved to `rt`. To what end? You are supporting a change (and proposing further changes) which breaks existing conventions and solves zero problems, while simultaneously opposing a change that would fix at least two situations that we know of (Digger and Jonathan's installation script).
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:36:14 UTC, Mike Franklin wrote: Many of the implementations in `rt/array` are templates, so the entire implementation should be available through object.d, not just declarations. The amount of templated code is still finite, otherwise you would have needed to include all of rt into the set of files to be made importable. In `core.internal`, I see utilities that appear to be intended for use only within runtime, or "privately" imported by Phobos. I do not see implementations for fundamental language features as can be found in `rt`. The code in `rt/array` implementations for fundamental language features, not utilities to be used privately with druntime. Please have a closer look: - core.internal.hash contains the implementation of hashing routines used for associative arrays. - core.internal.arrayop contains the implementation of array vector operations. This one doesn't seem to be too far from your work in question. "rt can import core, but core can't import rt." makes it pretty clear to me. Please see my other reply.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:40:50 UTC, Vladimir Panteleev wrote: - core.internal.hash contains the implementation of hashing routines used for associative arrays. - core.internal.arrayop contains the implementation of array vector operations. This one doesn't seem to be too far from your work in question. ... and are the exception, not the rule. I believe they should be moved to `rt`.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:19:02 UTC, Vladimir Panteleev wrote: You don't need to move the implementations themselves into core.internal. Adding declarations there for the rt implementations would suffice. Many of the implementations in `rt/array` are templates, so the entire implementation should be available through object.d, not just declarations. So is the rest of core.internal. Please take a few minutes to have a look at the contents of the core.internal package, as well as all the places in object.d where modules from core.internal are already imported. Would you say that those cases are very unlike the case of this particular change? If so, would you mind explaining how? In `core.internal`, I see utilities that appear to be intended for use only within runtime, or "privately" imported by Phobos. I do not see implementations for fundamental language features as can be found in `rt`. The code in `rt/array` implementations for fundamental language features, not utilities to be used privately with druntime. Based on what you and Vladimir have said, I'm even more confident I've made the right decision putting them in `rt`. I really don't see the series of logical steps one would need to take to reach that conclusion. "rt can import core, but core can't import rt." makes it pretty clear to me. Mike
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:14:16 UTC, Mike Franklin wrote: Many of the implementations in `rt/array` require importing or referencing other implementations in `rt` (e.g. `rt.lifetime`). If they were moved to `core.internal` they would require importing `rt` or peeking into `rt` with various hacks, which exactly what you've said should not be done. This isn't exactly true. The restriction is that core should not *import* rt. Have a look at all the extern(C) definitions in Druntime - using extern(C) functions to communicate between the compiler and rt, as well as core and rt, is not a "hack", but an established mechanism to invoke the low-level implementations in Druntime.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:14:16 UTC, Mike Franklin wrote: On Monday, 15 July 2019 at 12:02:35 UTC, Seb wrote: I think that fits core.internal better than rt. Have you considered that during said discussion? The implementations in `rt/array` contain templates that are ports of runtime hooks that used to reside in `rt`. The implementations also require importing implementations from `rt` so it makes sense to me that they should all stay within `rt`. You don't need to move the implementations themselves into core.internal. Adding declarations there for the rt implementations would suffice. `rt` is not imported by the user; it is imported by the compiler indirectly through `object.d`. So is the rest of core.internal. Please take a few minutes to have a look at the contents of the core.internal package, as well as all the places in object.d where modules from core.internal are already imported. Would you say that those cases are very unlike the case of this particular change? If so, would you mind explaining how? Based on what you and Vladimir have said, I'm even more confident I've made the right decision putting them in `rt`. I really don't see the series of logical steps one would need to take to reach that conclusion.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 12:02:35 UTC, Seb wrote: I think that fits core.internal better than rt. Have you considered that during said discussion? The implementations in `rt/array` contain templates that are ports of runtime hooks that used to reside in `rt`. The implementations also require importing implementations from `rt` so it makes sense to me that they should all stay within `rt`. For reference, the discussion was this one: http://imgur.com/a/HszdtHd I'm sorry that I missed it, because Vladimir makes a very good point. rt isn't supposed to be available nor importable by the user. `rt` is not imported by the user; it is imported by the compiler indirectly through `object.d`. rt can import core, but core can't import rt. Many of the implementations in `rt/array` require importing or referencing other implementations in `rt` (e.g. `rt.lifetime`). If they were moved to `core.internal` they would require importing `rt` or peeking into `rt` with various hacks, which exactly what you've said should not be done. Based on what you and Vladimir have said, I'm even more confident I've made the right decision putting them in `rt`. Mike
Re: Release D 2.087.0
On Monday, 15 July 2019 at 11:49:31 UTC, Vladimir Panteleev wrote: On Monday, 15 July 2019 at 11:33:44 UTC, Mike Franklin wrote: I discussed that briefly on Slack with a couple other developers. My understanding is the `rt` is the language implementation and `core` is the low level library for users. The code in `rt/array` are language implementations. They are not to be imported by the user. They are, however, required by the language to implement D's built-in arrays, so that is why they are publicly imported in object.d. I think that fits core.internal better than rt. Have you considered that during said discussion? For reference, the discussion was this one: http://imgur.com/a/HszdtHd I'm sorry that I missed it, because Vladimir makes a very good point. rt isn't supposed to be available nor importable by the user. rt can import core, but core can't import rt.
Re: Release D 2.087.0
On Monday, 15 July 2019 at 11:33:44 UTC, Mike Franklin wrote: I discussed that briefly on Slack with a couple other developers. My understanding is the `rt` is the language implementation and `core` is the low level library for users. The code in `rt/array` are language implementations. They are not to be imported by the user. They are, however, required by the language to implement D's built-in arrays, so that is why they are publicly imported in object.d. I think that fits core.internal better than rt. Have you considered that during said discussion?
Re: Release D 2.087.0
On Monday, 15 July 2019 at 10:52:08 UTC, Vladimir Panteleev wrote: On Thursday, 4 July 2019 at 12:57:43 UTC, Mike Franklin wrote: The copy should take place when building druntime from the makefiles. The files to be copied are listed at https://github.com/dlang/druntime/blob/12bcb73da97a0c26aaf4b943eabd3c25051a89da/mak/COPY#L405-L408 and, for Windows, should be copied https://github.com/dlang/druntime/blob/12bcb73da97a0c26aaf4b943eabd3c25051a89da/mak/WINDOWS#L1187-L1197 is executed. But, I don't know if that happens or not with digger. Why have these declarations been placed in the "rt" package, and not in "core.internal"? The "rt" package has historically always been for the Druntime implementation part which is not importable from D programs. I think "core.internal" would have been more suitable. I discussed that briefly on Slack with a couple other developers. My understanding is the `rt` is the language implementation and `core` is the low level library for users. The code in `rt/array` are language implementations. They are not to be imported by the user. They are, however, required by the language to implement D's built-in arrays, so that is why they are publicly imported in object.d. Mike
Re: Release D 2.087.0
On Friday, 5 July 2019 at 03:47:20 UTC, Jonathan M Davis wrote: Yeah. I ran into the same problem with my own build tool. There wasn't previously an rt folder in the imports. It was all hidden in the implementation, and my build tool didn't copy it over, resulting in confusing errors at first when druntime was recently changed to have an rt folder in the imports. I think that for druntime, it currently works to just copy over everything in the import folder, whereas with Phobos, you have to copy over specific directories (std and etc IIRC). So, for Phobos, you can't just grab everything from a single folder, and it may have been the case with druntime at one point that you couldn't either (I'm not sure). So, it doesn't really surprise me that digger broke. Any time that a tool is having to duplicate any logic from the build system (even if it's just which files to grab to install rather than for the build itself), it risks breaking any time that the build system is altered. Digger has a good reason and a not-so-good reason to avoid using the included build system logic. The good reason is that Digger attempts to support building a range of D versions as large as possible on modern systems. Sometimes, that means patching or entirely avoiding parts of the build system included with the D source code. However, this was not the case in this situation. The not-so-good reason, and the root issue here, is the "digger install" functionality, which attempts to merge an existing "standard" D installation with one built by Digger. At the time I implemented it, we didn't have a good well-defined process of building release archives (rather, it was the duty of the release manager at the time to get it right), so, there was no way for Digger to know how to correctly build a complete D distribution including all the extra files which are not versioned, but nevertheless included in the archives; "digger install" was thus created as a work-around for that situation. This Digger feature has a hard-coded list of objects it needs to patch / update, which as you can guess did not include the "rt" package from Druntime.
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 12:57:43 UTC, Mike Franklin wrote: The copy should take place when building druntime from the makefiles. The files to be copied are listed at https://github.com/dlang/druntime/blob/12bcb73da97a0c26aaf4b943eabd3c25051a89da/mak/COPY#L405-L408 and, for Windows, should be copied https://github.com/dlang/druntime/blob/12bcb73da97a0c26aaf4b943eabd3c25051a89da/mak/WINDOWS#L1187-L1197 is executed. But, I don't know if that happens or not with digger. Why have these declarations been placed in the "rt" package, and not in "core.internal"? The "rt" package has historically always been for the Druntime implementation part which is not importable from D programs. I think "core.internal" would have been more suitable.
Re: Release D 2.087.0
On Sunday, 7 July 2019 at 10:09:50 UTC, Johannes Pfau wrote:
Am Sun, 07 Jul 2019 08:06:57 + schrieb uranuz:
After updating compiler to 2.087 I got a lot of deprecation
warnings
linked to std.json module. I have found all of the usages of
deprecated
symbols in my project and changed them to the new ones. All
these
warnings are about changing JSON_TYPE to JSONType
JSON_TYPE.STRING to
JSONType.string and etc...
But after eleminating deprecated symbols from my project I
still have
deprecation warnings. Seems that these symbols being accessed
from
Phobos, because I am pretty sure that I don't have other
external
dependencies that use std.json. The problem it that because of
this
`spamming` messages I can miss `real` deprecation warnings. Is
there
some way to `fix` it? The is some part of compiler output (all
of it is
too long): /usr/include/dmd/phobos/std/conv.d(987,34):
Deprecation: enum
member `std.json.JSONType.INTEGER` is deprecated - Use .integer
/usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum
member
I think phobos does not explicitly use these deprecated
symbols, but the reflection code in format triggers the
deprecation messages:
import std.json, std.stdio;
void main()
{
JSONType c;
writefln("%s", c);
}
I'm not sure if this can be solved, maybe deprecated members
can be explicitly ignored in format, but maybe the reflection
code itself is triggering the deprecation (which would then
probably be a DMD bug).
Yes. It looks like it, because seems that `format` and `conv` use
the same approach to convert data to string and in process of
iterating reflection code `touches` deprecated symbols and it
`starts to smell`... Maybe some special `compilation context`
should exists where these deprecation warnings are ignored or
ignore them when we access or iterate symbols using __trait(...)
Should I file a bugreport?
Re: Release D 2.087.0
Am Sun, 07 Jul 2019 08:06:57 + schrieb uranuz:
> After updating compiler to 2.087 I got a lot of deprecation warnings
> linked to std.json module. I have found all of the usages of deprecated
> symbols in my project and changed them to the new ones. All these
> warnings are about changing JSON_TYPE to JSONType JSON_TYPE.STRING to
> JSONType.string and etc...
> But after eleminating deprecated symbols from my project I still have
> deprecation warnings. Seems that these symbols being accessed from
> Phobos, because I am pretty sure that I don't have other external
> dependencies that use std.json. The problem it that because of this
> `spamming` messages I can miss `real` deprecation warnings. Is there
> some way to `fix` it? The is some part of compiler output (all of it is
> too long): /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum
> member `std.json.JSONType.INTEGER` is deprecated - Use .integer
> /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member
I think phobos does not explicitly use these deprecated symbols, but the
reflection code in format triggers the deprecation messages:
import std.json, std.stdio;
void main()
{
JSONType c;
writefln("%s", c);
}
I'm not sure if this can be solved, maybe deprecated members can be
explicitly ignored in format, but maybe the reflection code itself is
triggering the deprecation (which would then probably be a DMD bug).
--
Johannes
Re: Release D 2.087.0
After updating compiler to 2.087 I got a lot of deprecation warnings linked to std.json module. I have found all of the usages of deprecated symbols in my project and changed them to the new ones. All these warnings are about changing JSON_TYPE to JSONType JSON_TYPE.STRING to JSONType.string and etc... But after eleminating deprecated symbols from my project I still have deprecation warnings. Seems that these symbols being accessed from Phobos, because I am pretty sure that I don't have other external dependencies that use std.json. The problem it that because of this `spamming` messages I can miss `real` deprecation warnings. Is there some way to `fix` it? The is some part of compiler output (all of it is too long): /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.INTEGER` is deprecated - Use .integer /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.INTEGER` is deprecated - Use .integer /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.UINTEGER` is deprecated - Use .uinteger /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.UINTEGER` is deprecated - Use .uinteger /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.FLOAT` is deprecated - Use .float_ /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.FLOAT` is deprecated - Use .float_ /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.ARRAY` is deprecated - Use .array /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.ARRAY` is deprecated - Use .array /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.OBJECT` is deprecated - Use .object /usr/include/dmd/phobos/std/conv.d(987,34): Deprecation: enum member `std.json.JSONType.OBJECT` is deprecated - Use .object /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.NULL` is deprecated - Use .null_ /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.NULL` is deprecated - Use .null_ /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.STRING` is deprecated - Use .string /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.STRING` is deprecated - Use .string /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.INTEGER` is deprecated - Use .integer /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.INTEGER` is deprecated - Use .integer /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.UINTEGER` is deprecated - Use .uinteger /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.UINTEGER` is deprecated - Use .uinteger /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.FLOAT` is deprecated - Use .float_ /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.FLOAT` is deprecated - Use .float_ /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.ARRAY` is deprecated - Use .array /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.ARRAY` is deprecated - Use .array /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.OBJECT` is deprecated - Use .object /usr/include/dmd/phobos/std/format.d(4373,24): Deprecation: enum member `std.json.JSONType.OBJECT` is deprecated - Use .object
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 08:11:26 UTC, Martin Nowak wrote: Glad to announce D 2.087.0, ♥ to the 63 contributors. This release comes with types matching single template alias parameters, nested template methods/local template functions, multi-threaded GC marking, and a phobos compiled with -preview=DIP1000. http://dlang.org/download.html http://dlang.org/changelog/2.087.0.html -Martin The blog announcement: https://dlang.org/blog/2019/07/05/dmd-2-087-0-released/ And on reddit: https://www.reddit.com/r/programming/comments/c9g5pg/d_20870_released/
Re: Release D 2.087.0
Another stat that might make sense to people who have the history of that stat, is views of videos in the foundation's channel and in the Dconf channels on YouTube.
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 09:13:39 UTC, Martin Tschierschke wrote: On Thursday, 4 July 2019 at 08:47:03 UTC, Martin Tschierschke wrote: I just looked at this again: http://erdani.com/d/downloads.daily.png and it should be a great reward to see how adoption of D increases in the long run. mt. Just repeating something already noted by Andrei about that graph ... It is only direct downloads, it doesn't count Linux distribution installs - atleast Gentoo has structured & fairly up-to-date packages, allowing to switch seamlessly between DMD, GDC & LDC; Debian/Ubuntu/Devuan probably have it too, Arch Linux too? The influx is likely bigger than that graph suggests :-)
Re: Release D 2.087.0
On Thursday, July 4, 2019 6:48:15 AM MDT Robert M. Münch via Digitalmars-d- announce wrote: > On 2019-07-04 10:11:18 +, Mike Franklin said: > > I don't know what digger is doing, but from the error messages, it > > appears that the new files in `rt/array` can't be found. I believe the > > build is trying to use a new compiler with an older or existing > > runtime. You'll need both the latest compiler and the latest runtime > > together for them to work. > > > > If you can identify a bug in the makefiles, or some other such problem > > preventing the build, let me know and I'll try to fix it right away. > > So, the problem is, that digger somehow misses to copy over the new > source to the install directory. It does for some parts (phobos, but > I'm not sure if for every file necessary) but not for druntime files. > > I just manually copied the files now. Yeah. I ran into the same problem with my own build tool. There wasn't previously an rt folder in the imports. It was all hidden in the implementation, and my build tool didn't copy it over, resulting in confusing errors at first when druntime was recently changed to have an rt folder in the imports. I think that for druntime, it currently works to just copy over everything in the import folder, whereas with Phobos, you have to copy over specific directories (std and etc IIRC). So, for Phobos, you can't just grab everything from a single folder, and it may have been the case with druntime at one point that you couldn't either (I'm not sure). So, it doesn't really surprise me that digger broke. Any time that a tool is having to duplicate any logic from the build system (even if it's just which files to grab to install rather than for the build itself), it risks breaking any time that the build system is altered. - Jonathan M Davis
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 12:48:15 UTC, Robert M. Münch wrote: So, the problem is, that digger somehow misses to copy over the new source to the install directory. It does for some parts (phobos, but I'm not sure if for every file necessary) but not for druntime files. I just manually copied the files now. The copy should take place when building druntime from the makefiles. The files to be copied are listed at https://github.com/dlang/druntime/blob/12bcb73da97a0c26aaf4b943eabd3c25051a89da/mak/COPY#L405-L408 and, for Windows, should be copied https://github.com/dlang/druntime/blob/12bcb73da97a0c26aaf4b943eabd3c25051a89da/mak/WINDOWS#L1187-L1197 is executed. But, I don't know if that happens or not with digger. Mike
Re: Release D 2.087.0
On 2019-07-04 10:11:18 +, Mike Franklin said: I don't know what digger is doing, but from the error messages, it appears that the new files in `rt/array` can't be found. I believe the build is trying to use a new compiler with an older or existing runtime. You'll need both the latest compiler and the latest runtime together for them to work. If you can identify a bug in the makefiles, or some other such problem preventing the build, let me know and I'll try to fix it right away. So, the problem is, that digger somehow misses to copy over the new source to the install directory. It does for some parts (phobos, but I'm not sure if for every file necessary) but not for druntime files. I just manually copied the files now. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 10:01:10 UTC, Robert M. Münch wrote: On 2019-07-04 08:11:26 +, Martin Nowak said: Glad to announce D 2.087.0, ♥ to the 63 contributors. This release comes with types matching single template alias parameters, nested template methods/local template functions, multi-threaded GC marking, and a phobos compiled with -preview=DIP1000. http://dlang.org/download.html http://dlang.org/changelog/2.087.0.html -Martin I just updated with Digger to the newest version and get: ...d-language\test> dub test -a=x86_64 -f Failed to invoke the compiler C:\D\dmd2\windows\bin\dmd.exe to determine the build platform: predefs DigitalMars Windows CRuntime_Microsoft CppRuntime_Microsoft LittleEndian D_Version2 all D_SIMD D_InlineAsm_X86_64 X86_64 Win64 D_LP64 assert D_ModuleInfo D_Exceptions D_TypeInfo D_HardFloat binaryC:\D\dmd2\windows\bin\dmd.exe version v2.087.0 configC:\D\dmd2\windows\bin\sc.ini DFLAGS-IC:\D\dmd2\windows\bin\..\..\src\phobos -IC:\D\dmd2\windows\bin\..\..\src\druntime\import -L/OPT:NOICF parse dub_platform_probe_956c8798_90a5_4300_b011_2b9e6cc8ace5 importall dub_platform_probe importobject (C:\D\dmd2\windows\bin\..\..\src\druntime\import\object.d) C:\D\dmd2\windows\bin\..\..\src\druntime\import\object.d(42): Error: module `comparison` is in file 'rt\array\comparison.d' which cannot be read import path[0] = C:\D\dmd2\windows\bin\..\..\src\phobos import path[1] = C:\D\dmd2\windows\bin\..\..\src\druntime\import Not sure what this is telling me. Related to the release? Yes, it's probably related to the release. Some runtime hooks were taken out of object.d and moved to `rt/array` to help set direction for a GSoC project that is in the process converting many runtime hooks to templates, and to help prevent object.d from becoming even more unwieldy than it already is. Move such refactorings are planned. I don't know what digger is doing, but from the error messages, it appears that the new files in `rt/array` can't be found. I believe the build is trying to use a new compiler with an older or existing runtime. You'll need both the latest compiler and the latest runtime together for them to work. If you can identify a bug in the makefiles, or some other such problem preventing the build, let me know and I'll try to fix it right away. Mike
Re: Release D 2.087.0
On 2019-07-04 08:11:26 +, Martin Nowak said: Glad to announce D 2.087.0, ♥ to the 63 contributors. This release comes with types matching single template alias parameters, nested template methods/local template functions, multi-threaded GC marking, and a phobos compiled with -preview=DIP1000. http://dlang.org/download.html http://dlang.org/changelog/2.087.0.html -Martin I just updated with Digger to the newest version and get: ...d-language\test> dub test -a=x86_64 -f Failed to invoke the compiler C:\D\dmd2\windows\bin\dmd.exe to determine the build platform: predefs DigitalMars Windows CRuntime_Microsoft CppRuntime_Microsoft LittleEndian D_Version2 all D_SIMD D_InlineAsm_X86_64 X86_64 Win64 D_LP64 assert D_ModuleInfo D_Exceptions D_TypeInfo D_HardFloat binaryC:\D\dmd2\windows\bin\dmd.exe version v2.087.0 configC:\D\dmd2\windows\bin\sc.ini DFLAGS-IC:\D\dmd2\windows\bin\..\..\src\phobos -IC:\D\dmd2\windows\bin\..\..\src\druntime\import -L/OPT:NOICF parse dub_platform_probe_956c8798_90a5_4300_b011_2b9e6cc8ace5 importall dub_platform_probe importobject (C:\D\dmd2\windows\bin\..\..\src\druntime\import\object.d) C:\D\dmd2\windows\bin\..\..\src\druntime\import\object.d(42): Error: module `comparison` is in file 'rt\array\comparison.d' which cannot be read import path[0] = C:\D\dmd2\windows\bin\..\..\src\phobos import path[1] = C:\D\dmd2\windows\bin\..\..\src\druntime\import Not sure what this is telling me. Related to the release? -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: Release D 2.087.0
Thank you, Martin!
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 08:47:03 UTC, Martin Tschierschke wrote: On Thursday, 4 July 2019 at 08:11:26 UTC, Martin Nowak wrote: Glad to announce D 2.087.0, ♥ to the 63 contributors. [...] http://dlang.org/download.html http://dlang.org/changelog/2.087.0.html -Martin Thank you, all 63! This continuous progress makes me expect a great future for D! I just looked at this again: http://erdani.com/d/downloads.daily.png and it should be a great reward to see how adoption of D increases in the long run. mt.
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 08:11:26 UTC, Martin Nowak wrote: Glad to announce D 2.087.0, ♥ to the 63 contributors. This release comes with types matching single template alias parameters, nested template methods/local template functions, multi-threaded GC marking, and a phobos compiled with -preview=DIP1000. http://dlang.org/download.html http://dlang.org/changelog/2.087.0.html -Martin Thank you, all 63! This continuous progress makes me expect a great future for D!
Re: Release D 2.087.0
On Thursday, 4 July 2019 at 08:11:26 UTC, Martin Nowak wrote: Glad to announce D 2.087.0, ♥ to the 63 contributors. This release comes with types matching single template alias parameters, nested template methods/local template functions, multi-threaded GC marking, and a phobos compiled with -preview=DIP1000. http://dlang.org/download.html http://dlang.org/changelog/2.087.0.html -Martin And this **long** awaited __traits/is extension : https://dlang.org/changelog/2.087.0.html#fix16002 yay
Release D 2.087.0
Glad to announce D 2.087.0, ♥ to the 63 contributors. This release comes with types matching single template alias parameters, nested template methods/local template functions, multi-threaded GC marking, and a phobos compiled with -preview=DIP1000. http://dlang.org/download.html http://dlang.org/changelog/2.087.0.html -Martin
