Re: code.dlang.org reliability update
On Monday, 2 March 2020 at 19:17:59 UTC, Sönke Ludwig wrote: As of yesterday, code.dlang.org now points to a more powerful dedicated server that can host the DUB registry without the danger of freezing due to excessive swapping - this is what happened on the 26th last month [1]. In addition to that, the server that previously hosted the registry is now used to run an official mirror, reachable at codemirror.dlang.org. This will be configured as a built-in fallback server starting with DMD 2.091.0/DUB 1.12.0 and, at least in theory, will lead to an uptime of virtually 100%. Excellent, thank you! Bastiaan.
Re: code.dlang.org reliability update
On Monday, 2 March 2020 at 19:17:59 UTC, Sönke Ludwig wrote: settings.json is found/needs to be created in %APPDATA%\dub\ on Windows I guess that should be %APPDATA%\Local\dub\
Re: code.dlang.org reliability update
Am 02.03.2020 um 23:50 schrieb MrSmith: On Monday, 2 March 2020 at 19:17:59 UTC, Sönke Ludwig wrote: settings.json is found/needs to be created in %APPDATA%\dub\ on Windows I guess that should be %APPDATA%\Local\dub\ Nope, should be right. At least on my machines, %APPDATA% points to AppData\Roaming, which is the right base folder.
Re: code.dlang.org reliability update
On Monday, 2 March 2020 at 19:17:59 UTC, Sönke Ludwig wrote: As of yesterday, code.dlang.org now points to a more powerful dedicated server that can host the DUB registry without the danger of freezing due to excessive swapping - this is what happened on the 26th last month [1]. Sönke, thanks a lot for all the effort you are putting into this over the years. Very much appreciated!
Re: code.dlang.org reliability update
On Tuesday, 3 March 2020 at 08:26:19 UTC, Sönke Ludwig wrote: Nope, should be right. At least on my machines, %APPDATA% points to AppData\Roaming, which is the right base folder. My bad, confused %APPDATA% with AppData. It is indeed AppData\Roaming\dub\
Re: code.dlang.org reliability update
On Monday, 2 March 2020 at 19:17:59 UTC, Sönke Ludwig wrote:
As of yesterday, code.dlang.org now points to a more powerful
dedicated server that can host the DUB registry without the
danger of freezing due to excessive swapping - this is what
happened on the 26th last month [1].
In addition to that, the server that previously hosted the
registry is now used to run an official mirror, reachable at
codemirror.dlang.org. This will be configured as a built-in
fallback server starting with DMD 2.091.0/DUB 1.12.0 and, at
least in theory, will lead to an uptime of virtually 100%.
To make use of the mirror today, it is also possible to
configure it in DUB's settings.json as a custom registry:
{
"registryUrls": ["https://codemirror.dlang.org/";]
}
settings.json is found/needs to be created in %APPDATA%\dub\ on
Windows and in ~/.dub/ on all other systems. The custom entry
should be removed once DUB 1.12.0 is used, to avoid redundant
requests in certain situations.
[1]:
https://forum.dlang.org/thread/ontwwoxuhnoczcoka...@forum.dlang.org
thank you very much for this Sönke! Is throwing so much more RAM
(= money) and power (= more money) at it going to be a good
solution in the long run though?
Is there maybe a plan for remaking the registry architecture like
adding better supported mirrors and load balancing to it?
Re: code.dlang.org reliability update
On 3/2/2020 11:17 AM, Sönke Ludwig wrote: As of yesterday, code.dlang.org now points to a more powerful dedicated server that can host the DUB registry without the danger of freezing due to excessive swapping - this is what happened on the 26th last month [1]. Wonderful news!
Re: code.dlang.org reliability update
Am 04.03.2020 um 20:06 schrieb WebFreak001:
On Monday, 2 March 2020 at 19:17:59 UTC, Sönke Ludwig wrote:
As of yesterday, code.dlang.org now points to a more powerful
dedicated server that can host the DUB registry without the danger of
freezing due to excessive swapping - this is what happened on the 26th
last month [1].
In addition to that, the server that previously hosted the registry is
now used to run an official mirror, reachable at codemirror.dlang.org.
This will be configured as a built-in fallback server starting with
DMD 2.091.0/DUB 1.12.0 and, at least in theory, will lead to an uptime
of virtually 100%.
To make use of the mirror today, it is also possible to configure it
in DUB's settings.json as a custom registry:
{
"registryUrls": ["https://codemirror.dlang.org/";]
}
settings.json is found/needs to be created in %APPDATA%\dub\ on
Windows and in ~/.dub/ on all other systems. The custom entry should
be removed once DUB 1.12.0 is used, to avoid redundant requests in
certain situations.
[1]: https://forum.dlang.org/thread/ontwwoxuhnoczcoka...@forum.dlang.org
thank you very much for this Sönke! Is throwing so much more RAM (=
money) and power (= more money) at it going to be a good solution in the
long run though?
Is there maybe a plan for remaking the registry architecture like adding
better supported mirrors and load balancing to it?
The server has a different main purpose, so this is just a nice benefit
for the registry in that regard. The main issue is that there is some
kind of GC related memory leak that is hard to debug with the basically
non-existent tooling. Decoupling the package scanning from the web
frontend into separate processes will play a major role in making that
much less of an issue, though.
There is at least one fully supported mirror now and the cache based
mirror by Sebastiaan could also be added as a reliable (albeit partial)
fallback. So in that regard, at least the pressure for taking more
involved steps has been mostly relieved now.
But making changes to the general architecture - be it in the form of
adding load-balancing, or more deeply rooted, such as using GIT for
storing and distributing the package index - is something that still is
definitely desirable to push forward.
Re: code.dlang.org reliability update
On Thursday, 5 March 2020 at 12:41:41 UTC, Sönke Ludwig wrote: The main issue is that there is some kind of GC related memory leak that is hard to debug with the basically non-existent tooling. What a shame for D, how is this possible?
Re: code.dlang.org reliability update
On Thursday, 5 March 2020 at 12:41:41 UTC, Sönke Ludwig wrote: But making changes to the general architecture - be it in the form of adding load-balancing, or more deeply rooted, such as using GIT for storing and distributing the package index - is something that still is definitely desirable to push forward. A week ago I created a git dub package index at https://github.com/skoppe/dub-packages-index it gets updated every 15 minutes by scraping code.dlang.org I use it to feed my registry-mirror, which is now fully standalone A possible next step is creating a PackageSupplier class in dub that clones and uses this repo for resolution. If I was aware of Sönke's work I might have skipped it, but it was a good exercise regardless.
