Re: Fluid 0.6.0 — UI library for D
On Thursday, 25 January 2024 at 12:33:31 UTC, cookiewitch wrote: Fluid is a flexible UI library for the D programming language. Minimal setup. Declarative. Non-intrusive. Awesome!
Re: Good News: Almost all druntime supported on arsd webassembly
On Thursday, 9 November 2023 at 04:02:48 UTC, Lingo Chen wrote: Yes, the wasmgc's main selling point is the sharing of GC and TypeInfo between Javascript and WASM. Not just javascript, but any WASM runtime. Sharing of data between the JS/WASM are problematic. JS has no destructor, so it practically impossible to do manual memory/resource management in JS side; One has to do a lot of works to pass data around. With spasm I was auto-generating all glue code based off of WebIDL files. The post build step was annoying but it worked pretty well. I even had a prototype that would do it for any typescript library. Of course using WASM ref types is way more ergonomic. One thing to note about WASMGC, it is MVP and missing a lot of things like support for interior pointers etc. There are workarounds but it's not so simple.
Re: Good News: Almost all druntime supported on arsd webassembly
On Monday, 6 November 2023 at 23:44:11 UTC, Lingo Chen wrote: On Monday, 6 November 2023 at 11:05:45 UTC, Sebastiaan Koppe wrote: https://llvm.org/devmtg/2022-11/slides/TechTalk3-ClangClang-WebAssembly.pdf llvm ir seems to have support for the reftype(gc). so it a compiler problem? Unsure how it's progressed since but as can be seen on the last slide of that pdf, it's work in progress. Kotlin abandoned llvm and implemented a new wasm backend. So waiting for llvm probably is not a good idea? Is taking D's frontend + binaryen's backend a feasible solution? If wasmgc's architecture is a good fit for D. Can it be done quickly? I have only looked approx 5 minutes at it, so its really hard to say. I expect it to be quite some effort. There are some things we can do today that are smaller in scope and will help any future effort, such as reviving my druntime port. You are going to need that anyway, unless you go with the custom druntime or betterC route.
Re: Good News: Almost all druntime supported on arsd webassembly
On Monday, 6 November 2023 at 00:47:43 UTC, Lingo Chen wrote: On Sunday, 5 November 2023 at 16:18:53 UTC, Sebastiaan Koppe wrote: On Sunday, 5 November 2023 at 01:28:25 UTC, Lingo Chen wrote: https://v8.dev/blog/wasm-gc-porting wasmgc is shipping. any plan for porting? My understanding is that it requires emitting dedicated instructions for allocating such objects, as well as dedicated instructions for accessing them. Since LLVM doesn't support them, I am not sure what a possible path to support should look like, aside from writing a complete fresh backend. https://llvm.org/devmtg/2022-11/slides/TechTalk3-ClangClang-WebAssembly.pdf llvm ir seems to have support for the reftype(gc). so it a compiler problem? Unsure how it's progressed since but as can be seen on the last slide of that pdf, it's work in progress.
Re: Good News: Almost all druntime supported on arsd webassembly
On Sunday, 5 November 2023 at 01:28:25 UTC, Lingo Chen wrote: https://v8.dev/blog/wasm-gc-porting wasmgc is shipping. any plan for porting? My understanding is that it requires emitting dedicated instructions for allocating such objects, as well as dedicated instructions for accessing them. Since LLVM doesn't support them, I am not sure what a possible path to support should look like, aside from writing a complete fresh backend.
Re: D Language Foundation Monthly Meeting Summary for May 2023
On Friday, 23 June 2023 at 14:32:51 UTC, Mike Parker wrote: [...] Walter brought up [Sebastiaan Koppe's presentation from last year](https://youtu.be/hJhNhIeq29U) on structured concurrency. He said he'd like to see Sebastiaan there this year for an update on the project, preferably as a follow-up presentation. He was excited about the project and felt it was a big deal in general and a big deal for D. Even if Sebastiaan decided not to move the project forward, Walter felt we should find someone to take it over. He said the ideas behind it were big and would be a nice thing for us. (We subsequently confirmed with Sebastiaan that he didn't have enough material yet for a follow-up talk). [...] For anyone interested, this is still very much alive. In fact, the lovely folks at https://github.com/NVIDIA/stdexec have validated several ideas I had for improving things, and I have been using any spare time I have left - which isn't much sadly - to incorporate these concepts. I hope to find some time to do a lighting talk.
Re: Release: serverino - please destroy it.
On Tuesday, 10 May 2022 at 10:49:06 UTC, Andrea Fontana wrote: On Tuesday, 10 May 2022 at 08:32:15 UTC, Sebastiaan Koppe wrote: The difference is that with the route uda you can *only* map routes 1:1 exhaustively. With your approach it is up to the programmer to avoid errors. It is also hard to reason about the flow of requests through all those functions, and you have to look at the body of them to determine what will happen. Sorry I don't follow you It is simple, since all your handler are effectively chained, any error in any one of them can cause later ones to fail or misbehave. This decreases locality and increases the things you have to reason about. There are other benefits to uda tagged endpoints too, for example they are easier to nest, or to programmatically generate them. In vibe-d I added the default option of generating OPTION handlers for every regular endpoint. This is required for CORS. In any case if you want to use a different routing strategy it's quite easy. I really don't like libraries that force you to use their own style/way. That is good.
Re: Release: serverino - please destroy it.
On Monday, 9 May 2022 at 20:37:50 UTC, Andrea Fontana wrote:
On Monday, 9 May 2022 at 20:08:38 UTC, Sebastiaan Koppe wrote:
As an example, how many requests per second can you manage if
all requests have to wait 100 msecs?
For non critical workload you will probably still hit good
enough performance though.
Firstly, it depends on how many workers you have.
Then you should consider that a lot of (most?) websites use
php-fpm, that works using the same approach (but php is much
slower than D). The same goes for cgi/fastcgi/scgi and so on.
Let's say you have just 20 workers. 100msecs for each request
(a lot of time for my standards, I would say). That means 20*10
= 200 webpages/s = 720k pages/h. I don't think your website has
so much traffic...
And I hope not every request will take 100msecs!
100msecs is on the upper end for sure, but if you add a database,
external service call, etc. it is not uncommon to reach that.
The point however, is that the architecture breaks down because
it is unable to do work concurrently. Every requests blocks a
worker from start to finish.
Unless it is CPU heavy the system will be under utilized. That is
not necessarily bad though. The simplicity has something going
for it, but it is definitely a tradeoff that you should consider
highlighting.
```
@endpoint void my_end(Request r, Output o)
{
if (r.uri == "/asd") // or whatever you want: regex,
or checking another field
return false; //
}
```
This is just like:
```
@matchuda(uri, "/asd") void my_end() { ... }
```
What's the difference? The first one is much more flexible,
IMHO.
The difference is that with the route uda you can *only* map
routes 1:1 exhaustively. With your approach it is up to the
programmer to avoid errors. It is also hard to reason about the
flow of requests through all those functions, and you have to
look at the body of them to determine what will happen.
Re: Release: serverino - please destroy it.
On Sunday, 8 May 2022 at 21:32:42 UTC, Andrea Fontana wrote: Every request is processed by a worker running in an isolated process, no fibers/threads, sorry (or thanks?) I did some tests and the performance sounds good: on a local machine it can handle more than 100_000 reqs/sec for a simple page containing just "hello world!".Of course that's not a good benchmark, if you can help me with other benchmarks it would be much appreciated (a big thanks to Tomáš Chaloupka who did some tests!) Typically server applications are IO heavy. I expect your isolated-process approach to break down with that kind of work. As an example, how many requests per second can you manage if all requests have to wait 100 msecs? For non critical workload you will probably still hit good enough performance though. Instead of using a lot of different UDAs to set routing rules, you can simply write them in your endpoint's body and exit from it to pass to the next endpoint. My experience is that exhaustive matching is easier to reason about at larger scale. Please help me testing it, I'm looking forward to receiving your shiny new issues on github. I noticed it has zero unittests, that is probably a good place to start.
Re: Release D 2.099.1
On Thursday, 7 April 2022 at 21:32:34 UTC, Martin Nowak wrote: Glad to announce D 2.099.1, ♥ to the 12 contributors. http://dlang.org/download.html This point release fixes a few issues over 2.099.0, see the changelog for more details. http://dlang.org/changelog/2.099.1.html -Martin Getting signature errors. ``` $ curl -fsS https://dlang.org/install.sh | bash -s dmd Downloading and unpacking http://downloads.dlang.org/releases/2.x/2.099.1/dmd.2.099.1.linux.tar.xz 100.0% gpg: Signature made Thu Apr 7 16:35:02 2022 UTC gpg:using EDDSA key 27637885C3CF8350732A1CA5723DC8887F97C07F gpg: Can't check signature: No public key Invalid signature http://downloads.dlang.org/releases/2.x/2.099.1/dmd.2.099.1.linux.tar.xz.sig ```
Re: [Semi-OT] Cross-Platform GitHub Action 0.3.0 - NetBSD
On Tuesday, 16 November 2021 at 13:34:49 UTC, Jacob Carlborg wrote: # Cross-Platform GitHub Action 0.3.0 I would like to announce a new release of [Cross-Platform GitHub Action](https://github.com/marketplace/actions/cross-platform-action), [0.3.0](https://github.com/cross-platform-actions/action/releases/tag/v0.3.0). I was reading https://github.com/marketplace/actions/cross-platform-action#under-the-hood and have to say I am impressed. Looks like a very well done library.
Re: DMD Frontend working in WebAssembly
On Friday, 15 October 2021 at 11:52:24 UTC, hatf0 wrote: Very cool! This all would not be possible with your wasm forks -- they are the saving grace here. Nice that you got it running. GC also needs some investigation (or malloc), as I keep get spurious OOM errors. Could be because dmd is one hell of a memory hog, but who knows. So far I haven't hit those when running druntime/phobos tests, but they could be caused by pointers being hidden from GC because they sit on the wasm stack/registers. The only way to solve that would be to (somehow) have LDC put them on the shadow stack whenever they aren't. I know of some experiments where they did this with a boehm style GC on emscripten. Maybe I need to look more into how other languages (like go) handle this.
Re: DMD Frontend working in WebAssembly
On Thursday, 14 October 2021 at 22:58:37 UTC, hatf0 wrote: On Thursday, 14 October 2021 at 22:56:07 UTC, hatf0 wrote: Hi all, I've just managed to get the full DMD front-end to work in WebAssembly (with skoppe's druntime fork). Cool stuff. I recently picked up my wasm forks again and updated to wasi 12, ldc 1.25.0 and the accompanying druntime/phobos versions and got all tests to pass (those that I didn't version out). I am currently in the process of updating to latest ldc and latest druntime/phobos. Phobos has a bunch of merge conflicts, but it should be doable if I find a few hours. Then there is the process of upstreaming a 5000+ line patch and I probably need some help there. I reckon the WASI bindings are pretty standalone so those should be easy to add, but there is a sprawl of additions across almost all files due to not supporting threads, fibers and catching exceptions.
Re: Computing the next SemVer for dlang packages with dsemver
On Thursday, 22 October 2020 at 08:59:18 UTC, Robert burner Schadek wrote: I should stop ranting now. Not at all, I love it. Nice project.
Re: addle 0.1.0 - argument-dependent lookup for UFCS functions
On Sunday, 21 June 2020 at 00:06:12 UTC, Paul Backus wrote: Now available on Dub, by "popular demand"! Links: - Documentation: https://addle.dpldocs.info/addle.html - Dub: https://code.dlang.org/packages/addle - Github: https://github.com/pbackus/addle Cool. Thanks.
Re: tardy v0.0.1 - Runtime polymorphism without inheritance
On Wednesday, 17 June 2020 at 16:01:29 UTC, Paul Backus wrote: A while ago, I collaborated briefly with Adam Kowalski from the Dlang discord server on some code to emulate C++-style argument-dependent lookup in D. Using that code, your example above would be written: Geometry.create(r.extended).measure; Geometry.create(c.extended).measure; Here's a gist with the code, along with a small example: https://gist.github.com/pbackus/0a70419eb8bece52f3a08edfe7b6019b If anyone thinks it's worthwhile, I can toss this up on Dub. Personally, I've never had much use for it in my own projects. Yeah I can see a use for this. If have had to refactor things before because I was hitting this restriction.
Re: tardy v0.0.1 - Runtime polymorphism without inheritance
On Saturday, 13 June 2020 at 15:11:49 UTC, Atila Neves wrote: Tardy lets users have their cake and eat it too by not making them have to use classes for runtime polymorphism. This is one of those things that is so obvious in hindsight. Genius.
Re: Rationale for accepting DIP 1028 as is
On Thursday, 28 May 2020 at 16:01:35 UTC, Johannes Pfau wrote: Am Thu, 28 May 2020 12:28:16 + schrieb Sebastiaan Koppe: If it does come back to haunt him, he can always add a DIP to make extern(!D) @system by default. It won't invalidate any work. This would be another round of massively breaking user code. The breakage will be split in two rounds, but the amount of code needed to be modified would be the same as with the safe-by-default-except-for-extern-non-c. With the possibility of not needing the second round. I know, arguing for a lost cause...
Re: Rationale for accepting DIP 1028 as is
On Thursday, 28 May 2020 at 09:21:09 UTC, Jonathan M Davis wrote: He did unfortunately manage to convince Atila, so the DIP has been accepted, but based on the discussions, I think that you may be the only person I've seen say anything positive about the DIP treating extern(C) functions as @safe. - Jonathan M Davis I think Walter had to make a tough call with many tradeoffs. The defining feature of engineering I would say. Is he wrong? Maybe, I don't know. The obvious path is far from always being a winner. If it does come back to haunt him, he can always add a DIP to make extern(!D) @system by default. It won't invalidate any work.
Re: LDC 1.21.0
On Friday, 24 April 2020 at 06:13:09 UTC, Paulo Pinto wrote: Great work! What is the status of WebAssembly support beyond betterC? Almost there. I originally planned to complete it last February. It turned out to be a bit more work because I didn't consider I would need to port parts of phobos as well. I then set a new deadline in March. Covid-19 happened and my wife and I have been homeschooling our kids in-between work. I haven't been able to find a lot of time to work on this. The druntime and phobos test-runners both compile, and I am using whatever spare time I have to fix or disable any tests that fail. Once that is done, I will open up PR's to both upstream repos. I do expect some discussions on that front so it probably won't be a one-day deal. After that there will be a little celebration since I have already have a PR to update LDC's CI to create prebuild releases.
Re: lookuptable
On Thursday, 23 April 2020 at 04:29:12 UTC, Steven Schveighoffer wrote: Hm... thanks for the suggestion. I'm not sure if it fits here, as the point is to avoid runtime cost and GC allocation, not make lookups uber-fast. Granted, it was far-fetched. These are meant to be short-lived things. My main target was e.g. mysql-native has an "asAA" function which generates an AA with keys being the column names. This is kind of crappy, because if you do that for each row, then you are generating and throwing away a LOT of data. Yeah, that is quite crappy. I remember mapping each row to a struct and use member access, but then you have to sync the struct and the query yourself. It doesn't have to be short-lived though. Most apps process the same set of queries, so pre-creating all your prepared statements and mappings might make sense. I am not sure about mysql, but postgres gives you back the columns and types after preparing a statement, so you could do some runtime mapping once at startup (and at each reconnect).
Re: lookuptable
On Wednesday, 22 April 2020 at 21:21:31 UTC, Sebastiaan Koppe wrote: For the indexLookup you can also use https://github.com/skoppe/perfect-hash That is not quite true, only for strings.
Re: lookuptable
On Wednesday, 22 April 2020 at 20:27:12 UTC, Steven Schveighoffer wrote: Just wanted to throw this out there on a slow day. I wrote this little utility to generate static lookup tables, because using an AA is too expensive for something like e.g. looking up database row data by column name. It's very crude, but very short and sweet. Hopefully people find a use for it. Let me know if you have any requests or find any bugs on github. https://code.dlang.org/packages/lookuptable -Steve Nice. For the indexLookup you can also use https://github.com/skoppe/perfect-hash
Re: DLS deprecation
On Tuesday, 7 April 2020 at 19:12:49 UTC, Laurent Tréguier wrote: So today, I am deprecating DLS, along with its editor extensions. If anyone was using them, be advised that they will not have any update or support from now on. Webfreak is still working on code-d/serve-d from what I gather, so hopefully, the handful of people who could be using DLS on VSCode can use it instead. Aww, too bad. Thanks for your effort. I was wondering how much time maintaining took? And what where the biggest things to upkeep?
Re: code.dlang.org reliability update
On Thursday, 5 March 2020 at 12:41:41 UTC, Sönke Ludwig wrote: But making changes to the general architecture - be it in the form of adding load-balancing, or more deeply rooted, such as using GIT for storing and distributing the package index - is something that still is definitely desirable to push forward. A week ago I created a git dub package index at https://github.com/skoppe/dub-packages-index it gets updated every 15 minutes by scraping code.dlang.org I use it to feed my registry-mirror, which is now fully standalone A possible next step is creating a PackageSupplier class in dub that clones and uses this repo for resolution. If I was aware of Sönke's work I might have skipped it, but it was a good exercise regardless.
Re: dub 502 bad gateway
On Tuesday, 19 November 2019 at 13:27:31 UTC, Sebastiaan Koppe wrote: After having had another day where my CI can't build because of gateway errors, this morning I had enough. (I setup an uptime robot so we can see how bad it actually is [1], it checks every 5 min and I just started it.) [...] Of course, this is a bandaid solution. Probably dub can be made to do this as well. And likely should. If it were up to me I would go serverless. [1] https://stats.uptimerobot.com/6mQX4Crw2L [2] https://github.com/skoppe/dub-registry-mirror I have updated the script to cache all routes. This means the registry mirror at dub.bytecraft.nl will also work when the registry at code.dlang.org is completely down, **if** your dependencies have been cached previously. The cache works in the following manner: 1. if a cache entry exists for a given route and it is no longer than 5 minutes old: return that 2. if a cache entry is older than 5 minutes: fetch from upstream, cache it and return that 3. if a cache entry is older than 5 minutes and upstream is down: reset cache expiry and return from cache 4. if a cache entry doesn't exist: fetch from upstream, cache it and return that 5. if a cache entry doesn't exist and upstream is down: return error On top of that there is a 5 second timeout on fetches from upstream. Essentially a cached entry is kept up-to-date (5 minute lag) whenever upstream is alive, otherwise stale data is returned if available. Use it like so: `dub build --registry=https://dub.bytecraft.nl` Code at https://github.com/skoppe/dub-registry-mirror/blob/master/worker.js
Re: dud: A dub replacement
On Thursday, 23 January 2020 at 17:05:10 UTC, Robert Schadek
wrote:
DESTROY!
Tried:
---
unittest {
immutable SemVer v1 = SemVer(1,1,1);
immutable SemVer v2 = SemVer(2,2,2);
immutable SemVer v3 = SemVer(3,3,3);
immutable SemVer v4 = SemVer(4,4,4);
immutable VersionRange a = VersionRange(v1, Inclusive.yes, v2,
Inclusive.yes);
immutable VersionRange b = VersionRange(v3, Inclusive.yes, v4,
Inclusive.yes);
immutable VersionRange c = VersionRange(v2, Inclusive.yes, v3,
Inclusive.yes);
assert(intersectionOf(a, invert(a)).ranges.length == 0);
assert(intersectionOf(a, a) == a);
assert(intersectionOf(a, b) != a);
assert(intersectionOf(a, b) != b);
assert(intersectionOf(b, a) != a);
assert(intersectionOf(b, a) != b);
assert(intersectionOf(invert(a), invert(a)) == invert(a));
assert(invert(intersectionOf(a, invert(b))) ==
invert(intersectionOf(a,a)));
auto add(T, P)(T a, P b) {
return invert(intersectionOf(invert(a), invert(b)));
}
auto remove(T, P)(T a, P b) {
return intersectionOf(a, invert(b));
}
assert(add(a,b) == add(b,a));
assert(invert(invert(a)) == add(intersectionOf(a,c),
remove(a,c)));
assert(differenceOf(a, a).ranges.length == 0);
assert(invert(intersectionOf(invert(a), invert(a))) ==
invert(invert(a)));
assert(invert(invert(differenceOf(a, invert(a ==
invert(invert(a)));
assert(invert(invert(intersectionOf(a, c))) ==
invert(invert(remove(a, remove(a, c);
}
---
Everything green. Nice work.
Haven't tried Inclusive.no yet. I'll leave that to someone else.
Re: Visual D 0.51.0 - semantic engine based on dmd frontend
On Saturday, 18 January 2020 at 14:22:41 UTC, Rainer Schuetze wrote: This version features a first version of an intellisense engine that is based on the DMD frontend (as of version 2.090) for semantic analysis. It is still considered experimental and has to be enabled on the respective language options page. Cheers, Rainer Nice, that is awesome. Could the engine also be re-used for in e.g. the D lsp server?
Re: Proposal for porting D runtime to WebAssembly
On Saturday, 4 January 2020 at 16:28:24 UTC, kinke wrote: On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe wrote: You can track the work here: https://github.com/skoppe/druntime/tree/wasm I gave it a quick glance; looks pretty good, and like pretty much work. ;) - Thx. Great. Thanks for looking. The compiler should probably help a bit by firstly predefining a version `CRuntime_WASI` (either for all wasm targets, or for triples like wasm32-unknown-unknown-wasi) and secondly emitting TLS globals as regular globals for now, so that you don't have to add `__gshared` everywhere. Yes. I will probably manage to do the first, but for the second one I definitely need some pointers. - reals (probably are going to be unsupported) It's probably just a matter of checking which type clang uses for C `long double` when targeting wasm, and making LDC use the same type. Could be. I personally prefer to avoid them because wasm only supports f32/f64, which I guess means they will be emulated (I have no idea though, maybe some wasm hosts do the right thing). But some people might need them, so if fixing the ABI is not a big deal, we could include them. - wasi libc needs to be distributed (either in source and compiled into wasm druntime) or statically linked I'd prefer a static lib (and referencing that one via `-defaultlib=druntime-ldc,phobos2-ldc,wasi` in ldc2.conf's wasm section). Good. Building it via LDC CI for inclusion in (some?) prebuilt LDC packages is probably not that much of a hassle with a clang host compiler. I don't think so either. I have already got it building, so I just need to go over my notes. once ldc-build-druntime works If you need some CMake help (excluding C files etc.), https://github.com/ldc-developers/ldc/pull/2787 might have something useful. Thanks. (_start is the wasm's equivalent of _Dmain) Not really; _start (in libc) is used on Linux too, which sets up the C runtime, then calls C main, which calls druntime's _d_run_main which in turn calls _Dmain. Ahh, fumbling as I go along. Thanks for the correction.
Re: Proposal for porting D runtime to WebAssembly
On Sunday, 5 January 2020 at 08:24:21 UTC, Denis Feklushkin wrote: On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe wrote: - reals (probably are going to be unsupported) It seems to me for now they can be threated as double without any problems Yeah, that is what I have done so far.
Re: Proposal for porting D runtime to WebAssembly
On Saturday, 23 November 2019 at 10:29:24 UTC, Johan Engelen wrote: On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d I'm assuming you already started some work in this area? Where can we track it? Great initiative! Johan You can track the work here: https://github.com/skoppe/druntime/tree/wasm Almost all unittests pass. I am in the process of getting `ldc-build-druntime` to build it, as well as hooking into main(). I really wanted to make a pr, so that others can build it as well, but I am pressed for time due to family weekend trip. It is on my list once I get back, as well as incorpareting all info from this thread back into the proposal. Some things to tackle before going beta: - AA unittests fail - reals (probably are going to be unsupported) - wasi libc needs to be distributed (either in source and compiled into wasm druntime) or statically linked - CI (but should be doable once ldc-build-druntime works) - hooking into main() (I thought about making a @weak _start() in druntime so that users can still override it when they want) (_start is the wasm's equivalent of _Dmain) - probably need help from LDC to spill i32 pointer on the shadow stack
Re: code.dlang.org downtime
On Wednesday, 18 December 2019 at 09:29:50 UTC, John Colvin wrote: This is still down for me, regardless of using the IP or address. I don't think it's just me either: https://stats.uptimerobot.com/6mQX4Crw2L/783838659 Anytime you see the metadata working you can add `--registry=https://dub.bytecraft.nl` to dub. I am really tempted to cache the metadata as well. Although that brings up the question of how to purge it when new versions are released. (I could setup a job to import the dump every now and then, hmm). This stuff just begs to be fixed.
Re: code.dlang.org downtime
On Tuesday, 17 December 2019 at 01:34:16 UTC, bachmeier wrote: Oh, I don't doubt that. My point was that it makes the D language project look like a small-scale open source project relying on volunteers (in this case Sonke) being generous with time and resources. What manager is going to trust a project like that with a major project? It seems unlikely that this would not be considered a major issue. I totally agree. I am very grateful for Sönke for everything he has and continues to do, but I believe we should move away from self hosting the dub registry. Not only will that increase the uptime, but it will also increase the bus factor and take some work off Sönke's hands into the d organisation. In another post I outlined a very crude draft proposal https://forum.dlang.org/post/arbvxihnxnboqcifh...@forum.dlang.org Since no one replied I assume everyone is onboard. (just kidding) The architecture isn't difficult, but the migration path is. In February I should have some time and I am willing to setup a proof of concept.
Re: Proposal for porting D runtime to WebAssembly
On Tuesday, 26 November 2019 at 09:18:05 UTC, Thomas Brix wrote: On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d An alternative idea, would be to use emscriptens fork of musl to have a full C-library. AFAIK this includes threading. LLVM is supposed to support TLS in wasm since version 9. Yes, indeed. https://reviews.llvm.org/D64537 gives a good overview. I believe it is best to first actually have a version of druntime on wasm, rather than eagerly pulling in all the latest features. I find the scope I set in the proposal to be quite reasonable. Adding tls, threading and exception handling would be much easier after this work is done and merged. And it would also be something others might want to contribute to.
Re: Proposal for porting D runtime to WebAssembly
On Monday, 25 November 2019 at 13:50:20 UTC, Georgi D wrote: Hi Sebastiaan, If you are looking at the C++ coroutines I would recommend looking into the proposal for "First-class symmetric coroutines in C++". The official paper can be found here: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1430r1.pdf There is also a presentation with some nice animations explaining the proposal here: https://docs.google.com/presentation/d/1B5My9nh-P2HLGI8Dtfm6Q7ZfHD9hJ27kJUgnfA2syv4/edit?usp=sharing There paper is still in early development, for example the syntax has changed since then as well as some other pieces. If you are interested I can connect you with the author of the paper who can explain it with more details. Georgi Thanks for that. It would be great, but I don't have time for that at the moment.
Re: Proposal for porting D runtime to WebAssembly
On Monday, 25 November 2019 at 18:44:01 UTC, thedeemon wrote: On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d Please correct me where I'm wrong, but on the level of WebAssembly there are no registers, there is an operand stack outside the address space, there are local variables to the current function, again outside the accessible address space of program's linear memory, and there is the linear memory itself. So scanning the stack becomes a really hard (should I say impossible?) part. What some compilers do is they organize another stack manually in the linear memory and store the values that would otherwise be on the normal stack, there. Yeah, that accurately describes the situation. I will update the wording in the document to use 'stack', 'shadow stack' (also sometimes called 'user stack') and the local variable. Thanks. One solution that I employed in spasm's experimental gc is to only run it directly from javascript. This way there can't be anything hiding in the stack or in a local variable. Although that approach doesn't work for all use cases. Which means in case of D you'll have to seriously change the codegen, to change how local variables are stored, and to use a kind of shadow stack for temporaries in expressions that may be pointers. Do you really have a plan about it? Well, no, not fully. That is why I said 'unknown'. But there must be a solution somewhere. LLVM already puts pointers to stack or local variables in the shadow stack. As well as for structs-by-val that don't fit the stack. We could adjust LDC to nudge LLVM to maintain live roots on the shadow stack as well. Go's approach is to put everything on the shadow stack. (see: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4/preview#heading=h.mjo1bish3xni) There is also the possibility of a code transformation. Binaryen has a spill-the-pointer pass that effectively gets you go's solution (but only for i32's) (see: https://github.com/WebAssembly/binaryen/blob/master/src/passes/pass.cpp#L310) I am favoring the first option, but I don't know how hard that would be. Will update the document with this info. Thank you for questioning this.
Re: Proposal for porting D runtime to WebAssembly
On Monday, 25 November 2019 at 13:28:17 UTC, Ola Fosheim Grøstad wrote: On Monday, 25 November 2019 at 12:52:46 UTC, Sebastiaan Koppe wrote: As an example, it is just a matter of time before a PaaS provider fully embraces wasm. This sounds interesting, I've been pondering about serverless FaaS (function as a service), where you basically (hopefully) get functions triggered by NoSQL database updates and not have to bother with your own webserver. This is already doable with dynamodb, or kinesis streams. Or google's dataflow. Using wasm just makes that more seamless (and faster). I see that CloudFlare has support for webassembly in their workers, but for Google Functions I only see Node10, but maybe they can run webassembly as well? I haven't found anything definitive on it though... Node has good wasm support, I don't know how you would get the wasm binary in, but it probably can be done. Instead of having docker containers you just compile to wasm, which will be pretty small and can boot in (sub) milli-seconds (plus they don't necessarily need a linux host kernel running and can run it closer to the hypervisor.) Yes, but the biggest potential I see is when you don't have to set up servers to process data. I rather not setup servers for anything. Just throw the data into the distributed database, which triggers a Function that updates other parts of the database and then triggers another function that push the resulting PDF (or whatever) to a service that serves the files directly (i.e. cached close to the user like CloudFlare). You don't have to wait for that. That future is already here. The in and output could also be distributed storage, event streams or some queue. The problem, however, is often when using those tools you get pushed into a small set of supported programming languages. Like AWS' glue that focuses on Scala or Python, or google's functions that only support js/python and go. Understandable, but I rather choose my own language. Wasm makes that possible.
Re: dud: A dub replacement
On Monday, 25 November 2019 at 12:15:42 UTC, Joseph Rushton Wakeling wrote: What's currently broken or impossible in DUB? What parts of that can be fixed without changing the config or CLI? And what improvements are most efficiently made via breaking changes? Please, let's bring our focus on that. The biggest thing for me would be incremental compilation. As well as a dub build and test 'watch' mode to avoid scanning the dependencies every time.
Re: Proposal for porting D runtime to WebAssembly
On Monday, 25 November 2019 at 12:19:30 UTC, Joseph Rushton Wakeling wrote: On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d Thanks for putting this together, it looks very carefully thought out. Thanks! Exceptions can be thrown but not catched. A thrown exception will terminate the program. Exceptions are still in the proposal phase. When the proposal is accepted exceptions can be fully supported. This would suggest that there may be some benefit in D providing improved support for return-type-based error propagation (as with `Result` from Rust), no ... ? Yes, definitely. But what do you mean with improved support? Like better pattern matching over either types?
Re: Proposal for porting D runtime to WebAssembly
On Monday, 25 November 2019 at 09:01:15 UTC, Dukc wrote: On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d This proposal is so perfectly balanced between value and implementability that I can find nothing to add or remove. Thanks! I'm interested, what's your motivation in doing all this? If I understood correctly, your primary motivation to write Spasm was to write better optimized front-end programs than you get with JS frameworks. That is a fair question. Spasm has been very successful if you look at rendering speed. It (almost) beats everything else out there [1]. Well, that is not surprising since everything is known at compile time; it literally compiles down to the same code as if you issued low-level dom calls manually. I am very happy about that. With regards to developer experience it is behind. First of all you have to deal with betterC. This alone is already a hurdle for many. Second is the DSL, or lack of it. It doesn't come close to something like e.g. SwiftUI. In fact, I wrote a (unfinished) material-ui component library on top of spasm and I was struggling at times. So it became clear to me I need to have druntime available. It will allow people to use the (almost) complete set of D features and it opens up some metaprogramming avenues that are closed off right now. With that I will be able to create some nice DSL, in line with JSX/SwiftUI or . There are plenty of opportunities here. It is not unfeasible to connect spasm to Qt, or dlangui, and create a cross-platform UI library, something like flutter. On the other hand, I am very excited about WebAssembly in general. It is certainly at the beginning of the hype curve and I suspect some very exciting things will appear in the future. Some of them are already here right now. For instance, you can target ARM by compiling D code to wasm and then use wasmer to compile it to ARM. With D connecting itself to the wasm world it exposes itself to a lot of cool things, which we mostly get for free. As an example, it is just a matter of time before a PaaS provider fully embraces wasm. Instead of having docker containers you just compile to wasm, which will be pretty small and can boot in (sub) milli-seconds (plus they don't necessarily need a linux host kernel running and can run it closer to the hypervisor.) There are tons of possibilities here, and I want D to be a viable option when that day comes. So it is not just about frontends anymore. But wouldn't it be easier to just use Rust since it has already implemented all this? All the rust frameworks for web apps that I have seen rely on runtime techniques like the virtual dom. As a consequence they spend more cpu time and result in bigger files. That may be perfectly fine for most (and it probably is), but I wanted to squeeze it as much as I could. Maybe it is possible to do that in rust as well, I don't know. D's metaprogramming seemed a more natural fit. [1] except Svelte, which is a little bit smaller in code size, and a tiny bit faster. But they build a whole compiler just for that. Lets wait for host bindings support in wasm and measure again.
Re: Proposal for porting D runtime to WebAssembly
On Sunday, 24 November 2019 at 18:46:04 UTC, Jacob Carlborg wrote: On 2019-11-23 10:51, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d What will happen to code that uses TLS? Will it be promoted to a global variable or will it fail to compile? LLVM errors out saying it can't select tls for wasm. We could modify ldc to not emit TLS instructions under WebAssembly. But yeah, right now, you need to __gshared everything. I know.
Re: Proposal for porting D runtime to WebAssembly
On Saturday, 23 November 2019 at 15:23:41 UTC, Alexandru Ermicioi wrote: On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d I was wondering whats your position on Fibers? I am not going to support them in this initial port. And to be honest I rather see us moving towards stackless coroutines. Can they be implemented in current WebAssembly? I haven't looked into it. I suppose they could be, since go has their goroutines supported in wasm as well. But I don't think it is easy. WebAssembly uses the Harvard architecture, which means code and data is separate and code isn't addressable. That is why wasm uses a function table and indexes instead of function pointer addresses. So things like moving the instruction pointer are out. If so I'd guess they would be a nice match for async related functionality javascript is known for. You can still use the JavaScript eventloop, either browser or node.
Re: Proposal for porting D runtime to WebAssembly
On Saturday, 23 November 2019 at 12:40:20 UTC, Ola Fosheim Gr wrote: On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe wrote: This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d Yes, if I read this right the plan is to keep the runtime small. That is good, small footprint is important. Small footprint is super important, especially when targeting the browser. The first stage is getting something to work though, but I will definitely chisel bytes off afterwards. Also, if applicable, structure the object file in way that compress well (gzip). E.g. the layout of compiler emitted data structures and constants on the heap. I don't know how much control we have (or want) over this. In the end LLVM and wasm-lld do that and we just piggyback that.
Re: Proposal for porting D runtime to WebAssembly
On Saturday, 23 November 2019 at 10:29:24 UTC, Johan Engelen wrote: Perhaps you can explicitly clarify that "port" in this context means that you will add the required version(WebAssembly) blocks in the official druntime, rather than in a fork of druntime. Indeed. It will not be a fork, but the changes will be upstreamed into the official druntime. (WebAssembly predefined version now explicitly mentions that it is for 32bit. Do you want to broaden this to 64bit aswell, or add a new version identifier?) I haven't seen anybody working on wasm64. I know it exists, but that is about it. I do not know what the future of wasm64 will hold. Probably there will come a time somebody needs it, but as of yet everybody focuses on wasm32, and I don't see that changing anytime soon. Still, I think it is a good idea to be prepared. Personally I would add wasm32 and wasm64 and also define WebAssembly whenever one of them is. Don't know if that is the smart thing to do. I read that Clang uses a triple with explicit mention of WASI: --target wasm32-wasi Are you planning for the same with LDC? Will you need a new predefined version identifier for WASI-libc? Perhaps group all required compiler features in a section (and move the `real` story there). Rust uses that as well. It would make sense for us to use that as well. Good idea. The ultimate goal is to not use libc, but directly call the wasi api. In the mean, yes, we should introduce the WASI-libc version. I have now put all that under the WebAssembly version, but that is conflating things. (although it is not a big deal, since the linker will strip them out if unused.) Will add to a separate compiler section in the gist. Can you elaborate on how you envision CI testing? We can use any of the WASI runtimes. I personally use Wasmer (written in rust, uses cranelift which is also used in Firefox). Another option (or in parallel) would be using the V8 in either node or an headless browser (although that would be better suited for testing JavaScript interoperability). I would go with wasmer first. Do you want to add that to LDC testing? (this may also mean that you first add a new change to LDC's druntime, confirming functionality with LDC CI, and then upstreaming the change) Yes, in fact, I am already targetting LDC's druntime. I'm assuming you already started some work in this area? Where can we track it? Will post the link here after some clean up. A few days. Great initiative! Johan Thanks, these are some very good points.
Proposal for porting D runtime to WebAssembly
This is my proposal for porting D runtime to WebAssembly. I would like to ask you to review it. You can find it here: https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
dub 502 bad gateway
After having had another day where my CI can't build because of gateway errors, this morning I had enough. (I setup an uptime robot so we can see how bad it actually is [1], it checks every 5 min and I just started it.) I know, I can setup my CI to cache dub packages. But that is just an optimisation. Things ought to just work. So this morning I started on a quick fix. I took a look at the source and concluded that for dub packages hosted on gitlab and github you can simply redirect to gitlab's and github's respective archives. That is what the dub registry does anyway. To that end I setup that logic in a cloudflare worker [2]. So for metadata it redirects to the api endpoint on code.dlang.org (which is pretty stable), and for to actual packages directly to gitlab or github. Use it like so: `dub build --registry=https://dub.bytecraft.nl` Of course, this is a bandaid solution. Probably dub can be made to do this as well. And likely should. If it were up to me I would go serverless. [1] https://stats.uptimerobot.com/6mQX4Crw2L [2] https://github.com/skoppe/dub-registry-mirror
Re: setup-dlang v0.4.0 released
On Monday, 18 November 2019 at 10:32:29 UTC, Mihails wrote: https://github.com/mihails-strasuns/setup-dlang/releases/tag/v0.4.0 Note: assuming no new issues will be found in a next few weeks, this will be eventually promoted to the 1.0.0 release. Fingers crossed. Thanks for this. It works perfectly.
Re: dud: A dub replacement
On Monday, 18 November 2019 at 09:53:56 UTC, Paolo Invernizzi wrote: A win-win move would be to have dud emit the other formats automatically as part of the compilation procedure, so to have always all of them present and synced on the content. I already regret starting about this. Instead of rooting for the SDL format, I should have just recommended to deprecate the json one.
Re: dud: A dub replacement
On Monday, 18 November 2019 at 08:57:58 UTC, Russel Winder wrote: Is SDL the right format? Cargo uses TOML to great effect. And TOML has, I suspect greater traction more widely than SDL. I personally prefer SDL when it comes to nested data, but yeah, that would work as well. The point I was making is to just have 1 format. With dud that should be possible.
Re: dud: A dub replacement
On Sunday, 17 November 2019 at 16:26:45 UTC, Denis Feklushkin wrote: On Thursday, 14 November 2019 at 23:33:06 UTC, Nick Sabalausky (Abscissa) wrote: Also, this, apparently, should lead to the fact that dud will have their own package description format. Almost inevitable. (This has already been discussed, perhaps.) SDL all the way please.
Re: dud: A dub replacement
On Monday, 11 November 2019 at 13:44:28 UTC, Robert Schadek wrote: So dub has some problems, and personally I find its code base very hard to get into. I understand what you mean. What I found most disruptive to my internal model of the code is the mutable state. It prevented me from a solid understanding of all the parts. The bigger goal, at least personally, is to have a code base of pure functions that is "trivial" to understand and debug. The rules of thumb is: "When your line gets longer than 80 characters, restructure your code!", "Branch statements are code smells." I certainly applaud your goal and I think many parts can be improved upon. However, not everything is accidental complexity; sometimes problems are hard and code is non trivial. Although in almost all cases it can be made nicer, it is just a matter of how much time you want to spend on it.
Re: sumtype 0.9.0: BetterC and DIP 1000 support
On Saturday, 28 September 2019 at 21:28:00 UTC, Paul Backus wrote: New since the last announced version, 0.8.3: - SumType is now fully compatible with DIP 1000 and BetterC! - Self-referential SumTypes can now be nested. Sweet! Keep up the good work.
Re: Demo: Calling Typescript library from D via WebAssembly
On Friday, 27 September 2019 at 10:07:08 UTC, Dennis wrote: On Thursday, 26 September 2019 at 09:25:28 UTC, Sebastiaan Koppe wrote: I would like to announce an example showcasing D interacting with a Typescript library via WebAssembly. I love how much work you put into WebAssembly support for D. Keep up the good work! Thanks, I hope to be able to spend more time on it in the future. There are still a lot of things that need to be improved. I have started work on a small project that generates these bindings automatically, leveraging the typescript compiler. I can imagine some basic types are easy (such as number -> double), but I wonder how you bind to functions that take sum types or 'any'. Are you going to use opaque JS objects on the D side or convert D structs/classes upon calling TS/JS functions? I am using opaque JS objects. Spasm already has Sumtypes and supports Any as well, so it is going to move forward from that. Having said that, the lineSeries data in the tradingview example is implemented with a proxy object that directly accessed memory from the wasm module. There are pros and cons to either approach. Also, I have to take into account the upcoming interface-types proposal https://github.com/WebAssembly/interface-types/blob/master/proposals/interface-types/Explainer.md
Demo: Calling Typescript library from D via WebAssembly
I would like to announce an example showcasing D interacting with a Typescript library via WebAssembly. https://github.com/skoppe/spasm-tradingview-example I have started work on a small project that generates these bindings automatically, leveraging the typescript compiler. This could also be used to interact with javascript libraries which have typescript bindings.
Re: SAOC - Looking for a mentor
On Friday, 9 August 2019 at 05:50:22 UTC, Ionuț Mihalache wrote: I have here [1] my project proposal, could you give me some opinions about it(if it's ok, what should be changed or added, maybe removed). Thanks, Ionuț I would first work out some real-world mixed-compilation scenario's that you would want to write support for. In the very least that will give you (and others) a clear and understandable goal. Right now I have no idea what you intend to support and what not. There have been countless post on the forum of compilation setups that dub doesn't support. I think the most common one (for me) is having a C/C++ library that first needs to be compiled into a static library before compiling the D code. A more advances use case for myself is a code generation scenario I often use. E.g. first compile one program (if modified), run it to generate code, then use that to compile another program. Here, either one could be in D or C/C++. (currently I have one in Typescript) With those scenario's you can structurize your proposal a bit better. For instance you can describe a common scenario (the problem), then describe what you want to support (scope it) and finally describe the method or way your are going to implement it (while also looking at more advanced scenario's that you don't support right away, but can be implemented in the future). Right now it is in a very draft form and goes too much into dub's details. Hope that helps.
Re: Spasm - webassembly libary for single page applications
On Tuesday, 6 August 2019 at 22:57:52 UTC, a11e99z wrote: tried. --combined => got error about time.d. could not fix, idk how to fix. If calling ldc2 manually you need to use the -betterC flag, see https://wiki.dlang.org/Generating_WebAssembly_with_LDC for the minimal working example. so, I think need to fix dub/sdl/json generating for wasm/spasm for Windows - -of=..WASM not EXE - libs as last with prefix -L="". probably all file names better put in quotes. probably with Unix-slashes I have opened a dub issue for the .lib .exe issue. https://github.com/dlang/dub/issues/1749
Re: Spasm - webassembly libary for single page applications
On Tuesday, 6 August 2019 at 19:02:09 UTC, a11e99z wrote: hi. can not compile for Windows LDC ver 1.16.0. Currently ldc 1.16.0 isn't supported. You can downgrade to ldc 1.15.0 spasm 0.1.13: target for configuration "library" is up to date. test_spasm ~master: building configuration "application"... Error: unrecognized file extension lib ?? ldc2 failed with exit code 1. Have you tried the github issues? I remember dukc having the same issue on windows as well. and I have some questions: - spasm contains some GC-allocator. can I use "hello" ~ 123.to!string or something? or I can use string literals only without some kind of string builder? No you cannot use the concat operator (~). Neither can you use new, class, AA or dynamic arrays among others. This is because Spasm uses betterC, and a lot of D features aren't available in betterC. I am working on a PR for druntime to at least compile to wasm without betterC. This will open the way to incrementally support more and more D features. In the meantime you need to get familiar with the betterC constraints. You can always look in the examples for workable code. Make sure to use the 0.1.13 tag, since master has some new unreleased stuff (which I hope to release in the coming month). There is a string builder in spasm as well as a betterC version of the phobos `text` function. The GC allocator is still unreleased and experimental.
Re: Redis client hunt-redis RC1 released
On Saturday, 27 July 2019 at 21:09:47 UTC, zoujiaqing wrote: I'm sorry. I'm durking to litght, I want to respect all software authors and keep copyright, but we don't know how to express it. In every source file you simply paste the original copyright at the top. And then maybe add "ported to D by ..."
Re: Ownership and Borrowing in D
On Tuesday, 23 July 2019 at 19:15:44 UTC, Olivier FAURE wrote: That's as good a time as any to plug my own proposal draft: https://gist.github.com/PoignardAzur/9896ddb17b9f6d6f3d0fa5e6fe1a7088 Any thoughts? So in a nutshell, a variable with the unique qualifier ensures that there are no other references to that data during the lifetime of said variable? In other words, you can only take 1 addressOf/ref at a time? Does it prevent more that just use-after-free? Random idea: what about an @owned destructor? One that only runs on the original object, not on any of its copies. Which implies scope of course, so that nothing of the copies or its internals are allowed to escape.
Re: Ownership and Borrowing in D
On Tuesday, 23 July 2019 at 04:02:50 UTC, Timon Gehr wrote: On 21.07.19 02:17, Walter Bright wrote: On 7/20/2019 3:39 PM, Sebastiaan Koppe wrote: Do you mean to keep track of ownership/borrowedness manually? No, that's what the copyctor/opAssign/dtor semantics so. This is not true. I thought as much. Thanks for the confirmation. I am considering moving to pointers to benefit from the future semantics. It's just that I don't like pointers that much...
Re: Ownership and Borrowing in D
On Saturday, 20 July 2019 at 21:56:50 UTC, Walter Bright wrote: On 7/20/2019 5:42 AM, Sebastiaan Koppe wrote: Anyway my question is whether the ownership and borrowing semantics ever gets applied to non-pointers? Currently, no. The programmer would be expected to include the correct copyctor/opAssign/destructors which would take care of it. Do you mean to keep track of ownership/borrowedness manually? Sure, that will work. But how would I then get the compiler to complain when a borrowed one outlives the owned one?
Re: Ownership and Borrowing in D
On Tuesday, 16 July 2019 at 06:12:42 UTC, Walter Bright wrote:
Now I just have to deliver the goods!
I am very excited about this, together with unique these are some
very desired semantics.
I have a practical question. In spasm I have to solve the
lifetime of javascript objects that are referenced from D. The
way it works is that objects in Javascript are put in a big array
and the index - the handle if you will - is passed to D. When D
code wants to do anything with the js object, it passes the
handle to a js glue function which looks up the object in the
table and does what it needs to.
The trick is finding an efficient mechanism to free the js
objects in the js array. At first I tried reference counting -
which worked, but introduced a lot of code bloat.
Now I have settled on having the wrapper struct (JsHandle) of the
handle be non-copyable, and pass it around by either moving or
using `ref scope` with dip1000 and @safe. Then if you need to
have the handle around for longer you can always wrap it in a RC
wrapper.
This works remarkably well. It gives cheap, efficient and safe
code and still allows multiple long-lived references with RC if
that is what is wanted.
There are 2 issues I currently have.
First some code:
---
alias Handle = uint;
struct JsHandle {
nothrow:
package Handle handle;
alias handle this;
~this() {
if (handle != invalid)
spasm_removeObject(handle);
}
@disable this(this);
}
struct HTMLElement {
spasm.bindings.dom.Element _parent;
alias _parent this;
this(JsHandle h) {
_parent = .Element(h);
}
// ... functions
}
struct HTMLCanvasElement {
spasm.bindings.html.HTMLElement _parent;
alias _parent this;
this(JsHandle h) {
_parent = .HTMLElement(h);
}
// ... functions
}
---
1) Sometimes the web api's accept an Nullable/Optional, which
means I have to wrap, for instance, the HTMLElement into an
Optional type and pass that. Of course, I want to maintain the
unique reference constraint and enforce that the Optional won't
outlive the original HTMLElement. Which I can't, since they are
all non-pointers and I can't store a scope ref inside a Optional.
The solution I have now is to create a Nullable!(T*). This does
solve the unique semantics, at the expense of having an api that
accepts both Nullable!(T) and Nullable!(T*). Pretty ugly.
2) the other issue is that I sometimes want to "upcast" a
HTMLElement to a HTMLCanvasElement. I can simply do this by
moving the handle around, but I want the HTMLCanvasElement not to
outlive the HTMLElement. Again, the solution is to involve
pointers. Since all wrapper structs have the same memory layout
(only a uint), I used this hack:
---
template as(Target) {
auto as(Source)(scope return ref Source s) {
return cast(Target*)
}
}
---
Anyway my question is whether the ownership and borrowing
semantics ever gets applied to non-pointers?
Re: S2Geometry Library Release v0.2
On Saturday, 25 May 2019 at 11:48:12 UTC, Vijay Nayar wrote: D's philosophy of having a large tool-box makes this work doable by a single person while other languages have spent many years with many contributors. Great work! Can you tell something about how you converted the code and how you plan to stay up-to-date?
Re: Phobos now compiling with -dip1000
On Saturday, 23 March 2019 at 03:06:37 UTC, Walter Bright wrote: Many thanks to Sebastian Wilzbach, Nicholas Wilson, Mike Franklin, and others! It's been a long and often frustrating endeavor, but we made it and I'm very pleased with the results. This is really nice. I congratulate your efforts. Safety for the win!
Re: DConf 2019 Schedule
On Sunday, 17 March 2019 at 22:43:28 UTC, Mike Parker wrote: We're extending the Early-Bird Discount until March 24, so if you haven't registered yet, you still have a chance to save. No definitive word yet on whether we'll be offering a 201 attendee discount, but I should know something this week. I was actually waiting for the early bird discount to end, but I guess I will just register now.
Re: LDC 1.15.0-beta1
On Saturday, 9 March 2019 at 19:57:36 UTC, kinke wrote: Please help test, and thanks to all contributors! Thank you very much guys! I opened an issue https://github.com/ldc-developers/ldc/issues/3023 about exports in WebAssembly. ldc 1.14's default is to export pretty much everything, and ldc 1.15-beta1's default is to export only the bare minimum. Of course I want something in between :) See the issue for more information.
Re: Project Highlight: Spasm
On Friday, 1 March 2019 at 13:26:58 UTC, Olivier FAURE wrote: Would the author be interested in structural level-feedback? As in, not "I wish there was this feature", but "I think the way you're doing X and Y is wrong, and the project would probably benefit from a complete refactoring". I realize this kind of feedback is pretty irritating to get and hard to act on several months into the project, hence why I'm asking. Maybe. But in the beginning no one is willing to give it :) I am very much interested in qualitative feedback on spasm's internals and its design. It is the main reason I announced the project. I am completely willing to overhaul each and every line if that benefits the project. The short version is, it's pretty clear Sebastiaan has designed spasm with the goal of giving the library compile-time information on the structure of the widgets to render, to avoid React's superfluous updates and prop comparison; that said, I think it's possible to give the library that information without losing React's "your components are all functions, don't worry about how the data is updated" simplicity, which I think is an area where spasm comes up short. That would be awesome. I initially tried very hard to stick to React/JSX functional rendering. I could not find a way to make it a zero-cost abstraction, but maybe you have more success!
Re: Project Highlight: Spasm
On Friday, 1 March 2019 at 07:28:06 UTC, Radu wrote: Dip1000 would make reference counting safe. Memory management should be handled by reference counting. RC should delegate malloc, free to an allocator. I see. Yeah, RC could work. Sure. Hence my suggestion to implement a typed malloc. Probably an allocator and memcpy friends that works in betterc without any C dependencies would worth its own dub package :) :) dip1000 is usable right?
Re: Project Highlight: Spasm
On Thursday, 28 February 2019 at 12:47:32 UTC, Radu wrote: Re. the memory management section, I wonder if reference counting using dip1000 would work for memory management. At least partly. One memory issue spasm has is to release JS objects once D code is done with them. The approach I am likely to take is to simply disallow more than 1 reference to a JS object, and then when that reference is destroyed, to also destroy the JS object. Basically a non-copyable struct with a destructor. Over time this can be expanded with dip1000 to allow having more references to the same JS object as long as the compiler can prove it doesn't escape and has shorter lifetime. I don't see how dip1000 can help free memory though. You could also use the type system information for a D malloc implementation that uses free lists for known sizes. Something in line with https://github.com/JinShil/memcpyD I saw memcpyD before. In fact I used part of the implementation in a (uncompleted) wasm port of druntime. But again, I don't see how it would help with freeing unused objects. It is quite interesting to see that WebAssembly lines up pretty close to bare metal programming, maybe there are some synergies here that can be exploited and create the minimum runtimes for both. Definitely.
Re: Spasm 0.1.3 released - with bindings to web apis
On Wednesday, 30 January 2019 at 00:22:15 UTC, WebFreak001 wrote: the underrun example looks really cool! I'm on linux but I don't use docker Wait, you are on linux. Why doesn't your ldc have wasm target? How did you install it? I will try out making something with spasm soon! Cool.
Re: Spasm 0.1.3 released - with bindings to web apis
On Saturday, 26 January 2019 at 15:34:15 UTC, WebFreak001 wrote: amazing! I would really like to try it but it seem the precompiled LDC version doesn't support the wasm output and I have no idea what that wercker stuff is you mentioned or how to use the container you sent with compiling on my local filesystem and not inside a sandbox :/ Make sure you have installed docker on your machine (are you running windows?), go inside your workspace directory and run `docker run --rm -ti -v $PWD:/app --workdir /app dlang2/ldc-ubuntu:1.13.0 /bin/bash` This will start a docker container with ldc 1.13.0 installed on ubuntu. It will also mount the current directory under `/app`. Inside the container you can just run dub, etc. And on your host you can just edit the files. Still looks great having this, especially now with all these APIs. I would really like to try making WebGL run with this in the future Have you seen the underrun example?
Re: Spasm 0.1.3 released - with bindings to web apis
On Saturday, 26 January 2019 at 23:40:01 UTC, Mike Franklin wrote: This is really cool; nice work! Thanks. Are you aware of SignalR, and do you see something similar eventually being added to Spasm? I don't use .NET myself, but I once reverse engineered a signalR client for some data feed I wanted to listen to, so I am a little familiar with it. But why not use gRPC over websockets, or real-time graphQL? Spasm has bindings to the browsers websocket and I don't think its hard to connect that to a D client implementation of gRPC or graphQL. Also, what are the limitations that prevent the D runtime (just druntime, not Phobos) from being ported to WASM? The real blocker was the GC. In wasm the address space starts at 0, so almost everything looks like a pointer :) But now that we have a precise GC, that should be solved. Maybe some issues with threaded since Webassembly is currently single-threaded (although multithreading is on the roadmap.) So yeah, there aren't many blockers. I did actually try it a couple of months ago, and got pretty far. But I wanted to keep things lean and mean and decided to constrain it to betterC. I am first going to see how well I can implement some material-ui components. That is going to be the litmus test for the SPA framework part of spasm. Again, nice work! Thanks. A lot of praise goes to the LDC team and the fact they build it on top of LLVM.
Spasm 0.1.3 released - with bindings to web apis
Spasm is a betterC library for web development that uses LDC to
compile to WebAssembly, and I just released a major update.
It now has bindings to most web api's, like the dom, fetch,
audio, webgl, etc.
So you can do things like this:
---
import spasm.bindings;
import spasm.dom;
import spasm.types;
extern (C) export void _start()
{
auto elem = document.createElement("div").as!HTMLElement;
elem.style.backgroundColor = "green";
elem.innerHTML = "BLA BLA!";
elem.addEventListener("mouseover",(event){
console.log("onmouseover");
console.log(event);
console.log(event.as!MouseEvent.clientX);
});
auto root = document.querySelector("body").front;
root.appendChild(elem);
}
---
And have it Just Work.
See the repo for more info: https://github.com/skoppe/spasm
It is still a WIP but I am getting there.
Re: Top Five World’s Most Underrated Programming Languages
On Wednesday, 23 January 2019 at 09:14:18 UTC, Bienlein wrote: On Friday, 18 January 2019 at 08:55:23 UTC, Paulo Pinto wrote: D really needs its killer use case if it is to move away from that list. D is a lot like Scala on the JVM: Both language have myriads of language features and bells and whistles, but there is no killer feature in the language itself. For me the killer feature is meta-programming. No other language comes close. Take the wasm-reader [1] library I recently wrote for instance. While not feature complete - nor a fair direct comparison - it is far less code than anything in C++ [2] or Rust [3]. [1] https://github.com/skoppe/wasm-reader/blob/master/source/wasm_reader/reader.d [2] https://github.com/WebAssembly/binaryen/blob/master/src/wasm/wasm-binary.cpp [3] https://github.com/rustwasm/twiggy/blob/master/parser/wasm_parse/mod.rs
Re: DConf 2019: Shepherd's Pie Edition
On Friday, 28 December 2018 at 16:31:01 UTC, Adam D. Ruppe wrote: For last year's meeting, my manager (the team I'm on has done our meetings differently for a while) convinced the CEO to try a more interactive approach for the org-wide meeting too. We did that speaker intro, small random group work, whole group conclusion pattern. It was a success. Everyone was more engaged, we had more cross-team collaboration (which has continued throughout the year as people are more comfortable with each other!), and people have shown better retention of the material. Staff surveys about subjective feelings about this meeting were up, too, people said it is more enjoyable. Nice! I have seen this work as well and it is amazing to see it when it happens. Great that your team could be that catalyst. In my experience it works great in trainings and workshops, but it might as well be very enjoyable and productive on dconf. Although I would keep at least 60% talks.
Re: Fuzzed - a program to find DMDFE parser crash
On Saturday, 15 December 2018 at 15:37:19 UTC, Basile B. wrote: I think this is what Walter calls "AST poisoning" (never understood how it worked before today). And the whole parser is like this. This poisoning kills the interest of using a fuzzer. 99% of the crashes will be in hdrgen. As is common with fuzzing, you'll need to ensure the program crashes. Sometimes that requires some tweaking. Regardless, you still have the input to investigate.
Re: Fuzzed - a program to find DMDFE parser crash
On Saturday, 15 December 2018 at 11:29:45 UTC, Basile B. wrote: Fuzzed [1] is a simple fuzzer for the D programming language. It allows to detect sequences of tokens that crash the parser. While the D front end is not yet used to make tools, if this ever happens the parser will have to accept invalid code. As experienced with dparse, invalid code tend to crash more a parser because of a cognitive bias that lead us, "hoomans", to prove that things work rather than the opposite. Nice. In my experience fuzzing parses works very well. I have good memories with afl. So much so that I once wrote a wrapper around it to handle running it distributed. See https://github.com/skoppe/afl-dist Could use a readme and a how-to though.
Re: DLP identify leaf functions
On Friday, 30 November 2018 at 20:10:05 UTC, Jacob Carlborg wrote: I would like to announce a new project I've started, called DLP (D Language Processing). Currently it's quite experimental but the idea is that it would contain a collection of commands for inspecting D code in various ways. It uses the DMD frontend as a library (the Dub package) to process D code. https://github.com/jacob-carlborg/dlp Really nice! I have some ideas about data-flow analysis and this allows some easy experimenting without forking the compiler.
Re: Ported js13k game underrun to D targeting webassembly
On Monday, 19 November 2018 at 11:13:52 UTC, Dukc wrote: I had a look at your code, and just now I realized that spasm can really call JavaScript without any glue from JavaScript side. I am not sure where you got that impression from, but I am afraid you'll still need js glue code. This is huge! Now I can start to think about expanding my usage of D at the web page I'm developing, potentially replacing parts of my application made with Bridge.NET. The only thing I did with spasm was to use a subset of the D language to define a declarative language for creating html components, and I used lots of introspection to optimise the rendering of them. Everything else comes from the wasm backend from llvm and it's support in ldc. I really appreciate you getting excited about it but look, Spasm is still in an early stage. The code is rough, the documentation poor. And it has plenty missing that needs to be fixed before you should consider writing anything serious with it. The two most important being bindings to all web apis and memory deallocation. Having said that, of course I want you to use it :) I know you did announce about Spasm earlier and I should have thanked you already. My fault: I realized that you made something that enables one to generate WebAsm, but didn't realize it enabled calling JS classes directly. Should have looked closer. The way calling js classes from D (or any language out there) works, is to insert the js object in a big associative array at the js side with an int as the key. The int (handle) is passed to D. Anytime D wants to call a function or retrieve a property of that class it calls some glue code with the handle as first parameter. In the javascript glue code the handle is used to lookup the object and the corresponding function is called. Since javascript allows you to call by string (e.g. `object["foobar"]()`), you can minimize glue code at the expense of decoding utf-8 strings. 1. Manually compile your code and parts of Druntime/Phobos targetting 32-bit X86, link them manually with llvm-link using --only-needed and only then compile to JavaScript (Haven't tried with WebAsm, but theoretically should compile the same way). This way you don't need to manually stub what you don't use -well, when the linkers have same ideas about what you're calling, which's not always the case. That might work, but for the end-user it really needs to be as simple as `dub build`. 2: Use Vladimir's DScripten. I haven't tried, as I was limited by not being aware of anything like spasm to call Web API, but probably easier than my method above and brings larger portions of Phobos into user's reach. Also IIRC Vladimir managed to get allocators working. Yep. The prototype version of spasm ran on top of dscripten-tools. I decided to move to native llvm wasm compilation instead, since I don't particularly like the emscripten toolchain. Even with my relatively meager method, pipeline-style programming with std.algorithm and std.range over (often static) arrays is doable and practical in my experience. As long as std.algorithms/ranges don't allocate, they work.
Re: Ported js13k game underrun to D targeting webassembly
On Sunday, 18 November 2018 at 19:51:06 UTC, Steven Schveighoffer wrote: Really cool. I didn't play the original, but I found this reasonably straightforward. I love this this webassembly stuff, I think it's a fantastic demonstration, and test for the power of D. Thanks, I think so too. Although there are definitely some hurdles to overcome. A system to do closures by hand should be doable with templates I would think. Do you have a quick example? For example https://github.com/skoppe/spasm/blob/master/example/underrun/source/game/terminal.d#L259 The whole Handler struct is just to call `terminal_run_story` after `terminal_write_text` completes. One thing that bugs me is that anything that allocates on the GC - like the closures here - can't be hooked into to call a custom allocator. That includes things like std.array.Appender, AA's, etc.
Re: Ported js13k game underrun to D targeting webassembly
On Sunday, 18 November 2018 at 10:33:14 UTC, blahness wrote: Good stuff. Any reason why the music & some of the sound effects are missing? Yep, I have only ported the most essential sound effects. The music I skipped entirely. It wouldn't be much work since the audio api / buffers are already there. I just didn't want to spend more time on an example. The text when you bring a system back online is corrupted. I know, I have looked at it for quite a while and I just can't figure out why it is broken.
Ported js13k game underrun to D targeting webassembly
Underrun is small game build by Dominic Szablewski for the 2018 js13kGames competition. I decided to port it to D and to target webassembly. You can play the game here https://skoppe.github.io/spasm/examples/underrun/ It is part of my endeavour into the wonderful world of webassembly. I specifically choose this project so I could get a feel for how to call web api's other than the dom, i.e. WebGL and AudioContext. Porting went pretty smooth, except for the fact that I wanted it to be done in betterC, which turned out to cripple lots of things I am used to rely on. Things like: - having to write closures by hand - not being able to use stdx.allocator - almost no use of phobos (even at CTFE) D could definitely use some love in that area. I was specifically stumped that stdx.allocator doesn't work in betterC; if somewhere is a good place to use stdx.allocator you would expect it to be betterC, but alas. Had lots of fun though. Hope you like it.
Re: Spasm - webassembly libary for single page applications
On Wednesday, 17 October 2018 at 19:07:16 UTC, aberba wrote: A common use case for wasm is to port C++ native apps to web. e.g. is the recent autoCAD web app which does almost everything the desktop app can. That's the only reason to IMO do stuff in wasm. Games, productivity software, etc...performance. Spasm might just be perfect for that kind of stuff There are issues getting the current GC ported to webassembly, so it is hard to port D code to wasm. That is one of the reasons why spasm has taken the betterC approach. But remember, spasm is just a library to render and update html, and to respond to dom events. It won't help you in anyway to port something to wasm. If you really want to port existing D code to wasm you either need to rewrite that in betterC or port druntime, which includes writing a precise GC. Dscripten-tools is a move in that direction. The reason spasm exists is because I wanted to optimise web application's rendering code at compile-time, to reduce the runtime (setup) costs and to deliver high performant web applications. I first tried to do that by writing a javascript optimiser that can take React code as input and spit out highly optimised js code. I got pretty far with that but at one point I realised that to do it well I needed advanced things like data-flow analysis and abstract interpretation. So I decided to ditched that and just use D's static introspection and LLVM's wasm target. A couple of weeks after that spasm was born.
Re: Spasm - webassembly libary for single page applications
On Tuesday, 16 October 2018 at 03:23:21 UTC, Jesse Phillips wrote: It would be cool if D provided the easiest way to develop webasm first to see if it could claim that market. If you have some minutes to spare it would be great if you could try it out. It should only take 10min to render your first divs, otherwise something is wrong. The major hurdle with any wasm dom framework is that there are no standard components to build on (like dropdowns, drag-n-drop, input validations, notifications, etc.), nor any css frameworks (like material ui, bootstrap). So'll need to build everything from scratch, and no sane person is likely to do that. What might be an option is to try to integrate with other wasm libraries out there, so at least we can use their components. But everyone does his own thing, so integrating is going to be hard as well.
Re: Spasm - webassembly libary for single page applications
On Sunday, 14 October 2018 at 06:03:10 UTC, Bogdan wrote: Awesome work! I remember that, at some point the https://glimmerjs.com/ authors wanted to write their vm in rust for better performance. It looks like D is a new option for such projects. Bogdan Thanks, a lot of credits go to LDC for supporting the webassembly backend of LLVM. I thought about doing a VM as well, specifically because I was afraid of the performance penalty of switching a lot between js and wasm. The idea was to emit all dom operations into one bytebuffer (possibly at compile-time) and then instruct js to execute that. It turns out jumping between wasm and js isn't really a big deal (at least not anymore), so I ditched that idea to keep things simple. Plus, there is a good chance that in the near future wasm will be able to call the browsers' api directly.
Re: Spasm - webassembly libary for single page applications
On Saturday, 13 October 2018 at 04:34:28 UTC, Nick Sabalausky (Abscissa) wrote: What are the main current limitations? Mainly that it is written in betterC, which is quite the limitation if you are used to freely call anything from phobos, or expect the GC to clean up. Also I currently don't free memory.
Re: Spasm - webassembly libary for single page applications
On Saturday, 13 October 2018 at 04:34:28 UTC, Nick Sabalausky (Abscissa) wrote: Nifty, I'll have to look into this. Any idea what it would take to get this doing some WebGL? (Or playing audio?) Or is this more for HTML-ish sorts of stuff? This is more focused on HTML rendering, but you can do anything that you can do from JS, you just have to write JS glue code. WebGL is based on and follows the OpenGL ES (Embedded Systems) spec, which is a subset of OpenGL. So if you have the API definitions you are already half done. The other half is writing that in JS and calling the actual WebGL. If you want to do it right now you can use vladimir's dscripten-tools project, which is based on emscripten. I just don't like emscripten because it has a complex toolchain (D->LDC->(patched)LLVM->asm.js->binaryen->wasm) and results in bloated code.
Spasm - webassembly libary for single page applications
I like to announce Spasm https://github.com/skoppe/spasm It is a webassembly library to develop single page applications and builds on my previous work (https://forum.dlang.org/post/eqneqstmwfzugymfe...@forum.dlang.org). It generates fast and small webassembly binaries. The example todo-mvc is only 5995 bytes (wasm) + 2199 bytes (html+js) when gzipped, and loads pretty fast (https://skoppe.github.io/spasm/examples/todo-mvc/) Spasm can be used with dub and only requires a recent version of ldc. See the readme on github how to get started. The library includes a webpack bootstrap project to build a production ready web page. Spasm is written in betterC. Next steps are: - more js host bindings (xhr, localstorage, etc.) - memory deallocation - gather an angry mob to make phobos more betterC compatible ;)
Re: Webassembly TodoMVC
On Sunday, 23 September 2018 at 18:36:11 UTC, Joakim wrote: Vladimir mentioned that there's a Musl port to wasm, have you tried it? https://github.com/jfbastien/musl Druntime and ldc support Musl. Thanks for the link, I will have a look at it!
Re: Webassembly TodoMVC
On Sunday, 23 September 2018 at 17:53:32 UTC, Suliman wrote: What do you think of the struct approach compared to a traditional jsx/virtual-dom? jsx is sucks. Look at Vue.js way, if you will able to fo you framework Vue-style it will be perfect! The reason I made this wasm experiment was not to port some js-framework over to D, just to do the same thing but then in D. If I like some js framework out there, I would just use that one. My goal here is to bring something new to the table.
Re: Webassembly TodoMVC
On Saturday, 22 September 2018 at 14:54:29 UTC, aberba wrote: Can the SPA code be released as a separate module for WebAssembly web app development? Currently the whole thing is not so developer-friendly, it was just the easiest way for me to get it up and running. Right now I am trying to ditch emscripten in favor of ldc's webassembly target. This will make it possible to publish it as a dub package (ldc only), as well as reduce some of the bloat. The downside is that ditching emscripten means I have to implement things like malloc and free myself. There is some obvious overlap between this and recent efforts by others (I remember D memcpy, and people trying to run it without libc, etc.), so I expect a situation in the future where all these efforts might be combined. Regardless, I don't need much from the C library, just enough to make (de)allocations and parts of the D standard library work. TL;DR I intend to publish it on dub, but it does takes some more time. What do you think of the struct approach compared to a traditional jsx/virtual-dom?
Webassembly TodoMVC
Hey guys,
Following the D->emscripten->wasm toolchain from CyberShadow and
Ace17 I created a proof of concept framework for creating single
page webassembly applications using D's compile time features.
This is a proof of concept to find out what is possible.
At https://skoppe.github.io/d-wasm-todomvc-poc/ you can find a
working demo and the repo can be found at
https://github.com/skoppe/d-wasm-todomvc-poc
Here is an example from the readme showing how to use it.
---
struct Button {
mixin Node!"button";
@prop innerText = "Click me!";
}
struct App {
mixin Node!"div";
@child Button button;
}
mixin Spa!App;
---
Re: asdf json library moved to libmir
On Wednesday, 11 July 2018 at 21:42:45 UTC, yannick wrote: Hi All, Since i'll be leaving Tamedia (who sponsored development of Asdf) and it's not under active use anymore here we decided to donate it to libmir. Asdf has a few nifty features that make it perfect for reading large amounts of json lines and transforming data. For an example use case see je, the json to csv transformer app ( https://github.com/tamediadigital/je ) I still think that it is by far the easiest to use and fastest production grade Json Library for D. Maybe it should be renamed to reflect that, proposals welcome. cheers, y Great! Thanks for keeping it alive. I use the library specifically for reading json files stored on s3 that were generated by aws' kinesis service. (The files have multiple json roots in them, but aren't separated by newlines; just one after another. I created a small custom parseJsonByLine for that).
Re: flycheck-dmd-dub v0.12 - Emacs on-the-fly syntax checking for D
On Thursday, 8 February 2018 at 17:05:32 UTC, Atila Neves wrote: https://melpa.org/#/flycheck-dmd-dub flycheck already works with D, but the problem is setting the right compiler flags for your project in order to able to compile properly. flycheck-dmd-dub does this automatically for dub projects. This new release fixes bugs and speeds up opening files by using `--nodeps --skip-registry=all` if the dependent packages have already been downloaded. Atila Really nice. Big thanks!
Re: Trip notes from Israel
On Monday, 22 May 2017 at 15:05:24 UTC, Andrei Alexandrescu wrote: http://dlang.org/blog/2017/05/22/introspection-introspection-everywhere/ -- Andrei Wow, that was really good. Love to read more of your trip notes.
Re: LDC 1.1.0 released
Awesome!
Re: Autotesting dub packages with dmd nightly
On Wednesday, 10 August 2016 at 18:35:03 UTC, Sebastiaan Koppe wrote: So true. Then I will do PR's first. I finally got around implementing running dmd/druntime/phobos pull requests against all dub packages. Thank you digger, for making it so easy. I did a run of master + druntime#1602. Then I ran a comparison with a batch from 2.071.2-b2. 108 packages had a different build result. I have no nice stats or pictures, but a quick glance over the raw data: 50 of them went from green unittests to a dmd exit code 1. 16 went from unknown build results to dmd exit code 1. 10 went from dmd exit code 255 to 1 9 of them are now green. 8 of them went from linker errors to a dmd exit code 1. 6 of then went from non-zero exit code during unittest run to a dmd exit code 1. 3 previous ran out of memory but now resulted in dmd exit code 1. etc. All in all I think +/- 96 package are affected. A little over 11%.
Re: Autotesting dub packages with dmd nightly
On Wednesday, 10 August 2016 at 10:32:24 UTC, Martin Nowak wrote: We want better ranking of dub packages (mostly by download, but for sure also showing CI results [¹]). I was also thinking about integrating results from CI builds that packages do themselves. But there is some 'impedance mismatch': those CI build are done on the master branch, not on the latest release that is on code.dlang.org. Is it already usable? Short answer: No. I am currently test running it on all packages against the 10 latest dmd releases (I have done 6k packages on and off since 2 days ago). But I am running into vibe.d issues/missing features. Things like not being able to use gzip with requestHttp (let alone with a RestInterfaceClient), invalid internal state with the http client pool on interrupting requests, and some other things. Also, I am writing a PR for vibe.d to send http request to unix sockets. How to deploy then? For the worker it's just a docker container. But until the unix sockets PR is done you do have to setup the docker daemon to listen on the docker0 interface. I need to test https://github.com/dlang/druntime/pull/1602 and otherwise have to resetup my project tester for that. I am using digger to build dmd, so adding in the pull request is trivial. I do need to adjust internals to properly handle it though. But alas, family is coming over so don't expect anything anytime soon. Yes, but from past experience we know that people don't look at results, if you don't make it part of PR acceptance. So true. Then I will do PR's first.
Re: Autotesting dub packages with dmd nightly
On Sunday, 7 August 2016 at 23:08:34 UTC, Martin Nowak wrote: I actually don't think this makes sense. You're not in the position to maintain 1K+ packages, it's the library owners that need to test their code. Thanks for taking the time to respond. I agree with you. Library owners should test their code themselves. But they don't. 24% of the packages don't build. Just this short list I'm using for the project tester is hardly maintainable. I don't need to maintain anything besides linker errors. It is quite simple, I just run `dub test` and see what happens. If that doesn't work I consider it a failed build. https://github.com/MartinNowak/project_tester (uses Jenkins, no need to write yet another CI). I would argue mine is simpler to deploy and have nodes join. I've already thought about many different aspects of this and here are the 2 things that are useful and might work out. - Implement a tester that runs for every PR (just the other testers) and tests the most popular/important dub packages. Once a day is not enough b/c will feel responsible for breakages, we really need feedback before merging. It is just a matter of resources. I choose nightly since it seemed doable using just my own resources. - Show test results of various CIs on code.dlang.org. Testing a dub package on Travis-CI is already a no-brainer. For example the following .travis.yml would test a package against all dmd release channels. ```yaml language: d d: [dmd, dmd-beta, dmd-nightly] ``` Yes, that is quite nice. But that only gets triggered when the repo is updated. All in all I understand your reservations, and I highly appreciate your feedback. I understand I won't bring the end-all solution to testing, but I do hope to reach the goals that I have set forth for myself. 1) catching (some) regressions, 2) giving insights into bit rot on code.dlang.org, 3) have fun. It might take a couple of months before I reach them, or I might not at all.
Re: Autotesting dub packages with dmd nightly
On Saturday, 6 August 2016 at 20:00:53 UTC, Basile B. wrote: On Saturday, 6 August 2016 at 19:06:34 UTC, Sebastiaan Koppe wrote: - code.dlang.org has an api but doesn't provide an endpoint to retrieve all packages/version. Now I just scrape the site instead (thanks Adam for your dom implementation). No endpoint but still possible in two steps. Yeah, I considered something like that myself. I find scraping to be better for both sides though, and simpler. A good api would certainly be better. On Saturday, 6 August 2016 at 20:08:47 UTC, Seb wrote: Why don't you make a PR to the dub registry (https://github.com/dlang/dub-registry) to get such an endpoint? Or at least open an issue ;-) https://github.com/dlang/dub-registry/issues/171
