Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: Hi all, Recently there have been inquiries about support for D on Alpine Linux, a distribution mostly used in combination with Docker to create lightweight container images for microservices. At BPF Korea, we're working on a blockchain written in D, and wanted to be able to easily test and distribute our node using Alpine images, but there was no package for it yet. However, thanks to the work of many contributors before (Joakim Noah, yshui, Petar Kirov/zombinedev, and many others), most of the porting was already done and it was just a matter of fixing a few small issues and and creating the package definitions. A package for `dub` (v1.18.0), `dtools` (ddemangle & rdmd), and `ldc` (v1.18.0) are now available in the `testing` repository of Alpine Linux edge. As `testing` is not enabled by default, you will need to specify the repository (or add it to your `/etc/apk/repositories`) when installing the packages. For example: ``` apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing ldc ldc-static dtools-rdmd dub ``` this not install the lto static lib, how can i build lto static lib for alpine ?
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Wednesday, 15 January 2020 at 11:48:29 UTC, kinke wrote: On Wednesday, 15 January 2020 at 04:00:26 UTC, Mathias Lang wrote: The LDC package is not going to be cross-architecture in the near future, but it should be able to correctly cross-compile once LDC a version matching 2.090.1 is released (most likely LDC 1.20.0). What's the reason for that requirement? I'll soon release LDC v1.20.0-beta1, so is there something from DMD stable you'd need as prerequisite? While I've never fully understood the popularity of Alpine-based docker images (okay, the base image is tiny, but if all images are based on the same, say, Ubuntu image, the few hundred MB only need to be stored once on each host), it'd probably be nice to have the official prebuilt LDC Linux packages linked against musl, to get rid of the glibc dependency. I assume that would be enough to make it runnable on almost all Linux x64 hosts. Nothing from DMD, but some fixes in Druntime. The `stat` struct definition was broken (among other things), so anything involving files was failing on other architecture. This was fixed in https://github.com/dlang/druntime/pull/2899 . Regarding Alpine's popularity: I think it's a matter of convenience. Just like distributing a single binary (or, like in your case, a standalone package) is easier to deal with, being able to distribute a tiny image packaging your application is great (because the economy of scale argument only holds if you have many images on the same host and don't wipe it frequently). Also, the Ubuntu base image is much slower to build, and just I like my compilation time to be low, I like my image build time to be low. I think the README of alpine covers it pretty well: https://github.com/alpinelinux/docker-alpine#why .
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Wednesday, 15 January 2020 at 04:00:26 UTC, Mathias Lang wrote: On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: [...] Time for an update! The GDC PR have been merged, and followed by another large fix to make it work on most architectures. So if you use Alpine edge, you can just run `apk add gcc-gdc` and you'll get GDC-9.2.0 (with a lot of patches), which is a 2.076.0 frontend, but allow you to bootstrap efficiently. The package is present on ARMv7, AArch64, x86, x86_64, and s390x. Only ppc64le is not supported as GDC hasn't been ported to it (at least not as of the time of 9.2.0 release). [...] It would be nice to have PBF Korea listed on https://dlang.org/orgs-using-d.html (organizations using D)
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Wednesday, 15 January 2020 at 04:00:26 UTC, Mathias Lang wrote: The LDC package is not going to be cross-architecture in the near future, but it should be able to correctly cross-compile once LDC a version matching 2.090.1 is released (most likely LDC 1.20.0). What's the reason for that requirement? I'll soon release LDC v1.20.0-beta1, so is there something from DMD stable you'd need as prerequisite? While I've never fully understood the popularity of Alpine-based docker images (okay, the base image is tiny, but if all images are based on the same, say, Ubuntu image, the few hundred MB only need to be stored once on each host), it'd probably be nice to have the official prebuilt LDC Linux packages linked against musl, to get rid of the glibc dependency. I assume that would be enough to make it runnable on almost all Linux x64 hosts.
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Wednesday, 15 January 2020 at 11:46:21 UTC, aberba wrote: On Wednesday, 15 January 2020 at 04:00:26 UTC, Mathias Lang wrote: On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: [...] Time for an update! The GDC PR have been merged, and followed by another large fix to make it work on most architectures. So if you use Alpine edge, you can just run `apk add gcc-gdc` and you'll get GDC-9.2.0 (with a lot of patches), which is a 2.076.0 frontend, but allow you to bootstrap efficiently. The package is present on ARMv7, AArch64, x86, x86_64, and s390x. Only ppc64le is not supported as GDC hasn't been ported to it (at least not as of the time of 9.2.0 release). [...] It would be nice to have PBF Korea listed on https://dlang.org/orgs-using-d.html (organizations using D) Oops, BPF, my bad
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: What's next ? 1) There is a pending PR (https://github.com/alpinelinux/aports/pull/12006) to have GDC working on all architectures alpine supports, not just x86_64. 2) Adding a package for gdmd 3) Rebuild packages based on GDC, so that all architectures are supported. 4) Move the packages to community so they are available out of the box. It would be great for it to happen by the end of the month, as the next alpine release would be around end of December according to their schedule, but that depends on how long PR take to be reviewed. 5) A DMD package for x86 and x86_64 shouldn't be hard to make either Time for an update! The GDC PR have been merged, and followed by another large fix to make it work on most architectures. So if you use Alpine edge, you can just run `apk add gcc-gdc` and you'll get GDC-9.2.0 (with a lot of patches), which is a 2.076.0 frontend, but allow you to bootstrap efficiently. The package is present on ARMv7, AArch64, x86, x86_64, and s390x. Only ppc64le is not supported as GDC hasn't been ported to it (at least not as of the time of 9.2.0 release). Additionally, `gdmd` has been added to `testing` to make it easier to build packages. LDC has been updated to v1.19.0 and the package definition fixed (only `ldc` and `ldc-runtime` present now, no more `ldc-static`). Dub v1.19.0 is also available, also built with GDC, on all the architectures GDC supports. The LDC package is not going to be cross-architecture in the near future, but it should be able to correctly cross-compile once LDC a version matching 2.090.1 is released (most likely LDC 1.20.0). I am now working on finalizing the LDC package so all tests pass on x86 & x86_64, and to provide a DMD package. While there are still a few details to flesh out, most things should work perfectly on x86_64 (we've been using it for months), so feel free to test it and report back. I also hope we can use it in some of the official dlang repositories (e.g. dlang-tour and the dub testsuite). The best place to report issues with the packages is on the Alpine Gitlab ( https://gitlab.alpinelinux.org/alpine/aports/issues) and to tag @Geod24.
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Wednesday, 13 November 2019 at 12:27:52 UTC, user wrote: On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: ``` apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing ldc ldc-static dtools-rdmd dub ``` A hello world vibe project doesn't build for me using a Dockerfile using your template. I tried to add missing deps, but I couldn't really figure out what's missing. I get an error during the `dub build` step: ``` eventcore 0.8.48: building configuration "epoll"... /root/.dub/packages/eventcore-0.8.48/eventcore/source/eventcore/drivers/posix/processes.d(316,10): Error: module `core.sys.posix.sys.wait` import `idtype_t` not found` ``` My full Dockefile: ``` FROM alpine:edge as builder RUN apk --no-cache add build-base git RUN apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing ldc ldc-static dtools-rdmd dub RUN apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing libevent ibevent-dev WORKDIR /tmp/app ADD source ./source ADD dub.json ./ RUN dub build --compiler=ldc2 ``` Unfortunately recent versions of Vibe.d seems to be broken with Alpine. We use eventcore v0.8.43, Vibe.d v0.8.6, vibe-core v1.7.0. I'll see if I can get a CI in place for Vibe (and fix the issues). Thanks for reporting!
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: ``` apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing ldc ldc-static dtools-rdmd dub ``` A hello world vibe project doesn't build for me using a Dockerfile using your template. I tried to add missing deps, but I couldn't really figure out what's missing. I get an error during the `dub build` step: ``` eventcore 0.8.48: building configuration "epoll"... /root/.dub/packages/eventcore-0.8.48/eventcore/source/eventcore/drivers/posix/processes.d(316,10): Error: module `core.sys.posix.sys.wait` import `idtype_t` not found` ``` My full Dockefile: ``` FROM alpine:edge as builder RUN apk --no-cache add build-base git RUN apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing ldc ldc-static dtools-rdmd dub RUN apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing libevent ibevent-dev WORKDIR /tmp/app ADD source ./source ADD dub.json ./ RUN dub build --compiler=ldc2 ```
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On 2019-11-06 02:02, sarn wrote: And the neat way to do that is with a multi-stage build: one Dockerfile, with an Alpine container building the binary, then copying to a FROM scratch container I've used the "smith" tool as well [1]. It has some additonal help with dynamically linked code. It will use "ldd", recursively, to track dependencies and automatically add those. It will also do a couple of other things, like making the filesystem read only. The musl build is practically necessary because glibc has effectively given up standalone static binary support. So, thanks BPF Korea :) Yeah, it helps. Although it still possible to statically link with glibc, although you might get some warnings. [1] https://github.com/oracle/smith -- /Jacob Carlborg
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 12:20:04 UTC, Jacob Carlborg wrote: On Tuesday, 5 November 2019 at 11:49:20 UTC, Daniel Kozak wrote: Generally no, because Apline use musl libc instead of glibc, so there are some issues with that The correct way is to use static linking and putting only the binary in a Docker image, i.e. "from scratch" [1] ;). But using Alpine and musl will help with building the binary. [1] https://hub.docker.com/_/scratch And the neat way to do that is with a multi-stage build: one Dockerfile, with an Alpine container building the binary, then copying to a FROM scratch container: https://docs.docker.com/develop/develop-images/multistage-build/ The musl build is practically necessary because glibc has effectively given up standalone static binary support. So, thanks BPF Korea :)
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: At BPF Korea, we're working on a blockchain written in D Hello, Sorry if this has been said already: would you consider being listed in https://dlang.org/orgs-using-d.html? Thanks
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
Am 05.11.2019 um 10:48 schrieb Petar Kirov [ZombineDev]: On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: [..] That's great news! Thanks a lot for your hard work! # Note: This will redownload your dependencies every time, which doesn't play well with docker I have been meaning to add a docker and CI friendly command to dub that fetches all dependencies without building them. In the Node.js world they have the `npm ci` [0] command. [0]: https://docs.npmjs.com/cli/ci.html Sounds like "dub upgrade --missing-only" will do that, as it implicitly downloads all dependencies in addition to filling the (non-existent in this case) gaps.
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 11:49:20 UTC, Daniel Kozak wrote: Generally no, because Apline use musl libc instead of glibc, so there are some issues with that The correct way is to use static linking and putting only the binary in a Docker image, i.e. "from scratch" [1] ;). But using Alpine and musl will help with building the binary. [1] https://hub.docker.com/_/scratch
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tue, Nov 5, 2019 at 12:05 PM Dejan Lekic via Digitalmars-d-announce wrote: > > On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: > > Hi all, > > Recently there have been inquiries about support for D on > > Alpine Linux, a distribution mostly used in combination with > > Docker to create lightweight container images for microservices. > > > > At BPF Korea, we're working on a blockchain written in D, and > > wanted to be able to easily test and distribute our node using > > Alpine images, but there was no package for it yet. > > I am confused... Why do you need LDC, DMD, dub, etc on Alpine? > Can't you build it anywhere and just put the final artifact (or > set of artifacts) inside an Alpine-based container? Generally no, because Apline use musl libc instead of glibc, so there are some issues with that
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: Hi all, Recently there have been inquiries about support for D on Alpine Linux, a distribution mostly used in combination with Docker to create lightweight container images for microservices. At BPF Korea, we're working on a blockchain written in D, and wanted to be able to easily test and distribute our node using Alpine images, but there was no package for it yet. I am confused... Why do you need LDC, DMD, dub, etc on Alpine? Can't you build it anywhere and just put the final artifact (or set of artifacts) inside an Alpine-based container?
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: [..] That's great news! Thanks a lot for your hard work! # Note: This will redownload your dependencies every time, which doesn't play well with docker I have been meaning to add a docker and CI friendly command to dub that fetches all dependencies without building them. In the Node.js world they have the `npm ci` [0] command. [0]: https://docs.npmjs.com/cli/ci.html
Re: D for microservices: ldc, rdmd, dub now available on Alpine x86_64
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote: Hi all, Recently there have been inquiries about support for D on Alpine Linux, a distribution mostly used in combination with Docker to create lightweight container images for microservices. [...] This is great! Much thanks to all for all the work towards this!
Re: D for microservices
On 10/23/2017 06:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Mysql-native is vibe.d-compatible, but also works when you have zero vibe.d dependencies (in which case it switches to Phobos's sockets instead of using Vibe's sockets.)
Re: D for microservices
On 05/05/2018 02:30 PM, Nick Sabalausky (Abscissa) wrote: On 10/23/2017 06:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Mysql-native is vibe.d-compatible, but also works when you have zero vibe.d dependencies (in which case it switches to Phobos's sockets instead of using Vibe's sockets.) Sorry, didn't notice this was a half-year-old thread.
Re: D for microservices
On Sunday, 11 March 2018 at 11:10:09 UTC, Tamas wrote: On Thursday, 8 March 2018 at 22:53:11 UTC, aberba wrote: On Thursday, 8 March 2018 at 09:35:08 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB. Building directly in alpine will simplify automated builds too. Simple Vibe.d app talking to Redis, packed into docker containers: https://github.com/tam4s/hello-redis The takeaway is that I could not use Alpine as a host image, because I could not build the app statically on ubuntu. warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking I've put up a new binary release of ldc 1.9 for Alpine, which fixes the vibe.d issues by pulling in two upstream commits from druntime, adds druntime and Phobos as shared libraries, and includes dub and rdmd: https://github.com/ldc-developers/ldc/releases I tested by unpacking that release in an Alpine VPS, adding its bin/ to my path, and simply running the following: dub fetch vibe-d dub build vibe-d The list of Alpine packages needed to run ldc are listed in the release notes. You can also cross-compile using the regular linux build of ldc by using ldc-build-runtime, the included tool to rebuild the stdlib for other platforms, and these instructions from the wiki: https://wiki.dlang.org/Building_LDC_runtime_libraries#Usage_for_cross-compilation I'm looking at adding a flag to dmd to enable building for the Musl C runtime: https://github.com/dlang/dmd/pull/8020 While the Musl port is mostly there, it appears that yshui didn't bother porting all of druntime, so there may still be other dub packages that need other missing C declarations. However, I'm done working on this port, beyond finishing off the above pull and the druntime pull linked from the release notes, as I don't use Alpine, containers, or microservices. I simply chipped in because I have porting experience and thought I could push D for microservices a bit farther along. I'll keep putting out official builds of ldc for Alpine though, as long as there's demand for them. If you'd like to use D in Alpine containers, now's the time to contribute to whatever else is missing. A good idea of the current status can be found in the first post of the dmd pull linked above. This may be a good project for the hackathon, as it's mostly polishing up a bunch of small things.
Re: D for microservices
On 2018-03-11 12:10, Tamas wrote: Simple Vibe.d app talking to Redis, packed into docker containers: https://github.com/tam4s/hello-redis The takeaway is that I could not use Alpine as a host image, because I could not build the app statically on ubuntu. warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking It's just a warning. If you're not using "dlopen" or any of your dependencies you can just ignore the warning. -- /Jacob Carlborg
Re: D for microservices
On Thursday, 8 March 2018 at 22:53:11 UTC, aberba wrote: On Thursday, 8 March 2018 at 09:35:08 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB. Building directly in alpine will simplify automated builds too. Simple Vibe.d app talking to Redis, packed into docker containers: https://github.com/tam4s/hello-redis The takeaway is that I could not use Alpine as a host image, because I could not build the app statically on ubuntu. warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Re: D for microservices
On Thursday, 8 March 2018 at 09:35:08 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB. Building directly in alpine will simplify automated builds too.
Re: D for microservices
On Thursday, 8 March 2018 at 09:28:02 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 08:07:23 UTC, Joakim wrote: [...] I built out LDC on Alpine using the v1.8.0 LDC release. It's available now through Docker Hub and Github. Docker: https://hub.docker.com/r/andrewbenton/alpine-ldc/ Github: https://github.com/andrewbenton/alpine-ldc/ I get errors like below when trying to compile dub-registry: [...] OK you found an issue :). Indeed the file only covered Glibc so that's why vibe.d didn't compile for Musl. I fixed the issue for uClibc (1) and it works now (builds and runs). Probably the fix for Musl is similar. 1. https://github.com/dlang/druntime/pull/2134
Re: D for microservices
On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB.
Re: D for microservices
On Thursday, 8 March 2018 at 08:07:23 UTC, Joakim wrote: On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [snip] What is the exact error? Maybe report it here: https://github.com/lindt/docker-dmd/issues/1 I built out LDC on Alpine using the v1.8.0 LDC release. It's available now through Docker Hub and Github. Docker: https://hub.docker.com/r/andrewbenton/alpine-ldc/ Github: https://github.com/andrewbenton/alpine-ldc/ I get errors like below when trying to compile dub-registry: /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/dns.d(11,9): Error: module core.sys.posix.netdb import 'AI_ADDRCONFIG' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/dns.d(11,9): Error: module core.sys.posix.netdb import 'AI_V4MAPPED' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(13,9): Error: module core.sys.posix.netdb import 'AI_ADDRCONFIG' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(13,9): Error: module core.sys.posix.netdb import 'AI_V4MAPPED' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(47,10): Error: module core.sys.linux.netinet.in_ import 'IP_ADD_MEMBERSHIP' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(47,10): Error: module core.sys.linux.netinet.in_ import 'IP_MULTICAST_LOOP' not found, did you mean enum member 'IPV6_MULTICAST_LOOP'? ldc2 failed with exit code 1.
Re: D for microservices
On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [snip] This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Unfortunately the link that's associated with is based on an ubuntu build which should be using glibc, not alpine w/ musl.
Re: D for microservices
On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [...] I took a shot at dockerizing dub-registry with the alpine build using multistage builds. I got ldc installed and dub built. Unfortunately when I tried to build dub-registry, I found that vibe-d's vibe-core, libevent, and libasync configurations can't run under musl due to some of the networking dependencies in druntime core, e.g. `core.sys.linux.netinet.in_`. I realize that's something that you can't really prevent as the packager, but it's still relevant to the status of D for microservices. Bad news: not being able to build vibe.d is a pretty big hit for this objective. Good news: there are other libraries we can try and we know how to fix this problem. What is the exact error? Maybe report it here: https://github.com/lindt/docker-dmd/issues/1
Re: D for microservices
On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg
Re: D for microservices
On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [...] I took a shot at dockerizing dub-registry with the alpine build using multistage builds. I got ldc installed and dub built. Unfortunately when I tried to build dub-registry, I found that vibe-d's vibe-core, libevent, and libasync configurations can't run under musl due to some of the networking dependencies in druntime core, e.g. `core.sys.linux.netinet.in_`. I realize that's something that you can't really prevent as the packager, but it's still relevant to the status of D for microservices. Bad news: not being able to build vibe.d is a pretty big hit for this objective. Good news: there are other libraries we can try and we know how to fix this problem. This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org
Re: D for microservices
On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: On Monday, 5 March 2018 at 14:34:44 UTC, aberba wrote: [snip] The Alpine build is up, let me know if you have any problems. Note the changelog entry that says you'll need to install llvm and maybe other packages from the Alpine package manager first. I took a shot at dockerizing dub-registry with the alpine build using multistage builds. I got ldc installed and dub built. Unfortunately when I tried to build dub-registry, I found that vibe-d's vibe-core, libevent, and libasync configurations can't run under musl due to some of the networking dependencies in druntime core, e.g. `core.sys.linux.netinet.in_`. I realize that's something that you can't really prevent as the packager, but it's still relevant to the status of D for microservices. Bad news: not being able to build vibe.d is a pretty big hit for this objective. Good news: there are other libraries we can try and we know how to fix this problem.
Re: D for microservices
On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: On Monday, 5 March 2018 at 14:34:44 UTC, aberba wrote: On Sunday, 25 February 2018 at 22:12:38 UTC, Joakim wrote: On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: [...] I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: [...] OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release. LDC 1.8 is out! The Alpine build is up, let me know if you have any problems. Note the changelog entry that says you'll need to install llvm and maybe other packages from the Alpine package manager first. Nice. I usually use the docker images though.
Re: D for microservices
On Monday, 5 March 2018 at 14:34:44 UTC, aberba wrote: On Sunday, 25 February 2018 at 22:12:38 UTC, Joakim wrote: On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks. OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release. LDC 1.8 is out! The Alpine build is up, let me know if you have any problems. Note the changelog entry that says you'll need to install llvm and maybe other packages from the Alpine package manager first.
Re: D for microservices
On Sunday, 25 February 2018 at 22:12:38 UTC, Joakim wrote: On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks. OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release. LDC 1.8 is out!
Re: D for microservices
On 2018-02-25 17:51, yawniek wrote: Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? Build a completely statically linked binary by compiling using LDC and add the "-static" flag. -- /Jacob Carlborg
Re: D for microservices
On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks. OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release.
Re: D for microservices
On Saturday, 24 February 2018 at 10:03:07 UTC, Joakim wrote: On Thursday, 22 February 2018 at 21:59:27 UTC, aberba wrote: On Thursday, 22 February 2018 at 08:54:04 UTC, Joakim wrote: [...] That makes me very happy!! Very much appreciated. I've updated ldc master to use Yuxuan's druntime port, which means the upcoming ldc 1.8 release will likely be a viable Alpine/Musl cross-compiler out of the box, provided you give it a C/Musl cross-compiler/linker to work with, by using the bundled ldc-build-runtime tool: https://wiki.dlang.org/Building_LDC_runtime_libraries Cross-compiling has not been tested however: do people normally cross-compile to Alpine containers or build their software in Alpine itself? If the latter, building ldc from master on Alpine/x64 3.7.0 passes all of the druntime/phobos stdlib unit tests but one, so you're good to go there, as long as you don't need a pre-built binary. Maybe we'll put one out for ldc with the upcoming 1.8 release. I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks.
Re: D for microservices
On Saturday, 24 February 2018 at 10:03:07 UTC, Joakim wrote: I've updated ldc master to use Yuxuan's druntime port, which means the upcoming ldc 1.8 release will likely be a viable Alpine/Musl cross-compiler out of the box, provided you give it a C/Musl cross-compiler/linker to work with, by using the bundled ldc-build-runtime tool: https://wiki.dlang.org/Building_LDC_runtime_libraries Cross-compiling has not been tested however: do people normally cross-compile to Alpine containers or build their software in Alpine itself? If the latter, building ldc from master on Alpine/x64 3.7.0 passes all of the druntime/phobos stdlib unit tests but one, so you're good to go there, as long as you don't need a pre-built binary. Maybe we'll put one out for ldc with the upcoming 1.8 release. great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container?
Re: D for microservices
On Thursday, 22 February 2018 at 21:59:27 UTC, aberba wrote: On Thursday, 22 February 2018 at 08:54:04 UTC, Joakim wrote: On Monday, 23 October 2017 at 18:18:28 UTC, Jacob Carlborg wrote: [...] Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices. That makes me very happy!! Very much appreciated. I've updated ldc master to use Yuxuan's druntime port, which means the upcoming ldc 1.8 release will likely be a viable Alpine/Musl cross-compiler out of the box, provided you give it a C/Musl cross-compiler/linker to work with, by using the bundled ldc-build-runtime tool: https://wiki.dlang.org/Building_LDC_runtime_libraries Cross-compiling has not been tested however: do people normally cross-compile to Alpine containers or build their software in Alpine itself? If the latter, building ldc from master on Alpine/x64 3.7.0 passes all of the druntime/phobos stdlib unit tests but one, so you're good to go there, as long as you don't need a pre-built binary. Maybe we'll put one out for ldc with the upcoming 1.8 release.
Re: D for microservices
On 2018-02-22 09:54, Joakim wrote: Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices. That's great. -- /Jacob Carlborg
Re: D for microservices
On Thursday, 22 February 2018 at 08:54:04 UTC, Joakim wrote: On Monday, 23 October 2017 at 18:18:28 UTC, Jacob Carlborg wrote: [...] Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices. That makes me very happy!! Very much appreciated.
Re: D for microservices
On 22/02/2018 10:17 PM, Suliman wrote: It would be nice if anyone will rewrite Musl to betterC :) Combine it with dmc's libc and we're starting to get a reasonable state.
Re: D for microservices
It would be nice if anyone will rewrite Musl to betterC :)
Re: D for microservices
On Monday, 23 October 2017 at 18:18:28 UTC, Jacob Carlborg wrote: On 2017-10-23 17:35, Joakim wrote: I'm sure someone could put these together if the above stuff worked. The question is who's interested in volunteering to help put this all together? Yeah, exactly. My plate if already full and all this is lower down on the priority list. Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices.
Re: D for microservices
On Saturday, 28 October 2017 at 14:55:25 UTC, aberba wrote: On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. Its the future. Highly doubt that. It really depend on the scale of your operations. Netflix, Google, Facebook, etc. all have open source tools around microservices. Its currently ruled by JavaScript > Go > Java. JavaScript being the leader. They have these in common: 1. Easy to deploy their code in docker containers including alpine Linux. Interestingly while Docker may seem all the rage in startups I find its use limited to test environments in the real world. Also Java fat jars were super easy to deploy ages before docker. They are also a great deal smaller. 2. They have APIs for major cloud services. Both official and third-party. 3. Good support for networking. HTTP, Websockets, IPC*, etc. Mostly HTTP. Honestly APIs these days can be written in anything that is able to put together a few HTTP responses. Unless you are doing serious work like: - DBs - Search engines - ML pipelines - Real-time event processing systems Any semimodern language/technology with a several hosts can manage to saturate 1Gbit link. Some take a certain amount of tuning others work out of the box. If you go for 40gbit/s it may be important to choose the right technology otherwise it’s all a matter of taste.
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. Its the future. Netflix, Google, Facebook, etc. all have open source tools around microservices. Its currently ruled by JavaScript > Go > Java. JavaScript being the leader. They have these in common: 1. Easy to deploy their code in docker containers including alpine Linux. 2. They have APIs for major cloud services. Both official and third-party. 3. Good support for networking. HTTP, Websockets, IPC*, etc. Mostly HTTP. That's it. The major advantages besides the hype. Lets see about D based on these requirements. 1. Kind of. There are dmd-ldc-dub docker images on docker hub. One by sociomantic. None using Alpine Linux as base though. Most people prefer alpine cus its lightweight (not a requirement). 2. Nope. Official APIs comes when there's mass adoption for that language and has good ROI. No complete and production ready cloud services API that I know of. Seems D users in that area roll out something on their own for private use with features they need. But most cloud PaaS provide support for custom runtimes which D has one for Heroku. Major candidate is Google AppEngine. 3. Phobos doesn't have a D HTTP API. Its sad but we have "requests" at code.dlang.org which works. We have vibe.d for http servers (aka nodejs of D) and seems popular based on threads about it and downloads. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? There several database APIs at cods.dlang.org. I don't know why some complain about no db APIs. This is a niche that D and all newer languages should target. How do we do it? Good question.
Re: D for microservices
On 10/27/17 00:18, Jacob Carlborg wrote: On 2017-10-26 12:25, Adam Wilson wrote: My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. My concern is not about the event loop, it's about asynchronous IO. vibe.d needs to use asynchronous IO and I assume that's regardless what kind of event loop implementation is used. Does all the existing database drivers that you want to use support asynchronous IO? PgSQL/MySQL/MSSQL all do, I think that covers about 90% of usage. IIRC Oracle does as well. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-26 12:25, Adam Wilson wrote: My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. My concern is not about the event loop, it's about asynchronous IO. vibe.d needs to use asynchronous IO and I assume that's regardless what kind of event loop implementation is used. Does all the existing database drivers that you want to use support asynchronous IO? -- /Jacob Carlborg
Re: D for microservices
On Thursday, October 26, 2017 19:19:57 Adam Wilson via Digitalmars-d wrote: > On 10/26/17 17:51, Jonathan M Davis wrote: > > On Thursday, October 26, 2017 03:25:24 Adam Wilson via Digitalmars-d wrote: > >> On 10/25/17 23:57, Jacob Carlborg wrote: > >>> I'm more concerned that I don't think we'll manage to implement a > >>> complete API and 100% bug free at the first try. > >> > >> Depends on how one defines first try. Phobos as a pretty solid process > >> for making sure these things are reasonably bug free. And near as I can > >> tell, Phobos is pretty good about accepting bug fixes. > > > > The bigger problem is API bugs. The review process is rigorous enough to > > weed out a lot of stuff, but the end result is typically an API that we > > _think_ looks good but usually hasn't seen much real world use (much as > > its design may be based on real world experience). And if it turns out > > that the API has problems, it can be very difficult to fix that in a > > way that doesn't break code. Sometimes, we're able to reasonably do > > something about it and sometimes not. In theory, std.experimental would > > mitigate that, and that's where anything that was accepted would go > > first, but the process for getting new modules into Phobos is very much > > geared towards getting an API right up front rather than getting > > something that's close and iterating towards where it ultimately should > > be. > > > > What would probably be better in general would be to be writing stuff > > that ends up on code.dlang.org first and gets some real world use there > > and then look at getting it into Phobos later rather than aiming > > directly for Phobos. Not only would that likely help lead towards > > better code being in Phobos, but we'd still get useful stuff even if it > > didn't make it into Phobos. > > > > - Jonathan M Davis > > I understand the concern, however, I can see potential mitigations. For > one, steal an API concept from somewhere else. I've had reasonable > success so far stealing ADO.NET and the refactoring it into something > more idiomatic. Using that as a starting point gave me a pretty good > understanding of what was needed and it gave me a prototype API that has > been battle-tested. > > Has anything from code.dlang.org been pulled into Phobos yet? I'm not > aware of anything. code.dlang.org is where Phobos projects go to quietly > die in obscurity. Nothing has been pulled into Phobos from anywhere in a while. The last thing was probably checkedint, and before that it was std.allocator - both of which came from Andrei. ndslice was in std.experimental for a little while but Ilya removed it, because he decided that it wasn't working to have it in Phobos, because he couldn't continue to improve on it there. Beyond that, I think that the last thing was the logger, which has just languished in std.experimental. It's my understanding that it needs some more work, but basically, once we added std.experimental, nothing has made it into Phobos proper. And over the last few years, there haven't been many attempts to get anything into Phobos, so not much has even made it into std.experimental. There was the GSoC project for a new XML parser, but that project seems to have died after the student got too busy with school after GSoC, and Sonke's std.json replacement didn't finish making its way through the review process, and I think that Sonke has given up on getting it in (if I understand correctly, there was just too much disagreement over what the std.json replacement should look like). Overall, people have just stopped trying to get major stuff into Phobos. New functions get added to existing modules, but pretty much no one seems to want to go through the Phobos review process to get anything into Phobos. code.dlang.org is where folks put anything that they're doing that they want to make available, and IIRC, the only item from there that's even attempted to make it into Phobos was Sonke's JSON parser. Much as some folks continue to talk about getting some piece of functionality into Phobos, no one is trying if it's anything major. So, it's not like Phobos projects are going to code.dlang.org to die. In general, they simply aren't even being attempted, and any serious projects that do exist are on code.dlang.org. Some do sit there unfinished (e.g. std_experimental_xml), but largely, the authors just don't want to go to the effort of getting the code into Phobos. And honestly, in general, at this point, I don't think that I'd want to bother either. It's quite a lot of work to get something through the Phobos review process, and once it's in Phobos, you lose control over it. If I work on something major, I can just put it up on code.dlang.org, and anyone who wants to use it can, and I have full control over what I do with the code base without having to get approval from Andrei or anyone else as to what I do with it. So, unless we're talking about something that practically needs to be in the
Re: D for microservices
On 10/26/17 17:51, Jonathan M Davis wrote: On Thursday, October 26, 2017 03:25:24 Adam Wilson via Digitalmars-d wrote: On 10/25/17 23:57, Jacob Carlborg wrote: I'm more concerned that I don't think we'll manage to implement a complete API and 100% bug free at the first try. Depends on how one defines first try. Phobos as a pretty solid process for making sure these things are reasonably bug free. And near as I can tell, Phobos is pretty good about accepting bug fixes. The bigger problem is API bugs. The review process is rigorous enough to weed out a lot of stuff, but the end result is typically an API that we _think_ looks good but usually hasn't seen much real world use (much as its design may be based on real world experience). And if it turns out that the API has problems, it can be very difficult to fix that in a way that doesn't break code. Sometimes, we're able to reasonably do something about it and sometimes not. In theory, std.experimental would mitigate that, and that's where anything that was accepted would go first, but the process for getting new modules into Phobos is very much geared towards getting an API right up front rather than getting something that's close and iterating towards where it ultimately should be. What would probably be better in general would be to be writing stuff that ends up on code.dlang.org first and gets some real world use there and then look at getting it into Phobos later rather than aiming directly for Phobos. Not only would that likely help lead towards better code being in Phobos, but we'd still get useful stuff even if it didn't make it into Phobos. - Jonathan M Davis I understand the concern, however, I can see potential mitigations. For one, steal an API concept from somewhere else. I've had reasonable success so far stealing ADO.NET and the refactoring it into something more idiomatic. Using that as a starting point gave me a pretty good understanding of what was needed and it gave me a prototype API that has been battle-tested. Has anything from code.dlang.org been pulled into Phobos yet? I'm not aware of anything. code.dlang.org is where Phobos projects go to quietly die in obscurity. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On Monday, October 23, 2017 12:13:12 Laeeth Isharc via Digitalmars-d wrote: > How can we generate a static binary ? I asked about this before, > and the response was that it's a bad idea because of security > vulns and so on. True if you are running on a conventional Linux > host. But on the other hand, it's not that great if when the > system phobos is upgraded all the D binaries break. You can > write a script using rdmd or dub scripting feature, but that > doesn't work for more complex programs. I believe that it works just fine if you do the linking step yourself, but dmd doesn't properly handle it with -L-static: https://issues.dlang.org/show_bug.cgi?id=6952 I have no idea how easy it would be to the linking manually with dub. Alternatively, I believe that you can statically link against any 3rd party libraries that you're using by linking to the .a file rather than by the library name. That won't work for glibc, but much as I'd love to statically link that, it's my understanding that doing so is generally a bad idea. I don't recall the reasons though - and with the increased stability in linux's ABI, I would have thought that it would be far less of an issue than it would have been in the past. Certainly, the concept of being able to statically link a program and then just run it on most any recent linux distro without recompiling it sounds very appealing. Yes, you'll have to recompile it for any security updates to any of your dependencies, whereas that can be avoided with dynamic linking, but depending on what you're doing, that often isn't really an issue, and it can be managed if it is (heck, you're doing basically the same thing when maintaining something like a docker image). - Jonathan M Davis
Re: D for microservices
On Thursday, October 26, 2017 03:25:24 Adam Wilson via Digitalmars-d wrote: > On 10/25/17 23:57, Jacob Carlborg wrote: > > I'm more concerned that I don't think we'll manage to implement a > > complete API and 100% bug free at the first try. > > Depends on how one defines first try. Phobos as a pretty solid process > for making sure these things are reasonably bug free. And near as I can > tell, Phobos is pretty good about accepting bug fixes. The bigger problem is API bugs. The review process is rigorous enough to weed out a lot of stuff, but the end result is typically an API that we _think_ looks good but usually hasn't seen much real world use (much as its design may be based on real world experience). And if it turns out that the API has problems, it can be very difficult to fix that in a way that doesn't break code. Sometimes, we're able to reasonably do something about it and sometimes not. In theory, std.experimental would mitigate that, and that's where anything that was accepted would go first, but the process for getting new modules into Phobos is very much geared towards getting an API right up front rather than getting something that's close and iterating towards where it ultimately should be. What would probably be better in general would be to be writing stuff that ends up on code.dlang.org first and gets some real world use there and then look at getting it into Phobos later rather than aiming directly for Phobos. Not only would that likely help lead towards better code being in Phobos, but we'd still get useful stuff even if it didn't make it into Phobos. - Jonathan M Davis
Re: D for microservices
On 26/10/2017 11:25 AM, Adam Wilson wrote: On 10/25/17 23:57, Jacob Carlborg wrote: On 2017-10-26 00:53, Adam Wilson wrote: This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I'm entirely against anything that is not compatible with vibe.d ;) My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. (Something about not supporting GUI loops, paging Mr. Cattermole). If that is really the case I don't see how being entirely vibe.d compatible and meeting the universal standard requirements of Phobos is possible. There would need to be a requirements gathering phase so that the community as a whole can bring their use-cases before we dove into code. The problem isn't the event loop design. Its a fairly solved problem. The way vibe.d's works is very specific to their use case (which isn't wrong if you only consider them). You can't 'hook' into it. Which makes it very undesirable for Phobos. Since it won't cover most use cases. Even if the source they are using is compatible with an external GUI event loop (which it should be for Windows from what I've read). So I wouldn't be starting with vibe.d's event loop model. Quite to the contrary, kill it. Build something that will last throughout the ages for everyone and put this problem to rest.
Re: D for microservices
On 10/25/17 23:57, Jacob Carlborg wrote: On 2017-10-26 00:53, Adam Wilson wrote: This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I'm entirely against anything that is not compatible with vibe.d ;) My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. (Something about not supporting GUI loops, paging Mr. Cattermole). If that is really the case I don't see how being entirely vibe.d compatible and meeting the universal standard requirements of Phobos is possible. There would need to be a requirements gathering phase so that the community as a whole can bring their use-cases before we dove into code. I actually don't think the slow updates are huge problem as these DB interface libraries are pretty slow to change themselves. For example, ADO.NET didn't change significantly from it's 1.0 release until the arrival of Async/Await in .NET 4.5, which was 10 years later. The biggest addition prior to Async was TVP support and that was tiny comparatively and came in 2005. The libpq5 interface has barely changed in years. I can't speak to MySQL but IIRC it hasn't changed much either, at least not in a way that would effect the abstraction layer. I'm more concerned that I don't think we'll manage to implement a complete API and 100% bug free at the first try. Depends on how one defines first try. Phobos as a pretty solid process for making sure these things are reasonably bug free. And near as I can tell, Phobos is pretty good about accepting bug fixes. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-26 00:53, Adam Wilson wrote: This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I'm entirely against anything that is not compatible with vibe.d ;) I actually don't think the slow updates are huge problem as these DB interface libraries are pretty slow to change themselves. For example, ADO.NET didn't change significantly from it's 1.0 release until the arrival of Async/Await in .NET 4.5, which was 10 years later. The biggest addition prior to Async was TVP support and that was tiny comparatively and came in 2005. The libpq5 interface has barely changed in years. I can't speak to MySQL but IIRC it hasn't changed much either, at least not in a way that would effect the abstraction layer. I'm more concerned that I don't think we'll manage to implement a complete API and 100% bug free at the first try. -- /Jacob Carlborg
Re: D for microservices
On Monday, 23 October 2017 at 12:13:12 UTC, Laeeth Isharc wrote: ... And on the other hand, for deployment, it's much easier to copy over one binary. (In a sense it's funny that the question was asked in the context of containers, because containers are kind of an alternative to having a single binary). When you're properly set-up of course it doesn't matter, but when you're moving towards that, a single binary is much easier. ... A bit OT but this part made me think of [1] of which you may appreciate the insight. Right now I kind of feel like with D we get the downside in size of fat binaries without the upside in portability (although solutions exist for the dedicated and patient ones). This might proove an important drawback in the future... we'll see. [1]: http://www.smashcompany.com/technology/why-would-anyone-choose-docker-over-fat-binaries
Re: D for microservices
On 10/23/17 23:29, Jacob Carlborg wrote: On 2017-10-24 00:02, Adam Wilson wrote: I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. It can be an optional dependency. Looking at ddb [1] it's not that much code that will be different if vibe.d is used or not. It's basically only reading and writing to the socket that is different [2]. Dub provides predefined version for different dependencies, i.e. Have_vibe_d_core. This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I had heard noises about that a few months ago. Anything happening on that front? No, not as far as I know. Sönke seems really busy recently. That is what I was afraid of. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? I would be really nice to database support directly in the standard library but it's not critical for me. It takes a lot of work with and massive overhead to get something into Phobos. It's also going to be really slow get in updates. I'm not sure if it's worth it. [1] https://github.com/pszturmaj/ddb [2] https://github.com/pszturmaj/ddb/blob/master/source/ddb/postgres.d#L189-L246 I actually don't think the slow updates are huge problem as these DB interface libraries are pretty slow to change themselves. For example, ADO.NET didn't change significantly from it's 1.0 release until the arrival of Async/Await in .NET 4.5, which was 10 years later. The biggest addition prior to Async was TVP support and that was tiny comparatively and came in 2005. The libpq5 interface has barely changed in years. I can't speak to MySQL but IIRC it hasn't changed much either, at least not in a way that would effect the abstraction layer. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 10/23/17 18:51, rikki cattermole wrote: On 23/10/2017 11:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. I had heard noises about that a few months ago. Anything happening on that front? An event loop is a key piece of a project (Async/Await) that I want to work on, and having it in DRuntime would make that project fantastically simpler. IMHO, the bulk of the time required is in getting an event loop into DRT, the rest is a LOT of relatively straightforward compiler lowering. IMHO, DRT is in significant need of an event loop system. This would allow us to simplify a large number of problems (Async/Await, GUI's, IO, etc). As near as I can tell, the problem isn't so much doing the work, but getting the required sign-off's for inclusion into DRT. Another problem that I've been made aware of is that vibe-core may not be ideal in certain situations. As this would be landed in DRT itself this would obviously need to be addressed. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? *whispers* heyyy, heard about SPEW[0]? [0] https://github.com/Devisualization/spew/blob/master/src/base/cf/spew/event_loop/defs.d You've mentioned it a few times. ;-) -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-24 00:02, Adam Wilson wrote: I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. It can be an optional dependency. Looking at ddb [1] it's not that much code that will be different if vibe.d is used or not. It's basically only reading and writing to the socket that is different [2]. Dub provides predefined version for different dependencies, i.e. Have_vibe_d_core. I had heard noises about that a few months ago. Anything happening on that front? No, not as far as I know. Sönke seems really busy recently. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? I would be really nice to database support directly in the standard library but it's not critical for me. It takes a lot of work with and massive overhead to get something into Phobos. It's also going to be really slow get in updates. I'm not sure if it's worth it. [1] https://github.com/pszturmaj/ddb [2] https://github.com/pszturmaj/ddb/blob/master/source/ddb/postgres.d#L189-L246 -- /Jacob Carlborg
Re: D for microservices
On 23/10/2017 11:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. I had heard noises about that a few months ago. Anything happening on that front? An event loop is a key piece of a project (Async/Await) that I want to work on, and having it in DRuntime would make that project fantastically simpler. IMHO, the bulk of the time required is in getting an event loop into DRT, the rest is a LOT of relatively straightforward compiler lowering. IMHO, DRT is in significant need of an event loop system. This would allow us to simplify a large number of problems (Async/Await, GUI's, IO, etc). As near as I can tell, the problem isn't so much doing the work, but getting the required sign-off's for inclusion into DRT. Another problem that I've been made aware of is that vibe-core may not be ideal in certain situations. As this would be landed in DRT itself this would obviously need to be addressed. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? *whispers* heyyy, heard about SPEW[0]? [0] https://github.com/Devisualization/spew/blob/master/src/base/cf/spew/event_loop/defs.d
Re: D for microservices
On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. I had heard noises about that a few months ago. Anything happening on that front? An event loop is a key piece of a project (Async/Await) that I want to work on, and having it in DRuntime would make that project fantastically simpler. IMHO, the bulk of the time required is in getting an event loop into DRT, the rest is a LOT of relatively straightforward compiler lowering. IMHO, DRT is in significant need of an event loop system. This would allow us to simplify a large number of problems (Async/Await, GUI's, IO, etc). As near as I can tell, the problem isn't so much doing the work, but getting the required sign-off's for inclusion into DRT. Another problem that I've been made aware of is that vibe-core may not be ideal in certain situations. As this would be landed in DRT itself this would obviously need to be addressed. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-23 14:17, Laeeth Isharc wrote:
Can you elaborate on how the TLS implementation needs to be changed?
# echo 'void main() {}' > main.d && dmd -c main.d && gcc main.o -o main
-m64 -static -L/root/.dvm/compilers/dmd-2.076.1/linux/bin/../lib64
-Xlinker -Bstatic -lphobos2 -lpthread -lm -lrt -ldl
/root/.dvm/compilers/dmd-2.076.1/linux/bin/../lib64/libphobos2.a(sections_elf_shared_782_420.o):
In function `_D2rt19sections_elf_shared11getTLSRangeFNbNimmZAv':
src/rt/sections_elf_shared.d:(.text._D2rt19sections_elf_shared11getTLSRangeFNbNimmZAv[_D2rt19sections_elf_shared11getTLSRangeFNbNimmZAv]+0x38):
undefined reference to `__tls_get_addr'
collect2: error: ld returned 1 exit status
I need to manually invoke GCC to link because DMD will pass "-Xlinker
--export-dynamic -Xlinker -Bdynamic" and I need to add "-static".
If someone wanted to work on rabbit bindings/wrapper, I might be open to
sponsoring that. I'm not in love with Rabbit (one node uses more than
40% of memory so the node goes down, taking the cluster with it.
really?), but we use it currently.
I made a start on bindings for librabbitmq here:
https://github.com/kaleidicassociates/rabbitmq-d
Is that compatible with vibe.d?
--
/Jacob Carlborg
Re: D for microservices
On 2017-10-23 14:13, Laeeth Isharc wrote: How can we generate a static binary ? It's already supported by LDC, using the -static flag. This Linux binary [1] is statically linked. [1] https://github.com/jacob-carlborg/remarkify/releases -- /Jacob Carlborg
Re: D for microservices
On 2017-10-23 17:35, Joakim wrote: I'm sure someone could put these together if the above stuff worked. The question is who's interested in volunteering to help put this all together? Yeah, exactly. My plate if already full and all this is lower down on the priority list. -- /Jacob Carlborg
Re: D for microservices
On Monday, 23 October 2017 at 12:08:52 UTC, Jacob Carlborg wrote: On 2017-10-22 04:48, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? * Support full static linking using DMD, which requires the TLS implementation to be modified * Support musl as the standard C library, I've discussed that before [1] I can probably whip this together in a couple days, as I added the Bionic support before, but it's not something I need, so it'd just be a donation for you and others. That's why I'm trying to figure out who exactly wants this and if others want to chip in. * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library * RabbitMQ library compatible with vibe.d * Serialization to/from JSON and YAML Some of this stuff is on dub, but nothing that interests me or that I'd chip in with. Good to see that RabbitMQ supports binary protocols like AMQP and MQTT though, would be nuts if it were all just text like STOMP or especially JSON/YAML. * Official Docker images with DMD and LDC wouldn't hurt * Pre-compiled DMD that works on Alpine. Fully statically linked DMD would help here I'm sure someone could put these together if the above stuff worked. The question is who's interested in volunteering to help put this all together?
Re: D for microservices
On Monday, 23 October 2017 at 12:08:52 UTC, Jacob Carlborg wrote: On 2017-10-22 04:48, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? * Support full static linking using DMD, which requires the TLS implementation to be modified * Support musl as the standard C library, I've discussed that before [1] * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library * RabbitMQ library compatible with vibe.d * Serialization to/from JSON and YAML * Official Docker images with DMD and LDC wouldn't hurt * Pre-compiled DMD that works on Alpine. Fully statically linked DMD would help here That's what I can think of for now. [1] http://forum.dlang.org/post/nhem1l$1ee3$1...@digitalmars.com Can you elaborate on how the TLS implementation needs to be changed? If someone wanted to work on rabbit bindings/wrapper, I might be open to sponsoring that. I'm not in love with Rabbit (one node uses more than 40% of memory so the node goes down, taking the cluster with it. really?), but we use it currently. I made a start on bindings for librabbitmq here: https://github.com/kaleidicassociates/rabbitmq-d Laeeth.
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? We're going a bit in that direction, although it's a different thing in our kind of industry from a web company - we have fewer services and many fewer requests, but latency matters more for example. I was thinking we might use Go for some services that integrate and monitor things, but we could also use D. https://jobs.github.com/positions/8e98eac8-b504-11e7-9da8-0737a3dcef18 From the comment: "Would be really cool to have LDC easily building alpine containers + static D binaries" How can we generate a static binary ? I asked about this before, and the response was that it's a bad idea because of security vulns and so on. True if you are running on a conventional Linux host. But on the other hand, it's not that great if when the system phobos is upgraded all the D binaries break. You can write a script using rdmd or dub scripting feature, but that doesn't work for more complex programs. And on the other hand, for deployment, it's much easier to copy over one binary. (In a sense it's funny that the question was asked in the context of containers, because containers are kind of an alternative to having a single binary). When you're properly set-up of course it doesn't matter, but when you're moving towards that, a single binary is much easier. I guess I can figure out the answer to this question easily enough, but I think giving people the option might help with adoption of D for micro services. For example it's really just not that fun to make an AWS Lambda using D - being able to easily build a static binary would make the process much more pleasant. I wrote this up a while back: http://awslambda-d.readthedocs.io/en/latest/ "Since dmd links phobos dynamically on linux, and phobos/druntime aren't installed on the AWS lambda server, we will need to upload these to the servers and tell the application where to find them. (I should really have appended to LD_LIBRARY_PATH as I did with PATH). Now one can follow the regular instructions for AWS Lambda: create a .zip file with the D binary, the JS file, and the following libraries (update version numbers as appropriate): libcrypto.so.1.0.0 libphobos2.so.0.67 libevent-2.0.so.5 libssl.so.1.0.0 " Alpine is nice - would be good to have this as a standard target in time. Laeeth.
Re: D for microservices
On 2017-10-22 04:48, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? * Support full static linking using DMD, which requires the TLS implementation to be modified * Support musl as the standard C library, I've discussed that before [1] * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library * RabbitMQ library compatible with vibe.d * Serialization to/from JSON and YAML * Official Docker images with DMD and LDC wouldn't hurt * Pre-compiled DMD that works on Alpine. Fully statically linked DMD would help here That's what I can think of for now. [1] http://forum.dlang.org/post/nhem1l$1ee3$1...@digitalmars.com -- /Jacob Carlborg
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: [...] rock solid, easy, common-dev-proof, huge std lib. like that of golang.
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: This is a niche that D and all newer languages should target. How do we do it? Optimize the TechEmpower benchmark? Vibe.d looks quite weak there. https://www.techempower.com/benchmarks/
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: [...] Rather a container with dub/ldc configured to compile with musl libc I think. What else, what kinds of libraries do you normally use? Given current selection I bet most are hand-rolled except maybe vibe.d and common serialization libs.
Re: D and microservices
On Thursday, 8 October 2015 at 03:12:03 UTC, Martin Nowak wrote: We might be able to reuse the existing delegate capture mechanism to create continuations. Then await would "simply" rewrite the rest of the body as delegate, similar to how the foreach body can be transformed into a delegate. Not sure how that works? What is needed is to preallocate a "continuation" that is large enough to cover all "stack allocated" state that has to be retained when the coroutine yields, at all yield points. But with reuse of memory for dead variables (like in optimized stack usage where you reuse stack memory that is dead). Then use those areas of the continuation during computation rather than the stack. So basically all functions that may yield have to be available for static analysis before you start up the coroutine and all reachable yield points have to be analysed. So you basically have one big stack frame at the "top of the stack" as a separate object (the "continuation") and everything that has to survive a yield has to go onto that frame. That's how I would do it anyway. There is some use-case for HPC code where stackless coroutines make a huge differences (used with a work stealing scheduler), for basic networking code it will only be a small difference. The current D implementation can run out of stack. That's much less of an issue if the system/thread stack is used in combination with a fixed size preallocated "continuation".
Re: D and microservices
On Wednesday, 7 October 2015 at 10:50:47 UTC, Russel Winder wrote: On Tue, 2015-10-06 at 18:56 +, Dicebot via Digitalmars-d wrote: On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote: > Has anyone got a small example of microservices using D, > with Vibe.d or otherwise, that I can make use of? I need > some examples of small microservices for a session at μCon > 2015. What do you mean by microservice examples? It is infrastructure methodology, not specific code thing, any simple network service can be viewed as microservice. At the Web services application level it is having a small framework for structuring the use of GET, POST, PUT, DELETE,… packets. Basically RESTful services. At a non HTTP level I guess you can say all networking is microservices except that you want a protocol over TCP/IP or UDP/IP and shouldn't there be frameworks to assist building these things? Zeromq plus msgpack or the like (maybe cerealed!) gives you quite a lot. I use nanomsg, which is still beta but nicer in some ways. In any case it gives you survey, request/reply, bus, publish / subscribe patterns. And it's very light and has decent throughput. What kind of framework were you thinking of ? (nanomsg doesn't do UDP, but I think zeromq does. You can write your own protocol or pattern if the existing ones don't work for you, and sounds like that's work, but not much more than it needs to be).
Re: D and microservices
On Thursday, 8 October 2015 at 06:43:24 UTC, Laeeth Isharc wrote: In any case, we have now inconceivable amounts of computing power on tap for very affordable prices and the tools to manage it. When a little instance is 0.7 cents an hour, and a usable one is 1.5 cents and you can scale up and down as quickly as you like, many things become possible that weren't before. The world is only slowly beginning to figure out what, judging by what I see in finance. It is amazing how much compute is on tap these days and we do almost nothing with it. Forget about the cloud, just look at how powerful all these mobile devices are. The problem is that software development has lagged leagues behind hardware development. Since I am foolish enough to run my own mail server, you can be sure that my sympathies are with you. I used to do that, but what would stop them from getting into your server either? Just as they go after email hosts, they can go after server hosts. Unless you're running it out of your own house, which has other problems, most notably port-blocking by ISPs. I think people are reckless - both individuals and corporations - in what they unthinkingly cede to companies that really have few incentives to act as one might wish. It's one thing to be the customer, and quite another to be the product. I saw Eric Schmidt speak some years back, and he really didn't give me a feeling that we have a similar idea about what evil means! They are reckless until they're personally hurt by it. There will be a backlash when some scandal or hack forces people to change. The recent ad-blocking and router security scandals are good examples, both of which have been festering for a _long_ time: http://fortune.com/2015/09/20/apple-ad-block-ios9/ http://blog.codinghorror.com/welcome-to-the-internet-of-compromised-things/ within the enterprise often being rather different. Things even in quite small firms of 150+ people start being more about internal bureaucratic logic and much less about ROI. And the problem is that it's hard to make a convincing case for something that doesn't yet exist, and yet many big things have the tiniest beginnings. So the current enterprise technology environment discourages those little experiments that drive real innovation, as well as getting in the way of larger projects. Interesting article along the same vein, that goes in a fairly radical direction until near the end, when it shifts course to point out how Apple is trying to account for this: http://www.asymco.com/2015/08/31/fluid-coupling/ The cloud, by commoditising compute, and making the price much more transparent then is a force for breaking down central structures that no longer serve their needs well. When you can get whatever you need from Amazon today, waiting months and paying 10x more starts to seem a little stiff, and people start asking questions when before they had no choice but to grin and bear it. Of course the questions about privacy and commercial confidentiality are there, but perhaps solutions are arriving, and business people need to be practical. Even an overpriced internal cloud gives much more power to internal entrepreneurs than did the old way. Heh, a more centralized external cloud allows more agility than the centralized internal IT systems that were in place before? Perhaps. Given all the data breaches, I don't think the problems with data security will ever go away though.
Re: D and microservices
On Wednesday, 7 October 2015 at 10:39:07 UTC, Russel Winder wrote: On Tue, 2015-10-06 at 15:07 -0400, Nick Sabalausky via (Kinda like how "cloud" sounds like a big fancy new revolution until you realize it's just the hip new word for "internet" or "hosted". Yes - technically it is nothing new perhaps. But it's a commercial phenomenon, and so whilst the marketing hype is regrettable, if inevitable, it's the commercial aspects that are important here, much more than technical ones. Big technological shifts (speaking about things viewed from a societal level, not from a practitioner's perspective) have two components - big relative price shift (which might be infinite in practical terms to a large fortune or from cheap to very cheap depending on the situation), and then the new activities and new ways of doing old things that become possible as a consequence of this price shift. Business people take a long time to figure it out (see Brynjolfsson's work on organisational architecture), and that's why we have a period of hand-wringing between the new technology arriving and us seeing its broader benefits. Solow, an expert on growth, observed in 1987 that "computers are everywhere but in the productivity statistics". (My theory then was that people were too busy fiddling with fonts on their Macs to actually do any work!). A decade later, we had a different view of their influence. Similarly we have people saying today "how many jobs has Twitter created?". But it's not the people that Twitter directly employ, but those that benefit both from using it, and from the broader set of shifts of which Twitter is only the beginning. Blyth Masters, for example, is doing some interesting work on exploring possibilities from using blockchain type technologies for wholesale finance. There are some obvious problems there, but I wouldn't care to bet nothing interesting comes out of it. But it's much bigger than that, even though I can only be aware of a part of it. In any case, we have now inconceivable amounts of computing power on tap for very affordable prices and the tools to manage it. When a little instance is 0.7 cents an hour, and a usable one is 1.5 cents and you can scale up and down as quickly as you like, many things become possible that weren't before. The world is only slowly beginning to figure out what, judging by what I see in finance. "Cloud" is really a destruction of personal computing in favour of re- centralization of all computing: put the computing power back in the hands of the people who want to control what you may or may not do with computers. Beyond this is gets political. Does that sound about accurate, or am I missing something? Since I am foolish enough to run my own mail server, you can be sure that my sympathies are with you. I think people are reckless - both individuals and corporations - in what they unthinkingly cede to companies that really have few incentives to act as one might wish. It's one thing to be the customer, and quite another to be the product. I saw Eric Schmidt speak some years back, and he really didn't give me a feeling that we have a similar idea about what evil means! This being said, the world has always been broken in some respect, and we shouldn't let that stop us looking at the situation objectively even if one would wish in some respects things were other than they are. The cloud may be centralising things for the individual, but at the corporate level it's less clear. In particular, it's a classic case of the accounting being pretty clear that the economies of scale mean you should centralise IT infrastructure and it's management, but the practical experience of the people actually generating value within the enterprise often being rather different. Things even in quite small firms of 150+ people start being more about internal bureaucratic logic and much less about ROI. And the problem is that it's hard to make a convincing case for something that doesn't yet exist, and yet many big things have the tiniest beginnings. So the current enterprise technology environment discourages those little experiments that drive real innovation, as well as getting in the way of larger projects. The cloud, by commoditising compute, and making the price much more transparent then is a force for breaking down central structures that no longer serve their needs well. When you can get whatever you need from Amazon today, waiting months and paying 10x more starts to seem a little stiff, and people start asking questions when before they had no choice but to grin and bear it. Of course the questions about privacy and commercial confidentiality are there, but perhaps solutions are arriving, and business people need to be practical. Even an overpriced internal cloud gives much more power to internal entrepreneurs than did the old way. I think you are missing some aspects of why
Re: D and microservices
On Wednesday, 7 October 2015 at 17:31:12 UTC, Atila Neves wrote: My "Go vs D vs C vs Erlang" MQTT shootout was based on a colleague claiming that Go would win because concurrency is its thing. I called his bluff since despite Go having a (AFAIK) very good scheduler, I didn't see how vibe.d wouldn't give me the same performance. I was right. Yes, Go is unlikely to have any throughput advantages compared to a systems level programming language. Go's advantages: - a more mature compiler - robust runtime features - lower latency GC - infrastructure support - simple syntactical constructs I would consider Go-routines a convenience feature. If C++17 does stackless coroutines right then it probably will surpass both Go and D in terms of memory locality, initialization performance and memory usage; and therefore throughput as well.
Re: D and microservices
On Tuesday, 6 October 2015 at 17:54:48 UTC, Russel Winder wrote: On Tue, 2015-10-06 at 16:21 +, Dejan Lekic via Digitalmars-d wrote: On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote: > [...] As far as I know, there is no implementation of microservices as we see in the Java world. IMHO, D community should come up with a good microservices architecture. As you pointed out, it could be based on vibed. Pity, microservices is a very fashionable thing just now, and Go is wiping the floor with Node and Java. Well that bit is opinion but… many people are getting into all this non-blocking, event-driven, shared memory stuff and boiling their brains, whereas the Go folk are doing blocking stuff using dataflow which is much easier to program. I have never used Go, but isn't what you describe exactly what vibe.d is doing using Fibers ?
Re: D and microservices
On Tuesday, 6 October 2015 at 23:01:43 UTC, Laeeth Isharc wrote: On Tuesday, 6 October 2015 at 19:31:20 UTC, Mengu wrote: a half of it is the buzz and other half of is not. remember people talking about reactjs, go and rails being buzz? they were the same. we have built an online payment gateway and we are about to decouple our application and switch to microservices architecture. we have an api, a dashboard, a checkout page, mobile flow. we have to deal with accounting and reporting as well. and there is no way that this application will turn into a giant monolith. i don't want that. nobody wants that. it will become something we cannot handle. now a question for you. do you wish you had built it from components from day one? or do you see creating it as a blob to start with and then when the right divisions are clear factoring these out into micro-services as simply a natural part of the design process? because you know much more by having started, and it's not so hard to refactor at this stage. it's the latter for me. i am glad we have everything coupled together. this way we are able to see what parts can live by themselves and what parts cannot. and when you're building a start-up, i hardly believe designing microservices is the way to go. if you have time, if you have resources then go and design your microservices architecture. take your time, use your resources. but i didn't. i was one man and i did not have the time. now we are 4 people. we are not adding new features anymore and we know what kind of future is out there for our application. right now almost 95% of our application can be decoupled, do not depend on each other in terms of code. they can just communicate and get things done. also it means more uptime, more developers, more resources, etc. we already have two microservices. one is for card vaulting and the other one is for end of day and cash report download/process service from the banks and other payment gateways. sometimes the industry you are in will push you that way. and... one of the most important things for me with microservices is that now I can get Haskell and D in our codebase. :-) another thing is whenever we do deployments we have to take down the whole application and go offline pretend I'm asking you before it was deployed in production... nobody suggests starting with microservices architecture because you'll never know where things will lead you however when it becomes a giant the suggestion is to use microservices. some people do. but I would have thought the point I made above is the real reason. it doesn't take very long to write it that way from the beginning IFF you know what you want it to look like before you start. and maybe you don't. but I am interested in what your experience has been. nobody, in their right minds, then. :)
Re: D and microservices
On Wednesday, 7 October 2015 at 19:58:55 UTC, Ola Fosheim Grøstad wrote: If C++17 does stackless coroutines right then it probably will surpass both Go and D in terms of memory locality, initialization performance and memory usage; and therefore throughput as well. We might be able to reuse the existing delegate capture mechanism to create continuations. Then await would "simply" rewrite the rest of the body as delegate, similar to how the foreach body can be transformed into a delegate. There is some use-case for HPC code where stackless coroutines make a huge differences (used with a work stealing scheduler), for basic networking code it will only be a small difference.
Re: D and microservices
On Tue, 2015-10-06 at 15:07 -0400, Nick Sabalausky via Digitalmars-d
wrote:
>
[…]
>
> Felt stupid for not being hip to this "microservices" thing you say,
> so
> just looked it up. But it sounds to me like it's basically just a
> buzz-driven rediscovery of the basic principles of proper
> encapsulation
> and Unix philosophy ("do one thing and do it well").
Yes there is an element of just using encapsulation and single purpose
tool, but there is a lot of trying to do cluster computing well by
keeping the components small and manageable. The downside is the
potential communications complexity.
BTW microservices doesn't mean just Web services/applications per se
(they are a subset), it is any protocol. The Web services/applications
people conveniently forget there are protocols other than HTTP.
> (Kinda like how "cloud" sounds like a big fancy new revolution until
> you
> realize it's just the hip new word for "internet" or "hosted". Or
> "Facade design pattern" vs plain old "It's a thin wrapper".)
"Cloud" is really a destruction of personal computing in favour of re-
centralization of all computing: put the computing power back in the
hands of the people who want to control what you may or may not do with
computers. Beyond this is gets political.
> Does that sound about accurate, or am I missing something?
I think you are missing some aspects of why there is sanity to what is
happening, but you are not wrong that there is a lot of "buzz" and
"hype", but that tends to go along with anything "trendy" and "hip".
And there is a lot of reinventing the wheel because the hipsters create
cool new stuff, but have failed to study the last 60 years of computing
before doing so.
--
Russel.
=
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
signature.asc
Description: This is a digitally signed message part
Re: D and microservices
On Wed, 2015-10-07 at 07:15 +, extrawurst via Digitalmars-d wrote: > […] > > I have never used Go, but isn't what you describe exactly what > vibe.d is doing using Fibers ? As I understand it, vibe.d is a single threaded event-loop with fibres. In this sense it is equivalent in architecture to Node except that vibe.d allows for blocking fibres where Node required non-blocking approaches. Thus I would choose vibe.d over Node any day, except… Go has a task pool (goroutines), but underneath is a thread pool not a single thread. This means that multiple goroutines can be executing concurrently, that means in parallel (*). GPars has an approach similar to Go. Go is way more efficient though as it uses the kernel threads directly, GPars has to fight with the JVM threads. (*) It is a pity that computing and the rest of the world have different meaning for concurrent and parallel, it leads to lots of wonderful confusion. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: D and microservices
On Tuesday, 6 October 2015 at 19:07:32 UTC, Nick Sabalausky wrote:
Felt stupid for not being hip to this "microservices" thing you
say, so just looked it up. But it sounds to me like it's
basically just a buzz-driven rediscovery of the basic
principles of proper encapsulation and Unix philosophy ("do one
thing and do it well").
(Kinda like how "cloud" sounds like a big fancy new revolution
until you realize it's just the hip new word for "internet" or
"hosted". Or "Facade design pattern" vs plain old "It's a thin
wrapper".)
Does that sound about accurate, or am I missing something?
Heh, funny you say that, since Russel's post just got me to look
up the term too. :) I had encountered the term before, when I was
talking to a dev shop about a project earlier this year, but
figured it was exactly what it sounded like and didn't bother
looking it up till now. Looks like I was right.
Whatever hype may be involved, at least they're re-branding
something _useful_, just be grateful for that. ;)
On Wednesday, 7 October 2015 at 10:39:07 UTC, Russel Winder wrote:
(Kinda like how "cloud" sounds like a big fancy new revolution
until
you
realize it's just the hip new word for "internet" or "hosted".
Or
"Facade design pattern" vs plain old "It's a thin wrapper".)
"Cloud" is really a destruction of personal computing in favour
of re- centralization of all computing: put the computing power
back in the hands of the people who want to control what you
may or may not do with computers. Beyond this is gets political.
I wouldn't go that far. Users want to access their data from
anywhere on their mobile devices, and given that those mobile
devices can't store much, the cloud is just the most convenient
place to put the data, now that 4G speeds are comparable to wired
broadband from just a couple years ago (except for the much lower
quotas on wireless, which only really limit you with video).
You could have made the same argument a couple decades ago, that
PCs were a step back from workstations, because the old UNIX
workstations came with compilers and a toolchain that let you
build your own software whereas you had to go get the tools for
your PC, ie they weren't bundled. Some people make the same
argument with the even more locked-down tablets today, where you
can't even develop software using them.
But the truth is that most PC users weren't developing software
or exerting much "control" over their desktops/laptops, and
almost nobody wants to develop software or create on a mobile
device. They welcome more locked-down systems, because it lets
them do the lighter tasks they actually want to get done much
easier, without worrying about configuring the system just right,
ie complexity, or viruses.
However, I do agree with you that the cloud is way overplayed,
just for different reasons. I think decentralized computing is
going to take off instead, with a decentralized social network,
like Diaspora except running peer-to-peer on your mobile devices,
replacing centralized Facebook. Cloud will merely be one of the
nodes in that decentralized network, one where you offload heavy
computing activity or data that you need from everywhere else,
but not the center of mass that many treat it as today.
Re: D and microservices
On Wednesday, 7 October 2015 at 10:50:47 UTC, Russel Winder wrote: What do you mean by microservice examples? It is infrastructure methodology, not specific code thing, any simple network service can be viewed as microservice. At the Web services application level it is having a small framework for structuring the use of GET, POST, PUT, DELETE,… packets. Basically RESTful services. Even web microservices are not necessarily RESTful ones. Nick has correctly pointed out that there is a lot of spiritual similariy between concept of microservice and traditional UNIX way of designing small single-purpose command lines apps. It basically gets down to "do as few things as possible in one application (but do it good) and build a pipeline". At a non HTTP level I guess you can say all networking is microservices except that you want a protocol over TCP/IP or UDP/IP and shouldn't there be frameworks to assist building these things? What utility would you expect from such framework?
Re: D and microservices
On Tue, 2015-10-06 at 18:56 +, Dicebot via Digitalmars-d wrote: > On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote: > > Has anyone got a small example of microservices using D, with > > Vibe.d or otherwise, that I can make use of? I need some > > examples of small microservices for a session at μCon 2015. > > What do you mean by microservice examples? It is infrastructure > methodology, not specific code thing, any simple network service > can be viewed as microservice. At the Web services application level it is having a small framework for structuring the use of GET, POST, PUT, DELETE,… packets. Basically RESTful services. At a non HTTP level I guess you can say all networking is microservices except that you want a protocol over TCP/IP or UDP/IP and shouldn't there be frameworks to assist building these things? -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: D and microservices
On Wednesday, 7 October 2015 at 10:48:02 UTC, Russel Winder wrote: On Wed, 2015-10-07 at 07:15 +, extrawurst via Digitalmars-d wrote: […] I have never used Go, but isn't what you describe exactly what vibe.d is doing using Fibers ? As I understand it, vibe.d is a single threaded event-loop with fibres. In this sense it is equivalent in architecture to Node except that vibe.d allows for blocking fibres where Node required non-blocking approaches. Thus I would choose vibe.d over Node any day, except… vibe.d supports multiple threads and will schedule tasks on separate threads if asked to. Each of those threads will in turn have several fibres. The main difference from Go's runtime for me are twofold: 1. vibe.d is a library, goroutines are part of the library 2. vibe.d (like D itself) uses the actor model, go is (sort of, according to some) CSP My "Go vs D vs C vs Erlang" MQTT shootout was based on a colleague claiming that Go would win because concurrency is its thing. I called his bluff since despite Go having a (AFAIK) very good scheduler, I didn't see how vibe.d wouldn't give me the same performance. I was right. Atila
Re: D and microservices
On Tue, 2015-10-06 at 16:21 +, Dejan Lekic via Digitalmars-d wrote: > On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote: > > Has anyone got a small example of microservices using D, with > > Vibe.d or otherwise, that I can make use of? I need some > > examples of small microservices for a session at μCon 2015. > > As far as I know, there is no implementation of microservices as > we see in the Java world. IMHO, D community should come up with a > good microservices architecture. As you pointed out, it could be > based on vibed. Pity, microservices is a very fashionable thing just now, and Go is wiping the floor with Node and Java. Well that bit is opinion but… many people are getting into all this non-blocking, event-driven, shared memory stuff and boiling their brains, whereas the Go folk are doing blocking stuff using dataflow which is much easier to program. -- Russel. = Dr Russel Winder t:+44 20 7585 2200 voip:sip: russel.win...@ekiga.net 41 Buckmaster Road m:+44 7770 465 077 xmpp:rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype:russel_winder signature.asc Description: This is a digitally signed message part
Re: D and microservices
On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote: Has anyone got a small example of microservices using D, with Vibe.d or otherwise, that I can make use of? I need some examples of small microservices for a session at μCon 2015. What do you mean by microservice examples? It is infrastructure methodology, not specific code thing, any simple network service can be viewed as microservice.
Re: D and microservices
On 10/06/2015 01:54 PM, Russel Winder via Digitalmars-d wrote:
On Tue, 2015-10-06 at 16:21 +, Dejan Lekic via Digitalmars-d wrote:
On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote:
Has anyone got a small example of microservices using D, with
Vibe.d or otherwise, that I can make use of? I need some
examples of small microservices for a session at μCon 2015.
As far as I know, there is no implementation of microservices as
we see in the Java world. IMHO, D community should come up with a
good microservices architecture. As you pointed out, it could be
based on vibed.
Pity, microservices is a very fashionable thing just now, and Go is wiping
the floor with Node and Java. Well that bit is opinion but… many people are
getting into all this non-blocking, event-driven, shared memory stuff and
boiling their brains, whereas the Go folk are doing blocking stuff using
dataflow which is much easier to program.
Felt stupid for not being hip to this "microservices" thing you say, so
just looked it up. But it sounds to me like it's basically just a
buzz-driven rediscovery of the basic principles of proper encapsulation
and Unix philosophy ("do one thing and do it well").
(Kinda like how "cloud" sounds like a big fancy new revolution until you
realize it's just the hip new word for "internet" or "hosted". Or
"Facade design pattern" vs plain old "It's a thin wrapper".)
Does that sound about accurate, or am I missing something?
Re: D and microservices
On Tuesday, 6 October 2015 at 19:07:32 UTC, Nick Sabalausky wrote:
On 10/06/2015 01:54 PM, Russel Winder via Digitalmars-d wrote:
On Tue, 2015-10-06 at 16:21 +, Dejan Lekic via
Digitalmars-d wrote:
On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder
wrote:
Has anyone got a small example of microservices using D, with
Vibe.d or otherwise, that I can make use of? I need some
examples of small microservices for a session at μCon 2015.
As far as I know, there is no implementation of microservices
as
we see in the Java world. IMHO, D community should come up
with a
good microservices architecture. As you pointed out, it could
be
based on vibed.
Pity, microservices is a very fashionable thing just now, and
Go is wiping
the floor with Node and Java. Well that bit is opinion but…
many people are
getting into all this non-blocking, event-driven, shared
memory stuff and
boiling their brains, whereas the Go folk are doing blocking
stuff using
dataflow which is much easier to program.
Felt stupid for not being hip to this "microservices" thing you
say, so just looked it up. But it sounds to me like it's
basically just a buzz-driven rediscovery of the basic
principles of proper encapsulation and Unix philosophy ("do one
thing and do it well").
(Kinda like how "cloud" sounds like a big fancy new revolution
until you realize it's just the hip new word for "internet" or
"hosted". Or "Facade design pattern" vs plain old "It's a thin
wrapper".)
Does that sound about accurate, or am I missing something?
a half of it is the buzz and other half of is not. remember
people talking about reactjs, go and rails being buzz? they were
the same. we have built an online payment gateway and we are
about to decouple our application and switch to microservices
architecture. we have an api, a dashboard, a checkout page,
mobile flow. we have to deal with accounting and reporting as
well. and there is no way that this application will turn into a
giant monolith. i don't want that. nobody wants that. it will
become something we cannot handle.
another thing is whenever we do deployments we have to take down
the whole application and go offline. i know there are other
workarounds but when we only want to deploy mobile payments or
the api, other functionalities should continue working and our
customers should be able to pay.
nobody suggests starting with microservices architecture because
you'll never know where things will lead you however when it
becomes a giant the suggestion is to use microservices.
one other benefit of using this microservice is that you don't
have to look for specific language programmers. you only need to
hire good programmers as the only requirement is to do one thing
and and doing it well. the rest is about communication.
Re: D and microservices
On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote: Has anyone got a small example of microservices using D, with Vibe.d or otherwise, that I can make use of? I need some examples of small microservices for a session at μCon 2015. on your email, in case it fits. nanomsg and a little vibed
Re: D and microservices
On Tuesday, 6 October 2015 at 19:31:20 UTC, Mengu wrote: a half of it is the buzz and other half of is not. remember people talking about reactjs, go and rails being buzz? they were the same. we have built an online payment gateway and we are about to decouple our application and switch to microservices architecture. we have an api, a dashboard, a checkout page, mobile flow. we have to deal with accounting and reporting as well. and there is no way that this application will turn into a giant monolith. i don't want that. nobody wants that. it will become something we cannot handle. now a question for you. do you wish you had built it from components from day one? or do you see creating it as a blob to start with and then when the right divisions are clear factoring these out into micro-services as simply a natural part of the design process? because you know much more by having started, and it's not so hard to refactor at this stage. another thing is whenever we do deployments we have to take down the whole application and go offline pretend I'm asking you before it was deployed in production... nobody suggests starting with microservices architecture because you'll never know where things will lead you however when it becomes a giant the suggestion is to use microservices. some people do. but I would have thought the point I made above is the real reason. it doesn't take very long to write it that way from the beginning IFF you know what you want it to look like before you start. and maybe you don't. but I am interested in what your experience has been.
Re: D and microservices
On Tuesday, 6 October 2015 at 16:12:12 UTC, Russel Winder wrote: Has anyone got a small example of microservices using D, with Vibe.d or otherwise, that I can make use of? I need some examples of small microservices for a session at μCon 2015. As far as I know, there is no implementation of microservices as we see in the Java world. IMHO, D community should come up with a good microservices architecture. As you pointed out, it could be based on vibed.
