Re: D for microservices
On 10/23/2017 06:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Mysql-native is vibe.d-compatible, but also works when you have zero vibe.d dependencies (in which case it switches to Phobos's sockets instead of using Vibe's sockets.)
Re: D for microservices
On 05/05/2018 02:30 PM, Nick Sabalausky (Abscissa) wrote: On 10/23/2017 06:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Mysql-native is vibe.d-compatible, but also works when you have zero vibe.d dependencies (in which case it switches to Phobos's sockets instead of using Vibe's sockets.) Sorry, didn't notice this was a half-year-old thread.
Re: D for microservices
On Sunday, 11 March 2018 at 11:10:09 UTC, Tamas wrote: On Thursday, 8 March 2018 at 22:53:11 UTC, aberba wrote: On Thursday, 8 March 2018 at 09:35:08 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB. Building directly in alpine will simplify automated builds too. Simple Vibe.d app talking to Redis, packed into docker containers: https://github.com/tam4s/hello-redis The takeaway is that I could not use Alpine as a host image, because I could not build the app statically on ubuntu. warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking I've put up a new binary release of ldc 1.9 for Alpine, which fixes the vibe.d issues by pulling in two upstream commits from druntime, adds druntime and Phobos as shared libraries, and includes dub and rdmd: https://github.com/ldc-developers/ldc/releases I tested by unpacking that release in an Alpine VPS, adding its bin/ to my path, and simply running the following: dub fetch vibe-d dub build vibe-d The list of Alpine packages needed to run ldc are listed in the release notes. You can also cross-compile using the regular linux build of ldc by using ldc-build-runtime, the included tool to rebuild the stdlib for other platforms, and these instructions from the wiki: https://wiki.dlang.org/Building_LDC_runtime_libraries#Usage_for_cross-compilation I'm looking at adding a flag to dmd to enable building for the Musl C runtime: https://github.com/dlang/dmd/pull/8020 While the Musl port is mostly there, it appears that yshui didn't bother porting all of druntime, so there may still be other dub packages that need other missing C declarations. However, I'm done working on this port, beyond finishing off the above pull and the druntime pull linked from the release notes, as I don't use Alpine, containers, or microservices. I simply chipped in because I have porting experience and thought I could push D for microservices a bit farther along. I'll keep putting out official builds of ldc for Alpine though, as long as there's demand for them. If you'd like to use D in Alpine containers, now's the time to contribute to whatever else is missing. A good idea of the current status can be found in the first post of the dmd pull linked above. This may be a good project for the hackathon, as it's mostly polishing up a bunch of small things.
Re: D for microservices
On 2018-03-11 12:10, Tamas wrote: Simple Vibe.d app talking to Redis, packed into docker containers: https://github.com/tam4s/hello-redis The takeaway is that I could not use Alpine as a host image, because I could not build the app statically on ubuntu. warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking It's just a warning. If you're not using "dlopen" or any of your dependencies you can just ignore the warning. -- /Jacob Carlborg
Re: D for microservices
On Thursday, 8 March 2018 at 22:53:11 UTC, aberba wrote: On Thursday, 8 March 2018 at 09:35:08 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB. Building directly in alpine will simplify automated builds too. Simple Vibe.d app talking to Redis, packed into docker containers: https://github.com/tam4s/hello-redis The takeaway is that I could not use Alpine as a host image, because I could not build the app statically on ubuntu. warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Re: D for microservices
On Thursday, 8 March 2018 at 09:35:08 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB. Building directly in alpine will simplify automated builds too.
Re: D for microservices
On Thursday, 8 March 2018 at 09:28:02 UTC, Andrew Benton wrote: On Thursday, 8 March 2018 at 08:07:23 UTC, Joakim wrote: [...] I built out LDC on Alpine using the v1.8.0 LDC release. It's available now through Docker Hub and Github. Docker: https://hub.docker.com/r/andrewbenton/alpine-ldc/ Github: https://github.com/andrewbenton/alpine-ldc/ I get errors like below when trying to compile dub-registry: [...] OK you found an issue :). Indeed the file only covered Glibc so that's why vibe.d didn't compile for Musl. I fixed the issue for uClibc (1) and it works now (builds and runs). Probably the fix for Musl is similar. 1. https://github.com/dlang/druntime/pull/2134
Re: D for microservices
On Thursday, 8 March 2018 at 07:54:29 UTC, Jacob Carlborg wrote: On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg Looks like that works out much nicer than using alpine as the first stage right now. I'm not sure anyone will be too upset about the extra 7MB.
Re: D for microservices
On Thursday, 8 March 2018 at 08:07:23 UTC, Joakim wrote: On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [snip] What is the exact error? Maybe report it here: https://github.com/lindt/docker-dmd/issues/1 I built out LDC on Alpine using the v1.8.0 LDC release. It's available now through Docker Hub and Github. Docker: https://hub.docker.com/r/andrewbenton/alpine-ldc/ Github: https://github.com/andrewbenton/alpine-ldc/ I get errors like below when trying to compile dub-registry: /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/dns.d(11,9): Error: module core.sys.posix.netdb import 'AI_ADDRCONFIG' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/dns.d(11,9): Error: module core.sys.posix.netdb import 'AI_V4MAPPED' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(13,9): Error: module core.sys.posix.netdb import 'AI_ADDRCONFIG' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(13,9): Error: module core.sys.posix.netdb import 'AI_V4MAPPED' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(47,10): Error: module core.sys.linux.netinet.in_ import 'IP_ADD_MEMBERSHIP' not found /root/.dub/packages/eventcore-0.8.28/eventcore/source/eventcore/drivers/posix/sockets.d(47,10): Error: module core.sys.linux.netinet.in_ import 'IP_MULTICAST_LOOP' not found, did you mean enum member 'IPV6_MULTICAST_LOOP'? ldc2 failed with exit code 1.
Re: D for microservices
On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [snip] This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Unfortunately the link that's associated with is based on an ubuntu build which should be using glibc, not alpine w/ musl.
Re: D for microservices
On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [...] I took a shot at dockerizing dub-registry with the alpine build using multistage builds. I got ldc installed and dub built. Unfortunately when I tried to build dub-registry, I found that vibe-d's vibe-core, libevent, and libasync configurations can't run under musl due to some of the networking dependencies in druntime core, e.g. `core.sys.linux.netinet.in_`. I realize that's something that you can't really prevent as the packager, but it's still relevant to the status of D for microservices. Bad news: not being able to build vibe.d is a pretty big hit for this objective. Good news: there are other libraries we can try and we know how to fix this problem. What is the exact error? Maybe report it here: https://github.com/lindt/docker-dmd/issues/1
Re: D for microservices
On Thursday, 8 March 2018 at 07:20:53 UTC, Radu wrote: This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org Yes, it's pretty straightforward: 1. Build on Ubuntu, or some other dist 2. Statically link the whole binary with LDC, then you don't need to use Musl 3. Run it on Alpine -- /Jacob Carlborg
Re: D for microservices
On Thursday, 8 March 2018 at 06:49:07 UTC, Andrew Benton wrote: On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: [...] I took a shot at dockerizing dub-registry with the alpine build using multistage builds. I got ldc installed and dub built. Unfortunately when I tried to build dub-registry, I found that vibe-d's vibe-core, libevent, and libasync configurations can't run under musl due to some of the networking dependencies in druntime core, e.g. `core.sys.linux.netinet.in_`. I realize that's something that you can't really prevent as the packager, but it's still relevant to the status of D for microservices. Bad news: not being able to build vibe.d is a pretty big hit for this objective. Good news: there are other libraries we can try and we know how to fix this problem. This guys says that vide.d works https://forum.dlang.org/thread/gikoeutmdyvolfshp...@forum.dlang.org
Re: D for microservices
On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: On Monday, 5 March 2018 at 14:34:44 UTC, aberba wrote: [snip] The Alpine build is up, let me know if you have any problems. Note the changelog entry that says you'll need to install llvm and maybe other packages from the Alpine package manager first. I took a shot at dockerizing dub-registry with the alpine build using multistage builds. I got ldc installed and dub built. Unfortunately when I tried to build dub-registry, I found that vibe-d's vibe-core, libevent, and libasync configurations can't run under musl due to some of the networking dependencies in druntime core, e.g. `core.sys.linux.netinet.in_`. I realize that's something that you can't really prevent as the packager, but it's still relevant to the status of D for microservices. Bad news: not being able to build vibe.d is a pretty big hit for this objective. Good news: there are other libraries we can try and we know how to fix this problem.
Re: D for microservices
On Monday, 5 March 2018 at 17:58:25 UTC, Joakim wrote: On Monday, 5 March 2018 at 14:34:44 UTC, aberba wrote: On Sunday, 25 February 2018 at 22:12:38 UTC, Joakim wrote: On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: [...] I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: [...] OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release. LDC 1.8 is out! The Alpine build is up, let me know if you have any problems. Note the changelog entry that says you'll need to install llvm and maybe other packages from the Alpine package manager first. Nice. I usually use the docker images though.
Re: D for microservices
On Monday, 5 March 2018 at 14:34:44 UTC, aberba wrote: On Sunday, 25 February 2018 at 22:12:38 UTC, Joakim wrote: On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks. OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release. LDC 1.8 is out! The Alpine build is up, let me know if you have any problems. Note the changelog entry that says you'll need to install llvm and maybe other packages from the Alpine package manager first.
Re: D for microservices
On Sunday, 25 February 2018 at 22:12:38 UTC, Joakim wrote: On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks. OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release. LDC 1.8 is out!
Re: D for microservices
On 2018-02-25 17:51, yawniek wrote: Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? Build a completely statically linked binary by compiling using LDC and add the "-static" flag. -- /Jacob Carlborg
Re: D for microservices
On Sunday, 25 February 2018 at 16:51:09 UTC, yawniek wrote: great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container? I don't know, presumably you're referring to the static linking support Jacob mentioned earlier in this thread. I have not tried that. On Sunday, 25 February 2018 at 17:48:34 UTC, aberba wrote: I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks. OK, I will look at releasing a native ldc binary for Alpine with the upcoming 1.8 release.
Re: D for microservices
On Saturday, 24 February 2018 at 10:03:07 UTC, Joakim wrote: On Thursday, 22 February 2018 at 21:59:27 UTC, aberba wrote: On Thursday, 22 February 2018 at 08:54:04 UTC, Joakim wrote: [...] That makes me very happy!! Very much appreciated. I've updated ldc master to use Yuxuan's druntime port, which means the upcoming ldc 1.8 release will likely be a viable Alpine/Musl cross-compiler out of the box, provided you give it a C/Musl cross-compiler/linker to work with, by using the bundled ldc-build-runtime tool: https://wiki.dlang.org/Building_LDC_runtime_libraries Cross-compiling has not been tested however: do people normally cross-compile to Alpine containers or build their software in Alpine itself? If the latter, building ldc from master on Alpine/x64 3.7.0 passes all of the druntime/phobos stdlib unit tests but one, so you're good to go there, as long as you don't need a pre-built binary. Maybe we'll put one out for ldc with the upcoming 1.8 release. I usually ship and compile code in Alpine itself. Once I have an ldc compiler with Alpine as base image, I'm good to go. Some platforms like OpenShift will rebuild when a release is triggered in git master... Copying binary require some hacks.
Re: D for microservices
On Saturday, 24 February 2018 at 10:03:07 UTC, Joakim wrote: I've updated ldc master to use Yuxuan's druntime port, which means the upcoming ldc 1.8 release will likely be a viable Alpine/Musl cross-compiler out of the box, provided you give it a C/Musl cross-compiler/linker to work with, by using the bundled ldc-build-runtime tool: https://wiki.dlang.org/Building_LDC_runtime_libraries Cross-compiling has not been tested however: do people normally cross-compile to Alpine containers or build their software in Alpine itself? If the latter, building ldc from master on Alpine/x64 3.7.0 passes all of the druntime/phobos stdlib unit tests but one, so you're good to go there, as long as you don't need a pre-built binary. Maybe we'll put one out for ldc with the upcoming 1.8 release. great stuff, thank you! this will be very useful! Q: what would be needed to build a single binary (a la golang) that works in a FROM SCRATCH docker container?
Re: D for microservices
On Thursday, 22 February 2018 at 21:59:27 UTC, aberba wrote: On Thursday, 22 February 2018 at 08:54:04 UTC, Joakim wrote: On Monday, 23 October 2017 at 18:18:28 UTC, Jacob Carlborg wrote: [...] Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices. That makes me very happy!! Very much appreciated. I've updated ldc master to use Yuxuan's druntime port, which means the upcoming ldc 1.8 release will likely be a viable Alpine/Musl cross-compiler out of the box, provided you give it a C/Musl cross-compiler/linker to work with, by using the bundled ldc-build-runtime tool: https://wiki.dlang.org/Building_LDC_runtime_libraries Cross-compiling has not been tested however: do people normally cross-compile to Alpine containers or build their software in Alpine itself? If the latter, building ldc from master on Alpine/x64 3.7.0 passes all of the druntime/phobos stdlib unit tests but one, so you're good to go there, as long as you don't need a pre-built binary. Maybe we'll put one out for ldc with the upcoming 1.8 release.
Re: D for microservices
On 2018-02-22 09:54, Joakim wrote: Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices. That's great. -- /Jacob Carlborg
Re: D for microservices
On Thursday, 22 February 2018 at 08:54:04 UTC, Joakim wrote: On Monday, 23 October 2017 at 18:18:28 UTC, Jacob Carlborg wrote: [...] Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices. That makes me very happy!! Very much appreciated.
Re: D for microservices
On 22/02/2018 10:17 PM, Suliman wrote: It would be nice if anyone will rewrite Musl to betterC :) Combine it with dmc's libc and we're starting to get a reasonable state.
Re: D for microservices
It would be nice if anyone will rewrite Musl to betterC :)
Re: D for microservices
On Monday, 23 October 2017 at 18:18:28 UTC, Jacob Carlborg wrote: On 2017-10-23 17:35, Joakim wrote: I'm sure someone could put these together if the above stuff worked. The question is who's interested in volunteering to help put this all together? Yeah, exactly. My plate if already full and all this is lower down on the priority list. Yuxuan Shui has ported druntime to Musl over the last couple months: https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ayshui+is%3Aclosed With his changes, all druntime unit tests pass, now only a few asserts tripping in the additional tests for shared libraries. I added a couple more tweaks to get all but one of the Phobos unit tests passing too. With a last tiny patch to add a command-line flag to dmd to easily use this port- already added to the latest ldc 1.8 beta, though it doesn't have all the druntime support: https://github.com/ldc-developers/ldc/pull/2373 - anyone can use D with Alpine/Musl containers for microservices.
Re: D for microservices
On Saturday, 28 October 2017 at 14:55:25 UTC, aberba wrote: On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. Its the future. Highly doubt that. It really depend on the scale of your operations. Netflix, Google, Facebook, etc. all have open source tools around microservices. Its currently ruled by JavaScript > Go > Java. JavaScript being the leader. They have these in common: 1. Easy to deploy their code in docker containers including alpine Linux. Interestingly while Docker may seem all the rage in startups I find its use limited to test environments in the real world. Also Java fat jars were super easy to deploy ages before docker. They are also a great deal smaller. 2. They have APIs for major cloud services. Both official and third-party. 3. Good support for networking. HTTP, Websockets, IPC*, etc. Mostly HTTP. Honestly APIs these days can be written in anything that is able to put together a few HTTP responses. Unless you are doing serious work like: - DBs - Search engines - ML pipelines - Real-time event processing systems Any semimodern language/technology with a several hosts can manage to saturate 1Gbit link. Some take a certain amount of tuning others work out of the box. If you go for 40gbit/s it may be important to choose the right technology otherwise it’s all a matter of taste.
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. Its the future. Netflix, Google, Facebook, etc. all have open source tools around microservices. Its currently ruled by JavaScript > Go > Java. JavaScript being the leader. They have these in common: 1. Easy to deploy their code in docker containers including alpine Linux. 2. They have APIs for major cloud services. Both official and third-party. 3. Good support for networking. HTTP, Websockets, IPC*, etc. Mostly HTTP. That's it. The major advantages besides the hype. Lets see about D based on these requirements. 1. Kind of. There are dmd-ldc-dub docker images on docker hub. One by sociomantic. None using Alpine Linux as base though. Most people prefer alpine cus its lightweight (not a requirement). 2. Nope. Official APIs comes when there's mass adoption for that language and has good ROI. No complete and production ready cloud services API that I know of. Seems D users in that area roll out something on their own for private use with features they need. But most cloud PaaS provide support for custom runtimes which D has one for Heroku. Major candidate is Google AppEngine. 3. Phobos doesn't have a D HTTP API. Its sad but we have "requests" at code.dlang.org which works. We have vibe.d for http servers (aka nodejs of D) and seems popular based on threads about it and downloads. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? There several database APIs at cods.dlang.org. I don't know why some complain about no db APIs. This is a niche that D and all newer languages should target. How do we do it? Good question.
Re: D for microservices
On 10/27/17 00:18, Jacob Carlborg wrote: On 2017-10-26 12:25, Adam Wilson wrote: My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. My concern is not about the event loop, it's about asynchronous IO. vibe.d needs to use asynchronous IO and I assume that's regardless what kind of event loop implementation is used. Does all the existing database drivers that you want to use support asynchronous IO? PgSQL/MySQL/MSSQL all do, I think that covers about 90% of usage. IIRC Oracle does as well. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-26 12:25, Adam Wilson wrote: My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. My concern is not about the event loop, it's about asynchronous IO. vibe.d needs to use asynchronous IO and I assume that's regardless what kind of event loop implementation is used. Does all the existing database drivers that you want to use support asynchronous IO? -- /Jacob Carlborg
Re: D for microservices
On Thursday, October 26, 2017 19:19:57 Adam Wilson via Digitalmars-d wrote: > On 10/26/17 17:51, Jonathan M Davis wrote: > > On Thursday, October 26, 2017 03:25:24 Adam Wilson via Digitalmars-d wrote: > >> On 10/25/17 23:57, Jacob Carlborg wrote: > >>> I'm more concerned that I don't think we'll manage to implement a > >>> complete API and 100% bug free at the first try. > >> > >> Depends on how one defines first try. Phobos as a pretty solid process > >> for making sure these things are reasonably bug free. And near as I can > >> tell, Phobos is pretty good about accepting bug fixes. > > > > The bigger problem is API bugs. The review process is rigorous enough to > > weed out a lot of stuff, but the end result is typically an API that we > > _think_ looks good but usually hasn't seen much real world use (much as > > its design may be based on real world experience). And if it turns out > > that the API has problems, it can be very difficult to fix that in a > > way that doesn't break code. Sometimes, we're able to reasonably do > > something about it and sometimes not. In theory, std.experimental would > > mitigate that, and that's where anything that was accepted would go > > first, but the process for getting new modules into Phobos is very much > > geared towards getting an API right up front rather than getting > > something that's close and iterating towards where it ultimately should > > be. > > > > What would probably be better in general would be to be writing stuff > > that ends up on code.dlang.org first and gets some real world use there > > and then look at getting it into Phobos later rather than aiming > > directly for Phobos. Not only would that likely help lead towards > > better code being in Phobos, but we'd still get useful stuff even if it > > didn't make it into Phobos. > > > > - Jonathan M Davis > > I understand the concern, however, I can see potential mitigations. For > one, steal an API concept from somewhere else. I've had reasonable > success so far stealing ADO.NET and the refactoring it into something > more idiomatic. Using that as a starting point gave me a pretty good > understanding of what was needed and it gave me a prototype API that has > been battle-tested. > > Has anything from code.dlang.org been pulled into Phobos yet? I'm not > aware of anything. code.dlang.org is where Phobos projects go to quietly > die in obscurity. Nothing has been pulled into Phobos from anywhere in a while. The last thing was probably checkedint, and before that it was std.allocator - both of which came from Andrei. ndslice was in std.experimental for a little while but Ilya removed it, because he decided that it wasn't working to have it in Phobos, because he couldn't continue to improve on it there. Beyond that, I think that the last thing was the logger, which has just languished in std.experimental. It's my understanding that it needs some more work, but basically, once we added std.experimental, nothing has made it into Phobos proper. And over the last few years, there haven't been many attempts to get anything into Phobos, so not much has even made it into std.experimental. There was the GSoC project for a new XML parser, but that project seems to have died after the student got too busy with school after GSoC, and Sonke's std.json replacement didn't finish making its way through the review process, and I think that Sonke has given up on getting it in (if I understand correctly, there was just too much disagreement over what the std.json replacement should look like). Overall, people have just stopped trying to get major stuff into Phobos. New functions get added to existing modules, but pretty much no one seems to want to go through the Phobos review process to get anything into Phobos. code.dlang.org is where folks put anything that they're doing that they want to make available, and IIRC, the only item from there that's even attempted to make it into Phobos was Sonke's JSON parser. Much as some folks continue to talk about getting some piece of functionality into Phobos, no one is trying if it's anything major. So, it's not like Phobos projects are going to code.dlang.org to die. In general, they simply aren't even being attempted, and any serious projects that do exist are on code.dlang.org. Some do sit there unfinished (e.g. std_experimental_xml), but largely, the authors just don't want to go to the effort of getting the code into Phobos. And honestly, in general, at this point, I don't think that I'd want to bother either. It's quite a lot of work to get something through the Phobos review process, and once it's in Phobos, you lose control over it. If I work on something major, I can just put it up on code.dlang.org, and anyone who wants to use it can, and I have full control over what I do with the code base without having to get approval from Andrei or anyone else as to what I do with it. So, unless we're talking about something that practically needs to be in the standa
Re: D for microservices
On 10/26/17 17:51, Jonathan M Davis wrote: On Thursday, October 26, 2017 03:25:24 Adam Wilson via Digitalmars-d wrote: On 10/25/17 23:57, Jacob Carlborg wrote: I'm more concerned that I don't think we'll manage to implement a complete API and 100% bug free at the first try. Depends on how one defines first try. Phobos as a pretty solid process for making sure these things are reasonably bug free. And near as I can tell, Phobos is pretty good about accepting bug fixes. The bigger problem is API bugs. The review process is rigorous enough to weed out a lot of stuff, but the end result is typically an API that we _think_ looks good but usually hasn't seen much real world use (much as its design may be based on real world experience). And if it turns out that the API has problems, it can be very difficult to fix that in a way that doesn't break code. Sometimes, we're able to reasonably do something about it and sometimes not. In theory, std.experimental would mitigate that, and that's where anything that was accepted would go first, but the process for getting new modules into Phobos is very much geared towards getting an API right up front rather than getting something that's close and iterating towards where it ultimately should be. What would probably be better in general would be to be writing stuff that ends up on code.dlang.org first and gets some real world use there and then look at getting it into Phobos later rather than aiming directly for Phobos. Not only would that likely help lead towards better code being in Phobos, but we'd still get useful stuff even if it didn't make it into Phobos. - Jonathan M Davis I understand the concern, however, I can see potential mitigations. For one, steal an API concept from somewhere else. I've had reasonable success so far stealing ADO.NET and the refactoring it into something more idiomatic. Using that as a starting point gave me a pretty good understanding of what was needed and it gave me a prototype API that has been battle-tested. Has anything from code.dlang.org been pulled into Phobos yet? I'm not aware of anything. code.dlang.org is where Phobos projects go to quietly die in obscurity. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On Monday, October 23, 2017 12:13:12 Laeeth Isharc via Digitalmars-d wrote: > How can we generate a static binary ? I asked about this before, > and the response was that it's a bad idea because of security > vulns and so on. True if you are running on a conventional Linux > host. But on the other hand, it's not that great if when the > system phobos is upgraded all the D binaries break. You can > write a script using rdmd or dub scripting feature, but that > doesn't work for more complex programs. I believe that it works just fine if you do the linking step yourself, but dmd doesn't properly handle it with -L-static: https://issues.dlang.org/show_bug.cgi?id=6952 I have no idea how easy it would be to the linking manually with dub. Alternatively, I believe that you can statically link against any 3rd party libraries that you're using by linking to the .a file rather than by the library name. That won't work for glibc, but much as I'd love to statically link that, it's my understanding that doing so is generally a bad idea. I don't recall the reasons though - and with the increased stability in linux's ABI, I would have thought that it would be far less of an issue than it would have been in the past. Certainly, the concept of being able to statically link a program and then just run it on most any recent linux distro without recompiling it sounds very appealing. Yes, you'll have to recompile it for any security updates to any of your dependencies, whereas that can be avoided with dynamic linking, but depending on what you're doing, that often isn't really an issue, and it can be managed if it is (heck, you're doing basically the same thing when maintaining something like a docker image). - Jonathan M Davis
Re: D for microservices
On Thursday, October 26, 2017 03:25:24 Adam Wilson via Digitalmars-d wrote: > On 10/25/17 23:57, Jacob Carlborg wrote: > > I'm more concerned that I don't think we'll manage to implement a > > complete API and 100% bug free at the first try. > > Depends on how one defines first try. Phobos as a pretty solid process > for making sure these things are reasonably bug free. And near as I can > tell, Phobos is pretty good about accepting bug fixes. The bigger problem is API bugs. The review process is rigorous enough to weed out a lot of stuff, but the end result is typically an API that we _think_ looks good but usually hasn't seen much real world use (much as its design may be based on real world experience). And if it turns out that the API has problems, it can be very difficult to fix that in a way that doesn't break code. Sometimes, we're able to reasonably do something about it and sometimes not. In theory, std.experimental would mitigate that, and that's where anything that was accepted would go first, but the process for getting new modules into Phobos is very much geared towards getting an API right up front rather than getting something that's close and iterating towards where it ultimately should be. What would probably be better in general would be to be writing stuff that ends up on code.dlang.org first and gets some real world use there and then look at getting it into Phobos later rather than aiming directly for Phobos. Not only would that likely help lead towards better code being in Phobos, but we'd still get useful stuff even if it didn't make it into Phobos. - Jonathan M Davis
Re: D for microservices
On 26/10/2017 11:25 AM, Adam Wilson wrote: On 10/25/17 23:57, Jacob Carlborg wrote: On 2017-10-26 00:53, Adam Wilson wrote: This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I'm entirely against anything that is not compatible with vibe.d ;) My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. (Something about not supporting GUI loops, paging Mr. Cattermole). If that is really the case I don't see how being entirely vibe.d compatible and meeting the universal standard requirements of Phobos is possible. There would need to be a requirements gathering phase so that the community as a whole can bring their use-cases before we dove into code. The problem isn't the event loop design. Its a fairly solved problem. The way vibe.d's works is very specific to their use case (which isn't wrong if you only consider them). You can't 'hook' into it. Which makes it very undesirable for Phobos. Since it won't cover most use cases. Even if the source they are using is compatible with an external GUI event loop (which it should be for Windows from what I've read). So I wouldn't be starting with vibe.d's event loop model. Quite to the contrary, kill it. Build something that will last throughout the ages for everyone and put this problem to rest.
Re: D for microservices
On 10/25/17 23:57, Jacob Carlborg wrote: On 2017-10-26 00:53, Adam Wilson wrote: This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I'm entirely against anything that is not compatible with vibe.d ;) My apologies, something rather the other direction. Instead of forcing compat with vibe.d, going to vibe.d and say: "here is our standard event-loop, it has everything you need, you'll need to use it for all the other goodness to work". I know others can make good arguments about why the vibe event-loop is insufficient, and I'll let them make them. (Something about not supporting GUI loops, paging Mr. Cattermole). If that is really the case I don't see how being entirely vibe.d compatible and meeting the universal standard requirements of Phobos is possible. There would need to be a requirements gathering phase so that the community as a whole can bring their use-cases before we dove into code. I actually don't think the slow updates are huge problem as these DB interface libraries are pretty slow to change themselves. For example, ADO.NET didn't change significantly from it's 1.0 release until the arrival of Async/Await in .NET 4.5, which was 10 years later. The biggest addition prior to Async was TVP support and that was tiny comparatively and came in 2005. The libpq5 interface has barely changed in years. I can't speak to MySQL but IIRC it hasn't changed much either, at least not in a way that would effect the abstraction layer. I'm more concerned that I don't think we'll manage to implement a complete API and 100% bug free at the first try. Depends on how one defines first try. Phobos as a pretty solid process for making sure these things are reasonably bug free. And near as I can tell, Phobos is pretty good about accepting bug fixes. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-26 00:53, Adam Wilson wrote: This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I'm entirely against anything that is not compatible with vibe.d ;) I actually don't think the slow updates are huge problem as these DB interface libraries are pretty slow to change themselves. For example, ADO.NET didn't change significantly from it's 1.0 release until the arrival of Async/Await in .NET 4.5, which was 10 years later. The biggest addition prior to Async was TVP support and that was tiny comparatively and came in 2005. The libpq5 interface has barely changed in years. I can't speak to MySQL but IIRC it hasn't changed much either, at least not in a way that would effect the abstraction layer. I'm more concerned that I don't think we'll manage to implement a complete API and 100% bug free at the first try. -- /Jacob Carlborg
Re: D for microservices
On Monday, 23 October 2017 at 12:13:12 UTC, Laeeth Isharc wrote: ... And on the other hand, for deployment, it's much easier to copy over one binary. (In a sense it's funny that the question was asked in the context of containers, because containers are kind of an alternative to having a single binary). When you're properly set-up of course it doesn't matter, but when you're moving towards that, a single binary is much easier. ... A bit OT but this part made me think of [1] of which you may appreciate the insight. Right now I kind of feel like with D we get the downside in size of fat binaries without the upside in portability (although solutions exist for the dedicated and patient ones). This might proove an important drawback in the future... we'll see. [1]: http://www.smashcompany.com/technology/why-would-anyone-choose-docker-over-fat-binaries
Re: D for microservices
On 10/23/17 23:29, Jacob Carlborg wrote: On 2017-10-24 00:02, Adam Wilson wrote: I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. It can be an optional dependency. Looking at ddb [1] it's not that much code that will be different if vibe.d is used or not. It's basically only reading and writing to the socket that is different [2]. Dub provides predefined version for different dependencies, i.e. Have_vibe_d_core. This of course makes the assumption that we clean-room our own protocol implementations which I am entirely against. Better to use what already exists. I had heard noises about that a few months ago. Anything happening on that front? No, not as far as I know. Sönke seems really busy recently. That is what I was afraid of. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? I would be really nice to database support directly in the standard library but it's not critical for me. It takes a lot of work with and massive overhead to get something into Phobos. It's also going to be really slow get in updates. I'm not sure if it's worth it. [1] https://github.com/pszturmaj/ddb [2] https://github.com/pszturmaj/ddb/blob/master/source/ddb/postgres.d#L189-L246 I actually don't think the slow updates are huge problem as these DB interface libraries are pretty slow to change themselves. For example, ADO.NET didn't change significantly from it's 1.0 release until the arrival of Async/Await in .NET 4.5, which was 10 years later. The biggest addition prior to Async was TVP support and that was tiny comparatively and came in 2005. The libpq5 interface has barely changed in years. I can't speak to MySQL but IIRC it hasn't changed much either, at least not in a way that would effect the abstraction layer. -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 10/23/17 18:51, rikki cattermole wrote: On 23/10/2017 11:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. I had heard noises about that a few months ago. Anything happening on that front? An event loop is a key piece of a project (Async/Await) that I want to work on, and having it in DRuntime would make that project fantastically simpler. IMHO, the bulk of the time required is in getting an event loop into DRT, the rest is a LOT of relatively straightforward compiler lowering. IMHO, DRT is in significant need of an event loop system. This would allow us to simplify a large number of problems (Async/Await, GUI's, IO, etc). As near as I can tell, the problem isn't so much doing the work, but getting the required sign-off's for inclusion into DRT. Another problem that I've been made aware of is that vibe-core may not be ideal in certain situations. As this would be landed in DRT itself this would obviously need to be addressed. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? *whispers* heyyy, heard about SPEW[0]? [0] https://github.com/Devisualization/spew/blob/master/src/base/cf/spew/event_loop/defs.d You've mentioned it a few times. ;-) -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-24 00:02, Adam Wilson wrote: I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. It can be an optional dependency. Looking at ddb [1] it's not that much code that will be different if vibe.d is used or not. It's basically only reading and writing to the socket that is different [2]. Dub provides predefined version for different dependencies, i.e. Have_vibe_d_core. I had heard noises about that a few months ago. Anything happening on that front? No, not as far as I know. Sönke seems really busy recently. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? I would be really nice to database support directly in the standard library but it's not critical for me. It takes a lot of work with and massive overhead to get something into Phobos. It's also going to be really slow get in updates. I'm not sure if it's worth it. [1] https://github.com/pszturmaj/ddb [2] https://github.com/pszturmaj/ddb/blob/master/source/ddb/postgres.d#L189-L246 -- /Jacob Carlborg
Re: D for microservices
On 23/10/2017 11:02 PM, Adam Wilson wrote: On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. I had heard noises about that a few months ago. Anything happening on that front? An event loop is a key piece of a project (Async/Await) that I want to work on, and having it in DRuntime would make that project fantastically simpler. IMHO, the bulk of the time required is in getting an event loop into DRT, the rest is a LOT of relatively straightforward compiler lowering. IMHO, DRT is in significant need of an event loop system. This would allow us to simplify a large number of problems (Async/Await, GUI's, IO, etc). As near as I can tell, the problem isn't so much doing the work, but getting the required sign-off's for inclusion into DRT. Another problem that I've been made aware of is that vibe-core may not be ideal in certain situations. As this would be landed in DRT itself this would obviously need to be addressed. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? *whispers* heyyy, heard about SPEW[0]? [0] https://github.com/Devisualization/spew/blob/master/src/base/cf/spew/event_loop/defs.d
Re: D for microservices
On 10/23/17 05:08, Jacob Carlborg wrote: * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library I've been looking pretty extensively at these two items recently. If the database drivers are compatible with Vibe.d AND we wish to provide a common abstraction layer for them (presumably via Phobos) then order for the abstraction layer to aware of the whether the driver is making a blocking or non-blocking call we must include Vibe.D in the abstraction layer. Ergo, we must include at least the vibe-core package in Phobos, or more preferably, DRT. I had heard noises about that a few months ago. Anything happening on that front? An event loop is a key piece of a project (Async/Await) that I want to work on, and having it in DRuntime would make that project fantastically simpler. IMHO, the bulk of the time required is in getting an event loop into DRT, the rest is a LOT of relatively straightforward compiler lowering. IMHO, DRT is in significant need of an event loop system. This would allow us to simplify a large number of problems (Async/Await, GUI's, IO, etc). As near as I can tell, the problem isn't so much doing the work, but getting the required sign-off's for inclusion into DRT. Another problem that I've been made aware of is that vibe-core may not be ideal in certain situations. As this would be landed in DRT itself this would obviously need to be addressed. What would the appetite be for working together to come up with a reasonably generic event loop for DRT that vibe and other systems could then leverage? -- Adam Wilson IRC: LightBender import quiet.dlang.dev;
Re: D for microservices
On 2017-10-23 14:17, Laeeth Isharc wrote:
Can you elaborate on how the TLS implementation needs to be changed?
# echo 'void main() {}' > main.d && dmd -c main.d && gcc main.o -o main
-m64 -static -L/root/.dvm/compilers/dmd-2.076.1/linux/bin/../lib64
-Xlinker -Bstatic -lphobos2 -lpthread -lm -lrt -ldl
/root/.dvm/compilers/dmd-2.076.1/linux/bin/../lib64/libphobos2.a(sections_elf_shared_782_420.o):
In function `_D2rt19sections_elf_shared11getTLSRangeFNbNimmZAv':
src/rt/sections_elf_shared.d:(.text._D2rt19sections_elf_shared11getTLSRangeFNbNimmZAv[_D2rt19sections_elf_shared11getTLSRangeFNbNimmZAv]+0x38):
undefined reference to `__tls_get_addr'
collect2: error: ld returned 1 exit status
I need to manually invoke GCC to link because DMD will pass "-Xlinker
--export-dynamic -Xlinker -Bdynamic" and I need to add "-static".
If someone wanted to work on rabbit bindings/wrapper, I might be open to
sponsoring that. I'm not in love with Rabbit (one node uses more than
40% of memory so the node goes down, taking the cluster with it.
really?), but we use it currently.
I made a start on bindings for librabbitmq here:
https://github.com/kaleidicassociates/rabbitmq-d
Is that compatible with vibe.d?
--
/Jacob Carlborg
Re: D for microservices
On 2017-10-23 14:13, Laeeth Isharc wrote: How can we generate a static binary ? It's already supported by LDC, using the -static flag. This Linux binary [1] is statically linked. [1] https://github.com/jacob-carlborg/remarkify/releases -- /Jacob Carlborg
Re: D for microservices
On 2017-10-23 17:35, Joakim wrote: I'm sure someone could put these together if the above stuff worked. The question is who's interested in volunteering to help put this all together? Yeah, exactly. My plate if already full and all this is lower down on the priority list. -- /Jacob Carlborg
Re: D for microservices
On Monday, 23 October 2017 at 12:08:52 UTC, Jacob Carlborg wrote: On 2017-10-22 04:48, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? * Support full static linking using DMD, which requires the TLS implementation to be modified * Support musl as the standard C library, I've discussed that before [1] I can probably whip this together in a couple days, as I added the Bionic support before, but it's not something I need, so it'd just be a donation for you and others. That's why I'm trying to figure out who exactly wants this and if others want to chip in. * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library * RabbitMQ library compatible with vibe.d * Serialization to/from JSON and YAML Some of this stuff is on dub, but nothing that interests me or that I'd chip in with. Good to see that RabbitMQ supports binary protocols like AMQP and MQTT though, would be nuts if it were all just text like STOMP or especially JSON/YAML. * Official Docker images with DMD and LDC wouldn't hurt * Pre-compiled DMD that works on Alpine. Fully statically linked DMD would help here I'm sure someone could put these together if the above stuff worked. The question is who's interested in volunteering to help put this all together?
Re: D for microservices
On Monday, 23 October 2017 at 12:08:52 UTC, Jacob Carlborg wrote: On 2017-10-22 04:48, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? * Support full static linking using DMD, which requires the TLS implementation to be modified * Support musl as the standard C library, I've discussed that before [1] * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library * RabbitMQ library compatible with vibe.d * Serialization to/from JSON and YAML * Official Docker images with DMD and LDC wouldn't hurt * Pre-compiled DMD that works on Alpine. Fully statically linked DMD would help here That's what I can think of for now. [1] http://forum.dlang.org/post/nhem1l$1ee3$1...@digitalmars.com Can you elaborate on how the TLS implementation needs to be changed? If someone wanted to work on rabbit bindings/wrapper, I might be open to sponsoring that. I'm not in love with Rabbit (one node uses more than 40% of memory so the node goes down, taking the cluster with it. really?), but we use it currently. I made a start on bindings for librabbitmq here: https://github.com/kaleidicassociates/rabbitmq-d Laeeth.
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? We're going a bit in that direction, although it's a different thing in our kind of industry from a web company - we have fewer services and many fewer requests, but latency matters more for example. I was thinking we might use Go for some services that integrate and monitor things, but we could also use D. https://jobs.github.com/positions/8e98eac8-b504-11e7-9da8-0737a3dcef18 From the comment: "Would be really cool to have LDC easily building alpine containers + static D binaries" How can we generate a static binary ? I asked about this before, and the response was that it's a bad idea because of security vulns and so on. True if you are running on a conventional Linux host. But on the other hand, it's not that great if when the system phobos is upgraded all the D binaries break. You can write a script using rdmd or dub scripting feature, but that doesn't work for more complex programs. And on the other hand, for deployment, it's much easier to copy over one binary. (In a sense it's funny that the question was asked in the context of containers, because containers are kind of an alternative to having a single binary). When you're properly set-up of course it doesn't matter, but when you're moving towards that, a single binary is much easier. I guess I can figure out the answer to this question easily enough, but I think giving people the option might help with adoption of D for micro services. For example it's really just not that fun to make an AWS Lambda using D - being able to easily build a static binary would make the process much more pleasant. I wrote this up a while back: http://awslambda-d.readthedocs.io/en/latest/ "Since dmd links phobos dynamically on linux, and phobos/druntime aren't installed on the AWS lambda server, we will need to upload these to the servers and tell the application where to find them. (I should really have appended to LD_LIBRARY_PATH as I did with PATH). Now one can follow the regular instructions for AWS Lambda: create a .zip file with the D binary, the JS file, and the following libraries (update version numbers as appropriate): libcrypto.so.1.0.0 libphobos2.so.0.67 libevent-2.0.so.5 libssl.so.1.0.0 " Alpine is nice - would be good to have this as a standard target in time. Laeeth.
Re: D for microservices
On 2017-10-22 04:48, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: "For now everything works(?) but I think the process could be improved.. Would be really cool to have LDC easily building alpine containers + static D binaries for microservice and tooling development. I'm pretty tired of reading Go code :)" https://github.com/ldc-developers/ldc/issues/2341#issuecomment-334626550 It strikes me that microservices are a great way for new programming languages like D to get tried and gain some uptake, but that D might not be that easy to deploy to that scenario yet. So this is a question for those deploying microservices, as I'm not in that field, what can the D devs do to make it as easy as possible to get D microservices up and running, make some Docker and Alpine containers with ldc/dub/vibe.d preinstalled publicly available? What else, what kinds of libraries do you normally use? This is a niche that D and all newer languages should target. How do we do it? * Support full static linking using DMD, which requires the TLS implementation to be modified * Support musl as the standard C library, I've discussed that before [1] * Database drivers for the common databases (PostgreSQL, MySQL, SQLite) compatible with vibe.d * Database driver abstraction on top of the above drivers, perhaps some lightweight ORM library * RabbitMQ library compatible with vibe.d * Serialization to/from JSON and YAML * Official Docker images with DMD and LDC wouldn't hurt * Pre-compiled DMD that works on Alpine. Fully statically linked DMD would help here That's what I can think of for now. [1] http://forum.dlang.org/post/nhem1l$1ee3$1...@digitalmars.com -- /Jacob Carlborg
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: [...] rock solid, easy, common-dev-proof, huge std lib. like that of golang.
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: This is a niche that D and all newer languages should target. How do we do it? Optimize the TechEmpower benchmark? Vibe.d looks quite weak there. https://www.techempower.com/benchmarks/
Re: D for microservices
On Sunday, 22 October 2017 at 02:48:57 UTC, Joakim wrote: I just read the following two week-old comment on the ldc issue tracker, when someone tried to run D on Alpine linux: [...] Rather a container with dub/ldc configured to compile with musl libc I think. What else, what kinds of libraries do you normally use? Given current selection I bet most are hand-rolled except maybe vibe.d and common serialization libs.
