Re: Answers needed from those using D for Web Development, Web APIs and Services
On Friday, 15 December 2017 at 08:13:25 UTC, aberba wrote: I'm going to do a writeup on the state of D in Web Development, APIs and Services for 2017. I need the perspective of the community too along with my personal experience. Please help out. More details the better. 0. Since when did you or company start using D in this area? I have been fooling around with D for web-related stuff for a year or so, but nothing terribly concrete. I've been actively migrating my RSS system to D mostly in the past month and a bit. 1. Do you use a framework? Which one? I use vibe.d. 2. Why that approach and what would have done otherwise? It was there. Without it, I probably would have looked at Hunt briefly and then cobbled together something based on CGI. 3. Which task exactly do you use D to accomplish? https://github.com/dhasenan/pierce/ I use D for all backend code, so: * reading feeds * mucking about with the database * authentication 4. Which (dub) packages do you use and for what purpose? * arsd-official:dom: an excellent HTML/XML parser * datefmt: date parsing, primarily, with a side of formatting * pbkdf2: password hashing * urld: to have consistent URL handling with other applications * vibe-d-postgresql: postgres 5. How do you host your software code (cloud platforms, vps, PaaS, docker, Openshift, kubernetes, etc)? I have a couple VPS boxes with linode that I deploy stuff to. PaaS providers do me a concern. Docker is 6. What are some constraints and problems in using D for such tasks? vibe.d is all non-blocking IO and it's not always easy to find a library that does a thing with non-blocking IO. You can dispatch tasks to a worker thread, but that uses D's `shared` system, which imposes barriers I'm not terribly familiar with. That also doesn't give you Futures. I believe my code still has blocking database access, at least in production. With my user volume (one for the D alpha, three for the C# version), this isn't yet a huge problem; however, I've had to fork a process for handling background tasks, since they can be long. dub doesn't know how to dynamically link dependencies. This means my binary is 38MB and takes half a minute to copy to the server. Since I worked out how to do dynamic linking manually, I'm going to add that in, and then I'll be able to rsync everything. That should reduce my application's binary size to a megabyte or less, which will be a lot nicer. vibe doesn't have a logging appender for std.experimental.logger, and vibe's logging system isn't terribly awesome. I wrote a vibe-compatible rolling file appender for std.experimental.logger in the end. It's kind of weird, though, that std.experimental.logger doesn't separate layout from the output type. It makes sense for some potential appenders to ignore the layout system you would give them -- like if you have an appender for some sort of structured logging API that accepts protobuf-encoded events. But most logs are just text, and if I don't like the layout, I need to write a whole logger. 7. What solutions do you recommend? vibe.d isn't a bad option. It's got a lot of stuff in it. However, it might be simpler on the whole to use fastcgi.
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Fri, Dec 22, 2017 at 06:21:04PM +, Russel Winder wrote: > On Fri, 2017-12-22 at 09:42 -0800, H. S. Teoh via Digitalmars-d wrote: > […] > > that are a pain to manage. Yes I know dub does it "automatically", > > but the problem with dub is that it tries to do too much -- it wants > > to be a build system in addition to being a packaging system. The > > former is OK, I guess, even though I really wish it was more > > configurable in terms of how it manages local repository caches. But > > as a build system, I'm sorry to say that dub sucks. Or at least, its > > docs suck, 'cos I can't figure out how to make it do what I want. > > After struggling with it for about a week or two, I threw in the > > towel and went back to SCons. Nowadays I only use dub for updating > > vibe.d via a dummy blank project. > > > […] > > Just to reiterate, SCons D support now has a ProgramAllAtOnce builder > for those that want to use Unit-Threaded in their D codebases using > SCons. For D projects, I've been finding that Command() has been the best tool for me in terms of configuring exactly how I want things built. I used to use (early versions of) your SCons D build tools (and thanks for that!), but ultimately went back to Command() because I found it very frustrating to have my builds break because of an incompatible change in the D tooling whenever I upgrade SCons. So until the SCons D tooling API has stabilized, I'll probably hold off for the time being. Also, for vibe.d projects, I've been finding the need to write my own scanner in order to pick up Diet template (*.dt) dependencies, so that builds would trigger correctly when Diet templates are changed. There is no standard way to do this, unfortunately; so far I've been scanning for `render!(.*)` lines, but this doesn't always work if `render` is instantiated with parameters generated from CTFE. Manual hardcoding has been necessary to get this part of my dependency tree to work. In the long term, I think an approach similar to tup will have to be adopted. O(n) dependency scanning just doesn't cut it anymore for code the size of today's large software projects. And with dynamic dependencies (e.g. CTFE-dependent imports) that are bound to happen in D code with heavy metaprogramming, there's really no sane way to manage dependencies explicitly; you really need to just instrument the compiler and record all input files it reads the way tup does. I shouldn't be needing to write custom scanners just to accomodate CTFE-generated imports that may change again after a few more commits. It's SSOT (single source of truth) all over again: the compiler is the ultimate authority that determines which file depends on what, and having to repeat this information in your build script (or independently derive it via scanners) introduces fragility / incompleteness into your build system. > Also I have the beginnings of a Dub SCons tool for using Dub as a > package manager in a SCons build. [...] That's nice, though for now, I'm sticking with manually updating my dependencies when needed. One thing I found annoying with dub was the sheer amount of time it spent at startup to scan all dependencies and packages and possibly downloading a whole bunch of stuff. The network latency really kills the compile-test-debug cycle time. I know there's a switch to suppress this behaviour, but the initial dependency scanning is still pretty slow even in spite of that. When a 1-line change requires waiting 15-20 seconds just to recompile, that really breaks my workflow. Plus, sometimes I *don't* want anything updated -- when debugging a program, the last thing I want is for dub or the build script or whatever to decide to link in a slightly different version of a library, and suddenly I'm no longer sure if the new crash is caused by the library or my own code, or the bug may now be masked by the slightly different behaviour of an upgraded library. I know that for people who want things done for them automatically and handed over on a silver platter, dub is great. Unfortunately, it doesn't work for me. (But I also know that I don't represent typical usage, so take all this with a grain of salt.) T -- Маленькие детки - маленькие бедки.
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Fri, 2017-12-22 at 09:42 -0800, H. S. Teoh via Digitalmars-d wrote: […] > that are a pain to manage. Yes I know dub does it "automatically", > but > the problem with dub is that it tries to do too much -- it wants to > be a > build system in addition to being a packaging system. The former is > OK, > I guess, even though I really wish it was more configurable in terms > of > how it manages local repository caches. But as a build system, I'm > sorry > to say that dub sucks. Or at least, its docs suck, 'cos I can't > figure > out how to make it do what I want. After struggling with it for about > a > week or two, I threw in the towel and went back to SCons. Nowadays I > only use dub for updating vibe.d via a dummy blank project. > […] Just to reiterate, SCons D support now has a ProgramAllAtOnce builder for those that want to use Unit-Threaded in their D codebases using SCons. Also I have the beginnings of a Dub SCons tool for using Dub as a package manager in a SCons build. Currently it does what I need, so until there are users requesting extensions and bug fixes it is "job done", though I may try to get enough tests to go for a pull request of the currently separate tool into the SCons distribution. https://github.com/russel/SCons_D_Experiment -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Friday, 22 December 2017 at 17:42:57 UTC, H. S. Teoh wrote: The main reason I went to vibe.d was because of HTTPS support, that cgi.d didn't have. You shouldn't be running a homemade HTTP server in public. The way you should do it is putting the application behind a real web server (you can do it with cgi mode or http using a reverse proxy, both of which cgi.d fully supports) which is responsible for the encryption.
Re: Answers needed from those using D for Web Development, Web APIs and Services
> On Friday, 15 December 2017 at 08:13:25 UTC, aberba wrote: > > 0. Since when did you or company start using D in this area? Unfortunately I'm presently not able to use D at my job. [...] > > 1. Do you use a framework? Which one? I've used Adam Ruppe's cgi.d for a bit, but recently switched to vibe.d. [...] > > 2. Why that approach and what would have done otherwise? I actually rather like cgi.d for its simplicity. Vibe.d is pretty cool and pretty powerful, but it comes with a complex set of dependencies that are a pain to manage. Yes I know dub does it "automatically", but the problem with dub is that it tries to do too much -- it wants to be a build system in addition to being a packaging system. The former is OK, I guess, even though I really wish it was more configurable in terms of how it manages local repository caches. But as a build system, I'm sorry to say that dub sucks. Or at least, its docs suck, 'cos I can't figure out how to make it do what I want. After struggling with it for about a week or two, I threw in the towel and went back to SCons. Nowadays I only use dub for updating vibe.d via a dummy blank project. The main reason I went to vibe.d was because of HTTPS support, that cgi.d didn't have. That, and also it was supposed to be the flagship web platform for D, so I figured I should at least give it a try. It does have some nice perks like less boilerplate for handling HTTP requests, I suppose, and Diet templates are kinda cool though also kinda klunky in certain details. The ubiquitous use of classes instead of structs rubbed me the wrong way somewhat, but I can live with it. Built-in support for databases was nice, but I didn't end up using it because (1) I needed a persistent database that's always consistent on-disk, so Redis was out, and I really don't like the idea of needing a separate database server just to run Mongo (managing a separate dependent service is way too much needless complexity for what I'm doing), so I went back to Adam Ruppe's sqlite.d instead. Overall, vibe.d is not bad. The docs could use improvement -- I struggled to find what I want for quite a few things, which wasted a lot of time. A lot of the frustration came from the docs being unclear about whether something was possible or not. Beyond the most basic examples, many things were implicit, or just plain not stated, leading to me spending far too much time trying to figure out whether module X supported feature Y, or if I should look elsewhere or write my own. I'm OK with writing a feature myself, but it's frustrating when I don't even know whether I need to. Other than that, though, vibe.d performance is pretty good and it does what I need, with a number of nice syntactic shortcuts to reduce boilerplate. So overall, I'm relatively happy with it. [...] > > 3. Which task exactly do you use D to accomplish? Just serving dynamic webpages on server-side. Basically as a PHP replacement. [...] > > 4. Which (dub) packages do you use and for what purpose? Just vibe.d and its dependencies. I struggled to make dub do what I want, so in general I avoided using it, sad to say. Currently the only thing I use dub for is to update vibe.d, via a dummy empty package that declares dependency on vibe.d. Once vibe.d is built, I link my code to its static libraries manually from SCons. > > 5. How do you host your software code (cloud platforms, vps, PaaS, > > docker, Openshift, kubernetes, etc)? On a Linux server. > > 6. What are some constraints and problems in using D for such tasks? Constraints? Problems? D has none. :-D Well OK, let's just say D fits the way I work very well, and has powerful tools for reducing boilerplate, so I find working in D highly productive. If there's any real issue with using D overall, it's in struggling to make dub do what I want. I still haven't figured out whether that's due to an inherent limitation in dub, or the docs are just that bad. [...] > > 7. What solutions do you recommend? [...] If you're starting from scratch, vibe.d + dub isn't a bad approach. My main issue with dub is in trying to integrate it with an existing codebase and with my style of working. It's somewhat Windows-like in that respect, if you follow its workflow style, everything Just Works(tm). But if you want to customize stuff like I do, be prepared for a less pleasant time that may involve doing lots of stuff on your own. Thankfully, doing stuff on your own in D is less painful than in other languages that I've programmed in, so even if you end up needing to do that, it's workable. T -- Frank disagreement binds closer than feigned agreement.
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Friday, 15 December 2017 at 08:13:25 UTC, aberba wrote: I'm going to do a writeup on the state of D in Web Development, APIs and Services for 2017. I need the perspective of the community too along with my personal experience. Please help out. More details the better. 0. Since when did you or company start using D in this area? 1. Do you use a framework? Which one? 2. Why that approach and what would have done otherwise? 3. Which task exactly do you use D to accomplish? 4. Which (dub) packages do you use and for what purpose? 5. How do you host your software code (cloud platforms, vps, PaaS, docker, Openshift, kubernetes, etc)? 6. What are some constraints and problems in using D for such tasks? 7. What solutions do you recommend? 0. Since over a year ago, I mostly do freelancing with it, so just me really. 1. Diamond -- https://github.com/diamondmvc/diamond 2. Would have done nothing different. 3. Every project I do using D, I pretty much use D only, so all tasks necessary; big or small. 4. mysql-native, vibe.d and diamond. 5. I don't host any projects myself. 6. None so far, except for webservices like SOAP. (Although I'm planning on implementing that for Diamond -- Don't know when however.) 7. Depends on the task.
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Friday, 15 December 2017 at 08:13:25 UTC, aberba wrote: I'm going to do a writeup on the state of D in Web Development, APIs and Services for 2017. I need the perspective of the community too along with my personal experience. Please help out. More details the better. 0. Since when did you or company start using D in this area? I converted 2 of my websites from PHP to D earlier this year. http://clarityfitidaho.com http://bobbiblu.net 1. Do you use a framework? Which one? I use my own "cgi.d" framework to create cgi scripts in D that are served up by apache. (https://github.com/marler8997/mored/blob/master/more/cgi.d) I started a project to replace apache for my applications, however, I have not finished or deployed it yet. 2. Why that approach and what would have done otherwise? PHP is a nightmare. I love the benefits of compiled languages, and D is both compiled and powerful enough to compete with dynamic languages like python/javascript/php. Go would also be a good candidate, but without all the power of D that I've come to know and love, I feel like someone's cut off my legs when I program in it. 3. Which task exactly do you use D to accomplish? In the web space, I use it to render web pages via cgi scripts, but I use D in many more areas outside of the web space. 4. Which (dub) packages do you use and for what purpose? I don't use any dub packages. 5. How do you host your software code (cloud platforms, vps, PaaS, docker, Openshift, kubernetes, etc)? I rent a debian server for $50 a month from serverpronto. I host a handful of websites on it and use it for other things as well. 6. What are some constraints and problems in using D for such tasks? No complaints from me. D works great especially on linux. 7. What solutions do you recommend?
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Friday, 15 December 2017 at 08:13:25 UTC, aberba wrote: 0. Since when did you or company start using D in this area? I used D for web in 2009 until about 2013 for work, and then changed jobs and didn't get back into using D until this year. 1. Do you use a framework? Which one? my own web.d 2. Why that approach and what would have done otherwise? Libraries suck. I avoid them unless they are ubiquitous. Using traditional cgi meant it could go with a well-tested production server. 3. Which task exactly do you use D to accomplish? everything server-side, json apis as well as html pages/form handling/websockets. 4. Which (dub) packages do you use and for what purpose? none 5. How do you host your software code (cloud platforms, vps, PaaS, docker, Openshift, kubernetes, etc)? on a linux server. 6. What are some constraints and problems in using D for such tasks? Nothing serious. Compile time got a little slow (~14 seconds at one point) but some refactoring improved it. 7. What solutions do you recommend? keep it simple to get work done.
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Thursday, 21 December 2017 at 19:58:44 UTC, WebFreak001 wrote: yes, I can see each response and export them to csv. I wanted I think if you can make csv with only answers related to WEB development it will be data relevant to what aberba wants. to make a summary of the results but I don't really know where to start As for me, the main summary of results already is on the /viewanalytics page: D User Survey 167 answers D even much more marginal than I expected. -- Узок их круг, страшно далеки они от народа
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Tuesday, 19 December 2017 at 06:58:06 UTC, crimaniak wrote: On Friday, 15 December 2017 at 08:13:25 UTC, aberba wrote: I'm going to do a writeup on the state of D in Web Development, APIs and Services for 2017. I need the perspective of the community too along with my personal experience. Please help out. More details the better. I think some questions already answered in this survey https://forum.dlang.org/thread/hrtakvaqrhvayeidq...@forum.dlang.org I wonder, it is possible to filter Google Forms result to see only results with relevant items in 'primary and secondary area of development' questions? yes, I can see each response and export them to csv. I wanted to make a summary of the results but I don't really know where to start
Re: Answers needed from those using D for Web Development, Web APIs and Services
On Friday, 15 December 2017 at 08:13:25 UTC, aberba wrote: I'm going to do a writeup on the state of D in Web Development, APIs and Services for 2017. I need the perspective of the community too along with my personal experience. Please help out. More details the better. I think some questions already answered in this survey https://forum.dlang.org/thread/hrtakvaqrhvayeidq...@forum.dlang.org I wonder, it is possible to filter Google Forms result to see only results with relevant items in 'primary and secondary area of development' questions?
