Re: [hackers] [quark] Thoughts on CGI and authentication?
On Mon, 26 Oct 2020 11:49:33 +0100 José Miguel Sánchez García wrote: Dear José, > Funny, that's my current use case. All my CGI is through forms, so > I'm currently running a separate server for the form handlers, > regenerating the HTML and then redirecting to the recently updated > page through a "303 See Other" code. > > My motivation behind integrating CGI into quark was leveraging the > quality of its implementation to avoid the security pitfalls of > badly-written HTTP servers out there. I would only have to worry > about writing a simple script to handle the form data. > > Also, if CGI was integrated into the web server itself, I could use > the same domain/port/endpoint to serve the static page (via a GET > request) and to handle the form (via a POST request). Moot point but > it goes a long way towards usability. another approach would be to have a very small interposer that splits GET and POST requests and forwards them to quark and the CGI-handler respectively. > Finally, CGI is often used to customize the content of a page for a > given user. Imagine a logged in user in a forum: they must see a link > that points to their profile. Anonymous users would see a login/signup > bar instead. > > I must say that, even with these advantages in mind, I've come to > think that CGI would not be appropriate for quark. Its goals are at > odds with the needs of a CGI implementation, and that's fine (there > are alternatives for those who want CGI). Feel free to prove me wrong > :) Software gets really complex if you try covering the last 5% of use-cases. Given the massive flexibility of the static web and how many CGI-applications really are just far away from the original idea of the web I really don't see a reason to tailor quark towards CGI. It was there before, but it just made everything really complicated. With best regards Laslo
Re: [hackers] [quark] Thoughts on CGI and authentication?
On 10/26/2020 8:41 AM, Laslo Hunhold wrote: Tell me one example where you need CGI which isn't a web forum? To give an example how you can solve something statically: A comment section could be built by having a static web server and also a very thin "handler" that is called when the form is submitted that adds the comment to a database and updates the static data on the fly. The advantage of this is that if someone manages to "crash" the comment-handler or kill the database process or something, the website is not affected. Funny, that's my current use case. All my CGI is through forms, so I'm currently running a separate server for the form handlers, regenerating the HTML and then redirecting to the recently updated page through a "303 See Other" code. My motivation behind integrating CGI into quark was leveraging the quality of its implementation to avoid the security pitfalls of badly-written HTTP servers out there. I would only have to worry about writing a simple script to handle the form data. Also, if CGI was integrated into the web server itself, I could use the same domain/port/endpoint to serve the static page (via a GET request) and to handle the form (via a POST request). Moot point but it goes a long way towards usability. Finally, CGI is often used to customize the content of a page for a given user. Imagine a logged in user in a forum: they must see a link that points to their profile. Anonymous users would see a login/signup bar instead. I must say that, even with these advantages in mind, I've come to think that CGI would not be appropriate for quark. Its goals are at odds with the needs of a CGI implementation, and that's fine (there are alternatives for those who want CGI). Feel free to prove me wrong :) Best regards, José Miguel
Re: [hackers] [quark] Thoughts on CGI and authentication?
On Sun, 25 Oct 2020 18:00:30 +0300 Platon Ryzhikov wrote: Dear Platon, > I've recently had an idea that instead of adding support for running > scripts by HTTP server (which in any case leads to new fork() calls) > one could use a library providing HTTP server itself while all the > logic is created separately and is performed using callbacks from > library main loop. In that case one could attempt to handle dynamic > (and static using proper callbacks) content within fixed number of > threads. there is theoretically no limit to that, but IPC is a difficult thing here given you are within a chroot. One could think of another Unix-domain socket (besides the one that would be created with the -U option) that could be used to "send" and "receive" data, but to be honest, it really is not withing quark's scope. Tell me one example where you need CGI which isn't a web forum? To give an example how you can solve something statically: A comment section could be built by having a static web server and also a very thin "handler" that is called when the form is submitted that adds the comment to a database and updates the static data on the fly. The advantage of this is that if someone manages to "crash" the comment-handler or kill the database process or something, the website is not affected. Still, maybe I'm missing something here. Please let me know what you need CGI for! With best regards Laslo
Re: [hackers] [quark] Thoughts on CGI and authentication?
On Thu, 22 Oct 2020 20:52:27 +0200 José Miguel Sánchez García wrote: > Hello, > > I'm currently serving my static webpage with quark. I want to add some > CGI, but the opinion here about CGI support in the server is pretty > low. I'm trying to update the basecgi patch, but the latest changes > adding interruptibility seem to complicate the design (the goals of > these two are kinda opposite to what CGI does right? I mean, these are > about memory-bounded operation and CGI scripts can generate > arbitrarily long data, which must be known before even generating the > response header). I have no experience with HTTP whatsoever, so I want > to hear better-informed opinions ;) Hi Jose, I've recently had an idea that instead of adding support for running scripts by HTTP server (which in any case leads to new fork() calls) one could use a library providing HTTP server itself while all the logic is created separately and is performed using callbacks from library main loop. In that case one could attempt to handle dynamic (and static using proper callbacks) content within fixed number of threads. -- Platon Ryzhikov
Re: [hackers] [quark] Thoughts on CGI and authentication?
On Fri, 23 Oct 2020 17:10:37 +0200 José Miguel Sánchez García wrote: Dear José, > That was the whole reasoning behind supporting digest authentication. > Sure, TLS protects the connection from third parties messing around > with your connection, but nothing prevents an evil/misconfigured > server from stealing your cleartext password. At least with digest > authentication, you know that the server is not seeing your password > either (at least you would if the login UI for HTTP auth were barely > usable and told you info about the security mechanism being used... > I'm getting off track sorry). I see what you mean. Still, when you go via TLS, it makes sure that the authenticity of the server is assured as well. > > Keeping with the spirit of the current set of command line arguments > > (e.g. -m for maps, of which you can specify as many as you want), > > one could have a flag -p (protect/password/whatever) that takes a > > group name and a cleartext password and applies it to all files > > matching that group in the serving folder, for example '-m "nogroup > > user:pw"' for example. > > I like that: simple and intuitive. Will do that, thanks! You might also go with "group user pw", which saves us one more "token"-format. > I hope it ends up being a drop-in solution, looking at the code it > seems like it will. We'll know when it's done ;) It most probably will be. With best regards Laslo
Re: [hackers] [quark] Thoughts on CGI and authentication?
On 10/23/2020 2:08 PM, Laslo Hunhold wrote: Dear Laslo, as RFC 7617 (Basic Authentication Scheme) states: This scheme is not considered to be a secure method of user authentication unless used in conjunction with some external secure system such as TLS (Transport Layer Security, [RFC5246]), as the user-id and password are passed over the network as cleartext. So yeah, it would only "really" work on an encrypted connection. However, when the data you want to protect is so sensitive, any authentication scheme over an unencrypted network won't help you protect the data. It may not leak the passwords, because non-Basic doesn't transfer it in cleartext, but that's just one detail. That was the whole reasoning behind supporting digest authentication. Sure, TLS protects the connection from third parties messing around with your connection, but nothing prevents an evil/misconfigured server from stealing your cleartext password. At least with digest authentication, you know that the server is not seeing your password either (at least you would if the login UI for HTTP auth were barely usable and told you info about the security mechanism being used... I'm getting off track sorry). Keeping with the spirit of the current set of command line arguments (e.g. -m for maps, of which you can specify as many as you want), one could have a flag -p (protect/password/whatever) that takes a group name and a cleartext password and applies it to all files matching that group in the serving folder, for example '-m "nogroup user:pw"' for example. I like that: simple and intuitive. Will do that, thanks! Given quark's static scope, it can be optimized to a point that no webserver before it really has gone to. I must admit that I rarely use HTTP-authentication in my web-projects, and I see it as half out of scope, unless it really is a drop-in solution that doesn't get in the way. I hope it ends up being a drop-in solution, looking at the code it seems like it will. We'll know when it's done ;) Best regards. José Miguel
Re: [hackers] [quark] Thoughts on CGI and authentication?
On Thu, 22 Oct 2020 20:52:27 +0200 José Miguel Sánchez García wrote: Dear José, > I'm currently serving my static webpage with quark. I want to add some > CGI, but the opinion here about CGI support in the server is pretty > low. I'm trying to update the basecgi patch, but the latest changes > adding interruptibility seem to complicate the design (the goals of > these two are kinda opposite to what CGI does right? I mean, these are > about memory-bounded operation and CGI scripts can generate > arbitrarily long data, which must be known before even generating the > response header). I have no experience with HTTP whatsoever, so I want > to hear better-informed opinions ;) to be honest, I don't see any way CGI would work with the latest architectural change. Quark's scope is static serving. > About authentication, I think it is a pretty useful feature. I'll > investigate how to implement it properly (abusing file system > ownership/permissions, through a table of auth realms in config.h, > command line parameters... Again, I'd like to hear more opinions) and > upload it as a patch. Is it interesting enough to include it upstream? Definitely! I think that using the file system is an excellent idea. With best regards Laslo
Re: [hackers] [quark] Thoughts on CGI and authentication?
On Fri, 23 Oct 2020 06:27:08 +0100 Nick wrote: Dear Nick, > I don't think it's insecure presuming the HTTP is being served > behind some TLS connection. And if you're doing authentication you > want that anyway. I haven't particularly thought it through, though, > maybe there's something dangerous about it. I mean, lack of browser > support for a straightforward "log out" function sucks, but hey, > it's the web, of course it's broken. as RFC 7617 (Basic Authentication Scheme) states: This scheme is not considered to be a secure method of user authentication unless used in conjunction with some external secure system such as TLS (Transport Layer Security, [RFC5246]), as the user-id and password are passed over the network as cleartext. So yeah, it would only "really" work on an encrypted connection. However, when the data you want to protect is so sensitive, any authentication scheme over an unencrypted network won't help you protect the data. It may not leak the passwords, because non-Basic doesn't transfer it in cleartext, but that's just one detail. > The filesystem based thing sounds odd to me, personally - I think > it's common for websites to have a quite different set of users to > those that exist on the server operating system. But I think setting > it in config.h is also a bad idea, as one of the nice design things > about quark is the ability to run it straight from the command line, > and needing to recompile to redo authentication would detract from > that. Maybe a simple authentication file with > usernamepassword one per line, which is passed to a flag, > would be good? If you want a system with different files accessible > to different users, though, then reusing filesystem permissions is > the only non-intrusive way I can imagine. Keeping with the spirit of the current set of command line arguments (e.g. -m for maps, of which you can specify as many as you want), one could have a flag -p (protect/password/whatever) that takes a group name and a cleartext password and applies it to all files matching that group in the serving folder, for example '-m "nogroup user:pw"' for example. > Just some early morning thoughts. I look forward to Anselm replying > and saying that authentication is out of scope for quark, keeping us > all honest ;) Quark is in a difficult position, because OpenBSD's httpd really is an excellent piece of software that provides all the functionality you would want in an all-purpose web server (CGI, Auth, etc.). Even though I intend quark to be web server that is quick and simple to deploy (i.e. in seconds from the command line to quickly share data on a local network), which would put in question many functions it has (vhosts, redirects, possibly auth later on), I have noticed that many people are just looking for a simple web server for their non-OpenBSD servers. And within that scope, quark fits its role, and because I like to tinker with things, I like to experiment with new things, e.g. the 100%-runtime-configuration. Given quark's static scope, it can be optimized to a point that no webserver before it really has gone to. I must admit that I rarely use HTTP-authentication in my web-projects, and I see it as half out of scope, unless it really is a drop-in solution that doesn't get in the way. With best regards Laslo
Re: [hackers] [quark] Thoughts on CGI and authentication?
On Thu, 22 Oct 2020 22:07:37 +0200 José Miguel Sánchez García wrote: Dear José, > I had HTTP basic and digest auth in mind. They both can be implemented > pretty easily without adding external dependencies and with a > reasonable amount of code. Everything more advanced is straight out of > scope for quark. > Thanks for suggesting basic! I wasn't sure about it, as it's pretty > insecure nowadays. But I acknowledge that, for quark's use cases, it > is perfectly reasonable. I agree with Hiltjo that Basic-authentication is the way to go. The reason is that without TLS, you can, as an evildoer, do whatever you want with a connection, and trying to "fix" it in a half-assed way doesn't work. The only way to fix it is using TLS, encapsulating the entire stream. Encryption, tamper-proofing, etc. are then handled by it one layer above in a much better way than all these pseudo-secure more complex authentication methods. With best regards Laslo
Re: [hackers] [quark] Thoughts on CGI and authentication?
Quoth José Miguel Sánchez García: > Thanks for suggesting basic! I wasn't sure about it, as it's pretty > insecure nowadays. But I acknowledge that, for quark's use cases, it > is perfectly reasonable. I don't think it's insecure presuming the HTTP is being served behind some TLS connection. And if you're doing authentication you want that anyway. I haven't particularly thought it through, though, maybe there's something dangerous about it. I mean, lack of browser support for a straightforward "log out" function sucks, but hey, it's the web, of course it's broken. The filesystem based thing sounds odd to me, personally - I think it's common for websites to have a quite different set of users to those that exist on the server operating system. But I think setting it in config.h is also a bad idea, as one of the nice design things about quark is the ability to run it straight from the command line, and needing to recompile to redo authentication would detract from that. Maybe a simple authentication file with usernamepassword one per line, which is passed to a flag, would be good? If you want a system with different files accessible to different users, though, then reusing filesystem permissions is the only non-intrusive way I can imagine. Just some early morning thoughts. I look forward to Anselm replying and saying that authentication is out of scope for quark, keeping us all honest ;) Nick
Re: [hackers] [quark] Thoughts on CGI and authentication?
Currently the idea I'm most attached to is using groups as realms, while keeping auth data in a separate file (or even hardcoded inside config.h). I think it's a pretty elegant solution (and it would match how I already manage shell user access to my server files), but relying on filesystem metadata can be problematic as it can get lost relatively easily (e.g: archive formats used for backup which do not store user info). Let me know how this ideas feels overall, I'm kinda biased as I love (ab)using the filesystem for my own benefit. Best regards. José Miguel
Re: [hackers] [quark] Thoughts on CGI and authentication?
I had HTTP basic and digest auth in mind. They both can be implemented pretty easily without adding external dependencies and with a reasonable amount of code. Everything more advanced is straight out of scope for quark. Thanks for suggesting basic! I wasn't sure about it, as it's pretty insecure nowadays. But I acknowledge that, for quark's use cases, it is perfectly reasonable. Best regards. José Miguel On Thu, Oct 22, 2020 at 9:42 PM Hiltjo Posthuma wrote: > > On Thu, Oct 22, 2020 at 08:52:27PM +0200, José Miguel Sánchez García wrote: > > Hello, > > > > I'm currently serving my static webpage with quark. I want to add some > > CGI, but the opinion here about CGI support in the server is pretty > > low. I'm trying to update the basecgi patch, but the latest changes > > adding interruptibility seem to complicate the design (the goals of > > these two are kinda opposite to what CGI does right? I mean, these are > > about memory-bounded operation and CGI scripts can generate > > arbitrarily long data, which must be known before even generating the > > response header). I have no experience with HTTP whatsoever, so I want > > to hear better-informed opinions ;) > > > > About authentication, I think it is a pretty useful feature. I'll > > investigate how to implement it properly (abusing file system > > ownership/permissions, through a table of auth realms in config.h, > > command line parameters... Again, I'd like to hear more opinions) and > > upload it as a patch. Is it interesting enough to include it upstream? > > > > Best regards. > > José Miguel > > > > Hi Miguel, > > I'd use HTTP Basic Auth. It's just a simple HTTP header. > > https://tools.ietf.org/html/rfc7617 > > -- > Kind regards, > Hiltjo >
Re: [hackers] [quark] Thoughts on CGI and authentication?
On Thu, Oct 22, 2020 at 08:52:27PM +0200, José Miguel Sánchez García wrote: > Hello, > > I'm currently serving my static webpage with quark. I want to add some > CGI, but the opinion here about CGI support in the server is pretty > low. I'm trying to update the basecgi patch, but the latest changes > adding interruptibility seem to complicate the design (the goals of > these two are kinda opposite to what CGI does right? I mean, these are > about memory-bounded operation and CGI scripts can generate > arbitrarily long data, which must be known before even generating the > response header). I have no experience with HTTP whatsoever, so I want > to hear better-informed opinions ;) > > About authentication, I think it is a pretty useful feature. I'll > investigate how to implement it properly (abusing file system > ownership/permissions, through a table of auth realms in config.h, > command line parameters... Again, I'd like to hear more opinions) and > upload it as a patch. Is it interesting enough to include it upstream? > > Best regards. > José Miguel > Hi Miguel, I'd use HTTP Basic Auth. It's just a simple HTTP header. https://tools.ietf.org/html/rfc7617 -- Kind regards, Hiltjo
Re: [hackers] [quark] Thoughts on CGI and authentication?
Hey Jose, I agree that the recent changes have made it more difficult to implement CGI. I've also wanted modify Quark to accept POST & PUT requests. With the help of Ryzhikov's Base-CGI patch, I've come up with the following: https://github.com/jeremybobbin/quark Basically, if you pass the `-x` option, when an executible file is requested, Quark executes the file, writing the request body to stdin, and writing stdout as the response. I haven't tried to implement user authorization via filesystem permissions, though I'd be curious to see what's possible. Jeremy
[hackers] [quark] Thoughts on CGI and authentication?
Hello, I'm currently serving my static webpage with quark. I want to add some CGI, but the opinion here about CGI support in the server is pretty low. I'm trying to update the basecgi patch, but the latest changes adding interruptibility seem to complicate the design (the goals of these two are kinda opposite to what CGI does right? I mean, these are about memory-bounded operation and CGI scripts can generate arbitrarily long data, which must be known before even generating the response header). I have no experience with HTTP whatsoever, so I want to hear better-informed opinions ;) About authentication, I think it is a pretty useful feature. I'll investigate how to implement it properly (abusing file system ownership/permissions, through a table of auth realms in config.h, command line parameters... Again, I'd like to hear more opinions) and upload it as a patch. Is it interesting enough to include it upstream? Best regards. José Miguel
