Re: [dev] [quark] Performance issues

[Anselm Garbe]

Hi Laslo,

On Tue, 24 Sep 2019 at 23:22, Laslo Hunhold  wrote:
> On Tue, 24 Sep 2019 11:17:09 -0700
> Anselm Garbe  wrote:
> > - Always run as current user, if root is used chroot() to current
> > directory.
>
> chroot() should never be optional. unveil() might bring the same
> effect, but the unveil()-wrapper in quark doesn't do anything on Linux.

My thinking is different, I'd rather discourage running quark as root
at all, if it detects uid==0 it should spit out a warning. The only
reason for root privileges would be port 80. Yes chroot() would
require this as well, but actually you should be able to make escaping
from served directories impossible by taking some precautions in terms
of not following links and rejecting .. etc. in urlinfos. The security
measure of operating quark should just be derived from the user that
executes it. That keeps it simple and allows all kinds of surroundings
if desired, and does not rely on its own implementation magic. All it
should ensure that a provided root directory is not escaped.

> > - Always serve current directory as root (no need to supply directory
> > option)
>
> I think that would overstretch it.

By default, there is nothing wrong with supporting -d override similar
to -p override.

> I understand this point, given port 80 cannot be bound when not root.
> But this is a permission thing and that's what doas and sudo are for.
> Especially doas allows you to fine-grain the permissions of specific
> applications.

That's what users are for. doas and sudo are just hacks/workarounds.

> > - Don't support vhosts, if you need that use different server, if you
> > need to run multiple instances, use different ports instead
>
> I don't understand how different ports would solve that problem unless
> you maybe have a relay that is connected to different quark instances
> on an internal network. Publicly, if you have a server serving e.g. 3
> virtual hosts, all accesses have to happen on one port.

Drop in by my definition means one does not run quark permanently, it
is just for temporary stuff. Like during development you serve some
generated documentation through it, or when you want to share some
files for the time being on a LAN. I would always warn about quark not
to be used in any production setup, as it has never been properly
reviewed or pen tested.

With this definition there is really no justification at all to
support vhosts. If you need to serve different directory roots (that
typically may translate to different vhosts in classic setups), just
run different quark instances on different ports, that's the point I'm
making here.

> > - Don't support unix sockets
>
> Agreed, they increase the complexity, but they make quark much more
> flexible. For instance, this makes it very easy to attach something like
> stunnel to it for TLS.

There is no reason to put some stunnel on an INET socket that talks to
a locally only bound quark INET socket. If one really wants to access
quark via a unix socket, use netcat to offer a unix socket to local
inet socket. No need to overload quark for this.

> > - Don't bother with TLS stuff, a drop-in server should not be required
> > to use TLS
>
> quark itself doesn't do any TLS, but it's capable of "dealing" with
> something like stunnel by giving proper redirects and so forth.

That's my whole point. If someone needs TLS, he can achieve that by
putting proxy in front of it.

> > - Eventually add content range support (which might become handy for
> > drop-in use cases)
>
> quark supports partial content. This is mandatory if you serve videos
> or audio files.

Good

> > That's it. I'm sure this can be done within 200 lines. And this should
> > be the aim for quark after all. It shouldn't be supposed to be used as
> > a general purpose server, as said initially, there are enough already
> > to choose from. But there are almost non that offer the feature set
> > described.
>
> The most substantial part of quark is not necessarily the host handling
> or something like directory-choice (the latter is done in like 4 lines
> during the ARG-loop). The HTTP-header parsing alone takes 200 lines and
> handles special cases like this crappy IPv6-bracket-notation which is
> too easy to miss. Another substantial part is actually the
> content-range handling, because it has some intricate details. I get
> your point of course, and in a world with only OpenBSD (sadly it isn't)
> I would strip out vhosts immediately.

If you remove vhost handling you will be able to remove all kinds of
cruft, like redirect handling etc.

> The main problem I see is that there is no "nice" httpd on Linux. We
> might move forward and try to port OpenBSD's httpd to Linux and get it
> packaged. However, up to this point, quark can act as a lightweight
> server even for static purposes on Linux.

There is no nice drop in on Linux or anywhere to the spec I described.
quark could fill this gap.

If I want to run a real httpd on Linux, I'd stick to nginx... but if 

Re: [dev] Setting keyboard shortcut for clicking and moving mouse

[Piotr Oleskiewicz]

Hi Mark,

I'm not sure this is implemented in dwm.

You can take a look at the wmp executable from the wmutils project 
(https://github.com/wmutils/core) and map it in your config.h by spawning wmp 
from there.


Best wishes,
Piotr

Re: [dev] Setting keyboard shortcut for clicking and moving mouse

[C.J. Wagenius]

Have a look at 
https://linuxreviews.org/HOWTO_use_the_numeric_keyboard_keys_as_mouse_in_XOrg 


/cjw

25 sep. 2019 18:30 av c...@riseup.net:

> Hello,
> Does anyone know or have sample config.h codes on how to set keyboard
> shutcut for left mouse click, middle mouse click, right mouse click,
> moving mouse in web broswer to right, up, left, down direction? 
>
> After setting these keyboard shortcut DWM config.h file, we probably no 
> longer need
> computer mouse.
> Mark
>

[dev] Setting keyboard shortcut for clicking and moving mouse

[CSO]

Hello,
Does anyone know or have sample config.h codes on how to set keyboard
shutcut for left mouse click, middle mouse click, right mouse click,
moving mouse in web broswer to right, up, left, down direction? 

After setting these keyboard shortcut DWM config.h file, we probably no longer 
need
computer mouse.
Mark

Re: [dev] [quark] Performance issues

[Markus Wichmann]

On Wed, Sep 25, 2019 at 04:32:45PM +0200, Laslo Hunhold wrote:
> please excuse the ignorant question, but why would you want to access
> /dev/null? The problem with /dev/urandom is well-known, which is why
> OpenBSD has arc4random(3).
>

You might not, but a library you use might. I remember musl testing for
open FDs 0, 1, and 2 and opening /dev/null to make up for missing FDs.
And crashing (deliberately) if that fails. But then, that only happens
for elevated security contexts, i.e. setuid or "secure" mode (which on
Linux means "file capabilities set").

/dev/null has the distinction of being the only device POSIX actually
requires. I don't know what you might use it for. (/dev/tty is
defined, but optional.)

Ciao,
Markus

Re: [dev] [quark] Performance issues

[Hiltjo Posthuma]

On Wed, Sep 25, 2019 at 04:32:45PM +0200, Laslo Hunhold wrote:
> On Wed, 25 Sep 2019 16:06:52 +0200
> Markus Wichmann  wrote:
> 
> Dear Markus,
> 
> > chroot() has several detrimental effects, most importantly making it
> > impossible to access /dev/null and /dev/urandom. Unless, of course,
> > measures are taken to replicate these devices underneath the new root.
> 
> please excuse the ignorant question, but why would you want to access
> /dev/null? The problem with /dev/urandom is well-known, which is why
> OpenBSD has arc4random(3).
> 
> > It is also not a security device. If a service in a chroot is
> > exploited with root privileges, it can mount procfs wherever, and
> > access /proc/1/root. It can also mount another instance of the rootfs
> > wherever and escape the jail that way.
> 
> You are absolutely right. chroot() on its own is not a security device,
> but it is when you drop your privileges after chrooting. Then it is not
> possible to mount procfs or whatnot.
> 
> After all, one measure alone is never enough, which is why I also use
> unveil(2) (at least on OpenBSD) to make any other accesses to the file
> system impossible.
> 
> With best regards
> 
> Laslo
> 

Yes and to add to it:
Of course unveil and pledge also don't need (initial) root
permissions, which makes locking down basic programs very easy.

-- 
Kind regards,
Hiltjo

Re: [dev] [quark] Performance issues

[Laslo Hunhold]

On Wed, 25 Sep 2019 16:06:52 +0200
Markus Wichmann  wrote:

Dear Markus,

> chroot() has several detrimental effects, most importantly making it
> impossible to access /dev/null and /dev/urandom. Unless, of course,
> measures are taken to replicate these devices underneath the new root.

please excuse the ignorant question, but why would you want to access
/dev/null? The problem with /dev/urandom is well-known, which is why
OpenBSD has arc4random(3).

> It is also not a security device. If a service in a chroot is
> exploited with root privileges, it can mount procfs wherever, and
> access /proc/1/root. It can also mount another instance of the rootfs
> wherever and escape the jail that way.

You are absolutely right. chroot() on its own is not a security device,
but it is when you drop your privileges after chrooting. Then it is not
possible to mount procfs or whatnot.

After all, one measure alone is never enough, which is why I also use
unveil(2) (at least on OpenBSD) to make any other accesses to the file
system impossible.

With best regards

Laslo

Re: [dev] [quark] Performance issues

[Markus Wichmann]

On Wed, Sep 25, 2019 at 08:20:52AM +0200, Laslo Hunhold wrote:
> chroot() should never be optional. unveil() might bring the same
> effect, but the unveil()-wrapper in quark doesn't do anything on Linux.
>

chroot() has several detrimental effects, most importantly making it
impossible to access /dev/null and /dev/urandom. Unless, of course,
measures are taken to replicate these devices underneath the new root.

It is also not a security device. If a service in a chroot is exploited
with root privileges, it can mount procfs wherever, and access
/proc/1/root. It can also mount another instance of the rootfs wherever
and escape the jail that way.

>
> With best regards
>
> Laslo
>

Ciao,
Markus

Re: [dev] [quark] Performance issues

[Laslo Hunhold]

On Tue, 24 Sep 2019 21:33:03 +0200
Hiltjo Posthuma  wrote:

Dear Hiltjo,

> > Thanks for the insight! I was thinking about using quark instead of
> > OpenBSD's httpd, because it simpler to use and probably even more
> > secure. I think ~1000 requests/second is still plenty for all my
> > projects, but in case I need the extra performance some day, I know
> > what the bottleneck is.
> >   
> 
> To setup OpenBSD httpd it is 3 lines of text for a static-serving
> httpd:
> 
>   server "default" {
>   listen on egress port 80
>   }
> 
> It can be used with a nicely priviledge-separated model with the
> slowcgi program too:
> 
>   server "default" {
>   listen on egress port 80
> 
>   location "/" {
>   root "/cgi-bin/program"
>   fastcgi
>   }
>   }
> 
> Regarding "performance": have you tried to look at login.conf
> resource-limits or tuning the system in other ways for your actual
> needs?

exactly! If you are using OpenBSD, I see no reason not to use httpd
over quark in any circumstance, but especially for "fixed" setups like
a webserver.
If you are at a conference and quickly want to share something on the
network, there's not much difference between

   $ vim /etc/httpd.conf
   ... adding three lines
   $ rcctl reload httpd

and

   $ quark -h ... -p 80 -d ...

On Linux the story is different and many of the common solutions either
involve lots of config, are bloated with lots of dependencies, are
not secure or even lack basic features.

So yeah, even though I'm the quark maintainer, I would not evangelize
it and see it more as a quick drop-in on non-OpenBSD machines.

With best regards

Laslo

Re: [dev] [quark] Performance issues

[Laslo Hunhold]

On Tue, 24 Sep 2019 19:58:01 +0200
maillists.rul...@mailbox.org wrote:

Dear Richard,

> Thanks for your assessments! I think you've hit the nail on the head.
> I got my initial test results on an OpenBSD machine. I tested the
> golang server and quark again on a Linux machine today and the
> difference is much less pronounced:
> 
> quark  (Linux): 14494 requests/second
> golang (Linux): 16601 requests/second
> 
> As you'd expect the difference also declines when the requested file
> is larger. With an index.html, that is 1MiB in size I got these
> results:
> 
> quark  (OpenBSD; 1MiB index.html): 160 requests/second
> golang (OpenBSD; 1MiB index.html): 198 requests/second

yes, that looks more like it. Thanks for testing it out!

> Thanks for the insight! I was thinking about using quark instead of
> OpenBSD's httpd, because it simpler to use and probably even more
> secure. I think ~1000 requests/second is still plenty for all my
> projects, but in case I need the extra performance some day, I know
> what the bottleneck is.

OpenBSD's httpd is a very fine piece of software and I see it as the
golden standard, also with regard to security. In a way it is also more
consistent being based on relayd(8) and pf(4).

With best regards

Laslo

Re: [dev] [quark] Performance issues

[Laslo Hunhold]

On Tue, 24 Sep 2019 21:16:05 +0200
Hiltjo Posthuma  wrote:

Dear Hiltjo,

> Sorry to go a bit off-topic here, but I quickly tested and reviewed
> the CGI patch. This CGI patch is broken. A basic `quark -h 127.0.0.1
> -p 8080` serving some page always returns HTTP 400 "Bad request".
> 
> A few bounds checks seem not to use good code practises like:
> 
>   * all other data will be later passed to script */
>   sprintf(r->cgicont, "%s", p);
> 
> It seems it allows only a maximum of PATH_MAX bytes of POST data too.
> 
> and:
> 
>   snprintf(realtarget, sizeof(tmptarget) + sizeof(s.cgi[i].dir)
> - 1, "%s%s", s.cgi[i].dir, tmptarget);
> 
> The patch filename is also named incorrectly on the wiki.
> 
> I CC'd the patch author.

thanks for the heads up! I think at least in regard to the HTTP 400 it
might be due to the pledge disallowing something, killing the
cgi-process. But this is just from the top of my head and I haven't
looked into it.

With best regards

Laslo

Re: [dev] [quark] Performance issues

[Laslo Hunhold]

On Tue, 24 Sep 2019 11:17:09 -0700
Anselm Garbe  wrote:

Dear Anselm,

> The issue is, there are enough http servers, but most of them are
> designed as daemons. As you say, quark could become a nice alternative
> for drop-in purporses. For this it is currently too feature rich
> however. I would suggest to strip it down as follows:

see my remarks below.
 
> - Keep it GET/HEAD based only

I agree of course. :)

> - Always run as current user, if root is used chroot() to current
> directory.

chroot() should never be optional. unveil() might bring the same
effect, but the unveil()-wrapper in quark doesn't do anything on Linux.

> - Always serve current directory as root (no need to supply directory
> option)

I think that would overstretch it.

> - If no explicit port is specified start from 8080 and take next
> available port

I understand this point, given port 80 cannot be bound when not root.
But this is a permission thing and that's what doas and sudo are for.
Especially doas allows you to fine-grain the permissions of specific
applications.

> - Don't support vhosts, if you need that use different server, if you
> need to run multiple instances, use different ports instead

I don't understand how different ports would solve that problem unless
you maybe have a relay that is connected to different quark instances
on an internal network. Publicly, if you have a server serving e.g. 3
virtual hosts, all accesses have to happen on one port.

> - Don't support unix sockets

Agreed, they increase the complexity, but they make quark much more
flexible. For instance, this makes it very easy to attach something like
stunnel to it for TLS.

> - Don't bother with TLS stuff, a drop-in server should not be required
> to use TLS

quark itself doesn't do any TLS, but it's capable of "dealing" with
something like stunnel by giving proper redirects and so forth.

> - Only log to stderr/stdout

That's what it does. :)

> - Eventually add content range support (which might become handy for
> drop-in use cases)

quark supports partial content. This is mandatory if you serve videos
or audio files.

> That's it. I'm sure this can be done within 200 lines. And this should
> be the aim for quark after all. It shouldn't be supposed to be used as
> a general purpose server, as said initially, there are enough already
> to choose from. But there are almost non that offer the feature set
> described.

The most substantial part of quark is not necessarily the host handling
or something like directory-choice (the latter is done in like 4 lines
during the ARG-loop). The HTTP-header parsing alone takes 200 lines and
handles special cases like this crappy IPv6-bracket-notation which is
too easy to miss. Another substantial part is actually the
content-range handling, because it has some intricate details. I get
your point of course, and in a world with only OpenBSD (sadly it isn't)
I would strip out vhosts immediately.

The main problem I see is that there is no "nice" httpd on Linux. We
might move forward and try to port OpenBSD's httpd to Linux and get it
packaged. However, up to this point, quark can act as a lightweight
server even for static purposes on Linux.

With best regards

Laslo