Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Am 11.02.2014 11:37, schrieb Julien Cristau: On Tue, Feb 11, 2014 at 10:38:39 +0100, Peter Palfrader wrote: On Tue, 11 Feb 2014, Christoph Haas wrote: Regarding the certificate: does Debian have resources to buy an SSL certificate? I usually use a free StartCom certificate for my own purposes but I am not sure whether it is suitable for such use. I don't think that the sponsor will donate an SSL certificate either but I'm willing to ask. jcristau recently got a cert for france.debian.net from gandi, using the put a file on the webserver method of authentication. Gandi certs are around 12 Euros a year. startcom requires control of the top level (debian.net) so that didn't work. So I went with gandi, and got a 1-year certificate for 14.40€. Done. I have bought a Gandi certificate and set up screenshots.debian.net via HTTPS. Looks like I'm using relative URLs everywhere so the site should work well. Feel free to try changing the URL in packages.debian.org to HTTPS. I'm curious about the effect of the encryption on the CPU load. Cheers Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL7mE8ACgkQCV53xXnMZYZmMwCfe7zmGqG6DbmwXo+1YH87jIOr UG0An0eYVQGh34AZKTx2OI29xK9Pbhsn =y1Xv -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fb9852.40...@debian.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Quick note… I have tried to… Redirect permanent / https://screenshots.debian.net/ But it only took 10 seconds and my (virtual) server started to respond unbearably slowly. The system load seemed okay (0.8 with 2 CPUs) but the response time was ~10 seconds per HTTPS request. I'm not yet sure what the problem is because KeepAlive on Apache level is on and there are enough sockets available according to apache2ctl fullstatus shows enough enough open slots. So I'll leave HTTPS on but I can't currently redirect everyone to use HTTPS. Ideas welcome. …Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL7n4MACgkQCV53xXnMZYYUxgCfbjINjFtOQy0jKdqXRuoLwEdF 2/8Anj06IxQqqkh/mAP3ThFlH3IQ3w+m =yp5z -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fb9f83.5090...@debian.org
Re: SSL for screenshots.debian.net?
Hi, cat /proc/sys/kernel/random/entropy_avail ? On Wed Feb 12, 2014 at 17:21:23 +0100, Christoph Haas wrote: Quick note… I have tried to… Redirect permanent / https://screenshots.debian.net/ But it only took 10 seconds and my (virtual) server started to respond unbearably slowly. The system load seemed okay (0.8 with 2 CPUs) but the response time was ~10 seconds per HTTPS request. I'm not yet sure what the problem is because KeepAlive on Apache level is on and there are enough sockets available according to apache2ctl fullstatus shows enough enough open slots. So I'll leave HTTPS on but I can't currently redirect everyone to use HTTPS. Ideas welcome. -- Martin Zobel-Helas zo...@debian.orgDebian System Administrator Debian GNU/Linux Developer Debian Listmaster http://about.me/zobel Debian Webmaster GPG Fingerprint: 6B18 5642 8E41 EC89 3D5D BDBB 53B1 AC6D B11B 627B -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140212175104.go7...@ftbfs.de
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Some raw data… $ while true ; do cat /proc/sys/kernel/random/entropy_avail ; sleep 1; done 129 153 134 180 183 174 174 134 192 175 179 188 135 183 $ time curl https://screenshots.debian.net/ /dev/null real0m11.148s user0m0.016s sys 0m0.012s $ apache2ctl status Apache Server Status for localhost Server Version: Apache/2.2.16 (Debian) mod_ssl/2.2.16 OpenSSL/0.9.8o mod_wsgi/ 3.3 Python/2.6.6 Server Built: Mar 3 2013 12:12:28 - --- Current Time: Wednesday, 12-Feb-2014 13:08:31 EST Restart Time: Wednesday, 12-Feb-2014 13:07:18 EST Parent Server Generation: 10 Server uptime: 1 minute 12 seconds Total accesses: 3461 - Total Traffic: 44.9 MB CPU Usage: u26.13 s2.46 cu0 cs0 - 39.7% CPU load 48.1 requests/sec - 0.6 MB/second - 13.3 kB/request 149 requests currently being processed, 1 idle workers RRWRCRKKKRWKKKWWK... CKRRKKCRCKWCK... CKKKWRKKCCRWKRCKK... CKCKKCRKCKKKRKKWWKWRK... _KWKKCKWRRRWWKWKK... KKCWKWKRKWKRKWKCKCWRKRKWK... Scoreboard Key: _ Waiting for Connection, S Starting up, R Reading Request, W Sending Reply, K Keepalive (read), D DNS Lookup, C Closing connection, L Logging, G Gracefully finishing, I Idle cleanup of worker, . Open slot with no current process The server sponsor from Vexxhost reacted very quickly though and has offered a faster server. So I will deploy the application there and try HTTPS globally again. I may also try to put an nginx in front. Somehow I have a bad feeling with mod_wsgi and Apache-mpm-worker. …Christoph On 12.02.2014 18:51, Martin Zobel-Helas wrote: Hi, cat /proc/sys/kernel/random/entropy_avail ? On Wed Feb 12, 2014 at 17:21:23 +0100, Christoph Haas wrote: Quick note… I have tried to… Redirect permanent / https://screenshots.debian.net/ But it only took 10 seconds and my (virtual) server started to respond unbearably slowly. The system load seemed okay (0.8 with 2 CPUs) but the response time was ~10 seconds per HTTPS request. I'm not yet sure what the problem is because KeepAlive on Apache level is on and there are enough sockets available according to apache2ctl fullstatus shows enough enough open slots. So I'll leave HTTPS on but I can't currently redirect everyone to use HTTPS. Ideas welcome. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL7uh4ACgkQCV53xXnMZYYfpgCgpedpVMdRESKUyzhjBf9PbVTt fMwAoK83RsX0ozn++LqO7m+MxJK40iP9 =cKNt -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fbba21.2090...@debian.org
Re: SSL for screenshots.debian.net?
Do you have haveged installed? On Wed, Feb 12, 2014 at 07:14:57PM +0100, Christoph Haas wrote: -BEGIN PGP SIGNED MESSAGE- The server sponsor from Vexxhost reacted very quickly though and has offered a faster server. So I will deploy the application there and try HTTPS globally again. I may also try to put an nginx in front. Somehow I have a bad feeling with mod_wsgi and Apache-mpm-worker. -- Luca Filipozzi http://www.crowdrise.com/SupportDebian -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140212181817.ga31...@emyr.net
Re: SSL for screenshots.debian.net?
Hi, On Wed Feb 12, 2014 at 18:18:17 +, Luca Filipozzi wrote: Do you have haveged installed? On Wed, Feb 12, 2014 at 07:14:57PM +0100, Christoph Haas wrote: -BEGIN PGP SIGNED MESSAGE- The server sponsor from Vexxhost reacted very quickly though and has offered a faster server. So I will deploy the application there and try HTTPS globally again. I may also try to put an nginx in front. Somehow I have a bad feeling with mod_wsgi and Apache-mpm-worker. How about moving the installation to DSA maintained hardware? -- Martin Zobel-Helas zo...@debian.orgDebian System Administrator Debian GNU/Linux Developer Debian Listmaster http://about.me/zobel Debian Webmaster GPG Fingerprint: 6B18 5642 8E41 EC89 3D5D BDBB 53B1 AC6D B11B 627B -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140212182639.gp7...@ftbfs.de
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12.02.2014 19:26, Martin Zobel-Helas wrote: On Wed Feb 12, 2014 at 18:18:17 +, Luca Filipozzi wrote: Do you have haveged installed? On Wed, Feb 12, 2014 at 07:14:57PM +0100, Christoph Haas wrote: -BEGIN PGP SIGNED MESSAGE- The server sponsor from Vexxhost reacted very quickly though and has offered a faster server. So I will deploy the application there and try HTTPS globally again. I may also try to put an nginx in front. Somehow I have a bad feeling with mod_wsgi and Apache-mpm-worker. How about moving the installation to DSA maintained hardware? What would that mean? I'm trying to be honest here. I like adding features every now and then and would like to be able to try them out without requesting anyone to do anything. From my understanding DDs do not have access to DSA-maintained hardware. As much as I understand the policy I prefer the flexibility of logging into the system and hacking the application. I'm serious about hosting the application though and the web site has a very good availability since 2008. You know what I mean. As much as you have a hard time helping me figure out the SSL performance issue as you don't have root access at the moment I would be in the same situation vice versa. That's one concern. The other concern is - as I mentioned (and nobody commented on it) - that I seriously intend to open the service for other Linux distributions so that the effort of taking screenshots can be used by several distributions. Debian users would be able to use screenshots uploaded by Ubuntu users. And CentOS users would see screenshots from OpenSuSE users. Is there a case where DSA-maintained hardware was used in such a context yet? Vexxhost as a sponsor (also for other related projects) has proven to be a reliable and committed ISP. And if performance is the issue it looks like they pimp the server until it fits again. ?Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL7wE0ACgkQCV53xXnMZYZh+gCg+AvZZj4sTC4axFvW/77rkiWk m78AoKGm1SpcdLjYVum107OHO85Yk2QH =kdnl -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fbc04d.1090...@debian.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Not yet. But /proc/sys/kernel/random/entropy_avail never dropped lower than 100. Do you still suspect a lack of entropy? ?Christoph On 12.02.2014 19:18, Luca Filipozzi wrote: Do you have haveged installed? On Wed, Feb 12, 2014 at 07:14:57PM +0100, Christoph Haas wrote: -BEGIN PGP SIGNED MESSAGE- The server sponsor from Vexxhost reacted very quickly though and has offered a faster server. So I will deploy the application there and try HTTPS globally again. I may also try to put an nginx in front. Somehow I have a bad feeling with mod_wsgi and Apache-mpm-worker. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL7wOAACgkQCV53xXnMZYZUqgCdHVP76KQHAqddx+qSQZmXriHt XhwAmwV2WAwT33wtXa5Crxeq0dKNghMn =AOs1 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fbc0e0.2090...@debian.org
Re: SSL for screenshots.debian.net?
On Wed, 12 Feb 2014, Christoph Haas wrote: What would that mean? I'm trying to be honest here. I like adding features every now and then and would like to be able to try them out without requesting anyone to do anything. From my understanding DDs do not have access to DSA-maintained hardware. Um. You couldn't be farther from the truth. Most of the hosts that run services are accessible to all DDs, and of the rest the hosts are at least accessible to the team running the service. The standard setup is that service stuff lives in /srv/$service. Teams tend to not have root. For most web based services they, however, still have access to update their apache vhost configuration. See for instance picconi.debian.org, which runs one the packages.d.o master, or glinka.debian.org which runs a bunch of things, or even buxtehude.debian.org which runs the BTS. That's one concern. The other concern is - as I mentioned (and nobody commented on it) - that I seriously intend to open the service for other Linux distributions so that the effort of taking screenshots can be used by several distributions. Debian users would be able to use screenshots uploaded by Ubuntu users. And CentOS users would see screenshots from OpenSuSE users. Is there a case where DSA-maintained hardware was used in such a context yet? I think Debian providing services to other distributions would be just fine. -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140212185109.gv14...@anguilla.noreply.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12.02.2014 20:18, Peter Palfrader wrote: On Wed, 12 Feb 2014, Christoph Haas wrote: Thanks for the clarification. I would like to consider that once I have the next major version of the underlying application ready to be deployed. Good to hear that the policy has become so liberal. What amount of lead time would be required to request such resources? Minutes to Days, maybe a tiny number of weeks. As you know, we're all volunteers and sometimes other commitments prevent us from doing the important things :) Perfect. 2-4 weeks is fine. I'm likely running thorough tests before anyway. This might be one of the easier deployments, as you probably already know exactly how much disk space and memory you'll need. Right. Although a rewrite of the application may have different requirements. But I've been in the sysadmin business for 18 years so I think I can give pretty good estimates and have the admin-side in mind. Have you given any consideration into making this service mirrorable? I.e. have one master instance through which updates are added, and more hosts that can then serve the content to end-users? Yes, I have. The challenge would mainly be the storage backends. PostgreSQL has been my favorite database. But replication was an issue as all mirrors would have to be able to write and send their updates to other servers. Besides the actual image (PNG) blobs would have to be copied, too. So I wondered if I have to implement replication on an application layer. An alternative would be MongoDB. (I once tried CouchDB but I'm pretty sure it's the wrong tool for the job.) MongoDB could even store the image blobs within the collection and replicate them. As far as I understand MongoDB though there is only one master at a time. So if screenshots.ubuntu.com would receive a new screenshot upload it would have to contact the master server which may be another node in the MongoDB cluster elsewhere in the world. Not only would all mirrors need to able to work in a cluster - but also the latency of the slave to the master server might become a problem. I have already put way more thought than code into the rewrite. :) Simple mirrors would work in a master/slave fashion. But that is not the concept of using a common screenshot database for multiple distributions. Suggestions welcome though… I have a good idea of the application. But at this stage I can still do some major design changes without losing much work. …Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL70M4ACgkQCV53xXnMZYZLfgCgiPnEw5suHBvFJXVV9duH4FA4 nxQAoK/CbnEAXQH8kt81N1MlIvvxqMkk =hCMz -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fbd0ce.7030...@debian.org
Re: SSL for screenshots.debian.net?
On Wed, 12 Feb 2014, Christoph Haas wrote: Have you given any consideration into making this service mirrorable? I.e. have one master instance through which updates are added, and more hosts that can then serve the content to end-users? Yes, I have. The challenge would mainly be the storage backends. PostgreSQL has been my favorite database. But replication was an issue as all mirrors would have to be able to write and send their updates to other servers. Besides the actual image (PNG) blobs would have to be copied, too. So I wondered if I have to implement replication on an application layer. Simple mirrors would work in a master/slave fashion. But that is not the concept of using a common screenshot database for multiple distributions. I was thinking of redirecting all udpates to https://screenshots-master.debian.org/, but serve the normal queries from https://screenshots.debian.org/. The -master is on a single host, the screenshots site could be replicated, i.e. a round robin of several machines. If you use pg and a filesystem as your storage backends, we can replicate the database pretty much instantly - the non-master clients will only need read-only access. The filesystem tree could just be replicated using rsync - for bugs.d.o we use a inotify based rsyncing strategy that runs an rsync over new/changed files within the minute they happen. This is, of course, not fully replicated as the -master hist is still a SPOF. But if that goes down we only lose the ability to add new screenshots, not the ability to serve them. Also, it seems like it ought to be relatively simple and straight forward, but then I don't know the intricate details of screenshots.d.n. Cheers, -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140212202119.gx14...@anguilla.noreply.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12.02.2014 21:21, Peter Palfrader wrote: On Wed, 12 Feb 2014, Christoph Haas wrote: Have you given any consideration into making this service mirrorable? I.e. have one master instance through which updates are added, and more hosts that can then serve the content to end-users? Yes, I have. The challenge would mainly be the storage backends. PostgreSQL has been my favorite database. But replication was an issue as all mirrors would have to be able to write and send their updates to other servers. Besides the actual image (PNG) blobs would have to be copied, too. So I wondered if I have to implement replication on an application layer. Simple mirrors would work in a master/slave fashion. But that is not the concept of using a common screenshot database for multiple distributions. I was thinking of redirecting all udpates to https://screenshots-master.debian.org/, but serve the normal queries from https://screenshots.debian.org/. The -master is on a single host, the screenshots site could be replicated, i.e. a round robin of several machines. If you use pg and a filesystem as your storage backends, we can replicate the database pretty much instantly - the non-master clients will only need read-only access. The filesystem tree could just be replicated using rsync - for bugs.d.o we use a inotify based rsyncing strategy that runs an rsync over new/changed files within the minute they happen. Thanks for sharing your ideas. Sending updates to a single server is indeed an interesting idea. Timely updates are not really an issue. If someone uploads a screenshot it does not have to visible instantly. (That's why I even considered MongoDB because the application does not require ACID.) It's indeed a simple mechanism that is easy to understand and operate. Just as you said… SPOF instantly popped up in my brain. :) I will try to make the implementation open enough to either use pg/rsync replication or some other mean like application-based replication. The only decent alternative to pg seems to be Cassandra and I'm not sure I want to ride that horse. As you don't mind providing a service for other free distributions as well the distribution issue surely won't a problem at this stage. Getting a successor version up and running needs a higher priority than winning the nobel price for the fanciest replication. :) As long as it can be implemented later without throwing everything away. The current application is based on a deprecated Python framework and I just hope that I get it up and running on Wheezy for the next months - currently it runs on Squeeze. This is, of course, not fully replicated as the -master hist is still a SPOF. But if that goes down we only lose the ability to add new screenshots, not the ability to serve them. Also, it seems like it ought to be relatively simple and straight forward, but then I don't know the intricate details of screenshots.d.n. Oh, the application is nothing magical really. It gets Packages.bz2 files from various DEB-based distributions (currently Ubuntu-integration is just a dirty hack that needs a proper implementation) and imports the metadata into a pg database. Users can then browse the site for screenshots by names and debtags (although for some reason the debtags support seems increasingly broken and I haven't found the cause yet) and upload screenshots if they are missing. And of course it features HTTP-based APIs for packages.debian.org and packages.ubuntu.com as well for Synaptic and the Ubuntu Software Center application with some smart logic in the backend for dealing with different versions and distributions. I will keep that basic concept in the rewrite. However I will use an actively maintained Ruby framework based on Rack so that the operation should be easy for the next 5+ years. And the major feature is support for multiple distributions to get more screenshots. Also I'm taking better care of performance issues like delivering static assets and caching. Other social features like commenting on packages are second class. For the curious… planning takes place here: https://trello.com/b/hRpf5XUQ/projekt-debshots Actually I'm still surprised of the success. screenshots.debian.net was a fun project and now delivers 40 screenshots/sec. So it's worth taking it serious in my opinion. It's very motivating seeing your support for the idea. Thanks. …Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL73YAACgkQCV53xXnMZYb5pgCglj11mDu1/qlI24CIrjt5YVUJ /d8AoL5C2II5nK67dxxMzlUdwcjR7cLl =UlJb -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fbdd80.6050...@debian.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12.02.2014 22:00, Peter Palfrader wrote: On Wed, 12 Feb 2014, Christoph Haas wrote: I will keep that basic concept in the rewrite. However I will use an actively maintained Ruby framework based on Rack so that the operation should be easy for the next 5+ years. I have yet to go through one stable update that doesn't break all of my ruby scripts. If you want stability, then this might not be the best choice. That bad? In my experience the situation isn't all roses and happiness in the Python ecosystem either. Suddenly the module versions from PyPi are no longer available. Or PIL (which I require a lot for image processing) becomes unavailable, gets forked and has an unclear future with Python 3. RubyGems or Pypi… I believe that both share the same problem of problematic long-term support of module versions. But I know what you mean. Ruby modules are a moving target. The reason I chose Ruby was to have a language others may be familiar with, too. Python is a nice language. I didn't find a web framework that was fun to code with. Pylons is deprecated. Pyramid is plain weird for common web applications (more like a framework for other frameworks and not for people who need to write actual working web applications). Django is nice and widespread but very narrow-minded and arguably not the best choice for an open-minded image database. Rails application seem to be deployable for several years from what I heard of other sysadmins. Of course after a few years you start running into problems again with missing gems, broken dependencies or changes Rails versions. The morale probably is that any non-trivial web application needs some constant love. And there is nothing worse than a deprecated framework. :( And I'm using Padrino at the moment which is much simpler and easier to understand than Rails but is based on Rack and other common and well-maintained gems. So as long as we can get mod_passenger on the server we should be fine. However the project should be fun to complete and maintain. So if anything should force me to program in PHP or Java I'll leave the project. Volunteers like us need to have fun with their work, too. :) With my dsa hat on I suspect we'd be ok to run ruby scripts if their dependencies are in stable and there is some wsgi like means to launch them from apache (or we just proxy to something that got started manually). mod_passenger should work. Preferably with RVM to fulfill the exact Gem requirements. Yes, I know, I'm using DEB packages wherever possible. But especially for non-trivial web applications and fast-moving scripting languages you rarely have the right set of module versions around to operate the application. Yes, reality is stupid. :) …Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL75HUACgkQCV53xXnMZYZghQCeKI3aHaBMv0EtpQh+sMgJJgQ2 adkAoNIxDcAVnyhVpHcpIxlJuXjrN002 =uRMN -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fbe475.1050...@debian.org
Re: SSL for screenshots.debian.net?
On Wed, 12 Feb 2014, Christoph Haas wrote: I will keep that basic concept in the rewrite. However I will use an actively maintained Ruby framework based on Rack so that the operation should be easy for the next 5+ years. I have yet to go through one stable update that doesn't break all of my ruby scripts. If you want stability, then this might not be the best choice. With my dsa hat on I suspect we'd be ok to run ruby scripts if their dependencies are in stable and there is some wsgi like means to launch them from apache (or we just proxy to something that got started manually). Other social features like commenting on packages are second class. Completely agree. Stick to one thing and do that well. Cheers, -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140212210026.gy14...@anguilla.noreply.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 FYI… I have made a little progress. HTTPS with redirection for all requests is now enabled. But I have tuned up the Apache parameters a bit. Now the status looks better: CPU Usage: u430.37 s31.58 cu0 cs0 - 43.3% CPU load 84.2 requests/sec - 1.2 MB/second - 14.3 kB/request 180 requests currently being processed, 170 idle workers __K_CW__K__K_K__K___K__K__KK_KK__KKK_K_R__.. _KKK_C__K_KRKK_K_K_C_RRK_KCK___KC_K_K_K_KK.. KC__KC_KRCK_KK__K__K_KKC__KKW_K_KK_R_CC_KK__K_.. _K_KK__RKKCK_KK__KCK_K___K_WC_K_KCR_K__K___KCK__KK.. KK_K_KKK___K___K_R_KRKK___KKK___KC_C_K_K_R.. ___K__K_KW___KCKKK_KKK_WKK__C_R_WR_K__C__KRK__.. _KRC_K___K_KRCK__KK_CRKK_KKCC__R_W_KK_KK_K__CK.. Scoreboard Key: _ Waiting for Connection, S Starting up, R Reading Request, W Sending Reply, K Keepalive (read), D DNS Lookup, C Closing connection, L Logging, G Gracefully finishing, I Idle cleanup of worker, . Open slot with no current process I was fooled earlier by thinking that . (open slot) means an available socket/listener/thread/whatever. My current configuration is now: IfModule mpm_worker_module StartServers 2 MinSpareThreads 75 MaxSpareThreads 250 ThreadLimit 64 ThreadsPerChild 50 MaxClients 500 MaxRequestsPerChild 0 /IfModule As I said I have never used Apache in a high-performance environment (if you consider 80 req/sec) high. I assume that I should have a considerable amount of connections in _ (waiting for connection) state. Now the system load is still much higher but that's probably because packages.debian.org and packages.ubuntu.com request screenshots as HTTP first and then get redirected which doubles the amount of requests. I should probably exclude /thumbnail and /thumbnail-with-version from the redirection to avoid duplicate requests. Anyway… feel free to make packages.debian.org use HTTPS now. …Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL75e0ACgkQCV53xXnMZYbqrQCdGD3kBqnrAVM1leAuZbNbqtWL L80An2IgO890m1LTRZItQEvrIm4UN3cu =i+0/ -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fbe5ed.3000...@debian.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Paul, Am 11.02.2014 04:39, schrieb Paul Wise: The Debian sysadmins got a report of an ISP in the UK doing DNS hijacks for debian.org, intercepting packages.d.o requests and blocking access to pages about some packages. As a result the Debian sysadmins have added SSL to packages.d.o. Unfortunately it references screenshots.d.n which doesn't have SSL, which means that people visiting over SSL will get mixed content warnings and not be able to view Debian screenshots. Are you able to add an SSL certificate to screenshots.d.n so that the Debian sysadmins can enable http - https redirects and HSTS? Alright, I understand the problem. I'm currently in the process of rewriting the web application behind screenshots.debian.net and expect the new version to go beta in mid-2014. But we probably need to act before that. Besides I don't think I'm using absolute URLs or rewrites anywhere so the current application is probably safe. My main concern is CPU power. The system is running on a sponsored virtual server from the ISP Vexxhost in Canada. And at peak times the load is already around 0.5. I can ask whether they have a kind of SSL accelerator at their disposal. Otherwise I could just set up HTTPS at the Apache level and see how serious the CPU usage will go up. Regarding the certificate: does Debian have resources to buy an SSL certificate? I usually use a free StartCom certificate for my own purposes but I am not sure whether it is suitable for such use. I don't think that the sponsor will donate an SSL certificate either but I'm willing to ask. As soon as we clarified that I will enable HTTPS at screenshots.debian.net in no time. Cheers Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL55zAACgkQCV53xXnMZYZJrwCglGl0LPu3QtGvJZWKzQjUeRIt QYcAoNIQ9zYDKJ9FS1YIEjTncyErxaJ1 =4Jtt -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52f9e733.6000...@debian.org
Re: SSL for screenshots.debian.net?
On Tue, 11 Feb 2014, Christoph Haas wrote: My main concern is CPU power. The system is running on a sponsored virtual server from the ISP Vexxhost in Canada. And at peak times the load is already around 0.5. I can ask whether they have a kind of SSL accelerator at their disposal. Otherwise I could just set up HTTPS at the Apache level and see how serious the CPU usage will go up. I don't think the SSL overhead will be noticeable compared to the load caused by modern scripting languages. Regarding the certificate: does Debian have resources to buy an SSL certificate? I usually use a free StartCom certificate for my own purposes but I am not sure whether it is suitable for such use. I don't think that the sponsor will donate an SSL certificate either but I'm willing to ask. jcristau recently got a cert for france.debian.net from gandi, using the put a file on the webserver method of authentication. Gandi certs are around 12 Euros a year. Cheers, -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140211093839.go14...@anguilla.noreply.org
Re: SSL for screenshots.debian.net?
On Tue, Feb 11, 2014 at 10:38:39 +0100, Peter Palfrader wrote: On Tue, 11 Feb 2014, Christoph Haas wrote: Regarding the certificate: does Debian have resources to buy an SSL certificate? I usually use a free StartCom certificate for my own purposes but I am not sure whether it is suitable for such use. I don't think that the sponsor will donate an SSL certificate either but I'm willing to ask. jcristau recently got a cert for france.debian.net from gandi, using the put a file on the webserver method of authentication. Gandi certs are around 12 Euros a year. startcom requires control of the top level (debian.net) so that didn't work. So I went with gandi, and got a 1-year certificate for 14.40€. Cheers, Julien signature.asc Description: Digital signature
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Am 11.02.2014 11:37, schrieb Julien Cristau: On Tue, Feb 11, 2014 at 10:38:39 +0100, Peter Palfrader wrote: On Tue, 11 Feb 2014, Christoph Haas wrote: Regarding the certificate: does Debian have resources to buy an SSL certificate? I usually use a free StartCom certificate for my own purposes but I am not sure whether it is suitable for such use. I don't think that the sponsor will donate an SSL certificate either but I'm willing to ask. jcristau recently got a cert for france.debian.net from gandi, using the put a file on the webserver method of authentication. Gandi certs are around 12 Euros a year. startcom requires control of the top level (debian.net) so that didn't work. So I went with gandi, and got a 1-year certificate for 14.40€. Thanks for the hint. I will get a Gandi certificate then and make screenshots.debian.net HTTPS-enabled. As soon as I'm done I'll get back to you so you can change the links. Cheers Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL6AKIACgkQCV53xXnMZYZLqQCg8T8NioghbxZpzxl8bI5QMzrR t8cAn3BNjbf51YmOpatS4Ql486mK867D =56LK -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fa00a2.9010...@debian.org
Re: SSL for screenshots.debian.net?
On Tue, 11 Feb 2014, Christoph Haas wrote: Thanks for the hint. I will get a Gandi certificate then and make screenshots.debian.net HTTPS-enabled. As soon as I'm done I'll get back to you so you can change the links. Great, thanks. I suspect long-term we might consider moving it onto d-a hardware. IIRC screenshots is a pylons project. Can it run out of apache using wsgi? Cheers, -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140211105929.gq14...@anguilla.noreply.org
Re: SSL for screenshots.debian.net?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Am 11.02.2014 11:59, schrieb Peter Palfrader: On Tue, 11 Feb 2014, Christoph Haas wrote: Thanks for the hint. I will get a Gandi certificate then and make screenshots.debian.net HTTPS-enabled. As soon as I'm done I'll get back to you so you can change the links. Great, thanks. I suspect long-term we might consider moving it onto d-a hardware. We can do that. However in the past there was little interest in running anything but trivial PHP or Perl/CGI applications on d-a hardware. Glad that the policy was reconsidered. :) Currently it's a WSGI app on Apache. Pylons however has been deprecated and I wasn't fond of the direction that the successor project Pyramid went. So currently I'm programming Ruby and am using the Padrino framework (something in between Sinatra and Rails following a similar mindset as Pylons - just in Ruby). But even then the application can be deployed with mod_passenger and still use the same PostgreSQL database in the background. Nothing too fancy. However the next release is supposed to satisfy constant streams of requests from other Linux distributions to use the web application for their distro - mainly OpenSuSE and CentOS. I considered designing the application so that each distribution could run their own copy but that would mean that every distribution collects their own screenshots and the participation depends on the user base. As you can imagine screenshots.debian.net gets way more uploads from Ubuntu users than from Debian users. There are probably just more desktop users. So without the Ubuntu users we would have fewer screenshots to show on packages.debian.org. Another approach would have been to design the storage backend to exchange screenshots between different instances of the server. For example screenshots.debian.net could get updated from screenshots.ubuntu.com and push them to screenshots.opensuse.org. After lengthy (months!) considerations I decided to write the application as a single-instance system that will be able to deal with virtual hosts properly. screenshots.debian.net and screenshots.ubuntu.com use the same instance and their visual appearance varies by the virtual host name. Also the backend algorithm tries to find screenshots of the requested distribution first before falling back to other distributions. Example: you are looking for a screenshot of Pidgin on screenshots.debian.net. Maybe there was no screenshot from Debian users but an Ubuntu user kindly uploaded a screenshots via screenshots.ubuntu.com. So instead of showing no screenshot the application will fall back to the Ubuntu screenshot. Oh, well, I could go on and on about the concepts… to cut it short: with the current design you would have to host screenshots for other distributions, too, and deal with my stream of updates and patches. Not sure you want to do that. Currently the sponsor is fine with providing the virtual server. Cheers Christoph -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL6B2sACgkQCV53xXnMZYYsfACfb3rRVirqzKHiIBr4x6EFVUYS sZwAnR6iy77+EFktKCSSDbjYOMsOlZBB =Pga6 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fa076b.30...@debian.org
Re: SSL for screenshots.debian.net?
On 11/02/14 at 11:51 +0100, Christoph Haas wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Am 11.02.2014 11:37, schrieb Julien Cristau: On Tue, Feb 11, 2014 at 10:38:39 +0100, Peter Palfrader wrote: On Tue, 11 Feb 2014, Christoph Haas wrote: Regarding the certificate: does Debian have resources to buy an SSL certificate? I usually use a free StartCom certificate for my own purposes but I am not sure whether it is suitable for such use. I don't think that the sponsor will donate an SSL certificate either but I'm willing to ask. jcristau recently got a cert for france.debian.net from gandi, using the put a file on the webserver method of authentication. Gandi certs are around 12 Euros a year. startcom requires control of the top level (debian.net) so that didn't work. So I went with gandi, and got a 1-year certificate for 14.40€. Thanks for the hint. I will get a Gandi certificate then and make screenshots.debian.net HTTPS-enabled. As soon as I'm done I'll get back to you so you can change the links. Hi Christoph, Debian could pay for the certificate, if needed. Especially if the medium-term plan is to move the service to Debian infrastructure. Lucas -- To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140211211532.ga25...@xanadu.blop.info
