Bug#851194: dgit: please document how to set up dgit infrastructure.
My previous server setup instructions had a problem. nginx and fcgiwrap have
undesirable buffering which causes clones of large repos to time out.
To fix this first apply the patch from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863478 to fcgiwrap and
build/install it.
Then modify the "location ~
^.*\.git/(HEAD|info/refs|objects/info/.*|git-(upload|receive)-pack)$ {" block in
your nginx conf. replace
include fastcgi_params;
with
#note: NO_BUFFERING relies on a patched fcgiwrap.
fastcgi_param NO_BUFFERING 1;
include fastcgi_params;
gzip off;
fastcgi_buffering off;
Restart nginx and fcgiwrap and you should be able to clone large repos.
You might also want to increase the number of allowed cgi processes. The
fcgiwrap default is to only allow one. The documentation for fcgiwrap says this
can be done though /etc/default/fcgiwrap but that doesn't seem to work with
systemd.
Instead it seems you need to
cp /lib/systemd/system/fcgiwrap.service /etc/systemd/system/fcgiwrap.service
edit /etc/systemd/system/fcgiwrap.service and add -c to the execstart
parameter
systemctl daemon-reload
service fcgiwrap restart
Bug#851194: dgit: please document how to set up dgit infrastructure.
And here is the stuff on the public side.
I assume you already have nginx working. You will need to change the distro
name and any IP addresses and hostnames to suit your setup.
Install fastcgiwrap git and gitweb
Point dns for your dgit hostname at the server.
Add a server block to your nginx config for the dgit server.
server {
listen 5.153.225.206:80;
listen [2001:41c9:1:3ce::10]:80;
listen 5.153.225.206:443 ssl;
listen [2001:41c9:1:3ce::10]:443 ssl;
server_name dgit.raspbian.org;
server_name dgit-bm.raspbian.org;
#static files needed by gitweb
location /static {
alias /usr/share/gitweb/static/;
autoindex on;
}
#config based on
http://weininger.net/configuration-of-nginx-for-gitweb-and-git-http-backend.html
# static repo files for cloning over https
location ~
^.*\.git/objects/([0-9a-f]+/[0-9a-f]+|pack/pack-[0-9a-f]+.(pack|idx))$ {
root /home/dgit/dispatch-dir/distro=raspbian/repos;
}
# requests that need to go to git-http-backend
location ~
^.*\.git/(HEAD|info/refs|objects/info/.*|git-(upload|receive)-pack)$ {
root /home/git/repositories;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
fastcgi_param SCRIPT_FILENAME /usr/lib/git-core/git-http-backend;
fastcgi_param PATH_INFO $uri;
fastcgi_param GIT_PROJECT_ROOT
/home/dgit/dispatch-dir/distro=raspbian/repos;
fastcgi_param GIT_HTTP_EXPORT_ALL "";
fastcgi_param REMOTE_USER $remote_user;
include fastcgi_params;
}
# send anything else to gitweb if it's not a real file
try_files $uri @gitweb;
location @gitweb {
fastcgi_pass unix:/var/run/fcgiwrap.socket;
fastcgi_param SCRIPT_FILENAME /usr/share/gitweb/gitweb.cgi;
fastcgi_param PATH_INFO $uri;
fastcgi_param GITWEB_CONFIG /etc/gitweb.conf;
include fastcgi_params;
}
}
edit gitweb.conf to point it at the git repos. In our case this was
$projectroot = "/home/dgit/dispatch-dir/distro=raspbian/repos";
Bug#851194: dgit: please document how to set up dgit infrastructure.
This is a draft of how I set up the private side of the dgit server and client
for raspbian. It may be incomplete. I have still to document the setup for the
public side of the dgit server.
I have acheived a succesful push of a patched xen package with this
configuration.
(replace raspbian and raspbian-related urls in these instructions with the name
and urls of your distro)
server push setup
add a user dgit
create /home/dgit/ssh-wrap with the following contents
#!/bin/sh
set -e
umask 002
srvdir=/home/dgit
dispatchdir=$srvdir/dispatch-dir
#dgitlive=$srvdir/dgit-live
PERLLIB="$dgitlive${PERLLIB+:}${PERLLIB}" \
#exec $dgitlive/infra/dgit-ssh-dispatch $dispatchdir
exec dgit-ssh-dispatch $dispatchdir
create /home/dgit/dispatch-dir/distro=raspbian
in that directory put
a subdirectory called repos with a subdirectory called _template containing a
bare git repo all owned by user dgit
a file called keyring.gpg containing the gpg keys with access to push to the
repo
(you can import keys to the keyring with gpg --no-default-keyring --keyring
dispatch-dir/distro\=raspbian/keyring.gpg --import )
a file called policy-hook containing a copy of
/usr/bin/dgit-repos-policy-trusting
a file called suites containing a list of allowed suites
in /home/dgit/.ssh/authorized-keys add lines like
command="/home/dgit/ssh-wrap" ssh-rsa
The dgit server has a commit check, unfortunately I found that this commit
check seems to be too strict (see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851716 ). There is supposedly
a way to disable this through the policy hook but I couldn't make that work.
So as a temporary soloution I just commented out that block of code (it can be
found in /usr/bin/dgit-repos-server and starts with if (!($policy &
NOCOMMITCHECK)) { )
Client setup
#!/bin/sh
git config dgit-distro.raspbian.git-url https://dgit.raspbian.org/
git config dgit-distro.raspbian.git-url-suffix .git
git config dgit-distro.raspbian/push.git-url ""
git config dgit-distro.raspbian/push.git-host dgit.raspbian.org
git config dgit-distro.raspbian/push.git-user-force dgit
git config dgit-distro.raspbian/push.git-proto "git+ssh://"
git config dgit-distro.raspbian/push.git-path "/dgit/raspbian/repos"
git config dgit-distro.raspbian.git-check "true"
git config dgit-distro.raspbian.git-check-suffix "/info/refs"
git config dgit-distro.raspbian/push.git-check "ssh-cmd"
git config dgit-distro.raspbian/push.git-create "true"
git config dgit-distro.raspbian.upload-host raspbian
git config dgit-distro.raspbian.mirror http://archive.raspbian.org/raspbian
git config dgit-distro.raspbian.archive-query "aptget:"
git config dgit-suite.wheezy-staging.distro raspbian
git config dgit-suite.jessie-staging.distro raspbian
git config dgit-suite.stretch-staging.distro raspbian
Bug#851194: dgit: please document how to set up dgit infrastructure.
Control: forcemerge -1 842643
peter green writes ("Bug#851194: dgit: please document how to set up dgit
infrastructure."):
> Please document how to set up a dgit server and how to configure the
> dgit client to use that server so that dgit can be used for
> downstream distros and local packages.
Since this is now in your way, I am implicitly raising the priority of
the other bug (#842643), which was previously regarded as a wishlist
item.
Regards,
Ian.
--
Ian Jackson <ijack...@chiark.greenend.org.uk> These opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Bug#851194: dgit: please document how to set up dgit infrastructure.
package: dgit Please document how to set up a dgit server and how to configure the dgit client to use that server so that dgit can be used for downstream distros and local packages.
