Package: gitlab Version: 8.4.3+dfsg-6 Severity: grave Justification: renders package unusable
Hi, this problem appeared after my upgrade to 8.4.3+dfsg-6 without me changing anything in my configuration, so I assume that this must be gitlab's fault, thus the high severity. The symptom is, that since after the upgrade (and after manually creating the /var/run/gitlab directories as I described in #814713), I only get the following message in my browser when trying to access the website: 502 Whoops, GitLab is taking too much time to respond. Try refreshing the page, or going back and attempting the action again. Please contact your GitLab administrator if this problem persists. Now I *am* the gitlab administrator and I fail to figure out how to find the source of the problem. My logs show the following: root@22gitlab:~# tail -n10000 /usr/share/gitlab/log/* /var/log/gitlab/* ==> /usr/share/gitlab/log/mail_room.log <== fatal: Not a git repository (or any of the parent directories): .git ==> /usr/share/gitlab/log/production.log <== ** [Raven] Raven 0.15.3 configured not to send errors. ** [Raven] Raven 0.15.3 configured not to send errors. ** [Raven] Raven 0.15.3 configured not to send errors. ** [Raven] Raven 0.15.3 configured not to send errors. Scoped order and limit are ignored, it's forced to be batch order and batch size ** [Raven] Raven 0.15.3 configured not to send errors. Scoped order and limit are ignored, it's forced to be batch order and batch size ==> /var/log/gitlab/sidekiq.log <== No such file or directory @ rb_sysopen - /var/run/gitlab/pids/sidekiq.pid /usr/lib/ruby/vendor_ruby/sidekiq/cli.rb:355:in `initialize' /usr/lib/ruby/vendor_ruby/sidekiq/cli.rb:355:in `open' /usr/lib/ruby/vendor_ruby/sidekiq/cli.rb:355:in `write_pid' /usr/lib/ruby/vendor_ruby/sidekiq/cli.rb:42:in `parse' /usr/bin/sidekiq:12:in `<main>' Pidfile /var/run/gitlab/pids/sidekiq.pid does not exist 2016-02-14T15:21:41.835Z 293 TID-b4ak0 INFO: Booting Sidekiq 4.0.1 with redis options {:url=>"redis://localhost:6379", :namespace=>"resque:gitlab"} 2016-02-14T15:21:41.983Z 293 TID-b4ak0 INFO: Cron Jobs - add job with name: stuck_ci_builds_worker fatal: Not a git repository (or any of the parent directories): .git 2016-02-14T15:21:57.436Z 293 TID-b4ak0 INFO: Running in ruby 2.2.4p230 (2015-12-16) [x86_64-linux-gnu] 2016-02-14T15:21:57.436Z 293 TID-b4ak0 INFO: See LICENSE and the LGPL-3.0 for licensing details. 2016-02-14T15:21:57.436Z 293 TID-b4ak0 INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org ==> /var/log/gitlab/unicorn.stderr.log <== I, [2016-02-14T16:21:07.730554 #287] INFO -- : Refreshing Gem list fatal: Not a git repository (or any of the parent directories): .git I, [2016-02-14T16:21:58.169242 #287] INFO -- : listening on addr=/var/run/gitlab/pids/gitlab.socket fd=12 I, [2016-02-14T16:21:58.170435 #287] INFO -- : listening on addr=127.0.0.1:8080 fd=15 I, [2016-02-14T16:21:58.195925 #346] INFO -- : worker=0 ready I, [2016-02-14T16:21:58.205334 #349] INFO -- : worker=1 ready I, [2016-02-14T16:21:58.220556 #287] INFO -- : master process ready I, [2016-02-14T16:21:58.223929 #352] INFO -- : worker=2 ready ==> /var/log/gitlab/unicorn.stdout.log <== Does there exist a way to increase the verbosity of the logging so that I can see what causes the 502 problem? Here is the (shortened, because I have lots of repositories) output of gitlab:check: fatal: Not a git repository (or any of the parent directories): .git Checking GitLab Shell ... GitLab Shell version >= 2.6.10 ? ... OK (2.6.10) Repo base directory exists? ... yes Repo base directory is a symlink? ... no Repo base owned by gitlab:gitlab? ... yes Repo base access is drwxrws---? ... yes hooks directories in repos are links: ... [...] Running /usr/share/gitlab-shell/bin/check Check GitLab API access: FAILED. code: 502 gitlab-shell self-check failed Try fixing it: Make sure GitLab is running; Check the gitlab-shell configuration file: sudo -u gitlab -H editor /usr/share/gitlab-shell/config.yml Please fix the error above and rerun the checks. Checking GitLab Shell ... Finished Checking Sidekiq ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking Reply by email ... Reply by email is disabled in config/gitlab.yml Checking Reply by email ... Finished Checking LDAP ... LDAP is disabled in config/gitlab.yml Checking LDAP ... Finished Checking GitLab ... Git configured with autocrlf=input? ... yes Database config exists? ... yes Database is SQLite ... no All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config outdated? ... no Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory setup correctly? ... skipped (no tmp uploads folder yet) Init script exists? ... yes Init script up-to-date? ... yes projects have namespace: ... [...] Redis version >= 2.8.0? ... yes Ruby version >= 2.1.0 ? ... yes (2.2.4) Your git bin path is "/usr/bin/git" Git version >= 1.7.10 ? ... yes (2.1.4) Active users: 16 Checking GitLab ... Finished So this is all green except for the gitlab-shell/bin/check output. But I guess it's expected that this is red because I cannot access the web interface through my browser either, right? I'd appreciate some advise in how to tell you more about the reasons for this problem. Thanks! cheers, josch