Your message dated Sun, 19 Feb 2023 01:03:08 +0000
with message-id <e1pty6e-00aph6...@fasolo.debian.org>
and subject line Bug#1031519: Removed package(s) from unstable
has caused the Debian Bug report #1015301,
regarding gitlab: Upgrade to 15.0.4 fails because BackgroundMigrations could 
not finalize
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
1015301: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1015301
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: gitlab
Version: 15.0.4+ds1-1~fto11+2
Severity: important

Dear Maintainer,

I just tried to upgrade to Gitlab 15.0.4 on my two instances. While I
had no issues in my smaller instance, my bigger one had some issues.

First, postinst failed with the message

    /etc/systemd/system/gitaly.service.d/override.conf already exist

I moved the file away and the upgrade could continue. At the end of
the upgrade, gitaly did not start anymore, so I restored the file (it
actually contains the setting of the user running gitaly).

Then, however, I stumbled upon a much more serious issue: The database
migration could not finish as there were pending background jobs that
failed to finalize:

    gitlab_production database is not empty, skipping gitlab setup
    Attention: used pure ruby version of MurmurHash3
    /usr/share/gitlab/lib/gitlab.rb:47: warning: already initialized constant 
Gitlab::APP_DIRS_PATTERN
    /usr/share/gitlab/lib/gitlab.rb:47: warning: previous definition of 
APP_DIRS_PATTERN was here
    /usr/share/gitlab/lib/gitlab.rb:48: warning: already initialized constant 
Gitlab::VERSION
    /usr/share/gitlab/lib/gitlab.rb:48: warning: previous definition of VERSION 
was here
    /usr/share/gitlab/lib/gitlab.rb:49: warning: already initialized constant 
Gitlab::INSTALLATION_TYPE
    /usr/share/gitlab/lib/gitlab.rb:49: warning: previous definition of 
INSTALLATION_TYPE was here
    /usr/share/gitlab/lib/gitlab.rb:50: warning: already initialized constant 
Gitlab::HTTP_PROXY_ENV_VARS
    /usr/share/gitlab/lib/gitlab.rb:50: warning: previous definition of 
HTTP_PROXY_ENV_VARS was here
    == 20220213103859 RemoveIntegrationsType: migrating 
===========================
    rake aborted!
    StandardError: An error has occurred, all later migrations canceled:

    
Gitlab::Database::BackgroundMigration::BatchedMigrationRunner::FailedToFinalize
    …

I did some research, and found somebody else having a similar issue
[1]. Unfortunately, manually running the background jobs [2] did not
work either as they continued to fail. In the database I identified
the following stuck background migrations [3]:


    gitlab_production=# select id,status,job_class_name, table_name, 
column_name, job_arguments from batched_background_migrations where status <> 3;
     id | status |                  job_class_name                  |  
table_name  | column_name | job_arguments 
    
----+--------+--------------------------------------------------+--------------+-------------+---------------
     17 |      4 | BackfillNamespaceIdForNamespaceRoute             | routes    
   | id          | []
     19 |      4 | BackfillMemberNamespaceForGroupMembers           | members   
   | id          | []
     20 |      4 | MigratePersonalNamespaceProjectMaintainerToOwner | members   
   | id          | []
     23 |      4 | BackfillGroupFeatures                            | 
namespaces   | id          | [10000]
     15 |      4 | BackfillIntegrationsTypeNew                      | 
integrations | id          | []
     16 |      4 | BackfillUserNamespace                            | 
namespaces   | id          | []
     18 |      4 | BackfillIssueSearchData                          | issues    
   | id          | []


I then proceeded to simply change the status of the jobs to 3 in the
database as proposed in the issue mentioned above [1]. I could then
finish the upgrade normally with `apt upgrade`.

Afterwards, I simply undid the database change (i.e., reverted the
status of the failed background migrations to 4) and then restarted
the migrations in the web UI.

As far as I can tell, everything seems normal now. The seven
background migration jobs are still pending, but I can continue to use
Gitlab normally.

Do you need any more information on the matter?

Thanks for your investigation into the issue!

Best,
Maximilian


[1]: https://gitlab.com/gitlab-org/gitlab/-/issues/340193
[2]: 
https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#manually-finishing-a-batched-background-migration
[3]: https://docs.gitlab.com/ee/update/index.html#batched-background-migrations

--- End Message ---
--- Begin Message ---
Version: 13.4.7-2+rm

Dear submitter,

as the package gitlab has just been removed from the Debian archive
unstable we hereby close the associated bug reports.  We are sorry
that we couldn't deal with your issue properly.

For details on the removal, please see https://bugs.debian.org/1031519

The version of this package that was in Debian prior to this removal
can still be found using https://snapshot.debian.org/.

Please note that the changes have been done on the master archive and
will not propagate to any mirrors until the next dinstall run at the
earliest.

This message was generated automatically; if you believe that there is
a problem with it please contact the archive administrators by mailing
ftpmas...@ftp-master.debian.org.

Debian distribution maintenance software
pp.
Scott Kitterman (the ftpmaster behind the curtain)

--- End Message ---

Reply via email to