Your message dated Sun, 19 Feb 2023 01:03:08 +0000
with message-id <e1pty6e-00aph6...@fasolo.debian.org>
and subject line Bug#1031519: Removed package(s) from unstable
has caused the Debian Bug report #852787,
regarding gitlab: systemd configuration for sidekiq tears down gitlab service
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)
--
852787: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852787
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: gitlab
Severity: important
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hello,
we have low mem instance of gitlab - it used to have 2GB of ram, now it has 3GB.
Sometimes, it runs out of memory. (usually at 22:xx )
OOM killer usually kills one of sidekiq children.
It triggers restart of sidekiq, but also of other gitlab services.
But gitlab-sidekiq is the only one, which is started, other are left dead.
Can this be somehow addressed in systemd service dependecies?
Libor
-----BEGIN PGP SIGNATURE-----
iQJJBAEBCAAzFiEEPGZVVU37tFmB0TQv8O+MbsKfR44FAliLGIQVHGxpYm9yLmts
ZXBhY0BiY29tLmN6AAoJEPDvjG7Cn0eOcGgP/3eJAMqQoBWxG48YXLnfx1D6YZ6h
K4CLe0fpz8rFG87v4qsfyszA5WF3qY3QTffYKXxDJq5Z9QAaV2V+k5Jn0j0Sqhes
IsZqN5g4XvOdyYB6UEy6IGdDhiogiqCeHTz/gRL83WYDb2c+aIykZVYUWSS2MELl
fIpDyIn5FvlRs42VQu0u6x/Wi6b1+j55Y7KWo3j/bET+8MQzg5fOEdIO3qXGTGrf
A5wM0QQtFsOmcvQNWz7hWn7Lm4Fd5Q5U48O0MRiaJNTOUA/OOfoHgRBwHcBAUtvR
2alVQwb9FlH90wdRhUHARJEX10m2+duTvRw4KrP2PHaFcy5l/GJiVp8RV5NrAkHF
Fo9xDrmNjDnCAhjT54gOaudAYW5P9MpqJDHOtcaenvsoK8OHaEFh85kBxYhDj4DA
T8f7qnoHzBsHXzDr3zLMas8TGVfWRc/H7HhhL19WTDwCyXJfm+NBcwYd5RUXysl/
TdZE9K09uhxKtMNbnhAqfPk08XTmTRLObRGx/2U8wWaU8f3X1aZOJLANOQUKuq2Q
57eEIQ3Ed8eY8AUN95jfxq5s9QbgLhwGf8hl6eyzXKrAtHxq/coeN2rNzdaEYIac
A7VRgk4id1qnwqdAFgy6DoNTzqK2vcAjy5Ix+DmoFPHmaxqIuXj6O9pNr5+/9lL4
/0DxTlrb5zieOvBi
=h2TC
-----END PGP SIGNATURE-----
Jan 26 22:15:18 gitlab kernel: kworker/0:0 invoked oom-killer:
gfp_mask=0x24200c2(GFP_HIGHUSER), order=0, oom_score_adj=0
Jan 26 22:15:18 gitlab kernel: kworker/0:0 cpuset=/ mems_allowed=0
Jan 26 22:15:18 gitlab kernel: CPU: 0 PID: 9360 Comm: kworker/0:0 Not tainted
4.8.0-2-amd64 #1 Debian 4.8.15-2
Jan 26 22:15:18 gitlab kernel: Hardware name: VMware, Inc. VMware Virtual
Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
Jan 26 22:15:18 gitlab kernel: Workqueue: events_freezable vmballoon_work
[vmw_balloon]
Jan 26 22:15:18 gitlab kernel: 0000000000000286 000000001c838855
ffffffff8831f925 ffff972ef1ab3d08
Jan 26 22:15:18 gitlab kernel: ffff972f38b02080 ffffffff881f7491
0000000000000206 ffffffff8818f2e9
Jan 26 22:15:18 gitlab kernel: ffff972f3fffccc0 0000000c00000246
ffff972f2d66c080 ffff972f38b02080
Jan 26 22:15:18 gitlab kernel: Call Trace:
Jan 26 22:15:18 gitlab kernel: [<ffffffff8831f925>] ? dump_stack+0x5c/0x77
Jan 26 22:15:18 gitlab kernel: [<ffffffff881f7491>] ? dump_header+0x59/0x1dc
Jan 26 22:15:18 gitlab kernel: [<ffffffff8818f2e9>] ?
do_try_to_free_pages+0x269/0x350
Jan 26 22:15:18 gitlab kernel: [<ffffffff8817a392>] ?
oom_kill_process+0x222/0x3e0
Jan 26 22:15:18 gitlab kernel: [<ffffffff8817a8fc>] ? out_of_memory+0x35c/0x3f0
Jan 26 22:15:18 gitlab kernel: [<ffffffff8817f987>] ?
__alloc_pages_nodemask+0xd67/0xda0
Jan 26 22:15:18 gitlab kernel: [<ffffffff880a7101>] ? update_curr+0xe1/0x160
Jan 26 22:15:18 gitlab kernel: [<ffffffff881d0221>] ?
alloc_pages_current+0x91/0x140
Jan 26 22:15:18 gitlab kernel: [<ffffffffc04fcf5f>] ?
vmballoon_work+0x47f/0x704 [vmw_balloon]
Jan 26 22:15:18 gitlab kernel: [<ffffffff8808dc90>] ?
process_one_work+0x160/0x410
Jan 26 22:15:18 gitlab kernel: [<ffffffff8808df8d>] ? worker_thread+0x4d/0x480
Jan 26 22:15:18 gitlab kernel: [<ffffffff8808df40>] ?
process_one_work+0x410/0x410
Jan 26 22:15:18 gitlab kernel: [<ffffffff88093d5d>] ? kthread+0xcd/0xf0
Jan 26 22:15:18 gitlab kernel: [<ffffffff885e8eef>] ? ret_from_fork+0x1f/0x40
Jan 26 22:15:18 gitlab kernel: [<ffffffff88093c90>] ?
kthread_create_on_node+0x190/0x190
Jan 26 22:15:18 gitlab kernel: Mem-Info:
Jan 26 22:15:18 gitlab kernel: active_anon:98986 inactive_anon:102318
isolated_anon:0
active_file:23 inactive_file:47
isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
slab_reclaimable:3358 slab_unreclaimable:9520
mapped:1580 shmem:1984 pagetables:3919
bounce:0
free:14226 free_pcp:30 free_cma:0
Jan 26 22:15:18 gitlab kernel: Node 0 active_anon:395944kB
inactive_anon:409272kB active_file:92kB inactive_file:188kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB mapped:6320kB dirty:0kB writeback:0kB
shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 7936kB
writeback_tmp:0kB unstable:0kB pages_scanned:0 all_unreclaimable? no
Jan 26 22:15:18 gitlab kernel: Node 0 DMA free:12136kB min:232kB low:288kB
high:344kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB
bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 26 22:15:18 gitlab kernel: lowmem_reserve[]: 0 2976 2976 2976 2976
Jan 26 22:15:18 gitlab kernel: Node 0 DMA32 free:44768kB min:44820kB
low:56024kB high:67228kB active_anon:395944kB inactive_anon:409272kB
active_file:92kB inactive_file:188kB unevictable:0kB writepending:0kB
present:3129280kB managed:3068696kB mlocked:0kB slab_reclaimable:13432kB
slab_unreclaimable:38080kB kernel_stack:3648kB pagetables:15676kB bounce:0kB
free_pcp:120kB local_pcp:120kB free_cma:0kB
Jan 26 22:15:18 gitlab kernel: lowmem_reserve[]: 0 0 0 0 0
Jan 26 22:15:18 gitlab kernel: Node 0 DMA: 0*4kB 1*8kB (U) 0*16kB 1*32kB (U)
1*64kB (U) 0*128kB 1*256kB (U) 1*512kB (U) 1*1024kB (U) 1*2048kB (M) 2*4096kB
(ME) = 12136kB
Jan 26 22:15:18 gitlab kernel: Node 0 DMA32: 10060*4kB (UME) 576*8kB (UM)
0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB =
44848kB
Jan 26 22:15:18 gitlab kernel: Node 0 hugepages_total=0 hugepages_free=0
hugepages_surp=0 hugepages_size=2048kB
Jan 26 22:15:18 gitlab kernel: 12168 total pagecache pages
Jan 26 22:15:18 gitlab kernel: 10120 pages in swap cache
Jan 26 22:15:18 gitlab kernel: Swap cache stats: add 987900, delete 977780,
find 2147165/2439256
Jan 26 22:15:18 gitlab kernel: Free swap = 0kB
Jan 26 22:15:18 gitlab kernel: Total swap = 999420kB
Jan 26 22:15:18 gitlab kernel: 786318 pages RAM
Jan 26 22:15:18 gitlab kernel: 0 pages HighMem/MovableOnly
Jan 26 22:15:18 gitlab kernel: 15167 pages reserved
Jan 26 22:15:18 gitlab kernel: 0 pages hwpoisoned
Jan 26 22:15:18 gitlab kernel: [ pid ] uid tgid total_vm rss nr_ptes
nr_pmds swapents oom_score_adj name
Jan 26 22:15:18 gitlab kernel: [ 253] 0 253 14533 80 24
3 82 0 systemd-journal
Jan 26 22:15:18 gitlab kernel: [ 276] 0 276 26800 0 19
4 85 0 lvmetad
Jan 26 22:15:18 gitlab kernel: [ 280] 0 280 11688 12 24
4 503 -1000 systemd-udevd
Jan 26 22:15:18 gitlab kernel: [ 475] 0 475 11639 32 27
3 129 0 systemd-logind
Jan 26 22:15:18 gitlab kernel: [ 476] 0 476 6991 0 19
4 54 0 atd
Jan 26 22:15:18 gitlab kernel: [ 477] 0 477 62528 62 29
4 226 0 rsyslogd
Jan 26 22:15:18 gitlab kernel: [ 482] 105 482 11278 50 26
3 67 -900 dbus-daemon
Jan 26 22:15:18 gitlab kernel: [ 513] 0 513 6997 32 18
3 46 0 cron
Jan 26 22:15:18 gitlab kernel: [ 530] 0 530 51506 102 57
3 258 0 vmtoolsd
Jan 26 22:15:18 gitlab kernel: [ 537] 0 537 1054 0 8
3 30 0 acpid
Jan 26 22:15:18 gitlab kernel: [ 719] 108 719 5949 14 18
3 90 -500 nrpe
Jan 26 22:15:18 gitlab kernel: [ 728] 0 728 17489 11 39
3 183 -1000 sshd
Jan 26 22:15:18 gitlab kernel: [ 737] 0 737 3662 0 11
3 34 0 agetty
Jan 26 22:15:18 gitlab kernel: [ 756] 110 756 25549 70 26
3 108 0 ntpd
Jan 26 22:15:18 gitlab kernel: [ 765] 0 765 12592 105 32
4 2247 0 munin-node
Jan 26 22:15:18 gitlab kernel: [ 818] 114 818 10828 365 19
3 1130 0 redis-server
Jan 26 22:15:18 gitlab kernel: [ 831] 113 831 68188 329 57
3 369 -900 postgres
Jan 26 22:15:18 gitlab kernel: [ 832] 113 832 72342 376 67
3 402 -900 postgres
Jan 26 22:15:18 gitlab kernel: [ 841] 0 841 20287 17 28
3 185 0 master
Jan 26 22:15:18 gitlab kernel: [ 852] 109 852 20845 20 32
3 199 0 qmgr
Jan 26 22:15:18 gitlab kernel: [ 859] 0 859 24571 26 53
3 453 0 apache2
Jan 26 22:15:18 gitlab kernel: [ 919] 33 919 5393 18 15
3 32 0 htcacheclean
Jan 26 22:15:18 gitlab kernel: [ 922] 113 922 72342 77 54
3 420 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 923] 113 923 72342 82 55
3 414 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 924] 113 924 72342 64 54
3 422 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 925] 113 925 72454 228 57
3 445 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 926] 113 926 36092 47 52
3 412 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 939] 113 939 68221 329 62
3 399 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 940] 113 940 68188 117 50
3 378 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 941] 113 941 68188 61 51
3 388 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 942] 113 942 68282 276 53
3 396 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 943] 113 943 32010 106 46
3 365 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1130] 115 1130 404015 64570 410
6 56881 0 sidekiq
Jan 26 22:15:18 gitlab kernel: [ 1133] 115 1133 160074 7295 269
4 55461 0 unicorn_rails
Jan 26 22:15:18 gitlab kernel: [ 1134] 115 1134 28119 23 58
3 5872 0 mail_room
Jan 26 22:15:18 gitlab kernel: [ 1135] 115 1135 31701 423 21
5 74 0 gitlab-workhors
Jan 26 22:15:18 gitlab kernel: [ 1613] 113 1613 70206 481 59
3 747 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1698] 115 1698 175756 53397 295
4 23918 0 unicorn_rails
Jan 26 22:15:18 gitlab kernel: [ 1701] 115 1701 178468 62348 298
4 16520 0 unicorn_rails
Jan 26 22:15:18 gitlab kernel: [ 1784] 113 1784 104096 4142 135
3 28366 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1793] 113 1793 109673 5080 148
4 33780 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1825] 113 1825 71767 854 71
3 1486 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1826] 113 1826 71698 891 71
3 1367 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1870] 113 1870 72056 853 72
3 1724 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1963] 113 1963 72127 730 72
3 1942 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1975] 113 1975 71909 970 71
3 1503 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1976] 113 1976 72064 759 72
3 1813 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1977] 113 1977 71127 721 70
3 1449 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1978] 113 1978 72159 905 72
3 1775 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1979] 113 1979 72078 929 72
3 1667 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1983] 113 1983 71024 850 70
3 1254 0 postgres
Jan 26 22:15:18 gitlab kernel: [ 1990] 113 1990 71668 831 71
3 1890 0 postgres
Jan 26 22:15:18 gitlab kernel: [30182] 33 30182 326710 450 97
5 1507 0 apache2
Jan 26 22:15:18 gitlab kernel: [30183] 33 30183 326630 477 97
4 1356 0 apache2
Jan 26 22:15:18 gitlab kernel: [18263] 109 18263 20803 194 31
3 0 0 pickup
Jan 26 22:15:18 gitlab kernel: Out of memory: Kill process 1130 (sidekiq) score
119 or sacrifice child
Jan 26 22:15:18 gitlab kernel: Killed process 1130 (sidekiq)
total-vm:1616060kB, anon-rss:258280kB, file-rss:0kB, shmem-rss:0kB
Jan 26 22:15:18 gitlab kernel: oom_reaper: reaped process 1130 (sidekiq), now
anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Jan 26 22:15:18 gitlab systemd[1]: gitlab-sidekiq.service: Main process exited,
code=killed, status=9/KILL
Jan 26 22:15:18 gitlab systemd[1]: gitlab-sidekiq.service: Unit entered failed
state.
Jan 26 22:15:18 gitlab systemd[1]: gitlab-sidekiq.service: Failed with result
'signal'.
Jan 26 22:15:18 gitlab systemd[1]: Stopped target GitLab Service.
Jan 26 22:15:18 gitlab systemd[1]: Stopped GitLab Services.
Jan 26 22:15:18 gitlab systemd[1]: Stopping Gitlab mailroom Worker...
Jan 26 22:15:18 gitlab systemd[1]: Stopping Gitlab Workhorse handles slow HTTP
requests for Gitlab....
Jan 26 22:15:18 gitlab systemd[1]: Stopped Gitlab Workhorse handles slow HTTP
requests for Gitlab..
Jan 26 22:15:19 gitlab systemd[1]: Stopped Gitlab mailroom Worker.
Jan 26 22:15:19 gitlab systemd[1]: Stopping GitLab Unicorn Server...
Jan 26 22:15:20 gitlab systemd[1]: Stopped GitLab Unicorn Server.
Jan 26 22:15:28 gitlab systemd[1]: gitlab-sidekiq.service: Service hold-off
time over, scheduling restart.
Jan 26 22:15:28 gitlab systemd[1]: Stopped GitLab Sidekiq Worker.
Jan 26 22:15:28 gitlab systemd[1]: Starting GitLab Sidekiq Worker...
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.243Z 18868
TID-gn57l97t8 INFO: Booting Sidekiq 4.2.3 with redis options
{:host=>"localhost", :port=>6379, :password=>nil, :namespace=>"resque:gitlab",
:url=>nil}
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.295Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: stuck_ci_builds_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.301Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: expire_build_artifacts_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.310Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: repository_check_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.320Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: admin_email_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.325Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name:
repository_archive_cache_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.330Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name:
import_export_project_cleanup_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.334Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: requests_profiles_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.339Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: remove_expired_members_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.344Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name:
remove_expired_group_links_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.347Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: prune_old_events_worker
Jan 26 22:15:47 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:15:47.354Z 18868
TID-gn57l97t8 INFO: Cron Jobs - add job with name: trending_projects_worker
Jan 26 22:16:01 gitlab systemd[1]: Started GitLab Sidekiq Worker.
Jan 26 22:16:04 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:16:04.347Z 18868
TID-gn57l97t8 INFO: Running in ruby 2.3.3p222 (2016-11-21) [x86_64-linux-gnu]
Jan 26 22:16:04 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:16:04.347Z 18868
TID-gn57l97t8 INFO: See LICENSE and the LGPL-3.0 for licensing details.
Jan 26 22:16:04 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:16:04.347Z 18868
TID-gn57l97t8 INFO: Upgrade to Sidekiq Pro for more features and support:
http://sidekiq.org
Jan 26 22:16:04 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:16:04.354Z 18868
TID-gn57l97t8 INFO: Starting processing, hit Ctrl-C to stop
Jan 26 22:20:05 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:20:05.289Z 18868
TID-gn58yenr0 RepositoryCheck::BatchWorker JID-5d784fb4b3f6f9c8f0381712 INFO:
start
Jan 26 22:20:05 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:20:05.292Z 18868
TID-gn58ydpxs INFO: Cron Jobs - add job with name: repository_check_worker
Jan 26 22:20:05 gitlab gitlab-sidekiq[18868]: 2017-01-26T21:20:05.325Z 18868
TID-gn58yenr0 RepositoryCheck::BatchWorker JID-5d784fb4b3f6f9c8f0381712 INFO:
done: 0.036 sec
--- End Message ---
--- Begin Message ---
Version: 13.4.7-2+rm
Dear submitter,
as the package gitlab has just been removed from the Debian archive
unstable we hereby close the associated bug reports. We are sorry
that we couldn't deal with your issue properly.
For details on the removal, please see https://bugs.debian.org/1031519
The version of this package that was in Debian prior to this removal
can still be found using https://snapshot.debian.org/.
Please note that the changes have been done on the master archive and
will not propagate to any mirrors until the next dinstall run at the
earliest.
This message was generated automatically; if you believe that there is
a problem with it please contact the archive administrators by mailing
ftpmas...@ftp-master.debian.org.
Debian distribution maintenance software
pp.
Scott Kitterman (the ftpmaster behind the curtain)
--- End Message ---
