[Sts-sponsors] [Bug 1910307] Re: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main)
I built 1.9.0-1ubuntu0+lp1910307v20210113b2 in https://launchpad.net/~mruffell/+archive/ubuntu/lp1910307-test, and contains the identical code to the debdiff above. It builds successfully on all supported architectures, which also includes riscv64, which means the patch we merged in still works as intended. Ready to merge. -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1910307 Title: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main) Status in librelp package in Ubuntu: In Progress Status in librelp source package in Hirsute: In Progress Bug description: [Impact] librelp 1.9.0-1 is available in Debian, and is much newer than what we carry in Ubuntu 1.5.0-1ubuntu2. This bug tracks the merge to the newer version. This bug is needed for the SRU of bug 1908473, which fixes a file descriptor leak in librelp. The leak was fixed in 1.7.0, so the merge to 1.9.0 contains the fix already. [Testcase] Test packages are available in the following ppa, with builds for all supported architectures, with debug symbols: https://launchpad.net/~mruffell/+archive/ubuntu/lp1910307-test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
Re: [Sts-sponsors] Please review and consider sponsoring LP1910307 and LP1908473 (librelp)
Hi Eric, I built the merge package in the below ppa with all architectures enabled, and I got William Grant to enable riscv64. https://launchpad.net/~mruffell/+archive/ubuntu/lp1910307-test All packages built successfully, so the riscv64 patch still works as intended. The changelog in the debdiff has been updated as well. Please sponsor the merge. Thanks, Matthew On Wed, Jan 13, 2021 at 9:23 AM Eric Desrochers wrote: > > Great, thanks! > > On Tue, Jan 12, 2021 at 3:20 PM Matthew Ruffell > wrote: >> >> > Did you talk to William Grant ? I would assume that if you did the merge, >> > it was okay with you doing it and he had no plan to merge it anytime soon ? >> > >> Yes, I talked to William. He did not have any attachment to the >> package at all, and wasn't planning on merging anytime soon. See >> attached picture. >> >> >> > On Tue, Jan 12, 2021 at 12:12 AM Matthew Ruffell >> > wrote: >> >> >> >> Hi Dan, >> >> >> >> If you have some time, can you please review my merge request for >> >> librelp 1.9 to hirsute? >> >> >> >> https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 >> >> >> >> Thanks, >> >> Matthew >> >> >> >> On Thu, Jan 7, 2021 at 1:45 AM Mauricio Oliveira >> >> wrote: >> >> > >> >> > Hey Matthew, >> >> > >> >> > Great work on those. >> >> > >> >> > I'd be happy to review/sponsor for the SRUs after the merge is released. >> >> > Could you please ping me once that happens? and I'll take care of it. >> >> > >> >> > cheers, >> >> > >> >> > On Wed, Jan 6, 2021 at 2:12 AM Matthew Ruffell >> >> > wrote: >> >> > > >> >> > > Hello Eric, >> >> > > >> >> > > I followed the instructions you posted on Mattermost yesterday on how >> >> > > to perform >> >> > > a merge. >> >> > > >> >> > > The merge bug for librelp is: >> >> > > >> >> > > https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 >> >> > > >> >> > > I have attached debdiffs from Debian, and the last Ubuntu version, >> >> > > just like >> >> > > the merge document mentioned. Please review and merge if it is okay. >> >> > > >> >> > > Once librelp has been merged, I also need sponsorship for patches to >> >> > > fix the >> >> > > file descriptor leak in librelp in Groovy and Focal. The LP bug is: >> >> > > >> >> > > https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1908473 >> >> > > >> >> > > Please review and consider sponsoring. I hope the versions are a bit >> >> > > more correct >> >> > > this time around. >> >> > > >> >> > > Thanks, >> >> > > Matthew >> >> > > >> >> > > -- >> >> > > Mailing list: https://launchpad.net/~sts-sponsors >> >> > > Post to : sts-sponsors@lists.launchpad.net >> >> > > Unsubscribe : https://launchpad.net/~sts-sponsors >> >> > > More help : https://help.launchpad.net/ListHelp >> >> > >> >> > >> >> > >> >> > -- >> >> > Mauricio Faria de Oliveira >> >> >> >> -- >> >> Mailing list: https://launchpad.net/~sts-sponsors >> >> Post to : sts-sponsors@lists.launchpad.net >> >> Unsubscribe : https://launchpad.net/~sts-sponsors >> >> More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1898129] Re: Cannot configure 'cryptsetup luksFormat' at install time
Hello Mauricio, or anyone else affected,
Accepted partman-crypto into focal-proposed. The package will build now
and be available at https://launchpad.net/ubuntu/+source/partman-
crypto/101ubuntu4.1 in a few hours, and then in the -proposed
repository.
Please help us by testing this new package. See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed. Your feedback will aid us getting this
update out to other Ubuntu users.
If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
focal to verification-done-focal. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-focal. In either case, without details of your testing we will
not be able to proceed.
Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
advance for helping!
N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.
** Changed in: partman-crypto (Ubuntu Focal)
Status: In Progress => Fix Committed
** Changed in: ubiquity (Ubuntu Focal)
Milestone: None => ubuntu-20.04.2
** Tags added: verification-needed verification-needed-focal
--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1898129
Title:
Cannot configure 'cryptsetup luksFormat' at install time
Status in partman-crypto package in Ubuntu:
Invalid
Status in ubiquity package in Ubuntu:
Fix Released
Status in partman-crypto source package in Focal:
Fix Committed
Status in ubiquity source package in Focal:
In Progress
Status in partman-crypto source package in Groovy:
Invalid
Status in ubiquity source package in Groovy:
Won't Fix
Status in partman-crypto source package in Hirsute:
Invalid
Status in ubiquity source package in Hirsute:
Fix Released
Status in partman-crypto package in Debian:
Unknown
Bug description:
[Impact]
* Users cannot specify options for 'cryptsetup luksFormat'
that is used by the installer.
* Some deployments need the installed disks in LUKS1 format
for backward compatibility with older releases that don't
support LUKS2, for backup/audit/management purposes.
* However, on Focal and later, cryptsetup defaults to LUKS2,
which broke that functionality.
* Currently it's not possible to request the LUKS format in
the installer, so this patch allows for that w/ a preseed
option ('partman-crypto/luksformat_options') for the user.
[Test Case]
* Default behavior: LUKS2
- Install Ubuntu (Focal/later); check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information
Version: 2
...
* Opt-in behavior: LUKS1 (for example; can use other options)
- Install Ubuntu (Focal/later) with preseed file/option:
ubiquity partman-crypto/luksformat_options string \
--type luks1
- Check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information for /dev/vda4
Version: 1
...
- Check install logs for confirmation:
$ grep luksFormat /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat:
'--type luks1'
[Where problems could occur]
* The changes are contained within the partman-crypto functionality,
so only install with encrypted disks should be affected by issues.
* Any additional options specified to 'cryptsetup luksFormat' are
opt-in _and_ specified by the user via the preseed option, thus
errors are probably tied to particular options (mis) used.
* If the preseed option is not specified, original behavior remains.
[Other Info]
* This patch is applied in Hirsute.
* This patch is not needed in Groovy (rationale in comment #15.)
* This patch is targeted at Focal (cryptsetup defaulted to LUKS2.)
* This patch is not needed in Bionic/earlier (^defaults to LUKS1.)
[Original Description]
Most users should be fine with the options to
'cryptsetup luksFormat' used by the installer.
However, some users may have reasons to use
other options, and that is not possible now.
Let's provide a new preseed option for that:
'partman-crypto/luksformat_options'
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/partman-crypto/+bug/1898129/+subscriptions
--
Mailing list: https://launchpad.net/~sts-sponsors
Post to : sts-sponsors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~sts-sponsors
More help :
[Sts-sponsors] [Bug 1903733] Re: Out of memory issue for websocket client
** Description changed: [Impact] Applications using package python-tornado v5.1.1 or earlier are susceptible to an out of memory error related to websockets. [Other Info] Upstream commit(s): https://github.com/tornadoweb/tornado/pull/2351/commits/20becca336caae61cd24f7afba0e177c0a210c70 $ git remote -v originhttps://github.com/tornadoweb/tornado.git (fetch) originhttps://github.com/tornadoweb/tornado.git (push) $ git describe --contains 20becca3 v5.1.0b1~28^2~1 $ rmadison python3-tornardo => python3-tornado | 4.2.1-1ubuntu3 | xenial python3-tornado | 4.5.3-1 | bionic/universe => python3-tornado | 4.5.3-1ubuntu0.1| bionic-updates/universe python3-tornado | 6.0.3+really5.1.1-3 | focal/universe python3-tornado | 6.0.4-2 | groovy/universe python3-tornado | 6.0.4-3 | hirsute/universe python3-tornado | 6.1.0-1 | hirsute-proposed/universe [Original Description] - Tornado has no 'flow control' for websockets. A websocket will receive data as fast as it can, and store the data in a deque. If that data is not consumed as fast as it is written, then that deque will grow in size indefinitely, ultimately leading to a memory error and killing the process. + Tornado has no 'flow control', [8] TCP flow control definition, for websockets. A websocket will receive data as fast as it can, and store the data in a deque. If that data is not consumed as fast as it is written, then that deque will grow in size indefinitely, ultimately leading to a memory error and killing the process. Fix is to use a Queue. Read and get messages from the queue on the client side. Patch file [0] Commit history [1] GitHub [2] Issue [3] [0] https://patch-diff.githubusercontent.com/raw/tornadoweb/tornado/pull/2351.patch [1] https://github.com/tornadoweb/tornado/pull/2351/commits [2] https://github.com/tornadoweb/tornado [3] https://github.com/tornadoweb/tornado/issues/2341 [Test Case] I will be attaching two python test files. client.py server.py # create lxc container & limits on memory and turn off swap $ sudo apt-get install lxc lxd $ lxd init $ lxc launch ubuntu:18.04 bionic-python-tornado # shrink server size lxc config set server limits.cpu 2 # changes ram setting lxc config set server limits.memory 150MB # severely limits amount of swap used [4] lxc config set bionic-py-tornado limits.memory.swap false # install dev tools and download source code $ lxc exec bionic-python-tornado bash $ apt-get update $ apt install ubuntu-dev-tools -y $ pull-lp-source python-tornado bionic $ sudo apt build-dep . # copy client.py and server.py to # $ ~/python-tornado-4.5.3/demos $ scp or touch client.py and server.py # build code $ python3 setup.py build $ python3 setup.py install # I have 3 terminals open 2 for executing python, one for the client and one for server and another one using top to view memory constraints # run server.py, client.py, and top in separate terminals $ python3 demos/client.py $ python3 demos/server.py $ top What gets print out in the client.py is the length of the collections.deque In the server.py prints out messages like: message: keep alive * press ctrl+E for showing memory in MB in the terminal with top top - shows that swap is off/ running very low and our memory is only 150MB Although I never hit the oom exception that is expected to be thrown, you can check dmesg $ sudo dmesg | grep -i python looks similar to this: [ 3250.067833] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=lxc.payload.iptest,mems_allowed=0,oom_memcg=/lxc.payload.iptest,task_memcg=/lxc.payload.iptest,task=python3,pid=44889,uid=100 [ 3250.067838] Memory cgroup out of memory: Killed process 44889 (python3) total-vm:304616kB, anon-rss:235152kB, file-rss:0kB, shmem-rss:0kB, UID:100 pgtables:628kB oom_score_adj:0 [ 3250.075096] oom_reaper: reaped process 44889 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0k After either adding the patch or running focal or later versions *pull-lp-source python-tornado focal We can run the exact same setup again, and this time it shows that the new queue object has only a length of 1. We have shown that before the patch, what was used to store messages in the queue was unbounded and could grow "If maxlen is not specified or is None, deques may grow to an arbitrary length over time."[6] Afterwards they decided using a blocking queue with size 1. (Queue(1)), where there is only ever 1 item in the queue at a time. [7] [4] https://discuss.linuxcontainers.org/t/limiting-swap-in-lxc-container/6343/4 [5] attached screenshot - [6] https://docs.python.org/3/library/collections.html?highlight=collections%20deque#collections.deque + [6]
[Sts-sponsors] [Bug 1910307] Re: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main)
Attached is the revised debdiff between librelp_1.9.0-1.dsc and librelp_1.9.0-1ubuntu1.dsc ** Patch removed: "Debdiff with the merge from Debian" https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307/+attachment/5449599/+files/librelp_diff_debian.debdiff ** Patch added: "Debdiff with the merge from Debian" https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307/+attachment/5452413/+files/librelp_diff_debian.debdiff -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1910307 Title: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main) Status in librelp package in Ubuntu: In Progress Status in librelp source package in Hirsute: In Progress Bug description: [Impact] librelp 1.9.0-1 is available in Debian, and is much newer than what we carry in Ubuntu 1.5.0-1ubuntu2. This bug tracks the merge to the newer version. This bug is needed for the SRU of bug 1908473, which fixes a file descriptor leak in librelp. The leak was fixed in 1.7.0, so the merge to 1.9.0 contains the fix already. [Testcase] Test packages are available in the following ppa, with builds for all supported architectures, with debug symbols: https://launchpad.net/~mruffell/+archive/ubuntu/lp1910307-test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1910307] Re: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main)
** Description changed: [Impact] librelp 1.9.0-1 is available in Debian, and is much newer than what we carry in Ubuntu 1.5.0-1ubuntu2. This bug tracks the merge to the newer version. This bug is needed for the SRU of bug 1908473, which fixes a file descriptor leak in librelp. The leak was fixed in 1.7.0, so the merge to 1.9.0 contains the fix already. + + [Testcase] + + Test packages are available in the following ppa, with builds for all + supported architectures, with debug symbols: + + https://launchpad.net/~mruffell/+archive/ubuntu/lp1910307-test -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1910307 Title: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main) Status in librelp package in Ubuntu: In Progress Status in librelp source package in Hirsute: In Progress Bug description: [Impact] librelp 1.9.0-1 is available in Debian, and is much newer than what we carry in Ubuntu 1.5.0-1ubuntu2. This bug tracks the merge to the newer version. This bug is needed for the SRU of bug 1908473, which fixes a file descriptor leak in librelp. The leak was fixed in 1.7.0, so the merge to 1.9.0 contains the fix already. [Testcase] Test packages are available in the following ppa, with builds for all supported architectures, with debug symbols: https://launchpad.net/~mruffell/+archive/ubuntu/lp1910307-test To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1903733] Re: Out of memory issue for websocket client
updated bionic python-tornado package ** Patch added: "python-tornado_4.5.3-1ubuntu0.1.1+bionic+lp1903733.debdiff" https://bugs.launchpad.net/ubuntu/bionic/+source/python-tornado/+bug/1903733/+attachment/5452411/+files/python-tornado_4.5.3-1ubuntu0.1.1+bionic+lp1903733.debdiff -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1903733 Title: Out of memory issue for websocket client Status in python-tornado package in Ubuntu: Fix Released Status in python-tornado source package in Xenial: In Progress Status in python-tornado source package in Bionic: In Progress Status in python-tornado source package in Focal: Fix Released Status in python-tornado source package in Groovy: Fix Released Status in python-tornado source package in Hirsute: Fix Released Bug description: [Impact] Applications using package python-tornado v5.1.1 or earlier are susceptible to an out of memory error related to websockets. [Other Info] Upstream commit(s): https://github.com/tornadoweb/tornado/pull/2351/commits/20becca336caae61cd24f7afba0e177c0a210c70 $ git remote -v originhttps://github.com/tornadoweb/tornado.git (fetch) originhttps://github.com/tornadoweb/tornado.git (push) $ git describe --contains 20becca3 v5.1.0b1~28^2~1 $ rmadison python3-tornardo => python3-tornado | 4.2.1-1ubuntu3 | xenial python3-tornado | 4.5.3-1 | bionic/universe => python3-tornado | 4.5.3-1ubuntu0.1| bionic-updates/universe python3-tornado | 6.0.3+really5.1.1-3 | focal/universe python3-tornado | 6.0.4-2 | groovy/universe python3-tornado | 6.0.4-3 | hirsute/universe python3-tornado | 6.1.0-1 | hirsute-proposed/universe [Original Description] Tornado has no 'flow control' for websockets. A websocket will receive data as fast as it can, and store the data in a deque. If that data is not consumed as fast as it is written, then that deque will grow in size indefinitely, ultimately leading to a memory error and killing the process. Fix is to use a Queue. Read and get messages from the queue on the client side. Patch file [0] Commit history [1] GitHub [2] Issue [3] [0] https://patch-diff.githubusercontent.com/raw/tornadoweb/tornado/pull/2351.patch [1] https://github.com/tornadoweb/tornado/pull/2351/commits [2] https://github.com/tornadoweb/tornado [3] https://github.com/tornadoweb/tornado/issues/2341 [Test Case] I will be attaching two python test files. client.py server.py # create lxc container & limits on memory and turn off swap $ sudo apt-get install lxc lxd $ lxd init $ lxc launch ubuntu:18.04 bionic-python-tornado # shrink server size lxc config set server limits.cpu 2 # changes ram setting lxc config set server limits.memory 150MB # severely limits amount of swap used [4] lxc config set bionic-py-tornado limits.memory.swap false # install dev tools and download source code $ lxc exec bionic-python-tornado bash $ apt-get update $ apt install ubuntu-dev-tools -y $ pull-lp-source python-tornado bionic $ sudo apt build-dep . # copy client.py and server.py to # $ ~/python-tornado-4.5.3/demos $ scp or touch client.py and server.py # build code $ python3 setup.py build $ python3 setup.py install # I have 3 terminals open 2 for executing python, one for the client and one for server and another one using top to view memory constraints # run server.py, client.py, and top in separate terminals $ python3 demos/client.py $ python3 demos/server.py $ top What gets print out in the client.py is the length of the collections.deque In the server.py prints out messages like: message: keep alive * press ctrl+E for showing memory in MB in the terminal with top top - shows that swap is off/ running very low and our memory is only 150MB Although I never hit the oom exception that is expected to be thrown, you can check dmesg $ sudo dmesg | grep -i python looks similar to this: [ 3250.067833] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=lxc.payload.iptest,mems_allowed=0,oom_memcg=/lxc.payload.iptest,task_memcg=/lxc.payload.iptest,task=python3,pid=44889,uid=100 [ 3250.067838] Memory cgroup out of memory: Killed process 44889 (python3) total-vm:304616kB, anon-rss:235152kB, file-rss:0kB, shmem-rss:0kB, UID:100 pgtables:628kB oom_score_adj:0 [ 3250.075096] oom_reaper: reaped process 44889 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0k After either adding the patch or running focal or later versions *pull-lp-source python-tornado focal We can run the exact same setup again, and this time it shows that the new queue object has only a length of 1. We have shown that before the patch, what was used to store messages in the queue was unbounded and could
Re: [Sts-sponsors] Please review and consider sponsoring LP1910307 and LP1908473 (librelp)
Great, thanks! On Tue, Jan 12, 2021 at 3:20 PM Matthew Ruffell < matthew.ruff...@canonical.com> wrote: > > Did you talk to William Grant ? I would assume that if you did the > merge, it was okay with you doing it and he had no plan to merge it anytime > soon ? > > > Yes, I talked to William. He did not have any attachment to the > package at all, and wasn't planning on merging anytime soon. See > attached picture. > > > > On Tue, Jan 12, 2021 at 12:12 AM Matthew Ruffell < > matthew.ruff...@canonical.com> wrote: > >> > >> Hi Dan, > >> > >> If you have some time, can you please review my merge request for > >> librelp 1.9 to hirsute? > >> > >> https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 > >> > >> Thanks, > >> Matthew > >> > >> On Thu, Jan 7, 2021 at 1:45 AM Mauricio Oliveira > >> wrote: > >> > > >> > Hey Matthew, > >> > > >> > Great work on those. > >> > > >> > I'd be happy to review/sponsor for the SRUs after the merge is > released. > >> > Could you please ping me once that happens? and I'll take care of it. > >> > > >> > cheers, > >> > > >> > On Wed, Jan 6, 2021 at 2:12 AM Matthew Ruffell > >> > wrote: > >> > > > >> > > Hello Eric, > >> > > > >> > > I followed the instructions you posted on Mattermost yesterday on > how to perform > >> > > a merge. > >> > > > >> > > The merge bug for librelp is: > >> > > > >> > > https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 > >> > > > >> > > I have attached debdiffs from Debian, and the last Ubuntu version, > just like > >> > > the merge document mentioned. Please review and merge if it is okay. > >> > > > >> > > Once librelp has been merged, I also need sponsorship for patches > to fix the > >> > > file descriptor leak in librelp in Groovy and Focal. The LP bug is: > >> > > > >> > > https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1908473 > >> > > > >> > > Please review and consider sponsoring. I hope the versions are a bit > >> > > more correct > >> > > this time around. > >> > > > >> > > Thanks, > >> > > Matthew > >> > > > >> > > -- > >> > > Mailing list: https://launchpad.net/~sts-sponsors > >> > > Post to : sts-sponsors@lists.launchpad.net > >> > > Unsubscribe : https://launchpad.net/~sts-sponsors > >> > > More help : https://help.launchpad.net/ListHelp > >> > > >> > > >> > > >> > -- > >> > Mauricio Faria de Oliveira > >> > >> -- > >> Mailing list: https://launchpad.net/~sts-sponsors > >> Post to : sts-sponsors@lists.launchpad.net > >> Unsubscribe : https://launchpad.net/~sts-sponsors > >> More help : https://help.launchpad.net/ListHelp > -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
Re: [Sts-sponsors] Please review and consider sponsoring LP1910307 and LP1908473 (librelp)
> Did you talk to William Grant ? I would assume that if you did the merge, it > was okay with you doing it and he had no plan to merge it anytime soon ? > Yes, I talked to William. He did not have any attachment to the package at all, and wasn't planning on merging anytime soon. See attached picture. > On Tue, Jan 12, 2021 at 12:12 AM Matthew Ruffell > wrote: >> >> Hi Dan, >> >> If you have some time, can you please review my merge request for >> librelp 1.9 to hirsute? >> >> https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 >> >> Thanks, >> Matthew >> >> On Thu, Jan 7, 2021 at 1:45 AM Mauricio Oliveira >> wrote: >> > >> > Hey Matthew, >> > >> > Great work on those. >> > >> > I'd be happy to review/sponsor for the SRUs after the merge is released. >> > Could you please ping me once that happens? and I'll take care of it. >> > >> > cheers, >> > >> > On Wed, Jan 6, 2021 at 2:12 AM Matthew Ruffell >> > wrote: >> > > >> > > Hello Eric, >> > > >> > > I followed the instructions you posted on Mattermost yesterday on how to >> > > perform >> > > a merge. >> > > >> > > The merge bug for librelp is: >> > > >> > > https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 >> > > >> > > I have attached debdiffs from Debian, and the last Ubuntu version, just >> > > like >> > > the merge document mentioned. Please review and merge if it is okay. >> > > >> > > Once librelp has been merged, I also need sponsorship for patches to fix >> > > the >> > > file descriptor leak in librelp in Groovy and Focal. The LP bug is: >> > > >> > > https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1908473 >> > > >> > > Please review and consider sponsoring. I hope the versions are a bit >> > > more correct >> > > this time around. >> > > >> > > Thanks, >> > > Matthew >> > > >> > > -- >> > > Mailing list: https://launchpad.net/~sts-sponsors >> > > Post to : sts-sponsors@lists.launchpad.net >> > > Unsubscribe : https://launchpad.net/~sts-sponsors >> > > More help : https://help.launchpad.net/ListHelp >> > >> > >> > >> > -- >> > Mauricio Faria de Oliveira >> >> -- >> Mailing list: https://launchpad.net/~sts-sponsors >> Post to : sts-sponsors@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~sts-sponsors >> More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1826737] Re: lshw does not list NVMe storage devices as "disk" nodes
** Patch added: "lshw-focal.debdiff" https://bugs.launchpad.net/ubuntu/+source/lshw/+bug/1826737/+attachment/5452324/+files/lshw-focal.debdiff -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1826737 Title: lshw does not list NVMe storage devices as "disk" nodes Status in lshw package in Ubuntu: Fix Released Status in lshw source package in Bionic: New Status in lshw source package in Focal: New Status in lshw source package in Groovy: New Status in lshw source package in Hirsute: Fix Released Status in lshw package in Debian: Unknown Bug description: [Impact] * NVMe devices are not recognized by lshw in Ubuntu [Test Case] * Running "lshw -C disk" or "lshw -C storage" does not show NVMe devices. Example: https://pastebin.ubuntu.com/p/FfKGNc7W6M/ [Where problems could occur] * This upload consists of four cherry-picked patches and the feature is self-contained, so the regression potential is quite low. If there's anything to happen, it would be in the network device scan, where the structure was altered by the main NVMe patch. * For those who does HW monitoring/inventory listing based on 'lshw' might observe changes in their inventory listing, it shouldn't be a 'problem' but I want to point this out. [Other information] # Redhat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1695343 [Original description] Ubuntu MATE 19.04, updated 2019-04-28 sudo lshw -class disk Expected : info on SSD Actual result : info on USB drive only. Note this is already reported to RedHat ProblemType: Bug DistroRelease: Ubuntu 19.04 Package: lshw 02.18-0.1ubuntu7 ProcVersionSignature: Ubuntu 5.0.0-13.14-generic 5.0.6 Uname: Linux 5.0.0-13-generic x86_64 NonfreeKernelModules: nvidia_modeset nvidia ApportVersion: 2.20.10-0ubuntu27 Architecture: amd64 CurrentDesktop: MATE Date: Sun Apr 28 07:11:45 2019 InstallationDate: Installed on 2019-04-25 (3 days ago) InstallationMedia: Ubuntu-MATE 19.04 "Disco Dingo" - Release amd64 (20190416) SourcePackage: lshw UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/lshw/+bug/1826737/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1826737] Re: lshw does not list NVMe storage devices as "disk" nodes
** Patch added: "lshw-groovy.debdiff" https://bugs.launchpad.net/ubuntu/+source/lshw/+bug/1826737/+attachment/5452323/+files/lshw-groovy.debdiff -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1826737 Title: lshw does not list NVMe storage devices as "disk" nodes Status in lshw package in Ubuntu: Fix Released Status in lshw source package in Bionic: New Status in lshw source package in Focal: New Status in lshw source package in Groovy: New Status in lshw source package in Hirsute: Fix Released Status in lshw package in Debian: Unknown Bug description: [Impact] * NVMe devices are not recognized by lshw in Ubuntu [Test Case] * Running "lshw -C disk" or "lshw -C storage" does not show NVMe devices. Example: https://pastebin.ubuntu.com/p/FfKGNc7W6M/ [Where problems could occur] * This upload consists of four cherry-picked patches and the feature is self-contained, so the regression potential is quite low. If there's anything to happen, it would be in the network device scan, where the structure was altered by the main NVMe patch. * For those who does HW monitoring/inventory listing based on 'lshw' might observe changes in their inventory listing, it shouldn't be a 'problem' but I want to point this out. [Other information] # Redhat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1695343 [Original description] Ubuntu MATE 19.04, updated 2019-04-28 sudo lshw -class disk Expected : info on SSD Actual result : info on USB drive only. Note this is already reported to RedHat ProblemType: Bug DistroRelease: Ubuntu 19.04 Package: lshw 02.18-0.1ubuntu7 ProcVersionSignature: Ubuntu 5.0.0-13.14-generic 5.0.6 Uname: Linux 5.0.0-13-generic x86_64 NonfreeKernelModules: nvidia_modeset nvidia ApportVersion: 2.20.10-0ubuntu27 Architecture: amd64 CurrentDesktop: MATE Date: Sun Apr 28 07:11:45 2019 InstallationDate: Installed on 2019-04-25 (3 days ago) InstallationMedia: Ubuntu-MATE 19.04 "Disco Dingo" - Release amd64 (20190416) SourcePackage: lshw UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/lshw/+bug/1826737/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1903733] Re: Out of memory issue for websocket client
** Patch removed: "python-tornado-xenial-lp1903733b0.debdiff" https://bugs.launchpad.net/ubuntu/+source/python-tornado/+bug/1903733/+attachment/5443587/+files/python-tornado-xenial-lp1903733b0.debdiff ** Patch removed: "python-tornadob0-lp1903733-bionic.debdiff" https://bugs.launchpad.net/ubuntu/+source/python-tornado/+bug/1903733/+attachment/5443547/+files/python-tornadob0-lp1903733-bionic.debdiff -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1903733 Title: Out of memory issue for websocket client Status in python-tornado package in Ubuntu: Fix Released Status in python-tornado source package in Xenial: In Progress Status in python-tornado source package in Bionic: In Progress Status in python-tornado source package in Focal: Fix Released Status in python-tornado source package in Groovy: Fix Released Status in python-tornado source package in Hirsute: Fix Released Bug description: [Impact] Applications using package python-tornado v5.1.1 or earlier are susceptible to an out of memory error related to websockets. [Other Info] Upstream commit(s): https://github.com/tornadoweb/tornado/pull/2351/commits/20becca336caae61cd24f7afba0e177c0a210c70 $ git remote -v originhttps://github.com/tornadoweb/tornado.git (fetch) originhttps://github.com/tornadoweb/tornado.git (push) $ git describe --contains 20becca3 v5.1.0b1~28^2~1 $ rmadison python3-tornardo => python3-tornado | 4.2.1-1ubuntu3 | xenial python3-tornado | 4.5.3-1 | bionic/universe => python3-tornado | 4.5.3-1ubuntu0.1| bionic-updates/universe python3-tornado | 6.0.3+really5.1.1-3 | focal/universe python3-tornado | 6.0.4-2 | groovy/universe python3-tornado | 6.0.4-3 | hirsute/universe python3-tornado | 6.1.0-1 | hirsute-proposed/universe [Original Description] Tornado has no 'flow control' for websockets. A websocket will receive data as fast as it can, and store the data in a deque. If that data is not consumed as fast as it is written, then that deque will grow in size indefinitely, ultimately leading to a memory error and killing the process. Fix is to use a Queue. Read and get messages from the queue on the client side. Patch file [0] Commit history [1] GitHub [2] Issue [3] [0] https://patch-diff.githubusercontent.com/raw/tornadoweb/tornado/pull/2351.patch [1] https://github.com/tornadoweb/tornado/pull/2351/commits [2] https://github.com/tornadoweb/tornado [3] https://github.com/tornadoweb/tornado/issues/2341 [Test Case] I will be attaching two python test files. client.py server.py # create lxc container & limits on memory and turn off swap $ sudo apt-get install lxc lxd $ lxd init $ lxc launch ubuntu:18.04 bionic-python-tornado # shrink server size lxc config set server limits.cpu 2 # changes ram setting lxc config set server limits.memory 150MB # severely limits amount of swap used [4] lxc config set bionic-py-tornado limits.memory.swap false # install dev tools and download source code $ lxc exec bionic-python-tornado bash $ apt-get update $ apt install ubuntu-dev-tools -y $ pull-lp-source python-tornado bionic $ sudo apt build-dep . # copy client.py and server.py to # $ ~/python-tornado-4.5.3/demos $ scp or touch client.py and server.py # build code $ python3 setup.py build $ python3 setup.py install # I have 3 terminals open 2 for executing python, one for the client and one for server and another one using top to view memory constraints # run server.py, client.py, and top in separate terminals $ python3 demos/client.py $ python3 demos/server.py $ top What gets print out in the client.py is the length of the collections.deque In the server.py prints out messages like: message: keep alive * press ctrl+E for showing memory in MB in the terminal with top top - shows that swap is off/ running very low and our memory is only 150MB Although I never hit the oom exception that is expected to be thrown, you can check dmesg $ sudo dmesg | grep -i python looks similar to this: [ 3250.067833] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=lxc.payload.iptest,mems_allowed=0,oom_memcg=/lxc.payload.iptest,task_memcg=/lxc.payload.iptest,task=python3,pid=44889,uid=100 [ 3250.067838] Memory cgroup out of memory: Killed process 44889 (python3) total-vm:304616kB, anon-rss:235152kB, file-rss:0kB, shmem-rss:0kB, UID:100 pgtables:628kB oom_score_adj:0 [ 3250.075096] oom_reaper: reaped process 44889 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0k After either adding the patch or running focal or later versions *pull-lp-source python-tornado focal We can run the exact same setup again, and this time it shows that the new queue object has only a
[Sts-sponsors] [Bug 1903733] Re: Out of memory issue for websocket client
** Description changed: [Impact] Applications using package python-tornado v5.1.1 or earlier are susceptible to an out of memory error related to websockets. [Other Info] Upstream commit(s): https://github.com/tornadoweb/tornado/pull/2351/commits/20becca336caae61cd24f7afba0e177c0a210c70 $ git remote -v originhttps://github.com/tornadoweb/tornado.git (fetch) originhttps://github.com/tornadoweb/tornado.git (push) $ git describe --contains 20becca3 v5.1.0b1~28^2~1 $ rmadison python3-tornardo => python3-tornado | 4.2.1-1ubuntu3 | xenial python3-tornado | 4.5.3-1 | bionic/universe => python3-tornado | 4.5.3-1ubuntu0.1| bionic-updates/universe python3-tornado | 6.0.3+really5.1.1-3 | focal/universe python3-tornado | 6.0.4-2 | groovy/universe python3-tornado | 6.0.4-3 | hirsute/universe python3-tornado | 6.1.0-1 | hirsute-proposed/universe [Original Description] Tornado has no 'flow control' for websockets. A websocket will receive data as fast as it can, and store the data in a deque. If that data is not consumed as fast as it is written, then that deque will grow in size indefinitely, ultimately leading to a memory error and killing the process. Fix is to use a Queue. Read and get messages from the queue on the client side. Patch file [0] Commit history [1] GitHub [2] Issue [3] [0] https://patch-diff.githubusercontent.com/raw/tornadoweb/tornado/pull/2351.patch [1] https://github.com/tornadoweb/tornado/pull/2351/commits [2] https://github.com/tornadoweb/tornado [3] https://github.com/tornadoweb/tornado/issues/2341 [Test Case] I will be attaching two python test files. client.py server.py # create lxc container & limits on memory and turn off swap $ sudo apt-get install lxc lxd $ lxd init $ lxc launch ubuntu:18.04 bionic-python-tornado # shrink server size lxc config set server limits.cpu 2 # changes ram setting lxc config set server limits.memory 150MB # severely limits amount of swap used [4] lxc config set bionic-py-tornado limits.memory.swap false # install dev tools and download source code $ lxc exec bionic-python-tornado bash $ apt-get update $ apt install ubuntu-dev-tools -y $ pull-lp-source python-tornado bionic $ sudo apt build-dep . # copy client.py and server.py to # $ ~/python-tornado-4.5.3/demos $ scp or touch client.py and server.py # build code $ python3 setup.py build $ python3 setup.py install # I have 3 terminals open 2 for executing python, one for the client and one for server and another one using top to view memory constraints # run server.py, client.py, and top in separate terminals $ python3 demos/client.py $ python3 demos/server.py $ top What gets print out in the client.py is the length of the collections.deque In the server.py prints out messages like: message: keep alive * press ctrl+E for showing memory in MB in the terminal with top top - shows that swap is off/ running very low and our memory is only 150MB Although I never hit the oom exception that is expected to be thrown, you can check dmesg $ sudo dmesg | grep -i python looks similar to this: [ 3250.067833] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=lxc.payload.iptest,mems_allowed=0,oom_memcg=/lxc.payload.iptest,task_memcg=/lxc.payload.iptest,task=python3,pid=44889,uid=100 [ 3250.067838] Memory cgroup out of memory: Killed process 44889 (python3) total-vm:304616kB, anon-rss:235152kB, file-rss:0kB, shmem-rss:0kB, UID:100 pgtables:628kB oom_score_adj:0 [ 3250.075096] oom_reaper: reaped process 44889 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0k After either adding the patch or running focal or later versions *pull-lp-source python-tornado focal We can run the exact same setup again, and this time it shows that the new queue object has only a length of 1. + We have shown that before the patch, what was used to store messages in + the queue was unbounded and could grow "If maxlen is not specified or is + None, deques may grow to an arbitrary length over time."[6] Afterwards + they decided using a blocking queue with size 1. (Queue(1)), where there + is only ever 1 item in the queue at a time. [7] [4] https://discuss.linuxcontainers.org/t/limiting-swap-in-lxc-container/6343/4 [5] attached screenshot + [6] https://docs.python.org/3/library/collections.html?highlight=collections%20deque#collections.deque + [7] https://www.tornadoweb.org/en/stable/queues.html -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1903733 Title: Out of memory issue for websocket client Status in python-tornado package in Ubuntu: Fix Released Status in python-tornado source package in
[Sts-sponsors] [Bug 1903733] Re: Out of memory issue for websocket client
attached screenshot of terminal setup and running ** Attachment added: "Screenshot from 2021-01-11 13-37-11.png" https://bugs.launchpad.net/ubuntu/+source/python-tornado/+bug/1903733/+attachment/5452293/+files/Screenshot%20from%202021-01-11%2013-37-11.png -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1903733 Title: Out of memory issue for websocket client Status in python-tornado package in Ubuntu: Fix Released Status in python-tornado source package in Xenial: In Progress Status in python-tornado source package in Bionic: In Progress Status in python-tornado source package in Focal: Fix Released Status in python-tornado source package in Groovy: Fix Released Status in python-tornado source package in Hirsute: Fix Released Bug description: [Impact] Applications using package python-tornado v5.1.1 or earlier are susceptible to an out of memory error related to websockets. [Other Info] Upstream commit(s): https://github.com/tornadoweb/tornado/pull/2351/commits/20becca336caae61cd24f7afba0e177c0a210c70 $ git remote -v originhttps://github.com/tornadoweb/tornado.git (fetch) originhttps://github.com/tornadoweb/tornado.git (push) $ git describe --contains 20becca3 v5.1.0b1~28^2~1 $ rmadison python3-tornardo => python3-tornado | 4.2.1-1ubuntu3 | xenial python3-tornado | 4.5.3-1 | bionic/universe => python3-tornado | 4.5.3-1ubuntu0.1| bionic-updates/universe python3-tornado | 6.0.3+really5.1.1-3 | focal/universe python3-tornado | 6.0.4-2 | groovy/universe python3-tornado | 6.0.4-3 | hirsute/universe python3-tornado | 6.1.0-1 | hirsute-proposed/universe [Original Description] Tornado has no 'flow control' for websockets. A websocket will receive data as fast as it can, and store the data in a deque. If that data is not consumed as fast as it is written, then that deque will grow in size indefinitely, ultimately leading to a memory error and killing the process. Fix is to use a Queue. Read and get messages from the queue on the client side. Patch file [0] Commit history [1] GitHub [2] Issue [3] [0] https://patch-diff.githubusercontent.com/raw/tornadoweb/tornado/pull/2351.patch [1] https://github.com/tornadoweb/tornado/pull/2351/commits [2] https://github.com/tornadoweb/tornado [3] https://github.com/tornadoweb/tornado/issues/2341 [Test Case] I will be attaching two python test files. client.py server.py # create lxc container & limits on memory and turn off swap $ sudo apt-get install lxc lxd $ lxd init $ lxc launch ubuntu:18.04 bionic-python-tornado # shrink server size lxc config set server limits.cpu 2 # changes ram setting lxc config set server limits.memory 150MB # severely limits amount of swap used [4] lxc config set bionic-py-tornado limits.memory.swap false # install dev tools and download source code $ lxc exec bionic-python-tornado bash $ apt-get update $ apt install ubuntu-dev-tools -y $ pull-lp-source python-tornado bionic $ sudo apt build-dep . # copy client.py and server.py to # $ ~/python-tornado-4.5.3/demos $ scp or touch client.py and server.py # build code $ python3 setup.py build $ python3 setup.py install # I have 3 terminals open 2 for executing python, one for the client and one for server and another one using top to view memory constraints # run server.py, client.py, and top in separate terminals $ python3 demos/client.py $ python3 demos/server.py $ top What gets print out in the client.py is the length of the collections.deque In the server.py prints out messages like: message: keep alive * press ctrl+E for showing memory in MB in the terminal with top top - shows that swap is off/ running very low and our memory is only 150MB Although I never hit the oom exception that is expected to be thrown, you can check dmesg $ sudo dmesg | grep -i python looks similar to this: [ 3250.067833] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=lxc.payload.iptest,mems_allowed=0,oom_memcg=/lxc.payload.iptest,task_memcg=/lxc.payload.iptest,task=python3,pid=44889,uid=100 [ 3250.067838] Memory cgroup out of memory: Killed process 44889 (python3) total-vm:304616kB, anon-rss:235152kB, file-rss:0kB, shmem-rss:0kB, UID:100 pgtables:628kB oom_score_adj:0 [ 3250.075096] oom_reaper: reaped process 44889 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0k After either adding the patch or running focal or later versions *pull-lp-source python-tornado focal We can run the exact same setup again, and this time it shows that the new queue object has only a length of 1. [4] https://discuss.linuxcontainers.org/t/limiting-swap-in-lxc-container/6343/4 [5] attached screenshot To manage notifications
[Sts-sponsors] [Bug 1910307] Re: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main)
@Matthew * Do you have a PPA where your merge of librelp can be found ? * Could you please update d/changelog and mention that 'python2.diff' has been dropped. Example: * Former patches: - d/p/python2.diff I see you have noted it in LP which is good, but would be even better to have it marked done in d/changelog as well. I'll continue to review once I have the PPA. - Eric -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1910307 Title: Please merge librelp 1.9.0-1 (universe) from Debian unstable (main) Status in librelp package in Ubuntu: In Progress Status in librelp source package in Hirsute: In Progress Bug description: [Impact] librelp 1.9.0-1 is available in Debian, and is much newer than what we carry in Ubuntu 1.5.0-1ubuntu2. This bug tracks the merge to the newer version. This bug is needed for the SRU of bug 1908473, which fixes a file descriptor leak in librelp. The leak was fixed in 1.7.0, so the merge to 1.9.0 contains the fix already. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1910307] [NEW] Please merge librelp 1.9.0-1 (universe) from Debian unstable (main)
You have been subscribed to a public bug by Eric Desrochers (slashd): [Impact] librelp 1.9.0-1 is available in Debian, and is much newer than what we carry in Ubuntu 1.5.0-1ubuntu2. This bug tracks the merge to the newer version. This bug is needed for the SRU of bug 1908473, which fixes a file descriptor leak in librelp. The leak was fixed in 1.7.0, so the merge to 1.9.0 contains the fix already. ** Affects: librelp (Ubuntu) Importance: Wishlist Assignee: Matthew Ruffell (mruffell) Status: In Progress ** Affects: librelp (Ubuntu Hirsute) Importance: Wishlist Assignee: Matthew Ruffell (mruffell) Status: In Progress ** Tags: patch sts sts-sponsor -- Please merge librelp 1.9.0-1 (universe) from Debian unstable (main) https://bugs.launchpad.net/bugs/1910307 You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1908473] [NEW] rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak
You have been subscribed to a public bug by Eric Desrochers (slashd):
[Impact]
In recent versions of rsyslog and librelp, the imrelp module leaks file
descriptors due to a bug where it does not correctly close sockets, and
instead, leaves them in the CLOSE_WAIT state.
This causes rsyslogd on busy servers to eventually hit the limit of
maximum open files allowed, which locks rsyslogd up until it is
restarted.
A workaround is to restart rsyslogd every month or so to manually close
all of the open sockets.
Only users of the imrelp module are affected, and not rsyslog users in
general.
[Testcase]
Install the rsyslog-relp module like so:
$ sudo apt install rsyslog rsyslog-relp
Next, generate a working directory, and make a config file that loads
the relp module.
$ sudo mkdir /workdir
$ cat << EOF >> ./spool.conf
\$LocalHostName spool
\$AbortOnUncleanConfig on
\$PreserveFQDN on
global(
workDirectory="/workdir"
maxMessageSize="256k"
)
main_queue(queue.type="Direct")
module(load="imrelp")
input(
type="imrelp"
name="imrelp"
port="601"
ruleset="spool"
MaxDataSize="256k"
)
ruleset(name="spool" queue.type="direct") {
}
# Just so rsyslog doesn't whine that we do not have outputs
ruleset(name="noop" queue.type="direct") {
action(
type="omfile"
name="omfile"
file="/workdir/spool.log"
)
}
EOF
Verify that the config is valid, then start a rsyslog server.
$ sudo rsyslogd -f ./spool.conf -N9
$ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid
Fetch the rsyslogd PID and check for open files.
$ RLOGPID=$(cat /workdir/rsyslogd.pid)
$ sudo ls -l /proc/$RLOGPID/fd
total 0
lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'
We have 3 sockets open by default. Next, use netcat to open 100
connections:
$ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done
Now check for open file descriptors, and there will be an extra 100 sockets
in the list:
$ sudo ls -l /proc/$RLOGPID/fd
https://paste.ubuntu.com/p/f6NQVNbZcR/
We can check the state of these sockets with:
$ ss -t
https://paste.ubuntu.com/p/7Ts2FbxJrg/
The listening sockets will be in CLOSE-WAIT, and the netcat sockets will
be in FIN-WAIT-2.
$ ss -t | grep CLOSE-WAIT | wc -l
100
If you install the test package available in the following ppa:
https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test
When you open connections with netcat, these will be closed properly,
and the file descriptor leak will be fixed.
[Where problems could occur]
If a regression were to occur, it would be limited to users of the
imrelp module, which is a part of the rsyslogd-relp package, and depends
on librelp.
rsyslog-relp is not part of a default installation of rsyslog, and is
opt in by changing a configuration file to enable imrelp.
The changes to rsyslog implement a testcase which exercises the
problematic code to ensure things are working as expected, and should
run during autopkgtest time.
[Other]
Upstream bug list:
https://github.com/rsyslog/rsyslog/issues/4350
https://github.com/rsyslog/rsyslog/issues/4005
https://github.com/rsyslog/librelp/issues/188
https://github.com/rsyslog/librelp/pull/193
The following commits fix the problem:
rsyslogd
commit baee0bd5420649329793746f0daf87c4f59fe6a6
Author: Andre lorbach
Date: Thu Apr 9 13:00:35 2020 +0200
Subject: testbench: Add test for imrelp to check broken session handling.
Link:
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6
librelp
===
commit 7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
Author: Andre lorbach
Date: Mon May 11 14:59:55 2020 +0200
Subject: fix memory leak on session break.
Link:
https://github.com/rsyslog/librelp/commit/7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
commit 4a6ad8637c244fd3a1caeb9a93950826f58e956a
Author: Andre lorbach
Date: Wed Apr 8 15:55:32 2020 +0200
Subject: replsess: fix double free of sendbuf in some cases.
Link:
https://github.com/rsyslog/librelp/commit/4a6ad8637c244fd3a1caeb9a93950826f58e956a
commit 3797944fb62273fa1164acd3104f0894b337c4d0
Author: Ognyan Kulev
Date: Mon Jun 15 14:10:08 2020 +0300
Subject: Fix FD leak when socket shutdown is one-sided
Link:
https://github.com/rsyslog/librelp/commit/3797944fb62273fa1164acd3104f0894b337c4d0
** Affects: librelp (Ubuntu)
Importance: Medium
Assignee: Matthew Ruffell (mruffell)
Status: In Progress
** Affects: rsyslog (Ubuntu)
Importance: Medium
Assignee: Matthew Ruffell (mruffell)
Status: Fix Released
** Affects: librelp (Ubuntu Focal)
Importance: Medium
Assignee: Matthew Ruffell (mruffell)
Status: In Progress
** Affects: rsyslog (Ubuntu Focal)
Importance: Medium
Assignee: Matthew
Re: [Sts-sponsors] Please review and consider sponsoring LP1910307 and LP1908473 (librelp)
Hi Matthew, Did you talk to William Grant ? I would assume that if you did the merge, it was okay with you doing it and he had no plan to merge it anytime soon ? On Tue, Jan 12, 2021 at 12:12 AM Matthew Ruffell < matthew.ruff...@canonical.com> wrote: > Hi Dan, > > If you have some time, can you please review my merge request for > librelp 1.9 to hirsute? > > https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 > > Thanks, > Matthew > > On Thu, Jan 7, 2021 at 1:45 AM Mauricio Oliveira > wrote: > > > > Hey Matthew, > > > > Great work on those. > > > > I'd be happy to review/sponsor for the SRUs after the merge is released. > > Could you please ping me once that happens? and I'll take care of it. > > > > cheers, > > > > On Wed, Jan 6, 2021 at 2:12 AM Matthew Ruffell > > wrote: > > > > > > Hello Eric, > > > > > > I followed the instructions you posted on Mattermost yesterday on how > to perform > > > a merge. > > > > > > The merge bug for librelp is: > > > > > > https://bugs.launchpad.net/ubuntu/+source/librelp/+bug/1910307 > > > > > > I have attached debdiffs from Debian, and the last Ubuntu version, > just like > > > the merge document mentioned. Please review and merge if it is okay. > > > > > > Once librelp has been merged, I also need sponsorship for patches to > fix the > > > file descriptor leak in librelp in Groovy and Focal. The LP bug is: > > > > > > https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1908473 > > > > > > Please review and consider sponsoring. I hope the versions are a bit > > > more correct > > > this time around. > > > > > > Thanks, > > > Matthew > > > > > > -- > > > Mailing list: https://launchpad.net/~sts-sponsors > > > Post to : sts-sponsors@lists.launchpad.net > > > Unsubscribe : https://launchpad.net/~sts-sponsors > > > More help : https://help.launchpad.net/ListHelp > > > > > > > > -- > > Mauricio Faria de Oliveira > > -- > Mailing list: https://launchpad.net/~sts-sponsors > Post to : sts-sponsors@lists.launchpad.net > Unsubscribe : https://launchpad.net/~sts-sponsors > More help : https://help.launchpad.net/ListHelp > -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
