Re: [lxc-users] I/O error for logrotate in LXC
On 18/06/20 1:11 am, Harald Dunkel wrote: > Hi folks, > > on some LXC containers I get I/O errors for the logrotate > service (systemd): > > root@il04:~# systemctl status logrotate > * logrotate.service - Rotate log files > Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor > preset: enabled) > Active: failed (Result: exit-code) since Wed 2020-06-17 00:00:11 > CEST; 15h ago > Docs: man:logrotate(8) > man:logrotate.conf(5) > Process: 359762 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf > (code=exited, status=1/FAILURE) > Main PID: 359762 (code=exited, status=1/FAILURE) > > Jun 17 00:00:00 il04.ac.aixigo.de systemd[1]: Starting Rotate log files... > Jun 17 00:00:11 il04.ac.aixigo.de logrotate[359762]: Failed to kill unit > rsyslog.service: Input/output error > Jun 17 00:00:11 il04.ac.aixigo.de logrotate[359762]: error: error > running non-shared postrotate script for /var/log/syslog of > '/var/log/syslog > Jun 17 00:00:11 il04.ac.aixigo.de logrotate[359762]: ' > Jun 17 00:00:11 il04.ac.aixigo.de systemd[1]: logrotate.service: Main > process exited, code=exited, status=1/FAILURE > Jun 17 00:00:11 il04.ac.aixigo.de systemd[1]: logrotate.service: Failed > with result 'exit-code'. > Jun 17 00:00:11 il04.ac.aixigo.de systemd[1]: Failed to start Rotate log > files. > > root@il04:~# ps -ef | grep rsyslog > root 146014 1 0 Jun08 ? 00:00:06 /usr/sbin/rsyslogd > -n -iNONE > root 377934 377898 0 15:09 pts/7 00:00:00 grep rsyslog > root@il04:~# kill -HUP 146014 > root@il04:~# echo $? > 0 > > > Host is Debian 10, LXC is version 4.0.2. > The client runs Debian 10 as well. > > This doesn't happen on all LXC clients and hosts, but I haven't > detected a scheme yet. On my "real" hosts there are no I/O errors, > AFAICT. > > > Regards > Harri Hi Harri, I've seen this as well, but haven't had a good resolution. I haven't seen it with buster on buster though. My hosts are Xen VMs (from a VPS provider) with LXC inside, and I put it down to some oddity of the provided image. I installed a new system with buster, and haven't seen it since. On the containers that were still stuck in that environment, I used a workaround ... but I can't find any of those still in use. It involved editing /etc/logrotate.d/rsyslog, and changing the postrotate hook. Normally it calls /usr/lib/rsyslog/rsyslog-rotate, which calls "systemctl kill -s HUP rsyslog.service", which fails. Switching to use the non-systemd alternative in that file "invoke-rc.d rsyslog rotate > /dev/null" might be the valid workaround. Hope that helps. If as you say it can still happen on a Debian 10 host, then I'm still interested in finding out more about what kind of permissions might be missing, and how to reinstate them. Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] systemctl kill in container?
Hi all, # systemctl kill -s HUP rsyslog.service Failed to kill unit rsyslog.service: Input/output error Is this a known/expected problem when running systemd stuff in a container? It's used by the standard logrotate config for rsyslog in Debian (buster at least), and means my log rotation doesn't work in containers. I've worked around it by using kill (though I should probably use pkill) instead. Cheers, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] LXC and Docker together?
On 24/05/19 10:28 AM, Andrey Repin wrote: > Greetings, Richard Hector! > >> I've been asked by my client to set up Docker on the host I'd set up for >> them with LXC (on Debian Stretch). This is because the applications that >> run in the containers (Atlassian suite, mostly) will in the future be >> managed by a different contractor, who is familiar with Docker. So >> eventually, I expect most or all of the LXC containers will be replaced >> with Docker ones. > >> It appears that I can install Docker from the Docker repo, so that's ok. > >> What I'm more worried about, is if they're both using the same >> underlying technology (are they?) whether they'll interact in bad and/or >> unexpected ways. > >> Am I likely to be safe doing this? Anything to watch out for? > >> And will I eventually be able to uninstall LXC without disrupting Docker? > > They are not related per se. They use similar kernel functionality, but > there's no conflict between them, if that's what you want to know. > Furthermore, with a little tweak you can run Docker inside LXC container. Thanks all, I installed a new KVM with LXC in it, and added Docker ... it seems that Docker's default iptables rules do interfere with the way I set up bridges for LXC, but with a bit more head scratching, I think I'll be able to make it work :-) Cheers, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] LXC and Docker together?
Hi all, I've been asked by my client to set up Docker on the host I'd set up for them with LXC (on Debian Stretch). This is because the applications that run in the containers (Atlassian suite, mostly) will in the future be managed by a different contractor, who is familiar with Docker. So eventually, I expect most or all of the LXC containers will be replaced with Docker ones. It appears that I can install Docker from the Docker repo, so that's ok. What I'm more worried about, is if they're both using the same underlying technology (are they?) whether they'll interact in bad and/or unexpected ways. Am I likely to be safe doing this? Anything to watch out for? And will I eventually be able to uninstall LXC without disrupting Docker? Thanks, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] future of lxc/lxd? snap?
On 25/02/19 4:52 PM, Fajar A. Nugraha wrote: > On Sat, Feb 23, 2019 at 9:46 PM Richard Hector <mailto:rich...@walnut.gen.nz>> wrote: > > Hi all, > > I see that lxd in ubuntu cosmic and disco is a transitional package for > snap - I see that lxd can be used for snap packages, but they're not the > same thing, right? > > > https://lists.ubuntu.com/archives/ubuntu-devel/2018-August/040455.html > https://bugs.launchpad.net/ubuntu/+source/lxd/+bug/1788040 > > > And Debian buster (even sid) still doesn't have lxd at all. > > > They have snapd. Which you can then use to instal lxd. > > > Is lxd not the future of lxc after all? At least in debian-based > distros? Or is it expected (by ubuntu) that we only use lxc for snap > packages? > > > You mean "use lxc from snap packages"? > Read the two links above. Ah. I see I had it all backwards. This is unfortunate for those of us who don't really like the snap(/flatpak/appimage/etc) model where we have to have (and keep current) multiple copies of the same software to get around dependency issues. It also doesn't fit very well with the Debian stable model, if the snap package is coming from outside Debian. Oh well - I guess the lxc package is sticking around, right? And maybe someone will pick up packaging lxd for Debian. Thanks, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] future of lxc/lxd? snap?
On 24/02/19 7:18 AM, Andrey Repin wrote: > Greetings, Richard Hector! > >> Yep, sure. But LXD is currently the most common way to manage LXC, >> right? > > Right? Right?! Riiight???!! > > Sorry, but your insistence is not going to change the facts. Steady on. I wasn't insisting on anything; I was checking my assumptions. Maybe my New Zealand English usage is more unusual than I thought. I guess you're saying my assumption was wrong. >> At least, when I ask LXC questions on here, people seem to be >> surprised that I'm not using LXD. > > I'm not using LXD. It's just not suitable for my use cases. Ok. I'm not either, but that's more because it's not in Debian, and I like to stick with packaged solutions where I can. I haven't even tried it yet. >> So with LXD transitioning to snap in Ubuntu, is it expected that LXD >> will only be used for snap packages? > >> And if I want to continue using LXC without snap, I won't get the >> advantages of managing it with LXD? > > I'd like to know the answer as well. Right. We can both wait and see if someone else answers :-) Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] future of lxc/lxd? snap?
On 24/02/19 5:17 AM, Andrey Repin wrote: > Greetings, Richard Hector! > >> Hi all, > >> I see that lxd in ubuntu cosmic and disco is a transitional package for >> snap - I see that lxd can be used for snap packages, but they're not the >> same thing, right? > >> And Debian buster (even sid) still doesn't have lxd at all. > >> Is lxd not the future of lxc after all? > >> At least in debian-based distros? Or is it expected (by ubuntu) that we only >> use lxc for snap packages? > > LXC is containers, LXD is orchestration tool. > You can't say that a flashlight is a future of lightbulb. > They are simply not the same nor a replacement of each other. Yep, sure. But LXD is currently the most common way to manage LXC, right? At least, when I ask LXC questions on here, people seem to be surprised that I'm not using LXD. So with LXD transitioning to snap in Ubuntu, is it expected that LXD will only be used for snap packages? And if I want to continue using LXC without snap, I won't get the advantages of managing it with LXD? Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] future of lxc/lxd? snap?
Hi all, I see that lxd in ubuntu cosmic and disco is a transitional package for snap - I see that lxd can be used for snap packages, but they're not the same thing, right? And Debian buster (even sid) still doesn't have lxd at all. Is lxd not the future of lxc after all? At least in debian-based distros? Or is it expected (by ubuntu) that we only use lxc for snap packages? I'm currently using lxc on debian, but wondering what happens next ... This all seems odd since the linuxcontainers site says the project is sponsored by Canonical ... Cheers, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] [Solved] Re: configuring ipv6 from container config file
On 9/02/19 4:45 PM, Richard Hector wrote: > Hi all, > > I have 2 similar container hosts - both debian stretch, both the same > lxc version. > > I'm not using lxd. > > On one, all my containers have their ip addresses configured via the > container config files, and I've been able to add ipv6 addresses that > way and it works fine. > > On the other, I used to set up the ip addresses within the container > (/etc/network/interfaces). When I add v6 on those, by that method, it's > fine. More recent containers have been set up using the container config > file, and ipv4 works but ipv6 doesn't. Found it - /etc/sysctl.d/disableipv6.conf existed. Somehow I thought that sysctl stuff wouldn't work in a container, but it does. Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] configuring ipv6 from container config file
Hi all, I have 2 similar container hosts - both debian stretch, both the same lxc version. I'm not using lxd. On one, all my containers have their ip addresses configured via the container config files, and I've been able to add ipv6 addresses that way and it works fine. On the other, I used to set up the ip addresses within the container (/etc/network/interfaces). When I add v6 on those, by that method, it's fine. More recent containers have been set up using the container config file, and ipv4 works but ipv6 doesn't. I can't see any obvious difference between the two hosts. Any tips on where I should be looking? Are there general preferences for each way of configuring networking? Here's a section of a sample config that doesn't work: lxc.network.type = veth lxc.network.flags = up lxc.network.link = br0 lxc.network.ipv4 = 192.168.122.112/24 lxc.network.ipv4.gateway = 192.168.122.1 lxc.network.ipv6 = fd75:5198:8dc2:7905::70/64 lxc.network.ipv6.gateway = fd75:5198:8dc2:7905::1 As I say, IPv4 is working, and has been for ages. IPv6 is newly added, and doesn't. 192.168.122.1 and fd75:5198:8dc2:7905::1 are both configured on the br0 in the host. Cheers, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] kernel messages appear in containers?
On 3/02/19 3:54 PM, Tomasz Chmielewski wrote: > On 2019-02-03 11:51, Richard Hector wrote: >> Hi all, >> >> I've noticed that some log messages that really belong to the host (like >> those from monthly RAID checks, for example) can appear in arbitrary >> containers instead - so they're spread all over the place. >> >> Is that normal/fixable? > > I'd say it's a "normal, bad default". > > You can fix it by adding this to your sysctl config file: > > kernel.dmesg_restrict = 1 On 3/02/19 9:47 PM, Stéphane Graber wrote:> Yes, this is normal and as pointed out already, can be tweaked with > dmesg_restrict. > > There was some interest a while back in implementing a logging > namespace, which would solve this cleanly, but it's never been enough > of a priority for anyone to actually do the kernel work for it. Thanks to both. Done; will watch for results :-) Cheers, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] kernel messages appear in containers?
Hi all, I've noticed that some log messages that really belong to the host (like those from monthly RAID checks, for example) can appear in arbitrary containers instead - so they're spread all over the place. Is that normal/fixable? Cheers, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Failed to reset devices.list (etc)
On 10/09/18 02:52, Christian Brauner wrote: >> So my understanding is: The systemd service file tries to restrict what >> the service can access, for security, by altering the devices.list >> >> The container won't let anything mess with devices.list, for security. >> >> The two can't both happen at the same time. > Yes. > >> Is that about right? >> >> Is it not possible to say "you can make it tighter, but not looser"? > Yes. > >> I guess I just add it to my logcheck ignores and carry on? > Yes. > > :) > Christian > Thanks :-) Where's the best place to read more about this? Thanks, Richard signature.asc Description: OpenPGP digital signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Failed to reset devices.list (etc)
On 10/09/18 01:40, Christian Brauner wrote: > On Mon, Sep 10, 2018 at 01:30:42AM +1200, Richard Hector wrote: >> Hi all, >> >> I have messages like this in the logs on several of my (lxc, not lxd) >> containers: >> >> systemd[1]: phpsessionclean.service: Failed to reset devices.list: >> Operation not permitted >> >> systemd[1]: run-user-1000.mount: Failed to reset devices.list: Operation >> not permitted >> >> systemd[1]: apt-daily.service: Failed to reset devices.list: Operation >> not permitted >> >> systemd[1]: Failed to reset devices.list on >> /system.slice/systemd-tmpfiles-clean.service: Operation not permitted >> >> systemd[1]: Failed to reset devices.list on >> /system.slice/apt-daily.service: Operation not permitted >> >> Host is debian stretch, guests are a mix of debian and ubuntu. >> >> Searching the web finds various results of various ages; some claim to >> be fixed, others not. >> >> Some claim it's an issue with unprivileged containers only, but AFAIK >> I'm using privileged containers only (how do I tell?) >> >> What I can't find is: >> >> What is devices.list, what specifically (in each case) wants to reset >> it, and why? >> >> Can and should I stop it, and how? > > No need to stop it. systemd will simply gracefully move one but report > an error. The devices.list regulates to what devices a privileged > container can have access to. The container not being able to mess with > it is very mucht wanted for security reasons. There's no way to stop it > from LXC's side. If you really care about this you could probably > disable all services that try to touch it. But it's really not needed. Thanks Christian, So my understanding is: The systemd service file tries to restrict what the service can access, for security, by altering the devices.list The container won't let anything mess with devices.list, for security. The two can't both happen at the same time. Is that about right? Is it not possible to say "you can make it tighter, but not looser"? I guess I just add it to my logcheck ignores and carry on? Thanks, Richard > Christian > >> >> There are some references to setting "PrivateNetwork=false" in the >> service file (for the phpsessionclean one, at least) - but that didn't >> seem to have any effect. >> >> Any tips? >> >> Thanks, >> Richard >> ___ >> lxc-users mailing list >> lxc-users@lists.linuxcontainers.org >> http://lists.linuxcontainers.org/listinfo/lxc-users >> >> >> ___ >> lxc-users mailing list >> lxc-users@lists.linuxcontainers.org >> http://lists.linuxcontainers.org/listinfo/lxc-users signature.asc Description: OpenPGP digital signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] Failed to reset devices.list (etc)
Hi all, I have messages like this in the logs on several of my (lxc, not lxd) containers: systemd[1]: phpsessionclean.service: Failed to reset devices.list: Operation not permitted systemd[1]: run-user-1000.mount: Failed to reset devices.list: Operation not permitted systemd[1]: apt-daily.service: Failed to reset devices.list: Operation not permitted systemd[1]: Failed to reset devices.list on /system.slice/systemd-tmpfiles-clean.service: Operation not permitted systemd[1]: Failed to reset devices.list on /system.slice/apt-daily.service: Operation not permitted Host is debian stretch, guests are a mix of debian and ubuntu. Searching the web finds various results of various ages; some claim to be fixed, others not. Some claim it's an issue with unprivileged containers only, but AFAIK I'm using privileged containers only (how do I tell?) What I can't find is: What is devices.list, what specifically (in each case) wants to reset it, and why? Can and should I stop it, and how? There are some references to setting "PrivateNetwork=false" in the service file (for the phpsessionclean one, at least) - but that didn't seem to have any effect. Any tips? Thanks, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] PAM-CGFS[xxx]: Failed to get list of controllers
On 09/07/18 21:45, Christian Brauner wrote: > On Mon, Jul 09, 2018 at 02:08:03AM +1200, Richard Hector wrote: >> Hi all, >> >> I'm seeing this in auth.log on an lxc host (no lxd). >> >> "PAM-CGFS[xxx]: Failed to get list of controllers" >> >> This is on a host I didn't set up; I've inherited maintenance for it, >> and recently added logcheck. >> >> On another host, which I did set up, I don't see that. I have noticed >> that the problem host has libpam-cgfs installed, whereas the other one >> doesn't. >> >> Both hosts are running Debian stretch, and IIRC both were upgraded from >> jessie. >> >> Any tips on what's causing this, and whether it's a problem? > > So, first of all let's determine if you need the pam module at all. :) > - Do you run unprivileged containers as an unprivileged user on the > affected host? If not, you can remove the pam module. > - What version is the package the pam module is installed from. It might > just be that it can't really handle the empty unified cgroup hierarchy > because it is too old. Apologies for not replying at the time - I just found this again ... I believe I solved it by removing libpam-cgfs. Thanks for your help :-) Richard signature.asc Description: OpenPGP digital signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] PAM-CGFS[xxx]: Failed to get list of controllers
Hi all, I'm seeing this in auth.log on an lxc host (no lxd). "PAM-CGFS[xxx]: Failed to get list of controllers" This is on a host I didn't set up; I've inherited maintenance for it, and recently added logcheck. On another host, which I did set up, I don't see that. I have noticed that the problem host has libpam-cgfs installed, whereas the other one doesn't. Both hosts are running Debian stretch, and IIRC both were upgraded from jessie. Any tips on what's causing this, and whether it's a problem? Thanks, Richard ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
