Re: [lxc-users] /proc/self/cgroup after setuid
> Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im Auftrag > > Hi! > > I have a small locally written app which performs setuid()/setgid() to a non- > root user, and then calls the lxc C api. > > I noticed that various of the calls would fail, and after sniffing around, I > have a > hint. Before looking into /proc//cgroup, the lxclib first checks > /proc/self/cgroup and fails if lacking write access to all it finds. That is > okay > except that /proc/self/cgroup "latches" to the invoking user and does not > change after setuid. I am not sure, if this is the reason for your problem but special files and SUID binaries can be quite dangerous. Therefore quite strict access limitations might be on those files, some even stricter than implied by file system permission. This was needed to e.g. mitigate privilege escalations via the proc file system, like the one described here https://lwn.net/Articles/476947/ Thus who may open/write what/when might seems quite counter-intuitive. Even open file descriptors may change their behaviour while open due to program calling exec() or set[ug]id. > ... ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxc-usernsexec not working any more (differently) in lxc2 when invoked as root user: better solutions?
> Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im > Auftrag > > Quoting Fiedler Roman (roman.fied...@ait.ac.at): > > Hello List, > > > > With LXC1 on Trusty following sequence was used to fill an unprivileged > > container as root, where only configuration exists but no content. With > LXC2 > > on Xenial, this results in an error: > > > > cd -- /var/lib/lxc/test/rootfs > > lxc-usernsexec -m u:0:296608:65536 -m g:0:296608:65536 -- tar > > Do you have /etc/subuid and /etc/subgid entries for root including > this range? > > > --numeric-owner --exclude=./dev -xjf > > [somepath]/ubuntuxenial1604-i386.tar.bz2 > > newuidmap: uid range [0-65536) -> [296608-362144) not allowed > > error mapping child > > > > Deleting the file "/usr/bin/newuidmap" fixes the problem, but I guess that > > is not the best idea :-) > > Right, that sounds like your lxc1 is so old that it defaults to not using > newuidmap when you're root, which was changed years ago to default to > using > newuidmap. (By requiring uid allocations for the root user, we prevent > accidental clashes with subuid allocations for non-root users ) But the uid-map supplied is correct for a given non-root user, just newuidmap assumes for some unknown reason, that there are clashes. When this procedure is prevented, because it is too risky, how to perform it correctly (without workarounds or downgrading to very old version)? Is the procedure depicted below the recommended, correct way to do it? > > Following command works also ... > > > > bzip2 -cd < [somepath]/ubuntuxenial1604-i386.tar.bz2 | PATH="" > > /usr/bin/lxc-usernsexec -m u:0:296608:65536 -m g:0:296608:65536 -- > /bin/tar > > --numeric-owner --exclude=./dev -x > > > > ... but maybe there is a smarter way to avoid that problem? Is there a way > > to use "lxc-create" in a way, that it does not touch any file-system > > property (mode/owner/xattrs) nor any file content EXCEPT extracting a tar > to > > the prepared directory? Using PATH does not seem very sensible as it could > > provoke regressions as it relies on undocumented internal function of " > > lxc-usernsexec". > > > > Kind regards, > > Roman > > > > PS: after UID-mapping the procedure should not attempt a chdir: when > mapped > > and not already inside, it will have no means to reach the container > > rootfs > > location any more (as no other non-host-root process has). smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] lxc-usernsexec not working any more (differently) in lxc2 when invoked as root user: better solutions?
Hello List, With LXC1 on Trusty following sequence was used to fill an unprivileged container as root, where only configuration exists but no content. With LXC2 on Xenial, this results in an error: cd -- /var/lib/lxc/test/rootfs lxc-usernsexec -m u:0:296608:65536 -m g:0:296608:65536 -- tar --numeric-owner --exclude=./dev -xjf [somepath]/ubuntuxenial1604-i386.tar.bz2 newuidmap: uid range [0-65536) -> [296608-362144) not allowed error mapping child Deleting the file "/usr/bin/newuidmap" fixes the problem, but I guess that is not the best idea :-) Following command works also ... bzip2 -cd < [somepath]/ubuntuxenial1604-i386.tar.bz2 | PATH="" /usr/bin/lxc-usernsexec -m u:0:296608:65536 -m g:0:296608:65536 -- /bin/tar --numeric-owner --exclude=./dev -x ... but maybe there is a smarter way to avoid that problem? Is there a way to use "lxc-create" in a way, that it does not touch any file-system property (mode/owner/xattrs) nor any file content EXCEPT extracting a tar to the prepared directory? Using PATH does not seem very sensible as it could provoke regressions as it relies on undocumented internal function of " lxc-usernsexec". Kind regards, Roman PS: after UID-mapping the procedure should not attempt a chdir: when mapped and not already inside, it will have no means to reach the container rootfs location any more (as no other non-host-root process has). DI Roman Fiedler Scientist Digital Safety & Security Department Assistive Healthcare Information Technology AIT Austrian Institute of Technology GmbH Reininghausstraße 13/1 | 8020 Graz | Austria T +43(0) 50550 2957 | M +43(0) 664 8561599 | F +43(0) 50550 2950 roman.fied...@ait.ac.at | http://www.ait.ac.at/ FN: 115980 i HG Wien | UID: ATU14703506 http://www.ait.ac.at/Email-Disclaimer smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] new lxd user (on Gentoo) - queries about container logging and configuration parameters
> Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im > Auftrag > > Hi all, > > I've found lxd recently and jumped on it. I've used vserver before but > this is a different league of simplicity and compatibility with stock > kernels. Great work indeed, many thanks to the developers and the > community for supporting and improving this product. > > There are a few issues that I couldn't figure out on my own by looking > on the Internet, so I'm putting them out to the experts: > > - One aspect that drew my attention was the fact that running 'dmesg' > on the containers shows the host's kernel log. This does not look > right to me, is it a configuration fault on the container or the lxd > system? No, it isn't, see http://seclists.org/fulldisclosure/2015/Mar/89 I find it also strange, that this is not disabled by default. Check http://unix.stackexchange.com/questions/103576/container-lockdown for more info on this and similar. > [Snip] > > Any pointers for addressing these issues would be much appreciated. > > Best regards, > Pedro. smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] apparmor kernel log entries
> Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im > > Quoting Fiedler Roman (roman.fied...@ait.ac.at): > > > Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im > > > Auftrag von Serge Hallyn > > > > > > Wait - are you saying you want tasks in the container to be able to > > > ptrace tasks on the host? > > > > Yes, is possible. Sounds like > > I'm asking whether he *wants* the tasks to be able to do that. Well, not knowing his exact setup, question might be if he wants the machine to want it. With an adversary on it already, it would be the the best way to hide the ptrace-exploit within a top/ps instance (thus hide also true positive failed exploitation attempt messages from auditd within false-positive ptrace-attach messages when accessing proc file system, making it easier to come to same FP-conclusion on a TP), best using a rouge shared library from tempfs with ld-preload (thus not really leaving traces on the disk), and as a whole, thus being hard to distinguish from real admin activity using only really trivial tricks. > If not, then apparmor is correctly stopping the exploit. If so, then I'd > be interested in the use case. Yes, if he did not modify apparmor and has the appropriate lxc-updates from last year in place mitigating the apparmor-escape exploits. smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] iptables-save not working in unprivileged containers?
> Von: Tomasz Chmielewski [mailto:man...@wpkg.org] > > On 2015-11-10 01:22, Fiedler Roman wrote: > > >> # iptables -A INPUT -p tcp --dport 22 -j ACCEPT > > > > Yes, also here. > > > > Compare > > > > iptables-save > > > > with > > > > iptables-save -t filter > > > > Later should work. I think, that some special tables cannot be read in > > unpiv > > (mangle perhaps). > > It seems to behave just like "iptables-save" executed by non-root user > (in non-container). Not on this side: * Normal user: $ iptables-save -t filter iptables-save v1.4.21: Cannot initialize: Permission denied (you must be root) * As root in unpriv container: # iptables-save -t filter # Generated by iptables-save v1.4.21 on Mon Nov 9 16:55:27 2015 *filter :INPUT DROP [0:0] smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] iptables-save not working in unprivileged containers?
> Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im > Auftrag > > For some, reason, iptables-save does not seem to be working in > unprivileged containers. > > To reproduce: > > - this adds a sample iptables rule: > > # iptables -A INPUT -p tcp --dport 22 -j ACCEPT Yes, also here. Compare iptables-save with iptables-save -t filter Later should work. I think, that some special tables cannot be read in unpiv (mangle perhaps). > [Snip] LG R smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] lxc-usernsexec completely unexpected behaviour, reproducible on trunk?
Hello List,
I've accidentally destroyed some files due to following unexpected
behaviour. Is this also reproducible on trunk?
# echo "content" > file
# cat file
content
# lxc-usernsexec -m u:0:851968:65536 -m g:0:851968:65536 -- /bin/echo xxx <
file
# cat file
xxx
ent
It seems, that the bad magic at least on Ubuntu trusty version is from:
readlink("/proc/self/fd/0", "/tmp/file", 256) = 40 # don't know why, but
stdin link is copied
pipe([3, 4])= 0
pipe([5, 6])= 0
clone(Process 8253 attached
# stdin link is opened RW and duped to stdin/out/err
[pid 8253] open("/tmp/file", O_RDWR|O_NONBLOCK) = 3
[pid 8253] fcntl(3, F_GETFL) = 0x8802 (flags
O_RDWR|O_NONBLOCK|O_LARGEFILE)
[pid 8253] fcntl(3, F_SETFL, O_RDWR|O_LARGEFILE) = 0
[pid 8253] close(0)= 0
[pid 8253] close(1)= 0
[pid 8253] close(2)= 0
[pid 8253] dup2(3, 0) = 0
[pid 8253] dup2(3, 1) = 1
[pid 8253] dup2(3, 2) = 2
DI Roman Fiedler
Scientist
Digital Safety & Security Department
Assistive Healthcare Information Technology
AIT Austrian Institute of Technology GmbH
Reininghausstraße 13/1 | 8020 Graz | Austria
T +43(0) 50550 2957 | M +43(0) 664 8561599 | F +43(0) 50550 2950
roman.fied...@ait.ac.at | http://www.ait.ac.at/
FN: 115980 i HG Wien | UID: ATU14703506
http://www.ait.ac.at/Email-Disclaimer
smime.p7s
Description: S/MIME cryptographic signature
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] monitoring containers using lxc-info (without being root)
Hi, Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im Auftrag On 05/11/15 20:35, Stéphane Graber wrote: lxc-info -c doesn't read the container configuration, instead it connects to the container's command socket and asks the container what's the running configuration. That means that you need to run lxc-info as the same user which started the container for it to be able to contact the command socket. Understood, but I cannot run all my containers as zabbix or nagios user just for monitoring. sudo would be an option, but I wonder if the socket could be created with a lxc group permission and to add the monitoring account to this group? Maybe a second read-only socket to obtain information from the container (without the control part) could do the trick? Just a suggestion, of course. Keep on your good work. I have similar problem with following solution: minimal monitoring components running in container, writing messages to container-internal spool. Those messages are forwarded via pipe (lxc-attach) to host and then to Zabbix instance via TLS and are injected to Zabbix using trapper. All the spooling/forwarding components are identical, so there is only need for 1 type of spooling component (~25k) and the minimal tests (~30kb). Another advantage: Full history of measurements event with network connectivity loss between central monitoring point and LXC-hosts/guest. Roman. smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] What's the best way to copy file from host to container?
Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im Auftrag Greetings, LXC users mailing-list! What about packing the files with e.g. cpio on the host and injecting them via pipe to ns-attached cpio-process running in guest scope. This also works around guest to host privilege escalation if malicious guest content moves around fs-tree parts while copying. Could you please elaborate that? May be give an example implementation? Example: I want to inject two files, one owned by root other one by user. All commands on host: # start in empty directory on host touch x touch y chown 1000.100 y find . | cpio -o | lxc-attach --name lxc-guest -- cpio -i -d As second cpio runs within guest, it will automatically pick up the correct uid namespace. With tar you can even select if you want to inject files by (guest) file-UIDs or use the intelligent tar username-to-uid mapping algorithm. Apart from that, as second cpio is not only chrooted, but running in unprivileged namespace of container, malicious container cannot escalate to host using the cpio via trivial symlinking. To my knowledge, malicious guest may still escalate to host due to TIOCSTI syscall if guest root user is malicious and e.g. places a malicious libc in container and injection command is called from terminal. But method is safe with malicious guest non-root-uid processes, escape should be impossible already in that case. By detaching the injection process on host from any controlling tty, you could even eliminate the last guest-host escalation risk also. To my knowledge, this is the only secure way to inject files into a running container just involving file system and pipes (no network). Thank you, much appreciated. It didn't occurred to me that I could pipe something to lxc-attach :) Yes, but be aware of the risks involved and use carefully. There should be quite some ways for a malicious guest to influence the host. If I'm not wrong, but changing the command from above to find . | cpio -o | lxc-attach --name lxc-guest -- cpio -i -d /dev/null might allow guest to cause service errors in arbitrary host processes when they try to access /dev/null. But perhaps LXC pros could confirm or dismiss this assumption. Roman smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] What's the best way to copy file from host to container?
Hello Andrey, Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im Auftrag Greetings, Fiedler Roman. Monday, April 27, 2015, 09:14:26 you wrote: This seems to be the perfect solution, I can just copy over to the path like proc/12423/root/usr/local, without worrying about snapshot clone uses delta0, rather than root in the container folder. After I moved the file, I still need to update the file permissions and ownership though. What about packing the files with e.g. cpio on the host and injecting them via pipe to ns-attached cpio-process running in guest scope. This also works around guest to host privilege escalation if malicious guest content moves around fs-tree parts while copying. Could you please elaborate that? May be give an example implementation? Example: I want to inject two files, one owned by root other one by user. All commands on host: # start in empty directory on host touch x touch y chown 1000.100 y find . | cpio -o | lxc-attach --name lxc-guest -- cpio -i -d As second cpio runs within guest, it will automatically pick up the correct uid namespace. With tar you can even select if you want to inject files by (guest) file-UIDs or use the intelligent tar username-to-uid mapping algorithm. Apart from that, as second cpio is not only chrooted, but running in unprivileged namespace of container, malicious container cannot escalate to host using the cpio via trivial symlinking. To my knowledge, malicious guest may still escalate to host due to TIOCSTI syscall if guest root user is malicious and e.g. places a malicious libc in container and injection command is called from terminal. But method is safe with malicious guest non-root-uid processes, escape should be impossible already in that case. By detaching the injection process on host from any controlling tty, you could even eliminate the last guest-host escalation risk also. To my knowledge, this is the only secure way to inject files into a running container just involving file system and pipes (no network). Roman smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] What's the best way to copy file from host to container?
Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im Auftrag Hi Andrey, This seems to be the perfect solution, I can just copy over to the path like proc/12423/root/usr/local, without worrying about snapshot clone uses delta0, rather than root in the container folder. After I moved the file, I still need to update the file permissions and ownership though. What about packing the files with e.g. cpio on the host and injecting them via pipe to ns-attached cpio-process running in guest scope. This also works around guest to host privilege escalation if malicious guest content moves around fs-tree parts while copying. Roman On Fri, Apr 24, 2015 at 2:43 PM, Andrey anrdae...@yandex.ru wrote: Greetings, Dan Shi. Saturday, April 25, 2015, 00:37:27 you wrote: DS I need to deploy some config files, e.g., .ssh config, key file etc, to DS container. I can copy the files to the absolute path in container, e.g., DS /usr/local/containers/base/rootfs/root/.ssh/ DS The problem is that, I have to change the owner and permission of the DS files manually. I'm wondering if there is a better way to do it, like scp? ls -l /proc/$(lxc-info -n $NAME -p -H)/ Courtesy @stgraber -- Sincerely, Andrey Saturday, April 25, 2015 00:41:52 Sorry for my terrible english... ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxc-security: iptables audit with nflog not working with default settings (insecure)
Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im Auftrag von Fajar A. Nugraha On Wed, Mar 11, 2015 at 8:03 PM, Fiedler Roman roman.fied...@ait.ac.at wrote: But the current issue is different: The guest can snoop on the NFLOG messages generated on host and destined for the host and hence can get knowledge of ANY NFLOGed connection of host or any guest, no matter if on same bridge or another one. Ah, sorry I misunderstood your problem. All I can say is that it works for me on my simple test. I have ulogd2 on both host and guest, and if you look at my iptables command on the host and guest, they are almost identical (including nflog group) except for chain names (forward/input/output). The logged packets are from the correct one (the one inside the container has in/out=eth0, while the one on the host has in/out=br0). That was on Ubuntu 14.10 (kernel 3.16) with lxc-1.1 from daily ppa, so you might want to try that before filing a bug report to ubuntu. OK, I think I understand it now: On my setup, the packets captured by iptables firewall use the correct NFLOG group and make it to the ulogd on the host BUT the log lines end up in kernel message ring buffer (dump with dmesg) and those messages can be read/manipulated by both host AND guest. As there is a rsyslog running in guest, extracting messages from kernel message buffer from time to time, this will consume those messages and write it to the guest logs. With that information, I found report [1] suggesting to use echo 1 /proc/sys/kernel/dmesg_restrict, but that somehow fails. At least we can now change the subject to dmesg insecure with default settings Roman [1] http://unix.stackexchange.com/questions/103576/container-lockdown smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
[lxc-users] lxc-security: iptables audit with nflog not working with default settings (insecure)
Hello list, Has someone managed to get reliable network traffic auditing with LXC up and running? That means, that it is possible to write a protocol of e.g. every new connection from and to host. On my setup (Ubuntu Trusty), both host and guest may have different iptables rulesets. But the guest NFLOG messages are lost completely, those from host are sometimes sent to the ulogd in the guest (time-race), so the host log is not trustworthy also. What could be the best solution to get trustworthy logs with LXC? Kind regards, Roma DI Roman Fiedler Scientist Digital Safety Security Department Assistive Healthcare Information Technology AIT Austrian Institute of Technology GmbH Reininghausstraße 13/1 | 8020 Graz | Austria T +43(0) 50550 2957 | M +43(0) 664 8561599 | F +43(0) 50550 2950 roman.fied...@ait.ac.at | http://www.ait.ac.at/ FN: 115980 i HG Wien | UID: ATU14703506 http://www.ait.ac.at/Email-Disclaimer smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxc-security: iptables audit with nflog not working with default settings (insecure)
Von: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] Im Auftrag On Wed, Mar 11, 2015 at 5:48 PM, Fiedler Roman roman.fied...@ait.ac.at wrote: Hello list, Has someone managed to get reliable network traffic auditing with LXC up and running? That means, that it is possible to write a protocol of e.g. every new connection from and to host. On my setup (Ubuntu Trusty), both host and guest may have different iptables rulesets. But the guest NFLOG messages are lost completely, those from host are sometimes sent to the ulogd in the guest (time-race), so the host log is not trustworthy also. What could be the best solution to get trustworthy logs with LXC? Try something like this on the host: echo 1 /proc/sys/net/bridge/bridge-nf-call-iptables iptables -I FORWARD 1 -d 192.168.124.173 -j NFLOG --nflog-group 0 --nflog-prefix lxc-v iptables -I FORWARD 1 -s 192.168.124.173 -j NFLOG --nflog-group 0 --nflog-prefix lxc-v with the default ulogd2 setup on ubuntu 14.10 (which already includes rules for nflog-group 0 logging to a file) you should then be able to get something like this when the container (192.168.124.173) pings another container (192.168.124.134) This should be exactly the configuration I have tested so far. But that did not yet solve my problem ... * If some process in guest registers for the same NFLOG queue, he can steal the messages from the host queue, thus removing traces of his activity from host logging. SECURITY-ASPECT: apart from log corruption, the guest can get knowledge about any other connection to/from other containers and the host and as they include sequence numbers, may be able to inject spoofed data into any other unencrypted TCP connection or at least interrupt the connection using another helper machine. * If a guest should be protected with iptables also, e.g. to avoid Apache or Tomcat to connect to the local SSH port, those error logs - which are useful to detect ongoing guest intrusions - do not make it to any log-file. [...snip...] You might only be missing the bridge-nf-call-iptables part. Note that you shouldn't need it IF you use a custom lxc network setup which doesn't use bridges: https://www.mail-archive.com/lxc-users@lists.linuxcontainers.org/msg02587.html I've tried various network configurations also. I fear that effort here is quite futile since I do not yet understand the core kernel namespace concepts, e.g. how nflink and user namespaces work together. Hence everything I could do is (inefficient) trial and error instead of controlled engineering. And in the end I cannot be sure that there are not reliability/security-relevant holes left open with trial/error. Without any other clues from the mailing list, I'll still try out this procedure also and see if it would change the nflog behavior. Roman smime.p7s Description: S/MIME cryptographic signature ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
