Re: Debian Stable server hacked
On Fri, Aug 22, 2003 at 10:32:27AM -0400, Matt Zimmerman wrote:
On Wed, Aug 20, 2003 at 05:23:30PM +0200, Adam ENDRODI wrote:
You don't need an executable stack to get control of execution, you only
need to be able to change the instruction pointer, which is stored on
the stack (as data).
PaX is not just about non-executable address regions, but address space
randomization. In my understanding, the attacker just doesn't know what
he should modify the IP to. Given this, are you certain that only a
narrow range of exploits (common implementations) can be killed via PaX?
It is often the case that the attacker doesn't know the exact location of
structures in memory; there are techniques for finding out. I'm sure that
the authors of PaX do not misrepresent it as complete protection.
Sure, that's why you have the ACL for the *just in case* scenario.
And yes, that includes a restrictive default ACL just in case someone
manages to go around the application specific ACL.
For instance, let's look at a very simple ACL I have for MySQL:
/usr/sbin/mysqld oX {
/usr/share/mysql r
/var/lib/mysql rw
/var/log/mysql a
/ h
-CAP_ALL
RES_CRASH 1 10m
connect {
disabled
}
bind {
disabled
}
}
So that everything is hidden except for 3 paths. And all those
are restricted somewhat. The only writable path is the mysql database path.
You cannot execute stuff there. Sure, I don't think that Grsec prevents
me from dynamically loaded an .so and running it, *BUT* then all of that
code gets executed in mysqld so you are still stuck with this ACL.
I think there would have to be some hole in Grsec for one to allow
you to break out of this ACL.
Furthermore, the point of the ACL system is not only to restrict specific applications,
but to restrict the root user itself so that someone breaking into root,
even getting shell, will not be able to do much to /bin or /usr/bin or /lib
or even /etc since (at least I) have /etc read-only.
Hence I would have to say that Pax is a good deterrent for script-kiddies
but an ACL with Pax (like in Grsec) makes it difficult even for the
best of them (I hope :) he he
It's pointless to argue about it; it's clear that PaX provides some value in
protection against security vulnerabilities, and I think it's also clear
that because it will break many existing applications, it is not suitable
for use by default. But there is no reason why a PaX-enabled kernel could
not be provided as an option. All it needs is someone willing to do the
work (hint, hint).
I would *not* recommend a Grsec or pax system for Desktop by default :)
I'm not even sure if it would be good to include it in Debian. IMHO, it is
best to build a monolithic kernel and disable all the options you don't
need for a specific machine when you patch the kernel with Grsec or pax.
- Adam
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Fri, Aug 22, 2003 at 10:32:27AM -0400, Matt Zimmerman wrote:
On Wed, Aug 20, 2003 at 05:23:30PM +0200, Adam ENDRODI wrote:
You don't need an executable stack to get control of execution, you only
need to be able to change the instruction pointer, which is stored on
the stack (as data).
PaX is not just about non-executable address regions, but address space
randomization. In my understanding, the attacker just doesn't know what
he should modify the IP to. Given this, are you certain that only a
narrow range of exploits (common implementations) can be killed via PaX?
It is often the case that the attacker doesn't know the exact location of
structures in memory; there are techniques for finding out. I'm sure that
the authors of PaX do not misrepresent it as complete protection.
Sure, that's why you have the ACL for the *just in case* scenario.
And yes, that includes a restrictive default ACL just in case someone
manages to go around the application specific ACL.
For instance, let's look at a very simple ACL I have for MySQL:
/usr/sbin/mysqld oX {
/usr/share/mysql r
/var/lib/mysql rw
/var/log/mysql a
/ h
-CAP_ALL
RES_CRASH 1 10m
connect {
disabled
}
bind {
disabled
}
}
So that everything is hidden except for 3 paths. And all those
are restricted somewhat. The only writable path is the mysql database path.
You cannot execute stuff there. Sure, I don't think that Grsec prevents
me from dynamically loaded an .so and running it, *BUT* then all of that
code gets executed in mysqld so you are still stuck with this ACL.
I think there would have to be some hole in Grsec for one to allow
you to break out of this ACL.
Furthermore, the point of the ACL system is not only to restrict specific
applications,
but to restrict the root user itself so that someone breaking into root,
even getting shell, will not be able to do much to /bin or /usr/bin or /lib
or even /etc since (at least I) have /etc read-only.
Hence I would have to say that Pax is a good deterrent for script-kiddies
but an ACL with Pax (like in Grsec) makes it difficult even for the
best of them (I hope :) he he
It's pointless to argue about it; it's clear that PaX provides some value in
protection against security vulnerabilities, and I think it's also clear
that because it will break many existing applications, it is not suitable
for use by default. But there is no reason why a PaX-enabled kernel could
not be provided as an option. All it needs is someone willing to do the
work (hint, hint).
I would *not* recommend a Grsec or pax system for Desktop by default :)
I'm not even sure if it would be good to include it in Debian. IMHO, it is
best to build a monolithic kernel and disable all the options you don't
need for a specific machine when you patch the kernel with Grsec or pax.
- Adam
Re: Debian Stable server hacked
On Fri, Aug 22, 2003 at 06:35:37PM -0400, Phillip Hofmeister wrote: On Fri, 22 Aug 2003 at 10:32:27AM -0400, Matt Zimmerman wrote: It is often the case that the attacker doesn't know the exact location of structures in memory; there are techniques for finding out. I'm sure that the authors of PaX do not misrepresent it as complete protection. It's pointless to argue about it; it's clear that PaX provides some value in protection against security vulnerabilities, and I think it's also clear that because it will break many existing applications, it is not suitable for use by default. But there is no reason why a PaX-enabled kernel could not be provided as an option. All it needs is someone willing to do the work (hint, hint). I would be willing to maintain a grsec kernel image with PaX and temp. file symlink blocking if someone would be willing to sponsor it (hint, hint) I really do not have the time to sponsor you, but would like to see this happen. If you put together reasonable packages and ask on the mailing lists, I don't think you'd have a problem finding a sponsor. There are a number developers who are interested in this. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Fri, Aug 22, 2003 at 06:35:37PM -0400, Phillip Hofmeister wrote: On Fri, 22 Aug 2003 at 10:32:27AM -0400, Matt Zimmerman wrote: It is often the case that the attacker doesn't know the exact location of structures in memory; there are techniques for finding out. I'm sure that the authors of PaX do not misrepresent it as complete protection. It's pointless to argue about it; it's clear that PaX provides some value in protection against security vulnerabilities, and I think it's also clear that because it will break many existing applications, it is not suitable for use by default. But there is no reason why a PaX-enabled kernel could not be provided as an option. All it needs is someone willing to do the work (hint, hint). I would be willing to maintain a grsec kernel image with PaX and temp. file symlink blocking if someone would be willing to sponsor it (hint, hint) I really do not have the time to sponsor you, but would like to see this happen. If you put together reasonable packages and ask on the mailing lists, I don't think you'd have a problem finding a sponsor. There are a number developers who are interested in this. -- - mdz
Re: Debian Stable server hacked
* Matt Zimmerman ([EMAIL PROTECTED]) wrote: On Fri, Aug 22, 2003 at 06:35:37PM -0400, Phillip Hofmeister wrote: I would be willing to maintain a grsec kernel image with PaX and temp. file symlink blocking if someone would be willing to sponsor it (hint, hint) I really do not have the time to sponsor you, but would like to see this happen. If you put together reasonable packages and ask on the mailing lists, I don't think you'd have a problem finding a sponsor. There are a number developers who are interested in this. I might be willing to sponsor it. Contact me off-list if interested. Stephen pgpsEdrw3Va80.pgp Description: PGP signature
Re: Debian Stable server hacked
On Sat, Aug 23, 2003 at 10:14:24AM +0100, Dale Amon wrote: Does anyone know when a grsec patch set will be available for 2.6.0t3 or know of one updated to work with 2.4.22rc2? Yeah, I know, they are still experimental... This would be a great question posed to the GrSecurity forum, http://forums.grsecurity.net/ and in fact there's a thread on there already about it. Their forums are excellent. Steve -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Fri, Aug 22, 2003 at 06:35:37PM -0400, Phillip Hofmeister wrote: On Fri, 22 Aug 2003 at 10:32:27AM -0400, Matt Zimmerman wrote: It is often the case that the attacker doesn't know the exact location of structures in memory; there are techniques for finding out. I'm sure that the authors of PaX do not misrepresent it as complete protection. It's pointless to argue about it; it's clear that PaX provides some value in protection against security vulnerabilities, and I think it's also clear that because it will break many existing applications, it is not suitable for use by default. But there is no reason why a PaX-enabled kernel could not be provided as an option. All it needs is someone willing to do the work (hint, hint). I would be willing to maintain a grsec kernel image with PaX and temp. file symlink blocking if someone would be willing to sponsor it (hint, hint) Does anyone know when a grsec patch set will be available for 2.6.0t3 or know of one updated to work with 2.4.22rc2? Yeah, I know, they are still experimental...
Re: Debian Stable server hacked
On Sat, Aug 23, 2003 at 10:14:24AM +0100, Dale Amon wrote: Does anyone know when a grsec patch set will be available for 2.6.0t3 or know of one updated to work with 2.4.22rc2? Yeah, I know, they are still experimental... This would be a great question posed to the GrSecurity forum, http://forums.grsecurity.net/ and in fact there's a thread on there already about it. Their forums are excellent. Steve
Re: Debian Stable server hacked
On Wed, Aug 20, 2003 at 05:23:30PM +0200, Adam ENDRODI wrote: On Thu, Aug 14, 2003 at 12:00:40PM -0400, Matt Zimmerman wrote: No, it really doesn't. It might stop some common implementations of exploits, but that's about it. There are many papers available which describe the shortcomings of this kind of prevention. Could you provide some pointers on the topic? Google can provide enough. The general idea of stack protection is to prevent common automated attacks from working. Given time and determination, it is usually possible to circumvent it. You don't need an executable stack to get control of execution, you only need to be able to change the instruction pointer, which is stored on the stack (as data). PaX is not just about non-executable address regions, but address space randomization. In my understanding, the attacker just doesn't know what he should modify the IP to. Given this, are you certain that only a narrow range of exploits (common implementations) can be killed via PaX? It is often the case that the attacker doesn't know the exact location of structures in memory; there are techniques for finding out. I'm sure that the authors of PaX do not misrepresent it as complete protection. It's pointless to argue about it; it's clear that PaX provides some value in protection against security vulnerabilities, and I think it's also clear that because it will break many existing applications, it is not suitable for use by default. But there is no reason why a PaX-enabled kernel could not be provided as an option. All it needs is someone willing to do the work (hint, hint). -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 22 Aug 2003 at 10:32:27AM -0400, Matt Zimmerman wrote: It is often the case that the attacker doesn't know the exact location of structures in memory; there are techniques for finding out. I'm sure that the authors of PaX do not misrepresent it as complete protection. It's pointless to argue about it; it's clear that PaX provides some value in protection against security vulnerabilities, and I think it's also clear that because it will break many existing applications, it is not suitable for use by default. But there is no reason why a PaX-enabled kernel could not be provided as an option. All it needs is someone willing to do the work (hint, hint). I would be willing to maintain a grsec kernel image with PaX and temp. file symlink blocking if someone would be willing to sponsor it (hint, hint) - -- Phillip Hofmeister PGP/GPG Key: http://www.zionlth.org/~plhofmei/ wget -O - http://www.zionlth.org/~plhofmei/key.txt | gpg --import - -- Excuse #100: We just switched to FDDI. -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.2 (GNU/Linux) iD8DBQE/Rpq3S3Jybf3L5MQRAqkxAJ96rsDDKGr583UiBxDZEiaPuiS0sACeKD0r 1VLdCtM3Kg1jQ/oztj24NFk= =mBQL -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 20, 2003 at 05:23:30PM +0200, Adam ENDRODI wrote: On Thu, Aug 14, 2003 at 12:00:40PM -0400, Matt Zimmerman wrote: No, it really doesn't. It might stop some common implementations of exploits, but that's about it. There are many papers available which describe the shortcomings of this kind of prevention. Could you provide some pointers on the topic? Google can provide enough. The general idea of stack protection is to prevent common automated attacks from working. Given time and determination, it is usually possible to circumvent it. You don't need an executable stack to get control of execution, you only need to be able to change the instruction pointer, which is stored on the stack (as data). PaX is not just about non-executable address regions, but address space randomization. In my understanding, the attacker just doesn't know what he should modify the IP to. Given this, are you certain that only a narrow range of exploits (common implementations) can be killed via PaX? It is often the case that the attacker doesn't know the exact location of structures in memory; there are techniques for finding out. I'm sure that the authors of PaX do not misrepresent it as complete protection. It's pointless to argue about it; it's clear that PaX provides some value in protection against security vulnerabilities, and I think it's also clear that because it will break many existing applications, it is not suitable for use by default. But there is no reason why a PaX-enabled kernel could not be provided as an option. All it needs is someone willing to do the work (hint, hint). -- - mdz
Re: Debian Stable server hacked
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 22 Aug 2003 at 10:32:27AM -0400, Matt Zimmerman wrote: It is often the case that the attacker doesn't know the exact location of structures in memory; there are techniques for finding out. I'm sure that the authors of PaX do not misrepresent it as complete protection. It's pointless to argue about it; it's clear that PaX provides some value in protection against security vulnerabilities, and I think it's also clear that because it will break many existing applications, it is not suitable for use by default. But there is no reason why a PaX-enabled kernel could not be provided as an option. All it needs is someone willing to do the work (hint, hint). I would be willing to maintain a grsec kernel image with PaX and temp. file symlink blocking if someone would be willing to sponsor it (hint, hint) - -- Phillip Hofmeister PGP/GPG Key: http://www.zionlth.org/~plhofmei/ wget -O - http://www.zionlth.org/~plhofmei/key.txt | gpg --import - -- Excuse #100: We just switched to FDDI. -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.2 (GNU/Linux) iD8DBQE/Rpq3S3Jybf3L5MQRAqkxAJ96rsDDKGr583UiBxDZEiaPuiS0sACeKD0r 1VLdCtM3Kg1jQ/oztj24NFk= =mBQL -END PGP SIGNATURE-
Re: Debian Stable server hacked
On Thu, Aug 14, 2003 at 12:00:40PM -0400, Matt Zimmerman wrote: On Wed, Aug 13, 2003 at 09:00:51PM -0400, valerian wrote: It actually does a very good job of stopping any kind of stack-smashing attack dead in its tracks (both the stack and heap are marked as non-executable). That takes care of most vulnerabilities, both known and unknown. No, it really doesn't. It might stop some common implementations of exploits, but that's about it. There are many papers available which describe the shortcomings of this kind of prevention. Could you provide some pointers on the topic? You don't need an executable stack to get control of execution, you only need to be able to change the instruction pointer, which is stored on the stack (as data). PaX is not just about non-executable address regions, but address space randomization. In my understanding, the attacker just doesn't know what he should modify the IP to. Given this, are you certain that only a narrow range of exploits (common implementations) can be killed via PaX? bit, adam -- 1024D/37B8D989 954B 998A E5F5 BA2A 3622 82DD 54C2 843D 37B8 D989 finger://[EMAIL PROTECTED] | Some days, my soul's confined http://www.keyserver.net | And out of mind Sleep forever -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 20, 2003 at 05:23:30PM +0200, Adam ENDRODI wrote: No, it really doesn't. It might stop some common implementations of exploits, but that's about it. There are many papers available which describe the shortcomings of this kind of prevention. Could you provide some pointers on the topic? There was recently a long thread on bugtraq about this very topic (Subject was Buffer overflow prevention). You'll find some valuable information in there. The thread got kicked off bugtraq to secprog by the moderator and may still be alive there. noah pgp0.pgp Description: PGP signature
Re: Debian Stable server hacked
On Wed, Aug 20, 2003 at 05:23:30PM +0200, Adam ENDRODI wrote: No, it really doesn't. It might stop some common implementations of exploits, but that's about it. There are many papers available which describe the shortcomings of this kind of prevention. Could you provide some pointers on the topic? There was recently a long thread on bugtraq about this very topic (Subject was Buffer overflow prevention). You'll find some valuable information in there. The thread got kicked off bugtraq to secprog by the moderator and may still be alive there. noah pgpmwBAqUcjtp.pgp Description: PGP signature
Re: Debian Stable server hacked
On Thu, 14 Aug 2003 at 08:22:37PM -0400, Colin Walters wrote: On Wed, 2003-08-13 at 21:00, valerian wrote: Well capabilities are only one of the things that grsec implements. You can also restrict a process to access various parts of the filesystem. There's no reason /usr/sbin/apache should have write access to /etc, so you just don't allow it. Right, but we were discussing the scenario where the attacker is able to execute another program, such as /bin/sh. In that case all is lost, because the security is only associated with the executable pathname. With grsecurity ACLs can be inherited (from a parent process) and over-ridden... -- Phillip Hofmeister PGP/GPG Key: http://www.zionlth.org/~plhofmei/ wget -O - http://www.zionlth.org/~plhofmei/key.txt | gpg --import -- Excuse #101: User to computer ratio too high. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Thu, 14 Aug 2003 at 10:12:06PM -0400, Colin Walters wrote: On Wed, 2003-08-13 at 00:20, Adam Majer wrote: So, now I don't run a Debian kernel at all - only a monolithic (no modules) kernel This doesn't provide very much security. For example: http://www.phrack.org/show.php?p=58a=7 This is why gr-security can be set up to prohibit writes to kernel memory (by anyone). -- Phillip Hofmeister PGP/GPG Key: http://www.zionlth.org/~plhofmei/ wget -O - http://www.zionlth.org/~plhofmei/key.txt | gpg --import -- Excuse #68: Yes yes its called a design limitation -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, 2003-08-13 at 16:02, Colin Walters wrote: Let me give an example of how SELinux protects my machine (verbum.org). My blog is a Python script (pyblosxom) which runs in a domain called httpd_user_script_t. Oh, and what I forgot to mention about this domain is that it doesn't have write access to any files, for example. Nor (obviously) can it use the mount syscall. And those restrictions carry over to any other program it executes, including /bin/sh, /bin/ls, /bin/mount, whatever. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
Hi, maybe a legitimate user account combined with a local root exploit have been used to crack the server. Does this server has any legitimate user accounts? Are you sure you trust this users? Are you sure they (or you) don't write their passwords on a piece of paper? Who has local access to the server (unprotected LILO/Grub, booting from CDROM (KNOPPIX), mount the hd on another system)? Even if it might be manipulated, you should check the uptime of the system. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Thu, 07 Aug 2003 03:00:12 +0200, Peter Cordes wrote: sshd logs IP addresses of connections. Was the IP address for those did not receive id connections inside your site, or does it belong to an ISP somewhere, or what? If it's a local address, and not a computer lab, that might give you some clues about whose door to knock on... A professional cracker would have cleaned the sshd logs. So you can't really trust this logfile. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: [d-security] Debian Stable server hacked
Hello On Wed, Aug 06, 2003 at 04:01:39PM +0200, Thijs Welman wrote: I'm puzzled about how they managed to get those processes running (as root). There are no local accounts, other than some accounts for the sysadmins. Does anyone have any idea how they might have done this? Most times, servers are not cracked by somebody logging in and get root permissions somehow but by somebody who convinces a running network daemon like a web, database or mail server via means of buffer overflows etc to execute arbitrary code instructions. This code will then e.g. write a shell script and executes it or spanws a shell. You will never see an atacker in your last log :-) Try nmap to see which services are reachable from the network. bye, -christian- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
*** REPLY SEPARATOR *** On 12.08.2003 at 23:20 Adam Majer wrote: On Thu, Aug 07, 2003 at 07:03:13PM +0200, Thijs Welman wrote: Hi, Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? Generally if there is a kernel exploit, it is only used to get root from some other account. The way they get in is though some server app with a hole in it (known or not known). snip This is why my personal favourite it the former trusted debian project, now kown as http://www.adamantix.org. Take a look at their site, they offer RSBAC, PaX, all the goodies for the Kernel AND: They recompile all packages to be buffer overflow proof and as secure as possible. Mixing with standard debian packages is not recommended of course, but so far I haven't encountered any problems. Nearly everything is there if You don't require X anyway. regards Martin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 13, 2003 at 09:00:51PM -0400, valerian wrote: It actually does a very good job of stopping any kind of stack-smashing attack dead in its tracks (both the stack and heap are marked as non-executable). That takes care of most vulnerabilities, both known and unknown. No, it really doesn't. It might stop some common implementations of exploits, but that's about it. There are many papers available which describe the shortcomings of this kind of prevention. You don't need an executable stack to get control of execution, you only need to be able to change the instruction pointer, which is stored on the stack (as data). -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Debian Stable server hacked
Hi, Last sunday, August 3rd 2003, one of my servers was hacked which i, by coincidence, was able to catch 'in progress'. My loganalyzer showed four Did not receive identification string from w.x.y.z logentries from sshd. This happens all the time and i certainly don't check all of them out, but i happen to do so this time. I noticed suspicious network connections with netstat[1]. Shortly thereafter i noticed i had two init processes and multiple syslogd processes. I killed the syslogd processes immediately, as the networktraffic appeared to be IRC-traffic. Then i practically sealed the machine from outside with my firewall, allowing me to do some further research. I found the following: - The extra init process was somehow spawned, but the originally binary seems to have been deleted [2]. - All base system programs where ok, including init and syslogd. Md5s matched. - in / there was rpm-4.0.4.i386.tar.gz. I found that the content was installed. It matches the archive on ftp.rpm.org (md5) - I didn't find any other out-of-the-ordinary files - chkrootkit didn't find anything but the extra init proces running. I'm puzzled about how they managed to get those processes running (as root). There are no local accounts, other than some accounts for the sysadmins. Does anyone have any idea how they might have done this? Anyone seen similar hacks recently? I'd sure like to solve this problem, but at this moment i wouldn't know how, so suggestions are more than welcome. Unfortunately i don't have the resources to get an IDS system up and running... regards and tia, Thijs Welman Delft University of Technology the Netherlands - [0] My server is running Debian stable with: - linux-2.4.21-ac4 custom compiled kernel without LKM-support - sshd - apache - apache-ssl - php4 - smbd/nmbd (firewalled at the university network border) - postfix (not accessible from outside) - bind9 (not accessible from outside) - mysql (firewalled) - proftpd (firewalled) - snmpd (firewalled) - amanda-client from inetd (firewalled) All packages are unmodified releases from Debian stable and, yes, i do update packes from security.debian.org as soon as there are any updates. :) [1] netstat -anp at that time: tcp 00 MYIP:36789 IP#1:21ESTABLISHED 12642/wget tcp 14480 MYIP:36790 IP#1:20ESTABLISHED 12642/wget tcp 00 MYIP:44367 IP#2:60666 ESTABLISHED 10051/syslogd tcp 00 MYIP:33397 IP#2:60666 ESTABLISHED 10051/syslogd tcp 0 80 MYIP:53731 IP#3:59780 ESTABLISHED 10764/init Note: i found out 'init' and 'syslogd' where 'extra' processes. My normal init and syslogd were running normally (seemed untouched) [2] lsof output: init 1 root cwdDIR3,34096 2 / init 1 root rtdDIR3,34096 2 / init 1 root txtREG3,3 27844 312195 /sbin/init init 1 root memREG3,3 90210 179291 /lib/ld-2.2.5.so init 1 root memREG3,3 1153784 179294 /lib/libc-2.2.5.so init 1 root 10u FIFO3,3 49116 /dev/initctl init 9 root cwdDIR3,34096 2 / init 9 root rtdDIR3,34096 2 / init 9 root txtREG3,3 29304 312205 /sbin/init (deleted) init 9 root0u CHR1,3 49079 /dev/null init 9 root1u CHR1,3 49079 /dev/null init 9 root2u CHR1,3 49079 /dev/null init 9 root3u CHR1,2 49078 /dev/kmem init 9 root4u sock0,0 19 can't identify protocol -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 13, 2003 at 07:08:59PM -0400, Colin Walters wrote: But Linux capabilities are so weak. They won't protect an apache master process that runs as root from scribbling over /etc/passwd and giving an attacker a new uid 0 shell account, for example. At that point it's really game over. The attacker then logs in, and has all the capabilities. From there, they have access to /dev/mem, where they can runtime patch the kernel and kill off grsecurity or do whatever else they want. Well capabilities are only one of the things that grsec implements. You can also restrict a process to access various parts of the filesystem. There's no reason /usr/sbin/apache should have write access to /etc, so you just don't allow it. You can also place restrictions based on resources (cpu, memory, etc.) and IP addresses as well. BTW, grsec (well gradm actually) has an intelligent self-learning mode that can help you to give a process only the minimum amount of access necessary to operate in. That way, even a novice sysadmin should be able to benefit from a higher level of security. And even for experienced sysadmins, it makes the process of setting up ACLs much less error-prone. Anyway, since grsec uses PaX, it's very unlikely that anyone will take control of apache through a buffer overflow. ;-) From what I understand it is often possible to evade these kinds of restrictions. It actually does a very good job of stopping any kind of stack-smashing attack dead in its tracks (both the stack and heap are marked as non-executable). That takes care of most vulnerabilities, both known and unknown. Anyways, I just wanted to point out that while grsecurity probably helps somewhat, it provides significantly less security than a system like SELinux. Its sole advantage as far as I can tell is that it's somewhat easier to set up. Why even jump to conclusions like this when you admitted a few messages back that you don't know much about grsec? ;-( -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 06, 2003 at 04:01:39PM +0200, Thijs Welman wrote: All packages are unmodified releases from Debian stable and, yes, i do update packes from security.debian.org as soon as there are any updates. :) If you don't also subscribe to debian-security-announce, then you are missing important things like kernel updates. There are several local root exploits in the stock woody kernel which have been fixed by security updates that would not be installed automatically. You cannot rely on apt alone to secure your system. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, 06 Aug 2003 16:01:39 +0200, Thijs Welman [EMAIL PROTECTED] wrote: My loganalyzer showed four Did not receive identification string from w.x.y.z logentries from sshd. This happens all the time and i certainly don't check all of them out, but i happen to do so this time. That's probably people testing to see if port 22 is open. I'm puzzled about how they managed to get those processes running (as root). There are no local accounts, other than some accounts for the sysadmins. Does anyone have any idea how they might have done this? Maybe they brute forced the root password ? Do you have PermitRootLogin yes in sshd_config ? I'd set up ssh to do protocol 2 only, no root logins, and no passwords/ public keys only if possible. You say that you have apache and php4 installed. Are you running any php applications that may have been compromised ? Although I'd expect those to leave the attacker with access to www-data rather than root. Alan. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Thu, Aug 07, 2003 at 07:03:13PM +0200, Thijs Welman wrote: Hi, Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? Generally if there is a kernel exploit, it is only used to get root from some other account. The way they get in is though some server app with a hole in it (known or not known). For over 2 years I've been running stable debian with Debian kernel without any problems, well, until someone broke in. So, now I don't run a Debian kernel at all - only a monolithic (no modules) kernel with grsecurity.net patches. Then I set up the ACL system (more or less) so that all of the services that can be used to break into the system are quite useless for the attacker. For example, apache can only execute from paths that it cannot write to. Heck, same for root but apache can't even see /bin, etc.. The only problem was with SSH since if that is compromised, you get root. So I would suggest to only allow selected IPs to access SSH to provent someone from the other side to loggin in. ACLs are a bitch to set up, but then even if an attacker manages to break though an app into into your box, they will not be allowed to do anything :) Well, at least with grsecurity it should be more difficult to compromise a box by a few orders of magnitude.. - Adam PS. Needless to say, I would recommend grsecurity for server machines :) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Thu, 7 Aug 2003, Thijs Welman wrote: Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? -- Thijs Since 7 years, I always use custom kernels, and I never had problems (bugs nor exploits). It's run very well and smoothly :) E. -- Eric LeBlanc [EMAIL PROTECTED] -- UNIX is user friendly. It's just selective about who its friends are. == -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, 2003-08-13 at 00:20, Adam Majer wrote: So, now I don't run a Debian kernel at all - only a monolithic (no modules) kernel with grsecurity.net patches. Then I set up the ACL system (more or less) so that all of the services that can be used to break into the system are quite useless for the attacker. For example, apache can only execute from paths that it cannot write to. Heck, same for root but apache can't even see /bin, etc.. I admit to not having investigated grsecurity very much. But one question that comes up occasionally on the SELinux list and on IRC is How do I hide files/directories from a process, or prevent it from executing them?. The answer for the former is that with SELinux you can't reliably hide things, and although you can prevent execution of normal binaries such as stuff in /bin, it's not widely used. For example, regular users have permission to execute /bin/mount. Why? Because SELinux doesn't solely associate security with executable pathnames. If someone takes over control of the apache process via a buffer overflow or whatever, they don't need /bin/ls to list a directory; they can just as easily use the opendir/readdir/stat system calls. Likewise, they don't need /bin/mount to mount filesystems; they can just as easily use the mount syscalls. So the whole grsecurity ACL system seems very weak in that respect. Let me give an example of how SELinux protects my machine (verbum.org). My blog is a Python script (pyblosxom) which runs in a domain called httpd_user_script_t. This domain can execute programs such as /bin/sh (I think pyblosxom might use system() in a place or two). Suppose an attacker somehow got pyblosxom to execute arbitrary commands to /bin/sh. With SELinux, that doesn't help an attacker at all, because the new /bin/sh process gets the same security context as the CGI script, so it doesn't have any more privileges. The security is at the level of the system calls it can make, not what binaries it can execute. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 13, 2003 at 04:02:41PM -0400, Colin Walters wrote: Why? Because SELinux doesn't solely associate security with executable pathnames. If someone takes over control of the apache process via a buffer overflow or whatever, they don't need /bin/ls to list a directory; they can just as easily use the opendir/readdir/stat system calls. Likewise, they don't need /bin/mount to mount filesystems; they can just as easily use the mount syscalls. So the whole grsecurity ACL system seems very weak in that respect. grsec handles this by allowing you to restrict Linux capabilities for a process. For example, there's no reason /usr/sbin/apache should have access to CAP_SYS_ADMIN (allows mount/umount, amongst other things) or CAP_SYS_PTRACE (run ptrace) or many others. Anyway, since grsec uses PaX, it's very unlikely that anyone will take control of apache through a buffer overflow. ;-) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, 2003-08-13 at 18:39, valerian wrote: grsec handles this by allowing you to restrict Linux capabilities for a process. For example, there's no reason /usr/sbin/apache should have access to CAP_SYS_ADMIN (allows mount/umount, amongst other things) or CAP_SYS_PTRACE (run ptrace) or many others. But Linux capabilities are so weak. They won't protect an apache master process that runs as root from scribbling over /etc/passwd and giving an attacker a new uid 0 shell account, for example. At that point it's really game over. The attacker then logs in, and has all the capabilities. From there, they have access to /dev/mem, where they can runtime patch the kernel and kill off grsecurity or do whatever else they want. Anyway, since grsec uses PaX, it's very unlikely that anyone will take control of apache through a buffer overflow. ;-) From what I understand it is often possible to evade these kinds of restrictions. Anyways, I just wanted to point out that while grsecurity probably helps somewhat, it provides significantly less security than a system like SELinux. Its sole advantage as far as I can tell is that it's somewhat easier to set up. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 13, 2003 at 09:00:51PM -0400, valerian wrote: It actually does a very good job of stopping any kind of stack-smashing attack dead in its tracks (both the stack and heap are marked as non-executable). That takes care of most vulnerabilities, both known and unknown. No, it really doesn't. It might stop some common implementations of exploits, but that's about it. There are many papers available which describe the shortcomings of this kind of prevention. You don't need an executable stack to get control of execution, you only need to be able to change the instruction pointer, which is stored on the stack (as data). -- - mdz
Re: Debian Stable server hacked
On Wed, 2003-08-13 at 00:20, Adam Majer wrote: So, now I don't run a Debian kernel at all - only a monolithic (no modules) kernel This doesn't provide very much security. For example: http://www.phrack.org/show.php?p=58a=7
Re: Debian Stable server hacked
On Thu, Aug 07, 2003 at 07:03:13PM +0200, Thijs Welman wrote: Hi, Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? Generally if there is a kernel exploit, it is only used to get root from some other account. The way they get in is though some server app with a hole in it (known or not known). For over 2 years I've been running stable debian with Debian kernel without any problems, well, until someone broke in. So, now I don't run a Debian kernel at all - only a monolithic (no modules) kernel with grsecurity.net patches. Then I set up the ACL system (more or less) so that all of the services that can be used to break into the system are quite useless for the attacker. For example, apache can only execute from paths that it cannot write to. Heck, same for root but apache can't even see /bin, etc.. The only problem was with SSH since if that is compromised, you get root. So I would suggest to only allow selected IPs to access SSH to provent someone from the other side to loggin in. ACLs are a bitch to set up, but then even if an attacker manages to break though an app into into your box, they will not be allowed to do anything :) Well, at least with grsecurity it should be more difficult to compromise a box by a few orders of magnitude.. - Adam PS. Needless to say, I would recommend grsecurity for server machines :)
Re: Debian Stable server hacked
*** REPLY SEPARATOR *** On 12.08.2003 at 23:20 Adam Majer wrote: On Thu, Aug 07, 2003 at 07:03:13PM +0200, Thijs Welman wrote: Hi, Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? Generally if there is a kernel exploit, it is only used to get root from some other account. The way they get in is though some server app with a hole in it (known or not known). snip This is why my personal favourite it the former trusted debian project, now kown as http://www.adamantix.org. Take a look at their site, they offer RSBAC, PaX, all the goodies for the Kernel AND: They recompile all packages to be buffer overflow proof and as secure as possible. Mixing with standard debian packages is not recommended of course, but so far I haven't encountered any problems. Nearly everything is there if You don't require X anyway. regards Martin
Re: Debian Stable server hacked
On Wed, 2003-08-13 at 00:20, Adam Majer wrote: So, now I don't run a Debian kernel at all - only a monolithic (no modules) kernel with grsecurity.net patches. Then I set up the ACL system (more or less) so that all of the services that can be used to break into the system are quite useless for the attacker. For example, apache can only execute from paths that it cannot write to. Heck, same for root but apache can't even see /bin, etc.. I admit to not having investigated grsecurity very much. But one question that comes up occasionally on the SELinux list and on IRC is How do I hide files/directories from a process, or prevent it from executing them?. The answer for the former is that with SELinux you can't reliably hide things, and although you can prevent execution of normal binaries such as stuff in /bin, it's not widely used. For example, regular users have permission to execute /bin/mount. Why? Because SELinux doesn't solely associate security with executable pathnames. If someone takes over control of the apache process via a buffer overflow or whatever, they don't need /bin/ls to list a directory; they can just as easily use the opendir/readdir/stat system calls. Likewise, they don't need /bin/mount to mount filesystems; they can just as easily use the mount syscalls. So the whole grsecurity ACL system seems very weak in that respect. Let me give an example of how SELinux protects my machine (verbum.org). My blog is a Python script (pyblosxom) which runs in a domain called httpd_user_script_t. This domain can execute programs such as /bin/sh (I think pyblosxom might use system() in a place or two). Suppose an attacker somehow got pyblosxom to execute arbitrary commands to /bin/sh. With SELinux, that doesn't help an attacker at all, because the new /bin/sh process gets the same security context as the CGI script, so it doesn't have any more privileges. The security is at the level of the system calls it can make, not what binaries it can execute.
Re: Debian Stable server hacked
On Wed, Aug 13, 2003 at 04:02:41PM -0400, Colin Walters wrote: Why? Because SELinux doesn't solely associate security with executable pathnames. If someone takes over control of the apache process via a buffer overflow or whatever, they don't need /bin/ls to list a directory; they can just as easily use the opendir/readdir/stat system calls. Likewise, they don't need /bin/mount to mount filesystems; they can just as easily use the mount syscalls. So the whole grsecurity ACL system seems very weak in that respect. grsec handles this by allowing you to restrict Linux capabilities for a process. For example, there's no reason /usr/sbin/apache should have access to CAP_SYS_ADMIN (allows mount/umount, amongst other things) or CAP_SYS_PTRACE (run ptrace) or many others. Anyway, since grsec uses PaX, it's very unlikely that anyone will take control of apache through a buffer overflow. ;-)
Re: Debian Stable server hacked
On Wed, 2003-08-13 at 18:39, valerian wrote: grsec handles this by allowing you to restrict Linux capabilities for a process. For example, there's no reason /usr/sbin/apache should have access to CAP_SYS_ADMIN (allows mount/umount, amongst other things) or CAP_SYS_PTRACE (run ptrace) or many others. But Linux capabilities are so weak. They won't protect an apache master process that runs as root from scribbling over /etc/passwd and giving an attacker a new uid 0 shell account, for example. At that point it's really game over. The attacker then logs in, and has all the capabilities. From there, they have access to /dev/mem, where they can runtime patch the kernel and kill off grsecurity or do whatever else they want. Anyway, since grsec uses PaX, it's very unlikely that anyone will take control of apache through a buffer overflow. ;-) From what I understand it is often possible to evade these kinds of restrictions. Anyways, I just wanted to point out that while grsecurity probably helps somewhat, it provides significantly less security than a system like SELinux. Its sole advantage as far as I can tell is that it's somewhat easier to set up.
Re: Debian Stable server hacked
On Wed, Aug 13, 2003 at 07:08:59PM -0400, Colin Walters wrote: But Linux capabilities are so weak. They won't protect an apache master process that runs as root from scribbling over /etc/passwd and giving an attacker a new uid 0 shell account, for example. At that point it's really game over. The attacker then logs in, and has all the capabilities. From there, they have access to /dev/mem, where they can runtime patch the kernel and kill off grsecurity or do whatever else they want. Well capabilities are only one of the things that grsec implements. You can also restrict a process to access various parts of the filesystem. There's no reason /usr/sbin/apache should have write access to /etc, so you just don't allow it. You can also place restrictions based on resources (cpu, memory, etc.) and IP addresses as well. BTW, grsec (well gradm actually) has an intelligent self-learning mode that can help you to give a process only the minimum amount of access necessary to operate in. That way, even a novice sysadmin should be able to benefit from a higher level of security. And even for experienced sysadmins, it makes the process of setting up ACLs much less error-prone. Anyway, since grsec uses PaX, it's very unlikely that anyone will take control of apache through a buffer overflow. ;-) From what I understand it is often possible to evade these kinds of restrictions. It actually does a very good job of stopping any kind of stack-smashing attack dead in its tracks (both the stack and heap are marked as non-executable). That takes care of most vulnerabilities, both known and unknown. Anyways, I just wanted to point out that while grsecurity probably helps somewhat, it provides significantly less security than a system like SELinux. Its sole advantage as far as I can tell is that it's somewhat easier to set up. Why even jump to conclusions like this when you admitted a few messages back that you don't know much about grsec? ;-(
Re: Debian Stable server hacked
On Wed, Aug 06, 2003 at 05:56:47PM +0200, Thijs Welman wrote: Alan James wrote: Maybe they brute forced the root password ? Do you have PermitRootLogin yes in sshd_config ? No, i didn't at that moment. But there's no sign of an succesfull root login. Not in ps aux, not in netstat and no ssh traffic other than my own session in tcpdump. I guess a brute-force would show up in the ssh logfiles. Only thing there is four times Did not receive identification string. sshd logs IP addresses of connections. Was the IP address for those did not receive id connections inside your site, or does it belong to an ISP somewhere, or what? If it's a local address, and not a computer lab, that might give you some clues about whose door to knock on... -- #define X(x,y) x##y Peter Cordes ; e-mail: X([EMAIL PROTECTED] , des.ca) The gods confound the man who first found out how to distinguish the hours! Confound him, too, who in this place set up a sundial, to cut and hack my day so wretchedly into small pieces! -- Plautus, 200 BC pgp0.pgp Description: PGP signature
Re: Debian Stable server hacked
On Thu, Aug 07, 2003 at 01:27:20PM -0400, Eric LeBlanc wrote: Since 7 years, I always use custom kernels, and I never had problems (bugs nor exploits). In 7 years, you've never encountered a bug in the kernel? You are fortunate indeed. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, 06 Aug 2003 17:50:06 +0200, Alan James wrote: You say that you have apache and php4 installed. Are you running any php applications that may have been compromised ? Although I'd expect those to leave the attacker with access to www-data rather than root. Maybe this has been combined with a local root exploit. Alan. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
Thanx for the replies so far. Christian Hammers wrote: Try nmap to see which services are reachable from the network. Port State Service 22/tcp openssh 80/tcp openhttp 443/tcpopenhttps from within the campus network adds: Port State Service 21/tcp openftp 139/tcpopennetbios-ssn Rich Puhek wrote: NOTE: Ok, firewalled at the network border, but could poorly-secured internal windows machines have been used as a springboard for an attack? The same goes for the below services, are you sure that all the machines and people on the same side of the firewall are completely trustworthy? This is a big hole if you're only firewalling at the border of your campus network, and have a wide variety of machines out there... It's likely that there are numerous compromised systems wihtin the campus, unfortunately. They could have used one of those, that's possible. That means they must have exploited sshd, apache, apache-ssl, proftpd or samba. bind9 is open to a local 172.20-network (student housing), so is also candidate... Can't rule it out, but i can't imagine i would be the only one having problems... mysql is only open to three of my other servers. snmpd is only open to my monitoring server Was anyone else logged in at the time? Perhaps one of your admins had a weak or compromised password? Nope. No one was logged in at that time. The few logins in the logfile are accounted for. Alan James wrote: Maybe they brute forced the root password ? Do you have PermitRootLogin yes in sshd_config ? No, i didn't at that moment. But there's no sign of an succesfull root login. Not in ps aux, not in netstat and no ssh traffic other than my own session in tcpdump. I guess a brute-force would show up in the ssh logfiles. Only thing there is four times Did not receive identification string. You say that you have apache and php4 installed. Are you running any php applications that may have been compromised ? Although I'd expect those to leave the attacker with access to www-data rather than root. Thought of that myself. Checked the apache logfiles and went through the scripts... i don't have any 'candidates' besides Horde-2.1/Imp-3.1 and squirrelmail-1.4.0. But then there's still the www-data - root question... regards, Thijs Welman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
Hi, Matt Zimmerman wrote: If you don't also subscribe to debian-security-announce, then you are missing important things like kernel updates. There are several local root exploits in the stock woody kernel which have been fixed by security updates that would not be installed automatically. You cannot rely on apt alone to secure your system. Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? -- Thijs -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Thu, Aug 07, 2003 at 07:03:13PM +0200, Thijs Welman wrote: Matt Zimmerman wrote: If you don't also subscribe to debian-security-announce, then you are missing important things like kernel updates. There are several local root exploits in the stock woody kernel which have been fixed by security updates that would not be installed automatically. You cannot rely on apt alone to secure your system. Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? If you build your own kernels, you are on your own as far as security. You need to keep track of all of the vulnerabilities and whether they affect what you're running, and what version you need to update to in order to get the fixes. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Stable server hacked
On Wed, Aug 06, 2003 at 04:01:39PM +0200, Thijs Welman wrote: All packages are unmodified releases from Debian stable and, yes, i do update packes from security.debian.org as soon as there are any updates. :) If you don't also subscribe to debian-security-announce, then you are missing important things like kernel updates. There are several local root exploits in the stock woody kernel which have been fixed by security updates that would not be installed automatically. You cannot rely on apt alone to secure your system. -- - mdz
Re: Debian Stable server hacked
Hi, Matt Zimmerman wrote: If you don't also subscribe to debian-security-announce, then you are missing important things like kernel updates. There are several local root exploits in the stock woody kernel which have been fixed by security updates that would not be installed automatically. You cannot rely on apt alone to secure your system. Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? -- Thijs
Re: Debian Stable server hacked
On Thu, 7 Aug 2003, Thijs Welman wrote: Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? -- Thijs Since 7 years, I always use custom kernels, and I never had problems (bugs nor exploits). It's run very well and smoothly :) E. -- Eric LeBlanc [EMAIL PROTECTED] -- UNIX is user friendly. It's just selective about who its friends are. ==
Re: Debian Stable server hacked
On Thu, Aug 07, 2003 at 07:03:13PM +0200, Thijs Welman wrote: Matt Zimmerman wrote: If you don't also subscribe to debian-security-announce, then you are missing important things like kernel updates. There are several local root exploits in the stock woody kernel which have been fixed by security updates that would not be installed automatically. You cannot rely on apt alone to secure your system. Thanks. I forgot to mantion that i am subscribed to debian-security-announce as well (ofcourse ;)). As far as the kernel updates are concerned: i use my own kernel. At this moment that's 2.4.21 with Alan Cox' patches (ac4). Could be there's an exploit in that kernelversion. Maybe i should consider to go back to a debian-packagekernel... Anyone any comment on or experience with debian vs custom kernels? If you build your own kernels, you are on your own as far as security. You need to keep track of all of the vulnerabilities and whether they affect what you're running, and what version you need to update to in order to get the fixes. -- - mdz
Re: Debian Stable server hacked
On Thu, Aug 07, 2003 at 01:27:20PM -0400, Eric LeBlanc wrote: Since 7 years, I always use custom kernels, and I never had problems (bugs nor exploits). In 7 years, you've never encountered a bug in the kernel? You are fortunate indeed. -- - mdz
Re: Debian Stable server hacked
On Thu, 07 Aug 2003 03:00:12 +0200, Peter Cordes wrote: sshd logs IP addresses of connections. Was the IP address for those did not receive id connections inside your site, or does it belong to an ISP somewhere, or what? If it's a local address, and not a computer lab, that might give you some clues about whose door to knock on... A professional cracker would have cleaned the sshd logs. So you can't really trust this logfile.
Re: Debian Stable server hacked
On Wed, 06 Aug 2003 17:50:06 +0200, Alan James wrote: You say that you have apache and php4 installed. Are you running any php applications that may have been compromised ? Although I'd expect those to leave the attacker with access to www-data rather than root. Maybe this has been combined with a local root exploit. Alan.
Re: Debian Stable server hacked
Hi, maybe a legitimate user account combined with a local root exploit have been used to crack the server. Does this server has any legitimate user accounts? Are you sure you trust this users? Are you sure they (or you) don't write their passwords on a piece of paper? Who has local access to the server (unprotected LILO/Grub, booting from CDROM (KNOPPIX), mount the hd on another system)? Even if it might be manipulated, you should check the uptime of the system.
Re: Debian Stable server hacked
- Original Message - From: Thijs Welman [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, August 06, 2003 5:56 PM Subject: Re: Debian Stable server hacked Thanx for the replies so far. [...] Thought of that myself. Checked the apache logfiles and went through the scripts... i don't have any 'candidates' besides Horde-2.1/Imp-3.1 and squirrelmail-1.4.0. But then there's still the www-data - root question... It is possible to write harmful php code which executes code on your server, and use that to trigger a local root exploit. I've seen one of those attempts one of my webservers, which tried to trigger a kernel exploit. Luckily we upgraded that kernel some days before the attempt. Regards, Teun -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Debian Stable server hacked
Hi, Last sunday, August 3rd 2003, one of my servers was hacked which i, by coincidence, was able to catch 'in progress'. My loganalyzer showed four Did not receive identification string from w.x.y.z logentries from sshd. This happens all the time and i certainly don't check all of them out, but i happen to do so this time. I noticed suspicious network connections with netstat[1]. Shortly thereafter i noticed i had two init processes and multiple syslogd processes. I killed the syslogd processes immediately, as the networktraffic appeared to be IRC-traffic. Then i practically sealed the machine from outside with my firewall, allowing me to do some further research. I found the following: - The extra init process was somehow spawned, but the originally binary seems to have been deleted [2]. - All base system programs where ok, including init and syslogd. Md5s matched. - in / there was rpm-4.0.4.i386.tar.gz. I found that the content was installed. It matches the archive on ftp.rpm.org (md5) - I didn't find any other out-of-the-ordinary files - chkrootkit didn't find anything but the extra init proces running. I'm puzzled about how they managed to get those processes running (as root). There are no local accounts, other than some accounts for the sysadmins. Does anyone have any idea how they might have done this? Anyone seen similar hacks recently? I'd sure like to solve this problem, but at this moment i wouldn't know how, so suggestions are more than welcome. Unfortunately i don't have the resources to get an IDS system up and running... regards and tia, Thijs Welman Delft University of Technology the Netherlands - [0] My server is running Debian stable with: - linux-2.4.21-ac4 custom compiled kernel without LKM-support - sshd - apache - apache-ssl - php4 - smbd/nmbd (firewalled at the university network border) - postfix (not accessible from outside) - bind9 (not accessible from outside) - mysql (firewalled) - proftpd (firewalled) - snmpd (firewalled) - amanda-client from inetd (firewalled) All packages are unmodified releases from Debian stable and, yes, i do update packes from security.debian.org as soon as there are any updates. :) [1] netstat -anp at that time: tcp 00 MYIP:36789 IP#1:21ESTABLISHED 12642/wget tcp 14480 MYIP:36790 IP#1:20ESTABLISHED 12642/wget tcp 00 MYIP:44367 IP#2:60666 ESTABLISHED 10051/syslogd tcp 00 MYIP:33397 IP#2:60666 ESTABLISHED 10051/syslogd tcp 0 80 MYIP:53731 IP#3:59780 ESTABLISHED 10764/init Note: i found out 'init' and 'syslogd' where 'extra' processes. My normal init and syslogd were running normally (seemed untouched) [2] lsof output: init 1 root cwdDIR3,34096 2 / init 1 root rtdDIR3,34096 2 / init 1 root txtREG3,3 27844 312195 /sbin/init init 1 root memREG3,3 90210 179291 /lib/ld-2.2.5.so init 1 root memREG3,3 1153784 179294 /lib/libc-2.2.5.so init 1 root 10u FIFO3,3 49116 /dev/initctl init 9 root cwdDIR3,34096 2 / init 9 root rtdDIR3,34096 2 / init 9 root txtREG3,3 29304 312205 /sbin/init (deleted) init 9 root0u CHR1,3 49079 /dev/null init 9 root1u CHR1,3 49079 /dev/null init 9 root2u CHR1,3 49079 /dev/null init 9 root3u CHR1,2 49078 /dev/kmem init 9 root4u sock0,0 19 can't identify protocol
Re: [d-security] Debian Stable server hacked
Hello On Wed, Aug 06, 2003 at 04:01:39PM +0200, Thijs Welman wrote: I'm puzzled about how they managed to get those processes running (as root). There are no local accounts, other than some accounts for the sysadmins. Does anyone have any idea how they might have done this? Most times, servers are not cracked by somebody logging in and get root permissions somehow but by somebody who convinces a running network daemon like a web, database or mail server via means of buffer overflows etc to execute arbitrary code instructions. This code will then e.g. write a shell script and executes it or spanws a shell. You will never see an atacker in your last log :-) Try nmap to see which services are reachable from the network. bye, -christian-
Re: Debian Stable server hacked
A few thoughts on potenital problems: Thijs Welman wrote: Unfortunately i don't have the resources to get an IDS system up and running... A bare-bones IDS isn't all thet extreme to build, especially if you are only interested in a single network. Debian stable + snort source package from unstable might be your best bet... regards and tia, Thijs Welman Delft University of Technology the Netherlands - [0] My server is running Debian stable with: - linux-2.4.21-ac4 custom compiled kernel without LKM-support - sshd - apache - apache-ssl - php4 - smbd/nmbd (firewalled at the university network border) NOTE: Ok, firewalled at the network border, but could poorly-secured internal windows machines have been used as a springboard for an attack? The same goes for the below services, are you sure that all the machines and people on the same side of the firewall are completely trustworthy? This is a big hole if you're only firewalling at the border of your campus network, and have a wide variety of machines out there... - postfix (not accessible from outside) - bind9 (not accessible from outside) - mysql (firewalled) - proftpd (firewalled) - snmpd (firewalled) - amanda-client from inetd (firewalled) All packages are unmodified releases from Debian stable and, yes, i do update packes from security.debian.org as soon as there are any updates. :) Was anyone else logged in at the time? Perhaps one of your admins had a weak or compromised password? --Rich _ Rich Puhek ETN Systems Inc. 2125 1st Ave East Hibbing MN 55746 tel: 218.262.1130 email: [EMAIL PROTECTED] _
Re: Debian Stable server hacked
On Wed, 06 Aug 2003 16:01:39 +0200, Thijs Welman [EMAIL PROTECTED] wrote: My loganalyzer showed four Did not receive identification string from w.x.y.z logentries from sshd. This happens all the time and i certainly don't check all of them out, but i happen to do so this time. That's probably people testing to see if port 22 is open. I'm puzzled about how they managed to get those processes running (as root). There are no local accounts, other than some accounts for the sysadmins. Does anyone have any idea how they might have done this? Maybe they brute forced the root password ? Do you have PermitRootLogin yes in sshd_config ? I'd set up ssh to do protocol 2 only, no root logins, and no passwords/ public keys only if possible. You say that you have apache and php4 installed. Are you running any php applications that may have been compromised ? Although I'd expect those to leave the attacker with access to www-data rather than root. Alan.
Re: Debian Stable server hacked
Hello, Was anyone else logged in at the time? Perhaps one of your admins had a weak or compromised password? Install johntheripper if you want to check for weak passwords :D a great program! Hobbs. FOR ALL YOUR UNIX/LINUX QUESTIONS, visit: http://unixforum.co.uk -- _-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_ || | Richard Hobbs[EMAIL PROTECTED]http://mongeese.co.uk | | http://unixforum.co.uk | || | Registered Linux User: 313906 (http://counter.li.org) | || | There's only one way of life, and that's your own| | The Levellers | '`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-'`-_-' __ Send all your jokes to : [EMAIL PROTECTED] !! To subscribe, email: [EMAIL PROTECTED]
Re: Debian Stable server hacked
Thanx for the replies so far. Christian Hammers wrote: Try nmap to see which services are reachable from the network. Port State Service 22/tcp openssh 80/tcp openhttp 443/tcpopenhttps from within the campus network adds: Port State Service 21/tcp openftp 139/tcpopennetbios-ssn Rich Puhek wrote: NOTE: Ok, firewalled at the network border, but could poorly-secured internal windows machines have been used as a springboard for an attack? The same goes for the below services, are you sure that all the machines and people on the same side of the firewall are completely trustworthy? This is a big hole if you're only firewalling at the border of your campus network, and have a wide variety of machines out there... It's likely that there are numerous compromised systems wihtin the campus, unfortunately. They could have used one of those, that's possible. That means they must have exploited sshd, apache, apache-ssl, proftpd or samba. bind9 is open to a local 172.20-network (student housing), so is also candidate... Can't rule it out, but i can't imagine i would be the only one having problems... mysql is only open to three of my other servers. snmpd is only open to my monitoring server Was anyone else logged in at the time? Perhaps one of your admins had a weak or compromised password? Nope. No one was logged in at that time. The few logins in the logfile are accounted for. Alan James wrote: Maybe they brute forced the root password ? Do you have PermitRootLogin yes in sshd_config ? No, i didn't at that moment. But there's no sign of an succesfull root login. Not in ps aux, not in netstat and no ssh traffic other than my own session in tcpdump. I guess a brute-force would show up in the ssh logfiles. Only thing there is four times Did not receive identification string. You say that you have apache and php4 installed. Are you running any php applications that may have been compromised ? Although I'd expect those to leave the attacker with access to www-data rather than root. Thought of that myself. Checked the apache logfiles and went through the scripts... i don't have any 'candidates' besides Horde-2.1/Imp-3.1 and squirrelmail-1.4.0. But then there's still the www-data - root question... regards, Thijs Welman
Re: Debian Stable server hacked
- Original Message - From: Thijs Welman [EMAIL PROTECTED] To: debian-security@lists.debian.org Sent: Wednesday, August 06, 2003 5:56 PM Subject: Re: Debian Stable server hacked Thanx for the replies so far. [...] Thought of that myself. Checked the apache logfiles and went through the scripts... i don't have any 'candidates' besides Horde-2.1/Imp-3.1 and squirrelmail-1.4.0. But then there's still the www-data - root question... It is possible to write harmful php code which executes code on your server, and use that to trigger a local root exploit. I've seen one of those attempts one of my webservers, which tried to trigger a kernel exploit. Luckily we upgraded that kernel some days before the attempt. Regards, Teun
