Re: chkrootkit has me worried!
hi ya kevin On Tue, 29 Nov 2005, kevin bailey wrote: > i have tried out lots of different things on this server and have made the > mistake of leaving unnecessary services running. everybody does that, one forgets to "undo the experiment environment" and restore back to secure mode > in this case i think that webmin was running the miniserv.pl server and this > had a security warning issued for the version i had. any cgi is a bad idea unless you can do sanity checking of what users can feed it to get unauthorized access > also - very luckily - no data on the server has been damaged. - how do you "know" that ?? - can you compare the files against backups on cdrom that shows no modified changes ?? - one doesn't need to check binaries unless you want to check it vs fresh installs - using existing binaries is fine if you have a way to verify its md5 of the binary and libs and directory tree ( i prefer to check everything.. automatically with scripts ) - fresh installs means you have to configure everything again from nothing .. maybe 1hr ..maybe 1 day .. maybe 1 week ( i somtimes spend a week to fix up distros from net/cd installs ) > it seems to > spawn lots of hidden processes and has had to be rebooted to get it under > control again. it is typical for the reboot process ( scripts ) to be modified with trojans and backdoors and other hidden goodies > main points learnt. > > make sure you have snapshot backups going back months. and saved on say dvd or disks on backup servers that is powered off never overwrite backups unless you're willing to forgo checking against that data history dating back that far - it might be hours, days, weeks, months before you find that you've been had .. > regularly run chkrootkit and get the server to email the results to you. i wouldn't place too much confidence on binaries running on suspect hacked boxes, even if its for sanity checking, and first pass notifications of something whacky going on rebooting and running off standalone cd is better, but it means your server/services will be offline for the duration any of these IDS is sorta too late .. telling you're [cr|h]acked i'd tend to spend the time upfront to harden the server as best as possible for the time/budget that they or you/we provide for so we can sleep at nights > if backing up to another server get that server to pull backups out. on my > new machines i was pushing out the backups from the primary server - this > would mean a cracker would then have an easy way in to the backup machine > because i was using authorized_keys so the backup would run in a script. i say never use keys to do backups ... - keys can be stolen ... and if you dont require additional authentication, you're shot - if you can do it, play with backups, the [cr|h]acker and do it too - if you lock things down to one ip# ... its less likely the [cr|h]acker can get that ip# assigned to themself to play with your boxes always push backups, since remote machines should NEVER have access to root-read-only files - push with secure NFS and ssh ... pc1# mount backup_pc:/Backup/PC1 pc1# scp " copy all new files to remote backup_pc " requiring manual authenticaion if you're paranoid pc1# pgp encrypt that new backup data pc1# chmod 400 "backup data" pc1# umount backup_pc:/Backup/PC1 > but mainly only run required services - and check them closely - and don't > rely on your distro to incorporate every single security patch required for > your server. bingo and there's gazillion different solutions .. most works equally well for each user .. untill they outgrow it for any numberof reasons, where being cracked will usually result in some new changes c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: chkrootkit has me worried!
Rick Moen wrote: > > Unsafe data passed to eval(). Sheesh! And awstats is so large, that it would require a lot of effort to do a proper audit of it. Are their any automated tools for auditing perl code? Or I wonder what would happen if you just switced on taint mode? > >>I would agree with that idea. In fact, I've just lodged a bug report >>along those lines. Bug #341308. > > > Thank you, Geoff! No worries. Jonas has already responded to the bug, he sounds in favour of it. I'm sure he'd appreciate patch suggestions on implementing it. -- Geoff Crompton Debian System Administrator Strategic Data +61 3 9340 9000 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: chkrootkit has me worried!
Quoting Geoff Crompton ([EMAIL PROTECTED]):
> The most recent vulnerability that I was aware of in Awstats can still
> work even in static mode. http://www.securityfocus.com/bid/14525. The
> referrer in the log file is not sanity checked.
Hmm. I note: "It should be noted this vulnerability is only possible
if the affected application has at least one URLPlugin enabled."
The iDefense advisory casts light on the problem Perl snippet:
the $url parameter contains unfiltered user-supplied
data that is used in a call to the Perl routine eval() on lines 4841
and 4842 of awstats.pl (version 6.4):
my $function="ShowInfoURL_$pluginname('$url')";
eval("$function");
The malicious referrer value will be included in the referrer
statistics portion of the AWStats report after AWStats has been run
to generate a new report including the tainted data. Once a user
visits the referrer statistics page, the injected perl code will
execute with permissions of the web service.
Unsafe data passed to eval(). Sheesh!
> I would agree with that idea. In fact, I've just lodged a bug report
> along those lines. Bug #341308.
Thank you, Geoff!
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: chkrootkit has me worried!
> So, here's my favourite example of the "bad implementation" problem: > AWstats. It's had a long history of: > > o Someone finds yet another way its stats-generating CGI can be subverted by >sending it aberrant URL information from the public. > o The upstream maintainer issues an update. > o Debian issues a new package. > o An exploit emerges and gets rolled into automated attack tools. > o Lather, rinse, repeat. > > If you look more closely at AWstats, you might start to wonder: "I > guess the author never quite got input validation right. But why > does it have to run as a CGI (awstats.pl) in the first place? Can't it > run as a cronjob, instead, generating the same stats page as static HTML > every hour, instead?" The most recent vulnerability that I was aware of in Awstats can still work even in static mode. http://www.securityfocus.com/bid/14525. The referrer in the log file is not sanity checked. Unfortunately awstats seems to have organically grown as a single perl script. This one script is upto almost 5 lines of code. I've looked a little at the code, and I can't say it is easy to follow. But it does seem to do a good job of generating stats. I just don't feel comfortable trusting it on my servers. > > And you'd be right to wonder. That's a solved problem, thanks to Steve > Kemp over at debian-administration.org: > http://www.debian-administration.org/articles/85 > > I keep meaning to file a very polite bug with Debian maintainer Jonas > Smedegaard, suggesting that static-page mode be the default since > upstream's CGI default is (in my opinion) too risky, but I haven't done > that yet. > I would agree with that idea. In fact, I've just lodged a bug report along those lines. Bug #341308. -- Geoff Crompton Debian System Administrator Strategic Data +61 3 9340 9000 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: chkrootkit has me worried!
On Tuesday 29 November 2005 14.04, kevin bailey wrote: > if backing up to another server get that server to pull backups out. on > my new machines i was pushing out the backups from the primary server - > this would mean a cracker would then have an easy way in to the backup > machine because i was using authorized_keys so the backup would run in a > script. Note that its not a question of push vs. pull but a question of where the authentication happens. In both cases you'll have some means (ssh key, hardcoded password etc.) to access the other machine - the decision should thus not be push vs. pull but to store the authentication information on the side where a compromise is less likely. Then, use tools like rssh to lock down the account used to transfer the back up data. Also allow log in on this account only from a fixed IP address. (Obviously not always possible in the hobbyist scenario when you're backing up your server to your home machine on DSL or so.) cheers -- vbi -- Beware of the FUD - know your enemies. This week * Patent Law, and how it is currently abused. * http://fortytwo.ch/opinion pgp0p59XJGkXF.pgp Description: PGP signature
Re: chkrootkit has me worried!
Quoting kevin bailey ([EMAIL PROTECTED]): > what with it being several different symptoms i tend to think this is not a > false positive. Concur. > cause: > > this is an old server which has been running for 4 years. If such an old server is maintained and administered properly, and if you doesn't get unlucky and suffer compromise because a remote user's login credentials were stolen elsewhere, then host age alone is not a problem. > i have tried out lots of different things on this server and have made the > mistake of leaving unnecessary services running. Whoops. That is a risk factor. > in this case i think that webmin was running the miniserv.pl server and this > had a security warning issued for the version i had. Yes. Be aware that many CGI-based services are just plain risky on account of bad implementation (e.g., failure to validate input), and that the security team can't save you from this. Only alert and cautious system administration can. > it doesn't seem to have been fixed in the debian security fixes - i'll > contact debian RE this. So, here's my favourite example of the "bad implementation" problem: AWstats. It's had a long history of: o Someone finds yet another way its stats-generating CGI can be subverted by sending it aberrant URL information from the public. o The upstream maintainer issues an update. o Debian issues a new package. o An exploit emerges and gets rolled into automated attack tools. o Lather, rinse, repeat. If you look more closely at AWstats, you might start to wonder: "I guess the author never quite got input validation right. But why does it have to run as a CGI (awstats.pl) in the first place? Can't it run as a cronjob, instead, generating the same stats page as static HTML every hour, instead?" And you'd be right to wonder. That's a solved problem, thanks to Steve Kemp over at debian-administration.org: http://www.debian-administration.org/articles/85 I keep meaning to file a very polite bug with Debian maintainer Jonas Smedegaard, suggesting that static-page mode be the default since upstream's CGI default is (in my opinion) too risky, but I haven't done that yet. > also - very luckily - no data on the server has been damaged. it seems to > spawn lots of hidden processes and has had to be rebooted to get it under > control again. With respect, you have rather little reason to believe that you yet have control. Since it is highly likely that your site was root-compromised, your best course of action is to rebuild with the same data files but entirely fresh software from trusted media, avoiding direct reuse of any of your existing configuration files or user dotfiles. http://www.cert.org/tech_tips/win-UNIX-system_compromise.html > main points learnt. > > make sure you have snapshot backups going back months. > > regularly run chkrootkit and get the server to email the results to you. chkrootkit is useful (as is rkhunter) as a last-gasp doublecheck to increase your confidence that your other security precautions have worked. It's a "canary". However, it had better not be used in place of those precautions, or you're already in trouble. Let's use an analogy from public health: chkrootkit is the blood test that informs you that you have a case of amoebic dysentary. Your suggestion amounts to "Well, then I mostly need bloodwork done every few months. Never mind that bit about being careful about eating raw shellfish caught near sewage outfalls and eating in restaurants with questionable sanitary practices." The Debian security team is your county's restaurant inspectors. AWstats's default CGI-generated mode is that bucket of raw oysters. > but mainly only run required services - and check them closely - and don't > rely on your distro to incorporate every single security patch required for > your server. Right, and remember that the health inspectors can't guarantee every oyster -- and that fugu from a reputable restaurant can still kill you. (I hope you don't mind if I publish our correspondence in Linux Gazette, http://linuxgazette.net/ .) -- Cheers, Rick Moen "Anger makes dull men witty, but it keeps them poor." [EMAIL PROTECTED] -- Elizabeth Tudor -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: chkrootkit has me worried!
thanks for the replies. what with it being several different symptoms i tend to think this is not a false positive. cause: this is an old server which has been running for 4 years. i have tried out lots of different things on this server and have made the mistake of leaving unnecessary services running. in this case i think that webmin was running the miniserv.pl server and this had a security warning issued for the version i had. it doesn't seem to have been fixed in the debian security fixes - i'll contact debian RE this. or it might have been a weakness in zope. luckily i was halfway through moving everything off this server to a new pair of servers and have been able to move the changeover forwards. also - very luckily - no data on the server has been damaged. it seems to spawn lots of hidden processes and has had to be rebooted to get it under control again. main points learnt. make sure you have snapshot backups going back months. regularly run chkrootkit and get the server to email the results to you. if backing up to another server get that server to pull backups out. on my new machines i was pushing out the backups from the primary server - this would mean a cracker would then have an easy way in to the backup machine because i was using authorized_keys so the backup would run in a script. but mainly only run required services - and check them closely - and don't rely on your distro to incorporate every single security patch required for your server. kev -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: chkrootkit has me worried!
On Tue, Nov 29, 2005 at 04:34:11AM +, kevin bailey wrote: > hi, > > the following output looks like i've been rooted. Yes, it doesn't look like a false positive: > Checking `ls'... INFECTED > Checking `netstat'... INFECTED > Checking `ps'... INFECTED > Checking `top'... INFECTED Nasty. > Searching for suspicious files and dirs, it may take a while... > /usr/lib/zope/lib/python/Products/DCWorkflow/.Xserver-lcd > /usr/lib/zope/lib/python/Products/ZnolkSQLWizard/.selectColumns.dtml.swo > /usr/lib/zope/lib/python/Products/ZnolkSQLWizard/.selectColumns.dtml.swp > /usr/lib/zope/lib/python/SearchIndex/.testinfo Those might be FP. > /usr/lib/nmh/include/lib/.sniffer This one looks nasty. > Searching for anomalies in shell history files... Warning: > `//root/.bash_history' file size is zero Nasty. > Checking `lkm'... You have 107 process hidden for readdir command > You have 113 process hidden for ps command Nasty. > Checking `sniffer'... eth0 is PROMISC You have several processes hidden and what looks like sniffer logs so be careful. Your passwords might be compromised either through a trojaned ssh client if you are using ssh or through the sniffer if you are using clear-text passwords. Sorry, Javierthrough the sniffer if you are using clear-text passwords. signature.asc Description: Digital signature
