Viewing Hidden or Cloaked Contents or Links in Forums

Ever since Google rolling out the Panda update which targets low quality sites, it has caused many webmasters to lose massive traffic and revenue as well. Many has claimed that they’re innocent and should not have been affected by the Panda but I strongly believe that they’ve unknowingly violated some guidelines that caused the penalty. One of the violation that is categorized as high risk is Cloaking. In simple terms, cloaking means showing different content to users than GoogleBot. This can be easily done by checking the Googlebot IP address or user-agent and feeding with different content. Many webmasters does not realize using some forum plugins that forces the users to register or login to view the content can be a cloaking violation.

For example, you searched for a keyword in Google and visited one of the webpage from the results. Then you are presented with annoying messages that restricts you from viewing the content.

Sorry, but you do not have permission to use this feature. If you are not logged in, you may do so using the form below if available.

Guests cannot see links. Registration or Login is required.

Now you have to go through the process of signing up in some forums that you would probably never visit again and you may even lose track of the origin URL at the end. The user will definitely start to blame Google on indexing useless restricted pages but the truth is, the forum has actually implemented some cloaking techniques where different contents are being presented to Googlebot and users. There are proper guidelines provided by Google on the correct implementation method for registration/subscription sites so a webmaster should definitely refer to it if they’re running such service on their website.

As for normal users, there are simple ways to view the cloaked contents or links without signing up or logging in forum. The first method that does not involve installing anything is to view the Google Cache. Click the arrow pointing to the right and then followed by clicking on the Cached hyperlink.

If for some reason you are unable to view the cached version of the webpage by Google, your next attempt is to visit the page using a Googlebot emulator. Smart IT Consulting has created a Spider Spoofer allowing you to view a webpage as Googlebot. All you need to do is enter the URL to the box and click the Submit button. By default the page will open in a new tab but probably blocked by your web browser’s popup blocker so you’ll need to click on the red colored link that says “Click here to view the page as Googlebot-2.1” if nothing is happening.

The links below are cloaked when viewing the webpage directly from web browser.

Links are uncloaked and revealed when viewing as Googlebot.

Finally, if you always end up visiting such websites and finds it troublesome to visit the page from the Spider Spoofer, you can install extensions that changes your web browser’s user-agent string to Googlebot. Here are the most popular extension to spoof your user agent called User Agent Switched for Chrome and Firefox. After installing the extension, click on the User Agent Switcher button and select Googlebot from the list. The change is only temporary and will revert back to the default if Chrome is restarted.

Other good uses of changing your user agent to Googlebot is to see if your website has been infected by the “pharma hack” where extra spam words and links are added to the page when it is visited by Googlebot. You can also switch your user agent to Android, iPhone or iPad to test the mobile version of a website.