I used to think that the more pages I have on Google index, the more traffic I am going to get.
But it’s not true. I realized that when I was checking Google indexed pages of RTB for the first time. What I found was embarrassing. A lot of useless pages got indexed by Google, even some of them had no content. This is may be one of the reasons why I got hit by Panda 4.0.
Having these low-quality pages can harm your SEO too. Besides, there is no reason to have these useless pages on Google index. In this post, I will be sharing how to block URLs from crawling and how to remove URLs from Google index.
Before going further, I’d like to share when to remove indexed pages and when not to remove.
When to Remove Indexed Pages?
There could a lot of reasons to remove URLs, but here are some of the most common.
- A page with confidential data got indexed by Google and you want to get that out of the index.
- You have deleted a post long ago, but it’s still showing in the search results.
- Your site got hit by Panda for low-quality contents.
- You don’t want some specific pages (like privacy policy page) to show on search results.
- Google Webmaster Tools notifies about a page violating Google’s guidelines.
When Not to Remove Indexed Pages?
Though you have the power to remove any URLs from Google Search results, there can be some situations when you shouldn’t. Google provides some guidelines about this. Here are the situations.
- Google’s crawlers naturally drop the outdated URLs from the index. So you don’t need to request an urgent update for those old pages.
- You don’t need to remove URLs because of crawl errors. They will drop out naturally over time.
- If you think your site might have a manual action, don’t remove URLs to start with a clean slate. Instead, file a reconsideration request after fixing it up.
- If your site got hacked and some bad URLs got indexed by Google, don’t block the entire site. Just block those bad URLs. But at first, clean up your site and let Google recrawl your site.
- Don’t use URL blocking tool to remove different versions of a page. Instead, use canonicalization of URLs.
How to Check Indexed Pages in Google
Before taking any action, you need to know which pages of your site Google has already indexed. To know this, go to Google. And search for
site:yoursite.com
You’ll see only pages from your site which Google has indexed so far. Now you need to determine which URLs you’ll be blocking/removing.
Here is an example of a useless page for RoadToBlogging.Com.
Make a list of the pages that you want to remove from Google search results.
How to Block URLs from Google’s Crawling
Before removing URLs from the search results, I’d suggest your to block the URLs from Google crawling. By blocking the URLs, you can stop Google from indexing the web page for display in search results. Here are 3 ways to stop Google crawling.
1. Add Noindex metatag
You can tell Googlebot not to index a particular page by adding noindex meta tag in page’s HTML code. When Googlebot finds the noindex meta tag, it will drop the page from Google search results.
Adding noindex meta tag is pretty simple. Just place the following meta tag into <head> section of your page.
<meta name=”robots” content=”noindex”>
If you are using WordPress, you can easily do it by using Yoast plugin. Once you’ve installed the Yoast plugin, go to the page on which you want to add noindex meta tag and edit the page/post. Scroll down to ‘Yoast SEO‘ and click on ‘Advanced’ tab. Select ‘noindex’ from ‘Meta Robots Index’ and update the post/page.
To remove your page faster, use fetch as Google.
2. Use Robots.txt File
Robots.txt file indicates the pages or folders of your site you don’t want crawled by Google bots. To be honest with you, adding a file or page to the “Disallow” list in the Robot.txt won’t remove it from Google index. But it will help Google bots to crawl the right content and index them.
If you haven’t configured Robots.txt file yet, read how to create and configure Robots.txt file.
https://support.google.com/webmasters/answer/6062608?vid=1-635796757441517520-3775289588
3. Delete the Page & Redirect it to new URL
If the page has no value to readers and search engine bots, then just delete the page from your site. But you have to redirect the deleted page’s URL to an existing URL. Whenever Google bot finds that the page is redirected to a new URL, it will drop the old URL and index the new URL.
You can use Redirection plugin to redirect the URL. Once you’ve installed the plugin, go to WordPress Dashboard > Tools > Redirection. Then you’ll get Add new redirection’‘ option to redirect old URL to new URL.
Enter the URL which you want to redirect to the new URL on the Source URL and enter your destination URL on the Target URL. Then click on ‘Add Redirection’ button.
How to Remove URLs Using Google Webmaster Tools
After blocking the URLs, you should wait for few days to let Google removes the pages automatically. If Google doesn’t remove the URLs after a while, use Google Webmaster Tools. Read: How to Verify WordPress blog with Google Webmaster Tools
Now let’s see how to remove URLs by using Google Webmaster Tools. Before going to the tutorial, I’d suggest you to use this tool with great care. Only use this tool when you know what you are doing. Doing it incorrectly can take your site out of the Google index.
Here are the steps.
At first go to your Webmaster Tools dashboard and select the proper site.
From the left navigation Click on “Google Index”. You’ll find four options there. Select “Remove URLs” from there. (Follow the image below)
Now click on ‘Temporarily hide’ and enter the URL that you want to hide. Then click on ‘Continue’.
On the next screen, you’ll see your URL and another field named “Request type”. Here you will find three options. Select the first one which is “Temporarily hide page from search results and remove for cache”. Then Click on “Submit Request”.
Your request to hide the URL has been submitted to Google. You’ll see pending on request Status. When the URL is removed, the status will be changed to ‘Removed’. You can also Cancel the request by clicking on ‘Cancel’ button.
Generally, Google does not take much time to process these requests. My one took less than one day. I submitted some other URLs along with this and all URLs were removed within a couple of hours.
Over To You
Hope this post helps you to remove unwanted pages from Google search results. If you have any question regarding this, feel free to ask us via comment.
Once again, use this tool with great care. It’s always a good idea to let Google removes the pages automatically by blocking URLs.
However, if you found this post useful, help me by sharing this post on Facebook, Twitter or Goole+.
Leave a Reply