Tag Archive - theme

Using the Robots Meta Tag to Reduce Duplicate Search Engine Data

29 April 2010 by TQuizzle, No Comments

Chris Pearson is a stud when it comes to all things web, not to mention the guy’s relentless pursuit of wordpress goodness keeps me coming back to his blog again and again.
Yesterday morning, a post from him via DIYThemes.com grabbed my attention. It was all about how to use the Robots Meta Tags option in the very popular theme Thesis. While I dont use Thesis, I still found it extreamly interesting and started to investigate its declaration:

One of the biggest problems with WordPress is the fact that it automatically generates different kinds of archive pages that can be indexed by search engines. From date-based archives (daily, monthly, yearly) to tags to categories, these auto-generated pages all contain duplicate content that doesn’t belong in search engines.

Like many times before, he’s exactly right.

Maybe you don’t think its a big deal. Okay, I get that…but think of this:

The job of search engines is to index your site’s pages and determine what your site is about. If you have 100 unique article pages, then ideally, search engines should only have to crawl 100 pages to index your site fully.

However, if you’re using categories, tags, and date-based archives (and depending on how overboard you tend to go with categorization and tagging), then you’re going to have at least one additional page per category, per tag, per month, etc.

Now, instead of having 100 pages to index, you may have 600 pages. Considering you only have 100 pages of unique content, forcing a search engine to index 600 pages to determine what your site is about just doesn’t make any sense.

Think of it this way: Would you rather read a 100-page book or a 600-page book that tells the exact same story? Further, which one do you think you’ll understand better? Which will hold your focus better?

Wow, now that’s heavy.

Functions Hack

So since its implemented with Thesis so beautifully, and since I love to hack my functions.php file…I figured, I’d do just that. I’d put together a super-tiny function that fixes this, albeit small, issue for my specific customization.

1
2
3
4
5
6
7
8
9
10
11
// META ROBOTS functions.php hack for proper indexing by Search Engines
 
function tq_meta_robots() {
// If you would like to disable the ODP or Yahoo! Directory
// keep the next two lines.
if( is_home() )
echo '<meta name="robots" content="index,follow,archive,noodp,noydir" />';
if( is_archive() || is_tax() ) 
echo '<meta name="robots" content="index,nofollow,noarchive" />';
}
add_action('wp_head','tq_meta_robots');

There…that’ll do it.

The first part checks to see if its the home page (since “noodp” and “noydir” are domain specific settings), then sets the proper search engine robot setting to allow indexing, link following, archiving and prevents search engines from pulling extra information regarding your site from the ODP or DMOZ and the Yahoo! Directory.
The second part checks to see if the page is any archive type page (Category, Tag, Author and Date based pages are all types of Archives), or any taxonomy related pages; and sets the information for those types of pages. This sets the search engine robot setting to allow indexing, tells it that links are to NOT be followed, and to not archive the page.

Done and Done.

BlueQ v1.0.0

25 October 2007 by TQuizzle, 3 Comments

BlueQ v 1.0.1 is finally here!

I’ve been hinting of it’s release for the better part of 4 months now, but it’s finally done.

There may still be a few minor bugs to work out, but overall it should work just as you’d expect.

Check it out on my BlueQ theme page or just skip right to the download.

This version changes quite a bit from the old BlueQ version (pre 1.0.0), so take it for a test drive (just by browsing my site) and let me know how you like it.

Update:

Fixed two little problems and added another feature to sidebar.php. Details are in the changelog on the theme page.