Keyword Research and Analysis Strategies (Part 2): Competition Analysis

Keyword research, with subscription based tools like Keyword Discovery or WordTracker usually ends up in a massive list of keywords and for some, the list can number in the thousands. Many of the keywords in the list form the “long tail” of search results, a term coined by Chris Anderson. It refers to the collective searches for “low traffic” searches outweighing the searches done for more popular searches. There will be some keywords that don’t even show up in the keyword research that get traffic in the long tail. If you haven’t read it, it’s a good idea to get your copy. It will certainly give you a whole new insight into online marketing. Understandably, it is difficult for many small or medium-sized businesses to attempt to rank for so many terms initially. We only have to choose a handful of the higher traffic, relevant keywords to start.

In the first article of this keyword research and analysis series, we discussed keyword lists and the smart selection of potential keywords suitable for further SEO analysis. Now that we have built up a massive list of possible keywords (hopefully), how do we whittle it down to the select few to target? What if we have hundreds of potential keywords for different products/services to sift through? Ironically, to select only the most targeted keywords that drive quality traffic to your site, you would have invest in a small pay-per-click search marketing campaign. We would only do this on a small scale and for a short duration of time to determine the true value of this massive list of keywords. You can get immediate feedback from the PPC traffic. Rather than guess which keyword phrases will drive quality traffic, we’ll know for certain which ones truly will from detailed web analytics. If you haven’t already opened up a Google Adwords campaign yet, now is a good time to do so for this test. For this trial, shutting off the content network where ads appear on Google Adsense publisher sites is highly recommended. We are trying to test keywords for traffic quality. Remember, proper testing can only be done on one variable at a time. If you don’t have use any web analytics for your site currently, sign up for a free analytics account with Google Analytics. Google Analytics complements Adwords nicely for tracking so it is an excellent way to get started. Investing a small amount to test the keywords using PPC only serves to improve the probabilities of only optimizing for the best keywords resulting in time and cost savings. The traffic should result in an action you want your visitors to take such as opting in or a purchase.

Once we have identified profitable keywords to optimize for, we are not finished yet. We still have to perform keyword competition analysis on the keywords. Again, what drives the most traffic is not necessarily the best to go after at this point of the site promotion process. If no SEO work has been done previously, it’s usually best to avoid very high traffic key terms because they typically have large amounts of competition. To be certain what competition exists for your search term, search each term in Google with each key phrase wrapped in double quotes (eg. “online dating service”). Obviously, with the large number of keywords, it’s best to keep all our data on an Excel spreadsheet to record all important data found during our keyword research.

Let’s run through an example. When we type in “online dating service” (with the quotes) in Google, notice that it reports that we have “Results 1-10 of about 1,570,000 for …” as the top right corner of your browser window. Hmm… That seems like a large amount of competititors to go up against.

We want to see if there really are over a million competitors for this phrase. Since Google only shows 10 pages of results at a time, we’ll have to adjust our browsing preferences to make our searches faster. First, set browsing preferences in Google to list 100 sites at a time by clicking on preferences near the Google search box at the top. This can be done in any browser. Note: cookies in your browser needs to be enabled. Now instead of showing 10 results per page, you can view 100 results at one time.

We click on the last page of the search engine results pages which should list the 901-1000th result. At this time of writing, when we click on the last page (in this case, 10) we find that we truly only have 818 “real competitors”! Google’s focus is on relevancy and it’s likely that these other pages were not included because Google deems them as unimportant to the searcher because of irrelevant content or duplicate content. These unlisted pages are in therefore in Google’s supplemental index. The supplemental index is different from the “main” search engine index and supplemental results usually show when there aren’t enough results to show or the terms are very obscure.

We now have the number of real competitors for this keyword phrase. We rinse and repeat the process until we have completed competition analysis for all the keywords in the list. Is this process time consuming? Yes! Worth the effort? Definitely, especially if you’re serious about long-term results.

The next step is going to make the difference in your SEO campaign. It is investigating keyword phrase difficulty. From our experience as SEOs, we can understand why many clients want fast results. From a business cashflow perspective, if it takes some time and a significant investment to get the site ranked with a reasonable level of traffic. If a website owner embarks on a campaign to achieve higher rankings for a very difficult search term, they are almost certainly doomed to fail. Sometimes it’s because the SEO techniques weren’t sound. Many times, it’s keyword choice. SEO is an art and takes quite a bit of skill to get it done right. Feedback is difficult to obtain with difficult terms therefore it’s hard to gauge what really is needed to accomplish the task. Because they do not see much progress even in a year, many give up before reaching the finish line.

Now, you can mitigate these risks. By comparing the difficulty level of your search terms on a whole, you’ll increase your chances of ranking higher in the search engines in a shorter period of time and with considerably less resources expended. We will covered how to do this in our next article.

If you want to get keyword research done but don’t want to spend the time to do it, let our SEO experts handle it. You can learn more about our keyword research services here.

Anthony Yap is the principal SEO expert for He delivers higher search engine rankings for his search engine optimization clients, driving quality website traffic that converts. For internet marketing consulting services, contact me.