14 Jul 08 Analysing online competitive intelligence

Many people wonder about how accurate many of the online competitive intelligence tools are since they all pull their data from such an expansive array of sources. Based on my own experience, I’m going to make a quick comparison of the online traffic tracking tools to try and divine exactly where the truth lies - pardon the pun.

For this exercise I’ll do a comparison of some websites that have an amount of traffic that is much smaller than the behemoths like Ebay and Google (but not necessarily small). This will show weaknesses of some tools.

 First, we have Hitwise - an expensive product choice, but the data, in my experience is fantastic - they track traffic from the ISP level and have a great team of analysts and techies to make sure this is accurate. They currently catch about 3 million of the 16 million Australian web surfers, which is great for achieving an accurate sample size.

The data is updated regularly (almost daily), you can see it at a daily granularity and break it down by a host of metrics like visits, page views, time on site….the list goes on - and that is just website traffic measurement. Since we are doing a comparison, that is all I’ll look at, but a lot of the value of Hitwise comes from the rest of the capabilities of the tool.

Now, looking at the same cross section through Compete.com, a free web tool with some paid services, we can see that the trend lines are dramatically different for Ticketek.

Compete get their data from several sources; from what I can tell it mainly comes from their own Compete toolbar. This is used as a statistical sample to generate the numbers of “People”we see here. As you can see, they acknowledge they do not have enough data here for accurate estimates except in the case of mytickets, which they seem to have done a fairly good job on. Hitwise on the other hand opt for accuracy with percentage market shares, rather than absolute numbers.

As a general rule, unless you are looking at sites that have massive amounts of traffic, compete can be OK for assessing relative trends, but for absolute numbers I wouldn’t trust it at all.

Compete also provides some top level keyword analysis which can give a couple of nice hints (I just hope “boners in speedos” is a band.)

Next we have Google trends for website. In Australia, and globally, Google has access to massive amounts of data on user behaviour. From what I can tell, the information for these trends comes from what Google sees through its toolbar users that share information, what it sees from people travelling through search, and what it sees from the users that share their analytics information.

 It seems that Google also produce estimates on actual visitor numbers, rather than go the Hitwise route of reporting market share as the default metric.  It has picked up the Ticketek decline well, but missed the spike in mytickets seen by Compete and Hitwise in January.  This would correlate with New Years gigs and I think there was also some integration with SMH to explain this. There is some top level click stream data (also provided in great depth from Hitwise) here which is great to see at a glance which sites people are moving between.

Finally, we have Alexa,  the pioneer of competitive analysis for the masses of the web. It seems that Alexa disagrees with everybody about inthemix, as they have inthemix doing almost as well as Ticketek. However, Alexa also seems to have missed the spike for mytickets in January. Alexa relies on Alexa toolbar users which has a significant amount of Australian users -my guess is that these are skewed to more web savvy users that are more likely to be interested in inthemix than the other two sites that were developed at later stages.

So, what does this all mean?

 Well, you need to be doing competitive tracking to complement your on site analytics. The tools you choose for this may change depending on your needs - but I do think for data reliability, you can’t go past Google or Hitwise. However, if you want to use Hitwise, it will cost you and you are going to need to find information  to strategically pay that off - it is possible to do this, but you need to find the right person, and be in the right industry to make that work.

In the mean time, I’d say Google is enough for starters, do some crosschecks against other tools for validity and aspire to Hitwise when you find that those trends aren’t enough, and the cost of not knowing is too great.