Web Analytics Process - Measurement & Optimization

This article is a review of a model proposed by me and Avinash Kaushik in a two-part paper we wrote for the Search Engine Marketing Journal on 2009 and 2010. If you wish to read the original papers, you can download them on pdf: Part I and Part II.

Below I propose a Web Analytics Process framework that shows the steps and the flow that should be present when implementing Web Analytics in organizations.

The objective of Web Analytics is to first and foremost improve the experience of online customers, it is not a technology to produce reports; it is a virtuous cycle for website optimization. The process proposed above will enable companies to create a powerful data-driven culture, to measure customer interaction with the website, to segment visitors and understand how each group behave, to analyze campaign Return On Investment (ROI), and to optimize the website in order to increase profitability.

1. Start with a clear definition of business goals
2. Build a set of Key Performance Indicators to track goal achievement
3. Collect accurate and complete data
4. Analyze Data to extract insights
5. Test Alternatives based on assumptions learned from data analysis.
6. Implement insights based on either data analysis or website testing

Following, I discuss each step in detail.

Defining Website Goals

This is the first step on any website optimization initiative, you must understand your goals in order to improve your website. The answer to the following question is critical in defining a website’s goals: why does your website exist? An anecdote that represents the importance of objectives can be found on Alice's Adventures in Wonderland, when she meets the Cheshire cat in a crossroad:

"Would you tell me, please, which way I ought to go from here?"
"That dependes a good deal on where you want to get to" said the Cat.
"I don't much care where-" said Alice.
"Then it doesn't matter which way you go," said the Cat.
"-so long as I get somewhere," Alice added as an explanation.
"Oh, you're sure to do that," said the Cat, "if you only walk long enough."

Each website will have its own unique objectives. For some, the objective will be to increase pages viewed in order to sell more advertising (media sites); for others, the objective will be to decrease pages viewed because they want their visitors to find answers (support sites). For some, the objective will be to buy as fast as possible (increase revenues); for others, the objective will be to sell only if the product fits the needs of the customer (decrease products returns). But, as Jim Sterne said in his Social Media Metrics book:

"Your focus should always be on either increasing revenue, lowering costs, or improving customer satisfaction. Doing all three would be just fine."

As we can see in the Web Analytics Process proposed above, the objectives are absolutely necessary in order to start the process, only after they are defined we can proceed to build the Key Performance Indicators. It is also very important to constantly revisit the goals in the light of website analyses and optimization to fine tune them.

Building Key Performance Indicators (KPIs)

In order to measure goal achievement, the marketer should create Key Performance Indicators (KPIs) to understand whether the website is going up or down. KPIs must be like a good work of art: it wakes you up. Sometimes it makes you happy and sometimes it makes you sad; but it should never leave you untouched, because if that is the case you are not using the right KPIs.

And good works of art are rare, you have just a few truly touching works of art per museum; and not every work of art touches the same people. The same applies to KPIs, there are just a few truly good KPIs per company, and each person (or hierarchy level) will be interested in a different set of KPIs, the one that relate to their day-to-day activities: upper-management is touched by the overall achievement of the website’s goals; mid-management is touched by campaign and site optimization results; and analysts are touched by every single metric on the world!

Good KPIs should contain four attributes:

1. Un-complex: decisions in companies are made by people in several departments with different backgrounds. If the web analyst is the only person that understands the KPIs, it is unlikely that decision makers across the company will use them.
2. Relevant: as we mention above, each company has its unique objectives, therefore they should also have their own set of KPIs to measure improvement.
3. Timely: great metrics must be provided promptly so that decision makers can make timely decisions. Even excellent KPIs are useless if it takes a month to get information when your industry changes every week.
4. Instantly useful: it is vital to understand quickly what the KPI is, so that one can find the first blush of insights as soon as s/he look at it.

Following the definition of the website objectives and the metrics that will be used to measure them, we will be in a much better condition to collect the data that will be needed.

Collecting Website Data



When a company starts to collect website data (or reviews its data collection), two questions should be asked:

• Is my data accurate? If your data is not accurate, it is like building an empire in the sand, your foundations can be shaken too easily
• Am I collecting all the data that I need? If data is not collected, you will not be able to understand customer behavior properly.

I will not go into data collection methods (Web Logs, JavaScript Tagging, Web Beacons and Packet Sniffing) as this has been extensively documented. You can find an explanation of each of the main four data collection methods currently used on Part I of the article I wrote with Avinash.

Analyzing Data



Following I provide a few ideas that can help on the conversion of data into insights, an essential step when optimizing any website.

• In the same way that a 100m freestyle swimmer won't compare his time to a 100m sprint runner, a website can't compare itself to different websites. The best way to proceed is to trend your metrics over time, to understand if the website is improving or not, like a swimmer is constantly struggling to break his own record.
• There is no absolute truth when it comes to engagement, a high number of pageviews can be a good sign (for a content website), a bad sign (for support websites) or no sign at all (for Flash and AJAX websites). However, time is probably a good indicator of visitor engagement.
• Bounce rates measure the quality of traffic you are attracting, if visitors are not interacting with your site at all, you are doing something wrong: either attracting the wrong audience or not leaving up to you promises.
• Keywords driving traffic to your site tell a lot about your visitor’s intent, the visitors are telling why they are coming to the website, and keywords with high bounce rates show where the intent is not met. It could be that the website is ranked for the wrong keywords. It could be that the pages these visitors are landing on don’t have the right calls to action.
• Internal site search tells you exactly what visitors want from you, so look for which searches provide engaging results (look at 'time on site after search' metric), and which search results are driving them away (look at '% search exits'). Maybe visitors are looking for a product you currently do not sell? Maybe your search is badly configured and they get 0 results for an important product search? Google has proved that search is really important for people!
• Data Visualization is like a car's windshield wiper on the rain, it enables the driver to see more clearly. You can chose from an endless pool of graphs and tools to visualize numbers; if you like to read numbers, good for you, but don't assume everyone does, in the majority of cases people just hate numbers. So use line graphs, pie charts, density reports, heatmaps...

Important to note that data analysis can lead to three different outcomes (as seen in the Web Analytics Process chart above):

1. To a discover of immediate insights for implementation such as website bugs or pages that do not convert for an obvious reason.
2. To a hypotheses regarding a low converting customer touch point - this will lead to a split test.
3. To an understanding of a data collection failure - either important data can be missing or inaccurate.

Testing Alternatives



In the spirit of the African proverb above, it is very unwise to change a website completely without first trying with the tip of your fingers. When we test, we lower the risk of a loss in revenue due to a poor new design and we bring science to the decision making process in the organization. But the most interesting outcome about experimenting is not the final result; it is the learning experience about the customer, a chance to understand what they like and dislike, which ultimately will lead to more or less conversions.

The web analyst must try endlessly and learn to be wrong quickly, learn to test everything and understand that the customer should choose, not the designer or the website manager. Experimenting and testing empowers an idea democracy, meaning that ideas can be created by anyone in the organization, and the customers (the market) will choose the best one; the winner is scientifically clear.

I have described the Website Testing Process before, and the advantages of A/B and Multivariate Testing and I also discussed the choice between testing and analysis, but here are a few tips when it comes to website testing:

1. Testing is not limited to landing pages: It should be implemented across the website, wherever visitors are abandoning the website and wherever the website is leaving money on the table.
2. Try your tool (and your skills) with a small experiment in a page that does not require the CEO approval to change. Sometimes it is wise to start small and then grow. Once you are familiar with your tool, try a test in an important page but for a small (or less profitable) segment. Then head for the jackpot!
3. Measure multiple goals: while you improve primary conversions you might be decreasing registrations or newsletter signups which might have a negative impact in the long run.
4. Test for different segments: as I have described on Test Segmentation for Higher Conversion Rates, segments such as GEO location and operating systems (Hi, I am a Mac, and I am a PC) can have completely different behaviors, so the tests should also be segmented in order to understand those differences.

Implementation



Brazilians have a popular saying that can be translated as "to die on the beach". It is used for situations where you are almost getting what you want (the sea) and then you lose it. A Web Analyst that overcomes all previous steps successfully and then gets stuck on the implementation of insights is dying on the beach. No implementation is a synonym of no Web Analytics. Below are some tips that can help you overcome implementation bottlenecks:

• Get C level support: this will be essential if you come to a point where organization priorities must be set and resources allocated.
• Start small: as I mentioned above, starting small helps setting the expectations, people understand the tools and what is required from them.
• Be friendly: being a nice person and smiling always help you getting things done, that's the way human nature works.

Conclusion

The big question is: how can a website manager convince surfers to buy a product or read an article? And the answer is: look at the data and understand what is happening in the website, listen to customers’ voices and optimize the website to better serve them; after all they are the reason for the website’s existence. Customers should tell us what to do, not consultants, friends or feelings; data and online surveys are the place to look for customers’ needs.