Task Performance Indicator
- Take your customers’ top tasks (typically about 10).
- Create task questions based on these top tasks
- Give these questions to a representative sample of your customers (typically between 13 and 18) in a moderated, remote testing environment.
- Measure whether they are able to complete the tasks, how long it is taking, and what the main problems are.
We call the overall process the Task Performance Indicator (TPI). The TPI has been used by Cisco, Microsoft, NetApp, Dropbox, VMware, European Union, Tetra Pak, OECD, etc. A typical TPI takes about 6-8 weeks to complete. The first one costs in the region of €27,000, and repeat ones cost about €21,000. This price is based on our most senior consultants being involved. We also teach the method so that you can run them mainly by yourself, thus significantly reducing costs.
The TPI allows you to clearly show that improvements you have made are resulting in higher task completion by your customers. Let’s say we have a 60% success rate for a particular task. In the data we will provide you with there will be a clear explanation of what caused the 40% failure rate. If the causes of the failure are addressed and fixed, then the next time we test this task there will be a clear improvement in the success rate.
This makes the TPI different from other customer satisfaction measures, where it is often difficult to move the needle, because even when you make improvements, it doesn’t necessarily improve the customer satisfaction score. If you make an improvement connected with a TPI task, then the score will improve the next time you run the test.
The essential question that the TPI seeks to address is:
How quickly and easily are your customers able to complete their top tasks?
The Task Performance Indicator has been specially designed to answer that exact question.
How the Task Performance Indicator works
The Task Performance Indicator provides a defined process to measure task efficiency improvements on your website. The stages of the Task Performance Indicator are:
- Identify tasks to be measured.
- Recruit participants.
- Carry out task measurements.
- Recommend improvements
- Repeat roughly every six to 12 months
Identify tasks to be measured
Based on the top tasks as selected by your customers in the Top Tasks project, we will carefully select 10 task questions. We have developed a substantial methodology for developing task questions, which are significantly different from tasks themselves. For example, a top task for the Organisation for Economic Co-operation and Development (OECD) was:
Compare country statistical data
While the above is a task, you can’t test it as it stands. You must turn it into a very clear and precise question, such as:
Did more people die of heart attacks in Canada than in France in 2014?
Framing the task questions exactly right is essential to the success of the measurement process.
You will identify and recruit a number of customers to take part in the measurement sessions, typically about 30. We require a minimum of 13 people to do each task, so we try and have a pool of about 30 in case some drop out, etc.
Participants in the testing must be representative. Getting the right customer mix is vital in ensuring we get accurate, defensible results that reflect the true nature of carrying out the identified tasks on your website / app. It is standard practice to offer a small incentive for participation. We recommend a $75 Amazon voucher.
We will assist with advice on how to obtain the correct participant mix and how to inform them about what happens during a measurement.
Carry out task measurements
In individual sessions lasting approximately 60 minutes, we will measure how long it takes each participant to complete the tasks. Each task is carried out by at least 13 participants (ideally 15).
Basically, we measure the time it takes a participant to complete each task from the moment they have understood it and started working on it until they come up with their definitive answer or have given up.
The measurements will be carried out remotely. In such sessions, we will communicate with the participants over the phone and record their screen with a screen sharing tool, like WebEx or GoToMeeting.
The procedure for each measurement is:
- Briefly inform the participants of what’s going to happen. The procedure is also described in an introductory email sent to each participant well ahead of the measurement session.
- Emphasize that we are measuring the performance of the website / app, not them.
- Give specific tasks to the participants one by one, and ask them to complete these tasks as best they can.
Repeat every 6 to 12 months
The Task Performance Indicator is an ongoing management model. You can track the performance of customer tasks over time, making improvements and then seeing how the score changes. In the following table, we see how the success rate for a particular task has increased from 36% to 86% over a two-year period.
The TPI puts you in control of your own destiny. If you make improvements to your website or app, then these improvements will show up as higher success rates and lower task times the next time you run a TPI.
Task Performance Indicator deliverables
The deliverables involve a presentation including:
- An overall Task Performance Indicator score which is a combination of the scores for the 10 tasks tested.
- Data on what was the first thing customers did when given the tasks. Did they use search or navigation as their first strategy?
- Key recommendations on how to improve the efficiency of the tasks measured. Both general and task specific recommendations are provided.
- Task-by-task analysis, with a particular focus on the worst performing tasks, with the objective of highlighting what went wrong and how to fix it.
- Participant videos on a task-by-task basis will be supplied. This makes it much easier to review particular tasks and spot trends and issues.
- The Task Performance Indicator metrics for each task. These include the individual scores for the success rate and completion time.
See also: Search Performance Indicator