1. Consider your problem
A problem is an issue that is negatively affecting people, groups, or organisations.
You should already have a well-defined problem. Begin the play by briefly summarising the problem, including what caused it and who it affects.
2. Craft a short statement that defines your intended outcome
The outcome is the impact of your project.
You should already have an idea of your intended outcome. Put your outcome into a short statement that answers the question “What does success look like for this project?”, and keep the following in mind as you craft your statement:
The outcome is the impact of your project.
The outcome is not an output or a metric of your project.
- An output is a product created by the project – for example, a workshop artefact, a tool, or a report.
- A metric is a measurement that indicates whether the impact of the project is being achieved.
Example
Problem:
Customers are unhappy about how long it takes to get their problem resolved through the call centre.
Outcome:
❌ A chatbot that customers can talk to instead of calling – This is an output. It may or may not lead to success.
❌ Reduced call centre volume – This is a metric. It could indicate success, but it could also be a result of other factors, including coincidence.
✅ Shorter customer problem resolution times and increased customer satisfaction when contact is made via the call centre – This is an intended outcome. When this statement is true, the project will have succeeded.
The outcome may need several elements.
If your project touches multiple services, groups of stakeholders, or groups of users, you will need to consider the impact on each one.
For example, if your customers are unhappy about support call wait times you could increase the number of call centre staff. This could be a success from the perspective of customer satisfaction, but would it meet the business’s financial needs?
The outcome must be clear enough that you can tie it to metrics.
Without this, it’s likely that ‘success’ won’t be clearly defined, and it will be difficult to measure the project’s effectiveness.
Example
Problem: Support centre costs are too high for our budget
Outcome:
❌ Improve call centre efficiency – ‘Efficiency’ is too vague to measure
✅ Reduce call centre costs – Costs can be measured using financial data.
3. Plan how to measure progress towards the outcome over time
Metrics are measurements that are used to track progress toward your intended outcome.
Checking your metrics regularly throughout your project helps you track your progress towards solving your problem and keep your project on track. For examples of metrics you might use, see the resources section below. The types of metrics you should choose depends on your project.
If you are re-designing a service, your measurement plan may build on what is already measured and reported.
Whatever measurements you consider, identify ones that will help you understand the outcomes of the service, not just its outputs. For example, rather than relying solely on the count of how many people use the service annually, consider measuring how interacting with a service impacted someone’s situation, business, or family. Context for the numbers will help you understand the true value or impact of a service.
Decide what to measure based on your problem and your intended outcomes.
- How do you know your problem is real (what data and user insights do you have that indicates a problem?)
- What kind of data would give you a clearer idea of the problem?
- What data can be tied to the root causes of your problem?
- What kind of data would be useful to track changes over time?
Make a list of what you are going to measure.
Consider the resources you already have, and what kind you might need to acquire
- What sources of data do you have access to? For example: digital (web) analytics, user feedback, site performance, call centre data, financial information
- What analytics tools does your organisation already have?
The Queensland Government uses Google Analytics to monitor digital services. - Are any relevant metrics currently being tracked?
- What kind of additional data might you need that doesn’t currently exist or isn’t easily accessible? How could you access or generate it?
Tip
Never use a single metric as a goal.
Relying on metrics as goals can lead to a narrow focus on short-term outcomes, and may not capture the full picture of what is happening.
Metrics can be misleading because they can be influenced by a range of factors. Focusing too much on a metric can lead to a distorted view of performance and may not reflect the reality of the situation.
Metrics can lead to a narrow view of performance. For example, a project that focuses solely on decreasing cost to serve may neglect other important factors such as customer satisfaction.
Metrics can create incentives for unwanted behaviour. For example, a call centre employee who is focused on hitting a calls-per-day target might transfer calls unnecessarily, leading to worse experiences for customers.
Metrics can stifle creativity. Focusing on hitting specific metrics may cause a team to neglect opportunities for innovation and improvement.
Add these details to your list.
4. Take a benchmark measurement
A benchmark is an initial set of measurements to be compared against later.
Now that you’ve planned how to measure your problem and progress towards the desired outcome, plan who will take a measurement (or set of measurements) to use as a benchmark and when that will be done.
- Which other teams do you have to collaborate with to collect the required measurements?
- When will you take the initial benchmark (this may be different for different parts of the service)?
Create a document to record your measurements and track them over time. You may need to end the session here and reconvene once the benchmark data has been collected.
5. Define your success criteria
Success criteria are measurements that indicate that the problem has been solved. Establishing clear goals and success criteria creates alignment and understanding across the team.
Read your intended outcome statement, then look at your benchmark data. If the problem was resolved, how would that be reflected in your metrics?
Estimate the difference you would expect to see, then document it.
If you defined multiple outcomes, establish clear success criteria for each one.
Remember, the success metrics are not the goal, they’re indicators that you’ve reached the goal.
6. Set up a measurement schedule
Decide on the frequency with which you will capture data to measure your project's progress. This can be weekly, bi-weekly, or monthly, depending on the project's timeline and the team's preference.
You do not need to review the data each time you collect it.
7. Allow for iteration
As you move through the project and uncover more information, you might need to update your desired outcomes. If that happens, you will also need to update your success criteria or even choose different metrics.
