Whether you are a commissioner in health or local government, the CEO of a VCFSE organisation, or a practitioner with your own company, it is now commonplace to measure and monitor the impact of your work. Deciding what to measure, however, is a whole different situation. In this blog we give you some questions, hints and tips that we think will help you.
1. Why are you measuring an outcome?
First and foremost, what is your reason for measurement?
- as a quality assurance process, i.e. to be able to audit your service
- as a routine part of your data monitoring
- as a discrete research or evaluation project to learn more about the impact of the service
The reason for starting at this point is to work out what a viable amount of data is to collect and exactly what type of data to collect. For example, if you are auditing your service, then you will have criteria and standards of performance. If you are monitoring your service, you are going to look for key data that is quick to collect in a consultation but provides key information on service impact. If you want to evaluate or research your service in more depth, you may choose more data to collect and use a range of measures for this. Collecting data from a range of measures takes up quite a bit of time so is not usually suitable to be collected as part of a consultation.
2. How do I capture the data that I need?
Interesting question! Some data you will use is the demographic data frequently collected when people attend their first appointment. But you also want to know if the support you offer has an effect on your client, right? Often people use patient reported outcome measures (PROMs) or patient reported experience measures (PREMs).These are questionnaires that capture a person's opinions on their health, social status or experience. Predominantly PROMs and PREMs are used in health care settings to enable the patient voice to be included in clinical decision making. In the last 5-10 years there has been a spread to non-healthcare settings as well.
3. Types of PROMs
PROMs have been designed with different intentions in mind and there are some broad categories that we can fit them into.
- General - these capture the overall health, quality of life or wellbeing status.
- Condition focused - these capture detailed information on one condition or situation.
- Person-centred - This category describes MYCaW and MYMOP questionnaires that Meaningful Measures use. These questionnaires avoid having pre-determined items listed and allow a person to nominate their key concerns or symptoms. This always enables the most relevant information to be captured using a person's own words.
4. Should I use a validated PROM?
Yes. Here at Meaningful Measures, we always advise you to use a validated measure, whether they are our own, or any other. It is important to know that the format and questions asked on a PROM relate to the population you are using it in. Validation is the approached used to determine this. If a PROM has not been tested in the population you want to use it for, you cannot be confident that is will be appropriate or that the score changes will be reliable or useful.
4. When is the best time to collect data?
Another interesting question! Well we always recommend you start your measures when you first see a person for an appointment. MYCaW and MYMOP are specifically designed to be used as part of a consultation without affecting the flow of discussion and relationship building with a client or service user. If you are using a questionnaire that has over 5 questions to it, you may want to allow extra time in an appointment for it to be complete or ask a client to fill it in before the appointment starts, maybe in the waiting room or emailed to them in advance.
When you want to collect follow up data, it is important to select a time period in which you think some change will have occurred so that this is reflected in the score changes you capture. This duration will be different in different settings, we are always happy to talk this through with you and help you work out what will be best.
5.What else should I consider when starting to collect data?
Quite a few things actually. Have you worked out how to fully comply with the Data Protection Act (2018), which includes GDPR regulations? Have you got a clear rationale for why you are collecting the data you are collecting? Are you adhering to a data minimisation approach? Are you clear on how you will safely store your data? Do you know when you do or don't need data sharing agreements in place? Do your service users or clients know how you are using their data? Do you know how to analyse the data you are collecting? Is the data collection and analysis part of some-one's job description specifically?
From our experience we often see that the biggest barriers to getting the process of data collection completed is:
- time within the working day for staff
- a lack of research knowledge and training to do the process with integrity
- a lack of knowledge on how to analyse and report the data
If you want some help with any of these points, whether it be a 1:1 chat to check you are doing things right, or you just want to hand your data over to experts to analyse for you, please get in touch with us via This email address is being protected from spambots. You need JavaScript enabled to view it.- we are only too pleased to be able to help you.