Research has potential to improve the lives of the world’s vulnerable people if it is appropriately referred to in decision making processes. While there is a significant industry of activity each year to communicate research findings, little systematic research has tested or compared the effectiveness of such efforts either for changing beliefs or for prompting action.
Using a randomised control design, this study explored the effectiveness of one popular research communication tool, a policy brief, and queried whether different versions of a brief bring about different results.
Key findings
- Influencing beliefs: the policy brief is more effective in creating ‘evidence-accurate’ beliefs amongst those with no prior opinion
- Messengers matter when it comes to people’s intended actions after reading the brief
- Influencing actions: gender and self-perceived levels of influence affect people’s intention to act after reading the brief
This first of its kind study has implications for how research communication experts design policy briefs, how they understand and enable readers to act as knowledge brokers in their particular environment and how we evaluate research communication going forward.
- Measuring the impact and influence of policy briefs (Research Summary) (PDF)
- What Difference Does a Policy Brief Make? (Research Report) (PDF)
Study design
The study used a multi-armed randomised controlled design to a) test the effectiveness of a policy brief overall for changing beliefs and prompting actions compared to a placebo policy brief delivered to a control group, and b) test whether different versions achieved greater or lesser effects.
The three treatments at the core of the study were different versions of policy brief that summarised findings from a systematic review of the effectiveness of food based agriculture interventions for improving nutritional status of children while the placebo brief was an existing issue from the IDS In Focus series that focused on priorities for delivering the Millennium Development Goals.
Data collection
The main data collection tool was a series of four surveys that sought to measure changes in participants beliefs and reported actions, administered in English through surveymonkey (a web based survey tool).
The study sample
Over 75,000 people were invited to the take part in the study, contacted via email, and 807 people opted in to the study by completing a baseline survey during the invitation period. The sample is composed of highly educated people, equally distributed between male and female. Most participants are working in government and non-government institutions and are aged between 25 and 55. Participants are from high income countries based on the World Bank classification in 46% of cases.
Limitations
The study’s limitations are around selection bias, its reliance on self-reporting, the nature of the original agriculture report in terms of its actionability, and the fact that a policy brief is rarely consulted in isolation.
Key contacts
