Monitoring Your Impact
Monitoring and Evaluating (M&E) Social Norms SBCC
Congratulations, you have programming in place addressing social norms and you want to monitor and evaluate it to make sure it’s working!
Remember that monitoring and evaluating your social norms programming should build on the earlier
work you have done to understand the social norms in your context.
This guidance is based on the worksheets and experience built under the FCDO-funded
Women’s Integrated Sexual Health (WISH) programme.
For more information, click here.
| Phase | Why? | How? | ||
| Formative | Identify possible social norms, reference groups and sanctions |
|
||
| Baseline | Verify social norms, assess their strength and opportunities |
|
||
| You are here | Monitoring | Capture indications of norm change, monitor backlash |
|
|
| Endline | Demonstrate changes in social norms, correlate with changes in behaviour and attitudes |
|
||
Monitoring social norms programming
Not all social norms programming will need an evaluation, but we should always at least monitor that things are working the way that we expect and document key operational learnings.
Key questions to address with monitoring and some suggested methods below, these should be easily integrated into existing project monitoring, tracking and supervision.
| Are activities taking place as planned? Are we reaching the right target groups with activities? | CBM tracking |
| Are activities creating the right types of discussion? | Observation, CBM feedback |
| Are we starting to see any change in participants reactions to the social norms? | Observation, CBM feedback |
| Has there been any negative feedback or reactions to the activities? | See section 8 for evaluating potential backlash |
Evaluating social norms programming
If you have used this resource hub to help design your social norms programming, then you already have a head start when it comes to evaluating. You will need to have identified and analysed your social norm(s) and have thought through how you will address them in your activities and you know how norm change works, and that it takes a long time. (if not – click through to the relevant section!)
A few key things to remember when evaluating social norms evaluations:
Social norms are not the only factors that drive behaviour, including behaviours linked to your outcomes of interest. Norms are almost always embedded in a system of structural drivers that intersect and sustain behaviour(s). Do not make the mistake of measuring only norms. You will have to make decisions about what other factors to measure, at the structural and individual levels. You will have listed many of these during the activities suggested in the section on identifying social norms.
Other factors you may want to measure will likely include individual attitudes but remember Norms can be aligned or misaligned with attitudes. Individual attitudes may be positively correlated (aligned), negatively correlated (misaligned), or possibly even uncorrelated with the social norm depending on the individual and the norm. If you did any of the insight gathering recommended in the related section, you will understand the other factors at play like attitudes.
Norms do not change overnight. Plan to measure at several time points, even as you measure other constructs that may be precursors to normative shifts and/or that may provide insight into why the desired behaviour change is or is not occurring. This section explains how norm change works, and how long this can take.
Demonstrating impact on social norms and linking this to behaviour and overall outcomes can be costly, time consuming and resource intensive. Consider the level of evidence you need for your project and design your evaluation accordingly.
- For example, do you need to demonstrate a robust quantified change and impact over time specifically on social norms to a donor? If so, you will likely need both quantitative data and qualitative data from baseline and endline.
- Are you evaluating a pilot, and want to understand if the approach will deliver the type of change you are seeking? If so, a qualitative evaluation is probably better suited to explore the ways change may (or may not) be happening and this level of evidence should be enough to inform decisions on scale-up.
- Do you need to demonstrate the importance or value of including social norms elements in your programming, but not direct impact? If so, you might be able to use quantitative data on behavioural outcomes (like FP uptake) and qualitative data on how your programming is influencing norms to provide this picture.
There are two main approaches to evaluating social norms programming, quantitative and qualitative.
Quantitative surveys
You can add social norms questions to quantitative questionnaires alongside questions on knowledge, attitudes and practice to enable a comparison between actual behaviour with empirical expectations, and individual attitudes with normative expectations.
For a quantitative survey to generate robust evidence you will need to be sure that it is representative of your target population (i.e., has a strong sampling approach) and you may need to capture data over time (i.e., use a longitudinal approach). Speak to your RME and/or the GSO Evidence and Impact team if you think you need this kind of evaluation.
Qualitative methods
Qualitative methods can also be used to explore shifts in social norms and whether such shifts are influenced by programme activities. Focus group discussions and/or in-depth interviews can be used to explore whether a community approves of disapproved of a certain behaviour, what happens when people breach and comply with certain expectations and whether this has changed over time, and what might have caused those changes.
| Area of measurement | Definition | Type of survey question |
|---|---|---|
| Knowledge | What I know | What is the legal age of marriage in xx country? |
| Attitude | What I think | Agreement with the statement: It is/ was important to me that I get married as soon as possible |
| Behaviour/ practice | What I do | At what age did you get married? |
| Empirical Expectations | What I think others do | What proportion of girls in your community are married by aged 20? All, Most, Some, Few, None |
| Normative Expectations | What others expect me to do | Agreement with the statement: Most people in my community think it’s important for a girl to get married as soon as possible |
| Sanctions/ rewards | The risks of breaching the norm/ the reward for complying | Agreement with the statement: In my community, if a girl is not married before she is 20 people will think there is something wrong with her In my community a girl who is married by the time she is 20 will be thought to have done well |
Retrospective questioning like this can overstate programme effects, but it has the advantage of offering an insight into perceptions of norm change without requiring baseline data. This approach could also be used at both baseline and endline, to provide a more robust indication of change
The MSI “Gagarabadau” project in Northern Nigeria used a vignette (short story) during qualitative discussions to explore participants’ reactions the norms targeted by the intervention related to men’s status and women’s role in decision-making.
By exploring whether participants felt others in their community behaved like those in the vignette the qualitative discussions were able to gauge the prevalence of certain norms. Further questions helped understand the potential sanctions and rewards at play. The facilitator was then able to probe whether participants felt that community attitudes towards certain topics had changed over time and if so, what had contributed to the change.
example: The MSI “Gagarabadau” project
Vignette: Hasan is 35 years old, he has two wives, Samirah (28) and Aisha (24). He has three children, two with Samirah and one with Aisha. He has decided that he would now like both of his wives to use family planning to avoid getting pregnant again for the moment. He wants to make sure that Aisha stays healthy and has time to take care of the younger children, while Samirah wants to start a small business to generate some income for the family. Hasan has discussed this with both Samirah and Aisha, and they agreed that this was a good thing for the success and well-being of their family.
Example follow-up questions
- How many other men in your community behave like Hasan? Why is that?
- Would other people in your community approve of the way Hasan has acted in this story?
- How would people in your community treat Hasan if they knew the things about him described in the story?
- Would the way that others treated him make Hasan change his mind about the decisions he made together with his wives about their family?
- What kinds of things make a man well-regarded and well-respected in your community?
- Has that changed at all / is that changing in any way? What is causing that change?