Monitoring Your Impact

Monitoring and Evaluating Social Norms Programming

Congratulations, you have programming in place addressing social norms and you want to monitor and evaluate it to make sure it’s working!

Remember that monitoring and evaluating your social norms programming should build on the earlier work you have done to understand the social norms in your context.

This guidance is based on the worksheets and experience built under the FCDO-funded Women’s Integrated Sexual Health (WISH) programme. For more information, click here.

Phase			Why?	How?	Covered in
Formative			Identify possible social norms, reference groups and sanctions	Desk review of existing literature and secondary data Informal discussions with your team Informal discussions key community informants (CRT) Qualitative discussions and interviews with community members	Section 4 Section 5
Baseline			Verify social norms, assess their strength and opportunities	Qualitative discussions and interviews with community members Quantitative surveys Analyse the norm (see section 4)	Section 4 Section 5
You are here	Monitoring	Capture indications of norm change, monitor backlash	Activity monitoring Observation	Section 8 Section 11
You are here		Endline	Demonstrate changes in social norms, correlate with changes in behaviour and attitudes	Qualitative discussions and interviews with community members Quantitative surveys	Section 11

Monitoring social norms programming

Not all social norms programming will need an evaluation, but we should always at least monitor that things are working the way that we expect and document key operational learnings.

Key questions to address with monitoring and some suggested methods below, these should be easily integrated into existing project monitoring, tracking and supervision.

Are activities taking place as planned? Are we reaching the right target groups with activities?	CBM tracking
Are activities creating the right types of discussion?	Observation, CBM feedback
Are we starting to see any change in participants reactions to the social norms?	Observation, CBM feedback
Has there been any negative feedback or reactions to the activities?	See section 8 for evaluating potential backlash

Evaluating social norms programming

If you have used this resource hub to help design your social norms programming, then you already have a head start when it comes to evaluating. You will need to have identified and analysed your social norm(s) and have thought through how you will address them in your activities and you know how norm change works, and that it takes a long time. (if not – click through to the relevant section!)

A few key things to remember when evaluating social norms evaluations:

Social norms are not the only factors that drive behaviour, including behaviours linked to your outcomes of interest. Norms are almost always embedded in a system of structural drivers that intersect and sustain behaviour(s). Do not make the mistake of measuring only norms. You will have to make decisions about what other factors to measure, at the structural and individual levels. You will have listed many of these during the activities suggested in the section on identifying social norms.

Other factors you may want to measure will likely include individual attitudes but remember Norms can be aligned or misaligned with attitudes. Individual attitudes may be positively correlated (aligned), negatively correlated (misaligned), or possibly even uncorrelated with the social norm depending on the individual and the norm. If you did any of the insight gathering recommended in the related section, you will understand the other factors at play like attitudes.

Norms do not change overnight. Plan to measure at several time points, even as you measure other constructs that may be precursors to normative shifts and/or that may provide insight into why the desired behaviour change is or is not occurring. This section explains how norm change works, and how long this can take.

Demonstrating impact on social norms and linking this to behaviour and overall outcomes can be costly, time consuming and resource intensive. Consider the level of evidence you need for your project and design your evaluation accordingly.

For example, do you need to demonstrate a robust quantified change and impact over time specifically on social norms to a donor? If so, you will likely need both quantitative data and qualitative data from baseline and endline.
Are you evaluating a pilot, and want to understand if the approach will deliver the type of change you are seeking? If so, a qualitative evaluation is probably better suited to explore the ways change may (or may not) be happening and this level of evidence should be enough to inform decisions on scale-up.
Do you need to demonstrate the importance or value of including social norms elements in your programming, but not direct impact? If so, you might be able to use quantitative data on behavioural outcomes (like FP uptake) and qualitative data on how your programming is influencing norms to provide this picture.

There are two main approaches to evaluating social norms programming, quantitative and qualitative.

Quantitative surveys
Qualitative methods

Quantitative surveys

You can add social norms questions to quantitative questionnaires alongside questions on knowledge, attitudes and practice to enable a comparison between actual behaviour with empirical expectations, and individual attitudes with normative expectations.

For a quantitative survey to generate robust evidence you will need to be sure that it is representative of your target population (i.e., has a strong sampling approach) and you may need to capture data over time (i.e., use a longitudinal approach). Speak to your RME and/or the GSO Evidence and Impact team if you think you need this kind of evaluation.

Qualitative methods

Qualitative methods can also be used to explore shifts in social norms and whether such shifts are influenced by programme activities. Focus group discussions and/or in-depth interviews can be used to explore whether a community approves of disapproved of a certain behaviour, what happens when people breach and comply with certain expectations and whether this has changed over time, and what might have caused those changes.

Area of measurement	Definition	Type of survey question
Knowledge	What I know	What is the legal age of marriage in xx country?
Attitude	What I think	Agreement with the statement: It is/ was important to me that I get married as soon as possible
Behaviour/ practice	What I do	At what age did you get married?
Empirical Expectations	What I think others do	What proportion of girls in your community are married by aged 20? All, Most, Some, Few, None
Normative Expectations	What others expect me to do	Agreement with the statement: Most people in my community think it’s important for a girl to get married as soon as possible
Sanctions/ rewards	The risks of breaching the norm/ the reward for complying	Agreement with the statement: In my community, if a girl is not married before she is 20 people will think there is something wrong with her In my community a girl who is married by the time she is 20 will be thought to have done well

Retrospective questioning like this can overstate programme effects, but it has the advantage of offering an insight into perceptions of norm change without requiring baseline data. This approach could also be used at both baseline and endline, to provide a more robust indication of change

The MSI “Gagarabadau” project in Northern Nigeria used a vignette (short story) during qualitative discussions to explore participants’ reactions the norms targeted by the intervention related to men’s status and women’s role in decision-making.

By exploring whether participants felt others in their community behaved like those in the vignette the qualitative discussions were able to gauge the prevalence of certain norms. Further questions helped understand the potential sanctions and rewards at play. The facilitator was then able to probe whether participants felt that community attitudes towards certain topics had changed over time and if so, what had contributed to the change.