Measuring social norms on sexual and reproductive health in Kenya

Context

The Agonyora – Breaking the Shackling Power of Harmful Norms program is an intervention implemented by Lwala Community Alliance. We are evaluating this program with the support of the Children’s Investment Fund Foundation (CIFF). The program aims to improve access to and uptake of family planning services among young people in Migori county, Kenya to meet their unmet need. Overall, Migori county performs slightly lower than the national level for sexual and reproductive health (SRH) indicators. They have higher rates of teenage pregnancy and experiences of sexual violence, and lower than average use of contraception among married women.

Laterite is uncovering whether the program is associated with changes in social norms among male and female role models. Between June and July 2023, Laterite conducted a phone survey targeting 305 respondents in Migori county who were perceived to be role models in the community. We assessed norms around three priority areas: (1) acceptability of youth access to contraceptives, (2) acceptability of intergenerational communication about sex and puberty and (3) attitudes towards early marriage and teenage pregnancy.

To measure these priority norms, we needed to develop “scales” for each. Scales are tools for measuring constructs like social norms that cannot be directly measured. Instead, respondents are asked a series of questions related to that concept, and responses are combined to provide an overall score. This case study focuses on the process of identifying and developing these scales.

Process

We used a five-step process to identify and develop scales to measure the priority norms:

1. Defining the social constructs – Firstly, our constructs were guided by the three priority norms related to the implementation. The final constructs were youth contraception use stigma, parent-child communication, and teenage pregnancy and early marriage.

2. Generating a pool of potential questions – Next, we reviewed literature to identify previously validated scales. For constructs without validated scales, we generated questions that could measure our constructs based on findings from previous qualitative research.

3. Question selection – We kept questions as is from previously validated scales. Newly generated questions were assessed for construct validity. We also re-worded them to avoid bias from leading questions, double-barreled questions and the presence of implicit assumptions.

4. Developing scoring instructions – In our study, we scored responses such that lower numeric values consistently indicated more open attitudes towards the priority norms. Consequently, positively worded statements (e.g., A girl who is pregnant should continue with her schooling) were scored such that “Strongly agree” received a value of 1 and “Strongly disagree” received a value of 5. Negatively worded statements (e.g., Getting married before the age of 18 is beneficial) were reverse coded so that a response of “Strongly disagree” received a value of 1 and “Strongly agree” received a value of 5.

5. Translations – Finally, we conducted forward-translations of the final questions from English to Luo, which was the local language of administration, and back-translations of the questions from Luo to English. Back translations are an important tool for ensuring the meaning of questions does not change during the translation process. A key insight during the back translation process was that the meaning of “girl” in Luo varied depending on age and marital status.

Outcome

The scale development process led to the identification of four scales. Two of these (hyperlinked) had been previously validated in the literature. Two were generated for this specific evaluation. Our final questionnaire included:

1. Contraception use stigma scale – 7 questions
2. Parent-child communication scale – 8 questions
3. Equity for girls’ scale – 4 questions
4. Girls’ empowerment scale – 12 questions

During data collection, we assessed the scale performances by monitoring how well questions within the scale correlated with each other, a marker of scale reliability. We also used factor analysis to confirm that the items in each of our scales corresponded to distinct constructs.

The findings from the study were that on average, role models had positive attitudes towards the priority norms. Some characteristics such as being married and having been involved in previous norms change activities were also predictors of having an open-minded attitude.

Beyond this, we also analyzed how the program is associated with increased family planning visits and changes in costs of family planning provision. We’re looking forward to gaining more insights from the program’s evaluation during midline and endline.

Often, our work requires us to measure complex constructs ranging from social norms about sexual and reproductive health, to women’s empowerment, to the self-efficacy of farmers. Measuring these constructs is too challenging for any single question. By developing survey tools that consist of carefully constructed, contextually appropriate scales we can accurately measure these outcomes and assess the real-world impact of programs.

This evaluation was funded by the Children’s Investment Fund Foundation (CIFF).