A multi-country evaluation of World Vision Youth Ready curriculum: Blended pedagogical approaches to Social Emotional Learning for vulnerable youths in Latin and Caribbean regions

Abstract

Before COVID, 60% of Latin Americans aged 15–29 were outside the school system, and one in five neither studied nor worked (CAF, 2020). Violence/gangs, poverty, gender-based violence, and discriminatory gender norms result in socio-emotional development challenges, depriving youth from reaching their full potential as educated, skilled, and productive citizens. World Vision (WV) pioneered Youth Ready (YR), a holistic multi-sectoral curriculum integrating social-emotional learning with second chance literacy and life-skills training to facilitate youth empowerment and citizenship education in the Latin/Caribbean regions. This program, guided by the Positive Youth Development (PYD) theoretical framework, targets vulnerable in and out-of-school youth who are unemployed, functionally illiterate, and at risk of violence. The PYD actively engages youth in building assets, agency, contribution, and enabling environment as vital to accessing education, employment, and entrepreneurship opportunities.  COVID-19 affected the implementation of YR, requiring innovative methods of reaching participants with alternative pedagogical approaches like integration of social media, and blended face-to-face/ virtual learning platforms. In April 2021, an independent comparative mixed-method evaluation across seven countries using the OECD criteria (Relevance, Coherence, Effectiveness, Efficiency, Impact, Sustainability) assessed the impact of YR delivery approaches. The results indicated that, even when adapted, YR made significant contributions to youth skill training and was highly effective.  Of the 1018 sample, four out of every five youth said it contributed to holistic skills development, was accessible, promoted a safe and gender-sensitive environment, and built collaborative relationships. The approach was 96% effectively developed social and communication skills including critical thinking and recognizing emotions. However, internet/technology challenges, such as lack of access and high associated costs, meant that some adaptations were more effective than others. The study documented success factors and challenges to guide future scale-up in other regions such as East and West Africa, using a blended curriculum delivery model that integrates emotional learning with literacy and life-skills training for at-risk youth.  
​
Keywords: Positive Youth Development, youth empowerment, citizenship education, and socio-emotional learning 
 
Introduction

15.5 % of young people constitute the world population (United Nations, 2018) and facing unprecedented challenges. Before the COVID-19 pandemic, nearly half the global youth population was out-of-school and unemployed (United Nations, 2018), exposing them to more violence, risky behaviour, irregular migration, trafficking, depression, suicide, and other anti-social and anti-thriving behaviorus. Consequently, for youths, this is also meaning lack of experience, low levels of education, mental health issues, and stigma from living in poverty and gang-affected communities affecting their contribution to their economies (World Vision, 2021). As a result, youth unemployment is one of the biggest challenges (United Nations, 2018). In Latin America and the Caribbean (LAC), nearly 20 million young people, are unemployed and/or not engaged in a learning program (Guyana Chronicles, March 17, 2020). The pandemic has further exacerbated these economic needs and declining mental health among the most affected youth. Before the pandemic, 60% of young Latin Americans (aged 15–29) were outside the school system and one in five neither studied nor worked (CAF, 2020). It is anticipated that the numbers have drastically increased leaving youths—especially girls and young women— at higher risk of violence, mental health, and psychosocial stress due to school closures, unemployment, and declining household income.   More so, the long-term consequences of the pandemic continue to threaten the ability of youths to participate fully, learn and develop skills required to become productive citizens. 

The need for positive youth development including transferable competencies such as socio-emotional skills, empowerment, and global citizenship education interventions for young people is now more apparent and urgent. Positive Youth Development (PYD) approaches have been proven successful in responding to at-risk youths, building their skills, fostering healthy relationships, and supporting them to be active citizens (Alvarado et.al.,2017; Catalano et.al., 2019; Ettekal & Agans, 2020). The Youth Ready program and curriculum, developed by World Vision (WV), presents this opportunity for young people across the globe. Globally, it has already helped thousands of vulnerable youths, with 60 000+ reached in the LAC, discover their potential, plan for their future, and gain the skills, support, resources, character, and confidence they need to succeed in both work and life.  

As has been the case in several education systems around the world, the COVID-19 pandemic upended the implementation of the Youth Ready curriculum. The constantly changing and uncertain contexts including school closures, unemployment, declining household income, mental health and psychosocial stress, lockdowns among many others continue to jeopardize normalcy in programming and youth access to education (Ettekal & Agans, 2020), employment and entrepreneurial pathways (Alvarado et.al.,2017).  Contextualized adaptations within 7 countries of the LAC region that WV is operating in with the YR program have been employed. As such, practitioner lessons learned and evidence sharing, on these adaptations, are valuable to support and enhance PYD interventions, especially in this unique and historical crisis, within similar comparative educational and global contexts. 

The current paper has developed under this premise. We recognize that the lessons drawn from a multi-country evaluation on youth programming adaptations conducted as part of the YR program build toward the scholarship and emerging body of research, practice, and knowledge within the field of comparative education, youth empowerment, and global citizenship. Limited evidence in Low Middle-Income Countries (LMICs) linking Social Emotional Learning (SEL) and PYD, even more within a restricted environment like that posed by the never-ending pandemic, has been documented. Although we focus on evidence from the LAC region, the timeliness of the evidence will be invaluable globally. Particularly, it will enhance diverse youth programming stakeholders’ work including academia, practitioners, donors, and policymakers as they navigate this catastrophic event and new normal in the future of young people. 

The paper will outline the intricate linkages between SEL and PYD approaches, evidence on the impact and adaptations, such as blended learning approaches, on youth program implementation, including specific implications on SEL curriculum components, because of the pandemic, and will conclude with concluding practices for consideration. 
The Youth Ready (YR) Model, Positive Youth Development and Socio-Emotional Learning: A Conceptual Framework 

The development of the YR curriculum began with the recognition that substantial investment and interventions in developing youth self-confidence, employability skills, and social assets and agency, guidance back to education, employment, or entrepreneurial opportunities are critical in enabling them to become active and positive contributors to their communities and countries. Centered within the global citizenship education and youth empowerment theories, WV works with the governments, schools, churches, and ecological systems in creating a supportive environment for youth and at-risk adolescents and youth with complex needs (World Vision, n.d.). The program and curriculum support adolescents and youths, both within and outside of school, between 14 and 29 years old, who live in vulnerable communities in El Salvador, Guatemala, Honduras, Peru, Colombia, Ecuador, and Bolivia. Adolescents and youths participating in the program go through a two-staged “Viability Journey”, a mix of training, activities, peer support, mentoring, linkages, internships, seed capital, and scholarships. In the two stages, as illustrated in Fig1, youth i) are linked to peer group support, a safe learning space, and support from adult mentors and trained facilitators, ii) create their livelihood pathway plan which they pursue in the second stage. The curriculum is designed to equip youth with new knowledge, attitudes, and skills related to positive self-identity, basic literacy and reading comprehension, savings and financial education, health, and self-care, life skills, social-emotional intelligence, entrepreneurship, employability, citizenship, and resilience. 

Fig. 1 - YR Curriculum- Viability Journey 

Various models of positive youth development models have been documented and studied in diverse contexts, within HICs (Catalano, Berglund et.al, 2002; Lerner et.al, 2018) and LMICs (Catalano et.al., 2018; Alvarado et.al.,2017, Fox & Kaul,2018), with many being used to anchor programming for adolescents 10–18 years and young people between 18–29 years (Shek et.al.,2019). And like most, the YR curriculum is grounded in the PYD conceptual framework defined and developed by USAID under their Youth Power Learning program. The USAID PYD is well-aligned to the infamous and traditional Five Cs Model (Lerner et al., 2005), which outlines five vital factors that support the well-being and development of young people which are competence (i.e. problem-solving, logical thinking, decision-making, conflict resolution), confidence (i.e. global positive value and capabilities), character (i.e. morality, integrity, respect for social and cultural regulations), connections (individual positive relationships) and compassion (i.e. capacity of sympathizing and empathizing for others) (Shek et.al.,2019; Arnold & Silliman, 2017; Dimitrova & Wiium, 2021).  

As such guided by the USAID PYD framework, the YR curriculum engages vulnerable youths (14-29 years) across four domains: (1) assets, (2) agency, (3) contribution, and their (4) enabling environment (Hinson et.al., 2016) to improve their well-being, increase their access to and completion of education creating employment and entrepreneurship pathways. Specifically, the PYD makes the hypothesis that young people will thrive as global citizens and adults including good health, economic success, and meaningful participation if they are provided with the knowledge, skills, and support they need (Alvarado et.al.,2017). These are the main pillars that form the foundation of WV’s YR program in Latin and the Caribbean regions. 

Positive correlations between Social and Emotional Learning (SEL) and PYD are also reflected within the YR curriculum. SEL is defined by Shek et.al. (2019) as a positive development model that develops the capabilities of youths to better understand, manage and express the social-emotional aspects of life including their interactions with the environments. Research on SEL interventions has shown that the positive effects of SEL preventive programming are particularly significant for students identified as high risk (Castro-Olivo, 2014)) and prevent problems such as violence, drug use, or dropping out (Weissberg & O’Brien, 2004, Taylor et.al, 2017). Thus, intertwined with the YR PYD agenda, the SEL components in the first stages of the ‘Viability Journey’ frameworks target directly the development of transferable competencies, problem-solving skills, self-awareness, self-determination, leadership, and positive behaviours that contribute to the increased agency of youth. 

This paper positions the Youth Ready curriculum, as a representative model strongly embedded in the integration of PYD and SEL principles and presents findings on how blended learning approaches during the COVID-19 pandemic can be adapted to maintain these principles and provides a useful conceptual framework for understanding youth empowerment programs in LMICs such as those within Latin and Caribbean regions during an emerging crisis. 

The YR Adaptation Evaluation Research Questions

The evaluation was conducted with three key objectives, to understand the impact of COVID-19 on youth cohorts; to document innovative and blended YR approaches used by each implementing country; and analyse of successes and shortcomings of the curriculum using the Organization for Economic Co-operation and Development (OECD) criteria relevance, coherence, effectiveness, efficiency, impact, and sustainability. For the sake of this paper, we focus on indicators reflecting relevance, effectiveness, impact, and sustainability as they presented evidence related to youth development and socio-emotional learning skills among youths. Some of the questions guiding the study were, 

To what extent are the blended pedagogical approaches of the YR and content delivery, in the context of the pandemic:

1. Relevant to each implementing country’s context given the socio-economic, political, and technological contexts 
2. Effective in delivering the content in an accessible and inclusive manner, creating a positive impact on the mental health, gender inclusion, and socio-economic conditions of the participating youth. 
3. Impact of COVID 19 on youth attrition rates from the program; and
4. Sustainable in a way that World Vision can successfully standardize elements of YR curriculum delivery processes 

Positionality statement and Limitations 

WV acknowledges that the Youth Ready project, one understudy, is the organization’s initiative, hence achieving full neutrality and objectivity may have been limited. Additionally, WV was conscious of the potential biases that could affect the integrity of the evaluation and evidence-based on their position with study participants within the constructs of identity, power, and access e.g., funder-beneficiary dynamics. As a result, certain measures were put in place which was using a third-party independent consultant to design, collect and analyse data, incorporating participatory and collaborative research methodologies with the youths, and establishing a research body, the Centre for Learning for Adolescents and Youth (CLAY) to ensure adherence to research standards and coordination across the 7 countries under study.
 
Methodology 

Research Design: A participatory evaluation with both quantitative and qualitative data collection methods was used to address the evaluation objectives and key questions. Composite indicators were developed to measure relevance, coherence, effectiveness, impact, and sustainability of the YR curriculum implementation using blended delivery methods including the development of appropriate data collection tools. 

Sample and Data collection: A finite population sample size calculation formula was used to define the representative sample size. Out of 2469 youth who were engaged in the YR program since the pandemic hit (March 2020) from the 7 study countries, 1018 youth (66% women) were sampled, according to the total number of active youth and dropouts in each country. In addition, 71 interviews and 18 focus group discussions were conducted with female and male in-school  youth, out-of-school  youth, key private and local government partners, facilitators, and project coordinators.

Data Analysis: SPSS, a statistical software, was used to analyze the quantitative survey based on the composite indicators/set of questions designed to answer the research questions whereas thematic analysis was used for the qualitative data analysis using Kobo.  

Key Findings and Discussions

1. YR program COVID-19 Modalities 

All projects in the seven countries responded to the pandemic-related closures and the need to ensure that young participants continued to benefit from the program during the social and economic instability. The responses of each of the implementing projects were innovative, pivoting towards a mixed pedagogical approach that involved contextualized approaches such as in-person teaching while adhering to COVID-19 protocols (social distancing, sanitary facilities, etc.), the integration of other social media applications for communication; radio programming where internet/cell phone connectivity is poor; and adoption of variations of face-to-face teaching with delivery through online platforms (blended). 49% completed the YR training in the blended method, 35% online only and 17% completed it in person with more than half of the young people (54%)’s motivation being to improve their skills including socio-emotional learning related. 
​
2. Impact 

The impact measurement was analysed using two variables, retention of participants before and after the pandemic and participants' perception of the impact of their participation on other aspects of their lives. The qualitative findings indicated that the perceived impact of the COVID-19 pandemic is multidimensional, it affected the economy of young people's families and communities due to job losses, their education access, interrupted their studies, increase in domestic violence, caused adversity, increased domestic workload for women, emotional problems, mental illnesses (depression, anxiety), among others (Fig 2 below). All these adversely impact PYD and SEL which are critical for the global citizenship capabilities required of youths. Despite these challenges, 91% of young people completed the training program, especially those who still had face-to-face modality access e.g., Peru and Bolivia. However, countries such as Honduras and Bolivia (67%), which virtualized, represented the lowest completion rate due to inaccessible internet or smartphones (23%), wide gender digital divide (40% female, 30%male), and technical difficulties. As highlighted earlier in the paper, the effects of COVID-19 affected the most vulnerable youths living in poverty-stricken and rural areas. 

Fig. 2 - Reasons for attrition rates  

3. Relevance 

Due to the changes forced upon the delivery of the program, it was essential to measure whether the program continued to be relevant in serving the needs of the youths and the development of their skills. A set of questions were asked to determine the relevance of training delivery methods with the shift to virtual and blended facilitation of the training including the use of SEL and user-friendly modalities such as interactive activities, videos, images, and easy-to-understand graphics, and how these facilitated collaborations of young people in projects, group work, etc.  One of the main aims of the YR, even during the pandemic, is to build social skills, healthy relationships, communication, and social awareness amongst youths. 88% (89% female and 86% male) of the young people who completed the YR curriculum through blended delivery or only online delivery considered the training delivery method and content? to be relevant. 85% reported that the online platform supports collaboration with other young people in projects, group work, and presentations, improved personal relationships, and fostered shared decision-making, respect for opinions, and equal responsibilities. However, the evidence highlighted that the main weakness in the virtual modality is that it is not possible to interact effectively as in the in-person modality.  Furthermore, for young people in rural areas, the process became more complex, both due to connectivity limitations and due to their characteristics: shyness and difficulty in interacting in virtuality. 
​
4. Effectiveness 

A composite indicator with four dimensions was constructed to measure effectiveness, assigning a specific weight to each dimension. Amongst them was an indicator on knowledge and skills acquired by the youth within the training, i.e., improvement of softs skills such as social and communication skills including critical thinking and recognizing emotions in a safe and gender-sensitive learning environment. 4 out of 5 youths who completed the program in the blended or online delivery method considered it effective in improving their skills, accessible and promoted a safe and gender-sensitive environment.  On leadership and life skills, 69% of youth (78% female and 52% male) highlighted the limited portrayal of females as leaders, promotion of women leadership, or explicit activities within the curriculum to address these aspects. 

Fig 3- Effectiveness in improving skills 

5. Sustainability 

For this criterion, it was interesting to document the sustainable and recommended execution methods for the context of each country from stakeholder perspectives and experiences. To answer this question, the preference of young people toward the virtual or combined modality was established as the first parameter. The preference of young people is mostly towards the face-to-face modality (61%), 33% prefer the combined modality and 6% prefer the virtual modality. According to the survey, the face-to-face mode was preferred because it provided more effective interactions with other young people and facilitators (74) %, it easier to understand the content (60%), and to ask questions or real-time feedback (56%). 

Conclusions

The COVID 19 crisis highlighted the need to incorporate into projects an analysis of the adverse country-specific scenarios and carry out contingency measures to respond to the scenarios. Despite the results indicating that there is a greater preference for the virtual modality, there is an agreement between young people and facilitators that a blended modality has certain benefits that should not be lost, such as saving time and the possibility of improving their skills to interact better in a virtualized environment. Also, the young people state that their abilities to interact in the virtual mode improved over time and the management of the platforms became easier as they had more interactions with them thus providing potential for them to continue building their strategic thinking, interpersonal skills, and social-emotional capacities. 

The virtualization experience in higher education that began in the 90s shows that this is an adaptation process that takes place gradually and that its sustainability lies in the creation of a different model of educational management that ensures quality in learning (Silvio, 1998). In this sense, through this study, some good practices can be feasibly standardized, adapted, and contextualized globally, especially in LMICs, as youth programs navigate this difficult and unprecedented context, have been documented. In conclusion, critical considerations in successful SEL and PYD approaches during a difficult crisis include:

•  Virtualisation: ‘Virtual meetings are the “bread and butter,” (Ettekal & Agans, 2020) of the pandemic The approaches adopted in all the countries were well adapted. The combination of training delivery methods during the Covid-19 pandemic and the use of different platforms allowed the continuity of the training processes and the majority of youth have completed the training.
• Tailored Adaptable Approaches: In the context of the pandemic, the approaches adopted in each of the7 countries were relevant for 88% of the target population.  However, tailoring virtual resources for the rural and low comprehension population using interactive and game-based activities is critical. 
• Considering Accessibility: Socio-economic conditions limited the successful implementation of a combined or online-only modality. In the context of the pandemic, the most vulnerable are those with limited resources (lacking devices) and are most likely to drop out of the program.

Written by: Vongaishe Changamire  and Lemlem Kebede -  World Vision Canada - please feel free to contact them if you have any further questions.

_______________________________________________________

References:

Alvarado, G., Skinner, M., Plaut, D., Moss, C., Kapungu, C., & Reavley, N. (2017). A systematic review of positive youth development programs in low-and middle-income countries. Washington, DC: YouthPower Learning, Making Cents International.
Arnold, M. E., & Silliman, B. (2017). From theory to practice: A critical review of positive youth development program frameworks. Journal of Youth Development, 12(2), 1-20.
Catalano, R. F., Berglund, M. L., Ryan, J. A., Lonczak, H. S., & Hawkins, J. D. (2002). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Prevention & Treatment, 5(1), 15a.
Catalano, R. F., Skinner, M. L., Alvarado, G., Kapungu, C., Reavley, N., Patton, G. C., ... & Petroni, S. (2019). Positive youth development programs in low-and middle-income countries: A conceptual framework and systematic review of efficacy. Journal of Adolescent Health, 65(1), 15-31.
Castro-Olivo, S. M. (2014). Promoting social-emotional learning in adolescent Latino ELLs: A study of the culturally adapted Strong Teens program. School Psychology Quarterly, 29(4), 567. 
Dimitrova, R., & Wiium, N. (Eds.). (2021). Handbook of positive youth development: Advancing research, policy, and practice in global contexts. Cham, Switzerland: Springer.
Ettekal, A. V., & Agans, J. P. (2020). Positive youth development through leisure: Confronting the COVID-19 pandemic. Journal of Youth Development, 15(2), 1-20.
Fox, L., & Kaul, U. (2018). The evidence is in: How should youth employment programs in low-income countries be designed?. World Bank Policy Research Working Paper, (8500).
Hinson, L., Kapungu, C., Jessee, C., Skinner, M., Bardini, M. & Evans-Whipp, T. (2016). Measuring Positive Youth Development Toolkit: A Guide for Implementers of Youth Programs. Washington, DC: YouthPower Learning, Making Cents International.
Lerner, R. M., & Lerner, J. V. (2013). The positive development of youth: Comprehensive findings from the 4-H study of positive youth development. Washington, DC: National 4-H Council.
Lerner, R. M., Lerner, J. V., Geldhof, G. J., Gestsdóttir, S., King, P. E., Sim, A. T., & Dowling, E. (2018). Studying positive youth development. Handbook of adolescent development research and its impact on global policy, 68-82.
Shek, D. T., Dou, D., Zhu, X., & Chai, W. (2019). Positive youth development: Current perspectives. Adolescent health, medicine and therapeutics, 10, 131.
Taylor, R. D., Oberle, E., Durlak, J. A., & Weissberg, R. P. (2017). Promoting positive youth development through school‐based social and emotional learning interventions: A meta‐analysis of follow‐up effects. Child development, 88(4), 1156-1171.
United Nations. Department of Economic and Social Affairs. (2018). World youth report: Youth and the 2030 agenda for sustainable development. United Nations Publications.
Weissberg, R. P., & O’Brien, M. U. (2004). What works in school-based social and emotional learning programs for positive youth development. The Annals of the American Academy of Political and Social Science, 591(1), 86-97.
World Vision International (n.d.) https://www.wvi.ngo/sites/default/files/Youth%20Ready%20Quick%20Guide.pdf