Summary of the Evaluation of the CanCode Program

Overview

CanCode is a national contribution program funded by Innovation, Science and Economic Development (ISED) Canada aimed at supporting the development of coding and digital skills among diverse K-12 youth and teachers. CanCode programming is currently delivered across Canada by 28 not-for-profit recipient organizations. CanCode training was delivered at no cost to participants in a variety of locations, modes and formats.

Between 2017-18 and 2022-23, ISED allocated funding of $190 Million to CanCode, including $50 Million in Budget 2017, $60 Million in Budget 2019, and a renewal of $80 Million over 3 years in Budget 2021.

The evaluation was conducted using multiple methodologies and sought to assess program relevance, performance and efficiency. The evaluation covered short-term outcomes for the period between April 1, 2017, to March 31, 2024.

Findings

Relevance

The evaluation found that CanCode addressed a demonstrated need to provide youth and teachers with access to coding and digital skills knowledge and training across Canada, in alignment with government priorities.

Labour market readiness:

• Labour market shifts driven by advanced technologies
• Need for coding and digital skills
• Prepare youth for the future workplace

Existing barriers:

• Inconsistent coding curricula
• Teacher discomfort with digital skills due to lack of experience
• Underrepresentation of certain sociodemographic groups in STEM

Funding importance:

• Without ISED funding, surveyed recipient organizations indicated that training might have been fee-based (63%) or not offered (38%)

Performance

CanCode has met their performance objective to increase access to coding and digital skills training for youth and teachers through their multimodal approach. Preliminary indications show that CanCode training is sparking curiosity in students towards STEM education and increased teachers' confidence to integrate digital skills training into the curriculum. The evaluation was unable to confirm the impact of CanCode training on youth's post-secondary and employment choices, and it also identified challenges with consistent performance measurement and reporting.

CanCode surpassed participation targets:

# of students reached	Target
7,621,161*	6,009,642

* 76% were youth with disabilities, girls, Indigenous, rural, remote, Northern and Black Youth

# of teachers reached	Target
405,599	270,669

98% of teachers surveyed agreed that CanCode training had contributed to developing teachers' and students' knowledge and confidence in coding and digital skills.

CanCode's multimodal approach & adaptation to the COVID-19 pandemic

CanCode's approach and adaptability to delivering digital skills learning contributed to a creative digital ecosystem and ensured that digital education remained accessible despite COVID-19 related disruptions.

Innovative engagement: Hands-on activities, like robot building and coding sessions tailored to various learner needs, or the use of mailed STEM kits during the pandemic.

Inclusivity and adaptability: The shift to virtual activities during the pandemic supported continued access to training, developed with culturally relevant content, especially for Indigenous communities.

Diverse learning methods: Incorporating digital literacy for youth with disabilities and integrating play and art with coding.

Targeted programs: Implementing all-girls programs and events to engage underrepresented groups in STEM, alongside general efforts to make digital education accessible and appealing to a broad audience.

85% of teachers surveyed indicated that CanCode had a moderate or major influence on students' intention to pursue STEM related education.

Performance measurement and reporting challenges:

• Inconsistent data collection methods across not-for-profit organizations
• Challenges in GBA Plus/EDI data collection due to youth privacy and sensitivity concerns
• Manual data entry which poses risks for errors
• Double-counting of participants due to recipient organizations collaborating
• Heavy focus on quantitative data

Efficiency

Overall, elements of CanCode's delivery model were found to be efficient through:

• Diverse partnerships: CanCode organizations formed public and private partnerships, exceeding objectives through collaborative efforts
• Leveraging of resources: Partnerships facilitated additional funding and in-kind contributions, pooling expertise for greater program reach and effectiveness
• Community of practice (best practice): Recipients were brought together to share experiences, optimize challenges, and adopt best practices, particularly benefiting underserved communities

The evaluation found that the reporting requirements were onerous and burdensome for recipient organizations. There were also challenges related to the application, approval, and funding process as recipient organizations noted a need for more guidance, feedback on unsuccessful applications, and transparency in fund distribution.

Recommendation

ISED Connected Canada Branch, Spectrum Telecommunications Sector should explore:

• Approaches to enhance the consistency of recipient-submitted data. Reliable and simple data tools as well as consistent methodologies would allow the program to better track its progress against outcomes, alleviate reporting requirements for recipient organizations and identify gaps in program reach relative to equity-deserving groups
• Strategic approaches and further leverage the Community of Practice forum to enhance knowledge sharing, networking initiatives and collaborative learning among recipient organizations. Such approaches could further enhance innovation, promote best practices and advance the goals of the program
• Opportunities to enhance the program application process to support applicants and communicate decisions