Podcast Archive
Evaluating Impact
This podcast explores the results of the first phase of evaluation for the 12 communities across Canada, known as Trailbuilders, who were involved with Vibrant Communities.
Take the Seminar!
Community members and national sponsors of the Vibrant Communities network have been committed to a rigorous approach to learning about and evaluating their efforts since they began in 2002. The report, Evaluating Vibrant Communities: 2002-2010, summarizes results from the first phase of the evaluation. In this podcast, Eric Leviten-Reid talks with Jamie Gamble about this evaulation.
Learning Objectives:
- To become familiar with Vibrant Communities’ approach
- To learn about the patterns, scope and scale of changes contained in Evaluating Vibrant Communities: 2002-2010
Access Podcast Highlights:
- Poverty is Complex
- Understanding Vibrant Communities
- Patterns, Scope and Scale of Change
- Factors for Success
- Conclusions and Next Steps
- Reflection Questions
- Links & Resources
- Meet Jamie Gamble
Eric explains that poverty is a complex problem. It is technically complex as it involves a wide range of interdependent factors - health, housing, education, employment and so on - and socially complex because people from different backgrounds often understand the problem very differently.
As a result, Eric suggests, we need to work in unconventional ways:
Arriving at a shared understanding of the problem is a significant challenge in and of itself.
We usually break a problem into smaller parts and solve each one individually. With poverty, the different factors are completely intertwined.
We expect that every problem has a complete and perfect solution. However, the shape of poverty changes as the conditions giving rise to it change.
Eric gives more details in this clip, on the “tricky issues” and unconventional approaches that arise because poverty is a complex issue.
Understanding Vibrant Communities
The Vibrant Communities approach uses five key factors that interlock like pieces of a puzzle:
- Poverty Reduction, not Alleviation
- Comprehensive Thinking and Action
- Multisectoral Collaboration
- Community Asset Building
- Community Learning and Change
Three national sponsors supported the 12 communities across Canada, known as Trailbuilders, who were involved with Vibrant Communities:
The J.W. McConnell Family Foundation provided a series of matching grants to support core operations and to encourage other funders to do the same.
The Caledon Institute contributed policy research and writing to help document the work of communities, bring the policy implications forward to government and to help communities think through the policy issues involved in their work.
Tamarack facilitated cross community learning and coaching which allowed communities to learn from one another and reflect on their own experiences as they evolved.
In this clip, Eric gives more details on the “five puzzle pieces” of the Vibrant Communities approach.
Patterns, Scope and Scale of Change
Jamie uses the metaphor of a music score to explain how the Vibrant Communities principles played out in different ways across the country. While the principles were the notes in a music score, communities arranged them in various ways, arriving at quite different results. However, four patterns of activity emerged:
- Transformational Change
- Programmatic Push
- Citizen Empowerment
- Policy Focused
A programmatic push, as in Edmonton, emphasizes projects with immediate and tangible outcomes, whereas Calgary is an example of a policy focused pattern where actions tend to focus on changing the policies and practices of larger systems or organizations.
Transformational change, as in Hamilton and Saint John, is about synergies between different kinds of action at different levels, and seems to require a special set of conditions. Citizen empowerment emphasizes grassroots engagement, as in Saint-Michel (a Montreal neighbourhood).
Changes happened at multiple levels within each initiative:
- Beneficial effects for households
- Changes in larger systems
- Changes in the overall capacity and will of communities
Evaluating Vibrant Communities: 2002-2010 uses three case examples to illustrate the differences between communities:
- The multiple and comprehensive benefits of a program like Project Comeback in Surrey, British Columbia
- The use of in depth research in Saint John, New Brunswick to frame their initiative around priority neighbourhoods
- Connecting over 700 organizations in Hamilton, Ontario to achieve a high aspiration- the best place to raise a child.
Jamie summarizes some of the key numbers reported by the 12 communities over nine years:
- 322,698 poverty reducing benefits to
- 170,903 households in Canada
- 164 poverty reducing initiatives completed or in progress by local Trail Builders
- $19.5 million invested in local Trail Builder activity
- 1690 organizations partnering in Trail Builder communities
- An additional 1080 individuals serving as partners, including 573 people living in poverty.
- 35 substantive government policy changes
Here, Jamie gives examples from Edmonton, Calgary, Hamilton and Saint Michel to illustrate the four patterns of change.
Jamie explains that, as different communities started at different times, some are not as far down the road as others. However, some common factors for successful initiatives are apparent:
- Vibrant Communities entity is firmly established in the community
- High degree of credibility and legitimacy
- Able to clearly articulate the purpose of their work and overall approach
- Strong convening organization
- Initiative does not have “competition” from other convening efforts in the community
- Leadership of the initiative includes all sectors
- The effort is able to attract influential members
- A high degree of resident mobilization is present in the work
- Group has high aspirations
- High use of research to inform the work
Here, Jamie gives more details and examples from communities about the factors that made them successful.
Jamie emphasizes that a Vibrant Communities approach does not ‘guarantee’ success, but creates conditions and improves the probabilities of positive outcomes. However, the Vibrant Communities approach seems to affect four areas:
Community Will and Capacity
E.g. more innovative ideas, a higher profile for poverty, attracts more resources and a broader array of stakeholders
Systems and Policy Change
E.g. influences government policies, thinking and practices, strengthens links, coordination and information flows and decision-making processes
Individual and Household Poverty Reduction
E.g. addresses large numbers of households and more than one root cause of poverty
Lessons about Effectiveness
E.g. a large number of factors are relevant and progress is uneven but communities with the greatest effects share common characteristics
In the next phase of the evaluation, Vibrant Communities will delve deeper into three questions:
- What are the deeper effects of Vibrant Communities work on households, local capacity and systems?
- What were the effects of national supports to local groups?
- What makes a local group effective? What can we do to improve the chances of good outcomes?
The next part of the evaluation will be complete late in 2011.
In this clip, Eric describes the three themes that will guide the next phase of the evaluation.
GOING DEEPER
- What patterns do poverty reduction initiatives you know belong to - Transformational Change, Programmatic Push, Citizen Empowerment or Policy Focused?
- Which numbers and kinds of results that Vibrant Communities reported are the most compelling to you?
- What further questions or evaluation do you think Vibrant Communities should pursue in the second phase of their evaluation?
Evaluating Vibrant Communities: 2002-2010 - This report summarizes results from the first phase of the evaluation of Vibrant Communities. Access it here.
Evaluating Vibrant Communities Summary: 2002-2010 - Access the report executive summary here. A summary in French is available on request by emailing us here.
Evaluating Vibrant Communities: 2002-2010 Webpage - This webpage contains links to the full evaluation report, an executive summary and to other resources that explain the Vibrant Communities approach and evaluation process. Access the webpage here.
Getting the Most Out of Evaluation - In this audio seminar, Mark and Jamie talk about designing the Vibrant Communities evaluation process with the end-user in mind. Vibrant Communities asked key stakeholders about the questions that should be asked, the format of the end product and how, as a participant in the evaluation process, they might use the results to inform their work. Access the seminar here.
Learning and Evaluation for Vibrant Communities Trail Builders - This is the evaluation package used by local communities participating in Vibrant Communities. Access the package here.
Developmental Evaluation Primer - The primer, prepared by Jamie Gamble, describes an alternative to formative and summative evaluation. Access the primer here.
Strategic Dialogue on Poverty - The J.W. McConnell Family Foundation and the Tamarack Institute hosted a strategic dialogue about place-based poverty reduction efforts in Canada. Forty-six business, government, philanthropic, academic and community sector leaders reviewed the strengths and challenges of working collaboratively to tackle poverty in Canada and the United States. Access the web page about the event here.
Comprehensive Community Initiatives: Lessons Learned, Potential and Opportunities Moving Forward - The Wellesley Institute was commissioned to provide an overview of place-based approaches to poverty reduction in Canada. This paper describes the approach, trends, enablers, and impact. The paper also poses questions about next steps. Access the paper here and the presentation here.
Meet Jamie Gamble
Jamie Gamble - Jamie Gamble is the owner and principal of the New Brunswick based consulting company, Imprint Consulting Inc. He holds a Masters of Management Degree from McGill University. Through his work he is active in fostering innovation and development in a wide range of issues including poverty reduction, environmental protection, food security, youth leadership, and the arts. Jamie has extensive international experience and has worked in India, New Zealand and Mexico, and all across Canada. In 2008 the J.W. McConnell Family Foundation published Jamie’s Developmental Evaluation Primer.
©2011 Tamarack — An Institute for Community Engagement