Evaluating Community Impact 2015: Resources

Submitted by natashapei on October 22, 2015 - 10:32am


DAY ONE CURRICULUM – Tuesday, October 27 


Welcome and Opening of the Community

 -Introductory Slides  

Module #1: The Dynamic Nature of Community Change and Impact

What is the “it” we are trying to evaluate? What are the implications of defining communities as “complex adaptive systems”??  What are the implications for developing, implementing, adapting and evaluating community change strategies? We need a strong grasp of these foundational ideas before we explore how we can assess our efforts to evaluate community impact. 


Module #2: Building your Community Change Strategy 

The work of evaluating community change efforts depends entirely on that challenge a group is trying to address and the particular strategy they are working towards.  This is the “it” that evaluation is meant to assess.  However, community change groups often struggle to craft a community change strategy that is compelling, actionable and engaging. In this module, we’ll explore the different ways to build and describe your community change strategy - including logic models, theories of change and frameworks for change – and the importance of using ‘strategic learning’ to inform the continual adaptation of strategies to reflect new learnings, shifts in context, and the arrival of new organizations to our efforts.


Module #3 Designing Evaluations that are Used 

Sometimes you need to go slow before you go fast. While many community change practitioners are eager to get to discussions of measurements and tools, it is important that they lay the groundwork for a robust evaluation process before they get into the micro-mechanics of the work. 

In this module, we’ll explore four critical pre-conditions for evaluating community change: clarifying your intervention, sorting out the role of evaluation users and evaluators, developing an evaluation scope of work and embracing some guidelines for developing good evaluations.


Module #4 Splash and Ripple Methods Carousel

All community change activities generate anticipated and unanticipated outcomes, and participants and evaluators need to try to capture both kinds of effects if they are serious about innovation and moving the needle on complex issues. 

In this module, we will explore the limitations of traditional evaluation practices in identifying outcomes and explore three practical resources, from the simple to the sophisticated, that you might use in your own work.

-Presentation: Outcome Harvesting by Mark Cabaj
-Handout: Outcome Harvesting 101
-Handout: Cases in Outcome Harvesting 

- Presentation: Most Significant Change by Liz Weaver

- Presentation: Theory of Change by Paule Langevin


DAY TWO CURRICULUM – Wednesday, October 28 


Module #5 Topic Population Level Change 

The ultimate goal of many would-be change makers is to ‘move the needle’ on complex community issues (e.g. high school graduation rates). While it is not always easy to measure population level changes – and even harder to link a group’s activities to these longer term changes – it is still important to have this as . This module will the value of population level data in informing and tracking the progress of community change efforts, three types of data strategies for , and the different role of strategists, implementers, and evaluation-researchers in 

-Presentation- Version 1

-Presentation- Version 2 (Adaptive Cycle)

Module #6: Shared Measurement

Collective Impact is an important new framework for communities that seek to move the needle on a complex community issue.  One of the core elements of collective impact is shared measurement where partners agree to a core set of measures which are designed to track positive and consistent progress at scale.  The development of a shared measurement framework will be explored in this module.  As well, the module will discuss both the benefits of a shared measurement practice and the challenges in this approach.  Finally, the module will explore the critical importance of building in learning and sense-making strategies in a shared measurement approach.  


-See Vibrant Communities' evolution of Shared Measurement, by Liz Weaver

Module #7: Evaluating Systems Change        

In order to move the needle on population-level outcomes, social innovators need to re-shape the (in)formal systems that shape those outcomes.  Programmatic interventions, on their own, are insufficient. In this module, we will explore several key concepts and frameworks for thinking about, planning and evaluating system changes. 

- Presentation

Module #8: Systems Change Methods Carousel  

In this module, we will explore three different methods to assess different aspects of systems change using a carousel method.  Specifically, participants will learn about three methods : understanding, influencing and evaluating policy change, building a public awareness profile for communicating evaluation results and building social networks and social capital.  

-Presentation: Carousel
-Presentation: Network Mapping
-Presentation: Evaluating Policy Change



DAY THREE CURRICULUM – Thursday, October 29 



Module #9: Contribution Analysis 

The people involved in any assessment of any local effort to change some part of the community for the better eventually turn their attention to one of the most vexing challenges in the evaluation of community change initiatives: How do we know if there is a link between our activities and observed results? Would the apartment complex have been developed without our efforts? Were there other factors that influenced the type, scale and timing of each of the changes?  Who can and should “take credit” for these outcomes?  Groups unable to answer these questions will struggle to (a) describe the success (or failure) of their efforts; (b) make evidence-based decisions on future directions; or, (c) demonstrate the value of their work to themselves or others.

This section will introduce participants to the concept and practice of contribution analysis, a systematic approach to assessing the “contribution” of the activities of local roundtables and networks to changes in their community.  It includes a general introduction to the questions of contribution, the steps of contribution analysis, and the opportunity to play with several techniques of contribution analysis.


Module #10: Making Sense of Outcomes and Communicating your Impact 

Data does not speak for itself. People must interpret it, draw conclusions, make judgements and recommendations. Without sense-making, data is noise. With sense-making, it become part o

This session will explore different processes for ‘making sense’ of data, and will include a brief exploration of some of the emerging techniques for data visualization, something that leading evaluator, Michael Quinn Patton, has called one of the most significant developments in evaluation and community change in the last twenty years. 

-Presentation: Evaluating Community Impact for Distribution 1
Presentation: Evaluating Community impact for Distribution 2

Module #11: Synthesizing our Learning 

In this interactive module, learning participants will synthesize the key evaluation themes that have been presented during the last three days.  They will be asked to reflect on each of the modules and determine how these lessons can be applied to their community change context.  Using a synthesis tool, from the book, Gamestorming – A Playbooks for Innovators, Rulebreakers and Changemakers by David Gray, Sunni Brown and James Macanufo, participants will work in learning pods to create a visual representation of our time together.  


Module #12: Key Themes and Learning Review 

13 modules in 3 days is a lot of material.  In this module, we’ll review the highlights of each, point out some new resources, and give you time to sort through how you will begin to use your learnings in your own community.  

-Summary Presentation

-Resource at a Glance: Policy Change

-Resource at a Glance: Capturing and Making Sense of Outcomes

-Resource at a Glance: Collective Impact

-Resource at a Glance: Developing Evaluations that are Used

-Resource at a Glance: Framework for Change

-Resource at a Glance: Phases of Collaborative Change Efforts

-Resource at a Glance: Network Mapping

-Resource at a Glance: MSC Stories 

module_2_-_building_your_community_change_strategy.pdf1.82 MB
module_3_-_designing_evaluations_that_are_used.pdf2.19 MB
module_4_-_splash_and_ripple_-_introduction.pdf1005.79 KB
module_4_-_splash_and_ripple_-_most_significant_change.pdf1.11 MB
module_5_-_population_level_changes.pdf1.86 MB
module_6_-_shared_measurement.pdf1.06 MB
module_8_-_systems_change_methods_carousel_-_public_awareness.pdf1.26 MB
module_11_-_synthesizing_our_learning.pdf859.81 KB
resource_at_a_glance_policy_change.pdf414.55 KB
resource_at_a_glance_capturing_and_making_sense_of_outcomes.pdf741.92 KB
resource_at_a_glance_collective-impact.pdf1.86 MB
resource_at_a_glance_developing_evaluations_that_are_used.pdf531.96 KB
resource_at_a_glance_framework_for_change.pdf179.5 KB
resource_at_a_glance_msc_stories.pdf257.88 KB
resource_at_a_glance_phases_of_collaborative_change_efforts.pdf571.25 KB
eci_oct_2015_-_welcome__learning_community.pdf1.35 MB
module_1-_the_dynamic_nature_of_community_change_and_impact.pdf1.84 MB
module_8_-_systems_change_-_network_mapping.pdf8.23 MB
module_8-_systems_change_methods_-evaluating_policy_change.pdf1.89 MB
module_10_-_data_viz_presentation_1_for_evaluating_community_impact_-_for_distribution.pdf12.3 MB
module_10_-_data_viz_presentation_2_for_evaluating_community_impact_-_for_distribution.pdf10.78 MB
aglance_networkmapping.pdf746.17 KB
module_4-splash__ripple-outcome_harvesting.pdf1.58 MB
outcome-harvesting-101-final1.pdf117.04 KB
cases_in_outcome_harvesting1.pdf1.84 MB
module_5_new.pdf14.95 MB
eci_2015_module_systems_change.pdf11.45 MB
eci_2015_-_contribution_analysis.pdf1.59 MB
eci_2015_-_summary.pdf766.08 KB
the_foundation_of_the_theory_of_change.pdf5.43 MB