• The Spark

Measuring the Impact of Digital Learning

Author:  Eric Sheninger, Senior Fellow, International Center for Leadership in Education (ICLE)  | 12/12/2017

As we move more toward digital learning and innovation in our schools, our ability to show the impact—not just talk about it—is at the root of any successful implementation. School leaders and teachers must be able to articulate the why, how, and what, and the detailed process that drives each respective change effort—as well as the resulting qualitative and quantitative data.

School and district leaders are continually in search of tools and processes that can help them measure the impact of the changes they are implementing. Practices such as Bring Your Own Device (BYOD), 1:1 technology, blended learning, personalized learning, classroom and school redesign, branding, makerspaces, and professional learning can all lead practitioners to ask: “How do we determine the effectiveness of our latest initiative?” Among the many services and tools available to help districts, schools, and organizations transform teaching, learning, and leadership, one that warrants close attention is the International Center for Leadership in Education’s (ICLE) Digital Practice Assessment (DPA).

What is the DPA and what is its value?

The DPA, aligned to the seven Pillars of Digital Leadership (below), provides authentic baseline data to support personalized professional learning and thus creates the context for work with leaders and teachers in a school. Through this proven model, our consultants help schools and districts identify opportunities to begin their transformation or take their digital and innovation goals to the next level, leveraging the knowledge, experience, and practice of ICLE’s thought leadership.

Districts use the DPA to develop, refine, measure, and then share examples that illustrate how an initiative has attained efficacy. These steps, which structure a critical look at practice, allow everyone to know where they are and how effective change is.

The DPA consists of a combination of a self-reflection questionnaire rubric, on-site observations, and online inventories comprised of data and evidence collection. Practitioners leverage evidence-based rubrics to observe leadership and instructional practices and collect artifacts to provide evidence of effective digital learning and innovative professional practice. Once these rubrics are applied and the artifacts are collected and analyzed, a detailed summary report outlining areas of success, focus opportunities, and recommended next steps guides the professional learning partnership with ICLE. The report, in turn, supports the development of a strategic professional learning and implementation plan.

The DPA Process

Step 1: Self-Reflection and Evidence Collection—Each school, usually designated by the district, completes the Pillars of Digital Leadership Questionnaire. This 18-question rubric asks school leaders to reflect on their perceptions as to where their school falls on a continuum, from not yet started to well developed. This questionnaire, along with aligned evidence for each item, is completed and archived in the Professional Learning Portal (PLP), a free, digital platform developed by ICLE to support schools in data collection, reflection, and goal setting to show growth and improvement over a period of time. Baseline evidence is in the context of digital leadership and learning and may include examples of student data, lesson plans, unit plans, student work, professional learning community meeting minutes, rigorous digital performance tasks, walk-through forms, assessments, sample observations/evaluations, portfolios, PD plans, social media accounts, pictures, videos, press releases, media coverage, and partnerships, among others.

Step 2: Stakeholder Interviews and Classroom Observations—Consultants conduct on-site observations and interviews to validate perceptions and evidence collected for the seven Pillars of Digital Leadership Questionnaire, as well as targeted classroom observations of student learning aligned to rigor, relevance and engagement. This data is collected and archived in the PLP during classroom observations. This allows the ICLE consultant to engage school leaders in dialogue about their culture, student learning, and digital integration, regardless of where they are with their digital transformation.

Step 3: Data Synthesis and Analysis—The data collected from classroom observations, which are tightly aligned to ICLE’s research-based rubrics, provide a detailed view of where a district or school is with their instruction and how teachers are leveraging technology in the classroom to increase student learning. Analysis of this data, combined with results from the self-reflective Pillars of Digital Leadership Questionnaire and other artifacts collected, lead to a summary report that details the current state of practice at each school in the district.

Step 4: Share the Findings and Plan Next Steps—The DPA report is shared and discussed with the school leadership team. In partnership with ICLE, observations about the evidence collected are shared and discussed. During the strategic planning process, discussions focus on areas of strength and improvements to develop a tailored and personalized implementation plan.

Step 5: Professional Learning and Continuous Monitoring—The personalized professional learning plan is implemented and progress is monitored and documented to determine the efficacy of the digital transformation. At specific points in the implementation, Steps 1-4 will be revisited to ensure continuous improvement and show growth along the way.

The DPA: Distinctive, Yet Aligned

The DPA process has been created to support districts and schools looking for ways to measure and articulate the impact of technology and innovation on practice. The DPA doesn’t look just at technology and innovation; it also provides insight on all elements of school culture and student learning. Data on its own is certainly valuable—and, the DPA model moves beyond looking at data alone as the only metric for success by actually taking a lens to an array of strategies and practices that combine to create a thriving learning culture.

In addition to being informed by a broad body of research and driven by evidence, the DPA process is aligned to the following:

The DPA process helps schools and districts develop, refine, measure, and then share amazing examples that illustrate how they’ve attained efficacy. By taking the steps to look critically at practice, we can see where we are and how effective change is on the journey toward system-wide digital transformation.

Learn more about how ICLE can support digital learning and innovation in your district.

# # #

I invite you to join ICLE in Orlando for our 26th annual Model Schools Conference, June 24–27, 2018, where participants will discuss effective education strategies and develop their own action plans for positive change.

A version of this post originally appeared on the blog ‘A Principal’s Reflections.’

  • Add your comment

    Your name will not be published to our website. Please enter comment name. We’re sorry—there are some invalid characters.


    Your nickname will appear with your comment. Please enter comment nickname. We’re sorry—there are some invalid characters.


    Please enter comment email.


    Please enter comment. We’re sorry—there are some invalid characters.