CDLA · Module 3 — Outcomes and authentic assessment
Practitioners rate themselves low on evaluating impact. Design evidence with LTEM; use Kirkpatrick as a translation for executives, working backward.
About 16 min
Practice in the lesson player
Sorting, matching, quizzes, and the module gate stay behind a free account so scored answers are not public. The teaching notes below are the same cited curriculum.
01
ATD Capability Model self-assessment, 31 Dec 2023, N = 29,473. Self-ratings.
Data and analytics: 33.5%
Evaluating impact: 40%
Instructional design: 45.6%
02
Executive question (Kirkpatrick) | Ask first | Evidence that answers it (LTEM)
Level 4: Did results change? | Which number do you already track? | Tier 8
Level 3: Did behaviour change? | Who can observe it, and how often? | Tier 7
Level 2: Did they learn? | What decision or task shows it? | Tiers 5 and 6, not tier 4 alone
Level 1: Did they react well? | Ask last, and about transfer factors | Tier 3
03
Executives often know Kirkpatrick's four levels, so use them to start the conversation. Work backward from results they already track, then behaviour, then learning, then reaction. But Kirkpatrick's Level 2 lumps recall and real competence together, which is exactly where most overclaiming happens. Design and report with LTEM; translate into Kirkpatrick for the audience.
04
When a full impact study is too expensive, Robert Brinkerhoff's Success Case Method is a defensible middle path. Survey all participants briefly to find the most and least successful users of the learning. Interview a small number from each end. Document what the successful ones did differently and what it produced, with evidence, and what blocked the unsuccessful ones. The output is not an average effect. It is a credible account of what the program can produce and what in the environment stops it, which is often more useful to a sponsor.
05
This module designs the evidence. The CLISS program goes further into instrumentation, xAPI and executive narratives. Architects own the assessment map; impact strategists own the measurement protocol.