The Kirkpatrick Model: Level 4

We are beginning to close up our discussion on the Kirkpatrick Model with a description of its last level, Level 4, which deals with the issue of Results.


Results . . . in short, yes, this is what administrators ultimately look at in the context of their training plans. Be it a commercial business or a not-for-profit, mindful organizations are extremely careful with costs that some might view as discretionary, such as training. For training to be of value, it ultimately needs to be translated into Results.

The Kirkpatrick Model characterizes Results as: “the degree to which targeted outcomes occur as a result of the training and support and accountability package.”

The KNWM adds another dimension to Level 4: Leading Indicators. This addition focuses on “short-term observations and measurements suggesting that critical behaviors are on track to create a positive impact on desired results” (

The last level should be analyzed and structured even before Level 1 and even before the training begins. Why? First, administrators find it difficult to determine useful metrics for measuring employee behavior. Attempting to create metrics after the completion of the training is problematic, because doing so can lead to accepting poor measures of outcomes or just accepting a “general sense” of the outcome without looking at how the training actually impacted the bottom line. Let’s look at an example of possible consequences:

Imagine that training costs $10,000; imagine that training only increased production value by $1,000 per year; and, imagine that there is a complete turnover of employees every eight years. Such measurements confirm that the training decreased overall organization income by at least $2,000.

Second, administrators need to be able to discuss not only the training with employees but also the means by which they plan on measuring its value. Educational Technologies confirm that consulting with employees makes the collection of data for the metric easier; problems identified with the collection process can be fed back into the training program to modify future assessments (

Educational Technologies also suggests training value can be determined by introducing a “control group,” as one might in a formal scientific experiment. Creating a control group might seem to be discriminatory toward those not included in the training. However, such need not be depending upon the structure and timing of the training. For example, if the training takes place over rotating, consecutive phases that last, say, over six months, then it would be possible to assess performance metrics of the first group versus that of the last group still awaiting training.

As noted at the onset, the Kirkpatrick Model changed its conceptualization from a hierarchical pyramid toward links in a chain. The notion of a chain connotes an interconnected process, but the Kirkpatrick Partners also use the notion of chain to develop the means of determining Results: a Chain of Evidence.


Those at Kirkpatrick Partners argue that the chain model needs to be followed while being mindful of five different principles:

  1. The end is the beginning
  2. Return on Expectations (ROE) is the ultimate indicator of value
  3. Business partnership is essential to bring about positive ROE
  4. Value must be created before it can be demonstrated, and
  5. A compelling chain of evidence demonstrates your bottom line value.

Ultimately, these principles have led to what the Kirkpatrick Partners term as the “true” model or “complete model,” the Kirkpatrick Business Partnership Model as depicted below (“The Kirkpatrick Four Levels: A Fresh Look After 50 Years, 1959-2009,” Jim Kirkpatrick and Wendy Kirkpatrick).

Next week, we’ll describe the Kirkpatrick Principles in detail and, in a final blog, discuss the critiques of the Kirkpatrick Model while placing it in the context of other models.

Craig Lee Keller, Ph.D., JAG Learning Strategist

The Kirkpatrick Model: Level 3

For many administrators and managers “in the trenches,” the notion of appreciating post-training behavior is a novel concept. They are consumed with responsibilities and tasks in the workplace; some may even believe that extra work was created by the detour from work to attend the training.


This level is fairly straight forward, but a key link in the original Kirkpatrick Model. Again, to state the obvious, trainings have extremely limited value if their intended purposes are not some how realized in the workplace. The Kirkpatrick model utilizes a single element for the third level and adds an additional one for the NWKM: Required Drivers (

A. Behavior

  1. Behavior is defined by “the degree to which participants apply what they have learned during the training when they are back at the job.”

B. Required Drivers

  1. Similarly, required drivers are “processes and systems that reinforce, encourage, and reward performance of critical behaviors on the job.”

The web site confirms that determining the level of staff application of key principles, mindsets, and skill sets is quite challenging at the onset. They argue that assessment of Level 3 Behavior should take place between three to six months after the training. Much of this assessment includes informal observations; however, discerning whether or not the training has truly taken root is determined through staff counseling and interviews. Using “tests” can be problematic, for as discussed, the ability to “know” information is very different from being able to “apply” training information in real-life job situations.

The Required Drivers as the second element of Level 3 is truly significant. Without administrative processes and systems affirming the training, many employees—perhaps most—will simply forget about the training and leave the materials under an ever-increasing pile of training materials never to be looked at again. So what does it mean to implement Required Drivers?

Required Drivers necessitate administrators and managers to become actively involved in the process of implementing the training in the workplace. The NWKM identifies three ways this can be accomplished: reinforcement, encouragement, and rewards.  Functionally speaking, what does this mean in the workplace?

Reinforcing training material requires administrators and managers to serve as a “coach.” Playing the role of the coach is essential; here instead of being a judge, coaches provide reminders and refreshers of training material in situations.

Encouragement requires management to be sympathetic to their employees. Such encouragement is founded on the management insight that knowledge levels and positive dispositions are not the only factor when organizations seek to implement new practices and work models. In short, employees may attempt doff off old practices in exchange for new ones, but old habits can be hard to break. Equally, the skill of recognizing when to apply the training is frequently accomplished through trial and error until a given employee can develop a sufficient level of skill.

Rewards make things easier for employees. While having an affirming manager/coach is essential, rewards offer the external incentives that can further motivate staff during periods when the training model has not been fully implemented.

In our wrap up of the Kirkpatrick Model, we’ll look at Level 4: Results. With this level, we’ll include a discussion of the Kirkpatrick Principles, which govern and provide direction for the different links in the model.

Craig Lee Keller, Ph.D., JAG Learning Strategist