The GAO Logic Model

Ever heard of the GAO Logic Model?

I wouldn’t be surprised if you haven’t heard of the GAO Logic Model because it isn’t labeled as such in the 2024 Yellow Book. However, I hear auditors talking about it here and there, so I thought I’d try my best to explain it.

The GAO Logic Model is a family of performance metrics that apply to government programs (and any other activity for that matter!) And if you follow the performance audit standards in the Yellow Book, the GAO advises you to get to know your audit subject using the model.

Check out section 8.38 of the Yellow Book for the detailed description of the components of the model:

• Input metrics
• Process metrics
• Output metrics
• Outcome metrics

These metrics tell those who run and oversee a program what went in, what was done, what came out, and whether any of it actually mattered.

You have to know the family

Isn’t it illuminating when you meet someone’s parents? A person’s family explains a lot about how a person thinks and acts… the good, the bad, and the ugly!

The metrics contained in the Logic Model work the same way. You need to know all the metrics to understand how well a program is performing. Not just the outputs of a program.

The GAO Logic Model explained with flu shots

For our example, we will pretend to measure a county program at a health clinic where low-income individuals receive free flu shots.

If I told you the county population was 150,000 folks and the clinic vaccinated 39,500 folks, would you be pleased?

Well, that depends on the rest of the family.

I have all kinds of questions about other metrics before I can decide if that is a reasonable number.

• What resources did the county have at its disposal?
• How many folks were eligible to participate?
• How long does it take to administer a vaccination?
• Are people any healthier after all that work?

So, let’s meet the family!

Inputs: What was thrown into the program?

The GAO says inputs are the amount of resources… put into a program, including money, people, and materials.

For the county clinic, lets pretend that looks like:

• $720,000 spent on vaccines, supplies, and staffing
• 6,200 staff hours dedicated to flu vaccinations
• 48,000 vaccine doses sitting in clinic refrigerators humming quietly
• 70,000 low-income residents eligible for services

Inputs are nice and concrete. We can measure them and gather evidence to support them easily. Auditors love them because they come with invoices, payroll records, and spreadsheets that behave.

Program Operations (a.k.a. Process Measures): What did you do all day?

The Yellow Book calls these the strategies, processes, and activities management uses to convert inputs into outputs.

Examples:

• Average patient wait time: 18 minutes
• Appointment utilization rate: 82%
• Vaccinations per staff hour: 6.4
• Average cost per vaccination: $18.23

This is where internal controls live. Scheduling systems. Staffing plans. The quiet nurse who figured out how to move patients in and out of the waiting room faster.

Outputs: What came out the other end of the process?

Per the GAO, outputs are the quantity of goods or services produced by a program.

Outputs are the participation trophies of performance measurement:

• 39,500 flu vaccinations administered
• 56% of the eligible population served

Outputs answer the question, “Did we do the thing we said we were going to do?”

They do not answer the question: “Did it help anyone?”

Which brings us to…

Outcomes: The holy grail

The GAO says outcomes are accomplishments or results of a program and they show progress made in achieving the stated program purposes.

This is the why:

• 14% reduction in flu-related ER visits
• Lower flu positivity rate among clinic-served residents (9.8% vs. 13.5% countywide)

Outcomes are hard. Really hard because they are influenced by cultural, economic, physical, or technological factors outside the program.

For instance, my family of four got the flu shot and then left a week later for an expensive trip to Disney World. We all came down with the flu and were quarantined in our hotel for a week. One of my daughters actually threw up on Cinderella before we realized what was going on! Bibbidi-bobbidi-blech. The flu strain we had wasn’t covered in the vaccination.

So a variety of factors can influence the successful outcome of a vaccination program like weather, school-aged children, and whether flu season showed up angry or just mildly annoyed.

Given all of those variables, is it reasonable to praise the clinic for their outcomes? I mean, did the flu shot actually work or did we just luck out?

On the other hand, imagine critiquing the clinic for their poor outcomes. Their response will likely be, “Hey! We can’t control everything! Give us a break.” And they will be right.

It is about control

Input and process metrics advise the reader on program constraints and costs. Output metrics count quantities produced. All of these measures can be directly impacted by working hard and conserving resources.

Outcome metrics report that the program made a difference. But the outcomes aren’t entirely in the control of the program managers the way that the sheer activity and busyness of the input, process, and outcome metrics are. That’s why outcomes are the holy grail of metrics.

So the next time someone says, “We vaccinated 40,000 people. We did a great job. Please give us more money so we can keep working.”

You can gently reply with, “Yes, but did fewer people end up in the ER?” Because if not, we should spend our time doing something else.

Looking for high-quality and convenient CPE?

We have you covered! Our live webinars are a great choice if you want the learning to come to you. Just log on at the scheduled time and enjoy wherever you are! Here are a few of our upcoming courses:

• Mar 24: Professional Skepticism: Critical Tool for Combating Errors, Scams & Lies (2 CPE Hours)
• Mar 25: Excel’s Hidden Gem: Analysis ToolPak Tips Every Auditor Should Know (2 CPE Hours)
• Mar 26: Audit Interviewing: The Known, Unknown & Getting the Facts (4 CPE Hours)
• Mar 31: Microsoft 365 Deep Dive for Government Auditors (4 CPE Hours)
• Apr 8: Introduction to Cybersecurity for Government Auditors (2 CPE Hours)

Need to do things at your own speed, but still get all your credits? Plan your CPE around your life, not the other way around! Yellowbook-CPE.com has dozens of self-study e-book and video courses, including the Essentials Skills Bundle. Are you a new auditor, or trying to get a new hire up to speed? This bundle of courses is for you! Through a combination of video and text, new auditors will learn how to think through an audit project on their own AND learn critical client relation skills.