Has the GAO gone too far with audit quality control?

Objectives: 

• Assess whether an audit team has met the Yellow Book requirements regarding quality control and peer review

At the most basic level, the quality control requirements are twofold. The Yellow Book requires that audit shops:

1. have an internal quality control system, and
2. undergo an external peer review that determines whether the audit shop’s quality control system is working

Here is exactly what the Yellow Book has to say:

5.02     An audit organization conducting engagements in accordance with GAGAS must establish and maintain a system of quality control that is designed to provide the audit organization with reasonable assurance that the organization and its personnel comply with professional standards and applicable legal and regulatory requirements. 

5.60     Each audit organization conducting engagements in accordance with GAGAS must obtain an external peer review conducted by reviewers independent of the audit organization being reviewed. The peer review should be sufficient in scope to provide a reasonable basis for determining whether, for the period under review, (1) the reviewed audit organization’s system of quality control was suitably designed and (2) the organization is complying with its quality control system so that it has reasonable assurance that it is performing and reporting in conformity with professional standards and applicable legal and regulatory requirements in all material respects. 

Again, the GAO is mimicking the AICPA with this standard. And if you are currently part of the AICPA’s peer review program, none of the quality control or peer review standards will surprise or annoy you. But, if you are an internal auditor, you are not going to be happy because this standard far exceeds the IIA’s quality control and peer review standards! For instance, the IIA requires a peer review every five years, not every three years as required by the Yellow Book.

Quality control and peer review should suit your situation

If you are a one-man audit shop, how do you make sure your audits are up to snuff? What does your quality control system look like? I have heard a wide variety of answers. One guy said that he put his working papers away in a box and then gave them the once over using a checklist six months later. Another one-man shop said that he had an agreement with another auditor to review each other’s working papers every year or so. Another guy hired a reviewer once a year.

Which of these practitioners is right? All of them are right. Who judges whether your procedure is adequate or not? Your peer reviewer does.

If I conducted a peer review of the first guy – the guy who looks at his own stuff six months later – I would not be happy. But that is me. And the Yellow Book doesn’t say that what he is doing is wrong or that I am right.

One of my clients is a huge audit shop with 200+ auditors. On each engagement, the in-charge conducts a review, as does another supervisor. The audit manager reviews the working papers and sometimes the audit director gets involved. The shop also has a two- or three-person team called the “quality assurance team” that is responsible for reviewing every set of working papers in detail before the report is issued. Whoa! That is a lot of review. This huge audit shop receives a peer review from a similarly large audit shop every three years.

Notice that this is a review by a peer: someone like you, in a similar situation as you. A one-man shop would not ask a huge audit shop to review their quality control system. Instead, he would ask a peer, another one-man shop, to conduct the review. If the peer reviewing the one-man shop’s system also puts his stuff in the closet and looks at it in six months, the one-man shop is golden!

5.03     An audit organization’s system of quality control encompasses the organization’s leadership, emphasis on performing high-quality work, and policies and procedures designed to provide reasonable assurance of complying with professional standards and applicable legal and regulatory requirements. The nature, extent, and formality of an audit organization’s quality control system will vary based on the audit organization’s circumstances, such as size, number of offices and geographic dispersion, knowledge and experience of its personnel, nature and complexity of its engagement work, and cost-benefit considerations. 

While the quality control system must be documented, the last sentence of the above paragraph does allow a little flexibility regarding the form and content of the documentation depending on the audit organization’s circumstances.

Now that you know that the GAO allows some flexibility here, let’s look at the ideal they set forth in this standard. First let’s talk about the requirements for a quality control system and then we will talk about the revised standards for peer review.

Six Elements of a Quality Control System

The quality control standards entail six specific components. Where did the GAO get these six components? You guessed it! The AICPA.

If you are responsible for the quality control system for your team, or if you are a peer reviewer, I recommend that you go to the AICPA standards instead of solely relying on what the GAO says in this section. Sometimes the GAO, in summarizing the AICPA literature, leaves a few significant details out. Internal auditors, I’m talking to you, too. The AICPA renames the requirements periodically, but as of the fall of 2018, they are entitled “Establishing and Maintaining a System of Quality Control for a CPA Firm’s Accounting and Auditing Practice” and are available online for free.

Here are the six elements of your quality control system that the GAO standards require you to document and implement:

1. leadership responsibilities for quality within the audit organization (sections 5.05-5.06)
2. independence, legal, and ethical requirements (sections 5.08-5.11)
3. initiation, acceptance, and continuance of audits (sections 5.12-5.14)
4. human resources (sections 5.15-5.21)
5. engagement performance (sections 5.22-5.41)
6. monitoring of quality (sections 5.42-5.59)

1. Leadership responsibilities
This component of quality control asks that you are clear who is responsible for audit quality and what their responsibilities are.

2. Independence, legal, and ethical responsibilities
The audit organization needs to have policies and procedures in place to make sure that the team maintains its integrity and professionalism all year long.

The GAO added this clause in 2018:

5.09     At least annually, the audit organization should obtain written affirmation of compliance with its policies and procedures on independence from all of its personnel required to be independent. 

3.Initiation, acceptance, and continuance of audits
The audit organization needs to make sure that it can comply with standards and has the capacity to do the work before it accepts an engagement.

4. Human resources and 5. Engagement performance
Both of these components address your staff’s competence and responsibilities.

The HR element of the quality control system requires audit teams to have processes for recruiting, developing, and evaluating team members.

And the engagement performance standards define what supervisors and reviewers are responsible for doing:

5.39     Engagement supervision includes the following: 

1. tracking the progress of the engagement; 
2. considering the competence of individual members of the engagement team, whether they understand their instructions, and whether the work is being carried out in accordance with the planned approach to the engagement; 
3. addressing significant findings and issues arising during the engagement, considering their significance, and modifying the planned approach appropriately; and 
4. identifying matters for consultation or consideration by engagement team members with appropriate levels of skill and proficiency in auditing, specialists, or both during the engagement. 

5.40     A review of the work performed includes consideration of whether 

1. the work has been performed in accordance with professional standards and applicable legal and regulatory requirements; 
2. significant findings and issues have been raised for further consideration; 
3. appropriate consultations have taken place and the resulting conclusions have been documented and implemented; 
4. the nature, timing, and extent of the work performed is appropriate and without need for revision; 
5. the work performed supports the conclusions reached and is appropriately documented; 
6. the evidence obtained is sufficient and appropriate to support the report; and 
7. the objectives of the engagement procedures have been achieved.

The definition of a supervisor and a reviewer were very helpful to me when I was hired to edit audit reports and review working papers for a federal inspector general. The inspector general wanted me to sign off as a supervisor because the Yellow Book documentation standards require supervisory review of audit documentation before the report is issued. But since I did not get to choose who was on the assignment, monitor the project, or shape the results, I did not agree that I was a supervisor. My work occurred on the tail-end of the engagement to make sure that the audit met standards.

Instead of agreeing in my engagement letter to fulfill the role of a supervisor, I listed all of the responsibilities under the above description for a reviewer and referred to myself as a reviewer. Thankfully, the inspector general agreed that the description of a reviewer’s role better suited my responsibilities.

6. Monitoring of Quality
Through monitoring procedures, the audit organization makes sure that all five of the previous elements of the quality control system are in place and working.

Have things gone a little too far?

By adding monitoring to our internal processes, it feels like we may have gone too far. We are reviewing ourselves repeatedly. I joke with new auditors that the working papers are like zombies! They can rise from the grave any time – up to three years after you have put them to rest.

This is what could happen to a set of working papers in a large audit organization:

• The staff person creates the working papers and draft report
• A supervisor reviews
• A manager, director, and/or partner reviews
• A quality control team reviews the working papers before the report is published
• The report is published
• Annually, a monitor inspects the finished working papers plus performs other procedures
• Every three years, a peer reviewer looks at the quality control system and may select the working papers for another review against standards

See? A little much.

How did we get here? I am going to blame it on the COSO model of internal controls. The COSO model (which we will discuss in detail in later chapters) says that we don’t just trust that the controls we designed are working, but that we verify that the controls are working by conducting monitoring procedures. If we are going to require the auditee to monitor their controls – to add another layer of controls on their controls – we should also because we auditors have to model good behavior!

Monitoring Can Be Ongoing and/or Annual

Monitoring can be an ongoing process. Or the audit organization can choose to perform monitoring annually. For both, a report is expected.

5.43 The audit organization should perform monitoring procedures that enable it to assess compliance with professional standards and quality control policies and procedures for GAGAS engagements. Individuals performing monitoring should have sufficient expertise and authority within the audit organization. 

5.44     The audit organization should analyze and summarize the results of its monitoring process at least annually, with identification of any systemic or repetitive issues needing improvement, along with recommendations for corrective action. The audit organization should communicate to the relevant engagement partner or director, and other appropriate personnel, any deficiencies noted during the monitoring process and recommend appropriate remedial action. This communication should be sufficient to enable the audit organization and appropriate personnel to take prompt corrective action related to deficiencies, when necessary, in accordance with their defined roles and responsibilities. Information communicated should include the following: 

1.a description of the monitoring procedures performed; 
2.the conclusions reached from the monitoring procedures; and 
3.when relevant, a description of systemic, repetitive, or other deficiencies and of the actions taken to resolve those deficiencies. 

Annual Monitoring Is Similar in Scope and Activities to a Peer Review

The scope of the annual monitoring of the quality control system sounds very much like a full-blown peer review! The monitor will touch on every aspect of the quality control system and bring up any weaknesses in the system to audit management.

5.51     Ongoing consideration and evaluation of the audit organization’s system of quality control may include matters such as the following: 

a.review of selected administrative and human resource records pertaining to the quality control elements; 
b.review of engagement documentation and reports; 
c.discussions with the audit organization’s personnel; 
d.determination of corrective actions to be taken and improvements to be made in the system, including providing feedback on the audit organization’s policies and procedures relating to education and training; 
e.communication to appropriate audit organization personnel of weaknesses identified in the system, in the level of understanding of the system, or compliance with the system; and
f.follow-up by appropriate audit organization personnel so that necessary modifications are promptly made to the quality control policies and procedures. 

5.52     Monitoring procedures may also include an assessment of the following: 

a.    the appropriateness of the audit organization’s guidance materials and any practice aids; 
b.new developments in professional standards and applicable legal and regulatory requirements and how they are reflected in the audit organization’s policies and procedures, when appropriate; 
c.written affirmation of compliance with policies and procedures on independence; 
d.the effectiveness of staff training; 
e.decisions related to acceptance and continuance of relationships 
f.      with audited entities and specific engagements; and 
g.audit organization personnel’s understanding of the organization’s quality control policies and procedures and implementation thereof. 

Supervision, Reviews, Monitoring, and Inspections

After reading all the roles and responsibilities of a supervisor, reviewer, and monitor, you might wonder how you can do audits without an army of auditors at your disposal! Someone on the team needs to fill the role of supervisor, reviewer, monitor, and inspector. And while there is a relationship between these roles, the roles are not interchangeable. For example, look at this quote from the Yellow Book.

5.53     Reviews of the work by engagement team members prior to the date of the report are not monitoring procedures. 

I think the GAO still has some work to do clarifying the meaning of these various quality control roles on an audit, but here is my stab at it: As audits are being conducted, they must be supervisedas described earlier in this chapter. Before the audit report is issued, someone must reviewthe audit documentation. At least annually, a monitorchecks that the quality control system is in place (this quality control system includes supervision and monitoring). And as a part of the monitoring activities, the monitor will inspecta few finished audit projects.

What Does the Monitoring Report Say?

The annual monitoring report has very similar contents to a peer review report. The GAO recommends that the report include:

• Descriptions of the monitoring procedures performed;
• Conclusions drawn from the monitoring system;
• Descriptions of systemic quality control problems; and
• Actions taken to resolve the deficiencies.

5.59 Appropriate documentation relating to monitoring may include, for example, the following: 

a.    monitoring procedures, including the procedure for selecting completed engagements to be inspected; 

b.    a record of the evaluation of the following: 

1.    adherence to professional standards and applicable legal and regulatory requirements, 

2.    whether the system of quality control has been appropriately designed and is effectively implemented and operating, and 

3.    whether the audit organization’s quality control policies and procedures have been appropriately applied so that the reports that are issued by the audit organization are appropriate in the circumstances; and 

c.     identification of the deficiencies noted, an evaluation of their effect, and the basis for determining whether and what further action is necessary. 

Peer Reviews

Now let’s switch gears and become familiar with the external review of our quality control system that occurs every three years.

Peer reviews often do trigger significant, positive change in an audit organization. For example, consider this story: The director of a prestigious audit team was notoriously hard to work with. I got a taste of his counterproductive behavior first hand when he heckled me and belittled his staff when I led a class on writing audit reports for his team.

Not surprisingly, his team barely functioned due to the constant turmoil and fear he churned out. Because of the high turnover, they were unable to finish significant projects or keep good auditors on board. The team’s quality control system was in shambles.

The peer reviewer discovered early in his review that this director was the cause of a slip in the quality of the audits, and he wrote as much in his peer review report. He recommended counseling and leadership training for the director, and the director took the recommendation to heart.

After a year of counseling and leadership coaching, the director was a changed man. His staff became productive again and reported a much more pleasant work environment.

I saw him speak at a conference a year or so later. Instead of wearing his usual intimidating dark suit and red power tie, he wore a soft pink button up and, with a tear in his eye, spoke of being honored that he was asked to share his experiences with the audience. I almost didn’t recognize him!

So, for all of you stinkers out there, you’d better take your staff to lunch and atone for any evil acts before the peer reviewer shows up, or you, too, will end up wearing pastels and crying in public!

Purpose of a Peer Review

The purpose of a peer review is to make sure that your quality control system is working. The purpose of a peer review is not to dig in to all of your engagements for the past three years and or judge whether your audit planning was on target! Peer reviews are not supposed to make you cry. That would be duplicating the work already done by the quality control system.

5.60     Each audit organization conducting engagements in accordance with GAGAS must obtain an external peer review conducted by reviewers independent of the audit organization being reviewed. The peer review should be sufficient in scope to provide a reasonable basis for determining whether, for the period under review, (1) the reviewed audit organization’s system of quality control was suitably designed and (2) the organization is complying with its quality control system so that it has reasonable assurance that it is performing and reporting in conformity with professional standards and applicable legal and regulatory requirements in all material respects. 

5.70     A peer review is designed to test significant risk areas where it is possible that engagements are not being conducted, reported on, or both in conformity with professional standards and applicable legal and regulatory requirements in all material respects. A peer review is not designed to test every engagement, compliance with every professional standard, or every detailed component of the audit organization’s system of quality control. 

Scope of the Peer Review

One of the most important things that a peer reviewer verifies is that the annual monitoring of the quality control system is performed and that the problems raised by the monitor are acted upon and resolved (5.82 b). The peer reviewer will also interview audit staff and management to make sure they understand and are playing along with the quality control system.

Here are the areas a peer reviewer will evaluate:

5.82     The peer review team should include the following elements in the scope of the peer review: 

1. review of the audit organization’s design of, and compliance with, quality control and related policies and procedures; 
2. consideration of the adequacy and results of the audit organization’s internal monitoring procedures; 
3. review of selected audit reports and related documentation and, if applicable, documentation related to selected terminated engagements prepared in accordance with paragraph 5.25, if any terminated engagements are selected from the universe of engagements used for the peer review sample; 
4. review of prior peer review reports, if applicable; 
5. review of other documents necessary for assessing compliance with standards, for example, independence documentation, CPE records, and relevant human resource management files; and 
6. interviews with selected members of the audit organization’s personnel in various roles to assess their understanding of and compliance with relevant quality control policies and procedures. 

I led a peer review recently so let me give you an idea of the work involved in reviewing an organization with 100 or so auditors. We had a team of three reviewers (including me). We were at the client’s office for a full week but did a good amount of work before we had the entrance conference and a little work after the exit conference.

I created an engagement letter. In it, we asked the client to fill out a self-assessment questionnaire. All three of us reviewed the results of the questionnaire and reviewed the policies and procedures regarding quality control before we got on site and compared both the questionnaire and the policies and procedures to a list of all must and should requirements in the Yellow Book. We discussed how we would divide the work and contacted the client to see if we could accomplish any tasks ahead of our visit.

When we got on site, one of my team members performed and documented the interviews. Another of my team dug into the CPE records and independence documentation a week ahead of our visit and worked with the organization to resolve any discrepancies once on site. This same person examined a few working paper files on site. I reviewed prior monitoring and peer review reports before our site visit and reviewed a few working paper files on site.

I met with the client to inform them of our progress and negotiated any issues. I drafted the peer review report and documented our process using a peer review checklist created by the Association of Local Government Auditors. Along the way, we consulted with each other and learned the client’s systems. We exited on Friday at 2 p.m. after a very, very busy week.

Either Join a Peer Review Program or Follow GAO Specifics

I was very glad to have the Association of Local Government Auditor’s tools to rely on in performing the peer review. And the GAO appreciates these tools, too. So much so, that they have referred to them by name in the Yellow Book.

The GAO has taken a new tack on peer reviews in the 2018 version of the Yellow Book. They are deferring to several peer review programs, including the ALGA program. Auditors who are part of this program are considered in compliance with the Yellow Book peer review standards. If the audit team is not a part of one of these programs, they must follow the Yellow Book peer review requirements laid out in sections 5.62–5.65.

Most of the peer review requirements have been in place for many years, but in 2018 the GAO has added a few new requirements.

Let’s talk about the approved peer review programs first. Then we will talk about the GAO requirements that audit teams must follow if they are not part of a program. As part of the discussion of GAO requirements, I will highlight the few new requirements.

Approved Peer Review Programs

The GAO is acknowledging several peer review programs that are administered by professional organizations.

5.61     Audit organizations affiliated with one of the following recognized organizations should comply with the respective organization’s peer review requirements and the requirements listed throughout paragraphs 5.66 through 5.80. 

1.American Institute of Certified Public Accountants 
2.Council of the Inspectors General on Integrity and Efficiency 
3.Association of Local Government Auditors 
4.International Organization of Supreme Audit Institutions 
5.National State Auditors Association 

These programs provide peer review training, provide tools, and model reports that the peer reviewer can use, and manage a pool of available peer reviewers to staff a peer review. All organizations on this list have similarly robust programs.

But many auditors are not affiliated with these organizations. This means they can, as I did, use the tools and attend the training, but not contribute to or draw from the pool. Or they can follow the Yellow Book rules for how to conduct a peer review that are outlined later in Chapter 5.

Basic Requirements for a Peer Review

If you do not follow one of the programs recommended by the GAO, make sure that your peer review report contains the following elements:

5.91     The peer review team should prepare one or more written reports communicating the results of the peer review, which collectively include the following elements: 

1. a description of the scope of the peer review, including any limitations;
2. a rating concluding on whether the system of quality control of the reviewed audit organization was adequately designed and complied with during the period reviewed and would provide the audit organization with reasonable assurance that it conformed to professional standards and applicable legal and regulatoryrequirements;
3. specification of the professional standards and applicable legal and regulatory requirements to which the reviewed audit organization is being held; 
4. reference to a separate written communication, if issued under the peer review program; 
5. a statement that the peer review was conducted in accordance with GAGAS peer review requirements; and 
6. a detailed description of the findings, conclusions, and recommendations related to any deficiencies or significant deficiencies identified in the review. 

Also make sure that these requirements are followed.

• An audit organization should undergo a peer review every three years (5.84 & 5.85)
• The peer reviewer should use a risk-based approach to choose audits to review (5.66-5.71)
• The peer reviewer will conclude that your team passes the peer review, passes with deficiencies, or fails the peer review (5.72-5.76)
• The peer review report should be shared with the public. There is an exemption for internal auditors. (5.77-5.81)
• The peer reviewer and the audit organization must agree to the terms of the peer review in writing (5.86-5.88)
• The peer review team should be qualified to conduct a peer review (5.89-5.90)
• The audit organization should respond to the peer review results in writing (5.93-5.95)

Which Requirements Are New?

The GAO has included plenty of new language in the peer review section – most of it in the guidance included under the boxed requirements. But two additional requirements (should statements) are completely new:

• Section 5.86 – Written Agreement for Peer Review
• Section 5.93 and 5.94 – Audit Organization Response to the Peer Review

Here are the requirements:

Requirement: Written Agreement for Peer Review 

5.86     The peer review team and the reviewed audit organization should incorporate their basic agreement on the peer review into a written agreement. The written agreement should be drafted by the peer review team, reviewed by the reviewed audit organization to ensure that it accurately describes the agreement between the parties, and signed by the authorized representatives of both the peer review team and the reviewed audit organization prior to the initiation of work under the agreement. The written agreement should state that the peer review will be conducted in accordance with GAGAS peer review requirements. 

Requirements: Audit Organization’s Response to the Peer Review Report 

5.93     If the reviewed audit organization receives a report with a peer review rating of pass with deficiencies or fail, the reviewed audit organization should respond in writing to the deficiencies or significant deficiencies and related recommendations identified in the report. 

5.94     With respect to each deficiency or significant deficiency in the report, the reviewed audit organization should describe in its letter of response the corrective actions already taken, target dates for planned corrective actions, or both. 

The Exhausting Life of a Working Paper, Revisited

In light of all of that we know from this chapter about quality control and peer review, let’s look again at what can happen to a set of working papers. Thankfully, not all working papers will suffer this much review because some will not be selected during the monitoring procedures or the peer review. Both the monitor and the peer reviewer pick a sample of projects to evaluate.

• The staff person creates the working papers and draft report
• A supervisor reviews
• A manager, director, and/or partner reviews
• A quality control team reviews the working papers before the report is published
• The report is published
• Annually, a monitor inspects the finished working papers plus performs other procedures
• Every three years, a peer reviewer looks at the quality control system and may select the working papers for another review against standards

Wow. I’m exhausted just thinking about it. Those poor working papers must be frazzled!