The policy has been approved. Staff have completed training. The latest review found no missing section. Customers still describe a step that the policy was meant to prevent. Where should the Chief Compliance Officer look next?
A policy records the institution's intended approach. The work instructions tell staff how to carry it out. Systems, queues and handoffs shape what actually happens. A gap can arise between any of these layers, even when each document looks reasonable on its own.
Complaint patterns can help locate that gap. They show where the customer's experience may differ from the expected process. Their value grows when the review connects the reported event to the relevant duty, the operating instruction and evidence of what staff or systems did.
Compare the duties behind the words
A document comparison can find a missing phrase or an old reference. The harder task is to decide whether the operating instructions preserve the meaning of the duty. Similar words can describe different actions. Different words can describe the same action well.
Consider a hypothetical policy that requires prompt review of a qualifying customer notice. A local procedure places new cases in a queue for manual assignment. A partner's script directs customers to submit the same concern through a second channel. The policy, procedure and script may all use the word prompt. Their combined effect may still be a delay.
The review needs to trace who receives the notice, which event starts the work and who acts when the usual route fails. A broad instruction to follow applicable law leaves those operating choices unresolved. A detailed procedure can also be wrong if it starts from an incorrect view of which notice matters.
Begin with one specific obligation and confirm its scope with the right specialists. Identify the covered activity, the conditions that trigger action, the steps required and any relevant timing rules. Then follow those duties into policy and into the instructions used by the people doing the work. Keep the source version and effective date visible.
A policy can restate a duty while leaving key choices unresolved. A procedure may explain the normal route but omit failed transfers or customers unable to use that channel. Review the exceptions that could prevent the duty from being met.
Use complaints to test the written account
The customer's narrative provides a way to test whether the documented process explains actual experience. In the hypothetical queue example, customers may say that each team sent them back to the other. Reviewers can compare those accounts with transfer records, receipt times and assignment history.
Several outcomes are possible. The written procedure may omit ownership after a failed transfer. Staff may be using an obsolete script. The current instructions may be correct, but the system may send the case to a queue that no one monitors. These findings require different changes.
There may also be a valid process that was poorly explained. A customer can believe no action began while the institution's records show that work was underway. That possibility should remain visible. The review must not turn every difference between a narrative and a policy into a confirmed breach.
Ask staff to walk through a real case using the instructions available at the time. Which choices required local knowledge? Where did they ask a colleague what to do? Which step happened outside the documented system? Informal workarounds can keep a process moving while hiding its dependence on particular people.
Include the partner's work where it affects the outcome. A complete internal procedure can depend on a record that the partner does not capture or a notice date it does not retain. The institution needs to understand that dependency before judging whether its own control can work.
Document agreement and operating evidence answer different questions. The first can show that the intended duties are consistent. The second can show whether those duties were carried out in the cases examined. Neither should be presented as a substitute for the other.
Keep the chain intact when the business changes
A new product or channel can change the facts that made an old procedure workable. An account process designed for branch staff may rely on a conversation that no longer occurs online. A partner change may move the point at which the institution first receives a customer concern. The policy can remain unchanged while its assumptions become outdated.
The CFPB's Compliance Management Review procedures connect change management to product changes and post-implementation review. They provide a supervisory basis for examining whether a change achieved its intended result. They do not prescribe a single method of comparing documents. [1]
A useful change record identifies the affected duties and the parts of the process that depend on them. The review can then check whether policy, procedures, scripts, system settings and training were updated together. A revised document is an incomplete milestone if an old script remains in use.
Set an operating test before closing the change. Select cases that exercise the altered step, including exceptions where they matter. Check the record of receipt, ownership and completion against the intended sequence. Where the evidence is missing, establish whether the step failed or whether the institution failed to retain proof.
Complaint patterns after the change can guide further testing. A new cluster may identify a condition that the original design overlooked. It should not be dismissed merely because the document review passed. The purpose of that review was to support sound execution, and later evidence can show where it fell short.
The CCO can begin with one recurring complaint pattern and ask for the complete chain: the duty being assessed, the instruction that should deliver it and the record showing what happened. A broken link gives the institution a concrete place to investigate. A complete chain gives another reviewer a basis for checking the conclusion.
If the policy describes the right outcome, can the institution show which daily steps produce it and how failures in those steps become visible?