How do you know your security policy and procedures are being followed?
A policy is not effective merely because it has been approved, distributed, and placed in a document repository. The useful question is: how would the organization know that everybody is following it?
That question should be answered while the policy is written. If a policy requires phishing-resistant multifactor authentication, the organization should know which system inventory or configuration report demonstrates coverage. If it requires annual access reviews, it should identify who performs the review, what records it produces, and who follows up on the exceptions. If it requires incident reporting, there should be a usable reporting path and evidence that reports receive a response.
Policy and procedure divide that work: a policy says what, and a procedure says how. The question of evidence reaches both. A policy without the procedure beneath it states an intention nobody is obliged to act on, and it is the procedure that produces the artifact the policy's claim depends on.
Start with claims that can be tested
For each important requirement, identify:
- the person or role responsible for carrying it out;
- the systems, people, or business processes in scope;
- the evidence the activity produces;
- how often someone checks that evidence;
- what happens when the evidence is absent or shows a failure.
The evidence does not always need to be a technical log—a record written by a system itself, showing that a particular event happened at a particular time, such as an authentication attempt, a configuration change, or an account creation. It might instead be a ticket, an approved review record, a training roster, a tabletop exercise report, or a sample showing that a procedure can be repeated. What matters is that another qualified person can reach the same conclusion from it.
Nor does all evidence carry the same weight. Auditing practice has long ranked it by how far it sits from the party with an interest in the outcome: what an assessor tests or observes directly is stronger than what the organization hands over, and a record a system produced in the course of its own operation is stronger than one assembled by hand for the occasion. This is worth knowing while the procedure is being written rather than afterward, because the strong kind of evidence is usually no harder to produce than the weak kind—it simply has to be chosen deliberately.
A worked example
Start with a fixed line of policy:
Remote access to all systems working with sensitive data requires phishing-resistant multifactor authentication. Coverage is verified regularly, and every exception is approved, compensated, and time-limited.
That is a good requirement, and on its own it cannot be verified. Nothing in it says who checks, against what, or how often. Those answers belong in the procedure the policy points to.
The procedure carries the operating detail the policy has no business holding: how the requirement is configured on each platform, how a new employee enrolls a security key and what becomes of the temporary credential used to do it, how the enrollment report is produced, and how an exception is requested. It also answers, explicitly, the five questions in the list above.
- Responsibility. The identity administrator configures and maintains the authentication requirement. The IT manager approves any exception.
- Scope. Every system reachable from outside the office—the identity provider and the applications behind it, the VPN, the firewall's administrative interface—and every account that can use them, including contractor, vendor support, and administrative accounts.
- Evidence. A configuration export showing which authentication methods each system permits, plus an enrollment report listing every active account and the method registered to it, compared against the current roster of people who should have access.
- Frequency. The enrollment report monthly; the configuration whenever it changes, and at least at each annual review.
- Correction. A missing enrollment is resolved or the account is disabled within a stated period. An approved exception carries a named owner, a business reason, a compensating measure, and an expiration date.
The useful part is what that comparison finds the first time someone runs it. The applications behind single sign-on are usually clean. The firewall's administrative interface may authenticate against a local account the identity provider never sees. A vendor support account may hold an exception granted years ago for a reason no one now remembers. One-time codes sent by text message may remain enabled as a fallback for people whose security keys fail—which is to say, for anyone who says their key failed.
The harder question is which systems belonged on the list in the first place. Workstations and servers are the tractable part: an organization running centralized configuration management can show that the setting is enforced and, more usefully, which machines the tool does not manage. The outsourced systems are where scope quietly ends. The HR platform holds employee records. The learning management system holds the role-based training completions that another control depends on. The external service a few engineers use to validate their designs may hold the most sensitive technical material the company owns, and was very likely bought on a card without passing through IT. Each of those is reachable from outside the office, and each is in scope only if it appears on a current inventory.
None of that means the policy is wrong. It means the policy covers less than it claims, and the organization now knows by how much and can decide what to do about each item.
A non-technical requirement traces the same way. Managers review their staff's access annually should produce a dated record for each manager naming the accounts examined, the changes requested, and confirmation that the changes were made. The evidence is a set of records rather than a configuration export, but the test does not change.
Evidence must support the policy—not replace it
A dashboard showing that a tool is installed does not necessarily prove the control is effective. A completed checklist does not prove the underlying work occurred. Evidence has to answer the policy's actual claim and cover the full scope, including exceptions.
This is one place where maturity becomes visible. A mature process does not depend on one employee remembering how things are done. Responsibilities, evidence, review, and correction are repeatable when people change roles or leave the organization.
A practical review
Choose a small number of consequential policy statements and trace each one from the document to current evidence. Ask the responsible person to explain the process, inspect a sample, and follow one exception through correction. The gaps will usually reveal whether the policy describes reality, an aspiration, or a process that existed when the document was last edited.
A policy that cannot be verified is close to useless. A policy tied to clear, proportionate evidence can guide work, survive personnel changes, and support an assessment without turning the organization into a paperwork factory.
Kenneth Ingham Consulting writes and reviews policies and procedures against exactly this test.