Kenneth Ingham Consulting, LLC

What counts as sensitive data

2026-09-08T00:00:00-06:00

Nearly every security requirement is written in terms of sensitive data. Protect it, inventory where it lives, restrict who can reach it, know when it has left. All of that assumes the organization has already answered a question that is rarely asked directly: which data, specifically, and how does anyone know?

Organizations that skip the question tend to land in one of two places. Either sensitivity is decided by instinct in the moment, which cannot be trained, audited, or applied consistently by two people on the same day. Or everything becomes nominally sensitive because nobody decided otherwise, which sounds cautious and generally means nothing receives particular care while staff quietly work around controls that apply to the cafeteria menu.

That second one has to be separated from something it resembles closely and is entirely defensible. An organization can decide, deliberately, to hold everything to the strictest standard it is subject to. That is a real strategy with real advantages, and for some organizations it is the only sane one. It is covered further down. The failure is not treating everything as sensitive. It is arriving there by default, having never established what the strictest standard requires.

The categories are genuinely different

Sensitive data is not one thing with one owner. The common categories differ in where the obligation comes from, who defines the boundary, and what happens when it is wrong.

Regulated personal information, commonly called personally identifiable information (PII), and protected health information (PHI) where health records are involved, carries obligations created by law. The organization did not agree to them and cannot negotiate them, they follow the person rather than any contract, and they frequently reach across state and national borders to wherever the person is. What counts is defined externally and changes without the organization being consulted.

There is no single authority to consult here, which is itself the difficulty. The United States regulates personal information by sector rather than comprehensively, so which law applies depends on what the organization does. Health records fall under HIPAA, whose Privacy and Security Rules sit at 45 CFR Part 164 and whose definition of protected health information is at 45 CFR 160.103. A financial institution holding customer information is under the Gramm-Leach-Bliley Act, which states an affirmative and continuing obligation to protect the security and confidentiality of nonpublic personal information. A federal agency, or a contractor operating a system of records on its behalf, is under the Privacy Act of 1974.

Underneath all of it sits breach notification, which is state law rather than federal, exists in every state, and differs between them on what counts as personal information and how quickly notice must be given. An organization with customers abroad acquires those regimes too. The practical consequence is that this category cannot be settled by reading one document, which is why it more often needs counsel than an engineer.

Controlled Unclassified Information (CUI) carries obligations created by a contract, and its definition belongs to the government rather than to the organization holding it. This is the category most often misunderstood, because the name sounds like a description of importance. It is not. CUI is a defined set of categories with an official registry and marking rules, published by the National Archives, which runs the CUI program and maintains the registry of categories. Information either falls inside a listed category or does not. An organization's belief that something feels sensitive has no bearing on whether it is CUI, and neither does its belief that something is routine. The program itself is established by 32 CFR Part 2002, and defense contractors have a second place to look, since the DoD CUI Program publishes its own category and marking abbreviations.

CUI is also not one obligation. The regulation divides it at 32 CFR 2002.4 into CUI Basic, where the authorizing law sets out no specific handling or dissemination controls and the uniform controls apply, and CUI Specified, where the authorizing law does set out its own handling controls. Specified controls are not merely stricter; they can simply differ. Treating the two as interchangeable is the most common and most expensive mistake in this area, because the consequences are not remotely comparable.

A spill of CUI Basic is a serious matter, handled through contract mechanisms and reporting obligations. Nobody is likely to go to prison over it. Export controlled information, marked with the export control category's abbreviation, is a different proposition. Where the underlying authority is the export control regime, disclosure to a foreign person can be a criminal violation carrying substantial fines and a real prospect of imprisonment—and "foreign person" includes an employee working lawfully in the organization's own office, and a cloud administrator abroad whose employer never told the customer where support is staffed.

The practical consequence is that a single control set applied to everything marked CUI will be simultaneously too heavy for the basic categories and too light for the specified ones.

The marking is not the requirement. The marking names a category, the registry entry for that category names the law or regulation that authorized it, and that authority is what the organization is actually obliged to follow. Looking the category up is the first step and by far the easier one. What follows is reading the authority itself, working out what it requires for this data in this situation, and then complying with it—which for a Specified category can mean obligations that appear nowhere in the general CUI guidance, nowhere in the contract clause that brought the data in, and nowhere in the control set the organization has already built.

That is the work. An organization that has looked up its categories and stopped there knows the names of its obligations and has not yet met any of them.

Company proprietary information has no external authority at all. Nobody outside the organization will define it, and no registry will settle an argument about it. That makes it the category most often left undone, because deciding requires judgment rather than a lookup.

Information belonging to somebody else, held under a nondisclosure agreement (NDA) or equivalent confidentiality terms, carries obligations the organization accepted deliberately and often years ago. Customer data held to deliver a service usually lands here, and the terms are in an agreement rather than in a statute.

The practical consequence is that these cannot be handled by one policy sentence. They have different definitions, different owners, different retention pressures, and different consequences for getting them wrong.

Every contract says something about protecting data

Organizations often assume that data protection obligations arrive only with the obviously regulated work. Reading the whole contract portfolio of a mid-sized company says otherwise. Every single agreement contained a data protection requirement. Not most of them, and not only the government ones.

What varied enormously was what the requirement said. At one end, a single sentence obliging the contractor to protect the customer's information. At the other, a Department of Homeland Security contract that incorporated by reference the department's own reimplementation of the federal control catalog—a policy directive, a handbook, and roughly thirty lettered attachments, one of which tailors the federal controls and another of which carries the department's own baselines and parameter values. The clause in the contract was a few lines long. The obligation it created ran to something on the order of a thousand pages.

Both are binding, and the short one is arguably harder to satisfy. A sentence requiring appropriate protection obliges the organization to decide what appropriate means, apply that consistently, and be able to defend the decision years later to somebody whose view of appropriate may differ.

"Best practices" is not a requirement

A clause requiring best practices, industry standard care, or commercially reasonable security has the same defect as one requiring appropriate protection. It is worth naming separately because the phrase is common enough that it reads as though it means something.

Best for whom? What is right for a four-person non-profit with a donor list is not what is right for a billion-dollar multinational, and neither answer is wrong. Best when? Practice moves, and what was defensible three years ago may not be defensible now. Best against what? An organization facing opportunistic crime and one facing a well-resourced adversary have different correct answers to the same question.

The phrase has no referent, so nothing can be measured against it. Both parties sign believing they know what it means and find out they disagree at the one moment it matters. The organization cannot demonstrate compliance, an assessor cannot test it, and a dispute ends up buying expert testimony to supply a meaning the contract declined to provide.

For anyone in a position to write the clause, the fix is to name a published standard and be specific about which part of it: the CIS Controls at a stated implementation group, NIST SP 800-171 at a stated revision, or whatever fits the work and the size of the organization doing it. That makes the obligation determinable before signature, so both sides can price it, and testable afterwards, so an argument about whether it was met has an answer.

For the organization receiving such a clause rather than writing it, "best practices" is the signal to ask which standard is meant, and to get the answer in writing. Same question, same technical contact, same record.

The conditional contract

Return to that Homeland Security contract, because length was not its real difficulty either. It was written to be usable across the whole department rather than for the work actually being performed, so much of it was conditional: if the system does this, and the data is that, then the following controls apply. Determining which conditions were met was not possible from the contract, because the contract did not say. The answer had to be obtained by having the program manager ask the customer.

A contracts lawyer's description of that drafting is that it is poor practice, and the security consequence is worse than the legal one. An organization cannot categorize data whose obligations it cannot determine, and it cannot determine them by reading.

What it can do is ask, and asking well matters. The question goes to the customer's technical point of contact rather than to the contracting officer. The contracting officer administers the agreement and is often genuinely unable to say which control set applies to which data on this particular effort; the technical contact usually can, or knows who can. Organizations reverse this regularly, get an unsatisfying answer from the person whose name is on the contract, and conclude that nobody knows.

Then write down what was asked, who answered, their role, the date, and what they said, and keep it with the contract. Two reasons, and the second is the important one. The first is that the record is the only place the applicable control set will ever exist in one piece; organizations that skip it spend the next three years re-deriving it, usually during an assessment, usually from somebody who has since changed jobs.

The second is that if a spill happens later, there is an enormous difference between an organization that assumed and one that asked, was told, and can produce the exchange. It costs an email and a file. It is the cheapest insurance available in this entire subject.

The same duty, for a different reason

That was one badly drafted contract. The obligation to ask also arrives in a structural form that has nothing to do with drafting quality, and any defense contractor will recognize it.

The Department of Defense includes DFARS 252.204-7012, "Safeguarding Covered Defense Information and Cyber Incident Reporting", in very nearly every contract as a matter of routine. It requires protections meeting NIST SP 800-171 and reporting of cyber incidents within 72 hours.

Its presence, though, does not establish that any covered defense information is actually involved in the work. Plenty of contracts carry the clause and never touch CUI at all. The clause is in the contract because it is in almost every contract, not because somebody determined that this effort handles protected information.

So the determination falls to the contractor, and it is the contractor's problem in both directions. Markings are not always applied, and when applied are not always applied correctly. The government does not reliably volunteer which categories are in play. Assume CUI is present when it is not, and the organization has built and priced a control set nobody required. Assume it is absent when it is present, and the organization has a spill, a reporting failure, and a contract problem, in that order and on a 72-hour clock.

The answer is the one above, for the same reasons: ask the technical point of contact, get the answer in writing, and keep it beside the contract. A clause that appears in every contract tells an organization nothing about its own work. Only asking does.

The practical lesson is that the question is not which contracts mention security. They all do. The question is what each one actually requires, and incorporation by reference is where that hides: the clause is short, and the obligation is not.

Who decides, and who does not

For regulated information and CUI, the organization does not get a vote on the definition. It only gets to decide where the data is, who touches it, and how it is protected. Arguing about whether something ought to count is time spent on a question that has already been answered elsewhere.

Proprietary information is the opposite: the organization is the only authority that exists, so a decision has to be made rather than discovered. The useful test is to name the harm. What would it cost, and to whom, if a competitor had this, or a customer, or the public? Information that survives that question belongs on the list. Information that does not is ordinary business information, and saying so out loud is what makes the protected list credible.

"Everything is confidential" is the same statement as "nothing is confidential", delivered with more confidence. That is a claim about what the information is, which is a separate question from how much of it to protect. An organization can conclude honestly that very little of its information is proprietary and still choose to handle everything to one standard, for reasons covered below.

A label has to change something

A classification scheme with four levels and one set of handling rules is decoration. Each label has to change what happens: where the data may be stored, who may see it, whether it may leave the organization, what may be done with it after the work is finished, and how it is disposed of.

The test is straightforward. Take any two labels and describe how handling differs between them. If the answer is difficult to produce, there is one label wearing two names, and the scheme should be simplified until every distinction does work.

Short schemes get followed. A scheme with three levels that people can recall without looking is applied far more consistently than a scheme with seven that requires a reference table, and consistency is most of the value.

One level is a legitimate answer

Taken to its conclusion, that reasoning sometimes lands on a single level. Find the strictest requirement the organization is subject to, apply it to everything, and stop classifying. For some organizations this is not laziness but the correct decision, and it deserves stating clearly because the literature tends to treat classification schemes as self-evidently good.

The advantages are substantial. There is one control set to build, document, and be assessed against. Nobody has to classify anything during ordinary work, which means nothing gets misclassified, and misclassification is the failure mode that actually causes spills. Training is one conversation rather than a decision tree. An assessor sampling any system finds the same controls, so evidence from anywhere is evidence about everywhere. For a small organization holding one regulated category, the cost of building and maintaining two handling regimes usually exceeds the cost of over-protecting the rest.

The cost is equally real and it grows with scale. Applying controls designed for the most sensitive category to everything means encryption, access restriction, retention limits, and disposal requirements on the cafeteria menu. That is money, and it is friction, and past a certain size it becomes an obstacle to doing the work. The point where it stops paying is roughly where the protected material becomes a small fraction of the whole, and the overhead on everything else exceeds what maintaining a boundary would cost. That is when a second level earns its keep.

Someone still has to track the floor. Whichever way the organization goes, the strictest applicable standard has to be identified and then watched, because it moves when a new contract is signed or a new line of business opens. Choosing a single level is a decision about handling, not an exemption from the analysis in the rest of this article. An organization that applies one standard everywhere without knowing which standard that is has not simplified anything; it has guessed, and it will find out whether it guessed high enough at the worst possible time.

Where the tooling question comes in, and where it does not

Some of this sounds like a description of data loss prevention (DLP) software, and it is worth being direct about the relationship. DLP tools watch data in motion and at rest—outbound email, uploads, removable media, cloud storage—and act on what they find, either by blocking it, quarantining it, or recording it for review. Some read the labels a classification scheme applies. Others infer sensitivity from the content itself, matching patterns such as account numbers or the markings on a document.

The important point is the order of operations. Such a tool enforces a decision about what is sensitive and where it is permitted to go. It does not make that decision, and it cannot be configured by an organization that has not made it. Buying one first produces either a policy of blocking nothing, or months of false positives that end with the tool being switched to monitoring and quietly ignored.

There is also a category of data these tools handle badly, and source code is the clearest example. Consider an organization holding a large body of source code: some written in-house, some open source under a permissive license, some under a reciprocal license with obligations attached, and some received from a customer or a partner under terms of its own. Those files carry genuinely different restrictions on where they may go and who may see them. Nothing in the text distinguishes them. Sensitivity here is a property of provenance and license, not of content, and content inspection is what these tools do.

The usual answer is that the files should be labeled, and this is where the approach quietly fails. Labeling that volume of code is work nobody has budgeted, it has to be redone as files are added, moved, refactored, and vendored in, and it depends on every developer applying the right label every time. No program manager is willing to spend the team's time on that, and saying so is not a criticism of the program manager. A control that requires sustained voluntary effort from people measured on something else stays accurate for about a quarter, if that long.

Where content inspection cannot categorize the data, the answer is usually to control the container rather than the contents: which repositories exist, where they are hosted, who can clone them, and what may leave. That is an access control problem with a reliable answer, rather than a classification problem with an unreliable one. Provenance and license are worth tracking too, but that is a software composition question answered by different tooling than the kind that watches outbound traffic.

DLP software is also not required. Nothing in this article assumes such a product exists. A small organization whose staff know which categories exist, where each is allowed to live, and who to ask when unsure may be genuinely well controlled with no tooling at all, and an assessor can be shown that through the procedures people actually follow. Tooling earns its place as scale grows, turnover rises, or the consequence of a single mistake becomes large enough that relying on everyone remembering stops being reasonable. That is a judgment about the organization and its people, not a requirement that follows automatically from holding sensitive data.

The categories overlap, and that is normal

A personnel record is both regulated personal information and company proprietary information at the same time. The salary, the performance review, and the disciplinary history are personal data about an identifiable employee, carrying whatever the applicable privacy law requires: access rights, retention limits, and notification duties if it is disclosed. The same file is also information the organization has its own reasons to protect, because a compensation structure is useful to a competitor recruiting from it. Treating the record as purely an HR privacy matter tends to secure it against outsiders while leaving it readable across the company. Treating it as purely proprietary tends to protect it well and ignore the employee's rights over it entirely.

A contract deliverable can be CUI and somebody else's confidential information at once. A report written for a defense customer may contain government information that falls in a CUI category, and alongside it the technical data a subcontractor supplied under a nondisclosure agreement. The CUI obligations arrive through the prime contract and specify how the information is stored, marked, and reported on if spilled. The subcontractor's obligations arrive through a separate agreement that may forbid disclosure to named competitors, require return or destruction at the end of the engagement, and say nothing about marking at all. Neither set is a subset of the other.

When categories overlap, the handling is the union of the obligations rather than an average of them. The strictest storage requirement applies, the shortest retention limit applies, and the broadest notification obligation applies. Organizations sometimes try to resolve overlap by picking the label that fits best, which reliably discards one of the obligations—usually the one that came from the quieter party, because the government sends assessors and the subcontractor does not.

Notification duties travel with the data

Every category above carries some duty to tell somebody when things go wrong, and those duties are the part most often missing from a categorization record. They get left out because they are not about protecting the data. They are about what happens once protecting it has failed, which feels like a different subject and is not: the obligation attaches to the category, arrives with it, and travels with it wherever it goes.

They also multiply in ways that surprise people. Federal agencies do not share one reporting requirement; a defense contract, a health regulator, and a grant-making agency each have their own trigger, their own recipient, and their own clock. Within a single state, different agencies can impose different duties on the same organization for the same event. And a commercial contract can add its own on top of all of it, frequently on a shorter deadline than any statute, because a customer who wants to be told within twenty-four hours can simply write that down and will.

The practical consequence is that "who has to be told, and how quickly" has a different answer for each category, and sometimes several answers for one category held under several agreements. Working that out while an incident is in progress is the worst available time, and it is when most organizations first attempt it.

Breach notification in its own right is a large subject and not this one. The point here is narrower: the duties belong in the categorization record next to the handling rules, because they are a property of the data rather than a property of the incident, and an organization that has categorized its data without recording them has left out the part with a clock attached.

Where it usually goes wrong

The copies are forgotten. The database is identified and protected while the extract someone made for a report, the backup, the test fixture built from production, the email attachment, and the meeting transcript are not. Data does not become less sensitive by being copied somewhere less formal, though it almost always becomes less protected.

Aggregation changes the answer. Fields that carry no obligation individually can identify a person when combined, and a list of ordinary facts about a project can describe a capability the organization would rather not publish. Sensitivity is a property of the collection, not only of the field.

Nobody owns a category. A list of categories with no named owner produces no decisions when an unusual case appears, and unusual cases are exactly when the decision matters.

The list is written once. New obligations arrive with new contracts and new lines of business, and a classification scheme that has not been revisited since a contract was signed is describing the organization that existed then.

That last one is why the major control standards do not treat this as a one-time exercise. They require the categorization to be reviewed on a defined schedule and again when something changes, and they say so in more than one place: in the requirements covering how information is categorized in the first place, in the system inventory requirements that expect the record to stay current, in the media and information handling requirements that determine marking and storage, and in the periodic review of policies and procedures. An organization that categorized its data once and has grown two lines of business since is not partially compliant with those. It is describing a company that no longer exists, and the assessment will be against the one that does.

What this should produce

A record naming each category of sensitive data the organization actually holds, and for each one: where the obligation comes from, who owns decisions about it, and what handling follows. That is enough to train against, enough to audit against, and enough to answer the question that starts every serious security conversation.

The word to resist is "document", because it suggests something written once and filed. This is a live record. Every new contract can change it, and most organizations sign contracts more often than they revise policies. Two fields carry most of the weight and are the ones usually missing.

The contract reference. Each obligation should name the agreement it came from, by number or identifier, because obligations end. When a contract closes, the requirements that arrived with it may lapse, and an organization that cannot trace which requirement came from where will keep applying all of them forever. That is a slow, invisible, permanent cost increase that nobody ever decides to accept.

The effective dates. When the obligation started, and when it ends or comes up for renewal. Without dates there is no way to answer what applied at a given moment, which is exactly the question asked after an incident, during a dispute, and in any assessment covering a period rather than a day.

The notification duties, as described above: who has to be told, on what trigger, within what period. This is the field with a clock attached, and the only one whose absence is discovered under time pressure.

For a small organization a maintained table is entirely adequate. Past a certain number of contracts, or when the same data category arrives under several agreements with different terms, a table stops working and a small database is the honest answer. The trigger is usually not size but overlap: the first time somebody has to determine which of three customers' terms governs a particular file, a spreadsheet has already failed.

It is also the prerequisite for nearly everything else. An inventory cannot record what the organization has not defined, a risk assessment cannot weigh consequences for data nobody has categorized, and an incident response plan cannot tell anyone whether an event is reportable. Those all depend on this record existing, which is why it is worth doing before the work that assumes it.

The control standards agree, in the sense that matters: categorization sits near the front of every one of them, and the requirements that follow are written as though it has already been done. An organization working through a control set from the top will meet the categorization requirement early, and an organization that skipped it will find every later requirement asking a question it cannot answer.

Kenneth Ingham Consulting helps organizations decide what their sensitive data actually is, and what handling each category should carry.

Not all evidence is equally believable

2026-09-01T00:00:00-06:00

Two organizations can hand over evidence for the same requirement and get noticeably different receptions. That is not inconsistency on the assessor's part. Evidence differs in how much weight it can bear, and audit practice has ranked it that way for a very long time.

The ranking is worth understanding before an assessment, because an organization has more control over what kind of evidence it produces than it usually realizes.

Roughly strongest to weakest

What the assessor obtains directly. A configuration the assessor reads from the system, a test they run, a process they watch being performed. Nothing sits between the fact and the person forming the conclusion.

What a system produced while doing its job. Logs, automatically generated reports, ticket histories with system timestamps. These were created for the organization's own operational purposes, not for the assessment, and they are usually inconvenient to alter selectively.

What the organization compiled and handed over. Exports, screenshots, spreadsheets assembled to answer the question that was asked. These might be entirely accurate, and they were produced by someone who knew what answer would be convenient.

That knowledge is the difficulty, and acting on it requires falsifying nothing. Someone who knows what the answer is supposed to be makes a series of small choices while gathering the data—which systems to pull from, which date range to cover, how to word the query, which of several exports to attach—and each of those choices has a defensible rationale and a direction. The result can be truthful in every particular and still not represent the population it appears to describe. It is also the kind of bias that does not feel like bias to the person doing it, which is why an assessor would rather specify the sample than receive one.

What someone says. An explanation of how the process works. Useful for understanding, necessary for context, and the weakest thing on which to rest a conclusion.

The principle underneath

Two things move evidence up or down that list.

The first is distance from the party with an interest in the result. This is not an accusation of dishonesty, and it is worth being clear about that, since organizations sometimes hear it as one. The ranking exists because incentives exist. A method that only works when nobody is tempted is not much of a method, so audit practice assumes the incentive and prefers evidence that does not depend on the auditee's disinterest.

The second is whether the record was created for its own purposes or for the assessment. A report the operations team has produced monthly for three years, and acted on, says something a report generated the week before the assessor arrives does not—even when both contain the same numbers.

Two related factors adjust the weight further. Contemporaneous records beat reconstructed ones: something written when the work happened is stronger than something assembled afterward from memory. And corroboration matters, because two independent sources agreeing is stronger than either alone, particularly when one of them is a system record and the other is a person's account.

The idea is not unique to security

ISACA's CISA material is where many security practitioners first meet this ranking, but it neither originated there nor is peculiar to information systems audit. Financial and internal audit reached the same conclusions, in some cases decades earlier, and phrase them in strikingly similar terms.

The AICPA's audit evidence standard, AU-C 500, long set out generalizations that will look familiar: evidence obtained directly by the auditor is more reliable than evidence obtained indirectly; evidence from independent sources outside the entity is more reliable than evidence from within it; and original documents are more reliable than photocopies or converted electronic copies, whose reliability depends on the controls over the conversion.

Statement on Auditing Standards No. 142 rewrote that section, effective for periods ending on or after December 15, 2022, and replaced the ranking with attributes of the information itself: accuracy, completeness, authenticity, and susceptibility to bias. That is less a reversal than a generalization of the same reasoning. Directness and independence were always proxies for those attributes, and naming the attributes directly travels better to forms of evidence the older wording never anticipated. The PCAOB's AS 1105 makes the same point for public company audits, tying reliability to the nature and source of the evidence and to whether that source is knowledgeable and independent of the company. ISA 500 covers the same ground internationally.

Internal audit frames it as properties rather than a ranking. Under the IIA's 2017 framework, standard 2310 required auditors to identify information that is sufficient, reliable, relevant, and useful, and defined sufficient as factual and convincing enough that a prudent, informed person would reach the same conclusion. The 2024 Global Internal Audit Standards, mandatory since January 2025, carry that requirement into Standard 14.1 on gathering information for analyses and evaluation, framed as relevance, reliability, and sufficiency. The numbering changed; the reasoning did not.

The convergence is the interesting part. Bodies with different mandates, overseeing different work, arrived at the same handful of principles: prefer what the auditor obtained directly, prefer what came from a source with no stake in the answer, prefer the original to the copy, and have enough of it to carry the conclusion. An organization assembling evidence for a CMMC assessment is being measured against reasoning that predates CMMC by a long way, which is also why the reasoning is unlikely to shift under it.

What this means for the organization

The useful consequence is that evidence strength is largely a design decision, made when procedures are written rather than when an assessor asks.

Prefer output a system generates on a schedule over something a person assembles on request. The first is stronger and, after setup, less work.
Keep what makes a record self-describing: the timestamp, the system it came from, the account that produced it, and the period it covers. An undated screenshot with no hostname is a picture of a screen.
Retain records where they cannot be quietly edited, and where the retention period outlasts the assessment window.
Where an assessor can be given read-only access to a live system rather than an export, offer it. It is stronger evidence, and it means the assessor is not carrying a copy of the organization's data.
Keep the record of the work, not only the result. That a review was completed is weaker than the dated record naming what was examined and what changed.

None of these is expensive when chosen at the outset. Retrofitting them under assessment pressure is expensive, and it produces exactly the reconstructed, compiled-on-request evidence that carries the least weight.

Weak evidence is not useless

An organization that can only offer a spreadsheet and an explanation has not failed. But it should understand what it has just set in motion.

The mechanism is simple, and worth stating as plainly as possible. When an assessor receives strong evidence that a requirement is met, the item is settled and they move to the next one. When the evidence is weak, or has to be produced during the assessment because it did not already exist, they do not move on. They ask another question, request another sample, look at a second system, and start wondering what else is in the same condition.

Nothing about that is punitive. An assessor's job is to reach a defensible conclusion, and weak evidence has not given them one yet, so they keep going until something does.

The cost lands in two places. The obvious one is time, and assessment time is expensive. The less obvious one matters more: a longer examination covers more ground than a short one, and findings are discovered in the ground that gets covered. Weak evidence rarely produces a finding by itself. It produces the extended look during which the actual findings turn up.

The organization that finishes quickly is usually not the one with fewer problems. It is the one that could answer each question the first time it was asked.

Kenneth Ingham Consulting reviews what an organization's procedures actually produce and what that evidence will be worth when someone independent examines it.

What an annual security policy review should produce

2026-08-25T00:00:00-06:00

An annual policy review should be more than changing the date and collecting a new approval signature. Technology, contracts, staff, vendors, threats, and business processes change during the year. The review is the point at which the organization checks whether the document still describes what it needs to do and what it actually does.

Why annual, and not merely "regularly"

Organizations sometimes treat the yearly cycle as a convention rather than a requirement. It is a requirement, in nearly every framework an organization is likely to be measured against, and the wording converges to a striking degree.

NIST SP 800-53 puts a review obligation in the first control of every family—AC-1, AT-1, AU-1, CA-1, CM-1, IA-1, IR-1 and the rest—each requiring that the policy and procedures be reviewed and updated at an organization-defined frequency, and following organization-defined events. It names neither the frequency nor the events itself, leaving both to be filled in elsewhere. Note that both halves of the pattern are already present in the most general of these frameworks: a schedule, and a trigger that does not wait for it.

That phrase—organization-defined—is misleading, and worth pausing on. It suggests the organization being assessed chooses the value. Usually it does not. The customer, the regulator, or the framework chooses, and the contractor inherits the result. An organization under no external obligation is genuinely free to pick. Every other organization should find out what has already been picked for it.

NIST SP 800-171 carries the requirement forward for organizations handling controlled unclassified information, and supplies the clearest example of that inheritance. In Revision 3 the requirement is 03.15.01: develop, document, and disseminate the policies and procedures needed to satisfy the requirements, then review and update them at an organization-defined frequency.

Fifty of Revision 3's requirements carry such a parameter, and for defense contractors they are not open questions. In April 2025 the department published DoD Organization-Defined Parameters for NIST SP 800-171 Revision 3, defining the values as policy in preparation for making Revision 3 the minimum requirement for contractors.

For the review requirement, the value is explicit. Parameter 03.15.01.b is assigned:

at least every 12 months, or when there are significant incidents or significant changes to risks

The memorandum defines significant in a footnote, as "having or likely to have influence or effect." Both halves of the pattern are there again: a maximum interval, and an event trigger that does not wait for it. Nor is twelve months a one-off choice—it is the value the department assigns to nearly every frequency parameter in the document.

A contractor reading Revision 3 should therefore read that list beside it. The values are a floor rather than a suggestion, and only in four instances did the department leave guidance in place of a specified value.

CMMC rests for now on Revision 2, where the obligation is distributed across the requirement families rather than stated once, and where the interval arrives through a definition rather than a parameter. The CMMC Assessment Guide for Level 2 defines the term the requirements use:

Periodically: Occurring at a regular interval as determined by the OSA that may not exceed one year.

That is the sentence to cite when someone proposes a longer cycle, and it also marks where the organization's discretion begins and ends. Anything shorter than a year is genuinely its own choice; a year is the outer limit of what it may choose. The assessment and scoping guides for every level are published on the DoD CIO's CMMC documentation page.

CIS Controls state it plainly and repeatedly. In version 8.1.2, twenty-one Safeguards close with the same clause:

Review and update documentation annually, or when significant enterprise changes occur that could impact this Safeguard.

Only the object varies—documentation, content, the inventory, the policy, the classification scheme. The trigger never does. Most of those Safeguards carry the Govern security function that version 8.1 introduced, which is CIS saying outright that this is governance work rather than paperwork.

Two others, 15.4 and 16.6, require the annual review without the event trigger. That is the exception worth noticing: a schedule with no trigger is the weaker of the two halves, and it is rare. The Controls themselves are free; downloading them requires registration.

PCI DSS is the most explicit of the group. In version 4.0, requirement 12.1.2 sets the interval at least once every twelve months, with updates as needed to reflect changes to business objectives or risks to the environment. Requirement 12.1.1, immediately above it, is the separate obligation to establish, publish, maintain, and disseminate the policy in the first place.

Version 3.2.1 numbered the review requirement 12.1.1, which is worth knowing before quoting a number from memory. A citation carried forward from the older standard now lands on the adjacent requirement.

ISO/IEC 27001 is the outlier, and instructively so. Annex A 5.1 asks that policies be reviewed at planned intervals, and additionally if significant changes occur, without naming an interval. Organizations are free to choose, and the interval they choose is almost always a year, because a longer one is difficult to defend to a certification auditor.

The convergence is the point. Two things recur everywhere: a maximum interval, stated outright by PCI DSS and CMMC and left to the organization by the others, and an event trigger that does not wait for it. A review conducted only when something happens has no floor, and a review conducted only on schedule misses the reorganization in March. Both are required.

An assessor's note on dates

Assessors read review dates as a signal about the organization, not only as a compliance fact.

A policy stating that it is reviewed annually, with a history showing intervals of about eleven months, reads well. It says someone owns the calendar and schedules the work with room to spare.

A history showing 366 days between reviews in a non-leap year reads quite differently. It is one day over, and it will rarely fail an assessment by itself. What it says is that the review happened because the deadline arrived, that nobody noticed it had already passed, and that the organization has no margin anywhere in the process. An assessor who sees that will tend to look more carefully at other practices, because the same habit of just-barely tends not to appear in only one place.

The remedy costs nothing: schedule the review at ten or eleven months and treat the anniversary as the deadline rather than the appointment.

Inputs to the review

The reviewer should consider what has changed since the previous approval. Each category below has a place it ought to be found, and the reviewer needing to ask around for it is itself a finding.

Laws, regulations, and contract clauses

New or changed laws, regulations, grants, customer terms, and standards all bear on whether the policy still says the right thing.

Where to look depends on the obligation. Legal or outside counsel should be tracking statutory and regulatory change for the jurisdictions and sectors the organization operates in. Sector regulators and state attorneys general publish changes directly. Trade associations and sector information sharing organizations frequently summarize them earlier and more usefully than the primary sources. For anything federal, the contracting activity is often the first to say so in writing.

Contract clauses deserve separate treatment, because they are the obligations most often missed. A cybersecurity requirement arriving through a contract or a flow-down is as binding as a regulation and considerably easier to overlook, since it enters through the sales or contracts function rather than through legal or IT. Any clause carrying a security requirement should already be tracked before signature—that is the point at which the organization can still decline it, price it, or negotiate it. The review then works from that register rather than from a fresh reading of every contract.

An organization without such a register has found its first piece of work. It is also, usually, the reason a requirement surfaces for the first time during an assessment.

Changes to systems, data, vendors, and roles

Changes to systems, data, locations, vendors, and organizational roles all potentially invalidate policy language.

These should be discoverable from the change control system rather than from memory. If the organization operates change management with any discipline, the year's changes are already enumerated, dated, and attributed, and the reviewer's job is to read the log rather than to reconstruct the year. Vendor and role changes may sit elsewhere—procurement records, the HR system—but the principle holds: the review consumes records that already exist.

Where the change log cannot answer the question, the gap is worth recording. A policy review is a reasonable place to discover that change control is not capturing what it should.

Incidents and exercises

Incidents and tabletop exercises belong together, because they answer the same question from opposite directions: what actually happens when the plan meets events.

Both should arrive with reports. An incident's final report says what occurred, what the response revealed, and what was recommended afterward. A tabletop exercise report does the same for a scenario that was not real, which is the cheaper way to learn it. Either may show that a policy requirement is unworkable in practice, that a responsibility is assigned to a role that cannot discharge it, or that a procedure everyone believed existed does not.

Recommendations from those reports that were never implemented are especially worth surfacing at review time. A recommendation made after an incident and quietly dropped is a decision the organization made without recording that it was making one.

Audits, assessments, and self-assessments

Audit findings should likewise arrive with a report, and the review should consider both the findings and what became of them.

Beyond external audits are self-assessments, whether conducted internally or by an outside firm. For defense contractors these are not optional exercises. CMMC requires an annual affirmation, submitted by a senior official under 32 CFR 170.22, that the organization has implemented and will continue to maintain the applicable requirements. That affirmation is a legal statement, not an administrative one.

The consequences of getting it wrong are worth stating plainly, because they are frequently underestimated. Misrepresenting compliance status to the government can expose the affirming official to criminal prosecution under 18 U.S.C. 1001, which addresses false statements to federal agencies and carries the possibility of imprisonment. Separately, the organization faces civil liability under the False Claims Act, which the Department of Justice has pursued against cybersecurity misrepresentation specifically since establishing its Civil Cyber-Fraud Initiative in 2021. Because the affirmation recurs every year, so does the exposure.

None of that argues for pessimism in the affirmation. It argues for knowing the answer before signing it, which is what a genuine self-assessment produces and what an optimistic one does not. The policy review is one of the places that knowledge is assembled or found to be missing.

Exceptions and accepted risks

Exceptions and accepted risks are the same subject seen from two angles, and both belong in the review.

A documented exception should already describe what it is an exception to, the risk that accepting it creates, the compensating measures taken, who approved it, and when it expires. Reviewing those is straightforward when the documentation is good: are they still needed, are the compensating measures still in place, and has anything expired without being noticed?

The rest of the organization's risk is a larger question, and it is answered by a risk assessment rather than by the policy review. Many standards require one periodically, CMMC and NIST SP 800-171 among them—which, under the CMMC definition above, means at least annually—and the sequencing matters. A policy review informed by a current risk assessment can ask whether the policy addresses the risks the organization actually has. One conducted without it is reduced to asking whether the document is internally consistent.

Measurements and evidence

Measurements and samples showing whether important requirements were actually met are the difference between reviewing a document and reviewing a practice. This is the same question as whether the policy can be verified at all, asked a year later and with a year of evidence available.

The evidence is often not named in the policy at all, and it usually should not be. The policy states the requirement; the procedure beneath it says what performing that requirement produces, where the artifact lands, and who checks it. So the reviewer's question is not whether the policy identifies its evidence, but whether the policy and its procedures together do.

That question is easy or hard depending on one thing. Where the organization has cross-referenced its documents—each policy statement pointing to the procedures that carry it out, each procedure naming the policy it serves—the reviewer follows the requirement to the procedure to the artifact, and the review moves quickly. Where those links do not exist, the reviewer is reconstructing which procedure implements which requirement before any evidence can be examined, and that reconstruction is the bulk of the work.

Where evidence does exist, the review is also the moment to ask how believable it is. A year of operation is long enough to show what a procedure really produces, and a review that never asks how strong those artifacts are will keep renewing its confidence in the weakest of them.

If no procedure produces evidence for a consequential requirement, that omission is the most useful finding the review will generate.

Overlaps and contradictions

Policies and procedures that overlap or contradict one another need to be reconciled, and the review is where that happens. This is also where confusion between the two documents becomes expensive, since a policy says what and a procedure says how, and a requirement that has migrated into a procedure has escaped the approval authority that was supposed to govern it.

Planned changes

Finally, planned changes that will make current language inaccurate. A policy rewritten to match a system being replaced next quarter has been written twice.

Where to look for these is genuinely organization-specific, and size drives the answer more than anything else. A small organization can find them by asking the handful of people who make such decisions, and that conversation is reliable because there are few enough of them to ask. A larger one needs deliberate sources: the IT and capital project portfolios, the procurement pipeline, budget submissions for the coming year, merger and acquisition activity, facility plans, and whatever governance body approves significant initiatives.

The failure mode differs by size too. Small organizations miss planned changes because nobody wrote them down. Large ones miss them because the person doing the review is not in the room where they were decided.

Speaking with the people who perform the work is essential regardless of size. A document owner may know what the process is supposed to be; operators know where it is unclear, impractical, bypassed, or dependent on undocumented knowledge.

What the report should say

A review that produces no record has not been performed, whatever was discussed. The report is the artifact, and next year's reviewer is one of its readers.

The policy identified precisely. Its title, version, effective date, and scope, along with the reviewer's name and the date the review was conducted. A report that does not identify which version was examined cannot be relied on later, and the review interval cannot be calculated from it.

The evidence and people consulted. What was read, which records were sampled, and who was interviewed. This is what allows someone else to judge how thorough the review was, and what lets a subsequent reviewer avoid repeating work or, more importantly, notice that a source consulted last year was skipped this year.

What is working and remains appropriate. Confirming that most of a policy is correct is a finding, not filler. It records that the language was examined and found accurate rather than merely left alone, and it is the part that makes "no change" defensible.

What should change, and why. Each recommendation with its reason, tied to the input that prompted it: a new contract clause, an incident report, a measurement that came back short. A recommendation without a stated cause is difficult for an approver to weigh and impossible for a successor to understand.

Conflicts, gaps, exceptions, and risks found. Including the uncomfortable ones. A review that surfaces nothing is either describing an unusually mature organization or was not really conducted, and assessors know the ratio between those two.

An owner and a target date for every follow-up action. Not "the IT department" and not "as soon as practical." A named role and a date, because work assigned to everyone is assigned to no one, and this is the part of the report that will be checked next year.

The approval decision and the next scheduled review. Who approved the outcome, on what date, and when the next review falls due—scheduled with margin rather than on the anniversary.

Not every review needs to produce a rewritten policy. "No change" can be a valid result when the report shows what was checked and why the existing language remains accurate.

Conversely, approving edits is not enough when the review uncovers operational work. Those actions need owners and follow-up outside the document workflow, and for anything that cannot be fixed immediately, that means a plan of action and milestones rather than an intention. The distinction matters: an unremediated gap with a documented plan, an owner, and a date is a managed problem, and an unremediated gap without one is simply a gap that has now been written down.

Review related documents together

Policies form a system. An AI policy may depend on privacy, data protection, acceptable-use, procurement, incident response, and records-management rules. Reviewing one in isolation can introduce contradictions or leave a new responsibility without a procedure.

The review is also the natural moment to have procedure owners confirm that their procedures are still accurate—that the steps match what is done, that the named roles still exist, and that the tools referenced are the tools in use. A procedure nobody has read in a year is usually wrong in at least one particular, and the owner is the person who will notice.

That confirmation is worth more when it produces something. Asking each owner to generate a current artifact—the report the procedure says it produces, a dated sample, a fresh export—turns an assertion that the procedure works into evidence that it does, and does it once a year at a predictable time rather than under assessment pressure.

Approval has to mean something

A policy review ends in an approval, and who signs matters as much as that someone did.

The approver must hold enough authority to require compliance from everyone the policy's scope covers. That is the whole function of approval: a policy binds people, and only someone with authority over those people can bind them. A security policy approved by the security manager governs the security team. A security policy that reaches every employee, contractor, and vendor with access needs an executive, an owner, or a board—someone whose authority is not questioned by the department that finds the requirement inconvenient.

This is also where scope and approval have to be checked against each other. A policy whose scope has grown during the review—now covering contractors, or a newly acquired division—may have outgrown the person who approved it last year.

An approver who cannot explain what the organization has committed to has been handed the wrong document, which is usually a sign that the policy has drifted into procedural detail.

The review as a maturity test

The annual review is a useful measure of the organization quite apart from the document it produces. If the organization cannot identify current ownership, evidence, exceptions, and prior corrective work, the problem is larger than stale wording. The report should make that visible rather than hiding it behind a renewed approval date.

The reviews that go quickly are the ones where every input already existed: the contract register, the change log, the incident reports, the risk assessment, the evidence the procedures were supposed to produce. The reviews that take weeks are the ones spent assembling those things for the first time. Which kind an organization has is not really a question about its policies.

Kenneth Ingham Consulting reviews existing policies and procedures and delivers a report of this kind: what is working, what should change, and why and how.

The systems inventory: local machines are the easy half

2026-08-18T00:00:00-06:00

Almost every security requirement contains a hidden dependency: the list of systems it applies to. A control can be implemented well, verified regularly, and still leave the organization exposed, because the verification ran against an inventory that was missing something. That makes the inventory worth treating as a deliverable in its own right, rather than as a spreadsheet someone assembles the week before an assessment.

The half that tooling solves

Workstations and servers are comparatively tractable, because the tool that configures them can usually enumerate them. An organization running centralized configuration management—the Unix-oriented tools or their Windows equivalents—can demonstrate that a setting is enforced across the estate and produce the list of machines it was enforced on.

The valuable output of that tool is not the list of managed machines. It is the difference between that list and reality: the laboratory system nobody enrolled, the contractor's laptop, the server stood up for a project that ended two years ago, the machine that has not checked in for months. Those gaps are findable, because the organization owns the network the machines sit on and the directory they authenticate against.

Finding that difference means holding several lists against each other, because each is built from a different vantage point. Configuration management knows what it manages. DHCP logs know what asked for an address, which includes everything the configuration tool never heard of. The vulnerability management system knows what it scanned. A network scan or switch tables know what is actually connected. And the inventory records what the organization believes it has.

In practice those lists rarely agree. That is not by itself alarming: a machine built this week may legitimately appear in some and not yet in others, and those short-term differences resolve as normal onboarding catches up. The signal worth acting on is duration. A system that has appeared in the DHCP logs and the scanner for months, and has never appeared in the inventory, is not a timing artifact. It is a system nobody is accounting for, and it has been running that way long enough for the omission to be the normal state.

Reconciling the lists periodically, and asking about anything that persists in one and not another, turns four partial pictures into something close to a complete one.

Access control at the network layer narrows the problem from the other direction. A small network can require that only known hardware addresses be permitted to connect, which is administratively simple at that scale and makes an unknown device conspicuous. Larger networks need something more robust than a list of hardware addresses, which are straightforward to observe and impersonate; the usual answer is port-based network access control to the IEEE 802.1X standard, authenticating the device or its user before the port carries traffic.

Straightforward is not the same as free. It still takes a decision about who owns enrollment and what happens when something appears unmanaged. But the mechanism exists, and the answer is knowable.

The layer below the machine

Most inventories stop at the machine and its installed applications. One level below that sits software the organization rarely counts and frequently cannot list at all: the extensions loaded into browsers, code editors, and office suites.

They deserve counting because they behave like separate systems. An extension has its own vendor, its own update channel that usually operates automatically and without review, its own permissions, and often its own network connections. A browser extension permitted to read and change data on every site can see everything the person using it sees. An editor extension can read an entire source tree, execute code, and talk to a remote service, and it is typically installed by a developer in a few seconds because it looked useful. Office add-ins sit in the same position with respect to documents, as do plugins in engineering and laboratory applications.

The practical consequence is that an inventory can be accurate about every machine and still be unable to answer what has access to the source code or the contract files. Browser extensions can be enumerated through managed browser policy. Editor extensions are harder, because they live in per-user directories rather than in the system package manager, so this usually means pointing the endpoint software inventory at those locations deliberately.

An organization that has never looked will find the answer surprising, which is the ordinary reason to look.

The half that tooling does not solve

The harder half is the systems the organization pays for and does not run.

Consider three that most organizations have. The outsourced HR platform holds employee records, and the payroll and benefits data around them. The learning management system tracks role-based training completion—which is to say, it holds the evidence for a control the organization has almost certainly committed to somewhere. The external design validation service that a handful of engineers use may hold the most sensitive technical material the company owns.

What those three have in common is more instructive than what they do. None appears in a configuration management console. Each may or may not authenticate through the corporate identity provider. And in each case the department that bought it thinks of it as a subscription rather than as a system, which is exactly why it does not appear on a list of systems.

How much IT knows varies, and the variation is the point. IT was probably consulted about the HR platform, since it had to be connected to something. It may have been consulted about the learning management system. It may know nothing at all about the service the engineers use, which was very likely bought on a card, approved by a manager who saw a reasonable business expense, and never described to anyone as a system.

That last case is shadow IT, and it is not a discipline problem. It is what happens when people who have work to do find a tool that does it faster than the procurement process moves. The organization still owns the consequences.

The design validation service is the instructive one. It is likely the smallest line item of the three, used by the fewest people, purchased with the least ceremony—and holding the material a competitor or a foreign intelligence service would most want.

Finding what was never registered

The methods that surface unregistered systems are the same ones that surface unregistered AI use: expense and corporate-card records, identity provider application lists, web proxy and DNS records, contract and renewal files, and conversations with departments about the work they are actually trying to get done.

No single source is complete. Purchasing records miss anything free. Identity provider records miss anything with its own local accounts, which tends to be the systems that matter most for authentication questions. Network records show that a service was reached, not what was sent to it.

What a record needs to carry

Organizations working to a standard do not have to invent this list. NIST SP 800-53 CM-8 requires an inventory that accurately reflects the system, includes all components within it, avoids double-counting components assigned to another system, sits at a granularity sufficient for tracking and reporting, and carries whatever information the organization defines as necessary for component accountability. The accountability information named in the guidance is concrete: the system name; software owners, version numbers, and license information; hardware inventory specifications; and for anything networked, machine names and addresses across every protocol in use. The inventory specifications are themselves enumerated—date of receipt, cost, model, serial number, manufacturer, supplier information, component type, and physical location.

NIST SP 800-171 states the same expectation more briefly, and where it states it moved between revisions. In Revision 2 the inventory is folded into 3.4.1 alongside baseline configuration, covering hardware, software, firmware, and documentation across the system life cycle. Revision 3 promotes it to its own requirement, 03.04.10, which asks the organization to develop and document the inventory, review and update it at a defined frequency, and update it as part of installations, removals, and system updates. The mapping is to CM-8 and CM-8(1).

Those requirements are written for components the organization runs, which is part of why the outsourced half falls through. Externally provided services sit under SA-9 instead, where the obligation is to require providers to meet the organization's requirements, define oversight roles, and monitor compliance on an ongoing basis. An organization that reads its inventory obligation narrowly can satisfy CM-8 completely and still have no record of the service holding its most sensitive data.

Working from that base, each entry is more useful carrying:

the operating system and version, which drives nearly every configuration and patching question asked later;
the installed software, or a reliable pointer to the configuration management system that holds it, which is usually the better answer since a static list is wrong within a week;
the business owner, who is usually not in IT;
what data the system holds, and whether that puts it in scope for a specific obligation;
what happens if it is unavailable, and for how long that is tolerable. This is the field that turns an inventory into something a risk assessment and an incident responder can both use, and it varies enormously between entries that otherwise look alike. A virtualization host carrying thirty workloads takes all thirty with it. A replicated DNS server with a secondary already answering queries may not be missed until the second one fails. Recording the judgment once is far better than forming it under pressure;
how people authenticate to it, and whether the provider supports the authentication the organization's policy requires;
who has administrative access, including at the provider;
the contract, its renewal date, and what security terms it contains;
when the entry was last confirmed, and by whom.

The authentication column is where inventories earn their cost. A requirement for phishing-resistant multifactor authentication is a claim about a set of systems, and it can only be as accurate as that set. A provider that does not support it, or supports it only on a subscription tier the organization did not buy, is a gap worth finding during selection rather than during an assessment.

What an assessor does with it

An assessment usually opens with a request for the inventory: either a copy, or read-only access to wherever the organization maintains it. The second is better for everyone. An assessor who takes a copy is now holding customer data and has to protect it, retain it, and dispose of it; an assessor who is given a read-only account is looking at the live record and holding nothing. Read-only access is not always possible, and a copy is a normal outcome, but an organization that can offer the account has made the engagement simpler.

What happens next is the part organizations tend not to anticipate. The inventory is not checked off and set aside. It drives the rest of the assessment: which people get interviewed, which systems get sampled, and what evidence gets requested.

An example. If the inventory shows both Windows and Linux systems, an assessor asking about a configuration control cannot sample only Windows machines and call the control met. Meeting it on one platform says nothing about the other, so the sample has to cover both, and the evidence has to come from systems of each type. The same reasoning applies to any split the inventory reveals: laptops against servers, on-premises against cloud, employee accounts against contractor accounts.

This is the difference between evidence that is adequate and evidence that is sufficient. A single screenshot may adequately show that a setting exists somewhere. Whether it is sufficient depends on how much of the scope it covers, and the inventory is what determines the scope.

A thin inventory therefore costs more than it appears to. It does not reduce the assessment; it moves the work into the assessment, where the organization is paying for the time and where an incomplete answer looks like an incomplete program.

Keeping it true

An inventory is accurate on the day it is built and decays from there. Subscriptions renew automatically, staff leave holding the only knowledge of a tool, departments adopt something new between reviews, and providers change what they offer.

Tying confirmation to events that already happen helps more than scheduling yet another review: contract renewals, budget cycles, onboarding and departures, and the annual policy review. The measure of a good inventory is not that it was complete when written. It is how quickly the organization would notice that it no longer is.

Kenneth Ingham Consulting helps organizations establish scope and inventory as part of readiness work, which is usually where the surprises turn up.

How do you know your security policy and procedures are being followed?

2026-08-11T00:00:00-06:00

A policy is not effective merely because it has been approved, distributed, and placed in a document repository. The useful question is: how would the organization know that everybody is following it?

That question should be answered while the policy is written. If a policy requires phishing-resistant multifactor authentication, the organization should know which system inventory or configuration report demonstrates coverage. If it requires annual access reviews, it should identify who performs the review, what records it produces, and who follows up on the exceptions. If it requires incident reporting, there should be a usable reporting path and evidence that reports receive a response.

Policy and procedure divide that work: a policy says what, and a procedure says how. The question of evidence reaches both. A policy without the procedure beneath it states an intention nobody is obliged to act on, and it is the procedure that produces the artifact the policy's claim depends on.

Start with claims that can be tested

For each important requirement, identify:

the person or role responsible for carrying it out;
the systems, people, or business processes in scope;
the evidence the activity produces;
how often someone checks that evidence;
what happens when the evidence is absent or shows a failure.

The evidence does not always need to be a technical log—a record written by a system itself, showing that a particular event happened at a particular time, such as an authentication attempt, a configuration change, or an account creation. It might instead be a ticket, an approved review record, a training roster, a tabletop exercise report, or a sample showing that a procedure can be repeated. What matters is that another qualified person can reach the same conclusion from it.

Nor does all evidence carry the same weight. Auditing practice has long ranked it by how far it sits from the party with an interest in the outcome: what an assessor tests or observes directly is stronger than what the organization hands over, and a record a system produced in the course of its own operation is stronger than one assembled by hand for the occasion. This is worth knowing while the procedure is being written rather than afterward, because the strong kind of evidence is usually no harder to produce than the weak kind—it simply has to be chosen deliberately.

A worked example

Start with a fixed line of policy:

Remote access to all systems working with sensitive data requires phishing-resistant multifactor authentication. Coverage is verified regularly, and every exception is approved, compensated, and time-limited.

That is a good requirement, and on its own it cannot be verified. Nothing in it says who checks, against what, or how often. Those answers belong in the procedure the policy points to.

The procedure carries the operating detail the policy has no business holding: how the requirement is configured on each platform, how a new employee enrolls a security key and what becomes of the temporary credential used to do it, how the enrollment report is produced, and how an exception is requested. It also answers, explicitly, the five questions in the list above.

Responsibility. The identity administrator configures and maintains the authentication requirement. The IT manager approves any exception.
Scope. Every system reachable from outside the office—the identity provider and the applications behind it, the VPN, the firewall's administrative interface—and every account that can use them, including contractor, vendor support, and administrative accounts.
Evidence. A configuration export showing which authentication methods each system permits, plus an enrollment report listing every active account and the method registered to it, compared against the current roster of people who should have access.
Frequency. The enrollment report monthly; the configuration whenever it changes, and at least at each annual review.
Correction. A missing enrollment is resolved or the account is disabled within a stated period. An approved exception carries a named owner, a business reason, a compensating measure, and an expiration date.

The useful part is what that comparison finds the first time someone runs it. The applications behind single sign-on are usually clean. The firewall's administrative interface may authenticate against a local account the identity provider never sees. A vendor support account may hold an exception granted years ago for a reason no one now remembers. One-time codes sent by text message may remain enabled as a fallback for people whose security keys fail—which is to say, for anyone who says their key failed.

The harder question is which systems belonged on the list in the first place. Workstations and servers are the tractable part: an organization running centralized configuration management can show that the setting is enforced and, more usefully, which machines the tool does not manage. The outsourced systems are where scope quietly ends. The HR platform holds employee records. The learning management system holds the role-based training completions that another control depends on. The external service a few engineers use to validate their designs may hold the most sensitive technical material the company owns, and was very likely bought on a card without passing through IT. Each of those is reachable from outside the office, and each is in scope only if it appears on a current inventory.

None of that means the policy is wrong. It means the policy covers less than it claims, and the organization now knows by how much and can decide what to do about each item.

A non-technical requirement traces the same way. Managers review their staff's access annually should produce a dated record for each manager naming the accounts examined, the changes requested, and confirmation that the changes were made. The evidence is a set of records rather than a configuration export, but the test does not change.

Evidence must support the policy—not replace it

A dashboard showing that a tool is installed does not necessarily prove the control is effective. A completed checklist does not prove the underlying work occurred. Evidence has to answer the policy's actual claim and cover the full scope, including exceptions.

This is one place where maturity becomes visible. A mature process does not depend on one employee remembering how things are done. Responsibilities, evidence, review, and correction are repeatable when people change roles or leave the organization.

A practical review

Choose a small number of consequential policy statements and trace each one from the document to current evidence. Ask the responsible person to explain the process, inspect a sample, and follow one exception through correction. The gaps will usually reveal whether the policy describes reality, an aspiration, or a process that existed when the document was last edited.

A policy that cannot be verified is close to useless. A policy tied to clear, proportionate evidence can guide work, survive personnel changes, and support an assessment without turning the organization into a paperwork factory.

Kenneth Ingham Consulting writes and reviews policies and procedures against exactly this test.

A policy says what; a procedure says how

2026-08-04T00:00:00-06:00

Many organizations own a shelf of security documents and still cannot answer a plain question about any one of them: is this telling someone what to achieve, or telling them how to do it? The distinction sounds academic until an assessment, a staff departure, or a new hire makes it expensive.

The division of labor

A policy states what the organization will do and why it matters. It should make sense to someone who does not administer the systems it governs.

It also has to state its scope: which people and which systems have to meet it. People means employees, but usually also contractors, temporary staff, and vendor personnel with access. Systems means the ones the organization runs and the ones it merely pays for. A requirement with no stated scope cannot be complied with or checked, because nobody can say what falls under it.

Scope is where policies fail quietly. "All employees" leaves out the contractors who often hold the most access. "All company systems" leaves out the ones the company does not own but depends on. The scope statement also sets the size of the inventory the organization needs in order to check its own compliance.

A procedure states how the work is done: the steps, the role that performs them, the tool involved, what the work produces, who checks it, and where the result is recorded. It should be specific enough that a competent new employee can follow it without being shown.

The same requirement, split

Consider phishing-resistant multifactor authentication.

The policy is one or two sentences: remote access to all systems working with sensitive data requires phishing-resistant multifactor authentication. Coverage is verified regularly, and exceptions are approved, compensated, and time-limited.

The procedures are everything that sentence assumes: which authentication methods are permitted and which are switched off; how a new employee enrolls a security key, and what happens to the temporary credential used to do it; how the identity administrator produces the monthly enrollment report; who compares it against the current roster of people who should have access; where that comparison is filed; what happens when an account turns up without an enrolled method; how an exception is requested, what it has to contain, and who approves it.

None of that detail belongs in the policy. All of it has to exist somewhere.

Systems the organization does not run

The split matters most where the organization does not own the computer. A cloud service or an external service provider holding company sensitive data is in scope, and the organization can configure only what the provider chose to expose.

The policy does not change: the requirement follows the data, not the question of who racks the hardware. The procedure changes considerably, because the mechanisms available are contractual as much as technical. The requirement has to appear in the agreement, the provider has to be asked what it actually supports, and the answer has to be verified rather than assumed.

That check is worth doing early. Plenty of providers are absorbed in their own product and have given little thought to what their customers are obliged to meet. A provider that cannot support phishing-resistant authentication, or supports it only on a plan the organization is not buying, is a finding to discover during selection rather than during an assessment.

Providers describe the division in a responsibility matrix: which controls the provider operates, which the customer operates, and which are shared. That document is where the organization's own procedure has to start, because it determines what is left to write a procedure about. A control the matrix marks as shared is not a control that is finished.

Why separating them is practical, not tidy

Four consequences show up in ordinary operation.

Approval and change rate. A policy is approved by someone with authority to commit the organization. A procedure changes when a vendor rearranges a settings page. Merge them and one of two things happens: trivial operational changes queue up for executive approval, or the document quietly stops matching reality because nobody wants to trigger the approval cycle.

Audience. An executive, a customer, or an assessor reads the policy to learn what the organization has committed to. The person doing the work reads the procedure. A document serving both audiences usually serves neither: too much detail to approve, too little to follow.

Handover. Turnover is where the distinction stops being a documentation question and starts costing money. An administrator leaving with a written procedure hands over a document. One leaving without hands over a conversation, and whatever gets forgotten in it leaves with them.

The bill arrives twice. First on the way out, in the scramble to capture what one person turned out to be the only one who knew, conducted during a notice period against someone whose attention has already moved on. Then on the way in, because a new hire with nothing to follow has to be taught by whoever knows the work—which means taking an experienced person off their own tasks, for every hire, with a result that varies according to who did the teaching and what they happened to remember that day. Written procedures turn all of that into reading.

Evidence. A policy makes a claim about coverage. The procedure is what produces the artifact that supports the claim, and it is the procedure that names the report, the schedule, and the file it lands in. Without that, the policy asserts something nobody can demonstrate.

The line is wide and gray

Where a given statement belongs depends on the organization. A twelve-person firm may reasonably keep one short document; a twelve-hundred-person one that tried would produce something no one could approve or follow.

A workable rule of thumb: if a sentence would have to be rewritten because the organization changed vendors, reorganized a department, or upgraded a product, it is procedure. A policy that names a particular identity provider has to go back for re-approval when the organization switches to a different one, even though the underlying requirement never changed.

The same test applies to named individuals. Policies assign responsibility to roles, because people change jobs and the requirement does not.

Checking frequency sits squarely on the line, and reasonable organizations put it on either side. How often compliance is verified is a real commitment: an assessor or a customer reads it as one, and a cadence stated in the policy cannot be quietly relaxed by whoever finds the check inconvenient. Stated in the procedure instead, it can be tightened after a bad finding without waiting on a re-approval cycle.

A workable compromise states a floor in the policy—verification happens at least annually—and lets the procedure set the working cadence above it. What does not work is leaving the frequency in neither document, which is the usual arrangement and the reason so many checks turn out to have last happened whenever someone last thought of it.

Deciding where the cadence lives is the easier half. Whether the check is actually happening, and whether the answer would convince anyone, is a separate question.

Two familiar failure modes

A policy with no procedure is an intention nobody is obliged to act on. This is the usual result of buying a template document set. It reads well, covers every heading an assessor might name, and when someone asks how a requirement is actually met, the answer lives in one administrator's memory or does not exist.

A procedure with no policy is a habit that changes when its author leaves. The work may be done well, but nothing states the requirement, so there is no basis for saying that a deviation is wrong, and no way to tell whether the practice still reflects a decision anyone made.

Link them to each other

Splitting the documents creates a navigation problem, and the fix is unglamorous: each policy statement should link to the procedures that carry it out, and each procedure should name the policy it serves. Often that is more than one. A procedure for reviewing account access may answer to an access control policy and to a personnel security policy at the same time, and it should cite both.

The organization gets the first benefit. Cross-references are how anyone discovers that a requirement has no procedure at all: a policy statement with nothing to link to is the first failure mode above, found before an assessor finds it.

The second benefit arrives on assessment day. An assessor working a control follows the policy to the procedure to the evidence. Where those links exist, that path takes minutes and the conversation moves on. Where they do not, the assessor asks, someone goes looking, and the organization spends its assessment hours demonstrating that it can find things rather than demonstrating that it is secure. Assessment time is not free, and neither is the impression left by an organization that cannot navigate its own documentation.

A test worth running

Take one requirement that matters. Hand the procedure to a competent person who has not done the job and see whether they can do the work. Hand the policy to an executive who administers nothing and see whether they can tell what the organization has committed to. If both hold, the pair is doing its job.

Neither document is paperwork for its own sake. Together they are what lets the work survive a resignation, an audit, and a change of vendor.

Kenneth Ingham Consulting writes and reviews security policies and procedures, and sorting out which statement belongs in which document is usually where that work starts.