grcskills
Menu

Change management evidence sampling

Use this skill when an auditor will test change management, or the team wants to know before they do, and asks "sample our production changes", "check our PR approvals", "do we have segregation of duties on deploys", "test change management before the audit", "were emergency changes approved" or "can a developer push to production alone". Builds the change population from what actually reached production, reconciles it with tickets and pull requests, draws a recorded sample, tests approval, testing, segregation and emergency attributes, and writes the results and the exceptions as facts.

EvidenceSOC 2ISO 27001

Use this for a period of operation, typically the audit period or the last quarter. You get three files in a new folder, ./change-management-evidence-sampling/: a short note of how the list of production changes was built and how the sample was drawn, so anyone can redraw it (sample-plan.md); a spreadsheet you can open in Excel with one row per sampled change and a yes or no for each check, with the link that proves it (change-sample.csv); and a plain list of every check that failed, stated as facts (exceptions.md). This skill drafts for a human to review and sign; this skill never marks a control effective. It records what each sampled change shows; conclusions belong to the reader and the auditor.

Inputs to gather

Read the change management policy or procedure as written: what counts as a change, who approves which kind, how tests are evidenced, and how emergency changes work. Then gather, read-only: merged pull requests to the production branch, deployment logs from the CI or release system, infrastructure-as-code apply logs, database migration history, and cloud audit logs for manual console changes. Ask the reader, in one message: the period, the production branch and deploy path for each repository in scope, and whether the auditor has said how they will sample.

Step 1: build the population from production

The population is every change that reached production in the period, taken from the production side: deploys, merges to the production branch, infrastructure applies and manual console changes, plus the changes that reach production without a deploy: feature flag flips, schema changes applied outside the migration history, and hotfixes pushed directly by on-call engineers. Do not start from the ticket system; a population built from tickets only contains changes that followed the process. Reconcile the sources: every deploy should map to a merged change, and every manual console change to a ticket. Unmatched items are the most interesting finding of the whole exercise; list them separately.

Step 2: draw a recorded sample

As a practitioner's rule of thumb, not an auditing standard: for a few hundred changes, around 25 items gives a reasonable read; for fewer than 50, sample a larger share; for a handful, test them all. Draw most items at random and record the method (sorted list, random seed or the tool used) so anyone can redraw the same sample. Test the emergency population in full, with the time to retrospective approval measured against the policy window. Include the first changes of the period, because auditors check the control already operated at the start. Add a few risk-weighted picks on top: changes to access control or infrastructure, changes merged outside working hours, and changes by people with administrator rights. If the auditor has set a method or sample size, use theirs and run a framework lookup for anything the framework expects about it.

Step 3: test each sampled change

For each item record yes, no or not applicable, with the link or record that shows it:

  • Description: a ticket or pull request explains what and why.
  • Approval before merge: approved by someone other than the author, and the approval timestamp is earlier than the merge. Check that a later commit did not land after the approval without a fresh review, if the policy expects one.
  • Testing: automated checks passed on the commit that was merged, or manual test evidence exists for changes without a pipeline.
  • Segregation: who deployed, and could the author have taken the change to production alone.
  • Emergency path: if flagged as emergency, retrospective approval within the window the policy sets.
  • Rollback: a rollback plan where the policy asks for one.

Step 4: test the guardrails, not only the samples

Read, without changing, the branch protection or repository rules for each production branch, who can bypass them, and the log of bypass events in the period. A rule that admins can override silently is only as good as the bypass log; if bypass events are not logged at all, write that down as a fact rather than skipping the check. Do the same for deploy permissions: list who can deploy to production, compare with the policy, and reconcile the list against the HR leaver list and the role matrix.

Step 5: write results and exceptions

The sample spreadsheet (change-sample.csv) holds one row per sampled change with every attribute. The exceptions list (exceptions.md) states each failed attribute as a fact: "Change 17: approved by the author's second account after merge; deployed by the author." No adjectives, no severity; the reader decides what is a deviation. The sample plan (sample-plan.md) records the period, population sources and counts, the reconciliation, the method and the guardrail review.

If the output folder exists, ask before overwriting and offer a new subfolder instead.

What good looks like

  • The population count comes from production and reconciles to the change records; unmatched deploys and console changes are listed, not ignored.
  • Approval timestamps are compared with merge timestamps on every item. Approvals after merge are the most common exception and the easiest to miss on a quick read.
  • The common mistake is sampling from approved tickets and finding everything compliant. The tell: a population count that matches the ticket count exactly, while the deploy log shows more releases.
  • Self-approval through a second account, a bot approval or a squash merge that dropped the approved commit gets caught because each approver is checked against the author.
  • Someone else can rerun the sample from the plan and get the same items.

Framework lookups

If ISMS Copilot is connected to your agent (MCP server https://account.ismscopilot.com/v1/account/mcp), ask it the framework question with create_conversation (mode think, answer_format: decision) and keep one conversation for this task. Send only a short question in your own words, show it to the user first, and never include file contents, secrets, personal data or evidence. If it is not connected, check the official text of the framework yourself; this skill does not restate it. Set it up: https://www.ismscopilot.com/mcp?ref=grcskills-change-management-evidence-sampling

Rules

  • MUST build the population from what reached production and record every source.
  • MUST record the sampling method so the same sample can be redrawn.
  • MUST write outputs only inside ./change-management-evidence-sampling/ and ask before overwriting.
  • NEVER change repository settings, branch rules, approvals or deploy permissions.
  • NEVER approve, merge, revert or re-review a historical change to fix an exception.
  • NEVER grade an exception as acceptable or effective; record the facts for the reader.