Skip to main content

    AI Controls

    AI Technical Controls Assessment: What to Check Before a Release

    In short

    Follow one AI-assisted change from ticket to production and see which records show the controls actually worked.

    a software change moving through review, automated tests and a release gate.

    Company names and figures are invented.

    An AI-assisted change can pass through a release pipeline while an access change goes unnoticed. Follow the ticket, code revision, review and deployment records to see what the controls actually caught.

    Why check an AI-assisted change?

    A release can succeed while an unauthorized access change goes unnoticed. Review the code, permissions and approvals for the version deployed.

    The NIST AI Risk Management Framework links accountability, understanding risks, measuring behavior and acting on findings. For a software change, that means reviewing the work request, code changes, approvals and release records together.

    Example: a small change creates extra access

    Link the review to the exact code revision and deployment. Looking at a current policy alone cannot show how that particular change was handled.

    Questions a review should answer

    a reviewer's checklist beside a laptop showing a code change moving through review, tests and release.
    Original working-paper illustration.

    Reference table · every row is included.

    Review areas for the fictional payout change
    AreaQuestionWhy it matters
    ApprovalDid the approved request include the new ledger access?A routine fix can introduce a different business risk.
    Second reviewDid another qualified person check the changes?An approval click alone does not explain the quality of a review.
    TestingDid the tested version match the version released?Passing tests for an earlier version may be misleading.
    ReleaseDid the approved version follow the normal release path?Manual changes can bypass the checks a team relies on.
    AccessCan the service do more than its job requires?Extra permissions increase the damage an error could cause.
    Connected toolsWho approved the integration and manages its credentials?An existing connection can gain new capabilities.

    Keep the basis of an answer clear. Self-reported means someone described the process. Inspected means a reviewer examined records or settings. Tested means a reviewer observed or repeated a control. Reliable records and tests support different conclusions; a policy or interview alone does not prove the process worked.

    A rule can exist without working well

    A repository can require a second approval and still receive a rushed review. In this example, approval arrived after 40 seconds. That timing is a reason to ask how the review was done, not proof that it was ineffective.

    The reviewer checks whether the control could address the risk and whether the available records show it worked. These are often called design effectiveness and operating effectiveness.

    What AI adds to the risk

    • More code can arrive faster than reviewers can check it.
    • A tool may change files or permissions beyond the original request.
    • A named person still needs to be responsible for approving the work.
    • The coding tool's own access needs review, as well as the application's access.

    For agents that act directly in business systems, see human oversight for AI agents.

    AI change approvals and access controls.

    Permission & data boundary

    Approved taskPurpose + owner
    Agent + toolsScoped access
    Human reviewChange + test results
    ReleaseRuntime records
    Example

    Define permitted agent actions, allowlisted capabilities, scoped identities and restricted data access. A human checkpoint inspects the exact change and test results before release. Trace approvals and actions, retaining stop, revoke and recovery records. A sandbox test supports only the scenario, permissions and limits actually exercised.

    What a useful finding looks like

    Our AI controls review helps identify these weaknesses. Start with the short AI controls readiness checklist or the AI Governance QuickScan to organize the questions you need answered.

    Key takeaways

    • Check the actual change, not just the policy.
    • Match approvals, tests and release records to the version deployed.
    • Give each finding a clear business risk, owner and next action.
    Related serviceAI Technical Controls AssessmentRelated resourceAI Controls Readiness ChecklistFree toolAI Governance QuickScan

    References

    Primary sources cited above. Standards are referenced by number and year; their text is licensed and is paraphrased, not reproduced.

    1. NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023 (opens in a new tab) — Govern, Map, Measure and Manage functions.
    2. NIST AI 600-1, AI RMF: Generative Artificial Intelligence Profile, July 2024 (opens in a new tab) — Risks and suggested actions specific to generative AI.
    3. NIST AI Risk Management Framework program page (opens in a new tab)
    4. NIST SP 800-53A Rev. 5, Assessing Security and Privacy Controls, January 2022 (opens in a new tab) — Examine, interview and test as assessment methods, the basis for the self-reported / inspected / tested distinction used here.
    5. ISO/IEC 42001:2023, Artificial intelligence management system — Requirements (opens in a new tab) — Licensed text; referenced for Annex A control objectives, not reproduced.

    The next step

    Discuss a professional assessment.

    Get a review of change approvals, testing and release controls for your automated workflows.