Zum Hauptinhalt springen

Secure Coding: AI-Assisted Development

Rui Dai
Rui Dai Engineer
Teilen

Secure Coding: AI-Assisted Development

An AI coding agent can add a valid input check and quietly change the dependency lockfile, a test assertion, or a deployment workflow in the same pull request. The security question is the whole change, including what the agent read and which commands it ran. These secure coding practices turn that work into a reviewable path from task input to human-approved merge.

This editorial checklist draws on OWASP, NIST, CWE, and source-control guidance checked September 23, 2026. It reports no penetration test and cannot guarantee a secure release.

Why AI-Assisted Development Changes the Review Surface

An AI assistant adds prompts and repository context to the review surface. It may propose commands, edit tests to match its implementation, or treat a retrieved file as an instruction. OWASP's Secure Coding with AI guidance calls out test weakening, context leakage, and build or CI edits.

Review one task, one candidate revision, its execution record, and approval of that revision. The broader threat model for an agent runtime has its own controls.

Protect Inputs, Context, and Credentials

Give the assistant only the paths and data needed. Remove customer records, keys, tokens, and production logs from prompts; use synthetic fixtures where values are irrelevant. Check what the tool sends, including open files and terminal output. .gitignore does not prevent an assistant from reading a file, as OWASP's AI coding guidance explains.

Treat issue text, code comments, retrieved pages, and tool output as data that may contain instructions. OWASP's prompt-injection analysis describes how untrusted content can redirect an LLM; in this workflow, the response is to scope the assistant's inputs and independently constrain what it can do. Keep real credentials in an approved secret store, inject them only where needed, and ensure logs and traces do not capture their values.

Protect Inputs, Context, and Credentials

Limit Agent Permissions and Tool Access

Start with read access and a bounded working copy. Limit write access to necessary files; grant commands separately, with constrained network and credentials. A branch or worktree organizes edits but does not isolate identity, secrets, or network.

Do not make the prompt the only permission boundary. A service token should have the smallest repository, package, or cloud scope that supports the task. Package publication, infrastructure changes, access-policy edits, and production deployment should reach a separate human-controlled gate. This is an application of least privilege to the coding workflow, consistent with NIST's Secure Software Development Framework, not a claim that one permission setting will stop every failure.

Review Generated Code and Dependencies

Read the diff against the requirement before the agent's explanation. Include hidden workflows, lockfiles, generated code, migrations, and test fixtures. A summary can miss an authorization check removed elsewhere.

Review Generated Code and Dependencies

Validate Data Flow and Authorization

Trace each new input from entry point through validation, transformation, storage, and output. Check server-side validation, parameterized queries, contextual output encoding, logging, and error paths. For every sensitive read or write, name the acting identity and the object it may access. Authentication alone does not prove authorization: CWE-862 describes missing authorization even when a user has signed in.

If the agent added GET /orders/{id}, can user A retrieve user B's order by changing the ID? Require an object-level ownership check and a denied-case test. OWASP's secure code review method follows data from sources to sinks across trust boundaries.

Validate Data Flow and Authorization

Inspect Packages, Licenses, and Supply-Chain Risk

For a new package, confirm why the existing stack cannot do the job, the package's identity and source, the pinned version, transitive additions, install scripts, and license. Review manifest and lockfile together; a lockfile-only or script-only change can still alter execution. On GitHub, dependency review surfaces added packages, known vulnerabilities, and available license data, but it may miss ecosystems or changes it cannot parse.

Apply the same scrutiny to CI actions, containers, code generators, and build scripts. A dependency scanner reports known information; it does not establish that a package name is the one the team intended or that its build-time behavior is acceptable.

Test Security-Critical Behavior

Ask the agent to propose tests, then have a reviewer decide the security assertions independently. For the order endpoint, test a permitted owner, a different authenticated user, an unauthenticated request, a malformed ID, and a record that no longer exists. Verify the response, side effects, and what reaches logs. A passing happy-path suite cannot establish a permission boundary.

Run static analysis, secret scanning, dependency checks, and relevant integration tests on the candidate revision. Inspect any deleted test, weaker assertion, new mock, or skipped check. OWASP specifically warns against letting generated tests validate only the behavior generated by the same assistant. Record failed checks and unresolved findings instead of quietly rerunning until a green badge appears.

Add Human Approval Before Merge and Release

The reviewer should receive a compact packet: requirement, allowed scope, changed-file list, security-sensitive paths, dependency changes, test commands and results, tested commit SHA, known gaps, and the proposed rollback. Require a human who can reject the patch. A second AI review can suggest findings but cannot supply the approval.

Use repository controls to make this real. GitHub protected-branch rules can require reviews and status checks and address stale approvals after a new push. Re-run affected checks when the revision changes. Treat deployment as a distinct decision if the release can change production data or permissions. This reflects NIST SSDF's emphasis on review, testing, and release integrity, while leaving authority with the team's designated owners.

Add Human Approval Before Merge and Release

A Reusable Secure Coding Checklist

Copy the following into the pull request template. Each box should point to an artifact or a named decision; an unchecked box is an open question, not an invitation to infer that the agent handled it.

  • Inputs: Task, allowed files, excluded data, and acceptance criteria are recorded; prompt context was checked for secrets and customer data.
  • Permissions: Agent identity, writable scope, commands, network access, and approval points were reviewed.
  • Diff: A human inspected all changed files, including tests, manifests, lockfiles, CI, migrations, and deployment configuration.
  • Data flow: Input validation, authorization, sensitive outputs, and logging were checked at each changed boundary.
  • Dependencies: New or changed packages, scripts, licenses, and known advisories were assessed.
  • Tests: Negative and cross-user cases, static analysis, secret scan, and applicable integration checks ran against the stated revision; skipped checks are explained.
  • Decision: The designated reviewer resolved findings, approved the final candidate, and recorded rollback or recovery needs.

This handoff does not replace design review or incident response.

FAQ

Should AI-Generated Code Be Marked in Commit History?

Record material AI involvement in the PR or commit description when team policy calls for provenance, especially if it helps a reviewer locate prompts, generated tests, and manual corrections. Avoid claiming precise line-by-line authorship if people edited the patch. The useful record is who accepted the final behavior and which revision they reviewed; the authoring tool does not take ownership of the merge.

When Should a Team Generate an SBOM for Agent-Written Changes?

Use the same SBOM trigger as for human-written changes: a releasable artifact, a customer or contract requirement, or a material dependency change under your supply-chain policy. Generate from the actual build when possible, because a source-only inventory can miss the artifact's final contents. The 2026 CISA and partner SBOM minimum elements include generation context: before build, during build, or after build. Label the context and tie the SBOM to the released artifact.

What Should Happen After a Secret Is Pasted into an AI Coding Tool?

Stop reusing the value and tell the credential owner or incident-response channel. Revoke or rotate it, determine where the prompt or transcript went, and assess the provider's retention and deletion options under the team's agreement. Remove copies where appropriate, but deletion alone cannot make an exposed credential safe. GitHub's secret-remediation guidance puts revocation or rotation first for a leaked repository secret; use the same containment priority for a pasted credential.

Can AI-Assisted Code Be Used in a Regulated Environment?

Possibly, if the organization's policies, vendor terms, data handling, evidence retention, human approval, and applicable rules permit that use. Keep sensitive context out of unapproved tools and preserve traceable review and release decisions. This article is not legal or compliance advice; the compliance owner must verify the rules applicable at publication and deployment time.

How Should Security Findings from an AI Reviewer Be Retained?

Keep the finding with its code location, model/tool version if relevant, tested revision, human disposition, and remediation evidence in the normal issue or PR system. Redact secrets and unnecessary customer data. A dismissed finding needs a reason; an accepted fix needs a new revision and affected checks. Set retention through organizational policy so a later reviewer can reconstruct the decision without retaining raw sensitive prompts by default.

Conclusion

AI-assisted coding becomes manageable when the team can trace a bounded request through a limited agent run, a complete diff, independent security tests, and a human decision on the final revision. The practical failure is an attractive green check attached to the wrong code or the wrong authority. Make the revision and the approver explicit before the change moves forward.

Rui Dai
Verfasst vonRui Dai Engineer

Hey there! I’m an engineer with experience testing, researching, and evaluating AI tools. I design experiments to assess AI model performance, benchmark large language models, and analyze multi-agent systems in real-world workflows. I’m skilled at capturing first-hand AI insights and applying them through hands-on research and experimentation, dedicated to exploring practical applications of cutting-edge AI.

Verwandte Leitfäden