Make a Skylos job a required status check on the pull request: Copilot's review comments, its optional approvals and its agent's CodeQL run don't fail a PR that deleted @login_required or a failing test. GitHub already checks a lot of Copilot's work; this page covers what it leaves open and how to close it.
What GitHub already does for Copilot pull requests
| GitHub feature | What it does | What it doesn't do |
|---|---|---|
| Copilot cloud agent validation | Runs CodeQL on the agent's new code, checks new dependencies against the GitHub Advisory Database, runs secret scanning, and tries to fix what it finds before finishing. Free, on by default, no Advanced Security licence. Since June 2026 it also covers third-party agents such as Claude and Codex working in GitHub | It isn't a check you control on the PR, and admins can switch individual tools off |
| Workflow approval | Treats Copilot like an outside contributor: your Actions workflows wait for a human with write access to approve the run (admins can turn this off since March 2026) | Approval is a click; it doesn't review anything |
| Copilot code review | AI review comments, steered by .github/copilot-instructions.md and path-scoped instruction files | It comments; a model's findings can vary between runs |
| Copilot approvals (public preview, opt-in) | Lets Copilot code review approve a PR; the approval can count toward required reviews | An AI approval is not a test result |
| Copilot hooks | .github/hooks/*.json commands on events such as preToolUse (can deny a tool call) for Copilot cloud agent and Copilot CLI; the file must be on the default branch for the cloud agent | Nothing is installed by default; Skylos has no installer for them |
That is a solid base. What's left open is a check you control on the PR, aimed at what the diff took away or faked: a protection that was removed, tests that were deleted or skipped, and package names that don't exist.
What to add: three jobs, all required
pip install skylos
skylos cicd init --no-upload
This writes .github/workflows/skylos.yml with:
- Skylos Quality Gate: a scan of the changed lines for security issues, secrets, AI-code defects and quality, with annotations and PR review comments.
- Skylos Done (when it finds a pytest project): runs your tests and fails if tests were deleted, skipped or emptied of assertions, or test settings or CI steps were loosened (what it checks).
It needs no Skylos account or API key. Then add the removed-controls job, because in 4.47.1 the generated scan uses --diff and did not report a deleted @login_required in our test. The removed-controls job's scan (--diff-base) reported it as:
app/views.py:5 SKY-L021 Security control regression: Auth decorator @login_required was removed
Make all three jobs required status checks in branch protection or a ruleset. Two Copilot-specific consequences:
- If Copilot approvals count toward required reviews, a required Skylos check is what stops an AI approval from merging a PR that fails it.
- If workflow approval is on (the default), the Skylos checks stay pending until someone approves the run, and a pending required check blocks the merge. That is the intended order: a person looks, approves the run, then the checks decide.
Tell Copilot what you expect, then check it
Custom instructions shape what Copilot writes and what its review comments on. Keep them short and concrete in .github/copilot-instructions.md:
- Routes that read customer or billing data keep their auth and permission checks.
- Request validation doesn't move without equivalent coverage.
- No hard-coded secrets; no
shell=Truehelpers; noverify=False. - Don't delete, skip or loosen tests to make CI pass.
Instructions state intent. The required checks above are what enforce it.
Run the same checks locally
Before you push, or when reviewing a Copilot branch by hand:
skylos . -a # security, secrets, dead code, AI defects, dependencies
skylos . --diff-base origin/main --danger --quality # what this branch changed, incl. removed controls
skylos done --base origin/main # tests deleted, skipped or weakened
-a includes --ai-defects, which reports packages that don't exist on PyPI (SKY-D222) and real ones your dependency files don't declare (SKY-D223) (details).
What this doesn't do
- No hooks inside Copilot. Skylos runs on the pull request, not in Copilot's session. If you want a check during the session, Copilot's
preToolUsehook is where your own script would go. - No application-specific authorization. Tenant filters, billing rules and CORS policy need tests and human review.
- It doesn't replace CodeQL. Run both; they model different things.
Where to go next
- Catch removed auth checks in AI-generated PRs
- Verify AI-generated code: check the agent didn't delete or skip tests
- GitHub Actions PR gate for AI-generated code
- Claude Code, Codex and Cursor hooks examples
- Python security scanner for GitHub Actions
References
- GitHub changelog: Copilot coding agent now automatically validates code security and quality
- GitHub changelog: Security validation for third-party coding agents
- GitHub changelog: Optionally skip approval for Copilot coding agent Actions workflows
- GitHub changelog: Copilot code review can now approve pull requests
- GitHub Docs: About hooks for GitHub Copilot
Running this across a team? These checks are free in the CLI. Skylos Cloud keeps the results in one place: the Free plan adds a pass/fail check on pull requests and scan history for one project, and the Workspace plan adds up to 10 projects, approvals for findings you decide not to fix, pull request comments, and audit exports.