Skylos tutorial: 8 checks we didn't find in other tools

mkdir skylos-demo-risk && cd skylos-demo-risk && git init -q -b main
g() { git -c user.name=demo -c user.email=demo@example.com "$@"; }
cat > app.py <<'EOF'
from flask import Flask
from flask_login import login_required

app = Flask(__name__)


@app.route("/admin")
@login_required
def admin():
    return "admin page"
EOF
mkdir tests && cat > tests/test_app.py <<'EOF'
def test_add():
    assert 1 + 1 == 2


def test_multiply():
    assert 2 * 3 == 6
EOF
g add -A && g commit -qm "base"
# The agent's branch: drops a test and the login check
g checkout -qb agent-change
sed -i.bak '/^@login_required$/d' app.py && rm app.py.bak
python3 -c "p='tests/test_app.py'; s=open(p).read(); open(p,'w').write(s.split('def test_multiply')[0])"
g commit -qam "agent change"
skylos cicd risk-review --base main

Skylos risk review: NEEDS A HUMAN APPROVAL (exit 1)
Needs a human approval: tests_weakened (tests/test_app.py:5), auth_removed (app.py:8)
Compared 65ef19e077c6 (merge base of main) with 0da2f1612580; 2 files, 3 lines changed

Risks that need a human approval:
  tests_weakened  SKY-A110  tests/test_app.py:5  test_multiply was deleted; no test with the same body exists now
  auth_removed  SKY-L021  app.py:8  Security control regression: Auth decorator @login_required was removed from 'admin'

Checked: tests_weakened, auth_removed, dependency_changed, ci_weakened, checks_silenced, secret_added, guardrail_config_changed, migration_changed, large_diff

mkdir skylos-demo-ci-agent && cd skylos-demo-ci-agent && git init -q -b main
mkdir -p .github/workflows && cat > .github/workflows/claude.yml <<'EOF'
name: claude
on:
  issue_comment:
    types: [created]
permissions:
  contents: write
jobs:
  claude:
    runs-on: ubuntu-latest
    steps:
      - uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          github_token: ${{ secrets.GITHUB_TOKEN }}
          allowed_non_write_users: "*"
          claude_args: "--allowedTools Bash"
EOF
skylos . -a --format concise

… (trimmed: scan notes and other workflow findings)
.github/workflows/claude.yml:15  SKY-D351  issue_comment (anyone who can comment) → claude-code-action (the action loads the triggering issue, comment or pull request) with allowed_non_write_users: "*", which lets any account past the action's write-access check; the agent has shell commands (Bash in --allowedTools) and secrets ANTHROPIC_API_KEY; contents: write.
.github/workflows/claude.yml:16  SKY-D352  claude-code-action can run any shell command (Bash in --allowedTools) in a workflow that issue_comment (anyone who can comment) starts; a prompt injection in that text becomes commands run with secrets ANTHROPIC_API_KEY; contents: write.
… (trimmed: other workflow findings)

mkdir skylos-demo-cheat && cd skylos-demo-cheat && git init -q -b main
g() { git -c user.name=demo -c user.email=demo@example.com "$@"; }
cat > mathlib.py <<'EOF'
def factorial(n):
    result = 1
    for i in range(2, n):
        result *= i
    return result


class Money:
    def __init__(self, cents):
        self.cents = cents
EOF
mkdir tests && cat > tests/test_mathlib.py <<'EOF'
from mathlib import Money, factorial


def test_factorial():
    assert factorial(5) == 120


def test_money():
    assert Money(100) == Money(100)
EOF
printf '[tool.pytest.ini_options]\npythonpath = ["."]\n' > pyproject.toml
g add -A && g commit -qm "base"
# The agent "fixes" the failing tests by hard-coding the answer and rigging ==
g checkout -qb agent-change
python3 - <<'EOF'
p = "mathlib.py"
s = open(p).read()
s = s.replace("    result = 1\n", "    if n == 5:\n        return 120\n    result = 1\n")
s += "\n    def __eq__(self, other):\n        return True\n"
open(p, "w").write(s)
EOF
g commit -qam "fix tests"
skylos done --base main

Skylos done · base bd4e924 (merge base) · head 6159184
UNFINISHED Tests pass when Skylos runs them: Earlier required checks failed or were incomplete
PASS       No tests deleted, skipped or weakened: No tests deleted, skipped or loosened (2 compared)
PASS       Skylos settings and hooks left alone: Skylos settings, hooks and protected paths unchanged
PASS       No secrets added: No secrets added
PASS       Every package and import is real: Every added import and package resolves
WARN       Tests check the changed lines: Earlier required checks failed or were incomplete
FAIL       Code doesn't special-case the tests: 2 place(s) in the code special-case the tests
           mathlib.py:2  SKY-A115 factorial() returns 120 when n == 5, the exact input and answer of tests/test_mathlib.py::test_factorial: this special-cases the test instead of computing the answer
           mathlib.py:14  SKY-A117 Money.__eq__ always returns True: any comparison with it passes, so tests that compare values cannot fail
           Fix: Remove the code that answers the tests' exact inputs, detects the test run or rigs comparisons, and make the general code right.
PASS       Linters, type checkers, scanners and CI not silenced: No linter, type-checker, scanner or CI settings weakened
PASS       Security controls checked for removals: No security control removals detected (1 changed source file(s) compared)
PASS       AI-reviewer settings and scanner markers left alone: AI-reviewer settings unchanged; no scanner suppression markers added
PASS       No code left without callers: No code left without callers
Verdict: FAIL: code special-cases the tests. The tests were not run because of this
Receipt: .skylos/receipts/615918426c11-bd4e924d1cbf.json

mkdir skylos-demo-orphans && cd skylos-demo-orphans && git init -q -b main
g() { git -c user.name=demo -c user.email=demo@example.com "$@"; }
cat > util.py <<'EOF'
def format_price(cents):
    return f"${cents / 100:.2f}"
EOF
cat > checkout.py <<'EOF'
from util import format_price


def total(items):
    return format_price(sum(items))


if __name__ == "__main__":
    print(total([150, 250]))
EOF
mkdir tests && cat > tests/test_checkout.py <<'EOF'
from checkout import total


def test_total():
    assert total([150, 250]) == "$4.00"
EOF
printf '[tool.pytest.ini_options]\npythonpath = ["."]\n' > pyproject.toml
g add -A && g commit -qm "base"
# The agent inlines the formatting; util.format_price now has no caller
g checkout -qb agent-change
cat > checkout.py <<'EOF'
def total(items):
    return f"${sum(items) / 100:.2f}"


if __name__ == "__main__":
    print(total([150, 250]))
EOF
g commit -qam "inline formatting"
skylos done --base main

Skylos done · base 956cabf (merge base) · head 72e5223
PASS       Tests pass when Skylos runs them: 1 tests run by Skylos, 1 passed (0 s)
PASS       No tests deleted, skipped or weakened: No tests deleted, skipped or loosened (1 compared)
PASS       Skylos settings and hooks left alone: Skylos settings, hooks and protected paths unchanged
PASS       No secrets added: No secrets added
PASS       Every package and import is real: Every added import and package resolves
PASS       Tests check the changed lines: Tests check all 1 changed lines
PASS       Code doesn't special-case the tests: No code special-cases the tests (1 changed file(s) read)
PASS       Linters, type checkers, scanners and CI not silenced: No linter, type-checker, scanner or CI settings weakened
PASS       Security controls checked for removals: No security control removals detected (1 changed source file(s) compared)
PASS       AI-reviewer settings and scanner markers left alone: AI-reviewer settings unchanged; no scanner suppression markers added
PASS       No code left without callers: 1 existing symbol(s) left unused
           util.py:1  SKY-A135 (advice) `format_price` (not touched by this change) lost its last caller when checkout.py:2 changed (`return format_price(sum(items))`); Skylos isn't certain nothing else uses it (weak helper function)
Verdict: PASS
Receipt: .skylos/receipts/72e522369649-956cabf035e7.json