One in 17 AI Agent Skills Tells Agents to Skip Human Oversight, Study Finds

i10X Quick Take

Quick Take

1. The finding: About one in 17 public agent skills (5.8%, roughly 88,000 of 1.53 million English SKILL.md files on GitHub) contains a directive that reduces human oversight, such as “commit without asking” or “pass –yes”.
2. The risk context: 70.1% of these directives apply in every context, and they cluster in skills that run shell commands, install packages or use sudo.
3. The detection gap: Skill scanners from Cisco and NVIDIA caught only 2% to 46% of such directives, but GPT-5.4 asked the right question reached 61% recall at 71% precision.

Agent skills are small instruction files that teach coding agents like Claude Code how to perform a task. They are shared, copied and installed by the million. A new preprint from Queen’s University in Canada, “When Skills Tell Agents Not to Ask: A Large-Scale Study of Oversight Directives in Agent Skills” by Timofey Sanko and Yuan Tian, looks at a part of these files that security tools mostly ignore: lines that tell an agent to act without asking, skip a safeguard, keep going or stay quiet.

What the study shows

The authors call such lines “oversight-loosening” directives. They are not malware. Most simply make a workflow run without interruption. But an agent told not to ask will also not ask before acting on a misread task or a malicious web page.

• Common but outnumbered: Among 1.37 million candidate text units with oversight vocabulary, 13.3% were loosening directives (95% CI 11.7 to 15.0%) and 36.4% were tightening ones, a ratio of 2.7 to 1 in favor of more oversight.
• Mostly convenience flags: The most frequent mechanisms were permission flags such as npx -y or –yes (33.9%), proceeding without confirmation (27.4%) and “don’t stop, keep going” instructions (16.0%). Skipping safeguards such as –no-verify made up 7.1%, auto-actions like auto-commit 6.2% and not reporting actions 4.5%.
• Rarely scoped: 70.1% of loosening directives carry no condition such as “only in CI”.
• Concentrated in risky skills: Skills that run shell commands or install packages had 1.82 times the odds of containing loosening; sudo 1.74, pipe-to-shell 1.31. Of skills with curl-to-shell, 40.6% contained loosening, against 16.9% of the rest. 12,484 skills (0.82%) combine loosening with destructive commands, pipe-to-shell or sudo.
• Spread through a few sources: Copying does not amplify loosening, but it is concentrated: 960 of 86,048 distinct loosening units (1.1%) account for half of all occurrences, and the 100 families carrying the most loosening reach almost a quarter of the affected skills.
• Scanners ask the wrong question: Offline, Cisco’s AI Defense skill-scanner 2.2.1 found 2% of loosening directives on the gold set and NVIDIA’s SkillSpector 2.12.0 found 21%, both below plain keyword matching on F1. With GPT-5.4 analysis, recall rose to 15% and 46%. Between 23% and 29% of scanner flags landed on guardrails instead.

Why it matters: Oversight is a security property

Security guidance lists human approval of high-risk actions as a key mitigation for prompt injection. Every unscoped “don’t ask” line removes that checkpoint, often in exactly the skills that run shell commands. The study shows that this is not a fringe pattern but an everyday convenience feature at large scale, and that the tools marketplaces rely on are not built to see it. SkillSpector with LLM analysis flagged 90% of gold-set skills whose unit was loosening, but also 79% and 85% of skills whose unit was tightening or neither, so a flag says little.

The most practical result is Finding 4.4: the same model behind the scanners’ LLM analysis, given the study’s codebook instead of a “is this malicious?” prompt, reached an F1 of 0.65, nearly twice the best scanner configuration (0.36).

Method in brief

• Data: GitSkills, which records 3,797,117 SKILL.md files from 282,200 public GitHub repositories; after filtering, 1,527,281 distinct English skills with 3,301,422 occurrences.
• Retrieval: A lexicon of 36 oversight patterns (e.g. “without asking”, –no-verify) yielded 1.37 million candidate units.
• Labelling: 2,097 units hand-labelled with a codebook; an LLM served as independent second coder on 1,000 of them (Cohen’s kappa 0.82 in the final round), with every disagreement adjudicated.
• Detector: A trained classifier labelled all candidates (accuracy 0.80, kappa 0.65 on 400 held-out units; loosening precision 0.70, recall 0.61).
• Scanner test: 400 gold units with 61 loosening directives, scanners run offline and with GPT-5.4.

What remains open

• Lower bounds: The lexicon is estimated to capture only about 32% of loosening, so the counts are likely conservative.
• Text, not behavior: The study measures what skills say, not whether agents actually follow those instructions.
• Scope: Only public, English skills on GitHub were analysed; private and marketplace-only skills are missing.
• Detector quality: With an F1 of 0.65, large-scale estimates carry classification noise. No code or data link is given in the preprint.

Context: Skills became the new supply chain

The Agent Skills format was originally developed by Anthropic and is now used across many coding agents. For background on one of the tested scanners, see our earlier report on NVIDIA SkillSpector. Earlier security studies looked for malicious skills, vulnerabilities and credential leaks. This paper adds a quieter category: benign files that lower the human checkpoint. It fits a wider line of research on how agents handle autonomy, collected in our Agents section, and on guardrails in Safety & Alignment.

What it means for skill authors, vendors and security teams

• Scope your shortcuts: A one-line clause such as “pass –yes only in CI” keeps most of the convenience.
• Warn at install time: Agent vendors could flag unscoped loosening, especially in skills that run shell commands.
• Fix upstream: Because 92 to 97% of copied variants leave oversight text untouched, editing a small set of popular source skills reaches most affected installs.
• Change the scanner prompt: Ask “does this line reduce human oversight?” instead of only “is this skill malicious?”, and treat a flag as a reason to review, not as a verdict.

About the analysis

Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.

This article is based on the full text of the preprint (arXiv:2610.37734v1, submitted October 27, 2026). All numbers come from the paper; i10X did not rerun the analysis. The chart was drawn by i10X from Table 6; the paper is licensed CC BY 4.0. AI tools assisted with research and drafting; the selection, fact-checking and final editing were done by the i10X editorial team.

i10x Perspective

Most debate about agent safety focuses on models. This study is a reminder that a lot of autonomy is granted in plain text, by people trying to save a few clicks, and then copied into thousands of projects. The fix is cheap: scope the shortcut, and make tools look for it.