Verdict: Yes — you should treat every third-party agent skill like a browser extension with shell access. NVIDIA's open-source SkillSpector scanner gives you a fast, repeatable pre-install check. Run it in static mode for speed, add the optional LLM pass for subtle mismatches, and gate installs on a score of 50 or below.
Last verified: 2026-06-17 · Static scan: free · LLM scan: needs OpenAI/Anthropic/NVIDIA key · Works with Claude Code, Codex CLI, Gemini CLI, Hermes Agent, and other Agent-Skills-compatible agents
Why agent skills are a hidden supply-chain risk
An agent skill is a small bundle of instructions, metadata, and sometimes helper scripts that an AI agent loads when it matches a task. Once loaded, it typically runs with the same permissions as the agent itself: file-system read/write, shell commands, network calls, and access to your environment variables.
A 2026 empirical study, "Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale" (Liu et al., 2026), analyzed 42,447 skills from two major marketplaces and 31,132 unique skills after filtering. It found:
| Finding | Value |
|---|---|
| Skills with at least one vulnerability | 26.1% |
| Skills showing likely malicious intent | 5.2% |
| Risk multiplier for skills with executable scripts | 2.12× |
| Most prevalent categories | Data exfiltration (13.3%), privilege escalation (11.8%) |
Source: arXiv:2601.10338
Those numbers are not theoretical. In December 2025, Cato CTRL researchers showed how a "GIF Creator" skill could download and execute the MedusaLocker ransomware, exploiting the gap between what a user approves and what the skill actually does. The OWASP Top 10 for Agentic Applications calls this pattern "Identity and Privilege Abuse."
What SkillSpector is
SkillSpector is NVIDIA's open-source security scanner for agent skills. It is licensed under Apache 2.0, requires Python 3.12 or later, and can scan a Git repository, URL, zip file, local directory, or single SKILL.md file.
It answers one question: "Is this skill safe to install?"
The scanner reports in four formats:
| Format | Best for |
|---|---|
| Terminal | Quick local checks |
| JSON | Automation and CI gates |
| Markdown | Human review packets |
| SARIF | GitHub Code Scanning / VS Code |
Source: NVIDIA SkillSpector GitHub
How the two-stage scan works
SkillSpector uses a LangGraph-based pipeline:
- Resolve input — clone repo or unpack archive.
- Build context — read files, parse manifest, flag executable scripts.
- Stage 1: Static analysis — 64 deterministic checks across 16 categories.
- Stage 2: Meta-analyzer — optional LLM semantic review to filter false positives and flag intent mismatches.
- Report — risk score, findings, and recommendations.
Stage 1: Static checks (no API key needed)
Static analysis covers 64 patterns grouped into 16 categories. The most operationally important ones are:
| Category | What it catches | Example |
|---|---|---|
| Prompt injection | Hidden instructions in comments, invisible characters, or encoded text | A skill tells the agent to ignore safety rules |
| Data exfiltration | Sending files, env vars, or conversation context to external URLs | Reading ~/.aws/credentials and posting it online |
| Privilege escalation | Sudo, root, or overbroad permission claims | Declaring * permissions in the skill manifest |
| Supply chain | Typosquatted packages, curl \| bash, unpinned dependencies, known CVEs |
Fetching a remote script on every run |
| Dangerous code (AST) | exec(), eval(), subprocess, os.system |
Running arbitrary code from a string |
| Taint tracking | Data flowing from sensitive sources to network sinks | Env var → HTTP request |
| YARA signatures | Known malware, webshells, cryptominers | Reverse shell fingerprints |
| MCP least privilege | Wildcard or under-declared capabilities | A skill says it only reads files but also writes them |
| MCP tool poisoning | Unicode homoglyphs, hidden metadata, tool-name spoofing | Naming a fake tool reаd with a Cyrillic а |
It also queries OSV.dev for live CVE data on dependencies, with an offline fallback.
Source: NVIDIA SkillSpector documentation
Stage 2: LLM semantic analysis (optional)
Static checks can miss cases where a skill's description says one thing and its code does another. The optional LLM pass evaluates context and intent. It is useful for:
- Description-behavior mismatches.
- Vague triggers that could fire too often.
- Subtle policy issues that regex cannot see.
The trade-off: it requires an API key. Supported providers out of the box are OpenAI, Anthropic, NVIDIA build.nvidia.com, and any OpenAI-compatible endpoint (including local Ollama).
| Provider | Environment variable | Default model |
|---|---|---|
| OpenAI | OPENAI_API_KEY |
gpt-5.4 |
| Anthropic | ANTHROPIC_API_KEY |
claude-opus-4-6 |
| NVIDIA build.nvidia.com | NVIDIA_INFERENCE_KEY |
deepseek-ai/deepseek-v4-flash |
| Local Ollama | OPENAI_API_KEY=ollama, OPENAI_BASE_URL |
Configurable |
Source: NVIDIA SkillSpector GitHub
How to read the risk score
SkillSpector returns a 0–100 score. NVIDIA maps it to clear recommendations:
| Score | Severity | Recommendation |
|---|---|---|
| 0–20 | Low | Safe |
| 21–50 | Medium | Caution — review before installing |
| 51–80 | High | Do not install |
| 81–100 | Critical | Do not install |
Source: PyShine analysis of SkillSpector
Installation and first scan
You have two paths: local Python or Docker.
Option A: Local Python (recommended for daily use)
git clone https://github.com/NVIDIA/SkillSpector.git
cd SkillSpector
uv venv .venv && source .venv/bin/activate
make install
# Static-only scan of a local skill
skillspector scan ./my-skill/ --no-llm
# Scan with LLM semantic review
export SKILLSPECTOR_PROVIDER=anthropic
export ANTHROPIC_API_KEY=sk-ant-...
skillspector scan ./my-skill/
Source: NVIDIA SkillSpector documentation
Option B: Docker (no local Python needed)
make docker-build
# Static scan
docker run --rm -v "$PWD:/scan" skillspector scan ./my-skill/ --no-llm
# Scan with LLM using a .env file
cat > .env << 'EOF'
SKILLSPECTOR_PROVIDER=anthropic
ANTHROPIC_API_KEY=sk-ant-...
EOF
docker run --rm -v "$PWD:/scan" --env-file .env skillspector scan ./my-skill/
Source: NVIDIA SkillSpector GitHub
Useful scan patterns
# Single SKILL.md file
skillspector scan ./SKILL.md --no-llm
# Remote Git repository
skillspector scan https://github.com/user/my-skill --no-llm
# JSON output for CI
skillspector scan ./my-skill/ --no-llm --format json --output report.json
# SARIF for GitHub Code Scanning
skillspector scan ./my-skill/ --no-llm --format sarif --output report.sarif
Source: NVIDIA SkillSpector documentation
A practical install workflow
Here is the workflow we use for any skill we consider adding to our stack:
- Isolate the target. Copy or clone the skill into a throwaway directory, not your main project.
- Run a static scan first.
skillspector scan ./skill/ --no-llm --format json --output scan.json - Check the score. If it is 51 or above, stop. Do not install.
- Review high and critical findings. Static analysis can flag legitimate code as risky, so read the file and line number.
- Run the LLM pass if the score is 21–50 or if the description feels too broad. This catches mismatches that static rules miss.
- Re-scan after any edits. Some tools can auto-patch findings; verify the score drops before install.
- Install only in a low-value workspace first. Run one narrow task, then promote to your main environment.
This is the same layered approach OpenClaw uses in its ClawHub verification gate. In their public analysis, VirusTotal, static analysis, and SkillSpector agreed on fewer than 10.5% of combined positives — each scanner sees a different risk surface. SkillSpector is specifically tuned for agentic risk, not just generic malware.
Source: OpenClaw blog on NVIDIA collaboration
What this means for you
If you use Claude Code, Codex CLI, or Hermes Agent, you are already in the target audience. Skills make agents far more capable, but they also create a supply-chain hole that most teams are not monitoring.
SkillSpector is free, open-source, and takes minutes to add to your workflow. For a small business or solo builder, the easiest win is to scan every skill before install and refuse anything above 50. For a team, wire the JSON/SARIF output into CI so a risky skill cannot enter a shared workspace without a human exception.
If you are new to agent security, start with our broader guide on AI risks for small business and our Building with AI in 2026 pillar.
FAQ
Q: What is an agent skill? A: A portable package of instructions, metadata, and sometimes helper scripts that extends what an AI agent can do. The open specification is maintained at agentskills.io and is supported by Claude Code, Codex CLI, Gemini CLI, and other agents.
Q: Is SkillSpector free to use? A: Yes. The project is open-source under Apache 2.0. The static scan needs no API key. The optional LLM semantic scan requires your own OpenAI, Anthropic, NVIDIA, or OpenAI-compatible endpoint key.
Q: Does SkillSpector run the skill it is scanning? A: No. It performs static and optional semantic analysis on the skill files. It does not execute the skill, so it cannot catch runtime-only behavior, but it also cannot infect your system.
Q: What score is safe enough to install? A: NVIDIA's mapping treats 0–20 as safe, 21–50 as caution, and 51+ as do-not-install. We recommend blocking installs above 50 and reviewing 21–50 manually with the LLM pass enabled.
Q: Can I use SkillSpector in CI/CD?
A: Yes. Use --format json or --format sarif. The CLI exits with code 1 when the score exceeds 50, which will fail a standard CI gate.
Q: Which agents and skill formats are supported? A: SkillSpector is built for Agent Skills-compatible skills and works across Claude Code, Codex CLI, Gemini CLI, Hermes Agent, OpenClaw, and similar runtimes.
Q: What does the LLM scan add that static analysis misses? A: It evaluates semantic intent, such as a skill whose description claims to "format Markdown" but whose code silently fetches remote scripts. It also helps filter false positives from legitimate code.
Discussion
0 comments