All Tools
P
MonitoringFreeOpen Source
PYRIT
Automated red teaming framework for generative AI
MIT
ABOUT
Generative AI systems introduce novel attack surfaces — multi-turn jailbreaks, prompt injection chains, and modality-specific exploits — that traditional security testing doesn't cover. Security teams need a structured, repeatable framework for red teaming that scales across models, providers, and attack types. PyRIT provides composable attack strategies, standardized scoring, and conversation memory with a plugin architecture for targets, converters, and scorers. It supports OpenAI, Azure, Anthropic, Google, HuggingFace, custom HTTP endpoints, and web app targets.
INSTALL
pip install pyritINTEGRATION GUIDE
1. Run automated multi-turn jailbreak attacks (Crescendo, TAP, Skeleton Key) against production LLMs
2. Benchmark model safety across providers with standardized scenarios and scoring rubrics
3. Identify data leakage risks by probing sensitive information extraction from model outputs
4. Test content safety guardrails for harmful output across text, image, and audio modalities
5. Integrate into CI/CD pipelines to catch regressions before model deployment
TAGS
red-teamingsecuritysafetymicrosoftllm-evaluationrisk-assessment