All Tools
L
MonitoringFreeOpen Source
LANGKIT
Open-source text metrics toolkit for LLM monitoring and guardrails
Apache-2.0
ABOUT
Monitoring LLM outputs in production requires detecting issues like toxic content, prompt injections, PII leaks, and quality degradation — but most teams lack the tooling to extract these signals from unstructured text at scale. LangKit solves this by providing a library of pre-built text metrics for LLM monitoring, including toxicity detection, sentiment analysis, readability scoring, prompt injection detection, and PII identification — all designed to integrate with whylogs for production logging, alerting, and dashboarding.
INSTALL
pip install langkitINTEGRATION GUIDE
1. Monitor LLM outputs for toxic language, hate speech, and harmful content with automated scoring and alerting
2. Detect prompt injection attacks by analyzing input text patterns and scoring injection likelihood
3. Identify PII leaks in LLM responses by scanning for email addresses, phone numbers, credit card numbers, and other sensitive data
4. Track text quality metrics like readability, repetition, and coherence across LLM responses over time
5. Integrate LLM text signals into existing monitoring pipelines using whylogs for data logging and dashboard visualization
TAGS
monitoringllmtext-metricsobservabilitywhylabsguardrailssafety