What if every statement an AI agent made came with a credit score — a single number that captured how confident you should be in its truthfulness, backed by evidence you can inspect? That's the idea behind SignalStack Trust Scoring. It's a standardized way to quantify the reliability of AI-generated claims, and it's changing how production systems handle model outputs.
Why Trust Needs a Score
Binary classification — hallucination or not hallucination — is the wrong abstraction for production AI. Real systems need nuance. A claim about a company's founding date might be verifiable with high confidence from Wikipedia. A claim about a competitor's internal strategy might be inherently uncertain. Treating both the same way is either too risky or too restrictive.
A trust score gives you that nuance. It's a continuous value (typically 0.0 to 1.0) that represents the confidence that a claim is factually correct, based on the weight of available evidence. Different applications can set different thresholds: a news summarization bot might accept 0.8, while a financial reporting agent might require 0.95.
"Trust scoring turns AI verification from a binary gate into a risk-calibrated signal. That's the difference between a demo and a production system." — CTO, AI-native compliance platform
The Dimensions of Trust
SignalStack Trust Scoring evaluates claims across multiple dimensions, each contributing to the overall score:
Factual Accuracy
The primary dimension. Does the claim match information in authoritative sources? This is assessed by cross-referencing against our multi-source pipeline, which pulls from web search, knowledge graphs, vector databases, structured APIs, document stores, and code execution. Sources must corroborate the claim with reasonable consistency.
Source Recency
A claim that was true in 2023 may be false in 2026. Source recency adjusts the trust score based on the timeliness of supporting evidence. A claim about current events requires sources from the past 24 hours. A claim about historical facts can rely on decade-old sources. The decay function is configurable per domain.
Source Authority
Not all sources are equal. A peer-reviewed journal carries more weight than a blog post. An official company website carries more weight than a forum comment. Authority scoring is based on domain reputation, citation networks, and editorial standards. Sources are weighted accordingly in the aggregate score.
Internal Consistency
Does the claim contradict other claims made by the same agent, either in the current response or in the conversation history? Inconsistency is a strong signal of hallucination. The scoring engine evaluates claims against the established context to flag contradictions.
How Scores Are Weighted
The final trust score is a weighted combination of individual dimension scores. The weights are not static — they adapt based on the claim type and domain. A medical claim weights factual accuracy and source authority heavily. A stock price claim weights factual accuracy and source recency. The weighting model was trained on a corpus of 500,000+ verified claims with human-labeled ground truth.
The scoring engine produces three verdict levels:
- Pass (score >= 0.85): The claim is well-supported by evidence. Safe to use in most contexts.
- Warn (0.70 <= score < 0.85): The claim has partial support but some evidence is missing or contradictory. Surface with caveats or request confirmation.
- Fail (score < 0.70): The claim cannot be verified with sufficient confidence. Do not use without human review.
Trust Scoring in Practice
Here's how the Trust Scoring API works in practice. Make a POST request with the claim you want to verify:
curl -X POST https://signal-stack-ten.vercel.app/api/v1/verify/business \
-H "Authorization: Bearer $SIGNALSTACK_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"claim": "SignalStack was founded in 2024 and is SOC 2 compliant.",
"context": {
"domain": "technology",
"sources": ["web", "knowledge-graph"],
"min_confidence": 0.8
}
}'The response includes the trust score, verdict, and evidence:
{
"claim": "SignalStack was founded in 2024 and is SOC 2 compliant.",
"score": 0.92,
"verdict": "pass",
"dimensions": {
"factual_accuracy": 0.95,
"source_recency": 0.88,
"source_authority": 0.91,
"internal_consistency": 1.0
},
"evidence": [
{
"source": "signal-stack-ten.vercel.app/about",
"snippet": "Founded in 2024, SignalStack...",
"relevance": 0.97,
"retrieved_at": "2026-04-14T10:00:00Z"
},
{
"source": "signal-stack-ten.vercel.app/security",
"snippet": "SignalStack is SOC 2 Type II certified...",
"relevance": 0.94,
"retrieved_at": "2026-04-14T10:00:00Z"
}
]
}The response includes the individual dimension scores so you can understand why a particular verdict was reached. If a claim fails due to low source authority, you know the claim might be true but lacks reliable corroboration — a different failure mode than factual contradiction.
Configuring Thresholds Per Use Case
Different use cases demand different levels of certainty. The /docs/guides/trust-scoring documentation covers how to configure thresholds per endpoint, claim type, or even per user. A customer-facing chatbot might warn at 0.7 and fail at 0.6. An internal data analysis agent might warn at 0.6 and fail at 0.5. A compliance reporting agent might warn at 0.9 and fail at 0.85.
Pricing is based on verification volume, not tiers of accuracy. See /pricing for details on how verification credits work across different plan levels.
When tuning thresholds, run a retrospective analysis on 1,000+ verified claims to find your optimal cutoff. Plot the precision-recall curve for your domain. The ideal threshold is where false positives (blocking a correct claim) and false negatives (passing a wrong claim) are balanced for your specific risk tolerance.
Conclusion
Trust scoring brings the rigor of financial credit scoring to AI verification. By evaluating claims across multiple dimensions, weighting evidence by quality and recency, and providing actionable verdicts with supporting citations, SignalStack gives AI engineers a tool they've never had before: a quantifiable, auditable measure of their agent's truthfulness. Stop treating verification as a binary gate and start managing it as a calibrated signal.
Luke Swestun is the founder of SignalStack. He writes about trust infrastructure, hallucination detection, and building AI agents that can verify before they act.