Building a Verification Pipeline for Your AI Agent

This guide walks through adding SignalStack verification to any AI agent framework. You'll go from zero to a fully functional verification pipeline in under 30 minutes. We'll use Python for the examples, but the same patterns apply to any language with HTTP support.

Prerequisites

Before you start, you'll need:

• A SignalStack account (free tier available at signal-stack-ten.vercel.app)
• Python 3.10+ installed
• An existing AI agent (any framework — LangChain, CrewAI, custom, etc.)
• Basic familiarity with HTTP APIs and JSON

Step 1: Sign Up and Get Your API Key

Head to /docs/getting-started and create an account. Once you're in the dashboard:

1. Navigate to Settings > API Keys
2. Click "Generate New Key"
3. Copy the key and store it securely — you'll need it for every API call
4. Set it as an environment variable: export SIGNALSTACK_API_KEY='sk-your-key-here'

The free tier includes 500 verification calls per month, which is enough to build and test your integration. See /pricing for production limits.

Step 2: Install the Python SDK

SignalStack provides native SDKs for Python, Node.js, Go, and Rust. The Python SDK is the fastest way to get started:

pip install signalstack-sdk

Alternatively, you can use the REST API directly. All SDK methods are thin wrappers around HTTP calls. See /docs/sdks/python for the full SDK reference.

Step 3: Make Your First Verification

Let's verify a simple claim. Create a new file called verify_demo.py:

from signalstack import SignalStackClient
import os

client = SignalStackClient(
    api_key=os.environ["SIGNALSTACK_API_KEY"]
)

response = client.verify(
    claim="SignalStack is a trust infrastructure API for AI agents.",
    context={
        "domain": "technology",
        "sources": ["web", "knowledge-graph"],
        "min_confidence": 0.8
    }
)

print(f"Score: {response.score}")
print(f"Verdict: {response.verdict}")
print(f"Evidence sources: {len(response.evidence)}")

for evidence in response.evidence:
    print(f"  - {evidence.source}: {evidence.snippet[:100]}...")

Run it:

python verify_demo.py
# Output:
# Score: 0.94
# Verdict: pass
# Evidence sources: 3
#   - signal-stack-ten.vercel.app: 'SignalStack is a trust infrastructure API...'
#   - signal-stack-ten.vercel.app/docs: 'SignalStack provides verification...'
#   - wikipedia.org: 'Trust infrastructure refers to...'

Congratulations — you've just run your first AI verification. The API returned a score of 0.94 (well above the 0.8 minimum), a verdict of "pass", and three pieces of supporting evidence with source URLs and snippets.

Step 4: Integrate Into Your Agent Pipeline

The real power comes from integrating verification into your agent's generation pipeline. Here's a pattern that works with any agent framework:

from signalstack import SignalStackClient
from typing import Any

client = SignalStackClient(
    api_key=os.environ["SIGNALSTACK_API_KEY"]
)


def agent_with_verification(user_query: str) -> dict[str, Any]:
    # Step 1: Generate response using your agent
    agent_response = your_agent.generate(user_query)

    # Step 2: Extract claims from the response
    claims = extract_claims(agent_response)

    # Step 3: Verify each claim
    verification_results = []
    for claim in claims:
        result = client.verify(
            claim=claim,
            context={
                "domain": infer_domain(user_query),
                "sources": ["web", "knowledge-graph", "vector-db"],
                "min_confidence": 0.8
            }
        )
        verification_results.append(result)

    # Step 4: Check if any claims failed verification
    failed = [r for r in verification_results if r.verdict == "fail"]
    warnings = [r for r in verification_results if r.verdict == "warn"]

    if failed:
        return {
            "response": None,
            "error": "Agent generated unverifiable claims",
            "details": [
                {"claim": r.claim, "score": r.score}
                for r in failed
            ]
        }

    if warnings:
        # Surface warnings but still return the response
        return {
            "response": agent_response,
            "warnings": [
                {"claim": r.claim, "score": r.score}
                for r in warnings
            ]
        }

    return {"response": agent_response, "verified": True}


def extract_claims(text: str) -> list[str]:
    """Split agent response into individual claims for verification."""
    # Simple sentence splitting — use your preferred NLP approach
    import re
    sentences = re.split(r'(?<=[.!?])\s+', text)
    return [s.strip() for s in sentences if len(s.strip()) > 20]

This pattern works with LangChain:

from langchain.tools import tool

@tool
def verify_claim(claim: str) -> str:
    """Verify a factual claim before presenting it to the user."""
    result = client.verify(claim=claim, context={
        "sources": ["web", "knowledge-graph"]
    })
    return f"Score: {result.score}, Verdict: {result.verdict}"

Step 5: Handle Verification Results

Different verdicts should trigger different behaviors in your agent:

• Pass — deliver the response normally. Optionally surface the verification score in your UI for transparency.
• Warn — deliver the response but add a confidence caveat. For example: "I'm fairly confident this is correct, but you may want to verify the specific numbers."
• Fail — don't deliver the response. Instead, ask for clarification, try rephrasing the query, or escalate to a human.

Step 6: Set Up Webhooks for Asynchronous Verification

For non-blocking workflows, SignalStack supports webhooks. When you submit a verification with async mode, the API returns immediately with a request ID and delivers the result to your webhook URL:

from signalstack import SignalStackClient

client = SignalStackClient(api_key=os.environ["SIGNALSTACK_API_KEY"])

# Submit async verification
response = client.verify(
    claim="...",
    context={"sources": ["web", "knowledge-graph"]},
    mode="async",
    webhook_url="https://your-app.com/webhooks/signalstack"
)

# The response includes a request_id immediately
request_id = response.request_id
print(f"Verification in progress: {request_id}")

# Later, your webhook endpoint receives:
# POST /webhooks/signalstack
# {
#   "request_id": "...",
#   "claim": "...",
#   "score": 0.92,
#   "verdict": "pass",
#   "evidence": [...]
# }

Webhooks are essential for production systems where you can't afford to block the response while verifying claims. The /docs/guides/webhooks guide covers setup, retry logic, and security (webhook signature verification).

Step 7: Production Readiness

Before deploying to production, review these best practices:

• Set appropriate confidence thresholds per use case — lower for internal tools, higher for customer-facing
• Implement rate limiting — the standard tier supports 100 req/s; contact us for higher limits
• Cache verification results for identical or semantically similar claims
• Log all verification events for audit trails and debugging
• Set up monitoring alerts on verification failure rates — a sudden spike may indicate a model or data quality issue

Conclusion

Adding verification to your AI agent is a straightforward process: sign up, get your API key, install the SDK, and integrate the verification call into your generation pipeline. The hardest part isn't the integration — it's deciding what to do with the results. Map verification verdicts to clear behaviors (pass/warn/fail), set up webhooks for async workflows, and monitor your verification failure rates to catch issues before your users do.

For the full getting-started guide, visit /docs/getting-started. For a 5-minute quickstart, see /docs/quickstart. And for language-specific SDK documentation, check out /docs/sdks/python and the other SDK pages.