Custom Trust Models

Trust scoring is not one-size-fits-all. A customer support agent routing refund requests has very different reliability requirements than a trading agent executing market orders. A healthcare agent handling patient data operates under different regulatory constraints than a content moderation agent. SignalStack's trust scoring system is designed to be customized — you set the thresholds, weights, and verification combinations that match your specific risk tolerance and use case.

Why Default Trust Scores Aren't Enough

Out-of-the-box trust scoring provides a useful baseline. SignalStack's default model weights each verification dimension equally and flags outputs below a 0.7 confidence threshold. For many use cases, this is a reasonable starting point. But default configurations optimize for the average case, and your use case is not average.

Consider two teams building on SignalStack. Team A runs an AI customer support agent for an e-commerce platform. Their agents handle refund requests, order status inquiries, and product recommendations. The cost of a hallucination is low — a wrong product recommendation might mean a return, not a lawsuit. Their optimal configuration weights claim verification at 40%, document verification at 30%, and business verification at 30%, with a threshold of 0.5. Anything below 0.5 gets a human review, but most outputs sail through at 0.85+.

Team B runs an AI financial analyst agent for a hedge fund. Their agents produce investment memos with specific revenue projections, competitive analysis, and regulatory risk assessments. A hallucination here could mean a multi-million dollar trading loss. Their configuration weights claim verification at 60%, document verification at 25%, and media provenance at 15%. Their threshold is 0.92. Any output below 0.92 is automatically held for human review and never reaches the trading desk.

"The right trust threshold is the one where your false positive rate (blocking valid outputs) equals your false negative rate (passing hallucinated outputs) at your acceptable risk level. There is no universal threshold — only the threshold calibrated to your specific cost of failure."

Adjusting Verification Weights

SignalStack's trust scoring system computes an overall trust score as a weighted combination of individual verification dimension scores. Each dimension — business verification, document analysis, media provenance, and claim verification — produces a normalized score between 0 and 1. The overall score is the weighted sum.

The weight configuration is expressed as a simple JSON object passed to the verification API. Here's how Team A and Team B configure their weights:

// Team A: E-commerce customer support
// Low-stakes, high-volume — prioritize speed over precision
{
  "weights": {
    "claim_verification": 0.4,
    "document_verification": 0.3,
    "business_verification": 0.3,
    "media_provenance": 0.0
  },
  "threshold": 0.5,
  "on_failure": "human_review"
}

// Team B: Financial analyst agent
// High-stakes, low-volume — prioritize precision over speed
{
  "weights": {
    "claim_verification": 0.6,
    "document_verification": 0.25,
    "business_verification": 0.0,
    "media_provenance": 0.15
  },
  "threshold": 0.92,
  "on_failure": "block"
}

Notice that Team A sets media provenance weight to 0 — their agents don't process media files, so there's no point scoring that dimension. Team B sets business verification weight to 0 because their agents don't make financial transactions with counterparties. Eliminating irrelevant dimensions reduces noise in the overall score.

Setting Custom Thresholds

The trust threshold defines the boundary between "acceptable" and "needs review." Setting this threshold requires understanding your operational risk curve. SignalStack provides tooling to help you calibrate thresholds based on historical data.

The calibration workflow starts with a review period where you run verification in monitor-only mode — scoring every output but not blocking anything. This generates a distribution of trust scores across your actual workload. You can then analyze the score distribution and identify the natural cutoff point. Most teams find that their score distribution is bimodal: a cluster of high-confidence outputs (0.8-1.0) and a tail of low-confidence outputs (0.1-0.4). The optimal threshold is the valley between these two clusters.

For teams without sufficient historical data, SignalStack provides recommended thresholds based on use case category. Verified deployments across 3,000+ agents have produced these baseline recommendations:

• Customer support and content generation: 0.5 — low risk, high throughput priority.
• E-commerce and procurement: 0.7 — moderate risk, financial exposure limited per transaction.
• Healthcare information and legal analysis: 0.85 — high risk, regulatory compliance requirements.
• Financial trading and autonomous payments: 0.92 — critical risk, direct financial exposure.
• Emergency response and safety-critical systems: 0.97 — maximum risk, human-in-the-loop required.

Combining Verification Types

The true power of custom trust models emerges when you configure verification type combinations for specific workflow stages. An agent doesn't need all four verification types on every action. Different stages of a workflow require different verification profiles.

SignalStack supports workflow-aware verification configuration: you define verification policies for each stage of your agent workflow, and the system applies the appropriate configuration automatically. The API reference at /docs/api-reference documents the full configuration surface:

// Workflow-aware verification configuration
{
  "stages": [
    {
      "name": "entity_discovery",
      "verification": ["business_verification"],
      "threshold": 0.7,
      "max_retries": 2
    },
    {
      "name": "document_processing",
      "verification": ["document_analysis", "media_provenance"],
      "threshold": 0.8
    },
    {
      "name": "decision_execution",
      "verification": ["claim_verification", "document_analysis"],
      "threshold": 0.92,
      "on_failure": "block",
      "require_human_approval": true
    }
  ]
}

Conditional Verification

Beyond static configurations, SignalStack supports conditional verification rules. These allow different verification depth based on context: if an agent is making a payment over $10,000, require business verification and claim verification with a 0.95 threshold. If the payment is under $100, skip business verification and use a 0.5 threshold. If the counterparty is a new entity (first interaction), require full four-type verification regardless of amount.

Conditional verification is where trust models become truly adaptive. Rather than applying the same scrutiny to every action, the system dynamically adjusts verification depth based on risk signals. This is how you scale autonomous systems from handling hundreds of decisions per day to hundreds of thousands.

Monitoring and Iterating Trust Model Performance

A trust model is not a set-it-and-forget-it configuration. Agent behavior changes as models are updated, use cases evolve, and new verification capabilities become available. Effective teams treat trust model configuration as a continuous optimization process.

SignalStack provides a monitoring dashboard that tracks three key metrics for your trust model: the verification pass rate (percentage of outputs exceeding your threshold), the human review rate (percentage of outputs below threshold that were reviewed and either confirmed or rejected), and the override rate (percentage of automated decisions that humans overruled). These metrics tell you whether your trust model is correctly calibrated. A very high pass rate (>98%) suggests your threshold might be too low — you're letting through outputs that should be checked. A very high override rate (>15%) suggests your threshold is set incorrectly, flagging valid outputs and wasting human reviewer time.

The monitoring system also tracks verification dimension performance over time. If you notice that claim verification scores are consistently lower than other dimensions, it might indicate that your claim verification sources need updating, or that your agents are generating outputs in domains where your knowledge base is thin. Trust scoring dashboard data feeds directly into model improvement decisions.

A/B Testing Trust Configurations

For teams that want to optimize their trust model systematically, SignalStack supports A/B testing of trust configurations. You can route a percentage of your agent's outputs through a candidate trust configuration while the rest goes through the current production configuration. The system tracks pass rates, review rates, and override rates for each configuration and reports the comparative performance.

A typical A/B testing workflow starts with cloning your production configuration and adjusting a single parameter — lowering the threshold from 0.7 to 0.65, or changing the claim verification weight from 40% to 50%. Run the experiment for enough volume (SignalStack recommends at least 1,000 verification events per variant) and compare the results. If the candidate configuration shows a better trade-off between pass rate and override rate, promote it to production.

Real-World Example: Multi-Stage Trust for a Procurement Agent

To illustrate how these concepts come together, consider a procurement agent that handles the full vendor lifecycle: discovering potential suppliers, evaluating proposals, negotiating contracts, and issuing purchase orders. Each stage requires a different trust configuration.

During supplier discovery, the agent searches for new vendors and performs initial business verification. The threshold is moderate (0.7) because the cost of a false positive is low — a bad lead just means wasted follow-up time. The verification is lightweight: only business verification is needed, checking entity registration and basic credentials.

During proposal evaluation, the agent receives and analyzes supplier proposals. This stage requires document verification (checking that uploaded proposals are genuine and unmodified) and claim verification (checking the factual claims in each proposal against market data). The threshold rises to 0.85 because acting on a fraudulent or inaccurate proposal wastes significant team effort.

During contract negotiation, the agent produces draft contract terms and verifies the supplier's counter-proposals. This stage engages all four verification types: business verification reconfirms the supplier's current standing, document verification checks the contract PDF for tampering, media provenance validates any attached product images or certification scans, and claim verification checks every factual assertion in the negotiation. The threshold is 0.92, and any output below threshold triggers human review.

During purchase order execution, the agent issues the final PO. This stage applies claim verification and business verification with the highest threshold (0.95). If the trust score falls below 0.95, the PO is blocked entirely and routed to a procurement manager. This multi-stage approach means the agent is efficient during low-risk stages and maximally cautious during high-risk ones.

Conclusion

Custom trust models transform SignalStack from a general-purpose verification platform into a precision instrument calibrated for your specific risk profile. By adjusting verification weights, setting use-case-appropriate thresholds, and configuring workflow-aware verification combinations, you can achieve the exact balance of safety and throughput that your autonomous system requires. The configuration is documented in full at /docs/api-reference and the trust scoring guide at /docs/guides/trust-scoring provides detailed calibration methodology.