In early 2025, a European manufacturing company paid a €47,000 invoice that appeared to come from a long-standing supplier. The invoice was perfect — correct logo, accurate bank details, proper VAT number. The only problem was that it was entirely AI-generated. The supplier never sent it. The bank account belonged to a shell company. The loss was discovered 72 days later during a routine reconciliation, by which point the funds had been laundered through six jurisdictions. This wasn't a targeted nation-state attack. It was a single person with a subscription to a synthetic media API.
The Generative Invoice Problem
Deepfake invoices represent the convergence of two trends: the democratization of generative AI and the antiquated state of accounts payable security. AP departments still rely on a combination of trust, pattern matching, and manual review — exactly the vulnerabilities that AI-generated content exploits best.
Modern image generation models can produce photorealistic document scans complete with watermarks, letterhead, and signatures. LLMs can generate boilerplate text that matches a supplier's writing style. Together, these tools enable invoice fraud at a scale and quality that traditional detection systems cannot catch. The FBI's Internet Crime Complaint Center reported a 25% increase in business email compromise losses in 2024, with AI-generated documents cited as an emerging vector.
Three Attack Vectors
Generative AI enables three distinct invoice fraud techniques, each with different detection characteristics:
- Synthetic Generation — The invoice is created entirely from scratch using generative models. No source document exists. The attacker describes the desired output and the model produces a realistic invoice. Detection requires analyzing pixel-level artifacts and generation fingerprints.
- Metadata Manipulation — A legitimate invoice is modified using inpainting or editing models. The attacker changes the bank account number, payment address, or amount while preserving the visual integrity of the document. Detection requires comparing metadata fields against visual rendering.
- Template-Based Forgery — The attacker uses a known supplier template (often obtained from a previous legitimate invoice) and fills in new details. Detection relies on cross-referencing the document against the supplier's known patterns and registered business data.
Why Traditional Detection Fails
Most AP departments rely on a few detection methods, all of which are increasingly ineffective against AI-generated forgeries:
- Optical character recognition (OCR) — Reads text but cannot distinguish between human-typed and AI-generated content. Both render as valid characters.
- Manual visual inspection — Humans are poor at detecting high-quality deepfakes. Controlled studies show that human reviewers catch less than 50% of AI-generated documents when the forger is competent.
- Database matching — Checks invoice numbers and amounts against expected values. Works for known discrepancies but misses entirely novel fraudulent invoices.
- Watermark detection — Only effective against naive forgery. Modern generative models can produce convincing fake watermarks or reproduce originals from training data.
"The problem is not that AI-generated invoices look fake. The problem is that they look more real than the real ones. Perfect formatting, zero typos, flawless logos — they tick every checkbox that human reviewers use as signal for legitimacy." — Head of AP Security, Fortune 500 retailer (anonymous interview, 2025)
How AI-Powered Verification Stops Deepfake Invoices
SignalStack's document analysis API (/product/document-analysis) and media provenance system (/product/media-provenance) work together to detect AI-generated invoices at multiple levels of analysis:
Pixel-Level Analysis
Generative models leave detectable artifacts. Diffusion models produce characteristic noise patterns in high-frequency image regions. GAN-generated images show statistical anomalies in color distributions. SignalStack's analysis pipeline applies a suite of forensic algorithms:
- Frequency domain analysis — AI-generated images have distinctive patterns in their Fourier transform spectra, particularly in phase coherence across color channels.
- Noise consistency checks — Real documents taken from scanners have a consistent noise profile. AI-generated images often have uniform noise or noise that varies unnaturally across the frame.
- JPEG artifact analysis — The compression artifacts in AI-generated images behave differently from camera-captured or scanner-captured images. The difference is measurable at the block boundary level.
- Metadata forensics — AI generation tools often leave identifiable fingerprints in EXIF data, XMP metadata, or embedded color profiles. Stripping metadata is itself a suspicious signal.
Content-Level Cross-Reference
Beyond pixel analysis, SignalStack verifies the content of the invoice against independent data sources. The business verification API (/product/business-verification) checks whether the supplier's name, VAT number, address, and bank account are consistent with registered business data. If the invoice claims to be from Acme Corp but the bank account on the invoice doesn't match Acme Corp's registered payment details, the document receives a low trust score regardless of how visually convincing it is.
This cross-reference layer is critical because it attacks the fraud from a different angle. The forger can generate a perfect visual replica, but they cannot control what the real supplier's bank account number is. The verification system doesn't need to detect the forgery in the document — it just needs to detect the inconsistency between the document and reality.
Generation Provenance Detection
Emerging research in media provenance detection enables classifiers that can identify which model generated a given image with over 90% accuracy for known architectures. SignalStack's /product/media-provenance module runs ensemble classifiers trained on outputs from major generation models (DALL-E, Midjourney, Stable Diffusion, Firefly) and can flag documents that show signs of synthetic generation, even when the artifacts are invisible to the human eye.
Integration into AP Workflows
The document analysis API is designed to integrate directly into existing AP automation pipelines. An invoice arrives as a PDF or image, is passed to the API, and returns within 500 milliseconds with a trust score, flagged dimensions, and an evidence report. Based on the score, the AP system can:
- Automatically approve invoices with trust scores above 85
- Flag invoices between 60 and 85 for enhanced supplier verification
- Block and escalate invoices below 60 for fraud investigation
- Extract and record all evidence chain data for audit compliance
Deploy document analysis in a shadow mode before enforcing blocks. Run the API alongside your existing AP process for two weeks and compare the trust scores against actual fraud outcomes. This gives you a baseline for your organization's specific fraud patterns and lets you calibrate thresholds without disrupting operations. Most teams find that the optimal threshold for automated approval lands between 80 and 90, but the right number depends on your fraud rate, invoice volume, and risk tolerance.
The Arms Race Ahead
Deepfake invoice fraud is not a problem that will be solved once and forever. It is an arms race. As detection methods improve, generation methods will adapt. SignalStack's approach to this arms race is structural rather than reactive: instead of trying to detect every individual forgery technique, we verify documents against ground truth — registered business data, known supplier relationships, cryptographic provenance chains. A deepfake can mimic a document perfectly, but it cannot control what the real supplier's bank account is.
This structural advantage means that even as generation quality improves, verification remains effective. The goal is not to build a better forgery detector — it is to build a system that doesn't need one, because it verifies the claim rather than the document.
Conclusion
Deepfake invoices are not a future threat. They are a present one, and they are growing in volume and sophistication. Traditional AP security — visual inspection, OCR matching, manual verification — is no longer sufficient against AI-generated forgeries. SignalStack's document analysis and media provenance APIs provide the technical foundation for a new approach: verify every invoice against independent ground truth, not against its visual appearance. Start with /product/document-analysis to secure your AP pipeline against the coming wave of generative fraud.
Luke Swestun is the founder of SignalStack. He writes about trust infrastructure, hallucination detection, and building AI agents that can verify before they act.