Most AutoQA engines score what they can read easily, which means they score text and skip everything else. When a customer sends a screenshot of a failed payment, a PDF bank statement, or a voice note explaining a broken feature, many automated quality assurance tools either ignore the attachment entirely or score the ticket as if the attachment never existed. That is a scoring gap, not a minor edge case: in fintech, travel, and e-commerce, attachments frequently carry the actual evidence the agent needed to resolve the case, so any AutoQA engine that cannot process them is scoring an incomplete conversation and calling it complete.

TL;DR

• Attachments (screenshots, PDFs, voice notes) often contain the decision-critical information in a ticket, yet many auto QA tools score only the text thread around them.
• Large language models face real technical constraints on non-text content: token limits, semantic search instead of full in-context reading, and parsing errors when PDFs are processed as images.
• There is no universal industry standard for scoring non-text interactions, so most teams still rely on manual, subjective review for anything outside plain text.
• A QA scorecard that skips attachments will systematically under-score agents who resolve cases correctly using visual or audio evidence, and over-score agents who miss what the attachment showed.
• AutoQA that scores 100% of conversations, including attachments, closes a gap that sampling-based manual QA never had the throughput to close in the first place.

About the Author: This article is written by the team at Revelir AI, which builds RevelirQA, an AutoQA scoring engine currently running in production at Xendit and Tiket.com, evaluating thousands of customer service conversations a week across text, image, and document-heavy support flows in fintech and travel.

What Is the Attachment and Screenshot Problem in AutoQA?

The attachment and screenshot problem is the gap that appears when a customer service conversation depends on a non-text artifact, an image, PDF, or voice note, but the AutoQA system scoring that conversation only processes the surrounding text. auto QA, at its core, is the automatic evaluation of recorded conversations against a defined QA scorecard, producing a per-call or per-ticket score along with the reasoning behind it [voxjar.com]. That definition assumes the system has access to the full conversation. In practice, a large share of support interactions do not resolve in text alone.

Consider a fintech dispute ticket. A customer sends a screenshot of a failed transfer, the agent asks a clarifying question, the customer replies "see attached," and the agent resolves the case by referencing details visible only in the screenshot. Read the text thread alone and the ticket looks thin, maybe even non-compliant. Read the attachment and the agent followed the SOP exactly. An AutoQA engine that only ingests text will score that agent incorrectly, and it will do so consistently, every time a similar case comes through.

Why Do LLM-Based AutoQA Tools Struggle With Images, PDFs, and Voice Notes?

Building on the scoring gap above, the harder question is why this happens at a technical level rather than assuming it is just a product oversight. Major LLMs face strict file size and token limits when processing non-text content, and they often rely on semantic search rather than full in-context reading when a document is large. That means a model may retrieve a relevant snippet of a PDF rather than reading the whole document the way a human reviewer would.

Three specific technical constraints explain most of the failure cases:

• Token consumption on images. Processing a PDF or screenshot as an image consumes significantly more tokens than processing the equivalent plain text, and it introduces parsing errors that plain text does not have.
• Layout blindness. Models struggle to parse complex multicolumn layouts and to interpret spatial relationships in diagrams, tables, or annotated screenshots, exactly the kind of content that appears in bank statements, error screens, and boarding pass confirmations.
• Retrieval instead of reading. For long documents, many systems fall back to retrieving the most relevant chunk rather than reasoning over the full file, which means a detail that matters to the QA scorecard but sits outside the retrieved chunk gets missed entirely.

Here is the mechanism worth understanding: a text transcript is already tokenized in the format a model reasons over natively, so scoring it is close to a direct read. An image or scanned PDF has to be converted into that format first, through OCR or a vision encoder, before the model can reason over it at all. Every conversion step is a place where information can be lost, garbled, or approximated. That is why an AutoQA engine that treats attachments as an afterthought is not being lazy, it is running into a real computational cost and accuracy tradeoff that text-only scoring never has to face.

Is There an Industry Standard for Scoring Non-Text Interactions?

No, and that absence is itself a useful data point. There is no single universally adopted industry standard for scoring non-text customer service interactions. Traditional quality assurance has typically relied on subjective individual interpretation, where a QA reviewer looks at a ticket, forms a judgment call about the screenshot or voice note, and scores based on that judgment.

Organizations that have tried to formalize this instead build custom frameworks: calibrated scorecards paired with AI speech analytics to measure specific metrics like sentiment, compliance, and resolution effectiveness. That custom-built approach works, but it means every company is solving the attachment problem from scratch, and most manual QA programs never get past the sampling stage to build it properly. auto QA is meant to replace manual sampling and scale quality checks across every ticket [fin.ai][gorgias.com], but if the "auto" part of AutoQA stops at plain text, the scorecard is only automated for the easy half of the conversation.

How Should an AutoQA Scorecard Actually Handle Attachments?

A related but distinct question from the technical constraints above is what a well-designed QA scorecard should require once those constraints are accounted for. Several vendors in the auto QA space have started building toward this, with context-aware checks that flag process gaps specific to a customer's situation before they become escalations [evaluagent.com], and evaluation of the full 100% of conversations rather than a sample [miarec.com][intercom.help]. The scorecard itself needs three things to handle attachments properly:

Scorecard Requirement	Why It Matters
Attachment ingestion, not just detection	Flagging "an attachment was sent" is not the same as reading its content against policy. The scorecard needs the actual content extracted and evaluated.
Policy retrieval before scoring	An agent's response to a screenshot is only correct or incorrect relative to the company's own SOP for that scenario, not a generic benchmark.
Traceable reasoning	If a score depends on a PDF or voice note, the QA team needs to see what was extracted and why the model scored it the way it did, especially in regulated industries.

Some AutoQA platforms also focus on ignoring internal notes, automated flows, and rules-based messages so the score reflects only genuine customer-facing conversation [docs.gorgias.com]. That same filtering discipline should apply in reverse to attachments: a customer-sent screenshot that carries the resolution evidence should never be filtered out just because it is not a text message.

How Does RevelirQA Approach Attachment-Heavy Conversations?

Stepping back from the general framework, this is where the design choices behind RevelirQA become relevant. RevelirQA is built to score 100% of customer service conversations against a company's own policies and SOPs, retrieved through a RAG pipeline before every evaluation, rather than against a generic benchmark. For teams in fintech and travel, where transaction screenshots, e-tickets, and payment confirmations are routine parts of a support thread, scoring only the text around those artifacts would miss the evidence that actually determines whether the agent handled the case correctly.

Every RevelirQA score carries a full reasoning trace: the model used, the documents retrieved, and the reasoning applied. For a ticket involving a PDF statement or a screenshot, that trace lets a QA lead see exactly what informed the score, which matters in compliance-sensitive environments like Xendit's, where RevelirQA runs against thousands of tickets a week, not a sample. The same consistent QA scorecard applies whether the conversation involved a human agent or an AI chatbot, so a company running both gets one coherent view of quality instead of two disconnected ones.

Frequently Asked Questions

Does AutoQA replace manual QA sampling entirely?
AutoQA is designed to replace sampling-based review by scoring every conversation instead of the 1-5% a manual team typically has time to check [fin.ai][gorgias.com]. Manual review still has a role in calibration and edge-case judgment, but as the primary coverage mechanism, sampling cannot compete with 100% automated scoring on volume.

Can auto QA tools read voice notes accurately?
Voice notes first need to be transcribed, then scored against the scorecard like any other text. Accuracy depends on transcription quality and, for non-English languages, on whether the underlying model has been proven in that language.

Why do some AutoQA tools skip attachments instead of scoring them?
Processing images and PDFs costs more in tokens and introduces more parsing error than plain text, so some systems deprioritize or skip non-text content to keep scoring fast and cheap. That tradeoff shows up as a coverage gap in the final QA scores.

What is the difference between "AutoQA" and "auto QA"?
They refer to the same category: automated quality assurance that scores customer service conversations against a defined scorecard without manual sampling [voxjar.com][fin.ai]. Both spellings appear across the industry, and both describe the same underlying practice.

Should QA scorecards weight attachments differently than text?
Not necessarily differently, but the scorecard criteria that reference an attachment (for example, "agent correctly identified the error shown in the screenshot") need to be scored against the attachment's actual content, not inferred from the surrounding text alone.

Is there a standard framework for scoring image or PDF-based support tickets?
No formal universal standard exists yet. Most organizations build custom scorecards calibrated internally, often combined with AI-driven analytics for sentiment and compliance metrics, rather than adopting an industry-wide framework.

About Revelir AI

Revelir AI builds RevelirQA, an AI AutoQA engine for customer service that scores 100% of support conversations against a company's own policies and SOPs, ingested via RAG rather than generic benchmarks. Founded in 2025 by Rasmus Chow and headquartered in Singapore, Revelir AI is built for global enterprise QA teams, with production deployments running at Xendit and Tiket.com, processing thousands of tickets a week across multiple languages including English, Indonesian, Thai, and Tagalog. Every RevelirQA score carries a full audit trail, including the model, retrieved documents, and reasoning, and the platform evaluates human agents and AI chatbots on the same consistent QA scorecard. Revelir AI integrates with any helpdesk via API, with particular strength in high-volume, digitally-native businesses across fintech, travel, and e-commerce.

If attachment-heavy conversations are slipping through your current QA process unscored, it's worth seeing how a scoring engine built for the full conversation, not just the text, handles them. Learn more at Revelir AI.