A claims adjuster opens a PDF that looks like a policy endorsement. The words resemble a regulatory affidavit filed last quarter. One misfile sends the document down the wrong compliance track, and the error surfaces weeks later as a processing delay or a regulator’s flag. Insurance teams sort thousands of these forms every day, and the documents that cause the most trouble are the ones that look alike.
AWS published a reference build that attacks this problem with three collaborating AI agents instead of one oversized model AWS Machine Learning. The design pairs Anthropic’s Claude Haiku 4.5 for reading text with Amazon Titan Multimodal Embeddings G1 for reading layout, then lets a third agent arbitrate between them. By the end of this guide you will have a working classifier you can point at your own PDFs.
Amazon Bedrock noted the failure mode directly: “In our testing, single-model approaches struggled with edge cases and complex documents that require both textual and visual analysis.” The three-agent design is built to answer exactly that limitation.
Why near-identical insurance forms defeat single models
Single-model classifiers break on documents that share vocabulary but serve different legal purposes. A policy endorsement and a sworn affidavit can use the same clause language, yet routing one to the other’s queue breaks the audit trail. Manual triage is slow and scales poorly past a few hundred files a day.
The cost shows up in audit findings, not just slow queues. A misrouted affidavit can sit in a policy workflow for months before anyone notices the mismatch, and fixing it after the fact costs far more than catching it at intake. That asymmetry is why teams keep throwing human reviewers at the problem instead of automating it.
The reference system treats text and visual structure as separate evidence streams. One agent reasons about what the words mean; another measures how the page is laid out. A third agent compares their verdicts and only escalates to a human when the two disagree. That separation is what keeps similar-looking forms from collapsing into the same bucket.
The three-agent architecture, drawn out
The orchestration hinges on three specialized agents built with the Strands Agents SDK AWS Machine Learning. The Document Analysis Agent runs Claude Haiku 4.5 to interpret legal language and label each file POLICY, AFFIDAVIT, or MISCELLANEOUS. The Vector Similarity Search Agent uses Titan Multimodal Embeddings G1 to match a page’s visual structure against known templates.
Amazon documents the Titan family’s multimodal embedding model as a purpose-built engine for turning images and text into comparable vectors Amazon Bedrock docs. That capability is what lets the Vector Similarity Search Agent recognize a form by its shape instead of its sentences.
Provision Bedrock models and the Strands SDK
You need an AWS account with Bedrock access and IAM permission to invoke foundation models, plus enabled access to both Claude Haiku 4.5 and Titan Multimodal Embeddings G1. The walkthrough also assumes a current Python 3 environment, AWS CLI version 2, and the Strands Agents SDK with the FAISS library installed.
Start by wrapping the Claude model in a small factory so every agent shares one configured client. Keep the temperature low so classification stays deterministic, and set a generous token ceiling for long filings.
def create_bedrock_model(
model_id: str = "anthropic.claude-haiku-4-5-v1:0",
temperature: float = 0.1,
max_tokens: int = 4096,
) -> BedrockModel:
return BedrockModel(
model_id=model_id,
region_name="us-east-1",
temperature=temperature,
max_tokens=max_tokens,
)
For lower latency and higher availability, the authors suggest a cross-Region inference profile by prefixing the model ID with a geographic hint such as us., eu., or ap. that matches your deployment Region.
Write the Document Analysis Agent
The text agent returns a structured object so downstream code can parse it without scraping prose. Define a Pydantic model that pins the allowed categories and forces the model to explain its reasoning.
class TextualAnalysisOutput(BaseModel):
classification: str = Field(description="POLICY, AFFIDAVIT, or MISCELLANEOUS")
confidence: float = Field(description="Confidence score from 0.0 to 1.0")
reasoning: str = Field(description="Why this classification was chosen")
@tool
def analyze_document_text(document_text: str) -> str:
agent = Agent(
model=create_bedrock_model(),
system_prompt=TEXTUAL_ANALYSIS_PROMPT,
conversation_manager=NullConversationManager(),
)
result = agent(document_text, structured_output_model=TextualAnalysisOutput)
return str(result.structured_output)
The prompt itself does the heavy lifting. It names the agent as a document analysis specialist, lists the three permitted categories with examples, and demands a confidence score plus the textual features that drove the decision. That reasoning field is what later lets a reviewer see why a file landed in a bucket, not just which bucket it landed in.
Add the Vector Similarity Search Agent
This agent ignores the words and judges the document by its visual skeleton. It embeds the first page with Titan Multimodal Embeddings, then queries a FAISS index built from your own known templates to find the closest matches.
The FAISS library performs the efficient nearest-neighbor lookup that makes layout matching fast even across thousands of templates Faiss documentation. The agent turns the top matches into a label and a confidence derived from how consistently those matches agree.
@tool
def analyze_document_text_layout(pdf_path: str) -> str:
base64_pages = pdf_to_base64(pdf_path)
first_embedding = create_multimodal_embedding(image_base64=base64_pages[0])
results = db.similarity_search_by_vector(embedding=first_embedding, k=3)
labels = [r.metadata["label"] for r in results]
classification = Counter(labels).most_common(1)[0][0]
confidence = labels.count(classification) / len(labels)
agent = Agent(model=create_bedrock_model(), system_prompt=VECTOR_ANALYSIS_PROMPT)
return str(agent(
f"Top match: {classification} (confidence: {confidence:.2f})",
structured_output_model=VectorAnalysisOutput,
))
Loading the FAISS index uses allow_dangerous_deserialization=True because FAISS pickles internally; the authors note this is safe only because you built the index yourself from your own training documents.
Build the template index the vector agent searches
The vector agent is only as good as the templates you give it. Before it can classify anything, you embed a set of known-good documents, one per type, and store each embedding alongside its label in a FAISS index. The agent then compares an incoming page against that index rather than against free text.
You build this index from your own corpus because the layout patterns that matter are specific to your forms, not to documents in general. The source walkthrough loads it once at module level and notes that the pickle deserialization is safe precisely because the index came from your own training documents. Refresh the index whenever your form designs change so the visual matcher does not drift.
Let the Validation Agent orchestrate both
The orchestrator is where the “agents as tools” pattern earns its keep. Strands lets you register the two specialist functions as tools the validator can call, so you write no custom coordination logic Strands Agents SDK. The validator calls both, then synthesizes a single answer.
When the two specialists agree, the validator reports high confidence; when they split, it trusts text for content-rich documents and visuals for distinctive layouts. Very short or low-confidence text defers to the vector agent, endorsements count as POLICY documents, and contradictory evidence triggers a human review flag.
class OrchestratorOutput(BaseModel):
final_classification: str
confidence: float
requires_human_review: bool
decision_logic: str
justification: str
class MultiAgentDocumentClassifier:
def __init__(self, model=None):
self.orchestrator = Agent(
model=model or create_bedrock_model(),
tools=[analyze_document_text, analyze_document_text_layout],
system_prompt=ORCHESTRATOR_PROMPT,
)
When the orchestrator returns, read requires_human_review before trusting final_classification. A false value means both specialists agreed or one clearly dominated; a true value means the evidence was contradictory and a person should decide. The justification field carries the combined reasoning from both agents, so a reviewer can act without re-running the pipeline.
Classify a real document and read the score
With the classifier constructed, classification is a single call. The method extracts the first 3,000 characters of text, passes the PDF path to the vector tool, and asks the orchestrator to use both specialists before deciding.
def classify_document(self, pdf_path: str) -> OrchestratorOutput:
text_content = extract_pdf_text(pdf_path, max_chars=3000)
query = f"Classify this insurance document: {os.path.basename(pdf_path)}\n{text_content}"
result = self.orchestrator(query, structured_output_model=OrchestratorOutput)
return result.structured_output
classifier = MultiAgentDocumentClassifier()
result = classifier.classify_document("documents/sample_policy.pdf")
print(result.final_classification, result.confidence, result.justification)
Route on the score, not the label. Treat any result the validator flagged for human review as a manual queue item, and let high-confidence matches flow through automatically. Your own corpus will set the real baseline once you measure a labeled sample, so instrument the review queue from day one.
The authors measured the workload at an average of 19.3 seconds per document while holding 93 percent confidence, the trade-off they accepted for Haiku 4.5’s lower latency and cost against larger models AWS Machine Learning.
The same multi-agent pattern is spreading across Bedrock builders. OpenAI’s Daybreak cyber models now run on the platform zBrandCo, and nOps published FinOps agents on Amazon Bedrock zBrandCo. The classifier here is a small, self-contained instance of that broader move toward specialized agents.
That measurement discipline is where this build stops being a demo and becomes operations. The average latency is a starting point on Haiku 4.5, not a promise for your documents or your Region, and the confidence band only means what your labeled test set says it means. Build the labeled eval set first, watch the human-review queue, and let the agents earn autonomy one clean batch at a time.
