Recently I created a proof of concept document translation pipeline for a global client, orchestrated via Optimizely Opal.
Effective automation of file translation is an interesting problem and one that applies in various forms to many customers so I was keen to see how agents and the orchestration workflows would rise to the challenge. Throughout, it was apparent that the main driver of success wasn’t how effective the agent or prompts were, but about looking at a problem, working out which parts of it genuinely require reasoning and which parts just need ‘computing’, and then building an orchestration that can deliver both.
The starting point was the obvious one, which was to throw a sizable document at a language model and see what happens.
Unsurprisingly, that failed comprehensively, and then the useful work began: taking the problem apart, classifying each piece, and assembling something where the agents do the part only agents can do and deterministic code does everything else.
This reflection is about how that classification went, what it cost to get wrong, and what the resulting workflow looks like.
The client produces training materials in a range of formats, including SCORM packages for their learning platform and most commonly consisting of large training presentations in PowerPoint and supporting collateral in Word.
For the proof of concept I focused on some PowerPoint decks and Word documents, because between them they offered the right mix of complexity and scale to surface the problems worth knowing about. A ninety-plus slide deck has nested shapes, tables, text boxes on masters and layouts, speaker notes, and enough volume to break anything that only works on a small file. A Word handout is simpler but structured differently enough to stop us building something that only worked for slides. If the approach held for both, it stood a reasonable chance of being scalable to the rest.
Another key element is that the output must preserve the original formatting exactly, because these are teaching materials used within accreditation contexts.
What makes this an architecture problem rather than a translation problem is repetition. The pilot language was French, but with an eventual target of ten to fifteen languages. Translating one deck by hand is tedious but perfectly possible. Translating a full syllabus into fifteen languages (and then doing it again each time the source material is revised) is something organisations would struggle to justify without effective automation. Any step requiring a person to intervene per document, or per language, fails at exactly the point it starts to matter.
The first approach treated the whole thing as an agent problem. An agent reads the document, an agent translates it, an agent writes the result back. That is the shape the tooling nudges you towards, and it came apart at the first step.
Asked to open a PowerPoint file and report its contents, the agent returned slide content that was well-formatted, plausible, and in places simply invented. Not garbled or truncated, but confidently wrong in ways that took a careful read of the original deck to catch. No amount of prompt work fixes this, because it is not a prompt problem. A binary Office file is not a text-reading task so a model asked to perform one will generate something shaped like the answer.
That failure was helpful, because it exposed the underlying question. Not "how do I get the model to read this properly", but "what kind of problem is reading a file, and is that the same kind of problem as translating a sentence?"
Before going through the pipeline, it’s worth unpacking the distinction, because it is more useful than the more typical framing of "things AI is good at" and "things it’s bad at".
A compute problem has exactly one correct output. Given the same input twice, the right answer is the same both times, and you can tell whether you got it by checking rather than by forming an opinion. Reading a file, counting records, matching a string to a position, converting a format. The solution (some kind of script) is cheap, runs in milliseconds, is testable, and when it fails it does so loudly and in the same way every time. It’s not about complexity – some of these processes can be extremely convoluted – it’s about the output “correctness” being objective. It’s either right or wrong.
A reasoning problem has more than one defensible output and no fixed way to pick between them. It needs context, and it needs something that can weigh one option against another. Two competent translators will render the same sentence differently and both be right. The solution (some form of model attached to a LLM), needs context and – critically – it carries AI consumption costs per operation. It is also the only kind of problem where you get value from the model rather than the convenience of not having to build something.
Most of the work in designing this pipeline was sorting one from the other. I went through each step and asked the same thing: is there exactly one correct output here, or does something have to be worked out ‘intelligently’?
Reading the source file: compute. Here there’s one correct outcome, so we want a scripted deterministic process. In the pipeline I created this using Python to read all text-based content within the file or it errors, and it cannot invent content. The extraction produces batched JSON that everything downstream works from.
Applying approved terminology: reason, but bounded. The content carries a large specialist vocabulary that must render consistently across every batch, document and eventually language. Spotting that a phrase is one of those terms, in context, where the same words might be ordinary prose elsewhere in the sentence, needs reasoning. Deciding what that specialist term should be in French does not, and should never be left to inference.
So the knowledge sits in layered reference files, global with locale-specific files taking precedence, and an agent scans each batch against them and annotates. Every agent prompt stays language-agnostic. Everything language-specific lives in a file a linguist can edit without touching the workflow or code.
Translating: reason. Pure language translation, how a phrase sits in the space a slide gives it, whether something that works in English has an equivalent that works at all. This is what the model is for.
Verifying the output is complete: compute. Counting how many segments went in and how many came back is maths. More on why this matters below.
Writing the translation back into the document: compute. There’s one correct position for each translated string in the output file, and getting it wrong could happen silently with an ‘intelligent’ approach. So again this should be tackled in a fixed process that escalates in the event of any mismatches.
Two of those five need a model. The other three are compute problems where an agent would only perform slower and more expensively, and likely give a poorer outcome.
The part I didn’t anticipate was the importance of correctly orchestrating the workflow, and this is where Opal really earned its place.
During early iterations with large PowerPoint files, I hit limits on how much data models will accept in one go. Things broke in seemingly unpredictable ways until this became apparent. Anything of real size has to be broken into sections and fed through in sequence.
Fixing this turned out to have a secondary benefit: once the work is sliced, you can run a single slice. Testing a change against ten slides rather than the full deck took the feedback loop from several minutes to something you could watch in real time, and a failure cost significantly less consumption than a whole run. This also gave a clean way to see where the money was going, because comparing the cost of a run at different batch sizes separates what you pay per unit of content from what you pay simply for making the call. That distinction turned out to matter more than I expected (I’ll come back to this later).
The workflow as it runs now is: a webhook trigger, a preparation step that sets up the batching, then a loop for the batches. Inside the loop, two agents run in sequence, the terminology scan and the translation. When the loop completes, a merge trigger fires, and then a reconstruction trigger. Both of those last two are Opal steps whose entire job is to invoke processes that run outside Opal entirely.
What the whole pipeline produces is then dropped into cloud storage where a web front end can reach via the same page the user uploaded the input file from. That round trip is deliberately crude, because it’s a proof of concept and a real deployment would likely handle delivery, notification and access control quite differently. But it closes the loop and means I can demonstrate the whole pipeline rather than fragmented elements of it.
I had initially assumed this workflow would be mostly agents with a few helper scripts attached. It turned out the other way round. Most of the actual work happens in Python running in GitHub Actions, and what Opal does is conduct it: batching, sequencing, looping, error boundaries, handoffs. The agents are two elements in that process, doing the things only they can do.
Getting to that shape meant taking things out. An earlier version had a separate agent whose only job was committing translated batches to GitHub, and another QA agent after translation. Both were removed, on the same reasoning: neither was doing anything that could not be done directly by deterministic code. The commit responsibility folded into the translation agent and the loading step folded into the terminology scan.
There was a second reason for removing them, which was to find out how much of the credit consumption came from having an agent in the workflow at all, as opposed to the work it was doing. That turned out to be worth measuring. Every agent call carries a fixed cost before it does anything useful: the instructions it needs, the context it has to be given, the overhead of the call itself. That floor is much the same whether the agent is doing something hard or something trivial. So a step that only commits a file or fetches a document is paying close to full price for very little, and a workflow with several of those can quickly become expensive.
This one cost a full production run to diagnose. A ninety-nine slide deck came back with slides 41 to 60 entirely untranslated. No error, no warning, no partial output. Those slides had passed through the workflow unchanged.
What happened was that the data moving from one agent to the next was being silently truncated. There’s a size ceiling on inline transfer between steps, the batch had grown past it, and everything beyond the cut-off simply stopped existing. No part of the workflow noticed, because from every step's point of view it had received a perfectly valid payload. It was just a shorter than the one that had been sent.
The reason this took longer for me (a non-developer) to diagnose is that it didn’t seem like an obvious place to start looking for the issue. On a platform where you assemble steps visually, the wiring between them presents itself as something the platform handles, and so it is the last place you think to look when output goes missing. And it failed in the worst way available, which is quietly and only at scale.
The simple solution was to stop moving payloads between steps and start moving references to them. Push the data somewhere durable, hand the next step an address, and let it fetch what it needs. That cost one extra round trip and removed an entire class of failure, and it has the useful side effect of leaving the intermediate state somewhere I could use for easier debugging.
The only reason that truncation surfaced is a check in the merge step that counts segments coming out of translation against the number that went in. Any shortfall fails the run with a non-zero exit code rather than producing a quietly incomplete document.
That check has now fired twice in production. Once for the truncation problem, and once for an ordinary case of the model omitting a single segment, recovered by re-running that batch alone.
This is the clearest case of the compute-versus-reason split paying off. Working out whether something is missing is a counting problem, and counting is cheap and reliable. It is also close to the worst thing you can ask a model to do, because an absence leaves nothing in the context for it to react to. A model reviewing a batch of translations will tell you about the translations in front of it. It has no way of knowing about the ones that never arrived.
Having to batch the work meant I could vary the batch size and watch what happened, and while I assumed smaller batches would be slightly more expensive due to the cost per loop, this came out to be far more significant that I’d expected. Batches of twenty-slides proved meaningfully more efficient than ten-slide batches, and so it was important to walk the line between what minimises the overall consumption and what risks failing by exceeding LLM limits. So if you’re managing credit consumption, measure it rather than making assumptions.
The pipeline was able to handle test PowerPoint decks around 100 slide at a 99.02% automated handle rate, and simpler Word documents at 100%. Upload the file, choose language, wait, and the same page offers back a translated document along with a correction report listing anything the reconstruction could not place.
The residual 0.98% covers text that’s structurally inaccessible to the libraries doing the reconstruction, such as text embedded in imagery or locked inside SmartArt graphics. These need manual handling or a change to how the source materials are authored.
As flagged at the start, this was always intended as a proof of concept. Two formats, one target language, one (non)developer, and using simple free infrastructure and access to Opal. It established that the architecture can really work. Turning this into something a business can rely on is a separate job, covering the remaining source formats, non-Latin scripts with their own font and layout problems, and non-technical aspects such as organisational ownership and governance.
But it’s easy to see how these issues are surmountable and so the principle of using Opal to run an end-to-end translation pipeline is very achievable. Coupling this to capabilities that are more aligned to Optimizely’s wheelhouse, such as pushing this into content operations and you can see how powerful this can become to drive value.
In approaching this challenge it was really helpful to determine which problems would result in a single correct output and which would require reasoning. Giving the wrong problems to agents makes outcomes slower, more expensive and less reliable. The value then came from putting each piece where it belonged, then using the agentic workflow to hold the whole thing together, including the parts that are not agentic at all. I assumed that the workflow was simply scaffolding around the interesting elements, but in this case it turned out to be the most critical part of the design work.
If you’re interested in finding out more about this project, agent creation, or agentic workflows in action then feel free to get in touch.