Teaching Without Translating

1. A media file — q3-planning-call.mp4 — open in the library with its duration, resolution and tags.
2. Transcript. Speaker-labelled utterances with timestamps. Marcus and Dana discuss a migration window; one block highlights as playback reaches it.
3. Subtitle generation. A language is typed into the picker — Vietnamese — and generation runs.
4. Subtitles. English and Vietnamese both listed as Ready, each offering Download and Export video.
5. Playback. The video plays with subtitles burned into the picture rather than layered over it.
6. Summary. A generated summary, a chapter list with real timestamps, and generated topics and tags.
7. Search. A term is typed and the transcript filters to matching blocks, each with its timestamp.

Source is public under MIT: github.com/dallanj/language-barrier.

Problem

The client needed subtitled video for a scheduled course. The platform was mid-build and nowhere near able to serve her. Two obvious options were both bad: rush the platform into a state that met her deadline and live with what that cost the architecture, or tell her the tool wasn’t ready.

Her problem was narrow, well-defined, and completely separable from the platform — one person, one machine, files in and files out. It needed no auth, no database, no UI, none of the things that make the platform take two years.

Solution

A CLI pipeline operating entirely on local files:

1. Upload to AssemblyAI, transcribe with automatic language detection
2. Poll the job asynchronously to completion
3. Fetch the transcript as WebVTT
4. Parse the cues, translate through Google Cloud Translation, rebuild the VTT
5. Burn the subtitles into the video with FFmpeg — one output per target language

Technical Highlights

• WebVTT as the interchange format, adopted immediately rather than at the end. The alternative was passing transcript JSON between translation and burn-in and converting last. Standardizing on VTT the moment a transcript arrives keeps every intermediate state human-readable — I could open a file mid-pipeline and see whether a translation was sane — and FFmpeg’s subtitles filter consumes VTT directly, so nothing needs converting before burn-in.
• Every stage independently re-runnable. One function per stage, each output feeding the next. The script prompts for an existing transcript ID so transcription can be skipped entirely, and translation checks for its own output before spending API quota. Not tidiness: transcription costs money and minutes, and the first several attempts at subtitle styling were wrong. Re-running only the burn-in stage is the difference between iterating on legibility in minutes and in hours.
• Translation batched in chunks of 128 cues rather than one request per cue — inside request-size and rate limits, with per-cue timestamp alignment intact.
• Live progress on both long operations. A CLI tool that goes silent for eleven minutes on a long video is a tool the operator kills.

Engineering Considerations

The difficult parts were not the API integrations.

Subtitle legibility took real trial and error. Burned-in text is permanent — there is no “close enough” to pass through. Font, outline and positioning all had to survive text sitting over a bright factory floor with white PPE in frame.

FFmpeg’s non-blocking stderr pipe had to be read for progress without hanging the process or pinning a core in a tight loop, resolved with a usleep(100000) interval.

Escaping paths for the subtitles= filter is a separate escaping layer from shell escaping — colons and backslashes are meaningful inside FFmpeg’s filter syntax. Getting one layer right and the other wrong produces an error that points at neither.

Outcome

• Delivered Vietnamese-subtitled video in time for the course. The instructor taught it without translating aloud over her own material for the first time.
• Kept the platform build uncompromised by a deadline that wasn’t the platform’s.
• Published under MIT and left standing rather than folded away — an honest artifact of what solving the problem urgently looked like.
• Its logic is now in production inside the platform. SubtitleBurnService carries a docblock naming the script as its origin: identical libx264/AAC re-encode and identical subtitle styling, with proc_open()/shell_exec() replaced by Symfony Process, which handles argument escaping itself and exposes a progress callback without the manual non-blocking pipe reads the script needed.

Problem

An upload is not one operation. It is: receive bytes, decide what kind of file this is, check whether the user is allowed to store it, probe it, transcode it, generate a thumbnail, a preview clip and an animated preview, persist several related rows, then hand off to transcription.

Any of those can fail. Several of them write files to disk before the failure. A database transaction rolls back rows; it does not roll back an 80MB preview clip that FFmpeg already wrote.

Three specific failures made this concrete:

• A QuickTime voice recording is video/quicktime with no video track. The pipeline was chosen from the MIME prefix, so it was bound to the video processor before anything inspected the file. Downstream, ->videos()->first() returned null and ->get() on it produced Call to a member function get() on null — a message describing the symptom and naming nothing.
• Photos never uploaded at all. The type column normalized image to photo; the pipeline selector didn’t, so getPipelines() fell through to its default and threw Unsupported media type: image for every image.
• Two concurrent uploads could both pass a storage quota check and both commit, because request-time validation reads a total that the other upload hasn’t written yet.

Solution

One resolver, consulted once. MediaTypeResolver is the single source of truth for which pipeline a file belongs to. It probes with php-ffmpeg — the same library the pipelines themselves use — so the routing decision and the processing agree by construction rather than by coincidence. Crucially it treats “tagged as video” and “carries a picture” as different questions: a track can be marked video and hold no image, which is exactly what QuickTime timecode and metadata tracks do.

return $this->hasPicture($storedPath) ? self::VIDEO : self::AUDIO;

MediaService resolves the type once and passes it into the pipeline as data. CreateMediaModel is explicitly forbidden from re-deriving it.

Quota checked under a row lock, inside the transaction. CheckStorageQuota runs first and takes lockForUpdate() on the user row, using it as a mutex for the transaction’s duration. A second concurrent upload for the same user blocks there until this one commits or rolls back — so by the time it re-reads used bytes, it sees this upload’s committed bytes. Request-time validation stays in place as fast feedback; it simply cannot close that window on its own.

Progress driven by named stages, not hand-guessed percentages. Stage weights live in config/processing_stages.php per job type and subtype, and pipeline stages call advanceStage('transcoding') rather than setting a number. Progress is the cumulative weight up to that stage, capped at 99 — only markCompleted() is allowed to say 100.

Technical Highlights

• tries = 1 on the upload job, deliberately, with the reason in the code. An uploaded file isn’t safely retryable without re-sending the bytes. Because that means a failure is final, the job cleans up its own stored file in every catch branch — otherwise every failure would leave the raw upload in uploads/pending forever.
• Quota rejection is logged at info, not error. Exceeding your plan’s limit is an expected, user-facing outcome, not a defect. Logging it as an error trains you to ignore errors.
• Failure logs record origin and a trimmed trace, because the message alone wasn’t diagnosable. "Call to a member function get() on null" names neither the probe that returned null nor the stage it happened in. Failures now capture the exception class, the base-path-relative file and line, and the first six frames — enough to diagnose from the log instead of by reproduction.
• The outer try/catch inside ProcessVideo was removed on purpose. Swallowing the error there produced a log line plus a confusing null-object failure further downstream. Letting it propagate means the transaction rolls back and the job records the real message.
• Files written before a failure are tracked explicitly. Pipeline stages call trackGeneratedFile() after each write. On success it’s informational; on failure it’s the list uploads:cleanup uses to delete every artifact the pipeline produced — precisely the files a DB rollback never touches.
• Cleanup runs in three passes. Files from failed jobs (a backstop for the inline cleanup, in case the job was killed before reaching its catch block), orphaned files no ProcessingJob references at all (crashes before the job row existed), and empty directories. Orphans younger than two hours are left alone so the sweep can’t race an upload that is mid-flight. --dry-run reports without deleting.
• One event, many consumers. markCompleted() fires a generic ProcessingJobCompleted, and listeners decide what happens next per job type. That is the extension point that dispatches transcription after an upload finishes, rather than the upload job knowing about transcription.

Engineering Considerations

The listener that chains upload into transcription originally checked for video alone. Once audio files were being classified correctly rather than mis-routed, that check became the bug: a plain MP3, or a QuickTime voice recording now properly identified as audio, was stored and then silently never transcribed. No error, no failed job — a file that simply never gained a transcript.

Fixing the classifier is what surfaced it. That is the ordinary shape of this kind of work: correcting one thing exposes the code downstream that was quietly relying on the original mistake.

Outcome

• Routing, the type column, and the chosen processor became a single decision that cannot drift.
• Closed the concurrent-upload quota race that request-time validation structurally cannot catch.
• Made failures diagnosable from logs — origin, stage, and real message rather than a downstream null.
• No orphaned media on disk after a failure, including files written before the failing step.
• Progress reporting that reflects actual stage weights instead of invented percentages.

Problem

Transcription is a long-running external operation. You submit a file, and minutes later the provider calls you back. Three things follow from that:

• The webhook is not guaranteed. It can be missed, dropped, delivered while the app is deploying, or never sent. A pipeline that treats the callback as its only completion path has jobs that hang forever.
• Not every file the user uploads is a file the API accepts. AssemblyAI’s supported container list doesn’t include QuickTime .qt, so a screen or voice recording in that container is rejected at upload.
• A raw transcript is not a product. Wall-to-wall text with no structure is barely more useful than the video. It needs chapters, a summary, searchable metadata, and word-level timings for playback.

Solution

Prepare files for the API without degrading them. AssemblyAI’s own guidance is to submit audio in its native format, because re-encoding costs quality and they downsample to 16kHz mono internally regardless. AudioTranscodeService therefore makes a three-way decision rather than converting everything to MP3:

The original upload is never modified; derived files land in a separate directory so the user’s file stays exactly as they sent it.

Assume the webhook won’t come. ProcessTranscription submits the job and ends — the webhook takes it from there. transcriptions:sweep runs every five minutes with withoutOverlapping(), finds transcript jobs untouched for five minutes, and asks AssemblyAI directly what their status is. Completed ones get dispatched onward, errored ones fail, in-progress ones reset the clock. Anything still open after two hours is given up on permanently, whatever the API says at that point.

Let the model group, not measure. Chapters are generated through AssemblyAI’s LLM Gateway, replacing their deprecated auto_chapters flag. The prompt hands the model numbered paragraphs and asks for paragraph index ranges — never timestamps:

/*
 * Deliberately does NOT ask the model to invent start/end millisecond
 * timestamps itself — LLMs are unreliable at producing exact numbers
 * for data they weren't given precisely. Instead it hands back
 * paragraph indices, and resolveTimestamps() below maps those back to
 * AssemblyAI's own real timestamps.
 */

The model does what it’s good at (deciding where a topic changes) and is structurally prevented from doing what it’s bad at (producing exact numbers). Chapter timings are therefore exact by construction, not by luck.

Technical Highlights

• Output shape matched to the thing it replaced. The generated chapters return {gist, headline, summary, start, end} — the same shape AssemblyAI’s deprecated auto_chapters returned — so swapping the implementation required no frontend change at all.
• Model output is parsed defensively. The response is stripped of stray markdown fences before decoding, and a chapter referencing an out-of-range paragraph index is skipped with a warning rather than crashing the job. You do not trust a model to honour a format instruction every time.
• Enrichment is layered so cheap inputs feed expensive ones. Metadata generation (title, description, tags, topics, keywords) is fed the chapter summaries — already condensed, cheap to re-send — and only falls back to raw transcript text if chapter generation produced nothing.
• Every enrichment step is independently failable. Transcript blocks, chapters and metadata each sit in their own try/catch, and whatever succeeded is saved. A failure in chapter generation costs you chapters; it does not cost you the transcript.
• Word-level blocks built for playback, not for reading. Sentences are regrouped into blocks of roughly eight words, preferring to break at punctuation once that target is reached and hard-capping at eleven. Each block keeps every word’s individual start/end in milliseconds, which is what makes clickable timestamps and word-by-word highlighting possible.
• The LLM client is a sibling of the transcription client, not a special case. LLMGatewayClient mirrors TranscriptsClient in structure but targets a different host, because LLM Gateway is a separate service that happens to share billing and auth with the rest of AssemblyAI.
• Speaker diarization with user-assigned names. Utterances carry speaker labels; users map those labels to real names, and exports resolve them at render time rather than rewriting the stored transcript.

Engineering Considerations

The reconciliation sweep is the same pattern I built into a claims platform at an agency, where a locking system depended on a third-party WebSocket service and a payment pipeline depended on a bank’s webhooks. In both cases the answer was identical: do not trust a single event to have fired correctly — verify independently on a schedule.

It is worth being precise about why the sweep isn’t redundant with the webhook. They fail differently. The webhook fails by not arriving; the sweep fails by being slow. Running both means the common case is fast and the failure case is bounded — five minutes to notice, two hours to give up — rather than unbounded.

Outcome

• Files the API would have rejected are accepted, without re-encoding anything that didn’t need it.
• A missed webhook costs five minutes, not a permanently stranded job.
• Chapter timestamps are exact because the model was never allowed near them.
• A partial enrichment failure degrades the transcript rather than losing it.
• Word-level timings power clickable transcript navigation and synchronized highlighting during playback.

Problem

The script did this for one person on one machine. As a product it has to handle several users asking for several languages on the same media, potentially at the same time, where each request is minutes of API calls and CPU-bound encoding — and where asking twice should not cost twice.

Solution

Two generators behind one interface. GenerateSubtitleJob picks a strategy by comparing the requested language against the transcript’s own:

• NativeSubtitleGenerator — fetch the VTT AssemblyAI already produced, store it. No translation, no cost.
• TranslatedSubtitleGenerator — take the native subtitle (reusing it if it exists, fetching it if not), parse the cues, translate in chunks of 128, rebuild the VTT with original timings and translated text.

The translated generator’s reuse of the native file is the script’s idempotency idea, kept: don’t pay for work already done.

Deduplication at dispatch. SubtitleGenerationPipeline::dispatchFor() looks for an existing queued or processing job for the same media and language before creating one, and returns the existing job if it finds it. Two users asking for Vietnamese on the same video get one job and both watch it.

Timing survives translation, because timing is never translated. VttParser extracts cues; only the text array goes to the translation API; VttBuilder reassembles using the original timestamps with translated text slotted in by index, falling back to the source text where a translation is missing. Cue timing cannot drift because it never leaves the application.

Technical Highlights

• The VTT parser handles input the spec permits but tools produce inconsistently. Optional hours, either . or , as the decimal separator (SRT-style commas normalized to VTT dots), trailing cue settings like align: and position: ignored, bare numeric cue identifiers skipped, and timestamps padded to HH:MM:SS.mmm when hours were omitted.
• Burn-in ported from the script with the mechanics upgraded. Same libx264/AAC re-encode and same subtitle styling; proc_open() replaced by Symfony Process, which handles argument escaping and exposes a progress callback natively. The FFmpeg filter-syntax escaping is still there and still commented, because it is still a separate concern from shell escaping.
• Encode progress parsed from FFmpeg’s stderr into a real percentage. Duration is probed with ffprobe first; the process callback matches time=HH:MM:SS.ms on each chunk and updates the job’s progress. The user watching a nine-minute encode sees it move.
• WebM inputs are re-wrapped to MP4 on export, because burning subtitles into WebM through this filter chain isn’t reliable — the same fix the script needed, carried across.
• Timeouts matched to the work. The burn job runs with tries = 2, backoff = 30, timeout = 3600, with a comment explaining the difference: video encodes are CPU-bound local work, unlike the transcription jobs which are just waiting on an external API.
• PDF export by dompdf, chosen after Browsershot failed. Headless Chrome’s crashpad handler wouldn’t launch in this Docker environment regardless of HOME overrides or disable flags — a well-documented, unresolved issue. dompdf is pure PHP: no Node, no browser binary, no Docker changes, nothing to break on an unrelated Chrome update. The trade-off is a weaker CSS engine, so the template is written defensively with tables and floats instead of flexbox.
• Two PDF layouts, same words. Grouped by time, or grouped by speaker — with an automatic fall back to time-grouping when the transcript has no diarization to group by.
• Language list cached in two layers. A static property memoizes within the request, and Cache::flexible keeps it warm across requests with a six-hour fresh window and a seven-day stale window, so a slow upstream call never blocks a page render.

Outcome

• Requesting a language you already have costs nothing; requesting one twice runs once.
• Subtitle timing is exact after translation because timings never leave the application.
• Long encodes report real progress instead of appearing hung.
• Three export paths — subtitle file, burned-in video, PDF transcript — from one transcript.

Problem

The moment media is shared, the product stops being a media tool and becomes an access-control system.

Sharing looks like CRUD: invite a user, store a row, let them in. It isn’t. Authorization failures don’t announce themselves — a missing condition on a list query returns a page that renders correctly and shows the wrong rows to the wrong person. Nothing turns red. The only way to find these is to use the running application and notice something out of place.

Twelve merged pull requests, three of which are bugfix/ branches against work from earlier in the same sequence:

#165  feature/164-folder-invites-schema      #181  feature/180-permission-sync-and-redirects
#167  feature/166-close-visitor-permission-gap  #182  bugfix/177-revoke-ownership-leaks
#169  feature/168-invite-and-revoke-endpoints   #184  feature/183-ownership-transfer-and-leaving
#171  feature/170-invitation-emails             #187  bugfix/185-user-route-name-collision
#174  feature/173-shared-with-me                #188  feature/186-sharing-changelog
#176  feature/175-broadcasting-invitation
#179  bugfix/178-login-log

Those bugfix branches are left visible on purpose. The defects were found by using the application, not by a later audit.

The escalation

Severity: highest. A user invited to contribute to a folder could take a private space inside it and remove the folder’s owner from that space entirely.

Reproduction — A owns parent and invites B as a contributor; B creates child inside it, then revokes A:

child.user_id=B? yes
B pivot role on child: contributor
B can INVITE on child (owner-only ability): YES     ← escalation

BEFORE: A can view child: yes
AFTER : A can view child: NO
AFTER : A can delete child: NO
AFTER : A still owns parent: yes                    ← A owns the parent
AFTER : child still inside parent: yes              ← and it is still in it
AFTER : B can view child: yes

Read the last three lines together. A owns the parent. The child is still inside it. A cannot see it.

How it was found: not by auditing the policy. A subfolder appeared under “Shared with me” where it had no business being — the subtree-collapsing query excluded children whose parent was also shared, but not children whose parent was owned. The visible bug looked cosmetic. Pulling that thread produced this.

Two compounding causes. FolderService::getContributors() was written assuming the parent’s owner owns a nested folder, while CreateFolder set user_id to the creator — two pieces of code holding different beliefs about one column. And FolderPolicy::isOwnerOrContributor() short-circuited on $folder->user_id === $user->id and returned before the $roles filter was considered, so being recorded as the creator granted every ability the policy could grant, invite included. Either alone is a bug. Together they are an escalation path.

The fix was to stop one column carrying two facts:

That distinction is what makes “a contributor may delete what they created, and nothing else” a sentence the code can express. Before the split it wasn’t expressible — no column meant “made it but doesn’t own it.”

child owner is A: yes
B can invite/revoke on child: no
after B attempts revoke -- A can view child: yes
after B attempts revoke -- A can delete child: yes

Test: FolderOwnershipTest::it does not let a contributor manage access to a subfolder they created. Reverting the single line that sets subfolder ownership fails three tests, this one among them.

Two rules where there should have been one

A separate defect, same root shape: a folder nobody could delete. The UI offered Delete because the policy allowed it; the API refused because the pipeline didn’t.

A can delete ZSub (policy): NO      ← A had removed themselves; no pivot row
B can delete ZSub (policy): yes     ← user_id short-circuit
B pivot role on ZSub: contributor   ← but CanDestroy required Owner
B attempted delete -> still there
A attempted delete -> still there

FolderPolicy::delete() and the CanDestroy pipeline stage each carried their own copy of the rule, and the copies had drifted. HasPermission had the same shape. A folder satisfying one and failing the other was permanently stranded — not a permissions error a user could act on, a dead object in their library.

The pipeline stages now delegate to the policy. One rule, one place, consulted by everything.

This is usually filed as a tidiness problem. It isn’t. Two rules that agree in every case you tested are one refactor away from disagreeing, and when authorization rules disagree the failure isn’t an exception — it’s an object your user can see and cannot touch.

Design decisions

• Invites require consent, even from people who already have accounts. Originally, inviting an existing user attached them to the folder immediately — faster, no email round-trip. Under that behaviour, anyone who knew your email could push a folder into your library with a name and description they controlled. That is a message channel, and message channels get abused. The fix removed a code path rather than adding one.
• Contributors cannot share. Sharing is not itself shareable. Someone who needs to bring others in must be given ownership — which is why transfer exists. This closes the class of problem where access spreads through a graph nobody is watching.
• Transfer re-parents and cascades. A folder lives in its owner’s tree, so handing it over moves it into the recipient’s library and carries every descendant. Leaving descendants behind would recreate the split-ownership bug one level down. Nested folders cannot be transferred at all — that would carve a hole in the current owner’s structure.
• An owner cannot leave a shared folder. They transfer it or unshare it. This guarantees every folder has exactly one owner who can reach it — the invariant preventing orphans, and a deliberate refusal to build a button users will occasionally look for.
• The changelog logs from services, not broadcast events. Reusing the events is tempting since they already fire on every access change, but they cannot do audit: leave() and revokeContributor() emit the identical event, and inviting an address with no account emits nothing at all. Keeping open tabs fresh is a different job from recording what happened.
• Notifications split actionable from informational, under one invariant. A badge once counted items nothing could display: invite notifications whose invites had been answered were filtered out of the actionable list, excluded from the informational feed, still counted unread, and skipped by both bulk-clear controls — four behaviours, each correct alone. Replaced by a single rule: actionable + unread means a decision is live; actionable + read means resolved. That also closed a slow leak, since the prune had been sparing all actionable rows and resolved ones would have accumulated forever.

Outcome

• Closed a privilege escalation before any user encountered it, and made the underlying confusion structurally impossible rather than patched.
• Eliminated the “two authorization paths disagree” class of defect by making the policy the single source.
• Made delegated write access safe on nested structures — the case every sharing model gets wrong first.
• 75 of the platform’s 129 tests cover sharing, permissions, invites and notifications.