Quick index
Purpose
Measure how closely a machine‑generated transcript preserves a human‑authenticated reference of the same interview. The app performs all work in your browser; files never leave your device.
What is compared
- Unit of comparison is the word.
- Before comparing, the tool can remove or normalize: speaker labels (e.g.,
BOYD:,Speaker 2:), timecodes ([00:02:06]), bracketed notes in the reference (e.g.,(laughs)), punctuation/casing, fillers (e.g., um, okay), and common spelling/variant forms (e.g., ok/okay). - After normalization, both texts are tokenized to words, then aligned.
WIP — Word Information Preserved
What it tells you: how much of the reference wording was retained and how much of the machine wording is confirmed by the reference. Higher is better.
How it’s computed: let N = reference words, M = machine words, C = aligned matches. Then WIP = (C/N) × (C/M), capped at 100%.
- Penalizes omissions and extras.
- Balances brevity vs verbosity.
- Limitation: paraphrases (different words, same meaning) can lower WIP.
WER — Word Error Rate
Classic ASR metric: WER = (S + D + I) ÷ N, where S = substitutions, D = deletions (missing words), I = insertions (extra words), and N = reference length. Lower is better.
Alignment method
- Dynamic‑programming word alignment (Levenshtein). Matches cost 0; mismatches/insertions/deletions cost 1.
- Tie rule: when costs tie, prefer diagonal (match/substitution) > deletion > insertion.
Configuration reference
- Ignore speaker tags & timecodes — strips names like
BOYD:,Speaker 1:and timecodes like[00:03:15]from both sides. - Reference: ignore bracketed/parenthetical notes — removes editorial notes in the reference so they don’t count as “missing.”
- Normalize typographic punctuation — converts curly quotes/dashes/ellipses into ASCII.
- Strip ASCII punctuation & symbols — removes commas, periods, em‑dashes, etc., so we compare words only.
- Remove common discourse markers — drops fillers like um, uh, okay.
- Lowercase before comparing — makes matching case‑insensitive.
- Normalize ok/um/hyphenation/conversational variants — groups frequent near‑equivalents (e.g., okay→ok).
- Soften very common little words in counts (non‑standard) — treats a lone and/the/of as aligned when the same word appears within a two‑word window on the other side. Can reduce insertions/deletions and lower WER; turn off for strict WER.
Interpreting your numbers
- WIP near 100% = strong overlap without excess verbosity.
- WER near 0% = few word‑level edits relative to the reference.
- S/D/I diagnose swaps vs drops vs extras.
Simple bands (guideline): WIP Excellent ≥95%; Good 90–95%; Fair 80–90%; Needs review <80% (and inversely for WER).
Known limitations
- Word‑overlap scoring can penalize paraphrase.
- Exotic markup or punctuation conventions may require toggling normalization.
- Very long disfluencies can still influence alignments.
FAQ
Does timecode impact my score?
No, not with the default Ignore speaker tags & timecodes enabled. Time ranges like [00:12:34] are removed before alignment.
Do speaker labels (e.g., “Speaker 2:”) affect results?
Not with the default setting. Labels such as BOYD: or Speaker 2: are stripped from both sides.
What about parenthetical notes like “(laughs)” or “[inaudible]”?
By default, those are removed from the reference only so a machine transcript that omits them isn’t penalized.
Do fillers (um/uh/okay) count?
With defaults on, common discourse markers are removed before scoring. You can disable this in Configure if your style guide requires keeping them.
Are punctuation and case considered?
Defaults strip punctuation and lowercase both sides so that scoring focuses on words. Turn these off if your evaluation needs punctuation/case.
What does “Soften very common little words” do?
It re‑labels isolated insertions/deletions of tiny words like and/the/of as matches when the same word appears within a small window nearby on the other side. This can modestly lower WER. Leave it off for strict, textbook WER.
Privacy
All processing runs entirely in your web browser. No files are uploaded or stored by this app.