Behind a recording card lie two tough challenges we strived to solve
Aishi Smart AI Recording Card: Technical Practices for Transcription and Speaker‑Identification
When it comes to audio recording, hardware ensures that sound “gets captured”. Yet what truly defines the usability of a recording card is what happens after recording finishes: who said what in the audio, whether in‑house proprietary terms are transcribed correctly, and whether files recorded by multiple devices in the same meeting can be merged into one complete meeting transcript.
None of these challenges can be solved simply by connecting to a general‑purpose speech large model. Recently, we built the full backend service from scratch to a shippable state. In essence, we focused solidly on two core capabilities: enabling accurate machine speech‑to‑text transcription and reliable speaker identification. This article focuses purely on implementation and testing practices rather than specification sheets.

Aishi Smart AI Recording Card: Transcription / Speaker Identification Dual-Pipeline Diagram
01 Why These Two Capabilities Matter Most
The first pain point is “raw transcripts that cannot be used directly”. Real‑world audio recordings differ greatly from clean lab‑grade WAV files. Far‑field phone recordings produce 16‑kHz mono AAC audio with telephone band‑pass filtering; built‑in laptop microphones output 44.1‑kHz stereo audio with reverberation; remote conference‑room feeds often contain background noise plus −8 dB attenuation. Meetings frequently run for dozens of minutes. Mis‑transcribed proprietary terms, product codenames or personal names force manual revision of entire transcript segments.
A second, less‑obvious but equally frustrating pain point relates to speaker labels: one speaker may be assigned different IDs across separate audio files, or two distinct speakers may be incorrectly merged under a single ID. When customers import and combine recordings from multiple voice recorders and computers, speaker attribution breaks down entirely. For meeting transcripts, this issue is more severe than occasional character‑level transcription errors — accountability for commitments and decisions hinges entirely on correct speaker tagging.
Therefore, our conclusion is that the product dividing‑line for recording‑card solutions lies not in raw recording performance, but in whether transcription outputs can feed directly into business workflows. Poor transcription accuracy and unreliable speaker‑identification are two major barriers to practical adoption.

02 Capability One: Building Production‑Ready Transcription Beyond Basic Text Output
Inference Engine and Processing Pipeline
We adopt Qwen3‑ASR‑1.7B as our transcription inference kernel. Word‑ and character‑level timestamps are generated by ForcedAligner‑0.6B. All input media is first probed via ffprobe and normalized with ffmpeg to 16‑kHz mono audio. Long‑duration audio is segmented prioritizing silence points, with fixed‑size windows as fallback, to avoid token‑limit and aligner‑duration constraints. Segmented outputs are reassembled into a continuous complete timeline. We expose outputs in plain‑text, JSONL and SRT formats alongside an OpenAI‑compatible API interface, enabling integration with legacy systems without modification.
We adopt Qwen3‑ASR‑1.7B as our transcription inference kernel. Word‑ and character‑level timestamps are generated by ForcedAligner‑0.6B. All input media is first probed via ffprobe and normalized with ffmpeg to 16‑kHz mono audio. Long‑duration audio is segmented prioritizing silence points, with fixed‑size windows as fallback, to avoid token‑limit and aligner‑duration constraints. Segmented outputs are reassembled into a continuous complete timeline. We expose outputs in plain‑text, JSONL and SRT formats alongside an OpenAI‑compatible API interface, enabling integration with legacy systems without modification.
Operable Hot‑word Management, Not Static Configuration
We implemented a two‑tier hot‑word mechanism: per‑request dynamic private vocabulary injection, plus named hot‑word sets switchable by customer, department, project or meeting. Critically, vocabulary updates require zero service restarts. Measured process uptime remained nearly unchanged (582.3 s → 582.7 s) upon updates, with vocabulary changes taking immediate effect. Every vocabulary revision generates immutable snapshots supporting rollback by revision number together with audit logging. Mis‑spelled product names within meeting minutes can be corrected by operators via vocabulary adjustments — no retraining or software release required.
A/B testing demonstrates tangible improvements. Our baseline transcription rendered “通义千问” incorrectly as “通一千问”, and “Qwen3‑ASR” as “Queen 三 ASR”. After enabling targeted hot‑word sets, both terms were transcribed accurately.
Quantitative Performance Metrics
Average Real‑Time‑Factor (RTF) for short audio clips: 0.0939 (~10× real‑time speed). For 6‑minute audio files, RTF reaches 0.013 with end‑to‑end processing completed in 4.7 seconds. Segmentation covers the full audio length rather than only early segments.
Passed validation across 11 media format test‑cases, sampling rates ranging from 8 kHz to 192 kHz, diverse bit‑depth / channel layouts, and clip durations from 0.3 seconds up to 6 minutes.
20 consecutive requests achieved 100 % success rate; concurrency levels of 1, 2 and 4 all operated stably. Peak VRAM consumption hit 12 785 MiB, fitting fully within a single 16‑GB GPU.
We tested on 34 real‑world public‑network audio files totalling 85.37 MB (maximum file size 17.79 MB; maximum duration 348 seconds; peak specs 192 kHz / 64‑bit). All 33 valid samples produced expected outputs
03 Capability Two: Beyond Speaker Counting — Consistent Global Speaker Identity

Multi‑device Identity‑Merging Workflow Diagram: Device1 / Device2 / Device3 / Device4 → Unified Preprocessing → One‑to‑One Matching → G01 / G02 / G03
We rejected one seemingly convenient shortcut. Certain end‑to‑end models generate transcription text, timestamps and speaker tags in a single inference pass. We evaluated OpenMOSS’s MOSS‑Transcribe‑Diarize, executed via FunASR AutoModel with backend="hf". It produces outputs formatted as [0.35][S01]……[3.77], offering syntactic elegance. However tags such as [S01] are anonymous labels valid only within one individual recording file. The model cannot recognize real‑world identities such as Manager Zhang, nor guarantee consistent IDs for the same human speaker across different files.
Treating these anonymous tags as reliable speaker‑recognition would produce broken results for multi‑device merged recordings. Instead we implemented a layered architecture:
Within‑file speaker diarization and timestamp alignment handled by MOSS / CAM++ to determine “who spoke when”.
Cross‑file speaker identity resolution handled by a standalone voice‑print embedding layer, assigning persistent global IDs for the same speaker across all recordings.
Both backends co‑exist behind a unified DiarizeBackend interface. Automatic fallback to local CAM++ occurs if primary workers become unreachable, preserving main‑chain availability. Conflicting transformers version requirements (4.57.6 and 5.22.2) between the two components are resolved via dual‑container isolation to prevent dependency pollution. One notable bug encountered: timestamps inside FunASR sentence_info are expressed in milliseconds. Missing unit conversion early in development caused timelines scaled erroneously by a factor of 1000.
Cross‑File Identity Consistency Implementation
Audio from every recording device is normalized to 16‑kHz mono with amplitude normalization. Leading‑ and trailing‑edge silence boundaries are intentionally preserved without cropping; cropping would introduce timeline drift across different recording sources. We then extract 192‑dimensional voice‑print embeddings. Local per‑file speaker labels are matched against global IDs through maximum‑similarity one‑to‑one assignment, fed with expected participant count as prior knowledge. A dual‑threshold logic (0.72 / 0.55) governs matching decisions: high‑confidence matches bind directly to existing global IDs. Low‑confidence matches reuse existing IDs with a review flag rather than spawning new IDs — a major source of divergent IDs for identical speakers. An upper merging‑limit also prevents incorrectly merging two different individuals.
Built upon this foundation lie product‑oriented functions: newly‑ingested audio automatically populates real names after voice‑print enrolment. Manual identity corrections update voice‑print centroids, propagating corrections forward for future files with full audit traceability. Overlapping speech receives structured tagging, and per‑speaker audio tracks can be exported independently.
Multi‑device testing provides rigorous validation. We created a master recording containing three human speakers across eight dialogue turns, then simulated four real‑world capture conditions: near‑field microphone; far‑field phone (16‑kHz / AAC / −10 dB / telephone band‑pass / light background noise); laptop built‑in microphone (44.1‑kHz / stereo / MP3 / +5 dB / reverberation); remote conference‑room feed (16‑kHz / −8 dB / band‑pass + reverberation + background noise). The four test files retain identical speakers, dialogue sequence and content, differing only in capture environment to faithfully reproduce real‑customer pain‑points.
Test Results:
All four files under one meeting session assigned consistent global identifiers G01 / G02 / G03, matching ground‑truth for every dialogue turn. Consistency rate = 1.0, identity stability = 1.0; total global speaker count = 3, zero review‑flagged entries.
Post‑preprocessing durations: 48.88 s / 48.88 s / 48.94 s / 48.97 s; no truncation of audio boundaries.
Hold‑out‑set validation: audio captured via a fifth physical device with entirely new spoken content. After voice‑print enrolment, outputs correctly populated real‑name tags:
[G01 Manager Zhang] [G02 Director Li] [G03 Engineer Wang], confirming that matching relies on voice characteristics rather than memorized utterance text.For a 45‑second two‑speaker audio sample: diarization consumed 1.58 seconds; combined transcription plus diarization took 2.455 seconds. Diarization backend runs entirely on CPU, adding zero extra GPU memory consumption (Δ0 MiB).
All 21 ASR segments carried valid speaker tags; zero unassigned segments, no fallback generic S00 labels.
04 Large‑Scale Validation & Testing

Technical development frequently falls into the trap of “ship once basic functionality runs”. Our internal rule avoids relying purely on health‑check probes. Every test invokes real business‑facing APIs using genuine audio assets, validating response payloads and persisted output files, and faithfully documenting any failures.
Cumulative multi‑phase testing results (test suites represent independent assertion sets, not repeated runs on identical datasets):
| Test Suite | Result |
| Unit Test | 87 Items OK |
| Phase 1 Full Acceptance | 60 PASS / 0 FAIL (Format, Signal, Duration, Content, Exception, Concurrency, Stability, VRAM) |
| Real Public-Downloaded Audio | 33 / 33 PASS (1 failed download sample skipped honestly, not counted) |
| Hotword & Speaker Diarization Special Test | 40 PASS / 0 FAIL |
| Phase 2 (Cross-File Speaker Identity / MOSS Integration) | 47 PASS / 0 FAIL |
| Docker Dual-Container Business Acceptance | 29 PASS / 0 FAIL (4 mandatory test categories: single-file transcription, hotword A/B test, multi-speaker recognition, voiceprint enrolment) |
| AI-Readable API Contract Regression | 114 PASS / 0 FAIL (Real requests per endpoint + schema field-by-field validation) |
| ndependent Python / JavaScript Client Direct Connection Demo | 19 PASS / 0 FAIL for each |
Total: 448 assertions / test‑cases, zero failures. Within Phase‑2 and Docker acceptance reports, raw .speakers.txt outputs from four recording devices are included verbatim. Each line contains global speaker ID alongside transcribed utterance text for line‑by‑line ground‑truth comparison, instead of publishing only aggregate accuracy percentages.
API contract quality also deserves attention. The auto‑generated server‑side /openapi.json describes four POST routes with params=0 / body=no, defining only two framework‑level error schemas, because request parsing manually handles multipart / JSON payloads. Code auto‑generators consuming this raw OpenAPI spec would produce non‑functional client code. We manually authored a complete OpenAPI 3.1 specification drawing upon 74 real‑world production request‑response samples: covering 12 paths, 16 operations, 34 schemas, 34 error‑codes and dual authentication schemes. This contract is enforced by 114 regression test‑cases. Readable, machine‑consumable APIs ready for third‑party integration constitute a critical dimension of deliverable quality.
