Written with AI assistance from the source video. The video is the authority.
Log in to leave a comment
No posts yet
If you try local speech recognition after watching a YouTube tutorial, nine times out of ten you will hit a roadblock. The presenter in the video double-clicks a Windows executable and calls it a day, but that file will not run on the Linux servers where we actually deploy our services or the MacBooks we use for daily work.
Using a cloud speech API is convenient, but every monthly bill stings. Once you start processing dozens of hour-long audio files, costs spiral quickly. Furthermore, some files—such as customer interviews or internal meeting minutes—cannot be sent out to external servers. This is why you need to build the source code yourself and push your computer's hardware resources to the absolute limit.
First, acquire the essential compilation tools via your package manager. These are required for compiling the C++ engine and audio resampling.
`bash
sudo apt-get update && sudo apt-get install -y
build-essential cmake git pkg-config libasound2-dev libssl-dev ffmpeg
xcode-select --install
brew install cmake pkg-config openssl ffmpeg
`
On Apple Silicon environments, you must bundle Metal shaders and the Neural Engine (ANE) together to achieve proper speeds without fan noise. Following the guide by whisper.cpp project maintainer Georgi Gerganov, combining the Core ML backend to offload encoder operations to the ANE accelerates decoding while keeping GPU heat under control.
`bash
git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
pip install ane_transformers openai-whisper coremltools
./models/generate-coreml-model.sh base.en
cmake -B build -DWHISPER_COREML=1
cmake --build build -j$(sysctl -n hw.ncpu) --config Release
`
If an NVIDIA graphics card is installed on your Linux server, enable CUDA acceleration. If you are using a Radeon card, you must use the ENGINE_ENABLE_HIP flag instead of the legacy GGML_HIPBLAS option to ensure it operates smoothly on the latest ROCm drivers.
`bash
cd whisper.cpp
cmake -B build -DGGML_CUDA=1
cmake --build build -j$(nproc) --config Release
ROCM_ARCH=$(rocminfo | grep gfx | head -1 | awk '{print $2}')
cmake -S . -B build_hip
-DENGINE_ENABLE_HIP=ON
-DGPU_TARGETS=${ROCM_ARCH}
-DCMAKE_C_COMPILER="$(hipconfig -l)/clang"
-DCMAKE_CXX_COMPILER="$(hipconfig -l)/clang++"
-DCMAKE_BUILD_TYPE=Release
cmake --build build_hip -j$(nproc)
`
When using GCC 12 or 13 on recent Ubuntu versions, you may occasionally run into the nvcc fatal: Host compiler unsupported error. Instead of struggling to downgrade your system compiler version entirely, simply append -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-11 to your CMake configuration to redirect the path that NVCC looks at. If you encounter architecture errors on a Jetson Orin board, plug in -DCMAKE_CUDA_ARCHITECTURES=87.
Once compilation is complete, verify that the binary was bundled correctly using a sample audio file.
`bash
bash ./models/download-ggml-model.sh tiny.en
./build/bin/whisper-cli -m models/ggml-tiny.en.bin -f samples/jfk.wav -otxt
cat samples/jfk.wav.txt
`
If President Kennedy's speech prints out cleanly in your terminal, the engine is ready.
There is no need to tear down the Python business logic you have already written. whisper.cpp already includes an HTTP server that mimics the OpenAI specification.
First, register a systemd unit so that the server runs in the background without dying.
`ini
[Unit]
Description=Local Audio Inference Daemon
After=network.target
[Service]
Type=simple
User=appuser
WorkingDirectory=/opt/whisper.cpp
ExecStart=/opt/whisper.cpp/build/bin/whisper-server
--model /opt/whisper.cpp/models/ggml-large-v3-turbo.bin
--host 127.0.0.1
--port 8080
--inference-path /v1/audio/transcriptions
--threads 8
--convert
Restart=always
RestartSec=3
LimitNOFILE=65535
[Install]
WantedBy=multi-user.target
`
Enable and start the daemon.
`bash
sudo systemctl daemon-reload
sudo systemctl enable --now audio-server.service
`
Now, you just need to tweak 5 lines of OpenAI client initialization parameters in your Python code.
`python
import httpx
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8080/v1",
api_key="local-mode-dummy-token",
timeout=httpx.Timeout(connect=10.0, read=300.0, write=30.0, pool=5.0)
)
with open("meeting_audio.wav", "rb") as audio_file:
transcription = client.audio.transcriptions.create(
model="whisper-1",
file=audio_file,
response_format="json"
)
print(transcription.text)
`
Since speech inference takes longer than text generation, you must set the read timeout to 300 seconds or more to prevent connections from dropping on long files. It is better to match the number of threads to your physical core count. Blindly setting threads to 16 on an 8-core, 16-thread CPU will cause L3 cache contention and degrade processing speed.
A 2-hour recording file is around 100MB when in MP3 format, but the moment it is expanded into 16-bit, 16kHz uncompressed PCM tensors for model computation, it instantly devours hundreds of megabytes of memory. Add the Transformer attention KV cache on top of this, and the process will crash due to insufficient VRAM.
It is safer to use ffmpeg to split the file into 5-minute (300-second) intervals and pass them sequentially.
`bash
#!/usr/bin/env bash
set -euo pipefail
INPUT_FILE="2"
CHUNK_DURATION=300
mkdir -p "${OUTPUT_DIR}"
ffmpeg -y -i "${INPUT_FILE}"
-vn
-acodec pcm_s16le
-ar 16000
-ac 1
-f segment
-segment_time ${CHUNK_DURATION}
-reset_timestamps 1
"${OUTPUT_DIR}/chunk_%04d.wav"
`
If you process the split chunks as-is, the subtitle (SRT) timestamps will reset to 00:00:00 for every segment. Attach a combination script that adds 300 seconds to each chunk's sequence number to align the subtitle times back into their proper place.
`python
import os
import re
from datetime import timedelta
from typing import List
TIMESTAMP_REGEX = re.compile(r"(\d{2}):(\d{2}):(\d{2}),(\d{3})")
def parse_srt_time(ts_str: str) -> timedelta:
h, m, s, ms = map(int, TIMESTAMP_REGEX.match(ts_str).groups())
return timedelta(hours=h, minutes=m, seconds=s, milliseconds=ms)
def format_srt_time(td: timedelta) -> str:
tot_sec = int(td.total_seconds())
return f"{tot_sec // 3600:02d}:{(tot_sec % 3600) // 60:02d}:{tot_sec % 60:02d},{int(td.microseconds / 1000):03d}"
def merge_srt_chunks(srt_paths: List[str], chunk_sec: float, output_path: str):
global_index = 1
with open(output_path, "w", encoding="utf-8") as outfile:
for chunk_idx, srt_file in enumerate(srt_paths):
if not os.path.exists(srt_file):
continue
offset = timedelta(seconds=chunk_idx * chunk_sec)
with open(srt_file, "r", encoding="utf-8") as infile:
for block in infile.read().strip().split("\n\n"):
lines = block.strip().split("\n")
if len(lines) < 2:
continue
matches = re.findall(TIMESTAMP_REGEX, lines[1])
if len(matches) == 2:
s_str = f"{matches[0][0]}:{matches[0][1]}:{matches[0][2]},{matches[0][3]}"
e_str = f"{matches[1][0]}:{matches[1][1]}:{matches[1][2]},{matches[1][3]}"
adj_s = format_srt_time(parse_srt_time(s_str) + offset)
adj_e = format_srt_time(parse_srt_time(e_str) + offset)
text = "\n".join(lines[2:])
outfile.write(f"{global_index}\n{adj_s} --> {adj_e}\n{text}\n\n")
global_index += 1
`
By using this approach, even a 10-hour recording file can keep its peak VRAM usage capped under 2.5GB.
When batch-processing hundreds of recording files, bad files with broken headers inevitably pop up and halt the entire batch. Set up a safety net that probes audio stream validity with ffprobe first and forcefully re-encodes problematic files. Also, include simple journaling so that you do not have to restart from scratch if the process crashes midway.
`python
import json
import subprocess
import time
from pathlib import Path
from openai import APIConnectionError, InternalServerError, OpenAI
def repair_and_transcribe(client: OpenAI, audio_path: Path, retries: int = 5) -> str:
probe_cmd = ["ffprobe", "-v", "error", "-select_streams", "a:0", str(audio_path)]
target_path = audio_path
if subprocess.run(probe_cmd).returncode != 0:
target_path = audio_path.with_suffix(".fixed.wav")
repair_cmd = ["ffmpeg", "-y", "-i", str(audio_path), "-ar", "16000", "-ac", "1", "-c:a", "pcm_s16le", str(target_path)]
subprocess.run(repair_cmd, check=True)
delay = 2.0
for attempt in range(1, retries + 1):
try:
with open(target_path, "rb") as stream:
return client.audio.transcriptions.create(model="whisper-1", file=stream, response_format="text")
except (APIConnectionError, InternalServerError) as exc:
if attempt == retries:
raise RuntimeError(f"Batch failed: {audio_path.name}") from exc
time.sleep(delay)
delay *= 2.0
class CheckpointJournal:
def init(self, path: Path):
self.path = path
self.state = json.loads(path.read_text()) if path.exists() else {"completed": []}
def is_done(self, chunk_name: str) -> bool:
return chunk_name in self.state["completed"]
def mark_done(self, chunk_name: str):
self.state["completed"].append(chunk_name)
self.path.write_text(json.dumps(self.state, indent=2))
`
Commercial cloud APIs cost around $0.36 per hour. Converting 1,000 hours per month runs you $360 a month, totaling $4,320 a year. On the other hand, running your own hardware can keep costs down to around $25 a month even after factoring in electricity bills and equipment depreciation.
| Hardware and Model Combination | Speed Multiplier | RTF Metric | Memory Usage | Measured Benchmark |
|---|---|---|---|---|
| NVIDIA RTX 5090 (Nemotron 3.5 ASR) | Approx. 150.9x | 0.0066 | 4.8 GB VRAM | Completed 327.6s audio in 2.17s |
| NVIDIA RTX 4090 (Whisper Large-v3 Turbo) | Approx. 45.0x | 0.0222 | 2.1 GB VRAM | 4-layer decoder FP16 computation |
| Apple M4 Pro (whisper.cpp CoreML+Metal) | Approx. 22.0x | 0.0454 | 3.2 GB Unified Memory | ANE encoder processing |
| Cloud API Call (Including Network Wait) | Approx. 1.5~3.0x | 0.3333~0.6666 | N/A | Dependent on network latency and queue wait time |
Even setting up just one RTX 4090 machine allows you to breeze through hundreds of hours of daily recordings without falling behind. Above all, knowing that customer voice data never leaves my room provides the greatest peace of mind.