SpeechAnalyzer, crash-safe recording, and an input node that lies

fastResults isn't a performance option

SpeechAnalyzer's ReportingOption.fastResults reads like a knob you'd reach for when you want to trade power for latency. That framing is what cost me the afternoon.

Without it, you don't get results that are merely slower. You get nothing for about five seconds, then a burst. Then nothing, then another burst. The analyzer batches internally and dumps, which for a live transcript means the UI sits dead through an entire sentence and then vomits a paragraph. It doesn't feel like a slow feature; it feels like a broken one — and because there's no error and no warning, you go looking in your own buffering code first.

let transcriber = SpeechTranscriber(
    locale: locale,
    transcriptionOptions: [],
    // fastResults is what makes it feel live — without it the
    // analyzer batches results every ~5 seconds.
    reportingOptions: [.volatileResults, .fastResults],
    attributeOptions: [.audioTimeRange]
)

So: if you are transcribing live, treat this as mandatory rather than optional. It behaves like a correctness switch that happens to be spelled like a performance one.

Only PCM CAF survives a crash

If a recording matters — and in a notes app, a lost meeting is the worst thing that can happen to a user — then the container you write while recording has to be PCM in CAF. Not m4a, and not AAC-in-CAF either.

The reason is structural. Both m4a and AAC-in-CAF need something written at close: the container needs finalizing, and the packet table for a compressed stream is only laid down when the file is closed properly. So if the process dies mid-write, the thing you lose is precisely the thing that makes the bytes on disk interpretable. You end up with a file full of perfectly good audio that nothing will open. The one property you actually wanted from a recorder — that it survives a crash — is the one property those containers can't give you.

PCM in CAF has no such dependency. A truncated file is just a shorter recording.

// PCM in CAF: crash-safe (readable even truncated), converted to
// AAC when the recording finishes cleanly.
audioFile = try AVAudioFile(forWriting: audioFileURL, settings: [
    AVFormatIDKey: kAudioFormatLinearPCM,
    AVSampleRateKey: micFormat.sampleRate,
    AVNumberOfChannelsKey: micFormat.channelCount,
    AVLinearPCMBitDepthKey: 16,
    AVLinearPCMIsFloatKey: false,
])

The cost is disk: PCM runs roughly an order of magnitude larger than AAC. I pay it only while recording, and transcode to AAC on a background queue after a clean stop, deleting the CAF once the m4a lands. An interrupted recording is recovered on next launch and transcoded then. So the expensive representation exists exactly as long as the window in which it might save you, and not a moment longer.

AVAudioEngine's inputNode silently ignores your device selection

This is the one that genuinely got me, because it fails without failing.

On the Mac, Nib captures the other side of a call — Zoom, Meet — so the whole conversation lands in the transcript rather than just my half of it. The standard approach is to build an aggregate device combining the microphone with a system audio tap, then point the engine at it by setting kAudioOutputUnitProperty_CurrentDevice on the input node's audio unit.

Every recording came back one channel, microphone only. No error. No warning. No thrown exception, no OSStatus worth checking. The property set appeared to succeed and the engine just… didn't use the device. I spent a while assuming my aggregate device was malformed, because that's the hypothesis that doesn't require believing the framework is lying to you.

The fix is to stop going through inputNode at all and drive the aggregate device directly with Core Audio:

var newProcID: AudioDeviceIOProcID?
let status = AudioDeviceCreateIOProcIDWithBlock(
    &newProcID, aggregateDeviceID, queue
) { _, inputData, _, _, _ in
    let buffers = UnsafeMutableAudioBufferListPointer(
        UnsafeMutablePointer(mutating: inputData)
    )
    // One stream per source (mic, tap): average each stream's own
    // channels, then sum the streams so neither side drowns the other.
    …
}

But the part I'd most want to pass on isn't the fix, it's the verification. When a failure mode is "silently gives you the microphone," any test where the microphone can hear the thing you're testing for will pass. Play a file through the speakers and record, and you'll get a beautiful transcript — through the mic, proving nothing.

So you have to test adversarially: set output volume to zero, run say, and record. If the phrase appears in the transcript, the audio could only have arrived digitally. If it doesn't, you never had the tap. Anything short of that is just confirming your microphone works.

A 4k context window is an architecture decision

I wanted summarization to run on-device. Everything else in Nib's transcription path does, the audio never leaves the machine, and it would have been a much cleaner story to tell.

The on-device Foundation Models context is 4,096 tokens. An hour-long meeting transcript is far past that before you've added the note body, the instructions, or any room for the output. This isn't a tuning problem or a prompt-engineering problem — there's no arrangement of a long transcript that fits in a window that small while still being a summary of the whole thing. Chunk-and-merge trades the exact quality that makes minutes worth reading: what got decided, and by whom, and what's still open, are all cross-cutting facts that live in the parts you'd have split.

So summarization goes to a server. I'd rather say that plainly than let "on-device" imply more than it should: in Nib, transcription is on-device and the audio never leaves your machine, and the AI summarization step sends the transcript text to a proxy I run, routed only to providers with a no-retention policy. Those are different claims and I think the distinction is worth being pedantic about, especially in a category where it gets blurred a lot.

The constraint decided the architecture. I don't love the answer, but I'd rather be honest about the seam than pretend it isn't there.