Before You Pay for a Faster AI Model, Ask This One Question

A founder showed me their AI feature earlier this year and asked why people kept closing the tab before it finished.

The feature worked fine and the answers were good. But every time someone asked a question, they stared at a blank screen for eight or nine seconds while the model generated the full response, and then the whole thing appeared at once. Users assumed it was broken and left.

The AI wasn’t the problem. The way we delivered it was. We were waiting for the model to finish the entire answer on the server, then sending it down in one payload. So the user sat there with nothing until all of it showed up at once.

We changed it to stream. Server-Sent Events push each piece of the answer to the screen the moment the model produces it, the same way you watch ChatGPT type word by word. It’s the same model and it finishes at the same time. But text shows up within half a second and the user reads along as it builds. Nobody thinks it’s broken when they can watch it work.

On a learning product I built more recently, I streamed each lesson as its own event as the batches finished, so a user waiting on feedback for ten questions sees the first lesson while the rest are still being written.

Your AI feature can be smart and still feel bad to use. How responsive it feels comes down to how you deliver the output, and that has almost nothing to do with which model you’re paying for.

If your AI product feels slow, ask your team one question before you pay for a faster model: are we streaming the response, or making the user wait for all of it?